2024-09-11T23:17:26.584 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T23:17:26.610 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T23:17:26.714 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iogen}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901477' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBB859WpOcC3+JmrV+2H3s8sRkPpqBLhGWSi1GxpvgDMrfHg0S3OOzJLGaLsCLeLQWXQ8NfwieDRqnGFXXCnAjw= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKupNFnnJ6c0eZtTV5d53kSJeyaBEw375hjnI9+APiV86EindBI4pbHt9RzM+bU8Sw2xj90h7e5i3OSbNRKQ1MY= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC3O1YCu+0KdgDs/rOXB11mmUsnVUgJIqYC4E0+Ws391btV4lIDvGYMPHLbiq++BCRJHT8F3fho1YbBek36PH/E= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T23:17:26.714 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-11T23:17:26.715 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-11T23:17:26.715 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T23:17:26.717 INFO:teuthology.task.internal:Checking packages... 2024-09-11T23:17:26.739 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-11T23:17:26.740 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T23:17:26.740 INFO:teuthology.packaging:ref: None 2024-09-11T23:17:26.740 INFO:teuthology.packaging:tag: None 2024-09-11T23:17:26.740 INFO:teuthology.packaging:branch: squid 2024-09-11T23:17:26.740 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:17:26.740 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-11T23:17:26.990 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-11T23:17:26.992 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T23:17:26.998 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T23:17:26.998 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T23:17:27.009 INFO:teuthology.task.internal:Saving configuration 2024-09-11T23:17:27.030 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T23:17:27.038 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T23:17:27.072 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 23:12:00.798621', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBB859WpOcC3+JmrV+2H3s8sRkPpqBLhGWSi1GxpvgDMrfHg0S3OOzJLGaLsCLeLQWXQ8NfwieDRqnGFXXCnAjw='} 2024-09-11T23:17:27.102 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 23:12:00.795034', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKupNFnnJ6c0eZtTV5d53kSJeyaBEw375hjnI9+APiV86EindBI4pbHt9RzM+bU8Sw2xj90h7e5i3OSbNRKQ1MY='} 2024-09-11T23:17:27.139 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 23:12:00.796953', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC3O1YCu+0KdgDs/rOXB11mmUsnVUgJIqYC4E0+Ws391btV4lIDvGYMPHLbiq++BCRJHT8F3fho1YbBek36PH/E='} 2024-09-11T23:17:27.139 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T23:17:27.144 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-11T23:17:27.145 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-11T23:17:27.145 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-11T23:17:27.145 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T23:17:27.250 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8cd241f5b0>, signals=[15]) 2024-09-11T23:17:27.250 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T23:17:27.255 INFO:teuthology.task.internal:Opening connections... 2024-09-11T23:17:27.255 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T23:17:27.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:17:27.358 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-09-11T23:17:27.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:17:27.442 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-09-11T23:17:27.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:17:27.527 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T23:17:27.533 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-09-11T23:17:27.551 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-09-11T23:17:27.551 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-09-11T23:17:27.609 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2024-09-11T23:17:27.609 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2024-09-11T23:17:27.609 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2024-09-11T23:17:27.609 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2024-09-11T23:17:27.609 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2024-09-11T23:17:27.609 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2024-09-11T23:17:27.609 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T23:17:27.609 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-09-11T23:17:27.610 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2024-09-11T23:17:27.610 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T23:17:27.610 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2024-09-11T23:17:27.610 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T23:17:27.610 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T23:17:27.610 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T23:17:27.611 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-09-11T23:17:27.639 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-09-11T23:17:27.658 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-09-11T23:17:27.658 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-09-11T23:17:27.716 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-09-11T23:17:27.716 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-09-11T23:17:27.716 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-09-11T23:17:27.716 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-09-11T23:17:27.716 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-09-11T23:17:27.716 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-09-11T23:17:27.716 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T23:17:27.717 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-09-11T23:17:27.717 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-09-11T23:17:27.717 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T23:17:27.717 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-09-11T23:17:27.717 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T23:17:27.717 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T23:17:27.717 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T23:17:27.717 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-09-11T23:17:27.738 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-09-11T23:17:27.757 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-09-11T23:17:27.757 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-09-11T23:17:27.815 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-09-11T23:17:27.815 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-09-11T23:17:27.815 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-09-11T23:17:27.815 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-09-11T23:17:27.815 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-09-11T23:17:27.815 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-09-11T23:17:27.815 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T23:17:27.815 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-09-11T23:17:27.815 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-09-11T23:17:27.815 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T23:17:27.816 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-09-11T23:17:27.816 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T23:17:27.816 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T23:17:27.816 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T23:17:27.816 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-09-11T23:17:27.847 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T23:17:27.858 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T23:17:27.863 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T23:17:27.863 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-09-11T23:17:27.866 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-09-11T23:17:27.868 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-09-11T23:17:27.885 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T23:17:27.891 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T23:17:27.891 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-09-11T23:17:27.924 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-09-11T23:17:27.926 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-09-11T23:17:27.957 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T23:17:28.066 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T23:17:28.080 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-11T23:17:28.081 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-11T23:17:28.081 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-11T23:17:28.081 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T23:17:28.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:17:28.098 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-09-11T23:17:28.157 INFO:teuthology.orchestra.run.smithi133.stdout:5.14.0-503.el9.x86_64 2024-09-11T23:17:28.158 INFO:teuthology.task.kernel:Running kernel on smithi133: 5.14.0-503.el9.x86_64 2024-09-11T23:17:28.158 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum install -y kernel 2024-09-11T23:17:30.613 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-11T23:17:32.471 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-11T23:17:38.401 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.5 MB 00:02 2024-09-11T23:17:39.907 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-11T23:17:41.136 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-11T23:17:46.550 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-11T23:17:48.556 INFO:teuthology.orchestra.run.smithi133.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T23:17:48.556 INFO:teuthology.orchestra.run.smithi133.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T23:17:48.636 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-09-11T23:17:48.646 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-11T23:17:48.647 INFO:teuthology.orchestra.run.smithi133.stdout:Install 4 Packages 2024-09-11T23:17:48.647 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:17:48.647 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 79 M 2024-09-11T23:17:48.647 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 126 M 2024-09-11T23:17:48.647 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-09-11T23:17:48.793 INFO:teuthology.orchestra.run.smithi133.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 181 kB/s | 23 kB 00:00 2024-09-11T23:17:49.110 INFO:teuthology.orchestra.run.smithi133.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-09-11T23:17:49.494 INFO:teuthology.orchestra.run.smithi133.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 42 MB/s | 35 MB 00:00 2024-09-11T23:17:49.952 INFO:teuthology.orchestra.run.smithi133.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-11T23:17:49.952 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:17:49.953 INFO:teuthology.orchestra.run.smithi133.stdout:Total 60 MB/s | 79 MB 00:01 2024-09-11T23:17:50.640 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-09-11T23:17:50.995 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-09-11T23:17:50.995 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-09-11T23:17:51.948 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-09-11T23:17:51.948 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-09-11T23:17:54.405 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-09-11T23:17:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T23:17:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T23:17:58.012 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T23:17:58.145 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T23:18:02.977 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T23:18:03.037 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:18:07.882 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:18:45.697 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:18:45.736 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:18:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:18:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T23:18:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T23:18:46.064 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T23:18:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:18:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:18:46.882 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-09-11T23:18:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T23:18:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T23:18:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T23:18:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T23:18:46.883 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:18:46.883 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-09-11T23:18:47.256 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T23:18:47.347 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T23:18:47.347 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T23:18:47.348 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T23:18:47.348 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T23:18:47.348 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T23:18:47.348 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-11T23:18:47.348 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T23:18:47.348 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T23:18:47.407 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T23:18:47.408 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T23:18:47.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -qi grub2-tools 2024-09-11T23:18:47.445 INFO:teuthology.orchestra.run.smithi133.stdout:Name : grub2-tools 2024-09-11T23:18:47.445 INFO:teuthology.orchestra.run.smithi133.stdout:Epoch : 1 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:Version : 2.06 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:Release : 82.el9 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: x86_64 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:Group : Unspecified 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:Size : 8274670 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:License : GPLv3+ 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T23:18:47.446 INFO:teuthology.orchestra.run.smithi133.stdout:Packager : builder@centos.org 2024-09-11T23:18:47.447 INFO:teuthology.orchestra.run.smithi133.stdout:Vendor : CentOS 2024-09-11T23:18:47.447 INFO:teuthology.orchestra.run.smithi133.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T23:18:47.447 INFO:teuthology.orchestra.run.smithi133.stdout:Summary : Support tools for GRUB. 2024-09-11T23:18:47.447 INFO:teuthology.orchestra.run.smithi133.stdout:Description : 2024-09-11T23:18:47.447 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:18:47.447 INFO:teuthology.orchestra.run.smithi133.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T23:18:47.447 INFO:teuthology.orchestra.run.smithi133.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T23:18:47.447 INFO:teuthology.orchestra.run.smithi133.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T23:18:47.447 INFO:teuthology.orchestra.run.smithi133.stdout:hardware devices. 2024-09-11T23:18:47.447 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:18:47.447 INFO:teuthology.orchestra.run.smithi133.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T23:18:47.448 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T23:18:47.448 INFO:teuthology.task.kernel:Updating grub on smithi133 to boot 5.14.0-505.el9.x86_64 2024-09-11T23:18:47.448 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T23:18:48.088 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2024-09-11T23:18:49.490 INFO:teuthology.orchestra.run.smithi133.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T23:18:49.526 INFO:teuthology.orchestra.run.smithi133.stderr:done 2024-09-11T23:18:49.529 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-11T23:18:49.544 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.mOZFSu7uNs 2024-09-11T23:18:49.544 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mOZFSu7uNs 2024-09-11T23:18:49.611 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.mOZFSu7uNs 2024-09-11T23:18:49.757 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.mOZFSu7uNs is 6KB 2024-09-11T23:18:49.807 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.mOZFSu7uNs 2024-09-11T23:18:49.824 DEBUG:teuthology.orchestra.run.smithi133:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T23:18:49.887 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T23:18:49.887 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T23:18:49.887 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T23:18:49.887 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T23:18:49.888 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T23:18:50.062 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2024-09-11T23:19:20.066 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T23:19:20.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T23:19:20.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:19:38.557 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-09-11T23:19:47.558 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T23:19:47.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:19:50.653 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-09-11T23:20:02.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T23:20:02.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:20:34.389 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-09-11T23:20:35.061 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T23:20:35.061 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-11T23:20:35.061 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-09-11T23:20:35.075 INFO:teuthology.orchestra.run.smithi133.stdout:5.14.0-505.el9.x86_64 2024-09-11T23:20:35.075 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T23:20:35.076 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T23:20:35.076 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T23:20:36.077 DEBUG:teuthology.parallel:result is None 2024-09-11T23:20:36.077 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T23:20:36.743 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T23:20:36.744 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:20:36.747 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:20:36.750 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:20:36.769 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T23:20:37.093 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T23:20:37.210 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T23:20:37.210 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:20:37.213 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:20:37.217 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:20:37.399 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T23:20:37.405 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T23:20:37.405 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T23:20:37.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:20:37.423 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T23:20:37.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:20:37.440 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:20:37.443 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:20:37.465 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:20:37.502 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:20:37.519 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:20:37.528 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:20:37.529 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:20:37.555 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:20:37.557 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:20:37.559 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T23:20:37.566 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T23:20:37.566 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:20:37.569 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:20:37.571 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:20:37.598 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T23:20:37.607 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T23:20:37.608 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:20:37.641 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:20:37.644 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:20:37.661 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:20:37.752 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:20:37.820 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:20:37.870 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:20:37.909 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:20:37.909 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:20:37.979 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:20:38.030 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:20:38.081 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:20:38.174 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:20:38.209 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-11T23:20:38.209 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:20:38.276 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:20:38.322 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:20:38.372 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:20:38.460 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:20:38.501 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-11T23:20:38.502 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:20:38.568 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-09-11T23:20:38.571 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-09-11T23:20:38.577 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-09-11T23:20:38.616 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:20:38.620 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:20:38.643 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:20:39.080 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T23:20:39.087 INFO:teuthology.task.internal:Starting timer... 2024-09-11T23:20:39.088 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T23:20:39.098 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T23:20:39.106 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-11T23:20:39.107 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-09-11T23:20:39.167 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-09-11T23:20:39.169 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-09-11T23:20:39.242 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-09-11T23:20:39.245 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-09-11T23:20:39.313 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-09-11T23:20:39.315 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T23:20:39.315 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-09-11T23:20:39.340 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-09-11T23:20:39.340 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-09-11T23:20:39.374 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-09-11T23:20:39.374 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-09-11T23:20:39.402 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-09-11T23:20:39.402 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi133.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2024-09-11T23:20:39.402 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:20:39.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:20:39.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:20:39.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:20:39.461 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:20:39.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:20:39.490 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T23:20:39.490 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:20:39.527 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:20:39.565 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:20:39.598 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T23:20:39.608 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T23:20:39.611 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T23:20:39.761 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T23:20:39.772 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T23:20:39.773 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi133.front.sepia.ceph.com,smithi138.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T23:24:40.617 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi133.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-09-11T23:24:40.619 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T23:24:40.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:24:40.706 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-09-11T23:24:40.784 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T23:24:40.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-11T23:24:40.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:24:40.886 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-09-11T23:24:40.970 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-11T23:24:40.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-11T23:24:40.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:24:41.056 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-09-11T23:24:41.094 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-11T23:24:41.094 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T23:24:41.106 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T23:24:41.106 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:24:41.107 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:24:41.109 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:24:41.109 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:24:41.112 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:24:41.112 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:24:41.136 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:24:41.143 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:24:41.149 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:24:41.165 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:24:41.166 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:24:41.173 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-09-11T23:24:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:24:41.184 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-09-11T23:24:41.188 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-09-11T23:24:41.198 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:24:41.200 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-09-11T23:24:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-09-11T23:24:41.212 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:24:41.215 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:24:41.220 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-09-11T23:24:41.229 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:24:41.235 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:24:41.250 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:24:41.271 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:24:41.275 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:24:41.275 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-09-11T23:24:41.275 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.275 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.275 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.276 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.289 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:24:41.293 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:24:41.293 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-09-11T23:24:41.293 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.293 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.293 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.293 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.311 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:24:41.314 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:24:41.315 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-11T23:24:41.315 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.315 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.315 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.315 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:24:41.316 INFO:teuthology.run_tasks:Running task install... 2024-09-11T23:24:41.324 DEBUG:teuthology.task.install:project ceph 2024-09-11T23:24:41.324 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-11T23:24:41.324 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-11T23:24:41.324 INFO:teuthology.task.install:Using flavor: default 2024-09-11T23:24:41.329 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T23:24:41.330 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-11T23:24:41.330 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:24:41.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:24:41.333 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:24:41.333 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:24:41.334 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:24:41.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:24:41.488 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:24:41.488 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:24:41.500 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:24:41.500 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:24:41.511 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:24:41.511 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:24:41.608 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:24:41.608 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:24:41.608 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:24:41.616 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:24:41.616 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-11T23:24:41.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:24:41.627 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:24:41.627 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-11T23:24:41.627 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:24:41.634 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:24:41.634 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:24:41.643 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:24:41.643 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:24:41.653 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:24:41.653 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:24:41.700 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:24:41.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:24:41.719 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:24:41.776 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:24:41.785 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:24:41.796 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:24:41.840 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-09-11T23:24:41.841 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-09-11T23:24:41.849 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-09-11T23:24:41.850 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-09-11T23:24:41.860 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-09-11T23:24:41.861 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-09-11T23:24:42.074 INFO:teuthology.orchestra.run.smithi138.stdout:45 files removed 2024-09-11T23:24:42.081 INFO:teuthology.orchestra.run.smithi133.stdout:45 files removed 2024-09-11T23:24:42.085 INFO:teuthology.orchestra.run.smithi178.stdout:45 files removed 2024-09-11T23:24:42.101 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:24:42.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:24:42.112 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:24:42.648 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 323 kB/s | 84 kB 00:00 2024-09-11T23:24:42.657 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-11T23:24:42.663 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-11T23:24:42.915 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-11T23:24:42.939 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-11T23:24:42.941 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-11T23:24:43.182 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-11T23:24:43.198 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-11T23:24:43.209 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-11T23:24:44.191 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.2 MB 00:00 2024-09-11T23:24:44.243 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.2 MB 00:01 2024-09-11T23:24:44.565 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.2 MB 00:01 2024-09-11T23:24:46.241 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-11T23:24:47.161 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-11T23:24:48.136 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 20 MB 00:02 2024-09-11T23:24:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-11T23:24:52.234 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 19 kB 00:00 2024-09-11T23:24:52.421 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-11T23:24:53.292 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.5 MB 00:01 2024-09-11T23:24:53.389 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-11T23:24:54.039 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-11T23:24:54.918 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-11T23:24:54.934 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-11T23:24:56.080 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-11T23:24:58.727 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-11T23:25:00.407 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-11T23:25:01.088 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:25:01.089 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:25:01.100 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:25:01.101 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:25:01.102 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:25:01.103 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:25:01.104 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:25:01.107 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:25:01.108 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:25:01.171 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-09-11T23:25:01.190 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================== 2024-09-11T23:25:01.190 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-09-11T23:25:01.190 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================== 2024-09-11T23:25:01.190 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-09-11T23:25:01.190 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:25:01.190 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:25:01.191 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:25:01.192 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:25:01.193 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:25:01.194 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:25:01.195 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:25:01.196 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:25:01.197 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:25:01.198 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:25:01.199 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:25:01.200 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:25:01.201 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout:Installing weak dependencies: 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================== 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout:Install 148 Packages 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 8 Packages 2024-09-11T23:25:01.202 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:01.203 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 226 M 2024-09-11T23:25:01.203 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-09-11T23:25:01.494 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-11T23:25:02.079 INFO:teuthology.orchestra.run.smithi133.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-11T23:25:02.312 INFO:teuthology.orchestra.run.smithi133.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.8 MB/s | 897 kB 00:00 2024-09-11T23:25:02.404 INFO:teuthology.orchestra.run.smithi133.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-11T23:25:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.2 MB/s | 145 kB 00:00 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi133.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 22 MB/s | 2.4 MB 00:00 2024-09-11T23:25:02.655 INFO:teuthology.orchestra.run.smithi133.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 6.8 MB/s | 1.5 MB 00:00 2024-09-11T23:25:02.669 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:25:02.670 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:25:02.683 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:25:02.684 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:25:02.685 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:25:02.686 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:25:02.688 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:25:02.690 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:25:02.691 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:25:02.755 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-11T23:25:02.763 INFO:teuthology.orchestra.run.smithi133.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 19 MB/s | 4.7 MB 00:00 2024-09-11T23:25:02.774 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:25:02.775 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:25:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:25:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:25:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:25:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:25:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:25:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:25:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:25:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:25:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:25:02.785 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:25:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout:Install 148 Packages 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 8 Packages 2024-09-11T23:25:02.787 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 226 M 2024-09-11T23:25:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-11T23:25:03.084 INFO:teuthology.orchestra.run.smithi138.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-11T23:25:03.172 INFO:teuthology.orchestra.run.smithi133.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 17 MB/s | 21 MB 00:01 2024-09-11T23:25:03.239 INFO:teuthology.orchestra.run.smithi133.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 380 kB/s | 25 kB 00:00 2024-09-11T23:25:03.459 INFO:teuthology.orchestra.run.smithi138.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.3 MB/s | 897 kB 00:00 2024-09-11T23:25:03.535 INFO:teuthology.orchestra.run.smithi138.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 9.5 MB/s | 5.4 MB 00:00 2024-09-11T23:25:03.548 INFO:teuthology.orchestra.run.smithi133.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 14 MB/s | 11 MB 00:00 2024-09-11T23:25:03.560 INFO:teuthology.orchestra.run.smithi138.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.4 MB/s | 145 kB 00:00 2024-09-11T23:25:03.640 INFO:teuthology.orchestra.run.smithi133.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 228 kB 00:00 2024-09-11T23:25:03.715 INFO:teuthology.orchestra.run.smithi133.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 361 kB/s | 27 kB 00:00 2024-09-11T23:25:03.735 INFO:teuthology.orchestra.run.smithi138.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 2.4 MB 00:00 2024-09-11T23:25:03.770 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:25:03.771 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:25:03.782 INFO:teuthology.orchestra.run.smithi133.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 477 kB/s | 32 kB 00:00 2024-09-11T23:25:03.782 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:25:03.782 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:25:03.783 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:25:03.784 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:25:03.786 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:25:03.788 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:25:03.789 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:25:03.850 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-11T23:25:03.872 INFO:teuthology.orchestra.run.smithi178.stdout:====================================================================================== 2024-09-11T23:25:03.872 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-11T23:25:03.872 INFO:teuthology.orchestra.run.smithi178.stdout:====================================================================================== 2024-09-11T23:25:03.872 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-11T23:25:03.872 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:25:03.872 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:25:03.872 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:25:03.872 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:25:03.872 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:25:03.872 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:25:03.872 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:25:03.873 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:25:03.874 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:25:03.875 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:25:03.876 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:25:03.876 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:25:03.876 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:25:03.876 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:25:03.876 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:25:03.876 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:25:03.876 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:25:03.876 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:25:03.876 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:25:03.876 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:25:03.877 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:25:03.878 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:25:03.879 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:25:03.880 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:25:03.881 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:25:03.882 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:25:03.883 INFO:teuthology.orchestra.run.smithi178.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout:====================================================================================== 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout:Install 148 Packages 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 8 Packages 2024-09-11T23:25:03.884 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:25:03.885 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 226 M 2024-09-11T23:25:03.885 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-11T23:25:03.957 INFO:teuthology.orchestra.run.smithi133.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 4.2 MB/s | 752 kB 00:00 2024-09-11T23:25:03.986 INFO:teuthology.orchestra.run.smithi138.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 3.6 MB/s | 1.5 MB 00:00 2024-09-11T23:25:04.041 INFO:teuthology.orchestra.run.smithi133.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.9 MB/s | 163 kB 00:00 2024-09-11T23:25:04.178 INFO:teuthology.orchestra.run.smithi138.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-11T23:25:04.233 INFO:teuthology.orchestra.run.smithi133.stdout:(16/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 17 MB 00:01 2024-09-11T23:25:04.258 INFO:teuthology.orchestra.run.smithi133.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 583 kB/s | 127 kB 00:00 2024-09-11T23:25:04.317 INFO:teuthology.orchestra.run.smithi133.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.9 MB/s | 506 kB 00:00 2024-09-11T23:25:04.417 INFO:teuthology.orchestra.run.smithi133.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 451 kB/s | 45 kB 00:00 2024-09-11T23:25:04.525 INFO:teuthology.orchestra.run.smithi133.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.2 MB/s | 136 kB 00:00 2024-09-11T23:25:04.634 INFO:teuthology.orchestra.run.smithi133.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.5 MB/s | 162 kB 00:00 2024-09-11T23:25:04.662 INFO:teuthology.orchestra.run.smithi178.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 19 kB/s | 6.5 kB 00:00 2024-09-11T23:25:04.793 INFO:teuthology.orchestra.run.smithi133.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 2.0 MB/s | 321 kB 00:00 2024-09-11T23:25:04.920 INFO:teuthology.orchestra.run.smithi133.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.4 MB/s | 301 kB 00:00 2024-09-11T23:25:04.978 INFO:teuthology.orchestra.run.smithi138.stdout:(8/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 13 MB/s | 11 MB 00:00 2024-09-11T23:25:05.001 INFO:teuthology.orchestra.run.smithi133.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.2 MB/s | 100 kB 00:00 2024-09-11T23:25:05.085 INFO:teuthology.orchestra.run.smithi133.stdout:(25/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 6.5 MB/s | 5.3 MB 00:00 2024-09-11T23:25:05.095 INFO:teuthology.orchestra.run.smithi178.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.0 MB/s | 897 kB 00:00 2024-09-11T23:25:05.110 INFO:teuthology.orchestra.run.smithi133.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 784 kB/s | 85 kB 00:00 2024-09-11T23:25:05.171 INFO:teuthology.orchestra.run.smithi178.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.9 MB/s | 145 kB 00:00 2024-09-11T23:25:05.177 INFO:teuthology.orchestra.run.smithi133.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-11T23:25:05.204 INFO:teuthology.orchestra.run.smithi138.stdout:(9/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 9.5 MB/s | 21 MB 00:02 2024-09-11T23:25:05.229 INFO:teuthology.orchestra.run.smithi138.stdout:(10/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9 100 kB/s | 25 kB 00:00 2024-09-11T23:25:05.253 INFO:teuthology.orchestra.run.smithi133.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 424 kB/s | 32 kB 00:00 2024-09-11T23:25:05.313 INFO:teuthology.orchestra.run.smithi138.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 228 kB 00:00 2024-09-11T23:25:05.413 INFO:teuthology.orchestra.run.smithi138.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 270 kB/s | 27 kB 00:00 2024-09-11T23:25:05.445 INFO:teuthology.orchestra.run.smithi133.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 763 kB/s | 146 kB 00:00 2024-09-11T23:25:05.503 INFO:teuthology.orchestra.run.smithi133.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 7.4 MB/s | 3.1 MB 00:00 2024-09-11T23:25:05.513 INFO:teuthology.orchestra.run.smithi138.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 317 kB/s | 32 kB 00:00 2024-09-11T23:25:05.813 INFO:teuthology.orchestra.run.smithi178.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.7 MB/s | 2.4 MB 00:00 2024-09-11T23:25:05.954 INFO:teuthology.orchestra.run.smithi133.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 7.0 MB/s | 3.6 MB 00:00 2024-09-11T23:25:06.005 INFO:teuthology.orchestra.run.smithi138.stdout:(14/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 8.3 MB/s | 17 MB 00:02 2024-09-11T23:25:06.054 INFO:teuthology.orchestra.run.smithi178.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 6.3 MB/s | 1.5 MB 00:00 2024-09-11T23:25:06.055 INFO:teuthology.orchestra.run.smithi138.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.2 MB/s | 163 kB 00:00 2024-09-11T23:25:06.106 INFO:teuthology.orchestra.run.smithi138.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-11T23:25:06.173 INFO:teuthology.orchestra.run.smithi138.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 7.4 MB/s | 506 kB 00:00 2024-09-11T23:25:06.198 INFO:teuthology.orchestra.run.smithi138.stdout:(18/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 1.1 MB/s | 752 kB 00:00 2024-09-11T23:25:06.257 INFO:teuthology.orchestra.run.smithi138.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 776 kB/s | 45 kB 00:00 2024-09-11T23:25:06.316 INFO:teuthology.orchestra.run.smithi138.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.3 MB/s | 136 kB 00:00 2024-09-11T23:25:06.330 INFO:teuthology.orchestra.run.smithi178.stdout:(6/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 2.7 MB/s | 5.4 MB 00:02 2024-09-11T23:25:06.366 INFO:teuthology.orchestra.run.smithi138.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-11T23:25:06.396 INFO:teuthology.orchestra.run.smithi133.stdout:(32/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 15 MB/s | 49 MB 00:03 2024-09-11T23:25:06.430 INFO:teuthology.orchestra.run.smithi133.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 524 kB/s | 249 kB 00:00 2024-09-11T23:25:06.441 INFO:teuthology.orchestra.run.smithi138.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.2 MB/s | 321 kB 00:00 2024-09-11T23:25:06.471 INFO:teuthology.orchestra.run.smithi133.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 658 kB/s | 49 kB 00:00 2024-09-11T23:25:06.497 INFO:teuthology.orchestra.run.smithi133.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 246 kB/s | 16 kB 00:00 2024-09-11T23:25:06.539 INFO:teuthology.orchestra.run.smithi133.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.2 MB/s | 285 kB 00:00 2024-09-11T23:25:06.550 INFO:teuthology.orchestra.run.smithi138.stdout:(23/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-11T23:25:06.575 INFO:teuthology.orchestra.run.smithi138.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.2 MB/s | 301 kB 00:00 2024-09-11T23:25:06.617 INFO:teuthology.orchestra.run.smithi138.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.5 MB/s | 100 kB 00:00 2024-09-11T23:25:06.643 INFO:teuthology.orchestra.run.smithi138.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-11T23:25:06.664 INFO:teuthology.orchestra.run.smithi133.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 6.4 MB/s | 7.4 MB 00:01 2024-09-11T23:25:06.689 INFO:teuthology.orchestra.run.smithi133.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 121 kB/s | 18 kB 00:00 2024-09-11T23:25:06.710 INFO:teuthology.orchestra.run.smithi138.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-11T23:25:06.723 INFO:teuthology.orchestra.run.smithi133.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 3.3 MB/s | 765 kB 00:00 2024-09-11T23:25:06.760 INFO:teuthology.orchestra.run.smithi138.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 640 kB/s | 32 kB 00:00 2024-09-11T23:25:06.813 INFO:teuthology.orchestra.run.smithi178.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 6.2 MB/s | 4.7 MB 00:00 2024-09-11T23:25:06.819 INFO:teuthology.orchestra.run.smithi138.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.4 MB/s | 146 kB 00:00 2024-09-11T23:25:06.869 INFO:teuthology.orchestra.run.smithi138.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-11T23:25:07.136 INFO:teuthology.orchestra.run.smithi138.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 11 MB/s | 3.6 MB 00:00 2024-09-11T23:25:07.148 INFO:teuthology.orchestra.run.smithi133.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 165 kB/s | 80 kB 00:00 2024-09-11T23:25:07.220 INFO:teuthology.orchestra.run.smithi138.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.9 MB/s | 249 kB 00:00 2024-09-11T23:25:07.265 INFO:teuthology.orchestra.run.smithi133.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 353 kB/s | 191 kB 00:00 2024-09-11T23:25:07.304 INFO:teuthology.orchestra.run.smithi138.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 591 kB/s | 49 kB 00:00 2024-09-11T23:25:07.379 INFO:teuthology.orchestra.run.smithi138.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 219 kB/s | 16 kB 00:00 2024-09-11T23:25:07.415 INFO:teuthology.orchestra.run.smithi133.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-11T23:25:07.441 INFO:teuthology.orchestra.run.smithi133.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 867 kB/s | 253 kB 00:00 2024-09-11T23:25:07.513 INFO:teuthology.orchestra.run.smithi138.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 12 MB/s | 7.4 MB 00:00 2024-09-11T23:25:07.532 INFO:teuthology.orchestra.run.smithi133.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 911 kB/s | 106 kB 00:00 2024-09-11T23:25:07.555 INFO:teuthology.orchestra.run.smithi138.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 1.6 MB/s | 285 kB 00:00 2024-09-11T23:25:07.558 INFO:teuthology.orchestra.run.smithi133.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-11T23:25:07.588 INFO:teuthology.orchestra.run.smithi178.stdout:(8/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 17 MB 00:01 2024-09-11T23:25:07.597 INFO:teuthology.orchestra.run.smithi138.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 9.0 MB/s | 765 kB 00:00 2024-09-11T23:25:07.600 INFO:teuthology.orchestra.run.smithi133.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 3.7 MB/s | 1.2 MB 00:00 2024-09-11T23:25:07.622 INFO:teuthology.orchestra.run.smithi138.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 271 kB/s | 18 kB 00:00 2024-09-11T23:25:07.638 INFO:teuthology.orchestra.run.smithi178.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 506 kB/s | 25 kB 00:00 2024-09-11T23:25:07.642 INFO:teuthology.orchestra.run.smithi133.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-11T23:25:07.675 INFO:teuthology.orchestra.run.smithi133.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-11T23:25:07.725 INFO:teuthology.orchestra.run.smithi133.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-09-11T23:25:07.764 INFO:teuthology.orchestra.run.smithi138.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-11T23:25:07.793 INFO:teuthology.orchestra.run.smithi133.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 914 kB/s | 107 kB 00:00 2024-09-11T23:25:07.798 INFO:teuthology.orchestra.run.smithi138.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-11T23:25:07.835 INFO:teuthology.orchestra.run.smithi133.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 199 kB/s | 22 kB 00:00 2024-09-11T23:25:07.865 INFO:teuthology.orchestra.run.smithi138.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-11T23:25:07.868 INFO:teuthology.orchestra.run.smithi133.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-11T23:25:07.894 INFO:teuthology.orchestra.run.smithi138.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 253 kB 00:00 2024-09-11T23:25:07.910 INFO:teuthology.orchestra.run.smithi133.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-11T23:25:07.944 INFO:teuthology.orchestra.run.smithi133.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 300 kB/s | 33 kB 00:00 2024-09-11T23:25:07.961 INFO:teuthology.orchestra.run.smithi138.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-11T23:25:08.003 INFO:teuthology.orchestra.run.smithi138.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.9 MB/s | 1.2 MB 00:00 2024-09-11T23:25:08.027 INFO:teuthology.orchestra.run.smithi133.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-09-11T23:25:08.028 INFO:teuthology.orchestra.run.smithi138.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-11T23:25:08.061 INFO:teuthology.orchestra.run.smithi133.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-09-11T23:25:08.079 INFO:teuthology.orchestra.run.smithi138.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-11T23:25:08.121 INFO:teuthology.orchestra.run.smithi138.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-11T23:25:08.121 INFO:teuthology.orchestra.run.smithi133.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-11T23:25:08.147 INFO:teuthology.orchestra.run.smithi133.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-11T23:25:08.172 INFO:teuthology.orchestra.run.smithi133.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 407 kB/s | 45 kB 00:00 2024-09-11T23:25:08.239 INFO:teuthology.orchestra.run.smithi133.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-11T23:25:08.264 INFO:teuthology.orchestra.run.smithi133.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 4.8 MB/s | 581 kB 00:00 2024-09-11T23:25:08.290 INFO:teuthology.orchestra.run.smithi133.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 209 kB/s | 24 kB 00:00 2024-09-11T23:25:08.305 INFO:teuthology.orchestra.run.smithi138.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-11T23:25:08.347 INFO:teuthology.orchestra.run.smithi138.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-11T23:25:08.348 INFO:teuthology.orchestra.run.smithi133.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-09-11T23:25:08.373 INFO:teuthology.orchestra.run.smithi138.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.5 MB/s | 107 kB 00:00 2024-09-11T23:25:08.382 INFO:teuthology.orchestra.run.smithi133.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-11T23:25:08.398 INFO:teuthology.orchestra.run.smithi138.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 430 kB/s | 22 kB 00:00 2024-09-11T23:25:08.415 INFO:teuthology.orchestra.run.smithi133.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-11T23:25:08.423 INFO:teuthology.orchestra.run.smithi138.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-11T23:25:08.448 INFO:teuthology.orchestra.run.smithi138.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-11T23:25:08.457 INFO:teuthology.orchestra.run.smithi133.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 353 kB/s | 38 kB 00:00 2024-09-11T23:25:08.490 INFO:teuthology.orchestra.run.smithi138.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-09-11T23:25:08.533 INFO:teuthology.orchestra.run.smithi138.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-09-11T23:25:08.591 INFO:teuthology.orchestra.run.smithi138.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-09-11T23:25:08.599 INFO:teuthology.orchestra.run.smithi133.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-11T23:25:08.617 INFO:teuthology.orchestra.run.smithi138.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-11T23:25:08.642 INFO:teuthology.orchestra.run.smithi138.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-09-11T23:25:08.684 INFO:teuthology.orchestra.run.smithi138.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-11T23:25:08.714 INFO:teuthology.orchestra.run.smithi178.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 5.6 MB/s | 11 MB 00:01 2024-09-11T23:25:08.717 INFO:teuthology.orchestra.run.smithi138.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-11T23:25:08.743 INFO:teuthology.orchestra.run.smithi138.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-11T23:25:08.768 INFO:teuthology.orchestra.run.smithi138.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-11T23:25:08.774 INFO:teuthology.orchestra.run.smithi133.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-11T23:25:08.793 INFO:teuthology.orchestra.run.smithi138.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-11T23:25:08.819 INFO:teuthology.orchestra.run.smithi138.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T23:25:08.841 INFO:teuthology.orchestra.run.smithi133.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 11 MB/s | 4.9 MB 00:00 2024-09-11T23:25:08.844 INFO:teuthology.orchestra.run.smithi138.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-11T23:25:08.867 INFO:teuthology.orchestra.run.smithi133.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 3.3 MB/s | 910 kB 00:00 2024-09-11T23:25:08.892 INFO:teuthology.orchestra.run.smithi133.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-11T23:25:08.964 INFO:teuthology.orchestra.run.smithi178.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 913 kB/s | 228 kB 00:00 2024-09-11T23:25:09.001 INFO:teuthology.orchestra.run.smithi133.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-09-11T23:25:09.026 INFO:teuthology.orchestra.run.smithi133.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 655 kB/s | 87 kB 00:00 2024-09-11T23:25:09.101 INFO:teuthology.orchestra.run.smithi133.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-11T23:25:09.143 INFO:teuthology.orchestra.run.smithi133.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 523 kB/s | 61 kB 00:00 2024-09-11T23:25:09.218 INFO:teuthology.orchestra.run.smithi133.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-11T23:25:09.294 INFO:teuthology.orchestra.run.smithi133.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-09-11T23:25:09.303 INFO:teuthology.orchestra.run.smithi138.stdout:(66/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 12 MB/s | 49 MB 00:04 2024-09-11T23:25:09.319 INFO:teuthology.orchestra.run.smithi133.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-09-11T23:25:09.344 INFO:teuthology.orchestra.run.smithi133.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 204 kB/s | 26 kB 00:00 2024-09-11T23:25:09.378 INFO:teuthology.orchestra.run.smithi138.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 8.8 MB/s | 4.9 MB 00:00 2024-09-11T23:25:09.403 INFO:teuthology.orchestra.run.smithi133.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-11T23:25:09.406 INFO:teuthology.orchestra.run.smithi178.stdout:(12/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 28 MB/s | 49 MB 00:01 2024-09-11T23:25:09.431 INFO:teuthology.orchestra.run.smithi178.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 58 kB/s | 27 kB 00:00 2024-09-11T23:25:09.436 INFO:teuthology.orchestra.run.smithi133.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-11T23:25:09.446 INFO:teuthology.orchestra.run.smithi138.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 7.4 MB/s | 4.4 MB 00:00 2024-09-11T23:25:09.457 INFO:teuthology.orchestra.run.smithi178.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 636 kB/s | 32 kB 00:00 2024-09-11T23:25:09.461 INFO:teuthology.orchestra.run.smithi133.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 296 kB/s | 35 kB 00:00 2024-09-11T23:25:09.479 INFO:teuthology.orchestra.run.smithi138.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-11T23:25:09.496 INFO:teuthology.orchestra.run.smithi138.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.7 MB/s | 137 kB 00:00 2024-09-11T23:25:09.516 INFO:teuthology.orchestra.run.smithi178.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-11T23:25:09.553 INFO:teuthology.orchestra.run.smithi133.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-11T23:25:09.565 INFO:teuthology.orchestra.run.smithi178.stdout:(16/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 5.5 MB/s | 752 kB 00:00 2024-09-11T23:25:09.587 INFO:teuthology.orchestra.run.smithi133.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-09-11T23:25:09.590 INFO:teuthology.orchestra.run.smithi178.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-11T23:25:09.605 INFO:teuthology.orchestra.run.smithi138.stdout:(71/156): postgresql-server-13.16-1.el9.x86_64. 46 MB/s | 5.8 MB 00:00 2024-09-11T23:25:09.638 INFO:teuthology.orchestra.run.smithi138.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-09-11T23:25:09.641 INFO:teuthology.orchestra.run.smithi178.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 6.6 MB/s | 506 kB 00:00 2024-09-11T23:25:09.662 INFO:teuthology.orchestra.run.smithi133.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-09-11T23:25:09.672 INFO:teuthology.orchestra.run.smithi138.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-11T23:25:09.687 INFO:teuthology.orchestra.run.smithi133.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 576 kB/s | 77 kB 00:00 2024-09-11T23:25:09.708 INFO:teuthology.orchestra.run.smithi178.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 678 kB/s | 45 kB 00:00 2024-09-11T23:25:09.713 INFO:teuthology.orchestra.run.smithi133.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 336 kB/s | 42 kB 00:00 2024-09-11T23:25:09.714 INFO:teuthology.orchestra.run.smithi138.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-11T23:25:09.765 INFO:teuthology.orchestra.run.smithi138.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-11T23:25:09.780 INFO:teuthology.orchestra.run.smithi133.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-11T23:25:09.791 INFO:teuthology.orchestra.run.smithi178.stdout:(20/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 27 MB/s | 5.3 MB 00:00 2024-09-11T23:25:09.805 INFO:teuthology.orchestra.run.smithi133.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-11T23:25:09.815 INFO:teuthology.orchestra.run.smithi138.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-11T23:25:09.816 INFO:teuthology.orchestra.run.smithi178.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 1.2 MB/s | 136 kB 00:00 2024-09-11T23:25:09.830 INFO:teuthology.orchestra.run.smithi133.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-11T23:25:09.842 INFO:teuthology.orchestra.run.smithi178.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-11T23:25:09.865 INFO:teuthology.orchestra.run.smithi138.stdout:(77/156): postgresql-13.16-1.el9.x86_64.rpm 2.8 MB/s | 1.6 MB 00:00 2024-09-11T23:25:09.867 INFO:teuthology.orchestra.run.smithi178.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.3 MB/s | 321 kB 00:00 2024-09-11T23:25:09.881 INFO:teuthology.orchestra.run.smithi133.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 507 kB/s | 51 kB 00:00 2024-09-11T23:25:09.890 INFO:teuthology.orchestra.run.smithi138.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 340 kB/s | 26 kB 00:00 2024-09-11T23:25:09.892 INFO:teuthology.orchestra.run.smithi178.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 301 kB 00:00 2024-09-11T23:25:09.906 INFO:teuthology.orchestra.run.smithi133.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-09-11T23:25:09.916 INFO:teuthology.orchestra.run.smithi138.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-11T23:25:09.918 INFO:teuthology.orchestra.run.smithi178.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-11T23:25:09.941 INFO:teuthology.orchestra.run.smithi138.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-11T23:25:09.943 INFO:teuthology.orchestra.run.smithi178.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-11T23:25:09.966 INFO:teuthology.orchestra.run.smithi138.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-11T23:25:09.990 INFO:teuthology.orchestra.run.smithi133.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-11T23:25:09.993 INFO:teuthology.orchestra.run.smithi178.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-11T23:25:10.006 INFO:teuthology.orchestra.run.smithi133.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 442 kB/s | 44 kB 00:00 2024-09-11T23:25:10.042 INFO:teuthology.orchestra.run.smithi138.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-11T23:25:10.043 INFO:teuthology.orchestra.run.smithi178.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 638 kB/s | 32 kB 00:00 2024-09-11T23:25:10.068 INFO:teuthology.orchestra.run.smithi138.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-11T23:25:10.093 INFO:teuthology.orchestra.run.smithi138.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-11T23:25:10.107 INFO:teuthology.orchestra.run.smithi133.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-11T23:25:10.110 INFO:teuthology.orchestra.run.smithi138.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-09-11T23:25:10.124 INFO:teuthology.orchestra.run.smithi133.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 307 kB/s | 36 kB 00:00 2024-09-11T23:25:10.135 INFO:teuthology.orchestra.run.smithi138.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-11T23:25:10.210 INFO:teuthology.orchestra.run.smithi138.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-09-11T23:25:10.211 INFO:teuthology.orchestra.run.smithi178.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 878 kB/s | 146 kB 00:00 2024-09-11T23:25:10.224 INFO:teuthology.orchestra.run.smithi133.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 406 kB/s | 47 kB 00:00 2024-09-11T23:25:10.236 INFO:teuthology.orchestra.run.smithi138.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-09-11T23:25:10.258 INFO:teuthology.orchestra.run.smithi133.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-11T23:25:10.261 INFO:teuthology.orchestra.run.smithi138.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-09-11T23:25:10.286 INFO:teuthology.orchestra.run.smithi178.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 8.4 MB/s | 3.1 MB 00:00 2024-09-11T23:25:10.287 INFO:teuthology.orchestra.run.smithi138.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-11T23:25:10.312 INFO:teuthology.orchestra.run.smithi138.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-11T23:25:10.337 INFO:teuthology.orchestra.run.smithi138.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-09-11T23:25:10.341 INFO:teuthology.orchestra.run.smithi133.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 494 kB/s | 58 kB 00:00 2024-09-11T23:25:10.361 INFO:teuthology.orchestra.run.smithi178.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 24 MB/s | 3.6 MB 00:00 2024-09-11T23:25:10.366 INFO:teuthology.orchestra.run.smithi133.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 883 kB/s | 96 kB 00:00 2024-09-11T23:25:10.370 INFO:teuthology.orchestra.run.smithi138.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-09-11T23:25:10.396 INFO:teuthology.orchestra.run.smithi138.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 754 kB/s | 44 kB 00:00 2024-09-11T23:25:10.411 INFO:teuthology.orchestra.run.smithi178.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-11T23:25:10.421 INFO:teuthology.orchestra.run.smithi138.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-11T23:25:10.447 INFO:teuthology.orchestra.run.smithi138.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-09-11T23:25:10.462 INFO:teuthology.orchestra.run.smithi178.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 989 kB/s | 49 kB 00:00 2024-09-11T23:25:10.472 INFO:teuthology.orchestra.run.smithi138.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 939 kB/s | 47 kB 00:00 2024-09-11T23:25:10.512 INFO:teuthology.orchestra.run.smithi178.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 330 kB/s | 16 kB 00:00 2024-09-11T23:25:10.514 INFO:teuthology.orchestra.run.smithi138.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-11T23:25:10.534 INFO:teuthology.orchestra.run.smithi133.stdout:(101/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-11T23:25:10.539 INFO:teuthology.orchestra.run.smithi138.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-11T23:25:10.559 INFO:teuthology.orchestra.run.smithi133.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 233 kB/s | 45 kB 00:00 2024-09-11T23:25:10.562 INFO:teuthology.orchestra.run.smithi178.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.6 MB/s | 285 kB 00:00 2024-09-11T23:25:10.565 INFO:teuthology.orchestra.run.smithi138.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-11T23:25:10.584 INFO:teuthology.orchestra.run.smithi133.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 263 kB/s | 64 kB 00:00 2024-09-11T23:25:10.598 INFO:teuthology.orchestra.run.smithi138.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-11T23:25:10.624 INFO:teuthology.orchestra.run.smithi138.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 763 kB/s | 45 kB 00:00 2024-09-11T23:25:10.629 INFO:teuthology.orchestra.run.smithi178.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 11 MB/s | 765 kB 00:00 2024-09-11T23:25:10.634 INFO:teuthology.orchestra.run.smithi133.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-11T23:25:10.668 INFO:teuthology.orchestra.run.smithi133.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-11T23:25:10.679 INFO:teuthology.orchestra.run.smithi178.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 366 kB/s | 18 kB 00:00 2024-09-11T23:25:10.685 INFO:teuthology.orchestra.run.smithi133.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-11T23:25:10.785 INFO:teuthology.orchestra.run.smithi133.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-11T23:25:10.819 INFO:teuthology.orchestra.run.smithi133.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.3 MB/s | 26 kB 00:00 2024-09-11T23:25:10.835 INFO:teuthology.orchestra.run.smithi133.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-11T23:25:10.861 INFO:teuthology.orchestra.run.smithi133.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T23:25:10.886 INFO:teuthology.orchestra.run.smithi133.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-11T23:25:10.919 INFO:teuthology.orchestra.run.smithi133.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 26 MB/s | 839 kB 00:00 2024-09-11T23:25:10.930 INFO:teuthology.orchestra.run.smithi178.stdout:(38/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 3.2 MB/s | 21 MB 00:06 2024-09-11T23:25:10.933 INFO:teuthology.orchestra.run.smithi138.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-11T23:25:10.953 INFO:teuthology.orchestra.run.smithi133.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-11T23:25:10.958 INFO:teuthology.orchestra.run.smithi138.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 62 kB/s | 22 kB 00:00 2024-09-11T23:25:10.978 INFO:teuthology.orchestra.run.smithi133.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T23:25:11.004 INFO:teuthology.orchestra.run.smithi133.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T23:25:11.008 INFO:teuthology.orchestra.run.smithi138.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-09-11T23:25:11.025 INFO:teuthology.orchestra.run.smithi138.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-11T23:25:11.029 INFO:teuthology.orchestra.run.smithi133.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-11T23:25:11.051 INFO:teuthology.orchestra.run.smithi138.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-09-11T23:25:11.054 INFO:teuthology.orchestra.run.smithi133.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-11T23:25:11.068 INFO:teuthology.orchestra.run.smithi138.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-11T23:25:11.084 INFO:teuthology.orchestra.run.smithi138.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-11T23:25:11.088 INFO:teuthology.orchestra.run.smithi133.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 10 MB/s | 358 kB 00:00 2024-09-11T23:25:11.110 INFO:teuthology.orchestra.run.smithi138.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T23:25:11.113 INFO:teuthology.orchestra.run.smithi133.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-09-11T23:25:11.130 INFO:teuthology.orchestra.run.smithi178.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 8.8 MB/s | 7.4 MB 00:00 2024-09-11T23:25:11.135 INFO:teuthology.orchestra.run.smithi138.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-11T23:25:11.147 INFO:teuthology.orchestra.run.smithi133.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-11T23:25:11.172 INFO:teuthology.orchestra.run.smithi133.stdout:(121/156): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-09-11T23:25:11.177 INFO:teuthology.orchestra.run.smithi138.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-11T23:25:11.219 INFO:teuthology.orchestra.run.smithi138.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-11T23:25:11.239 INFO:teuthology.orchestra.run.smithi133.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-09-11T23:25:11.244 INFO:teuthology.orchestra.run.smithi138.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T23:25:11.264 INFO:teuthology.orchestra.run.smithi133.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-11T23:25:11.270 INFO:teuthology.orchestra.run.smithi138.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T23:25:11.295 INFO:teuthology.orchestra.run.smithi138.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 561 kB/s | 14 kB 00:00 2024-09-11T23:25:11.307 INFO:teuthology.orchestra.run.smithi133.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-09-11T23:25:11.332 INFO:teuthology.orchestra.run.smithi133.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 263 kB/s | 18 kB 00:00 2024-09-11T23:25:11.354 INFO:teuthology.orchestra.run.smithi138.stdout:(117/156): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-11T23:25:11.357 INFO:teuthology.orchestra.run.smithi133.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-11T23:25:11.379 INFO:teuthology.orchestra.run.smithi138.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-11T23:25:11.382 INFO:teuthology.orchestra.run.smithi133.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-11T23:25:11.408 INFO:teuthology.orchestra.run.smithi133.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-11T23:25:11.413 INFO:teuthology.orchestra.run.smithi138.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-11T23:25:11.433 INFO:teuthology.orchestra.run.smithi133.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T23:25:11.447 INFO:teuthology.orchestra.run.smithi138.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-11T23:25:11.458 INFO:teuthology.orchestra.run.smithi133.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T23:25:11.472 INFO:teuthology.orchestra.run.smithi138.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.6 MB/s | 220 kB 00:00 2024-09-11T23:25:11.483 INFO:teuthology.orchestra.run.smithi133.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-11T23:25:11.506 INFO:teuthology.orchestra.run.smithi138.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-11T23:25:11.508 INFO:teuthology.orchestra.run.smithi133.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-09-11T23:25:11.548 INFO:teuthology.orchestra.run.smithi138.stdout:(123/156): protobuf-compiler-3.14.0-13.el9.x86_ 934 kB/s | 863 kB 00:00 2024-09-11T23:25:11.551 INFO:teuthology.orchestra.run.smithi133.stdout:(133/156): protobuf-compiler-3.14.0-13.el9.x86_ 871 kB/s | 863 kB 00:00 2024-09-11T23:25:11.574 INFO:teuthology.orchestra.run.smithi138.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 159 kB/s | 11 kB 00:00 2024-09-11T23:25:11.576 INFO:teuthology.orchestra.run.smithi133.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-09-11T23:25:11.597 INFO:teuthology.orchestra.run.smithi178.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 87 kB/s | 80 kB 00:00 2024-09-11T23:25:11.609 INFO:teuthology.orchestra.run.smithi133.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.0 MB/s | 28 kB 00:00 2024-09-11T23:25:11.615 INFO:teuthology.orchestra.run.smithi138.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 689 kB/s | 23 kB 00:00 2024-09-11T23:25:11.641 INFO:teuthology.orchestra.run.smithi138.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 837 kB/s | 19 kB 00:00 2024-09-11T23:25:11.651 INFO:teuthology.orchestra.run.smithi133.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 3.8 MB/s | 1.0 MB 00:00 2024-09-11T23:25:11.666 INFO:teuthology.orchestra.run.smithi138.stdout:(127/156): python3-jaraco-classes-3.2.1-5.el9.n 150 kB/s | 18 kB 00:00 2024-09-11T23:25:11.676 INFO:teuthology.orchestra.run.smithi133.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 472 kB/s | 31 kB 00:00 2024-09-11T23:25:11.691 INFO:teuthology.orchestra.run.smithi138.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-11T23:25:11.702 INFO:teuthology.orchestra.run.smithi133.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 596 kB/s | 90 kB 00:00 2024-09-11T23:25:11.717 INFO:teuthology.orchestra.run.smithi138.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T23:25:11.727 INFO:teuthology.orchestra.run.smithi133.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-11T23:25:11.742 INFO:teuthology.orchestra.run.smithi138.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T23:25:11.752 INFO:teuthology.orchestra.run.smithi133.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-11T23:25:11.767 INFO:teuthology.orchestra.run.smithi138.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-11T23:25:11.778 INFO:teuthology.orchestra.run.smithi133.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-11T23:25:11.803 INFO:teuthology.orchestra.run.smithi133.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-11T23:25:11.809 INFO:teuthology.orchestra.run.smithi138.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.6 MB/s | 2.0 MB 00:00 2024-09-11T23:25:11.836 INFO:teuthology.orchestra.run.smithi133.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-11T23:25:11.843 INFO:teuthology.orchestra.run.smithi138.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-09-11T23:25:11.860 INFO:teuthology.orchestra.run.smithi138.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-11T23:25:11.862 INFO:teuthology.orchestra.run.smithi133.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-11T23:25:11.872 INFO:teuthology.orchestra.run.smithi178.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 851 kB/s | 801 kB 00:00 2024-09-11T23:25:11.885 INFO:teuthology.orchestra.run.smithi138.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-09-11T23:25:11.887 INFO:teuthology.orchestra.run.smithi133.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-11T23:25:11.897 INFO:teuthology.orchestra.run.smithi178.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 252 kB/s | 191 kB 00:00 2024-09-11T23:25:11.911 INFO:teuthology.orchestra.run.smithi138.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-09-11T23:25:11.912 INFO:teuthology.orchestra.run.smithi133.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T23:25:11.936 INFO:teuthology.orchestra.run.smithi138.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-11T23:25:11.937 INFO:teuthology.orchestra.run.smithi133.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-11T23:25:11.961 INFO:teuthology.orchestra.run.smithi138.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 892 kB/s | 90 kB 00:00 2024-09-11T23:25:11.978 INFO:teuthology.orchestra.run.smithi138.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-11T23:25:11.995 INFO:teuthology.orchestra.run.smithi138.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-11T23:25:11.997 INFO:teuthology.orchestra.run.smithi133.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-11T23:25:12.021 INFO:teuthology.orchestra.run.smithi138.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-11T23:25:12.046 INFO:teuthology.orchestra.run.smithi138.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-11T23:25:12.071 INFO:teuthology.orchestra.run.smithi138.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T23:25:12.097 INFO:teuthology.orchestra.run.smithi138.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-11T23:25:12.122 INFO:teuthology.orchestra.run.smithi138.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-09-11T23:25:12.148 INFO:teuthology.orchestra.run.smithi138.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T23:25:12.173 INFO:teuthology.orchestra.run.smithi138.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-11T23:25:12.222 INFO:teuthology.orchestra.run.smithi133.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-11T23:25:12.223 INFO:teuthology.orchestra.run.smithi138.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-11T23:25:12.281 INFO:teuthology.orchestra.run.smithi133.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 9.2 MB/s | 3.2 MB 00:00 2024-09-11T23:25:12.340 INFO:teuthology.orchestra.run.smithi138.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 20 MB/s | 2.3 MB 00:00 2024-09-11T23:25:12.372 INFO:teuthology.orchestra.run.smithi178.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 327 kB/s | 253 kB 00:00 2024-09-11T23:25:12.389 INFO:teuthology.orchestra.run.smithi133.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 377 kB/s | 41 kB 00:00 2024-09-11T23:25:12.431 INFO:teuthology.orchestra.run.smithi133.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 5.3 MB/s | 2.3 MB 00:00 2024-09-11T23:25:12.441 INFO:teuthology.orchestra.run.smithi138.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-11T23:25:12.483 INFO:teuthology.orchestra.run.smithi138.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 13 MB/s | 1.8 MB 00:00 2024-09-11T23:25:12.508 INFO:teuthology.orchestra.run.smithi138.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 610 kB/s | 41 kB 00:00 2024-09-11T23:25:12.564 INFO:teuthology.orchestra.run.smithi133.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-09-11T23:25:12.567 INFO:teuthology.orchestra.run.smithi138.stdout:(153/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.0 MB/s | 3.2 MB 00:00 2024-09-11T23:25:12.592 INFO:teuthology.orchestra.run.smithi138.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 2.2 MB/s | 249 kB 00:00 2024-09-11T23:25:12.631 INFO:teuthology.orchestra.run.smithi178.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 145 kB/s | 106 kB 00:00 2024-09-11T23:25:12.634 INFO:teuthology.orchestra.run.smithi138.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-11T23:25:12.681 INFO:teuthology.orchestra.run.smithi133.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 851 kB/s | 249 kB 00:00 2024-09-11T23:25:12.732 INFO:teuthology.orchestra.run.smithi133.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-11T23:25:12.773 INFO:teuthology.orchestra.run.smithi133.stdout:(156/156): NetworkManager-libnm-1.49.90-1.el9.x 3.3 MB/s | 1.8 MB 00:00 2024-09-11T23:25:12.776 INFO:teuthology.orchestra.run.smithi138.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-09-11T23:25:12.779 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:25:12.779 INFO:teuthology.orchestra.run.smithi133.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-11T23:25:12.782 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:25:12.782 INFO:teuthology.orchestra.run.smithi138.stdout:Total 23 MB/s | 226 MB 00:09 2024-09-11T23:25:12.892 INFO:teuthology.orchestra.run.smithi178.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-09-11T23:25:13.124 INFO:teuthology.orchestra.run.smithi178.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 180 kB/s | 135 kB 00:00 2024-09-11T23:25:13.382 INFO:teuthology.orchestra.run.smithi178.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-09-11T23:25:13.416 INFO:teuthology.orchestra.run.smithi178.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 159 kB/s | 125 kB 00:00 2024-09-11T23:25:13.566 INFO:teuthology.orchestra.run.smithi178.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 713 kB/s | 107 kB 00:00 2024-09-11T23:25:13.591 INFO:teuthology.orchestra.run.smithi178.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-09-11T23:25:13.617 INFO:teuthology.orchestra.run.smithi178.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-11T23:25:13.667 INFO:teuthology.orchestra.run.smithi178.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-09-11T23:25:13.692 INFO:teuthology.orchestra.run.smithi178.stdout:(53/156): python3-urllib3-1.26.5-6.el9.noarch.r 270 kB/s | 215 kB 00:00 2024-09-11T23:25:13.718 INFO:teuthology.orchestra.run.smithi178.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-11T23:25:13.768 INFO:teuthology.orchestra.run.smithi178.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-11T23:25:13.793 INFO:teuthology.orchestra.run.smithi178.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-09-11T23:25:13.818 INFO:teuthology.orchestra.run.smithi178.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-11T23:25:13.877 INFO:teuthology.orchestra.run.smithi178.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-09-11T23:25:13.961 INFO:teuthology.orchestra.run.smithi178.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-09-11T23:25:14.007 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-09-11T23:25:14.011 INFO:teuthology.orchestra.run.smithi178.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 491 kB/s | 24 kB 00:00 2024-09-11T23:25:14.020 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-11T23:25:14.061 INFO:teuthology.orchestra.run.smithi178.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-11T23:25:14.095 INFO:teuthology.orchestra.run.smithi178.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-11T23:25:14.153 INFO:teuthology.orchestra.run.smithi178.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-11T23:25:14.157 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-09-11T23:25:14.157 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-09-11T23:25:14.169 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-11T23:25:14.169 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-11T23:25:14.178 INFO:teuthology.orchestra.run.smithi178.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-11T23:25:14.204 INFO:teuthology.orchestra.run.smithi178.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-11T23:25:14.288 INFO:teuthology.orchestra.run.smithi178.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.8 MB/s | 3.0 MB 00:00 2024-09-11T23:25:14.596 INFO:teuthology.orchestra.run.smithi178.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-09-11T23:25:14.813 INFO:teuthology.orchestra.run.smithi178.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.1 MB/s | 910 kB 00:00 2024-09-11T23:25:14.871 INFO:teuthology.orchestra.run.smithi178.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-11T23:25:14.955 INFO:teuthology.orchestra.run.smithi178.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-09-11T23:25:15.130 INFO:teuthology.orchestra.run.smithi178.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-09-11T23:25:15.180 INFO:teuthology.orchestra.run.smithi178.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-11T23:25:15.247 INFO:teuthology.orchestra.run.smithi178.stdout:(73/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.6 MB/s | 4.9 MB 00:01 2024-09-11T23:25:15.289 INFO:teuthology.orchestra.run.smithi178.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-11T23:25:15.373 INFO:teuthology.orchestra.run.smithi178.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-11T23:25:15.456 INFO:teuthology.orchestra.run.smithi178.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-11T23:25:15.507 INFO:teuthology.orchestra.run.smithi178.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-11T23:25:15.557 INFO:teuthology.orchestra.run.smithi178.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-11T23:25:15.641 INFO:teuthology.orchestra.run.smithi178.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-11T23:25:15.691 INFO:teuthology.orchestra.run.smithi178.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-11T23:25:15.976 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-11T23:25:15.976 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-11T23:25:16.033 INFO:teuthology.orchestra.run.smithi178.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 7.0 MB/s | 6.0 MB 00:00 2024-09-11T23:25:16.100 INFO:teuthology.orchestra.run.smithi178.stdout:(82/156): postgresql-server-13.16-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-09-11T23:25:16.125 INFO:teuthology.orchestra.run.smithi178.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-11T23:25:16.183 INFO:teuthology.orchestra.run.smithi178.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-11T23:25:16.209 INFO:teuthology.orchestra.run.smithi178.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-11T23:25:16.234 INFO:teuthology.orchestra.run.smithi178.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-11T23:25:16.293 INFO:teuthology.orchestra.run.smithi178.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-11T23:25:16.318 INFO:teuthology.orchestra.run.smithi178.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-11T23:25:16.368 INFO:teuthology.orchestra.run.smithi178.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-11T23:25:16.374 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-09-11T23:25:16.375 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-09-11T23:25:16.393 INFO:teuthology.orchestra.run.smithi178.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-11T23:25:16.419 INFO:teuthology.orchestra.run.smithi178.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-11T23:25:16.469 INFO:teuthology.orchestra.run.smithi178.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-09-11T23:25:16.519 INFO:teuthology.orchestra.run.smithi178.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-11T23:25:16.578 INFO:teuthology.orchestra.run.smithi178.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-11T23:25:16.628 INFO:teuthology.orchestra.run.smithi178.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-09-11T23:25:16.678 INFO:teuthology.orchestra.run.smithi178.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-11T23:25:16.804 INFO:teuthology.orchestra.run.smithi178.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.6 MB/s | 5.1 MB 00:01 2024-09-11T23:25:16.829 INFO:teuthology.orchestra.run.smithi178.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.0 MB/s | 304 kB 00:00 2024-09-11T23:25:16.854 INFO:teuthology.orchestra.run.smithi178.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T23:25:16.879 INFO:teuthology.orchestra.run.smithi178.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-11T23:25:16.905 INFO:teuthology.orchestra.run.smithi178.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T23:25:16.930 INFO:teuthology.orchestra.run.smithi178.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-11T23:25:17.472 INFO:teuthology.orchestra.run.smithi178.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-09-11T23:25:17.539 INFO:teuthology.orchestra.run.smithi178.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-09-11T23:25:17.573 INFO:teuthology.orchestra.run.smithi178.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-11T23:25:17.599 INFO:teuthology.orchestra.run.smithi178.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-11T23:25:17.715 INFO:teuthology.orchestra.run.smithi178.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-11T23:25:17.740 INFO:teuthology.orchestra.run.smithi178.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-11T23:25:17.765 INFO:teuthology.orchestra.run.smithi178.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-11T23:25:17.791 INFO:teuthology.orchestra.run.smithi178.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T23:25:17.816 INFO:teuthology.orchestra.run.smithi178.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-11T23:25:17.849 INFO:teuthology.orchestra.run.smithi178.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 939 kB/s | 863 kB 00:00 2024-09-11T23:25:17.891 INFO:teuthology.orchestra.run.smithi178.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-11T23:25:17.917 INFO:teuthology.orchestra.run.smithi178.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-09-11T23:25:17.942 INFO:teuthology.orchestra.run.smithi178.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T23:25:17.967 INFO:teuthology.orchestra.run.smithi178.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-09-11T23:25:17.993 INFO:teuthology.orchestra.run.smithi178.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.7 MB/s | 548 kB 00:00 2024-09-11T23:25:18.018 INFO:teuthology.orchestra.run.smithi178.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T23:25:18.051 INFO:teuthology.orchestra.run.smithi178.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-11T23:25:18.077 INFO:teuthology.orchestra.run.smithi178.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-11T23:25:18.110 INFO:teuthology.orchestra.run.smithi178.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-11T23:25:18.144 INFO:teuthology.orchestra.run.smithi178.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-11T23:25:18.169 INFO:teuthology.orchestra.run.smithi178.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-11T23:25:18.219 INFO:teuthology.orchestra.run.smithi178.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-09-11T23:25:18.236 INFO:teuthology.orchestra.run.smithi178.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-09-11T23:25:18.253 INFO:teuthology.orchestra.run.smithi178.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 689 kB/s | 23 kB 00:00 2024-09-11T23:25:18.270 INFO:teuthology.orchestra.run.smithi178.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 582 kB/s | 19 kB 00:00 2024-09-11T23:25:18.287 INFO:teuthology.orchestra.run.smithi178.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 590 kB/s | 20 kB 00:00 2024-09-11T23:25:18.313 INFO:teuthology.orchestra.run.smithi178.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T23:25:18.338 INFO:teuthology.orchestra.run.smithi178.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T23:25:18.355 INFO:teuthology.orchestra.run.smithi178.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-11T23:25:18.388 INFO:teuthology.orchestra.run.smithi178.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-11T23:25:18.422 INFO:teuthology.orchestra.run.smithi178.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 6.7 MB/s | 1.0 MB 00:00 2024-09-11T23:25:18.447 INFO:teuthology.orchestra.run.smithi178.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-09-11T23:25:18.473 INFO:teuthology.orchestra.run.smithi178.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-11T23:25:18.498 INFO:teuthology.orchestra.run.smithi178.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-11T23:25:18.523 INFO:teuthology.orchestra.run.smithi178.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-11T23:25:18.549 INFO:teuthology.orchestra.run.smithi178.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-11T23:25:18.574 INFO:teuthology.orchestra.run.smithi178.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-11T23:25:18.608 INFO:teuthology.orchestra.run.smithi178.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 607 kB/s | 36 kB 00:00 2024-09-11T23:25:18.633 INFO:teuthology.orchestra.run.smithi178.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-11T23:25:18.658 INFO:teuthology.orchestra.run.smithi178.stdout:(142/156): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-11T23:25:18.683 INFO:teuthology.orchestra.run.smithi178.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T23:25:18.709 INFO:teuthology.orchestra.run.smithi178.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-11T23:25:18.734 INFO:teuthology.orchestra.run.smithi178.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-11T23:25:18.893 INFO:teuthology.orchestra.run.smithi178.stdout:(146/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.5 MB/s | 16 MB 00:02 2024-09-11T23:25:18.918 INFO:teuthology.orchestra.run.smithi178.stdout:(147/156): re2-20211101-20.el9.x86_64.rpm 917 kB/s | 191 kB 00:00 2024-09-11T23:25:18.968 INFO:teuthology.orchestra.run.smithi178.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-11T23:25:19.952 INFO:teuthology.orchestra.run.smithi178.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 3.1 MB/s | 3.2 MB 00:01 2024-09-11T23:25:20.118 INFO:teuthology.orchestra.run.smithi178.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 2.8 MB/s | 3.4 MB 00:01 2024-09-11T23:25:20.160 INFO:teuthology.orchestra.run.smithi178.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 1.9 MB/s | 2.3 MB 00:01 2024-09-11T23:25:20.744 INFO:teuthology.orchestra.run.smithi178.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 65 kB/s | 41 kB 00:00 2024-09-11T23:25:20.952 INFO:teuthology.orchestra.run.smithi178.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-09-11T23:25:20.986 INFO:teuthology.orchestra.run.smithi178.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 301 kB/s | 249 kB 00:00 2024-09-11T23:25:21.803 INFO:teuthology.orchestra.run.smithi178.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.3 MB/s | 1.4 MB 00:01 2024-09-11T23:25:22.003 INFO:teuthology.orchestra.run.smithi178.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-11T23:25:22.009 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:25:22.010 INFO:teuthology.orchestra.run.smithi178.stdout:Total 12 MB/s | 226 MB 00:18 2024-09-11T23:25:22.334 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-11T23:25:22.438 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:25:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:25:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:25:22.581 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:25:22.607 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:25:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:25:22.696 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-09-11T23:25:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:25:22.828 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:25:22.893 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:25:22.944 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:25:22.969 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:25:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:25:23.021 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:25:23.100 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:25:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:25:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:25:23.238 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-11T23:25:23.326 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:25:23.356 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:25:23.363 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:25:23.387 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-11T23:25:23.387 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-11T23:25:23.479 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:25:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:25:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:25:23.581 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:25:23.601 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:25:23.646 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:25:23.686 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:25:23.708 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:25:23.739 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:25:23.770 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:25:23.776 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:25:23.792 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:25:23.819 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:25:23.937 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:25:23.997 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:25:24.011 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:25:24.034 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:25:24.121 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:25:24.145 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:25:24.230 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:25:24.260 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:25:24.280 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:25:24.288 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:25:24.390 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:25:24.481 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:25:24.486 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:25:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:25:24.715 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:25:24.771 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:25:24.829 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:25:24.897 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:25:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:25:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:25:24.985 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:25:24.994 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:25:25.067 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:25:25.112 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:25:25.144 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:25:25.188 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:25:25.220 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:25:25.221 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:25:25.248 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:25:25.301 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-11T23:25:25.301 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-11T23:25:25.322 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:25:25.335 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:25:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:25:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:25:25.428 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:25:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:25:25.462 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:25:25.500 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:25:25.525 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:25:25.560 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:25:25.565 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:25:25.604 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:25:25.617 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:25:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:25:25.667 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:25:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:25:25.709 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:25:25.743 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:25:25.756 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:25:25.770 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:25:25.830 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:25:25.837 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:25:25.891 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:25:25.923 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:25:25.963 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:25:25.973 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:25:26.017 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:25:26.041 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:25:26.085 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:25:26.129 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:25:26.158 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:25:26.227 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:25:26.272 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:25:26.345 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:25:26.486 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:25:26.535 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:25:26.541 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:25:26.614 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:25:26.652 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:25:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:25:26.733 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:25:26.775 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:25:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:25:27.004 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:25:27.059 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:25:27.059 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:25:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:25:27.145 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:25:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:25:27.182 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:25:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:25:27.225 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:25:27.252 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:25:27.263 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:25:27.288 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:25:27.314 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:25:27.331 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:25:27.375 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:25:27.469 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:25:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:25:27.525 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:25:27.560 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:25:27.579 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:25:27.610 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:25:27.652 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:25:27.684 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:25:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:25:27.720 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:25:27.744 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:25:27.766 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:25:27.819 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:25:27.871 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:25:27.884 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:25:27.921 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:25:28.056 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:25:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:25:28.098 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:25:28.118 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:25:28.139 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:25:28.196 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:25:28.241 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:25:28.282 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:25:28.320 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:25:28.360 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:25:28.377 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:25:28.388 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:25:28.431 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:25:28.471 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:25:28.502 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:25:28.532 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:25:28.541 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:25:28.567 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:25:28.600 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:25:28.642 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:25:28.677 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:25:28.763 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:25:28.773 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:25:28.816 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:25:28.847 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:25:28.950 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:25:28.962 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:25:28.999 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:25:29.093 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:25:29.129 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:25:29.173 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:25:29.213 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:25:29.259 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:25:29.290 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:25:29.320 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:25:29.365 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:25:29.396 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:25:29.438 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:25:29.472 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:25:29.566 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:25:29.627 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:25:29.659 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:25:29.713 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:25:29.755 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:25:29.761 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:25:29.789 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:25:29.809 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:25:29.889 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:25:29.944 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:25:29.990 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:25:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:25:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:25:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:25:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:25:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:25:30.527 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:25:30.573 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:25:30.615 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:25:30.722 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:25:30.770 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:25:30.815 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:25:30.909 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:25:31.048 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:25:31.159 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:25:31.173 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:25:31.485 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:25:31.524 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:25:31.614 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:25:31.681 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-11T23:25:31.773 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:25:31.811 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:25:31.872 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:25:31.907 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:25:31.932 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:25:31.983 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:25:32.274 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:25:32.313 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:25:32.385 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:25:32.388 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:25:32.433 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:25:32.466 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:25:32.493 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:25:32.528 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:25:32.547 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:25:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:25:32.650 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:25:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:25:32.830 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:25:32.840 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:25:32.882 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:25:32.893 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:25:32.922 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:25:32.982 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:25:33.012 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:25:33.033 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:25:33.048 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:25:33.056 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:25:33.135 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:25:33.157 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:25:33.181 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:25:33.376 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:25:33.501 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:25:33.535 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:25:33.622 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:25:33.652 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:25:33.663 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:25:33.714 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:25:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:25:33.743 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:25:33.754 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:25:33.771 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:25:33.791 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:25:33.804 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:25:34.009 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:25:34.112 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:25:34.114 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:25:34.159 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:25:34.208 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:25:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:25:34.241 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:25:34.263 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:25:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:25:34.308 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:25:34.337 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:25:34.390 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:25:34.422 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:25:34.465 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:25:34.498 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:25:34.537 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:25:34.606 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:25:34.659 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:25:34.690 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:25:34.734 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:25:34.789 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:25:34.811 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:25:34.812 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:25:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:25:34.883 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:25:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:25:34.913 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:25:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:25:34.988 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:25:35.032 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:25:35.109 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:25:35.242 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:25:35.301 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:25:35.838 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:25:35.921 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:25:35.959 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:25:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:25:36.048 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:25:36.071 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:25:36.090 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:25:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:25:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:25:36.309 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:25:36.364 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:25:36.437 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:25:36.469 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:25:36.505 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:25:36.552 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:25:36.587 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:25:36.630 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:25:36.648 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:25:36.673 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:25:36.682 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:25:36.771 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:25:36.771 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:25:36.771 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:25:36.771 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:36.809 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:25:36.817 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:25:36.858 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:25:36.899 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:25:37.002 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:25:37.041 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:25:37.080 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:25:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:25:37.131 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:25:37.206 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:25:37.234 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:25:37.276 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:25:37.521 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:25:37.564 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:25:37.591 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:25:37.626 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:25:37.665 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:25:37.730 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:25:37.759 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:25:37.794 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:25:37.826 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:25:37.827 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:25:37.865 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:25:37.868 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:25:37.903 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:25:37.953 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:25:37.953 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:25:37.953 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:25:37.953 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:37.990 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:25:37.997 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:25:38.035 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:25:38.069 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:25:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:25:38.226 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:25:38.950 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:25:38.997 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:25:39.031 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:25:39.130 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:25:39.195 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:25:39.239 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:25:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:25:39.367 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:25:39.550 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:25:39.581 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:25:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:25:40.310 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:25:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:25:41.642 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:25:41.726 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:25:41.754 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:25:41.787 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:25:42.097 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:25:42.133 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:25:42.241 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:25:42.272 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:25:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:25:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:25:42.842 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:25:42.900 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:25:42.930 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:25:45.201 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:25:45.243 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:25:45.791 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:25:45.838 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:25:45.954 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:25:45.954 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:25:45.954 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:25:45.954 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:25:45.981 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:25:48.997 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:25:48.997 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-09-11T23:25:48.997 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-09-11T23:25:48.997 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-09-11T23:25:48.997 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-09-11T23:25:48.997 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-09-11T23:25:48.997 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-09-11T23:25:48.997 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-09-11T23:25:48.997 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:49.180 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:25:49.197 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:25:49.197 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:49.198 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:25:49.198 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:49.616 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:25:49.616 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-09-11T23:25:49.616 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-09-11T23:25:49.616 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-09-11T23:25:49.616 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-09-11T23:25:49.616 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-09-11T23:25:49.617 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /root 2024-09-11T23:25:49.617 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-09-11T23:25:49.617 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:49.801 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:25:49.817 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:25:49.817 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:49.817 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:25:49.817 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:50.659 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:25:50.686 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:25:50.686 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:50.686 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:25:50.686 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:25:50.686 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:25:50.686 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:50.910 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:25:50.933 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:25:50.934 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:50.934 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:25:50.934 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:25:50.934 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:25:50.934 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:51.312 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:25:51.339 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:25:51.339 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:51.339 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:25:51.339 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:25:51.339 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:25:51.340 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:51.369 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:25:51.393 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:25:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:25:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:25:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:25:51.394 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:51.499 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:25:51.549 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:25:51.571 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:25:51.596 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:25:51.596 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:51.596 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:25:51.596 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:25:51.596 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:25:51.596 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:51.602 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:25:51.787 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:25:51.837 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:25:51.907 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:25:51.995 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:25:52.029 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:25:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:25:52.053 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:52.053 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:25:52.053 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:25:52.053 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:25:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:52.127 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:25:52.151 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:25:52.172 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:25:52.204 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:25:52.232 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:25:52.236 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:25:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:25:52.498 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:25:52.546 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:25:52.617 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:25:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:25:52.705 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:25:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:25:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:25:52.809 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:25:52.838 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:25:52.851 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:25:52.897 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:25:52.928 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:25:52.957 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:25:53.390 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:25:53.422 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:25:53.453 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:25:53.516 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:25:53.553 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:25:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:25:53.916 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:25:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:25:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:25:54.884 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:25:54.916 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:25:55.018 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:25:55.179 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:25:55.203 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:25:55.227 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:25:55.228 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:55.228 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:25:55.228 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:25:55.228 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:25:55.228 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:55.280 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:25:55.291 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:25:55.387 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:25:55.443 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:25:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:25:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:25:55.724 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:25:55.887 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:25:55.916 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:25:55.939 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:25:55.939 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:55.939 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:25:55.939 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:25:55.939 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:25:55.940 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:55.983 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:25:55.994 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:25:56.092 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:25:56.152 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:25:56.365 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:25:56.398 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:25:56.422 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:25:56.422 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:56.422 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:25:56.422 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:25:56.423 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:25:56.423 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:56.466 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:25:56.490 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:25:56.490 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:56.490 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:25:56.490 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:56.541 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:25:56.565 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:25:56.565 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:56.565 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:25:56.565 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:25:56.565 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:25:56.566 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:56.867 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:25:56.891 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:25:56.891 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:56.891 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:25:56.891 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:25:56.891 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:25:56.891 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:25:57.077 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:25:57.110 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:25:57.136 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:25:57.136 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:57.136 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:25:57.136 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:25:57.136 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:25:57.137 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:57.177 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:25:57.200 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:25:57.200 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:57.201 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:25:57.201 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:57.253 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:25:57.276 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:25:57.276 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:57.277 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:25:57.277 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:25:57.277 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:25:57.277 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:57.578 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:25:57.602 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:25:57.602 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:57.602 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:25:57.602 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:25:57.602 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:25:57.603 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:25:57.709 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:25:57.709 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-09-11T23:25:57.709 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-09-11T23:25:57.709 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-09-11T23:25:57.709 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-09-11T23:25:57.709 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-09-11T23:25:57.709 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-09-11T23:25:57.710 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-09-11T23:25:57.710 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:25:57.891 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:25:57.908 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:25:57.908 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:57.909 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:25:57.909 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:25:59.361 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:25:59.389 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:25:59.389 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:59.389 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:25:59.389 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:25:59.389 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:25:59.389 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:25:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:25:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:25:59.636 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:25:59.636 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:25:59.636 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:25:59.636 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:25:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:26:00.071 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:26:00.095 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:26:00.095 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:26:00.095 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:26:00.095 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:26:00.095 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:26:00.096 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:26:00.209 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:26:00.266 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:26:00.322 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:26:00.509 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:26:00.563 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:26:00.633 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:26:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:26:00.888 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:26:00.938 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:26:00.978 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:26:01.006 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:26:01.407 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:26:01.438 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:26:01.447 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:26:01.469 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:26:01.489 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:26:01.529 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:26:01.532 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:26:01.561 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:26:01.569 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:26:01.589 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:26:01.770 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:26:01.812 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:26:01.842 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:26:01.872 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:26:01.910 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:26:02.174 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:26:02.215 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:26:02.264 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:26:02.296 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:26:02.323 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:26:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:26:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:26:02.515 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:26:02.531 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:26:02.561 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:26:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:26:02.614 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:26:02.621 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:26:02.654 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:26:02.667 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:26:03.289 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:26:03.332 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:26:03.352 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:26:03.359 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:26:03.443 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:26:03.484 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:26:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:26:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:26:03.617 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:26:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:26:03.646 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:26:03.657 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:26:03.657 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:26:03.749 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:26:03.749 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:26:03.750 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:26:03.842 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:26:03.867 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:26:03.872 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:26:03.873 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:26:03.932 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:26:03.956 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:26:03.981 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:26:03.981 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:26:03.981 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:26:03.981 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:26:03.981 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:26:03.981 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:26:04.032 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:26:04.044 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:26:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:26:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:26:04.140 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:26:04.145 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:26:04.185 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:26:04.188 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:26:04.208 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:26:04.286 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:26:04.335 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:26:04.421 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:26:04.447 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:26:04.473 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:26:04.507 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:26:04.507 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:26:04.583 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:26:04.583 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:26:04.667 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:26:04.692 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:26:04.698 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:26:04.698 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:26:04.877 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:26:04.877 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:26:04.969 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:26:05.043 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:26:05.130 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:26:05.168 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:26:05.192 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:26:05.192 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:26:05.192 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:26:05.192 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:26:05.192 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:26:05.192 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:26:05.235 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:26:05.258 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:26:05.259 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:26:05.259 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:26:05.259 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:26:05.311 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:26:05.334 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:26:05.335 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:26:05.335 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:26:05.335 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:26:05.335 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:26:05.335 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:26:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:26:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:26:05.660 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:26:05.661 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:26:05.661 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:26:05.661 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:26:05.661 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:26:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:26:10.221 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:26:10.221 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:26:10.221 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:26:10.221 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:26:10.221 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:26:10.221 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:26:10.222 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:26:10.222 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:26:10.222 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:26:10.222 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:26:10.222 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:26:10.222 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:26:10.222 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:26:10.222 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:26:10.222 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:26:10.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:26:10.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:26:10.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:26:10.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:26:10.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:26:10.237 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:26:10.238 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:26:10.239 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:26:10.239 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:26:10.239 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:26:10.239 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:26:10.239 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:26:10.239 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:26:10.239 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:26:10.239 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:26:10.239 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:26:10.239 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:26:10.240 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:26:10.240 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:26:10.240 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:26:10.240 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:26:10.240 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:26:10.240 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:26:10.240 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:26:10.240 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:26:10.240 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:26:10.241 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:26:10.241 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:26:10.241 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:26:10.241 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:26:10.241 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:26:10.241 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:26:10.241 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:26:10.241 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:26:10.241 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:26:10.241 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:26:10.242 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:26:10.243 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:26:10.244 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:26:10.245 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:26:10.246 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:26:10.247 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:26:10.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:26:10.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:26:10.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:26:10.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:26:10.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:26:10.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:26:10.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:26:10.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:26:10.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:26:10.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:26:10.248 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:26:10.249 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:26:10.289 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:26:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:26:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:26:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:26:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:26:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:26:10.637 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:26:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:26:10.759 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:26:10.760 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:26:10.760 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:26:10.760 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:26:10.760 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:26:10.760 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:26:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:26:10.762 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:26:10.762 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:26:10.762 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:26:10.762 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:26:10.762 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:26:10.762 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:26:10.762 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:26:10.762 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:26:10.762 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:26:10.763 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:26:10.763 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:26:10.763 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:26:10.763 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:26:10.763 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:26:10.763 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:26:10.763 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:26:10.763 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:26:10.763 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:26:10.763 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:26:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:26:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:26:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:26:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:26:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:26:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:26:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:26:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:26:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:26:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:26:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:26:10.765 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:26:10.766 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:26:10.767 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:26:10.768 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:26:10.768 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:26:10.768 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:26:10.768 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:26:10.768 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:26:10.768 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:26:10.768 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:26:10.768 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:26:10.768 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:26:10.768 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:26:10.768 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:26:10.769 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:26:10.769 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:26:10.769 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:26:10.769 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:26:10.769 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:26:10.769 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:26:10.769 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:26:10.769 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:26:10.769 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:26:10.770 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:26:10.771 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:26:10.772 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:26:10.773 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:26:10.774 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:26:10.774 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:26:10.774 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:26:11.242 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:26:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:26:11.302 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:26:12.147 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:26:12.236 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:26:12.277 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:26:12.360 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:26:12.389 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:26:12.414 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:26:12.448 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:26:12.448 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:26:12.532 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:26:12.532 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:26:12.616 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:26:12.641 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:26:12.647 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:26:12.647 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:26:12.748 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:26:12.748 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:26:12.748 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-09-11T23:26:12.748 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:12.748 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:12.748 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.749 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.750 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.750 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.750 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:12.750 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.750 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.750 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.751 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:12.751 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:12.751 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:12.751 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:12.751 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:12.751 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.751 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.751 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:12.752 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.752 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.752 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.752 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:12.752 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:12.752 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.753 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.753 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:12.753 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:26:12.753 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:26:12.753 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:26:12.753 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:26:12.753 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:26:12.753 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:26:12.753 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:26:12.753 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:26:12.754 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:26:12.755 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:26:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:26:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:26:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:26:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:26:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:26:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:26:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:26:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:26:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:26:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:26:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:26:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:26:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:26:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:26:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:26:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:26:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:26:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:26:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:26:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:26:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:26:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:26:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:26:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:26:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.763 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:12.763 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:26:12.763 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:26:12.763 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:26:12.763 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:26:12.763 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:26:12.763 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:26:12.764 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:26:12.764 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:26:12.764 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-11T23:26:12.827 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:26:12.827 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:26:12.903 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:26:12.966 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:26:13.158 DEBUG:teuthology.parallel:result is None 2024-09-11T23:26:13.413 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:26:13.413 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:26:13.414 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-09-11T23:26:13.414 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:13.414 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:13.414 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:13.414 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:13.414 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.415 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.415 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:26:13.415 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:26:13.415 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-09-11T23:26:13.415 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:26:13.415 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:26:13.415 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:26:13.416 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:26:13.416 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.416 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.416 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.416 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.416 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:13.416 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.417 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.417 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.417 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:13.417 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:13.417 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:13.417 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:13.418 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:13.418 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.418 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.418 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:13.418 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.418 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.418 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.418 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:26:13.419 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.420 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.421 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:26:13.421 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.421 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:26:13.421 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:26:13.421 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:26:13.421 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:26:13.421 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:26:13.421 INFO:teuthology.orchestra.run.smithi133.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:26:13.421 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:26:13.421 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:26:13.421 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:26:13.422 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:26:13.422 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:26:13.422 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:26:13.422 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:26:13.422 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:26:13.422 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:26:13.422 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:26:13.422 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:26:13.422 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:26:13.422 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:26:13.422 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:26:13.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:26:13.424 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:26:13.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:26:13.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:26:13.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:26:13.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:26:13.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:26:13.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:26:13.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:26:13.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:26:13.425 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:26:13.426 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:26:13.427 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.428 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:13.429 INFO:teuthology.orchestra.run.smithi133.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:26:13.429 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:26:13.429 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:26:13.429 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:26:13.429 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:26:13.429 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:26:13.429 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:26:13.430 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:26:13.430 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-09-11T23:26:13.843 DEBUG:teuthology.parallel:result is None 2024-09-11T23:26:18.623 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:26:18.623 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:26:18.623 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:26:18.624 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:26:18.624 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:26:18.624 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:26:18.624 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:26:18.624 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:26:18.624 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:26:18.624 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:26:18.624 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:26:18.624 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:26:18.625 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:26:18.625 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:26:18.625 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:26:18.625 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:26:18.625 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:26:18.625 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:26:18.625 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:26:18.625 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:26:18.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:26:18.627 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:26:18.628 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:26:18.629 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:26:18.630 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:26:18.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:26:18.632 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:26:18.633 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:26:18.633 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:26:18.633 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:26:18.633 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:26:18.633 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:26:18.633 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:26:18.633 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:26:18.633 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:26:18.633 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:26:18.634 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:26:18.635 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:26:18.636 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:26:18.637 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:26:18.637 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:26:18.637 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:26:18.637 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:26:21.433 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:26:21.434 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:21.435 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:26:21.436 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.437 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:26:21.438 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:26:21.439 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:26:21.440 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:26:21.441 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:26:21.442 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:26:21.443 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:26:21.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:26:21.445 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:26:21.446 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:26:21.446 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:26:21.446 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-11T23:26:21.874 DEBUG:teuthology.parallel:result is None 2024-09-11T23:26:21.875 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:26:22.027 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:26:22.053 INFO:teuthology.orchestra.run.smithi133.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:26:22.053 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:26:22.054 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:26:22.055 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:26:22.204 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:26:22.231 INFO:teuthology.orchestra.run.smithi138.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:26:22.231 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:26:22.231 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:26:22.233 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:26:22.383 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:26:22.407 INFO:teuthology.orchestra.run.smithi178.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:26:22.407 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:26:22.407 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:26:22.409 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T23:26:22.409 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:26:22.409 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:26:22.442 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-11T23:26:22.442 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:26:22.475 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-11T23:26:22.475 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:26:22.505 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T23:26:22.505 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:26:22.505 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:26:22.538 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:26:22.607 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-11T23:26:22.607 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:26:22.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:26:22.705 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-11T23:26:22.705 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:26:22.732 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:26:22.799 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T23:26:22.799 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:26:22.799 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:26:22.829 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:26:22.898 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-11T23:26:22.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:26:22.928 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:26:22.997 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-11T23:26:22.997 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:26:23.024 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:26:23.091 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T23:26:23.091 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:26:23.091 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:26:23.120 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:26:23.189 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-11T23:26:23.189 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:26:23.220 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:26:23.288 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-11T23:26:23.288 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:26:23.316 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:26:23.383 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T23:26:23.478 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2}} 2024-09-11T23:26:23.478 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:26:23.479 INFO:tasks.cephadm:Cluster fsid is 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:26:23.479 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T23:26:23.479 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.133', 'mon.b': '172.21.15.138', 'mon.c': '172.21.15.178'} 2024-09-11T23:26:23.479 INFO:tasks.cephadm:First mon is mon.a on smithi133 2024-09-11T23:26:23.479 INFO:tasks.cephadm:First mgr is x 2024-09-11T23:26:23.480 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T23:26:23.480 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname $(hostname -s) 2024-09-11T23:26:23.510 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-09-11T23:26:23.540 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2024-09-11T23:26:23.568 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-11T23:26:23.568 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:26:23.720 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-11T23:26:23.874 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:26:23.875 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T23:26:23.875 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T23:26:23.876 DEBUG:teuthology.orchestra.run.smithi133:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:26:24.263 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:26 /home/ubuntu/cephtest/cephadm 2024-09-11T23:26:24.264 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:26:24.649 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:26 /home/ubuntu/cephtest/cephadm 2024-09-11T23:26:24.649 DEBUG:teuthology.orchestra.run.smithi178:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:26:25.018 INFO:teuthology.orchestra.run.smithi178.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:26 /home/ubuntu/cephtest/cephadm 2024-09-11T23:26:25.019 DEBUG:teuthology.orchestra.run.smithi133:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:26:25.038 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:26:25.058 DEBUG:teuthology.orchestra.run.smithi178:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:26:25.092 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-11T23:26:25.093 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:26:25.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:26:25.100 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:26:26.550 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:26:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:26:26.761 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:27:02.093 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:27:02.093 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:27:02.093 INFO:teuthology.orchestra.run.smithi133.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:27:02.094 INFO:teuthology.orchestra.run.smithi133.stdout: "repo_digests": [ 2024-09-11T23:27:02.094 INFO:teuthology.orchestra.run.smithi133.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:27:02.094 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:27:02.094 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:27:02.513 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-11T23:27:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:27:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:27:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-09-11T23:27:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:27:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-11T23:27:02.514 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-11T23:27:02.659 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-11T23:27:02.659 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:27:02.659 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:27:02.659 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2024-09-11T23:27:02.659 INFO:teuthology.orchestra.run.smithi178.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:27:02.660 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-11T23:27:02.660 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-11T23:27:02.687 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph 2024-09-11T23:27:02.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-09-11T23:27:02.747 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2024-09-11T23:27:02.777 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /etc/ceph 2024-09-11T23:27:02.805 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-09-11T23:27:02.832 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2024-09-11T23:27:02.858 INFO:tasks.cephadm:Writing seed config... 2024-09-11T23:27:02.859 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-11T23:27:02.859 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-11T23:27:02.859 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-11T23:27:02.860 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T23:27:02.861 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T23:27:02.862 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-11T23:27:02.863 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-11T23:27:02.863 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-11T23:27:02.863 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-11T23:27:02.863 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-11T23:27:02.863 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-11T23:27:02.863 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-11T23:27:02.863 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T23:27:02.863 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-11T23:27:02.864 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:27:02.864 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T23:27:02.881 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 42321b88-7095-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-11T23:27:02.882 DEBUG:teuthology.orchestra.run.smithi133:mon.a> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T23:27:02.925 DEBUG:teuthology.orchestra.run.smithi133:mgr.x> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T23:27:02.968 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T23:27:02.969 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 42321b88-7095-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.133 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:27:03.174 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:27:03.174 INFO:teuthology.orchestra.run.smithi133.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '42321b88-7095-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.133', '--skip-admin-label'] 2024-09-11T23:27:03.174 INFO:teuthology.orchestra.run.smithi133.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T23:27:03.174 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying podman|docker is present... 2024-09-11T23:27:03.197 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 5.2.2 2024-09-11T23:27:03.198 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying lvm2 is present... 2024-09-11T23:27:03.198 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying time synchronization is in place... 2024-09-11T23:27:03.205 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T23:27:03.205 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T23:27:03.211 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T23:27:03.211 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-09-11T23:27:03.218 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout enabled 2024-09-11T23:27:03.225 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout active 2024-09-11T23:27:03.225 INFO:teuthology.orchestra.run.smithi133.stdout:Unit chronyd.service is enabled and running 2024-09-11T23:27:03.225 INFO:teuthology.orchestra.run.smithi133.stdout:Repeating the final host check... 2024-09-11T23:27:03.247 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 5.2.2 2024-09-11T23:27:03.248 INFO:teuthology.orchestra.run.smithi133.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T23:27:03.248 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl is present 2024-09-11T23:27:03.248 INFO:teuthology.orchestra.run.smithi133.stdout:lvcreate is present 2024-09-11T23:27:03.254 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T23:27:03.254 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T23:27:03.260 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T23:27:03.260 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-09-11T23:27:03.266 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout enabled 2024-09-11T23:27:03.273 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout active 2024-09-11T23:27:03.273 INFO:teuthology.orchestra.run.smithi133.stdout:Unit chronyd.service is enabled and running 2024-09-11T23:27:03.273 INFO:teuthology.orchestra.run.smithi133.stdout:Host looks OK 2024-09-11T23:27:03.273 INFO:teuthology.orchestra.run.smithi133.stdout:Cluster fsid: 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:27:03.273 INFO:teuthology.orchestra.run.smithi133.stdout:Acquiring lock 139944178365440 on /run/cephadm/42321b88-7095-11ef-bcea-c7b262605968.lock 2024-09-11T23:27:03.273 INFO:teuthology.orchestra.run.smithi133.stdout:Lock 139944178365440 acquired on /run/cephadm/42321b88-7095-11ef-bcea-c7b262605968.lock 2024-09-11T23:27:03.274 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying IP 172.21.15.133 port 3300 ... 2024-09-11T23:27:03.274 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying IP 172.21.15.133 port 6789 ... 2024-09-11T23:27:03.274 INFO:teuthology.orchestra.run.smithi133.stdout:Base mon IP(s) is [172.21.15.133:3300, 172.21.15.133:6789], mon addrv is [v2:172.21.15.133:3300,v1:172.21.15.133:6789] 2024-09-11T23:27:03.277 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.133 metric 100 2024-09-11T23:27:03.277 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.133 metric 100 2024-09-11T23:27:03.280 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T23:27:03.280 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T23:27:03.280 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T23:27:03.283 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T23:27:03.284 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T23:27:03.284 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T23:27:03.284 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T23:27:03.284 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:8087/64 scope link noprefixroute 2024-09-11T23:27:03.284 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T23:27:03.284 INFO:teuthology.orchestra.run.smithi133.stdout:Mon IP `172.21.15.133` is in CIDR network `172.21.0.0/20` 2024-09-11T23:27:03.285 INFO:teuthology.orchestra.run.smithi133.stdout:Mon IP `172.21.15.133` is in CIDR network `172.21.0.0/20` 2024-09-11T23:27:03.285 INFO:teuthology.orchestra.run.smithi133.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T23:27:03.286 INFO:teuthology.orchestra.run.smithi133.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T23:27:03.286 INFO:teuthology.orchestra.run.smithi133.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:27:03.749 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T23:27:03.749 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:27:03.750 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T23:27:03.750 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-11T23:27:03.750 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T23:27:03.750 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T23:27:03.750 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T23:27:04.662 INFO:teuthology.orchestra.run.smithi133.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T23:27:04.663 INFO:teuthology.orchestra.run.smithi133.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T23:27:04.663 INFO:teuthology.orchestra.run.smithi133.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T23:27:05.919 INFO:teuthology.orchestra.run.smithi133.stdout:stat: stdout 167 167 2024-09-11T23:27:05.919 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial keys... 2024-09-11T23:27:06.890 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQBKJ+JmuKKJDxAAa0PbR4Tlj362ZmqC6h0waw== 2024-09-11T23:27:07.879 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQBLJ+Jm7jZzDxAAwfJZE/rpbJl7yt4ocG9/aQ== 2024-09-11T23:27:14.976 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQBMJ+Jm1QKzCRAAybbxi8+lG9SCDfkWrHErlg== 2024-09-11T23:27:14.977 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial monmap... 2024-09-11T23:27:19.119 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T23:27:19.119 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-11T23:27:19.119 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:27:19.119 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T23:27:19.119 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool for a [v2:172.21.15.133:3300,v1:172.21.15.133:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T23:27:19.119 INFO:teuthology.orchestra.run.smithi133.stdout:setting min_mon_release = quincy 2024-09-11T23:27:19.120 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: set fsid to 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:27:19.120 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T23:27:19.120 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:27:19.120 INFO:teuthology.orchestra.run.smithi133.stdout:Creating mon... 2024-09-11T23:27:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:create mon.a on 2024-09-11T23:27:20.204 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T23:27:20.341 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T23:27:20.521 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-42321b88-7095-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-42321b88-7095-11ef-bcea-c7b262605968.target. 2024-09-11T23:27:20.601 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-42321b88-7095-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-42321b88-7095-11ef-bcea-c7b262605968.target. 2024-09-11T23:27:20.911 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.a 2024-09-11T23:27:20.911 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to reset failed state of unit ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T23:27:21.086 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-42321b88-7095-11ef-bcea-c7b262605968.target.wants/ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-42321b88-7095-11ef-bcea-c7b262605968@.service. 2024-09-11T23:27:21.768 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:27:21.768 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2024-09-11T23:27:21.775 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:27:21.775 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-09-11T23:27:21.775 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2024-09-11T23:27:21.775 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T23:27:21.776 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mon to start... 2024-09-11T23:27:21.776 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mon... 2024-09-11T23:27:22.884 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:22 smithi133 ceph-mon[31061]: from='client.? 172.21.15.133:0/1928022537' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T23:27:22.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T23:27:22.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout id: 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:27:22.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T23:27:22.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:22.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout services: 2024-09-11T23:27:22.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.586015s) 2024-09-11T23:27:22.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T23:27:22.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T23:27:22.892 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout data: 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.217+0000 7f103f507640 1 Processor -- start 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.217+0000 7f103f507640 1 -- start start 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.217+0000 7f103f507640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f1038106b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.217+0000 7f103f507640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1038107150 con 0x7f1038106780 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.218+0000 7f103d2a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f1038106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.218+0000 7f103d2a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f1038106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51374/0 (socket says 172.21.15.133:51374) 2024-09-11T23:27:22.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.218+0000 7f103d2a3640 1 -- 172.21.15.133:0/3556614774 learned_addr learned my addr 172.21.15.133:0/3556614774 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:22.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.218+0000 7f103d2a3640 1 -- 172.21.15.133:0/3556614774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1038107980 con 0x7f1038106780 2024-09-11T23:27:22.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.220+0000 7f103d2a3640 1 --2- 172.21.15.133:0/3556614774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f1038106b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1028009820 tx=0x7f102802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f32736d35a77c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:22.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.220+0000 7f102ffff640 1 -- 172.21.15.133:0/3556614774 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1028037070 con 0x7f1038106780 2024-09-11T23:27:22.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.220+0000 7f102ffff640 1 -- 172.21.15.133:0/3556614774 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1028032440 con 0x7f1038106780 2024-09-11T23:27:22.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.220+0000 7f102ffff640 1 -- 172.21.15.133:0/3556614774 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1028032740 con 0x7f1038106780 2024-09-11T23:27:22.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.221+0000 7f103f507640 1 -- 172.21.15.133:0/3556614774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 msgr2=0x7f1038106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:22.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.221+0000 7f103f507640 1 --2- 172.21.15.133:0/3556614774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f1038106b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1028009820 tx=0x7f102802a270 comp rx=0 tx=0).stop 2024-09-11T23:27:22.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.221+0000 7f103f507640 1 -- 172.21.15.133:0/3556614774 shutdown_connections 2024-09-11T23:27:22.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.221+0000 7f103f507640 1 --2- 172.21.15.133:0/3556614774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f1038106b80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:22.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.221+0000 7f103f507640 1 -- 172.21.15.133:0/3556614774 >> 172.21.15.133:0/3556614774 conn(0x7f1038101f10 msgr2=0x7f1038104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:22.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.221+0000 7f103f507640 1 -- 172.21.15.133:0/3556614774 shutdown_connections 2024-09-11T23:27:22.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.221+0000 7f103f507640 1 -- 172.21.15.133:0/3556614774 wait complete. 2024-09-11T23:27:22.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.222+0000 7f103f507640 1 Processor -- start 2024-09-11T23:27:22.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.222+0000 7f103f507640 1 -- start start 2024-09-11T23:27:22.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.222+0000 7f103f507640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f103818f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:22.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.222+0000 7f103f507640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1038107f20 con 0x7f1038106780 2024-09-11T23:27:22.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.222+0000 7f103d2a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f103818f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:22.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.222+0000 7f103d2a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f103818f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51384/0 (socket says 172.21.15.133:51384) 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.222+0000 7f103d2a3640 1 -- 172.21.15.133:0/1928022537 learned_addr learned my addr 172.21.15.133:0/1928022537 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.222+0000 7f103d2a3640 1 -- 172.21.15.133:0/1928022537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103818f880 con 0x7f1038106780 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.223+0000 7f103d2a3640 1 --2- 172.21.15.133:0/1928022537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f103818f340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f10280090b0 tx=0x7f10280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.223+0000 7f102e7fc640 1 -- 172.21.15.133:0/1928022537 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1028041070 con 0x7f1038106780 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.223+0000 7f102e7fc640 1 -- 172.21.15.133:0/1928022537 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f102803c7a0 con 0x7f1038106780 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.223+0000 7f103f507640 1 -- 172.21.15.133:0/1928022537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103818da70 con 0x7f1038106780 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.223+0000 7f102e7fc640 1 -- 172.21.15.133:0/1928022537 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1028037040 con 0x7f1038106780 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.223+0000 7f103f507640 1 -- 172.21.15.133:0/1928022537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103818dfb0 con 0x7f1038106780 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.223+0000 7f102e7fc640 1 -- 172.21.15.133:0/1928022537 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1028044070 con 0x7f1038106780 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.223+0000 7f102e7fc640 1 -- 172.21.15.133:0/1928022537 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1028046450 con 0x7f1038106780 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.224+0000 7f103f507640 1 -- 172.21.15.133:0/1928022537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1038108570 con 0x7f1038106780 2024-09-11T23:27:22.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.226+0000 7f102e7fc640 1 -- 172.21.15.133:0/1928022537 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1028046650 con 0x7f1038106780 2024-09-11T23:27:22.897 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.285+0000 7f103f507640 1 -- 172.21.15.133:0/1928022537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f1038194490 con 0x7f1038106780 2024-09-11T23:27:22.897 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.286+0000 7f102e7fc640 1 -- 172.21.15.133:0/1928022537 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f102803caa0 con 0x7f1038106780 2024-09-11T23:27:22.897 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.286+0000 7f103f507640 1 -- 172.21.15.133:0/1928022537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 msgr2=0x7f103818f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:22.897 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.286+0000 7f103f507640 1 --2- 172.21.15.133:0/1928022537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f103818f340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f10280090b0 tx=0x7f10280046c0 comp rx=0 tx=0).stop 2024-09-11T23:27:22.897 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.286+0000 7f103f507640 1 -- 172.21.15.133:0/1928022537 shutdown_connections 2024-09-11T23:27:22.897 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.286+0000 7f103f507640 1 --2- 172.21.15.133:0/1928022537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1038106780 0x7f103818f340 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:22.897 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.286+0000 7f103f507640 1 -- 172.21.15.133:0/1928022537 >> 172.21.15.133:0/1928022537 conn(0x7f1038101f10 msgr2=0x7f103807b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:22.897 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.287+0000 7f103f507640 1 -- 172.21.15.133:0/1928022537 shutdown_connections 2024-09-11T23:27:22.897 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:22.287+0000 7f103f507640 1 -- 172.21.15.133:0/1928022537 wait complete. 2024-09-11T23:27:22.897 INFO:teuthology.orchestra.run.smithi133.stdout:mon is available 2024-09-11T23:27:22.897 INFO:teuthology.orchestra.run.smithi133.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T23:27:23.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:23 smithi133 ceph-mon[31061]: from='client.? 172.21.15.133:0/1234492889' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T23:27:23.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:23 smithi133 ceph-mon[31061]: from='client.? 172.21.15.133:0/1234492889' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T23:27:25.608 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:25.609 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T23:27:25.609 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout fsid = 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:27:25.609 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.133:3300,v1:172.21.15.133:6789] 2024-09-11T23:27:25.609 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T23:27:25.609 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T23:27:25.609 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T23:27:25.609 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T23:27:25.609 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:25.609 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T23:27:25.610 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T23:27:25.610 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:25.610 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T23:27:25.610 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T23:27:25.610 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T23:27:25.610 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T23:27:25.610 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T23:27:25.610 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.331+0000 7f6693c2e640 1 Processor -- start 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.331+0000 7f6693c2e640 1 -- start start 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.331+0000 7f6693c2e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c106560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.331+0000 7f6693c2e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f668c07bdc0 con 0x7f668c106160 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.332+0000 7f66919ca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c106560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.332+0000 7f66919ca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c106560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51386/0 (socket says 172.21.15.133:51386) 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.332+0000 7f66919ca640 1 -- 172.21.15.133:0/3230404880 learned_addr learned my addr 172.21.15.133:0/3230404880 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.332+0000 7f66919ca640 1 -- 172.21.15.133:0/3230404880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c106aa0 con 0x7f668c106160 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.333+0000 7f66919ca640 1 --2- 172.21.15.133:0/3230404880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c106560 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6688009820 tx=0x7f668802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9785835f9ed43f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.333+0000 7f66909c8640 1 -- 172.21.15.133:0/3230404880 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6688037070 con 0x7f668c106160 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.333+0000 7f66909c8640 1 -- 172.21.15.133:0/3230404880 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6688032440 con 0x7f668c106160 2024-09-11T23:27:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.333+0000 7f66909c8640 1 -- 172.21.15.133:0/3230404880 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6688032740 con 0x7f668c106160 2024-09-11T23:27:25.612 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.333+0000 7f6693c2e640 1 -- 172.21.15.133:0/3230404880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 msgr2=0x7f668c106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:25.612 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.333+0000 7f6693c2e640 1 --2- 172.21.15.133:0/3230404880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c106560 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6688009820 tx=0x7f668802a270 comp rx=0 tx=0).stop 2024-09-11T23:27:25.612 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.333+0000 7f6693c2e640 1 -- 172.21.15.133:0/3230404880 shutdown_connections 2024-09-11T23:27:25.612 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.333+0000 7f6693c2e640 1 --2- 172.21.15.133:0/3230404880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c106560 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.612 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.333+0000 7f6693c2e640 1 -- 172.21.15.133:0/3230404880 >> 172.21.15.133:0/3230404880 conn(0x7f668c101f10 msgr2=0x7f668c104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:25.612 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.333+0000 7f6693c2e640 1 -- 172.21.15.133:0/3230404880 shutdown_connections 2024-09-11T23:27:25.612 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.333+0000 7f6693c2e640 1 -- 172.21.15.133:0/3230404880 wait complete. 2024-09-11T23:27:25.612 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.334+0000 7f6693c2e640 1 Processor -- start 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.334+0000 7f6693c2e640 1 -- start start 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.334+0000 7f6693c2e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c19d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.334+0000 7f6693c2e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f668c107040 con 0x7f668c106160 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.334+0000 7f66919ca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c19d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.334+0000 7f66919ca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c19d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51392/0 (socket says 172.21.15.133:51392) 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.334+0000 7f66919ca640 1 -- 172.21.15.133:0/1234492889 learned_addr learned my addr 172.21.15.133:0/1234492889 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.335+0000 7f66919ca640 1 -- 172.21.15.133:0/1234492889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c19d740 con 0x7f668c106160 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.335+0000 7f66919ca640 1 --2- 172.21.15.133:0/1234492889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c19d200 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f66880090b0 tx=0x7f66880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.335+0000 7f667effd640 1 -- 172.21.15.133:0/1234492889 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6688041070 con 0x7f668c106160 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.335+0000 7f667effd640 1 -- 172.21.15.133:0/1234492889 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668803c7a0 con 0x7f668c106160 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.335+0000 7f667effd640 1 -- 172.21.15.133:0/1234492889 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6688037040 con 0x7f668c106160 2024-09-11T23:27:25.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.335+0000 7f6693c2e640 1 -- 172.21.15.133:0/1234492889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c19d9d0 con 0x7f668c106160 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.335+0000 7f6693c2e640 1 -- 172.21.15.133:0/1234492889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c1a06c0 con 0x7f668c106160 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.335+0000 7f667effd640 1 -- 172.21.15.133:0/1234492889 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6688044070 con 0x7f668c106160 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.335+0000 7f667effd640 1 -- 172.21.15.133:0/1234492889 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f668803d3f0 con 0x7f668c106160 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.336+0000 7f6693c2e640 1 -- 172.21.15.133:0/1234492889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668c106560 con 0x7f668c106160 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.338+0000 7f667effd640 1 -- 172.21.15.133:0/1234492889 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6688046450 con 0x7f668c106160 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.395+0000 7f6693c2e640 1 -- 172.21.15.133:0/1234492889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f668c1a0890 con 0x7f668c106160 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.398+0000 7f667effd640 1 -- 172.21.15.133:0/1234492889 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f668c1a0890 con 0x7f668c106160 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.398+0000 7f667effd640 1 -- 172.21.15.133:0/1234492889 <== mon.0 v2:172.21.15.133:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6688056070 con 0x7f668c106160 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.399+0000 7f6693c2e640 1 -- 172.21.15.133:0/1234492889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 msgr2=0x7f668c19d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.399+0000 7f6693c2e640 1 --2- 172.21.15.133:0/1234492889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c19d200 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f66880090b0 tx=0x7f66880046c0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.399+0000 7f6693c2e640 1 -- 172.21.15.133:0/1234492889 shutdown_connections 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.399+0000 7f6693c2e640 1 --2- 172.21.15.133:0/1234492889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f668c106160 0x7f668c19d200 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.399+0000 7f6693c2e640 1 -- 172.21.15.133:0/1234492889 >> 172.21.15.133:0/1234492889 conn(0x7f668c101f10 msgr2=0x7f668c1937e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:25.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.399+0000 7f6693c2e640 1 -- 172.21.15.133:0/1234492889 shutdown_connections 2024-09-11T23:27:25.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:23.399+0000 7f6693c2e640 1 -- 172.21.15.133:0/1234492889 wait complete. 2024-09-11T23:27:25.615 INFO:teuthology.orchestra.run.smithi133.stdout:Generating new minimal ceph.conf... 2024-09-11T23:27:26.424 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:26 smithi133 ceph-mon[31061]: from='client.? 172.21.15.133:0/2246532919' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:27:26.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.043+0000 7f3da59a0640 1 Processor -- start 2024-09-11T23:27:26.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.043+0000 7f3da59a0640 1 -- start start 2024-09-11T23:27:26.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.043+0000 7f3da59a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da0106b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:26.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.043+0000 7f3da59a0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0107130 con 0x7f3da0106760 2024-09-11T23:27:26.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.043+0000 7f3d9effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da0106b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:26.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.043+0000 7f3d9effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da0106b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51394/0 (socket says 172.21.15.133:51394) 2024-09-11T23:27:26.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.043+0000 7f3d9effd640 1 -- 172.21.15.133:0/3010774630 learned_addr learned my addr 172.21.15.133:0/3010774630 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:26.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.043+0000 7f3d9effd640 1 -- 172.21.15.133:0/3010774630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da0107960 con 0x7f3da0106760 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.044+0000 7f3d9effd640 1 --2- 172.21.15.133:0/3010774630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da0106b60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009820 tx=0x7f3d9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6aa8b4292b90be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.044+0000 7f3d9e7fc640 1 -- 172.21.15.133:0/3010774630 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d90037070 con 0x7f3da0106760 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.044+0000 7f3d9e7fc640 1 -- 172.21.15.133:0/3010774630 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3d90032900 con 0x7f3da0106760 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.044+0000 7f3d9e7fc640 1 -- 172.21.15.133:0/3010774630 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d90030850 con 0x7f3da0106760 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.045+0000 7f3da59a0640 1 -- 172.21.15.133:0/3010774630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 msgr2=0x7f3da0106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.045+0000 7f3da59a0640 1 --2- 172.21.15.133:0/3010774630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da0106b60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009820 tx=0x7f3d9002a270 comp rx=0 tx=0).stop 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.045+0000 7f3da59a0640 1 -- 172.21.15.133:0/3010774630 shutdown_connections 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.045+0000 7f3da59a0640 1 --2- 172.21.15.133:0/3010774630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da0106b60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.045+0000 7f3da59a0640 1 -- 172.21.15.133:0/3010774630 >> 172.21.15.133:0/3010774630 conn(0x7f3da0101f10 msgr2=0x7f3da0104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.045+0000 7f3da59a0640 1 -- 172.21.15.133:0/3010774630 shutdown_connections 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.045+0000 7f3da59a0640 1 -- 172.21.15.133:0/3010774630 wait complete. 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.046+0000 7f3da59a0640 1 Processor -- start 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.046+0000 7f3da59a0640 1 -- start start 2024-09-11T23:27:26.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.046+0000 7f3da59a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.046+0000 7f3da59a0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3da010ad10 con 0x7f3da0106760 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.046+0000 7f3d9effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.046+0000 7f3d9effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da019b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51406/0 (socket says 172.21.15.133:51406) 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.046+0000 7f3d9effd640 1 -- 172.21.15.133:0/2246532919 learned_addr learned my addr 172.21.15.133:0/2246532919 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.046+0000 7f3d9effd640 1 -- 172.21.15.133:0/2246532919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da019be40 con 0x7f3da0106760 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.046+0000 7f3d9effd640 1 --2- 172.21.15.133:0/2246532919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da019b900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009940 tx=0x7f3d900326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.047+0000 7f3da499e640 1 -- 172.21.15.133:0/2246532919 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d90042070 con 0x7f3da0106760 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.047+0000 7f3da499e640 1 -- 172.21.15.133:0/2246532919 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3d90032be0 con 0x7f3da0106760 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.047+0000 7f3da499e640 1 -- 172.21.15.133:0/2246532919 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d90037040 con 0x7f3da0106760 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.047+0000 7f3da59a0640 1 -- 172.21.15.133:0/2246532919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da019c0d0 con 0x7f3da0106760 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.047+0000 7f3da59a0640 1 -- 172.21.15.133:0/2246532919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da019edc0 con 0x7f3da0106760 2024-09-11T23:27:26.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.047+0000 7f3da499e640 1 -- 172.21.15.133:0/2246532919 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3d90045070 con 0x7f3da0106760 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.047+0000 7f3da499e640 1 -- 172.21.15.133:0/2246532919 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3d900479f0 con 0x7f3da0106760 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.048+0000 7f3da59a0640 1 -- 172.21.15.133:0/2246532919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da010ae90 con 0x7f3da0106760 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.050+0000 7f3da499e640 1 -- 172.21.15.133:0/2246532919 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3d90032d90 con 0x7f3da0106760 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.108+0000 7f3da59a0640 1 -- 172.21.15.133:0/2246532919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f3da0195830 con 0x7f3da0106760 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.108+0000 7f3da499e640 1 -- 172.21.15.133:0/2246532919 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f3d9003d430 con 0x7f3da0106760 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.109+0000 7f3da59a0640 1 -- 172.21.15.133:0/2246532919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 msgr2=0x7f3da019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.109+0000 7f3da59a0640 1 --2- 172.21.15.133:0/2246532919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da019b900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009940 tx=0x7f3d900326e0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.109+0000 7f3da59a0640 1 -- 172.21.15.133:0/2246532919 shutdown_connections 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.109+0000 7f3da59a0640 1 --2- 172.21.15.133:0/2246532919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da0106760 0x7f3da019b900 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.109+0000 7f3da59a0640 1 -- 172.21.15.133:0/2246532919 >> 172.21.15.133:0/2246532919 conn(0x7f3da0101f10 msgr2=0x7f3da0102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.109+0000 7f3da59a0640 1 -- 172.21.15.133:0/2246532919 shutdown_connections 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:26.109+0000 7f3da59a0640 1 -- 172.21.15.133:0/2246532919 wait complete. 2024-09-11T23:27:26.713 INFO:teuthology.orchestra.run.smithi133.stdout:Restarting the monitor... 2024-09-11T23:27:26.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:26 smithi133 systemd[1]: Stopping Ceph mon.a for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:27:27.382 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:26 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a[31049]: 2024-09-11T23:27:26.973+0000 7f2af94db640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:27:27.382 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:26 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a[31049]: 2024-09-11T23:27:26.973+0000 7f2af94db640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T23:27:27.382 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:27 smithi133 podman[31306]: 2024-09-11 23:27:27.106942442 +0000 UTC m=+0.235385531 container died 992e467dbb2821ae14f39e36d5a139a17444ce6eba0efed708830b5958c16338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T23:27:27.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:27 smithi133 podman[31306]: 2024-09-11 23:27:27.482509689 +0000 UTC m=+0.610952774 container cleanup 992e467dbb2821ae14f39e36d5a139a17444ce6eba0efed708830b5958c16338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:27:27.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:27 smithi133 bash[31306]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a 2024-09-11T23:27:27.986 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:27 smithi133 podman[31318]: 2024-09-11 23:27:27.733206778 +0000 UTC m=+0.626658752 container remove 992e467dbb2821ae14f39e36d5a139a17444ce6eba0efed708830b5958c16338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T23:27:27.986 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:27 smithi133 systemd[1]: ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T23:27:27.986 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:27 smithi133 systemd[1]: Stopped Ceph mon.a for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:27:27.986 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:27 smithi133 systemd[1]: Starting Ceph mon.a for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:27:28.240 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 podman[31411]: 2024-09-11 23:27:28.008760688 +0000 UTC m=+0.016133374 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:27:28.240 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 podman[31411]: 2024-09-11 23:27:28.117350607 +0000 UTC m=+0.124723291 container create 573b8e5cf42605cd63fa7fa87cea92bdca73da537420b5edbaf9e46d8317207e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T23:27:28.406 INFO:teuthology.orchestra.run.smithi133.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T23:27:28.641 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 podman[31411]: 2024-09-11 23:27:28.319709428 +0000 UTC m=+0.327082115 container init 573b8e5cf42605cd63fa7fa87cea92bdca73da537420b5edbaf9e46d8317207e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-11T23:27:28.641 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 podman[31411]: 2024-09-11 23:27:28.322846478 +0000 UTC m=+0.330219161 container start 573b8e5cf42605cd63fa7fa87cea92bdca73da537420b5edbaf9e46d8317207e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T23:27:28.641 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: pidfile_write: ignore empty --pid-file 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: load: jerasure load: lrc 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: RocksDB version: 7.9.2 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Git sha 0 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: DB SUMMARY 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: DB Session ID: 873IBFGAEN419TGTEC0Z 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: CURRENT file: CURRENT 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-11T23:27:28.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106692 ; 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.error_if_exists: 0 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.create_if_missing: 0 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.paranoid_checks: 1 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.env: 0x561685178940 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.fs: PosixFileSystem 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.info_log: 0x5616875dd6e0 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T23:27:28.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.statistics: (nil) 2024-09-11T23:27:28.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.use_fsync: 0 2024-09-11T23:27:28.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_log_file_size: 0 2024-09-11T23:27:28.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T23:27:28.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T23:27:28.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T23:27:28.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T23:27:28.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.allow_fallocate: 1 2024-09-11T23:27:28.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T23:27:28.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.use_direct_reads: 0 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.db_log_dir: 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.wal_dir: 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T23:27:28.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.write_buffer_manager: 0x5616875ecdc0 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.unordered_write: 0 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T23:27:28.647 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.row_cache: None 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.wal_filter: None 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.two_write_queues: 0 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.wal_compression: 0 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.atomic_flush: 0 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.log_readahead_size: 0 2024-09-11T23:27:28.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_background_jobs: 2 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_background_compactions: -1 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_subcompactions: 1 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T23:27:28.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_open_files: -1 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_background_flushes: -1 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Compression algorithms supported: 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: kZSTD supported: 0 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: kXpressCompression supported: 0 2024-09-11T23:27:28.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: kBZip2Compression supported: 0 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: kLZ4Compression supported: 1 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: kZlibCompression supported: 1 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: kSnappyCompression supported: 1 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.merge_operator: 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_filter: None 2024-09-11T23:27:28.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5616875dd320) 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout: cache_index_and_filter_blocks: 1 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout: pin_top_level_index_and_filter: 1 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_type: 0 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout: data_block_index_type: 0 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_shortening: 1 2024-09-11T23:27:28.652 INFO:journalctl@ceph.mon.a.smithi133.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: checksum: 4 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: no_block_cache: 0 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache: 0x5616875d11f0 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_name: BinnedLRUCache 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_options: 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: capacity : 536870912 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: num_shard_bits : 4 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: strict_capacity_limit : 0 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: high_pri_pool_ratio: 0.000 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_compressed: (nil) 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: persistent_cache: (nil) 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_size: 4096 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_size_deviation: 10 2024-09-11T23:27:28.653 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_restart_interval: 16 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_block_restart_interval: 1 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: metadata_block_size: 4096 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: partition_filters: 0 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: use_delta_encoding: 1 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: filter_policy: bloomfilter 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: whole_key_filtering: 1 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: verify_compression: 0 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: read_amp_bytes_per_bit: 0 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: format_version: 5 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: enable_index_compression: 1 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_align: 0 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: max_auto_readahead_size: 262144 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: prepopulate_block_cache: 0 2024-09-11T23:27:28.654 INFO:journalctl@ceph.mon.a.smithi133.stdout: initial_auto_readahead_size: 8192 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout: num_file_reads_for_auto_readahead: 2 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compression: NoCompression 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.num_levels: 7 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T23:27:28.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T23:27:28.656 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T23:27:28.657 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.arena_block_size: 1048576 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T23:27:28.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.inplace_update_support: 0 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T23:27:28.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.bloom_locality: 0 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.max_successive_merges: 0 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.ttl: 2592000 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.enable_blob_files: false 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.min_blob_size: 0 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T23:27:28.660 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T23:27:28.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T23:27:28.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-11T23:27:28.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-11T23:27:28.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.blob_file_starting_level: 0 2024-09-11T23:27:28.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-11T23:27:28.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-11T23:27:28.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-11T23:27:28.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9b641d2a-ac60-4aab-bdcd-b2ab337c00bc 2024-09-11T23:27:28.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726097248348197, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726097248349148, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101598, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 99481, "index_size": 309, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14649, "raw_average_key_size": 49, "raw_value_size": 91588, "raw_average_value_size": 312, "num_data_blocks": 12, "num_entries": 293, "num_filter_entries": 293, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726097248, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9b641d2a-ac60-4aab-bdcd-b2ab337c00bc", "db_session_id": "873IBFGAEN419TGTEC0Z", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726097248349243, "job": 1, "event": "recovery_finished"} 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5616875fee00 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: DB pointer 0x5616876fa000 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** DB Stats ** 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T23:27:28.662 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** Compaction Stats [default] ** 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: L0 2/0 101.08 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: Sum 2/0 101.08 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 164.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** Compaction Stats [default] ** 2024-09-11T23:27:28.663 INFO:journalctl@ceph.mon.a.smithi133.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 164.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative compaction: 0.00 GB write, 32.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval compaction: 0.00 GB write, 32.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T23:27:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout: Block cache BinnedLRUCache@0x5616875d11f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: starting mon.a rank 0 at public addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] at bind addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: mon.a@-1(???) e1 preinit fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: mon.a@-1(???).mds e1 new map 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: mon.a@-1(???).mds e1 print_map 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout: e1 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout: btime 2024-09-11T23:27:21:703865+0000 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:27:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout: legacy client fscid: -1 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout: No filesystems configured 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 bash[31411]: 573b8e5cf42605cd63fa7fa87cea92bdca73da537420b5edbaf9e46d8317207e 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 systemd[1]: Started Ceph mon.a for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: monmap epoch 1 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: last_changed 2024-09-11T23:27:17.802580+0000 2024-09-11T23:27:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: created 2024-09-11T23:27:17.802580+0000 2024-09-11T23:27:28.667 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: min_mon_release 19 (squid) 2024-09-11T23:27:28.667 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: election_strategy: 1 2024-09-11T23:27:28.667 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-09-11T23:27:28.667 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: fsmap 2024-09-11T23:27:28.667 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T23:27:28.667 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:28 smithi133 ceph-mon[31430]: mgrmap e1: no daemons active 2024-09-11T23:27:29.481 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.828+0000 7f78a6985640 1 Processor -- start 2024-09-11T23:27:29.481 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.828+0000 7f78a6985640 1 -- start start 2024-09-11T23:27:29.481 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.828+0000 7f78a6985640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a0107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:29.481 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.828+0000 7f78a6985640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78a01082a0 con 0x7f78a01078d0 2024-09-11T23:27:29.481 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.829+0000 7f789ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a0107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:29.481 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.829+0000 7f789ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a0107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51418/0 (socket says 172.21.15.133:51418) 2024-09-11T23:27:29.481 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.829+0000 7f789ffff640 1 -- 172.21.15.133:0/4023183574 learned_addr learned my addr 172.21.15.133:0/4023183574 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:29.481 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.829+0000 7f789ffff640 1 -- 172.21.15.133:0/4023183574 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a0108a80 con 0x7f78a01078d0 2024-09-11T23:27:29.482 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.831+0000 7f789ffff640 1 --2- 172.21.15.133:0/4023183574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a0107cd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7890009820 tx=0x7f789002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e980aa9a592c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:29.482 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.831+0000 7f789effd640 1 -- 172.21.15.133:0/4023183574 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7890037070 con 0x7f78a01078d0 2024-09-11T23:27:29.482 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.831+0000 7f789effd640 1 -- 172.21.15.133:0/4023183574 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7890032900 con 0x7f78a01078d0 2024-09-11T23:27:29.482 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.831+0000 7f789effd640 1 -- 172.21.15.133:0/4023183574 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7890030850 con 0x7f78a01078d0 2024-09-11T23:27:29.482 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.832+0000 7f78a6985640 1 -- 172.21.15.133:0/4023183574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 msgr2=0x7f78a0107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:29.482 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.832+0000 7f78a6985640 1 --2- 172.21.15.133:0/4023183574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a0107cd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7890009820 tx=0x7f789002a270 comp rx=0 tx=0).stop 2024-09-11T23:27:29.482 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.832+0000 7f78a6985640 1 -- 172.21.15.133:0/4023183574 shutdown_connections 2024-09-11T23:27:29.482 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.832+0000 7f78a6985640 1 --2- 172.21.15.133:0/4023183574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a0107cd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:29.482 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.832+0000 7f78a6985640 1 -- 172.21.15.133:0/4023183574 >> 172.21.15.133:0/4023183574 conn(0x7f78a0103080 msgr2=0x7f78a01054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:29.482 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.832+0000 7f78a6985640 1 -- 172.21.15.133:0/4023183574 shutdown_connections 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.832+0000 7f78a6985640 1 -- 172.21.15.133:0/4023183574 wait complete. 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.832+0000 7f78a6985640 1 Processor -- start 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.832+0000 7f78a6985640 1 -- start start 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.833+0000 7f78a6985640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.833+0000 7f78a6985640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78a010be30 con 0x7f78a01078d0 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.833+0000 7f789ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.833+0000 7f789ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a019b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51434/0 (socket says 172.21.15.133:51434) 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.833+0000 7f789ffff640 1 -- 172.21.15.133:0/747228694 learned_addr learned my addr 172.21.15.133:0/747228694 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.833+0000 7f789ffff640 1 -- 172.21.15.133:0/747228694 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a019be30 con 0x7f78a01078d0 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.833+0000 7f789ffff640 1 --2- 172.21.15.133:0/747228694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a019b8f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7890009940 tx=0x7f78900326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.833+0000 7f789d7fa640 1 -- 172.21.15.133:0/747228694 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7890042070 con 0x7f78a01078d0 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.833+0000 7f789d7fa640 1 -- 172.21.15.133:0/747228694 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7890032be0 con 0x7f78a01078d0 2024-09-11T23:27:29.483 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.834+0000 7f78a6985640 1 -- 172.21.15.133:0/747228694 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a019c0c0 con 0x7f78a01078d0 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.834+0000 7f789d7fa640 1 -- 172.21.15.133:0/747228694 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7890037040 con 0x7f78a01078d0 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.834+0000 7f78a6985640 1 -- 172.21.15.133:0/747228694 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a019edb0 con 0x7f78a01078d0 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.834+0000 7f789d7fa640 1 -- 172.21.15.133:0/747228694 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7890045070 con 0x7f78a01078d0 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.834+0000 7f789d7fa640 1 -- 172.21.15.133:0/747228694 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7890047a80 con 0x7f78a01078d0 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.835+0000 7f78a6985640 1 -- 172.21.15.133:0/747228694 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a010bfb0 con 0x7f78a01078d0 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.837+0000 7f789d7fa640 1 -- 172.21.15.133:0/747228694 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7890032d90 con 0x7f78a01078d0 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.893+0000 7f78a6985640 1 -- 172.21.15.133:0/747228694 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f78a019ef80 con 0x7f78a01078d0 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.898+0000 7f789d7fa640 1 -- 172.21.15.133:0/747228694 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f789003d430 con 0x7f78a01078d0 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.899+0000 7f78a6985640 1 -- 172.21.15.133:0/747228694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 msgr2=0x7f78a019b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.899+0000 7f78a6985640 1 --2- 172.21.15.133:0/747228694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a019b8f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7890009940 tx=0x7f78900326e0 comp rx=0 tx=0).stop 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.899+0000 7f78a6985640 1 -- 172.21.15.133:0/747228694 shutdown_connections 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.899+0000 7f78a6985640 1 --2- 172.21.15.133:0/747228694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78a01078d0 0x7f78a019b8f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:29.484 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.899+0000 7f78a6985640 1 -- 172.21.15.133:0/747228694 >> 172.21.15.133:0/747228694 conn(0x7f78a0103080 msgr2=0x7f78a0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:29.485 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.899+0000 7f78a6985640 1 -- 172.21.15.133:0/747228694 shutdown_connections 2024-09-11T23:27:29.485 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:28.899+0000 7f78a6985640 1 -- 172.21.15.133:0/747228694 wait complete. 2024-09-11T23:27:29.485 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T23:27:29.499 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:27:29.500 INFO:teuthology.orchestra.run.smithi133.stdout:Creating mgr... 2024-09-11T23:27:29.500 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T23:27:29.500 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T23:27:29.745 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.x 2024-09-11T23:27:29.746 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to reset failed state of unit ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-11T23:27:29.900 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-42321b88-7095-11ef-bcea-c7b262605968.target.wants/ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-42321b88-7095-11ef-bcea-c7b262605968@.service. 2024-09-11T23:27:30.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:29 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/747228694' entity='client.admin' 2024-09-11T23:27:30.135 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:29 smithi133 systemd[1]: Starting Ceph mgr.x for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:27:30.454 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:30 smithi133 podman[31645]: 2024-09-11 23:27:30.136072346 +0000 UTC m=+0.109511520 container create 85e3425cd9c0cb05067ae350762509afe76cfe703d68e9fd21647c960c4bb476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:27:30.454 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:30 smithi133 podman[31645]: 2024-09-11 23:27:30.042462855 +0000 UTC m=+0.015901997 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:27:30.454 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:30 smithi133 podman[31645]: 2024-09-11 23:27:30.354784423 +0000 UTC m=+0.328223557 container init 85e3425cd9c0cb05067ae350762509afe76cfe703d68e9fd21647c960c4bb476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-11T23:27:30.454 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:30 smithi133 podman[31645]: 2024-09-11 23:27:30.357507931 +0000 UTC m=+0.330947060 container start 85e3425cd9c0cb05067ae350762509afe76cfe703d68e9fd21647c960c4bb476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:27:30.455 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:30 smithi133 bash[31645]: 85e3425cd9c0cb05067ae350762509afe76cfe703d68e9fd21647c960c4bb476 2024-09-11T23:27:30.471 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:27:30.471 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2024-09-11T23:27:30.479 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:27:30.479 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-09-11T23:27:30.479 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2024-09-11T23:27:30.479 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T23:27:30.489 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:27:30.489 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2024-09-11T23:27:30.498 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:27:30.498 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-09-11T23:27:30.498 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2024-09-11T23:27:30.498 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T23:27:30.498 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr to start... 2024-09-11T23:27:30.498 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr... 2024-09-11T23:27:30.709 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:30 smithi133 systemd[1]: Started Ceph mgr.x for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:27:30.709 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:30.480+0000 7fc9a0e28040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:27:30.710 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:30.545+0000 7fc9a0e28040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:27:30.710 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:30.695+0000 7fc9a0e28040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:27:31.022 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:30.766+0000 7fc9a0e28040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:27:31.023 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:30.915+0000 7fc9a0e28040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:27:31.385 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:31 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3983445306' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T23:27:31.385 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:31 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:31.070+0000 7fc9a0e28040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:27:31.688 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:31 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:31.389+0000 7fc9a0e28040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:27:31.689 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:31 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:31.449+0000 7fc9a0e28040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:27:31.689 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:31 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:31.633+0000 7fc9a0e28040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:27:31.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:31.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:27:31.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:27:31.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:31.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T23:27:21:703865+0000", 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T23:27:31.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T23:27:21.704517+0000", 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.953+0000 7feb7061f640 1 Processor -- start 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.953+0000 7feb7061f640 1 -- start start 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.954+0000 7feb7061f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6810a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.954+0000 7feb7061f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feb6807be20 con 0x7feb68108310 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.954+0000 7feb6e3bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6810a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.954+0000 7feb6e3bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6810a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51474/0 (socket says 172.21.15.133:51474) 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.954+0000 7feb6e3bb640 1 -- 172.21.15.133:0/1403024015 learned_addr learned my addr 172.21.15.133:0/1403024015 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.954+0000 7feb6e3bb640 1 -- 172.21.15.133:0/1403024015 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6810ac40 con 0x7feb68108310 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.955+0000 7feb6e3bb640 1 --2- 172.21.15.133:0/1403024015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6810a700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb64009820 tx=0x7feb6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fedd995911a20c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:31.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.956+0000 7feb6d3b9640 1 -- 172.21.15.133:0/1403024015 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb64037070 con 0x7feb68108310 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.956+0000 7feb6d3b9640 1 -- 172.21.15.133:0/1403024015 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feb64032900 con 0x7feb68108310 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.956+0000 7feb6d3b9640 1 -- 172.21.15.133:0/1403024015 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb64030850 con 0x7feb68108310 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.956+0000 7feb7061f640 1 -- 172.21.15.133:0/1403024015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 msgr2=0x7feb6810a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.956+0000 7feb7061f640 1 --2- 172.21.15.133:0/1403024015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6810a700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb64009820 tx=0x7feb6402a270 comp rx=0 tx=0).stop 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.957+0000 7feb7061f640 1 -- 172.21.15.133:0/1403024015 shutdown_connections 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.957+0000 7feb7061f640 1 --2- 172.21.15.133:0/1403024015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6810a700 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.957+0000 7feb7061f640 1 -- 172.21.15.133:0/1403024015 >> 172.21.15.133:0/1403024015 conn(0x7feb68101f10 msgr2=0x7feb68104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.957+0000 7feb7061f640 1 -- 172.21.15.133:0/1403024015 shutdown_connections 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.957+0000 7feb7061f640 1 -- 172.21.15.133:0/1403024015 wait complete. 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.957+0000 7feb7061f640 1 Processor -- start 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.957+0000 7feb7061f640 1 -- start start 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.957+0000 7feb7061f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.957+0000 7feb7061f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feb6810bc10 con 0x7feb68108310 2024-09-11T23:27:31.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.958+0000 7feb6e3bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.958+0000 7feb6e3bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51490/0 (socket says 172.21.15.133:51490) 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.958+0000 7feb6e3bb640 1 -- 172.21.15.133:0/3983445306 learned_addr learned my addr 172.21.15.133:0/3983445306 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.958+0000 7feb6e3bb640 1 -- 172.21.15.133:0/3983445306 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6819bd90 con 0x7feb68108310 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.958+0000 7feb6e3bb640 1 --2- 172.21.15.133:0/3983445306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6819b850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb64031000 tx=0x7feb640326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.958+0000 7feb5f7fe640 1 -- 172.21.15.133:0/3983445306 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb64042070 con 0x7feb68108310 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.958+0000 7feb5f7fe640 1 -- 172.21.15.133:0/3983445306 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feb64032be0 con 0x7feb68108310 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.958+0000 7feb7061f640 1 -- 172.21.15.133:0/3983445306 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6819c080 con 0x7feb68108310 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.958+0000 7feb5f7fe640 1 -- 172.21.15.133:0/3983445306 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb64037040 con 0x7feb68108310 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.958+0000 7feb7061f640 1 -- 172.21.15.133:0/3983445306 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6819ed10 con 0x7feb68108310 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.959+0000 7feb5f7fe640 1 -- 172.21.15.133:0/3983445306 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feb64045070 con 0x7feb68108310 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.959+0000 7feb5f7fe640 1 -- 172.21.15.133:0/3983445306 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feb64047a90 con 0x7feb68108310 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.959+0000 7feb7061f640 1 -- 172.21.15.133:0/3983445306 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb68108770 con 0x7feb68108310 2024-09-11T23:27:31.710 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:30.961+0000 7feb5f7fe640 1 -- 172.21.15.133:0/3983445306 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7feb6404c030 con 0x7feb68108310 2024-09-11T23:27:31.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:31.022+0000 7feb7061f640 1 -- 172.21.15.133:0/3983445306 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7feb6819ef80 con 0x7feb68108310 2024-09-11T23:27:31.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:31.023+0000 7feb5f7fe640 1 -- 172.21.15.133:0/3983445306 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7feb64032d90 con 0x7feb68108310 2024-09-11T23:27:31.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:31.024+0000 7feb7061f640 1 -- 172.21.15.133:0/3983445306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 msgr2=0x7feb6819b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:31.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:31.024+0000 7feb7061f640 1 --2- 172.21.15.133:0/3983445306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6819b850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb64031000 tx=0x7feb640326e0 comp rx=0 tx=0).stop 2024-09-11T23:27:31.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:31.024+0000 7feb7061f640 1 -- 172.21.15.133:0/3983445306 shutdown_connections 2024-09-11T23:27:31.711 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:31.024+0000 7feb7061f640 1 --2- 172.21.15.133:0/3983445306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb68108310 0x7feb6819b850 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:31.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:31.024+0000 7feb7061f640 1 -- 172.21.15.133:0/3983445306 >> 172.21.15.133:0/3983445306 conn(0x7feb68101f10 msgr2=0x7feb68102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:31.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:31.025+0000 7feb7061f640 1 -- 172.21.15.133:0/3983445306 shutdown_connections 2024-09-11T23:27:31.712 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:31.025+0000 7feb7061f640 1 -- 172.21.15.133:0/3983445306 wait complete. 2024-09-11T23:27:31.712 INFO:teuthology.orchestra.run.smithi133.stdout:mgr not available, waiting (1/15)... 2024-09-11T23:27:32.959 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:32 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:32.509+0000 7fc9a0e28040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:27:32.959 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:32 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:32.576+0000 7fc9a0e28040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:27:33.546 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:33 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:33.287+0000 7fc9a0e28040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:27:33.547 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:33 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:33.404+0000 7fc9a0e28040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:27:33.547 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:33 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:33.546+0000 7fc9a0e28040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:27:33.816 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:33 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:33.609+0000 7fc9a0e28040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:27:33.816 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:33 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:33.761+0000 7fc9a0e28040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:27:34.097 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:33 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:33.829+0000 7fc9a0e28040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:27:34.097 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:33 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:33.889+0000 7fc9a0e28040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:27:34.097 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:34.053+0000 7fc9a0e28040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:27:34.350 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:34 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3961207847' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T23:27:34.350 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:34.123+0000 7fc9a0e28040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:27:34.350 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:34.192+0000 7fc9a0e28040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:27:34.351 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:34.285+0000 7fc9a0e28040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:27:34.709 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:34.350+0000 7fc9a0e28040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:27:34.710 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:34.410+0000 7fc9a0e28040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:27:34.794 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:34.797 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:27:34.797 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:27:34.797 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T23:27:34.798 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:34.799 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T23:27:34.799 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T23:27:34.799 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:34.799 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T23:27:34.799 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:34.799 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T23:27:34.799 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T23:27:34.799 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T23:27:34.807 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T23:27:34.807 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T23:27:34.807 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T23:27:34.807 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:34.807 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T23:27:34.807 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T23:27:21:703865+0000", 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T23:27:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T23:27:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T23:27:21.704517+0000", 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.107+0000 7fe53a9f7640 1 Processor -- start 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.107+0000 7fe53a9f7640 1 -- start start 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.107+0000 7fe53a9f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe534105f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.107+0000 7fe53a9f7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe534106500 con 0x7fe534105b30 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.107+0000 7fe533fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe534105f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.107+0000 7fe533fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe534105f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47872/0 (socket says 172.21.15.133:47872) 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.107+0000 7fe533fff640 1 -- 172.21.15.133:0/2660503114 learned_addr learned my addr 172.21.15.133:0/2660503114 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:34.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.108+0000 7fe533fff640 1 -- 172.21.15.133:0/2660503114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe534108b10 con 0x7fe534105b30 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.108+0000 7fe533fff640 1 --2- 172.21.15.133:0/2660503114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe534105f30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe524009820 tx=0x7fe52402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5c95d3f37126cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.109+0000 7fe532ffd640 1 -- 172.21.15.133:0/2660503114 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe524037070 con 0x7fe534105b30 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.109+0000 7fe532ffd640 1 -- 172.21.15.133:0/2660503114 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe524032900 con 0x7fe534105b30 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.109+0000 7fe532ffd640 1 -- 172.21.15.133:0/2660503114 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe524030850 con 0x7fe534105b30 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.109+0000 7fe53a9f7640 1 -- 172.21.15.133:0/2660503114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 msgr2=0x7fe534105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.109+0000 7fe53a9f7640 1 --2- 172.21.15.133:0/2660503114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe534105f30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe524009820 tx=0x7fe52402a270 comp rx=0 tx=0).stop 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.110+0000 7fe53a9f7640 1 -- 172.21.15.133:0/2660503114 shutdown_connections 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.110+0000 7fe53a9f7640 1 --2- 172.21.15.133:0/2660503114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe534105f30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.110+0000 7fe53a9f7640 1 -- 172.21.15.133:0/2660503114 >> 172.21.15.133:0/2660503114 conn(0x7fe5341016e0 msgr2=0x7fe534103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.110+0000 7fe53a9f7640 1 -- 172.21.15.133:0/2660503114 shutdown_connections 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.110+0000 7fe53a9f7640 1 -- 172.21.15.133:0/2660503114 wait complete. 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.110+0000 7fe53a9f7640 1 Processor -- start 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.110+0000 7fe53a9f7640 1 -- start start 2024-09-11T23:27:34.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe53a9f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe53419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe53a9f7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe53410bd40 con 0x7fe534105b30 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe533fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe53419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe533fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe53419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47882/0 (socket says 172.21.15.133:47882) 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe533fff640 1 -- 172.21.15.133:0/3961207847 learned_addr learned my addr 172.21.15.133:0/3961207847 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe533fff640 1 -- 172.21.15.133:0/3961207847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53419be30 con 0x7fe534105b30 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe533fff640 1 --2- 172.21.15.133:0/3961207847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe53419b8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe5240097f0 tx=0x7fe524030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe5317fa640 1 -- 172.21.15.133:0/3961207847 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe524042070 con 0x7fe534105b30 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe5317fa640 1 -- 172.21.15.133:0/3961207847 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe524032ca0 con 0x7fe534105b30 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe53a9f7640 1 -- 172.21.15.133:0/3961207847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53419c0c0 con 0x7fe534105b30 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe5317fa640 1 -- 172.21.15.133:0/3961207847 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe524037040 con 0x7fe534105b30 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.111+0000 7fe53a9f7640 1 -- 172.21.15.133:0/3961207847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53419edb0 con 0x7fe534105b30 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.112+0000 7fe5317fa640 1 -- 172.21.15.133:0/3961207847 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe524045070 con 0x7fe534105b30 2024-09-11T23:27:34.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.112+0000 7fe5317fa640 1 -- 172.21.15.133:0/3961207847 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe524047a40 con 0x7fe534105b30 2024-09-11T23:27:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.113+0000 7fe53a9f7640 1 -- 172.21.15.133:0/3961207847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe534105fb0 con 0x7fe534105b30 2024-09-11T23:27:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.115+0000 7fe5317fa640 1 -- 172.21.15.133:0/3961207847 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe52403da60 con 0x7fe534105b30 2024-09-11T23:27:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.176+0000 7fe53a9f7640 1 -- 172.21.15.133:0/3961207847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe53419ef80 con 0x7fe534105b30 2024-09-11T23:27:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.176+0000 7fe5317fa640 1 -- 172.21.15.133:0/3961207847 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fe52403d430 con 0x7fe534105b30 2024-09-11T23:27:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.178+0000 7fe53a9f7640 1 -- 172.21.15.133:0/3961207847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 msgr2=0x7fe53419b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.178+0000 7fe53a9f7640 1 --2- 172.21.15.133:0/3961207847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe53419b8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe5240097f0 tx=0x7fe524030ec0 comp rx=0 tx=0).stop 2024-09-11T23:27:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.178+0000 7fe53a9f7640 1 -- 172.21.15.133:0/3961207847 shutdown_connections 2024-09-11T23:27:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.178+0000 7fe53a9f7640 1 --2- 172.21.15.133:0/3961207847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe534105b30 0x7fe53419b8f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.178+0000 7fe53a9f7640 1 -- 172.21.15.133:0/3961207847 >> 172.21.15.133:0/3961207847 conn(0x7fe5341016e0 msgr2=0x7fe5341022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.178+0000 7fe53a9f7640 1 -- 172.21.15.133:0/3961207847 shutdown_connections 2024-09-11T23:27:34.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:34.178+0000 7fe53a9f7640 1 -- 172.21.15.133:0/3961207847 wait complete. 2024-09-11T23:27:34.814 INFO:teuthology.orchestra.run.smithi133.stdout:mgr not available, waiting (2/15)... 2024-09-11T23:27:35.459 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:35 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:35.204+0000 7fc9a0e28040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:27:35.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:35 smithi133 ceph-mon[31430]: Activating manager daemon x 2024-09-11T23:27:35.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:35 smithi133 ceph-mon[31430]: mgrmap e2: x(active, starting, since 0.00309776s) 2024-09-11T23:27:35.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:35 smithi133 ceph-mon[31430]: from='mgr.14100 172.21.15.133:0/2811284156' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:27:35.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:35 smithi133 ceph-mon[31430]: from='mgr.14100 172.21.15.133:0/2811284156' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:27:35.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:35 smithi133 ceph-mon[31430]: from='mgr.14100 172.21.15.133:0/2811284156' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:27:35.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:35 smithi133 ceph-mon[31430]: from='mgr.14100 172.21.15.133:0/2811284156' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:27:35.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:35 smithi133 ceph-mon[31430]: from='mgr.14100 172.21.15.133:0/2811284156' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:27:35.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:35 smithi133 ceph-mon[31430]: Manager daemon x is now available 2024-09-11T23:27:35.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:35 smithi133 ceph-mon[31430]: from='mgr.14100 172.21.15.133:0/2811284156' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:27:35.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:35 smithi133 ceph-mon[31430]: from='mgr.14100 172.21.15.133:0/2811284156' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:27:36.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:36 smithi133 ceph-mon[31430]: from='mgr.14100 172.21.15.133:0/2811284156' entity='mgr.x' 2024-09-11T23:27:36.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:36 smithi133 ceph-mon[31430]: from='mgr.14100 172.21.15.133:0/2811284156' entity='mgr.x' 2024-09-11T23:27:36.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:36 smithi133 ceph-mon[31430]: from='mgr.14100 172.21.15.133:0/2811284156' entity='mgr.x' 2024-09-11T23:27:36.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:36 smithi133 ceph-mon[31430]: mgrmap e3: x(active, since 1.00646s) 2024-09-11T23:27:38.044 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:38.044 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:27:38.044 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:27:38.044 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T23:27:38.044 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T23:27:38.045 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T23:27:38.046 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:38.046 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T23:27:38.046 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:38.046 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T23:27:38.046 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T23:27:38.046 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T23:27:38.054 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T23:27:38.054 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T23:27:38.054 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T23:27:38.054 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T23:27:21:703865+0000", 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T23:27:38.055 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:27:38.056 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T23:27:21.704517+0000", 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.216+0000 7fbe821cd640 1 Processor -- start 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.216+0000 7fbe821cd640 1 -- start start 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.216+0000 7fbe821cd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c10a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.216+0000 7fbe821cd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c07be00 con 0x7fbe7c108300 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.216+0000 7fbe7b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c10a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.216+0000 7fbe7b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c10a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47962/0 (socket says 172.21.15.133:47962) 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.216+0000 7fbe7b7fe640 1 -- 172.21.15.133:0/1186363784 learned_addr learned my addr 172.21.15.133:0/1186363784 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.216+0000 7fbe7b7fe640 1 -- 172.21.15.133:0/1186363784 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7c10ac30 con 0x7fbe7c108300 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.217+0000 7fbe7b7fe640 1 --2- 172.21.15.133:0/1186363784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c10a6f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009820 tx=0x7fbe6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6294ae12940f7636 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:38.057 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.217+0000 7fbe7a7fc640 1 -- 172.21.15.133:0/1186363784 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe6c037070 con 0x7fbe7c108300 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.217+0000 7fbe7a7fc640 1 -- 172.21.15.133:0/1186363784 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbe6c032900 con 0x7fbe7c108300 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.217+0000 7fbe7a7fc640 1 -- 172.21.15.133:0/1186363784 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe6c030850 con 0x7fbe7c108300 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.218+0000 7fbe821cd640 1 -- 172.21.15.133:0/1186363784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 msgr2=0x7fbe7c10a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.218+0000 7fbe821cd640 1 --2- 172.21.15.133:0/1186363784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c10a6f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009820 tx=0x7fbe6c02a270 comp rx=0 tx=0).stop 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.218+0000 7fbe821cd640 1 -- 172.21.15.133:0/1186363784 shutdown_connections 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.218+0000 7fbe821cd640 1 --2- 172.21.15.133:0/1186363784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c10a6f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.218+0000 7fbe821cd640 1 -- 172.21.15.133:0/1186363784 >> 172.21.15.133:0/1186363784 conn(0x7fbe7c101ee0 msgr2=0x7fbe7c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.218+0000 7fbe821cd640 1 -- 172.21.15.133:0/1186363784 shutdown_connections 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.218+0000 7fbe821cd640 1 -- 172.21.15.133:0/1186363784 wait complete. 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.218+0000 7fbe821cd640 1 Processor -- start 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.219+0000 7fbe821cd640 1 -- start start 2024-09-11T23:27:38.058 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.219+0000 7fbe821cd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c19b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.219+0000 7fbe821cd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c10bc70 con 0x7fbe7c108300 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.219+0000 7fbe7b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c19b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.219+0000 7fbe7b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c19b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47976/0 (socket says 172.21.15.133:47976) 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.219+0000 7fbe7b7fe640 1 -- 172.21.15.133:0/1966577432 learned_addr learned my addr 172.21.15.133:0/1966577432 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.219+0000 7fbe7b7fe640 1 -- 172.21.15.133:0/1966577432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7c19bd80 con 0x7fbe7c108300 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.219+0000 7fbe7b7fe640 1 --2- 172.21.15.133:0/1966577432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c19b840 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c032ce0 tx=0x7fbe6c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.220+0000 7fbe78ff9640 1 -- 172.21.15.133:0/1966577432 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe6c042070 con 0x7fbe7c108300 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.220+0000 7fbe78ff9640 1 -- 172.21.15.133:0/1966577432 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbe6c0308c0 con 0x7fbe7c108300 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.220+0000 7fbe821cd640 1 -- 172.21.15.133:0/1966577432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7c19c010 con 0x7fbe7c108300 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.220+0000 7fbe78ff9640 1 -- 172.21.15.133:0/1966577432 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe6c037040 con 0x7fbe7c108300 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.220+0000 7fbe821cd640 1 -- 172.21.15.133:0/1966577432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe7c19ed00 con 0x7fbe7c108300 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.220+0000 7fbe78ff9640 1 -- 172.21.15.133:0/1966577432 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fbe6c045070 con 0x7fbe7c108300 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.221+0000 7fbe78ff9640 1 --2- 172.21.15.133:0/1966577432 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fbe68038e60 0x7fbe6803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.221+0000 7fbe821cd640 1 -- 172.21.15.133:0/1966577432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7c108760 con 0x7fbe7c108300 2024-09-11T23:27:38.059 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.221+0000 7fbe78ff9640 1 -- 172.21.15.133:0/1966577432 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbe6c06bd80 con 0x7fbe7c108300 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.221+0000 7fbe7affd640 1 --2- 172.21.15.133:0/1966577432 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fbe68038e60 0x7fbe6803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.221+0000 7fbe7affd640 1 --2- 172.21.15.133:0/1966577432 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fbe68038e60 0x7fbe6803b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe640098c0 tx=0x7fbe64006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.226+0000 7fbe78ff9640 1 -- 172.21.15.133:0/1966577432 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe6c045360 con 0x7fbe7c108300 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.234+0000 7fbe78ff9640 1 -- 172.21.15.133:0/1966577432 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fbe6c03c760 con 0x7fbe7c108300 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.457+0000 7fbe821cd640 1 -- 172.21.15.133:0/1966577432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbe7c19eed0 con 0x7fbe7c108300 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.457+0000 7fbe78ff9640 1 -- 172.21.15.133:0/1966577432 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fbe6c02ed90 con 0x7fbe7c108300 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.459+0000 7fbe821cd640 1 -- 172.21.15.133:0/1966577432 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fbe68038e60 msgr2=0x7fbe6803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.459+0000 7fbe821cd640 1 --2- 172.21.15.133:0/1966577432 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fbe68038e60 0x7fbe6803b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe640098c0 tx=0x7fbe64006db0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.459+0000 7fbe821cd640 1 -- 172.21.15.133:0/1966577432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 msgr2=0x7fbe7c19b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.459+0000 7fbe821cd640 1 --2- 172.21.15.133:0/1966577432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c19b840 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c032ce0 tx=0x7fbe6c032d10 comp rx=0 tx=0).stop 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.459+0000 7fbe821cd640 1 -- 172.21.15.133:0/1966577432 shutdown_connections 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.459+0000 7fbe821cd640 1 --2- 172.21.15.133:0/1966577432 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fbe68038e60 0x7fbe6803b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.459+0000 7fbe821cd640 1 --2- 172.21.15.133:0/1966577432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe7c108300 0x7fbe7c19b840 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.459+0000 7fbe821cd640 1 -- 172.21.15.133:0/1966577432 >> 172.21.15.133:0/1966577432 conn(0x7fbe7c101ee0 msgr2=0x7fbe7c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:38.061 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.460+0000 7fbe821cd640 1 -- 172.21.15.133:0/1966577432 shutdown_connections 2024-09-11T23:27:38.061 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:37.460+0000 7fbe821cd640 1 -- 172.21.15.133:0/1966577432 wait complete. 2024-09-11T23:27:38.061 INFO:teuthology.orchestra.run.smithi133.stdout:mgr is available 2024-09-11T23:27:38.450 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:38 smithi133 ceph-mon[31430]: mgrmap e4: x(active, since 2s) 2024-09-11T23:27:38.451 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:38 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1966577432' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T23:27:39.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:39.384 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T23:27:39.384 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout fsid = 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:27:39.384 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.133:3300,v1:172.21.15.133:6789] 2024-09-11T23:27:39.384 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T23:27:39.384 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T23:27:39.385 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T23:27:39.385 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T23:27:39.385 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:39.385 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T23:27:39.385 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T23:27:39.385 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-09-11T23:27:39.385 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T23:27:39.385 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T23:27:39.385 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T23:27:39.385 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T23:27:39.385 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T23:27:39.386 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T23:27:39.386 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.527+0000 7f3a46e1b640 1 Processor -- start 2024-09-11T23:27:39.386 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.527+0000 7f3a46e1b640 1 -- start start 2024-09-11T23:27:39.386 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.527+0000 7f3a46e1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a40107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:39.386 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.527+0000 7f3a46e1b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a401082a0 con 0x7f3a401078d0 2024-09-11T23:27:39.386 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.527+0000 7f3a44bb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a40107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:39.386 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.527+0000 7f3a44bb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a40107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47978/0 (socket says 172.21.15.133:47978) 2024-09-11T23:27:39.386 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.527+0000 7f3a44bb7640 1 -- 172.21.15.133:0/2283143365 learned_addr learned my addr 172.21.15.133:0/2283143365 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:39.386 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.527+0000 7f3a44bb7640 1 -- 172.21.15.133:0/2283143365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a40108a80 con 0x7f3a401078d0 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.528+0000 7f3a44bb7640 1 --2- 172.21.15.133:0/2283143365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a40107cd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009820 tx=0x7f3a3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e74aa6e80f0b3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.528+0000 7f3a3f7fe640 1 -- 172.21.15.133:0/2283143365 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a34037070 con 0x7f3a401078d0 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.528+0000 7f3a3f7fe640 1 -- 172.21.15.133:0/2283143365 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a34032900 con 0x7f3a401078d0 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.528+0000 7f3a3f7fe640 1 -- 172.21.15.133:0/2283143365 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a34030850 con 0x7f3a401078d0 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.529+0000 7f3a46e1b640 1 -- 172.21.15.133:0/2283143365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 msgr2=0x7f3a40107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.529+0000 7f3a46e1b640 1 --2- 172.21.15.133:0/2283143365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a40107cd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009820 tx=0x7f3a3402a270 comp rx=0 tx=0).stop 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.529+0000 7f3a46e1b640 1 -- 172.21.15.133:0/2283143365 shutdown_connections 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.529+0000 7f3a46e1b640 1 --2- 172.21.15.133:0/2283143365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a40107cd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.529+0000 7f3a46e1b640 1 -- 172.21.15.133:0/2283143365 >> 172.21.15.133:0/2283143365 conn(0x7f3a40103080 msgr2=0x7f3a401054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.529+0000 7f3a46e1b640 1 -- 172.21.15.133:0/2283143365 shutdown_connections 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.529+0000 7f3a46e1b640 1 -- 172.21.15.133:0/2283143365 wait complete. 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.529+0000 7f3a46e1b640 1 Processor -- start 2024-09-11T23:27:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.529+0000 7f3a46e1b640 1 -- start start 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.529+0000 7f3a46e1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a4019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.529+0000 7f3a46e1b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4010be30 con 0x7f3a401078d0 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.530+0000 7f3a44bb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a4019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.530+0000 7f3a44bb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a4019b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47986/0 (socket says 172.21.15.133:47986) 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.530+0000 7f3a44bb7640 1 -- 172.21.15.133:0/3519811813 learned_addr learned my addr 172.21.15.133:0/3519811813 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.530+0000 7f3a44bb7640 1 -- 172.21.15.133:0/3519811813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4019be30 con 0x7f3a401078d0 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.530+0000 7f3a44bb7640 1 --2- 172.21.15.133:0/3519811813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a4019b8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009940 tx=0x7f3a340326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.530+0000 7f3a3dffb640 1 -- 172.21.15.133:0/3519811813 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a34042070 con 0x7f3a401078d0 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.530+0000 7f3a3dffb640 1 -- 172.21.15.133:0/3519811813 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a34032be0 con 0x7f3a401078d0 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.530+0000 7f3a46e1b640 1 -- 172.21.15.133:0/3519811813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4019c0c0 con 0x7f3a401078d0 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.530+0000 7f3a3dffb640 1 -- 172.21.15.133:0/3519811813 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a34037040 con 0x7f3a401078d0 2024-09-11T23:27:39.388 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.530+0000 7f3a46e1b640 1 -- 172.21.15.133:0/3519811813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4019edb0 con 0x7f3a401078d0 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.531+0000 7f3a3dffb640 1 -- 172.21.15.133:0/3519811813 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f3a34045070 con 0x7f3a401078d0 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.531+0000 7f3a46e1b640 1 -- 172.21.15.133:0/3519811813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4010bfb0 con 0x7f3a401078d0 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.531+0000 7f3a3dffb640 1 --2- 172.21.15.133:0/3519811813 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f3a30038f70 0x7f3a3003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.532+0000 7f3a3dffb640 1 -- 172.21.15.133:0/3519811813 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a3406be50 con 0x7f3a401078d0 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.532+0000 7f3a3ffff640 1 --2- 172.21.15.133:0/3519811813 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f3a30038f70 0x7f3a3003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.532+0000 7f3a3ffff640 1 --2- 172.21.15.133:0/3519811813 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f3a30038f70 0x7f3a3003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0098c0 tx=0x7f3a2c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.536+0000 7f3a3dffb640 1 -- 172.21.15.133:0/3519811813 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a3404b170 con 0x7f3a401078d0 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.699+0000 7f3a46e1b640 1 -- 172.21.15.133:0/3519811813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3a40107d50 con 0x7f3a401078d0 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.700+0000 7f3a3dffb640 1 -- 172.21.15.133:0/3519811813 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f3a34032d90 con 0x7f3a401078d0 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.703+0000 7f3a46e1b640 1 -- 172.21.15.133:0/3519811813 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f3a30038f70 msgr2=0x7f3a3003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.703+0000 7f3a46e1b640 1 --2- 172.21.15.133:0/3519811813 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f3a30038f70 0x7f3a3003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0098c0 tx=0x7f3a2c006db0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.703+0000 7f3a46e1b640 1 -- 172.21.15.133:0/3519811813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 msgr2=0x7f3a4019b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:39.389 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.703+0000 7f3a46e1b640 1 --2- 172.21.15.133:0/3519811813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a4019b8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009940 tx=0x7f3a340326e0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.390 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.703+0000 7f3a46e1b640 1 -- 172.21.15.133:0/3519811813 shutdown_connections 2024-09-11T23:27:39.390 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.703+0000 7f3a46e1b640 1 --2- 172.21.15.133:0/3519811813 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f3a30038f70 0x7f3a3003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.390 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.703+0000 7f3a46e1b640 1 --2- 172.21.15.133:0/3519811813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a401078d0 0x7f3a4019b8f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.390 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.703+0000 7f3a46e1b640 1 -- 172.21.15.133:0/3519811813 >> 172.21.15.133:0/3519811813 conn(0x7f3a40103080 msgr2=0x7f3a40103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:39.390 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.703+0000 7f3a46e1b640 1 -- 172.21.15.133:0/3519811813 shutdown_connections 2024-09-11T23:27:39.390 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:38.703+0000 7f3a46e1b640 1 -- 172.21.15.133:0/3519811813 wait complete. 2024-09-11T23:27:39.390 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling cephadm module... 2024-09-11T23:27:39.557 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:39 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3519811813' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T23:27:40.578 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:40 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1970665918' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T23:27:40.578 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:40 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: ignoring --setuser ceph since I am not root 2024-09-11T23:27:40.578 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:40 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: ignoring --setgroup ceph since I am not root 2024-09-11T23:27:40.578 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:40 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:40.373+0000 7f4510dc2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:27:40.579 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:40 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:40.429+0000 7f4510dc2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:27:40.579 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:40 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:40.578+0000 7f4510dc2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:27:40.877 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:40 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:40.645+0000 7f4510dc2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:27:40.877 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:40 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:40.794+0000 7f4510dc2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:27:40.884 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.734+0000 7fcdf17df640 1 Processor -- start 2024-09-11T23:27:40.884 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.734+0000 7fcdf17df640 1 -- start start 2024-09-11T23:27:40.884 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.734+0000 7fcdf17df640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec07c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.884 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.734+0000 7fcdf17df640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec07c7b0 con 0x7fcdec07ddc0 2024-09-11T23:27:40.884 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.734+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec07c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.884 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.734+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec07c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47994/0 (socket says 172.21.15.133:47994) 2024-09-11T23:27:40.884 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.734+0000 7fcdeaffd640 1 -- 172.21.15.133:0/3623499771 learned_addr learned my addr 172.21.15.133:0/3623499771 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.734+0000 7fcdeaffd640 1 -- 172.21.15.133:0/3623499771 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec07c930 con 0x7fcdec07ddc0 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.735+0000 7fcdeaffd640 1 --2- 172.21.15.133:0/3623499771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec07c270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009820 tx=0x7fcddc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71625aa81ad3e91c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.735+0000 7fcde9ffb640 1 -- 172.21.15.133:0/3623499771 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcddc037070 con 0x7fcdec07ddc0 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.735+0000 7fcde9ffb640 1 -- 172.21.15.133:0/3623499771 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcddc032900 con 0x7fcdec07ddc0 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.735+0000 7fcde9ffb640 1 -- 172.21.15.133:0/3623499771 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcddc030850 con 0x7fcdec07ddc0 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.736+0000 7fcdf17df640 1 -- 172.21.15.133:0/3623499771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 msgr2=0x7fcdec07c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.736+0000 7fcdf17df640 1 --2- 172.21.15.133:0/3623499771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec07c270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009820 tx=0x7fcddc02a270 comp rx=0 tx=0).stop 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.736+0000 7fcdf17df640 1 -- 172.21.15.133:0/3623499771 shutdown_connections 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.736+0000 7fcdf17df640 1 --2- 172.21.15.133:0/3623499771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec07c270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.736+0000 7fcdf17df640 1 -- 172.21.15.133:0/3623499771 >> 172.21.15.133:0/3623499771 conn(0x7fcdec103080 msgr2=0x7fcdec1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.736+0000 7fcdf17df640 1 -- 172.21.15.133:0/3623499771 shutdown_connections 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.736+0000 7fcdf17df640 1 -- 172.21.15.133:0/3623499771 wait complete. 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.736+0000 7fcdf17df640 1 Processor -- start 2024-09-11T23:27:40.885 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdf17df640 1 -- start start 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdf17df640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdf17df640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec10df00 con 0x7fcdec07ddc0 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec19dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47998/0 (socket says 172.21.15.133:47998) 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdeaffd640 1 -- 172.21.15.133:0/1970665918 learned_addr learned my addr 172.21.15.133:0/1970665918 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdeaffd640 1 -- 172.21.15.133:0/1970665918 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec19e000 con 0x7fcdec07ddc0 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdeaffd640 1 --2- 172.21.15.133:0/1970665918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec19dac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcddc031000 tx=0x7fcddc0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdd3fff640 1 -- 172.21.15.133:0/1970665918 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcddc042070 con 0x7fcdec07ddc0 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdd3fff640 1 -- 172.21.15.133:0/1970665918 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcddc032be0 con 0x7fcdec07ddc0 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdf17df640 1 -- 172.21.15.133:0/1970665918 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec19e290 con 0x7fcdec07ddc0 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdd3fff640 1 -- 172.21.15.133:0/1970665918 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcddc037040 con 0x7fcdec07ddc0 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.737+0000 7fcdf17df640 1 -- 172.21.15.133:0/1970665918 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1a0f80 con 0x7fcdec07ddc0 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.738+0000 7fcdd3fff640 1 -- 172.21.15.133:0/1970665918 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fcddc045070 con 0x7fcdec07ddc0 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.738+0000 7fcdd3fff640 1 --2- 172.21.15.133:0/1970665918 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fcdd4038f20 0x7fcdd403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.738+0000 7fcdd3fff640 1 -- 172.21.15.133:0/1970665918 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcddc06bf70 con 0x7fcdec07ddc0 2024-09-11T23:27:40.886 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.738+0000 7fcdea7fc640 1 --2- 172.21.15.133:0/1970665918 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fcdd4038f20 0x7fcdd403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.738+0000 7fcdf17df640 1 -- 172.21.15.133:0/1970665918 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec07c2f0 con 0x7fcdec07ddc0 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.739+0000 7fcdea7fc640 1 --2- 172.21.15.133:0/1970665918 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fcdd4038f20 0x7fcdd403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcde00098c0 tx=0x7fcde0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.743+0000 7fcdd3fff640 1 -- 172.21.15.133:0/1970665918 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcddc042210 con 0x7fcdec07ddc0 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:39.952+0000 7fcdf17df640 1 -- 172.21.15.133:0/1970665918 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fcdec1a1360 con 0x7fcdec07ddc0 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.237+0000 7fcdd3fff640 1 -- 172.21.15.133:0/1970665918 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fcddc032d90 con 0x7fcdec07ddc0 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.238+0000 7fcdd3fff640 1 -- 172.21.15.133:0/1970665918 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fcddc03c7b0 con 0x7fcdec07ddc0 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.240+0000 7fcdf17df640 1 -- 172.21.15.133:0/1970665918 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fcdd4038f20 msgr2=0x7fcdd403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.240+0000 7fcdf17df640 1 --2- 172.21.15.133:0/1970665918 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fcdd4038f20 0x7fcdd403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcde00098c0 tx=0x7fcde0006db0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.240+0000 7fcdf17df640 1 -- 172.21.15.133:0/1970665918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 msgr2=0x7fcdec19dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.240+0000 7fcdf17df640 1 --2- 172.21.15.133:0/1970665918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec19dac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcddc031000 tx=0x7fcddc0326e0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.240+0000 7fcdf17df640 1 -- 172.21.15.133:0/1970665918 shutdown_connections 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.240+0000 7fcdf17df640 1 --2- 172.21.15.133:0/1970665918 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7fcdd4038f20 0x7fcdd403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.240+0000 7fcdf17df640 1 --2- 172.21.15.133:0/1970665918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdec07ddc0 0x7fcdec19dac0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.240+0000 7fcdf17df640 1 -- 172.21.15.133:0/1970665918 >> 172.21.15.133:0/1970665918 conn(0x7fcdec103080 msgr2=0x7fcdec103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:40.887 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.241+0000 7fcdf17df640 1 -- 172.21.15.133:0/1970665918 shutdown_connections 2024-09-11T23:27:40.888 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:40.241+0000 7fcdf17df640 1 -- 172.21.15.133:0/1970665918 wait complete. 2024-09-11T23:27:41.209 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:40 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:40.939+0000 7f4510dc2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:27:41.498 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:41 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1970665918' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T23:27:41.498 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:41 smithi133 ceph-mon[31430]: mgrmap e5: x(active, since 5s) 2024-09-11T23:27:41.498 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:41 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:41.265+0000 7f4510dc2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:27:41.498 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:41 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:41.329+0000 7f4510dc2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:27:41.874 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:41 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:41.520+0000 7f4510dc2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:27:42.216 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.291+0000 7f1c1caee640 1 Processor -- start 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.291+0000 7f1c1caee640 1 -- start start 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.291+0000 7f1c1caee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c18106560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.291+0000 7f1c1caee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1807bdc0 con 0x7f1c18106160 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.292+0000 7f1c1659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c18106560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.292+0000 7f1c1659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c18106560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48018/0 (socket says 172.21.15.133:48018) 2024-09-11T23:27:42.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.292+0000 7f1c1659c640 1 -- 172.21.15.133:0/263884079 learned_addr learned my addr 172.21.15.133:0/263884079 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.292+0000 7f1c1659c640 1 -- 172.21.15.133:0/263884079 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c18106aa0 con 0x7f1c18106160 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.293+0000 7f1c1659c640 1 --2- 172.21.15.133:0/263884079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c18106560 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009820 tx=0x7f1c0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fadf9e28f3c6ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.293+0000 7f1c15d9b640 1 -- 172.21.15.133:0/263884079 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c08037070 con 0x7f1c18106160 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.293+0000 7f1c15d9b640 1 -- 172.21.15.133:0/263884079 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c08032900 con 0x7f1c18106160 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.293+0000 7f1c15d9b640 1 -- 172.21.15.133:0/263884079 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c08030850 con 0x7f1c18106160 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.294+0000 7f1c1caee640 1 -- 172.21.15.133:0/263884079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 msgr2=0x7f1c18106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.294+0000 7f1c1caee640 1 --2- 172.21.15.133:0/263884079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c18106560 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009820 tx=0x7f1c0802a270 comp rx=0 tx=0).stop 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.294+0000 7f1c1caee640 1 -- 172.21.15.133:0/263884079 shutdown_connections 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.294+0000 7f1c1caee640 1 --2- 172.21.15.133:0/263884079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c18106560 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.294+0000 7f1c1caee640 1 -- 172.21.15.133:0/263884079 >> 172.21.15.133:0/263884079 conn(0x7f1c18101f90 msgr2=0x7f1c181043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.294+0000 7f1c1caee640 1 -- 172.21.15.133:0/263884079 shutdown_connections 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.294+0000 7f1c1caee640 1 -- 172.21.15.133:0/263884079 wait complete. 2024-09-11T23:27:42.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.295+0000 7f1c1caee640 1 Processor -- start 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.295+0000 7f1c1caee640 1 -- start start 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.295+0000 7f1c1caee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c1819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.295+0000 7f1c1caee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18107ba0 con 0x7f1c18106160 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.295+0000 7f1c1659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c1819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.295+0000 7f1c1659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c1819da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48026/0 (socket says 172.21.15.133:48026) 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.295+0000 7f1c1659c640 1 -- 172.21.15.133:0/2150281121 learned_addr learned my addr 172.21.15.133:0/2150281121 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.296+0000 7f1c1659c640 1 -- 172.21.15.133:0/2150281121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1819dfa0 con 0x7f1c18106160 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.296+0000 7f1c1659c640 1 --2- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c1819da60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c08031000 tx=0x7f1c080326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.296+0000 7f1c0f7fe640 1 -- 172.21.15.133:0/2150281121 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c08042070 con 0x7f1c18106160 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.296+0000 7f1c1caee640 1 -- 172.21.15.133:0/2150281121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1819e230 con 0x7f1c18106160 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.296+0000 7f1c0f7fe640 1 -- 172.21.15.133:0/2150281121 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c08032be0 con 0x7f1c18106160 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.296+0000 7f1c0f7fe640 1 -- 172.21.15.133:0/2150281121 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c08037040 con 0x7f1c18106160 2024-09-11T23:27:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.296+0000 7f1c1caee640 1 -- 172.21.15.133:0/2150281121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c181a0f20 con 0x7f1c18106160 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.297+0000 7f1c0f7fe640 1 -- 172.21.15.133:0/2150281121 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f1c08045070 con 0x7f1c18106160 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.297+0000 7f1c1caee640 1 -- 172.21.15.133:0/2150281121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c18106560 con 0x7f1c18106160 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.297+0000 7f1c0f7fe640 1 --2- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f1c04038f70 0x7f1c0403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.297+0000 7f1c0f7fe640 1 -- 172.21.15.133:0/2150281121 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1c0806bfd0 con 0x7f1c18106160 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.297+0000 7f1c0ffff640 1 -- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f1c04038f70 msgr2=0x7f1c0403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2855546700 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.297+0000 7f1c0ffff640 1 --2- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f1c04038f70 0x7f1c0403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.302+0000 7f1c0f7fe640 1 -- 172.21.15.133:0/2150281121 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c08042210 con 0x7f1c18106160 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.498+0000 7f1c0ffff640 1 -- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f1c04038f70 msgr2=0x7f1c0403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2855546700 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.498+0000 7f1c0ffff640 1 --2- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f1c04038f70 0x7f1c0403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.520+0000 7f1c1caee640 1 -- 172.21.15.133:0/2150281121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1c1802d940 con 0x7f1c18106160 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.521+0000 7f1c0f7fe640 1 -- 172.21.15.133:0/2150281121 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f1c08032d90 con 0x7f1c18106160 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.523+0000 7f1c1caee640 1 -- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f1c04038f70 msgr2=0x7f1c0403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:27:42.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.523+0000 7f1c1caee640 1 --2- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f1c04038f70 0x7f1c0403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:42.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.523+0000 7f1c1caee640 1 -- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 msgr2=0x7f1c1819da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:42.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.523+0000 7f1c1caee640 1 --2- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c1819da60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c08031000 tx=0x7f1c080326e0 comp rx=0 tx=0).stop 2024-09-11T23:27:42.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.523+0000 7f1c1caee640 1 -- 172.21.15.133:0/2150281121 shutdown_connections 2024-09-11T23:27:42.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.523+0000 7f1c1caee640 1 --2- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f1c04038f70 0x7f1c0403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:42.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.523+0000 7f1c1caee640 1 --2- 172.21.15.133:0/2150281121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c18106160 0x7f1c1819da60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:42.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.523+0000 7f1c1caee640 1 -- 172.21.15.133:0/2150281121 >> 172.21.15.133:0/2150281121 conn(0x7f1c18101f90 msgr2=0x7f1c18194700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:42.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.523+0000 7f1c1caee640 1 -- 172.21.15.133:0/2150281121 shutdown_connections 2024-09-11T23:27:42.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:41.523+0000 7f1c1caee640 1 -- 172.21.15.133:0/2150281121 wait complete. 2024-09-11T23:27:42.221 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for the mgr to restart... 2024-09-11T23:27:42.221 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr epoch 5... 2024-09-11T23:27:42.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:42 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2150281121' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T23:27:42.459 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:42.411+0000 7f4510dc2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:27:42.821 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:42.481+0000 7f4510dc2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:27:43.446 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:43.190+0000 7f4510dc2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:27:43.446 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:43.306+0000 7f4510dc2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:27:43.709 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:43.446+0000 7f4510dc2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:27:43.709 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:43.506+0000 7f4510dc2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:27:43.709 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:43.653+0000 7f4510dc2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:27:44.000 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:43.721+0000 7f4510dc2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:27:44.000 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:43.780+0000 7f4510dc2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:27:44.000 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:43.936+0000 7f4510dc2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:27:44.000 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:44 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:43.999+0000 7f4510dc2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:27:44.268 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:44 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:44.067+0000 7f4510dc2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:27:44.268 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:44 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:44.150+0000 7f4510dc2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:27:44.268 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:44 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:44.208+0000 7f4510dc2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:27:44.268 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:44 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:44.268+0000 7f4510dc2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:27:45.459 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:45.043+0000 7f4510dc2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:27:45.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: Active manager daemon x restarted 2024-09-11T23:27:45.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: Activating manager daemon x 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: mgrmap e6: x(active, starting, since 0.0034007s) 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: Manager daemon x is now available 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:27:45.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:45 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:27:46.382 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:46 smithi133 ceph-mon[31430]: Found migration_current of "None". Setting to last migration. 2024-09-11T23:27:46.382 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:46 smithi133 ceph-mon[31430]: mgrmap e7: x(active, since 1.00498s) 2024-09-11T23:27:46.682 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:27:46.682 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T23:27:46.682 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T23:27:46.682 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:27:46.682 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.615+0000 7f94fb515640 1 Processor -- start 2024-09-11T23:27:46.682 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.615+0000 7f94fb515640 1 -- start start 2024-09-11T23:27:46.682 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.615+0000 7f94fb515640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f410a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.615+0000 7f94fb515640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94f407bd80 con 0x7f94f41083b0 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.615+0000 7f94f92b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f410a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.615+0000 7f94f92b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f410a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35738/0 (socket says 172.21.15.133:35738) 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.615+0000 7f94f92b1640 1 -- 172.21.15.133:0/3238422001 learned_addr learned my addr 172.21.15.133:0/3238422001 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.615+0000 7f94f92b1640 1 -- 172.21.15.133:0/3238422001 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f410ace0 con 0x7f94f41083b0 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.616+0000 7f94f92b1640 1 --2- 172.21.15.133:0/3238422001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f410a7a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009820 tx=0x7f94f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a547da600b86f67a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.616+0000 7f94ebfff640 1 -- 172.21.15.133:0/3238422001 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94f0037070 con 0x7f94f41083b0 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.616+0000 7f94ebfff640 1 -- 172.21.15.133:0/3238422001 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94f0032900 con 0x7f94f41083b0 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.616+0000 7f94ebfff640 1 -- 172.21.15.133:0/3238422001 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94f0030850 con 0x7f94f41083b0 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.617+0000 7f94fb515640 1 -- 172.21.15.133:0/3238422001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 msgr2=0x7f94f410a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.617+0000 7f94fb515640 1 --2- 172.21.15.133:0/3238422001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f410a7a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009820 tx=0x7f94f002a270 comp rx=0 tx=0).stop 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.617+0000 7f94fb515640 1 -- 172.21.15.133:0/3238422001 shutdown_connections 2024-09-11T23:27:46.683 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.617+0000 7f94fb515640 1 --2- 172.21.15.133:0/3238422001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f410a7a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.617+0000 7f94fb515640 1 -- 172.21.15.133:0/3238422001 >> 172.21.15.133:0/3238422001 conn(0x7f94f4101f10 msgr2=0x7f94f4104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.617+0000 7f94fb515640 1 -- 172.21.15.133:0/3238422001 shutdown_connections 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.617+0000 7f94fb515640 1 -- 172.21.15.133:0/3238422001 wait complete. 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.618+0000 7f94fb515640 1 Processor -- start 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.618+0000 7f94fb515640 1 -- start start 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.618+0000 7f94fb515640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.618+0000 7f94fb515640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94f410bcc0 con 0x7f94f41083b0 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.618+0000 7f94f92b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.618+0000 7f94f92b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35740/0 (socket says 172.21.15.133:35740) 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.618+0000 7f94f92b1640 1 -- 172.21.15.133:0/386229153 learned_addr learned my addr 172.21.15.133:0/386229153 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.618+0000 7f94f92b1640 1 -- 172.21.15.133:0/386229153 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f419be30 con 0x7f94f41083b0 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.618+0000 7f94f92b1640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f419b8f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f94f0031000 tx=0x7f94f00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.619+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94f0042070 con 0x7f94f41083b0 2024-09-11T23:27:46.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.619+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94f0032be0 con 0x7f94f41083b0 2024-09-11T23:27:46.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.619+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94f0037040 con 0x7f94f41083b0 2024-09-11T23:27:46.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.619+0000 7f94fb515640 1 -- 172.21.15.133:0/386229153 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f419c0c0 con 0x7f94f41083b0 2024-09-11T23:27:46.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.619+0000 7f94fb515640 1 -- 172.21.15.133:0/386229153 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f419edb0 con 0x7f94f41083b0 2024-09-11T23:27:46.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.620+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f94f0045070 con 0x7f94f41083b0 2024-09-11T23:27:46.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.620+0000 7f94ea7fc640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 0x7f94e003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.620+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 --> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f94e003bb90 con 0x7f94e0038fc0 2024-09-11T23:27:46.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.620+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f94f006bf80 con 0x7f94f41083b0 2024-09-11T23:27:46.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.620+0000 7f94f8ab0640 1 -- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 msgr2=0x7f94e003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2855546700 2024-09-11T23:27:46.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.620+0000 7f94f8ab0640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 0x7f94e003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.821+0000 7f94f8ab0640 1 -- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 msgr2=0x7f94e003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2855546700 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:42.821+0000 7f94f8ab0640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 0x7f94e003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:43.222+0000 7f94f8ab0640 1 -- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 msgr2=0x7f94e003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2855546700 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:43.222+0000 7f94f8ab0640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 0x7f94e003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:44.023+0000 7f94f8ab0640 1 -- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 msgr2=0x7f94e003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2855546700 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:44.023+0000 7f94f8ab0640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 0x7f94e003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:45.047+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f94f006aa40 con 0x7f94f41083b0 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:45.047+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 msgr2=0x7f94e003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:45.047+0000 7f94ea7fc640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 0x7f94e003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.048+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f94f003d900 con 0x7f94f41083b0 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.048+0000 7f94ea7fc640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f94e003c8f0 0x7f94e003ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.048+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 --> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f94e003bb90 con 0x7f94e003c8f0 2024-09-11T23:27:46.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.049+0000 7f94f8ab0640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f94e003c8f0 0x7f94e003ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.049+0000 7f94f8ab0640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f94e003c8f0 0x7f94e003ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f94e4003980 tx=0x7f94e40092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.050+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 <== mgr.14118 v2:172.21.15.133:6800/2152407978 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f94e003bb90 con 0x7f94e003c8f0 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.056+0000 7f94fb515640 1 -- 172.21.15.133:0/386229153 --> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f94f419c5a0 con 0x7f94e003c8f0 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94ea7fc640 1 -- 172.21.15.133:0/386229153 <== mgr.14118 v2:172.21.15.133:6800/2152407978 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f94f419c5a0 con 0x7f94e003c8f0 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94fb515640 1 -- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f94e003c8f0 msgr2=0x7f94e003ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94fb515640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f94e003c8f0 0x7f94e003ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f94e4003980 tx=0x7f94e40092b0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94fb515640 1 -- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 msgr2=0x7f94f419b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94fb515640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f419b8f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f94f0031000 tx=0x7f94f00326e0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94fb515640 1 -- 172.21.15.133:0/386229153 shutdown_connections 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94fb515640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f94e003c8f0 0x7f94e003ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94fb515640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:6800/2855546700,v1:172.21.15.133:6801/2855546700] conn(0x7f94e0038fc0 0x7f94e003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94fb515640 1 --2- 172.21.15.133:0/386229153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94f41083b0 0x7f94f419b8f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94fb515640 1 -- 172.21.15.133:0/386229153 >> 172.21.15.133:0/386229153 conn(0x7f94f4101f10 msgr2=0x7f94f4102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94fb515640 1 -- 172.21.15.133:0/386229153 shutdown_connections 2024-09-11T23:27:46.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:46.057+0000 7f94fb515640 1 -- 172.21.15.133:0/386229153 wait complete. 2024-09-11T23:27:46.688 INFO:teuthology.orchestra.run.smithi133.stdout:mgr epoch 5 is available 2024-09-11T23:27:46.688 INFO:teuthology.orchestra.run.smithi133.stdout:Setting orchestrator backend to cephadm... 2024-09-11T23:27:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-mon[31430]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T23:27:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-mon[31430]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T23:27:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:27:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:47.129+0000 7f447811c640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:27:47] ENGINE Error in HTTPServer.serve 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: Traceback (most recent call last): 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self._connections.run(self.expiration_interval) 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self._run(expiration_interval) 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:27:47.210 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: s = self.context.wrap_socket( 2024-09-11T23:27:47.211 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:27:47.211 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: return self.sslsocket_class._create( 2024-09-11T23:27:47.211 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:27:47.211 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self.do_handshake() 2024-09-11T23:27:47.211 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:27:47.211 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self._sslobj.do_handshake() 2024-09-11T23:27:47.211 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:27:47.211 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:27:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.112+0000 7f40b3e6f640 1 Processor -- start 2024-09-11T23:27:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.112+0000 7f40b3e6f640 1 -- start start 2024-09-11T23:27:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.113+0000 7f40b3e6f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.113+0000 7f40b3e6f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac1086f0 con 0x7f40ac107d20 2024-09-11T23:27:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.113+0000 7f40b1c0b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.113+0000 7f40b1c0b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35788/0 (socket says 172.21.15.133:35788) 2024-09-11T23:27:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.113+0000 7f40b1c0b640 1 -- 172.21.15.133:0/3595351166 learned_addr learned my addr 172.21.15.133:0/3595351166 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.113+0000 7f40b1c0b640 1 -- 172.21.15.133:0/3595351166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac106980 con 0x7f40ac107d20 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.114+0000 7f40b1c0b640 1 --2- 172.21.15.133:0/3595351166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac108120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009820 tx=0x7f40a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4700da2df60ce55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.114+0000 7f40b0c09640 1 -- 172.21.15.133:0/3595351166 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40a0037070 con 0x7f40ac107d20 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.114+0000 7f40b0c09640 1 -- 172.21.15.133:0/3595351166 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f40a0032900 con 0x7f40ac107d20 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.114+0000 7f40b0c09640 1 -- 172.21.15.133:0/3595351166 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40a0030850 con 0x7f40ac107d20 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.114+0000 7f40b3e6f640 1 -- 172.21.15.133:0/3595351166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 msgr2=0x7f40ac108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.114+0000 7f40b3e6f640 1 --2- 172.21.15.133:0/3595351166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac108120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009820 tx=0x7f40a002a270 comp rx=0 tx=0).stop 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.114+0000 7f40b3e6f640 1 -- 172.21.15.133:0/3595351166 shutdown_connections 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.114+0000 7f40b3e6f640 1 --2- 172.21.15.133:0/3595351166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac108120 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.114+0000 7f40b3e6f640 1 -- 172.21.15.133:0/3595351166 >> 172.21.15.133:0/3595351166 conn(0x7f40ac1016e0 msgr2=0x7f40ac103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.115+0000 7f40b3e6f640 1 -- 172.21.15.133:0/3595351166 shutdown_connections 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.115+0000 7f40b3e6f640 1 -- 172.21.15.133:0/3595351166 wait complete. 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.115+0000 7f40b3e6f640 1 Processor -- start 2024-09-11T23:27:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.115+0000 7f40b3e6f640 1 -- start start 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.115+0000 7f40b3e6f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.115+0000 7f40b3e6f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac10ad40 con 0x7f40ac107d20 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.115+0000 7f40b1c0b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.115+0000 7f40b1c0b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35790/0 (socket says 172.21.15.133:35790) 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.115+0000 7f40b1c0b640 1 -- 172.21.15.133:0/1933768519 learned_addr learned my addr 172.21.15.133:0/1933768519 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.116+0000 7f40b1c0b640 1 -- 172.21.15.133:0/1933768519 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac19ad20 con 0x7f40ac107d20 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.116+0000 7f40b1c0b640 1 --2- 172.21.15.133:0/1933768519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac19a7e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f40a0030460 tx=0x7f40a0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.116+0000 7f409effd640 1 -- 172.21.15.133:0/1933768519 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40a0042070 con 0x7f40ac107d20 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.116+0000 7f40b3e6f640 1 -- 172.21.15.133:0/1933768519 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ac19afb0 con 0x7f40ac107d20 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.116+0000 7f409effd640 1 -- 172.21.15.133:0/1933768519 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f40a0032be0 con 0x7f40ac107d20 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.116+0000 7f409effd640 1 -- 172.21.15.133:0/1933768519 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40a0037040 con 0x7f40ac107d20 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.116+0000 7f40b3e6f640 1 -- 172.21.15.133:0/1933768519 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ac19dca0 con 0x7f40ac107d20 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.117+0000 7f409effd640 1 -- 172.21.15.133:0/1933768519 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f40a0045070 con 0x7f40ac107d20 2024-09-11T23:27:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.117+0000 7f409effd640 1 --2- 172.21.15.133:0/1933768519 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f4098038e60 0x7f409803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.117+0000 7f409effd640 1 -- 172.21.15.133:0/1933768519 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f40a006bcf0 con 0x7f40ac107d20 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.117+0000 7f40b3e6f640 1 -- 172.21.15.133:0/1933768519 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ac108120 con 0x7f40ac107d20 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.117+0000 7f40b140a640 1 --2- 172.21.15.133:0/1933768519 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f4098038e60 0x7f409803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.117+0000 7f40b140a640 1 --2- 172.21.15.133:0/1933768519 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f4098038e60 0x7f409803b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40a80098c0 tx=0x7f40a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.122+0000 7f409effd640 1 -- 172.21.15.133:0/1933768519 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40a0045360 con 0x7f40ac107d20 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.306+0000 7f40b3e6f640 1 -- 172.21.15.133:0/1933768519 --> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f40ac102960 con 0x7f4098038e60 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.311+0000 7f409effd640 1 -- 172.21.15.133:0/1933768519 <== mgr.14118 v2:172.21.15.133:6800/2152407978 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f40ac102960 con 0x7f4098038e60 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.313+0000 7f40b3e6f640 1 -- 172.21.15.133:0/1933768519 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f4098038e60 msgr2=0x7f409803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.313+0000 7f40b3e6f640 1 --2- 172.21.15.133:0/1933768519 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f4098038e60 0x7f409803b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40a80098c0 tx=0x7f40a8006db0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.313+0000 7f40b3e6f640 1 -- 172.21.15.133:0/1933768519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 msgr2=0x7f40ac19a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.313+0000 7f40b3e6f640 1 --2- 172.21.15.133:0/1933768519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac19a7e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f40a0030460 tx=0x7f40a0030d80 comp rx=0 tx=0).stop 2024-09-11T23:27:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.313+0000 7f40b3e6f640 1 -- 172.21.15.133:0/1933768519 shutdown_connections 2024-09-11T23:27:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.313+0000 7f40b3e6f640 1 --2- 172.21.15.133:0/1933768519 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f4098038e60 0x7f409803b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.313+0000 7f40b3e6f640 1 --2- 172.21.15.133:0/1933768519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40ac107d20 0x7f40ac19a7e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.313+0000 7f40b3e6f640 1 -- 172.21.15.133:0/1933768519 >> 172.21.15.133:0/1933768519 conn(0x7f40ac1016e0 msgr2=0x7f40ac102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.314+0000 7f40b3e6f640 1 -- 172.21.15.133:0/1933768519 shutdown_connections 2024-09-11T23:27:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:47.314+0000 7f40b3e6f640 1 -- 172.21.15.133:0/1933768519 wait complete. 2024-09-11T23:27:48.198 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:48 smithi133 ceph-mon[31430]: [11/Sep/2024:23:27:46] ENGINE Bus STARTING 2024-09-11T23:27:48.198 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:48 smithi133 ceph-mon[31430]: [11/Sep/2024:23:27:47] ENGINE Serving on http://172.21.15.133:8765 2024-09-11T23:27:48.198 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:48 smithi133 ceph-mon[31430]: [11/Sep/2024:23:27:47] ENGINE Serving on https://172.21.15.133:7150 2024-09-11T23:27:48.198 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:48 smithi133 ceph-mon[31430]: [11/Sep/2024:23:27:47] ENGINE Bus STARTED 2024-09-11T23:27:48.198 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:48 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:27:48.198 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:48 smithi133 ceph-mon[31430]: [11/Sep/2024:23:27:47] ENGINE Error in HTTPServer.serve 2024-09-11T23:27:48.198 INFO:journalctl@ceph.mon.a.smithi133.stdout: Traceback (most recent call last): 2024-09-11T23:27:48.198 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:27:48.198 INFO:journalctl@ceph.mon.a.smithi133.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: self._run(expiration_interval) 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: s = self.context.wrap_socket( 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: return self.sslsocket_class._create( 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: self.do_handshake() 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: self._sslobj.do_handshake() 2024-09-11T23:27:48.199 INFO:journalctl@ceph.mon.a.smithi133.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:27:48.200 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:48 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:27:48.200 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:48 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:27:48.200 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:48 smithi133 ceph-mon[31430]: mgrmap e8: x(active, since 2s) 2024-09-11T23:27:49.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T23:27:49.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.275+0000 7f1b03d8a640 1 Processor -- start 2024-09-11T23:27:49.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.275+0000 7f1b03d8a640 1 -- start start 2024-09-11T23:27:49.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.275+0000 7f1b03d8a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc106310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:49.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.275+0000 7f1b03d8a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc106850 con 0x7f1afc105f10 2024-09-11T23:27:49.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.275+0000 7f1b01b26640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc106310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.275+0000 7f1b01b26640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc106310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35794/0 (socket says 172.21.15.133:35794) 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.275+0000 7f1b01b26640 1 -- 172.21.15.133:0/3348716943 learned_addr learned my addr 172.21.15.133:0/3348716943 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.276+0000 7f1b01b26640 1 -- 172.21.15.133:0/3348716943 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc107070 con 0x7f1afc105f10 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.276+0000 7f1b01b26640 1 --2- 172.21.15.133:0/3348716943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc106310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1af800a9c0 tx=0x7f1af802b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451288ca735c777c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.277+0000 7f1b00b24640 1 -- 172.21.15.133:0/3348716943 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1af800b040 con 0x7f1afc105f10 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.277+0000 7f1b00b24640 1 -- 172.21.15.133:0/3348716943 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1af8030b00 con 0x7f1afc105f10 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.277+0000 7f1b00b24640 1 -- 172.21.15.133:0/3348716943 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1af8032cb0 con 0x7f1afc105f10 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.277+0000 7f1b03d8a640 1 -- 172.21.15.133:0/3348716943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 msgr2=0x7f1afc106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.277+0000 7f1b03d8a640 1 --2- 172.21.15.133:0/3348716943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc106310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1af800a9c0 tx=0x7f1af802b610 comp rx=0 tx=0).stop 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.277+0000 7f1b03d8a640 1 -- 172.21.15.133:0/3348716943 shutdown_connections 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.277+0000 7f1b03d8a640 1 --2- 172.21.15.133:0/3348716943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc106310 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.277+0000 7f1b03d8a640 1 -- 172.21.15.133:0/3348716943 >> 172.21.15.133:0/3348716943 conn(0x7f1afc101700 msgr2=0x7f1afc103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.277+0000 7f1b03d8a640 1 -- 172.21.15.133:0/3348716943 shutdown_connections 2024-09-11T23:27:49.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.277+0000 7f1b03d8a640 1 -- 172.21.15.133:0/3348716943 wait complete. 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.278+0000 7f1b03d8a640 1 Processor -- start 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.278+0000 7f1b03d8a640 1 -- start start 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.278+0000 7f1b03d8a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.278+0000 7f1b03d8a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc1077e0 con 0x7f1afc105f10 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.278+0000 7f1b01b26640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.278+0000 7f1b01b26640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc19bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35800/0 (socket says 172.21.15.133:35800) 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.278+0000 7f1b01b26640 1 -- 172.21.15.133:0/2933987656 learned_addr learned my addr 172.21.15.133:0/2933987656 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.278+0000 7f1b01b26640 1 -- 172.21.15.133:0/2933987656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc19c090 con 0x7f1afc105f10 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.279+0000 7f1b01b26640 1 --2- 172.21.15.133:0/2933987656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc19bb50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1af80329c0 tx=0x7f1af8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.279+0000 7f1af2ffd640 1 -- 172.21.15.133:0/2933987656 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1af803f070 con 0x7f1afc105f10 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.279+0000 7f1af2ffd640 1 -- 172.21.15.133:0/2933987656 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1af803b8f0 con 0x7f1afc105f10 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.279+0000 7f1b03d8a640 1 -- 172.21.15.133:0/2933987656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc19c320 con 0x7f1afc105f10 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.279+0000 7f1af2ffd640 1 -- 172.21.15.133:0/2933987656 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1af803a890 con 0x7f1afc105f10 2024-09-11T23:27:49.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.279+0000 7f1b03d8a640 1 -- 172.21.15.133:0/2933987656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc19f010 con 0x7f1afc105f10 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.279+0000 7f1af2ffd640 1 -- 172.21.15.133:0/2933987656 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1af8042070 con 0x7f1afc105f10 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.280+0000 7f1b03d8a640 1 -- 172.21.15.133:0/2933987656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1afc04ffb0 con 0x7f1afc105f10 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.280+0000 7f1af2ffd640 1 --2- 172.21.15.133:0/2933987656 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f1ae8038fc0 0x7f1ae803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.280+0000 7f1af2ffd640 1 -- 172.21.15.133:0/2933987656 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1af806ccb0 con 0x7f1afc105f10 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.280+0000 7f1b01325640 1 --2- 172.21.15.133:0/2933987656 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f1ae8038fc0 0x7f1ae803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.280+0000 7f1b01325640 1 --2- 172.21.15.133:0/2933987656 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f1ae8038fc0 0x7f1ae803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0098c0 tx=0x7f1aec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.284+0000 7f1af2ffd640 1 -- 172.21.15.133:0/2933987656 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1af80313a0 con 0x7f1afc105f10 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.451+0000 7f1b03d8a640 1 -- 172.21.15.133:0/2933987656 --> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1afc074c70 con 0x7f1ae8038fc0 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.452+0000 7f1af2ffd640 1 -- 172.21.15.133:0/2933987656 <== mgr.14118 v2:172.21.15.133:6800/2152407978 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1afc074c70 con 0x7f1ae8038fc0 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.454+0000 7f1b03d8a640 1 -- 172.21.15.133:0/2933987656 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f1ae8038fc0 msgr2=0x7f1ae803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.454+0000 7f1b03d8a640 1 --2- 172.21.15.133:0/2933987656 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f1ae8038fc0 0x7f1ae803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0098c0 tx=0x7f1aec006db0 comp rx=0 tx=0).stop 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.454+0000 7f1b03d8a640 1 -- 172.21.15.133:0/2933987656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 msgr2=0x7f1afc19bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.454+0000 7f1b03d8a640 1 --2- 172.21.15.133:0/2933987656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc19bb50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1af80329c0 tx=0x7f1af8032e10 comp rx=0 tx=0).stop 2024-09-11T23:27:49.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.454+0000 7f1b03d8a640 1 -- 172.21.15.133:0/2933987656 shutdown_connections 2024-09-11T23:27:49.079 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.454+0000 7f1b03d8a640 1 --2- 172.21.15.133:0/2933987656 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f1ae8038fc0 0x7f1ae803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:49.079 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.454+0000 7f1b03d8a640 1 --2- 172.21.15.133:0/2933987656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1afc105f10 0x7f1afc19bb50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:49.079 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.454+0000 7f1b03d8a640 1 -- 172.21.15.133:0/2933987656 >> 172.21.15.133:0/2933987656 conn(0x7f1afc101700 msgr2=0x7f1afc18d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:49.079 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.454+0000 7f1b03d8a640 1 -- 172.21.15.133:0/2933987656 shutdown_connections 2024-09-11T23:27:49.079 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:48.454+0000 7f1b03d8a640 1 -- 172.21.15.133:0/2933987656 wait complete. 2024-09-11T23:27:49.079 INFO:teuthology.orchestra.run.smithi133.stdout:Generating ssh key... 2024-09-11T23:27:49.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-mon[31430]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: Generating public/private rsa key pair. 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: Your identification has been saved in /tmp/tmpcu0xb4__/key 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: Your public key has been saved in /tmp/tmpcu0xb4__/key.pub 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: The key fingerprint is: 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: SHA256:4DbUYZZzD77mjngPHBD97JjqPY8g0i2A0OF4piTMBg0 ceph-42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: The key's randomart image is: 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: +---[RSA 3072]----+ 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: |Eo. ..+. | 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: |++.. =+.o | 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: |+== + .* o | 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: |== o o + . | 2024-09-11T23:27:49.960 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: |o . + S+ . | 2024-09-11T23:27:49.961 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: | o o oo.+ | 2024-09-11T23:27:49.961 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: | . + o.oo | 2024-09-11T23:27:49.961 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: | . o.+o+. | 2024-09-11T23:27:49.961 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: | .o.+=+ | 2024-09-11T23:27:49.961 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:27:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: +----[SHA256]-----+ 2024-09-11T23:27:50.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:50 smithi133 ceph-mon[31430]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:27:50.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:50 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:27:50.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:50 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:27:50.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.473+0000 7fbd44970640 1 Processor -- start 2024-09-11T23:27:50.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.473+0000 7fbd44970640 1 -- start start 2024-09-11T23:27:50.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.473+0000 7fbd44970640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd40107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:50.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.473+0000 7fbd44970640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbd401082a0 con 0x7fbd401078d0 2024-09-11T23:27:50.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.473+0000 7fbd3e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd40107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:50.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.473+0000 7fbd3e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd40107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35802/0 (socket says 172.21.15.133:35802) 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.473+0000 7fbd3e59c640 1 -- 172.21.15.133:0/3130842221 learned_addr learned my addr 172.21.15.133:0/3130842221 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.474+0000 7fbd3e59c640 1 -- 172.21.15.133:0/3130842221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd40108a80 con 0x7fbd401078d0 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.474+0000 7fbd3e59c640 1 --2- 172.21.15.133:0/3130842221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd40107cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009820 tx=0x7fbd3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2fab16d454993d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.474+0000 7fbd3d59a640 1 -- 172.21.15.133:0/3130842221 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30037070 con 0x7fbd401078d0 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.474+0000 7fbd3d59a640 1 -- 172.21.15.133:0/3130842221 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd30032900 con 0x7fbd401078d0 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.474+0000 7fbd3d59a640 1 -- 172.21.15.133:0/3130842221 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30030850 con 0x7fbd401078d0 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.475+0000 7fbd44970640 1 -- 172.21.15.133:0/3130842221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 msgr2=0x7fbd40107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.475+0000 7fbd44970640 1 --2- 172.21.15.133:0/3130842221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd40107cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009820 tx=0x7fbd3002a270 comp rx=0 tx=0).stop 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.475+0000 7fbd44970640 1 -- 172.21.15.133:0/3130842221 shutdown_connections 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.475+0000 7fbd44970640 1 --2- 172.21.15.133:0/3130842221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd40107cd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.475+0000 7fbd44970640 1 -- 172.21.15.133:0/3130842221 >> 172.21.15.133:0/3130842221 conn(0x7fbd40103080 msgr2=0x7fbd401054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.475+0000 7fbd44970640 1 -- 172.21.15.133:0/3130842221 shutdown_connections 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.475+0000 7fbd44970640 1 -- 172.21.15.133:0/3130842221 wait complete. 2024-09-11T23:27:50.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.476+0000 7fbd44970640 1 Processor -- start 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.476+0000 7fbd44970640 1 -- start start 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.476+0000 7fbd44970640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd4019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.476+0000 7fbd44970640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4010be30 con 0x7fbd401078d0 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.476+0000 7fbd3e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd4019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.476+0000 7fbd3e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd4019b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35818/0 (socket says 172.21.15.133:35818) 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.476+0000 7fbd3e59c640 1 -- 172.21.15.133:0/1731813688 learned_addr learned my addr 172.21.15.133:0/1731813688 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.476+0000 7fbd3e59c640 1 -- 172.21.15.133:0/1731813688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4019be30 con 0x7fbd401078d0 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.477+0000 7fbd3e59c640 1 --2- 172.21.15.133:0/1731813688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd4019b8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbd300097f0 tx=0x7fbd30030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.477+0000 7fbd2f7fe640 1 -- 172.21.15.133:0/1731813688 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30042070 con 0x7fbd401078d0 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.477+0000 7fbd2f7fe640 1 -- 172.21.15.133:0/1731813688 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd30032ca0 con 0x7fbd401078d0 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.477+0000 7fbd2f7fe640 1 -- 172.21.15.133:0/1731813688 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30037040 con 0x7fbd401078d0 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.477+0000 7fbd44970640 1 -- 172.21.15.133:0/1731813688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4019c0c0 con 0x7fbd401078d0 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.477+0000 7fbd44970640 1 -- 172.21.15.133:0/1731813688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4019edb0 con 0x7fbd401078d0 2024-09-11T23:27:50.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.478+0000 7fbd2f7fe640 1 -- 172.21.15.133:0/1731813688 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbd30045070 con 0x7fbd401078d0 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.478+0000 7fbd44970640 1 -- 172.21.15.133:0/1731813688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4010bfb0 con 0x7fbd401078d0 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.478+0000 7fbd2f7fe640 1 --2- 172.21.15.133:0/1731813688 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbd28038fc0 0x7fbd2803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.478+0000 7fbd2f7fe640 1 -- 172.21.15.133:0/1731813688 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd3006bea0 con 0x7fbd401078d0 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.478+0000 7fbd3dd9b640 1 --2- 172.21.15.133:0/1731813688 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbd28038fc0 0x7fbd2803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.478+0000 7fbd3dd9b640 1 --2- 172.21.15.133:0/1731813688 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbd28038fc0 0x7fbd2803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd340098c0 tx=0x7fbd34006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.483+0000 7fbd2f7fe640 1 -- 172.21.15.133:0/1731813688 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd3004b170 con 0x7fbd401078d0 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.649+0000 7fbd44970640 1 -- 172.21.15.133:0/1731813688 --> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fbd401043c0 con 0x7fbd28038fc0 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.875+0000 7fbd2f7fe640 1 -- 172.21.15.133:0/1731813688 <== mgr.14118 v2:172.21.15.133:6800/2152407978 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbd401043c0 con 0x7fbd28038fc0 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.877+0000 7fbd44970640 1 -- 172.21.15.133:0/1731813688 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbd28038fc0 msgr2=0x7fbd2803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.877+0000 7fbd44970640 1 --2- 172.21.15.133:0/1731813688 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbd28038fc0 0x7fbd2803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd340098c0 tx=0x7fbd34006db0 comp rx=0 tx=0).stop 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.877+0000 7fbd44970640 1 -- 172.21.15.133:0/1731813688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 msgr2=0x7fbd4019b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.877+0000 7fbd44970640 1 --2- 172.21.15.133:0/1731813688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd4019b8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbd300097f0 tx=0x7fbd30030ec0 comp rx=0 tx=0).stop 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.877+0000 7fbd44970640 1 -- 172.21.15.133:0/1731813688 shutdown_connections 2024-09-11T23:27:50.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.877+0000 7fbd44970640 1 --2- 172.21.15.133:0/1731813688 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbd28038fc0 0x7fbd2803b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:50.510 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.877+0000 7fbd44970640 1 --2- 172.21.15.133:0/1731813688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd401078d0 0x7fbd4019b8f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:50.510 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.877+0000 7fbd44970640 1 -- 172.21.15.133:0/1731813688 >> 172.21.15.133:0/1731813688 conn(0x7fbd40103080 msgr2=0x7fbd40103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:50.510 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.877+0000 7fbd44970640 1 -- 172.21.15.133:0/1731813688 shutdown_connections 2024-09-11T23:27:50.510 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:49.877+0000 7fbd44970640 1 -- 172.21.15.133:0/1731813688 wait complete. 2024-09-11T23:27:51.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:51 smithi133 ceph-mon[31430]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:27:51.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:51 smithi133 ceph-mon[31430]: Generating ssh key... 2024-09-11T23:27:51.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmXJ7ZuLXqnx6VQwiRJCde3t5EEGFo6vYYYf1VRy2nAqb8U/UMc5wSkfpclZ4EFz9Hi0guh6raqkQ+lE6Zo6PZJg0uTFQJOqIoXyzzbI9CKZPtAiCa+1L81YEJLVCThlvFn0JasWRLZLNwiSQTbQxeCe+K3dWdBxZZqw0TRQ8nYYuFFUMljqwMKotOhhGoz/WCU0sKz/029h9DO+dLSb485X+CSW68ZCKaeLClI33KIdaJVdDCPuY66IUtJMT6LOXK76lTeuDqVvPq1emFHr5AwUQKzs/f8MfGI14Ap13WlJWE2AFrQ25v5Q5b7agQqyaz4BY3/L6w5F59FtZPgPpId3FjFt9GiOmjQPcQK5E4kkX1F6YTWpN8i0CdmO4CqCUQZqkaYzbm/ZxpRLtrTpNccJdBV559xUadt0qCP6KxxAOSpSM+0f3mChmyDtvrma8ym9jcP5tjhNq2ljFaySjZ3jiTtxiYT4yPvqRjJHZMVfIUO5qrM85s0U5sHUfqClU= ceph-42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:27:51.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.904+0000 7f17d690d640 1 Processor -- start 2024-09-11T23:27:51.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.904+0000 7f17d690d640 1 -- start start 2024-09-11T23:27:51.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.905+0000 7f17d690d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d0106b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:51.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.905+0000 7f17d690d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0107150 con 0x7f17d0106780 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.905+0000 7f17cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d0106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.905+0000 7f17cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d0106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35828/0 (socket says 172.21.15.133:35828) 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.905+0000 7f17cffff640 1 -- 172.21.15.133:0/4022369521 learned_addr learned my addr 172.21.15.133:0/4022369521 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.905+0000 7f17cffff640 1 -- 172.21.15.133:0/4022369521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d0107980 con 0x7f17d0106780 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.906+0000 7f17cffff640 1 --2- 172.21.15.133:0/4022369521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d0106b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009820 tx=0x7f17c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c89aa8fb2146325 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.906+0000 7f17ceffd640 1 -- 172.21.15.133:0/4022369521 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c0037070 con 0x7f17d0106780 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.906+0000 7f17ceffd640 1 -- 172.21.15.133:0/4022369521 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f17c0032900 con 0x7f17d0106780 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.906+0000 7f17ceffd640 1 -- 172.21.15.133:0/4022369521 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c0030850 con 0x7f17d0106780 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.907+0000 7f17d690d640 1 -- 172.21.15.133:0/4022369521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 msgr2=0x7f17d0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.907+0000 7f17d690d640 1 --2- 172.21.15.133:0/4022369521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d0106b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009820 tx=0x7f17c002a270 comp rx=0 tx=0).stop 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.907+0000 7f17d690d640 1 -- 172.21.15.133:0/4022369521 shutdown_connections 2024-09-11T23:27:51.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.907+0000 7f17d690d640 1 --2- 172.21.15.133:0/4022369521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d0106b80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.907+0000 7f17d690d640 1 -- 172.21.15.133:0/4022369521 >> 172.21.15.133:0/4022369521 conn(0x7f17d0101f10 msgr2=0x7f17d0104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:51.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.907+0000 7f17d690d640 1 -- 172.21.15.133:0/4022369521 shutdown_connections 2024-09-11T23:27:51.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.907+0000 7f17d690d640 1 -- 172.21.15.133:0/4022369521 wait complete. 2024-09-11T23:27:51.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.907+0000 7f17d690d640 1 Processor -- start 2024-09-11T23:27:51.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.907+0000 7f17d690d640 1 -- start start 2024-09-11T23:27:51.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.907+0000 7f17d690d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:51.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.908+0000 7f17d690d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f17d010ad30 con 0x7f17d0106780 2024-09-11T23:27:51.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.908+0000 7f17cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:51.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.908+0000 7f17cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d019b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35838/0 (socket says 172.21.15.133:35838) 2024-09-11T23:27:51.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.908+0000 7f17cffff640 1 -- 172.21.15.133:0/3989689381 learned_addr learned my addr 172.21.15.133:0/3989689381 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.908+0000 7f17cffff640 1 -- 172.21.15.133:0/3989689381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d019be50 con 0x7f17d0106780 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.908+0000 7f17cffff640 1 --2- 172.21.15.133:0/3989689381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d019b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009940 tx=0x7f17c00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.908+0000 7f17cd7fa640 1 -- 172.21.15.133:0/3989689381 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c0042070 con 0x7f17d0106780 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.908+0000 7f17cd7fa640 1 -- 172.21.15.133:0/3989689381 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f17c0032be0 con 0x7f17d0106780 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.908+0000 7f17d690d640 1 -- 172.21.15.133:0/3989689381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d019c0e0 con 0x7f17d0106780 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.908+0000 7f17cd7fa640 1 -- 172.21.15.133:0/3989689381 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c0037040 con 0x7f17d0106780 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.908+0000 7f17d690d640 1 -- 172.21.15.133:0/3989689381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d019edd0 con 0x7f17d0106780 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.909+0000 7f17cd7fa640 1 -- 172.21.15.133:0/3989689381 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f17c0045070 con 0x7f17d0106780 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.909+0000 7f17d690d640 1 -- 172.21.15.133:0/3989689381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d010aeb0 con 0x7f17d0106780 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.909+0000 7f17cd7fa640 1 --2- 172.21.15.133:0/3989689381 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f17bc038fc0 0x7f17bc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.909+0000 7f17cd7fa640 1 -- 172.21.15.133:0/3989689381 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f17c006bec0 con 0x7f17d0106780 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.909+0000 7f17cf7fe640 1 --2- 172.21.15.133:0/3989689381 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f17bc038fc0 0x7f17bc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:51.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.910+0000 7f17cf7fe640 1 --2- 172.21.15.133:0/3989689381 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f17bc038fc0 0x7f17bc03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f17b80098c0 tx=0x7f17b8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:51.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:50.914+0000 7f17cd7fa640 1 -- 172.21.15.133:0/3989689381 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17c004b170 con 0x7f17d0106780 2024-09-11T23:27:51.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.080+0000 7f17d690d640 1 -- 172.21.15.133:0/3989689381 --> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f17d002d9f0 con 0x7f17bc038fc0 2024-09-11T23:27:51.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.081+0000 7f17cd7fa640 1 -- 172.21.15.133:0/3989689381 <== mgr.14118 v2:172.21.15.133:6800/2152407978 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f17d002d9f0 con 0x7f17bc038fc0 2024-09-11T23:27:51.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.083+0000 7f17d690d640 1 -- 172.21.15.133:0/3989689381 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f17bc038fc0 msgr2=0x7f17bc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:51.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.083+0000 7f17d690d640 1 --2- 172.21.15.133:0/3989689381 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f17bc038fc0 0x7f17bc03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f17b80098c0 tx=0x7f17b8006db0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.083+0000 7f17d690d640 1 -- 172.21.15.133:0/3989689381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 msgr2=0x7f17d019b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:51.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.083+0000 7f17d690d640 1 --2- 172.21.15.133:0/3989689381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d019b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009940 tx=0x7f17c00326e0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.083+0000 7f17d690d640 1 -- 172.21.15.133:0/3989689381 shutdown_connections 2024-09-11T23:27:51.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.083+0000 7f17d690d640 1 --2- 172.21.15.133:0/3989689381 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f17bc038fc0 0x7f17bc03b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.083+0000 7f17d690d640 1 --2- 172.21.15.133:0/3989689381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17d0106780 0x7f17d019b910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.083+0000 7f17d690d640 1 -- 172.21.15.133:0/3989689381 >> 172.21.15.133:0/3989689381 conn(0x7f17d0101f10 msgr2=0x7f17d0102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:51.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.083+0000 7f17d690d640 1 -- 172.21.15.133:0/3989689381 shutdown_connections 2024-09-11T23:27:51.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:51.083+0000 7f17d690d640 1 -- 172.21.15.133:0/3989689381 wait complete. 2024-09-11T23:27:51.661 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:27:51.661 INFO:teuthology.orchestra.run.smithi133.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T23:27:51.661 INFO:teuthology.orchestra.run.smithi133.stdout:Adding host smithi133... 2024-09-11T23:27:52.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:52 smithi133 ceph-mon[31430]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:27:53.296 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:53 smithi133 ceph-mon[31430]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "addr": "172.21.15.133", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:27:54.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:54 smithi133 ceph-mon[31430]: Deploying cephadm binary to smithi133 2024-09-11T23:27:56.020 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Added host 'smithi133' with addr '172.21.15.133' 2024-09-11T23:27:56.021 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.070+0000 7f69c4c14640 1 Processor -- start 2024-09-11T23:27:56.021 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.070+0000 7f69c4c14640 1 -- start start 2024-09-11T23:27:56.021 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.071+0000 7f69c4c14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c0107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:56.021 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.071+0000 7f69c4c14640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f69c0108280 con 0x7f69c01078b0 2024-09-11T23:27:56.021 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.071+0000 7f69be59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c0107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:56.021 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.071+0000 7f69be59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c0107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50278/0 (socket says 172.21.15.133:50278) 2024-09-11T23:27:56.021 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.071+0000 7f69be59c640 1 -- 172.21.15.133:0/2046532827 learned_addr learned my addr 172.21.15.133:0/2046532827 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:56.021 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.071+0000 7f69be59c640 1 -- 172.21.15.133:0/2046532827 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c0108a60 con 0x7f69c01078b0 2024-09-11T23:27:56.021 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.072+0000 7f69be59c640 1 --2- 172.21.15.133:0/2046532827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c0107cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009820 tx=0x7f69b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e14876e537444d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:56.021 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.072+0000 7f69bd59a640 1 -- 172.21.15.133:0/2046532827 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69b0037070 con 0x7f69c01078b0 2024-09-11T23:27:56.021 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.072+0000 7f69bd59a640 1 -- 172.21.15.133:0/2046532827 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f69b0032900 con 0x7f69c01078b0 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.072+0000 7f69bd59a640 1 -- 172.21.15.133:0/2046532827 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69b0030850 con 0x7f69c01078b0 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.073+0000 7f69c4c14640 1 -- 172.21.15.133:0/2046532827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 msgr2=0x7f69c0107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.073+0000 7f69c4c14640 1 --2- 172.21.15.133:0/2046532827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c0107cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009820 tx=0x7f69b002a270 comp rx=0 tx=0).stop 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.073+0000 7f69c4c14640 1 -- 172.21.15.133:0/2046532827 shutdown_connections 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.073+0000 7f69c4c14640 1 --2- 172.21.15.133:0/2046532827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c0107cb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.073+0000 7f69c4c14640 1 -- 172.21.15.133:0/2046532827 >> 172.21.15.133:0/2046532827 conn(0x7f69c0103060 msgr2=0x7f69c0105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.073+0000 7f69c4c14640 1 -- 172.21.15.133:0/2046532827 shutdown_connections 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.073+0000 7f69c4c14640 1 -- 172.21.15.133:0/2046532827 wait complete. 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.074+0000 7f69c4c14640 1 Processor -- start 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.074+0000 7f69c4c14640 1 -- start start 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.074+0000 7f69c4c14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.074+0000 7f69c4c14640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f69c010be10 con 0x7f69c01078b0 2024-09-11T23:27:56.022 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.074+0000 7f69be59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.074+0000 7f69be59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c019b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50288/0 (socket says 172.21.15.133:50288) 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.074+0000 7f69be59c640 1 -- 172.21.15.133:0/3327682015 learned_addr learned my addr 172.21.15.133:0/3327682015 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.074+0000 7f69be59c640 1 -- 172.21.15.133:0/3327682015 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c019bdd0 con 0x7f69c01078b0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.074+0000 7f69be59c640 1 --2- 172.21.15.133:0/3327682015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c019b890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69b00097f0 tx=0x7f69b0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.075+0000 7f69b77fe640 1 -- 172.21.15.133:0/3327682015 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69b0042070 con 0x7f69c01078b0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.075+0000 7f69b77fe640 1 -- 172.21.15.133:0/3327682015 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f69b0032ca0 con 0x7f69c01078b0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.075+0000 7f69b77fe640 1 -- 172.21.15.133:0/3327682015 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69b0037040 con 0x7f69c01078b0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.075+0000 7f69c4c14640 1 -- 172.21.15.133:0/3327682015 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c019c060 con 0x7f69c01078b0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.075+0000 7f69c4c14640 1 -- 172.21.15.133:0/3327682015 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c019ed50 con 0x7f69c01078b0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.075+0000 7f69b77fe640 1 -- 172.21.15.133:0/3327682015 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f69b0045070 con 0x7f69c01078b0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.076+0000 7f69c4c14640 1 -- 172.21.15.133:0/3327682015 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c010bf90 con 0x7f69c01078b0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.076+0000 7f69b77fe640 1 --2- 172.21.15.133:0/3327682015 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f69ac038f70 0x7f69ac03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.076+0000 7f69b77fe640 1 -- 172.21.15.133:0/3327682015 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f69b006bf30 con 0x7f69c01078b0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.076+0000 7f69bdd9b640 1 --2- 172.21.15.133:0/3327682015 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f69ac038f70 0x7f69ac03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:56.023 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.076+0000 7f69bdd9b640 1 --2- 172.21.15.133:0/3327682015 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f69ac038f70 0x7f69ac03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f69a80098c0 tx=0x7f69a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.080+0000 7f69b77fe640 1 -- 172.21.15.133:0/3327682015 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69b003d430 con 0x7f69c01078b0 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:52.252+0000 7f69c4c14640 1 -- 172.21.15.133:0/3327682015 --> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi133", "addr": "172.21.15.133", "target": ["mon-mgr", ""]}) -- 0x7f69c01043a0 con 0x7f69ac038f70 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:55.439+0000 7f69b77fe640 1 -- 172.21.15.133:0/3327682015 <== mgr.14118 v2:172.21.15.133:6800/2152407978 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f69c01043a0 con 0x7f69ac038f70 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:55.442+0000 7f69c4c14640 1 -- 172.21.15.133:0/3327682015 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f69ac038f70 msgr2=0x7f69ac03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:55.442+0000 7f69c4c14640 1 --2- 172.21.15.133:0/3327682015 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f69ac038f70 0x7f69ac03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f69a80098c0 tx=0x7f69a8006db0 comp rx=0 tx=0).stop 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:55.442+0000 7f69c4c14640 1 -- 172.21.15.133:0/3327682015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 msgr2=0x7f69c019b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:55.442+0000 7f69c4c14640 1 --2- 172.21.15.133:0/3327682015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c019b890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69b00097f0 tx=0x7f69b0030ec0 comp rx=0 tx=0).stop 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:55.442+0000 7f69c4c14640 1 -- 172.21.15.133:0/3327682015 shutdown_connections 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:55.442+0000 7f69c4c14640 1 --2- 172.21.15.133:0/3327682015 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f69ac038f70 0x7f69ac03b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:55.442+0000 7f69c4c14640 1 --2- 172.21.15.133:0/3327682015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01078b0 0x7f69c019b890 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:55.442+0000 7f69c4c14640 1 -- 172.21.15.133:0/3327682015 >> 172.21.15.133:0/3327682015 conn(0x7f69c0103060 msgr2=0x7f69c0103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:55.442+0000 7f69c4c14640 1 -- 172.21.15.133:0/3327682015 shutdown_connections 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:55.442+0000 7f69c4c14640 1 -- 172.21.15.133:0/3327682015 wait complete. 2024-09-11T23:27:56.024 INFO:teuthology.orchestra.run.smithi133.stdout:Deploying unmanaged mon service... 2024-09-11T23:27:56.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:56 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:27:56.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:56 smithi133 ceph-mon[31430]: Added host smithi133 2024-09-11T23:27:56.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:56 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:27:57.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T23:27:57.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.587+0000 7f52de1f4640 1 Processor -- start 2024-09-11T23:27:57.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.587+0000 7f52de1f4640 1 -- start start 2024-09-11T23:27:57.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.587+0000 7f52de1f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d8107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:57.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.587+0000 7f52de1f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8108280 con 0x7f52d81078b0 2024-09-11T23:27:57.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.588+0000 7f52d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d8107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:57.613 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.588+0000 7f52d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d8107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50302/0 (socket says 172.21.15.133:50302) 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.588+0000 7f52d77fe640 1 -- 172.21.15.133:0/849611065 learned_addr learned my addr 172.21.15.133:0/849611065 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.588+0000 7f52d77fe640 1 -- 172.21.15.133:0/849611065 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d8108a60 con 0x7f52d81078b0 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.588+0000 7f52d77fe640 1 --2- 172.21.15.133:0/849611065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d8107cb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009820 tx=0x7f52c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c3f79ddc5dd3dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.589+0000 7f52d67fc640 1 -- 172.21.15.133:0/849611065 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52c8037070 con 0x7f52d81078b0 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.589+0000 7f52d67fc640 1 -- 172.21.15.133:0/849611065 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52c8032900 con 0x7f52d81078b0 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.589+0000 7f52d67fc640 1 -- 172.21.15.133:0/849611065 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52c8030850 con 0x7f52d81078b0 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.589+0000 7f52de1f4640 1 -- 172.21.15.133:0/849611065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 msgr2=0x7f52d8107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.589+0000 7f52de1f4640 1 --2- 172.21.15.133:0/849611065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d8107cb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009820 tx=0x7f52c802a270 comp rx=0 tx=0).stop 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.589+0000 7f52de1f4640 1 -- 172.21.15.133:0/849611065 shutdown_connections 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.589+0000 7f52de1f4640 1 --2- 172.21.15.133:0/849611065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d8107cb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.589+0000 7f52de1f4640 1 -- 172.21.15.133:0/849611065 >> 172.21.15.133:0/849611065 conn(0x7f52d8103060 msgr2=0x7f52d8105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:57.614 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.589+0000 7f52de1f4640 1 -- 172.21.15.133:0/849611065 shutdown_connections 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.589+0000 7f52de1f4640 1 -- 172.21.15.133:0/849611065 wait complete. 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.590+0000 7f52de1f4640 1 Processor -- start 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.590+0000 7f52de1f4640 1 -- start start 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.590+0000 7f52de1f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.590+0000 7f52de1f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52d810be10 con 0x7f52d81078b0 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.590+0000 7f52d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.590+0000 7f52d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d819b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50312/0 (socket says 172.21.15.133:50312) 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.590+0000 7f52d77fe640 1 -- 172.21.15.133:0/4147469125 learned_addr learned my addr 172.21.15.133:0/4147469125 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.590+0000 7f52d77fe640 1 -- 172.21.15.133:0/4147469125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d819bdd0 con 0x7f52d81078b0 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.591+0000 7f52d77fe640 1 --2- 172.21.15.133:0/4147469125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d819b890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009940 tx=0x7f52c80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.591+0000 7f52d4ff9640 1 -- 172.21.15.133:0/4147469125 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52c8042070 con 0x7f52d81078b0 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.591+0000 7f52d4ff9640 1 -- 172.21.15.133:0/4147469125 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52c8032be0 con 0x7f52d81078b0 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.591+0000 7f52de1f4640 1 -- 172.21.15.133:0/4147469125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d819c060 con 0x7f52d81078b0 2024-09-11T23:27:57.615 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.591+0000 7f52d4ff9640 1 -- 172.21.15.133:0/4147469125 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52c8037040 con 0x7f52d81078b0 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.591+0000 7f52de1f4640 1 -- 172.21.15.133:0/4147469125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d819ed50 con 0x7f52d81078b0 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.592+0000 7f52d4ff9640 1 -- 172.21.15.133:0/4147469125 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f52c8045070 con 0x7f52d81078b0 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.592+0000 7f52de1f4640 1 -- 172.21.15.133:0/4147469125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d810bf90 con 0x7f52d81078b0 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.592+0000 7f52d4ff9640 1 --2- 172.21.15.133:0/4147469125 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f52c0038f70 0x7f52c003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.592+0000 7f52d4ff9640 1 -- 172.21.15.133:0/4147469125 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f52c806c020 con 0x7f52d81078b0 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.592+0000 7f52d6ffd640 1 --2- 172.21.15.133:0/4147469125 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f52c0038f70 0x7f52c003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.593+0000 7f52d6ffd640 1 --2- 172.21.15.133:0/4147469125 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f52c0038f70 0x7f52c003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0098c0 tx=0x7f52cc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.598+0000 7f52d4ff9640 1 -- 172.21.15.133:0/4147469125 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52c8032e00 con 0x7f52d81078b0 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.794+0000 7f52de1f4640 1 -- 172.21.15.133:0/4147469125 --> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f52d81043a0 con 0x7f52c0038f70 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.799+0000 7f52d4ff9640 1 -- 172.21.15.133:0/4147469125 <== mgr.14118 v2:172.21.15.133:6800/2152407978 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f52d81043a0 con 0x7f52c0038f70 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.801+0000 7f52de1f4640 1 -- 172.21.15.133:0/4147469125 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f52c0038f70 msgr2=0x7f52c003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:57.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.801+0000 7f52de1f4640 1 --2- 172.21.15.133:0/4147469125 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f52c0038f70 0x7f52c003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0098c0 tx=0x7f52cc006db0 comp rx=0 tx=0).stop 2024-09-11T23:27:57.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.801+0000 7f52de1f4640 1 -- 172.21.15.133:0/4147469125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 msgr2=0x7f52d819b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:57.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.801+0000 7f52de1f4640 1 --2- 172.21.15.133:0/4147469125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d819b890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009940 tx=0x7f52c80326e0 comp rx=0 tx=0).stop 2024-09-11T23:27:57.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.801+0000 7f52de1f4640 1 -- 172.21.15.133:0/4147469125 shutdown_connections 2024-09-11T23:27:57.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.801+0000 7f52de1f4640 1 --2- 172.21.15.133:0/4147469125 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f52c0038f70 0x7f52c003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:57.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.801+0000 7f52de1f4640 1 --2- 172.21.15.133:0/4147469125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81078b0 0x7f52d819b890 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:57.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.801+0000 7f52de1f4640 1 -- 172.21.15.133:0/4147469125 >> 172.21.15.133:0/4147469125 conn(0x7f52d8103060 msgr2=0x7f52d8103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:57.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.801+0000 7f52de1f4640 1 -- 172.21.15.133:0/4147469125 shutdown_connections 2024-09-11T23:27:57.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:56.801+0000 7f52de1f4640 1 -- 172.21.15.133:0/4147469125 wait complete. 2024-09-11T23:27:57.617 INFO:teuthology.orchestra.run.smithi133.stdout:Deploying unmanaged mgr service... 2024-09-11T23:27:57.941 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:57 smithi133 ceph-mon[31430]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:27:57.941 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:57 smithi133 ceph-mon[31430]: Saving service mon spec with placement count:5 2024-09-11T23:27:57.941 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:57 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:27:59.155 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T23:27:59.155 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.020+0000 7fbc0b08d640 1 Processor -- start 2024-09-11T23:27:59.155 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.020+0000 7fbc0b08d640 1 -- start start 2024-09-11T23:27:59.155 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.020+0000 7fbc0b08d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc04107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.155 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.020+0000 7fbc0b08d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbc04108280 con 0x7fbc041078b0 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.020+0000 7fbc08e29640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc04107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.020+0000 7fbc08e29640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc04107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50322/0 (socket says 172.21.15.133:50322) 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.020+0000 7fbc08e29640 1 -- 172.21.15.133:0/454533091 learned_addr learned my addr 172.21.15.133:0/454533091 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.020+0000 7fbc08e29640 1 -- 172.21.15.133:0/454533091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc04108a60 con 0x7fbc041078b0 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.021+0000 7fbc08e29640 1 --2- 172.21.15.133:0/454533091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc04107cb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009820 tx=0x7fbbf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52f133276712b23 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.021+0000 7fbc037fe640 1 -- 172.21.15.133:0/454533091 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbf8037070 con 0x7fbc041078b0 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.021+0000 7fbc037fe640 1 -- 172.21.15.133:0/454533091 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbbf8032900 con 0x7fbc041078b0 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.021+0000 7fbc037fe640 1 -- 172.21.15.133:0/454533091 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbf8030850 con 0x7fbc041078b0 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.022+0000 7fbc0b08d640 1 -- 172.21.15.133:0/454533091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 msgr2=0x7fbc04107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.022+0000 7fbc0b08d640 1 --2- 172.21.15.133:0/454533091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc04107cb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009820 tx=0x7fbbf802a270 comp rx=0 tx=0).stop 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.022+0000 7fbc0b08d640 1 -- 172.21.15.133:0/454533091 shutdown_connections 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.022+0000 7fbc0b08d640 1 --2- 172.21.15.133:0/454533091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc04107cb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.156 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.022+0000 7fbc0b08d640 1 -- 172.21.15.133:0/454533091 >> 172.21.15.133:0/454533091 conn(0x7fbc04103060 msgr2=0x7fbc04105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.022+0000 7fbc0b08d640 1 -- 172.21.15.133:0/454533091 shutdown_connections 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.022+0000 7fbc0b08d640 1 -- 172.21.15.133:0/454533091 wait complete. 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.022+0000 7fbc0b08d640 1 Processor -- start 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.022+0000 7fbc0b08d640 1 -- start start 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc0b08d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc0407db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc0b08d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0410be10 con 0x7fbc041078b0 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc08e29640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc0407db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc08e29640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc0407db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50328/0 (socket says 172.21.15.133:50328) 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc08e29640 1 -- 172.21.15.133:0/3027047824 learned_addr learned my addr 172.21.15.133:0/3027047824 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc08e29640 1 -- 172.21.15.133:0/3027047824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0407e040 con 0x7fbc041078b0 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc08e29640 1 --2- 172.21.15.133:0/3027047824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc0407db00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009940 tx=0x7fbbf80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc01ffb640 1 -- 172.21.15.133:0/3027047824 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbf8042070 con 0x7fbc041078b0 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc01ffb640 1 -- 172.21.15.133:0/3027047824 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbbf8032be0 con 0x7fbc041078b0 2024-09-11T23:27:59.157 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc0b08d640 1 -- 172.21.15.133:0/3027047824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0407c230 con 0x7fbc041078b0 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc01ffb640 1 -- 172.21.15.133:0/3027047824 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbf8037040 con 0x7fbc041078b0 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.023+0000 7fbc0b08d640 1 -- 172.21.15.133:0/3027047824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0407c770 con 0x7fbc041078b0 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.024+0000 7fbc01ffb640 1 -- 172.21.15.133:0/3027047824 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbbf8045070 con 0x7fbc041078b0 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.024+0000 7fbc0b08d640 1 -- 172.21.15.133:0/3027047824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc04107d30 con 0x7fbc041078b0 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.024+0000 7fbc01ffb640 1 --2- 172.21.15.133:0/3027047824 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbbf4038f70 0x7fbbf403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.025+0000 7fbc01ffb640 1 -- 172.21.15.133:0/3027047824 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbbf806c060 con 0x7fbc041078b0 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.025+0000 7fbc03fff640 1 --2- 172.21.15.133:0/3027047824 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbbf4038f70 0x7fbbf403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.025+0000 7fbc03fff640 1 --2- 172.21.15.133:0/3027047824 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbbf4038f70 0x7fbbf403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098c0 tx=0x7fbbf0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.029+0000 7fbc01ffb640 1 -- 172.21.15.133:0/3027047824 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbbf8045e10 con 0x7fbc041078b0 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.213+0000 7fbc0b08d640 1 -- 172.21.15.133:0/3027047824 --> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbc041043a0 con 0x7fbbf4038f70 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.220+0000 7fbc01ffb640 1 -- 172.21.15.133:0/3027047824 <== mgr.14118 v2:172.21.15.133:6800/2152407978 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbc041043a0 con 0x7fbbf4038f70 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.222+0000 7fbc0b08d640 1 -- 172.21.15.133:0/3027047824 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbbf4038f70 msgr2=0x7fbbf403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.222+0000 7fbc0b08d640 1 --2- 172.21.15.133:0/3027047824 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbbf4038f70 0x7fbbf403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098c0 tx=0x7fbbf0006db0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.222+0000 7fbc0b08d640 1 -- 172.21.15.133:0/3027047824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 msgr2=0x7fbc0407db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:59.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.222+0000 7fbc0b08d640 1 --2- 172.21.15.133:0/3027047824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc0407db00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009940 tx=0x7fbbf80326e0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.222+0000 7fbc0b08d640 1 -- 172.21.15.133:0/3027047824 shutdown_connections 2024-09-11T23:27:59.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.222+0000 7fbc0b08d640 1 --2- 172.21.15.133:0/3027047824 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fbbf4038f70 0x7fbbf403b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.222+0000 7fbc0b08d640 1 --2- 172.21.15.133:0/3027047824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc041078b0 0x7fbc0407db00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:59.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.222+0000 7fbc0b08d640 1 -- 172.21.15.133:0/3027047824 >> 172.21.15.133:0/3027047824 conn(0x7fbc04103060 msgr2=0x7fbc04103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:59.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.222+0000 7fbc0b08d640 1 -- 172.21.15.133:0/3027047824 shutdown_connections 2024-09-11T23:27:59.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:58.222+0000 7fbc0b08d640 1 -- 172.21.15.133:0/3027047824 wait complete. 2024-09-11T23:27:59.458 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:59 smithi133 ceph-mon[31430]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:27:59.458 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:59 smithi133 ceph-mon[31430]: Saving service mgr spec with placement count:2 2024-09-11T23:27:59.458 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:27:59 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:28:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.542+0000 7f42b848d640 1 Processor -- start 2024-09-11T23:28:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.543+0000 7f42b848d640 1 -- start start 2024-09-11T23:28:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.543+0000 7f42b848d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b0107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.543+0000 7f42b848d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f42b01082a0 con 0x7f42b01078d0 2024-09-11T23:28:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.543+0000 7f42b6229640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b0107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.543+0000 7f42b6229640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b0107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50332/0 (socket says 172.21.15.133:50332) 2024-09-11T23:28:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.543+0000 7f42b6229640 1 -- 172.21.15.133:0/2452574861 learned_addr learned my addr 172.21.15.133:0/2452574861 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.543+0000 7f42b6229640 1 -- 172.21.15.133:0/2452574861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b0108a80 con 0x7f42b01078d0 2024-09-11T23:28:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.544+0000 7f42b6229640 1 --2- 172.21.15.133:0/2452574861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b0107cd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009820 tx=0x7f42ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8773eaf7f8c12a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.544+0000 7f42b5227640 1 -- 172.21.15.133:0/2452574861 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42ac037070 con 0x7f42b01078d0 2024-09-11T23:28:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.544+0000 7f42b5227640 1 -- 172.21.15.133:0/2452574861 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42ac032900 con 0x7f42b01078d0 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.544+0000 7f42b5227640 1 -- 172.21.15.133:0/2452574861 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42ac030850 con 0x7f42b01078d0 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.545+0000 7f42b848d640 1 -- 172.21.15.133:0/2452574861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 msgr2=0x7f42b0107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.545+0000 7f42b848d640 1 --2- 172.21.15.133:0/2452574861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b0107cd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009820 tx=0x7f42ac02a270 comp rx=0 tx=0).stop 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.545+0000 7f42b848d640 1 -- 172.21.15.133:0/2452574861 shutdown_connections 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.545+0000 7f42b848d640 1 --2- 172.21.15.133:0/2452574861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b0107cd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.545+0000 7f42b848d640 1 -- 172.21.15.133:0/2452574861 >> 172.21.15.133:0/2452574861 conn(0x7f42b0103080 msgr2=0x7f42b01054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.545+0000 7f42b848d640 1 -- 172.21.15.133:0/2452574861 shutdown_connections 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.545+0000 7f42b848d640 1 -- 172.21.15.133:0/2452574861 wait complete. 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.545+0000 7f42b848d640 1 Processor -- start 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.545+0000 7f42b848d640 1 -- start start 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42b848d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42b848d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f42b010be30 con 0x7f42b01078d0 2024-09-11T23:28:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42b6229640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42b6229640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b019b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50334/0 (socket says 172.21.15.133:50334) 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42b6229640 1 -- 172.21.15.133:0/1116072809 learned_addr learned my addr 172.21.15.133:0/1116072809 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42b6229640 1 -- 172.21.15.133:0/1116072809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b019be10 con 0x7f42b01078d0 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42b6229640 1 --2- 172.21.15.133:0/1116072809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b019b8d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f42ac0097f0 tx=0x7f42ac030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42a77fe640 1 -- 172.21.15.133:0/1116072809 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42ac042070 con 0x7f42b01078d0 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42a77fe640 1 -- 172.21.15.133:0/1116072809 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42ac032ca0 con 0x7f42b01078d0 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42a77fe640 1 -- 172.21.15.133:0/1116072809 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42ac037040 con 0x7f42b01078d0 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42b848d640 1 -- 172.21.15.133:0/1116072809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b019c0a0 con 0x7f42b01078d0 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.546+0000 7f42b848d640 1 -- 172.21.15.133:0/1116072809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b019ed90 con 0x7f42b01078d0 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.547+0000 7f42a77fe640 1 -- 172.21.15.133:0/1116072809 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f42ac045070 con 0x7f42b01078d0 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.547+0000 7f42a77fe640 1 --2- 172.21.15.133:0/1116072809 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f429c038fc0 0x7f429c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.547+0000 7f42b848d640 1 -- 172.21.15.133:0/1116072809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b010bfb0 con 0x7f42b01078d0 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.547+0000 7f42a77fe640 1 -- 172.21.15.133:0/1116072809 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f42ac06be90 con 0x7f42b01078d0 2024-09-11T23:28:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.547+0000 7f42b5a28640 1 --2- 172.21.15.133:0/1116072809 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f429c038fc0 0x7f429c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.548+0000 7f42b5a28640 1 --2- 172.21.15.133:0/1116072809 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f429c038fc0 0x7f429c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f42a00098c0 tx=0x7f42a0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.552+0000 7f42a77fe640 1 -- 172.21.15.133:0/1116072809 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42ac03d430 con 0x7f42b01078d0 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.719+0000 7f42b848d640 1 -- 172.21.15.133:0/1116072809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f42b0107d50 con 0x7f42b01078d0 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.724+0000 7f42a77fe640 1 -- 172.21.15.133:0/1116072809 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f42ac037d00 con 0x7f42b01078d0 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.726+0000 7f42b848d640 1 -- 172.21.15.133:0/1116072809 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f429c038fc0 msgr2=0x7f429c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.726+0000 7f42b848d640 1 --2- 172.21.15.133:0/1116072809 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f429c038fc0 0x7f429c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f42a00098c0 tx=0x7f42a0006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.726+0000 7f42b848d640 1 -- 172.21.15.133:0/1116072809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 msgr2=0x7f42b019b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.726+0000 7f42b848d640 1 --2- 172.21.15.133:0/1116072809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b019b8d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f42ac0097f0 tx=0x7f42ac030ec0 comp rx=0 tx=0).stop 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.726+0000 7f42b848d640 1 -- 172.21.15.133:0/1116072809 shutdown_connections 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.726+0000 7f42b848d640 1 --2- 172.21.15.133:0/1116072809 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f429c038fc0 0x7f429c03b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.726+0000 7f42b848d640 1 --2- 172.21.15.133:0/1116072809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b01078d0 0x7f42b019b8d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.726+0000 7f42b848d640 1 -- 172.21.15.133:0/1116072809 >> 172.21.15.133:0/1116072809 conn(0x7f42b0103080 msgr2=0x7f42b0103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.726+0000 7f42b848d640 1 -- 172.21.15.133:0/1116072809 shutdown_connections 2024-09-11T23:28:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:27:59.726+0000 7f42b848d640 1 -- 172.21.15.133:0/1116072809 wait complete. 2024-09-11T23:28:00.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:00 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1116072809' entity='client.admin' 2024-09-11T23:28:01.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.720+0000 7fea3d89c640 1 Processor -- start 2024-09-11T23:28:01.912 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.720+0000 7fea3d89c640 1 -- start start 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.720+0000 7fea3d89c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea38106b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.720+0000 7fea3d89c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea38107140 con 0x7fea38106770 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.721+0000 7fea36ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea38106b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.721+0000 7fea36ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea38106b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50350/0 (socket says 172.21.15.133:50350) 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.721+0000 7fea36ffd640 1 -- 172.21.15.133:0/2124695816 learned_addr learned my addr 172.21.15.133:0/2124695816 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.721+0000 7fea36ffd640 1 -- 172.21.15.133:0/2124695816 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea38107970 con 0x7fea38106770 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.721+0000 7fea36ffd640 1 --2- 172.21.15.133:0/2124695816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea38106b70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea28009e80 tx=0x7fea2802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32aef0cf2bcc401f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.722+0000 7fea35ffb640 1 -- 172.21.15.133:0/2124695816 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea28036070 con 0x7fea38106770 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.722+0000 7fea35ffb640 1 -- 172.21.15.133:0/2124695816 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea280318c0 con 0x7fea38106770 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.722+0000 7fea35ffb640 1 -- 172.21.15.133:0/2124695816 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea28007ab0 con 0x7fea38106770 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.722+0000 7fea3d89c640 1 -- 172.21.15.133:0/2124695816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 msgr2=0x7fea38106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.722+0000 7fea3d89c640 1 --2- 172.21.15.133:0/2124695816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea38106b70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea28009e80 tx=0x7fea2802a7a0 comp rx=0 tx=0).stop 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.722+0000 7fea3d89c640 1 -- 172.21.15.133:0/2124695816 shutdown_connections 2024-09-11T23:28:01.913 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.722+0000 7fea3d89c640 1 --2- 172.21.15.133:0/2124695816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea38106b70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.722+0000 7fea3d89c640 1 -- 172.21.15.133:0/2124695816 >> 172.21.15.133:0/2124695816 conn(0x7fea38101f40 msgr2=0x7fea38104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.722+0000 7fea3d89c640 1 -- 172.21.15.133:0/2124695816 shutdown_connections 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.722+0000 7fea3d89c640 1 -- 172.21.15.133:0/2124695816 wait complete. 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.723+0000 7fea3d89c640 1 Processor -- start 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.723+0000 7fea3d89c640 1 -- start start 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.723+0000 7fea3d89c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea3819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.723+0000 7fea3d89c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea3810ad20 con 0x7fea38106770 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.724+0000 7fea36ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea3819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.724+0000 7fea36ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea3819a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50356/0 (socket says 172.21.15.133:50356) 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.724+0000 7fea36ffd640 1 -- 172.21.15.133:0/2895458802 learned_addr learned my addr 172.21.15.133:0/2895458802 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.724+0000 7fea36ffd640 1 -- 172.21.15.133:0/2895458802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3819ad40 con 0x7fea38106770 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.724+0000 7fea36ffd640 1 --2- 172.21.15.133:0/2895458802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea3819a800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea28007ce0 tx=0x7fea28031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.727+0000 7fea3c89a640 1 -- 172.21.15.133:0/2895458802 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea28036050 con 0x7fea38106770 2024-09-11T23:28:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.727+0000 7fea3c89a640 1 -- 172.21.15.133:0/2895458802 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea28042070 con 0x7fea38106770 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.727+0000 7fea3c89a640 1 -- 172.21.15.133:0/2895458802 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea2803d8a0 con 0x7fea38106770 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.727+0000 7fea3d89c640 1 -- 172.21.15.133:0/2895458802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3819afd0 con 0x7fea38106770 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.727+0000 7fea3d89c640 1 -- 172.21.15.133:0/2895458802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3819dcc0 con 0x7fea38106770 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.728+0000 7fea3c89a640 1 -- 172.21.15.133:0/2895458802 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fea2803db50 con 0x7fea38106770 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.728+0000 7fea3c89a640 1 --2- 172.21.15.133:0/2895458802 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fea24038fc0 0x7fea2403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.728+0000 7fea3c89a640 1 -- 172.21.15.133:0/2895458802 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea28007e00 con 0x7fea38106770 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.728+0000 7fea3d89c640 1 -- 172.21.15.133:0/2895458802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3810aea0 con 0x7fea38106770 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.728+0000 7fea367fc640 1 --2- 172.21.15.133:0/2895458802 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fea24038fc0 0x7fea2403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.729+0000 7fea367fc640 1 --2- 172.21.15.133:0/2895458802 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fea24038fc0 0x7fea2403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fea200098c0 tx=0x7fea20006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.733+0000 7fea3c89a640 1 -- 172.21.15.133:0/2895458802 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea28044310 con 0x7fea38106770 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.901+0000 7fea3d89c640 1 -- 172.21.15.133:0/2895458802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fea38106bf0 con 0x7fea38106770 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.906+0000 7fea3c89a640 1 -- 172.21.15.133:0/2895458802 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v7)=0 v7) ==== 130+0+0 (secure 0 0 0) 0x7fea28045520 con 0x7fea38106770 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.909+0000 7fea3d89c640 1 -- 172.21.15.133:0/2895458802 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fea24038fc0 msgr2=0x7fea2403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.909+0000 7fea3d89c640 1 --2- 172.21.15.133:0/2895458802 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fea24038fc0 0x7fea2403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fea200098c0 tx=0x7fea20006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:01.915 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.909+0000 7fea3d89c640 1 -- 172.21.15.133:0/2895458802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 msgr2=0x7fea3819a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:01.916 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.909+0000 7fea3d89c640 1 --2- 172.21.15.133:0/2895458802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea3819a800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea28007ce0 tx=0x7fea28031ca0 comp rx=0 tx=0).stop 2024-09-11T23:28:01.916 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.909+0000 7fea3d89c640 1 -- 172.21.15.133:0/2895458802 shutdown_connections 2024-09-11T23:28:01.916 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.909+0000 7fea3d89c640 1 --2- 172.21.15.133:0/2895458802 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fea24038fc0 0x7fea2403b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:01.916 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.909+0000 7fea3d89c640 1 --2- 172.21.15.133:0/2895458802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38106770 0x7fea3819a800 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:01.916 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.909+0000 7fea3d89c640 1 -- 172.21.15.133:0/2895458802 >> 172.21.15.133:0/2895458802 conn(0x7fea38101f40 msgr2=0x7fea38102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:01.916 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.909+0000 7fea3d89c640 1 -- 172.21.15.133:0/2895458802 shutdown_connections 2024-09-11T23:28:01.916 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:00.909+0000 7fea3d89c640 1 -- 172.21.15.133:0/2895458802 wait complete. 2024-09-11T23:28:01.916 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling the dashboard module... 2024-09-11T23:28:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:01 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2895458802' entity='client.admin' 2024-09-11T23:28:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:01 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:28:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:01 smithi133 ceph-mon[31430]: from='mgr.14118 172.21.15.133:0/390552447' entity='mgr.x' 2024-09-11T23:28:03.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:02 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2983914370' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T23:28:03.123 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:02 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: ignoring --setuser ceph since I am not root 2024-09-11T23:28:03.123 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:02 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: ignoring --setgroup ceph since I am not root 2024-09-11T23:28:03.123 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:03.063+0000 7f628e416040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:28:03.123 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:03.122+0000 7f628e416040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:28:03.435 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:03.278+0000 7f628e416040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:28:03.436 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:03.346+0000 7f628e416040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:28:03.709 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:03.503+0000 7f628e416040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:28:03.709 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:03.646+0000 7f628e416040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:28:03.833 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.330+0000 7f05fa79c640 1 Processor -- start 2024-09-11T23:28:03.833 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.331+0000 7f05fa79c640 1 -- start start 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.331+0000 7f05fa79c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f4106b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.331+0000 7f05fa79c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f05f4107150 con 0x7f05f4106780 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.331+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f4106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.331+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f4106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48938/0 (socket says 172.21.15.133:48938) 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.331+0000 7f05f3fff640 1 -- 172.21.15.133:0/1701031186 learned_addr learned my addr 172.21.15.133:0/1701031186 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.331+0000 7f05f3fff640 1 -- 172.21.15.133:0/1701031186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f4107980 con 0x7f05f4106780 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.332+0000 7f05f3fff640 1 --2- 172.21.15.133:0/1701031186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f4106b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009820 tx=0x7f05e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7228a30fcb84e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.332+0000 7f05f37fe640 1 -- 172.21.15.133:0/1701031186 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4037070 con 0x7f05f4106780 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.332+0000 7f05f37fe640 1 -- 172.21.15.133:0/1701031186 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e4032920 con 0x7f05f4106780 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.332+0000 7f05f37fe640 1 -- 172.21.15.133:0/1701031186 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4030810 con 0x7f05f4106780 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.333+0000 7f05fa79c640 1 -- 172.21.15.133:0/1701031186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 msgr2=0x7f05f4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:03.834 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.333+0000 7f05fa79c640 1 --2- 172.21.15.133:0/1701031186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f4106b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009820 tx=0x7f05e402a270 comp rx=0 tx=0).stop 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.333+0000 7f05fa79c640 1 -- 172.21.15.133:0/1701031186 shutdown_connections 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.333+0000 7f05fa79c640 1 --2- 172.21.15.133:0/1701031186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f4106b80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.333+0000 7f05fa79c640 1 -- 172.21.15.133:0/1701031186 >> 172.21.15.133:0/1701031186 conn(0x7f05f4101f10 msgr2=0x7f05f4104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.333+0000 7f05fa79c640 1 -- 172.21.15.133:0/1701031186 shutdown_connections 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.333+0000 7f05fa79c640 1 -- 172.21.15.133:0/1701031186 wait complete. 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.333+0000 7f05fa79c640 1 Processor -- start 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05fa79c640 1 -- start start 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05fa79c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f41919b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05fa79c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f05f410ad30 con 0x7f05f4106780 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f41919b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f41919b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48944/0 (socket says 172.21.15.133:48944) 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05f3fff640 1 -- 172.21.15.133:0/2983914370 learned_addr learned my addr 172.21.15.133:0/2983914370 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05f3fff640 1 -- 172.21.15.133:0/2983914370 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f4191ef0 con 0x7f05f4106780 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05f3fff640 1 --2- 172.21.15.133:0/2983914370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f41919b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05e40097f0 tx=0x7f05e40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:03.835 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05f1ffb640 1 -- 172.21.15.133:0/2983914370 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4037040 con 0x7f05f4106780 2024-09-11T23:28:03.836 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05f1ffb640 1 -- 172.21.15.133:0/2983914370 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e4030c40 con 0x7f05f4106780 2024-09-11T23:28:03.836 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05fa79c640 1 -- 172.21.15.133:0/2983914370 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f41900e0 con 0x7f05f4106780 2024-09-11T23:28:03.836 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05f1ffb640 1 -- 172.21.15.133:0/2983914370 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e403a5b0 con 0x7f05f4106780 2024-09-11T23:28:03.836 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.334+0000 7f05fa79c640 1 -- 172.21.15.133:0/2983914370 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f4190620 con 0x7f05f4106780 2024-09-11T23:28:03.836 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.335+0000 7f05f1ffb640 1 -- 172.21.15.133:0/2983914370 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f05e403a7f0 con 0x7f05f4106780 2024-09-11T23:28:03.836 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.336+0000 7f05fa79c640 1 -- 172.21.15.133:0/2983914370 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f410aeb0 con 0x7f05f4106780 2024-09-11T23:28:03.836 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.336+0000 7f05f1ffb640 1 --2- 172.21.15.133:0/2983914370 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f05e0038f70 0x7f05e003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:03.836 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.336+0000 7f05f1ffb640 1 -- 172.21.15.133:0/2983914370 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f05e406c540 con 0x7f05f4106780 2024-09-11T23:28:03.836 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.336+0000 7f05ebfff640 1 --2- 172.21.15.133:0/2983914370 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f05e0038f70 0x7f05e003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:03.837 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.336+0000 7f05ebfff640 1 --2- 172.21.15.133:0/2983914370 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f05e0038f70 0x7f05e003b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0098c0 tx=0x7f05dc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:03.837 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.340+0000 7f05f1ffb640 1 -- 172.21.15.133:0/2983914370 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05e4031c00 con 0x7f05f4106780 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.570+0000 7f05fa79c640 1 -- 172.21.15.133:0/2983914370 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f05f4195630 con 0x7f05f4106780 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.910+0000 7f05f1ffb640 1 -- 172.21.15.133:0/2983914370 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f05e4044920 con 0x7f05f4106780 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.912+0000 7f05f1ffb640 1 -- 172.21.15.133:0/2983914370 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f05e4044400 con 0x7f05f4106780 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.913+0000 7f05fa79c640 1 -- 172.21.15.133:0/2983914370 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f05e0038f70 msgr2=0x7f05e003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.913+0000 7f05fa79c640 1 --2- 172.21.15.133:0/2983914370 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f05e0038f70 0x7f05e003b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0098c0 tx=0x7f05dc006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.913+0000 7f05fa79c640 1 -- 172.21.15.133:0/2983914370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 msgr2=0x7f05f41919b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.913+0000 7f05fa79c640 1 --2- 172.21.15.133:0/2983914370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f41919b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05e40097f0 tx=0x7f05e40327b0 comp rx=0 tx=0).stop 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.913+0000 7f05fa79c640 1 -- 172.21.15.133:0/2983914370 shutdown_connections 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.913+0000 7f05fa79c640 1 --2- 172.21.15.133:0/2983914370 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f05e0038f70 0x7f05e003b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.913+0000 7f05fa79c640 1 --2- 172.21.15.133:0/2983914370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f4106780 0x7f05f41919b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.913+0000 7f05fa79c640 1 -- 172.21.15.133:0/2983914370 >> 172.21.15.133:0/2983914370 conn(0x7f05f4101f10 msgr2=0x7f05f4102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.914+0000 7f05fa79c640 1 -- 172.21.15.133:0/2983914370 shutdown_connections 2024-09-11T23:28:03.838 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:02.914+0000 7f05fa79c640 1 -- 172.21.15.133:0/2983914370 wait complete. 2024-09-11T23:28:04.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:03 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2983914370' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T23:28:04.162 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:03 smithi133 ceph-mon[31430]: mgrmap e9: x(active, since 17s) 2024-09-11T23:28:04.162 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:03.972+0000 7f628e416040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:28:04.162 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:04 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:04.034+0000 7f628e416040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:28:04.447 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:04 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:04.227+0000 7f628e416040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:28:05.065 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:04 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2290067110' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T23:28:05.073 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.241+0000 7fb14a994640 1 Processor -- start 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.242+0000 7fb14a994640 1 -- start start 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.242+0000 7fb14a994640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb144106b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.242+0000 7fb14a994640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb144107150 con 0x7fb144106780 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.242+0000 7fb143fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb144106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.242+0000 7fb143fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb144106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48962/0 (socket says 172.21.15.133:48962) 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.242+0000 7fb143fff640 1 -- 172.21.15.133:0/2354524627 learned_addr learned my addr 172.21.15.133:0/2354524627 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.242+0000 7fb143fff640 1 -- 172.21.15.133:0/2354524627 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb144107980 con 0x7fb144106780 2024-09-11T23:28:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.243+0000 7fb143fff640 1 --2- 172.21.15.133:0/2354524627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb144106b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb134009820 tx=0x7fb13402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f316ef55811dbab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.243+0000 7fb142ffd640 1 -- 172.21.15.133:0/2354524627 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb134037070 con 0x7fb144106780 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.243+0000 7fb142ffd640 1 -- 172.21.15.133:0/2354524627 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb134032920 con 0x7fb144106780 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.243+0000 7fb142ffd640 1 -- 172.21.15.133:0/2354524627 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb134030810 con 0x7fb144106780 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.244+0000 7fb14a994640 1 -- 172.21.15.133:0/2354524627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 msgr2=0x7fb144106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.244+0000 7fb14a994640 1 --2- 172.21.15.133:0/2354524627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb144106b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb134009820 tx=0x7fb13402a270 comp rx=0 tx=0).stop 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.244+0000 7fb14a994640 1 -- 172.21.15.133:0/2354524627 shutdown_connections 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.244+0000 7fb14a994640 1 --2- 172.21.15.133:0/2354524627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb144106b80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.244+0000 7fb14a994640 1 -- 172.21.15.133:0/2354524627 >> 172.21.15.133:0/2354524627 conn(0x7fb144101f10 msgr2=0x7fb144104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.244+0000 7fb14a994640 1 -- 172.21.15.133:0/2354524627 shutdown_connections 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.244+0000 7fb14a994640 1 -- 172.21.15.133:0/2354524627 wait complete. 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.244+0000 7fb14a994640 1 Processor -- start 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.245+0000 7fb14a994640 1 -- start start 2024-09-11T23:28:05.075 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.245+0000 7fb14a994640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb14419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.245+0000 7fb14a994640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb14410ad30 con 0x7fb144106780 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.245+0000 7fb143fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb14419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.245+0000 7fb143fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb14419b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48974/0 (socket says 172.21.15.133:48974) 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.245+0000 7fb143fff640 1 -- 172.21.15.133:0/2290067110 learned_addr learned my addr 172.21.15.133:0/2290067110 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.245+0000 7fb143fff640 1 -- 172.21.15.133:0/2290067110 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14419be80 con 0x7fb144106780 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.245+0000 7fb143fff640 1 --2- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb14419b940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb134030560 tx=0x7fb134032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.245+0000 7fb1417fa640 1 -- 172.21.15.133:0/2290067110 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb134037040 con 0x7fb144106780 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.246+0000 7fb14a994640 1 -- 172.21.15.133:0/2290067110 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14419c110 con 0x7fb144106780 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.246+0000 7fb1417fa640 1 -- 172.21.15.133:0/2290067110 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb134040070 con 0x7fb144106780 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.246+0000 7fb1417fa640 1 -- 172.21.15.133:0/2290067110 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb13403a6d0 con 0x7fb144106780 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.246+0000 7fb14a994640 1 -- 172.21.15.133:0/2290067110 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14419ee00 con 0x7fb144106780 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.246+0000 7fb1417fa640 1 -- 172.21.15.133:0/2290067110 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb13403a940 con 0x7fb144106780 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.247+0000 7fb14a994640 1 -- 172.21.15.133:0/2290067110 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14410aeb0 con 0x7fb144106780 2024-09-11T23:28:05.076 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.247+0000 7fb1417fa640 1 --2- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fb12c038fc0 0x7fb12c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.247+0000 7fb1437fe640 1 -- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fb12c038fc0 msgr2=0x7fb12c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2152407978 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.247+0000 7fb1417fa640 1 -- 172.21.15.133:0/2290067110 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb13406c5a0 con 0x7fb144106780 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.247+0000 7fb1437fe640 1 --2- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fb12c038fc0 0x7fb12c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.251+0000 7fb1417fa640 1 -- 172.21.15.133:0/2290067110 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb134044c70 con 0x7fb144106780 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.448+0000 7fb1437fe640 1 -- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fb12c038fc0 msgr2=0x7fb12c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2152407978 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.448+0000 7fb1437fe640 1 --2- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fb12c038fc0 0x7fb12c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.472+0000 7fb14a994640 1 -- 172.21.15.133:0/2290067110 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb14402d900 con 0x7fb144106780 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.472+0000 7fb1417fa640 1 -- 172.21.15.133:0/2290067110 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fb134045980 con 0x7fb144106780 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.474+0000 7fb14a994640 1 -- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fb12c038fc0 msgr2=0x7fb12c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.474+0000 7fb14a994640 1 --2- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fb12c038fc0 0x7fb12c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.474+0000 7fb14a994640 1 -- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 msgr2=0x7fb14419b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.474+0000 7fb14a994640 1 --2- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb14419b940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb134030560 tx=0x7fb134032670 comp rx=0 tx=0).stop 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.474+0000 7fb14a994640 1 -- 172.21.15.133:0/2290067110 shutdown_connections 2024-09-11T23:28:05.077 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.474+0000 7fb14a994640 1 --2- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7fb12c038fc0 0x7fb12c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:05.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.474+0000 7fb14a994640 1 --2- 172.21.15.133:0/2290067110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb144106780 0x7fb14419b940 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:05.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.474+0000 7fb14a994640 1 -- 172.21.15.133:0/2290067110 >> 172.21.15.133:0/2290067110 conn(0x7fb144101f10 msgr2=0x7fb144102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:05.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.474+0000 7fb14a994640 1 -- 172.21.15.133:0/2290067110 shutdown_connections 2024-09-11T23:28:05.078 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:04.474+0000 7fb14a994640 1 -- 172.21.15.133:0/2290067110 wait complete. 2024-09-11T23:28:05.078 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for the mgr to restart... 2024-09-11T23:28:05.078 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr epoch 9... 2024-09-11T23:28:05.319 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:05 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:05.134+0000 7f628e416040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:28:05.319 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:05 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:05.202+0000 7f628e416040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:28:05.959 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:05 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:05.913+0000 7f628e416040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:28:06.373 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.026+0000 7f628e416040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:28:06.373 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.167+0000 7f628e416040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:28:06.373 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.226+0000 7f628e416040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:28:06.373 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.372+0000 7f628e416040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:28:06.654 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.439+0000 7f628e416040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:28:06.654 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.497+0000 7f628e416040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:28:06.655 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.653+0000 7f628e416040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:28:06.906 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.717+0000 7f628e416040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:28:06.906 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.782+0000 7f628e416040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:28:06.906 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.862+0000 7f628e416040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:28:07.209 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.921+0000 7f628e416040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:28:07.209 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:06.980+0000 7f628e416040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:28:08.209 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:07.756+0000 7f628e416040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:28:08.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: Active manager daemon x restarted 2024-09-11T23:28:08.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: Activating manager daemon x 2024-09-11T23:28:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T23:28:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: mgrmap e10: x(active, starting, since 0.00339602s) 2024-09-11T23:28:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:28:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:28:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:28:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:28:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:28:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: Manager daemon x is now available 2024-09-11T23:28:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:28:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:28:09.659 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:09.452+0000 7f61f972a640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:28:09] ENGINE Error in HTTPServer.serve 2024-09-11T23:28:09.659 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: Traceback (most recent call last): 2024-09-11T23:28:09.659 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:28:09.659 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self._connections.run(self.expiration_interval) 2024-09-11T23:28:09.659 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:28:09.659 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self._run(expiration_interval) 2024-09-11T23:28:09.659 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:28:09.659 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:28:09.659 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:28:09.659 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:28:09.659 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:28:09.660 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: s = self.context.wrap_socket( 2024-09-11T23:28:09.660 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:28:09.660 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: return self.sslsocket_class._create( 2024-09-11T23:28:09.660 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:28:09.660 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self.do_handshake() 2024-09-11T23:28:09.660 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:28:09.660 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self._sslobj.do_handshake() 2024-09-11T23:28:09.660 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:28:09.660 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:28:09.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:28:09.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T23:28:09.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T23:28:09.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:28:09.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.497+0000 7f54a9a91640 1 Processor -- start 2024-09-11T23:28:09.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.497+0000 7f54a9a91640 1 -- start start 2024-09-11T23:28:09.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.497+0000 7f54a9a91640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a410a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:09.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.497+0000 7f54a9a91640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f54a407bda0 con 0x7f54a4108390 2024-09-11T23:28:09.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.498+0000 7f54a37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a410a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:09.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.498+0000 7f54a37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a410a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48982/0 (socket says 172.21.15.133:48982) 2024-09-11T23:28:09.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.498+0000 7f54a37fe640 1 -- 172.21.15.133:0/1454411708 learned_addr learned my addr 172.21.15.133:0/1454411708 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.498+0000 7f54a37fe640 1 -- 172.21.15.133:0/1454411708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a410acc0 con 0x7f54a4108390 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.499+0000 7f54a37fe640 1 --2- 172.21.15.133:0/1454411708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a410a780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5498009820 tx=0x7f549802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fca94949a97f6ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.499+0000 7f54a27fc640 1 -- 172.21.15.133:0/1454411708 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5498037070 con 0x7f54a4108390 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.499+0000 7f54a27fc640 1 -- 172.21.15.133:0/1454411708 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5498032920 con 0x7f54a4108390 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.499+0000 7f54a27fc640 1 -- 172.21.15.133:0/1454411708 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5498030810 con 0x7f54a4108390 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.499+0000 7f54a9a91640 1 -- 172.21.15.133:0/1454411708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 msgr2=0x7f54a410a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.499+0000 7f54a9a91640 1 --2- 172.21.15.133:0/1454411708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a410a780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5498009820 tx=0x7f549802a270 comp rx=0 tx=0).stop 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.499+0000 7f54a9a91640 1 -- 172.21.15.133:0/1454411708 shutdown_connections 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.499+0000 7f54a9a91640 1 --2- 172.21.15.133:0/1454411708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a410a780 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.500+0000 7f54a9a91640 1 -- 172.21.15.133:0/1454411708 >> 172.21.15.133:0/1454411708 conn(0x7f54a4101f10 msgr2=0x7f54a4104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.500+0000 7f54a9a91640 1 -- 172.21.15.133:0/1454411708 shutdown_connections 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.500+0000 7f54a9a91640 1 -- 172.21.15.133:0/1454411708 wait complete. 2024-09-11T23:28:09.670 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.500+0000 7f54a9a91640 1 Processor -- start 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.500+0000 7f54a9a91640 1 -- start start 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.500+0000 7f54a9a91640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a4107c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.500+0000 7f54a9a91640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f54a410bce0 con 0x7f54a4108390 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.501+0000 7f54a37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a4107c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.501+0000 7f54a37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a4107c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48998/0 (socket says 172.21.15.133:48998) 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.501+0000 7f54a37fe640 1 -- 172.21.15.133:0/3685850613 learned_addr learned my addr 172.21.15.133:0/3685850613 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.501+0000 7f54a37fe640 1 -- 172.21.15.133:0/3685850613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a4108150 con 0x7f54a4108390 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.501+0000 7f54a37fe640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a4107c10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5498030420 tx=0x7f5498032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.501+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5498037040 con 0x7f54a4108390 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.501+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5498040070 con 0x7f54a4108390 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.501+0000 7f54a9a91640 1 -- 172.21.15.133:0/3685850613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a4106340 con 0x7f54a4108390 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.501+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f549803a6e0 con 0x7f54a4108390 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.501+0000 7f54a9a91640 1 -- 172.21.15.133:0/3685850613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a4106880 con 0x7f54a4108390 2024-09-11T23:28:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.502+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f549803a9e0 con 0x7f54a4108390 2024-09-11T23:28:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.502+0000 7f54a0ff9640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 0x7f548c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.502+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 --> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f548c03bb90 con 0x7f548c038fc0 2024-09-11T23:28:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.502+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f549806c740 con 0x7f54a4108390 2024-09-11T23:28:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.502+0000 7f54a2ffd640 1 -- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 msgr2=0x7f548c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2152407978 2024-09-11T23:28:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.502+0000 7f54a2ffd640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 0x7f548c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:28:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.703+0000 7f54a2ffd640 1 -- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 msgr2=0x7f548c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2152407978 2024-09-11T23:28:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:05.703+0000 7f54a2ffd640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 0x7f548c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:28:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:06.104+0000 7f54a2ffd640 1 -- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 msgr2=0x7f548c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2152407978 2024-09-11T23:28:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:06.104+0000 7f54a2ffd640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 0x7f548c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T23:28:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:06.905+0000 7f54a2ffd640 1 -- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 msgr2=0x7f548c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2152407978 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:06.905+0000 7f54a2ffd640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 0x7f548c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:07.759+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f549806b1f0 con 0x7f54a4108390 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:07.759+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 msgr2=0x7f548c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:07.759+0000 7f54a0ff9640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 0x7f548c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.761+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f549802ed50 con 0x7f54a4108390 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.761+0000 7f54a0ff9640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f548c03ca80 0x7f548c03ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.761+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f549803de70 con 0x7f548c03ca80 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.762+0000 7f54a2ffd640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f548c03ca80 0x7f548c03ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.763+0000 7f54a2ffd640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f548c03ca80 0x7f548c03ee70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5494003980 tx=0x7f5494006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.763+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f549803de70 con 0x7f548c03ca80 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.770+0000 7f54a9a91640 1 -- 172.21.15.133:0/3685850613 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f54a41087f0 con 0x7f548c03ca80 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.770+0000 7f54a0ff9640 1 -- 172.21.15.133:0/3685850613 <== mgr.14150 v2:172.21.15.133:6800/3324898762 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f54a41087f0 con 0x7f548c03ca80 2024-09-11T23:28:09.673 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.770+0000 7f54a9a91640 1 -- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f548c03ca80 msgr2=0x7f548c03ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.770+0000 7f54a9a91640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f548c03ca80 0x7f548c03ee70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5494003980 tx=0x7f5494006e10 comp rx=0 tx=0).stop 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.770+0000 7f54a9a91640 1 -- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 msgr2=0x7f54a4107c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.770+0000 7f54a9a91640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a4107c10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5498030420 tx=0x7f5498032670 comp rx=0 tx=0).stop 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.771+0000 7f54a9a91640 1 -- 172.21.15.133:0/3685850613 shutdown_connections 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.771+0000 7f54a9a91640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f548c03ca80 0x7f548c03ee70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.771+0000 7f54a9a91640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:6800/2152407978,v1:172.21.15.133:6801/2152407978] conn(0x7f548c038fc0 0x7f548c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.771+0000 7f54a9a91640 1 --2- 172.21.15.133:0/3685850613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54a4108390 0x7f54a4107c10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.771+0000 7f54a9a91640 1 -- 172.21.15.133:0/3685850613 >> 172.21.15.133:0/3685850613 conn(0x7f54a4101f10 msgr2=0x7f54a4102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.771+0000 7f54a9a91640 1 -- 172.21.15.133:0/3685850613 shutdown_connections 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:08.771+0000 7f54a9a91640 1 -- 172.21.15.133:0/3685850613 wait complete. 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:mgr epoch 9 is available 2024-09-11T23:28:09.674 INFO:teuthology.orchestra.run.smithi133.stdout:Generating a dashboard self-signed certificate... 2024-09-11T23:28:09.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-mon[31430]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T23:28:09.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-mon[31430]: mgrmap e11: x(active, since 1.00601s) 2024-09-11T23:28:09.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-mon[31430]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T23:28:09.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:09 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:10.975 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:10 smithi133 ceph-mon[31430]: [11/Sep/2024:23:28:09] ENGINE Bus STARTING 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:10 smithi133 ceph-mon[31430]: [11/Sep/2024:23:28:09] ENGINE Serving on https://172.21.15.133:7150 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:10 smithi133 ceph-mon[31430]: [11/Sep/2024:23:28:09] ENGINE Error in HTTPServer.serve 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: Traceback (most recent call last): 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: self._run(expiration_interval) 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: s = self.context.wrap_socket( 2024-09-11T23:28:10.976 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout: return self.sslsocket_class._create( 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout: self.do_handshake() 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout: self._sslobj.do_handshake() 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:10 smithi133 ceph-mon[31430]: [11/Sep/2024:23:28:09] ENGINE Serving on http://172.21.15.133:8765 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:10 smithi133 ceph-mon[31430]: [11/Sep/2024:23:28:09] ENGINE Bus STARTED 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:10 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:10 smithi133 ceph-mon[31430]: mgrmap e12: x(active, since 2s) 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:10 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:10.977 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:10 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:10.982 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T23:28:10.982 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.091+0000 7f77dbfa3640 1 Processor -- start 2024-09-11T23:28:10.982 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.091+0000 7f77dbfa3640 1 -- start start 2024-09-11T23:28:10.982 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.091+0000 7f77dbfa3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d407c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.091+0000 7f77dbfa3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77d407c7b0 con 0x7f77d407ddc0 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.092+0000 7f77d9d3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d407c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.092+0000 7f77d9d3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d407c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49042/0 (socket says 172.21.15.133:49042) 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.092+0000 7f77d9d3f640 1 -- 172.21.15.133:0/2521919772 learned_addr learned my addr 172.21.15.133:0/2521919772 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.092+0000 7f77d9d3f640 1 -- 172.21.15.133:0/2521919772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d407c930 con 0x7f77d407ddc0 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.092+0000 7f77d9d3f640 1 --2- 172.21.15.133:0/2521919772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d407c270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009820 tx=0x7f77d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b2b09894077dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.093+0000 7f77d8d3d640 1 -- 172.21.15.133:0/2521919772 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77d0037070 con 0x7f77d407ddc0 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.093+0000 7f77d8d3d640 1 -- 172.21.15.133:0/2521919772 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d0032920 con 0x7f77d407ddc0 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.093+0000 7f77d8d3d640 1 -- 172.21.15.133:0/2521919772 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77d0030810 con 0x7f77d407ddc0 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.093+0000 7f77dbfa3640 1 -- 172.21.15.133:0/2521919772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 msgr2=0x7f77d407c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.093+0000 7f77dbfa3640 1 --2- 172.21.15.133:0/2521919772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d407c270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009820 tx=0x7f77d002a270 comp rx=0 tx=0).stop 2024-09-11T23:28:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.093+0000 7f77dbfa3640 1 -- 172.21.15.133:0/2521919772 shutdown_connections 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.093+0000 7f77dbfa3640 1 --2- 172.21.15.133:0/2521919772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d407c270 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.093+0000 7f77dbfa3640 1 -- 172.21.15.133:0/2521919772 >> 172.21.15.133:0/2521919772 conn(0x7f77d4103080 msgr2=0x7f77d41054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.093+0000 7f77dbfa3640 1 -- 172.21.15.133:0/2521919772 shutdown_connections 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.093+0000 7f77dbfa3640 1 -- 172.21.15.133:0/2521919772 wait complete. 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.094+0000 7f77dbfa3640 1 Processor -- start 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.094+0000 7f77dbfa3640 1 -- start start 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.094+0000 7f77dbfa3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.094+0000 7f77dbfa3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77d410df00 con 0x7f77d407ddc0 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.094+0000 7f77d9d3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.094+0000 7f77d9d3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49044/0 (socket says 172.21.15.133:49044) 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.094+0000 7f77d9d3f640 1 -- 172.21.15.133:0/1155356734 learned_addr learned my addr 172.21.15.133:0/1155356734 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.094+0000 7f77d9d3f640 1 -- 172.21.15.133:0/1155356734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d419e020 con 0x7f77d407ddc0 2024-09-11T23:28:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.094+0000 7f77d9d3f640 1 --2- 172.21.15.133:0/1155356734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d419dae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f77d0030420 tx=0x7f77d0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.095+0000 7f77caffd640 1 -- 172.21.15.133:0/1155356734 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77d0037040 con 0x7f77d407ddc0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.095+0000 7f77caffd640 1 -- 172.21.15.133:0/1155356734 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d0040070 con 0x7f77d407ddc0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.095+0000 7f77dbfa3640 1 -- 172.21.15.133:0/1155356734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d419e2b0 con 0x7f77d407ddc0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.095+0000 7f77caffd640 1 -- 172.21.15.133:0/1155356734 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77d003a6e0 con 0x7f77d407ddc0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.095+0000 7f77dbfa3640 1 -- 172.21.15.133:0/1155356734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d41a0fa0 con 0x7f77d407ddc0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.096+0000 7f77caffd640 1 -- 172.21.15.133:0/1155356734 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f77d003bb20 con 0x7f77d407ddc0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.096+0000 7f77dbfa3640 1 -- 172.21.15.133:0/1155356734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d407c2f0 con 0x7f77d407ddc0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.096+0000 7f77caffd640 1 --2- 172.21.15.133:0/1155356734 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77c0038f00 0x7f77c003b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.096+0000 7f77caffd640 1 -- 172.21.15.133:0/1155356734 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f77d006c520 con 0x7f77d407ddc0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.096+0000 7f77d953e640 1 --2- 172.21.15.133:0/1155356734 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77c0038f00 0x7f77c003b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.097+0000 7f77d953e640 1 --2- 172.21.15.133:0/1155356734 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77c0038f00 0x7f77c003b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77c40098c0 tx=0x7f77c4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.101+0000 7f77caffd640 1 -- 172.21.15.133:0/1155356734 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77d004a200 con 0x7f77d407ddc0 2024-09-11T23:28:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.230+0000 7f77caffd640 1 -- 172.21.15.133:0/1155356734 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f77d003a8c0 con 0x7f77d407ddc0 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.283+0000 7f77dbfa3640 1 -- 172.21.15.133:0/1155356734 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f77d4104380 con 0x7f77c0038f00 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.359+0000 7f77caffd640 1 -- 172.21.15.133:0/1155356734 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f77d4104380 con 0x7f77c0038f00 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.361+0000 7f77dbfa3640 1 -- 172.21.15.133:0/1155356734 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77c0038f00 msgr2=0x7f77c003b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.361+0000 7f77dbfa3640 1 --2- 172.21.15.133:0/1155356734 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77c0038f00 0x7f77c003b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77c40098c0 tx=0x7f77c4006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.361+0000 7f77dbfa3640 1 -- 172.21.15.133:0/1155356734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 msgr2=0x7f77d419dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.361+0000 7f77dbfa3640 1 --2- 172.21.15.133:0/1155356734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d419dae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f77d0030420 tx=0x7f77d0032670 comp rx=0 tx=0).stop 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.362+0000 7f77dbfa3640 1 -- 172.21.15.133:0/1155356734 shutdown_connections 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.362+0000 7f77dbfa3640 1 --2- 172.21.15.133:0/1155356734 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77c0038f00 0x7f77c003b3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.362+0000 7f77dbfa3640 1 --2- 172.21.15.133:0/1155356734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77d407ddc0 0x7f77d419dae0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.362+0000 7f77dbfa3640 1 -- 172.21.15.133:0/1155356734 >> 172.21.15.133:0/1155356734 conn(0x7f77d4103080 msgr2=0x7f77d4103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.362+0000 7f77dbfa3640 1 -- 172.21.15.133:0/1155356734 shutdown_connections 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:10.362+0000 7f77dbfa3640 1 -- 172.21.15.133:0/1155356734 wait complete. 2024-09-11T23:28:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial admin user... 2024-09-11T23:28:11.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:11 smithi133 ceph-mon[31430]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:28:12.640 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$7xUcbyb7U7k8gVpKs1okHuRVpQIsgJYImElnZ0pIpvJHrZIoxBzGm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726097292, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T23:28:12.640 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.600+0000 7f73aefdb640 1 Processor -- start 2024-09-11T23:28:12.640 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.600+0000 7f73aefdb640 1 -- start start 2024-09-11T23:28:12.640 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.601+0000 7f73aefdb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a810a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.640 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.601+0000 7f73aefdb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73a807bde0 con 0x7f73a8108350 2024-09-11T23:28:12.640 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.601+0000 7f73acd77640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a810a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.640 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.601+0000 7f73acd77640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a810a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49052/0 (socket says 172.21.15.133:49052) 2024-09-11T23:28:12.640 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.601+0000 7f73acd77640 1 -- 172.21.15.133:0/2861472820 learned_addr learned my addr 172.21.15.133:0/2861472820 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:12.640 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.601+0000 7f73acd77640 1 -- 172.21.15.133:0/2861472820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a810ac80 con 0x7f73a8108350 2024-09-11T23:28:12.640 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.602+0000 7f73acd77640 1 --2- 172.21.15.133:0/2861472820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a810a740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009820 tx=0x7f73a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1ba2bd7045c8c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.602+0000 7f739f7fe640 1 -- 172.21.15.133:0/2861472820 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73a4037070 con 0x7f73a8108350 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.602+0000 7f739f7fe640 1 -- 172.21.15.133:0/2861472820 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a4032920 con 0x7f73a8108350 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.602+0000 7f739f7fe640 1 -- 172.21.15.133:0/2861472820 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73a4030810 con 0x7f73a8108350 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.602+0000 7f73aefdb640 1 -- 172.21.15.133:0/2861472820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 msgr2=0x7f73a810a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.602+0000 7f73aefdb640 1 --2- 172.21.15.133:0/2861472820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a810a740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009820 tx=0x7f73a402a270 comp rx=0 tx=0).stop 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.603+0000 7f73aefdb640 1 -- 172.21.15.133:0/2861472820 shutdown_connections 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.603+0000 7f73aefdb640 1 --2- 172.21.15.133:0/2861472820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a810a740 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.603+0000 7f73aefdb640 1 -- 172.21.15.133:0/2861472820 >> 172.21.15.133:0/2861472820 conn(0x7f73a8101f10 msgr2=0x7f73a8104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.603+0000 7f73aefdb640 1 -- 172.21.15.133:0/2861472820 shutdown_connections 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.603+0000 7f73aefdb640 1 -- 172.21.15.133:0/2861472820 wait complete. 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.603+0000 7f73aefdb640 1 Processor -- start 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.603+0000 7f73aefdb640 1 -- start start 2024-09-11T23:28:12.641 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.603+0000 7f73aefdb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a8107bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.603+0000 7f73aefdb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73a810bd00 con 0x7f73a8108350 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.603+0000 7f73acd77640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a8107bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.603+0000 7f73acd77640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a8107bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49068/0 (socket says 172.21.15.133:49068) 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.604+0000 7f73acd77640 1 -- 172.21.15.133:0/1711054363 learned_addr learned my addr 172.21.15.133:0/1711054363 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.604+0000 7f73acd77640 1 -- 172.21.15.133:0/1711054363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a8108110 con 0x7f73a8108350 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.604+0000 7f73acd77640 1 --2- 172.21.15.133:0/1711054363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a8107bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73a4030420 tx=0x7f73a4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.604+0000 7f739dffb640 1 -- 172.21.15.133:0/1711054363 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73a4037040 con 0x7f73a8108350 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.604+0000 7f73aefdb640 1 -- 172.21.15.133:0/1711054363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a8106300 con 0x7f73a8108350 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.604+0000 7f739dffb640 1 -- 172.21.15.133:0/1711054363 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a4040070 con 0x7f73a8108350 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.604+0000 7f739dffb640 1 -- 172.21.15.133:0/1711054363 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73a403a6e0 con 0x7f73a8108350 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.604+0000 7f73aefdb640 1 -- 172.21.15.133:0/1711054363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a8106840 con 0x7f73a8108350 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.605+0000 7f739dffb640 1 -- 172.21.15.133:0/1711054363 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f73a403a9e0 con 0x7f73a8108350 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.605+0000 7f739dffb640 1 --2- 172.21.15.133:0/1711054363 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7390038fc0 0x7f739003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:12.642 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.605+0000 7f739dffb640 1 -- 172.21.15.133:0/1711054363 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f73a406c750 con 0x7f73a8108350 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.605+0000 7f739ffff640 1 --2- 172.21.15.133:0/1711054363 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7390038fc0 0x7f739003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.605+0000 7f73aefdb640 1 -- 172.21.15.133:0/1711054363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a81087b0 con 0x7f73a8108350 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.606+0000 7f739ffff640 1 --2- 172.21.15.133:0/1711054363 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7390038fc0 0x7f739003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f73980098c0 tx=0x7f7398006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.610+0000 7f739dffb640 1 -- 172.21.15.133:0/1711054363 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73a4041270 con 0x7f73a8108350 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:11.799+0000 7f73aefdb640 1 -- 172.21.15.133:0/1711054363 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f73a8106ec0 con 0x7f7390038fc0 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:12.020+0000 7f739dffb640 1 -- 172.21.15.133:0/1711054363 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f73a8106ec0 con 0x7f7390038fc0 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:12.023+0000 7f73aefdb640 1 -- 172.21.15.133:0/1711054363 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7390038fc0 msgr2=0x7f739003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:12.023+0000 7f73aefdb640 1 --2- 172.21.15.133:0/1711054363 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7390038fc0 0x7f739003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f73980098c0 tx=0x7f7398006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:12.023+0000 7f73aefdb640 1 -- 172.21.15.133:0/1711054363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 msgr2=0x7f73a8107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:12.023+0000 7f73aefdb640 1 --2- 172.21.15.133:0/1711054363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a8107bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73a4030420 tx=0x7f73a4032670 comp rx=0 tx=0).stop 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:12.023+0000 7f73aefdb640 1 -- 172.21.15.133:0/1711054363 shutdown_connections 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:12.023+0000 7f73aefdb640 1 --2- 172.21.15.133:0/1711054363 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7390038fc0 0x7f739003b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:12.023+0000 7f73aefdb640 1 --2- 172.21.15.133:0/1711054363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a8108350 0x7f73a8107bd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:12.643 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:12.023+0000 7f73aefdb640 1 -- 172.21.15.133:0/1711054363 >> 172.21.15.133:0/1711054363 conn(0x7f73a8101f10 msgr2=0x7f73a8102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:12.644 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:12.023+0000 7f73aefdb640 1 -- 172.21.15.133:0/1711054363 shutdown_connections 2024-09-11T23:28:12.644 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:12.023+0000 7f73aefdb640 1 -- 172.21.15.133:0/1711054363 wait complete. 2024-09-11T23:28:12.644 INFO:teuthology.orchestra.run.smithi133.stdout:Fetching dashboard port number... 2024-09-11T23:28:12.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:12 smithi133 ceph-mon[31430]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:28:12.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:12 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:14.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:13 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1475470811' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T23:28:14.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T23:28:14.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.139+0000 7fc6b2067640 1 Processor -- start 2024-09-11T23:28:14.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.139+0000 7fc6b2067640 1 -- start start 2024-09-11T23:28:14.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.140+0000 7fc6b2067640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac10a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:14.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.140+0000 7fc6b2067640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac07be20 con 0x7fc6ac108310 2024-09-11T23:28:14.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.140+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac10a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:14.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.140+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac10a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57994/0 (socket says 172.21.15.133:57994) 2024-09-11T23:28:14.333 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.140+0000 7fc6ab7fe640 1 -- 172.21.15.133:0/2164538781 learned_addr learned my addr 172.21.15.133:0/2164538781 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.140+0000 7fc6ab7fe640 1 -- 172.21.15.133:0/2164538781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac10ac40 con 0x7fc6ac108310 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.141+0000 7fc6ab7fe640 1 --2- 172.21.15.133:0/2164538781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac10a700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009820 tx=0x7fc69c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1345678f960eb366 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.141+0000 7fc6aa7fc640 1 -- 172.21.15.133:0/2164538781 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc69c037070 con 0x7fc6ac108310 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.141+0000 7fc6aa7fc640 1 -- 172.21.15.133:0/2164538781 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc69c032920 con 0x7fc6ac108310 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.141+0000 7fc6aa7fc640 1 -- 172.21.15.133:0/2164538781 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc69c030810 con 0x7fc6ac108310 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.141+0000 7fc6b2067640 1 -- 172.21.15.133:0/2164538781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 msgr2=0x7fc6ac10a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.141+0000 7fc6b2067640 1 --2- 172.21.15.133:0/2164538781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac10a700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009820 tx=0x7fc69c02a270 comp rx=0 tx=0).stop 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.141+0000 7fc6b2067640 1 -- 172.21.15.133:0/2164538781 shutdown_connections 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.141+0000 7fc6b2067640 1 --2- 172.21.15.133:0/2164538781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac10a700 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.141+0000 7fc6b2067640 1 -- 172.21.15.133:0/2164538781 >> 172.21.15.133:0/2164538781 conn(0x7fc6ac101f10 msgr2=0x7fc6ac104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.141+0000 7fc6b2067640 1 -- 172.21.15.133:0/2164538781 shutdown_connections 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.141+0000 7fc6b2067640 1 -- 172.21.15.133:0/2164538781 wait complete. 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.142+0000 7fc6b2067640 1 Processor -- start 2024-09-11T23:28:14.334 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.142+0000 7fc6b2067640 1 -- start start 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.142+0000 7fc6b2067640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.142+0000 7fc6b2067640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac10bc60 con 0x7fc6ac108310 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.142+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.142+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac19b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58000/0 (socket says 172.21.15.133:58000) 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.142+0000 7fc6ab7fe640 1 -- 172.21.15.133:0/1475470811 learned_addr learned my addr 172.21.15.133:0/1475470811 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.143+0000 7fc6ab7fe640 1 -- 172.21.15.133:0/1475470811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac19bdc0 con 0x7fc6ac108310 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.143+0000 7fc6ab7fe640 1 --2- 172.21.15.133:0/1475470811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac19b880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc69c030420 tx=0x7fc69c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.143+0000 7fc6a8ff9640 1 -- 172.21.15.133:0/1475470811 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc69c037040 con 0x7fc6ac108310 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.143+0000 7fc6a8ff9640 1 -- 172.21.15.133:0/1475470811 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc69c040070 con 0x7fc6ac108310 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.143+0000 7fc6b2067640 1 -- 172.21.15.133:0/1475470811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ac19c050 con 0x7fc6ac108310 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.143+0000 7fc6a8ff9640 1 -- 172.21.15.133:0/1475470811 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc69c03a6e0 con 0x7fc6ac108310 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.143+0000 7fc6b2067640 1 -- 172.21.15.133:0/1475470811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ac19ed40 con 0x7fc6ac108310 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.144+0000 7fc6a8ff9640 1 -- 172.21.15.133:0/1475470811 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fc69c03a9e0 con 0x7fc6ac108310 2024-09-11T23:28:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.144+0000 7fc6b2067640 1 -- 172.21.15.133:0/1475470811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ac108770 con 0x7fc6ac108310 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.144+0000 7fc6a8ff9640 1 --2- 172.21.15.133:0/1475470811 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc694038fc0 0x7fc69403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.144+0000 7fc6a8ff9640 1 -- 172.21.15.133:0/1475470811 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc69c06c9c0 con 0x7fc6ac108310 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.144+0000 7fc6aaffd640 1 --2- 172.21.15.133:0/1475470811 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc694038fc0 0x7fc69403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.144+0000 7fc6aaffd640 1 --2- 172.21.15.133:0/1475470811 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc694038fc0 0x7fc69403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00098c0 tx=0x7fc6a0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.149+0000 7fc6a8ff9640 1 -- 172.21.15.133:0/1475470811 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc69c06ee20 con 0x7fc6ac108310 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.320+0000 7fc6b2067640 1 -- 172.21.15.133:0/1475470811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc6ac19ef10 con 0x7fc6ac108310 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.320+0000 7fc6a8ff9640 1 -- 172.21.15.133:0/1475470811 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fc69c03ae20 con 0x7fc6ac108310 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.322+0000 7fc6b2067640 1 -- 172.21.15.133:0/1475470811 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc694038fc0 msgr2=0x7fc69403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.322+0000 7fc6b2067640 1 --2- 172.21.15.133:0/1475470811 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc694038fc0 0x7fc69403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00098c0 tx=0x7fc6a0006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.322+0000 7fc6b2067640 1 -- 172.21.15.133:0/1475470811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 msgr2=0x7fc6ac19b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.322+0000 7fc6b2067640 1 --2- 172.21.15.133:0/1475470811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac19b880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc69c030420 tx=0x7fc69c032670 comp rx=0 tx=0).stop 2024-09-11T23:28:14.336 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.322+0000 7fc6b2067640 1 -- 172.21.15.133:0/1475470811 shutdown_connections 2024-09-11T23:28:14.337 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.322+0000 7fc6b2067640 1 --2- 172.21.15.133:0/1475470811 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc694038fc0 0x7fc69403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:14.337 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.322+0000 7fc6b2067640 1 --2- 172.21.15.133:0/1475470811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6ac108310 0x7fc6ac19b880 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:14.337 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.322+0000 7fc6b2067640 1 -- 172.21.15.133:0/1475470811 >> 172.21.15.133:0/1475470811 conn(0x7fc6ac101f10 msgr2=0x7fc6ac102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:14.337 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.322+0000 7fc6b2067640 1 -- 172.21.15.133:0/1475470811 shutdown_connections 2024-09-11T23:28:14.337 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:13.322+0000 7fc6b2067640 1 -- 172.21.15.133:0/1475470811 wait complete. 2024-09-11T23:28:14.342 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:28:14.343 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2024-09-11T23:28:14.350 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:28:14.350 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-09-11T23:28:14.350 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2024-09-11T23:28:14.350 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T23:28:14.352 INFO:teuthology.orchestra.run.smithi133.stdout:Ceph Dashboard is now available at: 2024-09-11T23:28:14.352 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:14.352 INFO:teuthology.orchestra.run.smithi133.stdout: URL: https://smithi133.front.sepia.ceph.com:8443/ 2024-09-11T23:28:14.352 INFO:teuthology.orchestra.run.smithi133.stdout: User: admin 2024-09-11T23:28:14.352 INFO:teuthology.orchestra.run.smithi133.stdout: Password: cycourlx47 2024-09-11T23:28:14.352 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:14.353 INFO:teuthology.orchestra.run.smithi133.stdout:Saving cluster configuration to /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config directory 2024-09-11T23:28:14.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:14 smithi133 ceph-mon[31430]: mgrmap e13: x(active, since 6s) 2024-09-11T23:28:15.581 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.726+0000 7f2f4da39640 1 Processor -- start 2024-09-11T23:28:15.582 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.726+0000 7f2f4da39640 1 -- start start 2024-09-11T23:28:15.582 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.726+0000 7f2f4da39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4810a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.582 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.726+0000 7f2f4da39640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4807be00 con 0x7f2f48108330 2024-09-11T23:28:15.582 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.727+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4810a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.582 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.727+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4810a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58008/0 (socket says 172.21.15.133:58008) 2024-09-11T23:28:15.582 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.727+0000 7f2f46ffd640 1 -- 172.21.15.133:0/591787303 learned_addr learned my addr 172.21.15.133:0/591787303 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:15.582 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.727+0000 7f2f46ffd640 1 -- 172.21.15.133:0/591787303 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4810ac60 con 0x7f2f48108330 2024-09-11T23:28:15.582 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.728+0000 7f2f46ffd640 1 --2- 172.21.15.133:0/591787303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4810a720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009820 tx=0x7f2f3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5d5652d428d569 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:15.582 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.728+0000 7f2f467fc640 1 -- 172.21.15.133:0/591787303 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f38037070 con 0x7f2f48108330 2024-09-11T23:28:15.583 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.728+0000 7f2f467fc640 1 -- 172.21.15.133:0/591787303 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f38032920 con 0x7f2f48108330 2024-09-11T23:28:15.583 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.728+0000 7f2f467fc640 1 -- 172.21.15.133:0/591787303 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f38030810 con 0x7f2f48108330 2024-09-11T23:28:15.583 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.728+0000 7f2f4da39640 1 -- 172.21.15.133:0/591787303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 msgr2=0x7f2f4810a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.583 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.728+0000 7f2f4da39640 1 --2- 172.21.15.133:0/591787303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4810a720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009820 tx=0x7f2f3802a270 comp rx=0 tx=0).stop 2024-09-11T23:28:15.583 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.728+0000 7f2f4da39640 1 -- 172.21.15.133:0/591787303 shutdown_connections 2024-09-11T23:28:15.583 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.728+0000 7f2f4da39640 1 --2- 172.21.15.133:0/591787303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4810a720 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.583 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.728+0000 7f2f4da39640 1 -- 172.21.15.133:0/591787303 >> 172.21.15.133:0/591787303 conn(0x7f2f48101f10 msgr2=0x7f2f48104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:15.583 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.729+0000 7f2f4da39640 1 -- 172.21.15.133:0/591787303 shutdown_connections 2024-09-11T23:28:15.583 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.729+0000 7f2f4da39640 1 -- 172.21.15.133:0/591787303 wait complete. 2024-09-11T23:28:15.583 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.729+0000 7f2f4da39640 1 Processor -- start 2024-09-11T23:28:15.583 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.729+0000 7f2f4da39640 1 -- start start 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.729+0000 7f2f4da39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.729+0000 7f2f4da39640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4810bce0 con 0x7f2f48108330 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.729+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.730+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4819b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58012/0 (socket says 172.21.15.133:58012) 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.730+0000 7f2f46ffd640 1 -- 172.21.15.133:0/2621246078 learned_addr learned my addr 172.21.15.133:0/2621246078 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.730+0000 7f2f46ffd640 1 -- 172.21.15.133:0/2621246078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4819bde0 con 0x7f2f48108330 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.730+0000 7f2f46ffd640 1 --2- 172.21.15.133:0/2621246078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4819b8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f380097f0 tx=0x7f2f380327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.730+0000 7f2f4ca37640 1 -- 172.21.15.133:0/2621246078 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f38037040 con 0x7f2f48108330 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.730+0000 7f2f4ca37640 1 -- 172.21.15.133:0/2621246078 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f38030c40 con 0x7f2f48108330 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.730+0000 7f2f4da39640 1 -- 172.21.15.133:0/2621246078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4819c070 con 0x7f2f48108330 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.730+0000 7f2f4ca37640 1 -- 172.21.15.133:0/2621246078 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f3803a5d0 con 0x7f2f48108330 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.730+0000 7f2f4da39640 1 -- 172.21.15.133:0/2621246078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4819ed60 con 0x7f2f48108330 2024-09-11T23:28:15.584 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.731+0000 7f2f4ca37640 1 -- 172.21.15.133:0/2621246078 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f2f3803b3f0 con 0x7f2f48108330 2024-09-11T23:28:15.585 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.731+0000 7f2f4da39640 1 -- 172.21.15.133:0/2621246078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f48108790 con 0x7f2f48108330 2024-09-11T23:28:15.585 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.731+0000 7f2f4ca37640 1 --2- 172.21.15.133:0/2621246078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2f340390c0 0x7f2f3403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.585 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.731+0000 7f2f4ca37640 1 -- 172.21.15.133:0/2621246078 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2f3806c650 con 0x7f2f48108330 2024-09-11T23:28:15.585 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.731+0000 7f2f3e7fc640 1 --2- 172.21.15.133:0/2621246078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2f340390c0 0x7f2f3403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.585 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.732+0000 7f2f3e7fc640 1 --2- 172.21.15.133:0/2621246078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2f340390c0 0x7f2f3403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2f300098c0 tx=0x7f2f30006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:15.585 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.736+0000 7f2f4ca37640 1 -- 172.21.15.133:0/2621246078 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f38041510 con 0x7f2f48108330 2024-09-11T23:28:15.585 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.991+0000 7f2f4da39640 1 -- 172.21.15.133:0/2621246078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f2f4819ef30 con 0x7f2f48108330 2024-09-11T23:28:15.585 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.994+0000 7f2f4ca37640 1 -- 172.21.15.133:0/2621246078 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f2f380412b0 con 0x7f2f48108330 2024-09-11T23:28:15.585 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T23:28:15.586 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.996+0000 7f2f4da39640 1 -- 172.21.15.133:0/2621246078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2f340390c0 msgr2=0x7f2f3403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.586 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.996+0000 7f2f4da39640 1 --2- 172.21.15.133:0/2621246078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2f340390c0 0x7f2f3403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2f300098c0 tx=0x7f2f30006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.586 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.996+0000 7f2f4da39640 1 -- 172.21.15.133:0/2621246078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 msgr2=0x7f2f4819b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.586 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.996+0000 7f2f4da39640 1 --2- 172.21.15.133:0/2621246078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4819b8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f380097f0 tx=0x7f2f380327b0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.586 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.996+0000 7f2f4da39640 1 -- 172.21.15.133:0/2621246078 shutdown_connections 2024-09-11T23:28:15.586 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.996+0000 7f2f4da39640 1 --2- 172.21.15.133:0/2621246078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2f340390c0 0x7f2f3403b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.587 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.996+0000 7f2f4da39640 1 --2- 172.21.15.133:0/2621246078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f48108330 0x7f2f4819b8a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.587 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.996+0000 7f2f4da39640 1 -- 172.21.15.133:0/2621246078 >> 172.21.15.133:0/2621246078 conn(0x7f2f48101f10 msgr2=0x7f2f48102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:15.587 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.996+0000 7f2f4da39640 1 -- 172.21.15.133:0/2621246078 shutdown_connections 2024-09-11T23:28:15.587 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-09-11T23:28:14.996+0000 7f2f4da39640 1 -- 172.21.15.133:0/2621246078 wait complete. 2024-09-11T23:28:15.587 INFO:teuthology.orchestra.run.smithi133.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T23:28:15.587 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:15.587 INFO:teuthology.orchestra.run.smithi133.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:28:15.588 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:15.588 INFO:teuthology.orchestra.run.smithi133.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T23:28:15.588 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:15.588 INFO:teuthology.orchestra.run.smithi133.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T23:28:15.588 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:15.588 INFO:teuthology.orchestra.run.smithi133.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T23:28:15.588 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:15.589 INFO:teuthology.orchestra.run.smithi133.stdout: ceph telemetry on 2024-09-11T23:28:15.589 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:15.589 INFO:teuthology.orchestra.run.smithi133.stdout:For more information see: 2024-09-11T23:28:15.589 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:15.589 INFO:teuthology.orchestra.run.smithi133.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-11T23:28:15.589 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:15.589 INFO:teuthology.orchestra.run.smithi133.stdout:Bootstrap complete. 2024-09-11T23:28:15.619 INFO:tasks.cephadm:Fetching config... 2024-09-11T23:28:15.619 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:28:15.620 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T23:28:15.635 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T23:28:15.636 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:28:15.636 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T23:28:15.691 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T23:28:15.691 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:28:15.691 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T23:28:15.755 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T23:28:15.755 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:28:15.755 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T23:28:15.811 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T23:28:15.812 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmXJ7ZuLXqnx6VQwiRJCde3t5EEGFo6vYYYf1VRy2nAqb8U/UMc5wSkfpclZ4EFz9Hi0guh6raqkQ+lE6Zo6PZJg0uTFQJOqIoXyzzbI9CKZPtAiCa+1L81YEJLVCThlvFn0JasWRLZLNwiSQTbQxeCe+K3dWdBxZZqw0TRQ8nYYuFFUMljqwMKotOhhGoz/WCU0sKz/029h9DO+dLSb485X+CSW68ZCKaeLClI33KIdaJVdDCPuY66IUtJMT6LOXK76lTeuDqVvPq1emFHr5AwUQKzs/f8MfGI14Ap13WlJWE2AFrQ25v5Q5b7agQqyaz4BY3/L6w5F59FtZPgPpId3FjFt9GiOmjQPcQK5E4kkX1F6YTWpN8i0CdmO4CqCUQZqkaYzbm/ZxpRLtrTpNccJdBV559xUadt0qCP6KxxAOSpSM+0f3mChmyDtvrma8ym9jcP5tjhNq2ljFaySjZ3jiTtxiYT4yPvqRjJHZMVfIUO5qrM85s0U5sHUfqClU= ceph-42321b88-7095-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:28:15.886 INFO:teuthology.orchestra.run.smithi133.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmXJ7ZuLXqnx6VQwiRJCde3t5EEGFo6vYYYf1VRy2nAqb8U/UMc5wSkfpclZ4EFz9Hi0guh6raqkQ+lE6Zo6PZJg0uTFQJOqIoXyzzbI9CKZPtAiCa+1L81YEJLVCThlvFn0JasWRLZLNwiSQTbQxeCe+K3dWdBxZZqw0TRQ8nYYuFFUMljqwMKotOhhGoz/WCU0sKz/029h9DO+dLSb485X+CSW68ZCKaeLClI33KIdaJVdDCPuY66IUtJMT6LOXK76lTeuDqVvPq1emFHr5AwUQKzs/f8MfGI14Ap13WlJWE2AFrQ25v5Q5b7agQqyaz4BY3/L6w5F59FtZPgPpId3FjFt9GiOmjQPcQK5E4kkX1F6YTWpN8i0CdmO4CqCUQZqkaYzbm/ZxpRLtrTpNccJdBV559xUadt0qCP6KxxAOSpSM+0f3mChmyDtvrma8ym9jcP5tjhNq2ljFaySjZ3jiTtxiYT4yPvqRjJHZMVfIUO5qrM85s0U5sHUfqClU= ceph-42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:15.897 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmXJ7ZuLXqnx6VQwiRJCde3t5EEGFo6vYYYf1VRy2nAqb8U/UMc5wSkfpclZ4EFz9Hi0guh6raqkQ+lE6Zo6PZJg0uTFQJOqIoXyzzbI9CKZPtAiCa+1L81YEJLVCThlvFn0JasWRLZLNwiSQTbQxeCe+K3dWdBxZZqw0TRQ8nYYuFFUMljqwMKotOhhGoz/WCU0sKz/029h9DO+dLSb485X+CSW68ZCKaeLClI33KIdaJVdDCPuY66IUtJMT6LOXK76lTeuDqVvPq1emFHr5AwUQKzs/f8MfGI14Ap13WlJWE2AFrQ25v5Q5b7agQqyaz4BY3/L6w5F59FtZPgPpId3FjFt9GiOmjQPcQK5E4kkX1F6YTWpN8i0CdmO4CqCUQZqkaYzbm/ZxpRLtrTpNccJdBV559xUadt0qCP6KxxAOSpSM+0f3mChmyDtvrma8ym9jcP5tjhNq2ljFaySjZ3jiTtxiYT4yPvqRjJHZMVfIUO5qrM85s0U5sHUfqClU= ceph-42321b88-7095-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:28:15.940 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmXJ7ZuLXqnx6VQwiRJCde3t5EEGFo6vYYYf1VRy2nAqb8U/UMc5wSkfpclZ4EFz9Hi0guh6raqkQ+lE6Zo6PZJg0uTFQJOqIoXyzzbI9CKZPtAiCa+1L81YEJLVCThlvFn0JasWRLZLNwiSQTbQxeCe+K3dWdBxZZqw0TRQ8nYYuFFUMljqwMKotOhhGoz/WCU0sKz/029h9DO+dLSb485X+CSW68ZCKaeLClI33KIdaJVdDCPuY66IUtJMT6LOXK76lTeuDqVvPq1emFHr5AwUQKzs/f8MfGI14Ap13WlJWE2AFrQ25v5Q5b7agQqyaz4BY3/L6w5F59FtZPgPpId3FjFt9GiOmjQPcQK5E4kkX1F6YTWpN8i0CdmO4CqCUQZqkaYzbm/ZxpRLtrTpNccJdBV559xUadt0qCP6KxxAOSpSM+0f3mChmyDtvrma8ym9jcP5tjhNq2ljFaySjZ3jiTtxiYT4yPvqRjJHZMVfIUO5qrM85s0U5sHUfqClU= ceph-42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:15.951 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmXJ7ZuLXqnx6VQwiRJCde3t5EEGFo6vYYYf1VRy2nAqb8U/UMc5wSkfpclZ4EFz9Hi0guh6raqkQ+lE6Zo6PZJg0uTFQJOqIoXyzzbI9CKZPtAiCa+1L81YEJLVCThlvFn0JasWRLZLNwiSQTbQxeCe+K3dWdBxZZqw0TRQ8nYYuFFUMljqwMKotOhhGoz/WCU0sKz/029h9DO+dLSb485X+CSW68ZCKaeLClI33KIdaJVdDCPuY66IUtJMT6LOXK76lTeuDqVvPq1emFHr5AwUQKzs/f8MfGI14Ap13WlJWE2AFrQ25v5Q5b7agQqyaz4BY3/L6w5F59FtZPgPpId3FjFt9GiOmjQPcQK5E4kkX1F6YTWpN8i0CdmO4CqCUQZqkaYzbm/ZxpRLtrTpNccJdBV559xUadt0qCP6KxxAOSpSM+0f3mChmyDtvrma8ym9jcP5tjhNq2ljFaySjZ3jiTtxiYT4yPvqRjJHZMVfIUO5qrM85s0U5sHUfqClU= ceph-42321b88-7095-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:28:15.992 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmXJ7ZuLXqnx6VQwiRJCde3t5EEGFo6vYYYf1VRy2nAqb8U/UMc5wSkfpclZ4EFz9Hi0guh6raqkQ+lE6Zo6PZJg0uTFQJOqIoXyzzbI9CKZPtAiCa+1L81YEJLVCThlvFn0JasWRLZLNwiSQTbQxeCe+K3dWdBxZZqw0TRQ8nYYuFFUMljqwMKotOhhGoz/WCU0sKz/029h9DO+dLSb485X+CSW68ZCKaeLClI33KIdaJVdDCPuY66IUtJMT6LOXK76lTeuDqVvPq1emFHr5AwUQKzs/f8MfGI14Ap13WlJWE2AFrQ25v5Q5b7agQqyaz4BY3/L6w5F59FtZPgPpId3FjFt9GiOmjQPcQK5E4kkX1F6YTWpN8i0CdmO4CqCUQZqkaYzbm/ZxpRLtrTpNccJdBV559xUadt0qCP6KxxAOSpSM+0f3mChmyDtvrma8ym9jcP5tjhNq2ljFaySjZ3jiTtxiYT4yPvqRjJHZMVfIUO5qrM85s0U5sHUfqClU= ceph-42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:16.004 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T23:28:16.126 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:15 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2621246078' entity='client.admin' 2024-09-11T23:28:16.254 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:16.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.764+0000 7fa63189b640 1 -- 172.21.15.133:0/2053235916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa62c102110 msgr2=0x7fa62c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.764+0000 7fa63189b640 1 --2- 172.21.15.133:0/2053235916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa62c102110 0x7fa62c104500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009880 tx=0x7fa61c02a5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.764+0000 7fa63189b640 1 -- 172.21.15.133:0/2053235916 shutdown_connections 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.764+0000 7fa63189b640 1 --2- 172.21.15.133:0/2053235916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa62c102110 0x7fa62c104500 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.764+0000 7fa63189b640 1 -- 172.21.15.133:0/2053235916 >> 172.21.15.133:0/2053235916 conn(0x7fa62c0fbcb0 msgr2=0x7fa62c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.764+0000 7fa63189b640 1 -- 172.21.15.133:0/2053235916 shutdown_connections 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.764+0000 7fa63189b640 1 -- 172.21.15.133:0/2053235916 wait complete. 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.765+0000 7fa63189b640 1 Processor -- start 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.765+0000 7fa63189b640 1 -- start start 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.765+0000 7fa63189b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa62c102110 0x7fa62c101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.765+0000 7fa63189b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c107720 con 0x7fa62c102110 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.765+0000 7fa62affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa62c102110 0x7fa62c101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:16.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.765+0000 7fa62affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa62c102110 0x7fa62c101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58026/0 (socket says 172.21.15.133:58026) 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.765+0000 7fa62affd640 1 -- 172.21.15.133:0/634669386 learned_addr learned my addr 172.21.15.133:0/634669386 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.765+0000 7fa62affd640 1 -- 172.21.15.133:0/634669386 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c101ed0 con 0x7fa62c102110 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.765+0000 7fa62affd640 1 --2- 172.21.15.133:0/634669386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa62c102110 0x7fa62c101990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa61c004f60 tx=0x7fa61c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.766+0000 7fa630899640 1 -- 172.21.15.133:0/634669386 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa61c03f070 con 0x7fa62c102110 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.766+0000 7fa630899640 1 -- 172.21.15.133:0/634669386 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa61c003ad0 con 0x7fa62c102110 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.766+0000 7fa63189b640 1 -- 172.21.15.133:0/634669386 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c1000c0 con 0x7fa62c102110 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.766+0000 7fa630899640 1 -- 172.21.15.133:0/634669386 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa61c035030 con 0x7fa62c102110 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.766+0000 7fa63189b640 1 -- 172.21.15.133:0/634669386 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c100600 con 0x7fa62c102110 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.766+0000 7fa630899640 1 -- 172.21.15.133:0/634669386 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fa61c042070 con 0x7fa62c102110 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.767+0000 7fa63189b640 1 -- 172.21.15.133:0/634669386 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62c107ea0 con 0x7fa62c102110 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.767+0000 7fa630899640 1 --2- 172.21.15.133:0/634669386 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa614039060 0x7fa61403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:16.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.767+0000 7fa630899640 1 -- 172.21.15.133:0/634669386 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa61c06cfe0 con 0x7fa62c102110 2024-09-11T23:28:16.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.767+0000 7fa62a7fc640 1 --2- 172.21.15.133:0/634669386 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa614039060 0x7fa61403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:16.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.767+0000 7fa62a7fc640 1 --2- 172.21.15.133:0/634669386 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa614039060 0x7fa61403b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6200098c0 tx=0x7fa620006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:16.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.771+0000 7fa630899640 1 -- 172.21.15.133:0/634669386 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa61c039350 con 0x7fa62c102110 2024-09-11T23:28:16.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.942+0000 7fa63189b640 1 -- 172.21.15.133:0/634669386 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fa62c1901e0 con 0x7fa62c102110 2024-09-11T23:28:16.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.945+0000 7fa630899640 1 -- 172.21.15.133:0/634669386 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fa61c02f300 con 0x7fa62c102110 2024-09-11T23:28:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.947+0000 7fa63189b640 1 -- 172.21.15.133:0/634669386 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa614039060 msgr2=0x7fa61403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.947+0000 7fa63189b640 1 --2- 172.21.15.133:0/634669386 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa614039060 0x7fa61403b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6200098c0 tx=0x7fa620006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.947+0000 7fa63189b640 1 -- 172.21.15.133:0/634669386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa62c102110 msgr2=0x7fa62c101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.947+0000 7fa63189b640 1 --2- 172.21.15.133:0/634669386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa62c102110 0x7fa62c101990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa61c004f60 tx=0x7fa61c0048f0 comp rx=0 tx=0).stop 2024-09-11T23:28:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.947+0000 7fa63189b640 1 -- 172.21.15.133:0/634669386 shutdown_connections 2024-09-11T23:28:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.947+0000 7fa63189b640 1 --2- 172.21.15.133:0/634669386 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa614039060 0x7fa61403b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.947+0000 7fa63189b640 1 --2- 172.21.15.133:0/634669386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa62c102110 0x7fa62c101990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.947+0000 7fa63189b640 1 -- 172.21.15.133:0/634669386 >> 172.21.15.133:0/634669386 conn(0x7fa62c0fbcb0 msgr2=0x7fa62c0fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.947+0000 7fa63189b640 1 -- 172.21.15.133:0/634669386 shutdown_connections 2024-09-11T23:28:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:16.947+0000 7fa63189b640 1 -- 172.21.15.133:0/634669386 wait complete. 2024-09-11T23:28:18.073 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T23:28:18.073 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T23:28:18.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:17 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/634669386' entity='client.admin' 2024-09-11T23:28:18.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:17 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:18.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:17 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:18.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:17 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:18.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:17 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:18.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:17 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:18.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:17 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:18.337 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:18.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.769+0000 7f195cad7640 1 -- 172.21.15.133:0/139982588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1958102160 msgr2=0x7f1958104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:18.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.769+0000 7f195cad7640 1 --2- 172.21.15.133:0/139982588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1958102160 0x7f1958104550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1948009880 tx=0x7f194802a5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:18.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.769+0000 7f195cad7640 1 -- 172.21.15.133:0/139982588 shutdown_connections 2024-09-11T23:28:18.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.769+0000 7f195cad7640 1 --2- 172.21.15.133:0/139982588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1958102160 0x7f1958104550 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:18.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.769+0000 7f195cad7640 1 -- 172.21.15.133:0/139982588 >> 172.21.15.133:0/139982588 conn(0x7f19580fbca0 msgr2=0x7f19580fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.769+0000 7f195cad7640 1 -- 172.21.15.133:0/139982588 shutdown_connections 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.769+0000 7f195cad7640 1 -- 172.21.15.133:0/139982588 wait complete. 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.770+0000 7f195cad7640 1 Processor -- start 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.770+0000 7f195cad7640 1 -- start start 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.770+0000 7f195cad7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1958102160 0x7f1958196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.770+0000 7f195cad7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1958107730 con 0x7f1958102160 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.770+0000 7f195659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1958102160 0x7f1958196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.770+0000 7f195659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1958102160 0x7f1958196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58044/0 (socket says 172.21.15.133:58044) 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.770+0000 7f195659c640 1 -- 172.21.15.133:0/2043075945 learned_addr learned my addr 172.21.15.133:0/2043075945 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.771+0000 7f195659c640 1 -- 172.21.15.133:0/2043075945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19581972f0 con 0x7f1958102160 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.771+0000 7f195659c640 1 --2- 172.21.15.133:0/2043075945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1958102160 0x7f1958196db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1948004f60 tx=0x7f19480048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.771+0000 7f19477fe640 1 -- 172.21.15.133:0/2043075945 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f194803f070 con 0x7f1958102160 2024-09-11T23:28:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.771+0000 7f19477fe640 1 -- 172.21.15.133:0/2043075945 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1948003ad0 con 0x7f1958102160 2024-09-11T23:28:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.771+0000 7f195cad7640 1 -- 172.21.15.133:0/2043075945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1958197580 con 0x7f1958102160 2024-09-11T23:28:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.771+0000 7f19477fe640 1 -- 172.21.15.133:0/2043075945 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1948035030 con 0x7f1958102160 2024-09-11T23:28:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.771+0000 7f195cad7640 1 -- 172.21.15.133:0/2043075945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195819a270 con 0x7f1958102160 2024-09-11T23:28:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.772+0000 7f19477fe640 1 -- 172.21.15.133:0/2043075945 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1948042070 con 0x7f1958102160 2024-09-11T23:28:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.772+0000 7f19477fe640 1 --2- 172.21.15.133:0/2043075945 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1940039060 0x7f194003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.772+0000 7f195cad7640 1 -- 172.21.15.133:0/2043075945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1958107eb0 con 0x7f1958102160 2024-09-11T23:28:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.772+0000 7f19477fe640 1 -- 172.21.15.133:0/2043075945 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f194806cef0 con 0x7f1958102160 2024-09-11T23:28:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.772+0000 7f1955d9b640 1 --2- 172.21.15.133:0/2043075945 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1940039060 0x7f194003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.773+0000 7f1955d9b640 1 --2- 172.21.15.133:0/2043075945 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1940039060 0x7f194003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f194c0098c0 tx=0x7f194c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.777+0000 7f19477fe640 1 -- 172.21.15.133:0/2043075945 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1948039350 con 0x7f1958102160 2024-09-11T23:28:18.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.951+0000 7f195cad7640 1 -- 172.21.15.133:0/2043075945 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f195802d830 con 0x7f1940039060 2024-09-11T23:28:18.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.954+0000 7f19477fe640 1 -- 172.21.15.133:0/2043075945 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f195802d830 con 0x7f1940039060 2024-09-11T23:28:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.957+0000 7f195cad7640 1 -- 172.21.15.133:0/2043075945 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1940039060 msgr2=0x7f194003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.957+0000 7f195cad7640 1 --2- 172.21.15.133:0/2043075945 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1940039060 0x7f194003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f194c0098c0 tx=0x7f194c006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.957+0000 7f195cad7640 1 -- 172.21.15.133:0/2043075945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1958102160 msgr2=0x7f1958196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.957+0000 7f195cad7640 1 --2- 172.21.15.133:0/2043075945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1958102160 0x7f1958196db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1948004f60 tx=0x7f19480048f0 comp rx=0 tx=0).stop 2024-09-11T23:28:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.957+0000 7f195cad7640 1 -- 172.21.15.133:0/2043075945 shutdown_connections 2024-09-11T23:28:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.957+0000 7f195cad7640 1 --2- 172.21.15.133:0/2043075945 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1940039060 0x7f194003b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.957+0000 7f195cad7640 1 --2- 172.21.15.133:0/2043075945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1958102160 0x7f1958196db0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:18.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.957+0000 7f195cad7640 1 -- 172.21.15.133:0/2043075945 >> 172.21.15.133:0/2043075945 conn(0x7f19580fbca0 msgr2=0x7f19580fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:18.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.957+0000 7f195cad7640 1 -- 172.21.15.133:0/2043075945 shutdown_connections 2024-09-11T23:28:18.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:18.957+0000 7f195cad7640 1 -- 172.21.15.133:0/2043075945 wait complete. 2024-09-11T23:28:19.785 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-09-11T23:28:19.785 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-11T23:28:19.785 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-09-11T23:28:19.805 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-11T23:28:19.805 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:28:19.863 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-09-11T23:28:19.864 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch host add smithi138 2024-09-11T23:28:20.103 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:20.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:19 smithi133 ceph-mon[31430]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:28:20.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:20.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:20.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:20.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:28:20.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:19 smithi133 ceph-mon[31430]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-11T23:28:20.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:20.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:20.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.556+0000 7fed5809f640 1 -- 172.21.15.133:0/1028859765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed50074590 msgr2=0x7fed50074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.556+0000 7fed5809f640 1 --2- 172.21.15.133:0/1028859765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed50074590 0x7fed50074970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009880 tx=0x7fed4c02a670 comp rx=0 tx=0).stop 2024-09-11T23:28:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.556+0000 7fed5809f640 1 -- 172.21.15.133:0/1028859765 shutdown_connections 2024-09-11T23:28:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.556+0000 7fed5809f640 1 --2- 172.21.15.133:0/1028859765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed50074590 0x7fed50074970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.556+0000 7fed5809f640 1 -- 172.21.15.133:0/1028859765 >> 172.21.15.133:0/1028859765 conn(0x7fed500fd2b0 msgr2=0x7fed500ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.556+0000 7fed5809f640 1 -- 172.21.15.133:0/1028859765 shutdown_connections 2024-09-11T23:28:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.556+0000 7fed5809f640 1 -- 172.21.15.133:0/1028859765 wait complete. 2024-09-11T23:28:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.557+0000 7fed5809f640 1 Processor -- start 2024-09-11T23:28:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.557+0000 7fed5809f640 1 -- start start 2024-09-11T23:28:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.557+0000 7fed5809f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed50074590 0x7fed501a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:20.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.557+0000 7fed5809f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed501122f0 con 0x7fed50074590 2024-09-11T23:28:20.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.557+0000 7fed55e3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed50074590 0x7fed501a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:20.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.557+0000 7fed55e3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed50074590 0x7fed501a33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58076/0 (socket says 172.21.15.133:58076) 2024-09-11T23:28:20.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.557+0000 7fed55e3b640 1 -- 172.21.15.133:0/4111728165 learned_addr learned my addr 172.21.15.133:0/4111728165 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:20.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.558+0000 7fed55e3b640 1 -- 172.21.15.133:0/4111728165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed501a3920 con 0x7fed50074590 2024-09-11T23:28:20.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.558+0000 7fed55e3b640 1 --2- 172.21.15.133:0/4111728165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed50074590 0x7fed501a33e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009230 tx=0x7fed4c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.558+0000 7fed46ffd640 1 -- 172.21.15.133:0/4111728165 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed4c03f070 con 0x7fed50074590 2024-09-11T23:28:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.558+0000 7fed46ffd640 1 -- 172.21.15.133:0/4111728165 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed4c03bc80 con 0x7fed50074590 2024-09-11T23:28:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.558+0000 7fed46ffd640 1 -- 172.21.15.133:0/4111728165 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed4c035030 con 0x7fed50074590 2024-09-11T23:28:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.558+0000 7fed5809f640 1 -- 172.21.15.133:0/4111728165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed501a3bb0 con 0x7fed50074590 2024-09-11T23:28:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.559+0000 7fed5809f640 1 -- 172.21.15.133:0/4111728165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed501a3f50 con 0x7fed50074590 2024-09-11T23:28:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.559+0000 7fed46ffd640 1 -- 172.21.15.133:0/4111728165 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fed4c042070 con 0x7fed50074590 2024-09-11T23:28:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.560+0000 7fed46ffd640 1 --2- 172.21.15.133:0/4111728165 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fed3c0390c0 0x7fed3c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:20.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.560+0000 7fed5809f640 1 -- 172.21.15.133:0/4111728165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed50075cd0 con 0x7fed50074590 2024-09-11T23:28:20.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.560+0000 7fed46ffd640 1 -- 172.21.15.133:0/4111728165 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fed4c06cc90 con 0x7fed50074590 2024-09-11T23:28:20.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.560+0000 7fed5563a640 1 --2- 172.21.15.133:0/4111728165 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fed3c0390c0 0x7fed3c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:20.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.560+0000 7fed5563a640 1 --2- 172.21.15.133:0/4111728165 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fed3c0390c0 0x7fed3c03b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fed400098c0 tx=0x7fed40006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:20.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.564+0000 7fed46ffd640 1 -- 172.21.15.133:0/4111728165 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed4c0314c0 con 0x7fed50074590 2024-09-11T23:28:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:20.732+0000 7fed5809f640 1 -- 172.21.15.133:0/4111728165 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}) -- 0x7fed5019eb70 con 0x7fed3c0390c0 2024-09-11T23:28:21.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:20 smithi133 ceph-mon[31430]: Updating smithi133:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:28:21.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:20 smithi133 ceph-mon[31430]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:28:21.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:20 smithi133 ceph-mon[31430]: Updating smithi133:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:28:22.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:21 smithi133 ceph-mon[31430]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:28:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:22 smithi133 ceph-mon[31430]: Deploying cephadm binary to smithi138 2024-09-11T23:28:23.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:23.959+0000 7fed46ffd640 1 -- 172.21.15.133:0/4111728165 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fed5019eb70 con 0x7fed3c0390c0 2024-09-11T23:28:23.960 INFO:teuthology.orchestra.run.smithi133.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-09-11T23:28:23.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:23.962+0000 7fed5809f640 1 -- 172.21.15.133:0/4111728165 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fed3c0390c0 msgr2=0x7fed3c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:23.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:23.962+0000 7fed5809f640 1 --2- 172.21.15.133:0/4111728165 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fed3c0390c0 0x7fed3c03b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fed400098c0 tx=0x7fed40006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:23.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:23.962+0000 7fed5809f640 1 -- 172.21.15.133:0/4111728165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed50074590 msgr2=0x7fed501a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:23.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:23.962+0000 7fed5809f640 1 --2- 172.21.15.133:0/4111728165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed50074590 0x7fed501a33e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009230 tx=0x7fed4c003d40 comp rx=0 tx=0).stop 2024-09-11T23:28:23.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:23.962+0000 7fed5809f640 1 -- 172.21.15.133:0/4111728165 shutdown_connections 2024-09-11T23:28:23.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:23.962+0000 7fed5809f640 1 --2- 172.21.15.133:0/4111728165 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fed3c0390c0 0x7fed3c03b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:23.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:23.962+0000 7fed5809f640 1 --2- 172.21.15.133:0/4111728165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed50074590 0x7fed501a33e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:23.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:23.962+0000 7fed5809f640 1 -- 172.21.15.133:0/4111728165 >> 172.21.15.133:0/4111728165 conn(0x7fed500fd2b0 msgr2=0x7fed500fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:23.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:23.962+0000 7fed5809f640 1 -- 172.21.15.133:0/4111728165 shutdown_connections 2024-09-11T23:28:23.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:23.962+0000 7fed5809f640 1 -- 172.21.15.133:0/4111728165 wait complete. 2024-09-11T23:28:24.598 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T23:28:24.838 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:24 smithi133 ceph-mon[31430]: Added host smithi138 2024-09-11T23:28:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:25.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.313+0000 7fc5b9daa640 1 -- 172.21.15.133:0/4012832100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5b4107690 msgr2=0x7fc5b4107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.313+0000 7fc5b9daa640 1 --2- 172.21.15.133:0/4012832100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5b4107690 0x7fc5b4107a70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009c20 tx=0x7fc5a402aa90 comp rx=0 tx=0).stop 2024-09-11T23:28:25.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.314+0000 7fc5b9daa640 1 -- 172.21.15.133:0/4012832100 shutdown_connections 2024-09-11T23:28:25.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.314+0000 7fc5b9daa640 1 --2- 172.21.15.133:0/4012832100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5b4107690 0x7fc5b4107a70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.314+0000 7fc5b9daa640 1 -- 172.21.15.133:0/4012832100 >> 172.21.15.133:0/4012832100 conn(0x7fc5b4076250 msgr2=0x7fc5b4076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:25.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.314+0000 7fc5b9daa640 1 -- 172.21.15.133:0/4012832100 shutdown_connections 2024-09-11T23:28:25.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.314+0000 7fc5b9daa640 1 -- 172.21.15.133:0/4012832100 wait complete. 2024-09-11T23:28:25.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.314+0000 7fc5b9daa640 1 Processor -- start 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.314+0000 7fc5b9daa640 1 -- start start 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b9daa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5b4107690 0x7fc5b4197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b9daa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b410bce0 con 0x7fc5b4107690 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5b4107690 0x7fc5b4197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5b4107690 0x7fc5b4197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56394/0 (socket says 172.21.15.133:56394) 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b37fe640 1 -- 172.21.15.133:0/3319654405 learned_addr learned my addr 172.21.15.133:0/3319654405 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b37fe640 1 -- 172.21.15.133:0/3319654405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b4197650 con 0x7fc5b4107690 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b37fe640 1 --2- 172.21.15.133:0/3319654405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5b4107690 0x7fc5b4197110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005e30 tx=0x7fc5a402fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b0ff9640 1 -- 172.21.15.133:0/3319654405 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a4047070 con 0x7fc5b4107690 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b0ff9640 1 -- 172.21.15.133:0/3319654405 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a4003df0 con 0x7fc5b4107690 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b9daa640 1 -- 172.21.15.133:0/3319654405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b41978e0 con 0x7fc5b4107690 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b0ff9640 1 -- 172.21.15.133:0/3319654405 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a4033050 con 0x7fc5b4107690 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.315+0000 7fc5b9daa640 1 -- 172.21.15.133:0/3319654405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b419b9e0 con 0x7fc5b4107690 2024-09-11T23:28:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.316+0000 7fc5b0ff9640 1 -- 172.21.15.133:0/3319654405 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc5a404a070 con 0x7fc5b4107690 2024-09-11T23:28:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.316+0000 7fc5b0ff9640 1 --2- 172.21.15.133:0/3319654405 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5a0039060 0x7fc5a003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.316+0000 7fc5b0ff9640 1 -- 172.21.15.133:0/3319654405 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc5a403e030 con 0x7fc5b4107690 2024-09-11T23:28:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.316+0000 7fc5b2ffd640 1 --2- 172.21.15.133:0/3319654405 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5a0039060 0x7fc5a003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.316+0000 7fc5b9daa640 1 -- 172.21.15.133:0/3319654405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b4102d30 con 0x7fc5b4107690 2024-09-11T23:28:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.317+0000 7fc5b2ffd640 1 --2- 172.21.15.133:0/3319654405 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5a0039060 0x7fc5a003b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0098c0 tx=0x7fc59c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.321+0000 7fc5b0ff9640 1 -- 172.21.15.133:0/3319654405 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5a4051080 con 0x7fc5b4107690 2024-09-11T23:28:25.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.492+0000 7fc5b9daa640 1 -- 172.21.15.133:0/3319654405 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5b4107af0 con 0x7fc5a0039060 2024-09-11T23:28:25.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.494+0000 7fc5b0ff9640 1 -- 172.21.15.133:0/3319654405 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fc5b4107af0 con 0x7fc5a0039060 2024-09-11T23:28:25.494 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:25.494 INFO:teuthology.orchestra.run.smithi133.stdout:[{"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-09-11T23:28:25.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.496+0000 7fc5b9daa640 1 -- 172.21.15.133:0/3319654405 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5a0039060 msgr2=0x7fc5a003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.496+0000 7fc5b9daa640 1 --2- 172.21.15.133:0/3319654405 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5a0039060 0x7fc5a003b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0098c0 tx=0x7fc59c006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.496+0000 7fc5b9daa640 1 -- 172.21.15.133:0/3319654405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5b4107690 msgr2=0x7fc5b4197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.496+0000 7fc5b9daa640 1 --2- 172.21.15.133:0/3319654405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5b4107690 0x7fc5b4197110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005e30 tx=0x7fc5a402fc10 comp rx=0 tx=0).stop 2024-09-11T23:28:25.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.496+0000 7fc5b9daa640 1 -- 172.21.15.133:0/3319654405 shutdown_connections 2024-09-11T23:28:25.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.496+0000 7fc5b9daa640 1 --2- 172.21.15.133:0/3319654405 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5a0039060 0x7fc5a003b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.496+0000 7fc5b9daa640 1 --2- 172.21.15.133:0/3319654405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5b4107690 0x7fc5b4197110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.496+0000 7fc5b9daa640 1 -- 172.21.15.133:0/3319654405 >> 172.21.15.133:0/3319654405 conn(0x7fc5b4076250 msgr2=0x7fc5b40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:25.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.496+0000 7fc5b9daa640 1 -- 172.21.15.133:0/3319654405 shutdown_connections 2024-09-11T23:28:25.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:25.496+0000 7fc5b9daa640 1 -- 172.21.15.133:0/3319654405 wait complete. 2024-09-11T23:28:26.194 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi178 2024-09-11T23:28:26.194 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-11T23:28:26.194 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.conf 2024-09-11T23:28:26.214 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-11T23:28:26.214 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:28:26.272 INFO:tasks.cephadm:Adding host smithi178 to orchestrator... 2024-09-11T23:28:26.272 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch host add smithi178 2024-09-11T23:28:26.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:26 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:26.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:26 smithi133 ceph-mon[31430]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:28:26.508 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.919+0000 7f06828ec640 1 -- 172.21.15.133:0/333895859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f067c107660 msgr2=0x7f067c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.919+0000 7f06828ec640 1 --2- 172.21.15.133:0/333895859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f067c107660 0x7f067c107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f066c009880 tx=0x7f066c02a5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.919+0000 7f06828ec640 1 -- 172.21.15.133:0/333895859 shutdown_connections 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.919+0000 7f06828ec640 1 --2- 172.21.15.133:0/333895859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f067c107660 0x7f067c107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.919+0000 7f06828ec640 1 -- 172.21.15.133:0/333895859 >> 172.21.15.133:0/333895859 conn(0x7f067c0fd2b0 msgr2=0x7f067c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.919+0000 7f06828ec640 1 -- 172.21.15.133:0/333895859 shutdown_connections 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.919+0000 7f06828ec640 1 -- 172.21.15.133:0/333895859 wait complete. 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.920+0000 7f06828ec640 1 Processor -- start 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.920+0000 7f06828ec640 1 -- start start 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.920+0000 7f06828ec640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f067c107660 0x7f067c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.920+0000 7f06828ec640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f067c10bc70 con 0x7f067c107660 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.920+0000 7f067bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f067c107660 0x7f067c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.920+0000 7f067bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f067c107660 0x7f067c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56400/0 (socket says 172.21.15.133:56400) 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.920+0000 7f067bfff640 1 -- 172.21.15.133:0/1257674769 learned_addr learned my addr 172.21.15.133:0/1257674769 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.921+0000 7f067bfff640 1 -- 172.21.15.133:0/1257674769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c0763e0 con 0x7f067c107660 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.921+0000 7f067bfff640 1 --2- 172.21.15.133:0/1257674769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f067c107660 0x7f067c0744f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f066c009230 tx=0x7f066c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.921+0000 7f06797fa640 1 -- 172.21.15.133:0/1257674769 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f066c03f070 con 0x7f067c107660 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.921+0000 7f06797fa640 1 -- 172.21.15.133:0/1257674769 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f066c03bc80 con 0x7f067c107660 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.921+0000 7f06828ec640 1 -- 172.21.15.133:0/1257674769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c074a90 con 0x7f067c107660 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.921+0000 7f06797fa640 1 -- 172.21.15.133:0/1257674769 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f066c035030 con 0x7f067c107660 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.921+0000 7f06828ec640 1 -- 172.21.15.133:0/1257674769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c074f50 con 0x7f067c107660 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.922+0000 7f06797fa640 1 -- 172.21.15.133:0/1257674769 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f066c042070 con 0x7f067c107660 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.922+0000 7f06828ec640 1 -- 172.21.15.133:0/1257674769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067c102d10 con 0x7f067c107660 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.922+0000 7f06797fa640 1 --2- 172.21.15.133:0/1257674769 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0664039060 0x7f066403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.922+0000 7f06797fa640 1 -- 172.21.15.133:0/1257674769 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f066c06cc60 con 0x7f067c107660 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.922+0000 7f067b7fe640 1 --2- 172.21.15.133:0/1257674769 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0664039060 0x7f066403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.923+0000 7f067b7fe640 1 --2- 172.21.15.133:0/1257674769 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0664039060 0x7f066403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06700098c0 tx=0x7f0670006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:26.927+0000 7f06797fa640 1 -- 172.21.15.133:0/1257674769 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f066c04c350 con 0x7f067c107660 2024-09-11T23:28:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:27.097+0000 7f06828ec640 1 -- 172.21.15.133:0/1257674769 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}) -- 0x7f067c075a90 con 0x7f0664039060 2024-09-11T23:28:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:28 smithi133 ceph-mon[31430]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:28:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:28 smithi133 ceph-mon[31430]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:29.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:28 smithi133 ceph-mon[31430]: Deploying cephadm binary to smithi178 2024-09-11T23:28:29.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:29.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:29.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:29.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:29.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:29.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:29.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:28:30.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:29 smithi133 ceph-mon[31430]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-11T23:28:30.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:29 smithi133 ceph-mon[31430]: Updating smithi138:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:28:30.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:29 smithi133 ceph-mon[31430]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:28:30.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:29 smithi133 ceph-mon[31430]: Updating smithi138:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:28:30.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:29 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:30.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:29 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:30.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:29 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:30.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:30.281+0000 7f06797fa640 1 -- 172.21.15.133:0/1257674769 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f067c075a90 con 0x7f0664039060 2024-09-11T23:28:30.281 INFO:teuthology.orchestra.run.smithi133.stdout:Added host 'smithi178' with addr '172.21.15.178' 2024-09-11T23:28:30.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:30.284+0000 7f06828ec640 1 -- 172.21.15.133:0/1257674769 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0664039060 msgr2=0x7f066403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:30.284+0000 7f06828ec640 1 --2- 172.21.15.133:0/1257674769 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0664039060 0x7f066403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06700098c0 tx=0x7f0670006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:30.284+0000 7f06828ec640 1 -- 172.21.15.133:0/1257674769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f067c107660 msgr2=0x7f067c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:30.284+0000 7f06828ec640 1 --2- 172.21.15.133:0/1257674769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f067c107660 0x7f067c0744f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f066c009230 tx=0x7f066c003d40 comp rx=0 tx=0).stop 2024-09-11T23:28:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:30.284+0000 7f06828ec640 1 -- 172.21.15.133:0/1257674769 shutdown_connections 2024-09-11T23:28:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:30.284+0000 7f06828ec640 1 --2- 172.21.15.133:0/1257674769 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0664039060 0x7f066403b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:30.284+0000 7f06828ec640 1 --2- 172.21.15.133:0/1257674769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f067c107660 0x7f067c0744f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:30.284+0000 7f06828ec640 1 -- 172.21.15.133:0/1257674769 >> 172.21.15.133:0/1257674769 conn(0x7f067c0fd2b0 msgr2=0x7f067c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:30.284+0000 7f06828ec640 1 -- 172.21.15.133:0/1257674769 shutdown_connections 2024-09-11T23:28:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:30.284+0000 7f06828ec640 1 -- 172.21.15.133:0/1257674769 wait complete. 2024-09-11T23:28:30.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:30 smithi133 ceph-mon[31430]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:30.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:30.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:30.865 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:30.895 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T23:28:31.134 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.564+0000 7f259e7e8640 1 -- 172.21.15.133:0/1987576875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2598100990 msgr2=0x7f2598100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.564+0000 7f259e7e8640 1 --2- 172.21.15.133:0/1987576875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2598100990 0x7f2598100d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2588009880 tx=0x7f258802a5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.564+0000 7f259e7e8640 1 -- 172.21.15.133:0/1987576875 shutdown_connections 2024-09-11T23:28:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.565+0000 7f259e7e8640 1 --2- 172.21.15.133:0/1987576875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2598100990 0x7f2598100d70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.565+0000 7f259e7e8640 1 -- 172.21.15.133:0/1987576875 >> 172.21.15.133:0/1987576875 conn(0x7f2598076250 msgr2=0x7f2598076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.565+0000 7f259e7e8640 1 -- 172.21.15.133:0/1987576875 shutdown_connections 2024-09-11T23:28:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.565+0000 7f259e7e8640 1 -- 172.21.15.133:0/1987576875 wait complete. 2024-09-11T23:28:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.565+0000 7f259e7e8640 1 Processor -- start 2024-09-11T23:28:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.565+0000 7f259e7e8640 1 -- start start 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.565+0000 7f259e7e8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2598100990 0x7f25981970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.565+0000 7f259e7e8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f259810ab90 con 0x7f2598100990 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.566+0000 7f2597fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2598100990 0x7f25981970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.566+0000 7f2597fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2598100990 0x7f25981970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56414/0 (socket says 172.21.15.133:56414) 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.566+0000 7f2597fff640 1 -- 172.21.15.133:0/1096054212 learned_addr learned my addr 172.21.15.133:0/1096054212 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.566+0000 7f2597fff640 1 -- 172.21.15.133:0/1096054212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2598197620 con 0x7f2598100990 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.566+0000 7f2597fff640 1 --2- 172.21.15.133:0/1096054212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2598100990 0x7f25981970e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2588009230 tx=0x7f2588003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.566+0000 7f25957fa640 1 -- 172.21.15.133:0/1096054212 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f258803f070 con 0x7f2598100990 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.566+0000 7f25957fa640 1 -- 172.21.15.133:0/1096054212 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258803bc80 con 0x7f2598100990 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.566+0000 7f259e7e8640 1 -- 172.21.15.133:0/1096054212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2598197910 con 0x7f2598100990 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.566+0000 7f25957fa640 1 -- 172.21.15.133:0/1096054212 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2588035030 con 0x7f2598100990 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.566+0000 7f259e7e8640 1 -- 172.21.15.133:0/1096054212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259819b9b0 con 0x7f2598100990 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.567+0000 7f25957fa640 1 -- 172.21.15.133:0/1096054212 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f2588042070 con 0x7f2598100990 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.567+0000 7f25957fa640 1 --2- 172.21.15.133:0/1096054212 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2580039060 0x7f258003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:31.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.567+0000 7f25957fa640 1 -- 172.21.15.133:0/1096054212 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f258806cf10 con 0x7f2598100990 2024-09-11T23:28:31.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.567+0000 7f259e7e8640 1 -- 172.21.15.133:0/1096054212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2598101c40 con 0x7f2598100990 2024-09-11T23:28:31.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.567+0000 7f25977fe640 1 --2- 172.21.15.133:0/1096054212 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2580039060 0x7f258003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:31.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.568+0000 7f25977fe640 1 --2- 172.21.15.133:0/1096054212 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2580039060 0x7f258003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f258c0098c0 tx=0x7f258c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:31.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.572+0000 7f25957fa640 1 -- 172.21.15.133:0/1096054212 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f258804c350 con 0x7f2598100990 2024-09-11T23:28:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.744+0000 7f259e7e8640 1 -- 172.21.15.133:0/1096054212 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2598198440 con 0x7f2580039060 2024-09-11T23:28:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.745+0000 7f25957fa640 1 -- 172.21.15.133:0/1096054212 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f2598198440 con 0x7f2580039060 2024-09-11T23:28:31.744 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:28:31.744 INFO:teuthology.orchestra.run.smithi133.stdout:[{"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}] 2024-09-11T23:28:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.747+0000 7f259e7e8640 1 -- 172.21.15.133:0/1096054212 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2580039060 msgr2=0x7f258003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.747+0000 7f259e7e8640 1 --2- 172.21.15.133:0/1096054212 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2580039060 0x7f258003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f258c0098c0 tx=0x7f258c006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.747+0000 7f259e7e8640 1 -- 172.21.15.133:0/1096054212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2598100990 msgr2=0x7f25981970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.747+0000 7f259e7e8640 1 --2- 172.21.15.133:0/1096054212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2598100990 0x7f25981970e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2588009230 tx=0x7f2588003d40 comp rx=0 tx=0).stop 2024-09-11T23:28:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.747+0000 7f259e7e8640 1 -- 172.21.15.133:0/1096054212 shutdown_connections 2024-09-11T23:28:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.747+0000 7f259e7e8640 1 --2- 172.21.15.133:0/1096054212 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2580039060 0x7f258003b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.747+0000 7f259e7e8640 1 --2- 172.21.15.133:0/1096054212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2598100990 0x7f25981970e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.747+0000 7f259e7e8640 1 -- 172.21.15.133:0/1096054212 >> 172.21.15.133:0/1096054212 conn(0x7f2598076250 msgr2=0x7f25980696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.747+0000 7f259e7e8640 1 -- 172.21.15.133:0/1096054212 shutdown_connections 2024-09-11T23:28:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:31.747+0000 7f259e7e8640 1 -- 172.21.15.133:0/1096054212 wait complete. 2024-09-11T23:28:32.080 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:31 smithi133 ceph-mon[31430]: Added host smithi178 2024-09-11T23:28:32.080 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:31 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:32.080 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:31 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:32.409 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T23:28:32.410 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T23:28:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:32.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:32 smithi133 ceph-mon[31430]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:28:32.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:32 smithi133 ceph-mon[31430]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.108+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3997956333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c8106590 msgr2=0x7ff5c8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.108+0000 7ff5cd92b640 1 --2- 172.21.15.133:0/3997956333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c8106590 0x7ff5c8106970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009880 tx=0x7ff5b802a5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.108+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3997956333 shutdown_connections 2024-09-11T23:28:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.108+0000 7ff5cd92b640 1 --2- 172.21.15.133:0/3997956333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c8106590 0x7ff5c8106970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.108+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3997956333 >> 172.21.15.133:0/3997956333 conn(0x7ff5c80fc1a0 msgr2=0x7ff5c80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.108+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3997956333 shutdown_connections 2024-09-11T23:28:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.109+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3997956333 wait complete. 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.109+0000 7ff5cd92b640 1 Processor -- start 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.109+0000 7ff5cd92b640 1 -- start start 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.109+0000 7ff5cd92b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c8106590 0x7ff5c8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.109+0000 7ff5cd92b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c810abc0 con 0x7ff5c8106590 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.109+0000 7ff5c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c8106590 0x7ff5c8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.109+0000 7ff5c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c8106590 0x7ff5c8197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37548/0 (socket says 172.21.15.133:37548) 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.110+0000 7ff5c6ffd640 1 -- 172.21.15.133:0/3831236927 learned_addr learned my addr 172.21.15.133:0/3831236927 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.110+0000 7ff5c6ffd640 1 -- 172.21.15.133:0/3831236927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c81984a0 con 0x7ff5c8106590 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.110+0000 7ff5c6ffd640 1 --2- 172.21.15.133:0/3831236927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c8106590 0x7ff5c8197f60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009230 tx=0x7ff5b8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.110+0000 7ff5cc929640 1 -- 172.21.15.133:0/3831236927 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5b803f070 con 0x7ff5c8106590 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.110+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3831236927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c8198730 con 0x7ff5c8106590 2024-09-11T23:28:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.110+0000 7ff5cc929640 1 -- 172.21.15.133:0/3831236927 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b803bc80 con 0x7ff5c8106590 2024-09-11T23:28:33.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.110+0000 7ff5cc929640 1 -- 172.21.15.133:0/3831236927 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5b8035030 con 0x7ff5c8106590 2024-09-11T23:28:33.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.110+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3831236927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c8198b90 con 0x7ff5c8106590 2024-09-11T23:28:33.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.111+0000 7ff5cc929640 1 -- 172.21.15.133:0/3831236927 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ff5b8042070 con 0x7ff5c8106590 2024-09-11T23:28:33.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.111+0000 7ff5cc929640 1 --2- 172.21.15.133:0/3831236927 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ff5b4039060 0x7ff5b403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.111+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3831236927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c8101c00 con 0x7ff5c8106590 2024-09-11T23:28:33.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.111+0000 7ff5cc929640 1 -- 172.21.15.133:0/3831236927 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff5b806cf80 con 0x7ff5c8106590 2024-09-11T23:28:33.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.112+0000 7ff5bffff640 1 --2- 172.21.15.133:0/3831236927 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ff5b4039060 0x7ff5b403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.112+0000 7ff5bffff640 1 --2- 172.21.15.133:0/3831236927 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ff5b4039060 0x7ff5b403b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00098c0 tx=0x7ff5b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:33.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.116+0000 7ff5cc929640 1 -- 172.21.15.133:0/3831236927 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5b804c350 con 0x7ff5c8106590 2024-09-11T23:28:33.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.293+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3831236927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff5c81069f0 con 0x7ff5c8106590 2024-09-11T23:28:33.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.858+0000 7ff5cc929640 1 -- 172.21.15.133:0/3831236927 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7ff5b8031e20 con 0x7ff5c8106590 2024-09-11T23:28:33.858 INFO:teuthology.orchestra.run.smithi133.stderr:adjusted tunables profile to default 2024-09-11T23:28:33.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.860+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3831236927 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ff5b4039060 msgr2=0x7ff5b403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.860+0000 7ff5cd92b640 1 --2- 172.21.15.133:0/3831236927 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ff5b4039060 0x7ff5b403b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00098c0 tx=0x7ff5b0006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.860+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3831236927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c8106590 msgr2=0x7ff5c8197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.860+0000 7ff5cd92b640 1 --2- 172.21.15.133:0/3831236927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c8106590 0x7ff5c8197f60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009230 tx=0x7ff5b8003d40 comp rx=0 tx=0).stop 2024-09-11T23:28:33.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.860+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3831236927 shutdown_connections 2024-09-11T23:28:33.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.860+0000 7ff5cd92b640 1 --2- 172.21.15.133:0/3831236927 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ff5b4039060 0x7ff5b403b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.860+0000 7ff5cd92b640 1 --2- 172.21.15.133:0/3831236927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c8106590 0x7ff5c8197f60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.860+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3831236927 >> 172.21.15.133:0/3831236927 conn(0x7ff5c80fc1a0 msgr2=0x7ff5c80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:33.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.860+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3831236927 shutdown_connections 2024-09-11T23:28:33.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:33.860+0000 7ff5cd92b640 1 -- 172.21.15.133:0/3831236927 wait complete. 2024-09-11T23:28:33.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:33 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3831236927' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T23:28:34.541 INFO:tasks.cephadm:Adding mon.a on smithi133 2024-09-11T23:28:34.541 INFO:tasks.cephadm:Adding mon.b on smithi138 2024-09-11T23:28:34.541 INFO:tasks.cephadm:Adding mon.c on smithi178 2024-09-11T23:28:34.541 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi133:172.21.15.133=a;smithi138:172.21.15.138=b;smithi178:172.21.15.178=c' 2024-09-11T23:28:34.793 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T23:28:34.839 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T23:28:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:34 smithi133 ceph-mon[31430]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:34 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3831236927' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T23:28:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:34 smithi133 ceph-mon[31430]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:28:35.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:35.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:35.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:35.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:35.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:28:35.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:35.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:28:35.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.304+0000 7f07285df640 1 -- 172.21.15.178:0/527807439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07201020b0 msgr2=0x7f07201044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.304+0000 7f07285df640 1 --2- 172.21.15.178:0/527807439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07201020b0 0x7f07201044a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f071c009880 tx=0x7f071c02a5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f07285df640 1 -- 172.21.15.178:0/527807439 shutdown_connections 2024-09-11T23:28:35.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f07285df640 1 --2- 172.21.15.178:0/527807439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07201020b0 0x7f07201044a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f07285df640 1 -- 172.21.15.178:0/527807439 >> 172.21.15.178:0/527807439 conn(0x7f07200fbcb0 msgr2=0x7f07200fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:35.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f07285df640 1 -- 172.21.15.178:0/527807439 shutdown_connections 2024-09-11T23:28:35.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f07285df640 1 -- 172.21.15.178:0/527807439 wait complete. 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f07285df640 1 Processor -- start 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f07285df640 1 -- start start 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f07285df640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07201020b0 0x7f0720196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f07285df640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f07201076c0 con 0x7f07201020b0 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f072637b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07201020b0 0x7f0720196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f072637b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07201020b0 0x7f0720196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.178:60066/0 (socket says 172.21.15.178:60066) 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.305+0000 7f072637b640 1 -- 172.21.15.178:0/2677751105 learned_addr learned my addr 172.21.15.178:0/2677751105 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.306+0000 7f072637b640 1 -- 172.21.15.178:0/2677751105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0720197230 con 0x7f07201020b0 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.306+0000 7f072637b640 1 --2- 172.21.15.178:0/2677751105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07201020b0 0x7f0720196cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f071c004f60 tx=0x7f071c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.306+0000 7f07177fe640 1 -- 172.21.15.178:0/2677751105 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f071c03f070 con 0x7f07201020b0 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.306+0000 7f07285df640 1 -- 172.21.15.178:0/2677751105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0720197520 con 0x7f07201020b0 2024-09-11T23:28:35.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.306+0000 7f07177fe640 1 -- 172.21.15.178:0/2677751105 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f071c003ad0 con 0x7f07201020b0 2024-09-11T23:28:35.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.306+0000 7f07177fe640 1 -- 172.21.15.178:0/2677751105 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f071c035030 con 0x7f07201020b0 2024-09-11T23:28:35.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.306+0000 7f07285df640 1 -- 172.21.15.178:0/2677751105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072019a1a0 con 0x7f07201020b0 2024-09-11T23:28:35.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.307+0000 7f07177fe640 1 -- 172.21.15.178:0/2677751105 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f071c042070 con 0x7f07201020b0 2024-09-11T23:28:35.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.307+0000 7f07177fe640 1 --2- 172.21.15.178:0/2677751105 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f070c039060 0x7f070c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.307+0000 7f07177fe640 1 -- 172.21.15.178:0/2677751105 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f071c06ce00 con 0x7f07201020b0 2024-09-11T23:28:35.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.307+0000 7f07285df640 1 -- 172.21.15.178:0/2677751105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0720107e40 con 0x7f07201020b0 2024-09-11T23:28:35.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.307+0000 7f0725b7a640 1 --2- 172.21.15.178:0/2677751105 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f070c039060 0x7f070c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.308+0000 7f0725b7a640 1 --2- 172.21.15.178:0/2677751105 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f070c039060 0x7f070c03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f07100098c0 tx=0x7f0710006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:35.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.312+0000 7f07177fe640 1 -- 172.21.15.178:0/2677751105 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f071c04c350 con 0x7f07201020b0 2024-09-11T23:28:35.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.499+0000 7f07285df640 1 -- 172.21.15.178:0/2677751105 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi138:172.21.15.138=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}) -- 0x7f072002d8f0 con 0x7f070c039060 2024-09-11T23:28:35.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.504+0000 7f07177fe640 1 -- 172.21.15.178:0/2677751105 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f072002d8f0 con 0x7f070c039060 2024-09-11T23:28:35.504 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mon update... 2024-09-11T23:28:35.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.506+0000 7f07285df640 1 -- 172.21.15.178:0/2677751105 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f070c039060 msgr2=0x7f070c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.506+0000 7f07285df640 1 --2- 172.21.15.178:0/2677751105 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f070c039060 0x7f070c03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f07100098c0 tx=0x7f0710006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.506+0000 7f07285df640 1 -- 172.21.15.178:0/2677751105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07201020b0 msgr2=0x7f0720196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.506+0000 7f07285df640 1 --2- 172.21.15.178:0/2677751105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07201020b0 0x7f0720196cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f071c004f60 tx=0x7f071c0048f0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.506+0000 7f07285df640 1 -- 172.21.15.178:0/2677751105 shutdown_connections 2024-09-11T23:28:35.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.506+0000 7f07285df640 1 --2- 172.21.15.178:0/2677751105 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f070c039060 0x7f070c03b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.506+0000 7f07285df640 1 --2- 172.21.15.178:0/2677751105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07201020b0 0x7f0720196cf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.506+0000 7f07285df640 1 -- 172.21.15.178:0/2677751105 >> 172.21.15.178:0/2677751105 conn(0x7f07200fbcb0 msgr2=0x7f07200fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:35.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.506+0000 7f07285df640 1 -- 172.21.15.178:0/2677751105 shutdown_connections 2024-09-11T23:28:35.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:35.507+0000 7f07285df640 1 -- 172.21.15.178:0/2677751105 wait complete. 2024-09-11T23:28:36.029 DEBUG:teuthology.orchestra.run.smithi138:mon.b> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T23:28:36.032 DEBUG:teuthology.orchestra.run.smithi178:mon.c> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T23:28:36.034 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:28:36.034 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:28:36.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-11T23:28:36.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: Updating smithi178:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:28:36.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:36.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:36.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:36.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:36.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:36.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:36.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:36.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:36.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:28:36.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:28:36.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:36.314 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:28:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.706+0000 7f12dc61b640 1 -- 172.21.15.178:0/3015321139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d4100990 msgr2=0x7f12d4100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.706+0000 7f12dc61b640 1 --2- 172.21.15.178:0/3015321139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d4100990 0x7f12d4100d70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009880 tx=0x7f12d002a5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.706+0000 7f12dc61b640 1 -- 172.21.15.178:0/3015321139 shutdown_connections 2024-09-11T23:28:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.706+0000 7f12dc61b640 1 --2- 172.21.15.178:0/3015321139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d4100990 0x7f12d4100d70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.706+0000 7f12dc61b640 1 -- 172.21.15.178:0/3015321139 >> 172.21.15.178:0/3015321139 conn(0x7f12d4076250 msgr2=0x7f12d4076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.706+0000 7f12dc61b640 1 -- 172.21.15.178:0/3015321139 shutdown_connections 2024-09-11T23:28:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.706+0000 7f12dc61b640 1 -- 172.21.15.178:0/3015321139 wait complete. 2024-09-11T23:28:36.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.707+0000 7f12dc61b640 1 Processor -- start 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.707+0000 7f12dc61b640 1 -- start start 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.707+0000 7f12dc61b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d4100990 0x7f12d41970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.707+0000 7f12dc61b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f12d410ab90 con 0x7f12d4100990 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.707+0000 7f12da3b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d4100990 0x7f12d41970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.707+0000 7f12da3b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d4100990 0x7f12d41970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.178:60072/0 (socket says 172.21.15.178:60072) 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.707+0000 7f12da3b7640 1 -- 172.21.15.178:0/3052668926 learned_addr learned my addr 172.21.15.178:0/3052668926 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.708+0000 7f12da3b7640 1 -- 172.21.15.178:0/3052668926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d4197620 con 0x7f12d4100990 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.708+0000 7f12da3b7640 1 --2- 172.21.15.178:0/3052668926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d4100990 0x7f12d41970e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009230 tx=0x7f12d0003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.708+0000 7f12cb7fe640 1 -- 172.21.15.178:0/3052668926 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12d003f070 con 0x7f12d4100990 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.708+0000 7f12cb7fe640 1 -- 172.21.15.178:0/3052668926 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12d003bd40 con 0x7f12d4100990 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.708+0000 7f12dc61b640 1 -- 172.21.15.178:0/3052668926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d4197910 con 0x7f12d4100990 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.708+0000 7f12cb7fe640 1 -- 172.21.15.178:0/3052668926 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12d0035030 con 0x7f12d4100990 2024-09-11T23:28:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.708+0000 7f12dc61b640 1 -- 172.21.15.178:0/3052668926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d419b9b0 con 0x7f12d4100990 2024-09-11T23:28:36.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.709+0000 7f12cb7fe640 1 -- 172.21.15.178:0/3052668926 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f12d0042070 con 0x7f12d4100990 2024-09-11T23:28:36.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.709+0000 7f12cb7fe640 1 --2- 172.21.15.178:0/3052668926 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f12c0039060 0x7f12c003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.709+0000 7f12cb7fe640 1 -- 172.21.15.178:0/3052668926 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f12d006ccd0 con 0x7f12d4100990 2024-09-11T23:28:36.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.709+0000 7f12dc61b640 1 -- 172.21.15.178:0/3052668926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d4101c40 con 0x7f12d4100990 2024-09-11T23:28:36.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.709+0000 7f12d9bb6640 1 --2- 172.21.15.178:0/3052668926 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f12c0039060 0x7f12c003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.710+0000 7f12d9bb6640 1 --2- 172.21.15.178:0/3052668926 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f12c0039060 0x7f12c003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f12c40098c0 tx=0x7f12c4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:36.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.714+0000 7f12cb7fe640 1 -- 172.21.15.178:0/3052668926 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12d003b860 con 0x7f12d4100990 2024-09-11T23:28:36.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.940+0000 7f12dc61b640 1 -- 172.21.15.178:0/3052668926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f12d4198440 con 0x7f12d4100990 2024-09-11T23:28:36.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.940+0000 7f12cb7fe640 1 -- 172.21.15.178:0/3052668926 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f12d003bb60 con 0x7f12d4100990 2024-09-11T23:28:36.941 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:28:36.941 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","modified":"2024-09-11T23:27:17.802580Z","created":"2024-09-11T23:27:17.802580Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T23:28:36.941 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-11T23:28:36.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.943+0000 7f12dc61b640 1 -- 172.21.15.178:0/3052668926 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f12c0039060 msgr2=0x7f12c003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.943+0000 7f12dc61b640 1 --2- 172.21.15.178:0/3052668926 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f12c0039060 0x7f12c003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f12c40098c0 tx=0x7f12c4006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.943+0000 7f12dc61b640 1 -- 172.21.15.178:0/3052668926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d4100990 msgr2=0x7f12d41970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.943+0000 7f12dc61b640 1 --2- 172.21.15.178:0/3052668926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d4100990 0x7f12d41970e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009230 tx=0x7f12d0003d80 comp rx=0 tx=0).stop 2024-09-11T23:28:36.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.943+0000 7f12dc61b640 1 -- 172.21.15.178:0/3052668926 shutdown_connections 2024-09-11T23:28:36.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.943+0000 7f12dc61b640 1 --2- 172.21.15.178:0/3052668926 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f12c0039060 0x7f12c003b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.943+0000 7f12dc61b640 1 --2- 172.21.15.178:0/3052668926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d4100990 0x7f12d41970e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.943+0000 7f12dc61b640 1 -- 172.21.15.178:0/3052668926 >> 172.21.15.178:0/3052668926 conn(0x7f12d4076250 msgr2=0x7f12d40696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:36.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.943+0000 7f12dc61b640 1 -- 172.21.15.178:0/3052668926 shutdown_connections 2024-09-11T23:28:36.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:36.943+0000 7f12dc61b640 1 -- 172.21.15.178:0/3052668926 wait complete. 2024-09-11T23:28:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:36 smithi133 ceph-mon[31430]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:28:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:36 smithi133 ceph-mon[31430]: Updating smithi178:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:28:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:36 smithi133 ceph-mon[31430]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi138:172.21.15.138=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:28:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:36 smithi133 ceph-mon[31430]: Saving service mon spec with placement smithi133:172.21.15.133=a;smithi138:172.21.15.138=b;smithi178:172.21.15.178=c;count:3 2024-09-11T23:28:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:36 smithi133 ceph-mon[31430]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T23:28:37.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:36 smithi133 ceph-mon[31430]: Reconfiguring daemon mon.a on smithi133 2024-09-11T23:28:37.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:36 smithi133 ceph-mon[31430]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:38.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:38.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:38.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:28:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:28:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:37 smithi133 ceph-mon[31430]: Deploying daemon mon.c on smithi178 2024-09-11T23:28:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:37 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/3052668926' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:28:38.593 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:28:38.593 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:28:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:38 smithi133 ceph-mon[31430]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:39.509 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:28:40.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.074+0000 7f8971dc0640 1 -- 172.21.15.178:0/1849301645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c102840 msgr2=0x7f896c102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.074+0000 7f8971dc0640 1 --2- 172.21.15.178:0/1849301645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c102840 0x7f896c102c20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8960009880 tx=0x7f896002a670 comp rx=0 tx=0).stop 2024-09-11T23:28:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.074+0000 7f8971dc0640 1 -- 172.21.15.178:0/1849301645 shutdown_connections 2024-09-11T23:28:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.074+0000 7f8971dc0640 1 --2- 172.21.15.178:0/1849301645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c102840 0x7f896c102c20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.074+0000 7f8971dc0640 1 -- 172.21.15.178:0/1849301645 >> 172.21.15.178:0/1849301645 conn(0x7f896c0fc280 msgr2=0x7f896c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.074+0000 7f8971dc0640 1 -- 172.21.15.178:0/1849301645 shutdown_connections 2024-09-11T23:28:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.074+0000 7f8971dc0640 1 -- 172.21.15.178:0/1849301645 wait complete. 2024-09-11T23:28:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.075+0000 7f8971dc0640 1 Processor -- start 2024-09-11T23:28:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.075+0000 7f8971dc0640 1 -- start start 2024-09-11T23:28:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.075+0000 7f8971dc0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c102840 0x7f896c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.075+0000 7f8971dc0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f896c10bdb0 con 0x7f896c102840 2024-09-11T23:28:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.075+0000 7f896b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c102840 0x7f896c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:40.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.075+0000 7f896b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c102840 0x7f896c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.178:60090/0 (socket says 172.21.15.178:60090) 2024-09-11T23:28:40.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.075+0000 7f896b7fe640 1 -- 172.21.15.178:0/2135396361 learned_addr learned my addr 172.21.15.178:0/2135396361 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:28:40.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.076+0000 7f896b7fe640 1 -- 172.21.15.178:0/2135396361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c1977e0 con 0x7f896c102840 2024-09-11T23:28:40.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.076+0000 7f896b7fe640 1 --2- 172.21.15.178:0/2135396361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c102840 0x7f896c1972a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8960009230 tx=0x7f8960003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:40.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.076+0000 7f8968ff9640 1 -- 172.21.15.178:0/2135396361 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f896003f070 con 0x7f896c102840 2024-09-11T23:28:40.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.076+0000 7f8968ff9640 1 -- 172.21.15.178:0/2135396361 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f896003bc80 con 0x7f896c102840 2024-09-11T23:28:40.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.076+0000 7f8971dc0640 1 -- 172.21.15.178:0/2135396361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896c197a70 con 0x7f896c102840 2024-09-11T23:28:40.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.076+0000 7f8968ff9640 1 -- 172.21.15.178:0/2135396361 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8960035030 con 0x7f896c102840 2024-09-11T23:28:40.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.076+0000 7f8971dc0640 1 -- 172.21.15.178:0/2135396361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896c197ed0 con 0x7f896c102840 2024-09-11T23:28:40.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.077+0000 7f8968ff9640 1 -- 172.21.15.178:0/2135396361 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8960042070 con 0x7f896c102840 2024-09-11T23:28:40.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.077+0000 7f8971dc0640 1 -- 172.21.15.178:0/2135396361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896c06a4f0 con 0x7f896c102840 2024-09-11T23:28:40.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.077+0000 7f8968ff9640 1 --2- 172.21.15.178:0/2135396361 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8954039060 0x7f895403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:40.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.077+0000 7f8968ff9640 1 -- 172.21.15.178:0/2135396361 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f896006c980 con 0x7f896c102840 2024-09-11T23:28:40.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.078+0000 7f896affd640 1 --2- 172.21.15.178:0/2135396361 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8954039060 0x7f895403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:40.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.078+0000 7f896affd640 1 --2- 172.21.15.178:0/2135396361 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8954039060 0x7f895403b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f895c0098c0 tx=0x7f895c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:40.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.083+0000 7f8968ff9640 1 -- 172.21.15.178:0/2135396361 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89600314c0 con 0x7f896c102840 2024-09-11T23:28:40.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.330+0000 7f8971dc0640 1 -- 172.21.15.178:0/2135396361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f896c198660 con 0x7f896c102840 2024-09-11T23:28:40.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.330+0000 7f8968ff9640 1 -- 172.21.15.178:0/2135396361 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f89600046c0 con 0x7f896c102840 2024-09-11T23:28:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:28:40.331 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","modified":"2024-09-11T23:27:17.802580Z","created":"2024-09-11T23:27:17.802580Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T23:28:40.331 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-11T23:28:40.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.333+0000 7f8971dc0640 1 -- 172.21.15.178:0/2135396361 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8954039060 msgr2=0x7f895403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:40.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.333+0000 7f8971dc0640 1 --2- 172.21.15.178:0/2135396361 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8954039060 0x7f895403b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f895c0098c0 tx=0x7f895c006db0 comp rx=0 tx=0).stop 2024-09-11T23:28:40.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.333+0000 7f8971dc0640 1 -- 172.21.15.178:0/2135396361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c102840 msgr2=0x7f896c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:40.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.333+0000 7f8971dc0640 1 --2- 172.21.15.178:0/2135396361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c102840 0x7f896c1972a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8960009230 tx=0x7f8960003d40 comp rx=0 tx=0).stop 2024-09-11T23:28:40.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.333+0000 7f8971dc0640 1 -- 172.21.15.178:0/2135396361 shutdown_connections 2024-09-11T23:28:40.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.333+0000 7f8971dc0640 1 --2- 172.21.15.178:0/2135396361 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8954039060 0x7f895403b520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:40.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.333+0000 7f8971dc0640 1 --2- 172.21.15.178:0/2135396361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c102840 0x7f896c1972a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:40.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.333+0000 7f8971dc0640 1 -- 172.21.15.178:0/2135396361 >> 172.21.15.178:0/2135396361 conn(0x7f896c0fc280 msgr2=0x7f896c105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:40.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.333+0000 7f8971dc0640 1 -- 172.21.15.178:0/2135396361 shutdown_connections 2024-09-11T23:28:40.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:40.333+0000 7f8971dc0640 1 -- 172.21.15.178:0/2135396361 wait complete. 2024-09-11T23:28:41.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:40 smithi133 ceph-mon[31430]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:41.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:40 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/2135396361' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:28:41.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:41 smithi178 systemd[1]: Started Ceph mon.c for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:28:41.919 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:28:41.920 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:28:42.176 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:28:46.692 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:28:46.692 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.692 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: mon.a calling monitor election 2024-09-11T23:28:46.692 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:46.692 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.692 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.692 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: mon.c calling monitor election 2024-09-11T23:28:46.692 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: monmap epoch 2 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: last_changed 2024-09-11T23:28:41.337419+0000 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: created 2024-09-11T23:27:17.802580+0000 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: min_mon_release 19 (squid) 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: election_strategy: 1 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-11T23:28:46.693 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: fsmap 2024-09-11T23:28:46.694 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:28:46.694 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: mgrmap e13: x(active, since 38s) 2024-09-11T23:28:46.694 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: overall HEALTH_OK 2024-09-11T23:28:46.694 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:46.694 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:46.694 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:46.694 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:28:46.694 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:46.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:28:46.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: mon.a calling monitor election 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: mon.c calling monitor election 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: monmap epoch 2 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: last_changed 2024-09-11T23:28:41.337419+0000 2024-09-11T23:28:46.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: created 2024-09-11T23:27:17.802580+0000 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: min_mon_release 19 (squid) 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: election_strategy: 1 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: fsmap 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: mgrmap e13: x(active, since 38s) 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: overall HEALTH_OK 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:28:46.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:46.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.953+0000 7fab96ee9640 1 -- 172.21.15.178:0/1457429160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab80003530 msgr2=0x7fab800059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:46.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.953+0000 7fab96ee9640 1 --2- 172.21.15.178:0/1457429160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab80003530 0x7fab800059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fab901016a0 tx=0x7fab8402c490 comp rx=0 tx=0).stop 2024-09-11T23:28:46.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.953+0000 7fab96ee9640 1 -- 172.21.15.178:0/1457429160 shutdown_connections 2024-09-11T23:28:46.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.953+0000 7fab96ee9640 1 --2- 172.21.15.178:0/1457429160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab80003530 0x7fab800059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:46.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.953+0000 7fab96ee9640 1 -- 172.21.15.178:0/1457429160 >> 172.21.15.178:0/1457429160 conn(0x7fab900fc220 msgr2=0x7fab900fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:46.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.953+0000 7fab96ee9640 1 -- 172.21.15.178:0/1457429160 shutdown_connections 2024-09-11T23:28:46.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.953+0000 7fab96ee9640 1 -- 172.21.15.178:0/1457429160 wait complete. 2024-09-11T23:28:46.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.954+0000 7fab96ee9640 1 Processor -- start 2024-09-11T23:28:46.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.954+0000 7fab96ee9640 1 -- start start 2024-09-11T23:28:46.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.954+0000 7fab96ee9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab80003530 0x7fab90197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:46.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.954+0000 7fab96ee9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab901977d0 0x7fab9019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:46.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.954+0000 7fab96ee9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fab90108670 con 0x7fab80003530 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.954+0000 7fab96ee9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fab901084f0 con 0x7fab901977d0 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab8ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab901977d0 0x7fab9019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab8ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab901977d0 0x7fab9019bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35290/0 (socket says 172.21.15.178:35290) 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab8ffff640 1 -- 172.21.15.178:0/3761147246 learned_addr learned my addr 172.21.15.178:0/3761147246 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab94c85640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab80003530 0x7fab90197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab8ffff640 1 -- 172.21.15.178:0/3761147246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab901977d0 msgr2=0x7fab9019bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab8ffff640 1 -- 172.21.15.178:0/3761147246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab901977d0 msgr2=0x7fab9019bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab8ffff640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab901977d0 0x7fab9019bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab8ffff640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab901977d0 0x7fab9019bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab94c85640 1 -- 172.21.15.178:0/3761147246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab901977d0 msgr2=0x7fab9019bb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab94c85640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab901977d0 0x7fab9019bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab94c85640 1 -- 172.21.15.178:0/3761147246 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab90197df0 con 0x7fab80003530 2024-09-11T23:28:46.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.955+0000 7fab94c85640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab80003530 0x7fab90197290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab84005f30 tx=0x7fab84033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:46.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.956+0000 7fab8dffb640 1 -- 172.21.15.178:0/3761147246 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fab84038040 con 0x7fab80003530 2024-09-11T23:28:46.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.956+0000 7fab8dffb640 1 -- 172.21.15.178:0/3761147246 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab84032e00 con 0x7fab80003530 2024-09-11T23:28:46.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.956+0000 7fab8dffb640 1 -- 172.21.15.178:0/3761147246 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fab84004a00 con 0x7fab80003530 2024-09-11T23:28:46.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.956+0000 7fab96ee9640 1 -- 172.21.15.178:0/3761147246 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9019c0a0 con 0x7fab80003530 2024-09-11T23:28:46.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.956+0000 7fab96ee9640 1 -- 172.21.15.178:0/3761147246 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9019c660 con 0x7fab80003530 2024-09-11T23:28:46.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.957+0000 7fab8dffb640 1 -- 172.21.15.178:0/3761147246 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fab84043020 con 0x7fab80003530 2024-09-11T23:28:46.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.958+0000 7fab8dffb640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fab74038e50 0x7fab7403b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:46.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.958+0000 7fab8dffb640 1 -- 172.21.15.178:0/3761147246 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fab8406e530 con 0x7fab80003530 2024-09-11T23:28:46.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.958+0000 7fab96ee9640 1 -- 172.21.15.178:0/3761147246 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab90101ca0 con 0x7fab80003530 2024-09-11T23:28:46.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.958+0000 7fab8ffff640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fab74038e50 0x7fab7403b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:46.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.958+0000 7fab8ffff640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fab74038e50 0x7fab7403b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009100 tx=0x7fab7c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:46.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:46.963+0000 7fab8dffb640 1 -- 172.21.15.178:0/3761147246 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab8404a020 con 0x7fab80003530 2024-09-11T23:28:47.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.197+0000 7fab96ee9640 1 -- 172.21.15.178:0/3761147246 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fab9019c910 con 0x7fab80003530 2024-09-11T23:28:47.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.198+0000 7fab8dffb640 1 -- 172.21.15.178:0/3761147246 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fab8404c970 con 0x7fab80003530 2024-09-11T23:28:47.199 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:28:47.199 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","modified":"2024-09-11T23:28:41.337419Z","created":"2024-09-11T23:27:17.802580Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:28:47.199 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-09-11T23:28:47.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.200+0000 7fab96ee9640 1 -- 172.21.15.178:0/3761147246 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fab74038e50 msgr2=0x7fab7403b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:47.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.200+0000 7fab96ee9640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fab74038e50 0x7fab7403b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009100 tx=0x7fab7c008040 comp rx=0 tx=0).stop 2024-09-11T23:28:47.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.200+0000 7fab96ee9640 1 -- 172.21.15.178:0/3761147246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab80003530 msgr2=0x7fab90197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:47.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.200+0000 7fab96ee9640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab80003530 0x7fab90197290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab84005f30 tx=0x7fab84033680 comp rx=0 tx=0).stop 2024-09-11T23:28:47.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.200+0000 7fab96ee9640 1 -- 172.21.15.178:0/3761147246 shutdown_connections 2024-09-11T23:28:47.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.200+0000 7fab96ee9640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fab74038e50 0x7fab7403b310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:47.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.200+0000 7fab96ee9640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab901977d0 0x7fab9019bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:47.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.200+0000 7fab96ee9640 1 --2- 172.21.15.178:0/3761147246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab80003530 0x7fab90197290 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:47.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.200+0000 7fab96ee9640 1 -- 172.21.15.178:0/3761147246 >> 172.21.15.178:0/3761147246 conn(0x7fab900fc220 msgr2=0x7fab9006a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:47.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.200+0000 7fab96ee9640 1 -- 172.21.15.178:0/3761147246 shutdown_connections 2024-09-11T23:28:47.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:47.200+0000 7fab96ee9640 1 -- 172.21.15.178:0/3761147246 wait complete. 2024-09-11T23:28:47.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:47 smithi178 ceph-mon[35882]: Deploying daemon mon.b on smithi138 2024-09-11T23:28:47.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:47 smithi178 ceph-mon[35882]: from='client.? 172.21.15.178:0/3761147246' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:28:47.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:47 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:47.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:47 smithi133 ceph-mon[31430]: Deploying daemon mon.b on smithi138 2024-09-11T23:28:47.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:47 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/3761147246' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:28:47.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:47 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:48.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:48 smithi133 ceph-mon[31430]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:48.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:48 smithi178 ceph-mon[35882]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:48.833 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:28:48.833 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:28:49.085 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:28:49.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.512+0000 7f7b79462640 1 -- 172.21.15.178:0/3682505857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b74106630 msgr2=0x7f7b74106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:49.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.512+0000 7f7b79462640 1 --2- 172.21.15.178:0/3682505857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b74106630 0x7f7b74106a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009880 tx=0x7f7b6402a5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.512+0000 7f7b79462640 1 -- 172.21.15.178:0/3682505857 shutdown_connections 2024-09-11T23:28:49.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.512+0000 7f7b79462640 1 --2- 172.21.15.178:0/3682505857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b74106630 0x7f7b74106a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.512+0000 7f7b79462640 1 -- 172.21.15.178:0/3682505857 >> 172.21.15.178:0/3682505857 conn(0x7f7b740fc220 msgr2=0x7f7b740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:49.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.512+0000 7f7b79462640 1 -- 172.21.15.178:0/3682505857 shutdown_connections 2024-09-11T23:28:49.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.512+0000 7f7b79462640 1 -- 172.21.15.178:0/3682505857 wait complete. 2024-09-11T23:28:49.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b79462640 1 Processor -- start 2024-09-11T23:28:49.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b79462640 1 -- start start 2024-09-11T23:28:49.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b79462640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b74106630 0x7f7b74195190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:49.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b79462640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b741956d0 0x7f7b74199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:49.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b79462640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74109b70 con 0x7f7b74106630 2024-09-11T23:28:49.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b79462640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741099f0 con 0x7f7b741956d0 2024-09-11T23:28:49.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b727fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b741956d0 0x7f7b74199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:49.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b727fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b741956d0 0x7f7b74199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35292/0 (socket says 172.21.15.178:35292) 2024-09-11T23:28:49.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b727fc640 1 -- 172.21.15.178:0/112466540 learned_addr learned my addr 172.21.15.178:0/112466540 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:28:49.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b727fc640 1 -- 172.21.15.178:0/112466540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b741956d0 msgr2=0x7f7b74199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T23:28:49.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b727fc640 1 -- 172.21.15.178:0/112466540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b741956d0 msgr2=0x7f7b74199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:28:49.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b727fc640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b741956d0 0x7f7b74199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:28:49.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b72ffd640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b74106630 0x7f7b74195190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:49.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.513+0000 7f7b727fc640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b741956d0 0x7f7b74199a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:28:49.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.514+0000 7f7b72ffd640 1 -- 172.21.15.178:0/112466540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b741956d0 msgr2=0x7f7b74199a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:28:49.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.514+0000 7f7b72ffd640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b741956d0 0x7f7b74199a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.514+0000 7f7b72ffd640 1 -- 172.21.15.178:0/112466540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b74195cf0 con 0x7f7b74106630 2024-09-11T23:28:49.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.514+0000 7f7b72ffd640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b74106630 0x7f7b74195190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b64005ab0 tx=0x7f7b64032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:49.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.514+0000 7f7b6bfff640 1 -- 172.21.15.178:0/112466540 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7b64035070 con 0x7f7b74106630 2024-09-11T23:28:49.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.514+0000 7f7b79462640 1 -- 172.21.15.178:0/112466540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b74199fa0 con 0x7f7b74106630 2024-09-11T23:28:49.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.514+0000 7f7b6bfff640 1 -- 172.21.15.178:0/112466540 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b64032e40 con 0x7f7b74106630 2024-09-11T23:28:49.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.514+0000 7f7b6bfff640 1 -- 172.21.15.178:0/112466540 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7b6403bc10 con 0x7f7b74106630 2024-09-11T23:28:49.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.514+0000 7f7b79462640 1 -- 172.21.15.178:0/112466540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7419a500 con 0x7f7b74106630 2024-09-11T23:28:49.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.515+0000 7f7b6bfff640 1 -- 172.21.15.178:0/112466540 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7b64042030 con 0x7f7b74106630 2024-09-11T23:28:49.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.515+0000 7f7b79462640 1 -- 172.21.15.178:0/112466540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b74101ca0 con 0x7f7b74106630 2024-09-11T23:28:49.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.515+0000 7f7b6bfff640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7b6003b010 0x7f7b6003d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:49.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.515+0000 7f7b6bfff640 1 -- 172.21.15.178:0/112466540 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7b64040070 con 0x7f7b74106630 2024-09-11T23:28:49.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.516+0000 7f7b727fc640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7b6003b010 0x7f7b6003d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:49.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.516+0000 7f7b727fc640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7b6003b010 0x7f7b6003d4d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009100 tx=0x7f7b5c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:49.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.522+0000 7f7b6bfff640 1 -- 172.21.15.178:0/112466540 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b6403a3c0 con 0x7f7b74106630 2024-09-11T23:28:49.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.746+0000 7f7b79462640 1 -- 172.21.15.178:0/112466540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7b740008d0 con 0x7f7b74106630 2024-09-11T23:28:49.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.746+0000 7f7b6bfff640 1 -- 172.21.15.178:0/112466540 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f7b64003800 con 0x7f7b74106630 2024-09-11T23:28:49.747 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:28:49.747 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","modified":"2024-09-11T23:28:41.337419Z","created":"2024-09-11T23:27:17.802580Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:28:49.747 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-09-11T23:28:49.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.748+0000 7f7b79462640 1 -- 172.21.15.178:0/112466540 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7b6003b010 msgr2=0x7f7b6003d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:49.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.748+0000 7f7b79462640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7b6003b010 0x7f7b6003d4d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009100 tx=0x7f7b5c008040 comp rx=0 tx=0).stop 2024-09-11T23:28:49.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.748+0000 7f7b79462640 1 -- 172.21.15.178:0/112466540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b74106630 msgr2=0x7f7b74195190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:49.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.748+0000 7f7b79462640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b74106630 0x7f7b74195190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b64005ab0 tx=0x7f7b64032c50 comp rx=0 tx=0).stop 2024-09-11T23:28:49.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.748+0000 7f7b79462640 1 -- 172.21.15.178:0/112466540 shutdown_connections 2024-09-11T23:28:49.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.748+0000 7f7b79462640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7b6003b010 0x7f7b6003d4d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.748+0000 7f7b79462640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b741956d0 0x7f7b74199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.748+0000 7f7b79462640 1 --2- 172.21.15.178:0/112466540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b74106630 0x7f7b74195190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.748+0000 7f7b79462640 1 -- 172.21.15.178:0/112466540 >> 172.21.15.178:0/112466540 conn(0x7f7b740fc220 msgr2=0x7f7b740fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:49.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.748+0000 7f7b79462640 1 -- 172.21.15.178:0/112466540 shutdown_connections 2024-09-11T23:28:49.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:49.748+0000 7f7b79462640 1 -- 172.21.15.178:0/112466540 wait complete. 2024-09-11T23:28:50.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:49 smithi178 ceph-mon[35882]: from='client.? 172.21.15.178:0/112466540' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:28:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:49 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/112466540' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:28:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:50 smithi138 systemd[1]: Started Ceph mon.b for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:28:51.307 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:28:51.307 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:28:51.552 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:28:55.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.154+0000 7f87985ef640 1 -- 172.21.15.178:0/4100587389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f877c003530 msgr2=0x7f877c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.154+0000 7f87985ef640 1 --2- 172.21.15.178:0/4100587389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f877c003530 0x7f877c0059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8790102b00 tx=0x7f878c02c150 comp rx=0 tx=0).stop 2024-09-11T23:28:55.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.154+0000 7f87985ef640 1 -- 172.21.15.178:0/4100587389 shutdown_connections 2024-09-11T23:28:55.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.154+0000 7f87985ef640 1 --2- 172.21.15.178:0/4100587389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f877c003530 0x7f877c0059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.154+0000 7f87985ef640 1 -- 172.21.15.178:0/4100587389 >> 172.21.15.178:0/4100587389 conn(0x7f87900fd330 msgr2=0x7f87900ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:55.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.155+0000 7f87985ef640 1 -- 172.21.15.178:0/4100587389 shutdown_connections 2024-09-11T23:28:55.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.155+0000 7f87985ef640 1 -- 172.21.15.178:0/4100587389 wait complete. 2024-09-11T23:28:55.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.156+0000 7f87985ef640 1 Processor -- start 2024-09-11T23:28:55.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.156+0000 7f87985ef640 1 -- start start 2024-09-11T23:28:55.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.156+0000 7f87985ef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f877c003530 0x7f8790197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.156+0000 7f87985ef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8790197850 0x7f879019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.156+0000 7f87985ef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879019c120 0x7f879019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.156+0000 7f87985ef640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8790109790 con 0x7f877c003530 2024-09-11T23:28:55.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.156+0000 7f87985ef640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8790109610 con 0x7f879019c120 2024-09-11T23:28:55.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.156+0000 7f87985ef640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8790109910 con 0x7f8790197850 2024-09-11T23:28:55.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8795b8a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8790197850 0x7f879019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8795b8a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8790197850 0x7f879019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35924/0 (socket says 172.21.15.178:35924) 2024-09-11T23:28:55.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8795b8a640 1 -- 172.21.15.178:0/497816267 learned_addr learned my addr 172.21.15.178:0/497816267 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f879638b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f877c003530 0x7f8790197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8796b8c640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879019c120 0x7f879019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8795b8a640 1 -- 172.21.15.178:0/497816267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8790197850 msgr2=0x7f879019bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8795b8a640 1 -- 172.21.15.178:0/497816267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8790197850 msgr2=0x7f879019bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8795b8a640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8790197850 0x7f879019bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8796b8c640 1 -- 172.21.15.178:0/497816267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879019c120 msgr2=0x7f879019c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8796b8c640 1 -- 172.21.15.178:0/497816267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879019c120 msgr2=0x7f879019c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8796b8c640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879019c120 0x7f879019c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8795b8a640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8790197850 0x7f879019bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f8796b8c640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879019c120 0x7f879019c5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f879638b640 1 -- 172.21.15.178:0/497816267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8790197850 msgr2=0x7f879019bbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:28:55.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f879638b640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8790197850 0x7f879019bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f879638b640 1 -- 172.21.15.178:0/497816267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879019c120 msgr2=0x7f879019c5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:28:55.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f879638b640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879019c120 0x7f879019c5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f879638b640 1 -- 172.21.15.178:0/497816267 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87901a3e90 con 0x7f877c003530 2024-09-11T23:28:55.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f879638b640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f877c003530 0x7f8790197310 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f878c036000 tx=0x7f878c004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f87877fe640 1 -- 172.21.15.178:0/497816267 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c032040 con 0x7f877c003530 2024-09-11T23:28:55.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f87877fe640 1 -- 172.21.15.178:0/497816267 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878c043070 con 0x7f877c003530 2024-09-11T23:28:55.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f87985ef640 1 -- 172.21.15.178:0/497816267 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87901a4060 con 0x7f877c003530 2024-09-11T23:28:55.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f87877fe640 1 -- 172.21.15.178:0/497816267 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c036670 con 0x7f877c003530 2024-09-11T23:28:55.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.157+0000 7f87985ef640 1 -- 172.21.15.178:0/497816267 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87901a45c0 con 0x7f877c003530 2024-09-11T23:28:55.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.158+0000 7f87877fe640 1 -- 172.21.15.178:0/497816267 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f878c036810 con 0x7f877c003530 2024-09-11T23:28:55.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.159+0000 7f87985ef640 1 -- 172.21.15.178:0/497816267 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879010a8c0 con 0x7f877c003530 2024-09-11T23:28:55.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.159+0000 7f87877fe640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f87740394f0 0x7f877403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.159+0000 7f87877fe640 1 -- 172.21.15.178:0/497816267 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f878c06df10 con 0x7f877c003530 2024-09-11T23:28:55.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.159+0000 7f8795b8a640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f87740394f0 0x7f877403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.159+0000 7f8795b8a640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f87740394f0 0x7f877403b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8780009100 tx=0x7f8780008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.164+0000 7f87877fe640 1 -- 172.21.15.178:0/497816267 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f878c045bd0 con 0x7f877c003530 2024-09-11T23:28:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.404+0000 7f87985ef640 1 -- 172.21.15.178:0/497816267 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8790106060 con 0x7f877c003530 2024-09-11T23:28:55.404 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:28:55.404 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.404 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: mon.a calling monitor election 2024-09-11T23:28:55.404 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: mon.c calling monitor election 2024-09-11T23:28:55.404 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:55.404 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.405 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:55.405 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.405 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.405 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:55.405 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.405 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.405 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:28:55.405 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: monmap epoch 3 2024-09-11T23:28:55.405 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:55.405 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: last_changed 2024-09-11T23:28:50.066250+0000 2024-09-11T23:28:55.405 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: created 2024-09-11T23:27:17.802580+0000 2024-09-11T23:28:55.406 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: min_mon_release 19 (squid) 2024-09-11T23:28:55.406 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: election_strategy: 1 2024-09-11T23:28:55.406 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-09-11T23:28:55.406 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-11T23:28:55.406 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-11T23:28:55.406 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: fsmap 2024-09-11T23:28:55.406 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:28:55.406 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: mgrmap e13: x(active, since 47s) 2024-09-11T23:28:55.406 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: overall HEALTH_OK 2024-09-11T23:28:55.407 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:55.407 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:55.407 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:55.407 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:55.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.405+0000 7f87877fe640 1 -- 172.21.15.178:0/497816267 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f878c03d380 con 0x7f877c003530 2024-09-11T23:28:55.407 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:28:55.408 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":3,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","modified":"2024-09-11T23:28:50.066250Z","created":"2024-09-11T23:27:17.802580Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:28:55.408 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 3 2024-09-11T23:28:55.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 -- 172.21.15.178:0/497816267 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f87740394f0 msgr2=0x7f877403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f87740394f0 0x7f877403b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8780009100 tx=0x7f8780008040 comp rx=0 tx=0).stop 2024-09-11T23:28:55.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 -- 172.21.15.178:0/497816267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f877c003530 msgr2=0x7f8790197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f877c003530 0x7f8790197310 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f878c036000 tx=0x7f878c004d30 comp rx=0 tx=0).stop 2024-09-11T23:28:55.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 -- 172.21.15.178:0/497816267 shutdown_connections 2024-09-11T23:28:55.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f87740394f0 0x7f877403b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879019c120 0x7f879019c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8790197850 0x7f879019bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 --2- 172.21.15.178:0/497816267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f877c003530 0x7f8790197310 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 -- 172.21.15.178:0/497816267 >> 172.21.15.178:0/497816267 conn(0x7f87900fd330 msgr2=0x7f87900ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:55.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 -- 172.21.15.178:0/497816267 shutdown_connections 2024-09-11T23:28:55.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:55.407+0000 7f87985ef640 1 -- 172.21.15.178:0/497816267 wait complete. 2024-09-11T23:28:55.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: mon.a calling monitor election 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: mon.c calling monitor election 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:28:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: monmap epoch 3 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: last_changed 2024-09-11T23:28:50.066250+0000 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: created 2024-09-11T23:27:17.802580+0000 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: min_mon_release 19 (squid) 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: election_strategy: 1 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: fsmap 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: mgrmap e13: x(active, since 47s) 2024-09-11T23:28:55.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: overall HEALTH_OK 2024-09-11T23:28:55.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:55.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:55.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:55.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:56.001 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T23:28:56.001 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T23:28:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:56 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:56 smithi178 ceph-mon[35882]: from='client.? 172.21.15.178:0/497816267' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:28:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:56 smithi178 ceph-mon[35882]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:56 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:56.379 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:56 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:56.379 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:56 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/497816267' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:28:56.379 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:56 smithi133 ceph-mon[31430]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:56.379 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:56 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.701+0000 7f16c6a1f640 1 -- 172.21.15.133:0/1710024860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16c0100430 msgr2=0x7f16c0100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.701+0000 7f16c6a1f640 1 --2- 172.21.15.133:0/1710024860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16c0100430 0x7f16c0100810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009880 tx=0x7f16b002a5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.701+0000 7f16c6a1f640 1 -- 172.21.15.133:0/1710024860 shutdown_connections 2024-09-11T23:28:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.701+0000 7f16c6a1f640 1 --2- 172.21.15.133:0/1710024860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16c0100430 0x7f16c0100810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.701+0000 7f16c6a1f640 1 -- 172.21.15.133:0/1710024860 >> 172.21.15.133:0/1710024860 conn(0x7f16c00fc280 msgr2=0x7f16c00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:56.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.701+0000 7f16c6a1f640 1 -- 172.21.15.133:0/1710024860 shutdown_connections 2024-09-11T23:28:56.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.701+0000 7f16c6a1f640 1 -- 172.21.15.133:0/1710024860 wait complete. 2024-09-11T23:28:56.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.702+0000 7f16c6a1f640 1 Processor -- start 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.702+0000 7f16c6a1f640 1 -- start start 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.702+0000 7f16c6a1f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c0100430 0x7f16c0199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.702+0000 7f16c6a1f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c0199c70 0x7f16c019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.702+0000 7f16c6a1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16c019e540 0x7f16c019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.702+0000 7f16c6a1f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f16c010df40 con 0x7f16c019e540 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.702+0000 7f16c6a1f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16c010ddc0 con 0x7f16c0100430 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.702+0000 7f16c6a1f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16c010e0c0 con 0x7f16c0199c70 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16c4fbc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16c019e540 0x7f16c019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c0199c70 0x7f16c019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c0100430 0x7f16c0199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c0199c70 0x7f16c019e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:43162/0 (socket says 172.21.15.133:43162) 2024-09-11T23:28:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16c4fbc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16c019e540 0x7f16c019e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41262/0 (socket says 172.21.15.133:41262) 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bf7fe640 1 -- 172.21.15.133:0/75163266 learned_addr learned my addr 172.21.15.133:0/75163266 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bffff640 1 -- 172.21.15.133:0/75163266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c0100430 msgr2=0x7f16c0199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bffff640 1 -- 172.21.15.133:0/75163266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c0100430 msgr2=0x7f16c0199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bffff640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c0100430 0x7f16c0199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bffff640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c0100430 0x7f16c0199730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bf7fe640 1 -- 172.21.15.133:0/75163266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c0100430 msgr2=0x7f16c0199730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bf7fe640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c0100430 0x7f16c0199730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bf7fe640 1 -- 172.21.15.133:0/75163266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16c019e540 msgr2=0x7f16c019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bf7fe640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16c019e540 0x7f16c019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bf7fe640 1 -- 172.21.15.133:0/75163266 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c01a62b0 con 0x7f16c0199c70 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16c4fbc640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16c019e540 0x7f16c019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bf7fe640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c0199c70 0x7f16c019e000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f16b400b520 tx=0x7f16b400b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bd7fa640 1 -- 172.21.15.133:0/75163266 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4004230 con 0x7f16c0199c70 2024-09-11T23:28:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16c6a1f640 1 -- 172.21.15.133:0/75163266 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c01a64e0 con 0x7f16c0199c70 2024-09-11T23:28:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bd7fa640 1 -- 172.21.15.133:0/75163266 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b40043d0 con 0x7f16c0199c70 2024-09-11T23:28:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.703+0000 7f16bd7fa640 1 -- 172.21.15.133:0/75163266 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4003d60 con 0x7f16c0199c70 2024-09-11T23:28:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.704+0000 7f16c6a1f640 1 -- 172.21.15.133:0/75163266 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c01a6a40 con 0x7f16c0199c70 2024-09-11T23:28:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.704+0000 7f16bd7fa640 1 -- 172.21.15.133:0/75163266 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f16b400e070 con 0x7f16c0199c70 2024-09-11T23:28:56.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.704+0000 7f16c6a1f640 1 -- 172.21.15.133:0/75163266 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c0100890 con 0x7f16c0199c70 2024-09-11T23:28:56.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.705+0000 7f16bd7fa640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f16a80394f0 0x7f16a803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:56.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.705+0000 7f16bd7fa640 1 -- 172.21.15.133:0/75163266 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f16b4010790 con 0x7f16c0199c70 2024-09-11T23:28:56.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.705+0000 7f16bffff640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f16a80394f0 0x7f16a803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:56.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.705+0000 7f16bffff640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f16a80394f0 0x7f16a803b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f16b0031550 tx=0x7f16b00058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:56.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.710+0000 7f16bd7fa640 1 -- 172.21.15.133:0/75163266 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16b4014020 con 0x7f16c0199c70 2024-09-11T23:28:56.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.886+0000 7f16c6a1f640 1 -- 172.21.15.133:0/75163266 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f16c01a6da0 con 0x7f16c0199c70 2024-09-11T23:28:56.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.887+0000 7f16bd7fa640 1 -- 172.21.15.133:0/75163266 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f16b4010df0 con 0x7f16c0199c70 2024-09-11T23:28:56.887 INFO:teuthology.orchestra.run.smithi133.stdout:# minimal ceph.conf for 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:56.887 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-09-11T23:28:56.887 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:56.887 INFO:teuthology.orchestra.run.smithi133.stdout: mon_host = [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-11T23:28:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.889+0000 7f16c6a1f640 1 -- 172.21.15.133:0/75163266 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f16a80394f0 msgr2=0x7f16a803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.890+0000 7f16c6a1f640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f16a80394f0 0x7f16a803b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f16b0031550 tx=0x7f16b00058b0 comp rx=0 tx=0).stop 2024-09-11T23:28:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.890+0000 7f16c6a1f640 1 -- 172.21.15.133:0/75163266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c0199c70 msgr2=0x7f16c019e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.890+0000 7f16c6a1f640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c0199c70 0x7f16c019e000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f16b400b520 tx=0x7f16b400b9e0 comp rx=0 tx=0).stop 2024-09-11T23:28:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.890+0000 7f16c6a1f640 1 -- 172.21.15.133:0/75163266 shutdown_connections 2024-09-11T23:28:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.890+0000 7f16c6a1f640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f16a80394f0 0x7f16a803b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.890+0000 7f16c6a1f640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16c019e540 0x7f16c019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.890+0000 7f16c6a1f640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c0199c70 0x7f16c019e000 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.890+0000 7f16c6a1f640 1 --2- 172.21.15.133:0/75163266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c0100430 0x7f16c0199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.890+0000 7f16c6a1f640 1 -- 172.21.15.133:0/75163266 >> 172.21.15.133:0/75163266 conn(0x7f16c00fc280 msgr2=0x7f16c00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.890+0000 7f16c6a1f640 1 -- 172.21.15.133:0/75163266 shutdown_connections 2024-09-11T23:28:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:28:56.890+0000 7f16c6a1f640 1 -- 172.21.15.133:0/75163266 wait complete. 2024-09-11T23:28:57.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:28:57.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:57.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: mon.a calling monitor election 2024-09-11T23:28:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: mon.c calling monitor election 2024-09-11T23:28:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:28:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:28:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: monmap epoch 3 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: last_changed 2024-09-11T23:28:50.066250+0000 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: created 2024-09-11T23:27:17.802580+0000 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: min_mon_release 19 (squid) 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: election_strategy: 1 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: fsmap 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: mgrmap e13: x(active, since 47s) 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: overall HEALTH_OK 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:57.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='client.? 172.21.15.178:0/497816267' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:28:57.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:57.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:57.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: mon.b calling monitor election 2024-09-11T23:28:57.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: mon.b calling monitor election 2024-09-11T23:28:57.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: mon.c calling monitor election 2024-09-11T23:28:57.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: mon.a calling monitor election 2024-09-11T23:28:57.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:28:57.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: monmap epoch 3 2024-09-11T23:28:57.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:57.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: last_changed 2024-09-11T23:28:50.066250+0000 2024-09-11T23:28:57.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: created 2024-09-11T23:27:17.802580+0000 2024-09-11T23:28:57.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: min_mon_release 19 (squid) 2024-09-11T23:28:57.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: election_strategy: 1 2024-09-11T23:28:57.685 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-09-11T23:28:57.685 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-11T23:28:57.685 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-11T23:28:57.685 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: fsmap 2024-09-11T23:28:57.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:28:57.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: mgrmap e13: x(active, since 49s) 2024-09-11T23:28:57.689 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: overall HEALTH_OK 2024-09-11T23:28:57.689 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.689 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.689 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.689 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.689 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.689 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.689 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.689 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:28:57.689 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:28:57.689 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:57 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:57.713 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T23:28:57.713 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:28:57.713 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:28:57.742 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:28:57.742 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:28:57.808 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-11T23:28:57.808 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:28:57.836 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-11T23:28:57.836 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:28:57.901 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-11T23:28:57.901 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:28:57.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: mon.b calling monitor election 2024-09-11T23:28:57.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: mon.b calling monitor election 2024-09-11T23:28:57.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: mon.c calling monitor election 2024-09-11T23:28:57.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: mon.a calling monitor election 2024-09-11T23:28:57.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:28:57.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: monmap epoch 3 2024-09-11T23:28:57.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:57.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: last_changed 2024-09-11T23:28:50.066250+0000 2024-09-11T23:28:57.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: created 2024-09-11T23:27:17.802580+0000 2024-09-11T23:28:57.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: min_mon_release 19 (squid) 2024-09-11T23:28:57.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: election_strategy: 1 2024-09-11T23:28:57.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-09-11T23:28:57.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-11T23:28:57.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: fsmap 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: mgrmap e13: x(active, since 49s) 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: overall HEALTH_OK 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:28:57.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:28:57.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:57 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:57.930 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-11T23:28:57.930 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:28:57.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: mon.b calling monitor election 2024-09-11T23:28:57.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: mon.b calling monitor election 2024-09-11T23:28:57.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: mon.c calling monitor election 2024-09-11T23:28:57.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: mon.a calling monitor election 2024-09-11T23:28:57.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:28:57.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: monmap epoch 3 2024-09-11T23:28:57.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:28:57.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: last_changed 2024-09-11T23:28:50.066250+0000 2024-09-11T23:28:57.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: created 2024-09-11T23:27:17.802580+0000 2024-09-11T23:28:57.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: min_mon_release 19 (squid) 2024-09-11T23:28:57.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: election_strategy: 1 2024-09-11T23:28:57.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-09-11T23:28:57.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-11T23:28:57.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-11T23:28:57.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: fsmap 2024-09-11T23:28:57.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:28:57.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: mgrmap e13: x(active, since 49s) 2024-09-11T23:28:57.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: overall HEALTH_OK 2024-09-11T23:28:57.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:57.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:28:57.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:28:57.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:57.997 INFO:tasks.cephadm:Adding mgr.x on smithi133 2024-09-11T23:28:57.998 INFO:tasks.cephadm:Adding mgr.y on smithi138 2024-09-11T23:28:57.998 INFO:tasks.cephadm:Adding mgr.z on smithi178 2024-09-11T23:28:57.998 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi133=x;smithi138=y;smithi178=z' 2024-09-11T23:28:58.272 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:28:58.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.667+0000 7fc76dbc3640 1 -- 172.21.15.178:0/2562448694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7681021e0 msgr2=0x7fc7681045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.667+0000 7fc76dbc3640 1 --2- 172.21.15.178:0/2562448694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7681021e0 0x7fc7681045d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009880 tx=0x7fc75c02a5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.667+0000 7fc76dbc3640 1 -- 172.21.15.178:0/2562448694 shutdown_connections 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.667+0000 7fc76dbc3640 1 --2- 172.21.15.178:0/2562448694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7681021e0 0x7fc7681045d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.667+0000 7fc76dbc3640 1 -- 172.21.15.178:0/2562448694 >> 172.21.15.178:0/2562448694 conn(0x7fc7680fbda0 msgr2=0x7fc7680fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.667+0000 7fc76dbc3640 1 -- 172.21.15.178:0/2562448694 shutdown_connections 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.667+0000 7fc76dbc3640 1 -- 172.21.15.178:0/2562448694 wait complete. 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.667+0000 7fc76dbc3640 1 Processor -- start 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc76dbc3640 1 -- start start 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc76dbc3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7681021e0 0x7fc768196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc76dbc3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7681974c0 0x7fc76819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc76dbc3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc768197940 0x7fc76819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:58.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc76dbc3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681077f0 con 0x7fc768197940 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc76dbc3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc768107670 con 0x7fc7681021e0 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc76dbc3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc768107970 con 0x7fc7681974c0 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc766ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7681974c0 0x7fc76819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc766ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7681974c0 0x7fc76819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35926/0 (socket says 172.21.15.178:35926) 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc766ffd640 1 -- 172.21.15.178:0/3696447283 learned_addr learned my addr 172.21.15.178:0/3696447283 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc767fff640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc768197940 0x7fc76819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc7677fe640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7681021e0 0x7fc768196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc766ffd640 1 -- 172.21.15.178:0/3696447283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7681021e0 msgr2=0x7fc768196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc766ffd640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7681021e0 0x7fc768196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc766ffd640 1 -- 172.21.15.178:0/3696447283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc768197940 msgr2=0x7fc76819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc766ffd640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc768197940 0x7fc76819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc766ffd640 1 -- 172.21.15.178:0/3696447283 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76819f380 con 0x7fc7681974c0 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc767fff640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc768197940 0x7fc76819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:58.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.668+0000 7fc766ffd640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7681974c0 0x7fc76819c530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc75800b520 tx=0x7fc75800b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:58.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.669+0000 7fc764ff9640 1 -- 172.21.15.178:0/3696447283 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758004230 con 0x7fc7681974c0 2024-09-11T23:28:58.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.669+0000 7fc764ff9640 1 -- 172.21.15.178:0/3696447283 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7580043d0 con 0x7fc7681974c0 2024-09-11T23:28:58.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.669+0000 7fc76dbc3640 1 -- 172.21.15.178:0/3696447283 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc768100030 con 0x7fc7681974c0 2024-09-11T23:28:58.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.669+0000 7fc764ff9640 1 -- 172.21.15.178:0/3696447283 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758003d60 con 0x7fc7681974c0 2024-09-11T23:28:58.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.669+0000 7fc76dbc3640 1 -- 172.21.15.178:0/3696447283 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc768100640 con 0x7fc7681974c0 2024-09-11T23:28:58.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.670+0000 7fc764ff9640 1 -- 172.21.15.178:0/3696447283 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc75800e070 con 0x7fc7681974c0 2024-09-11T23:28:58.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.670+0000 7fc76dbc3640 1 -- 172.21.15.178:0/3696447283 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76802d7c0 con 0x7fc7681974c0 2024-09-11T23:28:58.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.670+0000 7fc764ff9640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc7500394f0 0x7fc75003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:58.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.670+0000 7fc764ff9640 1 -- 172.21.15.178:0/3696447283 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc75804b950 con 0x7fc7681974c0 2024-09-11T23:28:58.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.670+0000 7fc7677fe640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc7500394f0 0x7fc75003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:58.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.670+0000 7fc7677fe640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc7500394f0 0x7fc75003b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc75c031610 tx=0x7fc75c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:58.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.675+0000 7fc764ff9640 1 -- 172.21.15.178:0/3696447283 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc758014020 con 0x7fc7681974c0 2024-09-11T23:28:58.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.854+0000 7fc76dbc3640 1 -- 172.21.15.178:0/3696447283 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi138=y;smithi178=z", "target": ["mon-mgr", ""]}) -- 0x7fc7681901c0 con 0x7fc7500394f0 2024-09-11T23:28:58.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.860+0000 7fc764ff9640 1 -- 172.21.15.178:0/3696447283 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc7681901c0 con 0x7fc7500394f0 2024-09-11T23:28:58.859 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mgr update... 2024-09-11T23:28:58.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 -- 172.21.15.178:0/3696447283 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc7500394f0 msgr2=0x7fc75003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:58.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc7500394f0 0x7fc75003b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc75c031610 tx=0x7fc75c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:28:58.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 -- 172.21.15.178:0/3696447283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7681974c0 msgr2=0x7fc76819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:58.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7681974c0 0x7fc76819c530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc75800b520 tx=0x7fc75800b9e0 comp rx=0 tx=0).stop 2024-09-11T23:28:58.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 -- 172.21.15.178:0/3696447283 shutdown_connections 2024-09-11T23:28:58.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc7500394f0 0x7fc75003b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:58.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc768197940 0x7fc76819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:58.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7681974c0 0x7fc76819c530 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:58.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 --2- 172.21.15.178:0/3696447283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7681021e0 0x7fc768196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:58.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 -- 172.21.15.178:0/3696447283 >> 172.21.15.178:0/3696447283 conn(0x7fc7680fbda0 msgr2=0x7fc7680fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:58.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 -- 172.21.15.178:0/3696447283 shutdown_connections 2024-09-11T23:28:58.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:28:58.862+0000 7fc76dbc3640 1 -- 172.21.15.178:0/3696447283 wait complete. 2024-09-11T23:28:58.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:58 smithi133 ceph-mon[31430]: Reconfiguring mon.a (monmap changed)... 2024-09-11T23:28:58.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:58 smithi133 ceph-mon[31430]: Reconfiguring daemon mon.a on smithi133 2024-09-11T23:28:58.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:58 smithi133 ceph-mon[31430]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:58.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:58 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:58.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:58 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:59.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:58 smithi178 ceph-mon[35882]: Reconfiguring mon.a (monmap changed)... 2024-09-11T23:28:59.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:58 smithi178 ceph-mon[35882]: Reconfiguring daemon mon.a on smithi133 2024-09-11T23:28:59.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:58 smithi178 ceph-mon[35882]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:59.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:58 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:59.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:58 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:58 smithi138 ceph-mon[35142]: Reconfiguring mon.a (monmap changed)... 2024-09-11T23:28:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:58 smithi138 ceph-mon[35142]: Reconfiguring daemon mon.a on smithi133 2024-09-11T23:28:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:58 smithi138 ceph-mon[35142]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:28:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:58 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:28:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:58 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:28:59.400 DEBUG:teuthology.orchestra.run.smithi138:mgr.y> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T23:28:59.403 DEBUG:teuthology.orchestra.run.smithi178:mgr.z> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.z.service 2024-09-11T23:28:59.405 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T23:28:59.405 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:28:59.405 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:28:59.422 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:28:59.422 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-09-11T23:28:59.478 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:28:59.478 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-11T23:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-11 23:28:16.437792147 +0000 2024-09-11T23:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-11 23:24:03.057066535 +0000 2024-09-11T23:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-11 23:24:03.057066535 +0000 2024-09-11T23:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-11 23:24:03.057066535 +0000 2024-09-11T23:28:59.479 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:28:59.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-11T23:28:59.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-11T23:28:59.544 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000189315 s, 2.7 MB/s 2024-09-11T23:28:59.545 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:28:59.602 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-09-11T23:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-11T23:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:28:59.658 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-11 23:28:16.462791674 +0000 2024-09-11T23:28:59.658 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-11 23:24:03.387060334 +0000 2024-09-11T23:28:59.658 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-11 23:24:03.387060334 +0000 2024-09-11T23:28:59.658 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-11 23:24:03.387060334 +0000 2024-09-11T23:28:59.658 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:28:59.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-11T23:28:59.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-11T23:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000193998 s, 2.6 MB/s 2024-09-11T23:28:59.722 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:28:59.778 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-09-11T23:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-11T23:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-11 23:28:16.486791221 +0000 2024-09-11T23:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-11 23:24:03.713054208 +0000 2024-09-11T23:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-11 23:24:03.713054208 +0000 2024-09-11T23:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-11 23:24:03.713054208 +0000 2024-09-11T23:28:59.834 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:28:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-11T23:28:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-11T23:28:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000187572 s, 2.7 MB/s 2024-09-11T23:28:59.898 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:28:59.955 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-09-11T23:29:00.011 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:29:00.011 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-11T23:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-11 23:28:16.513790711 +0000 2024-09-11T23:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-11 23:24:04.063047631 +0000 2024-09-11T23:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-11 23:24:04.063047631 +0000 2024-09-11T23:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-11 23:24:04.063047631 +0000 2024-09-11T23:29:00.012 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:29:00.075 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-11T23:29:00.075 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-11T23:29:00.075 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000270597 s, 1.9 MB/s 2024-09-11T23:29:00.076 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:29:00.133 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-11T23:29:00.133 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:29:00.152 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:29:00.152 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-09-11T23:29:00.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:59 smithi138 ceph-mon[35142]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi138=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:00.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:59 smithi138 ceph-mon[35142]: Saving service mgr spec with placement smithi133=x;smithi138=y;smithi178=z;count:3 2024-09-11T23:29:00.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:59 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:00.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:59 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:00.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:59 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:00.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:59 smithi138 ceph-mon[35142]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:29:00.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:59 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:29:00.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:59 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:29:00.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:59 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:00.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:28:59 smithi138 ceph-mon[35142]: Reconfiguring daemon mgr.x on smithi133 2024-09-11T23:29:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:59 smithi133 ceph-mon[31430]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi138=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:59 smithi133 ceph-mon[31430]: Saving service mgr spec with placement smithi133=x;smithi138=y;smithi178=z;count:3 2024-09-11T23:29:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:59 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:59 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:59 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:59 smithi133 ceph-mon[31430]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:29:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:59 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:29:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:59 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:29:00.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:59 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:00.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:28:59 smithi133 ceph-mon[31430]: Reconfiguring daemon mgr.x on smithi133 2024-09-11T23:29:00.212 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:29:00.212 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:29:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-11T23:29:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:29:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:29:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-11 23:28:49.514665865 +0000 2024-09-11T23:29:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-11 23:24:17.838316582 +0000 2024-09-11T23:29:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-11 23:24:17.838316582 +0000 2024-09-11T23:29:00.212 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-11 23:24:17.838316582 +0000 2024-09-11T23:29:00.213 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:29:00.279 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-11T23:29:00.279 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-11T23:29:00.279 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000281107 s, 1.8 MB/s 2024-09-11T23:29:00.281 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:29:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:59 smithi178 ceph-mon[35882]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi138=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:59 smithi178 ceph-mon[35882]: Saving service mgr spec with placement smithi133=x;smithi138=y;smithi178=z;count:3 2024-09-11T23:29:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:59 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:59 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:59 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:59 smithi178 ceph-mon[35882]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:29:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:59 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:29:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:59 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:29:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:59 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:28:59 smithi178 ceph-mon[35882]: Reconfiguring daemon mgr.x on smithi133 2024-09-11T23:29:00.339 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-09-11T23:29:00.397 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:29:00.397 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:29:00.397 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-11T23:29:00.397 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:29:00.397 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:29:00.397 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-11 23:28:49.515665844 +0000 2024-09-11T23:29:00.397 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-11 23:24:18.163309690 +0000 2024-09-11T23:29:00.397 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-11 23:24:18.163309690 +0000 2024-09-11T23:29:00.397 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-11 23:24:18.163309690 +0000 2024-09-11T23:29:00.397 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:29:00.461 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-11T23:29:00.461 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-11T23:29:00.462 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000221003 s, 2.3 MB/s 2024-09-11T23:29:00.463 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:29:00.520 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-09-11T23:29:00.576 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:29:00.576 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:29:00.577 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-11T23:29:00.577 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:29:00.577 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:29:00.577 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-11 23:28:49.515665844 +0000 2024-09-11T23:29:00.577 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-11 23:24:18.485302861 +0000 2024-09-11T23:29:00.577 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-11 23:24:18.485302861 +0000 2024-09-11T23:29:00.577 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-11 23:24:18.485302861 +0000 2024-09-11T23:29:00.577 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:29:00.641 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-11T23:29:00.641 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-11T23:29:00.641 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000207596 s, 2.5 MB/s 2024-09-11T23:29:00.643 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:29:00.700 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-09-11T23:29:00.755 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:29:00.755 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:29:00.755 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-11T23:29:00.755 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:29:00.755 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:29:00.755 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-11 23:28:49.515665844 +0000 2024-09-11T23:29:00.756 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-11 23:24:18.811295947 +0000 2024-09-11T23:29:00.756 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-11 23:24:18.811295947 +0000 2024-09-11T23:29:00.756 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-11 23:24:18.811295947 +0000 2024-09-11T23:29:00.756 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:29:00.819 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-11T23:29:00.819 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-11T23:29:00.819 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000245359 s, 2.1 MB/s 2024-09-11T23:29:00.820 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:29:00.877 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-11T23:29:00.877 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:29:00.897 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:29:00.897 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-09-11T23:29:00.956 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:29:00.957 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:29:00.957 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-11T23:29:00.957 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:29:00.957 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:29:00.957 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-11 23:28:40.437715455 +0000 2024-09-11T23:29:00.957 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-11 23:24:22.298222681 +0000 2024-09-11T23:29:00.957 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-11 23:24:22.298222681 +0000 2024-09-11T23:29:00.957 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-11 23:24:22.298222681 +0000 2024-09-11T23:29:00.957 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:29:01.024 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-11T23:29:01.024 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-11T23:29:01.024 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000107226 s, 4.8 MB/s 2024-09-11T23:29:01.026 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:29:01.085 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-09-11T23:29:01.143 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:29:01.143 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:29:01.143 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-11T23:29:01.143 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:29:01.143 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:29:01.143 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-11 23:28:40.438715436 +0000 2024-09-11T23:29:01.143 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-11 23:24:22.607216973 +0000 2024-09-11T23:29:01.143 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-11 23:24:22.607216973 +0000 2024-09-11T23:29:01.143 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-11 23:24:22.607216973 +0000 2024-09-11T23:29:01.144 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:29:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:00 smithi138 ceph-mon[35142]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:29:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:29:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:01.207 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:00 smithi178 ceph-mon[35882]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:01.207 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:01.207 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:01.207 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:29:01.207 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:29:01.207 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:01.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:00 smithi133 ceph-mon[31430]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:01.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:01.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:01.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:29:01.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:29:01.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:01.210 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-11T23:29:01.210 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-11T23:29:01.210 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000123123 s, 4.2 MB/s 2024-09-11T23:29:01.211 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:29:01.269 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-09-11T23:29:01.326 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:29:01.326 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:29:01.326 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-11T23:29:01.326 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:29:01.326 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:29:01.326 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-11 23:28:40.438715436 +0000 2024-09-11T23:29:01.326 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-11 23:24:22.994209823 +0000 2024-09-11T23:29:01.326 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-11 23:24:22.994209823 +0000 2024-09-11T23:29:01.326 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-11 23:24:22.994209823 +0000 2024-09-11T23:29:01.326 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:29:01.391 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-11T23:29:01.391 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-11T23:29:01.392 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000122954 s, 4.2 MB/s 2024-09-11T23:29:01.393 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:29:01.451 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-09-11T23:29:01.507 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:29:01.507 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:29:01.508 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-11T23:29:01.508 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:29:01.508 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:29:01.508 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-11 23:28:40.439715418 +0000 2024-09-11T23:29:01.508 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-11 23:24:23.316203875 +0000 2024-09-11T23:29:01.508 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-11 23:24:23.316203875 +0000 2024-09-11T23:29:01.508 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-11 23:24:23.316203875 +0000 2024-09-11T23:29:01.508 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:29:01.573 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-11T23:29:01.573 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-11T23:29:01.573 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 9.667e-05 s, 5.3 MB/s 2024-09-11T23:29:01.575 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:29:01.632 INFO:tasks.cephadm:Deploying osd.0 on smithi133 with /dev/vg_nvme/lv_4... 2024-09-11T23:29:01.633 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:29:01.888 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:02.189 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:01 smithi178 ceph-mon[35882]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:29:02.189 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:01 smithi178 ceph-mon[35882]: Reconfiguring daemon mon.b on smithi138 2024-09-11T23:29:02.190 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:01 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:02.190 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:01 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:02.190 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:01 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:29:02.190 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:01 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:29:02.190 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:01 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:02.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:01 smithi138 ceph-mon[35142]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:29:02.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:01 smithi138 ceph-mon[35142]: Reconfiguring daemon mon.b on smithi138 2024-09-11T23:29:02.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:01 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:02.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:01 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:02.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:01 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:29:02.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:01 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:29:02.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:01 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:01 smithi133 ceph-mon[31430]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:29:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:01 smithi133 ceph-mon[31430]: Reconfiguring daemon mon.b on smithi138 2024-09-11T23:29:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:01 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:01 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:01 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:29:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:01 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:29:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:01 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:03.193 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:02 smithi133 ceph-mon[31430]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:29:03.193 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:02 smithi133 ceph-mon[31430]: Reconfiguring daemon mon.c on smithi178 2024-09-11T23:29:03.193 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:02 smithi133 ceph-mon[31430]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:03.193 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:03.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:02 smithi138 ceph-mon[35142]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:29:03.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:02 smithi138 ceph-mon[35142]: Reconfiguring daemon mon.c on smithi178 2024-09-11T23:29:03.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:02 smithi138 ceph-mon[35142]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:03.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:02 smithi178 ceph-mon[35882]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:29:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:02 smithi178 ceph-mon[35882]: Reconfiguring daemon mon.c on smithi178 2024-09-11T23:29:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:02 smithi178 ceph-mon[35882]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:04.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:04.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:04.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:04.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:04.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:04.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:29:04.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:29:04.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:29:04.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:04.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:03 smithi138 ceph-mon[35142]: Deploying daemon mgr.z on smithi178 2024-09-11T23:29:04.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:04.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:04.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:04.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:04.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:04.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:29:04.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:29:04.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:29:04.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:04.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:03 smithi133 ceph-mon[31430]: Deploying daemon mgr.z on smithi178 2024-09-11T23:29:04.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:04.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:04.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:04.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:04.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:04.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:29:04.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:29:04.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:29:04.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:04.243 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:03 smithi178 ceph-mon[35882]: Deploying daemon mgr.z on smithi178 2024-09-11T23:29:04.275 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:29:04.305 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_4 2024-09-11T23:29:04.559 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:05.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.992+0000 7f45eac2e640 1 -- 172.21.15.133:0/1823394499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e4101d10 msgr2=0x7f45e4102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:05.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.992+0000 7f45eac2e640 1 --2- 172.21.15.133:0/1823394499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e4101d10 0x7f45e4102190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009900 tx=0x7f45d002a410 comp rx=0 tx=0).stop 2024-09-11T23:29:05.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 -- 172.21.15.133:0/1823394499 shutdown_connections 2024-09-11T23:29:05.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 --2- 172.21.15.133:0/1823394499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45e41026d0 0x7f45e4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 --2- 172.21.15.133:0/1823394499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e4101d10 0x7f45e4102190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 --2- 172.21.15.133:0/1823394499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45e4100b10 0x7f45e4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 -- 172.21.15.133:0/1823394499 >> 172.21.15.133:0/1823394499 conn(0x7f45e40fc2a0 msgr2=0x7f45e40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:05.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 -- 172.21.15.133:0/1823394499 shutdown_connections 2024-09-11T23:29:05.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 -- 172.21.15.133:0/1823394499 wait complete. 2024-09-11T23:29:05.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 Processor -- start 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 -- start start 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45e4100b10 0x7f45e4197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e4101d10 0x7f45e41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45e41026d0 0x7f45e419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45e410d8c0 con 0x7f45e4101d10 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45e410d740 con 0x7f45e41026d0 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.993+0000 7f45eac2e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45e410da40 con 0x7f45e4100b10 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e4101d10 0x7f45e41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e4101d10 0x7f45e41983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35354/0 (socket says 172.21.15.133:35354) 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e3fff640 1 -- 172.21.15.133:0/1371662727 learned_addr learned my addr 172.21.15.133:0/1371662727 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e89ca640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45e4100b10 0x7f45e4197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e3fff640 1 -- 172.21.15.133:0/1371662727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45e4100b10 msgr2=0x7f45e4197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:05.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e3fff640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45e4100b10 0x7f45e4197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e91cb640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45e41026d0 0x7f45e419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e3fff640 1 -- 172.21.15.133:0/1371662727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45e41026d0 msgr2=0x7f45e419f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e3fff640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45e41026d0 0x7f45e419f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e3fff640 1 -- 172.21.15.133:0/1371662727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e419faa0 con 0x7f45e4101d10 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e89ca640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45e4100b10 0x7f45e4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e3fff640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e4101d10 0x7f45e41983b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009230 tx=0x7f45d0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e1ffb640 1 -- 172.21.15.133:0/1371662727 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d003f030 con 0x7f45e4101d10 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e1ffb640 1 -- 172.21.15.133:0/1371662727 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d0003ce0 con 0x7f45e4101d10 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45e1ffb640 1 -- 172.21.15.133:0/1371662727 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0035070 con 0x7f45e4101d10 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.994+0000 7f45eac2e640 1 -- 172.21.15.133:0/1371662727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e419fd30 con 0x7f45e4101d10 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.995+0000 7f45eac2e640 1 -- 172.21.15.133:0/1371662727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e41a0200 con 0x7f45e4101d10 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.996+0000 7f45e1ffb640 1 -- 172.21.15.133:0/1371662727 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f45d0042030 con 0x7f45e4101d10 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.996+0000 7f45eac2e640 1 -- 172.21.15.133:0/1371662727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e4107ec0 con 0x7f45e4101d10 2024-09-11T23:29:05.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.996+0000 7f45e1ffb640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f45d40392c0 0x7f45d403b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:05.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.996+0000 7f45e1ffb640 1 -- 172.21.15.133:0/1371662727 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f45d006c9b0 con 0x7f45e4101d10 2024-09-11T23:29:05.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.996+0000 7f45e89ca640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f45d40392c0 0x7f45d403b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:05.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:04.996+0000 7f45e89ca640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f45d40392c0 0x7f45d403b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f45e4101b70 tx=0x7f45d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:05.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:05.001+0000 7f45e1ffb640 1 -- 172.21.15.133:0/1371662727 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f45d004c350 con 0x7f45e4101d10 2024-09-11T23:29:05.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:05.178+0000 7f45eac2e640 1 -- 172.21.15.133:0/1371662727 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f45e4062630 con 0x7f45d40392c0 2024-09-11T23:29:05.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:04 smithi133 ceph-mon[31430]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:05.210 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:04 smithi178 ceph-mon[35882]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:05.210 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:05.148+0000 7f929f401040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:29:05.210 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:05.210+0000 7f929f401040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:29:05.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:04 smithi138 ceph-mon[35142]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:05.540 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:05.360+0000 7f929f401040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:29:05.540 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:05.425+0000 7f929f401040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:29:05.949 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:05.570+0000 7f929f401040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:29:05.950 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:05.709+0000 7f929f401040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:29:06.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:29:06.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:29:06.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:06.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:06.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:06.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:06.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:29:06.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:29:06.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:29:06.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:06.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:29:06.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:29:06.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:06.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:06.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:06.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:06.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:29:06.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:29:06.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:29:06.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:06.268 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:06 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:06.023+0000 7f929f401040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:29:06.268 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:06 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:06.083+0000 7f929f401040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:29:06.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:29:06.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:29:06.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:06.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:06.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:06.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:06.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:29:06.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:29:06.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:29:06.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:06.540 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:06 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:06.268+0000 7f929f401040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:29:07.100 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:06 smithi133 ceph-mon[31430]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:07.100 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:06 smithi133 ceph-mon[31430]: Deploying daemon mgr.y on smithi138 2024-09-11T23:29:07.100 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:06 smithi133 ceph-mon[31430]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:07.141 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:06 smithi138 ceph-mon[35142]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:07.142 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:06 smithi138 ceph-mon[35142]: Deploying daemon mgr.y on smithi138 2024-09-11T23:29:07.142 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:06 smithi138 ceph-mon[35142]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:07.202 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:06 smithi178 ceph-mon[35882]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:07.203 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:06 smithi178 ceph-mon[35882]: Deploying daemon mgr.y on smithi138 2024-09-11T23:29:07.203 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:06 smithi178 ceph-mon[35882]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:07.203 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:07.137+0000 7f929f401040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:29:07.203 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:07.203+0000 7f929f401040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:29:07.406 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:07 smithi138 podman[36498]: 2024-09-11 23:29:07.383014396 +0000 UTC m=+0.353744089 container start 1b7d14b3bbac10b61420cec7375b522ca8aa4dea65c1ef77fd3a581538e3a3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T23:29:07.698 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-mgr[36517]: -- 172.21.15.138:0/571296886 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55adda2f50e0 con 0x55adda2d2800 2024-09-11T23:29:07.698 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:07 smithi138 bash[36498]: 1b7d14b3bbac10b61420cec7375b522ca8aa4dea65c1ef77fd3a581538e3a3b2 2024-09-11T23:29:07.698 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:07 smithi138 systemd[1]: Started Ceph mgr.y for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:29:07.698 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:07.507+0000 7f61ba5fb040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:29:07.698 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:07.566+0000 7f61ba5fb040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:29:08.101 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:07.722+0000 7f61ba5fb040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:29:08.101 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:07.792+0000 7f61ba5fb040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:29:08.102 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:07.953+0000 7f61ba5fb040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:29:08.102 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:08.102+0000 7f61ba5fb040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:29:08.102 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2508898622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ba73339-5c3f-4621-85af-3c3addd0583c"}]: dispatch 2024-09-11T23:29:08.103 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2508898622' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ba73339-5c3f-4621-85af-3c3addd0583c"}]': finished 2024-09-11T23:29:08.103 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-mon[35142]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:29:08.103 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:08.103 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.103 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.103 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.103 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.103 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:08.103 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:07 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2508898622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ba73339-5c3f-4621-85af-3c3addd0583c"}]: dispatch 2024-09-11T23:29:08.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:07 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2508898622' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ba73339-5c3f-4621-85af-3c3addd0583c"}]': finished 2024-09-11T23:29:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:07 smithi133 ceph-mon[31430]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:29:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:08.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.212 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:07.945+0000 7f929f401040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:29:08.212 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:08.072+0000 7f929f401040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:29:08.212 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2508898622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ba73339-5c3f-4621-85af-3c3addd0583c"}]: dispatch 2024-09-11T23:29:08.212 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2508898622' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ba73339-5c3f-4621-85af-3c3addd0583c"}]': finished 2024-09-11T23:29:08.212 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-mon[35882]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:29:08.212 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:08.212 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.212 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.213 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.213 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.213 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:08.213 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:08.424 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:08.424+0000 7f61ba5fb040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:29:08.502 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:08.215+0000 7f929f401040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:29:08.502 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:08.279+0000 7f929f401040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:29:08.502 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:08.434+0000 7f929f401040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:29:08.502 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:08.502+0000 7f929f401040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:29:08.684 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:08.486+0000 7f61ba5fb040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:29:08.790 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:08.563+0000 7f929f401040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:29:08.790 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:08.729+0000 7f929f401040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:29:08.948 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:08.685+0000 7f61ba5fb040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:29:09.063 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:09.063 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2757447407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:29:09.063 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.063 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.063 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.063 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:08.793+0000 7f929f401040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:08.861+0000 7f929f401040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:08.946+0000 7f929f401040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:09 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:09.006+0000 7f929f401040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:29:09.064 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:09 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:09.063+0000 7f929f401040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:29:09.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2757447407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:09.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:09.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2757447407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:29:09.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:09.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:09.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:09.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:09.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:09.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:09.948 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:09 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:09.574+0000 7f61ba5fb040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:29:09.948 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:09 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:09.640+0000 7f61ba5fb040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:29:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:09.962+0000 7f45e1ffb640 1 -- 172.21.15.133:0/1371662727 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f45d006cd10 con 0x7f45e4101d10 2024-09-11T23:29:10.290 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:29:09 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:29:09.840+0000 7f929f401040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:29:10.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:09 smithi178 ceph-mon[35882]: Standby manager daemon z started 2024-09-11T23:29:10.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:09 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:29:10.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:09 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:29:10.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:09 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:29:10.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:09 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:29:10.358 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:09 smithi138 ceph-mon[35142]: Standby manager daemon z started 2024-09-11T23:29:10.359 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:09 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:29:10.359 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:09 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:29:10.359 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:09 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:29:10.359 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:09 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:29:10.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:09 smithi133 ceph-mon[31430]: Standby manager daemon z started 2024-09-11T23:29:10.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:09 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:29:10.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:09 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:29:10.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:09 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:29:10.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:09 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.178:0/3844168834' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:29:10.616 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:10.359+0000 7f61ba5fb040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:29:10.616 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:10.475+0000 7f61ba5fb040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:29:10.616 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:10.617+0000 7f61ba5fb040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:29:10.895 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:10.678+0000 7f61ba5fb040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:29:10.895 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:10.828+0000 7f61ba5fb040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:29:10.895 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:10.895+0000 7f61ba5fb040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:29:11.177 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:10 smithi138 ceph-mon[35142]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:11.178 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:10 smithi138 ceph-mon[35142]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T23:29:11.178 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:10 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:29:11.178 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:10.955+0000 7f61ba5fb040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:29:11.178 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:11.113+0000 7f61ba5fb040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:29:11.178 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:11.178+0000 7f61ba5fb040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:29:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:10 smithi178 ceph-mon[35882]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:10 smithi178 ceph-mon[35882]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T23:29:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:10 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:29:11.444 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:11.244+0000 7f61ba5fb040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:29:11.444 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:11.327+0000 7f61ba5fb040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:29:11.444 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:11.386+0000 7f61ba5fb040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:29:11.444 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:11.444+0000 7f61ba5fb040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:29:11.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:10 smithi133 ceph-mon[31430]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:11.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:10 smithi133 ceph-mon[31430]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T23:29:11.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:10 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:29:12.698 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:29:12 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:29:12.229+0000 7f61ba5fb040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:29:12.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:12.973+0000 7f45e1ffb640 1 -- 172.21.15.133:0/1371662727 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f45d003b460 con 0x7f45e4101d10 2024-09-11T23:29:13.121 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:12 smithi133 ceph-mon[31430]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:13.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:12 smithi133 ceph-mon[31430]: Standby manager daemon y started 2024-09-11T23:29:13.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:12 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:29:13.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:12 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:29:13.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:12 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:29:13.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:12 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:29:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:12 smithi178 ceph-mon[35882]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:12 smithi178 ceph-mon[35882]: Standby manager daemon y started 2024-09-11T23:29:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:12 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:29:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:12 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:29:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:12 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:29:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:12 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:29:13.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:12 smithi138 ceph-mon[35142]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:13.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:12 smithi138 ceph-mon[35142]: Standby manager daemon y started 2024-09-11T23:29:13.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:12 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:29:13.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:12 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:29:13.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:12 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:29:13.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:12 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.138:0/747825118' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:29:14.208 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:13 smithi133 ceph-mon[31430]: mgrmap e15: x(active, since 65s), standbys: y, z 2024-09-11T23:29:14.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:13 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:29:14.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:13 smithi178 ceph-mon[35882]: mgrmap e15: x(active, since 65s), standbys: y, z 2024-09-11T23:29:14.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:13 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:29:14.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:13 smithi138 ceph-mon[35142]: mgrmap e15: x(active, since 65s), standbys: y, z 2024-09-11T23:29:14.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:13 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:29:15.081 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:14 smithi133 ceph-mon[31430]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:15.292 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:14 smithi178 ceph-mon[35882]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:15.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:14 smithi138 ceph-mon[35142]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:16 smithi133 ceph-mon[31430]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:29:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:17.251 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:16 smithi138 ceph-mon[35142]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:17.251 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:29:17.251 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:17.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:16 smithi178 ceph-mon[35882]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:17.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:29:17.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:18.016 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:17 smithi133 ceph-mon[31430]: Deploying daemon osd.0 on smithi133 2024-09-11T23:29:18.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:17 smithi178 ceph-mon[35882]: Deploying daemon osd.0 on smithi133 2024-09-11T23:29:18.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:17 smithi138 ceph-mon[35142]: Deploying daemon osd.0 on smithi133 2024-09-11T23:29:19.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:18 smithi133 ceph-mon[31430]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:18 smithi178 ceph-mon[35882]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:19.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:18 smithi138 ceph-mon[35142]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:20.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:19 smithi178 ceph-mon[35882]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:20.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:19 smithi138 ceph-mon[35142]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:20.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:19 smithi133 ceph-mon[31430]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:23.100 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:22 smithi133 ceph-mon[31430]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:23.101 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:22 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:23.101 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:22 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:23.101 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:22 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:23.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:22 smithi138 ceph-mon[35142]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:23.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:22 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:23.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:22 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:23.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:22 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:22 smithi178 ceph-mon[35882]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:22 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:22 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:22 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:25.044 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:24 smithi133 ceph-mon[31430]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:24 smithi138 ceph-mon[35142]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:25.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:24 smithi178 ceph-mon[35882]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:26.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:26.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:26.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:26.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:26.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:26.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.745+0000 7f45e1ffb640 1 -- 172.21.15.133:0/1371662727 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f45e4062630 con 0x7f45d40392c0 2024-09-11T23:29:26.745 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 0 on host 'smithi133' 2024-09-11T23:29:26.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.747+0000 7f45eac2e640 1 -- 172.21.15.133:0/1371662727 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f45d40392c0 msgr2=0x7f45d403b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:26.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.747+0000 7f45eac2e640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f45d40392c0 0x7f45d403b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f45e4101b70 tx=0x7f45d8005f90 comp rx=0 tx=0).stop 2024-09-11T23:29:26.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.747+0000 7f45eac2e640 1 -- 172.21.15.133:0/1371662727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e4101d10 msgr2=0x7f45e41983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:26.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.747+0000 7f45eac2e640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e4101d10 0x7f45e41983b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009230 tx=0x7f45d0004360 comp rx=0 tx=0).stop 2024-09-11T23:29:26.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.747+0000 7f45eac2e640 1 -- 172.21.15.133:0/1371662727 shutdown_connections 2024-09-11T23:29:26.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.747+0000 7f45eac2e640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f45d40392c0 0x7f45d403b780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:26.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.747+0000 7f45eac2e640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45e41026d0 0x7f45e419f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:26.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.747+0000 7f45eac2e640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e4101d10 0x7f45e41983b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:26.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.747+0000 7f45eac2e640 1 --2- 172.21.15.133:0/1371662727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45e4100b10 0x7f45e4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:26.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.747+0000 7f45eac2e640 1 -- 172.21.15.133:0/1371662727 >> 172.21.15.133:0/1371662727 conn(0x7f45e40fc2a0 msgr2=0x7f45e4105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:26.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.748+0000 7f45eac2e640 1 -- 172.21.15.133:0/1371662727 shutdown_connections 2024-09-11T23:29:26.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:29:26.748+0000 7f45eac2e640 1 -- 172.21.15.133:0/1371662727 wait complete. 2024-09-11T23:29:27.049 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:26 smithi133 ceph-mon[31430]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:27.050 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:26 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:27.050 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:26 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:27.050 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:26 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:26 smithi138 ceph-mon[35142]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:26 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:26 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:26 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:27.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:26 smithi178 ceph-mon[35882]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:27.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:26 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:27.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:26 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:27.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:26 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:27.390 DEBUG:teuthology.orchestra.run.smithi133:osd.0> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T23:29:27.392 INFO:tasks.cephadm:Deploying osd.1 on smithi138 with /dev/vg_nvme/lv_4... 2024-09-11T23:29:27.392 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:29:27.631 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:29:28.803 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:29:28 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0[42562]: 2024-09-11T23:29:28.685+0000 7f314f7d6640 -1 osd.0 0 log_to_monitors true 2024-09-11T23:29:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:28 smithi138 ceph-mon[35142]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:28 smithi138 ceph-mon[35142]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:29:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:28 smithi133 ceph-mon[31430]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:28 smithi133 ceph-mon[31430]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:29:29.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:28 smithi178 ceph-mon[35882]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:29.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:28 smithi178 ceph-mon[35882]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:29:29.916 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:29:29.953 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-09-11T23:29:30.129 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:29 smithi133 ceph-mon[31430]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:29:30.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:29 smithi133 ceph-mon[31430]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:29:30.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:29 smithi133 ceph-mon[31430]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:29:30.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:29 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:30.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:29 smithi133 ceph-mon[31430]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:29:30.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:29 smithi138 ceph-mon[35142]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:29:30.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:29 smithi138 ceph-mon[35142]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:29:30.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:29 smithi138 ceph-mon[35142]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:29:30.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:29 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:30.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:29 smithi138 ceph-mon[35142]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:29:30.202 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:29:30.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:29 smithi178 ceph-mon[35882]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:29:30.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:29 smithi178 ceph-mon[35882]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:29:30.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:29 smithi178 ceph-mon[35882]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:29:30.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:29 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:30.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:29 smithi178 ceph-mon[35882]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:29:30.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.694+0000 7f11dd1fc640 1 -- 172.21.15.138:0/3175623629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d8101d10 msgr2=0x7f11d8102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:30.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.694+0000 7f11dd1fc640 1 --2- 172.21.15.138:0/3175623629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d8101d10 0x7f11d8102190 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009900 tx=0x7f11c002a410 comp rx=0 tx=0).stop 2024-09-11T23:29:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.694+0000 7f11dd1fc640 1 -- 172.21.15.138:0/3175623629 shutdown_connections 2024-09-11T23:29:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.694+0000 7f11dd1fc640 1 --2- 172.21.15.138:0/3175623629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d81026d0 0x7f11d8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.694+0000 7f11dd1fc640 1 --2- 172.21.15.138:0/3175623629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d8101d10 0x7f11d8102190 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.694+0000 7f11dd1fc640 1 --2- 172.21.15.138:0/3175623629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d8100b10 0x7f11d8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.694+0000 7f11dd1fc640 1 -- 172.21.15.138:0/3175623629 >> 172.21.15.138:0/3175623629 conn(0x7f11d80fc2a0 msgr2=0x7f11d80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.694+0000 7f11dd1fc640 1 -- 172.21.15.138:0/3175623629 shutdown_connections 2024-09-11T23:29:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.694+0000 7f11dd1fc640 1 -- 172.21.15.138:0/3175623629 wait complete. 2024-09-11T23:29:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.694+0000 7f11dd1fc640 1 Processor -- start 2024-09-11T23:29:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11dd1fc640 1 -- start start 2024-09-11T23:29:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11dd1fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d8100b10 0x7f11d806b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11dd1fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d8101d10 0x7f11d819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11dd1fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d81026d0 0x7f11d819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11dd1fc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f11d810c790 con 0x7f11d8100b10 2024-09-11T23:29:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11dd1fc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11d810c610 con 0x7f11d81026d0 2024-09-11T23:29:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11dd1fc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11d810c910 con 0x7f11d8101d10 2024-09-11T23:29:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d759e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d81026d0 0x7f11d819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d8100b10 0x7f11d806b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d759e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d81026d0 0x7f11d819f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55376/0 (socket says 172.21.15.138:55376) 2024-09-11T23:29:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d759e640 1 -- 172.21.15.138:0/630161462 learned_addr learned my addr 172.21.15.138:0/630161462 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-11T23:29:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d8100b10 0x7f11d806b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.138:46262/0 (socket says 172.21.15.138:46262) 2024-09-11T23:29:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d659c640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d8101d10 0x7f11d819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:30.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d759e640 1 -- 172.21.15.138:0/630161462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d8101d10 msgr2=0x7f11d819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:30.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d759e640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d8101d10 0x7f11d819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:30.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d759e640 1 -- 172.21.15.138:0/630161462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d8100b10 msgr2=0x7f11d806b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:30.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d759e640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d8100b10 0x7f11d806b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:30.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d759e640 1 -- 172.21.15.138:0/630161462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d819fb50 con 0x7f11d81026d0 2024-09-11T23:29:30.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d659c640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d8101d10 0x7f11d819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:30.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.695+0000 7f11d6d9d640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d8100b10 0x7f11d806b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:30.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.696+0000 7f11d759e640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d81026d0 0x7f11d819f290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11d000cad0 tx=0x7f11d000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:30.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.696+0000 7f11cffff640 1 -- 172.21.15.138:0/630161462 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d000ae20 con 0x7f11d81026d0 2024-09-11T23:29:30.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.696+0000 7f11cffff640 1 -- 172.21.15.138:0/630161462 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d0005020 con 0x7f11d81026d0 2024-09-11T23:29:30.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.696+0000 7f11dd1fc640 1 -- 172.21.15.138:0/630161462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d819fe40 con 0x7f11d81026d0 2024-09-11T23:29:30.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.696+0000 7f11cffff640 1 -- 172.21.15.138:0/630161462 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d000f6f0 con 0x7f11d81026d0 2024-09-11T23:29:30.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.696+0000 7f11dd1fc640 1 -- 172.21.15.138:0/630161462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d81a02f0 con 0x7f11d81026d0 2024-09-11T23:29:30.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.697+0000 7f11dd1fc640 1 -- 172.21.15.138:0/630161462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d8107f60 con 0x7f11d81026d0 2024-09-11T23:29:30.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.698+0000 7f11cffff640 1 -- 172.21.15.138:0/630161462 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f11d00051d0 con 0x7f11d81026d0 2024-09-11T23:29:30.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.699+0000 7f11cffff640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f11c40a2b70 0x7f11c40a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:30.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.699+0000 7f11cffff640 1 -- 172.21.15.138:0/630161462 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f11d00cd700 con 0x7f11d81026d0 2024-09-11T23:29:30.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.699+0000 7f11d6d9d640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f11c40a2b70 0x7f11c40a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:30.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.699+0000 7f11d6d9d640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f11c40a2b70 0x7f11c40a5030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f11d806b870 tx=0x7f11c800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:30.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.703+0000 7f11cffff640 1 -- 172.21.15.138:0/630161462 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11d0014030 con 0x7f11d81026d0 2024-09-11T23:29:30.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:30.878+0000 7f11dd1fc640 1 -- 172.21.15.138:0/630161462 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f11d8062630 con 0x7f11c40a2b70 2024-09-11T23:29:30.959 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0[42562]: 2024-09-11T23:29:30.753+0000 7f314af36640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T23:29:30.959 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0[42562]: 2024-09-11T23:29:30.757+0000 7f3143568640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:29:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:29:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi133 2024-09-11T23:29:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi133 to 17088M 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:29:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi133 2024-09-11T23:29:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:29:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi133 to 17088M 2024-09-11T23:29:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' 2024-09-11T23:29:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:29:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:29:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:31.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:31.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi133 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi133 to 17088M 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877]' entity='osd.0' 2024-09-11T23:29:31.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:31.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:29:31.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:29:31.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:31.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:31 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:29:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:31 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:29:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:31 smithi178 ceph-mon[35882]: osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877] boot 2024-09-11T23:29:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:31 smithi178 ceph-mon[35882]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:29:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:31 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:31 smithi178 ceph-mon[35882]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:32.311 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:31 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:29:32.311 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:31 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:29:32.312 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:31 smithi138 ceph-mon[35142]: osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877] boot 2024-09-11T23:29:32.312 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:31 smithi138 ceph-mon[35142]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:29:32.312 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:31 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:32.312 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:31 smithi138 ceph-mon[35142]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:32.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:31 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:29:32.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:31 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:29:32.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:31 smithi133 ceph-mon[31430]: osd.0 [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877] boot 2024-09-11T23:29:32.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:31 smithi133 ceph-mon[31430]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:29:32.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:31 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:29:32.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:31 smithi133 ceph-mon[31430]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:29:33.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:33 smithi138 ceph-mon[35142]: from='client.? 172.21.15.138:0/2577254439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19075246-d736-48fd-b96d-68395df7eb8b"}]: dispatch 2024-09-11T23:29:33.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:33 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19075246-d736-48fd-b96d-68395df7eb8b"}]: dispatch 2024-09-11T23:29:33.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:33 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19075246-d736-48fd-b96d-68395df7eb8b"}]': finished 2024-09-11T23:29:33.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:33 smithi138 ceph-mon[35142]: osdmap e9: 2 total, 1 up, 2 in 2024-09-11T23:29:33.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:33 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:33.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:33 smithi138 ceph-mon[35142]: from='client.? 172.21.15.138:0/2103189899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:29:34.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:33 smithi178 ceph-mon[35882]: from='client.? 172.21.15.138:0/2577254439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19075246-d736-48fd-b96d-68395df7eb8b"}]: dispatch 2024-09-11T23:29:34.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:33 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19075246-d736-48fd-b96d-68395df7eb8b"}]: dispatch 2024-09-11T23:29:34.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:33 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19075246-d736-48fd-b96d-68395df7eb8b"}]': finished 2024-09-11T23:29:34.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:33 smithi178 ceph-mon[35882]: osdmap e9: 2 total, 1 up, 2 in 2024-09-11T23:29:34.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:33 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:34.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:33 smithi178 ceph-mon[35882]: from='client.? 172.21.15.138:0/2103189899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:29:34.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:33 smithi133 ceph-mon[31430]: from='client.? 172.21.15.138:0/2577254439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19075246-d736-48fd-b96d-68395df7eb8b"}]: dispatch 2024-09-11T23:29:34.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:33 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19075246-d736-48fd-b96d-68395df7eb8b"}]: dispatch 2024-09-11T23:29:34.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:33 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19075246-d736-48fd-b96d-68395df7eb8b"}]': finished 2024-09-11T23:29:34.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:33 smithi133 ceph-mon[31430]: osdmap e9: 2 total, 1 up, 2 in 2024-09-11T23:29:34.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:33 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:34.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:33 smithi133 ceph-mon[31430]: from='client.? 172.21.15.138:0/2103189899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:29:35.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:34 smithi178 ceph-mon[35882]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:34 smithi138 ceph-mon[35142]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:34 smithi133 ceph-mon[31430]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:36 smithi138 ceph-mon[35142]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:36 smithi133 ceph-mon[31430]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:36 smithi178 ceph-mon[35882]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:39.065 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:38 smithi138 ceph-mon[35142]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:38 smithi133 ceph-mon[31430]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:38 smithi178 ceph-mon[35882]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:40.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:40 smithi133 ceph-mon[31430]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:41.037 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:40 smithi138 ceph-mon[35142]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:40 smithi178 ceph-mon[35882]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:42.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:41 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:29:42.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:41 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:42.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:41 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:29:42.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:41 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:41 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:29:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:41 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:43.063 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:42 smithi138 ceph-mon[35142]: Deploying daemon osd.1 on smithi138 2024-09-11T23:29:43.063 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:42 smithi138 ceph-mon[35142]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:43.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:42 smithi133 ceph-mon[31430]: Deploying daemon osd.1 on smithi138 2024-09-11T23:29:43.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:42 smithi133 ceph-mon[31430]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:43.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:42 smithi178 ceph-mon[35882]: Deploying daemon osd.1 on smithi138 2024-09-11T23:29:43.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:42 smithi178 ceph-mon[35882]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:45.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:44 smithi138 ceph-mon[35142]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:45.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:44 smithi133 ceph-mon[31430]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:44 smithi178 ceph-mon[35882]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:46.982 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:46 smithi138 ceph-mon[35142]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:46 smithi133 ceph-mon[31430]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:47.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:46 smithi178 ceph-mon[35882]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:48.188 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:47 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:48.188 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:47 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:48.188 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:47 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:48.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:47 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:48.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:47 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:48.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:47 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:48.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:47 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:48.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:47 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:48.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:47 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:49.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:48 smithi138 ceph-mon[35142]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:49.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:48 smithi133 ceph-mon[31430]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:48 smithi178 ceph-mon[35882]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:51.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:50 smithi138 ceph-mon[35142]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:51.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:51.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:51.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:51.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:51.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:50 smithi178 ceph-mon[35882]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:51.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:50 smithi133 ceph-mon[31430]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:51.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:51.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:51.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:51.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:51.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:51.508 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 1 on host 'smithi138' 2024-09-11T23:29:51.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.508+0000 7f11cffff640 1 -- 172.21.15.138:0/630161462 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f11d8062630 con 0x7f11c40a2b70 2024-09-11T23:29:51.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 -- 172.21.15.138:0/630161462 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f11c40a2b70 msgr2=0x7f11c40a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:51.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f11c40a2b70 0x7f11c40a5030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f11d806b870 tx=0x7f11c800b410 comp rx=0 tx=0).stop 2024-09-11T23:29:51.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 -- 172.21.15.138:0/630161462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d81026d0 msgr2=0x7f11d819f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:51.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d81026d0 0x7f11d819f290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11d000cad0 tx=0x7f11d000cf90 comp rx=0 tx=0).stop 2024-09-11T23:29:51.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 -- 172.21.15.138:0/630161462 shutdown_connections 2024-09-11T23:29:51.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f11c40a2b70 0x7f11c40a5030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d81026d0 0x7f11d819f290 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d8101d10 0x7f11d819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 --2- 172.21.15.138:0/630161462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d8100b10 0x7f11d806b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 -- 172.21.15.138:0/630161462 >> 172.21.15.138:0/630161462 conn(0x7f11d80fc2a0 msgr2=0x7f11d8105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:51.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 -- 172.21.15.138:0/630161462 shutdown_connections 2024-09-11T23:29:51.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:29:51.510+0000 7f11dd1fc640 1 -- 172.21.15.138:0/630161462 wait complete. 2024-09-11T23:29:52.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:51 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:52.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:51 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:52.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:51 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:52.158 DEBUG:teuthology.orchestra.run.smithi138:osd.1> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T23:29:52.161 INFO:tasks.cephadm:Deploying osd.2 on smithi178 with /dev/vg_nvme/lv_4... 2024-09-11T23:29:52.161 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:29:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:51 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:51 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:51 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:52.426 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:29:52.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:51 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:52.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:51 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:52.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:51 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:53.228 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:52 smithi138 ceph-mon[35142]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:52 smithi178 ceph-mon[35882]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:53.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:52 smithi133 ceph-mon[31430]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:54.173 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:53 smithi178 ceph-mon[35882]: from='osd.1 [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:29:54.173 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:53 smithi178 ceph-mon[35882]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:29:54.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:53 smithi138 ceph-mon[35142]: from='osd.1 [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:29:54.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:53 smithi138 ceph-mon[35142]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:29:54.198 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:29:53 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1[39607]: 2024-09-11T23:29:53.902+0000 7fe2c971b640 -1 osd.1 0 log_to_monitors true 2024-09-11T23:29:54.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:53 smithi133 ceph-mon[31430]: from='osd.1 [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:29:54.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:53 smithi133 ceph-mon[31430]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:29:54.891 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:29:54.916 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2024-09-11T23:29:55.161 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:29:55.283 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:55.283 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:29:55.283 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: from='osd.1 [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:29:55.283 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:29:55.283 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:55.283 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:29:55.283 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.284 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.284 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:29:55.284 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.284 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:55.284 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:55.284 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.448 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1[39607]: 2024-09-11T23:29:54.985+0000 7fe2c4e7b640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T23:29:55.448 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1[39607]: 2024-09-11T23:29:54.987+0000 7fe2c14b5640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:29:55.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:55.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:29:55.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: from='osd.1 [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:29:55.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:29:55.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:55.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:29:55.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:29:55.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:55.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:55.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:55.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:29:55.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: from='osd.1 [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:29:55.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:29:55.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:29:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:29:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:55.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:29:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.609+0000 7f3de017c640 1 -- 172.21.15.178:0/3309208384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd8076220 msgr2=0x7f3dd8074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.609+0000 7f3de017c640 1 --2- 172.21.15.178:0/3309208384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd8076220 0x7f3dd8074680 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009ca0 tx=0x7f3dc802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:29:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.609+0000 7f3de017c640 1 -- 172.21.15.178:0/3309208384 shutdown_connections 2024-09-11T23:29:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.609+0000 7f3de017c640 1 --2- 172.21.15.178:0/3309208384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dd8109a80 0x7f3dd810bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.609+0000 7f3de017c640 1 --2- 172.21.15.178:0/3309208384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dd8074c50 0x7f3dd80750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.609+0000 7f3de017c640 1 --2- 172.21.15.178:0/3309208384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd8076220 0x7f3dd8074680 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.609+0000 7f3de017c640 1 -- 172.21.15.178:0/3309208384 >> 172.21.15.178:0/3309208384 conn(0x7f3dd80fd390 msgr2=0x7f3dd80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.609+0000 7f3de017c640 1 -- 172.21.15.178:0/3309208384 shutdown_connections 2024-09-11T23:29:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.609+0000 7f3de017c640 1 -- 172.21.15.178:0/3309208384 wait complete. 2024-09-11T23:29:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3de017c640 1 Processor -- start 2024-09-11T23:29:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3de017c640 1 -- start start 2024-09-11T23:29:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3de017c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dd8074c50 0x7f3dd819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3de017c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd8076220 0x7f3dd819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3de017c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dd8109a80 0x7f3dd81a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3de017c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd810fae0 con 0x7f3dd8109a80 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3de017c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd810f960 con 0x7f3dd8074c50 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3de017c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd810fc60 con 0x7f3dd8076220 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3ddd717640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd8076220 0x7f3dd819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3ddd717640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd8076220 0x7f3dd819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49886/0 (socket says 172.21.15.178:49886) 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3ddd717640 1 -- 172.21.15.178:0/4013100210 learned_addr learned my addr 172.21.15.178:0/4013100210 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3dde719640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dd8109a80 0x7f3dd81a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.610+0000 7f3dddf18640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dd8074c50 0x7f3dd819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3ddd717640 1 -- 172.21.15.178:0/4013100210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dd8074c50 msgr2=0x7f3dd819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3ddd717640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dd8074c50 0x7f3dd819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3ddd717640 1 -- 172.21.15.178:0/4013100210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dd8109a80 msgr2=0x7f3dd81a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3ddd717640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dd8109a80 0x7f3dd81a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3ddd717640 1 -- 172.21.15.178:0/4013100210 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd81a1da0 con 0x7f3dd8076220 2024-09-11T23:29:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3dde719640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dd8109a80 0x7f3dd81a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3ddd717640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd8076220 0x7f3dd819a6b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00b4a0 tx=0x7f3dcc00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3dd6ffd640 1 -- 172.21.15.178:0/4013100210 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc014070 con 0x7f3dd8076220 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3dd6ffd640 1 -- 172.21.15.178:0/4013100210 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dcc010af0 con 0x7f3dd8076220 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3dd6ffd640 1 -- 172.21.15.178:0/4013100210 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc017d10 con 0x7f3dd8076220 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3de017c640 1 -- 172.21.15.178:0/4013100210 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd81a2090 con 0x7f3dd8076220 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.611+0000 7f3de017c640 1 -- 172.21.15.178:0/4013100210 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd81a2650 con 0x7f3dd8076220 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.612+0000 7f3de017c640 1 -- 172.21.15.178:0/4013100210 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd8108800 con 0x7f3dd8076220 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.613+0000 7f3dd6ffd640 1 -- 172.21.15.178:0/4013100210 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f3dcc017070 con 0x7f3dd8076220 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.614+0000 7f3dd6ffd640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3dc00a2ba0 0x7f3dc00a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.614+0000 7f3dd6ffd640 1 -- 172.21.15.178:0/4013100210 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2029+0+0 (secure 0 0 0) 0x7f3dcc0cca60 con 0x7f3dd8076220 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.614+0000 7f3dddf18640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3dc00a2ba0 0x7f3dc00a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.615+0000 7f3dddf18640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3dc00a2ba0 0x7f3dc00a5060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8006010 tx=0x7f3dc8005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.618+0000 7f3dd6ffd640 1 -- 172.21.15.178:0/4013100210 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3dcc09b330 con 0x7f3dd8076220 2024-09-11T23:29:55.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:29:55.793+0000 7f3de017c640 1 -- 172.21.15.178:0/4013100210 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3dd8062630 con 0x7f3dc00a2ba0 2024-09-11T23:29:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:55 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi138 2024-09-11T23:29:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:55 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi138 to 17088M 2024-09-11T23:29:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:55 smithi178 ceph-mon[35882]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:29:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:55 smithi178 ceph-mon[35882]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:29:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:29:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:29:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:55 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi138 2024-09-11T23:29:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:55 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi138 to 17088M 2024-09-11T23:29:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:55 smithi138 ceph-mon[35142]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:29:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:55 smithi138 ceph-mon[35142]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:29:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:55 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:55 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:55 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:29:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:55 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:29:56.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:55 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:55 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi138 2024-09-11T23:29:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:55 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi138 to 17088M 2024-09-11T23:29:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:55 smithi133 ceph-mon[31430]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:29:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:55 smithi133 ceph-mon[31430]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:29:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:56.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:56.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:29:56.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:29:56.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:57.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:56 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:29:57.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:56 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:29:57.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:56 smithi178 ceph-mon[35882]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:57.261 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:56 smithi178 ceph-mon[35882]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:57.262 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:56 smithi178 ceph-mon[35882]: osd.1 [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926] boot 2024-09-11T23:29:57.262 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:56 smithi178 ceph-mon[35882]: osdmap e12: 2 total, 2 up, 2 in 2024-09-11T23:29:57.262 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:56 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:57.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:56 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:29:57.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:56 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:29:57.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:56 smithi138 ceph-mon[35142]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:57.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:56 smithi138 ceph-mon[35142]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:57.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:56 smithi138 ceph-mon[35142]: osd.1 [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926] boot 2024-09-11T23:29:57.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:56 smithi138 ceph-mon[35142]: osdmap e12: 2 total, 2 up, 2 in 2024-09-11T23:29:57.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:56 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:57.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:56 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:29:57.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:56 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:29:57.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:56 smithi133 ceph-mon[31430]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:29:57.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:56 smithi133 ceph-mon[31430]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:29:57.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:56 smithi133 ceph-mon[31430]: osd.1 [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926] boot 2024-09-11T23:29:57.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:56 smithi133 ceph-mon[31430]: osdmap e12: 2 total, 2 up, 2 in 2024-09-11T23:29:57.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:56 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:29:58.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:57 smithi178 ceph-mon[35882]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:29:58.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:57 smithi178 ceph-mon[35882]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:29:58.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:57 smithi178 ceph-mon[35882]: from='client.? 172.21.15.178:0/3387056257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ae0f14e-2263-4fd2-9848-7b23b8e2638c"}]: dispatch 2024-09-11T23:29:58.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:57 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ae0f14e-2263-4fd2-9848-7b23b8e2638c"}]: dispatch 2024-09-11T23:29:58.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:57 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ae0f14e-2263-4fd2-9848-7b23b8e2638c"}]': finished 2024-09-11T23:29:58.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:57 smithi178 ceph-mon[35882]: osdmap e14: 3 total, 2 up, 3 in 2024-09-11T23:29:58.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:57 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:29:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:57 smithi138 ceph-mon[35142]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:29:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:57 smithi138 ceph-mon[35142]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:29:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:57 smithi138 ceph-mon[35142]: from='client.? 172.21.15.178:0/3387056257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ae0f14e-2263-4fd2-9848-7b23b8e2638c"}]: dispatch 2024-09-11T23:29:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:57 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ae0f14e-2263-4fd2-9848-7b23b8e2638c"}]: dispatch 2024-09-11T23:29:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:57 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ae0f14e-2263-4fd2-9848-7b23b8e2638c"}]': finished 2024-09-11T23:29:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:57 smithi138 ceph-mon[35142]: osdmap e14: 3 total, 2 up, 3 in 2024-09-11T23:29:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:57 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:29:58.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:57 smithi133 ceph-mon[31430]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:29:58.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:57 smithi133 ceph-mon[31430]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:29:58.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:57 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/3387056257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ae0f14e-2263-4fd2-9848-7b23b8e2638c"}]: dispatch 2024-09-11T23:29:58.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:57 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ae0f14e-2263-4fd2-9848-7b23b8e2638c"}]: dispatch 2024-09-11T23:29:58.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:57 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ae0f14e-2263-4fd2-9848-7b23b8e2638c"}]': finished 2024-09-11T23:29:58.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:57 smithi133 ceph-mon[31430]: osdmap e14: 3 total, 2 up, 3 in 2024-09-11T23:29:58.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:57 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:29:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:29:59 smithi138 ceph-mon[35142]: from='client.? 172.21.15.178:0/2952320244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:29:59.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:29:59 smithi178 ceph-mon[35882]: from='client.? 172.21.15.178:0/2952320244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:29:59.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:29:58 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/2952320244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:30:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:00 smithi178 ceph-mon[35882]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:00 smithi138 ceph-mon[35142]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:00.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:00 smithi133 ceph-mon[31430]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:01 smithi178 ceph-mon[35882]: overall HEALTH_OK 2024-09-11T23:30:01.360 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:01 smithi133 ceph-mon[31430]: overall HEALTH_OK 2024-09-11T23:30:01.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:01 smithi138 ceph-mon[35142]: overall HEALTH_OK 2024-09-11T23:30:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:02 smithi178 ceph-mon[35882]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:02 smithi138 ceph-mon[35142]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:02.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:02 smithi133 ceph-mon[31430]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:04.963 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:04 smithi178 ceph-mon[35882]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:04 smithi138 ceph-mon[35142]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:05.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:04 smithi133 ceph-mon[31430]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:06 smithi178 ceph-mon[35882]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:06 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:30:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:06 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:06 smithi178 ceph-mon[35882]: Deploying daemon osd.2 on smithi178 2024-09-11T23:30:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:06 smithi138 ceph-mon[35142]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:06 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:30:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:06 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:06 smithi138 ceph-mon[35142]: Deploying daemon osd.2 on smithi178 2024-09-11T23:30:07.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:06 smithi133 ceph-mon[31430]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:07.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:06 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:30:07.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:06 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:07.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:06 smithi133 ceph-mon[31430]: Deploying daemon osd.2 on smithi178 2024-09-11T23:30:09.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:08 smithi178 ceph-mon[35882]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:08 smithi138 ceph-mon[35142]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:09.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:08 smithi133 ceph-mon[31430]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:10 smithi178 ceph-mon[35882]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:11.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:10 smithi138 ceph-mon[35142]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:11.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:10 smithi133 ceph-mon[31430]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:13.014 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:12 smithi178 ceph-mon[35882]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:13.015 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:12 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:13.015 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:12 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:13.015 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:12 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:12 smithi138 ceph-mon[35142]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:12 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:12 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:12 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:12 smithi133 ceph-mon[31430]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:12 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:12 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:12 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:14.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:14.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:14.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:14.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:14.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:14.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:14 smithi178 ceph-mon[35882]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:14 smithi138 ceph-mon[35142]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:14.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:14.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:14.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:14.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:14.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:14.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:14 smithi133 ceph-mon[31430]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:16.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.055+0000 7f3dd6ffd640 1 -- 172.21.15.178:0/4013100210 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3dd8062630 con 0x7f3dc00a2ba0 2024-09-11T23:30:16.056 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 2 on host 'smithi178' 2024-09-11T23:30:16.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 -- 172.21.15.178:0/4013100210 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3dc00a2ba0 msgr2=0x7f3dc00a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:16.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3dc00a2ba0 0x7f3dc00a5060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8006010 tx=0x7f3dc8005f80 comp rx=0 tx=0).stop 2024-09-11T23:30:16.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 -- 172.21.15.178:0/4013100210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd8076220 msgr2=0x7f3dd819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:16.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd8076220 0x7f3dd819a6b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00b4a0 tx=0x7f3dcc00b960 comp rx=0 tx=0).stop 2024-09-11T23:30:16.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 -- 172.21.15.178:0/4013100210 shutdown_connections 2024-09-11T23:30:16.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3dc00a2ba0 0x7f3dc00a5060 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:16.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dd8109a80 0x7f3dd81a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:16.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dd8076220 0x7f3dd819a6b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:16.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 --2- 172.21.15.178:0/4013100210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dd8074c50 0x7f3dd819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:16.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 -- 172.21.15.178:0/4013100210 >> 172.21.15.178:0/4013100210 conn(0x7f3dd80fd390 msgr2=0x7f3dd81059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:16.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 -- 172.21.15.178:0/4013100210 shutdown_connections 2024-09-11T23:30:16.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:30:16.058+0000 7f3de017c640 1 -- 172.21.15.178:0/4013100210 wait complete. 2024-09-11T23:30:16.671 DEBUG:teuthology.orchestra.run.smithi178:osd.2> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T23:30:16.673 INFO:tasks.cephadm:Deploying osd.3 on smithi133 with /dev/vg_nvme/lv_3... 2024-09-11T23:30:16.674 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:30:16.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:16 smithi178 ceph-mon[35882]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:16.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:16.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:16.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:16.932 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:30:16.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:16 smithi133 ceph-mon[31430]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:16.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:16.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:16.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:17.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:16 smithi138 ceph-mon[35142]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:17.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:17.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:17.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:18.540 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:30:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2[40845]: 2024-09-11T23:30:18.171+0000 7fc54f80d640 -1 osd.2 0 log_to_monitors true 2024-09-11T23:30:19.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:18 smithi178 ceph-mon[35882]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:19.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:18 smithi178 ceph-mon[35882]: from='osd.2 [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:30:19.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:18 smithi178 ceph-mon[35882]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:30:19.111 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:18 smithi133 ceph-mon[31430]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:19.111 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:18 smithi133 ceph-mon[31430]: from='osd.2 [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:30:19.111 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:18 smithi133 ceph-mon[31430]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:30:19.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:18 smithi138 ceph-mon[35142]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:19.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:18 smithi138 ceph-mon[35142]: from='osd.2 [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:30:19.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:18 smithi138 ceph-mon[35142]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:30:19.373 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:30:19.399 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_3 2024-09-11T23:30:19.655 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:30:19.954 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:30:19.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: from='osd.2 [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:30:19.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:30:19.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:19.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:30:19.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:19.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:19.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:30:19.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:19.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:19.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:19.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:20.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:30:20.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: from='osd.2 [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:30:20.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:30:20.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:20.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:30:20.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:20.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:20.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:30:20.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:20.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:20.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:20.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:20.041 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2[40845]: 2024-09-11T23:30:19.857+0000 7fc54af6d640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T23:30:20.042 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:30:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2[40845]: 2024-09-11T23:30:19.860+0000 7fc5475a7640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:30:20.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.154+0000 7f0bf2228640 1 -- 172.21.15.133:0/2670491216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bec104fd0 msgr2=0x7f0bec1073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:20.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.154+0000 7f0bf2228640 1 --2- 172.21.15.133:0/2670491216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bec104fd0 0x7f0bec1073c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009900 tx=0x7f0bd402a410 comp rx=0 tx=0).stop 2024-09-11T23:30:20.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.154+0000 7f0bf2228640 1 -- 172.21.15.133:0/2670491216 shutdown_connections 2024-09-11T23:30:20.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.154+0000 7f0bf2228640 1 --2- 172.21.15.133:0/2670491216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec107900 0x7f0bec109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:20.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.154+0000 7f0bf2228640 1 --2- 172.21.15.133:0/2670491216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bec104fd0 0x7f0bec1073c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:20.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.154+0000 7f0bf2228640 1 --2- 172.21.15.133:0/2670491216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bec1026a0 0x7f0bec104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:20.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.154+0000 7f0bf2228640 1 -- 172.21.15.133:0/2670491216 >> 172.21.15.133:0/2670491216 conn(0x7f0bec0fc2a0 msgr2=0x7f0bec0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:20.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.154+0000 7f0bf2228640 1 -- 172.21.15.133:0/2670491216 shutdown_connections 2024-09-11T23:30:20.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.154+0000 7f0bf2228640 1 -- 172.21.15.133:0/2670491216 wait complete. 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0bf2228640 1 Processor -- start 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0bf2228640 1 -- start start 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0bf2228640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bec1026a0 0x7f0bec197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0bf2228640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec104fd0 0x7f0bec198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0bf2228640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bec107900 0x7f0bec19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0bf2228640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec10d8a0 con 0x7f0bec1026a0 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0bf2228640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec10d720 con 0x7f0bec107900 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0bf2228640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec10da20 con 0x7f0bec104fd0 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0beb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bec1026a0 0x7f0bec197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0beb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bec1026a0 0x7f0bec197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60152/0 (socket says 172.21.15.133:60152) 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0beb7fe640 1 -- 172.21.15.133:0/849219873 learned_addr learned my addr 172.21.15.133:0/849219873 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:30:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0be3fff640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec104fd0 0x7f0bec198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0bebfff640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bec107900 0x7f0bec19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0beb7fe640 1 -- 172.21.15.133:0/849219873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec104fd0 msgr2=0x7f0bec198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0beb7fe640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec104fd0 0x7f0bec198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0beb7fe640 1 -- 172.21.15.133:0/849219873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bec107900 msgr2=0x7f0bec19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0beb7fe640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bec107900 0x7f0bec19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0beb7fe640 1 -- 172.21.15.133:0/849219873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bec19fb50 con 0x7f0bec1026a0 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0bebfff640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bec107900 0x7f0bec19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0be3fff640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec104fd0 0x7f0bec198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.155+0000 7f0beb7fe640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bec1026a0 0x7f0bec197f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00d950 tx=0x7f0bdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.156+0000 7f0be97fa640 1 -- 172.21.15.133:0/849219873 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc017030 con 0x7f0bec1026a0 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.156+0000 7f0be97fa640 1 -- 172.21.15.133:0/849219873 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bdc004020 con 0x7f0bec1026a0 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.156+0000 7f0bf2228640 1 -- 172.21.15.133:0/849219873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bec19fe40 con 0x7f0bec1026a0 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.156+0000 7f0be97fa640 1 -- 172.21.15.133:0/849219873 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc0047a0 con 0x7f0bec1026a0 2024-09-11T23:30:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.156+0000 7f0bf2228640 1 -- 172.21.15.133:0/849219873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bec1a02a0 con 0x7f0bec1026a0 2024-09-11T23:30:20.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.157+0000 7f0bf2228640 1 -- 172.21.15.133:0/849219873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bec0fdd90 con 0x7f0bec1026a0 2024-09-11T23:30:20.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.158+0000 7f0be97fa640 1 -- 172.21.15.133:0/849219873 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f0bdc0041d0 con 0x7f0bec1026a0 2024-09-11T23:30:20.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.158+0000 7f0be97fa640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0bd80a2ba0 0x7f0bd80a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:20.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.158+0000 7f0be3fff640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0bd80a2ba0 0x7f0bd80a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:20.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.159+0000 7f0be97fa640 1 -- 172.21.15.133:0/849219873 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2571+0+0 (secure 0 0 0) 0x7f0bdc0cd270 con 0x7f0bec1026a0 2024-09-11T23:30:20.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.159+0000 7f0be3fff640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0bd80a2ba0 0x7f0bd80a5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4005ab0 tx=0x7f0bd4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:20.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.163+0000 7f0be97fa640 1 -- 172.21.15.133:0/849219873 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bdc09b920 con 0x7f0bec1026a0 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: from='osd.2 [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:19 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:20.344+0000 7f0bf2228640 1 -- 172.21.15.133:0/849219873 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0bec062630 con 0x7f0bd80a2ba0 2024-09-11T23:30:20.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:20 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi178 2024-09-11T23:30:20.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:20 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi178 to 17088M 2024-09-11T23:30:20.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:20 smithi133 ceph-mon[31430]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:20.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:20 smithi133 ceph-mon[31430]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:30:20.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:20 smithi133 ceph-mon[31430]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:30:20.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:20.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:20.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:30:20.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:30:20.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:20.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:21.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:20 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi178 2024-09-11T23:30:21.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:20 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi178 to 17088M 2024-09-11T23:30:21.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:20 smithi178 ceph-mon[35882]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:21.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:20 smithi178 ceph-mon[35882]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:30:21.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:20 smithi178 ceph-mon[35882]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:30:21.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:21.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:21.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:30:21.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:30:21.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:21.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:20 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi178 2024-09-11T23:30:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:20 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi178 to 17088M 2024-09-11T23:30:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:20 smithi138 ceph-mon[35142]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:20 smithi138 ceph-mon[35142]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:30:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:20 smithi138 ceph-mon[35142]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:30:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:30:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:30:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:22.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:22 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:30:22.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:22 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:30:22.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:22 smithi133 ceph-mon[31430]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:30:22.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:22 smithi133 ceph-mon[31430]: osd.2 [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816] boot 2024-09-11T23:30:22.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:22 smithi133 ceph-mon[31430]: osdmap e17: 3 total, 3 up, 3 in 2024-09-11T23:30:22.327 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:22 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:22.327 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:22 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:30:22.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:22 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:30:22.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:22 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:30:22.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:22 smithi138 ceph-mon[35142]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:30:22.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:22 smithi138 ceph-mon[35142]: osd.2 [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816] boot 2024-09-11T23:30:22.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:22 smithi138 ceph-mon[35142]: osdmap e17: 3 total, 3 up, 3 in 2024-09-11T23:30:22.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:22 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:22.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:22 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:30:22.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:22 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:30:22.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:22 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:30:22.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:22 smithi178 ceph-mon[35882]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:30:22.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:22 smithi178 ceph-mon[35882]: osd.2 [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816] boot 2024-09-11T23:30:22.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:22 smithi178 ceph-mon[35882]: osdmap e17: 3 total, 3 up, 3 in 2024-09-11T23:30:22.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:22 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:30:22.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:22 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:30:23.298 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:30:23 smithi133 sudo[47177]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:30:23.299 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:30:23 smithi133 sudo[47177]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:23.299 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:30:23 smithi133 sudo[47177]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:23.299 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:30:23 smithi133 sudo[47177]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:23.299 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:23 smithi133 ceph-mon[31430]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:23.299 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:30:23.299 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:23 smithi133 ceph-mon[31430]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:30:23.299 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:30:23.299 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:23 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1718340973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2392e1a-af87-4af1-a362-f17ce49444bb"}]: dispatch 2024-09-11T23:30:23.299 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:30:23.299 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:23 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1718340973' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2392e1a-af87-4af1-a362-f17ce49444bb"}]': finished 2024-09-11T23:30:23.300 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:23 smithi133 ceph-mon[31430]: osdmap e19: 4 total, 3 up, 4 in 2024-09-11T23:30:23.300 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:23.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:23 smithi138 ceph-mon[35142]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:23.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:30:23.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:23 smithi138 ceph-mon[35142]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:30:23.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:30:23.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:23 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1718340973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2392e1a-af87-4af1-a362-f17ce49444bb"}]: dispatch 2024-09-11T23:30:23.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:30:23.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:23 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1718340973' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2392e1a-af87-4af1-a362-f17ce49444bb"}]': finished 2024-09-11T23:30:23.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:23 smithi138 ceph-mon[35142]: osdmap e19: 4 total, 3 up, 4 in 2024-09-11T23:30:23.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:23.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:23 smithi178 ceph-mon[35882]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:30:23.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:30:23.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:23 smithi178 ceph-mon[35882]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:30:23.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:30:23.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:23 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1718340973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2392e1a-af87-4af1-a362-f17ce49444bb"}]: dispatch 2024-09-11T23:30:23.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:30:23.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:23 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1718340973' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2392e1a-af87-4af1-a362-f17ce49444bb"}]': finished 2024-09-11T23:30:23.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:23 smithi178 ceph-mon[35882]: osdmap e19: 4 total, 3 up, 4 in 2024-09-11T23:30:23.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:23.709 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:30:23 smithi133 sudo[47188]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:30:23.709 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:30:23 smithi133 sudo[47188]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:23.709 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:30:23 smithi133 sudo[47188]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:23.709 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:30:23 smithi133 sudo[47188]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:23.889 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:30:23 smithi138 sudo[43334]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:30:23.889 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:30:23 smithi138 sudo[43334]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:23.889 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:30:23 smithi138 sudo[43334]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:23.889 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:30:23 smithi138 sudo[43334]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:24.198 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:30:23 smithi138 sudo[43337]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:30:24.198 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:30:23 smithi138 sudo[43337]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:24.198 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:30:23 smithi138 sudo[43337]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:24.198 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:30:23 smithi138 sudo[43337]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:24 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3199683768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:30:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:24 smithi138 ceph-mon[35142]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:24 smithi138 ceph-mon[35142]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T23:30:24.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:24.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:24.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:24 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3199683768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:30:24.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:24.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:24 smithi133 ceph-mon[31430]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:24.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:24 smithi133 ceph-mon[31430]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T23:30:24.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:24.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:24.482 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:24 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3199683768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:30:24.482 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:24.482 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:24 smithi178 ceph-mon[35882]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:24.482 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:24 smithi178 ceph-mon[35882]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T23:30:24.483 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:24.483 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:24.483 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:30:24 smithi178 sudo[44776]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:30:24.483 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:30:24 smithi178 sudo[44776]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:24.483 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:30:24 smithi178 sudo[44776]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:24.483 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:30:24 smithi178 sudo[44776]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:24.790 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:30:24 smithi178 sudo[44779]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:30:24.790 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:30:24 smithi178 sudo[44779]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:24.790 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:30:24 smithi178 sudo[44779]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:24.790 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:30:24 smithi178 sudo[44779]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:25.018 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:24 smithi133 sudo[47423]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:30:25.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:24 smithi133 sudo[47423]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:25.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:24 smithi133 sudo[47423]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:25.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:24 smithi133 sudo[47423]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:25.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:25.035+0000 7f0be97fa640 1 -- 172.21.15.133:0/849219873 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0bdc097890 con 0x7f0bec1026a0 2024-09-11T23:30:25.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 sudo[47610]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:30:25.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 sudo[47610]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:25.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 sudo[47610]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:25.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 sudo[47610]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:25.521 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 sudo[44782]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:30:25.522 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 sudo[44782]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:25.522 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 sudo[44782]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:25.522 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 sudo[44782]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:25.783 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:25.783 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 ceph-mon[35142]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:30:25.783 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 ceph-mon[35142]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:30:25.783 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:30:25.783 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:30:25.783 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:30:25.783 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 ceph-mon[35142]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-09-11T23:30:25.783 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:30:25.783 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:30:25.783 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:30:25.784 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 sudo[43340]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:30:25.784 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 sudo[43340]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:25.784 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 sudo[43340]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:25.784 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:25 smithi138 sudo[43340]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:25.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 sudo[44785]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:30:25.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 sudo[44785]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:25.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 sudo[44785]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:25.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 sudo[44785]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:25.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:25.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 ceph-mon[35882]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:30:25.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 ceph-mon[35882]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:30:25.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:30:25.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:30:25.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:30:25.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 ceph-mon[35882]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-09-11T23:30:25.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:30:25.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:30:25.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:30:25.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:25.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 ceph-mon[31430]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:30:25.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 ceph-mon[31430]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:30:25.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:30:25.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:30:25.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:30:25.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 ceph-mon[31430]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-09-11T23:30:25.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:30:25.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:30:25.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:30:26.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 sudo[43343]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:30:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 sudo[43343]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:30:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 sudo[43343]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:30:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 sudo[43343]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:30:26.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:26 smithi178 ceph-mon[35882]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:30:26.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:26 smithi178 ceph-mon[35882]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:30:26.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:26 smithi178 ceph-mon[35882]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:30:26.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:26 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:30:26.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:26 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:30:26.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:26 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:30:26.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:26 smithi178 ceph-mon[35882]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:26.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:26 smithi178 ceph-mon[35882]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:30:26.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 ceph-mon[35142]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:30:26.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 ceph-mon[35142]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:30:26.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 ceph-mon[35142]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:30:26.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:30:26.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:30:26.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:30:26.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 ceph-mon[35142]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:26.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:26 smithi138 ceph-mon[35142]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:30:26.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:26 smithi133 ceph-mon[31430]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:30:26.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:26 smithi133 ceph-mon[31430]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:30:26.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:26 smithi133 ceph-mon[31430]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:30:26.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:26 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:30:26.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:26 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:30:26.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:26 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:30:26.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:26 smithi133 ceph-mon[31430]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:26.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:26 smithi133 ceph-mon[31430]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:30:29.183 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:28 smithi133 ceph-mon[31430]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:28 smithi138 ceph-mon[35142]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:29.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:28 smithi178 ceph-mon[35882]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:31.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:30 smithi178 ceph-mon[35882]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:31.185 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:30 smithi133 ceph-mon[31430]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:31.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:30 smithi138 ceph-mon[35142]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:32.186 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:31 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:30:32.187 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:31 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:32.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:31 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:30:32.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:31 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:31 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:30:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:31 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:33.052 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:32 smithi133 ceph-mon[31430]: Deploying daemon osd.3 on smithi133 2024-09-11T23:30:33.052 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:32 smithi133 ceph-mon[31430]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:33.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:32 smithi138 ceph-mon[35142]: Deploying daemon osd.3 on smithi133 2024-09-11T23:30:33.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:32 smithi138 ceph-mon[35142]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:32 smithi178 ceph-mon[35882]: Deploying daemon osd.3 on smithi133 2024-09-11T23:30:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:32 smithi178 ceph-mon[35882]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:34 smithi138 ceph-mon[35142]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:34 smithi133 ceph-mon[31430]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:35.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:34 smithi178 ceph-mon[35882]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:37.046 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:36 smithi133 ceph-mon[31430]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:37.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:36 smithi138 ceph-mon[35142]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:36 smithi178 ceph-mon[35882]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:38.018 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:38.018 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:38.018 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:38.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:37 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:38.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:37 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:38.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:37 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:38.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:37 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:38.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:37 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:38.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:37 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:39.091 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:38 smithi133 ceph-mon[31430]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:38 smithi138 ceph-mon[35142]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:38 smithi178 ceph-mon[35882]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:40.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:40 smithi133 ceph-mon[31430]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:40.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:40.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:40.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:40.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:40.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:40 smithi138 ceph-mon[35142]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:40 smithi178 ceph-mon[35882]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:41.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.326+0000 7f0be97fa640 1 -- 172.21.15.133:0/849219873 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0bec062630 con 0x7f0bd80a2ba0 2024-09-11T23:30:41.326 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 3 on host 'smithi133' 2024-09-11T23:30:41.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 -- 172.21.15.133:0/849219873 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0bd80a2ba0 msgr2=0x7f0bd80a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:41.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0bd80a2ba0 0x7f0bd80a5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4005ab0 tx=0x7f0bd4000bc0 comp rx=0 tx=0).stop 2024-09-11T23:30:41.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 -- 172.21.15.133:0/849219873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bec1026a0 msgr2=0x7f0bec197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:41.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bec1026a0 0x7f0bec197f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00d950 tx=0x7f0bdc00de10 comp rx=0 tx=0).stop 2024-09-11T23:30:41.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 -- 172.21.15.133:0/849219873 shutdown_connections 2024-09-11T23:30:41.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0bd80a2ba0 0x7f0bd80a5060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:41.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bec107900 0x7f0bec19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:41.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bec104fd0 0x7f0bec198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 --2- 172.21.15.133:0/849219873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bec1026a0 0x7f0bec197f20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 -- 172.21.15.133:0/849219873 >> 172.21.15.133:0/849219873 conn(0x7f0bec0fc2a0 msgr2=0x7f0bec1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 -- 172.21.15.133:0/849219873 shutdown_connections 2024-09-11T23:30:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:30:41.328+0000 7f0bf2228640 1 -- 172.21.15.133:0/849219873 wait complete. 2024-09-11T23:30:41.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:41 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:41.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:41 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:41.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:41 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:42.029 DEBUG:teuthology.orchestra.run.smithi133:osd.3> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T23:30:42.031 INFO:tasks.cephadm:Deploying osd.4 on smithi138 with /dev/vg_nvme/lv_3... 2024-09-11T23:30:42.032 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:30:42.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:41 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:42.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:41 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:42.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:41 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:42.286 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:30:42.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:41 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:41 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:41 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:43.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:42 smithi138 ceph-mon[35142]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:43.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:42 smithi133 ceph-mon[31430]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:43.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:42 smithi178 ceph-mon[35882]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:43.542 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:30:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3[49209]: 2024-09-11T23:30:43.404+0000 7fe874c19640 -1 osd.3 0 log_to_monitors true 2024-09-11T23:30:44.127 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:43 smithi133 ceph-mon[31430]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:30:44.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:43 smithi138 ceph-mon[35142]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:30:44.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:43 smithi178 ceph-mon[35882]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:30:44.616 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:30:44.642 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-09-11T23:30:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:30:45.154 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:44 smithi138 ceph-mon[35142]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:45.154 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:44 smithi138 ceph-mon[35142]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:30:45.154 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:44 smithi138 ceph-mon[35142]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T23:30:45.154 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:44 smithi138 ceph-mon[35142]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:30:45.154 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:44 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:45.209 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:30:44 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3[49209]: 2024-09-11T23:30:44.872+0000 7fe870379640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T23:30:45.209 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:30:44 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3[49209]: 2024-09-11T23:30:44.883+0000 7fe86c9b3640 -1 osd.3 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:30:45.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:44 smithi133 ceph-mon[31430]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:45.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:44 smithi133 ceph-mon[31430]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:30:45.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:44 smithi133 ceph-mon[31430]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T23:30:45.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:44 smithi133 ceph-mon[31430]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:30:45.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:44 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:44 smithi178 ceph-mon[35882]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:44 smithi178 ceph-mon[35882]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:30:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:44 smithi178 ceph-mon[35882]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T23:30:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:44 smithi178 ceph-mon[35882]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:30:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:44 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:45.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.358+0000 7f3813b80640 1 -- 172.21.15.138:0/1594050153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f380c102e00 msgr2=0x7f380c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:45.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.358+0000 7f3813b80640 1 --2- 172.21.15.138:0/1594050153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f380c102e00 0x7f380c103280 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3808009900 tx=0x7f380802a410 comp rx=0 tx=0).stop 2024-09-11T23:30:45.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.358+0000 7f3813b80640 1 -- 172.21.15.138:0/1594050153 shutdown_connections 2024-09-11T23:30:45.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.358+0000 7f3813b80640 1 --2- 172.21.15.138:0/1594050153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c1037c0 0x7f380c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:45.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.358+0000 7f3813b80640 1 --2- 172.21.15.138:0/1594050153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f380c102e00 0x7f380c103280 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:45.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.358+0000 7f3813b80640 1 --2- 172.21.15.138:0/1594050153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c101c00 0x7f380c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:45.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.358+0000 7f3813b80640 1 -- 172.21.15.138:0/1594050153 >> 172.21.15.138:0/1594050153 conn(0x7f380c0fd390 msgr2=0x7f380c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:45.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.358+0000 7f3813b80640 1 -- 172.21.15.138:0/1594050153 shutdown_connections 2024-09-11T23:30:45.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.358+0000 7f3813b80640 1 -- 172.21.15.138:0/1594050153 wait complete. 2024-09-11T23:30:45.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f3813b80640 1 Processor -- start 2024-09-11T23:30:45.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f3813b80640 1 -- start start 2024-09-11T23:30:45.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f3813b80640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c101c00 0x7f380c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:45.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f3813b80640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c102e00 0x7f380c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:45.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f3813b80640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f380c1037c0 0x7f380c19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:45.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f3813b80640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f380c10d860 con 0x7f380c101c00 2024-09-11T23:30:45.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f3813b80640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f380c10d6e0 con 0x7f380c1037c0 2024-09-11T23:30:45.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f3813b80640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f380c10d9e0 con 0x7f380c102e00 2024-09-11T23:30:45.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f381211d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f380c1037c0 0x7f380c19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:45.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f381211d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f380c1037c0 0x7f380c19f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54280/0 (socket says 172.21.15.138:54280) 2024-09-11T23:30:45.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f381211d640 1 -- 172.21.15.138:0/1400501037 learned_addr learned my addr 172.21.15.138:0/1400501037 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-11T23:30:45.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f381111b640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c102e00 0x7f380c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:45.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.359+0000 7f381191c640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c101c00 0x7f380c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.360+0000 7f381211d640 1 -- 172.21.15.138:0/1400501037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c102e00 msgr2=0x7f380c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.360+0000 7f381211d640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c102e00 0x7f380c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.360+0000 7f381211d640 1 -- 172.21.15.138:0/1400501037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c101c00 msgr2=0x7f380c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.360+0000 7f381211d640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c101c00 0x7f380c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.360+0000 7f381211d640 1 -- 172.21.15.138:0/1400501037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c19fad0 con 0x7f380c1037c0 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.360+0000 7f381211d640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f380c1037c0 0x7f380c19f460 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f380400eaa0 tx=0x7f380400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.360+0000 7f3802ffd640 1 -- 172.21.15.138:0/1400501037 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380400cdb0 con 0x7f380c1037c0 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.360+0000 7f3813b80640 1 -- 172.21.15.138:0/1400501037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c19fdc0 con 0x7f380c1037c0 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.360+0000 7f3802ffd640 1 -- 172.21.15.138:0/1400501037 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3804005020 con 0x7f380c1037c0 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.360+0000 7f3802ffd640 1 -- 172.21.15.138:0/1400501037 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38040116b0 con 0x7f380c1037c0 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.360+0000 7f3813b80640 1 -- 172.21.15.138:0/1400501037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c1a0270 con 0x7f380c1037c0 2024-09-11T23:30:45.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.361+0000 7f3813b80640 1 -- 172.21.15.138:0/1400501037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380c108fa0 con 0x7f380c1037c0 2024-09-11T23:30:45.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.362+0000 7f3802ffd640 1 -- 172.21.15.138:0/1400501037 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3804005560 con 0x7f380c1037c0 2024-09-11T23:30:45.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.363+0000 7f3802ffd640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f37f80a2c30 0x7f37f80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:45.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.363+0000 7f3802ffd640 1 -- 172.21.15.138:0/1400501037 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3389+0+0 (secure 0 0 0) 0x7f38040ce1c0 con 0x7f380c1037c0 2024-09-11T23:30:45.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.363+0000 7f381191c640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f37f80a2c30 0x7f37f80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:45.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.363+0000 7f381191c640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f37f80a2c30 0x7f37f80a50f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f380c102c60 tx=0x7f37fc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:45.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.366+0000 7f3802ffd640 1 -- 172.21.15.138:0/1400501037 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3804016030 con 0x7f380c1037c0 2024-09-11T23:30:45.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:30:45.541+0000 7f3813b80640 1 -- 172.21.15.138:0/1400501037 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f380c062630 con 0x7f37f80a2c30 2024-09-11T23:30:46.186 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:30:46.186 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:30:46.186 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:46.186 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:46.186 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi133 2024-09-11T23:30:46.186 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi133 to 8544M 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874] boot 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: osdmap e23: 4 total, 4 up, 4 in 2024-09-11T23:30:46.187 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:46.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:30:46.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi133 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi133 to 8544M 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:30:46.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:46.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874] boot 2024-09-11T23:30:46.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: osdmap e23: 4 total, 4 up, 4 in 2024-09-11T23:30:46.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:46.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:30:46.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:30:46.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi133 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi133 to 8544M 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:46.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: osd.3 [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874] boot 2024-09-11T23:30:46.292 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: osdmap e23: 4 total, 4 up, 4 in 2024-09-11T23:30:46.292 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:30:47.032 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:46 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:30:47.033 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:46 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:30:47.033 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:46 smithi138 ceph-mon[35142]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:30:47.033 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:46 smithi138 ceph-mon[35142]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:47.033 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:46 smithi138 ceph-mon[35142]: osdmap e24: 4 total, 4 up, 4 in 2024-09-11T23:30:47.191 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:46 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:30:47.191 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:46 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:30:47.191 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:46 smithi178 ceph-mon[35882]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:30:47.192 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:46 smithi178 ceph-mon[35882]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:47.192 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:46 smithi178 ceph-mon[35882]: osdmap e24: 4 total, 4 up, 4 in 2024-09-11T23:30:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:46 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:30:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:46 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:30:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:46 smithi133 ceph-mon[31430]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:30:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:46 smithi133 ceph-mon[31430]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:30:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:46 smithi133 ceph-mon[31430]: osdmap e24: 4 total, 4 up, 4 in 2024-09-11T23:30:48.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:47 smithi138 ceph-mon[35142]: from='client.? 172.21.15.138:0/1717002973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5414771-6d7e-4ea1-9b0a-50175189b9a9"}]: dispatch 2024-09-11T23:30:48.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:47 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5414771-6d7e-4ea1-9b0a-50175189b9a9"}]: dispatch 2024-09-11T23:30:48.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:47 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5414771-6d7e-4ea1-9b0a-50175189b9a9"}]': finished 2024-09-11T23:30:48.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:47 smithi138 ceph-mon[35142]: osdmap e25: 5 total, 4 up, 5 in 2024-09-11T23:30:48.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:47 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:30:48.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:47 smithi133 ceph-mon[31430]: from='client.? 172.21.15.138:0/1717002973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5414771-6d7e-4ea1-9b0a-50175189b9a9"}]: dispatch 2024-09-11T23:30:48.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:47 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5414771-6d7e-4ea1-9b0a-50175189b9a9"}]: dispatch 2024-09-11T23:30:48.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:47 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5414771-6d7e-4ea1-9b0a-50175189b9a9"}]': finished 2024-09-11T23:30:48.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:47 smithi133 ceph-mon[31430]: osdmap e25: 5 total, 4 up, 5 in 2024-09-11T23:30:48.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:47 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:30:48.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:47 smithi178 ceph-mon[35882]: from='client.? 172.21.15.138:0/1717002973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5414771-6d7e-4ea1-9b0a-50175189b9a9"}]: dispatch 2024-09-11T23:30:48.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:47 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5414771-6d7e-4ea1-9b0a-50175189b9a9"}]: dispatch 2024-09-11T23:30:48.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:47 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5414771-6d7e-4ea1-9b0a-50175189b9a9"}]': finished 2024-09-11T23:30:48.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:47 smithi178 ceph-mon[35882]: osdmap e25: 5 total, 4 up, 5 in 2024-09-11T23:30:48.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:47 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:30:49.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:48 smithi138 ceph-mon[35142]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:30:49.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:48 smithi138 ceph-mon[35142]: from='client.? 172.21.15.138:0/4207928300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:30:49.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:48 smithi138 ceph-mon[35142]: osdmap e26: 5 total, 4 up, 5 in 2024-09-11T23:30:49.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:30:49.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:48 smithi133 ceph-mon[31430]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:30:49.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:48 smithi133 ceph-mon[31430]: from='client.? 172.21.15.138:0/4207928300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:30:49.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:48 smithi133 ceph-mon[31430]: osdmap e26: 5 total, 4 up, 5 in 2024-09-11T23:30:49.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:30:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:48 smithi178 ceph-mon[35882]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:30:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:48 smithi178 ceph-mon[35882]: from='client.? 172.21.15.138:0/4207928300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:30:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:48 smithi178 ceph-mon[35882]: osdmap e26: 5 total, 4 up, 5 in 2024-09-11T23:30:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:30:50.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:50 smithi138 ceph-mon[35142]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T23:30:50.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:30:50.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:50 smithi138 ceph-mon[35142]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:30:50.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:50 smithi133 ceph-mon[31430]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T23:30:50.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:30:50.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:50 smithi133 ceph-mon[31430]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:30:50.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:50 smithi178 ceph-mon[35882]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T23:30:50.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:30:50.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:50 smithi178 ceph-mon[35882]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:30:53.141 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:52 smithi138 ceph-mon[35142]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:30:53.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:52 smithi133 ceph-mon[31430]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:30:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:52 smithi178 ceph-mon[35882]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:30:55.030 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:54 smithi138 ceph-mon[35142]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-11T23:30:55.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:54 smithi133 ceph-mon[31430]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-11T23:30:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:54 smithi178 ceph-mon[35882]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-11T23:30:56.889 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:56 smithi138 ceph-mon[35142]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:30:56.889 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:56 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:30:56.889 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:56 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:57.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:56 smithi133 ceph-mon[31430]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:30:57.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:56 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:30:57.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:56 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:57.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:56 smithi178 ceph-mon[35882]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:30:57.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:56 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:30:57.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:56 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:58.068 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:57 smithi138 ceph-mon[35142]: Deploying daemon osd.4 on smithi138 2024-09-11T23:30:58.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:57 smithi133 ceph-mon[31430]: Deploying daemon osd.4 on smithi138 2024-09-11T23:30:58.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:57 smithi178 ceph-mon[35882]: Deploying daemon osd.4 on smithi138 2024-09-11T23:30:59.069 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:30:58 smithi138 ceph-mon[35142]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T23:30:59.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:30:58 smithi133 ceph-mon[31430]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T23:30:59.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:30:58 smithi178 ceph-mon[35882]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T23:31:01.112 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:00 smithi138 ceph-mon[35142]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T23:31:01.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:00 smithi133 ceph-mon[31430]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T23:31:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:00 smithi178 ceph-mon[35882]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T23:31:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:02 smithi138 ceph-mon[35142]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:03.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:02 smithi133 ceph-mon[31430]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:03.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:03.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:03.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:02 smithi178 ceph-mon[35882]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:04 smithi138 ceph-mon[35142]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:04 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:04 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:04 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:04 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:04 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:05.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:04 smithi133 ceph-mon[31430]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:05.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:04 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:05.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:04 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:05.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:04 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:05.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:04 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:05.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:04 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:04 smithi178 ceph-mon[35882]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:04 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:04 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:04 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:04 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:04 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:06.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.158+0000 7f3802ffd640 1 -- 172.21.15.138:0/1400501037 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f380c062630 con 0x7f37f80a2c30 2024-09-11T23:31:06.158 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 4 on host 'smithi138' 2024-09-11T23:31:06.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 -- 172.21.15.138:0/1400501037 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f37f80a2c30 msgr2=0x7f37f80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:06.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f37f80a2c30 0x7f37f80a50f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f380c102c60 tx=0x7f37fc00b390 comp rx=0 tx=0).stop 2024-09-11T23:31:06.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 -- 172.21.15.138:0/1400501037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f380c1037c0 msgr2=0x7f380c19f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:06.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f380c1037c0 0x7f380c19f460 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f380400eaa0 tx=0x7f380400ef60 comp rx=0 tx=0).stop 2024-09-11T23:31:06.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 -- 172.21.15.138:0/1400501037 shutdown_connections 2024-09-11T23:31:06.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f37f80a2c30 0x7f37f80a50f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:06.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f380c1037c0 0x7f380c19f460 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:06.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c102e00 0x7f380c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:06.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 --2- 172.21.15.138:0/1400501037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c101c00 0x7f380c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:06.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 -- 172.21.15.138:0/1400501037 >> 172.21.15.138:0/1400501037 conn(0x7f380c0fd390 msgr2=0x7f380c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:06.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 -- 172.21.15.138:0/1400501037 shutdown_connections 2024-09-11T23:31:06.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:06.160+0000 7f3813b80640 1 -- 172.21.15.138:0/1400501037 wait complete. 2024-09-11T23:31:06.741 DEBUG:teuthology.orchestra.run.smithi138:osd.4> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T23:31:06.744 INFO:tasks.cephadm:Deploying osd.5 on smithi178 with /dev/vg_nvme/lv_3... 2024-09-11T23:31:06.744 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:31:06.997 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:31:07.015 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:06 smithi138 ceph-mon[35142]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:07.015 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:06 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:07.015 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:06 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:07.015 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:06 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:06 smithi178 ceph-mon[35882]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:06 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:06 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:06 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:07.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:06 smithi133 ceph-mon[31430]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:07.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:06 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:07.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:06 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:07.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:06 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:08.698 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:31:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4[46102]: 2024-09-11T23:31:08.376+0000 7fa82b99d640 -1 osd.4 0 log_to_monitors true 2024-09-11T23:31:09.018 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:08 smithi138 ceph-mon[35142]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:09.019 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:08 smithi138 ceph-mon[35142]: from='osd.4 [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:31:09.019 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:08 smithi138 ceph-mon[35142]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:31:09.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:08 smithi133 ceph-mon[31430]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:09.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:08 smithi133 ceph-mon[31430]: from='osd.4 [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:31:09.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:08 smithi133 ceph-mon[31430]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:31:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:08 smithi178 ceph-mon[35882]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:08 smithi178 ceph-mon[35882]: from='osd.4 [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:31:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:08 smithi178 ceph-mon[35882]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:31:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:31:09.538 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2024-09-11T23:31:09.789 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:31:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:31:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='osd.4 [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:31:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T23:31:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:31:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:10.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:10.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:10.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:10.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:09 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:31:10.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='osd.4 [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:31:10.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T23:31:10.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:10.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:31:10.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:10.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:10.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:10.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:10.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:09 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:31:10.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='osd.4 [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:31:10.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T23:31:10.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:10.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:31:10.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:10.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:10.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:10.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:10.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:09 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:10.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.253+0000 7f82bd6d0640 1 -- 172.21.15.178:0/851568167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82b8105010 msgr2=0x7f82b8107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:10.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.253+0000 7f82bd6d0640 1 --2- 172.21.15.178:0/851568167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82b8105010 0x7f82b8107400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009900 tx=0x7f82a802a410 comp rx=0 tx=0).stop 2024-09-11T23:31:10.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.253+0000 7f82bd6d0640 1 -- 172.21.15.178:0/851568167 shutdown_connections 2024-09-11T23:31:10.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.253+0000 7f82bd6d0640 1 --2- 172.21.15.178:0/851568167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b8107940 0x7f82b8109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:10.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.253+0000 7f82bd6d0640 1 --2- 172.21.15.178:0/851568167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82b8105010 0x7f82b8107400 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:10.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.253+0000 7f82bd6d0640 1 --2- 172.21.15.178:0/851568167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b81026e0 0x7f82b8104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:10.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.253+0000 7f82bd6d0640 1 -- 172.21.15.178:0/851568167 >> 172.21.15.178:0/851568167 conn(0x7f82b80fc280 msgr2=0x7f82b80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:10.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.253+0000 7f82bd6d0640 1 -- 172.21.15.178:0/851568167 shutdown_connections 2024-09-11T23:31:10.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.253+0000 7f82bd6d0640 1 -- 172.21.15.178:0/851568167 wait complete. 2024-09-11T23:31:10.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.253+0000 7f82bd6d0640 1 Processor -- start 2024-09-11T23:31:10.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.253+0000 7f82bd6d0640 1 -- start start 2024-09-11T23:31:10.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82bd6d0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b81026e0 0x7f82b8102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:10.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82bd6d0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8105010 0x7f82b8100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:10.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82bd6d0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82b8107940 0x7f82b8100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:10.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82bd6d0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82b810d870 con 0x7f82b8105010 2024-09-11T23:31:10.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82bd6d0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82b810d6f0 con 0x7f82b81026e0 2024-09-11T23:31:10.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82bd6d0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82b810d9f0 con 0x7f82b8107940 2024-09-11T23:31:10.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b81026e0 0x7f82b8102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:10.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b77fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82b8107940 0x7f82b8100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:10.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b81026e0 0x7f82b8102040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.178:36540/0 (socket says 172.21.15.178:36540) 2024-09-11T23:31:10.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b6ffd640 1 -- 172.21.15.178:0/2419231107 learned_addr learned my addr 172.21.15.178:0/2419231107 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:31:10.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b67fc640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8105010 0x7f82b8100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:10.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b6ffd640 1 -- 172.21.15.178:0/2419231107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82b8107940 msgr2=0x7f82b8100bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:10.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b6ffd640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82b8107940 0x7f82b8100bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:10.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b6ffd640 1 -- 172.21.15.178:0/2419231107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8105010 msgr2=0x7f82b8100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:10.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b6ffd640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8105010 0x7f82b8100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:10.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b6ffd640 1 -- 172.21.15.178:0/2419231107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b8101490 con 0x7f82b81026e0 2024-09-11T23:31:10.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b77fe640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82b8107940 0x7f82b8100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:31:10.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b67fc640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8105010 0x7f82b8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:10.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.254+0000 7f82b6ffd640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b81026e0 0x7f82b8102040 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00d950 tx=0x7f82ac00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:10.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.255+0000 7f829ffff640 1 -- 172.21.15.178:0/2419231107 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac017070 con 0x7f82b81026e0 2024-09-11T23:31:10.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.255+0000 7f82bd6d0640 1 -- 172.21.15.178:0/2419231107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b81a20e0 con 0x7f82b81026e0 2024-09-11T23:31:10.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.255+0000 7f82bd6d0640 1 -- 172.21.15.178:0/2419231107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b81a2640 con 0x7f82b81026e0 2024-09-11T23:31:10.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.255+0000 7f829ffff640 1 -- 172.21.15.178:0/2419231107 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ac009c30 con 0x7f82b81026e0 2024-09-11T23:31:10.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.255+0000 7f829ffff640 1 -- 172.21.15.178:0/2419231107 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac020060 con 0x7f82b81026e0 2024-09-11T23:31:10.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.256+0000 7f82bd6d0640 1 -- 172.21.15.178:0/2419231107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a40052c0 con 0x7f82b81026e0 2024-09-11T23:31:10.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.257+0000 7f829ffff640 1 -- 172.21.15.178:0/2419231107 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f82ac005030 con 0x7f82b81026e0 2024-09-11T23:31:10.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.258+0000 7f829ffff640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f82a00a2c00 0x7f82a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:10.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.258+0000 7f829ffff640 1 -- 172.21.15.178:0/2419231107 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3821+0+0 (secure 0 0 0) 0x7f82ac0cda00 con 0x7f82b81026e0 2024-09-11T23:31:10.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.258+0000 7f82b67fc640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f82a00a2c00 0x7f82a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:10.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.259+0000 7f82b67fc640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f82a00a2c00 0x7f82a00a50c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f82a8006010 tx=0x7f82a8032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:10.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.264+0000 7f829ffff640 1 -- 172.21.15.178:0/2419231107 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82ac09bad0 con 0x7f82b81026e0 2024-09-11T23:31:10.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:10.445+0000 7f82bd6d0640 1 -- 172.21.15.178:0/2419231107 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f82a4000bf0 con 0x7f82a00a2c00 2024-09-11T23:31:10.948 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4[46102]: 2024-09-11T23:31:10.649+0000 7fa8270fd640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T23:31:10.948 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4[46102]: 2024-09-11T23:31:10.655+0000 7fa81f72f640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:31:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi138 2024-09-11T23:31:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi138 to 8544M 2024-09-11T23:31:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-mon[35142]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-mon[35142]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:31:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-mon[35142]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T23:31:10.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:10.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:10.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:31:10.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:31:10.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:10.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:10 smithi138 ceph-mon[35142]: from='osd.4 ' entity='osd.4' 2024-09-11T23:31:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:10 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi138 2024-09-11T23:31:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:10 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi138 to 8544M 2024-09-11T23:31:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:10 smithi178 ceph-mon[35882]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:10 smithi178 ceph-mon[35882]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:31:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:10 smithi178 ceph-mon[35882]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T23:31:11.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:10 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:11.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:10 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:11.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:10 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:31:11.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:10 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:31:11.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:10 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:11.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:10 smithi178 ceph-mon[35882]: from='osd.4 ' entity='osd.4' 2024-09-11T23:31:11.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:10 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi138 2024-09-11T23:31:11.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:10 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi138 to 8544M 2024-09-11T23:31:11.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:10 smithi133 ceph-mon[31430]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:11.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:10 smithi133 ceph-mon[31430]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:31:11.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:10 smithi133 ceph-mon[31430]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T23:31:11.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:10 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:11.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:10 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:11.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:10 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:31:11.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:10 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:31:11.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:10 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:11.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:10 smithi133 ceph-mon[31430]: from='osd.4 ' entity='osd.4' 2024-09-11T23:31:12.010 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:11 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:31:12.010 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:11 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:31:12.010 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:11 smithi178 ceph-mon[35882]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:31:12.011 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:12.011 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:11 smithi178 ceph-mon[35882]: osd.4 [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734] boot 2024-09-11T23:31:12.011 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:11 smithi178 ceph-mon[35882]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T23:31:12.011 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:11 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:31:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:11 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:31:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:11 smithi138 ceph-mon[35142]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:31:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:11 smithi138 ceph-mon[35142]: osd.4 [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734] boot 2024-09-11T23:31:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:11 smithi138 ceph-mon[35142]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T23:31:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:12.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:11 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:31:12.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:11 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:31:12.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:11 smithi133 ceph-mon[31430]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:31:12.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:12.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:11 smithi133 ceph-mon[31430]: osd.4 [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734] boot 2024-09-11T23:31:12.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:11 smithi133 ceph-mon[31430]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T23:31:12.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:31:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:12 smithi138 ceph-mon[35142]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:12 smithi138 ceph-mon[35142]: from='client.? 172.21.15.178:0/3651115166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d1bd3f9-6661-405e-ad49-71be51a3034f"}]: dispatch 2024-09-11T23:31:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:12 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d1bd3f9-6661-405e-ad49-71be51a3034f"}]: dispatch 2024-09-11T23:31:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:12 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d1bd3f9-6661-405e-ad49-71be51a3034f"}]': finished 2024-09-11T23:31:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:12 smithi138 ceph-mon[35142]: osdmap e31: 6 total, 5 up, 6 in 2024-09-11T23:31:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:12 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:13.203 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:12 smithi178 ceph-mon[35882]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:13.203 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:12 smithi178 ceph-mon[35882]: from='client.? 172.21.15.178:0/3651115166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d1bd3f9-6661-405e-ad49-71be51a3034f"}]: dispatch 2024-09-11T23:31:13.203 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:12 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d1bd3f9-6661-405e-ad49-71be51a3034f"}]: dispatch 2024-09-11T23:31:13.203 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:12 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d1bd3f9-6661-405e-ad49-71be51a3034f"}]': finished 2024-09-11T23:31:13.203 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:12 smithi178 ceph-mon[35882]: osdmap e31: 6 total, 5 up, 6 in 2024-09-11T23:31:13.203 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:12 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:12 smithi133 ceph-mon[31430]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:31:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:12 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/3651115166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d1bd3f9-6661-405e-ad49-71be51a3034f"}]: dispatch 2024-09-11T23:31:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:12 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d1bd3f9-6661-405e-ad49-71be51a3034f"}]: dispatch 2024-09-11T23:31:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:12 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d1bd3f9-6661-405e-ad49-71be51a3034f"}]': finished 2024-09-11T23:31:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:12 smithi133 ceph-mon[31430]: osdmap e31: 6 total, 5 up, 6 in 2024-09-11T23:31:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:12 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:14.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:13 smithi138 ceph-mon[35142]: from='client.? 172.21.15.178:0/2792449393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:31:14.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:13 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/2792449393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:31:14.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:13 smithi178 ceph-mon[35882]: from='client.? 172.21.15.178:0/2792449393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:31:15.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:14 smithi138 ceph-mon[35142]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:15.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:14 smithi133 ceph-mon[31430]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:15.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:14 smithi178 ceph-mon[35882]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:17.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:16 smithi138 ceph-mon[35142]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:16 smithi133 ceph-mon[31430]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:17.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:16 smithi178 ceph-mon[35882]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:19.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:18 smithi138 ceph-mon[35142]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:19.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:18 smithi133 ceph-mon[31430]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:19.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:18 smithi178 ceph-mon[35882]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:20 smithi138 ceph-mon[35142]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:21.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:20 smithi133 ceph-mon[31430]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:21.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:20 smithi178 ceph-mon[35882]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:22.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:21 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:31:22.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:21 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:22.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:21 smithi178 ceph-mon[35882]: Deploying daemon osd.5 on smithi178 2024-09-11T23:31:22.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:21 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:31:22.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:21 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:22.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:21 smithi138 ceph-mon[35142]: Deploying daemon osd.5 on smithi178 2024-09-11T23:31:22.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:21 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:31:22.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:21 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:22.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:21 smithi133 ceph-mon[31430]: Deploying daemon osd.5 on smithi178 2024-09-11T23:31:23.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:22 smithi138 ceph-mon[35142]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:22 smithi133 ceph-mon[31430]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:22 smithi178 ceph-mon[35882]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:24 smithi138 ceph-mon[35142]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:24 smithi133 ceph-mon[31430]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:25.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:24 smithi178 ceph-mon[35882]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:27.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:26 smithi178 ceph-mon[35882]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:26 smithi138 ceph-mon[35142]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:27.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:26 smithi133 ceph-mon[31430]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:28.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:27 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:28.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:27 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:28.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:27 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:28.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:27 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:28.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:27 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:28.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:27 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:28.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:27 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:28.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:27 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:28.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:27 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:29.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:28 smithi178 ceph-mon[35882]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:28 smithi138 ceph-mon[35142]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:28 smithi133 ceph-mon[31430]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:30.852 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:30.852 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:30.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:30.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:30.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:30.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:30 smithi178 ceph-mon[35882]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.893+0000 7f829ffff640 1 -- 172.21.15.178:0/2419231107 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f82a4000bf0 con 0x7f82a00a2c00 2024-09-11T23:31:30.893 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 5 on host 'smithi178' 2024-09-11T23:31:30.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 -- 172.21.15.178:0/2419231107 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f82a00a2c00 msgr2=0x7f82a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:30.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f82a00a2c00 0x7f82a00a50c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f82a8006010 tx=0x7f82a8032cf0 comp rx=0 tx=0).stop 2024-09-11T23:31:30.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 -- 172.21.15.178:0/2419231107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b81026e0 msgr2=0x7f82b8102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:30.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b81026e0 0x7f82b8102040 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00d950 tx=0x7f82ac00de10 comp rx=0 tx=0).stop 2024-09-11T23:31:30.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 -- 172.21.15.178:0/2419231107 shutdown_connections 2024-09-11T23:31:30.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f82a00a2c00 0x7f82a00a50c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:30.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82b8107940 0x7f82b8100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:30.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8105010 0x7f82b8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:30.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 --2- 172.21.15.178:0/2419231107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b81026e0 0x7f82b8102040 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:30.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 -- 172.21.15.178:0/2419231107 >> 172.21.15.178:0/2419231107 conn(0x7f82b80fc280 msgr2=0x7f82b8103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:30.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 -- 172.21.15.178:0/2419231107 shutdown_connections 2024-09-11T23:31:30.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:31:30.895+0000 7f82bd6d0640 1 -- 172.21.15.178:0/2419231107 wait complete. 2024-09-11T23:31:30.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:30.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:30.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:30.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:30.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:30.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:30 smithi138 ceph-mon[35142]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:30.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:30.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:30.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:30.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:30.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:30.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:30 smithi133 ceph-mon[31430]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:31.541 DEBUG:teuthology.orchestra.run.smithi178:osd.5> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T23:31:31.543 INFO:tasks.cephadm:Deploying osd.6 on smithi133 with /dev/vg_nvme/lv_2... 2024-09-11T23:31:31.544 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:31:31.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:31 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:31.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:31 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:31.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:31 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:31.801 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:31.860 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:31 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:31.861 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:31 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:31.861 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:31 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:31.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:31 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:31.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:31 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:31.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:31 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:32.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:32 smithi178 ceph-mon[35882]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:32.856 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:32 smithi133 ceph-mon[31430]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:32.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:32 smithi138 ceph-mon[35142]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:33.291 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:31:33 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:31:33.079+0000 7f0d14bbf640 -1 osd.5 0 log_to_monitors true 2024-09-11T23:31:33.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:33 smithi178 ceph-mon[35882]: from='osd.5 [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:31:33.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:33 smithi178 ceph-mon[35882]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:31:33.937 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:33 smithi133 ceph-mon[31430]: from='osd.5 [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:31:33.938 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:33 smithi133 ceph-mon[31430]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:31:33.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:33 smithi138 ceph-mon[35142]: from='osd.5 [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:31:33.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:33 smithi138 ceph-mon[35142]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:31:34.200 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:31:34.225 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_2 2024-09-11T23:31:34.487 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:34.775 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:31:34.775 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='osd.5 [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:31:34.775 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: osdmap e32: 6 total, 5 up, 6 in 2024-09-11T23:31:34.775 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:34.775 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:31:34.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:34.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:34.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:34.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:34.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:34.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:34.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:34.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:34.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:34 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:34.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:31:34.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='osd.5 [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:31:34.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: osdmap e32: 6 total, 5 up, 6 in 2024-09-11T23:31:34.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:34.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:31:34.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:34.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:34.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:34.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:34.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:34.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:34.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:34.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:34.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:34 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.978+0000 7f521cd34640 1 -- 172.21.15.133:0/1247388242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218102690 msgr2=0x7f5218108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.978+0000 7f521cd34640 1 --2- 172.21.15.133:0/1247388242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218102690 0x7f5218108f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5210009ca0 tx=0x7f521002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:31:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.978+0000 7f521cd34640 1 -- 172.21.15.133:0/1247388242 shutdown_connections 2024-09-11T23:31:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.978+0000 7f521cd34640 1 --2- 172.21.15.133:0/1247388242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218102690 0x7f5218108f20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.978+0000 7f521cd34640 1 --2- 172.21.15.133:0/1247388242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218101cd0 0x7f5218102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.978+0000 7f521cd34640 1 --2- 172.21.15.133:0/1247388242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5218100ad0 0x7f5218100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.978+0000 7f521cd34640 1 -- 172.21.15.133:0/1247388242 >> 172.21.15.133:0/1247388242 conn(0x7f52180fc280 msgr2=0x7f52180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.978+0000 7f521cd34640 1 -- 172.21.15.133:0/1247388242 shutdown_connections 2024-09-11T23:31:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.978+0000 7f521cd34640 1 -- 172.21.15.133:0/1247388242 wait complete. 2024-09-11T23:31:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.978+0000 7f521cd34640 1 Processor -- start 2024-09-11T23:31:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f521cd34640 1 -- start start 2024-09-11T23:31:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f521cd34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218100ad0 0x7f521806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f521cd34640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218101cd0 0x7f521819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f521cd34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5218102690 0x7f521819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f521cd34640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f521810c750 con 0x7f5218101cd0 2024-09-11T23:31:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f521cd34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f521810c5d0 con 0x7f5218100ad0 2024-09-11T23:31:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f521cd34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f521810c8d0 con 0x7f5218102690 2024-09-11T23:31:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f520ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218101cd0 0x7f521819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f520ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218101cd0 0x7f521819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33726/0 (socket says 172.21.15.133:33726) 2024-09-11T23:31:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f520ffff640 1 -- 172.21.15.133:0/4123665901 learned_addr learned my addr 172.21.15.133:0/4123665901 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:31:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f5216d9d640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5218102690 0x7f521819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f521659c640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218100ad0 0x7f521806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f520ffff640 1 -- 172.21.15.133:0/4123665901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5218102690 msgr2=0x7f521819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f520ffff640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5218102690 0x7f521819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f520ffff640 1 -- 172.21.15.133:0/4123665901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218100ad0 msgr2=0x7f521806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f520ffff640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218100ad0 0x7f521806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f520ffff640 1 -- 172.21.15.133:0/4123665901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521819fb30 con 0x7f5218101cd0 2024-09-11T23:31:34.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f521659c640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218100ad0 0x7f521806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f520ffff640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218101cd0 0x7f521819cc00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f520000e9f0 tx=0x7f520000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.979+0000 7f520f7fe640 1 -- 172.21.15.133:0/4123665901 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520000cd80 con 0x7f5218101cd0 2024-09-11T23:31:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.980+0000 7f521cd34640 1 -- 172.21.15.133:0/4123665901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521819fdc0 con 0x7f5218101cd0 2024-09-11T23:31:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.980+0000 7f520f7fe640 1 -- 172.21.15.133:0/4123665901 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5200010910 con 0x7f5218101cd0 2024-09-11T23:31:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.980+0000 7f520f7fe640 1 -- 172.21.15.133:0/4123665901 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52000188f0 con 0x7f5218101cd0 2024-09-11T23:31:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.980+0000 7f521cd34640 1 -- 172.21.15.133:0/4123665901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52181a0270 con 0x7f5218101cd0 2024-09-11T23:31:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.981+0000 7f521cd34640 1 -- 172.21.15.133:0/4123665901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5218107ea0 con 0x7f5218101cd0 2024-09-11T23:31:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.982+0000 7f520f7fe640 1 -- 172.21.15.133:0/4123665901 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5200010ac0 con 0x7f5218101cd0 2024-09-11T23:31:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.983+0000 7f520f7fe640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f52040a2b80 0x7f52040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.983+0000 7f521659c640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f52040a2b80 0x7f52040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.983+0000 7f520f7fe640 1 -- 172.21.15.133:0/4123665901 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4253+0+0 (secure 0 0 0) 0x7f52000ce8b0 con 0x7f5218101cd0 2024-09-11T23:31:34.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.984+0000 7f521659c640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f52040a2b80 0x7f52040a5040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5218101b30 tx=0x7f5208005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:34.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:34.986+0000 7f520f7fe640 1 -- 172.21.15.133:0/4123665901 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f520009c750 con 0x7f5218101cd0 2024-09-11T23:31:35.040 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:31:34.606+0000 7f0d1031f640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T23:31:35.040 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:31:34.612+0000 7f0d0c959640 -1 osd.5 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:31:35.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:31:35.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='osd.5 [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:31:35.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: osdmap e32: 6 total, 5 up, 6 in 2024-09-11T23:31:35.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:35.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:31:35.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:31:35.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:35.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:35.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:35.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:31:35.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:35.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:35.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:35.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:34 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:35.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:35.170+0000 7f521cd34640 1 -- 172.21.15.133:0/4123665901 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5218062630 con 0x7f52040a2b80 2024-09-11T23:31:35.819 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi178 2024-09-11T23:31:35.819 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi178 to 8544M 2024-09-11T23:31:35.819 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:31:35.819 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T23:31:35.819 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:35.819 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:35.820 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:31:35.820 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:31:35.820 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:35.820 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: osd.5 [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536] boot 2024-09-11T23:31:35.820 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: osdmap e34: 6 total, 6 up, 6 in 2024-09-11T23:31:35.820 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:35.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi178 2024-09-11T23:31:35.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi178 to 8544M 2024-09-11T23:31:35.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:31:35.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T23:31:35.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:35.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:35.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:31:35.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:31:35.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:35.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: osd.5 [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536] boot 2024-09-11T23:31:35.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: osdmap e34: 6 total, 6 up, 6 in 2024-09-11T23:31:35.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:36.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi178 2024-09-11T23:31:36.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi178 to 8544M 2024-09-11T23:31:36.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:31:36.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T23:31:36.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:36.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:36.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:31:36.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:31:36.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:36.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: osd.5 [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536] boot 2024-09-11T23:31:36.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: osdmap e34: 6 total, 6 up, 6 in 2024-09-11T23:31:36.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:31:36.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:36 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:31:36.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:36 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:31:36.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:36 smithi133 ceph-mon[31430]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:31:36.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:36 smithi133 ceph-mon[31430]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:36.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:36 smithi133 ceph-mon[31430]: osdmap e35: 6 total, 6 up, 6 in 2024-09-11T23:31:36.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:36 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:31:36.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:36 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:31:36.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:36 smithi138 ceph-mon[35142]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:31:36.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:36 smithi138 ceph-mon[35142]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:36.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:36 smithi138 ceph-mon[35142]: osdmap e35: 6 total, 6 up, 6 in 2024-09-11T23:31:37.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:36 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:31:37.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:36 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:31:37.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:36 smithi178 ceph-mon[35882]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:31:37.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:36 smithi178 ceph-mon[35882]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:37.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:36 smithi178 ceph-mon[35882]: osdmap e35: 6 total, 6 up, 6 in 2024-09-11T23:31:37.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:37 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3458181641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92c712b8-c5b7-4097-b108-1df572e8f155"}]: dispatch 2024-09-11T23:31:37.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:37 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3458181641' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92c712b8-c5b7-4097-b108-1df572e8f155"}]': finished 2024-09-11T23:31:37.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:37 smithi138 ceph-mon[35142]: osdmap e36: 7 total, 6 up, 7 in 2024-09-11T23:31:37.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:37 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:31:37.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:37 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3458181641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92c712b8-c5b7-4097-b108-1df572e8f155"}]: dispatch 2024-09-11T23:31:37.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:37 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3458181641' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92c712b8-c5b7-4097-b108-1df572e8f155"}]': finished 2024-09-11T23:31:37.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:37 smithi133 ceph-mon[31430]: osdmap e36: 7 total, 6 up, 7 in 2024-09-11T23:31:37.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:31:38.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:37 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3458181641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92c712b8-c5b7-4097-b108-1df572e8f155"}]: dispatch 2024-09-11T23:31:38.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:37 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3458181641' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92c712b8-c5b7-4097-b108-1df572e8f155"}]': finished 2024-09-11T23:31:38.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:37 smithi178 ceph-mon[35882]: osdmap e36: 7 total, 6 up, 7 in 2024-09-11T23:31:38.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:37 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:31:38.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:38 smithi133 ceph-mon[31430]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:38.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:38 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/671590481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:31:38.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:38 smithi133 ceph-mon[31430]: osdmap e37: 7 total, 6 up, 7 in 2024-09-11T23:31:38.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:38 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:31:39.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:38 smithi178 ceph-mon[35882]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:39.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:38 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/671590481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:31:39.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:38 smithi178 ceph-mon[35882]: osdmap e37: 7 total, 6 up, 7 in 2024-09-11T23:31:39.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:38 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:31:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:38 smithi138 ceph-mon[35142]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:38 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/671590481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:31:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:38 smithi138 ceph-mon[35142]: osdmap e37: 7 total, 6 up, 7 in 2024-09-11T23:31:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:38 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:31:40.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:40 smithi138 ceph-mon[35142]: osdmap e38: 7 total, 6 up, 7 in 2024-09-11T23:31:40.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:31:40.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:40 smithi138 ceph-mon[35142]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:40.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:40 smithi133 ceph-mon[31430]: osdmap e38: 7 total, 6 up, 7 in 2024-09-11T23:31:40.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:31:40.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:40 smithi133 ceph-mon[31430]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:40.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:40 smithi178 ceph-mon[35882]: osdmap e38: 7 total, 6 up, 7 in 2024-09-11T23:31:40.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:31:40.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:40 smithi178 ceph-mon[35882]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:43.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:42 smithi138 ceph-mon[35142]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:43.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:42 smithi133 ceph-mon[31430]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:43.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:42 smithi178 ceph-mon[35882]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:45.162 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:44 smithi133 ceph-mon[31430]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T23:31:45.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:44 smithi138 ceph-mon[35142]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T23:31:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:44 smithi178 ceph-mon[35882]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T23:31:46.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:46 smithi133 ceph-mon[31430]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:31:46.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:31:46.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:47.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:46 smithi178 ceph-mon[35882]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:31:47.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:31:47.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:47.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:46 smithi138 ceph-mon[35142]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:31:47.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:46 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:31:47.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:46 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:48.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:47 smithi133 ceph-mon[31430]: Deploying daemon osd.6 on smithi133 2024-09-11T23:31:48.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:47 smithi138 ceph-mon[35142]: Deploying daemon osd.6 on smithi133 2024-09-11T23:31:48.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:47 smithi178 ceph-mon[35882]: Deploying daemon osd.6 on smithi133 2024-09-11T23:31:48.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:48 smithi133 ceph-mon[31430]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T23:31:49.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:48 smithi138 ceph-mon[35142]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T23:31:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:48 smithi178 ceph-mon[35882]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T23:31:51.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:50 smithi133 ceph-mon[31430]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T23:31:51.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:50 smithi138 ceph-mon[35142]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T23:31:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:50 smithi178 ceph-mon[35882]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T23:31:53.083 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:52 smithi133 ceph-mon[31430]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:53.083 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:52 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:53.083 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:52 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:53.083 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:52 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:52 smithi138 ceph-mon[35142]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:52 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:52 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:52 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:52 smithi178 ceph-mon[35882]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:52 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:52 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:52 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:55.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:54 smithi138 ceph-mon[35142]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:55.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:54 smithi133 ceph-mon[31430]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:54 smithi178 ceph-mon[35882]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:31:56.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.097+0000 7f520f7fe640 1 -- 172.21.15.133:0/4123665901 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5218062630 con 0x7f52040a2b80 2024-09-11T23:31:56.097 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 6 on host 'smithi133' 2024-09-11T23:31:56.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 -- 172.21.15.133:0/4123665901 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f52040a2b80 msgr2=0x7f52040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:56.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f52040a2b80 0x7f52040a5040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5218101b30 tx=0x7f5208005f90 comp rx=0 tx=0).stop 2024-09-11T23:31:56.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 -- 172.21.15.133:0/4123665901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218101cd0 msgr2=0x7f521819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:56.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218101cd0 0x7f521819cc00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f520000e9f0 tx=0x7f520000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:31:56.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 -- 172.21.15.133:0/4123665901 shutdown_connections 2024-09-11T23:31:56.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f52040a2b80 0x7f52040a5040 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:56.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5218102690 0x7f521819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:56.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218101cd0 0x7f521819cc00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:56.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 --2- 172.21.15.133:0/4123665901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218100ad0 0x7f521806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:56.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 -- 172.21.15.133:0/4123665901 >> 172.21.15.133:0/4123665901 conn(0x7f52180fc280 msgr2=0x7f5218105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:56.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 -- 172.21.15.133:0/4123665901 shutdown_connections 2024-09-11T23:31:56.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:31:56.100+0000 7f521cd34640 1 -- 172.21.15.133:0/4123665901 wait complete. 2024-09-11T23:31:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:55 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:55 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:55 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:55 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:55 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:55 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:55 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:56.760 DEBUG:teuthology.orchestra.run.smithi133:osd.6> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T23:31:56.763 INFO:tasks.cephadm:Deploying osd.7 on smithi138 with /dev/vg_nvme/lv_2... 2024-09-11T23:31:56.763 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:31:57.017 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:31:57.039 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:56 smithi133 ceph-mon[31430]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:57.039 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:56 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:57.040 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:56 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:57.040 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:56 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:57.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:56 smithi138 ceph-mon[35142]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:57.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:56 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:57.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:56 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:57.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:56 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:57.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:56 smithi178 ceph-mon[35882]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:57.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:56 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:57.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:56 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:57.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:56 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:31:58.960 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:31:58 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:31:58.644+0000 7fb20d30b640 -1 osd.6 0 log_to_monitors true 2024-09-11T23:31:59.272 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:59 smithi138 ceph-mon[35142]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:59.272 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:31:59 smithi138 ceph-mon[35142]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:31:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:31:59.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:59 smithi178 ceph-mon[35882]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:59.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:31:59 smithi178 ceph-mon[35882]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:31:59.300 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-09-11T23:31:59.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:59 smithi133 ceph-mon[31430]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:31:59.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:31:59 smithi133 ceph-mon[31430]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:31:59.554 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:31:59.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.994+0000 7f814e77c640 1 -- 172.21.15.138:0/2693018576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8148101c00 msgr2=0x7f8148102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:59.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.994+0000 7f814e77c640 1 --2- 172.21.15.138:0/2693018576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8148101c00 0x7f8148102000 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8138009900 tx=0x7f813802a410 comp rx=0 tx=0).stop 2024-09-11T23:31:59.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.994+0000 7f814e77c640 1 -- 172.21.15.138:0/2693018576 shutdown_connections 2024-09-11T23:31:59.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.994+0000 7f814e77c640 1 --2- 172.21.15.138:0/2693018576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81481037c0 0x7f814810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:59.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.994+0000 7f814e77c640 1 --2- 172.21.15.138:0/2693018576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8148102e00 0x7f8148103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:59.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.994+0000 7f814e77c640 1 --2- 172.21.15.138:0/2693018576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8148101c00 0x7f8148102000 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:59.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.994+0000 7f814e77c640 1 -- 172.21.15.138:0/2693018576 >> 172.21.15.138:0/2693018576 conn(0x7f81480fd390 msgr2=0x7f81480ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:59.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.994+0000 7f814e77c640 1 -- 172.21.15.138:0/2693018576 shutdown_connections 2024-09-11T23:31:59.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.994+0000 7f814e77c640 1 -- 172.21.15.138:0/2693018576 wait complete. 2024-09-11T23:31:59.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f814e77c640 1 Processor -- start 2024-09-11T23:31:59.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f814e77c640 1 -- start start 2024-09-11T23:31:59.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f814e77c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8148101c00 0x7f8148197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:59.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f814e77c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8148102e00 0x7f8148198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:59.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f814e77c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81481037c0 0x7f814819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:59.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f814e77c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f814810d890 con 0x7f8148101c00 2024-09-11T23:31:59.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f814e77c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f814810d710 con 0x7f81481037c0 2024-09-11T23:31:59.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f814e77c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f814810da10 con 0x7f8148102e00 2024-09-11T23:31:59.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f814cd19640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81481037c0 0x7f814819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:59.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f814cd19640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81481037c0 0x7f814819f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47298/0 (socket says 172.21.15.138:47298) 2024-09-11T23:31:59.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f814cd19640 1 -- 172.21.15.138:0/1135747433 learned_addr learned my addr 172.21.15.138:0/1135747433 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f8147fff640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8148101c00 0x7f8148197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.995+0000 7f81477fe640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8148102e00 0x7f8148198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.996+0000 7f814cd19640 1 -- 172.21.15.138:0/1135747433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8148102e00 msgr2=0x7f8148198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.996+0000 7f814cd19640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8148102e00 0x7f8148198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.996+0000 7f814cd19640 1 -- 172.21.15.138:0/1135747433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8148101c00 msgr2=0x7f8148197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.996+0000 7f814cd19640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8148101c00 0x7f8148197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.996+0000 7f814cd19640 1 -- 172.21.15.138:0/1135747433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814819fb40 con 0x7f81481037c0 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.996+0000 7f814cd19640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81481037c0 0x7f814819f4d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f814000eb10 tx=0x7f814000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.996+0000 7f81457fa640 1 -- 172.21.15.138:0/1135747433 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140004b20 con 0x7f81481037c0 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.996+0000 7f81457fa640 1 -- 172.21.15.138:0/1135747433 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8140004cc0 con 0x7f81481037c0 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.996+0000 7f814e77c640 1 -- 172.21.15.138:0/1135747433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814819fe30 con 0x7f81481037c0 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.996+0000 7f81457fa640 1 -- 172.21.15.138:0/1135747433 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140010760 con 0x7f81481037c0 2024-09-11T23:31:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.996+0000 7f814e77c640 1 -- 172.21.15.138:0/1135747433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81481a02e0 con 0x7f81481037c0 2024-09-11T23:32:00.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.997+0000 7f814e77c640 1 -- 172.21.15.138:0/1135747433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8148109020 con 0x7f81481037c0 2024-09-11T23:32:00.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.998+0000 7f81457fa640 1 -- 172.21.15.138:0/1135747433 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8140004e70 con 0x7f81481037c0 2024-09-11T23:32:00.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.999+0000 7f81457fa640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f81300a2bb0 0x7f81300a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.999+0000 7f81457fa640 1 -- 172.21.15.138:0/1135747433 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4669+0+0 (secure 0 0 0) 0x7f81400ceb60 con 0x7f81481037c0 2024-09-11T23:32:00.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:31:59.999+0000 7f8147fff640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f81300a2bb0 0x7f81300a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:00.000+0000 7f8147fff640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f81300a2bb0 0x7f81300a5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8138005ab0 tx=0x7f8138000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:00.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:00.006+0000 7f81457fa640 1 -- 172.21.15.138:0/1135747433 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f814009c900 con 0x7f81481037c0 2024-09-11T23:32:00.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:00.188+0000 7f814e77c640 1 -- 172.21.15.138:0/1135747433 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8148062630 con 0x7f81300a2bb0 2024-09-11T23:32:00.189 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:32:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T23:32:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:32:00.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:00.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:00.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:00.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:00.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:00.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:00.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:00.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:00.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:00.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:00.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:32:00.015+0000 7fb208a6b640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T23:32:00.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:32:00.022+0000 7fb2050a5640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:32:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:32:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T23:32:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:32:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:00.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:00.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:01 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi133 2024-09-11T23:32:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:01 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi133 to 5696M 2024-09-11T23:32:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:01 smithi138 ceph-mon[35142]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:32:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:01 smithi138 ceph-mon[35142]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:32:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:01 smithi138 ceph-mon[35142]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T23:32:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:01 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:01.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:01 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:01.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:01 smithi138 ceph-mon[35142]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:01.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:01 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:32:01.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:01 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:32:01.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:01 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:01 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi133 2024-09-11T23:32:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:01 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi133 to 5696M 2024-09-11T23:32:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:01 smithi178 ceph-mon[35882]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:32:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:01 smithi178 ceph-mon[35882]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:32:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:01 smithi178 ceph-mon[35882]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T23:32:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:01 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:01.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:01 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:01.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:01 smithi178 ceph-mon[35882]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:01.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:01 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:32:01.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:01 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:32:01.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:01 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:01.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:01 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi133 2024-09-11T23:32:01.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:01 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi133 to 5696M 2024-09-11T23:32:01.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:01 smithi133 ceph-mon[31430]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:32:01.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:01 smithi133 ceph-mon[31430]: from='osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:32:01.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:01 smithi133 ceph-mon[31430]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T23:32:01.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:01 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:01.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:01 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:01.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:01 smithi133 ceph-mon[31430]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:01.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:01 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:32:01.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:01 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:32:01.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:01 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:02.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:02 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:32:02.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:02 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:32:02.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:02 smithi138 ceph-mon[35142]: osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573] boot 2024-09-11T23:32:02.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:02 smithi138 ceph-mon[35142]: osdmap e41: 7 total, 7 up, 7 in 2024-09-11T23:32:02.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:02.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:02 smithi138 ceph-mon[35142]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:32:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:02 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:32:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:02 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:32:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:02 smithi178 ceph-mon[35882]: osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573] boot 2024-09-11T23:32:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:02 smithi178 ceph-mon[35882]: osdmap e41: 7 total, 7 up, 7 in 2024-09-11T23:32:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:02 smithi178 ceph-mon[35882]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:32:02.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:02 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:32:02.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:02 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:32:02.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:02 smithi133 ceph-mon[31430]: osd.6 [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573] boot 2024-09-11T23:32:02.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:02 smithi133 ceph-mon[31430]: osdmap e41: 7 total, 7 up, 7 in 2024-09-11T23:32:02.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:32:02.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:02 smithi133 ceph-mon[31430]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:32:03.173 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:03 smithi138 ceph-mon[35142]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T23:32:03.173 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:03 smithi138 ceph-mon[35142]: from='client.? 172.21.15.138:0/264991584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46533b61-85a2-4c27-8fb0-140a2e34f03c"}]: dispatch 2024-09-11T23:32:03.174 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:03 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46533b61-85a2-4c27-8fb0-140a2e34f03c"}]: dispatch 2024-09-11T23:32:03.174 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:03 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46533b61-85a2-4c27-8fb0-140a2e34f03c"}]': finished 2024-09-11T23:32:03.174 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:03 smithi138 ceph-mon[35142]: osdmap e43: 8 total, 7 up, 8 in 2024-09-11T23:32:03.174 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:03 smithi178 ceph-mon[35882]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T23:32:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:03 smithi178 ceph-mon[35882]: from='client.? 172.21.15.138:0/264991584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46533b61-85a2-4c27-8fb0-140a2e34f03c"}]: dispatch 2024-09-11T23:32:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:03 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46533b61-85a2-4c27-8fb0-140a2e34f03c"}]: dispatch 2024-09-11T23:32:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:03 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46533b61-85a2-4c27-8fb0-140a2e34f03c"}]': finished 2024-09-11T23:32:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:03 smithi178 ceph-mon[35882]: osdmap e43: 8 total, 7 up, 8 in 2024-09-11T23:32:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:03.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:03 smithi133 ceph-mon[31430]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T23:32:03.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:03 smithi133 ceph-mon[31430]: from='client.? 172.21.15.138:0/264991584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46533b61-85a2-4c27-8fb0-140a2e34f03c"}]: dispatch 2024-09-11T23:32:03.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:03 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46533b61-85a2-4c27-8fb0-140a2e34f03c"}]: dispatch 2024-09-11T23:32:03.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:03 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46533b61-85a2-4c27-8fb0-140a2e34f03c"}]': finished 2024-09-11T23:32:03.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:03 smithi133 ceph-mon[31430]: osdmap e43: 8 total, 7 up, 8 in 2024-09-11T23:32:03.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:04.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:04 smithi178 ceph-mon[35882]: from='client.? 172.21.15.138:0/2212339042' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:32:04.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:04 smithi178 ceph-mon[35882]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:04.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:04 smithi138 ceph-mon[35142]: from='client.? 172.21.15.138:0/2212339042' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:32:04.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:04 smithi138 ceph-mon[35142]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:04.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:04 smithi133 ceph-mon[31430]: from='client.? 172.21.15.138:0/2212339042' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:32:04.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:04 smithi133 ceph-mon[31430]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:06 smithi138 ceph-mon[35142]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:07.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:06 smithi133 ceph-mon[31430]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:07.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:06 smithi178 ceph-mon[35882]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:09.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:08 smithi138 ceph-mon[35142]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:09.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:08 smithi133 ceph-mon[31430]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:08 smithi178 ceph-mon[35882]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:10.945 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:10 smithi138 ceph-mon[35142]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:11.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:10 smithi133 ceph-mon[31430]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:10 smithi178 ceph-mon[35882]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:11.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:32:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:11 smithi138 ceph-mon[35142]: Deploying daemon osd.7 on smithi138 2024-09-11T23:32:12.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:32:12.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:12.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:11 smithi133 ceph-mon[31430]: Deploying daemon osd.7 on smithi138 2024-09-11T23:32:12.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:32:12.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:12.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:11 smithi178 ceph-mon[35882]: Deploying daemon osd.7 on smithi138 2024-09-11T23:32:12.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:12 smithi133 ceph-mon[31430]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:12 smithi138 ceph-mon[35142]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:12 smithi178 ceph-mon[35882]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:15.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:14 smithi138 ceph-mon[35142]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:15.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:14 smithi133 ceph-mon[31430]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:15.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:14 smithi178 ceph-mon[35882]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:17.150 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:16 smithi138 ceph-mon[35142]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:17.150 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:17.150 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:17.150 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:16 smithi133 ceph-mon[31430]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:17.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:16 smithi178 ceph-mon[35882]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:17.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:17.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:17.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:19.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:18 smithi138 ceph-mon[35142]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:19.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:18 smithi133 ceph-mon[31430]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:18 smithi178 ceph-mon[35882]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.248+0000 7f81457fa640 1 -- 172.21.15.138:0/1135747433 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8148062630 con 0x7f81300a2bb0 2024-09-11T23:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 7 on host 'smithi138' 2024-09-11T23:32:20.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 -- 172.21.15.138:0/1135747433 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f81300a2bb0 msgr2=0x7f81300a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:20.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f81300a2bb0 0x7f81300a5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8138005ab0 tx=0x7f8138000bc0 comp rx=0 tx=0).stop 2024-09-11T23:32:20.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 -- 172.21.15.138:0/1135747433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81481037c0 msgr2=0x7f814819f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:20.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81481037c0 0x7f814819f4d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f814000eb10 tx=0x7f814000c2d0 comp rx=0 tx=0).stop 2024-09-11T23:32:20.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 -- 172.21.15.138:0/1135747433 shutdown_connections 2024-09-11T23:32:20.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f81300a2bb0 0x7f81300a5070 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:20.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81481037c0 0x7f814819f4d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:20.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8148102e00 0x7f8148198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:20.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 --2- 172.21.15.138:0/1135747433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8148101c00 0x7f8148197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:20.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 -- 172.21.15.138:0/1135747433 >> 172.21.15.138:0/1135747433 conn(0x7f81480fd390 msgr2=0x7f81481061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:20.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 -- 172.21.15.138:0/1135747433 shutdown_connections 2024-09-11T23:32:20.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:32:20.251+0000 7f814e77c640 1 -- 172.21.15.138:0/1135747433 wait complete. 2024-09-11T23:32:20.542 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.543 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.543 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:20.543 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:20.543 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.543 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:20 smithi138 ceph-mon[35142]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:20.543 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:20.543 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.543 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:20 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:20.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:20.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:20 smithi133 ceph-mon[31430]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:20.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:20.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:20 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:20.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:20.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:20 smithi178 ceph-mon[35882]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:20.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:20.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:20 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:20.804 DEBUG:teuthology.orchestra.run.smithi138:osd.7> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T23:32:20.806 INFO:tasks.cephadm:Deploying osd.8 on smithi178 with /dev/vg_nvme/lv_2... 2024-09-11T23:32:20.806 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:32:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:32:22.698 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:32:22 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:32:22.409+0000 7fb525654640 -1 osd.7 0 log_to_monitors true 2024-09-11T23:32:23.063 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:22 smithi138 ceph-mon[35142]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:23.064 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:22 smithi138 ceph-mon[35142]: from='osd.7 [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:32:23.064 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:22 smithi138 ceph-mon[35142]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:32:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:22 smithi133 ceph-mon[31430]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:22 smithi133 ceph-mon[31430]: from='osd.7 [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:32:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:22 smithi133 ceph-mon[31430]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:32:23.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:22 smithi178 ceph-mon[35882]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:23.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:22 smithi178 ceph-mon[35882]: from='osd.7 [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:32:23.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:22 smithi178 ceph-mon[35882]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:32:23.622 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:32:23.646 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2024-09-11T23:32:23.889 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='osd.7 [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:24.138 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:24.138 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:32:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T23:32:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='osd.7 [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:32:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:32:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:24.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:24.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:24.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:24.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:24.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:32:24.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T23:32:24.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:24.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='osd.7 [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:32:24.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:32:24.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:24.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:24.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:24.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:24.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:24.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:24.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.355+0000 7f9b77308640 1 -- 172.21.15.178:0/4196150057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b70101d00 msgr2=0x7f9b70102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:24.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.355+0000 7f9b77308640 1 --2- 172.21.15.178:0/4196150057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b70101d00 0x7f9b70102180 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009900 tx=0x7f9b6002a410 comp rx=0 tx=0).stop 2024-09-11T23:32:24.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.355+0000 7f9b77308640 1 -- 172.21.15.178:0/4196150057 shutdown_connections 2024-09-11T23:32:24.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.355+0000 7f9b77308640 1 --2- 172.21.15.178:0/4196150057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b701026c0 0x7f9b70108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:24.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.355+0000 7f9b77308640 1 --2- 172.21.15.178:0/4196150057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b70101d00 0x7f9b70102180 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:24.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.355+0000 7f9b77308640 1 --2- 172.21.15.178:0/4196150057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b70100b00 0x7f9b70100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:24.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.355+0000 7f9b77308640 1 -- 172.21.15.178:0/4196150057 >> 172.21.15.178:0/4196150057 conn(0x7f9b700fc270 msgr2=0x7f9b700fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.355+0000 7f9b77308640 1 -- 172.21.15.178:0/4196150057 shutdown_connections 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.355+0000 7f9b77308640 1 -- 172.21.15.178:0/4196150057 wait complete. 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b77308640 1 Processor -- start 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b77308640 1 -- start start 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b77308640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b70100b00 0x7f9b70104a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b77308640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b70101d00 0x7f9b701030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b77308640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b701026c0 0x7f9b70103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b77308640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010c780 con 0x7f9b701026c0 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b77308640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010c600 con 0x7f9b70101d00 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b77308640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010c900 con 0x7f9b70100b00 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b750a4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b70100b00 0x7f9b70104a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b750a4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b70100b00 0x7f9b70104a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:34632/0 (socket says 172.21.15.178:34632) 2024-09-11T23:32:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b750a4640 1 -- 172.21.15.178:0/237362688 learned_addr learned my addr 172.21.15.178:0/237362688 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b748a3640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b70101d00 0x7f9b701030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b758a5640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b701026c0 0x7f9b70103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b750a4640 1 -- 172.21.15.178:0/237362688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b70101d00 msgr2=0x7f9b701030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b750a4640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b70101d00 0x7f9b701030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b750a4640 1 -- 172.21.15.178:0/237362688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b701026c0 msgr2=0x7f9b70103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b750a4640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b701026c0 0x7f9b70103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b750a4640 1 -- 172.21.15.178:0/237362688 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b70103ee0 con 0x7f9b70100b00 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b758a5640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b701026c0 0x7f9b70103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b750a4640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b70100b00 0x7f9b70104a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00e9c0 tx=0x7f9b6c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.356+0000 7f9b667fc640 1 -- 172.21.15.178:0/237362688 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c00cd80 con 0x7f9b70100b00 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.357+0000 7f9b667fc640 1 -- 172.21.15.178:0/237362688 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b6c0035f0 con 0x7f9b70100b00 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.357+0000 7f9b667fc640 1 -- 172.21.15.178:0/237362688 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c010690 con 0x7f9b70100b00 2024-09-11T23:32:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.357+0000 7f9b77308640 1 -- 172.21.15.178:0/237362688 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b701a21d0 con 0x7f9b70100b00 2024-09-11T23:32:24.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.357+0000 7f9b77308640 1 -- 172.21.15.178:0/237362688 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b701a2690 con 0x7f9b70100b00 2024-09-11T23:32:24.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.358+0000 7f9b77308640 1 -- 172.21.15.178:0/237362688 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b701a2920 con 0x7f9b70100b00 2024-09-11T23:32:24.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.359+0000 7f9b667fc640 1 -- 172.21.15.178:0/237362688 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9b6c0037a0 con 0x7f9b70100b00 2024-09-11T23:32:24.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.359+0000 7f9b667fc640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9b5c0a2b80 0x7f9b5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:24.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.360+0000 7f9b667fc640 1 -- 172.21.15.178:0/237362688 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5117+0+0 (secure 0 0 0) 0x7f9b6c0ce860 con 0x7f9b70100b00 2024-09-11T23:32:24.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.360+0000 7f9b748a3640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9b5c0a2b80 0x7f9b5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:24.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.360+0000 7f9b748a3640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9b5c0a2b80 0x7f9b5c0a5040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b60000c00 tx=0x7f9b600315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:24.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.363+0000 7f9b667fc640 1 -- 172.21.15.178:0/237362688 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b6c09c440 con 0x7f9b70100b00 2024-09-11T23:32:24.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:24.537+0000 7f9b77308640 1 -- 172.21.15.178:0/237362688 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9b7006af30 con 0x7f9b5c0a2b80 2024-09-11T23:32:24.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:32:24.645+0000 7fb520db4640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T23:32:24.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:32:24.656+0000 7fb5193e6640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:32:24.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi138 2024-09-11T23:32:24.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi138 to 5696M 2024-09-11T23:32:24.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-mon[35142]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:24.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-mon[35142]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:32:24.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-mon[35142]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T23:32:24.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:24.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:24.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:32:24.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:32:24.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:24.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:24 smithi138 ceph-mon[35142]: from='osd.7 ' entity='osd.7' 2024-09-11T23:32:25.183 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:24 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi138 2024-09-11T23:32:25.184 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:24 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi138 to 5696M 2024-09-11T23:32:25.184 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:24 smithi178 ceph-mon[35882]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:25.184 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:24 smithi178 ceph-mon[35882]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:32:25.184 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:24 smithi178 ceph-mon[35882]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T23:32:25.184 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:25.184 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:25.184 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:32:25.184 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:32:25.184 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:25.184 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:24 smithi178 ceph-mon[35882]: from='osd.7 ' entity='osd.7' 2024-09-11T23:32:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:24 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi138 2024-09-11T23:32:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:24 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi138 to 5696M 2024-09-11T23:32:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:24 smithi133 ceph-mon[31430]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:24 smithi133 ceph-mon[31430]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:32:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:24 smithi133 ceph-mon[31430]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T23:32:25.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:25.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:25.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:32:25.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:32:25.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:25.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:24 smithi133 ceph-mon[31430]: from='osd.7 ' entity='osd.7' 2024-09-11T23:32:25.937 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:25 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:32:25.937 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:25 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:32:25.937 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:25 smithi178 ceph-mon[35882]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:25.937 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:25 smithi178 ceph-mon[35882]: osd.7 [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854] boot 2024-09-11T23:32:25.937 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:25 smithi178 ceph-mon[35882]: osdmap e46: 8 total, 8 up, 8 in 2024-09-11T23:32:25.937 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:25 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:32:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:25 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:32:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:25 smithi138 ceph-mon[35142]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:25 smithi138 ceph-mon[35142]: osd.7 [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854] boot 2024-09-11T23:32:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:25 smithi138 ceph-mon[35142]: osdmap e46: 8 total, 8 up, 8 in 2024-09-11T23:32:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:25 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:32:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:25 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:32:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:25 smithi133 ceph-mon[31430]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:25 smithi133 ceph-mon[31430]: osd.7 [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854] boot 2024-09-11T23:32:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:25 smithi133 ceph-mon[31430]: osdmap e46: 8 total, 8 up, 8 in 2024-09-11T23:32:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:32:27.152 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:26 smithi178 ceph-mon[35882]: pgmap v165: 1 pgs: 1 remapped; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:27.153 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:26 smithi178 ceph-mon[35882]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T23:32:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:26 smithi138 ceph-mon[35142]: pgmap v165: 1 pgs: 1 remapped; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:27.200 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:26 smithi138 ceph-mon[35142]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T23:32:27.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:26 smithi133 ceph-mon[31430]: pgmap v165: 1 pgs: 1 remapped; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:32:27.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:26 smithi133 ceph-mon[31430]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T23:32:28.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:27 smithi138 ceph-mon[35142]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T23:32:28.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:27 smithi138 ceph-mon[35142]: from='client.? 172.21.15.178:0/4146610075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e16c39c-28cf-427c-b40a-73cffe154908"}]: dispatch 2024-09-11T23:32:28.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:27 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e16c39c-28cf-427c-b40a-73cffe154908"}]: dispatch 2024-09-11T23:32:28.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:27 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e16c39c-28cf-427c-b40a-73cffe154908"}]': finished 2024-09-11T23:32:28.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:27 smithi138 ceph-mon[35142]: osdmap e49: 9 total, 8 up, 9 in 2024-09-11T23:32:28.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:27 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:28.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:27 smithi138 ceph-mon[35142]: from='client.? 172.21.15.178:0/2360689957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:32:28.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:27 smithi133 ceph-mon[31430]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T23:32:28.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:27 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/4146610075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e16c39c-28cf-427c-b40a-73cffe154908"}]: dispatch 2024-09-11T23:32:28.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:27 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e16c39c-28cf-427c-b40a-73cffe154908"}]: dispatch 2024-09-11T23:32:28.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:27 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e16c39c-28cf-427c-b40a-73cffe154908"}]': finished 2024-09-11T23:32:28.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:27 smithi133 ceph-mon[31430]: osdmap e49: 9 total, 8 up, 9 in 2024-09-11T23:32:28.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:27 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:28.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:27 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/2360689957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:32:28.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:27 smithi178 ceph-mon[35882]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T23:32:28.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:27 smithi178 ceph-mon[35882]: from='client.? 172.21.15.178:0/4146610075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e16c39c-28cf-427c-b40a-73cffe154908"}]: dispatch 2024-09-11T23:32:28.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:27 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e16c39c-28cf-427c-b40a-73cffe154908"}]: dispatch 2024-09-11T23:32:28.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:27 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e16c39c-28cf-427c-b40a-73cffe154908"}]': finished 2024-09-11T23:32:28.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:27 smithi178 ceph-mon[35882]: osdmap e49: 9 total, 8 up, 9 in 2024-09-11T23:32:28.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:27 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:28.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:27 smithi178 ceph-mon[35882]: from='client.? 172.21.15.178:0/2360689957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:32:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:28 smithi138 ceph-mon[35142]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:28 smithi138 ceph-mon[35142]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T23:32:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:28 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:28 smithi133 ceph-mon[31430]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:28 smithi133 ceph-mon[31430]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T23:32:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:29.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:28 smithi178 ceph-mon[35882]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:29.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:28 smithi178 ceph-mon[35882]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T23:32:29.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:28 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:30.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:29 smithi138 ceph-mon[35142]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T23:32:30.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:29 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:30.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:29 smithi133 ceph-mon[31430]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T23:32:30.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:29 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:30.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:29 smithi178 ceph-mon[35882]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T23:32:30.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:29 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:30 smithi138 ceph-mon[35142]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:30 smithi133 ceph-mon[31430]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:30 smithi178 ceph-mon[35882]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:33.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:32 smithi178 ceph-mon[35882]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:33.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:32 smithi138 ceph-mon[35142]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:32 smithi133 ceph-mon[31430]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:35.201 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:34 smithi138 ceph-mon[35142]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:34 smithi133 ceph-mon[31430]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:35.250 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:34 smithi178 ceph-mon[35882]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:35.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:32:35.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:36.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:32:36.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:36.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:32:36.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:36 smithi138 ceph-mon[35142]: Deploying daemon osd.8 on smithi178 2024-09-11T23:32:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:36 smithi138 ceph-mon[35142]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:37.201 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:36 smithi178 ceph-mon[35882]: Deploying daemon osd.8 on smithi178 2024-09-11T23:32:37.201 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:36 smithi178 ceph-mon[35882]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:36 smithi133 ceph-mon[31430]: Deploying daemon osd.8 on smithi178 2024-09-11T23:32:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:36 smithi133 ceph-mon[31430]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:38 smithi138 ceph-mon[35142]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:38 smithi133 ceph-mon[31430]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:39.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:38 smithi178 ceph-mon[35882]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:40.974 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:40 smithi178 ceph-mon[35882]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:40 smithi138 ceph-mon[35142]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:41.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:40 smithi133 ceph-mon[31430]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:42.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:41 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:42.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:41 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:42.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:41 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:42.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:41 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:42.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:41 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:42.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:41 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:41 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:41 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:41 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:43.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:42 smithi178 ceph-mon[35882]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:43.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:42 smithi138 ceph-mon[35142]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:43.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:42 smithi133 ceph-mon[31430]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:44.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.364+0000 7f9b667fc640 1 -- 172.21.15.178:0/237362688 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9b7006af30 con 0x7f9b5c0a2b80 2024-09-11T23:32:44.364 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 8 on host 'smithi178' 2024-09-11T23:32:44.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 -- 172.21.15.178:0/237362688 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9b5c0a2b80 msgr2=0x7f9b5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:44.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9b5c0a2b80 0x7f9b5c0a5040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b60000c00 tx=0x7f9b600315b0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 -- 172.21.15.178:0/237362688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b70100b00 msgr2=0x7f9b70104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:44.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b70100b00 0x7f9b70104a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00e9c0 tx=0x7f9b6c00ee80 comp rx=0 tx=0).stop 2024-09-11T23:32:44.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 -- 172.21.15.178:0/237362688 shutdown_connections 2024-09-11T23:32:44.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9b5c0a2b80 0x7f9b5c0a5040 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b701026c0 0x7f9b70103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b70101d00 0x7f9b701030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 --2- 172.21.15.178:0/237362688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b70100b00 0x7f9b70104a90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:44.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 -- 172.21.15.178:0/237362688 >> 172.21.15.178:0/237362688 conn(0x7f9b700fc270 msgr2=0x7f9b7006a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:44.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 -- 172.21.15.178:0/237362688 shutdown_connections 2024-09-11T23:32:44.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:32:44.367+0000 7f9b77308640 1 -- 172.21.15.178:0/237362688 wait complete. 2024-09-11T23:32:44.991 DEBUG:teuthology.orchestra.run.smithi178:osd.8> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.8.service 2024-09-11T23:32:44.993 INFO:tasks.cephadm:Deploying osd.9 on smithi133 with /dev/vg_nvme/lv_1... 2024-09-11T23:32:44.993 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:32:45.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:44 smithi133 ceph-mon[31430]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:45.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:44 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:45.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:44 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:45.258 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:45.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:44 smithi178 ceph-mon[35882]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:44 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:44 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:45.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:44 smithi138 ceph-mon[35142]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:45.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:44 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:45.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:44 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:46.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:46 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:46.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:46 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:46.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:46 smithi138 ceph-mon[35142]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:46.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:46.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:46.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:46 smithi133 ceph-mon[31430]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:46.528 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:46.528 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:46.528 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:46 smithi178 ceph-mon[35882]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:47.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:47 smithi178 ceph-mon[35882]: from='osd.8 [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:32:47.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:47 smithi178 ceph-mon[35882]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:32:47.790 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:32:47 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:32:47.445+0000 7fa78abfc640 -1 osd.8 0 log_to_monitors true 2024-09-11T23:32:47.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:47 smithi138 ceph-mon[35142]: from='osd.8 [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:32:47.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:47 smithi138 ceph-mon[35142]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:32:47.957 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:47 smithi133 ceph-mon[31430]: from='osd.8 [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:32:47.957 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:47 smithi133 ceph-mon[31430]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:32:47.960 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:32:47.987 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_1 2024-09-11T23:32:48.249 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:48.693 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:32:48.693 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T23:32:48.693 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='osd.8 [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:48.694 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.695 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:48.695 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:48.695 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:48.695 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.776+0000 7fb92e417640 1 -- 172.21.15.133:0/4030591175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb928101c00 msgr2=0x7fb928102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.776+0000 7fb92e417640 1 --2- 172.21.15.133:0/4030591175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb928101c00 0x7fb928102000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb918009900 tx=0x7fb91802a410 comp rx=0 tx=0).stop 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.776+0000 7fb92e417640 1 -- 172.21.15.133:0/4030591175 shutdown_connections 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.776+0000 7fb92e417640 1 --2- 172.21.15.133:0/4030591175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9281037c0 0x7fb92810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.776+0000 7fb92e417640 1 --2- 172.21.15.133:0/4030591175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928102e00 0x7fb928103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.776+0000 7fb92e417640 1 --2- 172.21.15.133:0/4030591175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb928101c00 0x7fb928102000 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.776+0000 7fb92e417640 1 -- 172.21.15.133:0/4030591175 >> 172.21.15.133:0/4030591175 conn(0x7fb9280fd390 msgr2=0x7fb9280ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.776+0000 7fb92e417640 1 -- 172.21.15.133:0/4030591175 shutdown_connections 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.776+0000 7fb92e417640 1 -- 172.21.15.133:0/4030591175 wait complete. 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.776+0000 7fb92e417640 1 Processor -- start 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.776+0000 7fb92e417640 1 -- start start 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92e417640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb928101c00 0x7fb928197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:48.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92e417640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928102e00 0x7fb928198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92e417640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9281037c0 0x7fb92819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92e417640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb92810d860 con 0x7fb928101c00 2024-09-11T23:32:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92e417640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb92810d6e0 con 0x7fb9281037c0 2024-09-11T23:32:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92e417640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb92810d9e0 con 0x7fb928102e00 2024-09-11T23:32:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb927fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb928101c00 0x7fb928197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb927fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb928101c00 0x7fb928197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35122/0 (socket says 172.21.15.133:35122) 2024-09-11T23:32:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb927fff640 1 -- 172.21.15.133:0/2481452982 learned_addr learned my addr 172.21.15.133:0/2481452982 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:32:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92c9b4640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9281037c0 0x7fb92819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb9277fe640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928102e00 0x7fb928198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92c9b4640 1 -- 172.21.15.133:0/2481452982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928102e00 msgr2=0x7fb928198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92c9b4640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928102e00 0x7fb928198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:48.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92c9b4640 1 -- 172.21.15.133:0/2481452982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb928101c00 msgr2=0x7fb928197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:48.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92c9b4640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb928101c00 0x7fb928197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:48.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92c9b4640 1 -- 172.21.15.133:0/2481452982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92819fb40 con 0x7fb9281037c0 2024-09-11T23:32:48.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb9277fe640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928102e00 0x7fb928198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:48.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb927fff640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb928101c00 0x7fb928197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:48.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.777+0000 7fb92c9b4640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9281037c0 0x7fb92819f4d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb920009fd0 tx=0x7fb92000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:48.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.778+0000 7fb9257fa640 1 -- 172.21.15.133:0/2481452982 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92000ce20 con 0x7fb9281037c0 2024-09-11T23:32:48.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.778+0000 7fb92e417640 1 -- 172.21.15.133:0/2481452982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92819fe30 con 0x7fb9281037c0 2024-09-11T23:32:48.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.778+0000 7fb9257fa640 1 -- 172.21.15.133:0/2481452982 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb920005020 con 0x7fb9281037c0 2024-09-11T23:32:48.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.778+0000 7fb9257fa640 1 -- 172.21.15.133:0/2481452982 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920010720 con 0x7fb9281037c0 2024-09-11T23:32:48.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.778+0000 7fb92e417640 1 -- 172.21.15.133:0/2481452982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9281a02e0 con 0x7fb9281037c0 2024-09-11T23:32:48.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.779+0000 7fb92e417640 1 -- 172.21.15.133:0/2481452982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb928109050 con 0x7fb9281037c0 2024-09-11T23:32:48.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.780+0000 7fb9257fa640 1 -- 172.21.15.133:0/2481452982 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb920005540 con 0x7fb9281037c0 2024-09-11T23:32:48.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.780+0000 7fb9257fa640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb9100a2c30 0x7fb9100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:48.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.781+0000 7fb927fff640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb9100a2c30 0x7fb9100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:48.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.781+0000 7fb9257fa640 1 -- 172.21.15.133:0/2481452982 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5549+0+0 (secure 0 0 0) 0x7fb9200ceae0 con 0x7fb9281037c0 2024-09-11T23:32:48.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.781+0000 7fb927fff640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb9100a2c30 0x7fb9100a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb918000c00 tx=0x7fb9180046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:48.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.785+0000 7fb9257fa640 1 -- 172.21.15.133:0/2481452982 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb92009c510 con 0x7fb9281037c0 2024-09-11T23:32:48.790 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:32:48.517+0000 7fa78635c640 -1 osd.8 0 waiting for initial osdmap 2024-09-11T23:32:48.790 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:32:48.527+0000 7fa782996640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:32:48.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:32:48.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T23:32:48.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:48.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='osd.8 [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:32:48.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:48.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:32:48.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T23:32:48.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:48.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='osd.8 [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:32:48.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:32:48.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:48.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:48.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:48.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:48.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:48.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:48.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:48.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:48.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:48.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:32:48.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:32:48.960+0000 7fb92e417640 1 -- 172.21.15.133:0/2481452982 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb928062630 con 0x7fb9100a2c30 2024-09-11T23:32:49.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:49 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi178 2024-09-11T23:32:49.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:49 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi178 to 5696M 2024-09-11T23:32:49.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:49 smithi133 ceph-mon[31430]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:32:49.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:49 smithi133 ceph-mon[31430]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:32:49.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:49.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:49.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:49 smithi133 ceph-mon[31430]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:49.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:32:49.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:32:49.688 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:49.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:49 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi178 2024-09-11T23:32:49.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:49 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi178 to 5696M 2024-09-11T23:32:49.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:49 smithi178 ceph-mon[35882]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:32:49.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:49 smithi178 ceph-mon[35882]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:32:49.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:49.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:49.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:49 smithi178 ceph-mon[35882]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:49.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:32:49.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:32:49.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:49.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:49 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi178 2024-09-11T23:32:49.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:49 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi178 to 5696M 2024-09-11T23:32:49.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:49 smithi138 ceph-mon[35142]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:32:49.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:49 smithi138 ceph-mon[35142]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:32:49.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:49.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:49.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:49 smithi138 ceph-mon[35142]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:49.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:32:49.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:32:49.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:50.699 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:50 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:32:50.699 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:50 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:32:50.700 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:50.700 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:50 smithi133 ceph-mon[31430]: osd.8 [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700] boot 2024-09-11T23:32:50.700 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:50 smithi133 ceph-mon[31430]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T23:32:50.700 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:50.700 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:50 smithi133 ceph-mon[31430]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:50.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:50 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:32:50.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:50 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:32:50.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:50.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:50 smithi178 ceph-mon[35882]: osd.8 [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700] boot 2024-09-11T23:32:50.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:50 smithi178 ceph-mon[35882]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T23:32:50.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:50.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:50 smithi178 ceph-mon[35882]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:50 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:32:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:50 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:32:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:50 smithi138 ceph-mon[35142]: osd.8 [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700] boot 2024-09-11T23:32:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:50 smithi138 ceph-mon[35142]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T23:32:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:32:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:50 smithi138 ceph-mon[35142]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:32:51.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:51 smithi133 ceph-mon[31430]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T23:32:51.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:51 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1555554897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1436e241-5439-49db-a07a-8b80f4499da2"}]: dispatch 2024-09-11T23:32:51.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:51 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1555554897' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1436e241-5439-49db-a07a-8b80f4499da2"}]': finished 2024-09-11T23:32:51.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:51 smithi133 ceph-mon[31430]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T23:32:51.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:51 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:32:51.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:51 smithi178 ceph-mon[35882]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T23:32:51.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:51 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1555554897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1436e241-5439-49db-a07a-8b80f4499da2"}]: dispatch 2024-09-11T23:32:51.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:51 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1555554897' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1436e241-5439-49db-a07a-8b80f4499da2"}]': finished 2024-09-11T23:32:51.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:51 smithi178 ceph-mon[35882]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T23:32:51.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:51 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:32:51.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:51 smithi138 ceph-mon[35142]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T23:32:51.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:51 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1555554897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1436e241-5439-49db-a07a-8b80f4499da2"}]: dispatch 2024-09-11T23:32:51.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:51 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1555554897' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1436e241-5439-49db-a07a-8b80f4499da2"}]': finished 2024-09-11T23:32:51.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:51 smithi138 ceph-mon[35142]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T23:32:51.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:51 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:32:52.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:52 smithi138 ceph-mon[35142]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:32:52.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:52 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1728816611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:32:52.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:52 smithi138 ceph-mon[35142]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T23:32:52.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:52 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:32:52.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:52 smithi133 ceph-mon[31430]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:32:52.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:52 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1728816611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:32:52.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:52 smithi133 ceph-mon[31430]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T23:32:52.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:52 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:32:53.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:52 smithi178 ceph-mon[35882]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:32:53.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:52 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1728816611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:32:53.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:52 smithi178 ceph-mon[35882]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T23:32:53.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:52 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:32:54.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:54 smithi178 ceph-mon[35882]: osdmap e58: 10 total, 9 up, 10 in 2024-09-11T23:32:54.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:32:54.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:54 smithi178 ceph-mon[35882]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:32:54.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:54 smithi138 ceph-mon[35142]: osdmap e58: 10 total, 9 up, 10 in 2024-09-11T23:32:54.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:32:54.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:54 smithi138 ceph-mon[35142]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:32:54.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:54 smithi133 ceph-mon[31430]: osdmap e58: 10 total, 9 up, 10 in 2024-09-11T23:32:54.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:32:54.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:54 smithi133 ceph-mon[31430]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:32:57.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:56 smithi133 ceph-mon[31430]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:32:57.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:56 smithi138 ceph-mon[35142]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:32:57.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:56 smithi178 ceph-mon[35882]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:32:59.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:32:58 smithi138 ceph-mon[35142]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-11T23:32:59.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:32:58 smithi133 ceph-mon[31430]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-11T23:32:59.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:32:58 smithi178 ceph-mon[35882]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-11T23:33:01.185 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:00 smithi133 ceph-mon[31430]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:33:01.185 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:33:01.185 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:00 smithi138 ceph-mon[35142]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:33:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:33:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:00 smithi178 ceph-mon[35882]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:33:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:33:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:02.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:01 smithi138 ceph-mon[35142]: Deploying daemon osd.9 on smithi133 2024-09-11T23:33:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:01 smithi133 ceph-mon[31430]: Deploying daemon osd.9 on smithi133 2024-09-11T23:33:02.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:01 smithi178 ceph-mon[35882]: Deploying daemon osd.9 on smithi133 2024-09-11T23:33:03.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:02 smithi138 ceph-mon[35142]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:33:03.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:02 smithi133 ceph-mon[31430]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:33:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:02 smithi178 ceph-mon[35882]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:33:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:04 smithi133 ceph-mon[31430]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-11T23:33:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:04 smithi138 ceph-mon[35142]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-11T23:33:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:04 smithi178 ceph-mon[35882]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-11T23:33:07.118 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:06 smithi133 ceph-mon[31430]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:33:07.118 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:06 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:07.118 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:06 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:07.118 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:06 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:06 smithi138 ceph-mon[35142]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:33:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:06 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:06 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:06 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:07.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:06 smithi178 ceph-mon[35882]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:33:07.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:06 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:07.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:06 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:07.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:06 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:08 smithi138 ceph-mon[35142]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:33:09.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:08 smithi133 ceph-mon[31430]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:33:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:08 smithi178 ceph-mon[35882]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:33:09.497 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 9 on host 'smithi133' 2024-09-11T23:33:09.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.497+0000 7fb9257fa640 1 -- 172.21.15.133:0/2481452982 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb928062630 con 0x7fb9100a2c30 2024-09-11T23:33:09.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 -- 172.21.15.133:0/2481452982 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb9100a2c30 msgr2=0x7fb9100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:09.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb9100a2c30 0x7fb9100a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb918000c00 tx=0x7fb9180046c0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 -- 172.21.15.133:0/2481452982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9281037c0 msgr2=0x7fb92819f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:09.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9281037c0 0x7fb92819f4d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb920009fd0 tx=0x7fb92000ef10 comp rx=0 tx=0).stop 2024-09-11T23:33:09.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 -- 172.21.15.133:0/2481452982 shutdown_connections 2024-09-11T23:33:09.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb9100a2c30 0x7fb9100a50f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9281037c0 0x7fb92819f4d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928102e00 0x7fb928198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 --2- 172.21.15.133:0/2481452982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb928101c00 0x7fb928197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 -- 172.21.15.133:0/2481452982 >> 172.21.15.133:0/2481452982 conn(0x7fb9280fd390 msgr2=0x7fb928106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:09.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 -- 172.21.15.133:0/2481452982 shutdown_connections 2024-09-11T23:33:09.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:09.500+0000 7fb92e417640 1 -- 172.21.15.133:0/2481452982 wait complete. 2024-09-11T23:33:10.538 DEBUG:teuthology.orchestra.run.smithi133:osd.9> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.9.service 2024-09-11T23:33:10.541 INFO:tasks.cephadm:Deploying osd.10 on smithi138 with /dev/vg_nvme/lv_1... 2024-09-11T23:33:10.541 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:33:10.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:10 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:10.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:10 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:10.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:10 smithi178 ceph-mon[35882]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:33:10.797 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:33:10.918 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:10 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:10.918 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:10 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:10.918 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:10 smithi138 ceph-mon[35142]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:33:10.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:10 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:10.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:10 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:10.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:10 smithi133 ceph-mon[31430]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:33:11.788 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:11.788 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:12.025 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:12.025 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:12.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:33:12 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:33:12.398+0000 7f52ab318640 -1 osd.9 0 log_to_monitors true 2024-09-11T23:33:13.027 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:12 smithi138 ceph-mon[35142]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:33:13.027 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:12 smithi138 ceph-mon[35142]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:33:13.029 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-11T23:33:13.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:12 smithi178 ceph-mon[35882]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:33:13.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:12 smithi178 ceph-mon[35882]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:33:13.055 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_1 2024-09-11T23:33:13.072 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:12 smithi133 ceph-mon[31430]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:33:13.072 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:12 smithi133 ceph-mon[31430]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:33:13.313 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:33:13.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.750+0000 7fc0fafea640 1 -- 172.21.15.138:0/108441584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f4102fb0 msgr2=0x7fc0f4103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:13.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.750+0000 7fc0fafea640 1 --2- 172.21.15.138:0/108441584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f4102fb0 0x7fc0f4103430 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009930 tx=0x7fc0e002a420 comp rx=0 tx=0).stop 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.750+0000 7fc0fafea640 1 -- 172.21.15.138:0/108441584 shutdown_connections 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.750+0000 7fc0fafea640 1 --2- 172.21.15.138:0/108441584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0f4103970 0x7fc0f410a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.750+0000 7fc0fafea640 1 --2- 172.21.15.138:0/108441584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f4102fb0 0x7fc0f4103430 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.750+0000 7fc0fafea640 1 --2- 172.21.15.138:0/108441584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f40fff30 0x7fc0f4100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.750+0000 7fc0fafea640 1 -- 172.21.15.138:0/108441584 >> 172.21.15.138:0/108441584 conn(0x7fc0f40fba80 msgr2=0x7fc0f40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.750+0000 7fc0fafea640 1 -- 172.21.15.138:0/108441584 shutdown_connections 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.750+0000 7fc0fafea640 1 -- 172.21.15.138:0/108441584 wait complete. 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0fafea640 1 Processor -- start 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0fafea640 1 -- start start 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0fafea640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0f40fff30 0x7fc0f4102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0fafea640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f4102fb0 0x7fc0f4100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:13.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0fafea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f4103970 0x7fc0f41014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0fafea640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f410d880 con 0x7fc0f4102fb0 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0fafea640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f410d700 con 0x7fc0f4103970 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0fafea640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f410da00 con 0x7fc0f40fff30 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f9587640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f4103970 0x7fc0f41014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f9587640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f4103970 0x7fc0f41014a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43366/0 (socket says 172.21.15.138:43366) 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f9587640 1 -- 172.21.15.138:0/943143156 learned_addr learned my addr 172.21.15.138:0/943143156 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f8d86640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0f40fff30 0x7fc0f4102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f3fff640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f4102fb0 0x7fc0f4100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f9587640 1 -- 172.21.15.138:0/943143156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0f40fff30 msgr2=0x7fc0f4102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f9587640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0f40fff30 0x7fc0f4102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f9587640 1 -- 172.21.15.138:0/943143156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f4102fb0 msgr2=0x7fc0f4100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:13.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f9587640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f4102fb0 0x7fc0f4100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f9587640 1 -- 172.21.15.138:0/943143156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f4101d60 con 0x7fc0f4103970 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f9587640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f4103970 0x7fc0f41014a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00eaa0 tx=0x7fc0ec00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.751+0000 7fc0f1ffb640 1 -- 172.21.15.138:0/943143156 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00cdb0 con 0x7fc0f4103970 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.752+0000 7fc0f1ffb640 1 -- 172.21.15.138:0/943143156 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ec005020 con 0x7fc0f4103970 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.752+0000 7fc0fafea640 1 -- 172.21.15.138:0/943143156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f41a20f0 con 0x7fc0f4103970 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.752+0000 7fc0f1ffb640 1 -- 172.21.15.138:0/943143156 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec0116b0 con 0x7fc0f4103970 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.752+0000 7fc0fafea640 1 -- 172.21.15.138:0/943143156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f41a2600 con 0x7fc0f4103970 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.753+0000 7fc0fafea640 1 -- 172.21.15.138:0/943143156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f4109250 con 0x7fc0f4103970 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.754+0000 7fc0f1ffb640 1 -- 172.21.15.138:0/943143156 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc0ec005560 con 0x7fc0f4103970 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.754+0000 7fc0f1ffb640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc0e40a2b80 0x7fc0e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.755+0000 7fc0f1ffb640 1 -- 172.21.15.138:0/943143156 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5965+0+0 (secure 0 0 0) 0x7fc0ec0ce960 con 0x7fc0f4103970 2024-09-11T23:33:13.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.755+0000 7fc0f8d86640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc0e40a2b80 0x7fc0e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:13.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.755+0000 7fc0f8d86640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc0e40a2b80 0x7fc0e40a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4069cb0 tx=0x7fc0e800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:13.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.759+0000 7fc0f1ffb640 1 -- 172.21.15.138:0/943143156 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0ec016030 con 0x7fc0f4103970 2024-09-11T23:33:13.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:13.941+0000 7fc0fafea640 1 -- 172.21.15.138:0/943143156 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc0f4062630 con 0x7fc0e40a2b80 2024-09-11T23:33:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:13 smithi138 ceph-mon[35142]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:33:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:13 smithi138 ceph-mon[35142]: osdmap e59: 10 total, 9 up, 10 in 2024-09-11T23:33:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:13 smithi138 ceph-mon[35142]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:33:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:13 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:13.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:13 smithi133 ceph-mon[31430]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:33:13.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:13 smithi133 ceph-mon[31430]: osdmap e59: 10 total, 9 up, 10 in 2024-09-11T23:33:13.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:13 smithi133 ceph-mon[31430]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:33:13.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:13 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:14.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:13 smithi178 ceph-mon[35882]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:33:14.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:13 smithi178 ceph-mon[35882]: osdmap e59: 10 total, 9 up, 10 in 2024-09-11T23:33:14.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:13 smithi178 ceph-mon[35882]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-11T23:33:14.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:13 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:14.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:33:14.566+0000 7f52a6a78640 -1 osd.9 0 waiting for initial osdmap 2024-09-11T23:33:14.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:33:14.582+0000 7f529f0aa640 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:33:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:33:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T23:33:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:33:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:14.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:14.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:14.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' 2024-09-11T23:33:14.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:14.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:14.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:14.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:14.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:14 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:15.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:33:15.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T23:33:15.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:15.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:15.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:15.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:15.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' 2024-09-11T23:33:15.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:15.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:15.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:15.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:15.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:14 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:15.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574]' entity='osd.9' 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:15.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:14 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:16.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:15 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:33:16.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:15 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:33:16.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:15 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi133 2024-09-11T23:33:16.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:15 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi133 to 4272M 2024-09-11T23:33:16.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:15 smithi178 ceph-mon[35882]: osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574] boot 2024-09-11T23:33:16.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:15 smithi178 ceph-mon[35882]: osdmap e61: 10 total, 10 up, 10 in 2024-09-11T23:33:16.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:15 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:16.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:15 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:33:16.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:15 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:33:16.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:15 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi133 2024-09-11T23:33:16.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:15 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi133 to 4272M 2024-09-11T23:33:16.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:15 smithi138 ceph-mon[35142]: osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574] boot 2024-09-11T23:33:16.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:15 smithi138 ceph-mon[35142]: osdmap e61: 10 total, 10 up, 10 in 2024-09-11T23:33:16.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:15 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:16.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:15 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:33:16.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:15 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:33:16.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:15 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi133 2024-09-11T23:33:16.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:15 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi133 to 4272M 2024-09-11T23:33:16.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:15 smithi133 ceph-mon[31430]: osd.9 [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574] boot 2024-09-11T23:33:16.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:15 smithi133 ceph-mon[31430]: osdmap e61: 10 total, 10 up, 10 in 2024-09-11T23:33:16.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:15 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:33:16.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:16 smithi138 ceph-mon[35142]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:16.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:16 smithi138 ceph-mon[35142]: from='client.? 172.21.15.138:0/3774617890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46e606d4-591e-415d-8aae-06d311761798"}]: dispatch 2024-09-11T23:33:16.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:16 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46e606d4-591e-415d-8aae-06d311761798"}]: dispatch 2024-09-11T23:33:16.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:16 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46e606d4-591e-415d-8aae-06d311761798"}]': finished 2024-09-11T23:33:16.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:16 smithi138 ceph-mon[35142]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T23:33:16.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:16.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:16 smithi138 ceph-mon[35142]: from='client.? 172.21.15.138:0/3661289041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:33:17.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:16 smithi178 ceph-mon[35882]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:17.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:16 smithi178 ceph-mon[35882]: from='client.? 172.21.15.138:0/3774617890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46e606d4-591e-415d-8aae-06d311761798"}]: dispatch 2024-09-11T23:33:17.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:16 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46e606d4-591e-415d-8aae-06d311761798"}]: dispatch 2024-09-11T23:33:17.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:16 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46e606d4-591e-415d-8aae-06d311761798"}]': finished 2024-09-11T23:33:17.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:16 smithi178 ceph-mon[35882]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T23:33:17.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:17.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:16 smithi178 ceph-mon[35882]: from='client.? 172.21.15.138:0/3661289041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:33:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:16 smithi133 ceph-mon[31430]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:16 smithi133 ceph-mon[31430]: from='client.? 172.21.15.138:0/3774617890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46e606d4-591e-415d-8aae-06d311761798"}]: dispatch 2024-09-11T23:33:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:16 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46e606d4-591e-415d-8aae-06d311761798"}]: dispatch 2024-09-11T23:33:17.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:16 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46e606d4-591e-415d-8aae-06d311761798"}]': finished 2024-09-11T23:33:17.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:16 smithi133 ceph-mon[31430]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T23:33:17.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:17.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:16 smithi133 ceph-mon[31430]: from='client.? 172.21.15.138:0/3661289041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:33:18.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:18 smithi138 ceph-mon[35142]: osdmap e63: 11 total, 10 up, 11 in 2024-09-11T23:33:18.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:18 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:18.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:18 smithi138 ceph-mon[35142]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:18.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:18 smithi133 ceph-mon[31430]: osdmap e63: 11 total, 10 up, 11 in 2024-09-11T23:33:18.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:18 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:18.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:18 smithi133 ceph-mon[31430]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:18.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:18 smithi178 ceph-mon[35882]: osdmap e63: 11 total, 10 up, 11 in 2024-09-11T23:33:18.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:18 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:18.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:18 smithi178 ceph-mon[35882]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:20 smithi138 ceph-mon[35142]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:21.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:20 smithi133 ceph-mon[31430]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:21.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:20 smithi178 ceph-mon[35882]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:23.127 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:22 smithi138 ceph-mon[35142]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:22 smithi133 ceph-mon[31430]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:22 smithi178 ceph-mon[35882]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:24 smithi138 ceph-mon[35142]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:33:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:24 smithi133 ceph-mon[31430]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:33:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:25.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:24 smithi178 ceph-mon[35882]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:25.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:33:25.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:26.160 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:25 smithi138 ceph-mon[35142]: Deploying daemon osd.10 on smithi138 2024-09-11T23:33:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:25 smithi133 ceph-mon[31430]: Deploying daemon osd.10 on smithi138 2024-09-11T23:33:26.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:25 smithi178 ceph-mon[35882]: Deploying daemon osd.10 on smithi138 2024-09-11T23:33:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:26 smithi138 ceph-mon[35142]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:27.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:26 smithi133 ceph-mon[31430]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:27.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:26 smithi178 ceph-mon[35882]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:28 smithi138 ceph-mon[35142]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:28 smithi133 ceph-mon[31430]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:29.275 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:28 smithi178 ceph-mon[35882]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:31.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:30 smithi138 ceph-mon[35142]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:30 smithi133 ceph-mon[31430]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:30 smithi178 ceph-mon[35882]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:33.177 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:32 smithi138 ceph-mon[35142]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:33.178 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:32 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:33.178 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:32 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:33.178 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:32 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:33.178 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:32 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:33.178 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:32 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:32 smithi133 ceph-mon[31430]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:32 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:32 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:32 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:32 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:32 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:32 smithi178 ceph-mon[35882]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:32 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:32 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:32 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:32 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:32 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.451+0000 7fc0f1ffb640 1 -- 172.21.15.138:0/943143156 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc0f4062630 con 0x7fc0e40a2b80 2024-09-11T23:33:33.451 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 10 on host 'smithi138' 2024-09-11T23:33:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.453+0000 7fc0fafea640 1 -- 172.21.15.138:0/943143156 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc0e40a2b80 msgr2=0x7fc0e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.453+0000 7fc0fafea640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc0e40a2b80 0x7fc0e40a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4069cb0 tx=0x7fc0e800a2d0 comp rx=0 tx=0).stop 2024-09-11T23:33:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.453+0000 7fc0fafea640 1 -- 172.21.15.138:0/943143156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f4103970 msgr2=0x7fc0f41014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.453+0000 7fc0fafea640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f4103970 0x7fc0f41014a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00eaa0 tx=0x7fc0ec00ef60 comp rx=0 tx=0).stop 2024-09-11T23:33:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.453+0000 7fc0fafea640 1 -- 172.21.15.138:0/943143156 shutdown_connections 2024-09-11T23:33:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.453+0000 7fc0fafea640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc0e40a2b80 0x7fc0e40a5040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.453+0000 7fc0fafea640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f4103970 0x7fc0f41014a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.453+0000 7fc0fafea640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f4102fb0 0x7fc0f4100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.454+0000 7fc0fafea640 1 --2- 172.21.15.138:0/943143156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0f40fff30 0x7fc0f4102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.454+0000 7fc0fafea640 1 -- 172.21.15.138:0/943143156 >> 172.21.15.138:0/943143156 conn(0x7fc0f40fba80 msgr2=0x7fc0f4106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.454+0000 7fc0fafea640 1 -- 172.21.15.138:0/943143156 shutdown_connections 2024-09-11T23:33:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-11T23:33:33.454+0000 7fc0fafea640 1 -- 172.21.15.138:0/943143156 wait complete. 2024-09-11T23:33:34.032 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:33 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:34.032 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:33 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:34.032 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:33 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:34.058 DEBUG:teuthology.orchestra.run.smithi138:osd.10> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.10.service 2024-09-11T23:33:34.061 INFO:tasks.cephadm:Deploying osd.11 on smithi178 with /dev/vg_nvme/lv_1... 2024-09-11T23:33:34.061 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:33:34.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:33 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:34.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:33 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:34.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:33 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:34.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:33 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:34.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:33 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:34.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:33 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:34.317 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:33:35.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:34 smithi178 ceph-mon[35882]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:35.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:34 smithi138 ceph-mon[35142]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:34 smithi133 ceph-mon[31430]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:36.448 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:33:36.236+0000 7f88b264b640 -1 osd.10 0 log_to_monitors true 2024-09-11T23:33:36.891 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T23:33:36.923 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_1 2024-09-11T23:33:37.172 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:33:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='osd.10 [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:33:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:33:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:37.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:37.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:36 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='osd.10 [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:33:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:33:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:37.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:37.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:36 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:37.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='osd.10 [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:33:37.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:33:37.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:33:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:33:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:36 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:37.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.617+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1045692025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd94102690 msgr2=0x7fcd94108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.617+0000 7fcd9b30e640 1 --2- 172.21.15.178:0/1045692025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd94102690 0x7fcd94108f20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009ca0 tx=0x7fcd8c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.617+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1045692025 shutdown_connections 2024-09-11T23:33:37.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.617+0000 7fcd9b30e640 1 --2- 172.21.15.178:0/1045692025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd94102690 0x7fcd94108f20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.617+0000 7fcd9b30e640 1 --2- 172.21.15.178:0/1045692025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd94101cd0 0x7fcd94102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.617+0000 7fcd9b30e640 1 --2- 172.21.15.178:0/1045692025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94100ad0 0x7fcd94100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.617+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1045692025 >> 172.21.15.178:0/1045692025 conn(0x7fcd940fc280 msgr2=0x7fcd940fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.617+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1045692025 shutdown_connections 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.617+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1045692025 wait complete. 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.617+0000 7fcd9b30e640 1 Processor -- start 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.617+0000 7fcd9b30e640 1 -- start start 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd9b30e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd94100ad0 0x7fcd94197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd9b30e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94101cd0 0x7fcd941983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd9b30e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd94102690 0x7fcd9419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd9b30e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9410c7d0 con 0x7fcd94100ad0 2024-09-11T23:33:37.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd9b30e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9410c650 con 0x7fcd94101cd0 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd9b30e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9410c950 con 0x7fcd94102690 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd998ab640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd94102690 0x7fcd9419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd998ab640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd94102690 0x7fcd9419f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51120/0 (socket says 172.21.15.178:51120) 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd998ab640 1 -- 172.21.15.178:0/1479348078 learned_addr learned my addr 172.21.15.178:0/1479348078 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd988a9640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94101cd0 0x7fcd941983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd990aa640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd94100ad0 0x7fcd94197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd998ab640 1 -- 172.21.15.178:0/1479348078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94101cd0 msgr2=0x7fcd941983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd998ab640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94101cd0 0x7fcd941983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd998ab640 1 -- 172.21.15.178:0/1479348078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd94100ad0 msgr2=0x7fcd94197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd998ab640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd94100ad0 0x7fcd94197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd998ab640 1 -- 172.21.15.178:0/1479348078 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9419fae0 con 0x7fcd94102690 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd990aa640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd94100ad0 0x7fcd94197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:33:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd998ab640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd94102690 0x7fcd9419f470 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0098d0 tx=0x7fcd8c02f840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd8a7fc640 1 -- 172.21.15.178:0/1479348078 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c02fc30 con 0x7fcd94102690 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1479348078 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9419fd70 con 0x7fcd94102690 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd8a7fc640 1 -- 172.21.15.178:0/1479348078 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd8c003e50 con 0x7fcd94102690 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd8a7fc640 1 -- 172.21.15.178:0/1479348078 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c004810 con 0x7fcd94102690 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.618+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1479348078 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd941a02f0 con 0x7fcd94102690 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.619+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1479348078 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd941a0550 con 0x7fcd94102690 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.620+0000 7fcd8a7fc640 1 -- 172.21.15.178:0/1479348078 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcd8c004b40 con 0x7fcd94102690 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.621+0000 7fcd8a7fc640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd800a2bb0 0x7fcd800a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.621+0000 7fcd8a7fc640 1 -- 172.21.15.178:0/1479348078 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6398+0+0 (secure 0 0 0) 0x7fcd8c0efdb0 con 0x7fcd94102690 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.621+0000 7fcd990aa640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd800a2bb0 0x7fcd800a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.622+0000 7fcd990aa640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd800a2bb0 0x7fcd800a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcd90006fd0 tx=0x7fcd90008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.624+0000 7fcd8a7fc640 1 -- 172.21.15.178:0/1479348078 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd8c0bd490 con 0x7fcd94102690 2024-09-11T23:33:37.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:37.799+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1479348078 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcd941a0720 con 0x7fcd800a2bb0 2024-09-11T23:33:38.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:33:37.943+0000 7f88addab640 -1 osd.10 0 waiting for initial osdmap 2024-09-11T23:33:38.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:33:37.957+0000 7f88aa3e5640 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:33:38.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi138 2024-09-11T23:33:38.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi138 to 4272M 2024-09-11T23:33:38.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-mon[35142]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:33:38.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-mon[35142]: osdmap e64: 11 total, 10 up, 11 in 2024-09-11T23:33:38.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:38.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-mon[35142]: from='osd.10 [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:33:38.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-mon[35142]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:33:38.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:33:38.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:33:38.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:37 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:38.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:37 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi138 2024-09-11T23:33:38.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:37 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi138 to 4272M 2024-09-11T23:33:38.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:37 smithi133 ceph-mon[31430]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:33:38.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:37 smithi133 ceph-mon[31430]: osdmap e64: 11 total, 10 up, 11 in 2024-09-11T23:33:38.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:37 smithi133 ceph-mon[31430]: from='osd.10 [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:33:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:37 smithi133 ceph-mon[31430]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:33:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:33:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:33:38.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:37 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:38.210 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:37 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi138 2024-09-11T23:33:38.210 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:37 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi138 to 4272M 2024-09-11T23:33:38.210 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:37 smithi178 ceph-mon[35882]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:33:38.210 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:37 smithi178 ceph-mon[35882]: osdmap e64: 11 total, 10 up, 11 in 2024-09-11T23:33:38.210 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:37 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:38.211 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:37 smithi178 ceph-mon[35882]: from='osd.10 [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:33:38.211 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:37 smithi178 ceph-mon[35882]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-09-11T23:33:38.211 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:37 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:33:38.211 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:37 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:33:38.211 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:37 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:38 smithi138 ceph-mon[35142]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:33:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:38 smithi138 ceph-mon[35142]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:38 smithi138 ceph-mon[35142]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:33:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:38 smithi138 ceph-mon[35142]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T23:33:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:38 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:38 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:38 smithi138 ceph-mon[35142]: osd.10 [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034] boot 2024-09-11T23:33:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:38 smithi138 ceph-mon[35142]: osdmap e66: 11 total, 11 up, 11 in 2024-09-11T23:33:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:38 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:38 smithi133 ceph-mon[31430]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:33:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:38 smithi133 ceph-mon[31430]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:38 smithi133 ceph-mon[31430]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:33:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:38 smithi133 ceph-mon[31430]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T23:33:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:38 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:38 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:38 smithi133 ceph-mon[31430]: osd.10 [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034] boot 2024-09-11T23:33:39.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:38 smithi133 ceph-mon[31430]: osdmap e66: 11 total, 11 up, 11 in 2024-09-11T23:33:39.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:38 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:39.282 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:38 smithi178 ceph-mon[35882]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:33:39.282 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:38 smithi178 ceph-mon[35882]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:33:39.282 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:38 smithi178 ceph-mon[35882]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-11T23:33:39.282 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:38 smithi178 ceph-mon[35882]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T23:33:39.282 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:38 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:39.282 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:38 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:39.282 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:38 smithi178 ceph-mon[35882]: osd.10 [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034] boot 2024-09-11T23:33:39.282 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:38 smithi178 ceph-mon[35882]: osdmap e66: 11 total, 11 up, 11 in 2024-09-11T23:33:39.282 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:38 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:33:40.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:39 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:33:40.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:39 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:33:40.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:39 smithi138 ceph-mon[35142]: osdmap e67: 11 total, 11 up, 11 in 2024-09-11T23:33:40.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:39 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:33:40.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:39 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:33:40.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:39 smithi133 ceph-mon[31430]: osdmap e67: 11 total, 11 up, 11 in 2024-09-11T23:33:40.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:39 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:33:40.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:39 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:33:40.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:39 smithi178 ceph-mon[35882]: osdmap e67: 11 total, 11 up, 11 in 2024-09-11T23:33:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:40 smithi138 ceph-mon[35142]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:40 smithi138 ceph-mon[35142]: from='client.? 172.21.15.178:0/407504173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "652ee9fd-003e-4bc5-80db-25e7880754d3"}]: dispatch 2024-09-11T23:33:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:40 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "652ee9fd-003e-4bc5-80db-25e7880754d3"}]: dispatch 2024-09-11T23:33:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:40 smithi138 ceph-mon[35142]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "652ee9fd-003e-4bc5-80db-25e7880754d3"}]': finished 2024-09-11T23:33:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:40 smithi138 ceph-mon[35142]: osdmap e68: 12 total, 11 up, 12 in 2024-09-11T23:33:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:33:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:40 smithi138 ceph-mon[35142]: from='client.? 172.21.15.178:0/4235405253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:33:41.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:40 smithi133 ceph-mon[31430]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:41.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:40 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/407504173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "652ee9fd-003e-4bc5-80db-25e7880754d3"}]: dispatch 2024-09-11T23:33:41.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:40 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "652ee9fd-003e-4bc5-80db-25e7880754d3"}]: dispatch 2024-09-11T23:33:41.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:40 smithi133 ceph-mon[31430]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "652ee9fd-003e-4bc5-80db-25e7880754d3"}]': finished 2024-09-11T23:33:41.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:40 smithi133 ceph-mon[31430]: osdmap e68: 12 total, 11 up, 12 in 2024-09-11T23:33:41.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:33:41.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:40 smithi133 ceph-mon[31430]: from='client.? 172.21.15.178:0/4235405253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:33:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:40 smithi178 ceph-mon[35882]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:40 smithi178 ceph-mon[35882]: from='client.? 172.21.15.178:0/407504173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "652ee9fd-003e-4bc5-80db-25e7880754d3"}]: dispatch 2024-09-11T23:33:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:40 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "652ee9fd-003e-4bc5-80db-25e7880754d3"}]: dispatch 2024-09-11T23:33:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:40 smithi178 ceph-mon[35882]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "652ee9fd-003e-4bc5-80db-25e7880754d3"}]': finished 2024-09-11T23:33:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:40 smithi178 ceph-mon[35882]: osdmap e68: 12 total, 11 up, 12 in 2024-09-11T23:33:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:33:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:40 smithi178 ceph-mon[35882]: from='client.? 172.21.15.178:0/4235405253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:33:42.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:42 smithi178 ceph-mon[35882]: osdmap e69: 12 total, 11 up, 12 in 2024-09-11T23:33:42.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:42 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:33:42.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:42 smithi178 ceph-mon[35882]: pgmap v226: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-09-11T23:33:42.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:42 smithi138 ceph-mon[35142]: osdmap e69: 12 total, 11 up, 12 in 2024-09-11T23:33:42.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:42 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:33:42.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:42 smithi138 ceph-mon[35142]: pgmap v226: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-09-11T23:33:42.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:42 smithi133 ceph-mon[31430]: osdmap e69: 12 total, 11 up, 12 in 2024-09-11T23:33:42.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:42 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:33:42.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:42 smithi133 ceph-mon[31430]: pgmap v226: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-09-11T23:33:43.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:43 smithi178 ceph-mon[35882]: osdmap e70: 12 total, 11 up, 12 in 2024-09-11T23:33:43.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:43 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:33:43.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:43 smithi138 ceph-mon[35142]: osdmap e70: 12 total, 11 up, 12 in 2024-09-11T23:33:43.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:43 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:33:43.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:43 smithi133 ceph-mon[31430]: osdmap e70: 12 total, 11 up, 12 in 2024-09-11T23:33:43.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:43 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:33:44.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:44 smithi178 ceph-mon[35882]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-11T23:33:44.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:44 smithi138 ceph-mon[35142]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-11T23:33:44.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:44 smithi133 ceph-mon[31430]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-11T23:33:47.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:46 smithi138 ceph-mon[35142]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:33:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:46 smithi133 ceph-mon[31430]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:33:47.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:46 smithi178 ceph-mon[35882]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:33:49.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:48 smithi138 ceph-mon[35142]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-11T23:33:49.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:48 smithi133 ceph-mon[31430]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-11T23:33:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:48 smithi178 ceph-mon[35882]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-11T23:33:50.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:33:50.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:50.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:49 smithi178 ceph-mon[35882]: Deploying daemon osd.11 on smithi178 2024-09-11T23:33:50.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:33:50.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:50.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:49 smithi138 ceph-mon[35142]: Deploying daemon osd.11 on smithi178 2024-09-11T23:33:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:33:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:33:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:49 smithi133 ceph-mon[31430]: Deploying daemon osd.11 on smithi178 2024-09-11T23:33:51.156 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:50 smithi178 ceph-mon[35882]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:51.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:50 smithi138 ceph-mon[35142]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:51.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:50 smithi133 ceph-mon[31430]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:52 smithi138 ceph-mon[35142]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:53.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:52 smithi133 ceph-mon[31430]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:52 smithi178 ceph-mon[35882]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:55.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:54 smithi138 ceph-mon[35142]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:55.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:55.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:55.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:54 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:55.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:54 smithi133 ceph-mon[31430]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:55.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:55.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:55.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:54 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:55.255 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:54 smithi178 ceph-mon[35882]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:55.255 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:33:55.255 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:55.255 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:54 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:57.156 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:56 smithi178 ceph-mon[35882]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:57.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:56 smithi138 ceph-mon[35142]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:57.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:56 smithi133 ceph-mon[31430]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:58.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.138+0000 7fcd8a7fc640 1 -- 172.21.15.178:0/1479348078 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcd941a0720 con 0x7fcd800a2bb0 2024-09-11T23:33:58.139 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 11 on host 'smithi178' 2024-09-11T23:33:58.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1479348078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd800a2bb0 msgr2=0x7fcd800a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:58.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd800a2bb0 0x7fcd800a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcd90006fd0 tx=0x7fcd90008040 comp rx=0 tx=0).stop 2024-09-11T23:33:58.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1479348078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd94102690 msgr2=0x7fcd9419f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:58.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd94102690 0x7fcd9419f470 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0098d0 tx=0x7fcd8c02f840 comp rx=0 tx=0).stop 2024-09-11T23:33:58.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1479348078 shutdown_connections 2024-09-11T23:33:58.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd800a2bb0 0x7fcd800a5070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:58.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd94102690 0x7fcd9419f470 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:58.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94101cd0 0x7fcd941983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:58.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 --2- 172.21.15.178:0/1479348078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd94100ad0 0x7fcd94197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:58.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1479348078 >> 172.21.15.178:0/1479348078 conn(0x7fcd940fc280 msgr2=0x7fcd940fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:58.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1479348078 shutdown_connections 2024-09-11T23:33:58.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:33:58.141+0000 7fcd9b30e640 1 -- 172.21.15.178:0/1479348078 wait complete. 2024-09-11T23:33:58.872 DEBUG:teuthology.orchestra.run.smithi178:osd.11> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.11.service 2024-09-11T23:33:58.874 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-11T23:33:58.875 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T23:33:59.145 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:58 smithi138 ceph-mon[35142]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:58 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:33:58 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:59.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:58 smithi133 ceph-mon[31430]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:59.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:58 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:59.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:33:58 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:59.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:58 smithi178 ceph-mon[35882]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:33:59.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:58 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:59.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:33:58 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:33:59.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.608+0000 7f4f13cbd640 1 -- 172.21.15.133:0/1010488992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f0c100450 msgr2=0x7f4f0c102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:59.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.608+0000 7f4f13cbd640 1 --2- 172.21.15.133:0/1010488992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f0c100450 0x7f4f0c102860 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009900 tx=0x7f4f0802a410 comp rx=0 tx=0).stop 2024-09-11T23:33:59.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.608+0000 7f4f13cbd640 1 -- 172.21.15.133:0/1010488992 shutdown_connections 2024-09-11T23:33:59.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.608+0000 7f4f13cbd640 1 --2- 172.21.15.133:0/1010488992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f0c105750 0x7f4f0c105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:59.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.608+0000 7f4f13cbd640 1 --2- 172.21.15.133:0/1010488992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f0c102da0 0x7f4f0c105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:59.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.608+0000 7f4f13cbd640 1 --2- 172.21.15.133:0/1010488992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f0c100450 0x7f4f0c102860 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:59.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.608+0000 7f4f13cbd640 1 -- 172.21.15.133:0/1010488992 >> 172.21.15.133:0/1010488992 conn(0x7f4f0c0fc280 msgr2=0x7f4f0c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:59.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.608+0000 7f4f13cbd640 1 -- 172.21.15.133:0/1010488992 shutdown_connections 2024-09-11T23:33:59.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.608+0000 7f4f13cbd640 1 -- 172.21.15.133:0/1010488992 wait complete. 2024-09-11T23:33:59.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.608+0000 7f4f13cbd640 1 Processor -- start 2024-09-11T23:33:59.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.608+0000 7f4f13cbd640 1 -- start start 2024-09-11T23:33:59.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f13cbd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f0c100450 0x7f4f0c19e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:59.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f13cbd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f0c102da0 0x7f4f0c19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:59.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f13cbd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f0c105750 0x7f4f0c1a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:59.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f13cbd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c113e60 con 0x7f4f0c105750 2024-09-11T23:33:59.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f13cbd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c113ce0 con 0x7f4f0c102da0 2024-09-11T23:33:59.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f13cbd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c113fe0 con 0x7f4f0c100450 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f11a59640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f0c100450 0x7f4f0c19e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f1225a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f0c105750 0x7f4f0c1a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f11a59640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f0c100450 0x7f4f0c19e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57794/0 (socket says 172.21.15.133:57794) 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f1225a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f0c105750 0x7f4f0c1a5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45930/0 (socket says 172.21.15.133:45930) 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f11a59640 1 -- 172.21.15.133:0/2331001377 learned_addr learned my addr 172.21.15.133:0/2331001377 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f11258640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f0c102da0 0x7f4f0c19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f1225a640 1 -- 172.21.15.133:0/2331001377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f0c100450 msgr2=0x7f4f0c19e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f1225a640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f0c100450 0x7f4f0c19e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f1225a640 1 -- 172.21.15.133:0/2331001377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f0c102da0 msgr2=0x7f4f0c19e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f1225a640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f0c102da0 0x7f4f0c19e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f1225a640 1 -- 172.21.15.133:0/2331001377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c1a60d0 con 0x7f4f0c105750 2024-09-11T23:33:59.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f11a59640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f0c100450 0x7f4f0c19e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:33:59.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.609+0000 7f4f1225a640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f0c105750 0x7f4f0c1a5a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009fd0 tx=0x7f4f0400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:59.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.610+0000 7f4f02ffd640 1 -- 172.21.15.133:0/2331001377 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400ce20 con 0x7f4f0c105750 2024-09-11T23:33:59.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.610+0000 7f4f13cbd640 1 -- 172.21.15.133:0/2331001377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0c1a63c0 con 0x7f4f0c105750 2024-09-11T23:33:59.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.610+0000 7f4f02ffd640 1 -- 172.21.15.133:0/2331001377 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f04005020 con 0x7f4f0c105750 2024-09-11T23:33:59.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.610+0000 7f4f02ffd640 1 -- 172.21.15.133:0/2331001377 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04010720 con 0x7f4f0c105750 2024-09-11T23:33:59.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.610+0000 7f4f13cbd640 1 -- 172.21.15.133:0/2331001377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0c1a6950 con 0x7f4f0c105750 2024-09-11T23:33:59.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.611+0000 7f4f13cbd640 1 -- 172.21.15.133:0/2331001377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0c0fdd70 con 0x7f4f0c105750 2024-09-11T23:33:59.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.612+0000 7f4f02ffd640 1 -- 172.21.15.133:0/2331001377 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4f04005540 con 0x7f4f0c105750 2024-09-11T23:33:59.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.613+0000 7f4f02ffd640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4ef80a2bb0 0x7f4ef80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:59.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.613+0000 7f4f02ffd640 1 -- 172.21.15.133:0/2331001377 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6809+0+0 (secure 0 0 0) 0x7f4f040cf110 con 0x7f4f0c105750 2024-09-11T23:33:59.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.613+0000 7f4f11a59640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4ef80a2bb0 0x7f4ef80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:59.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.613+0000 7f4f11a59640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4ef80a2bb0 0x7f4ef80a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f08031440 tx=0x7f4f080315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:59.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.620+0000 7f4f02ffd640 1 -- 172.21.15.133:0/2331001377 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f0409c650 con 0x7f4f0c105750 2024-09-11T23:33:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.790+0000 7f4f13cbd640 1 -- 172.21.15.133:0/2331001377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4f0c062df0 con 0x7f4f0c105750 2024-09-11T23:33:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.790+0000 7f4f02ffd640 1 -- 172.21.15.133:0/2331001377 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f4f0409bf50 con 0x7f4f0c105750 2024-09-11T23:33:59.790 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:33:59.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 -- 172.21.15.133:0/2331001377 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4ef80a2bb0 msgr2=0x7f4ef80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:59.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4ef80a2bb0 0x7f4ef80a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f08031440 tx=0x7f4f080315b0 comp rx=0 tx=0).stop 2024-09-11T23:33:59.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 -- 172.21.15.133:0/2331001377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f0c105750 msgr2=0x7f4f0c1a5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:59.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f0c105750 0x7f4f0c1a5a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009fd0 tx=0x7f4f0400ef10 comp rx=0 tx=0).stop 2024-09-11T23:33:59.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 -- 172.21.15.133:0/2331001377 shutdown_connections 2024-09-11T23:33:59.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4ef80a2bb0 0x7f4ef80a5070 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:59.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f0c105750 0x7f4f0c1a5a60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:59.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f0c102da0 0x7f4f0c19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:59.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 --2- 172.21.15.133:0/2331001377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f0c100450 0x7f4f0c19e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:59.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 -- 172.21.15.133:0/2331001377 >> 172.21.15.133:0/2331001377 conn(0x7f4f0c0fc280 msgr2=0x7f4f0c103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:59.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 -- 172.21.15.133:0/2331001377 shutdown_connections 2024-09-11T23:33:59.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:33:59.792+0000 7f4f13cbd640 1 -- 172.21.15.133:0/2331001377 wait complete. 2024-09-11T23:34:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:00 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2331001377' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:34:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:00 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2331001377' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:34:00.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:00 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2331001377' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:34:00.640 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726097618,"num_in_osds":12,"osd_in_since":1726097620,"num_remapped_pgs":0} 2024-09-11T23:34:01.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:01 smithi178 ceph-mon[35882]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:34:01.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:34:01 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:34:01.190+0000 7fd5f34a8640 -1 osd.11 0 log_to_monitors true 2024-09-11T23:34:01.302 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:01 smithi138 ceph-mon[35142]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:34:01.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:01 smithi133 ceph-mon[31430]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:34:01.641 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T23:34:01.900 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:02.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='osd.11 [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:34:02.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:34:02.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:34:02.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: Detected new or changed devices on smithi178 2024-09-11T23:34:02.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi178 to 4272M 2024-09-11T23:34:02.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:02.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:02.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:02 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='osd.11 [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:34:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:34:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:34:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: Detected new or changed devices on smithi178 2024-09-11T23:34:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi178 to 4272M 2024-09-11T23:34:02.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:02.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:02.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:02 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.372+0000 7fc69ec6f640 1 -- 172.21.15.133:0/2933443528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698100ad0 msgr2=0x7fc698100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:02.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.372+0000 7fc69ec6f640 1 --2- 172.21.15.133:0/2933443528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698100ad0 0x7fc698100ed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009900 tx=0x7fc68c02a410 comp rx=0 tx=0).stop 2024-09-11T23:34:02.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.372+0000 7fc69ec6f640 1 -- 172.21.15.133:0/2933443528 shutdown_connections 2024-09-11T23:34:02.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.372+0000 7fc69ec6f640 1 --2- 172.21.15.133:0/2933443528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc698102690 0x7fc698108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:02.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.372+0000 7fc69ec6f640 1 --2- 172.21.15.133:0/2933443528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc698101cd0 0x7fc698102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.372+0000 7fc69ec6f640 1 --2- 172.21.15.133:0/2933443528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698100ad0 0x7fc698100ed0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.372+0000 7fc69ec6f640 1 -- 172.21.15.133:0/2933443528 >> 172.21.15.133:0/2933443528 conn(0x7fc6980fc280 msgr2=0x7fc6980fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.372+0000 7fc69ec6f640 1 -- 172.21.15.133:0/2933443528 shutdown_connections 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.372+0000 7fc69ec6f640 1 -- 172.21.15.133:0/2933443528 wait complete. 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.372+0000 7fc69ec6f640 1 Processor -- start 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.372+0000 7fc69ec6f640 1 -- start start 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ec6f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698100ad0 0x7fc698197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ec6f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc698101cd0 0x7fc698198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ec6f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc698102690 0x7fc69819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ec6f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810c730 con 0x7fc698100ad0 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ec6f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810c5b0 con 0x7fc698102690 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ec6f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810c8b0 con 0x7fc698101cd0 2024-09-11T23:34:02.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ca0b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698100ad0 0x7fc698197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ca0b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698100ad0 0x7fc698197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54940/0 (socket says 172.21.15.133:54940) 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ca0b640 1 -- 172.21.15.133:0/3825302194 learned_addr learned my addr 172.21.15.133:0/3825302194 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc697fff640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc698101cd0 0x7fc698198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69d20c640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc698102690 0x7fc69819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ca0b640 1 -- 172.21.15.133:0/3825302194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc698101cd0 msgr2=0x7fc698198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ca0b640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc698101cd0 0x7fc698198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ca0b640 1 -- 172.21.15.133:0/3825302194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc698102690 msgr2=0x7fc69819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ca0b640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc698102690 0x7fc69819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ca0b640 1 -- 172.21.15.133:0/3825302194 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69819fb30 con 0x7fc698100ad0 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69d20c640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc698102690 0x7fc69819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ca0b640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698100ad0 0x7fc698197f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc68c031610 tx=0x7fc68c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:02.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc695ffb640 1 -- 172.21.15.133:0/3825302194 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c03f030 con 0x7fc698100ad0 2024-09-11T23:34:02.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc695ffb640 1 -- 172.21.15.133:0/3825302194 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc68c003ce0 con 0x7fc698100ad0 2024-09-11T23:34:02.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc69ec6f640 1 -- 172.21.15.133:0/3825302194 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69819fdc0 con 0x7fc698100ad0 2024-09-11T23:34:02.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.373+0000 7fc695ffb640 1 -- 172.21.15.133:0/3825302194 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c035070 con 0x7fc698100ad0 2024-09-11T23:34:02.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.374+0000 7fc69ec6f640 1 -- 172.21.15.133:0/3825302194 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6981a0210 con 0x7fc698100ad0 2024-09-11T23:34:02.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.375+0000 7fc69ec6f640 1 -- 172.21.15.133:0/3825302194 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6981a0500 con 0x7fc698100ad0 2024-09-11T23:34:02.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.376+0000 7fc695ffb640 1 -- 172.21.15.133:0/3825302194 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc68c042030 con 0x7fc698100ad0 2024-09-11T23:34:02.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.376+0000 7fc695ffb640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6880a2c30 0x7fc6880a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:02.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.376+0000 7fc697fff640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6880a2c30 0x7fc6880a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:02.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.377+0000 7fc695ffb640 1 -- 172.21.15.133:0/3825302194 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6831+0+0 (secure 0 0 0) 0x7fc68c0eef40 con 0x7fc698100ad0 2024-09-11T23:34:02.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.377+0000 7fc697fff640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6880a2c30 0x7fc6880a50f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc698199420 tx=0x7fc68400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:02.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.381+0000 7fc695ffb640 1 -- 172.21.15.133:0/3825302194 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc68c0bc470 con 0x7fc698100ad0 2024-09-11T23:34:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='osd.11 [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:34:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:34:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:34:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: Detected new or changed devices on smithi178 2024-09-11T23:34:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:34:02.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi178 to 4272M 2024-09-11T23:34:02.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:02.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:02.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:02 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:02.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.547+0000 7fc69ec6f640 1 -- 172.21.15.133:0/3825302194 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc6981a07f0 con 0x7fc698100ad0 2024-09-11T23:34:02.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.548+0000 7fc695ffb640 1 -- 172.21.15.133:0/3825302194 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7fc68c0bbd70 con 0x7fc698100ad0 2024-09-11T23:34:02.547 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:02.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 -- 172.21.15.133:0/3825302194 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6880a2c30 msgr2=0x7fc6880a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:02.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6880a2c30 0x7fc6880a50f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc698199420 tx=0x7fc68400b390 comp rx=0 tx=0).stop 2024-09-11T23:34:02.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 -- 172.21.15.133:0/3825302194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698100ad0 msgr2=0x7fc698197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:02.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698100ad0 0x7fc698197f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc68c031610 tx=0x7fc68c0043b0 comp rx=0 tx=0).stop 2024-09-11T23:34:02.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 -- 172.21.15.133:0/3825302194 shutdown_connections 2024-09-11T23:34:02.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6880a2c30 0x7fc6880a50f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:02.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc698102690 0x7fc69819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:02.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc698101cd0 0x7fc698198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:02.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 --2- 172.21.15.133:0/3825302194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698100ad0 0x7fc698197f00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:02.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 -- 172.21.15.133:0/3825302194 >> 172.21.15.133:0/3825302194 conn(0x7fc6980fc280 msgr2=0x7fc698069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:02.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 -- 172.21.15.133:0/3825302194 shutdown_connections 2024-09-11T23:34:02.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:02.550+0000 7fc69ec6f640 1 -- 172.21.15.133:0/3825302194 wait complete. 2024-09-11T23:34:03.295 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:03 smithi133 ceph-mon[31430]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:34:03.295 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:03 smithi133 ceph-mon[31430]: from='osd.11 [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:34:03.295 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:03 smithi133 ceph-mon[31430]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T23:34:03.295 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:03.295 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:03 smithi133 ceph-mon[31430]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:34:03.295 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:03.296 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:03.296 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:03.296 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:03 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:03.296 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:03 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3825302194' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:34:03.322 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726097618,"num_in_osds":12,"osd_in_since":1726097620,"num_remapped_pgs":0} 2024-09-11T23:34:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:03 smithi138 ceph-mon[35142]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:34:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:03 smithi138 ceph-mon[35142]: from='osd.11 [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:34:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:03 smithi138 ceph-mon[35142]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T23:34:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:03 smithi138 ceph-mon[35142]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:34:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:03 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:03 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3825302194' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:34:03.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:34:03.045+0000 7fd5eec08640 -1 osd.11 0 waiting for initial osdmap 2024-09-11T23:34:03.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:34:03.059+0000 7fd5eb242640 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:34:03.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-mon[35882]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:34:03.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-mon[35882]: from='osd.11 [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:34:03.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-mon[35882]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T23:34:03.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:03.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-mon[35882]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-11T23:34:03.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:03.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:03.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:03.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:03.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:03 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3825302194' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:34:04.323 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T23:34:04.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:04 smithi133 ceph-mon[31430]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:34:04.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:04 smithi133 ceph-mon[31430]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:34:04.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:04 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:04.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:04 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:04.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:04 smithi133 ceph-mon[31430]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:34:04.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:04 smithi138 ceph-mon[35142]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:34:04.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:04 smithi138 ceph-mon[35142]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:34:04.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:04 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:04.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:04 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:04.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:04 smithi138 ceph-mon[35142]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:34:04.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:04 smithi178 ceph-mon[35882]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T23:34:04.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:04 smithi178 ceph-mon[35882]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:34:04.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:04 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:04.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:04 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:04.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:04 smithi178 ceph-mon[35882]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:34:04.582 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:05.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.028+0000 7f096317e640 1 -- 172.21.15.133:0/4082641572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095c101d10 msgr2=0x7f095c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:05.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.028+0000 7f096317e640 1 --2- 172.21.15.133:0/4082641572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095c101d10 0x7f095c102190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0948009900 tx=0x7f094802a410 comp rx=0 tx=0).stop 2024-09-11T23:34:05.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.028+0000 7f096317e640 1 -- 172.21.15.133:0/4082641572 shutdown_connections 2024-09-11T23:34:05.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.028+0000 7f096317e640 1 --2- 172.21.15.133:0/4082641572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095c1026d0 0x7f095c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:05.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.028+0000 7f096317e640 1 --2- 172.21.15.133:0/4082641572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095c101d10 0x7f095c102190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:05.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.028+0000 7f096317e640 1 --2- 172.21.15.133:0/4082641572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f095c100b10 0x7f095c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:05.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.028+0000 7f096317e640 1 -- 172.21.15.133:0/4082641572 >> 172.21.15.133:0/4082641572 conn(0x7f095c0fc2a0 msgr2=0x7f095c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:05.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.028+0000 7f096317e640 1 -- 172.21.15.133:0/4082641572 shutdown_connections 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.028+0000 7f096317e640 1 -- 172.21.15.133:0/4082641572 wait complete. 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f096317e640 1 Processor -- start 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f096317e640 1 -- start start 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f096317e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095c100b10 0x7f095c06b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f096317e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095c101d10 0x7f095c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f096317e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f095c1026d0 0x7f095c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f096317e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10c770 con 0x7f095c100b10 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f096317e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10c5f0 con 0x7f095c101d10 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f096317e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10c8f0 con 0x7f095c1026d0 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f0960f1a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095c100b10 0x7f095c06b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f0960f1a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095c100b10 0x7f095c06b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54958/0 (socket says 172.21.15.133:54958) 2024-09-11T23:34:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f0960f1a640 1 -- 172.21.15.133:0/2327479528 learned_addr learned my addr 172.21.15.133:0/2327479528 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f0960f1a640 1 -- 172.21.15.133:0/2327479528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f095c1026d0 msgr2=0x7f095c19f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f0960f1a640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f095c1026d0 0x7f095c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f0960f1a640 1 -- 172.21.15.133:0/2327479528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095c101d10 msgr2=0x7f095c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f0960f1a640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095c101d10 0x7f095c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f0960f1a640 1 -- 172.21.15.133:0/2327479528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c19fa90 con 0x7f095c100b10 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.029+0000 7f0960f1a640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095c100b10 0x7f095c06b330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f095000c970 tx=0x7f095000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.030+0000 7f0959ffb640 1 -- 172.21.15.133:0/2327479528 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950007c10 con 0x7f095c100b10 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.030+0000 7f096317e640 1 -- 172.21.15.133:0/2327479528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c19fd80 con 0x7f095c100b10 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.030+0000 7f0959ffb640 1 -- 172.21.15.133:0/2327479528 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0950007db0 con 0x7f095c100b10 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.030+0000 7f0959ffb640 1 -- 172.21.15.133:0/2327479528 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095000f8a0 con 0x7f095c100b10 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.030+0000 7f096317e640 1 -- 172.21.15.133:0/2327479528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c1a01e0 con 0x7f095c100b10 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.031+0000 7f096317e640 1 -- 172.21.15.133:0/2327479528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f095c107ec0 con 0x7f095c100b10 2024-09-11T23:34:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.032+0000 7f0959ffb640 1 -- 172.21.15.133:0/2327479528 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f095000fb20 con 0x7f095c100b10 2024-09-11T23:34:05.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.032+0000 7f0959ffb640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f094c0a2b80 0x7f094c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:05.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.033+0000 7f095bfff640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f094c0a2b80 0x7f094c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:05.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.033+0000 7f0959ffb640 1 -- 172.21.15.133:0/2327479528 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f09500cef90 con 0x7f095c100b10 2024-09-11T23:34:05.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.033+0000 7f095bfff640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f094c0a2b80 0x7f094c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f09480096c0 tx=0x7f0948031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:05.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.037+0000 7f0959ffb640 1 -- 172.21.15.133:0/2327479528 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f095009c420 con 0x7f095c100b10 2024-09-11T23:34:05.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.204+0000 7f096317e640 1 -- 172.21.15.133:0/2327479528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f095c02d7c0 con 0x7f095c100b10 2024-09-11T23:34:05.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.205+0000 7f0959ffb640 1 -- 172.21.15.133:0/2327479528 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f095009bd20 con 0x7f095c100b10 2024-09-11T23:34:05.205 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 -- 172.21.15.133:0/2327479528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f094c0a2b80 msgr2=0x7f094c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f094c0a2b80 0x7f094c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f09480096c0 tx=0x7f0948031400 comp rx=0 tx=0).stop 2024-09-11T23:34:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 -- 172.21.15.133:0/2327479528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095c100b10 msgr2=0x7f095c06b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095c100b10 0x7f095c06b330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f095000c970 tx=0x7f095000ce30 comp rx=0 tx=0).stop 2024-09-11T23:34:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 -- 172.21.15.133:0/2327479528 shutdown_connections 2024-09-11T23:34:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f094c0a2b80 0x7f094c0a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f095c1026d0 0x7f095c19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095c101d10 0x7f095c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 --2- 172.21.15.133:0/2327479528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095c100b10 0x7f095c06b330 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 -- 172.21.15.133:0/2327479528 >> 172.21.15.133:0/2327479528 conn(0x7f095c0fc2a0 msgr2=0x7f095c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 -- 172.21.15.133:0/2327479528 shutdown_connections 2024-09-11T23:34:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:05.207+0000 7f096317e640 1 -- 172.21.15.133:0/2327479528 wait complete. 2024-09-11T23:34:05.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:05 smithi138 ceph-mon[35142]: purged_snaps scrub starts 2024-09-11T23:34:05.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:05 smithi138 ceph-mon[35142]: purged_snaps scrub ok 2024-09-11T23:34:05.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:05.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:05 smithi138 ceph-mon[35142]: osd.11 [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962] boot 2024-09-11T23:34:05.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:05 smithi138 ceph-mon[35142]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T23:34:05.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:05 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:05.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:05 smithi133 ceph-mon[31430]: purged_snaps scrub starts 2024-09-11T23:34:05.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:05 smithi133 ceph-mon[31430]: purged_snaps scrub ok 2024-09-11T23:34:05.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:05.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:05 smithi133 ceph-mon[31430]: osd.11 [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962] boot 2024-09-11T23:34:05.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:05 smithi133 ceph-mon[31430]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T23:34:05.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:05 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:05.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:05 smithi178 ceph-mon[35882]: purged_snaps scrub starts 2024-09-11T23:34:05.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:05 smithi178 ceph-mon[35882]: purged_snaps scrub ok 2024-09-11T23:34:05.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:05.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:05 smithi178 ceph-mon[35882]: osd.11 [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962] boot 2024-09-11T23:34:05.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:05 smithi178 ceph-mon[35882]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T23:34:05.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:05 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:34:05.898 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":74,"num_osds":12,"num_up_osds":12,"osd_up_since":1726097644,"num_in_osds":12,"osd_in_since":1726097620,"num_remapped_pgs":0} 2024-09-11T23:34:05.898 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:34:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:06.306 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:06 smithi133 ceph-mon[31430]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T23:34:06.306 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:06 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2327479528' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:34:06.307 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:06 smithi133 ceph-mon[31430]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:06.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:06 smithi138 ceph-mon[35142]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T23:34:06.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:06 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2327479528' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:34:06.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:06 smithi138 ceph-mon[35142]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:06.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:06 smithi178 ceph-mon[35882]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T23:34:06.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:06 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2327479528' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:34:06.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:06 smithi178 ceph-mon[35882]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:06.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.643+0000 7f03fff6a640 1 -- 172.21.15.133:0/3660155868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f8106a80 msgr2=0x7f03f8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:06.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.643+0000 7f03fff6a640 1 --2- 172.21.15.133:0/3660155868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f8106a80 0x7f03f8106e60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009930 tx=0x7f03f002a420 comp rx=0 tx=0).stop 2024-09-11T23:34:06.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 -- 172.21.15.133:0/3660155868 shutdown_connections 2024-09-11T23:34:06.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 --2- 172.21.15.133:0/3660155868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f8101420 0x7f03f810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:06.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 --2- 172.21.15.133:0/3660155868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f8100a80 0x7f03f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:06.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 --2- 172.21.15.133:0/3660155868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f8106a80 0x7f03f8106e60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:06.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 -- 172.21.15.133:0/3660155868 >> 172.21.15.133:0/3660155868 conn(0x7f03f80fc7a0 msgr2=0x7f03f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:06.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 -- 172.21.15.133:0/3660155868 shutdown_connections 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 -- 172.21.15.133:0/3660155868 wait complete. 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 Processor -- start 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 -- start start 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f8100a80 0x7f03f81970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f8101420 0x7f03f81975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f8106a80 0x7f03f8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8110e80 con 0x7f03f8100a80 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8110d00 con 0x7f03f8106a80 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.644+0000 7f03fff6a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8111000 con 0x7f03f8101420 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fdd06640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f8100a80 0x7f03f81970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fdd06640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f8100a80 0x7f03f81970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54980/0 (socket says 172.21.15.133:54980) 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fdd06640 1 -- 172.21.15.133:0/3414717354 learned_addr learned my addr 172.21.15.133:0/3414717354 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:06.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fe507640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f8106a80 0x7f03f8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fd505640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f8101420 0x7f03f81975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fdd06640 1 -- 172.21.15.133:0/3414717354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f8101420 msgr2=0x7f03f81975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fdd06640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f8101420 0x7f03f81975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fdd06640 1 -- 172.21.15.133:0/3414717354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f8106a80 msgr2=0x7f03f8197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fdd06640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f8106a80 0x7f03f8197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fdd06640 1 -- 172.21.15.133:0/3414717354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f81a6910 con 0x7f03f8100a80 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fe507640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f8106a80 0x7f03f8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fdd06640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f8100a80 0x7f03f81970a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009230 tx=0x7f03f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03f6ffd640 1 -- 172.21.15.133:0/3414717354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f003f030 con 0x7f03f8100a80 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fff6a640 1 -- 172.21.15.133:0/3414717354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f81a6ba0 con 0x7f03f8100a80 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03f6ffd640 1 -- 172.21.15.133:0/3414717354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0003ad0 con 0x7f03f8100a80 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03f6ffd640 1 -- 172.21.15.133:0/3414717354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0035070 con 0x7f03f8100a80 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.645+0000 7f03fff6a640 1 -- 172.21.15.133:0/3414717354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f81a7080 con 0x7f03f8100a80 2024-09-11T23:34:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.646+0000 7f03fff6a640 1 -- 172.21.15.133:0/3414717354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f8100ee0 con 0x7f03f8100a80 2024-09-11T23:34:06.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.647+0000 7f03f6ffd640 1 -- 172.21.15.133:0/3414717354 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f03f0042030 con 0x7f03f8100a80 2024-09-11T23:34:06.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.648+0000 7f03f6ffd640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f03e00a2b80 0x7f03e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:06.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.648+0000 7f03fd505640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f03e00a2b80 0x7f03e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:06.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.648+0000 7f03f6ffd640 1 -- 172.21.15.133:0/3414717354 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f03f00ef5b0 con 0x7f03f8100a80 2024-09-11T23:34:06.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.648+0000 7f03fd505640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f03e00a2b80 0x7f03e00a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f03ec005fd0 tx=0x7f03ec005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:06.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.651+0000 7f03f6ffd640 1 -- 172.21.15.133:0/3414717354 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03f00bc910 con 0x7f03f8100a80 2024-09-11T23:34:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.819+0000 7f03fff6a640 1 -- 172.21.15.133:0/3414717354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03f802d7c0 con 0x7f03f8100a80 2024-09-11T23:34:06.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.820+0000 7f03f6ffd640 1 -- 172.21.15.133:0/3414717354 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18525 (secure 0 0 0) 0x7f03f00bc210 con 0x7f03f8100a80 2024-09-11T23:34:06.820 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:06.820 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":74,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:34:05.050623+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:34:06.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 -- 172.21.15.133:0/3414717354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f03e00a2b80 msgr2=0x7f03e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:06.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f03e00a2b80 0x7f03e00a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f03ec005fd0 tx=0x7f03ec005850 comp rx=0 tx=0).stop 2024-09-11T23:34:06.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 -- 172.21.15.133:0/3414717354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f8100a80 msgr2=0x7f03f81970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:06.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f8100a80 0x7f03f81970a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009230 tx=0x7f03f0004270 comp rx=0 tx=0).stop 2024-09-11T23:34:06.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 -- 172.21.15.133:0/3414717354 shutdown_connections 2024-09-11T23:34:06.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f03e00a2b80 0x7f03e00a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:06.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f8106a80 0x7f03f8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:06.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f8101420 0x7f03f81975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:06.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 --2- 172.21.15.133:0/3414717354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f8100a80 0x7f03f81970a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:06.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 -- 172.21.15.133:0/3414717354 >> 172.21.15.133:0/3414717354 conn(0x7f03f80fc7a0 msgr2=0x7f03f80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:06.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 -- 172.21.15.133:0/3414717354 shutdown_connections 2024-09-11T23:34:06.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:06.823+0000 7f03fff6a640 1 -- 172.21.15.133:0/3414717354 wait complete. 2024-09-11T23:34:07.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:07 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3414717354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:34:07.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:07 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3414717354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:34:07.508 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:30:22.061012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:34:07.508 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T23:34:07.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:07 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3414717354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:34:07.749 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:08.120 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:08 smithi133 ceph-mon[31430]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:08.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.203+0000 7f2805b92640 1 -- 172.21.15.133:0/1262177041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2800101420 msgr2=0x7f280010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:08.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.203+0000 7f2805b92640 1 --2- 172.21.15.133:0/1262177041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2800101420 0x7f280010d950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009cd0 tx=0x7f27f802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.203+0000 7f2805b92640 1 -- 172.21.15.133:0/1262177041 shutdown_connections 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.203+0000 7f2805b92640 1 --2- 172.21.15.133:0/1262177041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2800101420 0x7f280010d950 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.203+0000 7f2805b92640 1 --2- 172.21.15.133:0/1262177041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2800100a80 0x7f2800100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.203+0000 7f2805b92640 1 --2- 172.21.15.133:0/1262177041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2800106a80 0x7f2800106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.203+0000 7f2805b92640 1 -- 172.21.15.133:0/1262177041 >> 172.21.15.133:0/1262177041 conn(0x7f28000fc7a0 msgr2=0x7f28000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.203+0000 7f2805b92640 1 -- 172.21.15.133:0/1262177041 shutdown_connections 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.203+0000 7f2805b92640 1 -- 172.21.15.133:0/1262177041 wait complete. 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.203+0000 7f2805b92640 1 Processor -- start 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.203+0000 7f2805b92640 1 -- start start 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f2805b92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2800100a80 0x7f2800198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f2805b92640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2800101420 0x7f2800198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f2805b92640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2800106a80 0x7f280019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f2805b92640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2800110e80 con 0x7f2800106a80 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f2805b92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2800110d00 con 0x7f2800100a80 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f2805b92640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2800111000 con 0x7f2800101420 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2800106a80 0x7f280019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2800106a80 0x7f280019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54992/0 (socket says 172.21.15.133:54992) 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fffff640 1 -- 172.21.15.133:0/902630588 learned_addr learned my addr 172.21.15.133:0/902630588 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27ff7fe640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2800100a80 0x7f2800198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27feffd640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2800101420 0x7f2800198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fffff640 1 -- 172.21.15.133:0/902630588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2800101420 msgr2=0x7f2800198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fffff640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2800101420 0x7f2800198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fffff640 1 -- 172.21.15.133:0/902630588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2800100a80 msgr2=0x7f2800198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fffff640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2800100a80 0x7f2800198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fffff640 1 -- 172.21.15.133:0/902630588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280019d3c0 con 0x7f2800106a80 2024-09-11T23:34:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fffff640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2800106a80 0x7f280019cce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27f8005cc0 tx=0x7f27f8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fcff9640 1 -- 172.21.15.133:0/902630588 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8004040 con 0x7f2800106a80 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fcff9640 1 -- 172.21.15.133:0/902630588 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f80046c0 con 0x7f2800106a80 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f2805b92640 1 -- 172.21.15.133:0/902630588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280019d650 con 0x7f2800106a80 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.204+0000 7f27fcff9640 1 -- 172.21.15.133:0/902630588 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8046480 con 0x7f2800106a80 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.205+0000 7f2805b92640 1 -- 172.21.15.133:0/902630588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28001a4f90 con 0x7f2800106a80 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.206+0000 7f2805b92640 1 -- 172.21.15.133:0/902630588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2800100ee0 con 0x7f2800106a80 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.207+0000 7f27fcff9640 1 -- 172.21.15.133:0/902630588 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f27f80305a0 con 0x7f2800106a80 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.207+0000 7f27fcff9640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f27e80a2b80 0x7f27e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.207+0000 7f27ff7fe640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f27e80a2b80 0x7f27e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.207+0000 7f27fcff9640 1 -- 172.21.15.133:0/902630588 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f27f802e030 con 0x7f2800106a80 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.208+0000 7f27ff7fe640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f27e80a2b80 0x7f27e80a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27f0005fd0 tx=0x7f27f000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.211+0000 7f27fcff9640 1 -- 172.21.15.133:0/902630588 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27f804b060 con 0x7f2800106a80 2024-09-11T23:34:08.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.377+0000 7f2805b92640 1 -- 172.21.15.133:0/902630588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28001a51c0 con 0x7f2800106a80 2024-09-11T23:34:08.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.377+0000 7f27fcff9640 1 -- 172.21.15.133:0/902630588 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f27f80bd710 con 0x7f2800106a80 2024-09-11T23:34:08.377 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-09-11T23:34:08.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 -- 172.21.15.133:0/902630588 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f27e80a2b80 msgr2=0x7f27e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f27e80a2b80 0x7f27e80a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27f0005fd0 tx=0x7f27f000b390 comp rx=0 tx=0).stop 2024-09-11T23:34:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 -- 172.21.15.133:0/902630588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2800106a80 msgr2=0x7f280019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2800106a80 0x7f280019cce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27f8005cc0 tx=0x7f27f8003970 comp rx=0 tx=0).stop 2024-09-11T23:34:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 -- 172.21.15.133:0/902630588 shutdown_connections 2024-09-11T23:34:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f27e80a2b80 0x7f27e80a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2800106a80 0x7f280019cce0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2800101420 0x7f2800198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 --2- 172.21.15.133:0/902630588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2800100a80 0x7f2800198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 -- 172.21.15.133:0/902630588 >> 172.21.15.133:0/902630588 conn(0x7f28000fc7a0 msgr2=0x7f2800104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 -- 172.21.15.133:0/902630588 shutdown_connections 2024-09-11T23:34:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:08.380+0000 7f2805b92640 1 -- 172.21.15.133:0/902630588 wait complete. 2024-09-11T23:34:08.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:08 smithi138 ceph-mon[35142]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:08.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:08 smithi178 ceph-mon[35882]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:09.053 INFO:tasks.cephadm:Adding mds.a on smithi133 2024-09-11T23:34:09.053 INFO:tasks.cephadm:Adding mds.d on smithi133 2024-09-11T23:34:09.053 INFO:tasks.cephadm:Adding mds.g on smithi133 2024-09-11T23:34:09.053 INFO:tasks.cephadm:Adding mds.j on smithi133 2024-09-11T23:34:09.053 INFO:tasks.cephadm:Adding mds.b on smithi138 2024-09-11T23:34:09.053 INFO:tasks.cephadm:Adding mds.e on smithi138 2024-09-11T23:34:09.053 INFO:tasks.cephadm:Adding mds.h on smithi138 2024-09-11T23:34:09.053 INFO:tasks.cephadm:Adding mds.k on smithi138 2024-09-11T23:34:09.054 INFO:tasks.cephadm:Adding mds.c on smithi178 2024-09-11T23:34:09.054 INFO:tasks.cephadm:Adding mds.f on smithi178 2024-09-11T23:34:09.054 INFO:tasks.cephadm:Adding mds.i on smithi178 2024-09-11T23:34:09.054 INFO:tasks.cephadm:Adding mds.l on smithi178 2024-09-11T23:34:09.054 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l' 2024-09-11T23:34:09.313 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:34:09.434 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:09 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/902630588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:34:09.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:09 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/902630588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:34:09.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:09 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/902630588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:34:09.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.804+0000 7fdc06eee640 1 -- 172.21.15.178:0/3788381244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc00101cd0 msgr2=0x7fdc00102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:09.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.804+0000 7fdc06eee640 1 --2- 172.21.15.178:0/3788381244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc00101cd0 0x7fdc00102150 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009900 tx=0x7fdbec02a410 comp rx=0 tx=0).stop 2024-09-11T23:34:09.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.804+0000 7fdc06eee640 1 -- 172.21.15.178:0/3788381244 shutdown_connections 2024-09-11T23:34:09.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.804+0000 7fdc06eee640 1 --2- 172.21.15.178:0/3788381244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc00102690 0x7fdc00108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:09.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.804+0000 7fdc06eee640 1 --2- 172.21.15.178:0/3788381244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc00101cd0 0x7fdc00102150 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:09.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.804+0000 7fdc06eee640 1 --2- 172.21.15.178:0/3788381244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc00100ad0 0x7fdc00100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:09.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.804+0000 7fdc06eee640 1 -- 172.21.15.178:0/3788381244 >> 172.21.15.178:0/3788381244 conn(0x7fdc000fc280 msgr2=0x7fdc000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:09.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.804+0000 7fdc06eee640 1 -- 172.21.15.178:0/3788381244 shutdown_connections 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.804+0000 7fdc06eee640 1 -- 172.21.15.178:0/3788381244 wait complete. 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc06eee640 1 Processor -- start 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc06eee640 1 -- start start 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc06eee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc00100ad0 0x7fdc0006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc06eee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc00101cd0 0x7fdc0019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc06eee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc00102690 0x7fdc0019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc06eee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0010c750 con 0x7fdc00100ad0 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc06eee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0010c5d0 con 0x7fdc00101cd0 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc06eee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0010c8d0 con 0x7fdc00102690 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc0548b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc00102690 0x7fdc0019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc0548b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc00102690 0x7fdc0019f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49812/0 (socket says 172.21.15.178:49812) 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc0548b640 1 -- 172.21.15.178:0/292403439 learned_addr learned my addr 172.21.15.178:0/292403439 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T23:34:09.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc04c8a640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc00100ad0 0x7fdc0006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdbfffff640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc00101cd0 0x7fdc0019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc0548b640 1 -- 172.21.15.178:0/292403439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc00101cd0 msgr2=0x7fdc0019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc0548b640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc00101cd0 0x7fdc0019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc0548b640 1 -- 172.21.15.178:0/292403439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc00100ad0 msgr2=0x7fdc0006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc0548b640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc00100ad0 0x7fdc0006aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdc0548b640 1 -- 172.21.15.178:0/292403439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0019fb70 con 0x7fdc00102690 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.805+0000 7fdbfffff640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc00101cd0 0x7fdc0019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.806+0000 7fdc0548b640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc00102690 0x7fdc0019f260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800eb10 tx=0x7fdbf800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.806+0000 7fdbfdffb640 1 -- 172.21.15.178:0/292403439 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8004b20 con 0x7fdc00102690 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.806+0000 7fdbfdffb640 1 -- 172.21.15.178:0/292403439 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf8004cc0 con 0x7fdc00102690 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.806+0000 7fdc06eee640 1 -- 172.21.15.178:0/292403439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0019fe00 con 0x7fdc00102690 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.806+0000 7fdbfdffb640 1 -- 172.21.15.178:0/292403439 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8010760 con 0x7fdc00102690 2024-09-11T23:34:09.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.806+0000 7fdc06eee640 1 -- 172.21.15.178:0/292403439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc001a0250 con 0x7fdc00102690 2024-09-11T23:34:09.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.807+0000 7fdc06eee640 1 -- 172.21.15.178:0/292403439 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc00107ec0 con 0x7fdc00102690 2024-09-11T23:34:09.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.808+0000 7fdbfdffb640 1 -- 172.21.15.178:0/292403439 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdbf8005020 con 0x7fdc00102690 2024-09-11T23:34:09.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.809+0000 7fdbfdffb640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdbf00a2b80 0x7fdbf00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:09.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.809+0000 7fdbfdffb640 1 -- 172.21.15.178:0/292403439 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fdbf80cf170 con 0x7fdc00102690 2024-09-11T23:34:09.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.809+0000 7fdc04c8a640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdbf00a2b80 0x7fdbf00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:09.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.810+0000 7fdc04c8a640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdbf00a2b80 0x7fdbf00a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc00101b30 tx=0x7fdbf4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:09.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:09.813+0000 7fdbfdffb640 1 -- 172.21.15.178:0/292403439 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdbf809c570 con 0x7fdc00102690 2024-09-11T23:34:10.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.008+0000 7fdc06eee640 1 -- 172.21.15.178:0/292403439 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}) -- 0x7fdc000628f0 con 0x7fdbf00a2b80 2024-09-11T23:34:10.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.016+0000 7fdbfdffb640 1 -- 172.21.15.178:0/292403439 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fdc000628f0 con 0x7fdbf00a2b80 2024-09-11T23:34:10.017 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mds.all update... 2024-09-11T23:34:10.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 -- 172.21.15.178:0/292403439 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdbf00a2b80 msgr2=0x7fdbf00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:10.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdbf00a2b80 0x7fdbf00a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc00101b30 tx=0x7fdbf4005f90 comp rx=0 tx=0).stop 2024-09-11T23:34:10.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 -- 172.21.15.178:0/292403439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc00102690 msgr2=0x7fdc0019f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:10.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc00102690 0x7fdc0019f260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800eb10 tx=0x7fdbf800c2d0 comp rx=0 tx=0).stop 2024-09-11T23:34:10.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 -- 172.21.15.178:0/292403439 shutdown_connections 2024-09-11T23:34:10.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdbf00a2b80 0x7fdbf00a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:10.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc00102690 0x7fdc0019f260 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:10.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc00101cd0 0x7fdc0019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:10.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 --2- 172.21.15.178:0/292403439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc00100ad0 0x7fdc0006aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:10.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 -- 172.21.15.178:0/292403439 >> 172.21.15.178:0/292403439 conn(0x7fdc000fc280 msgr2=0x7fdc00105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:10.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 -- 172.21.15.178:0/292403439 shutdown_connections 2024-09-11T23:34:10.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T23:34:10.019+0000 7fdc06eee640 1 -- 172.21.15.178:0/292403439 wait complete. 2024-09-11T23:34:10.756 DEBUG:teuthology.orchestra.run.smithi133:mds.a> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.a.service 2024-09-11T23:34:10.759 DEBUG:teuthology.orchestra.run.smithi133:mds.d> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.d.service 2024-09-11T23:34:10.803 DEBUG:teuthology.orchestra.run.smithi133:mds.g> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.g.service 2024-09-11T23:34:10.846 DEBUG:teuthology.orchestra.run.smithi133:mds.j> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.j.service 2024-09-11T23:34:10.889 DEBUG:teuthology.orchestra.run.smithi138:mds.b> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.b.service 2024-09-11T23:34:10.892 DEBUG:teuthology.orchestra.run.smithi138:mds.e> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.e.service 2024-09-11T23:34:10.935 DEBUG:teuthology.orchestra.run.smithi138:mds.h> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.h.service 2024-09-11T23:34:10.978 DEBUG:teuthology.orchestra.run.smithi138:mds.k> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.k.service 2024-09-11T23:34:11.021 DEBUG:teuthology.orchestra.run.smithi178:mds.c> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.c.service 2024-09-11T23:34:11.023 DEBUG:teuthology.orchestra.run.smithi178:mds.f> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.f.service 2024-09-11T23:34:11.066 DEBUG:teuthology.orchestra.run.smithi178:mds.i> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.i.service 2024-09-11T23:34:11.109 DEBUG:teuthology.orchestra.run.smithi178:mds.l> sudo journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.l.service 2024-09-11T23:34:11.152 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-11T23:34:11.152 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:34:11.175 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:11 smithi133 ceph-mon[31430]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:11.175 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:11 smithi133 ceph-mon[31430]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:34:11.175 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:11 smithi133 ceph-mon[31430]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-11T23:34:11.175 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:11.175 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:11.175 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:11.175 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:11.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:11.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:11.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:11.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:11 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:11.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:11 smithi138 ceph-mon[35142]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:11.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:11 smithi138 ceph-mon[35142]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:34:11.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:11 smithi138 ceph-mon[35142]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-11T23:34:11.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:11.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:11.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:11.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:11.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:11.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:11.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:11.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:11 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:11 smithi178 ceph-mon[35882]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:11 smithi178 ceph-mon[35882]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:34:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:11 smithi178 ceph-mon[35882]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-11T23:34:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:11.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:11 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:11.395 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.831+0000 7f090247e640 1 -- 172.21.15.133:0/1812199139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc101420 msgr2=0x7f08fc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.831+0000 7f090247e640 1 --2- 172.21.15.133:0/1812199139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc101420 0x7f08fc10d950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009cd0 tx=0x7f08f402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:34:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.831+0000 7f090247e640 1 -- 172.21.15.133:0/1812199139 shutdown_connections 2024-09-11T23:34:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.831+0000 7f090247e640 1 --2- 172.21.15.133:0/1812199139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc101420 0x7f08fc10d950 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.832+0000 7f090247e640 1 --2- 172.21.15.133:0/1812199139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc100a80 0x7f08fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.832+0000 7f090247e640 1 --2- 172.21.15.133:0/1812199139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08fc106a80 0x7f08fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.832+0000 7f090247e640 1 -- 172.21.15.133:0/1812199139 >> 172.21.15.133:0/1812199139 conn(0x7f08fc0fc7a0 msgr2=0x7f08fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.832+0000 7f090247e640 1 -- 172.21.15.133:0/1812199139 shutdown_connections 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.832+0000 7f090247e640 1 -- 172.21.15.133:0/1812199139 wait complete. 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.832+0000 7f090247e640 1 Processor -- start 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.832+0000 7f090247e640 1 -- start start 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f090247e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08fc100a80 0x7f08fc1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f090247e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc101420 0x7f08fc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f090247e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc106a80 0x7f08fc19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f090247e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc110e80 con 0x7f08fc106a80 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f090247e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc110d00 con 0x7f08fc100a80 2024-09-11T23:34:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f090247e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc111000 con 0x7f08fc101420 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f0900a1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc106a80 0x7f08fc19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f0900a1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc106a80 0x7f08fc19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55012/0 (socket says 172.21.15.133:55012) 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f0900a1b640 1 -- 172.21.15.133:0/35260723 learned_addr learned my addr 172.21.15.133:0/35260723 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f08f3fff640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc101420 0x7f08fc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f08fbfff640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08fc100a80 0x7f08fc1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f0900a1b640 1 -- 172.21.15.133:0/35260723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc101420 msgr2=0x7f08fc197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f0900a1b640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc101420 0x7f08fc197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f0900a1b640 1 -- 172.21.15.133:0/35260723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08fc100a80 msgr2=0x7f08fc1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f0900a1b640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08fc100a80 0x7f08fc1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f0900a1b640 1 -- 172.21.15.133:0/35260723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc19c270 con 0x7f08fc106a80 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f0900a1b640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc106a80 0x7f08fc19bb90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f08f4006010 tx=0x7f08f4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.833+0000 7f08f9ffb640 1 -- 172.21.15.133:0/35260723 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4004450 con 0x7f08fc106a80 2024-09-11T23:34:11.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.834+0000 7f08f9ffb640 1 -- 172.21.15.133:0/35260723 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f4003e10 con 0x7f08fc106a80 2024-09-11T23:34:11.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.834+0000 7f08f9ffb640 1 -- 172.21.15.133:0/35260723 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4046480 con 0x7f08fc106a80 2024-09-11T23:34:11.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.834+0000 7f090247e640 1 -- 172.21.15.133:0/35260723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc19c500 con 0x7f08fc106a80 2024-09-11T23:34:11.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.834+0000 7f090247e640 1 -- 172.21.15.133:0/35260723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc1a3e40 con 0x7f08fc106a80 2024-09-11T23:34:11.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.835+0000 7f090247e640 1 -- 172.21.15.133:0/35260723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08fc1021c0 con 0x7f08fc106a80 2024-09-11T23:34:11.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.836+0000 7f08f9ffb640 1 -- 172.21.15.133:0/35260723 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f08f40045f0 con 0x7f08fc106a80 2024-09-11T23:34:11.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.836+0000 7f08f9ffb640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f08e80a2b80 0x7f08e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:11.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.837+0000 7f08fbfff640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f08e80a2b80 0x7f08e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:11.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.837+0000 7f08f9ffb640 1 -- 172.21.15.133:0/35260723 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f08f402e030 con 0x7f08fc106a80 2024-09-11T23:34:11.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.837+0000 7f08fbfff640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f08e80a2b80 0x7f08e80a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0058c0 tx=0x7f08ec00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:11.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:11.840+0000 7f08f9ffb640 1 -- 172.21.15.133:0/35260723 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08f404b060 con 0x7f08fc106a80 2024-09-11T23:34:12.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.042+0000 7f090247e640 1 -- 172.21.15.133:0/35260723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08fc100ee0 con 0x7f08fc106a80 2024-09-11T23:34:12.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.042+0000 7f08f9ffb640 1 -- 172.21.15.133:0/35260723 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f08fc100ee0 con 0x7f08fc106a80 2024-09-11T23:34:12.102 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:12.102 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":1,"btime":"2024-09-11T23:27:21:703865+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-11T23:34:12.102 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 1 2024-09-11T23:34:12.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 -- 172.21.15.133:0/35260723 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f08e80a2b80 msgr2=0x7f08e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:12.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f08e80a2b80 0x7f08e80a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0058c0 tx=0x7f08ec00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:34:12.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 -- 172.21.15.133:0/35260723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc106a80 msgr2=0x7f08fc19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:12.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc106a80 0x7f08fc19bb90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f08f4006010 tx=0x7f08f4003c20 comp rx=0 tx=0).stop 2024-09-11T23:34:12.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 -- 172.21.15.133:0/35260723 shutdown_connections 2024-09-11T23:34:12.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f08e80a2b80 0x7f08e80a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:12.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08fc106a80 0x7f08fc19bb90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:12.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc101420 0x7f08fc197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:12.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 --2- 172.21.15.133:0/35260723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08fc100a80 0x7f08fc1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:12.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 -- 172.21.15.133:0/35260723 >> 172.21.15.133:0/35260723 conn(0x7f08fc0fc7a0 msgr2=0x7f08fc0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:12.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 -- 172.21.15.133:0/35260723 shutdown_connections 2024-09-11T23:34:12.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:12.044+0000 7f090247e640 1 -- 172.21.15.133:0/35260723 wait complete. 2024-09-11T23:34:12.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:12 smithi133 ceph-mon[31430]: Deploying daemon mds.l on smithi178 2024-09-11T23:34:12.224 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:12 smithi178 ceph-mon[35882]: Deploying daemon mds.l on smithi178 2024-09-11T23:34:12.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:12 smithi138 ceph-mon[35142]: Deploying daemon mds.l on smithi178 2024-09-11T23:34:12.540 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:34:12 smithi178 systemd[1]: Starting Ceph mds.l for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:34:12.871 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:34:12 smithi178 podman[65157]: 2024-09-11 23:34:12.520574187 +0000 UTC m=+0.015597275 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:12.871 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:34:12 smithi178 podman[65157]: 2024-09-11 23:34:12.626369729 +0000 UTC m=+0.121392812 container create d605213c28229bc6a123fb2068816a8750826699b59129e4bb8328d9ae5f7649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T23:34:12.871 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:34:12 smithi178 podman[65157]: 2024-09-11 23:34:12.845334194 +0000 UTC m=+0.340357273 container init d605213c28229bc6a123fb2068816a8750826699b59129e4bb8328d9ae5f7649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T23:34:12.871 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:34:12 smithi178 podman[65157]: 2024-09-11 23:34:12.848846842 +0000 UTC m=+0.343869921 container start d605213c28229bc6a123fb2068816a8750826699b59129e4bb8328d9ae5f7649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-11T23:34:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:13 smithi133 ceph-mon[31430]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:13 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/35260723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:13 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6832/3569562128,v1:172.21.15.178:6833/3569562128] up:boot 2024-09-11T23:34:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:13 smithi133 ceph-mon[31430]: fsmap 1 up:standby 2024-09-11T23:34:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:13 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:34:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:13 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:13 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:13 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:13 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:13 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:13 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:13.290 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:34:12 smithi178 ceph-mds[65176]: -- 172.21.15.178:0/30175861 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d713929860 con 0x55d713907000 2024-09-11T23:34:13.290 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:34:12 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l[65172]: starting mds.l at 2024-09-11T23:34:13.290 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:34:12 smithi178 bash[65157]: d605213c28229bc6a123fb2068816a8750826699b59129e4bb8328d9ae5f7649 2024-09-11T23:34:13.290 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:34:12 smithi178 systemd[1]: Started Ceph mds.l for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:13 smithi178 ceph-mon[35882]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:13 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/35260723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:13 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6832/3569562128,v1:172.21.15.178:6833/3569562128] up:boot 2024-09-11T23:34:13.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:13 smithi178 ceph-mon[35882]: fsmap 1 up:standby 2024-09-11T23:34:13.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:13 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:34:13.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:13 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:13.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:13 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:13.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:13 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:13.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:13 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:13.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:13 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:13.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:13 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:13.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:13 smithi138 ceph-mon[35142]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:13.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:13 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/35260723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:13.420 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:13 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6832/3569562128,v1:172.21.15.178:6833/3569562128] up:boot 2024-09-11T23:34:13.420 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:13 smithi138 ceph-mon[35142]: fsmap 1 up:standby 2024-09-11T23:34:13.420 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:13 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:34:13.420 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:13 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:13.420 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:13 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:13.420 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:13 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:13.420 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:13 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:13.420 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:13 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:13.420 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:13 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:14.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:14 smithi178 ceph-mon[35882]: Deploying daemon mds.k on smithi138 2024-09-11T23:34:14.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:14 smithi138 ceph-mon[35142]: Deploying daemon mds.k on smithi138 2024-09-11T23:34:14.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:14 smithi133 ceph-mon[31430]: Deploying daemon mds.k on smithi138 2024-09-11T23:34:14.798 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:34:14.948 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:34:14 smithi138 systemd[1]: Starting Ceph mds.k for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:34:15.055 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:15.175 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:15 smithi133 ceph-mon[31430]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:15.281 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:15 smithi138 ceph-mon[35142]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:15.282 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:34:15 smithi138 podman[64307]: 2024-09-11 23:34:14.929751784 +0000 UTC m=+0.015836995 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:15.282 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:34:15 smithi138 podman[64307]: 2024-09-11 23:34:15.038603258 +0000 UTC m=+0.124688462 container create 53cbe519fe351e375efd60720fcc9844fea3cdc63ebdfa6e1e5554f96c1632ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T23:34:15.282 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:34:15 smithi138 podman[64307]: 2024-09-11 23:34:15.282918952 +0000 UTC m=+0.369004153 container init 53cbe519fe351e375efd60720fcc9844fea3cdc63ebdfa6e1e5554f96c1632ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T23:34:15.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:15 smithi178 ceph-mon[35882]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:15.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.500+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3729226898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2ac100e20 msgr2=0x7fa2ac101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:15.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.500+0000 7fa2b2e28640 1 --2- 172.21.15.133:0/3729226898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2ac100e20 0x7fa2ac101200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009900 tx=0x7fa2a002a480 comp rx=0 tx=0).stop 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.500+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3729226898 shutdown_connections 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.500+0000 7fa2b2e28640 1 --2- 172.21.15.133:0/3729226898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2ac102ef0 0x7fa2ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.500+0000 7fa2b2e28640 1 --2- 172.21.15.133:0/3729226898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2ac102570 0x7fa2ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.500+0000 7fa2b2e28640 1 --2- 172.21.15.133:0/3729226898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2ac100e20 0x7fa2ac101200 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.500+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3729226898 >> 172.21.15.133:0/3729226898 conn(0x7fa2ac0fc7d0 msgr2=0x7fa2ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.500+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3729226898 shutdown_connections 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.500+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3729226898 wait complete. 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b2e28640 1 Processor -- start 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b2e28640 1 -- start start 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b2e28640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2ac100e20 0x7fa2ac197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b2e28640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2ac102570 0x7fa2ac1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b2e28640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2ac102ef0 0x7fa2ac197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b2e28640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac110f50 con 0x7fa2ac100e20 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b2e28640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac110dd0 con 0x7fa2ac102ef0 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b2e28640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1110d0 con 0x7fa2ac102570 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b0bc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2ac100e20 0x7fa2ac197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b0bc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2ac100e20 0x7fa2ac197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38360/0 (socket says 172.21.15.133:38360) 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b0bc4640 1 -- 172.21.15.133:0/3296083214 learned_addr learned my addr 172.21.15.133:0/3296083214 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b13c5640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2ac102ef0 0x7fa2ac197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2abfff640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2ac102570 0x7fa2ac1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b0bc4640 1 -- 172.21.15.133:0/3296083214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2ac102570 msgr2=0x7fa2ac1976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b0bc4640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2ac102570 0x7fa2ac1976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b0bc4640 1 -- 172.21.15.133:0/3296083214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2ac102ef0 msgr2=0x7fa2ac197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b0bc4640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2ac102ef0 0x7fa2ac197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.501+0000 7fa2b0bc4640 1 -- 172.21.15.133:0/3296083214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ac1a69e0 con 0x7fa2ac100e20 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.502+0000 7fa2b13c5640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2ac102ef0 0x7fa2ac197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.502+0000 7fa2b0bc4640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2ac100e20 0x7fa2ac197170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0006010 tx=0x7fa2a0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.502+0000 7fa2a9ffb640 1 -- 172.21.15.133:0/3296083214 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a003f070 con 0x7fa2ac100e20 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.502+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3296083214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ac1a6c70 con 0x7fa2ac100e20 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.502+0000 7fa2a9ffb640 1 -- 172.21.15.133:0/3296083214 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a0031e30 con 0x7fa2ac100e20 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.502+0000 7fa2a9ffb640 1 -- 172.21.15.133:0/3296083214 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0035030 con 0x7fa2ac100e20 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.502+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3296083214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ac1a7150 con 0x7fa2ac100e20 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.503+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3296083214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2ac101200 con 0x7fa2ac100e20 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.504+0000 7fa2a9ffb640 1 -- 172.21.15.133:0/3296083214 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa2a0042070 con 0x7fa2ac100e20 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.505+0000 7fa2a9ffb640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa29c0a2b80 0x7fa29c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.505+0000 7fa2a9ffb640 1 -- 172.21.15.133:0/3296083214 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fa2a00ef5a0 con 0x7fa2ac100e20 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.505+0000 7fa2abfff640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa29c0a2b80 0x7fa29c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.506+0000 7fa2abfff640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa29c0a2b80 0x7fa29c0a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa298005e10 tx=0x7fa298005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:15.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.509+0000 7fa2a9ffb640 1 -- 172.21.15.133:0/3296083214 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2a00bc9a0 con 0x7fa2ac100e20 2024-09-11T23:34:15.697 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:34:15 smithi138 podman[64307]: 2024-09-11 23:34:15.28594903 +0000 UTC m=+0.372034231 container start 53cbe519fe351e375efd60720fcc9844fea3cdc63ebdfa6e1e5554f96c1632ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T23:34:15.698 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:34:15 smithi138 ceph-mds[64326]: -- 172.21.15.138:0/3632516525 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5642d6249860 con 0x5642d6227400 2024-09-11T23:34:15.698 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:34:15 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k[64322]: starting mds.k at 2024-09-11T23:34:15.698 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:34:15 smithi138 bash[64307]: 53cbe519fe351e375efd60720fcc9844fea3cdc63ebdfa6e1e5554f96c1632ce 2024-09-11T23:34:15.698 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:34:15 smithi138 systemd[1]: Started Ceph mds.k for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:15.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.727+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3296083214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2ac108b00 con 0x7fa2ac100e20 2024-09-11T23:34:15.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.728+0000 7fa2a9ffb640 1 -- 172.21.15.133:0/3296083214 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fa2a00bc2a0 con 0x7fa2ac100e20 2024-09-11T23:34:15.729 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:15.729 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"btime":"2024-09-11T23:34:12:877042+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T23:34:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 2 2024-09-11T23:34:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3296083214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa29c0a2b80 msgr2=0x7fa29c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa29c0a2b80 0x7fa29c0a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa298005e10 tx=0x7fa298005da0 comp rx=0 tx=0).stop 2024-09-11T23:34:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3296083214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2ac100e20 msgr2=0x7fa2ac197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2ac100e20 0x7fa2ac197170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0006010 tx=0x7fa2a0003c80 comp rx=0 tx=0).stop 2024-09-11T23:34:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3296083214 shutdown_connections 2024-09-11T23:34:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa29c0a2b80 0x7fa29c0a5040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2ac102ef0 0x7fa2ac197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2ac102570 0x7fa2ac1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 --2- 172.21.15.133:0/3296083214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2ac100e20 0x7fa2ac197170 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3296083214 >> 172.21.15.133:0/3296083214 conn(0x7fa2ac0fc7d0 msgr2=0x7fa2ac105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3296083214 shutdown_connections 2024-09-11T23:34:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:15.730+0000 7fa2b2e28640 1 -- 172.21.15.133:0/3296083214 wait complete. 2024-09-11T23:34:16.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:16.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:16.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:16.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:16.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:16.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:16.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: Deploying daemon mds.j on smithi133 2024-09-11T23:34:16.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3296083214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:16.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:16.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] up:boot 2024-09-11T23:34:16.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: fsmap 2 up:standby 2024-09-11T23:34:16.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:16 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:34:16.683 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:16.683 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:16.683 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:16.683 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:16.683 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:16.683 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:16.684 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: Deploying daemon mds.j on smithi133 2024-09-11T23:34:16.684 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3296083214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:16.684 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:16.684 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] up:boot 2024-09-11T23:34:16.684 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: fsmap 2 up:standby 2024-09-11T23:34:16.684 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:16 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:34:16.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:16.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:16.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:16.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:16.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:16.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:16.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: Deploying daemon mds.j on smithi133 2024-09-11T23:34:16.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3296083214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:16.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:16.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] up:boot 2024-09-11T23:34:16.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: fsmap 2 up:standby 2024-09-11T23:34:16.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:16 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:34:18.459 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:34:18 smithi133 systemd[1]: Starting Ceph mds.j for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:34:18.744 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:34:18 smithi133 podman[68608]: 2024-09-11 23:34:18.378533141 +0000 UTC m=+0.015963224 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:18.745 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:34:18 smithi133 podman[68608]: 2024-09-11 23:34:18.507860029 +0000 UTC m=+0.145290117 container create 34093403e5e3a7ec22f8899242df800d9566e1884540e185666d3169935ac3f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T23:34:18.745 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:34:18 smithi133 podman[68608]: 2024-09-11 23:34:18.719040393 +0000 UTC m=+0.356470477 container init 34093403e5e3a7ec22f8899242df800d9566e1884540e185666d3169935ac3f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T23:34:18.745 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:34:18 smithi133 podman[68608]: 2024-09-11 23:34:18.721979667 +0000 UTC m=+0.359409754 container start 34093403e5e3a7ec22f8899242df800d9566e1884540e185666d3169935ac3f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:34:18.745 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:34:18 smithi133 ceph-mds[68627]: -- 172.21.15.133:0/2919388348 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649c09f1860 con 0x5649c09cf000 2024-09-11T23:34:18.902 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:34:19.160 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:19.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:18 smithi138 ceph-mon[35142]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:19.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:18 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:19.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:18 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:19.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:18 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:19.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:18 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:19.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:18 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:19.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:18 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:19.209 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:34:18 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-j[68623]: starting mds.j at 2024-09-11T23:34:19.209 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:34:18 smithi133 bash[68608]: 34093403e5e3a7ec22f8899242df800d9566e1884540e185666d3169935ac3f6 2024-09-11T23:34:19.209 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:34:18 smithi133 systemd[1]: Started Ceph mds.j for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:19.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:18 smithi133 ceph-mon[31430]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:19.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:18 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:19.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:18 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:19.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:18 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:19.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:18 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:19.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:18 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:19.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:18 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:18 smithi178 ceph-mon[35882]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:18 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:18 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:18 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:18 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:18 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:18 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:19 smithi138 ceph-mon[35142]: Deploying daemon mds.d on smithi133 2024-09-11T23:34:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:19 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.133:6834/1603982530,v1:172.21.15.133:6835/1603982530] up:boot 2024-09-11T23:34:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:19 smithi138 ceph-mon[35142]: fsmap 3 up:standby 2024-09-11T23:34:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:19 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:34:20.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:19 smithi133 ceph-mon[31430]: Deploying daemon mds.d on smithi133 2024-09-11T23:34:20.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:19 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.133:6834/1603982530,v1:172.21.15.133:6835/1603982530] up:boot 2024-09-11T23:34:20.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:19 smithi133 ceph-mon[31430]: fsmap 3 up:standby 2024-09-11T23:34:20.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:19 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:34:20.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:19 smithi178 ceph-mon[35882]: Deploying daemon mds.d on smithi133 2024-09-11T23:34:20.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:19 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.133:6834/1603982530,v1:172.21.15.133:6835/1603982530] up:boot 2024-09-11T23:34:20.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:19 smithi178 ceph-mon[35882]: fsmap 3 up:standby 2024-09-11T23:34:20.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:19 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:34:20.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.686+0000 7fd194706640 1 -- 172.21.15.133:0/2651620701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd18c102ef0 msgr2=0x7fd18c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:20.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.686+0000 7fd194706640 1 --2- 172.21.15.133:0/2651620701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd18c102ef0 0x7fd18c10da40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd184009ca0 tx=0x7fd18402a950 comp rx=0 tx=0).stop 2024-09-11T23:34:20.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.686+0000 7fd194706640 1 -- 172.21.15.133:0/2651620701 shutdown_connections 2024-09-11T23:34:20.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.686+0000 7fd194706640 1 --2- 172.21.15.133:0/2651620701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd18c102ef0 0x7fd18c10da40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:20.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.686+0000 7fd194706640 1 --2- 172.21.15.133:0/2651620701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd18c102570 0x7fd18c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:20.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.686+0000 7fd194706640 1 --2- 172.21.15.133:0/2651620701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd18c100e20 0x7fd18c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:20.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.686+0000 7fd194706640 1 -- 172.21.15.133:0/2651620701 >> 172.21.15.133:0/2651620701 conn(0x7fd18c0fc7d0 msgr2=0x7fd18c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:20.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.686+0000 7fd194706640 1 -- 172.21.15.133:0/2651620701 shutdown_connections 2024-09-11T23:34:20.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.686+0000 7fd194706640 1 -- 172.21.15.133:0/2651620701 wait complete. 2024-09-11T23:34:20.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd194706640 1 Processor -- start 2024-09-11T23:34:20.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd194706640 1 -- start start 2024-09-11T23:34:20.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd194706640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd18c100e20 0x7fd18c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:20.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd194706640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd18c102570 0x7fd18c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:20.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd194706640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd18c102ef0 0x7fd18c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:20.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd194706640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c110f80 con 0x7fd18c100e20 2024-09-11T23:34:20.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd194706640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c110e00 con 0x7fd18c102570 2024-09-11T23:34:20.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd194706640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c111100 con 0x7fd18c102ef0 2024-09-11T23:34:20.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd1924a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd18c100e20 0x7fd18c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:20.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd1924a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd18c100e20 0x7fd18c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38390/0 (socket says 172.21.15.133:38390) 2024-09-11T23:34:20.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd1924a2640 1 -- 172.21.15.133:0/3354523324 learned_addr learned my addr 172.21.15.133:0/3354523324 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:20.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd191ca1640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd18c102570 0x7fd18c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:20.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd192ca3640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd18c102ef0 0x7fd18c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd1924a2640 1 -- 172.21.15.133:0/3354523324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd18c102ef0 msgr2=0x7fd18c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd1924a2640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd18c102ef0 0x7fd18c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd1924a2640 1 -- 172.21.15.133:0/3354523324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd18c102570 msgr2=0x7fd18c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd1924a2640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd18c102570 0x7fd18c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd1924a2640 1 -- 172.21.15.133:0/3354523324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c19f6c0 con 0x7fd18c100e20 2024-09-11T23:34:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.687+0000 7fd192ca3640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd18c102ef0 0x7fd18c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:34:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.688+0000 7fd1924a2640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd18c100e20 0x7fd18c19a710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c970 tx=0x7fd17c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.688+0000 7fd1837fe640 1 -- 172.21.15.133:0/3354523324 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c007bb0 con 0x7fd18c100e20 2024-09-11T23:34:20.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.688+0000 7fd194706640 1 -- 172.21.15.133:0/3354523324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c19f9b0 con 0x7fd18c100e20 2024-09-11T23:34:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.688+0000 7fd1837fe640 1 -- 172.21.15.133:0/3354523324 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c007d50 con 0x7fd18c100e20 2024-09-11T23:34:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.688+0000 7fd1837fe640 1 -- 172.21.15.133:0/3354523324 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00f450 con 0x7fd18c100e20 2024-09-11T23:34:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.688+0000 7fd194706640 1 -- 172.21.15.133:0/3354523324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c1a7290 con 0x7fd18c100e20 2024-09-11T23:34:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.689+0000 7fd194706640 1 -- 172.21.15.133:0/3354523324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18c101200 con 0x7fd18c100e20 2024-09-11T23:34:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.691+0000 7fd1837fe640 1 -- 172.21.15.133:0/3354523324 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd17c004fc0 con 0x7fd18c100e20 2024-09-11T23:34:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.691+0000 7fd1837fe640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd1780a2b80 0x7fd1780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.692+0000 7fd191ca1640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd1780a2b80 0x7fd1780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.692+0000 7fd1837fe640 1 -- 172.21.15.133:0/3354523324 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fd17c0ced70 con 0x7fd18c100e20 2024-09-11T23:34:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.692+0000 7fd191ca1640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd1780a2b80 0x7fd1780a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0ff0c0 tx=0x7fd18800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:20.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.695+0000 7fd1837fe640 1 -- 172.21.15.133:0/3354523324 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd17c014030 con 0x7fd18c100e20 2024-09-11T23:34:20.919 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:20.919 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":4,"btime":"2024-09-11T23:34:18:886990+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T23:34:20.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.918+0000 7fd194706640 1 -- 172.21.15.133:0/3354523324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd18c19ba70 con 0x7fd18c100e20 2024-09-11T23:34:20.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.919+0000 7fd1837fe640 1 -- 172.21.15.133:0/3354523324 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7fd18c19ba70 con 0x7fd18c100e20 2024-09-11T23:34:20.919 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 4 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 -- 172.21.15.133:0/3354523324 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd1780a2b80 msgr2=0x7fd1780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd1780a2b80 0x7fd1780a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0ff0c0 tx=0x7fd18800a2d0 comp rx=0 tx=0).stop 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 -- 172.21.15.133:0/3354523324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd18c100e20 msgr2=0x7fd18c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd18c100e20 0x7fd18c19a710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c970 tx=0x7fd17c00ce30 comp rx=0 tx=0).stop 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 -- 172.21.15.133:0/3354523324 shutdown_connections 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd1780a2b80 0x7fd1780a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd18c102ef0 0x7fd18c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd18c102570 0x7fd18c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 --2- 172.21.15.133:0/3354523324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd18c100e20 0x7fd18c19a710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 -- 172.21.15.133:0/3354523324 >> 172.21.15.133:0/3354523324 conn(0x7fd18c0fc7d0 msgr2=0x7fd18c06a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 -- 172.21.15.133:0/3354523324 shutdown_connections 2024-09-11T23:34:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:20.921+0000 7fd194706640 1 -- 172.21.15.133:0/3354523324 wait complete. 2024-09-11T23:34:21.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:20 smithi138 ceph-mon[35142]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:21.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:20 smithi133 ceph-mon[31430]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:21.209 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:34:20 smithi133 systemd[1]: Starting Ceph mds.d for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:34:21.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:20 smithi178 ceph-mon[35882]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:21.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:21 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3354523324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:21.959 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:34:21 smithi133 podman[69022]: 2024-09-11 23:34:21.767679874 +0000 UTC m=+0.016074733 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:21.959 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:34:21 smithi133 podman[69022]: 2024-09-11 23:34:21.885332201 +0000 UTC m=+0.133727056 container create bca343033f22a6c507425826e32a0330e3c71d21b5980fa76fb2241beb9af19e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T23:34:22.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:21 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3354523324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:22.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:21 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3354523324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:22.459 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:34:22 smithi133 podman[69022]: 2024-09-11 23:34:22.112837769 +0000 UTC m=+0.361232612 container init bca343033f22a6c507425826e32a0330e3c71d21b5980fa76fb2241beb9af19e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:34:22.459 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:34:22 smithi133 podman[69022]: 2024-09-11 23:34:22.115727394 +0000 UTC m=+0.364122254 container start bca343033f22a6c507425826e32a0330e3c71d21b5980fa76fb2241beb9af19e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-d, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-11T23:34:22.459 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:34:22 smithi133 ceph-mds[69041]: -- 172.21.15.133:0/1706985065 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646dae83860 con 0x5646dae61400 2024-09-11T23:34:22.459 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:34:22 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-d[69037]: starting mds.d at 2024-09-11T23:34:22.459 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:34:22 smithi133 bash[69022]: bca343033f22a6c507425826e32a0330e3c71d21b5980fa76fb2241beb9af19e 2024-09-11T23:34:22.460 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:34:22 smithi133 systemd[1]: Started Ceph mds.d for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:22.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:22 smithi138 ceph-mon[35142]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:22.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:22 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:22.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:22 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:22.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:22 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:22.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:22 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:22.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:22 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:22.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:22 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:22 smithi133 ceph-mon[31430]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:22 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:22 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:22 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:23.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:22 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:23.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:22 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:23.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:22 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:22 smithi178 ceph-mon[35882]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:22 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:22 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:22 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:22 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:22 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:22 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:23.731 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:34:23.998 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:24.118 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:23 smithi133 ceph-mon[31430]: Deploying daemon mds.e on smithi138 2024-09-11T23:34:24.118 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:23 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.133:6836/3098733497,v1:172.21.15.133:6837/3098733497] up:boot 2024-09-11T23:34:24.118 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:23 smithi133 ceph-mon[31430]: fsmap 4 up:standby 2024-09-11T23:34:24.118 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:23 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:34:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:23 smithi138 ceph-mon[35142]: Deploying daemon mds.e on smithi138 2024-09-11T23:34:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:23 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.133:6836/3098733497,v1:172.21.15.133:6837/3098733497] up:boot 2024-09-11T23:34:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:23 smithi138 ceph-mon[35142]: fsmap 4 up:standby 2024-09-11T23:34:24.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:23 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:34:24.198 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:34:24 smithi138 systemd[1]: Starting Ceph mds.e for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:34:24.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:23 smithi178 ceph-mon[35882]: Deploying daemon mds.e on smithi138 2024-09-11T23:34:24.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:23 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.133:6836/3098733497,v1:172.21.15.133:6837/3098733497] up:boot 2024-09-11T23:34:24.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:23 smithi178 ceph-mon[35882]: fsmap 4 up:standby 2024-09-11T23:34:24.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:23 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:34:24.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.505+0000 7f3a772e5640 1 -- 172.21.15.133:0/2077641343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a70102790 msgr2=0x7f3a701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:24.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.505+0000 7f3a772e5640 1 --2- 172.21.15.133:0/2077641343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a70102790 0x7f3a701067e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009900 tx=0x7f3a6002a410 comp rx=0 tx=0).stop 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.505+0000 7f3a772e5640 1 -- 172.21.15.133:0/2077641343 shutdown_connections 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.505+0000 7f3a772e5640 1 --2- 172.21.15.133:0/2077641343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70106f10 0x7f3a7010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.505+0000 7f3a772e5640 1 --2- 172.21.15.133:0/2077641343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a70102790 0x7f3a701067e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.505+0000 7f3a772e5640 1 --2- 172.21.15.133:0/2077641343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a70101de0 0x7f3a701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.505+0000 7f3a772e5640 1 -- 172.21.15.133:0/2077641343 >> 172.21.15.133:0/2077641343 conn(0x7f3a700fd8c0 msgr2=0x7f3a700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.505+0000 7f3a772e5640 1 -- 172.21.15.133:0/2077641343 shutdown_connections 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.505+0000 7f3a772e5640 1 -- 172.21.15.133:0/2077641343 wait complete. 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.506+0000 7f3a772e5640 1 Processor -- start 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.506+0000 7f3a772e5640 1 -- start start 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.506+0000 7f3a772e5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a70101de0 0x7f3a70198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.506+0000 7f3a772e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70102790 0x7f3a70198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.506+0000 7f3a772e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a70106f10 0x7f3a7019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.506+0000 7f3a772e5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70111f80 con 0x7f3a70106f10 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.506+0000 7f3a772e5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70111e00 con 0x7f3a70102790 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.506+0000 7f3a772e5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70112100 con 0x7f3a70101de0 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.506+0000 7f3a75882640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a70106f10 0x7f3a7019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.506+0000 7f3a75882640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a70106f10 0x7f3a7019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52478/0 (socket says 172.21.15.133:52478) 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.506+0000 7f3a75882640 1 -- 172.21.15.133:0/1824512011 learned_addr learned my addr 172.21.15.133:0/1824512011 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a75081640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a70101de0 0x7f3a70198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a74880640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70102790 0x7f3a70198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a75882640 1 -- 172.21.15.133:0/1824512011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a70101de0 msgr2=0x7f3a70198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a75882640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a70101de0 0x7f3a70198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a75882640 1 -- 172.21.15.133:0/1824512011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70102790 msgr2=0x7f3a70198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:24.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a75882640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70102790 0x7f3a70198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a75882640 1 -- 172.21.15.133:0/1824512011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7019d3d0 con 0x7f3a70106f10 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a74880640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70102790 0x7f3a70198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a75882640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a70106f10 0x7f3a7019ccf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800cce0 tx=0x7f3a6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a667fc640 1 -- 172.21.15.133:0/1824512011 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800f450 con 0x7f3a70106f10 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a667fc640 1 -- 172.21.15.133:0/1824512011 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a6800fad0 con 0x7f3a70106f10 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a772e5640 1 -- 172.21.15.133:0/1824512011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7019d6c0 con 0x7f3a70106f10 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a667fc640 1 -- 172.21.15.133:0/1824512011 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68018c10 con 0x7f3a70106f10 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.507+0000 7f3a772e5640 1 -- 172.21.15.133:0/1824512011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a701a4fa0 con 0x7f3a70106f10 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.508+0000 7f3a772e5640 1 -- 172.21.15.133:0/1824512011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a701021c0 con 0x7f3a70106f10 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.509+0000 7f3a667fc640 1 -- 172.21.15.133:0/1824512011 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3a6800f5f0 con 0x7f3a70106f10 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.510+0000 7f3a667fc640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a5c0a2b80 0x7f3a5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.510+0000 7f3a75081640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a5c0a2b80 0x7f3a5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.510+0000 7f3a667fc640 1 -- 172.21.15.133:0/1824512011 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f3a680cf290 con 0x7f3a70106f10 2024-09-11T23:34:24.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.510+0000 7f3a75081640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a5c0a2b80 0x7f3a5c0a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c005fd0 tx=0x7f3a6c00b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:24.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.514+0000 7f3a667fc640 1 -- 172.21.15.133:0/1824512011 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a68014030 con 0x7f3a70106f10 2024-09-11T23:34:24.543 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:34:24 smithi138 podman[64617]: 2024-09-11 23:34:24.180261793 +0000 UTC m=+0.015939419 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:24.543 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:34:24 smithi138 podman[64617]: 2024-09-11 23:34:24.308306956 +0000 UTC m=+0.143984569 container create 4a6d2a34381e67a76b11d4afe9149ac32fc17811a4bfcf14361ee42cfcfddd42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:34:24.543 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:34:24 smithi138 podman[64617]: 2024-09-11 23:34:24.544420441 +0000 UTC m=+0.380098055 container init 4a6d2a34381e67a76b11d4afe9149ac32fc17811a4bfcf14361ee42cfcfddd42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T23:34:24.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.724+0000 7f3a772e5640 1 -- 172.21.15.133:0/1824512011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a7002d7c0 con 0x7f3a70106f10 2024-09-11T23:34:24.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.725+0000 7f3a667fc640 1 -- 172.21.15.133:0/1824512011 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7f3a6809c690 con 0x7f3a70106f10 2024-09-11T23:34:24.724 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:24.724 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":5,"btime":"2024-09-11T23:34:22:903592+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T23:34:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 5 2024-09-11T23:34:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 -- 172.21.15.133:0/1824512011 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a5c0a2b80 msgr2=0x7f3a5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a5c0a2b80 0x7f3a5c0a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c005fd0 tx=0x7f3a6c00b590 comp rx=0 tx=0).stop 2024-09-11T23:34:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 -- 172.21.15.133:0/1824512011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a70106f10 msgr2=0x7f3a7019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a70106f10 0x7f3a7019ccf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800cce0 tx=0x7f3a6800a620 comp rx=0 tx=0).stop 2024-09-11T23:34:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 -- 172.21.15.133:0/1824512011 shutdown_connections 2024-09-11T23:34:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a5c0a2b80 0x7f3a5c0a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a70106f10 0x7f3a7019ccf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70102790 0x7f3a70198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 --2- 172.21.15.133:0/1824512011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a70101de0 0x7f3a70198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 -- 172.21.15.133:0/1824512011 >> 172.21.15.133:0/1824512011 conn(0x7f3a700fd8c0 msgr2=0x7f3a700fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:24.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 -- 172.21.15.133:0/1824512011 shutdown_connections 2024-09-11T23:34:24.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:24.727+0000 7f3a772e5640 1 -- 172.21.15.133:0/1824512011 wait complete. 2024-09-11T23:34:24.914 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:34:24 smithi138 podman[64617]: 2024-09-11 23:34:24.547328963 +0000 UTC m=+0.383006575 container start 4a6d2a34381e67a76b11d4afe9149ac32fc17811a4bfcf14361ee42cfcfddd42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:34:24.914 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:34:24 smithi138 ceph-mds[64636]: -- 172.21.15.138:0/3169327428 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b398c1f860 con 0x55b398bfd400 2024-09-11T23:34:24.914 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:34:24 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e[64632]: starting mds.e at 2024-09-11T23:34:24.914 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:34:24 smithi138 bash[64617]: 4a6d2a34381e67a76b11d4afe9149ac32fc17811a4bfcf14361ee42cfcfddd42 2024-09-11T23:34:24.914 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:34:24 smithi138 systemd[1]: Started Ceph mds.e for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:25.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:24 smithi133 ceph-mon[31430]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:25.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:25.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:25.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:25.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:25.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:25.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:24 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:25.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:24 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1824512011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:24 smithi138 ceph-mon[35142]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:24 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:24 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1824512011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:25.284 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:24 smithi178 ceph-mon[35882]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:25.285 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:25.285 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:25.285 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:25.285 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:25.285 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:25.285 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:24 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:25.285 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:24 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1824512011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:25 smithi138 ceph-mon[35142]: Deploying daemon mds.f on smithi178 2024-09-11T23:34:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:25 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] up:boot 2024-09-11T23:34:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:25 smithi138 ceph-mon[35142]: fsmap 5 up:standby 2024-09-11T23:34:26.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:25 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:34:26.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:25 smithi133 ceph-mon[31430]: Deploying daemon mds.f on smithi178 2024-09-11T23:34:26.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:25 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] up:boot 2024-09-11T23:34:26.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:25 smithi133 ceph-mon[31430]: fsmap 5 up:standby 2024-09-11T23:34:26.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:25 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:34:26.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:25 smithi178 ceph-mon[35882]: Deploying daemon mds.f on smithi178 2024-09-11T23:34:26.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:25 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] up:boot 2024-09-11T23:34:26.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:25 smithi178 ceph-mon[35882]: fsmap 5 up:standby 2024-09-11T23:34:26.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:25 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:34:26.925 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:34:26 smithi178 podman[65456]: 2024-09-11 23:34:26.869905758 +0000 UTC m=+0.134586967 container create 486f09cfb7211226893b4a388965c75d5895c2265d4eb7dfdbcc962b73abfb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:34:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:26 smithi138 ceph-mon[35142]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:27.204 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:26 smithi178 ceph-mon[35882]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:27.205 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:34:27 smithi178 podman[65456]: 2024-09-11 23:34:27.097387773 +0000 UTC m=+0.362068984 container init 486f09cfb7211226893b4a388965c75d5895c2265d4eb7dfdbcc962b73abfb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:34:27.205 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:34:27 smithi178 podman[65456]: 2024-09-11 23:34:27.100216466 +0000 UTC m=+0.364897685 container start 486f09cfb7211226893b4a388965c75d5895c2265d4eb7dfdbcc962b73abfb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T23:34:27.205 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:34:27 smithi178 ceph-mds[65475]: -- 172.21.15.178:0/205538513 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e250ec1c20 con 0x55e250e9f400 2024-09-11T23:34:27.205 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:34:27 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: starting mds.f at 2024-09-11T23:34:27.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:26 smithi133 ceph-mon[31430]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:27.506 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:34:27.540 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:34:27 smithi178 bash[65456]: 486f09cfb7211226893b4a388965c75d5895c2265d4eb7dfdbcc962b73abfb9d 2024-09-11T23:34:27.540 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:34:27 smithi178 systemd[1]: Started Ceph mds.f for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:27.775 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:28.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.295+0000 7fdf73593640 1 -- 172.21.15.133:0/3018250078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf6c1027a0 msgr2=0x7fdf6c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.295+0000 7fdf73593640 1 --2- 172.21.15.133:0/3018250078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf6c1027a0 0x7fdf6c1067f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009900 tx=0x7fdf6002a410 comp rx=0 tx=0).stop 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.295+0000 7fdf73593640 1 -- 172.21.15.133:0/3018250078 shutdown_connections 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.295+0000 7fdf73593640 1 --2- 172.21.15.133:0/3018250078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c106f20 0x7fdf6c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.295+0000 7fdf73593640 1 --2- 172.21.15.133:0/3018250078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf6c1027a0 0x7fdf6c1067f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.295+0000 7fdf73593640 1 --2- 172.21.15.133:0/3018250078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c101df0 0x7fdf6c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf73593640 1 -- 172.21.15.133:0/3018250078 >> 172.21.15.133:0/3018250078 conn(0x7fdf6c0fd8f0 msgr2=0x7fdf6c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf73593640 1 -- 172.21.15.133:0/3018250078 shutdown_connections 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf73593640 1 -- 172.21.15.133:0/3018250078 wait complete. 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf73593640 1 Processor -- start 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf73593640 1 -- start start 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf73593640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c101df0 0x7fdf6c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:28.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf73593640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c1027a0 0x7fdf6c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf73593640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf6c106f20 0x7fdf6c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf73593640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c111f80 con 0x7fdf6c106f20 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf73593640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c111e00 con 0x7fdf6c1027a0 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf73593640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c112100 con 0x7fdf6c101df0 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.296+0000 7fdf71b30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf6c106f20 0x7fdf6c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf71b30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf6c106f20 0x7fdf6c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52496/0 (socket says 172.21.15.133:52496) 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf71b30640 1 -- 172.21.15.133:0/827979948 learned_addr learned my addr 172.21.15.133:0/827979948 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf7132f640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c101df0 0x7fdf6c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf70b2e640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c1027a0 0x7fdf6c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf71b30640 1 -- 172.21.15.133:0/827979948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c101df0 msgr2=0x7fdf6c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf71b30640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c101df0 0x7fdf6c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf71b30640 1 -- 172.21.15.133:0/827979948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c1027a0 msgr2=0x7fdf6c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:28.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf71b30640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c1027a0 0x7fdf6c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf71b30640 1 -- 172.21.15.133:0/827979948 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c19d390 con 0x7fdf6c106f20 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf71b30640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf6c106f20 0x7fdf6c19ccb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00cad0 tx=0x7fdf5c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf6a7fc640 1 -- 172.21.15.133:0/827979948 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00ae20 con 0x7fdf6c106f20 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf73593640 1 -- 172.21.15.133:0/827979948 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6c19d680 con 0x7fdf6c106f20 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf6a7fc640 1 -- 172.21.15.133:0/827979948 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5c005020 con 0x7fdf6c106f20 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf6a7fc640 1 -- 172.21.15.133:0/827979948 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00f6b0 con 0x7fdf6c106f20 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.297+0000 7fdf73593640 1 -- 172.21.15.133:0/827979948 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6c1a4f60 con 0x7fdf6c106f20 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.298+0000 7fdf73593640 1 -- 172.21.15.133:0/827979948 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6c1021d0 con 0x7fdf6c106f20 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.299+0000 7fdf6a7fc640 1 -- 172.21.15.133:0/827979948 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdf5c0051d0 con 0x7fdf6c106f20 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.300+0000 7fdf6a7fc640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdf540a2b80 0x7fdf540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.300+0000 7fdf7132f640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdf540a2b80 0x7fdf540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:28.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.300+0000 7fdf6a7fc640 1 -- 172.21.15.133:0/827979948 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fdf5c0cef30 con 0x7fdf6c106f20 2024-09-11T23:34:28.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.300+0000 7fdf7132f640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdf540a2b80 0x7fdf540a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdf64005e10 tx=0x7fdf64005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:28.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.304+0000 7fdf6a7fc640 1 -- 172.21.15.133:0/827979948 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf5c014030 con 0x7fdf6c106f20 2024-09-11T23:34:28.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.506+0000 7fdf73593640 1 -- 172.21.15.133:0/827979948 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf6c02d7c0 con 0x7fdf6c106f20 2024-09-11T23:34:28.506 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:28.506 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:28.506 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:28.506 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:28.506 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:28.506 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:28.507 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:28 smithi133 ceph-mon[31430]: Deploying daemon mds.c on smithi178 2024-09-11T23:34:28.507 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:28 smithi133 ceph-mon[31430]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:28.507 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:28 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6834/1681049778,v1:172.21.15.178:6835/1681049778] up:boot 2024-09-11T23:34:28.507 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:28 smithi133 ceph-mon[31430]: fsmap 6 up:standby 2024-09-11T23:34:28.507 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:28 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:34:28.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.506+0000 7fdf6a7fc640 1 -- 172.21.15.133:0/827979948 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5575 (secure 0 0 0) 0x7fdf5c09c330 con 0x7fdf6c106f20 2024-09-11T23:34:28.508 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:28.508 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"btime":"2024-09-11T23:34:27:926396+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-11T23:34:28.508 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-09-11T23:34:28.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.508+0000 7fdf73593640 1 -- 172.21.15.133:0/827979948 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdf540a2b80 msgr2=0x7fdf540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:28.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.508+0000 7fdf73593640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdf540a2b80 0x7fdf540a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdf64005e10 tx=0x7fdf64005da0 comp rx=0 tx=0).stop 2024-09-11T23:34:28.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.508+0000 7fdf73593640 1 -- 172.21.15.133:0/827979948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf6c106f20 msgr2=0x7fdf6c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:28.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.508+0000 7fdf73593640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf6c106f20 0x7fdf6c19ccb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00cad0 tx=0x7fdf5c00cf90 comp rx=0 tx=0).stop 2024-09-11T23:34:28.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.509+0000 7fdf73593640 1 -- 172.21.15.133:0/827979948 shutdown_connections 2024-09-11T23:34:28.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.509+0000 7fdf73593640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdf540a2b80 0x7fdf540a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:28.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.509+0000 7fdf73593640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf6c106f20 0x7fdf6c19ccb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:28.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.509+0000 7fdf73593640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c1027a0 0x7fdf6c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:28.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.509+0000 7fdf73593640 1 --2- 172.21.15.133:0/827979948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c101df0 0x7fdf6c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:28.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.509+0000 7fdf73593640 1 -- 172.21.15.133:0/827979948 >> 172.21.15.133:0/827979948 conn(0x7fdf6c0fd8f0 msgr2=0x7fdf6c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:28.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.509+0000 7fdf73593640 1 -- 172.21.15.133:0/827979948 shutdown_connections 2024-09-11T23:34:28.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:28.509+0000 7fdf73593640 1 -- 172.21.15.133:0/827979948 wait complete. 2024-09-11T23:34:28.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:28 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:28.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:28 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:28.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:28 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:28.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:28 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:28.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:28 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:28.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:28 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:28.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:28 smithi178 ceph-mon[35882]: Deploying daemon mds.c on smithi178 2024-09-11T23:34:28.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:28 smithi178 ceph-mon[35882]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:28.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:28 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6834/1681049778,v1:172.21.15.178:6835/1681049778] up:boot 2024-09-11T23:34:28.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:28 smithi178 ceph-mon[35882]: fsmap 6 up:standby 2024-09-11T23:34:28.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:28 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:34:28.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:28 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:28.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:28 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:28.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:28 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:28.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:28 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:28.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:28 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:28.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:28 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:28.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:28 smithi138 ceph-mon[35142]: Deploying daemon mds.c on smithi178 2024-09-11T23:34:28.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:28 smithi138 ceph-mon[35142]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:28.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:28 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6834/1681049778,v1:172.21.15.178:6835/1681049778] up:boot 2024-09-11T23:34:28.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:28 smithi138 ceph-mon[35142]: fsmap 6 up:standby 2024-09-11T23:34:28.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:28 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:34:29.270 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:34:29 smithi178 systemd[1]: Starting Ceph mds.c for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:34:29.538 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:29 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/827979948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:29.539 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:34:29 smithi178 podman[65757]: 2024-09-11 23:34:29.293725385 +0000 UTC m=+0.016288479 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:29.539 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:34:29 smithi178 podman[65757]: 2024-09-11 23:34:29.431885677 +0000 UTC m=+0.154448754 container create 0440d39b47ca86517301520df1f869db6203a87a4254b429b2001437e729ea4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:34:29.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:29 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/827979948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:29.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:29 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/827979948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:30.040 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:34:29 smithi178 podman[65757]: 2024-09-11 23:34:29.684300591 +0000 UTC m=+0.406863674 container init 0440d39b47ca86517301520df1f869db6203a87a4254b429b2001437e729ea4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T23:34:30.040 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:34:29 smithi178 podman[65757]: 2024-09-11 23:34:29.687115427 +0000 UTC m=+0.409678505 container start 0440d39b47ca86517301520df1f869db6203a87a4254b429b2001437e729ea4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T23:34:30.040 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:34:29 smithi178 ceph-mds[65776]: -- 172.21.15.178:0/698216421 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567cf2f7860 con 0x5567cf2d5400 2024-09-11T23:34:30.040 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:34:29 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c[65772]: starting mds.c at 2024-09-11T23:34:30.040 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:34:29 smithi178 bash[65757]: 0440d39b47ca86517301520df1f869db6203a87a4254b429b2001437e729ea4e 2024-09-11T23:34:30.040 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:34:29 smithi178 systemd[1]: Started Ceph mds.c for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:30 smithi138 ceph-mon[35142]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:30 smithi138 ceph-mon[35142]: Deploying daemon mds.h on smithi138 2024-09-11T23:34:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:30 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] up:boot 2024-09-11T23:34:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:30 smithi138 ceph-mon[35142]: fsmap 7 up:standby 2024-09-11T23:34:31.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:30 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:34:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:30 smithi133 ceph-mon[31430]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:31.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:31.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:31.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:30 smithi133 ceph-mon[31430]: Deploying daemon mds.h on smithi138 2024-09-11T23:34:31.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:30 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] up:boot 2024-09-11T23:34:31.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:30 smithi133 ceph-mon[31430]: fsmap 7 up:standby 2024-09-11T23:34:31.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:30 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:34:31.265 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:34:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:30 smithi178 ceph-mon[35882]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:30 smithi178 ceph-mon[35882]: Deploying daemon mds.h on smithi138 2024-09-11T23:34:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:30 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] up:boot 2024-09-11T23:34:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:30 smithi178 ceph-mon[35882]: fsmap 7 up:standby 2024-09-11T23:34:31.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:30 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:34:31.533 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:31.948 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:34:31 smithi138 systemd[1]: Starting Ceph mds.h for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:34:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.047+0000 7fba8cab6640 1 -- 172.21.15.133:0/331754702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88102570 msgr2=0x7fba881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.047+0000 7fba8cab6640 1 --2- 172.21.15.133:0/331754702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88102570 0x7fba881029b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009900 tx=0x7fba7c02a480 comp rx=0 tx=0).stop 2024-09-11T23:34:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.047+0000 7fba8cab6640 1 -- 172.21.15.133:0/331754702 shutdown_connections 2024-09-11T23:34:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.047+0000 7fba8cab6640 1 --2- 172.21.15.133:0/331754702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba88102ef0 0x7fba8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.047+0000 7fba8cab6640 1 --2- 172.21.15.133:0/331754702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88102570 0x7fba881029b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.047+0000 7fba8cab6640 1 --2- 172.21.15.133:0/331754702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba88100e20 0x7fba88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.047+0000 7fba8cab6640 1 -- 172.21.15.133:0/331754702 >> 172.21.15.133:0/331754702 conn(0x7fba880fc7d0 msgr2=0x7fba880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.047+0000 7fba8cab6640 1 -- 172.21.15.133:0/331754702 shutdown_connections 2024-09-11T23:34:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.047+0000 7fba8cab6640 1 -- 172.21.15.133:0/331754702 wait complete. 2024-09-11T23:34:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.048+0000 7fba8cab6640 1 Processor -- start 2024-09-11T23:34:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.048+0000 7fba8cab6640 1 -- start start 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.048+0000 7fba8cab6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba88100e20 0x7fba88198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.048+0000 7fba8cab6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88102570 0x7fba88198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba8cab6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba88102ef0 0x7fba8819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba8cab6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fba88110fa0 con 0x7fba88102570 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba8cab6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba88110e20 con 0x7fba88102ef0 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba8cab6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fba88111120 con 0x7fba88100e20 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba86d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba88102ef0 0x7fba8819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba86d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba88102ef0 0x7fba8819ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58162/0 (socket says 172.21.15.133:58162) 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba86d9d640 1 -- 172.21.15.133:0/3637792931 learned_addr learned my addr 172.21.15.133:0/3637792931 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba85d9b640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88102570 0x7fba88198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba8659c640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba88100e20 0x7fba88198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba86d9d640 1 -- 172.21.15.133:0/3637792931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba88100e20 msgr2=0x7fba88198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:32.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba86d9d640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba88100e20 0x7fba88198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:32.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba86d9d640 1 -- 172.21.15.133:0/3637792931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88102570 msgr2=0x7fba88198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:32.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba86d9d640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88102570 0x7fba88198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:32.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba86d9d640 1 -- 172.21.15.133:0/3637792931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8819d5b0 con 0x7fba88102ef0 2024-09-11T23:34:32.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba86d9d640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba88102ef0 0x7fba8819ced0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fba8000b970 tx=0x7fba8000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:32.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba777fe640 1 -- 172.21.15.133:0/3637792931 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80019070 con 0x7fba88102ef0 2024-09-11T23:34:32.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba777fe640 1 -- 172.21.15.133:0/3637792931 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba8000cb40 con 0x7fba88102ef0 2024-09-11T23:34:32.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba8cab6640 1 -- 172.21.15.133:0/3637792931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8819d8a0 con 0x7fba88102ef0 2024-09-11T23:34:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.049+0000 7fba777fe640 1 -- 172.21.15.133:0/3637792931 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80007650 con 0x7fba88102ef0 2024-09-11T23:34:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.050+0000 7fba8cab6640 1 -- 172.21.15.133:0/3637792931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba881a5180 con 0x7fba88102ef0 2024-09-11T23:34:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.050+0000 7fba8cab6640 1 -- 172.21.15.133:0/3637792931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba88101200 con 0x7fba88102ef0 2024-09-11T23:34:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.052+0000 7fba777fe640 1 -- 172.21.15.133:0/3637792931 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fba8001c070 con 0x7fba88102ef0 2024-09-11T23:34:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.052+0000 7fba777fe640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fba700a2b80 0x7fba700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.052+0000 7fba8659c640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fba700a2b80 0x7fba700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.052+0000 7fba777fe640 1 -- 172.21.15.133:0/3637792931 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fba800ce6e0 con 0x7fba88102ef0 2024-09-11T23:34:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.053+0000 7fba8659c640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fba700a2b80 0x7fba700a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba78009b60 tx=0x7fba78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.055+0000 7fba777fe640 1 -- 172.21.15.133:0/3637792931 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba8009b3f0 con 0x7fba88102ef0 2024-09-11T23:34:32.248 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:34:31 smithi138 podman[64917]: 2024-09-11 23:34:31.996450826 +0000 UTC m=+0.112596927 container create f93610347ef8ddfd745a23897a6ef05ad3d850d91d1a8b1853fa6e7a52be216d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T23:34:32.248 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:34:31 smithi138 podman[64917]: 2024-09-11 23:34:31.89980228 +0000 UTC m=+0.015948377 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.264+0000 7fba8cab6640 1 -- 172.21.15.133:0/3637792931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba881a5350 con 0x7fba88102ef0 2024-09-11T23:34:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.265+0000 7fba777fe640 1 -- 172.21.15.133:0/3637792931 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6403 (secure 0 0 0) 0x7fba8009f6f0 con 0x7fba88102ef0 2024-09-11T23:34:32.265 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:32.265 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"btime":"2024-09-11T23:34:30:271305+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-11T23:34:32.265 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2024-09-11T23:34:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 -- 172.21.15.133:0/3637792931 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fba700a2b80 msgr2=0x7fba700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fba700a2b80 0x7fba700a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba78009b60 tx=0x7fba78005f90 comp rx=0 tx=0).stop 2024-09-11T23:34:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 -- 172.21.15.133:0/3637792931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba88102ef0 msgr2=0x7fba8819ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba88102ef0 0x7fba8819ced0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fba8000b970 tx=0x7fba8000be30 comp rx=0 tx=0).stop 2024-09-11T23:34:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 -- 172.21.15.133:0/3637792931 shutdown_connections 2024-09-11T23:34:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fba700a2b80 0x7fba700a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba88102ef0 0x7fba8819ced0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88102570 0x7fba88198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 --2- 172.21.15.133:0/3637792931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba88100e20 0x7fba88198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 -- 172.21.15.133:0/3637792931 >> 172.21.15.133:0/3637792931 conn(0x7fba880fc7d0 msgr2=0x7fba88069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 -- 172.21.15.133:0/3637792931 shutdown_connections 2024-09-11T23:34:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:32.267+0000 7fba8cab6640 1 -- 172.21.15.133:0/3637792931 wait complete. 2024-09-11T23:34:32.698 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:34:32 smithi138 podman[64917]: 2024-09-11 23:34:32.24900987 +0000 UTC m=+0.365155977 container init f93610347ef8ddfd745a23897a6ef05ad3d850d91d1a8b1853fa6e7a52be216d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-11T23:34:32.698 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:34:32 smithi138 podman[64917]: 2024-09-11 23:34:32.252153299 +0000 UTC m=+0.368299404 container start f93610347ef8ddfd745a23897a6ef05ad3d850d91d1a8b1853fa6e7a52be216d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T23:34:32.698 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:34:32 smithi138 ceph-mds[64936]: -- 172.21.15.138:0/4056420006 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f02b501860 con 0x55f02b4df400 2024-09-11T23:34:32.698 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:34:32 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h[64932]: starting mds.h at 2024-09-11T23:34:32.698 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:34:32 smithi138 bash[64917]: f93610347ef8ddfd745a23897a6ef05ad3d850d91d1a8b1853fa6e7a52be216d 2024-09-11T23:34:32.698 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:34:32 smithi138 systemd[1]: Started Ceph mds.h for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:33.161 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:32 smithi178 ceph-mon[35882]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:33.161 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:32 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3637792931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:33.161 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:32 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:33.161 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:32 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:33.161 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:32 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:33.162 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:32 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:33.162 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:32 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:33.162 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:32 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:33.200 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:32 smithi138 ceph-mon[35142]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:33.201 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:32 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3637792931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:33.201 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:32 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:33.201 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:32 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:33.201 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:32 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:33.201 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:32 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:33.201 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:32 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:33.201 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:32 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:32 smithi133 ceph-mon[31430]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:32 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3637792931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:32 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:32 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:32 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:32 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:32 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:33.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:32 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:34.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:33 smithi178 ceph-mon[35882]: Deploying daemon mds.i on smithi178 2024-09-11T23:34:34.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:33 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.138:6836/1115908278,v1:172.21.15.138:6837/1115908278] up:boot 2024-09-11T23:34:34.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:33 smithi178 ceph-mon[35882]: fsmap 8 up:standby 2024-09-11T23:34:34.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:33 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:34:34.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:33 smithi138 ceph-mon[35142]: Deploying daemon mds.i on smithi178 2024-09-11T23:34:34.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:33 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.138:6836/1115908278,v1:172.21.15.138:6837/1115908278] up:boot 2024-09-11T23:34:34.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:33 smithi138 ceph-mon[35142]: fsmap 8 up:standby 2024-09-11T23:34:34.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:33 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:34:34.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:33 smithi133 ceph-mon[31430]: Deploying daemon mds.i on smithi178 2024-09-11T23:34:34.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:33 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.138:6836/1115908278,v1:172.21.15.138:6837/1115908278] up:boot 2024-09-11T23:34:34.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:33 smithi133 ceph-mon[31430]: fsmap 8 up:standby 2024-09-11T23:34:34.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:33 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:34:34.499 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:34:34 smithi178 systemd[1]: Starting Ceph mds.i for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:34:34.761 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:34:34 smithi178 podman[66055]: 2024-09-11 23:34:34.402845494 +0000 UTC m=+0.016097723 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:34.761 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:34:34 smithi178 podman[66055]: 2024-09-11 23:34:34.508215837 +0000 UTC m=+0.121468056 container create ac19066487aa6cab926dbd00506e81016cef9f4dc0fc551b746319d27db0f232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T23:34:34.761 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:34:34 smithi178 podman[66055]: 2024-09-11 23:34:34.735603997 +0000 UTC m=+0.348856216 container init ac19066487aa6cab926dbd00506e81016cef9f4dc0fc551b746319d27db0f232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T23:34:34.761 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:34:34 smithi178 podman[66055]: 2024-09-11 23:34:34.738381471 +0000 UTC m=+0.351633705 container start ac19066487aa6cab926dbd00506e81016cef9f4dc0fc551b746319d27db0f232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-11T23:34:35.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:34 smithi178 ceph-mon[35882]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:35.040 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:34:34 smithi178 ceph-mds[66073]: -- 172.21.15.178:0/3637232410 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5612943d3860 con 0x5612943b1000 2024-09-11T23:34:35.040 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:34:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i[66069]: starting mds.i at 2024-09-11T23:34:35.040 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:34:34 smithi178 bash[66055]: ac19066487aa6cab926dbd00506e81016cef9f4dc0fc551b746319d27db0f232 2024-09-11T23:34:35.040 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:34:34 smithi178 systemd[1]: Started Ceph mds.i for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:35.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:34:35.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:34 smithi138 ceph-mon[35142]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:34 smithi133 ceph-mon[31430]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:35.949 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:35.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:35.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:35.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:35 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] up:boot 2024-09-11T23:34:35.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:35 smithi133 ceph-mon[31430]: fsmap 9 up:standby 2024-09-11T23:34:35.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:34:35.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:35.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:35.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:35.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:35 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:35.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:35 smithi133 ceph-mon[31430]: Deploying daemon mds.g on smithi133 2024-09-11T23:34:36.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:36.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:36.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:35 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] up:boot 2024-09-11T23:34:36.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:35 smithi138 ceph-mon[35142]: fsmap 9 up:standby 2024-09-11T23:34:36.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:34:36.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:36.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:36.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:36.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:35 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:36.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:35 smithi138 ceph-mon[35142]: Deploying daemon mds.g on smithi133 2024-09-11T23:34:36.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:36.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:36.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:35 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] up:boot 2024-09-11T23:34:36.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:35 smithi178 ceph-mon[35882]: fsmap 9 up:standby 2024-09-11T23:34:36.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:34:36.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:36.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:36.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:36.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:35 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:36.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:35 smithi178 ceph-mon[35882]: Deploying daemon mds.g on smithi133 2024-09-11T23:34:36.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.911+0000 7f1a2dc09640 1 -- 172.21.15.133:0/3384200077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a28102ef0 msgr2=0x7f1a2810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:36.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.911+0000 7f1a2dc09640 1 --2- 172.21.15.133:0/3384200077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a28102ef0 0x7f1a2810da40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009ca0 tx=0x7f1a2002a950 comp rx=0 tx=0).stop 2024-09-11T23:34:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.911+0000 7f1a2dc09640 1 -- 172.21.15.133:0/3384200077 shutdown_connections 2024-09-11T23:34:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.911+0000 7f1a2dc09640 1 --2- 172.21.15.133:0/3384200077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a28102ef0 0x7f1a2810da40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.911+0000 7f1a2dc09640 1 --2- 172.21.15.133:0/3384200077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a28102570 0x7f1a281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.911+0000 7f1a2dc09640 1 --2- 172.21.15.133:0/3384200077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28100e20 0x7f1a28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.911+0000 7f1a2dc09640 1 -- 172.21.15.133:0/3384200077 >> 172.21.15.133:0/3384200077 conn(0x7f1a280fc7d0 msgr2=0x7f1a280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.911+0000 7f1a2dc09640 1 -- 172.21.15.133:0/3384200077 shutdown_connections 2024-09-11T23:34:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.911+0000 7f1a2dc09640 1 -- 172.21.15.133:0/3384200077 wait complete. 2024-09-11T23:34:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.912+0000 7f1a2dc09640 1 Processor -- start 2024-09-11T23:34:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.912+0000 7f1a2dc09640 1 -- start start 2024-09-11T23:34:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.912+0000 7f1a2dc09640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a28100e20 0x7f1a2819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.912+0000 7f1a2dc09640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a28102570 0x7f1a2819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.912+0000 7f1a2dc09640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28102ef0 0x7f1a281963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.912+0000 7f1a2dc09640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28110f80 con 0x7f1a28100e20 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.912+0000 7f1a2dc09640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28110e00 con 0x7f1a28102ef0 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.912+0000 7f1a2dc09640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28111100 con 0x7f1a28102570 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a277fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a28100e20 0x7f1a2819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a277fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a28100e20 0x7f1a2819e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35082/0 (socket says 172.21.15.133:35082) 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a277fe640 1 -- 172.21.15.133:0/2619847214 learned_addr learned my addr 172.21.15.133:0/2619847214 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a26ffd640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a28102570 0x7f1a2819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a27fff640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28102ef0 0x7f1a281963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a277fe640 1 -- 172.21.15.133:0/2619847214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a28102570 msgr2=0x7f1a2819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a277fe640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a28102570 0x7f1a2819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a277fe640 1 -- 172.21.15.133:0/2619847214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28102ef0 msgr2=0x7f1a281963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a277fe640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28102ef0 0x7f1a281963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:36.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a277fe640 1 -- 172.21.15.133:0/2619847214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a28196bf0 con 0x7f1a28100e20 2024-09-11T23:34:36.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a27fff640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28102ef0 0x7f1a281963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:34:36.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a277fe640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a28100e20 0x7f1a2819e070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c890 tx=0x7f1a1800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:36.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a24ff9640 1 -- 172.21.15.133:0/2619847214 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18004220 con 0x7f1a28100e20 2024-09-11T23:34:36.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a2dc09640 1 -- 172.21.15.133:0/2619847214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a28196ee0 con 0x7f1a28100e20 2024-09-11T23:34:36.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a24ff9640 1 -- 172.21.15.133:0/2619847214 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a180043c0 con 0x7f1a28100e20 2024-09-11T23:34:36.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a24ff9640 1 -- 172.21.15.133:0/2619847214 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18004a00 con 0x7f1a28100e20 2024-09-11T23:34:36.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.913+0000 7f1a2dc09640 1 -- 172.21.15.133:0/2619847214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a281b3e00 con 0x7f1a28100e20 2024-09-11T23:34:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.915+0000 7f1a2dc09640 1 -- 172.21.15.133:0/2619847214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a28101200 con 0x7f1a28100e20 2024-09-11T23:34:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.916+0000 7f1a24ff9640 1 -- 172.21.15.133:0/2619847214 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1a180079b0 con 0x7f1a28100e20 2024-09-11T23:34:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.916+0000 7f1a24ff9640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1a100a2b80 0x7f1a100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.916+0000 7f1a26ffd640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1a100a2b80 0x7f1a100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.916+0000 7f1a24ff9640 1 -- 172.21.15.133:0/2619847214 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f1a180ce880 con 0x7f1a28100e20 2024-09-11T23:34:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.917+0000 7f1a26ffd640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1a100a2b80 0x7f1a100a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a2810dd10 tx=0x7f1a1c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:36.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:36.921+0000 7f1a24ff9640 1 -- 172.21.15.133:0/2619847214 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a18014030 con 0x7f1a28100e20 2024-09-11T23:34:36.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:36 smithi133 ceph-mon[31430]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:36.959 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:34:36 smithi133 systemd[1]: Starting Ceph mds.g for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:34:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.138+0000 7f1a2dc09640 1 -- 172.21.15.133:0/2619847214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a281b3fd0 con 0x7f1a28100e20 2024-09-11T23:34:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.139+0000 7f1a24ff9640 1 -- 172.21.15.133:0/2619847214 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8061 (secure 0 0 0) 0x7f1a1809bc80 con 0x7f1a28100e20 2024-09-11T23:34:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2024-09-11T23:34:37.140 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:37.140 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"btime":"2024-09-11T23:34:34:903972+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T23:34:37.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 -- 172.21.15.133:0/2619847214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1a100a2b80 msgr2=0x7f1a100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:37.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1a100a2b80 0x7f1a100a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a2810dd10 tx=0x7f1a1c00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:34:37.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 -- 172.21.15.133:0/2619847214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a28100e20 msgr2=0x7f1a2819e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:37.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a28100e20 0x7f1a2819e070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c890 tx=0x7f1a1800cc90 comp rx=0 tx=0).stop 2024-09-11T23:34:37.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 -- 172.21.15.133:0/2619847214 shutdown_connections 2024-09-11T23:34:37.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1a100a2b80 0x7f1a100a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:37.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28102ef0 0x7f1a281963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:37.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a28102570 0x7f1a2819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:37.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 --2- 172.21.15.133:0/2619847214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a28100e20 0x7f1a2819e070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:37.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 -- 172.21.15.133:0/2619847214 >> 172.21.15.133:0/2619847214 conn(0x7f1a280fc7d0 msgr2=0x7f1a28069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 -- 172.21.15.133:0/2619847214 shutdown_connections 2024-09-11T23:34:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:37.142+0000 7f1a2dc09640 1 -- 172.21.15.133:0/2619847214 wait complete. 2024-09-11T23:34:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:36 smithi138 ceph-mon[35142]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:36 smithi178 ceph-mon[35882]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:37.459 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:34:37 smithi133 podman[69782]: 2024-09-11 23:34:37.138611825 +0000 UTC m=+0.111568217 container create 0fa68f80ac2e774301cbfa86b677e307c55c76f559cf4bb6bd6031c3bc953353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:34:37.459 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:34:37 smithi133 podman[69782]: 2024-09-11 23:34:37.044954268 +0000 UTC m=+0.017910681 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:37.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:37 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2619847214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:38.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:37 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2619847214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:38.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:37 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2619847214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:38.459 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:34:38 smithi133 podman[69782]: 2024-09-11 23:34:38.020264924 +0000 UTC m=+0.993221316 container init 0fa68f80ac2e774301cbfa86b677e307c55c76f559cf4bb6bd6031c3bc953353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T23:34:38.459 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:34:38 smithi133 podman[69782]: 2024-09-11 23:34:38.023468115 +0000 UTC m=+0.996424511 container start 0fa68f80ac2e774301cbfa86b677e307c55c76f559cf4bb6bd6031c3bc953353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:34:38.459 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:34:38 smithi133 ceph-mds[69818]: -- 172.21.15.133:0/582008793 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ed91ab860 con 0x557ed9189000 2024-09-11T23:34:38.459 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:34:38 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g[69809]: starting mds.g at 2024-09-11T23:34:38.460 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:34:38 smithi133 bash[69782]: 0fa68f80ac2e774301cbfa86b677e307c55c76f559cf4bb6bd6031c3bc953353 2024-09-11T23:34:38.460 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:34:38 smithi133 systemd[1]: Started Ceph mds.g for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:38 smithi138 ceph-mon[35142]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:38 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:38 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:38 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:38 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:38 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:38 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:38 smithi133 ceph-mon[31430]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:38 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:38 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:39.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:38 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:39.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:38 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:39.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:38 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:39.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:38 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:38 smithi178 ceph-mon[35882]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:38 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:38 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:38 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:38 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:38 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:38 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:39.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:34:40.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:39 smithi138 ceph-mon[35142]: Deploying daemon mds.a on smithi133 2024-09-11T23:34:40.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:39 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.133:6838/570951791,v1:172.21.15.133:6839/570951791] up:boot 2024-09-11T23:34:40.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:39 smithi138 ceph-mon[35142]: fsmap 10 up:standby 2024-09-11T23:34:40.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:39 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:34:40.204 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:39 smithi133 ceph-mon[31430]: Deploying daemon mds.a on smithi133 2024-09-11T23:34:40.204 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:39 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.133:6838/570951791,v1:172.21.15.133:6839/570951791] up:boot 2024-09-11T23:34:40.204 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:39 smithi133 ceph-mon[31430]: fsmap 10 up:standby 2024-09-11T23:34:40.204 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:39 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:34:40.205 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:34:39 smithi133 systemd[1]: Starting Ceph mds.a for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:34:40.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:39 smithi178 ceph-mon[35882]: Deploying daemon mds.a on smithi133 2024-09-11T23:34:40.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:39 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.133:6838/570951791,v1:172.21.15.133:6839/570951791] up:boot 2024-09-11T23:34:40.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:39 smithi178 ceph-mon[35882]: fsmap 10 up:standby 2024-09-11T23:34:40.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:39 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:34:40.459 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:34:40 smithi133 podman[70119]: 2024-09-11 23:34:40.107678571 +0000 UTC m=+0.016545542 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:40.459 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:34:40 smithi133 podman[70119]: 2024-09-11 23:34:40.212485467 +0000 UTC m=+0.121352423 container create f47ac7b17b43902a563b4032b6dceb62030ec34a22edc7a4c8e8271449649595 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T23:34:40.460 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:34:40 smithi133 podman[70119]: 2024-09-11 23:34:40.448314664 +0000 UTC m=+0.357181623 container init f47ac7b17b43902a563b4032b6dceb62030ec34a22edc7a4c8e8271449649595 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T23:34:40.460 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:34:40 smithi133 podman[70119]: 2024-09-11 23:34:40.451093353 +0000 UTC m=+0.359960310 container start f47ac7b17b43902a563b4032b6dceb62030ec34a22edc7a4c8e8271449649595 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-11T23:34:40.614 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:40.736 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:34:40 smithi133 ceph-mds[70162]: -- 172.21.15.133:0/2319494966 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5611edcd3860 con 0x5611edcb0800 2024-09-11T23:34:40.736 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:34:40 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a[70146]: starting mds.a at 2024-09-11T23:34:40.736 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:34:40 smithi133 bash[70119]: f47ac7b17b43902a563b4032b6dceb62030ec34a22edc7a4c8e8271449649595 2024-09-11T23:34:40.736 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:34:40 smithi133 systemd[1]: Started Ceph mds.a for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:41.018 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:40 smithi133 ceph-mon[31430]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:41.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:41.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:41.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:41.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:41.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:41.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:40 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:41.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.099+0000 7eff9375f640 1 -- 172.21.15.133:0/3693123641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff8c1011e0 msgr2=0x7eff8c101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:41.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.099+0000 7eff9375f640 1 --2- 172.21.15.133:0/3693123641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff8c1011e0 0x7eff8c101640 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff80009900 tx=0x7eff8002a410 comp rx=0 tx=0).stop 2024-09-11T23:34:41.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.099+0000 7eff9375f640 1 -- 172.21.15.133:0/3693123641 shutdown_connections 2024-09-11T23:34:41.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.099+0000 7eff9375f640 1 --2- 172.21.15.133:0/3693123641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff8c101b80 0x7eff8c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:41.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.099+0000 7eff9375f640 1 --2- 172.21.15.133:0/3693123641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff8c1011e0 0x7eff8c101640 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:41.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.099+0000 7eff9375f640 1 --2- 172.21.15.133:0/3693123641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c105ab0 0x7eff8c105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:41.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.099+0000 7eff9375f640 1 -- 172.21.15.133:0/3693123641 >> 172.21.15.133:0/3693123641 conn(0x7eff8c0fbee0 msgr2=0x7eff8c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.099+0000 7eff9375f640 1 -- 172.21.15.133:0/3693123641 shutdown_connections 2024-09-11T23:34:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.099+0000 7eff9375f640 1 -- 172.21.15.133:0/3693123641 wait complete. 2024-09-11T23:34:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff9375f640 1 Processor -- start 2024-09-11T23:34:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff9375f640 1 -- start start 2024-09-11T23:34:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff9375f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1011e0 0x7eff8c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff9375f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff8c101b80 0x7eff8c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff9375f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff8c105ab0 0x7eff8c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff9375f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c110ed0 con 0x7eff8c105ab0 2024-09-11T23:34:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff9375f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c110d50 con 0x7eff8c1011e0 2024-09-11T23:34:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff9375f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c111050 con 0x7eff8c101b80 2024-09-11T23:34:41.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff91cfc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff8c105ab0 0x7eff8c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:41.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff91cfc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff8c105ab0 0x7eff8c19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35122/0 (socket says 172.21.15.133:35122) 2024-09-11T23:34:41.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff91cfc640 1 -- 172.21.15.133:0/2855765406 learned_addr learned my addr 172.21.15.133:0/2855765406 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:41.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff914fb640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1011e0 0x7eff8c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:41.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.100+0000 7eff90cfa640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff8c101b80 0x7eff8c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:41.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff91cfc640 1 -- 172.21.15.133:0/2855765406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff8c101b80 msgr2=0x7eff8c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:41.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff91cfc640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff8c101b80 0x7eff8c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:41.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff91cfc640 1 -- 172.21.15.133:0/2855765406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1011e0 msgr2=0x7eff8c1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:41.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff91cfc640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1011e0 0x7eff8c1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff91cfc640 1 -- 172.21.15.133:0/2855765406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c19c260 con 0x7eff8c105ab0 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff914fb640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1011e0 0x7eff8c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff91cfc640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff8c105ab0 0x7eff8c19bb80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00e9d0 tx=0x7eff7c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff8a7fc640 1 -- 172.21.15.133:0/2855765406 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00cd80 con 0x7eff8c105ab0 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff9375f640 1 -- 172.21.15.133:0/2855765406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c19c550 con 0x7eff8c105ab0 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff8a7fc640 1 -- 172.21.15.133:0/2855765406 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff7c005020 con 0x7eff8c105ab0 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff8a7fc640 1 -- 172.21.15.133:0/2855765406 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0116d0 con 0x7eff8c105ab0 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.101+0000 7eff9375f640 1 -- 172.21.15.133:0/2855765406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c1a3e30 con 0x7eff8c105ab0 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.102+0000 7eff9375f640 1 -- 172.21.15.133:0/2855765406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8c101640 con 0x7eff8c105ab0 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.104+0000 7eff8a7fc640 1 -- 172.21.15.133:0/2855765406 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7eff7c0051d0 con 0x7eff8c105ab0 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.104+0000 7eff8a7fc640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7eff740a2b80 0x7eff740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.105+0000 7eff914fb640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7eff740a2b80 0x7eff740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:41.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.105+0000 7eff8a7fc640 1 -- 172.21.15.133:0/2855765406 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7eff7c0cf070 con 0x7eff8c105ab0 2024-09-11T23:34:41.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.105+0000 7eff914fb640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7eff740a2b80 0x7eff740a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7eff84009fc0 tx=0x7eff84005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:41.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.108+0000 7eff8a7fc640 1 -- 172.21.15.133:0/2855765406 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff7c016030 con 0x7eff8c105ab0 2024-09-11T23:34:41.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:40 smithi138 ceph-mon[35142]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:41.200 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:41.200 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:41.200 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:41.200 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:41.200 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:41.200 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:40 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:40 smithi178 ceph-mon[35882]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:34:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:34:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:40 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.306+0000 7eff9375f640 1 -- 172.21.15.133:0/2855765406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff8c198610 con 0x7eff8c105ab0 2024-09-11T23:34:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.307+0000 7eff8a7fc640 1 -- 172.21.15.133:0/2855765406 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9716 (secure 0 0 0) 0x7eff8c198610 con 0x7eff8c105ab0 2024-09-11T23:34:41.306 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:41.307 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":12,"btime":"2024-09-11T23:34:40:939317+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T23:34:41.307 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 12 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 -- 172.21.15.133:0/2855765406 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7eff740a2b80 msgr2=0x7eff740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7eff740a2b80 0x7eff740a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7eff84009fc0 tx=0x7eff84005f90 comp rx=0 tx=0).stop 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 -- 172.21.15.133:0/2855765406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff8c105ab0 msgr2=0x7eff8c19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff8c105ab0 0x7eff8c19bb80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00e9d0 tx=0x7eff7c00ee90 comp rx=0 tx=0).stop 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 -- 172.21.15.133:0/2855765406 shutdown_connections 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7eff740a2b80 0x7eff740a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff8c105ab0 0x7eff8c19bb80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff8c101b80 0x7eff8c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 --2- 172.21.15.133:0/2855765406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1011e0 0x7eff8c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 -- 172.21.15.133:0/2855765406 >> 172.21.15.133:0/2855765406 conn(0x7eff8c0fbee0 msgr2=0x7eff8c0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 -- 172.21.15.133:0/2855765406 shutdown_connections 2024-09-11T23:34:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:41.309+0000 7eff9375f640 1 -- 172.21.15.133:0/2855765406 wait complete. 2024-09-11T23:34:42.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:41 smithi133 ceph-mon[31430]: Deploying daemon mds.b on smithi138 2024-09-11T23:34:42.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:41 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.133:6840/1294883246,v1:172.21.15.133:6841/1294883246] up:boot 2024-09-11T23:34:42.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:41 smithi133 ceph-mon[31430]: fsmap 11 up:standby 2024-09-11T23:34:42.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:41 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:34:42.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:41 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2855765406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:41 smithi178 ceph-mon[35882]: Deploying daemon mds.b on smithi138 2024-09-11T23:34:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:41 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.133:6840/1294883246,v1:172.21.15.133:6841/1294883246] up:boot 2024-09-11T23:34:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:41 smithi178 ceph-mon[35882]: fsmap 11 up:standby 2024-09-11T23:34:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:41 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:34:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:41 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2855765406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:42.327 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:41 smithi138 ceph-mon[35142]: Deploying daemon mds.b on smithi138 2024-09-11T23:34:42.327 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:41 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.133:6840/1294883246,v1:172.21.15.133:6841/1294883246] up:boot 2024-09-11T23:34:42.327 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:41 smithi138 ceph-mon[35142]: fsmap 11 up:standby 2024-09-11T23:34:42.327 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:41 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:34:42.327 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:41 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2855765406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:42.698 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:34:42 smithi138 systemd[1]: Starting Ceph mds.b for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:34:42.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:42 smithi138 ceph-mon[35142]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:42.985 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:34:42 smithi138 podman[65218]: 2024-09-11 23:34:42.630299127 +0000 UTC m=+0.015437347 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:42.985 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:34:42 smithi138 podman[65218]: 2024-09-11 23:34:42.750638316 +0000 UTC m=+0.135776598 container create dfb088c5ff20c2c81fc92f76bf589f8bd98a68ab3af5b3dea38a9876cee53bd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:34:42.985 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:34:42 smithi138 podman[65218]: 2024-09-11 23:34:42.986313194 +0000 UTC m=+0.371451408 container init dfb088c5ff20c2c81fc92f76bf589f8bd98a68ab3af5b3dea38a9876cee53bd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T23:34:43.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:42 smithi133 ceph-mon[31430]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:43.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:42 smithi178 ceph-mon[35882]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:43.448 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:34:42 smithi138 podman[65218]: 2024-09-11 23:34:42.989322135 +0000 UTC m=+0.374460351 container start dfb088c5ff20c2c81fc92f76bf589f8bd98a68ab3af5b3dea38a9876cee53bd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T23:34:43.448 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:34:43 smithi138 ceph-mds[65237]: -- 172.21.15.138:0/3492295487 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56477ba67c20 con 0x56477ba44800 2024-09-11T23:34:43.448 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:34:43 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b[65233]: starting mds.b at 2024-09-11T23:34:43.448 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:34:43 smithi138 bash[65218]: dfb088c5ff20c2c81fc92f76bf589f8bd98a68ab3af5b3dea38a9876cee53bd4 2024-09-11T23:34:43.448 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:34:43 smithi138 systemd[1]: Started Ceph mds.b for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:34:43.993 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:34:44.355 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:44 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:44 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:44 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:44 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:44 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:44 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:44 smithi138 ceph-mon[35142]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:44 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] up:boot 2024-09-11T23:34:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:44 smithi138 ceph-mon[35142]: fsmap 12 up:standby 2024-09-11T23:34:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:44 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:34:44.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:44 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:44 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:44 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:44 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:44 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:44 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:44.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:44 smithi133 ceph-mon[31430]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:44.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:44 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] up:boot 2024-09-11T23:34:44.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:44 smithi133 ceph-mon[31430]: fsmap 12 up:standby 2024-09-11T23:34:44.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:44 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:34:44.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:44 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:44 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:44 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:44 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:44 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:44.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:44 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:44.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:44 smithi178 ceph-mon[35882]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:44.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:44 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] up:boot 2024-09-11T23:34:44.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:44 smithi178 ceph-mon[35882]: fsmap 12 up:standby 2024-09-11T23:34:44.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:44 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:34:44.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.851+0000 7f77f9ac9640 1 -- 172.21.15.133:0/1290701884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f4102ef0 msgr2=0x7f77f410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.851+0000 7f77f9ac9640 1 --2- 172.21.15.133:0/1290701884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f4102ef0 0x7f77f410da40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009ca0 tx=0x7f77ec02a950 comp rx=0 tx=0).stop 2024-09-11T23:34:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.851+0000 7f77f9ac9640 1 -- 172.21.15.133:0/1290701884 shutdown_connections 2024-09-11T23:34:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.851+0000 7f77f9ac9640 1 --2- 172.21.15.133:0/1290701884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f4102ef0 0x7f77f410da40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.851+0000 7f77f9ac9640 1 --2- 172.21.15.133:0/1290701884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f4102570 0x7f77f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.851+0000 7f77f9ac9640 1 --2- 172.21.15.133:0/1290701884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f4100e20 0x7f77f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.851+0000 7f77f9ac9640 1 -- 172.21.15.133:0/1290701884 >> 172.21.15.133:0/1290701884 conn(0x7f77f40fc7d0 msgr2=0x7f77f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.851+0000 7f77f9ac9640 1 -- 172.21.15.133:0/1290701884 shutdown_connections 2024-09-11T23:34:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.851+0000 7f77f9ac9640 1 -- 172.21.15.133:0/1290701884 wait complete. 2024-09-11T23:34:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f9ac9640 1 Processor -- start 2024-09-11T23:34:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f9ac9640 1 -- start start 2024-09-11T23:34:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f9ac9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f4100e20 0x7f77f40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f9ac9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f4102570 0x7f77f40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f9ac9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f4102ef0 0x7f77f40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f9ac9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4110f80 con 0x7f77f4100e20 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f9ac9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4110e00 con 0x7f77f4102ef0 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f9ac9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4111100 con 0x7f77f4102570 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f4100e20 0x7f77f40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f4100e20 0x7f77f40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47344/0 (socket says 172.21.15.133:47344) 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f37fe640 1 -- 172.21.15.133:0/572211980 learned_addr learned my addr 172.21.15.133:0/572211980 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77ebfff640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f4102570 0x7f77f40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f3fff640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f4102ef0 0x7f77f40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f37fe640 1 -- 172.21.15.133:0/572211980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f4102570 msgr2=0x7f77f40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f37fe640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f4102570 0x7f77f40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f37fe640 1 -- 172.21.15.133:0/572211980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f4102ef0 msgr2=0x7f77f40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:44.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f37fe640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f4102ef0 0x7f77f40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.852+0000 7f77f37fe640 1 -- 172.21.15.133:0/572211980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f41a69f0 con 0x7f77f4100e20 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.853+0000 7f77f37fe640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f4100e20 0x7f77f40fcc80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f77e400e9f0 tx=0x7f77e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.853+0000 7f77f17fa640 1 -- 172.21.15.133:0/572211980 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e400cd80 con 0x7f77f4100e20 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.853+0000 7f77f17fa640 1 -- 172.21.15.133:0/572211980 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e4010910 con 0x7f77f4100e20 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.853+0000 7f77f9ac9640 1 -- 172.21.15.133:0/572211980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f41a6ce0 con 0x7f77f4100e20 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.853+0000 7f77f17fa640 1 -- 172.21.15.133:0/572211980 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4018990 con 0x7f77f4100e20 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.853+0000 7f77f9ac9640 1 -- 172.21.15.133:0/572211980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f41a7210 con 0x7f77f4100e20 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.854+0000 7f77f9ac9640 1 -- 172.21.15.133:0/572211980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f4101200 con 0x7f77f4100e20 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.855+0000 7f77f17fa640 1 -- 172.21.15.133:0/572211980 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f77e4010ac0 con 0x7f77f4100e20 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.856+0000 7f77f17fa640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77e00a2c00 0x7f77e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.856+0000 7f77ebfff640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77e00a2c00 0x7f77e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.856+0000 7f77f17fa640 1 -- 172.21.15.133:0/572211980 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f77e40ceec0 con 0x7f77f4100e20 2024-09-11T23:34:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.856+0000 7f77ebfff640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77e00a2c00 0x7f77e00a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77f40fe710 tx=0x7f77dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:44.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:44.860+0000 7f77f17fa640 1 -- 172.21.15.133:0/572211980 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77e409c2c0 con 0x7f77f4100e20 2024-09-11T23:34:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.106+0000 7f77f9ac9640 1 -- 172.21.15.133:0/572211980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77f410a0a0 con 0x7f77f4100e20 2024-09-11T23:34:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.106+0000 7f77f17fa640 1 -- 172.21.15.133:0/572211980 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10545 (secure 0 0 0) 0x7f77e409bbc0 con 0x7f77f4100e20 2024-09-11T23:34:45.106 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:45.106 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":13,"btime":"2024-09-11T23:34:43:954935+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T23:34:45.107 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 13 2024-09-11T23:34:45.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.109+0000 7f77f9ac9640 1 -- 172.21.15.133:0/572211980 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77e00a2c00 msgr2=0x7f77e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:45.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.109+0000 7f77f9ac9640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77e00a2c00 0x7f77e00a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77f40fe710 tx=0x7f77dc005f90 comp rx=0 tx=0).stop 2024-09-11T23:34:45.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.109+0000 7f77f9ac9640 1 -- 172.21.15.133:0/572211980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f4100e20 msgr2=0x7f77f40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:45.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.109+0000 7f77f9ac9640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f4100e20 0x7f77f40fcc80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f77e400e9f0 tx=0x7f77e400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:34:45.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.110+0000 7f77f9ac9640 1 -- 172.21.15.133:0/572211980 shutdown_connections 2024-09-11T23:34:45.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.110+0000 7f77f9ac9640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77e00a2c00 0x7f77e00a50c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:45.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.110+0000 7f77f9ac9640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f4102ef0 0x7f77f40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:45.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.110+0000 7f77f9ac9640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f4102570 0x7f77f40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:45.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.110+0000 7f77f9ac9640 1 --2- 172.21.15.133:0/572211980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f4100e20 0x7f77f40fcc80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:45.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.110+0000 7f77f9ac9640 1 -- 172.21.15.133:0/572211980 >> 172.21.15.133:0/572211980 conn(0x7f77f40fc7d0 msgr2=0x7f77f406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:45.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.110+0000 7f77f9ac9640 1 -- 172.21.15.133:0/572211980 shutdown_connections 2024-09-11T23:34:45.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:45.110+0000 7f77f9ac9640 1 -- 172.21.15.133:0/572211980 wait complete. 2024-09-11T23:34:45.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:45.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:45.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:45.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:45 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:45.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:45 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/572211980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:45.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-11T23:34:45.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:45.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:45.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:45.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:45 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:45.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:45 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/572211980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:46.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:46.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:46.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:46.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:45 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:46.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:45 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/572211980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:34:46.040 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:46.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.457+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/2515629536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c58101430 msgr2=0x7f9c5810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:46.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.457+0000 7f9c5f2aa640 1 --2- 172.21.15.133:0/2515629536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c58101430 0x7f9c5810d960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009cd0 tx=0x7f9c5002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:34:46.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.457+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/2515629536 shutdown_connections 2024-09-11T23:34:46.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.457+0000 7f9c5f2aa640 1 --2- 172.21.15.133:0/2515629536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c58101430 0x7f9c5810d960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:46.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.457+0000 7f9c5f2aa640 1 --2- 172.21.15.133:0/2515629536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c58100a90 0x7f9c58100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:46.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.457+0000 7f9c5f2aa640 1 --2- 172.21.15.133:0/2515629536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c58106a90 0x7f9c58106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:46.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.457+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/2515629536 >> 172.21.15.133:0/2515629536 conn(0x7f9c580fc7d0 msgr2=0x7f9c580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:46.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.457+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/2515629536 shutdown_connections 2024-09-11T23:34:46.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.457+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/2515629536 wait complete. 2024-09-11T23:34:46.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5f2aa640 1 Processor -- start 2024-09-11T23:34:46.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5f2aa640 1 -- start start 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5f2aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c58100a90 0x7f9c58198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5f2aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c58101430 0x7f9c58198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5f2aa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c58106a90 0x7f9c5819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5f2aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58110e90 con 0x7f9c58100a90 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5f2aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58110d10 con 0x7f9c58101430 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5f2aa640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58111010 con 0x7f9c58106a90 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5d046640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c58100a90 0x7f9c58198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5d046640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c58100a90 0x7f9c58198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47358/0 (socket says 172.21.15.133:47358) 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5d046640 1 -- 172.21.15.133:0/1061173528 learned_addr learned my addr 172.21.15.133:0/1061173528 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5c845640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c58101430 0x7f9c58198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5d847640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c58106a90 0x7f9c5819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5d046640 1 -- 172.21.15.133:0/1061173528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c58106a90 msgr2=0x7f9c5819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:46.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5d046640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c58106a90 0x7f9c5819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:46.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5d046640 1 -- 172.21.15.133:0/1061173528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c58101430 msgr2=0x7f9c58198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:46.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5d046640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c58101430 0x7f9c58198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:46.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.458+0000 7f9c5d046640 1 -- 172.21.15.133:0/1061173528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5819d3c0 con 0x7f9c58100a90 2024-09-11T23:34:46.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.459+0000 7f9c5d046640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c58100a90 0x7f9c58198410 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400d950 tx=0x7f9c5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:46.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.459+0000 7f9c4e7fc640 1 -- 172.21.15.133:0/1061173528 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54017070 con 0x7f9c58100a90 2024-09-11T23:34:46.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.459+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/1061173528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5819d6b0 con 0x7f9c58100a90 2024-09-11T23:34:46.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.459+0000 7f9c4e7fc640 1 -- 172.21.15.133:0/1061173528 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c54009c30 con 0x7f9c58100a90 2024-09-11T23:34:46.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.459+0000 7f9c4e7fc640 1 -- 172.21.15.133:0/1061173528 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400f6b0 con 0x7f9c58100a90 2024-09-11T23:34:46.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.459+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/1061173528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c581a4f90 con 0x7f9c58100a90 2024-09-11T23:34:46.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.460+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/1061173528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c58100ef0 con 0x7f9c58100a90 2024-09-11T23:34:46.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.461+0000 7f9c4e7fc640 1 -- 172.21.15.133:0/1061173528 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9c540109d0 con 0x7f9c58100a90 2024-09-11T23:34:46.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.462+0000 7f9c4e7fc640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9c440a2b80 0x7f9c440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:46.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.462+0000 7f9c5c845640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9c440a2b80 0x7f9c440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:46.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.462+0000 7f9c4e7fc640 1 -- 172.21.15.133:0/1061173528 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f9c540ce6b0 con 0x7f9c58100a90 2024-09-11T23:34:46.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.463+0000 7f9c5c845640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9c440a2b80 0x7f9c440a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9c580ff1b0 tx=0x7f9c4800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:46.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.467+0000 7f9c4e7fc640 1 -- 172.21.15.133:0/1061173528 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c54009de0 con 0x7f9c58100a90 2024-09-11T23:34:46.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.646+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/1061173528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9c58109aa0 con 0x7f9c58100a90 2024-09-11T23:34:46.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.647+0000 7f9c4e7fc640 1 -- 172.21.15.133:0/1061173528 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7f9c5409b460 con 0x7f9c58100a90 2024-09-11T23:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-09-11T23:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-09-11T23:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-09-11T23:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:34:46.647 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-09-11T23:34:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/1061173528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9c440a2b80 msgr2=0x7f9c440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9c440a2b80 0x7f9c440a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9c580ff1b0 tx=0x7f9c4800b390 comp rx=0 tx=0).stop 2024-09-11T23:34:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/1061173528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c58100a90 msgr2=0x7f9c58198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c58100a90 0x7f9c58198410 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400d950 tx=0x7f9c5400de10 comp rx=0 tx=0).stop 2024-09-11T23:34:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/1061173528 shutdown_connections 2024-09-11T23:34:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9c440a2b80 0x7f9c440a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:46.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c58106a90 0x7f9c5819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:46.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c58101430 0x7f9c58198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:46.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 --2- 172.21.15.133:0/1061173528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c58100a90 0x7f9c58198410 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:46.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/1061173528 >> 172.21.15.133:0/1061173528 conn(0x7f9c580fc7d0 msgr2=0x7f9c5806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:46.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/1061173528 shutdown_connections 2024-09-11T23:34:46.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:46.649+0000 7f9c5f2aa640 1 -- 172.21.15.133:0/1061173528 wait complete. 2024-09-11T23:34:47.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:46 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:47.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:46 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:47.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:46 smithi138 ceph-mon[35142]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:47.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:46 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:47.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:46 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:47.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:46 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:47.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:46 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:47.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:46 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1061173528' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:34:47.157 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:47.157 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:47.157 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:46 smithi178 ceph-mon[35882]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:47.157 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:47.157 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:47.157 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:47.157 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:46 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:47.157 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:46 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1061173528' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:34:47.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:47.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:47.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:46 smithi133 ceph-mon[31430]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:47.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:47.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:47.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:47.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:46 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:34:47.162 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:46 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1061173528' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:34:47.479 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T23:34:47.480 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T23:34:48.195 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:48.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.718+0000 7f8bc42b0640 1 -- 172.21.15.133:0/2324009174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bbc100a80 msgr2=0x7f8bbc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:48.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.718+0000 7f8bc42b0640 1 --2- 172.21.15.133:0/2324009174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bbc100a80 0x7f8bbc100ee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009930 tx=0x7f8bac02a420 comp rx=0 tx=0).stop 2024-09-11T23:34:48.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.718+0000 7f8bc42b0640 1 -- 172.21.15.133:0/2324009174 shutdown_connections 2024-09-11T23:34:48.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.718+0000 7f8bc42b0640 1 --2- 172.21.15.133:0/2324009174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bbc101420 0x7f8bbc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:48.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.718+0000 7f8bc42b0640 1 --2- 172.21.15.133:0/2324009174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bbc100a80 0x7f8bbc100ee0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:48.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.718+0000 7f8bc42b0640 1 --2- 172.21.15.133:0/2324009174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bbc106a80 0x7f8bbc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:48.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.718+0000 7f8bc42b0640 1 -- 172.21.15.133:0/2324009174 >> 172.21.15.133:0/2324009174 conn(0x7f8bbc0fc7a0 msgr2=0x7f8bbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:48.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.718+0000 7f8bc42b0640 1 -- 172.21.15.133:0/2324009174 shutdown_connections 2024-09-11T23:34:48.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.718+0000 7f8bc42b0640 1 -- 172.21.15.133:0/2324009174 wait complete. 2024-09-11T23:34:48.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc42b0640 1 Processor -- start 2024-09-11T23:34:48.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc42b0640 1 -- start start 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc42b0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bbc100a80 0x7f8bbc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc42b0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bbc101420 0x7f8bbc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc42b0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bbc106a80 0x7f8bbc1a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc42b0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc110ec0 con 0x7f8bbc106a80 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc42b0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc110d40 con 0x7f8bbc101420 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc42b0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc111040 con 0x7f8bbc100a80 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc284d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bbc106a80 0x7f8bbc1a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc284d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bbc106a80 0x7f8bbc1a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47388/0 (socket says 172.21.15.133:47388) 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc284d640 1 -- 172.21.15.133:0/939882878 learned_addr learned my addr 172.21.15.133:0/939882878 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc204c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bbc100a80 0x7f8bbc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.719+0000 7f8bc184b640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bbc101420 0x7f8bbc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:48.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.720+0000 7f8bc284d640 1 -- 172.21.15.133:0/939882878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bbc100a80 msgr2=0x7f8bbc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.720+0000 7f8bc284d640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bbc100a80 0x7f8bbc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.720+0000 7f8bc284d640 1 -- 172.21.15.133:0/939882878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bbc101420 msgr2=0x7f8bbc199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.720+0000 7f8bc284d640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bbc101420 0x7f8bbc199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.720+0000 7f8bc284d640 1 -- 172.21.15.133:0/939882878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc1a4910 con 0x7f8bbc106a80 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.720+0000 7f8bc284d640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bbc106a80 0x7f8bbc1a43d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400b970 tx=0x7f8bb400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.720+0000 7f8bb37fe640 1 -- 172.21.15.133:0/939882878 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4019070 con 0x7f8bbc106a80 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.720+0000 7f8bc42b0640 1 -- 172.21.15.133:0/939882878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bbc1a4ae0 con 0x7f8bbc106a80 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.720+0000 7f8bb37fe640 1 -- 172.21.15.133:0/939882878 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb400cb40 con 0x7f8bbc106a80 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.720+0000 7f8bb37fe640 1 -- 172.21.15.133:0/939882878 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4007650 con 0x7f8bbc106a80 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.720+0000 7f8bc42b0640 1 -- 172.21.15.133:0/939882878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bbc1a4fa0 con 0x7f8bbc106a80 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.721+0000 7f8bc42b0640 1 -- 172.21.15.133:0/939882878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bbc100ee0 con 0x7f8bbc106a80 2024-09-11T23:34:48.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.722+0000 7f8bb37fe640 1 -- 172.21.15.133:0/939882878 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8bb401c070 con 0x7f8bbc106a80 2024-09-11T23:34:48.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.723+0000 7f8bb37fe640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8ba80a2b80 0x7f8ba80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:48.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.723+0000 7f8bb37fe640 1 -- 172.21.15.133:0/939882878 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f8bb40ce650 con 0x7f8bbc106a80 2024-09-11T23:34:48.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.723+0000 7f8bc204c640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8ba80a2b80 0x7f8ba80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:48.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.723+0000 7f8bc204c640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8ba80a2b80 0x7f8ba80a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005fd0 tx=0x7f8bb800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:48.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.726+0000 7f8bb37fe640 1 -- 172.21.15.133:0/939882878 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8bb409b400 con 0x7f8bbc106a80 2024-09-11T23:34:48.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:48.914+0000 7f8bc42b0640 1 -- 172.21.15.133:0/939882878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8bbc109010 con 0x7f8bbc106a80 2024-09-11T23:34:48.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:48.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:48.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:48.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:48.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:48 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:48.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:48 smithi133 ceph-mon[31430]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:48.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:48.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:48.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:48.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:48.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:48 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:48.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:48 smithi138 ceph-mon[35142]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:49.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:49.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:49.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:49.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:49.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:48 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:49.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:48 smithi178 ceph-mon[35882]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.833+0000 7f8bb37fe640 1 -- 172.21.15.133:0/939882878 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7f8bb409f700 con 0x7f8bbc106a80 2024-09-11T23:34:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-09-11T23:34:49.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 -- 172.21.15.133:0/939882878 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8ba80a2b80 msgr2=0x7f8ba80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:49.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8ba80a2b80 0x7f8ba80a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005fd0 tx=0x7f8bb800b390 comp rx=0 tx=0).stop 2024-09-11T23:34:49.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 -- 172.21.15.133:0/939882878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bbc106a80 msgr2=0x7f8bbc1a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:49.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bbc106a80 0x7f8bbc1a43d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400b970 tx=0x7f8bb400be30 comp rx=0 tx=0).stop 2024-09-11T23:34:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 -- 172.21.15.133:0/939882878 shutdown_connections 2024-09-11T23:34:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8ba80a2b80 0x7f8ba80a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bbc106a80 0x7f8bbc1a43d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bbc101420 0x7f8bbc199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 --2- 172.21.15.133:0/939882878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bbc100a80 0x7f8bbc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 -- 172.21.15.133:0/939882878 >> 172.21.15.133:0/939882878 conn(0x7f8bbc0fc7a0 msgr2=0x7f8bbc0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 -- 172.21.15.133:0/939882878 shutdown_connections 2024-09-11T23:34:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:49.836+0000 7f8bc42b0640 1 -- 172.21.15.133:0/939882878 wait complete. 2024-09-11T23:34:50.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:49 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/939882878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:34:50.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:50.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:50.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:50.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:50.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:49 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/939882878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:34:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:50.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:49 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/939882878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:34:50.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:50.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:50.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:34:50.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:34:50.270 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:34:50.542 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T23:34:50.789 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:51.096 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:50 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/939882878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:34:51.097 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:50 smithi133 ceph-mon[31430]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T23:34:51.097 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:50 smithi133 ceph-mon[31430]: pgmap v266: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:51.119 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:50 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/939882878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:34:51.119 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:50 smithi138 ceph-mon[35142]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T23:34:51.119 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:50 smithi138 ceph-mon[35142]: pgmap v266: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:50 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/939882878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:34:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:50 smithi178 ceph-mon[35882]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T23:34:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:50 smithi178 ceph-mon[35882]: pgmap v266: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:51.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.316+0000 7f8d462f9640 1 -- 172.21.15.133:0/3887914586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d40108300 msgr2=0x7f8d4010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:51.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.316+0000 7f8d462f9640 1 --2- 172.21.15.133:0/3887914586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d40108300 0x7f8d4010ec70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009cd0 tx=0x7f8d3802a8d0 comp rx=0 tx=0).stop 2024-09-11T23:34:51.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.316+0000 7f8d462f9640 1 -- 172.21.15.133:0/3887914586 shutdown_connections 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.316+0000 7f8d462f9640 1 --2- 172.21.15.133:0/3887914586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d40108300 0x7f8d4010ec70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.316+0000 7f8d462f9640 1 --2- 172.21.15.133:0/3887914586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d40103780 0x7f8d40107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.316+0000 7f8d462f9640 1 --2- 172.21.15.133:0/3887914586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40102dd0 0x7f8d401031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.316+0000 7f8d462f9640 1 -- 172.21.15.133:0/3887914586 >> 172.21.15.133:0/3887914586 conn(0x7f8d400fc7d0 msgr2=0x7f8d400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.316+0000 7f8d462f9640 1 -- 172.21.15.133:0/3887914586 shutdown_connections 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.316+0000 7f8d462f9640 1 -- 172.21.15.133:0/3887914586 wait complete. 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d462f9640 1 Processor -- start 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d462f9640 1 -- start start 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d462f9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d40102dd0 0x7f8d40100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d462f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d40103780 0x7f8d40100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d462f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40108300 0x7f8d401014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d462f9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401120d0 con 0x7f8d40103780 2024-09-11T23:34:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d462f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d40111f50 con 0x7f8d40108300 2024-09-11T23:34:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d462f9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d40112250 con 0x7f8d40102dd0 2024-09-11T23:34:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d3f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d40103780 0x7f8d40100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d3f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d40103780 0x7f8d40100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47400/0 (socket says 172.21.15.133:47400) 2024-09-11T23:34:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d3f7fe640 1 -- 172.21.15.133:0/1458991578 learned_addr learned my addr 172.21.15.133:0/1458991578 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d44896640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40108300 0x7f8d401014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.317+0000 7f8d3ffff640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d40102dd0 0x7f8d40100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.318+0000 7f8d3f7fe640 1 -- 172.21.15.133:0/1458991578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d40102dd0 msgr2=0x7f8d40100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.318+0000 7f8d3f7fe640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d40102dd0 0x7f8d40100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.318+0000 7f8d3f7fe640 1 -- 172.21.15.133:0/1458991578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40108300 msgr2=0x7f8d401014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.318+0000 7f8d3f7fe640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40108300 0x7f8d401014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.318+0000 7f8d3f7fe640 1 -- 172.21.15.133:0/1458991578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d401a6a30 con 0x7f8d40103780 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.318+0000 7f8d3f7fe640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d40103780 0x7f8d40100f70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d950 tx=0x7f8d3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.318+0000 7f8d3d7fa640 1 -- 172.21.15.133:0/1458991578 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34017070 con 0x7f8d40103780 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.318+0000 7f8d3d7fa640 1 -- 172.21.15.133:0/1458991578 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d34009c30 con 0x7f8d40103780 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.318+0000 7f8d3d7fa640 1 -- 172.21.15.133:0/1458991578 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340047a0 con 0x7f8d40103780 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.318+0000 7f8d462f9640 1 -- 172.21.15.133:0/1458991578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d401a6d20 con 0x7f8d40103780 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.318+0000 7f8d462f9640 1 -- 172.21.15.133:0/1458991578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d401a7200 con 0x7f8d40103780 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.319+0000 7f8d462f9640 1 -- 172.21.15.133:0/1458991578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d401031b0 con 0x7f8d40103780 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.320+0000 7f8d3d7fa640 1 -- 172.21.15.133:0/1458991578 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8d34004020 con 0x7f8d40103780 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.321+0000 7f8d3d7fa640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8d280a2b80 0x7f8d280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.321+0000 7f8d3ffff640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8d280a2b80 0x7f8d280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.321+0000 7f8d3d7fa640 1 -- 172.21.15.133:0/1458991578 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f8d340cec00 con 0x7f8d40103780 2024-09-11T23:34:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.321+0000 7f8d3ffff640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8d280a2b80 0x7f8d280a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000bbe0 tx=0x7f8d3000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:51.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.324+0000 7f8d3d7fa640 1 -- 172.21.15.133:0/1458991578 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d34020020 con 0x7f8d40103780 2024-09-11T23:34:51.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.491+0000 7f8d462f9640 1 -- 172.21.15.133:0/1458991578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8d4010a290 con 0x7f8d40103780 2024-09-11T23:34:51.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.845+0000 7f8d3d7fa640 1 -- 172.21.15.133:0/1458991578 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7f8d3409be90 con 0x7f8d40103780 2024-09-11T23:34:51.845 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-09-11T23:34:51.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.847+0000 7f8d462f9640 1 -- 172.21.15.133:0/1458991578 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8d280a2b80 msgr2=0x7f8d280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:51.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.847+0000 7f8d462f9640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8d280a2b80 0x7f8d280a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000bbe0 tx=0x7f8d3000b390 comp rx=0 tx=0).stop 2024-09-11T23:34:51.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.847+0000 7f8d462f9640 1 -- 172.21.15.133:0/1458991578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d40103780 msgr2=0x7f8d40100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:51.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.847+0000 7f8d462f9640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d40103780 0x7f8d40100f70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d950 tx=0x7f8d3400de10 comp rx=0 tx=0).stop 2024-09-11T23:34:51.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.848+0000 7f8d462f9640 1 -- 172.21.15.133:0/1458991578 shutdown_connections 2024-09-11T23:34:51.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.848+0000 7f8d462f9640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8d280a2b80 0x7f8d280a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:51.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.848+0000 7f8d462f9640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40108300 0x7f8d401014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:51.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.848+0000 7f8d462f9640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d40103780 0x7f8d40100f70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:51.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.848+0000 7f8d462f9640 1 --2- 172.21.15.133:0/1458991578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d40102dd0 0x7f8d40100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:51.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.848+0000 7f8d462f9640 1 -- 172.21.15.133:0/1458991578 >> 172.21.15.133:0/1458991578 conn(0x7f8d400fc7d0 msgr2=0x7f8d401063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:51.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.848+0000 7f8d462f9640 1 -- 172.21.15.133:0/1458991578 shutdown_connections 2024-09-11T23:34:51.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:51.848+0000 7f8d462f9640 1 -- 172.21.15.133:0/1458991578 wait complete. 2024-09-11T23:34:52.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:51 smithi138 ceph-mon[35142]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:34:52.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:51 smithi138 ceph-mon[35142]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T23:34:52.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:51 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1458991578' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:34:52.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:51 smithi133 ceph-mon[31430]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:34:52.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:51 smithi133 ceph-mon[31430]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T23:34:52.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:51 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1458991578' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:34:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:51 smithi178 ceph-mon[35882]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:34:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:51 smithi178 ceph-mon[35882]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T23:34:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:51 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1458991578' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:34:52.545 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T23:34:52.790 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:53.163 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:52 smithi133 ceph-mon[31430]: pgmap v268: 33 pgs: 5 active+clean, 1 creating+activating, 14 creating+peering, 13 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:53.163 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:52 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1458991578' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:34:53.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:52 smithi133 ceph-mon[31430]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:34:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:52 smithi138 ceph-mon[35142]: pgmap v268: 33 pgs: 5 active+clean, 1 creating+activating, 14 creating+peering, 13 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:52 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1458991578' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:34:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:52 smithi138 ceph-mon[35142]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:34:53.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.245+0000 7f96aea5e640 1 -- 172.21.15.133:0/2588016779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a8101df0 msgr2=0x7f96a81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:53.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.245+0000 7f96aea5e640 1 --2- 172.21.15.133:0/2588016779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a8101df0 0x7f96a81021d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9698009900 tx=0x7f969802a410 comp rx=0 tx=0).stop 2024-09-11T23:34:53.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.245+0000 7f96aea5e640 1 -- 172.21.15.133:0/2588016779 shutdown_connections 2024-09-11T23:34:53.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.245+0000 7f96aea5e640 1 --2- 172.21.15.133:0/2588016779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a8106f20 0x7f96a810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:53.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.245+0000 7f96aea5e640 1 --2- 172.21.15.133:0/2588016779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a81027a0 0x7f96a81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:53.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.245+0000 7f96aea5e640 1 --2- 172.21.15.133:0/2588016779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a8101df0 0x7f96a81021d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:53.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.245+0000 7f96aea5e640 1 -- 172.21.15.133:0/2588016779 >> 172.21.15.133:0/2588016779 conn(0x7f96a80fd8b0 msgr2=0x7f96a80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:53.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.245+0000 7f96aea5e640 1 -- 172.21.15.133:0/2588016779 shutdown_connections 2024-09-11T23:34:53.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.245+0000 7f96aea5e640 1 -- 172.21.15.133:0/2588016779 wait complete. 2024-09-11T23:34:53.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.246+0000 7f96aea5e640 1 Processor -- start 2024-09-11T23:34:53.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.246+0000 7f96aea5e640 1 -- start start 2024-09-11T23:34:53.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.246+0000 7f96aea5e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a8101df0 0x7f96a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.246+0000 7f96aea5e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a81027a0 0x7f96a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.246+0000 7f96aea5e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a8106f20 0x7f96a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.246+0000 7f96aea5e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8112020 con 0x7f96a8106f20 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.246+0000 7f96aea5e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8111ea0 con 0x7f96a8101df0 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.246+0000 7f96aea5e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96a81121a0 con 0x7f96a81027a0 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96acffb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a8106f20 0x7f96a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96acffb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a8106f20 0x7f96a8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35066/0 (socket says 172.21.15.133:35066) 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96acffb640 1 -- 172.21.15.133:0/1880403560 learned_addr learned my addr 172.21.15.133:0/1880403560 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96a7fff640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a8101df0 0x7f96a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96a77fe640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a81027a0 0x7f96a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96acffb640 1 -- 172.21.15.133:0/1880403560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a81027a0 msgr2=0x7f96a8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96acffb640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a81027a0 0x7f96a8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96acffb640 1 -- 172.21.15.133:0/1880403560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a8101df0 msgr2=0x7f96a80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:53.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96acffb640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a8101df0 0x7f96a80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96acffb640 1 -- 172.21.15.133:0/1880403560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a81a6a60 con 0x7f96a8106f20 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96acffb640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a8106f20 0x7f96a8074f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96a000b970 tx=0x7f96a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96a57fa640 1 -- 172.21.15.133:0/1880403560 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0019070 con 0x7f96a8106f20 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96aea5e640 1 -- 172.21.15.133:0/1880403560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a81a6d50 con 0x7f96a8106f20 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96a57fa640 1 -- 172.21.15.133:0/1880403560 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a0014440 con 0x7f96a8106f20 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96a57fa640 1 -- 172.21.15.133:0/1880403560 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a00075d0 con 0x7f96a8106f20 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.247+0000 7f96aea5e640 1 -- 172.21.15.133:0/1880403560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a81a7240 con 0x7f96a8106f20 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.248+0000 7f96aea5e640 1 -- 172.21.15.133:0/1880403560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a8102250 con 0x7f96a8106f20 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.250+0000 7f96a57fa640 1 -- 172.21.15.133:0/1880403560 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f96a001c070 con 0x7f96a8106f20 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.250+0000 7f96a57fa640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f96900a2b80 0x7f96900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.250+0000 7f96a7fff640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f96900a2b80 0x7f96900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.250+0000 7f96a57fa640 1 -- 172.21.15.133:0/1880403560 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f96a00ce8b0 con 0x7f96a8106f20 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.251+0000 7f96a7fff640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f96900a2b80 0x7f96900a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9698000c00 tx=0x7f96980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:53.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.254+0000 7f96a57fa640 1 -- 172.21.15.133:0/1880403560 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96a009b370 con 0x7f96a8106f20 2024-09-11T23:34:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:52 smithi178 ceph-mon[35882]: pgmap v268: 33 pgs: 5 active+clean, 1 creating+activating, 14 creating+peering, 13 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:52 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1458991578' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:34:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:52 smithi178 ceph-mon[35882]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:34:53.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:53.461+0000 7f96aea5e640 1 -- 172.21.15.133:0/1880403560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f96a8109b70 con 0x7f96a8106f20 2024-09-11T23:34:54.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:53 smithi138 ceph-mon[35142]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:34:54.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:53 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1880403560' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:34:54.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:53 smithi138 ceph-mon[35142]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-11T23:34:54.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:53 smithi133 ceph-mon[31430]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:34:54.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:53 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1880403560' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:34:54.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:53 smithi133 ceph-mon[31430]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-11T23:34:54.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:53 smithi178 ceph-mon[35882]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:34:54.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:53 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1880403560' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:34:54.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:53 smithi178 ceph-mon[35882]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-11T23:34:54.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.439+0000 7f96a57fa640 1 -- 172.21.15.133:0/1880403560 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f96a009f670 con 0x7f96a8106f20 2024-09-11T23:34:54.439 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T23:34:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 -- 172.21.15.133:0/1880403560 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f96900a2b80 msgr2=0x7f96900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f96900a2b80 0x7f96900a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9698000c00 tx=0x7f96980058d0 comp rx=0 tx=0).stop 2024-09-11T23:34:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 -- 172.21.15.133:0/1880403560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a8106f20 msgr2=0x7f96a8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a8106f20 0x7f96a8074f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96a000b970 tx=0x7f96a000bd70 comp rx=0 tx=0).stop 2024-09-11T23:34:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 -- 172.21.15.133:0/1880403560 shutdown_connections 2024-09-11T23:34:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f96900a2b80 0x7f96900a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a8106f20 0x7f96a8074f70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a81027a0 0x7f96a8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 --2- 172.21.15.133:0/1880403560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a8101df0 0x7f96a80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 -- 172.21.15.133:0/1880403560 >> 172.21.15.133:0/1880403560 conn(0x7f96a80fd8b0 msgr2=0x7f96a80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 -- 172.21.15.133:0/1880403560 shutdown_connections 2024-09-11T23:34:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:54.443+0000 7f96aea5e640 1 -- 172.21.15.133:0/1880403560 wait complete. 2024-09-11T23:34:55.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:54 smithi133 ceph-mon[31430]: pgmap v271: 97 pgs: 15 active+clean, 1 creating+activating, 40 creating+peering, 41 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:55.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:54 smithi133 ceph-mon[31430]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:34:55.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:54 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1880403560' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:34:55.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:54 smithi133 ceph-mon[31430]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-11T23:34:55.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:54 smithi133 ceph-mon[31430]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-11T23:34:55.166 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:34:55.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:54 smithi138 ceph-mon[35142]: pgmap v271: 97 pgs: 15 active+clean, 1 creating+activating, 40 creating+peering, 41 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:55.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:54 smithi138 ceph-mon[35142]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:34:55.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:54 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1880403560' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:34:55.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:54 smithi138 ceph-mon[35142]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-11T23:34:55.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:54 smithi138 ceph-mon[35142]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-11T23:34:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:54 smithi178 ceph-mon[35882]: pgmap v271: 97 pgs: 15 active+clean, 1 creating+activating, 40 creating+peering, 41 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:34:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:54 smithi178 ceph-mon[35882]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:34:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:54 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1880403560' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:34:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:54 smithi178 ceph-mon[35882]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-11T23:34:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:54 smithi178 ceph-mon[35882]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-11T23:34:55.413 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:55.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.876+0000 7f3cba131640 1 -- 172.21.15.133:0/4050252202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cb4102de0 msgr2=0x7f3cb41031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.876+0000 7f3cba131640 1 --2- 172.21.15.133:0/4050252202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cb4102de0 0x7f3cb41031c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009ca0 tx=0x7f3ca402a950 comp rx=0 tx=0).stop 2024-09-11T23:34:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.876+0000 7f3cba131640 1 -- 172.21.15.133:0/4050252202 shutdown_connections 2024-09-11T23:34:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.876+0000 7f3cba131640 1 --2- 172.21.15.133:0/4050252202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb410c180 0x7f3cb410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.876+0000 7f3cba131640 1 --2- 172.21.15.133:0/4050252202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cb4103700 0x7f3cb410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.876+0000 7f3cba131640 1 --2- 172.21.15.133:0/4050252202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cb4102de0 0x7f3cb41031c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.876+0000 7f3cba131640 1 -- 172.21.15.133:0/4050252202 >> 172.21.15.133:0/4050252202 conn(0x7f3cb40fc7a0 msgr2=0x7f3cb40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.876+0000 7f3cba131640 1 -- 172.21.15.133:0/4050252202 shutdown_connections 2024-09-11T23:34:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.876+0000 7f3cba131640 1 -- 172.21.15.133:0/4050252202 wait complete. 2024-09-11T23:34:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cba131640 1 Processor -- start 2024-09-11T23:34:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cba131640 1 -- start start 2024-09-11T23:34:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cba131640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cb4102de0 0x7f3cb41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cba131640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb4103700 0x7f3cb4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cba131640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cb410c180 0x7f3cb419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cba131640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4111f30 con 0x7f3cb410c180 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cba131640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4111db0 con 0x7f3cb4102de0 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cba131640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb41120b0 con 0x7f3cb4103700 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cb3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cb410c180 0x7f3cb419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cb3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cb410c180 0x7f3cb419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35086/0 (socket says 172.21.15.133:35086) 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cb3fff640 1 -- 172.21.15.133:0/4286568354 learned_addr learned my addr 172.21.15.133:0/4286568354 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cb2ffd640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb4103700 0x7f3cb4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.877+0000 7f3cb37fe640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cb4102de0 0x7f3cb41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.878+0000 7f3cb3fff640 1 -- 172.21.15.133:0/4286568354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb4103700 msgr2=0x7f3cb4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.878+0000 7f3cb3fff640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb4103700 0x7f3cb4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.878+0000 7f3cb3fff640 1 -- 172.21.15.133:0/4286568354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cb4102de0 msgr2=0x7f3cb41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.878+0000 7f3cb3fff640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cb4102de0 0x7f3cb41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.878+0000 7f3cb3fff640 1 -- 172.21.15.133:0/4286568354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb419d370 con 0x7f3cb410c180 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.878+0000 7f3cb3fff640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cb410c180 0x7f3cb419cc90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00c970 tx=0x7f3cac00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.878+0000 7f3cb0ff9640 1 -- 172.21.15.133:0/4286568354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00ae60 con 0x7f3cb410c180 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.878+0000 7f3cb0ff9640 1 -- 172.21.15.133:0/4286568354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cac005500 con 0x7f3cb410c180 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.878+0000 7f3cb0ff9640 1 -- 172.21.15.133:0/4286568354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00f450 con 0x7f3cb410c180 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.878+0000 7f3cba131640 1 -- 172.21.15.133:0/4286568354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb419d660 con 0x7f3cb410c180 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.878+0000 7f3cba131640 1 -- 172.21.15.133:0/4286568354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb41a4f40 con 0x7f3cb410c180 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.879+0000 7f3cba131640 1 -- 172.21.15.133:0/4286568354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb41031c0 con 0x7f3cb410c180 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.880+0000 7f3cb0ff9640 1 -- 172.21.15.133:0/4286568354 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3cac004ee0 con 0x7f3cb410c180 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.881+0000 7f3cb0ff9640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3c9c0a2c00 0x7f3c9c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.881+0000 7f3cb37fe640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3c9c0a2c00 0x7f3c9c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.881+0000 7f3cb0ff9640 1 -- 172.21.15.133:0/4286568354 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f3cac0ce9e0 con 0x7f3cb410c180 2024-09-11T23:34:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.881+0000 7f3cb37fe640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3c9c0a2c00 0x7f3c9c0a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40093d0 tx=0x7f3ca4009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:55.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:55.885+0000 7f3cb0ff9640 1 -- 172.21.15.133:0/4286568354 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3cac014030 con 0x7f3cb410c180 2024-09-11T23:34:56.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.054+0000 7f3cba131640 1 -- 172.21.15.133:0/4286568354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cb4106cd0 con 0x7f3cb410c180 2024-09-11T23:34:56.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.056+0000 7f3cb0ff9640 1 -- 172.21.15.133:0/4286568354 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22336 (secure 0 0 0) 0x7f3cac09ba90 con 0x7f3cb410c180 2024-09-11T23:34:56.056 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:56.056 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:34:53.462686+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:34:56.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.058+0000 7f3cba131640 1 -- 172.21.15.133:0/4286568354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3c9c0a2c00 msgr2=0x7f3c9c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:56.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.058+0000 7f3cba131640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3c9c0a2c00 0x7f3c9c0a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40093d0 tx=0x7f3ca4009360 comp rx=0 tx=0).stop 2024-09-11T23:34:56.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.058+0000 7f3cba131640 1 -- 172.21.15.133:0/4286568354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cb410c180 msgr2=0x7f3cb419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:56.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.058+0000 7f3cba131640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cb410c180 0x7f3cb419cc90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00c970 tx=0x7f3cac00cd70 comp rx=0 tx=0).stop 2024-09-11T23:34:56.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.058+0000 7f3cba131640 1 -- 172.21.15.133:0/4286568354 shutdown_connections 2024-09-11T23:34:56.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.058+0000 7f3cba131640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3c9c0a2c00 0x7f3c9c0a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:56.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.058+0000 7f3cba131640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cb410c180 0x7f3cb419cc90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:56.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.058+0000 7f3cba131640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb4103700 0x7f3cb4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:56.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.058+0000 7f3cba131640 1 --2- 172.21.15.133:0/4286568354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cb4102de0 0x7f3cb41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:56.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.058+0000 7f3cba131640 1 -- 172.21.15.133:0/4286568354 >> 172.21.15.133:0/4286568354 conn(0x7f3cb40fc7a0 msgr2=0x7f3cb40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:56.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.059+0000 7f3cba131640 1 -- 172.21.15.133:0/4286568354 shutdown_connections 2024-09-11T23:34:56.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:56.059+0000 7f3cba131640 1 -- 172.21.15.133:0/4286568354 wait complete. 2024-09-11T23:34:56.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:56 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] up:active 2024-09-11T23:34:56.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:56 smithi133 ceph-mon[31430]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:34:56.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:56 smithi133 ceph-mon[31430]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:34:56.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:56 smithi133 ceph-mon[31430]: pgmap v273: 97 pgs: 54 active+clean, 1 creating+activating, 26 creating+peering, 16 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s wr, 5 op/s 2024-09-11T23:34:56.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:56 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/4286568354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:34:56.695 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:34:56.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:56 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] up:active 2024-09-11T23:34:56.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:56 smithi138 ceph-mon[35142]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:34:56.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:56 smithi138 ceph-mon[35142]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:34:56.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:56 smithi138 ceph-mon[35142]: pgmap v273: 97 pgs: 54 active+clean, 1 creating+activating, 26 creating+peering, 16 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s wr, 5 op/s 2024-09-11T23:34:56.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:56 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/4286568354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:34:56.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:56 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] up:active 2024-09-11T23:34:56.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:56 smithi178 ceph-mon[35882]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:34:56.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:56 smithi178 ceph-mon[35882]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:34:56.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:56 smithi178 ceph-mon[35882]: pgmap v273: 97 pgs: 54 active+clean, 1 creating+activating, 26 creating+peering, 16 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s wr, 5 op/s 2024-09-11T23:34:56.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:56 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/4286568354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:34:56.943 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:57.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.349+0000 7f39cc87c640 1 -- 172.21.15.133:0/2994104996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39c8102570 msgr2=0x7f39c81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:57.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.349+0000 7f39cc87c640 1 --2- 172.21.15.133:0/2994104996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39c8102570 0x7f39c81029b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009900 tx=0x7f39b002a480 comp rx=0 tx=0).stop 2024-09-11T23:34:57.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.349+0000 7f39cc87c640 1 -- 172.21.15.133:0/2994104996 shutdown_connections 2024-09-11T23:34:57.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.349+0000 7f39cc87c640 1 --2- 172.21.15.133:0/2994104996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39c8102ef0 0x7f39c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.349+0000 7f39cc87c640 1 --2- 172.21.15.133:0/2994104996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39c8102570 0x7f39c81029b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.349+0000 7f39cc87c640 1 --2- 172.21.15.133:0/2994104996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c8100e20 0x7f39c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.349+0000 7f39cc87c640 1 -- 172.21.15.133:0/2994104996 >> 172.21.15.133:0/2994104996 conn(0x7f39c80fc7d0 msgr2=0x7f39c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:57.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.349+0000 7f39cc87c640 1 -- 172.21.15.133:0/2994104996 shutdown_connections 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.349+0000 7f39cc87c640 1 -- 172.21.15.133:0/2994104996 wait complete. 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39cc87c640 1 Processor -- start 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39cc87c640 1 -- start start 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39cc87c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c8100e20 0x7f39c819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39cc87c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39c8102570 0x7f39c819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39cc87c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39c8102ef0 0x7f39c819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39cc87c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81120a0 con 0x7f39c8102570 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39cc87c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8111f20 con 0x7f39c8100e20 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39cc87c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8112220 con 0x7f39c8102ef0 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39c5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39c8102570 0x7f39c819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39c5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39c8102570 0x7f39c819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35110/0 (socket says 172.21.15.133:35110) 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39c5d9b640 1 -- 172.21.15.133:0/308155526 learned_addr learned my addr 172.21.15.133:0/308155526 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39c6d9d640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39c8102ef0 0x7f39c819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39c659c640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c8100e20 0x7f39c819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39c5d9b640 1 -- 172.21.15.133:0/308155526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39c8102ef0 msgr2=0x7f39c819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39c5d9b640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39c8102ef0 0x7f39c819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39c5d9b640 1 -- 172.21.15.133:0/308155526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c8100e20 msgr2=0x7f39c819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39c5d9b640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c8100e20 0x7f39c819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39c5d9b640 1 -- 172.21.15.133:0/308155526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c819f6f0 con 0x7f39c8102570 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.350+0000 7f39c5d9b640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39c8102570 0x7f39c819ac80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f39b0006010 tx=0x7f39b0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.351+0000 7f39bf7fe640 1 -- 172.21.15.133:0/308155526 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b003f070 con 0x7f39c8102570 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.351+0000 7f39cc87c640 1 -- 172.21.15.133:0/308155526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c819f980 con 0x7f39c8102570 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.351+0000 7f39bf7fe640 1 -- 172.21.15.133:0/308155526 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b0031e30 con 0x7f39c8102570 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.351+0000 7f39bf7fe640 1 -- 172.21.15.133:0/308155526 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0035030 con 0x7f39c8102570 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.351+0000 7f39cc87c640 1 -- 172.21.15.133:0/308155526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c81a72c0 con 0x7f39c8102570 2024-09-11T23:34:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.352+0000 7f39cc87c640 1 -- 172.21.15.133:0/308155526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c8101200 con 0x7f39c8102570 2024-09-11T23:34:57.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.354+0000 7f39bf7fe640 1 -- 172.21.15.133:0/308155526 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f39b0042070 con 0x7f39c8102570 2024-09-11T23:34:57.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.354+0000 7f39bf7fe640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39b40a2b80 0x7f39b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:57.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.355+0000 7f39c659c640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39b40a2b80 0x7f39b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:57.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.355+0000 7f39bf7fe640 1 -- 172.21.15.133:0/308155526 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f39b00efee0 con 0x7f39c8102570 2024-09-11T23:34:57.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.355+0000 7f39c659c640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39b40a2b80 0x7f39b40a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009f20 tx=0x7f39c00094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:57.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.357+0000 7f39bf7fe640 1 -- 172.21.15.133:0/308155526 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39b00bd020 con 0x7f39c8102570 2024-09-11T23:34:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.525+0000 7f39cc87c640 1 -- 172.21.15.133:0/308155526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39c810d4a0 con 0x7f39c8102570 2024-09-11T23:34:57.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.526+0000 7f39bf7fe640 1 -- 172.21.15.133:0/308155526 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22336 (secure 0 0 0) 0x7f39b00bc920 con 0x7f39c8102570 2024-09-11T23:34:57.526 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:34:57.526 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:34:53.462686+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:34:57.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 -- 172.21.15.133:0/308155526 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39b40a2b80 msgr2=0x7f39b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:57.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39b40a2b80 0x7f39b40a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009f20 tx=0x7f39c00094f0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 -- 172.21.15.133:0/308155526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39c8102570 msgr2=0x7f39c819ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39c8102570 0x7f39c819ac80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f39b0006010 tx=0x7f39b0003c80 comp rx=0 tx=0).stop 2024-09-11T23:34:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 -- 172.21.15.133:0/308155526 shutdown_connections 2024-09-11T23:34:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39b40a2b80 0x7f39b40a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39c8102ef0 0x7f39c819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39c8102570 0x7f39c819ac80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 --2- 172.21.15.133:0/308155526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c8100e20 0x7f39c819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 -- 172.21.15.133:0/308155526 >> 172.21.15.133:0/308155526 conn(0x7f39c80fc7d0 msgr2=0x7f39c8105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 -- 172.21.15.133:0/308155526 shutdown_connections 2024-09-11T23:34:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:57.529+0000 7f39cc87c640 1 -- 172.21.15.133:0/308155526 wait complete. 2024-09-11T23:34:57.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:57 smithi138 ceph-mon[35142]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:34:57.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:57 smithi138 ceph-mon[35142]: Cluster is now healthy 2024-09-11T23:34:57.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:57 smithi133 ceph-mon[31430]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:34:57.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:57 smithi133 ceph-mon[31430]: Cluster is now healthy 2024-09-11T23:34:57.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:57 smithi178 ceph-mon[35882]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:34:57.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:57 smithi178 ceph-mon[35882]: Cluster is now healthy 2024-09-11T23:34:58.194 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-11T23:34:58.441 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:34:58.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:58 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/308155526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:34:58.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:58 smithi133 ceph-mon[31430]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-11T23:34:58.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:58 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/308155526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:34:58.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:58 smithi178 ceph-mon[35882]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-11T23:34:58.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.893+0000 7f397dfa0640 1 -- 172.21.15.133:0/952894251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3978106f20 msgr2=0x7f397810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:58.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.893+0000 7f397dfa0640 1 --2- 172.21.15.133:0/952894251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3978106f20 0x7f397810eaa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3970009ca0 tx=0x7f397002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.893+0000 7f397dfa0640 1 -- 172.21.15.133:0/952894251 shutdown_connections 2024-09-11T23:34:58.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.893+0000 7f397dfa0640 1 --2- 172.21.15.133:0/952894251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3978106f20 0x7f397810eaa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.893+0000 7f397dfa0640 1 --2- 172.21.15.133:0/952894251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39781027a0 0x7f39781067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.893+0000 7f397dfa0640 1 --2- 172.21.15.133:0/952894251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978101df0 0x7f39781021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.893+0000 7f397dfa0640 1 -- 172.21.15.133:0/952894251 >> 172.21.15.133:0/952894251 conn(0x7f39780fd8f0 msgr2=0x7f39780ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.893+0000 7f397dfa0640 1 -- 172.21.15.133:0/952894251 shutdown_connections 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.893+0000 7f397dfa0640 1 -- 172.21.15.133:0/952894251 wait complete. 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.893+0000 7f397dfa0640 1 Processor -- start 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.893+0000 7f397dfa0640 1 -- start start 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f397dfa0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978101df0 0x7f3978198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f397dfa0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39781027a0 0x7f39781989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f397dfa0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3978106f20 0x7f397819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f397dfa0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3978111fa0 con 0x7f39781027a0 2024-09-11T23:34:58.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f397dfa0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3978111e20 con 0x7f3978106f20 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f397dfa0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3978112120 con 0x7f3978101df0 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f396ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39781027a0 0x7f39781989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f396ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39781027a0 0x7f39781989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35128/0 (socket says 172.21.15.133:35128) 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f396ffff640 1 -- 172.21.15.133:0/71810177 learned_addr learned my addr 172.21.15.133:0/71810177 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f39777fe640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978101df0 0x7f3978198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f3977fff640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3978106f20 0x7f397819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f396ffff640 1 -- 172.21.15.133:0/71810177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978101df0 msgr2=0x7f3978198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f396ffff640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978101df0 0x7f3978198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f396ffff640 1 -- 172.21.15.133:0/71810177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3978106f20 msgr2=0x7f397819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f396ffff640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3978106f20 0x7f397819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f396ffff640 1 -- 172.21.15.133:0/71810177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397819d410 con 0x7f39781027a0 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f39777fe640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978101df0 0x7f3978198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:34:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f396ffff640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39781027a0 0x7f39781989a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f396000d950 tx=0x7f396000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f39757fa640 1 -- 172.21.15.133:0/71810177 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960017070 con 0x7f39781027a0 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f39757fa640 1 -- 172.21.15.133:0/71810177 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3960009c30 con 0x7f39781027a0 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f397dfa0640 1 -- 172.21.15.133:0/71810177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397819d700 con 0x7f39781027a0 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f39757fa640 1 -- 172.21.15.133:0/71810177 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396000f6b0 con 0x7f39781027a0 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.894+0000 7f397dfa0640 1 -- 172.21.15.133:0/71810177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39781a4fe0 con 0x7f39781027a0 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.895+0000 7f397dfa0640 1 -- 172.21.15.133:0/71810177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39781021d0 con 0x7f39781027a0 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.896+0000 7f39757fa640 1 -- 172.21.15.133:0/71810177 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f39600109d0 con 0x7f39781027a0 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.897+0000 7f39757fa640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39640a2c00 0x7f39640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.897+0000 7f39777fe640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39640a2c00 0x7f39640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.897+0000 7f39757fa640 1 -- 172.21.15.133:0/71810177 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f39600ceb90 con 0x7f39781027a0 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.897+0000 7f39777fe640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39640a2c00 0x7f39640a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3968009930 tx=0x7f3968008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:34:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:58.901+0000 7f39757fa640 1 -- 172.21.15.133:0/71810177 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3960009de0 con 0x7f39781027a0 2024-09-11T23:34:58.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:58 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/308155526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:34:58.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:58 smithi138 ceph-mon[35142]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-11T23:34:59.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.103+0000 7f397dfa0640 1 -- 172.21.15.133:0/71810177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f397802d7c0 con 0x7f39781027a0 2024-09-11T23:34:59.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.453+0000 7f39757fa640 1 -- 172.21.15.133:0/71810177 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f396009bc40 con 0x7f39781027a0 2024-09-11T23:34:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 -- 172.21.15.133:0/71810177 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39640a2c00 msgr2=0x7f39640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39640a2c00 0x7f39640a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3968009930 tx=0x7f3968008040 comp rx=0 tx=0).stop 2024-09-11T23:34:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 -- 172.21.15.133:0/71810177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39781027a0 msgr2=0x7f39781989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:34:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39781027a0 0x7f39781989a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f396000d950 tx=0x7f396000de10 comp rx=0 tx=0).stop 2024-09-11T23:34:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 -- 172.21.15.133:0/71810177 shutdown_connections 2024-09-11T23:34:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f39640a2c00 0x7f39640a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3978106f20 0x7f397819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39781027a0 0x7f39781989a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 --2- 172.21.15.133:0/71810177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978101df0 0x7f3978198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:34:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 -- 172.21.15.133:0/71810177 >> 172.21.15.133:0/71810177 conn(0x7f39780fd8f0 msgr2=0x7f39780fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:34:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 -- 172.21.15.133:0/71810177 shutdown_connections 2024-09-11T23:34:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:34:59.456+0000 7f397dfa0640 1 -- 172.21.15.133:0/71810177 wait complete. 2024-09-11T23:34:59.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:34:59 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/71810177' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:34:59.781 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:34:59 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/71810177' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:34:59.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:34:59 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/71810177' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:35:00.040 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:34:59 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i[66069]: 2024-09-11T23:34:59.914+0000 7fa988a4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T23:35:00.191 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-11T23:35:00.191 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-11T23:35:00.442 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:00.724 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:00 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/71810177' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:35:00.724 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:00 smithi133 ceph-mon[31430]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:35:00.725 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:00 smithi133 ceph-mon[31430]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T23:35:00.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:00 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/71810177' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:35:00.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:00 smithi178 ceph-mon[35882]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:35:00.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:00 smithi178 ceph-mon[35882]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T23:35:00.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.901+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/921372411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b34106080 msgr2=0x7f6b341125d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:00.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.901+0000 7f6b3c1b1640 1 --2- 172.21.15.133:0/921372411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b34106080 0x7f6b341125d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009900 tx=0x7f6b2402a480 comp rx=0 tx=0).stop 2024-09-11T23:35:00.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.901+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/921372411 shutdown_connections 2024-09-11T23:35:00.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.901+0000 7f6b3c1b1640 1 --2- 172.21.15.133:0/921372411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b34112b10 0x7f6b34114f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.901+0000 7f6b3c1b1640 1 --2- 172.21.15.133:0/921372411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b34106080 0x7f6b341125d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.901+0000 7f6b3c1b1640 1 --2- 172.21.15.133:0/921372411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b34105760 0x7f6b34105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.901+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/921372411 >> 172.21.15.133:0/921372411 conn(0x7f6b34077210 msgr2=0x7f6b34077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.901+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/921372411 shutdown_connections 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.901+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/921372411 wait complete. 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3c1b1640 1 Processor -- start 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3c1b1640 1 -- start start 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3c1b1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b34105760 0x7f6b341a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3c1b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b34106080 0x7f6b341a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3c1b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b34112b10 0x7f6b3419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3c1b1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6b341185a0 con 0x7f6b34106080 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3c1b1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34118420 con 0x7f6b34112b10 2024-09-11T23:35:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3c1b1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34118720 con 0x7f6b34105760 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3974c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b34106080 0x7f6b341a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3974c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b34106080 0x7f6b341a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35150/0 (socket says 172.21.15.133:35150) 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3974c640 1 -- 172.21.15.133:0/2665788747 learned_addr learned my addr 172.21.15.133:0/2665788747 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3a74e640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b34112b10 0x7f6b3419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b39f4d640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b34105760 0x7f6b341a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3974c640 1 -- 172.21.15.133:0/2665788747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b34105760 msgr2=0x7f6b341a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3974c640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b34105760 0x7f6b341a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3974c640 1 -- 172.21.15.133:0/2665788747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b34112b10 msgr2=0x7f6b3419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.902+0000 7f6b3974c640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b34112b10 0x7f6b3419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.903+0000 7f6b3974c640 1 -- 172.21.15.133:0/2665788747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3419f000 con 0x7f6b34106080 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.903+0000 7f6b3974c640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b34106080 0x7f6b341a4c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6b24005ab0 tx=0x7f6b24004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.903+0000 7f6b2affd640 1 -- 172.21.15.133:0/2665788747 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2403f030 con 0x7f6b34106080 2024-09-11T23:35:00.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.903+0000 7f6b2affd640 1 -- 172.21.15.133:0/2665788747 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b24003ce0 con 0x7f6b34106080 2024-09-11T23:35:00.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.903+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/2665788747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3419f290 con 0x7f6b34106080 2024-09-11T23:35:00.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.903+0000 7f6b2affd640 1 -- 172.21.15.133:0/2665788747 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24035070 con 0x7f6b34106080 2024-09-11T23:35:00.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.903+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/2665788747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b341ab5e0 con 0x7f6b34106080 2024-09-11T23:35:00.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.904+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/2665788747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b34105b40 con 0x7f6b34106080 2024-09-11T23:35:00.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.905+0000 7f6b2affd640 1 -- 172.21.15.133:0/2665788747 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6b24042030 con 0x7f6b34106080 2024-09-11T23:35:00.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.906+0000 7f6b2affd640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6b200a2b80 0x7f6b200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:00.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.906+0000 7f6b39f4d640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6b200a2b80 0x7f6b200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:00.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.906+0000 7f6b2affd640 1 -- 172.21.15.133:0/2665788747 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f6b240ef3e0 con 0x7f6b34106080 2024-09-11T23:35:00.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.906+0000 7f6b39f4d640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6b200a2b80 0x7f6b200a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6b300058c0 tx=0x7f6b3000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:00.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:00.910+0000 7f6b2affd640 1 -- 172.21.15.133:0/2665788747 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b240bbe40 con 0x7f6b34106080 2024-09-11T23:35:00.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:00 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/71810177' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:35:00.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:00 smithi138 ceph-mon[35142]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:35:00.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:00 smithi138 ceph-mon[35142]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T23:35:01.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.111+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/2665788747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f6b341ab7b0 con 0x7f6b34106080 2024-09-11T23:35:01.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.461+0000 7f6b2affd640 1 -- 172.21.15.133:0/2665788747 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f6b240c0140 con 0x7f6b34106080 2024-09-11T23:35:01.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.463+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/2665788747 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6b200a2b80 msgr2=0x7f6b200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:01.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.463+0000 7f6b3c1b1640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6b200a2b80 0x7f6b200a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6b300058c0 tx=0x7f6b3000a2d0 comp rx=0 tx=0).stop 2024-09-11T23:35:01.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.463+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/2665788747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b34106080 msgr2=0x7f6b341a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:01.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.463+0000 7f6b3c1b1640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b34106080 0x7f6b341a4c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6b24005ab0 tx=0x7f6b24004360 comp rx=0 tx=0).stop 2024-09-11T23:35:01.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.463+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/2665788747 shutdown_connections 2024-09-11T23:35:01.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.463+0000 7f6b3c1b1640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6b200a2b80 0x7f6b200a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:01.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.463+0000 7f6b3c1b1640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b34112b10 0x7f6b3419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:01.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.463+0000 7f6b3c1b1640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b34106080 0x7f6b341a4c50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:01.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.463+0000 7f6b3c1b1640 1 --2- 172.21.15.133:0/2665788747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b34105760 0x7f6b341a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:01.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.463+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/2665788747 >> 172.21.15.133:0/2665788747 conn(0x7f6b34077210 msgr2=0x7f6b34111550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:01.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.464+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/2665788747 shutdown_connections 2024-09-11T23:35:01.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:01.464+0000 7f6b3c1b1640 1 -- 172.21.15.133:0/2665788747 wait complete. 2024-09-11T23:35:01.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:01 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2665788747' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T23:35:01.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:01 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2665788747' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T23:35:01.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:01 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2665788747' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T23:35:02.150 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-11T23:35:02.403 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:02.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:02 smithi178 ceph-mon[35882]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:35:02.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:02 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2665788747' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T23:35:02.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:02 smithi178 ceph-mon[35882]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:35:02.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:02 smithi178 ceph-mon[35882]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:35:02.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:02 smithi178 ceph-mon[35882]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-11T23:35:02.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:02 smithi178 ceph-mon[35882]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-11T23:35:02.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:02 smithi178 ceph-mon[35882]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-11T23:35:02.834 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:02 smithi133 ceph-mon[31430]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:35:02.834 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:02 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2665788747' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T23:35:02.834 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:02 smithi133 ceph-mon[31430]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:35:02.834 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:02 smithi133 ceph-mon[31430]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:35:02.834 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:02 smithi133 ceph-mon[31430]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-11T23:35:02.834 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:02 smithi133 ceph-mon[31430]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-11T23:35:02.834 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:02 smithi133 ceph-mon[31430]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-11T23:35:02.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.919+0000 7fcd34231640 1 -- 172.21.15.133:0/1368379663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd2c102ef0 msgr2=0x7fcd2c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:02.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.919+0000 7fcd34231640 1 --2- 172.21.15.133:0/1368379663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd2c102ef0 0x7fcd2c10da40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcd24009ca0 tx=0x7fcd2402a950 comp rx=0 tx=0).stop 2024-09-11T23:35:02.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.919+0000 7fcd34231640 1 -- 172.21.15.133:0/1368379663 shutdown_connections 2024-09-11T23:35:02.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.919+0000 7fcd34231640 1 --2- 172.21.15.133:0/1368379663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd2c102ef0 0x7fcd2c10da40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.919+0000 7fcd34231640 1 --2- 172.21.15.133:0/1368379663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c102570 0x7fcd2c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.919+0000 7fcd34231640 1 --2- 172.21.15.133:0/1368379663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd2c100e20 0x7fcd2c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.919+0000 7fcd34231640 1 -- 172.21.15.133:0/1368379663 >> 172.21.15.133:0/1368379663 conn(0x7fcd2c0fc7d0 msgr2=0x7fcd2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.919+0000 7fcd34231640 1 -- 172.21.15.133:0/1368379663 shutdown_connections 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.919+0000 7fcd34231640 1 -- 172.21.15.133:0/1368379663 wait complete. 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd34231640 1 Processor -- start 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd34231640 1 -- start start 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd34231640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c100e20 0x7fcd2c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd34231640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd2c102570 0x7fcd2c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd34231640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd2c102ef0 0x7fcd2c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd34231640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c110f80 con 0x7fcd2c102570 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd34231640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c110e00 con 0x7fcd2c100e20 2024-09-11T23:35:02.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd34231640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c111100 con 0x7fcd2c102ef0 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd317cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd2c102570 0x7fcd2c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd317cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd2c102570 0x7fcd2c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53028/0 (socket says 172.21.15.133:53028) 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd317cc640 1 -- 172.21.15.133:0/2986378090 learned_addr learned my addr 172.21.15.133:0/2986378090 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd31fcd640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c100e20 0x7fcd2c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd327ce640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd2c102ef0 0x7fcd2c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd317cc640 1 -- 172.21.15.133:0/2986378090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd2c102ef0 msgr2=0x7fcd2c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd317cc640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd2c102ef0 0x7fcd2c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd317cc640 1 -- 172.21.15.133:0/2986378090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c100e20 msgr2=0x7fcd2c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd317cc640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c100e20 0x7fcd2c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd317cc640 1 -- 172.21.15.133:0/2986378090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c19d470 con 0x7fcd2c102570 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.920+0000 7fcd317cc640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd2c102570 0x7fcd2c198a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800d950 tx=0x7fcd2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.921+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2986378090 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28017030 con 0x7fcd2c102570 2024-09-11T23:35:02.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.921+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2986378090 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd28004020 con 0x7fcd2c102570 2024-09-11T23:35:02.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.921+0000 7fcd34231640 1 -- 172.21.15.133:0/2986378090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2c19d760 con 0x7fcd2c102570 2024-09-11T23:35:02.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.921+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2986378090 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd280047a0 con 0x7fcd2c102570 2024-09-11T23:35:02.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.921+0000 7fcd34231640 1 -- 172.21.15.133:0/2986378090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2c1a5040 con 0x7fcd2c102570 2024-09-11T23:35:02.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.922+0000 7fcd34231640 1 -- 172.21.15.133:0/2986378090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2c101200 con 0x7fcd2c102570 2024-09-11T23:35:02.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.923+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2986378090 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcd280041d0 con 0x7fcd2c102570 2024-09-11T23:35:02.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.923+0000 7fcd22ffd640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd180a2b80 0x7fcd180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:02.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.923+0000 7fcd31fcd640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd180a2b80 0x7fcd180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:02.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.923+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2986378090 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fcd280ce970 con 0x7fcd2c102570 2024-09-11T23:35:02.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.924+0000 7fcd31fcd640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd180a2b80 0x7fcd180a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0058c0 tx=0x7fcd1c00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:02.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:02.928+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2986378090 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd2809ba20 con 0x7fcd2c102570 2024-09-11T23:35:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:02 smithi138 ceph-mon[35142]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:35:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:02 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2665788747' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T23:35:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:02 smithi138 ceph-mon[35142]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:35:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:02 smithi138 ceph-mon[35142]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-11T23:35:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:02 smithi138 ceph-mon[35142]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-11T23:35:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:02 smithi138 ceph-mon[35142]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-11T23:35:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:02 smithi138 ceph-mon[35142]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-11T23:35:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.130+0000 7fcd34231640 1 -- 172.21.15.133:0/2986378090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fcd2c10a0a0 con 0x7fcd2c102570 2024-09-11T23:35:03.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.434+0000 7fcd22ffd640 1 -- 172.21.15.133:0/2986378090 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fcd2809b320 con 0x7fcd2c102570 2024-09-11T23:35:03.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 -- 172.21.15.133:0/2986378090 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd180a2b80 msgr2=0x7fcd180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:03.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd180a2b80 0x7fcd180a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0058c0 tx=0x7fcd1c00a4c0 comp rx=0 tx=0).stop 2024-09-11T23:35:03.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 -- 172.21.15.133:0/2986378090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd2c102570 msgr2=0x7fcd2c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:03.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd2c102570 0x7fcd2c198a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800d950 tx=0x7fcd2800de10 comp rx=0 tx=0).stop 2024-09-11T23:35:03.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 -- 172.21.15.133:0/2986378090 shutdown_connections 2024-09-11T23:35:03.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcd180a2b80 0x7fcd180a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:03.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd2c102ef0 0x7fcd2c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:03.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd2c102570 0x7fcd2c198a00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:03.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 --2- 172.21.15.133:0/2986378090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c100e20 0x7fcd2c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:03.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 -- 172.21.15.133:0/2986378090 >> 172.21.15.133:0/2986378090 conn(0x7fcd2c0fc7d0 msgr2=0x7fcd2c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:03.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 -- 172.21.15.133:0/2986378090 shutdown_connections 2024-09-11T23:35:03.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:03.437+0000 7fcd34231640 1 -- 172.21.15.133:0/2986378090 wait complete. 2024-09-11T23:35:03.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] up:active 2024-09-11T23:35:03.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:35:03.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-11T23:35:03.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-09-11T23:35:03.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-09-11T23:35:03.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2986378090' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:35:03.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2986378090' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:35:03.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] up:active 2024-09-11T23:35:03.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T23:35:03.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:35:03.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-09-11T23:35:03.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:03 smithi133 ceph-mon[31430]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-11T23:35:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] up:active 2024-09-11T23:35:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:35:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-11T23:35:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-09-11T23:35:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-09-11T23:35:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2986378090' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:35:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2986378090' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:35:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] up:active 2024-09-11T23:35:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T23:35:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:35:03.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-09-11T23:35:03.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:03 smithi178 ceph-mon[35882]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-11T23:35:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] up:active 2024-09-11T23:35:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:35:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-11T23:35:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-09-11T23:35:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-09-11T23:35:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2986378090' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:35:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2986378090' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:35:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] up:active 2024-09-11T23:35:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T23:35:03.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:35:03.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-09-11T23:35:03.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:03 smithi138 ceph-mon[35142]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-11T23:35:04.066 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-11T23:35:04.313 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:04.698 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:04 smithi133 ceph-mon[31430]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-11T23:35:04.698 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:04 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] up:active 2024-09-11T23:35:04.698 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:04 smithi133 ceph-mon[31430]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T23:35:04.698 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:04 smithi133 ceph-mon[31430]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:35:04.698 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:04 smithi133 ceph-mon[31430]: Cluster is now healthy 2024-09-11T23:35:04.698 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:04 smithi133 ceph-mon[31430]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-09-11T23:35:04.698 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:04 smithi133 ceph-mon[31430]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-11T23:35:04.698 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:04 smithi133 ceph-mon[31430]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-11T23:35:04.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.780+0000 7f30ca713640 1 -- 172.21.15.133:0/3730630119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30c4102790 msgr2=0x7f30c41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:04.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.780+0000 7f30ca713640 1 --2- 172.21.15.133:0/3730630119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30c4102790 0x7f30c41067e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009900 tx=0x7f30b802a410 comp rx=0 tx=0).stop 2024-09-11T23:35:04.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.780+0000 7f30ca713640 1 -- 172.21.15.133:0/3730630119 shutdown_connections 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.780+0000 7f30ca713640 1 --2- 172.21.15.133:0/3730630119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c4106f10 0x7f30c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.780+0000 7f30ca713640 1 --2- 172.21.15.133:0/3730630119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30c4102790 0x7f30c41067e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.780+0000 7f30ca713640 1 --2- 172.21.15.133:0/3730630119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4101de0 0x7f30c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.780+0000 7f30ca713640 1 -- 172.21.15.133:0/3730630119 >> 172.21.15.133:0/3730630119 conn(0x7f30c40fd8c0 msgr2=0x7f30c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.780+0000 7f30ca713640 1 -- 172.21.15.133:0/3730630119 shutdown_connections 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.780+0000 7f30ca713640 1 -- 172.21.15.133:0/3730630119 wait complete. 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.781+0000 7f30ca713640 1 Processor -- start 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.781+0000 7f30ca713640 1 -- start start 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30ca713640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30c4101de0 0x7f30c4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30ca713640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4102790 0x7f30c41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30ca713640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c4106f10 0x7f30c419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30ca713640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4111f90 con 0x7f30c4101de0 2024-09-11T23:35:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30ca713640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4111e10 con 0x7f30c4102790 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30ca713640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4112110 con 0x7f30c4106f10 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30c4101de0 0x7f30c4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30c4101de0 0x7f30c4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53046/0 (socket says 172.21.15.133:53046) 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c3fff640 1 -- 172.21.15.133:0/2126494756 learned_addr learned my addr 172.21.15.133:0/2126494756 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c8cb0640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c4106f10 0x7f30c419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c3fff640 1 -- 172.21.15.133:0/2126494756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c4106f10 msgr2=0x7f30c419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c3fff640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c4106f10 0x7f30c419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c3fff640 1 -- 172.21.15.133:0/2126494756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4102790 msgr2=0x7f30c41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c3fff640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4102790 0x7f30c41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c3fff640 1 -- 172.21.15.133:0/2126494756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c419d420 con 0x7f30c4101de0 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c8cb0640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c4106f10 0x7f30c419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c3fff640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30c4101de0 0x7f30c4198470 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f30b400c890 tx=0x7f30b400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c17fa640 1 -- 172.21.15.133:0/2126494756 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4004220 con 0x7f30c4101de0 2024-09-11T23:35:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c17fa640 1 -- 172.21.15.133:0/2126494756 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30b40043c0 con 0x7f30c4101de0 2024-09-11T23:35:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30ca713640 1 -- 172.21.15.133:0/2126494756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c419d710 con 0x7f30c4101de0 2024-09-11T23:35:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30c17fa640 1 -- 172.21.15.133:0/2126494756 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b40049e0 con 0x7f30c4101de0 2024-09-11T23:35:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.782+0000 7f30ca713640 1 -- 172.21.15.133:0/2126494756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c41a4ff0 con 0x7f30c4101de0 2024-09-11T23:35:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.783+0000 7f30ca713640 1 -- 172.21.15.133:0/2126494756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c41021c0 con 0x7f30c4101de0 2024-09-11T23:35:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.785+0000 7f30c17fa640 1 -- 172.21.15.133:0/2126494756 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f30b40079b0 con 0x7f30c4101de0 2024-09-11T23:35:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.785+0000 7f30c17fa640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f30ac0a2b80 0x7f30ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.785+0000 7f30c37fe640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f30ac0a2b80 0x7f30ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.785+0000 7f30c17fa640 1 -- 172.21.15.133:0/2126494756 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f30b40cf050 con 0x7f30c4101de0 2024-09-11T23:35:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.786+0000 7f30c37fe640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f30ac0a2b80 0x7f30ac0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30b802a930 tx=0x7f30b8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.788+0000 7f30c17fa640 1 -- 172.21.15.133:0/2126494756 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30b4014030 con 0x7f30c4101de0 2024-09-11T23:35:04.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:04 smithi178 ceph-mon[35882]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-11T23:35:04.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:04 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] up:active 2024-09-11T23:35:04.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:04 smithi178 ceph-mon[35882]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T23:35:04.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:04 smithi178 ceph-mon[35882]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:35:04.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:04 smithi178 ceph-mon[35882]: Cluster is now healthy 2024-09-11T23:35:04.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:04 smithi178 ceph-mon[35882]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-09-11T23:35:04.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:04 smithi178 ceph-mon[35882]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-11T23:35:04.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:04 smithi178 ceph-mon[35882]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-11T23:35:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:04 smithi138 ceph-mon[35142]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-11T23:35:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:04 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] up:active 2024-09-11T23:35:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:04 smithi138 ceph-mon[35142]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T23:35:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:04 smithi138 ceph-mon[35142]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:35:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:04 smithi138 ceph-mon[35142]: Cluster is now healthy 2024-09-11T23:35:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:04 smithi138 ceph-mon[35142]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-09-11T23:35:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:04 smithi138 ceph-mon[35142]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-11T23:35:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:04 smithi138 ceph-mon[35142]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-11T23:35:04.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:04.991+0000 7f30ca713640 1 -- 172.21.15.133:0/2126494756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f30c41997d0 con 0x7f30c4101de0 2024-09-11T23:35:05.290 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:35:04 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i[66069]: 2024-09-11T23:35:04.914+0000 7fa988a4f640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-11T23:35:05.290 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:35:04 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i[66069]: 2024-09-11T23:35:04.914+0000 7fa988a4f640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-11T23:35:05.290 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:35:04 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i[66069]: 2024-09-11T23:35:04.914+0000 7fa988a4f640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-11T23:35:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.446+0000 7f30c17fa640 1 -- 172.21.15.133:0/2126494756 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f30b409c100 con 0x7f30c4101de0 2024-09-11T23:35:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 -- 172.21.15.133:0/2126494756 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f30ac0a2b80 msgr2=0x7f30ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f30ac0a2b80 0x7f30ac0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30b802a930 tx=0x7f30b8000bc0 comp rx=0 tx=0).stop 2024-09-11T23:35:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 -- 172.21.15.133:0/2126494756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30c4101de0 msgr2=0x7f30c4198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30c4101de0 0x7f30c4198470 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f30b400c890 tx=0x7f30b400cd50 comp rx=0 tx=0).stop 2024-09-11T23:35:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 -- 172.21.15.133:0/2126494756 shutdown_connections 2024-09-11T23:35:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f30ac0a2b80 0x7f30ac0a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c4106f10 0x7f30c419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c4102790 0x7f30c41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 --2- 172.21.15.133:0/2126494756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30c4101de0 0x7f30c4198470 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 -- 172.21.15.133:0/2126494756 >> 172.21.15.133:0/2126494756 conn(0x7f30c40fd8c0 msgr2=0x7f30c40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 -- 172.21.15.133:0/2126494756 shutdown_connections 2024-09-11T23:35:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:05.448+0000 7f30ca713640 1 -- 172.21.15.133:0/2126494756 wait complete. 2024-09-11T23:35:05.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:05 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2126494756' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:35:05.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:05 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2126494756' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:35:05.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:05 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] up:active 2024-09-11T23:35:05.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:05 smithi133 ceph-mon[31430]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-11T23:35:05.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:05 smithi133 ceph-mon[31430]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:05.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:05 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2126494756' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:35:05.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:05 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2126494756' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:35:05.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:05 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] up:active 2024-09-11T23:35:05.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:05 smithi178 ceph-mon[35882]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-11T23:35:05.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:05 smithi178 ceph-mon[35882]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:05.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:05 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2126494756' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:35:05.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:05 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2126494756' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:35:05.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:05 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] up:active 2024-09-11T23:35:05.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:05 smithi138 ceph-mon[35142]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-11T23:35:05.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:05 smithi138 ceph-mon[35142]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:35:06.193 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-11T23:35:06.193 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-11T23:35:06.443 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:06.787 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:06 smithi133 ceph-mon[31430]: pgmap v278: 97 pgs: 97 active+clean; 582 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-11T23:35:06.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:06 smithi178 ceph-mon[35882]: pgmap v278: 97 pgs: 97 active+clean; 582 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-11T23:35:06.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.869+0000 7f25cb636640 1 -- 172.21.15.133:0/1412665204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25c4100a60 msgr2=0x7f25c4100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:06.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.869+0000 7f25cb636640 1 --2- 172.21.15.133:0/1412665204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25c4100a60 0x7f25c4100ec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f25b4009930 tx=0x7f25b402a420 comp rx=0 tx=0).stop 2024-09-11T23:35:06.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.869+0000 7f25cb636640 1 -- 172.21.15.133:0/1412665204 shutdown_connections 2024-09-11T23:35:06.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.869+0000 7f25cb636640 1 --2- 172.21.15.133:0/1412665204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25c4101400 0x7f25c410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:06.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.869+0000 7f25cb636640 1 --2- 172.21.15.133:0/1412665204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25c4100a60 0x7f25c4100ec0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:06.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.869+0000 7f25cb636640 1 --2- 172.21.15.133:0/1412665204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c4106a60 0x7f25c4106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:06.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.869+0000 7f25cb636640 1 -- 172.21.15.133:0/1412665204 >> 172.21.15.133:0/1412665204 conn(0x7f25c40fc780 msgr2=0x7f25c40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:06.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.869+0000 7f25cb636640 1 -- 172.21.15.133:0/1412665204 shutdown_connections 2024-09-11T23:35:06.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.869+0000 7f25cb636640 1 -- 172.21.15.133:0/1412665204 wait complete. 2024-09-11T23:35:06.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25cb636640 1 Processor -- start 2024-09-11T23:35:06.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25cb636640 1 -- start start 2024-09-11T23:35:06.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25cb636640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c4100a60 0x7f25c419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:06.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25cb636640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25c4101400 0x7f25c419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25cb636640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25c4106a60 0x7f25c419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25cb636640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f25c4110ea0 con 0x7f25c4101400 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25cb636640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25c4110d20 con 0x7f25c4100a60 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25cb636640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25c4111020 con 0x7f25c4106a60 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25c8bd1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25c4101400 0x7f25c419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25c8bd1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25c4101400 0x7f25c419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53062/0 (socket says 172.21.15.133:53062) 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25c8bd1640 1 -- 172.21.15.133:0/2337536830 learned_addr learned my addr 172.21.15.133:0/2337536830 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25c93d2640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c4100a60 0x7f25c419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.870+0000 7f25c9bd3640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25c4106a60 0x7f25c419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.871+0000 7f25c8bd1640 1 -- 172.21.15.133:0/2337536830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25c4106a60 msgr2=0x7f25c419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.871+0000 7f25c8bd1640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25c4106a60 0x7f25c419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.871+0000 7f25c8bd1640 1 -- 172.21.15.133:0/2337536830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c4100a60 msgr2=0x7f25c419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:06.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.871+0000 7f25c8bd1640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c4100a60 0x7f25c419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.871+0000 7f25c8bd1640 1 -- 172.21.15.133:0/2337536830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c419f660 con 0x7f25c4101400 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.871+0000 7f25c8bd1640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25c4101400 0x7f25c419abf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f25b4009230 tx=0x7f25b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.871+0000 7f25ba7fc640 1 -- 172.21.15.133:0/2337536830 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b403f030 con 0x7f25c4101400 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.871+0000 7f25ba7fc640 1 -- 172.21.15.133:0/2337536830 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b4003ad0 con 0x7f25c4101400 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.871+0000 7f25cb636640 1 -- 172.21.15.133:0/2337536830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c419f8f0 con 0x7f25c4101400 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.871+0000 7f25ba7fc640 1 -- 172.21.15.133:0/2337536830 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4035070 con 0x7f25c4101400 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.871+0000 7f25cb636640 1 -- 172.21.15.133:0/2337536830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c41a7230 con 0x7f25c4101400 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.872+0000 7f25cb636640 1 -- 172.21.15.133:0/2337536830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c41021a0 con 0x7f25c4101400 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.873+0000 7f25ba7fc640 1 -- 172.21.15.133:0/2337536830 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f25b4042030 con 0x7f25c4101400 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.874+0000 7f25ba7fc640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f25b00a2b80 0x7f25b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.874+0000 7f25ba7fc640 1 -- 172.21.15.133:0/2337536830 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f25b40ef750 con 0x7f25c4101400 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.874+0000 7f25c93d2640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f25b00a2b80 0x7f25b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.874+0000 7f25c93d2640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f25b00a2b80 0x7f25b00a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009990 tx=0x7f25c0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:06.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:06.877+0000 7f25ba7fc640 1 -- 172.21.15.133:0/2337536830 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25b40bc800 con 0x7f25c4101400 2024-09-11T23:35:06.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:06 smithi138 ceph-mon[35142]: pgmap v278: 97 pgs: 97 active+clean; 582 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-11T23:35:07.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.067+0000 7f25cb636640 1 -- 172.21.15.133:0/2337536830 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f25c40fd4e0 con 0x7f25b00a2b80 2024-09-11T23:35:07.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.094+0000 7f25ba7fc640 1 -- 172.21.15.133:0/2337536830 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25c40fd4e0 con 0x7f25b00a2b80 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 -- 172.21.15.133:0/2337536830 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f25b00a2b80 msgr2=0x7f25b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f25b00a2b80 0x7f25b00a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009990 tx=0x7f25c0009110 comp rx=0 tx=0).stop 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 -- 172.21.15.133:0/2337536830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25c4101400 msgr2=0x7f25c419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25c4101400 0x7f25c419abf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f25b4009230 tx=0x7f25b4004270 comp rx=0 tx=0).stop 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 -- 172.21.15.133:0/2337536830 shutdown_connections 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f25b00a2b80 0x7f25b00a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25c4106a60 0x7f25c419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25c4101400 0x7f25c419abf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 --2- 172.21.15.133:0/2337536830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c4100a60 0x7f25c419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 -- 172.21.15.133:0/2337536830 >> 172.21.15.133:0/2337536830 conn(0x7f25c40fc780 msgr2=0x7f25c40fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 -- 172.21.15.133:0/2337536830 shutdown_connections 2024-09-11T23:35:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:07.096+0000 7f25cb636640 1 -- 172.21.15.133:0/2337536830 wait complete. 2024-09-11T23:35:07.417 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:35:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:35:07.071+0000 7f61f9fab640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:35:07.417 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:35:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:35:07.072+0000 7f61f9fab640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:35:07.417 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:35:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:35:07.072+0000 7f61f9fab640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:35:07.417 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:35:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:35:07.072+0000 7f61f9fab640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:35:07.417 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:35:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:35:07.072+0000 7f61f9fab640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:35:07.747 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-11T23:35:08.004 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:08.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-11T23:35:08.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-11T23:35:08.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-11T23:35:08.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-11T23:35:08.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-11T23:35:08.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:07 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-11T23:35:08.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-11T23:35:08.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-11T23:35:08.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-11T23:35:08.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-11T23:35:08.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-11T23:35:08.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:07 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-11T23:35:08.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-11T23:35:08.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-11T23:35:08.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-11T23:35:08.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-11T23:35:08.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-11T23:35:08.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:07 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-11T23:35:08.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.463+0000 7f76d9274640 1 -- 172.21.15.133:0/21279441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d4101dd0 msgr2=0x7f76d41021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:08.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.463+0000 7f76d9274640 1 --2- 172.21.15.133:0/21279441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d4101dd0 0x7f76d41021b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009ca0 tx=0x7f76c402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:35:08.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.463+0000 7f76d9274640 1 -- 172.21.15.133:0/21279441 shutdown_connections 2024-09-11T23:35:08.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.463+0000 7f76d9274640 1 --2- 172.21.15.133:0/21279441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d4106f00 0x7f76d410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:08.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.463+0000 7f76d9274640 1 --2- 172.21.15.133:0/21279441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4102780 0x7f76d41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:08.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.463+0000 7f76d9274640 1 --2- 172.21.15.133:0/21279441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d4101dd0 0x7f76d41021b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:08.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.463+0000 7f76d9274640 1 -- 172.21.15.133:0/21279441 >> 172.21.15.133:0/21279441 conn(0x7f76d40fd8b0 msgr2=0x7f76d40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:08.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.463+0000 7f76d9274640 1 -- 172.21.15.133:0/21279441 shutdown_connections 2024-09-11T23:35:08.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.463+0000 7f76d9274640 1 -- 172.21.15.133:0/21279441 wait complete. 2024-09-11T23:35:08.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.464+0000 7f76d9274640 1 Processor -- start 2024-09-11T23:35:08.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.464+0000 7f76d9274640 1 -- start start 2024-09-11T23:35:08.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.464+0000 7f76d9274640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4101dd0 0x7f76d41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:08.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.464+0000 7f76d9274640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d4102780 0x7f76d4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:08.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.464+0000 7f76d9274640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d4106f00 0x7f76d419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:08.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.464+0000 7f76d9274640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4111fa0 con 0x7f76d4102780 2024-09-11T23:35:08.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.464+0000 7f76d9274640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4111e20 con 0x7f76d4106f00 2024-09-11T23:35:08.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.464+0000 7f76d9274640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4112120 con 0x7f76d4101dd0 2024-09-11T23:35:08.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d4102780 0x7f76d4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:08.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d4102780 0x7f76d4198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53090/0 (socket says 172.21.15.133:53090) 2024-09-11T23:35:08.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d27fc640 1 -- 172.21.15.133:0/680498871 learned_addr learned my addr 172.21.15.133:0/680498871 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:08.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d37fe640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d4106f00 0x7f76d419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:08.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d2ffd640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4101dd0 0x7f76d41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d27fc640 1 -- 172.21.15.133:0/680498871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4101dd0 msgr2=0x7f76d41984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d27fc640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4101dd0 0x7f76d41984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d27fc640 1 -- 172.21.15.133:0/680498871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d4106f00 msgr2=0x7f76d419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d27fc640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d4106f00 0x7f76d419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d27fc640 1 -- 172.21.15.133:0/680498871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d419d470 con 0x7f76d4102780 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d27fc640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d4102780 0x7f76d4198a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f76c800c970 tx=0x7f76c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76bbfff640 1 -- 172.21.15.133:0/680498871 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8007bb0 con 0x7f76d4102780 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d9274640 1 -- 172.21.15.133:0/680498871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d419d760 con 0x7f76d4102780 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76bbfff640 1 -- 172.21.15.133:0/680498871 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c8007d50 con 0x7f76d4102780 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76bbfff640 1 -- 172.21.15.133:0/680498871 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c800f450 con 0x7f76d4102780 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.465+0000 7f76d9274640 1 -- 172.21.15.133:0/680498871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41a5040 con 0x7f76d4102780 2024-09-11T23:35:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.466+0000 7f76d9274640 1 -- 172.21.15.133:0/680498871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d41021b0 con 0x7f76d4102780 2024-09-11T23:35:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.467+0000 7f76bbfff640 1 -- 172.21.15.133:0/680498871 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f76c8005020 con 0x7f76d4102780 2024-09-11T23:35:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.468+0000 7f76bbfff640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f76bc0a2b80 0x7f76bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.468+0000 7f76d2ffd640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f76bc0a2b80 0x7f76bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.468+0000 7f76bbfff640 1 -- 172.21.15.133:0/680498871 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8153+0+0 (secure 0 0 0) 0x7f76c80cecd0 con 0x7f76d4102780 2024-09-11T23:35:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.468+0000 7f76d2ffd640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f76bc0a2b80 0x7f76bc0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76c4006010 tx=0x7f76c4009900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.473+0000 7f76bbfff640 1 -- 172.21.15.133:0/680498871 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76c8014030 con 0x7f76d4102780 2024-09-11T23:35:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.665+0000 7f76d9274640 1 -- 172.21.15.133:0/680498871 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f76d4062630 con 0x7f76bc0a2b80 2024-09-11T23:35:08.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.689+0000 7f76bbfff640 1 -- 172.21.15.133:0/680498871 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f76d4062630 con 0x7f76bc0a2b80 2024-09-11T23:35:08.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 -- 172.21.15.133:0/680498871 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f76bc0a2b80 msgr2=0x7f76bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:08.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f76bc0a2b80 0x7f76bc0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76c4006010 tx=0x7f76c4009900 comp rx=0 tx=0).stop 2024-09-11T23:35:08.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 -- 172.21.15.133:0/680498871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d4102780 msgr2=0x7f76d4198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:08.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d4102780 0x7f76d4198a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f76c800c970 tx=0x7f76c800ce30 comp rx=0 tx=0).stop 2024-09-11T23:35:08.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 -- 172.21.15.133:0/680498871 shutdown_connections 2024-09-11T23:35:08.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f76bc0a2b80 0x7f76bc0a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:08.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d4106f00 0x7f76d419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:08.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d4102780 0x7f76d4198a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:08.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 --2- 172.21.15.133:0/680498871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4101dd0 0x7f76d41984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:08.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 -- 172.21.15.133:0/680498871 >> 172.21.15.133:0/680498871 conn(0x7f76d40fd8b0 msgr2=0x7f76d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:08.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 -- 172.21.15.133:0/680498871 shutdown_connections 2024-09-11T23:35:08.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:08.692+0000 7f76d9274640 1 -- 172.21.15.133:0/680498871 wait complete. 2024-09-11T23:35:08.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:08 smithi133 ceph-mon[31430]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:08.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-11T23:35:08.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-11T23:35:08.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-11T23:35:08.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-11T23:35:08.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-11T23:35:08.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:08 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-11T23:35:08.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:08 smithi133 ceph-mon[31430]: pgmap v280: 97 pgs: 97 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.1 KiB/s wr, 21 op/s 2024-09-11T23:35:08.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:08 smithi133 ceph-mon[31430]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:35:08.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:08 smithi133 ceph-mon[31430]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:35:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:08 smithi138 ceph-mon[35142]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-11T23:35:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-11T23:35:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-11T23:35:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-11T23:35:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-11T23:35:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:08 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-11T23:35:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:08 smithi138 ceph-mon[35142]: pgmap v280: 97 pgs: 97 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.1 KiB/s wr, 21 op/s 2024-09-11T23:35:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:08 smithi138 ceph-mon[35142]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:35:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:08 smithi138 ceph-mon[35142]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:35:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:08 smithi178 ceph-mon[35882]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-11T23:35:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-11T23:35:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-11T23:35:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-11T23:35:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-11T23:35:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:08 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-11T23:35:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:08 smithi178 ceph-mon[35882]: pgmap v280: 97 pgs: 97 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.1 KiB/s wr, 21 op/s 2024-09-11T23:35:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:08 smithi178 ceph-mon[35882]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:35:09.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:08 smithi178 ceph-mon[35882]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:35:09.337 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-11T23:35:09.586 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:09.957 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:09 smithi133 ceph-mon[31430]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:09.957 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:09 smithi133 ceph-mon[31430]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-09-11T23:35:10.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.037+0000 7f59668b3640 1 -- 172.21.15.133:0/3966891925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960102ef0 msgr2=0x7f596010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:10.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.037+0000 7f59668b3640 1 --2- 172.21.15.133:0/3966891925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960102ef0 0x7f596010da40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5958009ca0 tx=0x7f595802a950 comp rx=0 tx=0).stop 2024-09-11T23:35:10.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.037+0000 7f59668b3640 1 -- 172.21.15.133:0/3966891925 shutdown_connections 2024-09-11T23:35:10.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.037+0000 7f59668b3640 1 --2- 172.21.15.133:0/3966891925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960102ef0 0x7f596010da40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.037+0000 7f59668b3640 1 --2- 172.21.15.133:0/3966891925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5960102570 0x7f59601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.037+0000 7f59668b3640 1 --2- 172.21.15.133:0/3966891925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5960100e20 0x7f5960101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.037+0000 7f59668b3640 1 -- 172.21.15.133:0/3966891925 >> 172.21.15.133:0/3966891925 conn(0x7f59600fc7d0 msgr2=0x7f59600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.037+0000 7f59668b3640 1 -- 172.21.15.133:0/3966891925 shutdown_connections 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.037+0000 7f59668b3640 1 -- 172.21.15.133:0/3966891925 wait complete. 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f59668b3640 1 Processor -- start 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f59668b3640 1 -- start start 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f59668b3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960100e20 0x7f596019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f59668b3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5960102570 0x7f596019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f59668b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5960102ef0 0x7f596019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f59668b3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5960110f80 con 0x7f5960100e20 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f59668b3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5960110e00 con 0x7f5960102570 2024-09-11T23:35:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f59668b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5960111100 con 0x7f5960102ef0 2024-09-11T23:35:10.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f595ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960100e20 0x7f596019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:10.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f595ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960100e20 0x7f596019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53104/0 (socket says 172.21.15.133:53104) 2024-09-11T23:35:10.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f595ffff640 1 -- 172.21.15.133:0/3942102713 learned_addr learned my addr 172.21.15.133:0/3942102713 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:10.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f5957fff640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5960102570 0x7f596019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:10.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f5964e50640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5960102ef0 0x7f596019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:10.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f595ffff640 1 -- 172.21.15.133:0/3942102713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5960102ef0 msgr2=0x7f596019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:10.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f595ffff640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5960102ef0 0x7f596019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:10.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f595ffff640 1 -- 172.21.15.133:0/3942102713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5960102570 msgr2=0x7f596019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:10.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f595ffff640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5960102570 0x7f596019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:10.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f595ffff640 1 -- 172.21.15.133:0/3942102713 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596019f610 con 0x7f5960100e20 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.038+0000 7f5957fff640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5960102570 0x7f596019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.039+0000 7f595ffff640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960100e20 0x7f596019a660 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f595000c890 tx=0x7f595000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.039+0000 7f595dffb640 1 -- 172.21.15.133:0/3942102713 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950004220 con 0x7f5960100e20 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.039+0000 7f59668b3640 1 -- 172.21.15.133:0/3942102713 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596019f900 con 0x7f5960100e20 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.039+0000 7f595dffb640 1 -- 172.21.15.133:0/3942102713 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59500043c0 con 0x7f5960100e20 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.039+0000 7f595dffb640 1 -- 172.21.15.133:0/3942102713 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950004a00 con 0x7f5960100e20 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.039+0000 7f59668b3640 1 -- 172.21.15.133:0/3942102713 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59601a71e0 con 0x7f5960100e20 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.040+0000 7f59668b3640 1 -- 172.21.15.133:0/3942102713 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5960101200 con 0x7f5960100e20 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.041+0000 7f595dffb640 1 -- 172.21.15.133:0/3942102713 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f59500079b0 con 0x7f5960100e20 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.042+0000 7f595dffb640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f594c0a2c10 0x7f594c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.042+0000 7f5957fff640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f594c0a2c10 0x7f594c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.042+0000 7f595dffb640 1 -- 172.21.15.133:0/3942102713 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f59500cee80 con 0x7f5960100e20 2024-09-11T23:35:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.042+0000 7f5957fff640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f594c0a2c10 0x7f594c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f596010ddf0 tx=0x7f5948005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:10.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.046+0000 7f595dffb640 1 -- 172.21.15.133:0/3942102713 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5950014030 con 0x7f5960100e20 2024-09-11T23:35:10.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:09 smithi138 ceph-mon[35142]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:10.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:09 smithi138 ceph-mon[35142]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-09-11T23:35:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.240+0000 7f59668b3640 1 -- 172.21.15.133:0/3942102713 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f596006a680 con 0x7f594c0a2c10 2024-09-11T23:35:10.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.245+0000 7f595dffb640 1 -- 172.21.15.133:0/3942102713 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f596006a680 con 0x7f594c0a2c10 2024-09-11T23:35:10.244 INFO:teuthology.orchestra.run.smithi133.stdout:/volumes/qa/sv_0/c5800482-6837-487d-8e92-f6eb01265746 2024-09-11T23:35:10.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.246+0000 7f59668b3640 1 -- 172.21.15.133:0/3942102713 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f594c0a2c10 msgr2=0x7f594c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:10.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.246+0000 7f59668b3640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f594c0a2c10 0x7f594c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f596010ddf0 tx=0x7f5948005d30 comp rx=0 tx=0).stop 2024-09-11T23:35:10.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.247+0000 7f59668b3640 1 -- 172.21.15.133:0/3942102713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960100e20 msgr2=0x7f596019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:10.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.247+0000 7f59668b3640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960100e20 0x7f596019a660 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f595000c890 tx=0x7f595000cd50 comp rx=0 tx=0).stop 2024-09-11T23:35:10.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.247+0000 7f59668b3640 1 -- 172.21.15.133:0/3942102713 shutdown_connections 2024-09-11T23:35:10.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.247+0000 7f59668b3640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f594c0a2c10 0x7f594c0a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:10.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.247+0000 7f59668b3640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5960102ef0 0x7f596019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:10.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.247+0000 7f59668b3640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5960102570 0x7f596019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:10.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.247+0000 7f59668b3640 1 --2- 172.21.15.133:0/3942102713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960100e20 0x7f596019a660 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:10.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.247+0000 7f59668b3640 1 -- 172.21.15.133:0/3942102713 >> 172.21.15.133:0/3942102713 conn(0x7f59600fc7d0 msgr2=0x7f5960069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:10.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.247+0000 7f59668b3640 1 -- 172.21.15.133:0/3942102713 shutdown_connections 2024-09-11T23:35:10.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:10.247+0000 7f59668b3640 1 -- 172.21.15.133:0/3942102713 wait complete. 2024-09-11T23:35:10.290 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:35:09 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i[66069]: 2024-09-11T23:35:09.915+0000 7fa988a4f640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-11T23:35:10.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:09 smithi178 ceph-mon[35882]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:10.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:09 smithi178 ceph-mon[35882]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-09-11T23:35:10.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:10 smithi133 ceph-mon[31430]: pgmap v282: 97 pgs: 97 active+clean; 605 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.9 KiB/s wr, 29 op/s 2024-09-11T23:35:11.032 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-11T23:35:11.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:10 smithi138 ceph-mon[35142]: pgmap v282: 97 pgs: 97 active+clean; 605 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.9 KiB/s wr, 29 op/s 2024-09-11T23:35:11.285 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:10 smithi178 ceph-mon[35882]: pgmap v282: 97 pgs: 97 active+clean; 605 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.9 KiB/s wr, 29 op/s 2024-09-11T23:35:11.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.753+0000 7fdc70358640 1 -- 172.21.15.133:0/3807757574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc68101df0 msgr2=0x7fdc681021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:11.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.753+0000 7fdc70358640 1 --2- 172.21.15.133:0/3807757574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc68101df0 0x7fdc681021d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009900 tx=0x7fdc6402a410 comp rx=0 tx=0).stop 2024-09-11T23:35:11.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.753+0000 7fdc70358640 1 -- 172.21.15.133:0/3807757574 shutdown_connections 2024-09-11T23:35:11.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.753+0000 7fdc70358640 1 --2- 172.21.15.133:0/3807757574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc68106f20 0x7fdc6810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.753+0000 7fdc70358640 1 --2- 172.21.15.133:0/3807757574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc681027a0 0x7fdc681067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.753+0000 7fdc70358640 1 --2- 172.21.15.133:0/3807757574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc68101df0 0x7fdc681021d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.753+0000 7fdc70358640 1 -- 172.21.15.133:0/3807757574 >> 172.21.15.133:0/3807757574 conn(0x7fdc680fd8b0 msgr2=0x7fdc680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:11.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.753+0000 7fdc70358640 1 -- 172.21.15.133:0/3807757574 shutdown_connections 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.753+0000 7fdc70358640 1 -- 172.21.15.133:0/3807757574 wait complete. 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.754+0000 7fdc70358640 1 Processor -- start 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.754+0000 7fdc70358640 1 -- start start 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.754+0000 7fdc70358640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc68101df0 0x7fdc68198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.754+0000 7fdc70358640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc681027a0 0x7fdc68198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.754+0000 7fdc70358640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68106f20 0x7fdc6819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.754+0000 7fdc70358640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68111f40 con 0x7fdc68101df0 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.754+0000 7fdc70358640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68111dc0 con 0x7fdc681027a0 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.754+0000 7fdc70358640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdc681120c0 con 0x7fdc68106f20 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.754+0000 7fdc6e0f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc68101df0 0x7fdc68198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc6e0f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc68101df0 0x7fdc68198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53132/0 (socket says 172.21.15.133:53132) 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc6e0f4640 1 -- 172.21.15.133:0/592774451 learned_addr learned my addr 172.21.15.133:0/592774451 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:11.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc6d8f3640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc681027a0 0x7fdc68198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc6e8f5640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68106f20 0x7fdc6819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc6e0f4640 1 -- 172.21.15.133:0/592774451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68106f20 msgr2=0x7fdc6819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc6e0f4640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68106f20 0x7fdc6819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc6e0f4640 1 -- 172.21.15.133:0/592774451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc681027a0 msgr2=0x7fdc68198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc6e0f4640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc681027a0 0x7fdc68198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc6e0f4640 1 -- 172.21.15.133:0/592774451 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6819d4b0 con 0x7fdc68101df0 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc6e8f5640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68106f20 0x7fdc6819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc6e0f4640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc68101df0 0x7fdc68198500 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdc64006010 tx=0x7fdc640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc5f7fe640 1 -- 172.21.15.133:0/592774451 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6403f030 con 0x7fdc68101df0 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc5f7fe640 1 -- 172.21.15.133:0/592774451 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc64003ce0 con 0x7fdc68101df0 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc70358640 1 -- 172.21.15.133:0/592774451 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6819d740 con 0x7fdc68101df0 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc5f7fe640 1 -- 172.21.15.133:0/592774451 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64035070 con 0x7fdc68101df0 2024-09-11T23:35:11.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.755+0000 7fdc70358640 1 -- 172.21.15.133:0/592774451 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc681a5080 con 0x7fdc68101df0 2024-09-11T23:35:11.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.756+0000 7fdc70358640 1 -- 172.21.15.133:0/592774451 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc681032d0 con 0x7fdc68101df0 2024-09-11T23:35:11.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.757+0000 7fdc5f7fe640 1 -- 172.21.15.133:0/592774451 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdc64042030 con 0x7fdc68101df0 2024-09-11T23:35:11.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.758+0000 7fdc5f7fe640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc540a2c10 0x7fdc540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:11.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.758+0000 7fdc6d8f3640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc540a2c10 0x7fdc540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:11.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.758+0000 7fdc5f7fe640 1 -- 172.21.15.133:0/592774451 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fdc640efa30 con 0x7fdc68101df0 2024-09-11T23:35:11.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.758+0000 7fdc6d8f3640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc540a2c10 0x7fdc540a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc680fde10 tx=0x7fdc5800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:11.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.761+0000 7fdc5f7fe640 1 -- 172.21.15.133:0/592774451 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc640bc9f0 con 0x7fdc68101df0 2024-09-11T23:35:11.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.954+0000 7fdc70358640 1 -- 172.21.15.133:0/592774451 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdc68062630 con 0x7fdc540a2c10 2024-09-11T23:35:11.954 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:11 smithi133 ceph-mon[31430]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.980+0000 7fdc5f7fe640 1 -- 172.21.15.133:0/592774451 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdc68062630 con 0x7fdc540a2c10 2024-09-11T23:35:11.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 -- 172.21.15.133:0/592774451 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc540a2c10 msgr2=0x7fdc540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:11.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc540a2c10 0x7fdc540a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc680fde10 tx=0x7fdc5800b390 comp rx=0 tx=0).stop 2024-09-11T23:35:11.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 -- 172.21.15.133:0/592774451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc68101df0 msgr2=0x7fdc68198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:11.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc68101df0 0x7fdc68198500 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdc64006010 tx=0x7fdc640042f0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 -- 172.21.15.133:0/592774451 shutdown_connections 2024-09-11T23:35:11.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc540a2c10 0x7fdc540a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68106f20 0x7fdc6819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc681027a0 0x7fdc68198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 --2- 172.21.15.133:0/592774451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc68101df0 0x7fdc68198500 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:11.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 -- 172.21.15.133:0/592774451 >> 172.21.15.133:0/592774451 conn(0x7fdc680fd8b0 msgr2=0x7fdc680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:11.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 -- 172.21.15.133:0/592774451 shutdown_connections 2024-09-11T23:35:11.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:11.982+0000 7fdc70358640 1 -- 172.21.15.133:0/592774451 wait complete. 2024-09-11T23:35:12.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:11 smithi138 ceph-mon[35142]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:12.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:11 smithi178 ceph-mon[35882]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:12.676 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-11T23:35:12.920 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:12.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:12 smithi133 ceph-mon[31430]: pgmap v283: 97 pgs: 97 active+clean; 605 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.5 KiB/s wr, 30 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:35:12.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:12 smithi133 ceph-mon[31430]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:12 smithi138 ceph-mon[35142]: pgmap v283: 97 pgs: 97 active+clean; 605 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.5 KiB/s wr, 30 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:35:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:12 smithi138 ceph-mon[35142]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:12 smithi178 ceph-mon[35882]: pgmap v283: 97 pgs: 97 active+clean; 605 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.5 KiB/s wr, 30 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:35:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:12 smithi178 ceph-mon[35882]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:13.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.386+0000 7f92459db640 1 -- 172.21.15.133:0/2124729053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240102570 msgr2=0x7f92401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:13.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.386+0000 7f92459db640 1 --2- 172.21.15.133:0/2124729053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240102570 0x7f92401029b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9228009900 tx=0x7f922802a490 comp rx=0 tx=0).stop 2024-09-11T23:35:13.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.386+0000 7f92459db640 1 -- 172.21.15.133:0/2124729053 shutdown_connections 2024-09-11T23:35:13.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.386+0000 7f92459db640 1 --2- 172.21.15.133:0/2124729053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9240102ef0 0x7f924010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:13.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.386+0000 7f92459db640 1 --2- 172.21.15.133:0/2124729053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240102570 0x7f92401029b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.386+0000 7f92459db640 1 --2- 172.21.15.133:0/2124729053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9240100e20 0x7f9240101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.386+0000 7f92459db640 1 -- 172.21.15.133:0/2124729053 >> 172.21.15.133:0/2124729053 conn(0x7f92400fc7d0 msgr2=0x7f92400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.386+0000 7f92459db640 1 -- 172.21.15.133:0/2124729053 shutdown_connections 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.386+0000 7f92459db640 1 -- 172.21.15.133:0/2124729053 wait complete. 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.386+0000 7f92459db640 1 Processor -- start 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f92459db640 1 -- start start 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f92459db640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240100e20 0x7f9240198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f92459db640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9240102570 0x7f9240198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f92459db640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9240102ef0 0x7f924019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f92459db640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9240110fa0 con 0x7f9240100e20 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f92459db640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9240110e20 con 0x7f9240102ef0 2024-09-11T23:35:13.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f92459db640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9240111120 con 0x7f9240102570 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f923effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240100e20 0x7f9240198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f923effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240100e20 0x7f9240198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43128/0 (socket says 172.21.15.133:43128) 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f923effd640 1 -- 172.21.15.133:0/767841984 learned_addr learned my addr 172.21.15.133:0/767841984 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f923f7fe640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9240102ef0 0x7f924019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f9237fff640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9240102570 0x7f9240198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f923effd640 1 -- 172.21.15.133:0/767841984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9240102570 msgr2=0x7f9240198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f923effd640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9240102570 0x7f9240198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f923effd640 1 -- 172.21.15.133:0/767841984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9240102ef0 msgr2=0x7f924019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f923effd640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9240102ef0 0x7f924019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f923effd640 1 -- 172.21.15.133:0/767841984 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924019d510 con 0x7f9240100e20 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f9237fff640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9240102570 0x7f9240198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:13.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f923f7fe640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9240102ef0 0x7f924019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:13.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.387+0000 7f923effd640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240100e20 0x7f9240198560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f923000b4a0 tx=0x7f923000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:13.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.388+0000 7f92449d9640 1 -- 172.21.15.133:0/767841984 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230014070 con 0x7f9240100e20 2024-09-11T23:35:13.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.388+0000 7f92449d9640 1 -- 172.21.15.133:0/767841984 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9230010af0 con 0x7f9240100e20 2024-09-11T23:35:13.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.388+0000 7f92459db640 1 -- 172.21.15.133:0/767841984 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924019d800 con 0x7f9240100e20 2024-09-11T23:35:13.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.388+0000 7f92449d9640 1 -- 172.21.15.133:0/767841984 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230017d10 con 0x7f9240100e20 2024-09-11T23:35:13.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.388+0000 7f92459db640 1 -- 172.21.15.133:0/767841984 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92401a50e0 con 0x7f9240100e20 2024-09-11T23:35:13.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.389+0000 7f92459db640 1 -- 172.21.15.133:0/767841984 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9240101200 con 0x7f9240100e20 2024-09-11T23:35:13.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.390+0000 7f92449d9640 1 -- 172.21.15.133:0/767841984 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9230017070 con 0x7f9240100e20 2024-09-11T23:35:13.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.390+0000 7f92449d9640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f922c0a2c10 0x7f922c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:13.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.391+0000 7f9237fff640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f922c0a2c10 0x7f922c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:13.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.391+0000 7f92449d9640 1 -- 172.21.15.133:0/767841984 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f92300ce450 con 0x7f9240100e20 2024-09-11T23:35:13.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.391+0000 7f9237fff640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f922c0a2c10 0x7f922c0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9228000c00 tx=0x7f9228004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:13.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.394+0000 7f92449d9640 1 -- 172.21.15.133:0/767841984 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f923009b460 con 0x7f9240100e20 2024-09-11T23:35:13.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.592+0000 7f92459db640 1 -- 172.21.15.133:0/767841984 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f9240062630 con 0x7f922c0a2c10 2024-09-11T23:35:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.596+0000 7f92449d9640 1 -- 172.21.15.133:0/767841984 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9240062630 con 0x7f922c0a2c10 2024-09-11T23:35:13.596 INFO:teuthology.orchestra.run.smithi133.stdout:/volumes/qa/sv_1/53fd07fa-2161-401a-8fee-11858bbc78a1 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 -- 172.21.15.133:0/767841984 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f922c0a2c10 msgr2=0x7f922c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f922c0a2c10 0x7f922c0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9228000c00 tx=0x7f9228004100 comp rx=0 tx=0).stop 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 -- 172.21.15.133:0/767841984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240100e20 msgr2=0x7f9240198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240100e20 0x7f9240198560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f923000b4a0 tx=0x7f923000b960 comp rx=0 tx=0).stop 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 -- 172.21.15.133:0/767841984 shutdown_connections 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f922c0a2c10 0x7f922c0a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9240102ef0 0x7f924019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9240102570 0x7f9240198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 --2- 172.21.15.133:0/767841984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240100e20 0x7f9240198560 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 -- 172.21.15.133:0/767841984 >> 172.21.15.133:0/767841984 conn(0x7f92400fc7d0 msgr2=0x7f924006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 -- 172.21.15.133:0/767841984 shutdown_connections 2024-09-11T23:35:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:13.598+0000 7f92459db640 1 -- 172.21.15.133:0/767841984 wait complete. 2024-09-11T23:35:14.308 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:35:14.708 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:15.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:14 smithi133 ceph-mon[31430]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:15.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:14 smithi133 ceph-mon[31430]: pgmap v284: 97 pgs: 97 active+clean; 605 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.6 KiB/s wr, 19 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:35:15.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:14 smithi138 ceph-mon[35142]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:15.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:14 smithi138 ceph-mon[35142]: pgmap v284: 97 pgs: 97 active+clean; 605 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.6 KiB/s wr, 19 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:35:15.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.244+0000 7f365b085640 1 -- 172.21.15.133:0/2005745887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3654100a80 msgr2=0x7f3654100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:15.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.244+0000 7f365b085640 1 --2- 172.21.15.133:0/2005745887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3654100a80 0x7f3654100ee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3640009930 tx=0x7f364002a420 comp rx=0 tx=0).stop 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.244+0000 7f365b085640 1 -- 172.21.15.133:0/2005745887 shutdown_connections 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.244+0000 7f365b085640 1 --2- 172.21.15.133:0/2005745887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3654101420 0x7f365410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.244+0000 7f365b085640 1 --2- 172.21.15.133:0/2005745887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3654100a80 0x7f3654100ee0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.244+0000 7f365b085640 1 --2- 172.21.15.133:0/2005745887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3654106a80 0x7f3654106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.244+0000 7f365b085640 1 -- 172.21.15.133:0/2005745887 >> 172.21.15.133:0/2005745887 conn(0x7f36540fc7a0 msgr2=0x7f36540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.244+0000 7f365b085640 1 -- 172.21.15.133:0/2005745887 shutdown_connections 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.244+0000 7f365b085640 1 -- 172.21.15.133:0/2005745887 wait complete. 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f365b085640 1 Processor -- start 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f365b085640 1 -- start start 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f365b085640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3654100a80 0x7f3654197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f365b085640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3654101420 0x7f3654197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f365b085640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3654106a80 0x7f365419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f365b085640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3654110e80 con 0x7f3654100a80 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f365b085640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3654110d00 con 0x7f3654101420 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f365b085640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3654111000 con 0x7f3654106a80 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3658e21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3654100a80 0x7f3654197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3658e21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3654100a80 0x7f3654197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43148/0 (socket says 172.21.15.133:43148) 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3658e21640 1 -- 172.21.15.133:0/1978222731 learned_addr learned my addr 172.21.15.133:0/1978222731 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3653fff640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3654101420 0x7f3654197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3659622640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3654106a80 0x7f365419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3658e21640 1 -- 172.21.15.133:0/1978222731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3654106a80 msgr2=0x7f365419bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3658e21640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3654106a80 0x7f365419bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3658e21640 1 -- 172.21.15.133:0/1978222731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3654101420 msgr2=0x7f3654197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3658e21640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3654101420 0x7f3654197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3658e21640 1 -- 172.21.15.133:0/1978222731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365419c2e0 con 0x7f3654100a80 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3653fff640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3654101420 0x7f3654197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.245+0000 7f3658e21640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3654100a80 0x7f3654197330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f364800d950 tx=0x7f364800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.246+0000 7f3651ffb640 1 -- 172.21.15.133:0/1978222731 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3648017030 con 0x7f3654100a80 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.246+0000 7f365b085640 1 -- 172.21.15.133:0/1978222731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365419c5d0 con 0x7f3654100a80 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.246+0000 7f3651ffb640 1 -- 172.21.15.133:0/1978222731 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3648004020 con 0x7f3654100a80 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.246+0000 7f3651ffb640 1 -- 172.21.15.133:0/1978222731 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36480047a0 con 0x7f3654100a80 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.246+0000 7f365b085640 1 -- 172.21.15.133:0/1978222731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36541a3eb0 con 0x7f3654100a80 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.247+0000 7f365b085640 1 -- 172.21.15.133:0/1978222731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36541021c0 con 0x7f3654100a80 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.248+0000 7f3651ffb640 1 -- 172.21.15.133:0/1978222731 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f36480041d0 con 0x7f3654100a80 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.248+0000 7f3651ffb640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f36440a2c10 0x7f36440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.248+0000 7f3653fff640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f36440a2c10 0x7f36440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.248+0000 7f3651ffb640 1 -- 172.21.15.133:0/1978222731 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f36480ceb40 con 0x7f3654100a80 2024-09-11T23:35:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.249+0000 7f3653fff640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f36440a2c10 0x7f36440a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3640000c00 tx=0x7f3640005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:15.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.253+0000 7f3651ffb640 1 -- 172.21.15.133:0/1978222731 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f364809ba40 con 0x7f3654100a80 2024-09-11T23:35:15.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:14 smithi178 ceph-mon[35882]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:35:15.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:14 smithi178 ceph-mon[35882]: pgmap v284: 97 pgs: 97 active+clean; 605 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.6 KiB/s wr, 19 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:35:15.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.454+0000 7f365b085640 1 -- 172.21.15.133:0/1978222731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3654109010 con 0x7f3654100a80 2024-09-11T23:35:15.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.455+0000 7f3651ffb640 1 -- 172.21.15.133:0/1978222731 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f364809b340 con 0x7f3654100a80 2024-09-11T23:35:15.456 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:35:15.456 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"btime":"2024-09-11T23:35:05:451373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:35:05.451342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24472,24464,24419,24401]},"id":1}]} 2024-09-11T23:35:15.457 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-09-11T23:35:15.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 -- 172.21.15.133:0/1978222731 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f36440a2c10 msgr2=0x7f36440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:15.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f36440a2c10 0x7f36440a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3640000c00 tx=0x7f3640005a20 comp rx=0 tx=0).stop 2024-09-11T23:35:15.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 -- 172.21.15.133:0/1978222731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3654100a80 msgr2=0x7f3654197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:15.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3654100a80 0x7f3654197330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f364800d950 tx=0x7f364800de10 comp rx=0 tx=0).stop 2024-09-11T23:35:15.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 -- 172.21.15.133:0/1978222731 shutdown_connections 2024-09-11T23:35:15.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f36440a2c10 0x7f36440a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:15.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3654106a80 0x7f365419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:15.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3654101420 0x7f3654197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:15.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 --2- 172.21.15.133:0/1978222731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3654100a80 0x7f3654197330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:15.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 -- 172.21.15.133:0/1978222731 >> 172.21.15.133:0/1978222731 conn(0x7f36540fc7a0 msgr2=0x7f36540fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:15.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 -- 172.21.15.133:0/1978222731 shutdown_connections 2024-09-11T23:35:15.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:15.458+0000 7f365b085640 1 -- 172.21.15.133:0/1978222731 wait complete. 2024-09-11T23:35:16.077 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:15 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1978222731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:35:16.105 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:35:16.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:15 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1978222731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:35:16.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:15 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1978222731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:35:16.349 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:16.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.827+0000 7fa6771e5640 1 -- 172.21.15.133:0/2560068009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa670102790 msgr2=0x7fa6701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:16.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.827+0000 7fa6771e5640 1 --2- 172.21.15.133:0/2560068009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa670102790 0x7fa6701067e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa660009ca0 tx=0x7fa66002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:35:16.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.827+0000 7fa6771e5640 1 -- 172.21.15.133:0/2560068009 shutdown_connections 2024-09-11T23:35:16.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.827+0000 7fa6771e5640 1 --2- 172.21.15.133:0/2560068009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa670106f10 0x7fa67010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:16.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.827+0000 7fa6771e5640 1 --2- 172.21.15.133:0/2560068009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa670102790 0x7fa6701067e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:16.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.827+0000 7fa6771e5640 1 --2- 172.21.15.133:0/2560068009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa670101de0 0x7fa6701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:16.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.827+0000 7fa6771e5640 1 -- 172.21.15.133:0/2560068009 >> 172.21.15.133:0/2560068009 conn(0x7fa6700fd8c0 msgr2=0x7fa6700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:16.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.827+0000 7fa6771e5640 1 -- 172.21.15.133:0/2560068009 shutdown_connections 2024-09-11T23:35:16.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.827+0000 7fa6771e5640 1 -- 172.21.15.133:0/2560068009 wait complete. 2024-09-11T23:35:16.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.828+0000 7fa6771e5640 1 Processor -- start 2024-09-11T23:35:16.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.828+0000 7fa6771e5640 1 -- start start 2024-09-11T23:35:16.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa6771e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa670101de0 0x7fa670198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:16.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa6771e5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa670102790 0x7fa6701989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:16.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa6771e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa670106f10 0x7fa67019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:16.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa6771e5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa670111fb0 con 0x7fa670101de0 2024-09-11T23:35:16.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa6771e5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa670111e30 con 0x7fa670106f10 2024-09-11T23:35:16.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa6771e5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa670112130 con 0x7fa670102790 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa674f81640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa670101de0 0x7fa670198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa674f81640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa670101de0 0x7fa670198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43164/0 (socket says 172.21.15.133:43164) 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa674f81640 1 -- 172.21.15.133:0/2456507958 learned_addr learned my addr 172.21.15.133:0/2456507958 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa66ffff640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa670102790 0x7fa6701989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa674f81640 1 -- 172.21.15.133:0/2456507958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa670102790 msgr2=0x7fa6701989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa674f81640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa670102790 0x7fa6701989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa674f81640 1 -- 172.21.15.133:0/2456507958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa670106f10 msgr2=0x7fa67019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa674f81640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa670106f10 0x7fa67019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa674f81640 1 -- 172.21.15.133:0/2456507958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67019d440 con 0x7fa670101de0 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa66ffff640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa670102790 0x7fa6701989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa674f81640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa670101de0 0x7fa670198490 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c970 tx=0x7fa65800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa66dffb640 1 -- 172.21.15.133:0/2456507958 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658007bb0 con 0x7fa670101de0 2024-09-11T23:35:16.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa66dffb640 1 -- 172.21.15.133:0/2456507958 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa658007d50 con 0x7fa670101de0 2024-09-11T23:35:16.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa66dffb640 1 -- 172.21.15.133:0/2456507958 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65800f450 con 0x7fa670101de0 2024-09-11T23:35:16.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.829+0000 7fa6771e5640 1 -- 172.21.15.133:0/2456507958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67019d730 con 0x7fa670101de0 2024-09-11T23:35:16.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.830+0000 7fa6771e5640 1 -- 172.21.15.133:0/2456507958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6701a5010 con 0x7fa670101de0 2024-09-11T23:35:16.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.830+0000 7fa6771e5640 1 -- 172.21.15.133:0/2456507958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6701021c0 con 0x7fa670101de0 2024-09-11T23:35:16.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.831+0000 7fa66dffb640 1 -- 172.21.15.133:0/2456507958 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa658005020 con 0x7fa670101de0 2024-09-11T23:35:16.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.832+0000 7fa66dffb640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa65c0a2c90 0x7fa65c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:16.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.832+0000 7fa66ffff640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa65c0a2c90 0x7fa65c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:16.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.832+0000 7fa66dffb640 1 -- 172.21.15.133:0/2456507958 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fa6580cefe0 con 0x7fa670101de0 2024-09-11T23:35:16.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.833+0000 7fa66ffff640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa65c0a2c90 0x7fa65c0a5150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa6700fdce0 tx=0x7fa660000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:16.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:16.836+0000 7fa66dffb640 1 -- 172.21.15.133:0/2456507958 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa658014030 con 0x7fa670101de0 2024-09-11T23:35:17.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.007+0000 7fa6771e5640 1 -- 172.21.15.133:0/2456507958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6701997f0 con 0x7fa670101de0 2024-09-11T23:35:17.007 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:16 smithi133 ceph-mon[31430]: pgmap v285: 97 pgs: 97 active+clean; 624 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 14 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:35:17.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.008+0000 7fa66dffb640 1 -- 172.21.15.133:0/2456507958 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22677 (secure 0 0 0) 0x7fa65809bde0 con 0x7fa670101de0 2024-09-11T23:35:17.008 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:35:17.008 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":81,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:35:08.840852+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:35:17.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 -- 172.21.15.133:0/2456507958 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa65c0a2c90 msgr2=0x7fa65c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:17.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa65c0a2c90 0x7fa65c0a5150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa6700fdce0 tx=0x7fa660000f40 comp rx=0 tx=0).stop 2024-09-11T23:35:17.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 -- 172.21.15.133:0/2456507958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa670101de0 msgr2=0x7fa670198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:17.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa670101de0 0x7fa670198490 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c970 tx=0x7fa65800ce30 comp rx=0 tx=0).stop 2024-09-11T23:35:17.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 -- 172.21.15.133:0/2456507958 shutdown_connections 2024-09-11T23:35:17.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa65c0a2c90 0x7fa65c0a5150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:17.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa670106f10 0x7fa67019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:17.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa670102790 0x7fa6701989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:17.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 --2- 172.21.15.133:0/2456507958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa670101de0 0x7fa670198490 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:17.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 -- 172.21.15.133:0/2456507958 >> 172.21.15.133:0/2456507958 conn(0x7fa6700fd8c0 msgr2=0x7fa6700fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:17.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 -- 172.21.15.133:0/2456507958 shutdown_connections 2024-09-11T23:35:17.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:17.011+0000 7fa6771e5640 1 -- 172.21.15.133:0/2456507958 wait complete. 2024-09-11T23:35:17.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:16 smithi138 ceph-mon[35142]: pgmap v285: 97 pgs: 97 active+clean; 624 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 14 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:35:17.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:16 smithi178 ceph-mon[35882]: pgmap v285: 97 pgs: 97 active+clean; 624 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 14 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:35:17.916 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:35:17.916 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:35:18.161 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:18.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:17 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2456507958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:35:18.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:17 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2456507958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:35:18.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:17 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2456507958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:35:18.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.601+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2732616390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0103660 msgr2=0x7fe0e010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.601+0000 7fe0e4bf8640 1 --2- 172.21.15.133:0/2732616390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0103660 0x7fe0e010bb50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009900 tx=0x7fe0d402a480 comp rx=0 tx=0).stop 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.601+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2732616390 shutdown_connections 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.601+0000 7fe0e4bf8640 1 --2- 172.21.15.133:0/2732616390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e010c170 0x7fe0e010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.601+0000 7fe0e4bf8640 1 --2- 172.21.15.133:0/2732616390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0103660 0x7fe0e010bb50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.601+0000 7fe0e4bf8640 1 --2- 172.21.15.133:0/2732616390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0e0102d40 0x7fe0e0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.601+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2732616390 >> 172.21.15.133:0/2732616390 conn(0x7fe0e00fc7a0 msgr2=0x7fe0e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.601+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2732616390 shutdown_connections 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.601+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2732616390 wait complete. 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.601+0000 7fe0e4bf8640 1 Processor -- start 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0e4bf8640 1 -- start start 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0e4bf8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0e0102d40 0x7fe0e01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:18.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0e4bf8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e0103660 0x7fe0e0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:18.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0e4bf8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e010c170 0x7fe0e0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:18.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0e4bf8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e0111fb0 con 0x7fe0e010c170 2024-09-11T23:35:18.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0e4bf8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e0111e30 con 0x7fe0e0103660 2024-09-11T23:35:18.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0e4bf8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e0112130 con 0x7fe0e0102d40 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0ded9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e010c170 0x7fe0e0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0ded9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e010c170 0x7fe0e0101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43188/0 (socket says 172.21.15.133:43188) 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0ded9d640 1 -- 172.21.15.133:0/2651874546 learned_addr learned my addr 172.21.15.133:0/2651874546 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0ddd9b640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e0103660 0x7fe0e0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0de59c640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0e0102d40 0x7fe0e01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0ded9d640 1 -- 172.21.15.133:0/2651874546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0e0102d40 msgr2=0x7fe0e01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0ded9d640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0e0102d40 0x7fe0e01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0ded9d640 1 -- 172.21.15.133:0/2651874546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e0103660 msgr2=0x7fe0e0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0ded9d640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e0103660 0x7fe0e0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0ded9d640 1 -- 172.21.15.133:0/2651874546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e01a6920 con 0x7fe0e010c170 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0de59c640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0e0102d40 0x7fe0e01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.602+0000 7fe0ded9d640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e010c170 0x7fe0e0101420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800c970 tx=0x7fe0d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.603+0000 7fe0cf7fe640 1 -- 172.21.15.133:0/2651874546 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8007bb0 con 0x7fe0e010c170 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.603+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2651874546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e01a6c10 con 0x7fe0e010c170 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.603+0000 7fe0cf7fe640 1 -- 172.21.15.133:0/2651874546 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0d8007d50 con 0x7fe0e010c170 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.603+0000 7fe0cf7fe640 1 -- 172.21.15.133:0/2651874546 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8018800 con 0x7fe0e010c170 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.603+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2651874546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e01a7150 con 0x7fe0e010c170 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.604+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2651874546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e0103120 con 0x7fe0e010c170 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.605+0000 7fe0cf7fe640 1 -- 172.21.15.133:0/2651874546 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe0d800fa50 con 0x7fe0e010c170 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.605+0000 7fe0cf7fe640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0c80a2c10 0x7fe0c80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.605+0000 7fe0de59c640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0c80a2c10 0x7fe0c80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.606+0000 7fe0cf7fe640 1 -- 172.21.15.133:0/2651874546 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fe0d80cf2e0 con 0x7fe0e010c170 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.606+0000 7fe0de59c640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0c80a2c10 0x7fe0c80a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0009fa0 tx=0x7fe0d0009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:18.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.610+0000 7fe0cf7fe640 1 -- 172.21.15.133:0/2651874546 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0d8014030 con 0x7fe0e010c170 2024-09-11T23:35:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.772+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2651874546 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0e0062630 con 0x7fe0c80a2c10 2024-09-11T23:35:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.779+0000 7fe0cf7fe640 1 -- 172.21.15.133:0/2651874546 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258761 (secure 0 0 0) 0x7fe0e0062630 con 0x7fe0c80a2c10 2024-09-11T23:35:18.781 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:35:18.783 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T23:35:18.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2651874546 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0c80a2c10 msgr2=0x7fe0c80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:18.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0c80a2c10 0x7fe0c80a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0009fa0 tx=0x7fe0d0009570 comp rx=0 tx=0).stop 2024-09-11T23:35:18.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2651874546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e010c170 msgr2=0x7fe0e0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:18.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e010c170 0x7fe0e0101420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800c970 tx=0x7fe0d800ce30 comp rx=0 tx=0).stop 2024-09-11T23:35:18.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2651874546 shutdown_connections 2024-09-11T23:35:18.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0c80a2c10 0x7fe0c80a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:18.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e010c170 0x7fe0e0101420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:18.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e0103660 0x7fe0e0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:18.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 --2- 172.21.15.133:0/2651874546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0e0102d40 0x7fe0e01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:18.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2651874546 >> 172.21.15.133:0/2651874546 conn(0x7fe0e00fc7a0 msgr2=0x7fe0e00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:18.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2651874546 shutdown_connections 2024-09-11T23:35:18.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:18.782+0000 7fe0e4bf8640 1 -- 172.21.15.133:0/2651874546 wait complete. 2024-09-11T23:35:19.191 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:18 smithi178 ceph-mon[35882]: pgmap v286: 97 pgs: 97 active+clean; 624 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.1 KiB/s wr, 13 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:35:19.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:18 smithi138 ceph-mon[35142]: pgmap v286: 97 pgs: 97 active+clean; 624 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.1 KiB/s wr, 13 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:35:19.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:18 smithi133 ceph-mon[31430]: pgmap v286: 97 pgs: 97 active+clean; 624 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.1 KiB/s wr, 13 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:35:19.545 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-11T23:35:17.850024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638875,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":289,"num_read_kb":247,"num_write":415,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357704,"kb_used_data":12592,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584120,"statfs":{"total":1151940427776,"available":1151574138880,"internally_reserved":0,"allocated":12894208,"data_stored":7131729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40473,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":89,"num_write":32,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004538"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845014+0000","last_change":"2024-09-11T23:34:52.867694+0000","last_active":"2024-09-11T23:35:08.845014+0000","last_peered":"2024-09-11T23:35:08.845014+0000","last_clean":"2024-09-11T23:35:08.845014+0000","last_became_active":"2024-09-11T23:34:52.867368+0000","last_became_peered":"2024-09-11T23:34:52.867368+0000","last_unstale":"2024-09-11T23:35:08.845014+0000","last_undegraded":"2024-09-11T23:35:08.845014+0000","last_fullsized":"2024-09-11T23:35:08.845014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:46:07.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872241+0000","last_change":"2024-09-11T23:34:52.871881+0000","last_active":"2024-09-11T23:35:08.872241+0000","last_peered":"2024-09-11T23:35:08.872241+0000","last_clean":"2024-09-11T23:35:08.872241+0000","last_became_active":"2024-09-11T23:34:52.871609+0000","last_became_peered":"2024-09-11T23:34:52.871609+0000","last_unstale":"2024-09-11T23:35:08.872241+0000","last_undegraded":"2024-09-11T23:35:08.872241+0000","last_fullsized":"2024-09-11T23:35:08.872241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.938584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872054+0000","last_change":"2024-09-11T23:34:52.863931+0000","last_active":"2024-09-11T23:35:08.872054+0000","last_peered":"2024-09-11T23:35:08.872054+0000","last_clean":"2024-09-11T23:35:08.872054+0000","last_became_active":"2024-09-11T23:34:52.863637+0000","last_became_peered":"2024-09-11T23:34:52.863637+0000","last_unstale":"2024-09-11T23:35:08.872054+0000","last_undegraded":"2024-09-11T23:35:08.872054+0000","last_fullsized":"2024-09-11T23:35:08.872054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:44.118865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846293+0000","last_change":"2024-09-11T23:34:52.879619+0000","last_active":"2024-09-11T23:35:08.846293+0000","last_peered":"2024-09-11T23:35:08.846293+0000","last_clean":"2024-09-11T23:35:08.846293+0000","last_became_active":"2024-09-11T23:34:52.878695+0000","last_became_peered":"2024-09-11T23:34:52.878695+0000","last_unstale":"2024-09-11T23:35:08.846293+0000","last_undegraded":"2024-09-11T23:35:08.846293+0000","last_fullsized":"2024-09-11T23:35:08.846293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:30.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847368+0000","last_change":"2024-09-11T23:34:52.869363+0000","last_active":"2024-09-11T23:35:08.847368+0000","last_peered":"2024-09-11T23:35:08.847368+0000","last_clean":"2024-09-11T23:35:08.847368+0000","last_became_active":"2024-09-11T23:34:52.869027+0000","last_became_peered":"2024-09-11T23:34:52.869027+0000","last_unstale":"2024-09-11T23:35:08.847368+0000","last_undegraded":"2024-09-11T23:35:08.847368+0000","last_fullsized":"2024-09-11T23:35:08.847368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:15.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848328+0000","last_change":"2024-09-11T23:34:52.880665+0000","last_active":"2024-09-11T23:35:08.848328+0000","last_peered":"2024-09-11T23:35:08.848328+0000","last_clean":"2024-09-11T23:35:08.848328+0000","last_became_active":"2024-09-11T23:34:52.880172+0000","last_became_peered":"2024-09-11T23:34:52.880172+0000","last_unstale":"2024-09-11T23:35:08.848328+0000","last_undegraded":"2024-09-11T23:35:08.848328+0000","last_fullsized":"2024-09-11T23:35:08.848328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:12.025877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846502+0000","last_change":"2024-09-11T23:34:52.873897+0000","last_active":"2024-09-11T23:35:08.846502+0000","last_peered":"2024-09-11T23:35:08.846502+0000","last_clean":"2024-09-11T23:35:08.846502+0000","last_became_active":"2024-09-11T23:34:52.873219+0000","last_became_peered":"2024-09-11T23:34:52.873219+0000","last_unstale":"2024-09-11T23:35:08.846502+0000","last_undegraded":"2024-09-11T23:35:08.846502+0000","last_fullsized":"2024-09-11T23:35:08.846502+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:47.125682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847314+0000","last_change":"2024-09-11T23:34:52.870699+0000","last_active":"2024-09-11T23:35:08.847314+0000","last_peered":"2024-09-11T23:35:08.847314+0000","last_clean":"2024-09-11T23:35:08.847314+0000","last_became_active":"2024-09-11T23:34:52.870314+0000","last_became_peered":"2024-09-11T23:34:52.870314+0000","last_unstale":"2024-09-11T23:35:08.847314+0000","last_undegraded":"2024-09-11T23:35:08.847314+0000","last_fullsized":"2024-09-11T23:35:08.847314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:41.263159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.851159+0000","last_change":"2024-09-11T23:34:52.869700+0000","last_active":"2024-09-11T23:35:08.851159+0000","last_peered":"2024-09-11T23:35:08.851159+0000","last_clean":"2024-09-11T23:35:08.851159+0000","last_became_active":"2024-09-11T23:34:52.869398+0000","last_became_peered":"2024-09-11T23:34:52.869398+0000","last_unstale":"2024-09-11T23:35:08.851159+0000","last_undegraded":"2024-09-11T23:35:08.851159+0000","last_fullsized":"2024-09-11T23:35:08.851159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:49.776525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848589+0000","last_change":"2024-09-11T23:34:52.879999+0000","last_active":"2024-09-11T23:35:08.848589+0000","last_peered":"2024-09-11T23:35:08.848589+0000","last_clean":"2024-09-11T23:35:08.848589+0000","last_became_active":"2024-09-11T23:34:52.879646+0000","last_became_peered":"2024-09-11T23:34:52.879646+0000","last_unstale":"2024-09-11T23:35:08.848589+0000","last_undegraded":"2024-09-11T23:35:08.848589+0000","last_fullsized":"2024-09-11T23:35:08.848589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.058704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846881+0000","last_change":"2024-09-11T23:34:52.878740+0000","last_active":"2024-09-11T23:35:08.846881+0000","last_peered":"2024-09-11T23:35:08.846881+0000","last_clean":"2024-09-11T23:35:08.846881+0000","last_became_active":"2024-09-11T23:34:52.878120+0000","last_became_peered":"2024-09-11T23:34:52.878120+0000","last_unstale":"2024-09-11T23:35:08.846881+0000","last_undegraded":"2024-09-11T23:35:08.846881+0000","last_fullsized":"2024-09-11T23:35:08.846881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846051+0000","last_change":"2024-09-11T23:34:52.873409+0000","last_active":"2024-09-11T23:35:08.846051+0000","last_peered":"2024-09-11T23:35:08.846051+0000","last_clean":"2024-09-11T23:35:08.846051+0000","last_became_active":"2024-09-11T23:34:52.872959+0000","last_became_peered":"2024-09-11T23:34:52.872959+0000","last_unstale":"2024-09-11T23:35:08.846051+0000","last_undegraded":"2024-09-11T23:35:08.846051+0000","last_fullsized":"2024-09-11T23:35:08.846051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:38.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844266+0000","last_change":"2024-09-11T23:34:52.869164+0000","last_active":"2024-09-11T23:35:08.844266+0000","last_peered":"2024-09-11T23:35:08.844266+0000","last_clean":"2024-09-11T23:35:08.844266+0000","last_became_active":"2024-09-11T23:34:52.868847+0000","last_became_peered":"2024-09-11T23:34:52.868847+0000","last_unstale":"2024-09-11T23:35:08.844266+0000","last_undegraded":"2024-09-11T23:35:08.844266+0000","last_fullsized":"2024-09-11T23:35:08.844266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.764510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872027+0000","last_change":"2024-09-11T23:34:52.869494+0000","last_active":"2024-09-11T23:35:08.872027+0000","last_peered":"2024-09-11T23:35:08.872027+0000","last_clean":"2024-09-11T23:35:08.872027+0000","last_became_active":"2024-09-11T23:34:52.869178+0000","last_became_peered":"2024-09-11T23:34:52.869178+0000","last_unstale":"2024-09-11T23:35:08.872027+0000","last_undegraded":"2024-09-11T23:35:08.872027+0000","last_fullsized":"2024-09-11T23:35:08.872027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:23.431564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872033+0000","last_change":"2024-09-11T23:34:52.866269+0000","last_active":"2024-09-11T23:35:08.872033+0000","last_peered":"2024-09-11T23:35:08.872033+0000","last_clean":"2024-09-11T23:35:08.872033+0000","last_became_active":"2024-09-11T23:34:52.865980+0000","last_became_peered":"2024-09-11T23:34:52.865980+0000","last_unstale":"2024-09-11T23:35:08.872033+0000","last_undegraded":"2024-09-11T23:35:08.872033+0000","last_fullsized":"2024-09-11T23:35:08.872033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:48.492252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850466+0000","last_change":"2024-09-11T23:34:52.862192+0000","last_active":"2024-09-11T23:35:08.850466+0000","last_peered":"2024-09-11T23:35:08.850466+0000","last_clean":"2024-09-11T23:35:08.850466+0000","last_became_active":"2024-09-11T23:34:52.861841+0000","last_became_peered":"2024-09-11T23:34:52.861841+0000","last_unstale":"2024-09-11T23:35:08.850466+0000","last_undegraded":"2024-09-11T23:35:08.850466+0000","last_fullsized":"2024-09-11T23:35:08.850466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.258948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845097+0000","last_change":"2024-09-11T23:34:52.870984+0000","last_active":"2024-09-11T23:35:08.845097+0000","last_peered":"2024-09-11T23:35:08.845097+0000","last_clean":"2024-09-11T23:35:08.845097+0000","last_became_active":"2024-09-11T23:34:52.870435+0000","last_became_peered":"2024-09-11T23:34:52.870435+0000","last_unstale":"2024-09-11T23:35:08.845097+0000","last_undegraded":"2024-09-11T23:35:08.845097+0000","last_fullsized":"2024-09-11T23:35:08.845097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:34.391537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847466+0000","last_change":"2024-09-11T23:34:52.873176+0000","last_active":"2024-09-11T23:35:08.847466+0000","last_peered":"2024-09-11T23:35:08.847466+0000","last_clean":"2024-09-11T23:35:08.847466+0000","last_became_active":"2024-09-11T23:34:52.872862+0000","last_became_peered":"2024-09-11T23:34:52.872862+0000","last_unstale":"2024-09-11T23:35:08.847466+0000","last_undegraded":"2024-09-11T23:35:08.847466+0000","last_fullsized":"2024-09-11T23:35:08.847466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:38.058806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847715+0000","last_change":"2024-09-11T23:34:52.871346+0000","last_active":"2024-09-11T23:35:08.847715+0000","last_peered":"2024-09-11T23:35:08.847715+0000","last_clean":"2024-09-11T23:35:08.847715+0000","last_became_active":"2024-09-11T23:34:52.870981+0000","last_became_peered":"2024-09-11T23:34:52.870981+0000","last_unstale":"2024-09-11T23:35:08.847715+0000","last_undegraded":"2024-09-11T23:35:08.847715+0000","last_fullsized":"2024-09-11T23:35:08.847715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:56.973998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844737+0000","last_change":"2024-09-11T23:34:52.867058+0000","last_active":"2024-09-11T23:35:08.844737+0000","last_peered":"2024-09-11T23:35:08.844737+0000","last_clean":"2024-09-11T23:35:08.844737+0000","last_became_active":"2024-09-11T23:34:52.866687+0000","last_became_peered":"2024-09-11T23:34:52.866687+0000","last_unstale":"2024-09-11T23:35:08.844737+0000","last_undegraded":"2024-09-11T23:35:08.844737+0000","last_fullsized":"2024-09-11T23:35:08.844737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:09.019987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844347+0000","last_change":"2024-09-11T23:34:52.870882+0000","last_active":"2024-09-11T23:35:08.844347+0000","last_peered":"2024-09-11T23:35:08.844347+0000","last_clean":"2024-09-11T23:35:08.844347+0000","last_became_active":"2024-09-11T23:34:52.870667+0000","last_became_peered":"2024-09-11T23:34:52.870667+0000","last_unstale":"2024-09-11T23:35:08.844347+0000","last_undegraded":"2024-09-11T23:35:08.844347+0000","last_fullsized":"2024-09-11T23:35:08.844347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:08.386703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850359+0000","last_change":"2024-09-11T23:34:52.874070+0000","last_active":"2024-09-11T23:35:08.850359+0000","last_peered":"2024-09-11T23:35:08.850359+0000","last_clean":"2024-09-11T23:35:08.850359+0000","last_became_active":"2024-09-11T23:34:52.873772+0000","last_became_peered":"2024-09-11T23:34:52.873772+0000","last_unstale":"2024-09-11T23:35:08.850359+0000","last_undegraded":"2024-09-11T23:35:08.850359+0000","last_fullsized":"2024-09-11T23:35:08.850359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:49.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847766+0000","last_change":"2024-09-11T23:34:52.875502+0000","last_active":"2024-09-11T23:35:08.847766+0000","last_peered":"2024-09-11T23:35:08.847766+0000","last_clean":"2024-09-11T23:35:08.847766+0000","last_became_active":"2024-09-11T23:34:52.875193+0000","last_became_peered":"2024-09-11T23:34:52.875193+0000","last_unstale":"2024-09-11T23:35:08.847766+0000","last_undegraded":"2024-09-11T23:35:08.847766+0000","last_fullsized":"2024-09-11T23:35:08.847766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:37.213828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455450+0000","last_change":"2024-09-11T23:34:52.869015+0000","last_active":"2024-09-11T23:35:09.455450+0000","last_peered":"2024-09-11T23:35:09.455450+0000","last_clean":"2024-09-11T23:35:09.455450+0000","last_became_active":"2024-09-11T23:34:52.868731+0000","last_became_peered":"2024-09-11T23:34:52.868731+0000","last_unstale":"2024-09-11T23:35:09.455450+0000","last_undegraded":"2024-09-11T23:35:09.455450+0000","last_fullsized":"2024-09-11T23:35:09.455450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:12:15.993203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848557+0000","last_change":"2024-09-11T23:34:52.875555+0000","last_active":"2024-09-11T23:35:08.848557+0000","last_peered":"2024-09-11T23:35:08.848557+0000","last_clean":"2024-09-11T23:35:08.848557+0000","last_became_active":"2024-09-11T23:34:52.875219+0000","last_became_peered":"2024-09-11T23:34:52.875219+0000","last_unstale":"2024-09-11T23:35:08.848557+0000","last_undegraded":"2024-09-11T23:35:08.848557+0000","last_fullsized":"2024-09-11T23:35:08.848557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:16.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848205+0000","last_change":"2024-09-11T23:34:52.875629+0000","last_active":"2024-09-11T23:35:08.848205+0000","last_peered":"2024-09-11T23:35:08.848205+0000","last_clean":"2024-09-11T23:35:08.848205+0000","last_became_active":"2024-09-11T23:34:52.875302+0000","last_became_peered":"2024-09-11T23:34:52.875302+0000","last_unstale":"2024-09-11T23:35:08.848205+0000","last_undegraded":"2024-09-11T23:35:08.848205+0000","last_fullsized":"2024-09-11T23:35:08.848205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:12.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844886+0000","last_change":"2024-09-11T23:34:52.868613+0000","last_active":"2024-09-11T23:35:08.844886+0000","last_peered":"2024-09-11T23:35:08.844886+0000","last_clean":"2024-09-11T23:35:08.844886+0000","last_became_active":"2024-09-11T23:34:52.868184+0000","last_became_peered":"2024-09-11T23:34:52.868184+0000","last_unstale":"2024-09-11T23:35:08.844886+0000","last_undegraded":"2024-09-11T23:35:08.844886+0000","last_fullsized":"2024-09-11T23:35:08.844886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:25.990002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.873390+0000","last_change":"2024-09-11T23:35:08.873390+0000","last_active":"2024-09-11T23:35:08.873390+0000","last_peered":"2024-09-11T23:35:08.873390+0000","last_clean":"2024-09-11T23:35:08.873390+0000","last_became_active":"2024-09-11T23:35:08.873052+0000","last_became_peered":"2024-09-11T23:35:08.873052+0000","last_unstale":"2024-09-11T23:35:08.873390+0000","last_undegraded":"2024-09-11T23:35:08.873390+0000","last_fullsized":"2024-09-11T23:35:08.873390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.937951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872829+0000","last_change":"2024-09-11T23:34:52.873604+0000","last_active":"2024-09-11T23:35:08.872829+0000","last_peered":"2024-09-11T23:35:08.872829+0000","last_clean":"2024-09-11T23:35:08.872829+0000","last_became_active":"2024-09-11T23:34:52.873220+0000","last_became_peered":"2024-09-11T23:34:52.873220+0000","last_unstale":"2024-09-11T23:35:08.872829+0000","last_undegraded":"2024-09-11T23:35:08.872829+0000","last_fullsized":"2024-09-11T23:35:08.872829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:38.480630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847599+0000","last_change":"2024-09-11T23:34:52.872902+0000","last_active":"2024-09-11T23:35:08.847599+0000","last_peered":"2024-09-11T23:35:08.847599+0000","last_clean":"2024-09-11T23:35:08.847599+0000","last_became_active":"2024-09-11T23:34:52.872571+0000","last_became_peered":"2024-09-11T23:34:52.872571+0000","last_unstale":"2024-09-11T23:35:08.847599+0000","last_undegraded":"2024-09-11T23:35:08.847599+0000","last_fullsized":"2024-09-11T23:35:08.847599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:21.844995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.457056+0000","last_change":"2024-09-11T23:35:09.457056+0000","last_active":"2024-09-11T23:35:09.457056+0000","last_peered":"2024-09-11T23:35:09.457056+0000","last_clean":"2024-09-11T23:35:09.457056+0000","last_became_active":"2024-09-11T23:35:09.456657+0000","last_became_peered":"2024-09-11T23:35:09.456657+0000","last_unstale":"2024-09-11T23:35:09.457056+0000","last_undegraded":"2024-09-11T23:35:09.457056+0000","last_fullsized":"2024-09-11T23:35:09.457056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:12:56.121280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844633+0000","last_change":"2024-09-11T23:34:52.875312+0000","last_active":"2024-09-11T23:35:08.844633+0000","last_peered":"2024-09-11T23:35:08.844633+0000","last_clean":"2024-09-11T23:35:08.844633+0000","last_became_active":"2024-09-11T23:34:52.874873+0000","last_became_peered":"2024-09-11T23:34:52.874873+0000","last_unstale":"2024-09-11T23:35:08.844633+0000","last_undegraded":"2024-09-11T23:35:08.844633+0000","last_fullsized":"2024-09-11T23:35:08.844633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:18.894713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847244+0000","last_change":"2024-09-11T23:34:52.862266+0000","last_active":"2024-09-11T23:35:08.847244+0000","last_peered":"2024-09-11T23:35:08.847244+0000","last_clean":"2024-09-11T23:35:08.847244+0000","last_became_active":"2024-09-11T23:34:52.861923+0000","last_became_peered":"2024-09-11T23:34:52.861923+0000","last_unstale":"2024-09-11T23:35:08.847244+0000","last_undegraded":"2024-09-11T23:35:08.847244+0000","last_fullsized":"2024-09-11T23:35:08.847244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:23.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:14.462701+0000","last_change":"2024-09-11T23:34:50.854020+0000","last_active":"2024-09-11T23:35:14.462701+0000","last_peered":"2024-09-11T23:35:14.462701+0000","last_clean":"2024-09-11T23:35:14.462701+0000","last_became_active":"2024-09-11T23:34:50.853683+0000","last_became_peered":"2024-09-11T23:34:50.853683+0000","last_unstale":"2024-09-11T23:35:14.462701+0000","last_undegraded":"2024-09-11T23:35:14.462701+0000","last_fullsized":"2024-09-11T23:35:14.462701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:18.725495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455234+0000","last_change":"2024-09-11T23:34:52.874016+0000","last_active":"2024-09-11T23:35:09.455234+0000","last_peered":"2024-09-11T23:35:09.455234+0000","last_clean":"2024-09-11T23:35:09.455234+0000","last_became_active":"2024-09-11T23:34:52.873675+0000","last_became_peered":"2024-09-11T23:34:52.873675+0000","last_unstale":"2024-09-11T23:35:09.455234+0000","last_undegraded":"2024-09-11T23:35:09.455234+0000","last_fullsized":"2024-09-11T23:35:09.455234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:41:28.955548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848555+0000","last_change":"2024-09-11T23:34:50.855391+0000","last_active":"2024-09-11T23:35:08.848555+0000","last_peered":"2024-09-11T23:35:08.848555+0000","last_clean":"2024-09-11T23:35:08.848555+0000","last_became_active":"2024-09-11T23:34:50.855052+0000","last_became_peered":"2024-09-11T23:34:50.855052+0000","last_unstale":"2024-09-11T23:35:08.848555+0000","last_undegraded":"2024-09-11T23:35:08.848555+0000","last_fullsized":"2024-09-11T23:35:08.848555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.972239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844031+0000","last_change":"2024-09-11T23:34:50.848985+0000","last_active":"2024-09-11T23:35:08.844031+0000","last_peered":"2024-09-11T23:35:08.844031+0000","last_clean":"2024-09-11T23:35:08.844031+0000","last_became_active":"2024-09-11T23:34:50.848614+0000","last_became_peered":"2024-09-11T23:34:50.848614+0000","last_unstale":"2024-09-11T23:35:08.844031+0000","last_undegraded":"2024-09-11T23:35:08.844031+0000","last_fullsized":"2024-09-11T23:35:08.844031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:43:04.723136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847368+0000","last_change":"2024-09-11T23:34:52.866933+0000","last_active":"2024-09-11T23:35:08.847368+0000","last_peered":"2024-09-11T23:35:08.847368+0000","last_clean":"2024-09-11T23:35:08.847368+0000","last_became_active":"2024-09-11T23:34:52.866559+0000","last_became_peered":"2024-09-11T23:34:52.866559+0000","last_unstale":"2024-09-11T23:35:08.847368+0000","last_undegraded":"2024-09-11T23:35:08.847368+0000","last_fullsized":"2024-09-11T23:35:08.847368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:27.745118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456265+0000","last_change":"2024-09-11T23:34:50.849787+0000","last_active":"2024-09-11T23:35:09.456265+0000","last_peered":"2024-09-11T23:35:09.456265+0000","last_clean":"2024-09-11T23:35:09.456265+0000","last_became_active":"2024-09-11T23:34:50.849368+0000","last_became_peered":"2024-09-11T23:34:50.849368+0000","last_unstale":"2024-09-11T23:35:09.456265+0000","last_undegraded":"2024-09-11T23:35:09.456265+0000","last_fullsized":"2024-09-11T23:35:09.456265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:26.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847291+0000","last_change":"2024-09-11T23:34:52.870832+0000","last_active":"2024-09-11T23:35:08.847291+0000","last_peered":"2024-09-11T23:35:08.847291+0000","last_clean":"2024-09-11T23:35:08.847291+0000","last_became_active":"2024-09-11T23:34:52.870474+0000","last_became_peered":"2024-09-11T23:34:52.870474+0000","last_unstale":"2024-09-11T23:35:08.847291+0000","last_undegraded":"2024-09-11T23:35:08.847291+0000","last_fullsized":"2024-09-11T23:35:08.847291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:30.196114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":46,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:13.460750+0000","last_change":"2024-09-11T23:34:50.859074+0000","last_active":"2024-09-11T23:35:13.460750+0000","last_peered":"2024-09-11T23:35:13.460750+0000","last_clean":"2024-09-11T23:35:13.460750+0000","last_became_active":"2024-09-11T23:34:50.858664+0000","last_became_peered":"2024-09-11T23:34:50.858664+0000","last_unstale":"2024-09-11T23:35:13.460750+0000","last_undegraded":"2024-09-11T23:35:13.460750+0000","last_fullsized":"2024-09-11T23:35:13.460750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:04.077813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850853+0000","last_change":"2024-09-11T23:34:52.875901+0000","last_active":"2024-09-11T23:35:08.850853+0000","last_peered":"2024-09-11T23:35:08.850853+0000","last_clean":"2024-09-11T23:35:08.850853+0000","last_became_active":"2024-09-11T23:34:52.875568+0000","last_became_peered":"2024-09-11T23:34:52.875568+0000","last_unstale":"2024-09-11T23:35:08.850853+0000","last_undegraded":"2024-09-11T23:35:08.850853+0000","last_fullsized":"2024-09-11T23:35:08.850853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:54.173548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846938+0000","last_change":"2024-09-11T23:34:50.854957+0000","last_active":"2024-09-11T23:35:08.846938+0000","last_peered":"2024-09-11T23:35:08.846938+0000","last_clean":"2024-09-11T23:35:08.846938+0000","last_became_active":"2024-09-11T23:34:50.854626+0000","last_became_peered":"2024-09-11T23:34:50.854626+0000","last_unstale":"2024-09-11T23:35:08.846938+0000","last_undegraded":"2024-09-11T23:35:08.846938+0000","last_fullsized":"2024-09-11T23:35:08.846938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:53.738966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872227+0000","last_change":"2024-09-11T23:34:52.868850+0000","last_active":"2024-09-11T23:35:08.872227+0000","last_peered":"2024-09-11T23:35:08.872227+0000","last_clean":"2024-09-11T23:35:08.872227+0000","last_became_active":"2024-09-11T23:34:52.868536+0000","last_became_peered":"2024-09-11T23:34:52.868536+0000","last_unstale":"2024-09-11T23:35:08.872227+0000","last_undegraded":"2024-09-11T23:35:08.872227+0000","last_fullsized":"2024-09-11T23:35:08.872227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:11.746821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456837+0000","last_change":"2024-09-11T23:34:50.924819+0000","last_active":"2024-09-11T23:35:09.456837+0000","last_peered":"2024-09-11T23:35:09.456837+0000","last_clean":"2024-09-11T23:35:09.456837+0000","last_became_active":"2024-09-11T23:34:50.924283+0000","last_became_peered":"2024-09-11T23:34:50.924283+0000","last_unstale":"2024-09-11T23:35:09.456837+0000","last_undegraded":"2024-09-11T23:35:09.456837+0000","last_fullsized":"2024-09-11T23:35:09.456837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:41.345233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850516+0000","last_change":"2024-09-11T23:34:52.867113+0000","last_active":"2024-09-11T23:35:08.850516+0000","last_peered":"2024-09-11T23:35:08.850516+0000","last_clean":"2024-09-11T23:35:08.850516+0000","last_became_active":"2024-09-11T23:34:52.866125+0000","last_became_peered":"2024-09-11T23:34:52.866125+0000","last_unstale":"2024-09-11T23:35:08.850516+0000","last_undegraded":"2024-09-11T23:35:08.850516+0000","last_fullsized":"2024-09-11T23:35:08.850516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:18:55.615079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'23","reported_seq":73,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:13.461701+0000","last_change":"2024-09-11T23:34:50.857272+0000","last_active":"2024-09-11T23:35:13.461701+0000","last_peered":"2024-09-11T23:35:13.461701+0000","last_clean":"2024-09-11T23:35:13.461701+0000","last_became_active":"2024-09-11T23:34:50.855701+0000","last_became_peered":"2024-09-11T23:34:50.855701+0000","last_unstale":"2024-09-11T23:35:13.461701+0000","last_undegraded":"2024-09-11T23:35:13.461701+0000","last_fullsized":"2024-09-11T23:35:13.461701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:49.997617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41065,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":53,"num_write":25,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848168+0000","last_change":"2024-09-11T23:34:52.876928+0000","last_active":"2024-09-11T23:35:08.848168+0000","last_peered":"2024-09-11T23:35:08.848168+0000","last_clean":"2024-09-11T23:35:08.848168+0000","last_became_active":"2024-09-11T23:34:52.876578+0000","last_became_peered":"2024-09-11T23:34:52.876578+0000","last_unstale":"2024-09-11T23:35:08.848168+0000","last_undegraded":"2024-09-11T23:35:08.848168+0000","last_fullsized":"2024-09-11T23:35:08.848168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:07.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.870264+0000","last_change":"2024-09-11T23:35:08.869275+0000","last_active":"2024-09-11T23:35:08.870264+0000","last_peered":"2024-09-11T23:35:08.870264+0000","last_clean":"2024-09-11T23:35:08.870264+0000","last_became_active":"2024-09-11T23:35:08.850112+0000","last_became_peered":"2024-09-11T23:35:08.850112+0000","last_unstale":"2024-09-11T23:35:08.870264+0000","last_undegraded":"2024-09-11T23:35:08.870264+0000","last_fullsized":"2024-09-11T23:35:08.870264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:36.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844084+0000","last_change":"2024-09-11T23:34:52.871970+0000","last_active":"2024-09-11T23:35:08.844084+0000","last_peered":"2024-09-11T23:35:08.844084+0000","last_clean":"2024-09-11T23:35:08.844084+0000","last_became_active":"2024-09-11T23:34:52.871488+0000","last_became_peered":"2024-09-11T23:34:52.871488+0000","last_unstale":"2024-09-11T23:35:08.844084+0000","last_undegraded":"2024-09-11T23:35:08.844084+0000","last_fullsized":"2024-09-11T23:35:08.844084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:08.266431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456386+0000","last_change":"2024-09-11T23:34:50.854023+0000","last_active":"2024-09-11T23:35:09.456386+0000","last_peered":"2024-09-11T23:35:09.456386+0000","last_clean":"2024-09-11T23:35:09.456386+0000","last_became_active":"2024-09-11T23:34:50.853661+0000","last_became_peered":"2024-09-11T23:34:50.853661+0000","last_unstale":"2024-09-11T23:35:09.456386+0000","last_undegraded":"2024-09-11T23:35:09.456386+0000","last_fullsized":"2024-09-11T23:35:09.456386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:25.281976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.849648+0000","last_change":"2024-09-11T23:35:08.849648+0000","last_active":"2024-09-11T23:35:08.849648+0000","last_peered":"2024-09-11T23:35:08.849648+0000","last_clean":"2024-09-11T23:35:08.849648+0000","last_became_active":"2024-09-11T23:35:08.849246+0000","last_became_peered":"2024-09-11T23:35:08.849246+0000","last_unstale":"2024-09-11T23:35:08.849648+0000","last_undegraded":"2024-09-11T23:35:08.849648+0000","last_fullsized":"2024-09-11T23:35:08.849648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:17.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:15.463533+0000","last_change":"2024-09-11T23:34:50.853855+0000","last_active":"2024-09-11T23:35:15.463533+0000","last_peered":"2024-09-11T23:35:15.463533+0000","last_clean":"2024-09-11T23:35:15.463533+0000","last_became_active":"2024-09-11T23:34:50.853493+0000","last_became_peered":"2024-09-11T23:34:50.853493+0000","last_unstale":"2024-09-11T23:35:15.463533+0000","last_undegraded":"2024-09-11T23:35:15.463533+0000","last_fullsized":"2024-09-11T23:35:15.463533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:23.262472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846505+0000","last_change":"2024-09-11T23:34:52.875835+0000","last_active":"2024-09-11T23:35:08.846505+0000","last_peered":"2024-09-11T23:35:08.846505+0000","last_clean":"2024-09-11T23:35:08.846505+0000","last_became_active":"2024-09-11T23:34:52.875525+0000","last_became_peered":"2024-09-11T23:34:52.875525+0000","last_unstale":"2024-09-11T23:35:08.846505+0000","last_undegraded":"2024-09-11T23:35:08.846505+0000","last_fullsized":"2024-09-11T23:35:08.846505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:41.031816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847151+0000","last_change":"2024-09-11T23:34:50.924803+0000","last_active":"2024-09-11T23:35:08.847151+0000","last_peered":"2024-09-11T23:35:08.847151+0000","last_clean":"2024-09-11T23:35:08.847151+0000","last_became_active":"2024-09-11T23:34:50.924217+0000","last_became_peered":"2024-09-11T23:34:50.924217+0000","last_unstale":"2024-09-11T23:35:08.847151+0000","last_undegraded":"2024-09-11T23:35:08.847151+0000","last_fullsized":"2024-09-11T23:35:08.847151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:45.790592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.852839+0000","last_change":"2024-09-11T23:35:08.851856+0000","last_active":"2024-09-11T23:35:08.852839+0000","last_peered":"2024-09-11T23:35:08.852839+0000","last_clean":"2024-09-11T23:35:08.852839+0000","last_became_active":"2024-09-11T23:35:08.851490+0000","last_became_peered":"2024-09-11T23:35:08.851490+0000","last_unstale":"2024-09-11T23:35:08.852839+0000","last_undegraded":"2024-09-11T23:35:08.852839+0000","last_fullsized":"2024-09-11T23:35:08.852839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:46.269022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847890+0000","last_change":"2024-09-11T23:34:50.848385+0000","last_active":"2024-09-11T23:35:08.847890+0000","last_peered":"2024-09-11T23:35:08.847890+0000","last_clean":"2024-09-11T23:35:08.847890+0000","last_became_active":"2024-09-11T23:34:50.847808+0000","last_became_peered":"2024-09-11T23:34:50.847808+0000","last_unstale":"2024-09-11T23:35:08.847890+0000","last_undegraded":"2024-09-11T23:35:08.847890+0000","last_fullsized":"2024-09-11T23:35:08.847890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:38.796622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844817+0000","last_change":"2024-09-11T23:34:52.867341+0000","last_active":"2024-09-11T23:35:08.844817+0000","last_peered":"2024-09-11T23:35:08.844817+0000","last_clean":"2024-09-11T23:35:08.844817+0000","last_became_active":"2024-09-11T23:34:52.866161+0000","last_became_peered":"2024-09-11T23:34:52.866161+0000","last_unstale":"2024-09-11T23:35:08.844817+0000","last_undegraded":"2024-09-11T23:35:08.844817+0000","last_fullsized":"2024-09-11T23:35:08.844817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:27.913705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844372+0000","last_change":"2024-09-11T23:34:50.851466+0000","last_active":"2024-09-11T23:35:08.844372+0000","last_peered":"2024-09-11T23:35:08.844372+0000","last_clean":"2024-09-11T23:35:08.844372+0000","last_became_active":"2024-09-11T23:34:50.851191+0000","last_became_peered":"2024-09-11T23:34:50.851191+0000","last_unstale":"2024-09-11T23:35:08.844372+0000","last_undegraded":"2024-09-11T23:35:08.844372+0000","last_fullsized":"2024-09-11T23:35:08.844372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:00.328658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847008+0000","last_change":"2024-09-11T23:34:52.872290+0000","last_active":"2024-09-11T23:35:08.847008+0000","last_peered":"2024-09-11T23:35:08.847008+0000","last_clean":"2024-09-11T23:35:08.847008+0000","last_became_active":"2024-09-11T23:34:52.871935+0000","last_became_peered":"2024-09-11T23:34:52.871935+0000","last_unstale":"2024-09-11T23:35:08.847008+0000","last_undegraded":"2024-09-11T23:35:08.847008+0000","last_fullsized":"2024-09-11T23:35:08.847008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:08.217548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456729+0000","last_change":"2024-09-11T23:34:50.857763+0000","last_active":"2024-09-11T23:35:09.456729+0000","last_peered":"2024-09-11T23:35:09.456729+0000","last_clean":"2024-09-11T23:35:09.456729+0000","last_became_active":"2024-09-11T23:34:50.857427+0000","last_became_peered":"2024-09-11T23:34:50.857427+0000","last_unstale":"2024-09-11T23:35:09.456729+0000","last_undegraded":"2024-09-11T23:35:09.456729+0000","last_fullsized":"2024-09-11T23:35:09.456729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:59.703580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846908+0000","last_change":"2024-09-11T23:34:52.872426+0000","last_active":"2024-09-11T23:35:08.846908+0000","last_peered":"2024-09-11T23:35:08.846908+0000","last_clean":"2024-09-11T23:35:08.846908+0000","last_became_active":"2024-09-11T23:34:52.872090+0000","last_became_peered":"2024-09-11T23:34:52.872090+0000","last_unstale":"2024-09-11T23:35:08.846908+0000","last_undegraded":"2024-09-11T23:35:08.846908+0000","last_fullsized":"2024-09-11T23:35:08.846908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:04:00.963963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'6","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:15.474038+0000","last_change":"2024-09-11T23:34:50.924792+0000","last_active":"2024-09-11T23:35:15.474038+0000","last_peered":"2024-09-11T23:35:15.474038+0000","last_clean":"2024-09-11T23:35:15.474038+0000","last_became_active":"2024-09-11T23:34:50.924444+0000","last_became_peered":"2024-09-11T23:34:50.924444+0000","last_unstale":"2024-09-11T23:35:15.474038+0000","last_undegraded":"2024-09-11T23:35:15.474038+0000","last_fullsized":"2024-09-11T23:35:15.474038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:12.855314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846881+0000","last_change":"2024-09-11T23:34:52.872079+0000","last_active":"2024-09-11T23:35:08.846881+0000","last_peered":"2024-09-11T23:35:08.846881+0000","last_clean":"2024-09-11T23:35:08.846881+0000","last_became_active":"2024-09-11T23:34:52.871755+0000","last_became_peered":"2024-09-11T23:34:52.871755+0000","last_unstale":"2024-09-11T23:35:08.846881+0000","last_undegraded":"2024-09-11T23:35:08.846881+0000","last_fullsized":"2024-09-11T23:35:08.846881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:05.990832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":344,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846729+0000","last_change":"2024-09-11T23:33:42.256521+0000","last_active":"2024-09-11T23:35:08.846729+0000","last_peered":"2024-09-11T23:35:08.846729+0000","last_clean":"2024-09-11T23:35:08.846729+0000","last_became_active":"2024-09-11T23:33:42.256041+0000","last_became_peered":"2024-09-11T23:33:42.256041+0000","last_unstale":"2024-09-11T23:35:08.846729+0000","last_undegraded":"2024-09-11T23:35:08.846729+0000","last_fullsized":"2024-09-11T23:35:08.846729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_clean_scrub_stamp":"2024-09-11T23:30:22.061515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:21.391839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845844+0000","last_change":"2024-09-11T23:34:50.854087+0000","last_active":"2024-09-11T23:35:08.845844+0000","last_peered":"2024-09-11T23:35:08.845844+0000","last_clean":"2024-09-11T23:35:08.845844+0000","last_became_active":"2024-09-11T23:34:50.852346+0000","last_became_peered":"2024-09-11T23:34:50.852346+0000","last_unstale":"2024-09-11T23:35:08.845844+0000","last_undegraded":"2024-09-11T23:35:08.845844+0000","last_fullsized":"2024-09-11T23:35:08.845844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.287119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847645+0000","last_change":"2024-09-11T23:34:52.869198+0000","last_active":"2024-09-11T23:35:08.847645+0000","last_peered":"2024-09-11T23:35:08.847645+0000","last_clean":"2024-09-11T23:35:08.847645+0000","last_became_active":"2024-09-11T23:34:52.868621+0000","last_became_peered":"2024-09-11T23:34:52.868621+0000","last_unstale":"2024-09-11T23:35:08.847645+0000","last_undegraded":"2024-09-11T23:35:08.847645+0000","last_fullsized":"2024-09-11T23:35:08.847645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:17.430654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846715+0000","last_change":"2024-09-11T23:34:50.852752+0000","last_active":"2024-09-11T23:35:08.846715+0000","last_peered":"2024-09-11T23:35:08.846715+0000","last_clean":"2024-09-11T23:35:08.846715+0000","last_became_active":"2024-09-11T23:34:50.851895+0000","last_became_peered":"2024-09-11T23:34:50.851895+0000","last_unstale":"2024-09-11T23:35:08.846715+0000","last_undegraded":"2024-09-11T23:35:08.846715+0000","last_fullsized":"2024-09-11T23:35:08.846715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:46.894432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844043+0000","last_change":"2024-09-11T23:34:52.866191+0000","last_active":"2024-09-11T23:35:08.844043+0000","last_peered":"2024-09-11T23:35:08.844043+0000","last_clean":"2024-09-11T23:35:08.844043+0000","last_became_active":"2024-09-11T23:34:52.865858+0000","last_became_peered":"2024-09-11T23:34:52.865858+0000","last_unstale":"2024-09-11T23:35:08.844043+0000","last_undegraded":"2024-09-11T23:35:08.844043+0000","last_fullsized":"2024-09-11T23:35:08.844043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:21.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:15.462941+0000","last_change":"2024-09-11T23:34:50.924125+0000","last_active":"2024-09-11T23:35:15.462941+0000","last_peered":"2024-09-11T23:35:15.462941+0000","last_clean":"2024-09-11T23:35:15.462941+0000","last_became_active":"2024-09-11T23:34:50.923780+0000","last_became_peered":"2024-09-11T23:34:50.923780+0000","last_unstale":"2024-09-11T23:35:15.462941+0000","last_undegraded":"2024-09-11T23:35:15.462941+0000","last_fullsized":"2024-09-11T23:35:15.462941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:39.774827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455096+0000","last_change":"2024-09-11T23:34:52.868364+0000","last_active":"2024-09-11T23:35:09.455096+0000","last_peered":"2024-09-11T23:35:09.455096+0000","last_clean":"2024-09-11T23:35:09.455096+0000","last_became_active":"2024-09-11T23:34:52.867913+0000","last_became_peered":"2024-09-11T23:34:52.867913+0000","last_unstale":"2024-09-11T23:35:09.455096+0000","last_undegraded":"2024-09-11T23:35:09.455096+0000","last_fullsized":"2024-09-11T23:35:09.455096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.571825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:14.472171+0000","last_change":"2024-09-11T23:34:50.852421+0000","last_active":"2024-09-11T23:35:14.472171+0000","last_peered":"2024-09-11T23:35:14.472171+0000","last_clean":"2024-09-11T23:35:14.472171+0000","last_became_active":"2024-09-11T23:34:50.851898+0000","last_became_peered":"2024-09-11T23:34:50.851898+0000","last_unstale":"2024-09-11T23:35:14.472171+0000","last_undegraded":"2024-09-11T23:35:14.472171+0000","last_fullsized":"2024-09-11T23:35:14.472171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.661276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848710+0000","last_change":"2024-09-11T23:34:52.879995+0000","last_active":"2024-09-11T23:35:08.848710+0000","last_peered":"2024-09-11T23:35:08.848710+0000","last_clean":"2024-09-11T23:35:08.848710+0000","last_became_active":"2024-09-11T23:34:52.879678+0000","last_became_peered":"2024-09-11T23:34:52.879678+0000","last_unstale":"2024-09-11T23:35:08.848710+0000","last_undegraded":"2024-09-11T23:35:08.848710+0000","last_fullsized":"2024-09-11T23:35:08.848710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:47.456674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847399+0000","last_change":"2024-09-11T23:34:50.924957+0000","last_active":"2024-09-11T23:35:08.847399+0000","last_peered":"2024-09-11T23:35:08.847399+0000","last_clean":"2024-09-11T23:35:08.847399+0000","last_became_active":"2024-09-11T23:34:50.924500+0000","last_became_peered":"2024-09-11T23:34:50.924500+0000","last_unstale":"2024-09-11T23:35:08.847399+0000","last_undegraded":"2024-09-11T23:35:08.847399+0000","last_fullsized":"2024-09-11T23:35:08.847399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:39.268842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848946+0000","last_change":"2024-09-11T23:34:52.867337+0000","last_active":"2024-09-11T23:35:08.848946+0000","last_peered":"2024-09-11T23:35:08.848946+0000","last_clean":"2024-09-11T23:35:08.848946+0000","last_became_active":"2024-09-11T23:34:52.866799+0000","last_became_peered":"2024-09-11T23:34:52.866799+0000","last_unstale":"2024-09-11T23:35:08.848946+0000","last_undegraded":"2024-09-11T23:35:08.848946+0000","last_fullsized":"2024-09-11T23:35:08.848946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:38.177135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848411+0000","last_change":"2024-09-11T23:34:50.925156+0000","last_active":"2024-09-11T23:35:08.848411+0000","last_peered":"2024-09-11T23:35:08.848411+0000","last_clean":"2024-09-11T23:35:08.848411+0000","last_became_active":"2024-09-11T23:34:50.924282+0000","last_became_peered":"2024-09-11T23:34:50.924282+0000","last_unstale":"2024-09-11T23:35:08.848411+0000","last_undegraded":"2024-09-11T23:35:08.848411+0000","last_fullsized":"2024-09-11T23:35:08.848411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:47.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844337+0000","last_change":"2024-09-11T23:34:52.864993+0000","last_active":"2024-09-11T23:35:08.844337+0000","last_peered":"2024-09-11T23:35:08.844337+0000","last_clean":"2024-09-11T23:35:08.844337+0000","last_became_active":"2024-09-11T23:34:52.864267+0000","last_became_peered":"2024-09-11T23:34:52.864267+0000","last_unstale":"2024-09-11T23:35:08.844337+0000","last_undegraded":"2024-09-11T23:35:08.844337+0000","last_fullsized":"2024-09-11T23:35:08.844337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:29.912068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:14.449437+0000","last_change":"2024-09-11T23:34:50.849864+0000","last_active":"2024-09-11T23:35:14.449437+0000","last_peered":"2024-09-11T23:35:14.449437+0000","last_clean":"2024-09-11T23:35:14.449437+0000","last_became_active":"2024-09-11T23:34:50.849577+0000","last_became_peered":"2024-09-11T23:34:50.849577+0000","last_unstale":"2024-09-11T23:35:14.449437+0000","last_undegraded":"2024-09-11T23:35:14.449437+0000","last_fullsized":"2024-09-11T23:35:14.449437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:06.845000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848578+0000","last_change":"2024-09-11T23:34:52.872028+0000","last_active":"2024-09-11T23:35:08.848578+0000","last_peered":"2024-09-11T23:35:08.848578+0000","last_clean":"2024-09-11T23:35:08.848578+0000","last_became_active":"2024-09-11T23:34:52.871723+0000","last_became_peered":"2024-09-11T23:34:52.871723+0000","last_unstale":"2024-09-11T23:35:08.848578+0000","last_undegraded":"2024-09-11T23:35:08.848578+0000","last_fullsized":"2024-09-11T23:35:08.848578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:44.656218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846269+0000","last_change":"2024-09-11T23:34:50.860314+0000","last_active":"2024-09-11T23:35:08.846269+0000","last_peered":"2024-09-11T23:35:08.846269+0000","last_clean":"2024-09-11T23:35:08.846269+0000","last_became_active":"2024-09-11T23:34:50.859953+0000","last_became_peered":"2024-09-11T23:34:50.859953+0000","last_unstale":"2024-09-11T23:35:08.846269+0000","last_undegraded":"2024-09-11T23:35:08.846269+0000","last_fullsized":"2024-09-11T23:35:08.846269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:46.705864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872092+0000","last_change":"2024-09-11T23:34:52.867190+0000","last_active":"2024-09-11T23:35:08.872092+0000","last_peered":"2024-09-11T23:35:08.872092+0000","last_clean":"2024-09-11T23:35:08.872092+0000","last_became_active":"2024-09-11T23:34:52.866829+0000","last_became_peered":"2024-09-11T23:34:52.866829+0000","last_unstale":"2024-09-11T23:35:08.872092+0000","last_undegraded":"2024-09-11T23:35:08.872092+0000","last_fullsized":"2024-09-11T23:35:08.872092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.752006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850343+0000","last_change":"2024-09-11T23:34:50.924566+0000","last_active":"2024-09-11T23:35:08.850343+0000","last_peered":"2024-09-11T23:35:08.850343+0000","last_clean":"2024-09-11T23:35:08.850343+0000","last_became_active":"2024-09-11T23:34:50.924284+0000","last_became_peered":"2024-09-11T23:34:50.924284+0000","last_unstale":"2024-09-11T23:35:08.850343+0000","last_undegraded":"2024-09-11T23:35:08.850343+0000","last_fullsized":"2024-09-11T23:35:08.850343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:54.246154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847042+0000","last_change":"2024-09-11T23:34:52.869302+0000","last_active":"2024-09-11T23:35:08.847042+0000","last_peered":"2024-09-11T23:35:08.847042+0000","last_clean":"2024-09-11T23:35:08.847042+0000","last_became_active":"2024-09-11T23:34:52.868960+0000","last_became_peered":"2024-09-11T23:34:52.868960+0000","last_unstale":"2024-09-11T23:35:08.847042+0000","last_undegraded":"2024-09-11T23:35:08.847042+0000","last_fullsized":"2024-09-11T23:35:08.847042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.561712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847953+0000","last_change":"2024-09-11T23:34:50.854573+0000","last_active":"2024-09-11T23:35:08.847953+0000","last_peered":"2024-09-11T23:35:08.847953+0000","last_clean":"2024-09-11T23:35:08.847953+0000","last_became_active":"2024-09-11T23:34:50.854165+0000","last_became_peered":"2024-09-11T23:34:50.854165+0000","last_unstale":"2024-09-11T23:35:08.847953+0000","last_undegraded":"2024-09-11T23:35:08.847953+0000","last_fullsized":"2024-09-11T23:35:08.847953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:57:31.589495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844752+0000","last_change":"2024-09-11T23:34:52.875987+0000","last_active":"2024-09-11T23:35:08.844752+0000","last_peered":"2024-09-11T23:35:08.844752+0000","last_clean":"2024-09-11T23:35:08.844752+0000","last_became_active":"2024-09-11T23:34:52.875662+0000","last_became_peered":"2024-09-11T23:34:52.875662+0000","last_unstale":"2024-09-11T23:35:08.844752+0000","last_undegraded":"2024-09-11T23:35:08.844752+0000","last_fullsized":"2024-09-11T23:35:08.844752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:16.514938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847766+0000","last_change":"2024-09-11T23:34:50.850726+0000","last_active":"2024-09-11T23:35:08.847766+0000","last_peered":"2024-09-11T23:35:08.847766+0000","last_clean":"2024-09-11T23:35:08.847766+0000","last_became_active":"2024-09-11T23:34:50.850381+0000","last_became_peered":"2024-09-11T23:34:50.850381+0000","last_unstale":"2024-09-11T23:35:08.847766+0000","last_undegraded":"2024-09-11T23:35:08.847766+0000","last_fullsized":"2024-09-11T23:35:08.847766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:50.431293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847564+0000","last_change":"2024-09-11T23:34:52.870652+0000","last_active":"2024-09-11T23:35:08.847564+0000","last_peered":"2024-09-11T23:35:08.847564+0000","last_clean":"2024-09-11T23:35:08.847564+0000","last_became_active":"2024-09-11T23:34:52.870303+0000","last_became_peered":"2024-09-11T23:34:52.870303+0000","last_unstale":"2024-09-11T23:35:08.847564+0000","last_undegraded":"2024-09-11T23:35:08.847564+0000","last_fullsized":"2024-09-11T23:35:08.847564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:17.670981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.855045+0000","last_change":"2024-09-11T23:35:08.854490+0000","last_active":"2024-09-11T23:35:08.855045+0000","last_peered":"2024-09-11T23:35:08.855045+0000","last_clean":"2024-09-11T23:35:08.855045+0000","last_became_active":"2024-09-11T23:35:08.849820+0000","last_became_peered":"2024-09-11T23:35:08.849820+0000","last_unstale":"2024-09-11T23:35:08.855045+0000","last_undegraded":"2024-09-11T23:35:08.855045+0000","last_fullsized":"2024-09-11T23:35:08.855045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:09.509867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846681+0000","last_change":"2024-09-11T23:34:52.869695+0000","last_active":"2024-09-11T23:35:08.846681+0000","last_peered":"2024-09-11T23:35:08.846681+0000","last_clean":"2024-09-11T23:35:08.846681+0000","last_became_active":"2024-09-11T23:34:52.869367+0000","last_became_peered":"2024-09-11T23:34:52.869367+0000","last_unstale":"2024-09-11T23:35:08.846681+0000","last_undegraded":"2024-09-11T23:35:08.846681+0000","last_fullsized":"2024-09-11T23:35:08.846681+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:16.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455461+0000","last_change":"2024-09-11T23:34:52.872778+0000","last_active":"2024-09-11T23:35:09.455461+0000","last_peered":"2024-09-11T23:35:09.455461+0000","last_clean":"2024-09-11T23:35:09.455461+0000","last_became_active":"2024-09-11T23:34:52.872451+0000","last_became_peered":"2024-09-11T23:34:52.872451+0000","last_unstale":"2024-09-11T23:35:09.455461+0000","last_undegraded":"2024-09-11T23:35:09.455461+0000","last_fullsized":"2024-09-11T23:35:09.455461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:51.769927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847892+0000","last_change":"2024-09-11T23:34:50.849675+0000","last_active":"2024-09-11T23:35:08.847892+0000","last_peered":"2024-09-11T23:35:08.847892+0000","last_clean":"2024-09-11T23:35:08.847892+0000","last_became_active":"2024-09-11T23:34:50.849228+0000","last_became_peered":"2024-09-11T23:34:50.849228+0000","last_unstale":"2024-09-11T23:35:08.847892+0000","last_undegraded":"2024-09-11T23:35:08.847892+0000","last_fullsized":"2024-09-11T23:35:08.847892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:50.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850319+0000","last_change":"2024-09-11T23:34:52.867739+0000","last_active":"2024-09-11T23:35:08.850319+0000","last_peered":"2024-09-11T23:35:08.850319+0000","last_clean":"2024-09-11T23:35:08.850319+0000","last_became_active":"2024-09-11T23:34:52.866613+0000","last_became_peered":"2024-09-11T23:34:52.866613+0000","last_unstale":"2024-09-11T23:35:08.850319+0000","last_undegraded":"2024-09-11T23:35:08.850319+0000","last_fullsized":"2024-09-11T23:35:08.850319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:49.348386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848269+0000","last_change":"2024-09-11T23:34:50.851401+0000","last_active":"2024-09-11T23:35:08.848269+0000","last_peered":"2024-09-11T23:35:08.848269+0000","last_clean":"2024-09-11T23:35:08.848269+0000","last_became_active":"2024-09-11T23:34:50.851068+0000","last_became_peered":"2024-09-11T23:34:50.851068+0000","last_unstale":"2024-09-11T23:35:08.848269+0000","last_undegraded":"2024-09-11T23:35:08.848269+0000","last_fullsized":"2024-09-11T23:35:08.848269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:09.673864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872365+0000","last_change":"2024-09-11T23:34:52.865216+0000","last_active":"2024-09-11T23:35:08.872365+0000","last_peered":"2024-09-11T23:35:08.872365+0000","last_clean":"2024-09-11T23:35:08.872365+0000","last_became_active":"2024-09-11T23:34:52.864917+0000","last_became_peered":"2024-09-11T23:34:52.864917+0000","last_unstale":"2024-09-11T23:35:08.872365+0000","last_undegraded":"2024-09-11T23:35:08.872365+0000","last_fullsized":"2024-09-11T23:35:08.872365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:36.469793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848560+0000","last_change":"2024-09-11T23:34:50.924291+0000","last_active":"2024-09-11T23:35:08.848560+0000","last_peered":"2024-09-11T23:35:08.848560+0000","last_clean":"2024-09-11T23:35:08.848560+0000","last_became_active":"2024-09-11T23:34:50.923892+0000","last_became_peered":"2024-09-11T23:34:50.923892+0000","last_unstale":"2024-09-11T23:35:08.848560+0000","last_undegraded":"2024-09-11T23:35:08.848560+0000","last_fullsized":"2024-09-11T23:35:08.848560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:52.266778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848164+0000","last_change":"2024-09-11T23:34:52.875995+0000","last_active":"2024-09-11T23:35:08.848164+0000","last_peered":"2024-09-11T23:35:08.848164+0000","last_clean":"2024-09-11T23:35:08.848164+0000","last_became_active":"2024-09-11T23:34:52.875654+0000","last_became_peered":"2024-09-11T23:34:52.875654+0000","last_unstale":"2024-09-11T23:35:08.848164+0000","last_undegraded":"2024-09-11T23:35:08.848164+0000","last_fullsized":"2024-09-11T23:35:08.848164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:19.218291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:14.471624+0000","last_change":"2024-09-11T23:34:50.853881+0000","last_active":"2024-09-11T23:35:14.471624+0000","last_peered":"2024-09-11T23:35:14.471624+0000","last_clean":"2024-09-11T23:35:14.471624+0000","last_became_active":"2024-09-11T23:34:50.853576+0000","last_became_peered":"2024-09-11T23:34:50.853576+0000","last_unstale":"2024-09-11T23:35:14.471624+0000","last_undegraded":"2024-09-11T23:35:14.471624+0000","last_fullsized":"2024-09-11T23:35:14.471624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:38.550587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48273,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":153,"num_read_kb":128,"num_write":232,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":161895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612625,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":434822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841557,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1572864,"data_stored":1066848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005083,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":434291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":434669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495653,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":901120,"data_stored":434008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888110,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1026474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018932,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":434262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247863,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":433833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444094,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":485069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607618,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":484524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":921600,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":51355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":51900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:35:19.550 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:35:19.794 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:20.171 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:19 smithi133 ceph-mon[31430]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:35:20.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:19 smithi138 ceph-mon[35142]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:35:20.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.250+0000 7fc602c22640 1 -- 172.21.15.133:0/964637711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5fc100a80 msgr2=0x7fc5fc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:20.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.250+0000 7fc602c22640 1 --2- 172.21.15.133:0/964637711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5fc100a80 0x7fc5fc100ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009930 tx=0x7fc5e802a420 comp rx=0 tx=0).stop 2024-09-11T23:35:20.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.250+0000 7fc602c22640 1 -- 172.21.15.133:0/964637711 shutdown_connections 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.250+0000 7fc602c22640 1 --2- 172.21.15.133:0/964637711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5fc101420 0x7fc5fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.250+0000 7fc602c22640 1 --2- 172.21.15.133:0/964637711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5fc100a80 0x7fc5fc100ee0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.250+0000 7fc602c22640 1 --2- 172.21.15.133:0/964637711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5fc106a80 0x7fc5fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.250+0000 7fc602c22640 1 -- 172.21.15.133:0/964637711 >> 172.21.15.133:0/964637711 conn(0x7fc5fc0fc7a0 msgr2=0x7fc5fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.250+0000 7fc602c22640 1 -- 172.21.15.133:0/964637711 shutdown_connections 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.250+0000 7fc602c22640 1 -- 172.21.15.133:0/964637711 wait complete. 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc602c22640 1 Processor -- start 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc602c22640 1 -- start start 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc602c22640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5fc100a80 0x7fc5fc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc602c22640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5fc101420 0x7fc5fc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc602c22640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5fc106a80 0x7fc5fc1a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc602c22640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc110e40 con 0x7fc5fc100a80 2024-09-11T23:35:20.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc602c22640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc110cc0 con 0x7fc5fc101420 2024-09-11T23:35:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc602c22640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc110fc0 con 0x7fc5fc106a80 2024-09-11T23:35:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc6009be640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5fc100a80 0x7fc5fc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc6009be640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5fc100a80 0x7fc5fc1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43208/0 (socket says 172.21.15.133:43208) 2024-09-11T23:35:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc6009be640 1 -- 172.21.15.133:0/4054454314 learned_addr learned my addr 172.21.15.133:0/4054454314 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc6011bf640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5fc106a80 0x7fc5fc1a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc5fbfff640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5fc101420 0x7fc5fc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc6009be640 1 -- 172.21.15.133:0/4054454314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5fc106a80 msgr2=0x7fc5fc1a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc6009be640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5fc106a80 0x7fc5fc1a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc6009be640 1 -- 172.21.15.133:0/4054454314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5fc101420 msgr2=0x7fc5fc199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc6009be640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5fc101420 0x7fc5fc199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.251+0000 7fc6009be640 1 -- 172.21.15.133:0/4054454314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc1a48b0 con 0x7fc5fc100a80 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.252+0000 7fc6009be640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5fc100a80 0x7fc5fc1992f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c890 tx=0x7fc5f000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.252+0000 7fc5f9ffb640 1 -- 172.21.15.133:0/4054454314 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0004220 con 0x7fc5fc100a80 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.252+0000 7fc5f9ffb640 1 -- 172.21.15.133:0/4054454314 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f00043c0 con 0x7fc5fc100a80 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.252+0000 7fc602c22640 1 -- 172.21.15.133:0/4054454314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc1a4a80 con 0x7fc5fc100a80 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.252+0000 7fc5f9ffb640 1 -- 172.21.15.133:0/4054454314 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0004a00 con 0x7fc5fc100a80 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.252+0000 7fc602c22640 1 -- 172.21.15.133:0/4054454314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc1a4f40 con 0x7fc5fc100a80 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.253+0000 7fc602c22640 1 -- 172.21.15.133:0/4054454314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5fc100ee0 con 0x7fc5fc100a80 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.254+0000 7fc5f9ffb640 1 -- 172.21.15.133:0/4054454314 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc5f00079b0 con 0x7fc5fc100a80 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.255+0000 7fc5f9ffb640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5ec0a2c10 0x7fc5ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.255+0000 7fc5fbfff640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5ec0a2c10 0x7fc5ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.255+0000 7fc5f9ffb640 1 -- 172.21.15.133:0/4054454314 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fc5f00cee10 con 0x7fc5fc100a80 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.255+0000 7fc5fbfff640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5ec0a2c10 0x7fc5ec0a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000c00 tx=0x7fc5e8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.259+0000 7fc5f9ffb640 1 -- 172.21.15.133:0/4054454314 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5f0014030 con 0x7fc5fc100a80 2024-09-11T23:35:20.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:19 smithi178 ceph-mon[35882]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:35:20.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.427+0000 7fc602c22640 1 -- 172.21.15.133:0/4054454314 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5fc109080 con 0x7fc5ec0a2c10 2024-09-11T23:35:20.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.433+0000 7fc5f9ffb640 1 -- 172.21.15.133:0/4054454314 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258763 (secure 0 0 0) 0x7fc5e80315b0 con 0x7fc5ec0a2c10 2024-09-11T23:35:20.434 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:35:20.435 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T23:35:20.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 -- 172.21.15.133:0/4054454314 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5ec0a2c10 msgr2=0x7fc5ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5ec0a2c10 0x7fc5ec0a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000c00 tx=0x7fc5e8005a20 comp rx=0 tx=0).stop 2024-09-11T23:35:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 -- 172.21.15.133:0/4054454314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5fc100a80 msgr2=0x7fc5fc1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5fc100a80 0x7fc5fc1992f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c890 tx=0x7fc5f000cd50 comp rx=0 tx=0).stop 2024-09-11T23:35:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 -- 172.21.15.133:0/4054454314 shutdown_connections 2024-09-11T23:35:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5ec0a2c10 0x7fc5ec0a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5fc106a80 0x7fc5fc1a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5fc101420 0x7fc5fc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 --2- 172.21.15.133:0/4054454314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5fc100a80 0x7fc5fc1992f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 -- 172.21.15.133:0/4054454314 >> 172.21.15.133:0/4054454314 conn(0x7fc5fc0fc7a0 msgr2=0x7fc5fc0fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 -- 172.21.15.133:0/4054454314 shutdown_connections 2024-09-11T23:35:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:20.436+0000 7fc602c22640 1 -- 172.21.15.133:0/4054454314 wait complete. 2024-09-11T23:35:21.075 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:20 smithi133 ceph-mon[31430]: pgmap v287: 97 pgs: 97 active+clean; 632 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 14 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:35:21.103 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-11T23:35:19.850522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647413,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":310,"num_read_kb":271,"num_write":419,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357712,"kb_used_data":12600,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584112,"statfs":{"total":1151940427776,"available":1151574130688,"internally_reserved":0,"allocated":12902400,"data_stored":7140267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49011,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":113,"num_write":36,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.007698"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845014+0000","last_change":"2024-09-11T23:34:52.867694+0000","last_active":"2024-09-11T23:35:08.845014+0000","last_peered":"2024-09-11T23:35:08.845014+0000","last_clean":"2024-09-11T23:35:08.845014+0000","last_became_active":"2024-09-11T23:34:52.867368+0000","last_became_peered":"2024-09-11T23:34:52.867368+0000","last_unstale":"2024-09-11T23:35:08.845014+0000","last_undegraded":"2024-09-11T23:35:08.845014+0000","last_fullsized":"2024-09-11T23:35:08.845014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:46:07.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872241+0000","last_change":"2024-09-11T23:34:52.871881+0000","last_active":"2024-09-11T23:35:08.872241+0000","last_peered":"2024-09-11T23:35:08.872241+0000","last_clean":"2024-09-11T23:35:08.872241+0000","last_became_active":"2024-09-11T23:34:52.871609+0000","last_became_peered":"2024-09-11T23:34:52.871609+0000","last_unstale":"2024-09-11T23:35:08.872241+0000","last_undegraded":"2024-09-11T23:35:08.872241+0000","last_fullsized":"2024-09-11T23:35:08.872241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.938584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872054+0000","last_change":"2024-09-11T23:34:52.863931+0000","last_active":"2024-09-11T23:35:08.872054+0000","last_peered":"2024-09-11T23:35:08.872054+0000","last_clean":"2024-09-11T23:35:08.872054+0000","last_became_active":"2024-09-11T23:34:52.863637+0000","last_became_peered":"2024-09-11T23:34:52.863637+0000","last_unstale":"2024-09-11T23:35:08.872054+0000","last_undegraded":"2024-09-11T23:35:08.872054+0000","last_fullsized":"2024-09-11T23:35:08.872054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:44.118865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846293+0000","last_change":"2024-09-11T23:34:52.879619+0000","last_active":"2024-09-11T23:35:08.846293+0000","last_peered":"2024-09-11T23:35:08.846293+0000","last_clean":"2024-09-11T23:35:08.846293+0000","last_became_active":"2024-09-11T23:34:52.878695+0000","last_became_peered":"2024-09-11T23:34:52.878695+0000","last_unstale":"2024-09-11T23:35:08.846293+0000","last_undegraded":"2024-09-11T23:35:08.846293+0000","last_fullsized":"2024-09-11T23:35:08.846293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:30.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847368+0000","last_change":"2024-09-11T23:34:52.869363+0000","last_active":"2024-09-11T23:35:08.847368+0000","last_peered":"2024-09-11T23:35:08.847368+0000","last_clean":"2024-09-11T23:35:08.847368+0000","last_became_active":"2024-09-11T23:34:52.869027+0000","last_became_peered":"2024-09-11T23:34:52.869027+0000","last_unstale":"2024-09-11T23:35:08.847368+0000","last_undegraded":"2024-09-11T23:35:08.847368+0000","last_fullsized":"2024-09-11T23:35:08.847368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:15.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848328+0000","last_change":"2024-09-11T23:34:52.880665+0000","last_active":"2024-09-11T23:35:08.848328+0000","last_peered":"2024-09-11T23:35:08.848328+0000","last_clean":"2024-09-11T23:35:08.848328+0000","last_became_active":"2024-09-11T23:34:52.880172+0000","last_became_peered":"2024-09-11T23:34:52.880172+0000","last_unstale":"2024-09-11T23:35:08.848328+0000","last_undegraded":"2024-09-11T23:35:08.848328+0000","last_fullsized":"2024-09-11T23:35:08.848328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:12.025877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846502+0000","last_change":"2024-09-11T23:34:52.873897+0000","last_active":"2024-09-11T23:35:08.846502+0000","last_peered":"2024-09-11T23:35:08.846502+0000","last_clean":"2024-09-11T23:35:08.846502+0000","last_became_active":"2024-09-11T23:34:52.873219+0000","last_became_peered":"2024-09-11T23:34:52.873219+0000","last_unstale":"2024-09-11T23:35:08.846502+0000","last_undegraded":"2024-09-11T23:35:08.846502+0000","last_fullsized":"2024-09-11T23:35:08.846502+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:47.125682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847314+0000","last_change":"2024-09-11T23:34:52.870699+0000","last_active":"2024-09-11T23:35:08.847314+0000","last_peered":"2024-09-11T23:35:08.847314+0000","last_clean":"2024-09-11T23:35:08.847314+0000","last_became_active":"2024-09-11T23:34:52.870314+0000","last_became_peered":"2024-09-11T23:34:52.870314+0000","last_unstale":"2024-09-11T23:35:08.847314+0000","last_undegraded":"2024-09-11T23:35:08.847314+0000","last_fullsized":"2024-09-11T23:35:08.847314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:41.263159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.851159+0000","last_change":"2024-09-11T23:34:52.869700+0000","last_active":"2024-09-11T23:35:08.851159+0000","last_peered":"2024-09-11T23:35:08.851159+0000","last_clean":"2024-09-11T23:35:08.851159+0000","last_became_active":"2024-09-11T23:34:52.869398+0000","last_became_peered":"2024-09-11T23:34:52.869398+0000","last_unstale":"2024-09-11T23:35:08.851159+0000","last_undegraded":"2024-09-11T23:35:08.851159+0000","last_fullsized":"2024-09-11T23:35:08.851159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:49.776525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848589+0000","last_change":"2024-09-11T23:34:52.879999+0000","last_active":"2024-09-11T23:35:08.848589+0000","last_peered":"2024-09-11T23:35:08.848589+0000","last_clean":"2024-09-11T23:35:08.848589+0000","last_became_active":"2024-09-11T23:34:52.879646+0000","last_became_peered":"2024-09-11T23:34:52.879646+0000","last_unstale":"2024-09-11T23:35:08.848589+0000","last_undegraded":"2024-09-11T23:35:08.848589+0000","last_fullsized":"2024-09-11T23:35:08.848589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.058704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846881+0000","last_change":"2024-09-11T23:34:52.878740+0000","last_active":"2024-09-11T23:35:08.846881+0000","last_peered":"2024-09-11T23:35:08.846881+0000","last_clean":"2024-09-11T23:35:08.846881+0000","last_became_active":"2024-09-11T23:34:52.878120+0000","last_became_peered":"2024-09-11T23:34:52.878120+0000","last_unstale":"2024-09-11T23:35:08.846881+0000","last_undegraded":"2024-09-11T23:35:08.846881+0000","last_fullsized":"2024-09-11T23:35:08.846881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846051+0000","last_change":"2024-09-11T23:34:52.873409+0000","last_active":"2024-09-11T23:35:08.846051+0000","last_peered":"2024-09-11T23:35:08.846051+0000","last_clean":"2024-09-11T23:35:08.846051+0000","last_became_active":"2024-09-11T23:34:52.872959+0000","last_became_peered":"2024-09-11T23:34:52.872959+0000","last_unstale":"2024-09-11T23:35:08.846051+0000","last_undegraded":"2024-09-11T23:35:08.846051+0000","last_fullsized":"2024-09-11T23:35:08.846051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:38.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844266+0000","last_change":"2024-09-11T23:34:52.869164+0000","last_active":"2024-09-11T23:35:08.844266+0000","last_peered":"2024-09-11T23:35:08.844266+0000","last_clean":"2024-09-11T23:35:08.844266+0000","last_became_active":"2024-09-11T23:34:52.868847+0000","last_became_peered":"2024-09-11T23:34:52.868847+0000","last_unstale":"2024-09-11T23:35:08.844266+0000","last_undegraded":"2024-09-11T23:35:08.844266+0000","last_fullsized":"2024-09-11T23:35:08.844266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.764510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872027+0000","last_change":"2024-09-11T23:34:52.869494+0000","last_active":"2024-09-11T23:35:08.872027+0000","last_peered":"2024-09-11T23:35:08.872027+0000","last_clean":"2024-09-11T23:35:08.872027+0000","last_became_active":"2024-09-11T23:34:52.869178+0000","last_became_peered":"2024-09-11T23:34:52.869178+0000","last_unstale":"2024-09-11T23:35:08.872027+0000","last_undegraded":"2024-09-11T23:35:08.872027+0000","last_fullsized":"2024-09-11T23:35:08.872027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:23.431564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872033+0000","last_change":"2024-09-11T23:34:52.866269+0000","last_active":"2024-09-11T23:35:08.872033+0000","last_peered":"2024-09-11T23:35:08.872033+0000","last_clean":"2024-09-11T23:35:08.872033+0000","last_became_active":"2024-09-11T23:34:52.865980+0000","last_became_peered":"2024-09-11T23:34:52.865980+0000","last_unstale":"2024-09-11T23:35:08.872033+0000","last_undegraded":"2024-09-11T23:35:08.872033+0000","last_fullsized":"2024-09-11T23:35:08.872033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:48.492252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850466+0000","last_change":"2024-09-11T23:34:52.862192+0000","last_active":"2024-09-11T23:35:08.850466+0000","last_peered":"2024-09-11T23:35:08.850466+0000","last_clean":"2024-09-11T23:35:08.850466+0000","last_became_active":"2024-09-11T23:34:52.861841+0000","last_became_peered":"2024-09-11T23:34:52.861841+0000","last_unstale":"2024-09-11T23:35:08.850466+0000","last_undegraded":"2024-09-11T23:35:08.850466+0000","last_fullsized":"2024-09-11T23:35:08.850466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.258948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845097+0000","last_change":"2024-09-11T23:34:52.870984+0000","last_active":"2024-09-11T23:35:08.845097+0000","last_peered":"2024-09-11T23:35:08.845097+0000","last_clean":"2024-09-11T23:35:08.845097+0000","last_became_active":"2024-09-11T23:34:52.870435+0000","last_became_peered":"2024-09-11T23:34:52.870435+0000","last_unstale":"2024-09-11T23:35:08.845097+0000","last_undegraded":"2024-09-11T23:35:08.845097+0000","last_fullsized":"2024-09-11T23:35:08.845097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:34.391537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847466+0000","last_change":"2024-09-11T23:34:52.873176+0000","last_active":"2024-09-11T23:35:08.847466+0000","last_peered":"2024-09-11T23:35:08.847466+0000","last_clean":"2024-09-11T23:35:08.847466+0000","last_became_active":"2024-09-11T23:34:52.872862+0000","last_became_peered":"2024-09-11T23:34:52.872862+0000","last_unstale":"2024-09-11T23:35:08.847466+0000","last_undegraded":"2024-09-11T23:35:08.847466+0000","last_fullsized":"2024-09-11T23:35:08.847466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:38.058806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847715+0000","last_change":"2024-09-11T23:34:52.871346+0000","last_active":"2024-09-11T23:35:08.847715+0000","last_peered":"2024-09-11T23:35:08.847715+0000","last_clean":"2024-09-11T23:35:08.847715+0000","last_became_active":"2024-09-11T23:34:52.870981+0000","last_became_peered":"2024-09-11T23:34:52.870981+0000","last_unstale":"2024-09-11T23:35:08.847715+0000","last_undegraded":"2024-09-11T23:35:08.847715+0000","last_fullsized":"2024-09-11T23:35:08.847715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:56.973998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844737+0000","last_change":"2024-09-11T23:34:52.867058+0000","last_active":"2024-09-11T23:35:08.844737+0000","last_peered":"2024-09-11T23:35:08.844737+0000","last_clean":"2024-09-11T23:35:08.844737+0000","last_became_active":"2024-09-11T23:34:52.866687+0000","last_became_peered":"2024-09-11T23:34:52.866687+0000","last_unstale":"2024-09-11T23:35:08.844737+0000","last_undegraded":"2024-09-11T23:35:08.844737+0000","last_fullsized":"2024-09-11T23:35:08.844737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:09.019987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844347+0000","last_change":"2024-09-11T23:34:52.870882+0000","last_active":"2024-09-11T23:35:08.844347+0000","last_peered":"2024-09-11T23:35:08.844347+0000","last_clean":"2024-09-11T23:35:08.844347+0000","last_became_active":"2024-09-11T23:34:52.870667+0000","last_became_peered":"2024-09-11T23:34:52.870667+0000","last_unstale":"2024-09-11T23:35:08.844347+0000","last_undegraded":"2024-09-11T23:35:08.844347+0000","last_fullsized":"2024-09-11T23:35:08.844347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:08.386703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850359+0000","last_change":"2024-09-11T23:34:52.874070+0000","last_active":"2024-09-11T23:35:08.850359+0000","last_peered":"2024-09-11T23:35:08.850359+0000","last_clean":"2024-09-11T23:35:08.850359+0000","last_became_active":"2024-09-11T23:34:52.873772+0000","last_became_peered":"2024-09-11T23:34:52.873772+0000","last_unstale":"2024-09-11T23:35:08.850359+0000","last_undegraded":"2024-09-11T23:35:08.850359+0000","last_fullsized":"2024-09-11T23:35:08.850359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:49.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847766+0000","last_change":"2024-09-11T23:34:52.875502+0000","last_active":"2024-09-11T23:35:08.847766+0000","last_peered":"2024-09-11T23:35:08.847766+0000","last_clean":"2024-09-11T23:35:08.847766+0000","last_became_active":"2024-09-11T23:34:52.875193+0000","last_became_peered":"2024-09-11T23:34:52.875193+0000","last_unstale":"2024-09-11T23:35:08.847766+0000","last_undegraded":"2024-09-11T23:35:08.847766+0000","last_fullsized":"2024-09-11T23:35:08.847766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:37.213828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455450+0000","last_change":"2024-09-11T23:34:52.869015+0000","last_active":"2024-09-11T23:35:09.455450+0000","last_peered":"2024-09-11T23:35:09.455450+0000","last_clean":"2024-09-11T23:35:09.455450+0000","last_became_active":"2024-09-11T23:34:52.868731+0000","last_became_peered":"2024-09-11T23:34:52.868731+0000","last_unstale":"2024-09-11T23:35:09.455450+0000","last_undegraded":"2024-09-11T23:35:09.455450+0000","last_fullsized":"2024-09-11T23:35:09.455450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:12:15.993203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848557+0000","last_change":"2024-09-11T23:34:52.875555+0000","last_active":"2024-09-11T23:35:08.848557+0000","last_peered":"2024-09-11T23:35:08.848557+0000","last_clean":"2024-09-11T23:35:08.848557+0000","last_became_active":"2024-09-11T23:34:52.875219+0000","last_became_peered":"2024-09-11T23:34:52.875219+0000","last_unstale":"2024-09-11T23:35:08.848557+0000","last_undegraded":"2024-09-11T23:35:08.848557+0000","last_fullsized":"2024-09-11T23:35:08.848557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:16.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848205+0000","last_change":"2024-09-11T23:34:52.875629+0000","last_active":"2024-09-11T23:35:08.848205+0000","last_peered":"2024-09-11T23:35:08.848205+0000","last_clean":"2024-09-11T23:35:08.848205+0000","last_became_active":"2024-09-11T23:34:52.875302+0000","last_became_peered":"2024-09-11T23:34:52.875302+0000","last_unstale":"2024-09-11T23:35:08.848205+0000","last_undegraded":"2024-09-11T23:35:08.848205+0000","last_fullsized":"2024-09-11T23:35:08.848205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:12.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844886+0000","last_change":"2024-09-11T23:34:52.868613+0000","last_active":"2024-09-11T23:35:08.844886+0000","last_peered":"2024-09-11T23:35:08.844886+0000","last_clean":"2024-09-11T23:35:08.844886+0000","last_became_active":"2024-09-11T23:34:52.868184+0000","last_became_peered":"2024-09-11T23:34:52.868184+0000","last_unstale":"2024-09-11T23:35:08.844886+0000","last_undegraded":"2024-09-11T23:35:08.844886+0000","last_fullsized":"2024-09-11T23:35:08.844886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:25.990002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.873390+0000","last_change":"2024-09-11T23:35:08.873390+0000","last_active":"2024-09-11T23:35:08.873390+0000","last_peered":"2024-09-11T23:35:08.873390+0000","last_clean":"2024-09-11T23:35:08.873390+0000","last_became_active":"2024-09-11T23:35:08.873052+0000","last_became_peered":"2024-09-11T23:35:08.873052+0000","last_unstale":"2024-09-11T23:35:08.873390+0000","last_undegraded":"2024-09-11T23:35:08.873390+0000","last_fullsized":"2024-09-11T23:35:08.873390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.937951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872829+0000","last_change":"2024-09-11T23:34:52.873604+0000","last_active":"2024-09-11T23:35:08.872829+0000","last_peered":"2024-09-11T23:35:08.872829+0000","last_clean":"2024-09-11T23:35:08.872829+0000","last_became_active":"2024-09-11T23:34:52.873220+0000","last_became_peered":"2024-09-11T23:34:52.873220+0000","last_unstale":"2024-09-11T23:35:08.872829+0000","last_undegraded":"2024-09-11T23:35:08.872829+0000","last_fullsized":"2024-09-11T23:35:08.872829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:38.480630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847599+0000","last_change":"2024-09-11T23:34:52.872902+0000","last_active":"2024-09-11T23:35:08.847599+0000","last_peered":"2024-09-11T23:35:08.847599+0000","last_clean":"2024-09-11T23:35:08.847599+0000","last_became_active":"2024-09-11T23:34:52.872571+0000","last_became_peered":"2024-09-11T23:34:52.872571+0000","last_unstale":"2024-09-11T23:35:08.847599+0000","last_undegraded":"2024-09-11T23:35:08.847599+0000","last_fullsized":"2024-09-11T23:35:08.847599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:21.844995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.457056+0000","last_change":"2024-09-11T23:35:09.457056+0000","last_active":"2024-09-11T23:35:09.457056+0000","last_peered":"2024-09-11T23:35:09.457056+0000","last_clean":"2024-09-11T23:35:09.457056+0000","last_became_active":"2024-09-11T23:35:09.456657+0000","last_became_peered":"2024-09-11T23:35:09.456657+0000","last_unstale":"2024-09-11T23:35:09.457056+0000","last_undegraded":"2024-09-11T23:35:09.457056+0000","last_fullsized":"2024-09-11T23:35:09.457056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:12:56.121280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844633+0000","last_change":"2024-09-11T23:34:52.875312+0000","last_active":"2024-09-11T23:35:08.844633+0000","last_peered":"2024-09-11T23:35:08.844633+0000","last_clean":"2024-09-11T23:35:08.844633+0000","last_became_active":"2024-09-11T23:34:52.874873+0000","last_became_peered":"2024-09-11T23:34:52.874873+0000","last_unstale":"2024-09-11T23:35:08.844633+0000","last_undegraded":"2024-09-11T23:35:08.844633+0000","last_fullsized":"2024-09-11T23:35:08.844633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:18.894713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847244+0000","last_change":"2024-09-11T23:34:52.862266+0000","last_active":"2024-09-11T23:35:08.847244+0000","last_peered":"2024-09-11T23:35:08.847244+0000","last_clean":"2024-09-11T23:35:08.847244+0000","last_became_active":"2024-09-11T23:34:52.861923+0000","last_became_peered":"2024-09-11T23:34:52.861923+0000","last_unstale":"2024-09-11T23:35:08.847244+0000","last_undegraded":"2024-09-11T23:35:08.847244+0000","last_fullsized":"2024-09-11T23:35:08.847244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:23.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:14.462701+0000","last_change":"2024-09-11T23:34:50.854020+0000","last_active":"2024-09-11T23:35:14.462701+0000","last_peered":"2024-09-11T23:35:14.462701+0000","last_clean":"2024-09-11T23:35:14.462701+0000","last_became_active":"2024-09-11T23:34:50.853683+0000","last_became_peered":"2024-09-11T23:34:50.853683+0000","last_unstale":"2024-09-11T23:35:14.462701+0000","last_undegraded":"2024-09-11T23:35:14.462701+0000","last_fullsized":"2024-09-11T23:35:14.462701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:18.725495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455234+0000","last_change":"2024-09-11T23:34:52.874016+0000","last_active":"2024-09-11T23:35:09.455234+0000","last_peered":"2024-09-11T23:35:09.455234+0000","last_clean":"2024-09-11T23:35:09.455234+0000","last_became_active":"2024-09-11T23:34:52.873675+0000","last_became_peered":"2024-09-11T23:34:52.873675+0000","last_unstale":"2024-09-11T23:35:09.455234+0000","last_undegraded":"2024-09-11T23:35:09.455234+0000","last_fullsized":"2024-09-11T23:35:09.455234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:41:28.955548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848555+0000","last_change":"2024-09-11T23:34:50.855391+0000","last_active":"2024-09-11T23:35:08.848555+0000","last_peered":"2024-09-11T23:35:08.848555+0000","last_clean":"2024-09-11T23:35:08.848555+0000","last_became_active":"2024-09-11T23:34:50.855052+0000","last_became_peered":"2024-09-11T23:34:50.855052+0000","last_unstale":"2024-09-11T23:35:08.848555+0000","last_undegraded":"2024-09-11T23:35:08.848555+0000","last_fullsized":"2024-09-11T23:35:08.848555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.972239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844031+0000","last_change":"2024-09-11T23:34:50.848985+0000","last_active":"2024-09-11T23:35:08.844031+0000","last_peered":"2024-09-11T23:35:08.844031+0000","last_clean":"2024-09-11T23:35:08.844031+0000","last_became_active":"2024-09-11T23:34:50.848614+0000","last_became_peered":"2024-09-11T23:34:50.848614+0000","last_unstale":"2024-09-11T23:35:08.844031+0000","last_undegraded":"2024-09-11T23:35:08.844031+0000","last_fullsized":"2024-09-11T23:35:08.844031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:43:04.723136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847368+0000","last_change":"2024-09-11T23:34:52.866933+0000","last_active":"2024-09-11T23:35:08.847368+0000","last_peered":"2024-09-11T23:35:08.847368+0000","last_clean":"2024-09-11T23:35:08.847368+0000","last_became_active":"2024-09-11T23:34:52.866559+0000","last_became_peered":"2024-09-11T23:34:52.866559+0000","last_unstale":"2024-09-11T23:35:08.847368+0000","last_undegraded":"2024-09-11T23:35:08.847368+0000","last_fullsized":"2024-09-11T23:35:08.847368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:27.745118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456265+0000","last_change":"2024-09-11T23:34:50.849787+0000","last_active":"2024-09-11T23:35:09.456265+0000","last_peered":"2024-09-11T23:35:09.456265+0000","last_clean":"2024-09-11T23:35:09.456265+0000","last_became_active":"2024-09-11T23:34:50.849368+0000","last_became_peered":"2024-09-11T23:34:50.849368+0000","last_unstale":"2024-09-11T23:35:09.456265+0000","last_undegraded":"2024-09-11T23:35:09.456265+0000","last_fullsized":"2024-09-11T23:35:09.456265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:26.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847291+0000","last_change":"2024-09-11T23:34:52.870832+0000","last_active":"2024-09-11T23:35:08.847291+0000","last_peered":"2024-09-11T23:35:08.847291+0000","last_clean":"2024-09-11T23:35:08.847291+0000","last_became_active":"2024-09-11T23:34:52.870474+0000","last_became_peered":"2024-09-11T23:34:52.870474+0000","last_unstale":"2024-09-11T23:35:08.847291+0000","last_undegraded":"2024-09-11T23:35:08.847291+0000","last_fullsized":"2024-09-11T23:35:08.847291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:30.196114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":56,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:18.467544+0000","last_change":"2024-09-11T23:34:50.859074+0000","last_active":"2024-09-11T23:35:18.467544+0000","last_peered":"2024-09-11T23:35:18.467544+0000","last_clean":"2024-09-11T23:35:18.467544+0000","last_became_active":"2024-09-11T23:34:50.858664+0000","last_became_peered":"2024-09-11T23:34:50.858664+0000","last_unstale":"2024-09-11T23:35:18.467544+0000","last_undegraded":"2024-09-11T23:35:18.467544+0000","last_fullsized":"2024-09-11T23:35:18.467544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:04.077813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850853+0000","last_change":"2024-09-11T23:34:52.875901+0000","last_active":"2024-09-11T23:35:08.850853+0000","last_peered":"2024-09-11T23:35:08.850853+0000","last_clean":"2024-09-11T23:35:08.850853+0000","last_became_active":"2024-09-11T23:34:52.875568+0000","last_became_peered":"2024-09-11T23:34:52.875568+0000","last_unstale":"2024-09-11T23:35:08.850853+0000","last_undegraded":"2024-09-11T23:35:08.850853+0000","last_fullsized":"2024-09-11T23:35:08.850853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:54.173548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846938+0000","last_change":"2024-09-11T23:34:50.854957+0000","last_active":"2024-09-11T23:35:08.846938+0000","last_peered":"2024-09-11T23:35:08.846938+0000","last_clean":"2024-09-11T23:35:08.846938+0000","last_became_active":"2024-09-11T23:34:50.854626+0000","last_became_peered":"2024-09-11T23:34:50.854626+0000","last_unstale":"2024-09-11T23:35:08.846938+0000","last_undegraded":"2024-09-11T23:35:08.846938+0000","last_fullsized":"2024-09-11T23:35:08.846938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:53.738966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872227+0000","last_change":"2024-09-11T23:34:52.868850+0000","last_active":"2024-09-11T23:35:08.872227+0000","last_peered":"2024-09-11T23:35:08.872227+0000","last_clean":"2024-09-11T23:35:08.872227+0000","last_became_active":"2024-09-11T23:34:52.868536+0000","last_became_peered":"2024-09-11T23:34:52.868536+0000","last_unstale":"2024-09-11T23:35:08.872227+0000","last_undegraded":"2024-09-11T23:35:08.872227+0000","last_fullsized":"2024-09-11T23:35:08.872227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:11.746821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456837+0000","last_change":"2024-09-11T23:34:50.924819+0000","last_active":"2024-09-11T23:35:09.456837+0000","last_peered":"2024-09-11T23:35:09.456837+0000","last_clean":"2024-09-11T23:35:09.456837+0000","last_became_active":"2024-09-11T23:34:50.924283+0000","last_became_peered":"2024-09-11T23:34:50.924283+0000","last_unstale":"2024-09-11T23:35:09.456837+0000","last_undegraded":"2024-09-11T23:35:09.456837+0000","last_fullsized":"2024-09-11T23:35:09.456837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:41.345233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850516+0000","last_change":"2024-09-11T23:34:52.867113+0000","last_active":"2024-09-11T23:35:08.850516+0000","last_peered":"2024-09-11T23:35:08.850516+0000","last_clean":"2024-09-11T23:35:08.850516+0000","last_became_active":"2024-09-11T23:34:52.866125+0000","last_became_peered":"2024-09-11T23:34:52.866125+0000","last_unstale":"2024-09-11T23:35:08.850516+0000","last_undegraded":"2024-09-11T23:35:08.850516+0000","last_fullsized":"2024-09-11T23:35:08.850516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:18:55.615079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'27","reported_seq":88,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:18.468346+0000","last_change":"2024-09-11T23:34:50.857272+0000","last_active":"2024-09-11T23:35:18.468346+0000","last_peered":"2024-09-11T23:35:18.468346+0000","last_clean":"2024-09-11T23:35:18.468346+0000","last_became_active":"2024-09-11T23:34:50.855701+0000","last_became_peered":"2024-09-11T23:34:50.855701+0000","last_unstale":"2024-09-11T23:35:18.468346+0000","last_undegraded":"2024-09-11T23:35:18.468346+0000","last_fullsized":"2024-09-11T23:35:18.468346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:49.997617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49603,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":67,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848168+0000","last_change":"2024-09-11T23:34:52.876928+0000","last_active":"2024-09-11T23:35:08.848168+0000","last_peered":"2024-09-11T23:35:08.848168+0000","last_clean":"2024-09-11T23:35:08.848168+0000","last_became_active":"2024-09-11T23:34:52.876578+0000","last_became_peered":"2024-09-11T23:34:52.876578+0000","last_unstale":"2024-09-11T23:35:08.848168+0000","last_undegraded":"2024-09-11T23:35:08.848168+0000","last_fullsized":"2024-09-11T23:35:08.848168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:07.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.870264+0000","last_change":"2024-09-11T23:35:08.869275+0000","last_active":"2024-09-11T23:35:08.870264+0000","last_peered":"2024-09-11T23:35:08.870264+0000","last_clean":"2024-09-11T23:35:08.870264+0000","last_became_active":"2024-09-11T23:35:08.850112+0000","last_became_peered":"2024-09-11T23:35:08.850112+0000","last_unstale":"2024-09-11T23:35:08.870264+0000","last_undegraded":"2024-09-11T23:35:08.870264+0000","last_fullsized":"2024-09-11T23:35:08.870264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:36.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844084+0000","last_change":"2024-09-11T23:34:52.871970+0000","last_active":"2024-09-11T23:35:08.844084+0000","last_peered":"2024-09-11T23:35:08.844084+0000","last_clean":"2024-09-11T23:35:08.844084+0000","last_became_active":"2024-09-11T23:34:52.871488+0000","last_became_peered":"2024-09-11T23:34:52.871488+0000","last_unstale":"2024-09-11T23:35:08.844084+0000","last_undegraded":"2024-09-11T23:35:08.844084+0000","last_fullsized":"2024-09-11T23:35:08.844084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:08.266431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456386+0000","last_change":"2024-09-11T23:34:50.854023+0000","last_active":"2024-09-11T23:35:09.456386+0000","last_peered":"2024-09-11T23:35:09.456386+0000","last_clean":"2024-09-11T23:35:09.456386+0000","last_became_active":"2024-09-11T23:34:50.853661+0000","last_became_peered":"2024-09-11T23:34:50.853661+0000","last_unstale":"2024-09-11T23:35:09.456386+0000","last_undegraded":"2024-09-11T23:35:09.456386+0000","last_fullsized":"2024-09-11T23:35:09.456386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:25.281976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.849648+0000","last_change":"2024-09-11T23:35:08.849648+0000","last_active":"2024-09-11T23:35:08.849648+0000","last_peered":"2024-09-11T23:35:08.849648+0000","last_clean":"2024-09-11T23:35:08.849648+0000","last_became_active":"2024-09-11T23:35:08.849246+0000","last_became_peered":"2024-09-11T23:35:08.849246+0000","last_unstale":"2024-09-11T23:35:08.849648+0000","last_undegraded":"2024-09-11T23:35:08.849648+0000","last_fullsized":"2024-09-11T23:35:08.849648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:17.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:15.463533+0000","last_change":"2024-09-11T23:34:50.853855+0000","last_active":"2024-09-11T23:35:15.463533+0000","last_peered":"2024-09-11T23:35:15.463533+0000","last_clean":"2024-09-11T23:35:15.463533+0000","last_became_active":"2024-09-11T23:34:50.853493+0000","last_became_peered":"2024-09-11T23:34:50.853493+0000","last_unstale":"2024-09-11T23:35:15.463533+0000","last_undegraded":"2024-09-11T23:35:15.463533+0000","last_fullsized":"2024-09-11T23:35:15.463533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:23.262472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846505+0000","last_change":"2024-09-11T23:34:52.875835+0000","last_active":"2024-09-11T23:35:08.846505+0000","last_peered":"2024-09-11T23:35:08.846505+0000","last_clean":"2024-09-11T23:35:08.846505+0000","last_became_active":"2024-09-11T23:34:52.875525+0000","last_became_peered":"2024-09-11T23:34:52.875525+0000","last_unstale":"2024-09-11T23:35:08.846505+0000","last_undegraded":"2024-09-11T23:35:08.846505+0000","last_fullsized":"2024-09-11T23:35:08.846505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:41.031816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847151+0000","last_change":"2024-09-11T23:34:50.924803+0000","last_active":"2024-09-11T23:35:08.847151+0000","last_peered":"2024-09-11T23:35:08.847151+0000","last_clean":"2024-09-11T23:35:08.847151+0000","last_became_active":"2024-09-11T23:34:50.924217+0000","last_became_peered":"2024-09-11T23:34:50.924217+0000","last_unstale":"2024-09-11T23:35:08.847151+0000","last_undegraded":"2024-09-11T23:35:08.847151+0000","last_fullsized":"2024-09-11T23:35:08.847151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:45.790592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.852839+0000","last_change":"2024-09-11T23:35:08.851856+0000","last_active":"2024-09-11T23:35:08.852839+0000","last_peered":"2024-09-11T23:35:08.852839+0000","last_clean":"2024-09-11T23:35:08.852839+0000","last_became_active":"2024-09-11T23:35:08.851490+0000","last_became_peered":"2024-09-11T23:35:08.851490+0000","last_unstale":"2024-09-11T23:35:08.852839+0000","last_undegraded":"2024-09-11T23:35:08.852839+0000","last_fullsized":"2024-09-11T23:35:08.852839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:46.269022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847890+0000","last_change":"2024-09-11T23:34:50.848385+0000","last_active":"2024-09-11T23:35:08.847890+0000","last_peered":"2024-09-11T23:35:08.847890+0000","last_clean":"2024-09-11T23:35:08.847890+0000","last_became_active":"2024-09-11T23:34:50.847808+0000","last_became_peered":"2024-09-11T23:34:50.847808+0000","last_unstale":"2024-09-11T23:35:08.847890+0000","last_undegraded":"2024-09-11T23:35:08.847890+0000","last_fullsized":"2024-09-11T23:35:08.847890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:38.796622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844817+0000","last_change":"2024-09-11T23:34:52.867341+0000","last_active":"2024-09-11T23:35:08.844817+0000","last_peered":"2024-09-11T23:35:08.844817+0000","last_clean":"2024-09-11T23:35:08.844817+0000","last_became_active":"2024-09-11T23:34:52.866161+0000","last_became_peered":"2024-09-11T23:34:52.866161+0000","last_unstale":"2024-09-11T23:35:08.844817+0000","last_undegraded":"2024-09-11T23:35:08.844817+0000","last_fullsized":"2024-09-11T23:35:08.844817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:27.913705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844372+0000","last_change":"2024-09-11T23:34:50.851466+0000","last_active":"2024-09-11T23:35:08.844372+0000","last_peered":"2024-09-11T23:35:08.844372+0000","last_clean":"2024-09-11T23:35:08.844372+0000","last_became_active":"2024-09-11T23:34:50.851191+0000","last_became_peered":"2024-09-11T23:34:50.851191+0000","last_unstale":"2024-09-11T23:35:08.844372+0000","last_undegraded":"2024-09-11T23:35:08.844372+0000","last_fullsized":"2024-09-11T23:35:08.844372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:00.328658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847008+0000","last_change":"2024-09-11T23:34:52.872290+0000","last_active":"2024-09-11T23:35:08.847008+0000","last_peered":"2024-09-11T23:35:08.847008+0000","last_clean":"2024-09-11T23:35:08.847008+0000","last_became_active":"2024-09-11T23:34:52.871935+0000","last_became_peered":"2024-09-11T23:34:52.871935+0000","last_unstale":"2024-09-11T23:35:08.847008+0000","last_undegraded":"2024-09-11T23:35:08.847008+0000","last_fullsized":"2024-09-11T23:35:08.847008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:08.217548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456729+0000","last_change":"2024-09-11T23:34:50.857763+0000","last_active":"2024-09-11T23:35:09.456729+0000","last_peered":"2024-09-11T23:35:09.456729+0000","last_clean":"2024-09-11T23:35:09.456729+0000","last_became_active":"2024-09-11T23:34:50.857427+0000","last_became_peered":"2024-09-11T23:34:50.857427+0000","last_unstale":"2024-09-11T23:35:09.456729+0000","last_undegraded":"2024-09-11T23:35:09.456729+0000","last_fullsized":"2024-09-11T23:35:09.456729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:59.703580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846908+0000","last_change":"2024-09-11T23:34:52.872426+0000","last_active":"2024-09-11T23:35:08.846908+0000","last_peered":"2024-09-11T23:35:08.846908+0000","last_clean":"2024-09-11T23:35:08.846908+0000","last_became_active":"2024-09-11T23:34:52.872090+0000","last_became_peered":"2024-09-11T23:34:52.872090+0000","last_unstale":"2024-09-11T23:35:08.846908+0000","last_undegraded":"2024-09-11T23:35:08.846908+0000","last_fullsized":"2024-09-11T23:35:08.846908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:04:00.963963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'6","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:15.474038+0000","last_change":"2024-09-11T23:34:50.924792+0000","last_active":"2024-09-11T23:35:15.474038+0000","last_peered":"2024-09-11T23:35:15.474038+0000","last_clean":"2024-09-11T23:35:15.474038+0000","last_became_active":"2024-09-11T23:34:50.924444+0000","last_became_peered":"2024-09-11T23:34:50.924444+0000","last_unstale":"2024-09-11T23:35:15.474038+0000","last_undegraded":"2024-09-11T23:35:15.474038+0000","last_fullsized":"2024-09-11T23:35:15.474038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:12.855314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846881+0000","last_change":"2024-09-11T23:34:52.872079+0000","last_active":"2024-09-11T23:35:08.846881+0000","last_peered":"2024-09-11T23:35:08.846881+0000","last_clean":"2024-09-11T23:35:08.846881+0000","last_became_active":"2024-09-11T23:34:52.871755+0000","last_became_peered":"2024-09-11T23:34:52.871755+0000","last_unstale":"2024-09-11T23:35:08.846881+0000","last_undegraded":"2024-09-11T23:35:08.846881+0000","last_fullsized":"2024-09-11T23:35:08.846881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:05.990832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":344,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846729+0000","last_change":"2024-09-11T23:33:42.256521+0000","last_active":"2024-09-11T23:35:08.846729+0000","last_peered":"2024-09-11T23:35:08.846729+0000","last_clean":"2024-09-11T23:35:08.846729+0000","last_became_active":"2024-09-11T23:33:42.256041+0000","last_became_peered":"2024-09-11T23:33:42.256041+0000","last_unstale":"2024-09-11T23:35:08.846729+0000","last_undegraded":"2024-09-11T23:35:08.846729+0000","last_fullsized":"2024-09-11T23:35:08.846729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_clean_scrub_stamp":"2024-09-11T23:30:22.061515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:21.391839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845844+0000","last_change":"2024-09-11T23:34:50.854087+0000","last_active":"2024-09-11T23:35:08.845844+0000","last_peered":"2024-09-11T23:35:08.845844+0000","last_clean":"2024-09-11T23:35:08.845844+0000","last_became_active":"2024-09-11T23:34:50.852346+0000","last_became_peered":"2024-09-11T23:34:50.852346+0000","last_unstale":"2024-09-11T23:35:08.845844+0000","last_undegraded":"2024-09-11T23:35:08.845844+0000","last_fullsized":"2024-09-11T23:35:08.845844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.287119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847645+0000","last_change":"2024-09-11T23:34:52.869198+0000","last_active":"2024-09-11T23:35:08.847645+0000","last_peered":"2024-09-11T23:35:08.847645+0000","last_clean":"2024-09-11T23:35:08.847645+0000","last_became_active":"2024-09-11T23:34:52.868621+0000","last_became_peered":"2024-09-11T23:34:52.868621+0000","last_unstale":"2024-09-11T23:35:08.847645+0000","last_undegraded":"2024-09-11T23:35:08.847645+0000","last_fullsized":"2024-09-11T23:35:08.847645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:17.430654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846715+0000","last_change":"2024-09-11T23:34:50.852752+0000","last_active":"2024-09-11T23:35:08.846715+0000","last_peered":"2024-09-11T23:35:08.846715+0000","last_clean":"2024-09-11T23:35:08.846715+0000","last_became_active":"2024-09-11T23:34:50.851895+0000","last_became_peered":"2024-09-11T23:34:50.851895+0000","last_unstale":"2024-09-11T23:35:08.846715+0000","last_undegraded":"2024-09-11T23:35:08.846715+0000","last_fullsized":"2024-09-11T23:35:08.846715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:46.894432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844043+0000","last_change":"2024-09-11T23:34:52.866191+0000","last_active":"2024-09-11T23:35:08.844043+0000","last_peered":"2024-09-11T23:35:08.844043+0000","last_clean":"2024-09-11T23:35:08.844043+0000","last_became_active":"2024-09-11T23:34:52.865858+0000","last_became_peered":"2024-09-11T23:34:52.865858+0000","last_unstale":"2024-09-11T23:35:08.844043+0000","last_undegraded":"2024-09-11T23:35:08.844043+0000","last_fullsized":"2024-09-11T23:35:08.844043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:21.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:15.462941+0000","last_change":"2024-09-11T23:34:50.924125+0000","last_active":"2024-09-11T23:35:15.462941+0000","last_peered":"2024-09-11T23:35:15.462941+0000","last_clean":"2024-09-11T23:35:15.462941+0000","last_became_active":"2024-09-11T23:34:50.923780+0000","last_became_peered":"2024-09-11T23:34:50.923780+0000","last_unstale":"2024-09-11T23:35:15.462941+0000","last_undegraded":"2024-09-11T23:35:15.462941+0000","last_fullsized":"2024-09-11T23:35:15.462941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:39.774827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455096+0000","last_change":"2024-09-11T23:34:52.868364+0000","last_active":"2024-09-11T23:35:09.455096+0000","last_peered":"2024-09-11T23:35:09.455096+0000","last_clean":"2024-09-11T23:35:09.455096+0000","last_became_active":"2024-09-11T23:34:52.867913+0000","last_became_peered":"2024-09-11T23:34:52.867913+0000","last_unstale":"2024-09-11T23:35:09.455096+0000","last_undegraded":"2024-09-11T23:35:09.455096+0000","last_fullsized":"2024-09-11T23:35:09.455096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.571825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:14.472171+0000","last_change":"2024-09-11T23:34:50.852421+0000","last_active":"2024-09-11T23:35:14.472171+0000","last_peered":"2024-09-11T23:35:14.472171+0000","last_clean":"2024-09-11T23:35:14.472171+0000","last_became_active":"2024-09-11T23:34:50.851898+0000","last_became_peered":"2024-09-11T23:34:50.851898+0000","last_unstale":"2024-09-11T23:35:14.472171+0000","last_undegraded":"2024-09-11T23:35:14.472171+0000","last_fullsized":"2024-09-11T23:35:14.472171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.661276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848710+0000","last_change":"2024-09-11T23:34:52.879995+0000","last_active":"2024-09-11T23:35:08.848710+0000","last_peered":"2024-09-11T23:35:08.848710+0000","last_clean":"2024-09-11T23:35:08.848710+0000","last_became_active":"2024-09-11T23:34:52.879678+0000","last_became_peered":"2024-09-11T23:34:52.879678+0000","last_unstale":"2024-09-11T23:35:08.848710+0000","last_undegraded":"2024-09-11T23:35:08.848710+0000","last_fullsized":"2024-09-11T23:35:08.848710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:47.456674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847399+0000","last_change":"2024-09-11T23:34:50.924957+0000","last_active":"2024-09-11T23:35:08.847399+0000","last_peered":"2024-09-11T23:35:08.847399+0000","last_clean":"2024-09-11T23:35:08.847399+0000","last_became_active":"2024-09-11T23:34:50.924500+0000","last_became_peered":"2024-09-11T23:34:50.924500+0000","last_unstale":"2024-09-11T23:35:08.847399+0000","last_undegraded":"2024-09-11T23:35:08.847399+0000","last_fullsized":"2024-09-11T23:35:08.847399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:39.268842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848946+0000","last_change":"2024-09-11T23:34:52.867337+0000","last_active":"2024-09-11T23:35:08.848946+0000","last_peered":"2024-09-11T23:35:08.848946+0000","last_clean":"2024-09-11T23:35:08.848946+0000","last_became_active":"2024-09-11T23:34:52.866799+0000","last_became_peered":"2024-09-11T23:34:52.866799+0000","last_unstale":"2024-09-11T23:35:08.848946+0000","last_undegraded":"2024-09-11T23:35:08.848946+0000","last_fullsized":"2024-09-11T23:35:08.848946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:38.177135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848411+0000","last_change":"2024-09-11T23:34:50.925156+0000","last_active":"2024-09-11T23:35:08.848411+0000","last_peered":"2024-09-11T23:35:08.848411+0000","last_clean":"2024-09-11T23:35:08.848411+0000","last_became_active":"2024-09-11T23:34:50.924282+0000","last_became_peered":"2024-09-11T23:34:50.924282+0000","last_unstale":"2024-09-11T23:35:08.848411+0000","last_undegraded":"2024-09-11T23:35:08.848411+0000","last_fullsized":"2024-09-11T23:35:08.848411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:47.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844337+0000","last_change":"2024-09-11T23:34:52.864993+0000","last_active":"2024-09-11T23:35:08.844337+0000","last_peered":"2024-09-11T23:35:08.844337+0000","last_clean":"2024-09-11T23:35:08.844337+0000","last_became_active":"2024-09-11T23:34:52.864267+0000","last_became_peered":"2024-09-11T23:34:52.864267+0000","last_unstale":"2024-09-11T23:35:08.844337+0000","last_undegraded":"2024-09-11T23:35:08.844337+0000","last_fullsized":"2024-09-11T23:35:08.844337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:29.912068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:14.449437+0000","last_change":"2024-09-11T23:34:50.849864+0000","last_active":"2024-09-11T23:35:14.449437+0000","last_peered":"2024-09-11T23:35:14.449437+0000","last_clean":"2024-09-11T23:35:14.449437+0000","last_became_active":"2024-09-11T23:34:50.849577+0000","last_became_peered":"2024-09-11T23:34:50.849577+0000","last_unstale":"2024-09-11T23:35:14.449437+0000","last_undegraded":"2024-09-11T23:35:14.449437+0000","last_fullsized":"2024-09-11T23:35:14.449437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:06.845000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848578+0000","last_change":"2024-09-11T23:34:52.872028+0000","last_active":"2024-09-11T23:35:08.848578+0000","last_peered":"2024-09-11T23:35:08.848578+0000","last_clean":"2024-09-11T23:35:08.848578+0000","last_became_active":"2024-09-11T23:34:52.871723+0000","last_became_peered":"2024-09-11T23:34:52.871723+0000","last_unstale":"2024-09-11T23:35:08.848578+0000","last_undegraded":"2024-09-11T23:35:08.848578+0000","last_fullsized":"2024-09-11T23:35:08.848578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:44.656218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846269+0000","last_change":"2024-09-11T23:34:50.860314+0000","last_active":"2024-09-11T23:35:08.846269+0000","last_peered":"2024-09-11T23:35:08.846269+0000","last_clean":"2024-09-11T23:35:08.846269+0000","last_became_active":"2024-09-11T23:34:50.859953+0000","last_became_peered":"2024-09-11T23:34:50.859953+0000","last_unstale":"2024-09-11T23:35:08.846269+0000","last_undegraded":"2024-09-11T23:35:08.846269+0000","last_fullsized":"2024-09-11T23:35:08.846269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:46.705864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872092+0000","last_change":"2024-09-11T23:34:52.867190+0000","last_active":"2024-09-11T23:35:08.872092+0000","last_peered":"2024-09-11T23:35:08.872092+0000","last_clean":"2024-09-11T23:35:08.872092+0000","last_became_active":"2024-09-11T23:34:52.866829+0000","last_became_peered":"2024-09-11T23:34:52.866829+0000","last_unstale":"2024-09-11T23:35:08.872092+0000","last_undegraded":"2024-09-11T23:35:08.872092+0000","last_fullsized":"2024-09-11T23:35:08.872092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.752006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850343+0000","last_change":"2024-09-11T23:34:50.924566+0000","last_active":"2024-09-11T23:35:08.850343+0000","last_peered":"2024-09-11T23:35:08.850343+0000","last_clean":"2024-09-11T23:35:08.850343+0000","last_became_active":"2024-09-11T23:34:50.924284+0000","last_became_peered":"2024-09-11T23:34:50.924284+0000","last_unstale":"2024-09-11T23:35:08.850343+0000","last_undegraded":"2024-09-11T23:35:08.850343+0000","last_fullsized":"2024-09-11T23:35:08.850343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:54.246154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847042+0000","last_change":"2024-09-11T23:34:52.869302+0000","last_active":"2024-09-11T23:35:08.847042+0000","last_peered":"2024-09-11T23:35:08.847042+0000","last_clean":"2024-09-11T23:35:08.847042+0000","last_became_active":"2024-09-11T23:34:52.868960+0000","last_became_peered":"2024-09-11T23:34:52.868960+0000","last_unstale":"2024-09-11T23:35:08.847042+0000","last_undegraded":"2024-09-11T23:35:08.847042+0000","last_fullsized":"2024-09-11T23:35:08.847042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.561712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847953+0000","last_change":"2024-09-11T23:34:50.854573+0000","last_active":"2024-09-11T23:35:08.847953+0000","last_peered":"2024-09-11T23:35:08.847953+0000","last_clean":"2024-09-11T23:35:08.847953+0000","last_became_active":"2024-09-11T23:34:50.854165+0000","last_became_peered":"2024-09-11T23:34:50.854165+0000","last_unstale":"2024-09-11T23:35:08.847953+0000","last_undegraded":"2024-09-11T23:35:08.847953+0000","last_fullsized":"2024-09-11T23:35:08.847953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:57:31.589495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844752+0000","last_change":"2024-09-11T23:34:52.875987+0000","last_active":"2024-09-11T23:35:08.844752+0000","last_peered":"2024-09-11T23:35:08.844752+0000","last_clean":"2024-09-11T23:35:08.844752+0000","last_became_active":"2024-09-11T23:34:52.875662+0000","last_became_peered":"2024-09-11T23:34:52.875662+0000","last_unstale":"2024-09-11T23:35:08.844752+0000","last_undegraded":"2024-09-11T23:35:08.844752+0000","last_fullsized":"2024-09-11T23:35:08.844752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:16.514938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847766+0000","last_change":"2024-09-11T23:34:50.850726+0000","last_active":"2024-09-11T23:35:08.847766+0000","last_peered":"2024-09-11T23:35:08.847766+0000","last_clean":"2024-09-11T23:35:08.847766+0000","last_became_active":"2024-09-11T23:34:50.850381+0000","last_became_peered":"2024-09-11T23:34:50.850381+0000","last_unstale":"2024-09-11T23:35:08.847766+0000","last_undegraded":"2024-09-11T23:35:08.847766+0000","last_fullsized":"2024-09-11T23:35:08.847766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:50.431293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847564+0000","last_change":"2024-09-11T23:34:52.870652+0000","last_active":"2024-09-11T23:35:08.847564+0000","last_peered":"2024-09-11T23:35:08.847564+0000","last_clean":"2024-09-11T23:35:08.847564+0000","last_became_active":"2024-09-11T23:34:52.870303+0000","last_became_peered":"2024-09-11T23:34:52.870303+0000","last_unstale":"2024-09-11T23:35:08.847564+0000","last_undegraded":"2024-09-11T23:35:08.847564+0000","last_fullsized":"2024-09-11T23:35:08.847564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:17.670981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.855045+0000","last_change":"2024-09-11T23:35:08.854490+0000","last_active":"2024-09-11T23:35:08.855045+0000","last_peered":"2024-09-11T23:35:08.855045+0000","last_clean":"2024-09-11T23:35:08.855045+0000","last_became_active":"2024-09-11T23:35:08.849820+0000","last_became_peered":"2024-09-11T23:35:08.849820+0000","last_unstale":"2024-09-11T23:35:08.855045+0000","last_undegraded":"2024-09-11T23:35:08.855045+0000","last_fullsized":"2024-09-11T23:35:08.855045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:09.509867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846681+0000","last_change":"2024-09-11T23:34:52.869695+0000","last_active":"2024-09-11T23:35:08.846681+0000","last_peered":"2024-09-11T23:35:08.846681+0000","last_clean":"2024-09-11T23:35:08.846681+0000","last_became_active":"2024-09-11T23:34:52.869367+0000","last_became_peered":"2024-09-11T23:34:52.869367+0000","last_unstale":"2024-09-11T23:35:08.846681+0000","last_undegraded":"2024-09-11T23:35:08.846681+0000","last_fullsized":"2024-09-11T23:35:08.846681+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:16.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455461+0000","last_change":"2024-09-11T23:34:52.872778+0000","last_active":"2024-09-11T23:35:09.455461+0000","last_peered":"2024-09-11T23:35:09.455461+0000","last_clean":"2024-09-11T23:35:09.455461+0000","last_became_active":"2024-09-11T23:34:52.872451+0000","last_became_peered":"2024-09-11T23:34:52.872451+0000","last_unstale":"2024-09-11T23:35:09.455461+0000","last_undegraded":"2024-09-11T23:35:09.455461+0000","last_fullsized":"2024-09-11T23:35:09.455461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:51.769927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847892+0000","last_change":"2024-09-11T23:34:50.849675+0000","last_active":"2024-09-11T23:35:08.847892+0000","last_peered":"2024-09-11T23:35:08.847892+0000","last_clean":"2024-09-11T23:35:08.847892+0000","last_became_active":"2024-09-11T23:34:50.849228+0000","last_became_peered":"2024-09-11T23:34:50.849228+0000","last_unstale":"2024-09-11T23:35:08.847892+0000","last_undegraded":"2024-09-11T23:35:08.847892+0000","last_fullsized":"2024-09-11T23:35:08.847892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:50.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850319+0000","last_change":"2024-09-11T23:34:52.867739+0000","last_active":"2024-09-11T23:35:08.850319+0000","last_peered":"2024-09-11T23:35:08.850319+0000","last_clean":"2024-09-11T23:35:08.850319+0000","last_became_active":"2024-09-11T23:34:52.866613+0000","last_became_peered":"2024-09-11T23:34:52.866613+0000","last_unstale":"2024-09-11T23:35:08.850319+0000","last_undegraded":"2024-09-11T23:35:08.850319+0000","last_fullsized":"2024-09-11T23:35:08.850319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:49.348386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848269+0000","last_change":"2024-09-11T23:34:50.851401+0000","last_active":"2024-09-11T23:35:08.848269+0000","last_peered":"2024-09-11T23:35:08.848269+0000","last_clean":"2024-09-11T23:35:08.848269+0000","last_became_active":"2024-09-11T23:34:50.851068+0000","last_became_peered":"2024-09-11T23:34:50.851068+0000","last_unstale":"2024-09-11T23:35:08.848269+0000","last_undegraded":"2024-09-11T23:35:08.848269+0000","last_fullsized":"2024-09-11T23:35:08.848269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:09.673864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872365+0000","last_change":"2024-09-11T23:34:52.865216+0000","last_active":"2024-09-11T23:35:08.872365+0000","last_peered":"2024-09-11T23:35:08.872365+0000","last_clean":"2024-09-11T23:35:08.872365+0000","last_became_active":"2024-09-11T23:34:52.864917+0000","last_became_peered":"2024-09-11T23:34:52.864917+0000","last_unstale":"2024-09-11T23:35:08.872365+0000","last_undegraded":"2024-09-11T23:35:08.872365+0000","last_fullsized":"2024-09-11T23:35:08.872365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:36.469793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848560+0000","last_change":"2024-09-11T23:34:50.924291+0000","last_active":"2024-09-11T23:35:08.848560+0000","last_peered":"2024-09-11T23:35:08.848560+0000","last_clean":"2024-09-11T23:35:08.848560+0000","last_became_active":"2024-09-11T23:34:50.923892+0000","last_became_peered":"2024-09-11T23:34:50.923892+0000","last_unstale":"2024-09-11T23:35:08.848560+0000","last_undegraded":"2024-09-11T23:35:08.848560+0000","last_fullsized":"2024-09-11T23:35:08.848560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:52.266778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848164+0000","last_change":"2024-09-11T23:34:52.875995+0000","last_active":"2024-09-11T23:35:08.848164+0000","last_peered":"2024-09-11T23:35:08.848164+0000","last_clean":"2024-09-11T23:35:08.848164+0000","last_became_active":"2024-09-11T23:34:52.875654+0000","last_became_peered":"2024-09-11T23:34:52.875654+0000","last_unstale":"2024-09-11T23:35:08.848164+0000","last_undegraded":"2024-09-11T23:35:08.848164+0000","last_fullsized":"2024-09-11T23:35:08.848164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:19.218291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:14.471624+0000","last_change":"2024-09-11T23:34:50.853881+0000","last_active":"2024-09-11T23:35:14.471624+0000","last_peered":"2024-09-11T23:35:14.471624+0000","last_clean":"2024-09-11T23:35:14.471624+0000","last_became_active":"2024-09-11T23:34:50.853576+0000","last_became_peered":"2024-09-11T23:34:50.853576+0000","last_unstale":"2024-09-11T23:35:14.471624+0000","last_undegraded":"2024-09-11T23:35:14.471624+0000","last_fullsized":"2024-09-11T23:35:14.471624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:38.550587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56811,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":174,"num_read_kb":152,"num_write":236,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":170433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612625,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":434822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1075386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005083,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":434291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":434669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495653,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":901120,"data_stored":434008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888110,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1026474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018932,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":434262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247863,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":433833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444094,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":485069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607618,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":484524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":921600,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":51355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":51900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":51732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:35:21.107 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:35:21.108 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T23:35:21.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T23:35:21.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:20 smithi138 ceph-mon[35142]: pgmap v287: 97 pgs: 97 active+clean; 632 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 14 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:35:21.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:20 smithi178 ceph-mon[35882]: pgmap v287: 97 pgs: 97 active+clean; 632 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 14 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:35:21.358 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:21.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.816+0000 7f1ec09d0640 1 -- 172.21.15.133:0/1301487663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ebc102690 msgr2=0x7f1ebc108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:21.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.816+0000 7f1ec09d0640 1 --2- 172.21.15.133:0/1301487663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ebc102690 0x7f1ebc108f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009ca0 tx=0x7f1eb402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:35:21.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.816+0000 7f1ec09d0640 1 -- 172.21.15.133:0/1301487663 shutdown_connections 2024-09-11T23:35:21.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.816+0000 7f1ec09d0640 1 --2- 172.21.15.133:0/1301487663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ebc102690 0x7f1ebc108f20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:21.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.816+0000 7f1ec09d0640 1 --2- 172.21.15.133:0/1301487663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ebc101cd0 0x7f1ebc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:21.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.816+0000 7f1ec09d0640 1 --2- 172.21.15.133:0/1301487663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ebc100ad0 0x7f1ebc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:21.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.816+0000 7f1ec09d0640 1 -- 172.21.15.133:0/1301487663 >> 172.21.15.133:0/1301487663 conn(0x7f1ebc0fc280 msgr2=0x7f1ebc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:21.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.816+0000 7f1ec09d0640 1 -- 172.21.15.133:0/1301487663 shutdown_connections 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.816+0000 7f1ec09d0640 1 -- 172.21.15.133:0/1301487663 wait complete. 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.817+0000 7f1ec09d0640 1 Processor -- start 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.817+0000 7f1ec09d0640 1 -- start start 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.817+0000 7f1ec09d0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ebc100ad0 0x7f1ebc06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.817+0000 7f1ec09d0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ebc101cd0 0x7f1ebc19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.817+0000 7f1ec09d0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ebc102690 0x7f1ebc19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1ec09d0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc10c750 con 0x7f1ebc101cd0 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1ec09d0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc10c5d0 con 0x7f1ebc100ad0 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1ec09d0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc10c8d0 con 0x7f1ebc102690 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ebc101cd0 0x7f1ebc19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ebc101cd0 0x7f1ebc19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43228/0 (socket says 172.21.15.133:43228) 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb1d9b640 1 -- 172.21.15.133:0/2959066259 learned_addr learned my addr 172.21.15.133:0/2959066259 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1ebad9d640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ebc102690 0x7f1ebc19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb1d9b640 1 -- 172.21.15.133:0/2959066259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ebc102690 msgr2=0x7f1ebc19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb1d9b640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ebc102690 0x7f1ebc19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eba59c640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ebc100ad0 0x7f1ebc06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb1d9b640 1 -- 172.21.15.133:0/2959066259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ebc100ad0 msgr2=0x7f1ebc06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb1d9b640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ebc100ad0 0x7f1ebc06aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb1d9b640 1 -- 172.21.15.133:0/2959066259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ebc19fb00 con 0x7f1ebc101cd0 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb1d9b640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ebc101cd0 0x7f1ebc19cbd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400d770 tx=0x7f1ea400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb3fff640 1 -- 172.21.15.133:0/2959066259 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4017070 con 0x7f1ebc101cd0 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1ec09d0640 1 -- 172.21.15.133:0/2959066259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ebc19fd90 con 0x7f1ebc101cd0 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb3fff640 1 -- 172.21.15.133:0/2959066259 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea4005500 con 0x7f1ebc101cd0 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1eb3fff640 1 -- 172.21.15.133:0/2959066259 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4004e00 con 0x7f1ebc101cd0 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.818+0000 7f1ec09d0640 1 -- 172.21.15.133:0/2959066259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ebc1a0220 con 0x7f1ebc101cd0 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.819+0000 7f1ec09d0640 1 -- 172.21.15.133:0/2959066259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ebc107e50 con 0x7f1ebc101cd0 2024-09-11T23:35:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.820+0000 7f1eb3fff640 1 -- 172.21.15.133:0/2959066259 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1ea4004fa0 con 0x7f1ebc101cd0 2024-09-11T23:35:21.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.821+0000 7f1eb3fff640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1ea80a2c10 0x7f1ea80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:21.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.821+0000 7f1eba59c640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1ea80a2c10 0x7f1ea80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:21.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.821+0000 7f1eb3fff640 1 -- 172.21.15.133:0/2959066259 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f1ea40cef60 con 0x7f1ebc101cd0 2024-09-11T23:35:21.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.822+0000 7f1eba59c640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1ea80a2c10 0x7f1ea80a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005fd0 tx=0x7f1eac005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:21.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:21.824+0000 7f1eb3fff640 1 -- 172.21.15.133:0/2959066259 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ea409bd60 con 0x7f1ebc101cd0 2024-09-11T23:35:22.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.058+0000 7f1ec09d0640 1 -- 172.21.15.133:0/2959066259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1ebc02d7c0 con 0x7f1ebc101cd0 2024-09-11T23:35:22.058 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:21 smithi133 ceph-mon[31430]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:35:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.060+0000 7f1eb3fff640 1 -- 172.21.15.133:0/2959066259 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) ==== 170+0+59 (secure 0 0 0) 0x7f1ea409b660 con 0x7f1ebc101cd0 2024-09-11T23:35:22.060 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-09-11T23:35:22.060 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQA6KeJmkrWLAxAALnIXGXfxenON5p0gTJEdzQ== 2024-09-11T23:35:22.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 -- 172.21.15.133:0/2959066259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1ea80a2c10 msgr2=0x7f1ea80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:22.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1ea80a2c10 0x7f1ea80a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005fd0 tx=0x7f1eac005c70 comp rx=0 tx=0).stop 2024-09-11T23:35:22.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 -- 172.21.15.133:0/2959066259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ebc101cd0 msgr2=0x7f1ebc19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:22.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ebc101cd0 0x7f1ebc19cbd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400d770 tx=0x7f1ea400dc30 comp rx=0 tx=0).stop 2024-09-11T23:35:22.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 -- 172.21.15.133:0/2959066259 shutdown_connections 2024-09-11T23:35:22.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f1ea80a2c10 0x7f1ea80a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:22.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ebc102690 0x7f1ebc19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:22.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ebc101cd0 0x7f1ebc19cbd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:22.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 --2- 172.21.15.133:0/2959066259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ebc100ad0 0x7f1ebc06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:22.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 -- 172.21.15.133:0/2959066259 >> 172.21.15.133:0/2959066259 conn(0x7f1ebc0fc280 msgr2=0x7f1ebc105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:22.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 -- 172.21.15.133:0/2959066259 shutdown_connections 2024-09-11T23:35:22.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:22.063+0000 7f1ec09d0640 1 -- 172.21.15.133:0/2959066259 wait complete. 2024-09-11T23:35:22.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:21 smithi138 ceph-mon[35142]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:35:22.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:21 smithi178 ceph-mon[35882]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:35:22.885 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:35:22.885 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T23:35:22.885 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T23:35:22.921 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T23:35:22.921 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T23:35:22.922 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T23:35:23.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:22 smithi138 ceph-mon[35142]: pgmap v288: 97 pgs: 97 active+clean; 632 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.2 KiB/s wr, 12 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:35:23.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:22 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2959066259' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:35:23.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:22 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2959066259' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:35:23.205 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:22 smithi133 ceph-mon[31430]: pgmap v288: 97 pgs: 97 active+clean; 632 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.2 KiB/s wr, 12 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:35:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:22 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2959066259' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:35:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:22 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2959066259' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:35:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:22 smithi178 ceph-mon[35882]: pgmap v288: 97 pgs: 97 active+clean; 632 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.2 KiB/s wr, 12 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:35:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:22 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2959066259' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:35:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:22 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2959066259' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:35:23.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.666+0000 7fe07139e640 1 -- 172.21.15.133:0/2760427096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe06c101de0 msgr2=0x7fe06c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:23.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.666+0000 7fe07139e640 1 --2- 172.21.15.133:0/2760427096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe06c101de0 0x7fe06c1021c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009900 tx=0x7fe05c02a410 comp rx=0 tx=0).stop 2024-09-11T23:35:23.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.666+0000 7fe07139e640 1 -- 172.21.15.133:0/2760427096 shutdown_connections 2024-09-11T23:35:23.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.666+0000 7fe07139e640 1 --2- 172.21.15.133:0/2760427096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe06c106f10 0x7fe06c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:23.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.666+0000 7fe07139e640 1 --2- 172.21.15.133:0/2760427096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c102790 0x7fe06c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:23.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.666+0000 7fe07139e640 1 --2- 172.21.15.133:0/2760427096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe06c101de0 0x7fe06c1021c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:23.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.666+0000 7fe07139e640 1 -- 172.21.15.133:0/2760427096 >> 172.21.15.133:0/2760427096 conn(0x7fe06c0fd8c0 msgr2=0x7fe06c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:23.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.666+0000 7fe07139e640 1 -- 172.21.15.133:0/2760427096 shutdown_connections 2024-09-11T23:35:23.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.666+0000 7fe07139e640 1 -- 172.21.15.133:0/2760427096 wait complete. 2024-09-11T23:35:23.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe07139e640 1 Processor -- start 2024-09-11T23:35:23.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe07139e640 1 -- start start 2024-09-11T23:35:23.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe07139e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe06c101de0 0x7fe06c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:23.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe07139e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c102790 0x7fe06c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:23.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe07139e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe06c106f10 0x7fe06c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:23.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe07139e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c111f70 con 0x7fe06c106f10 2024-09-11T23:35:23.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe07139e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c111df0 con 0x7fe06c101de0 2024-09-11T23:35:23.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe07139e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c1120f0 con 0x7fe06c102790 2024-09-11T23:35:23.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe06b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe06c106f10 0x7fe06c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:23.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe06b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe06c106f10 0x7fe06c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52378/0 (socket says 172.21.15.133:52378) 2024-09-11T23:35:23.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe06b7fe640 1 -- 172.21.15.133:0/2194960814 learned_addr learned my addr 172.21.15.133:0/2194960814 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:23.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe06affd640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe06c101de0 0x7fe06c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:23.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe06b7fe640 1 -- 172.21.15.133:0/2194960814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c102790 msgr2=0x7fe06c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:23.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe0627fc640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c102790 0x7fe06c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:23.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe06b7fe640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c102790 0x7fe06c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:23.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe06b7fe640 1 -- 172.21.15.133:0/2194960814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe06c101de0 msgr2=0x7fe06c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:23.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe06b7fe640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe06c101de0 0x7fe06c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:23.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe06b7fe640 1 -- 172.21.15.133:0/2194960814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06c19d380 con 0x7fe06c106f10 2024-09-11T23:35:23.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe06affd640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe06c101de0 0x7fe06c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:23.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.667+0000 7fe0627fc640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c102790 0x7fe06c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:23.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.668+0000 7fe06b7fe640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe06c106f10 0x7fe06c19cca0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe06400cce0 tx=0x7fe06400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:23.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.668+0000 7fe068ff9640 1 -- 172.21.15.133:0/2194960814 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06400f450 con 0x7fe06c106f10 2024-09-11T23:35:23.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.668+0000 7fe068ff9640 1 -- 172.21.15.133:0/2194960814 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06400fad0 con 0x7fe06c106f10 2024-09-11T23:35:23.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.668+0000 7fe07139e640 1 -- 172.21.15.133:0/2194960814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06c19d670 con 0x7fe06c106f10 2024-09-11T23:35:23.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.668+0000 7fe068ff9640 1 -- 172.21.15.133:0/2194960814 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064018c10 con 0x7fe06c106f10 2024-09-11T23:35:23.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.668+0000 7fe07139e640 1 -- 172.21.15.133:0/2194960814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06c1a4f50 con 0x7fe06c106f10 2024-09-11T23:35:23.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.669+0000 7fe07139e640 1 -- 172.21.15.133:0/2194960814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe06c1021c0 con 0x7fe06c106f10 2024-09-11T23:35:23.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.670+0000 7fe068ff9640 1 -- 172.21.15.133:0/2194960814 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe06400f5f0 con 0x7fe06c106f10 2024-09-11T23:35:23.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.671+0000 7fe068ff9640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0580a2c10 0x7fe0580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:23.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.671+0000 7fe06affd640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0580a2c10 0x7fe0580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:23.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.671+0000 7fe068ff9640 1 -- 172.21.15.133:0/2194960814 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fe0640cf7c0 con 0x7fe06c106f10 2024-09-11T23:35:23.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.671+0000 7fe06affd640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0580a2c10 0x7fe0580a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe05c000c00 tx=0x7fe05c032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:23.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.675+0000 7fe068ff9640 1 -- 172.21.15.133:0/2194960814 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe064014030 con 0x7fe06c106f10 2024-09-11T23:35:23.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.887+0000 7fe07139e640 1 -- 172.21.15.133:0/2194960814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe06c109b60 con 0x7fe06c106f10 2024-09-11T23:35:23.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.891+0000 7fe068ff9640 1 -- 172.21.15.133:0/2194960814 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263209 (secure 0 0 0) 0x7fe06409c6c0 con 0x7fe06c106f10 2024-09-11T23:35:23.894 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:35:23.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 -- 172.21.15.133:0/2194960814 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0580a2c10 msgr2=0x7fe0580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:23.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0580a2c10 0x7fe0580a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe05c000c00 tx=0x7fe05c032d50 comp rx=0 tx=0).stop 2024-09-11T23:35:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 -- 172.21.15.133:0/2194960814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe06c106f10 msgr2=0x7fe06c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe06c106f10 0x7fe06c19cca0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe06400cce0 tx=0x7fe06400a620 comp rx=0 tx=0).stop 2024-09-11T23:35:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 -- 172.21.15.133:0/2194960814 shutdown_connections 2024-09-11T23:35:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe0580a2c10 0x7fe0580a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe06c106f10 0x7fe06c19cca0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c102790 0x7fe06c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 --2- 172.21.15.133:0/2194960814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe06c101de0 0x7fe06c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 -- 172.21.15.133:0/2194960814 >> 172.21.15.133:0/2194960814 conn(0x7fe06c0fd8c0 msgr2=0x7fe06c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 -- 172.21.15.133:0/2194960814 shutdown_connections 2024-09-11T23:35:23.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:23.894+0000 7fe07139e640 1 -- 172.21.15.133:0/2194960814 wait complete. 2024-09-11T23:35:24.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:23 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2194960814' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:35:24.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:23 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2194960814' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:35:24.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:23 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2194960814' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:35:24.584 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3324898762},{"type":"v1","addr":"172.21.15.133:6801","nonce":3324898762}]},"active_addr":"172.21.15.133:6801/3324898762","active_change":"2024-09-11T23:28:07.759637+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.133:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":3947064111}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":3596945316}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":802317131}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":1658494809}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":2553627143}]}]} 2024-09-11T23:35:24.590 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T23:35:24.590 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T23:35:24.590 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:35:24.833 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:25.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:24 smithi138 ceph-mon[35142]: pgmap v289: 97 pgs: 97 active+clean; 632 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-11T23:35:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:24 smithi133 ceph-mon[31430]: pgmap v289: 97 pgs: 97 active+clean; 632 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-11T23:35:25.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:24 smithi178 ceph-mon[35882]: pgmap v289: 97 pgs: 97 active+clean; 632 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-11T23:35:25.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.305+0000 7fa568d22640 1 -- 172.21.15.133:0/1991995608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa564100480 msgr2=0x7fa564100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:25.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.305+0000 7fa568d22640 1 --2- 172.21.15.133:0/1991995608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa564100480 0x7fa564100860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009ca0 tx=0x7fa55c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.305+0000 7fa568d22640 1 -- 172.21.15.133:0/1991995608 shutdown_connections 2024-09-11T23:35:25.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.305+0000 7fa568d22640 1 --2- 172.21.15.133:0/1991995608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5641055b0 0x7fa56410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.305+0000 7fa568d22640 1 --2- 172.21.15.133:0/1991995608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa564100e30 0x7fa564104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.305+0000 7fa568d22640 1 --2- 172.21.15.133:0/1991995608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa564100480 0x7fa564100860 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.305+0000 7fa568d22640 1 -- 172.21.15.133:0/1991995608 >> 172.21.15.133:0/1991995608 conn(0x7fa564077210 msgr2=0x7fa5640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.305+0000 7fa568d22640 1 -- 172.21.15.133:0/1991995608 shutdown_connections 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.305+0000 7fa568d22640 1 -- 172.21.15.133:0/1991995608 wait complete. 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa568d22640 1 Processor -- start 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa568d22640 1 -- start start 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa568d22640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa564100480 0x7fa5641972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa568d22640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa564100e30 0x7fa564197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa568d22640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5641055b0 0x7fa56419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa568d22640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa564110e90 con 0x7fa5641055b0 2024-09-11T23:35:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa568d22640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa564110d10 con 0x7fa564100e30 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa568d22640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa564111010 con 0x7fa564100480 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa562d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5641055b0 0x7fa56419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa562d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5641055b0 0x7fa56419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52400/0 (socket says 172.21.15.133:52400) 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa562d9d640 1 -- 172.21.15.133:0/4042149565 learned_addr learned my addr 172.21.15.133:0/4042149565 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa561d9b640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa564100e30 0x7fa564197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa56259c640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa564100480 0x7fa5641972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa562d9d640 1 -- 172.21.15.133:0/4042149565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa564100480 msgr2=0x7fa5641972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa562d9d640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa564100480 0x7fa5641972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa562d9d640 1 -- 172.21.15.133:0/4042149565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa564100e30 msgr2=0x7fa564197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.306+0000 7fa562d9d640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa564100e30 0x7fa564197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.307+0000 7fa562d9d640 1 -- 172.21.15.133:0/4042149565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56419c280 con 0x7fa5641055b0 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.307+0000 7fa562d9d640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5641055b0 0x7fa56419bba0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa554007900 tx=0x7fa554007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.307+0000 7fa55b7fe640 1 -- 172.21.15.133:0/4042149565 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55400e6f0 con 0x7fa5641055b0 2024-09-11T23:35:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.307+0000 7fa568d22640 1 -- 172.21.15.133:0/4042149565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56419c570 con 0x7fa5641055b0 2024-09-11T23:35:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.307+0000 7fa55b7fe640 1 -- 172.21.15.133:0/4042149565 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa55400ed70 con 0x7fa5641055b0 2024-09-11T23:35:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.307+0000 7fa55b7fe640 1 -- 172.21.15.133:0/4042149565 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55400ad10 con 0x7fa5641055b0 2024-09-11T23:35:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.307+0000 7fa568d22640 1 -- 172.21.15.133:0/4042149565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56410d9f0 con 0x7fa5641055b0 2024-09-11T23:35:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.308+0000 7fa568d22640 1 -- 172.21.15.133:0/4042149565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa564101960 con 0x7fa5641055b0 2024-09-11T23:35:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.309+0000 7fa55b7fe640 1 -- 172.21.15.133:0/4042149565 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa55400e890 con 0x7fa5641055b0 2024-09-11T23:35:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.310+0000 7fa55b7fe640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa5500a2c10 0x7fa5500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.310+0000 7fa56259c640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa5500a2c10 0x7fa5500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.310+0000 7fa55b7fe640 1 -- 172.21.15.133:0/4042149565 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fa5540d0b60 con 0x7fa5641055b0 2024-09-11T23:35:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.310+0000 7fa56259c640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa5500a2c10 0x7fa5500a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa55c000f80 tx=0x7fa55c0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.314+0000 7fa55b7fe640 1 -- 172.21.15.133:0/4042149565 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa55409da60 con 0x7fa5641055b0 2024-09-11T23:35:25.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.481+0000 7fa568d22640 1 -- 172.21.15.133:0/4042149565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa56410e410 con 0x7fa5641055b0 2024-09-11T23:35:25.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.482+0000 7fa55b7fe640 1 -- 172.21.15.133:0/4042149565 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22677 (secure 0 0 0) 0x7fa55409d360 con 0x7fa5641055b0 2024-09-11T23:35:25.482 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:35:25.482 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":81,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:35:08.840852+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:35:25.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 -- 172.21.15.133:0/4042149565 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa5500a2c10 msgr2=0x7fa5500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:25.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa5500a2c10 0x7fa5500a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa55c000f80 tx=0x7fa55c0093f0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 -- 172.21.15.133:0/4042149565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5641055b0 msgr2=0x7fa56419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:25.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5641055b0 0x7fa56419bba0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa554007900 tx=0x7fa554007dc0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 -- 172.21.15.133:0/4042149565 shutdown_connections 2024-09-11T23:35:25.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa5500a2c10 0x7fa5500a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5641055b0 0x7fa56419bba0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa564100e30 0x7fa564197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 --2- 172.21.15.133:0/4042149565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa564100480 0x7fa5641972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:25.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 -- 172.21.15.133:0/4042149565 >> 172.21.15.133:0/4042149565 conn(0x7fa564077210 msgr2=0x7fa564103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:25.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 -- 172.21.15.133:0/4042149565 shutdown_connections 2024-09-11T23:35:25.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:25.485+0000 7fa568d22640 1 -- 172.21.15.133:0/4042149565 wait complete. 2024-09-11T23:35:26.098 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:25 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/4042149565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:35:26.124 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T23:35:26.124 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:35:26.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:25 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/4042149565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:35:26.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:25 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/4042149565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:35:26.367 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.837+0000 7f32bf998640 1 -- 172.21.15.133:0/124869957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b8101de0 msgr2=0x7f32b81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.837+0000 7f32bf998640 1 --2- 172.21.15.133:0/124869957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b8101de0 0x7f32b81021c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009900 tx=0x7f32a802a410 comp rx=0 tx=0).stop 2024-09-11T23:35:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.837+0000 7f32bf998640 1 -- 172.21.15.133:0/124869957 shutdown_connections 2024-09-11T23:35:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.837+0000 7f32bf998640 1 --2- 172.21.15.133:0/124869957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b8106f10 0x7f32b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.837+0000 7f32bf998640 1 --2- 172.21.15.133:0/124869957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32b8102790 0x7f32b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.837+0000 7f32bf998640 1 --2- 172.21.15.133:0/124869957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b8101de0 0x7f32b81021c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.837+0000 7f32bf998640 1 -- 172.21.15.133:0/124869957 >> 172.21.15.133:0/124869957 conn(0x7f32b80fd8c0 msgr2=0x7f32b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.837+0000 7f32bf998640 1 -- 172.21.15.133:0/124869957 shutdown_connections 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.837+0000 7f32bf998640 1 -- 172.21.15.133:0/124869957 wait complete. 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.837+0000 7f32bf998640 1 Processor -- start 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.837+0000 7f32bf998640 1 -- start start 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bf998640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b8101de0 0x7f32b80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bf998640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b8102790 0x7f32b8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bf998640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32b8106f10 0x7f32b8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bf998640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8112010 con 0x7f32b8102790 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bf998640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8111e90 con 0x7f32b8106f10 2024-09-11T23:35:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bf998640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8112190 con 0x7f32b8101de0 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bcf33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b8102790 0x7f32b8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bcf33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b8102790 0x7f32b8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52420/0 (socket says 172.21.15.133:52420) 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bcf33640 1 -- 172.21.15.133:0/1101638582 learned_addr learned my addr 172.21.15.133:0/1101638582 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bd734640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b8101de0 0x7f32b80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bdf35640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32b8106f10 0x7f32b8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bcf33640 1 -- 172.21.15.133:0/1101638582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b8101de0 msgr2=0x7f32b80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bcf33640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b8101de0 0x7f32b80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bcf33640 1 -- 172.21.15.133:0/1101638582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32b8106f10 msgr2=0x7f32b8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bcf33640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32b8106f10 0x7f32b8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bcf33640 1 -- 172.21.15.133:0/1101638582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b81a6a70 con 0x7f32b8102790 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bdf35640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32b8106f10 0x7f32b8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bcf33640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b8102790 0x7f32b8074a30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32b400b4a0 tx=0x7f32b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32ae7fc640 1 -- 172.21.15.133:0/1101638582 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4014070 con 0x7f32b8102790 2024-09-11T23:35:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32bf998640 1 -- 172.21.15.133:0/1101638582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b81a6d60 con 0x7f32b8102790 2024-09-11T23:35:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32ae7fc640 1 -- 172.21.15.133:0/1101638582 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b4010af0 con 0x7f32b8102790 2024-09-11T23:35:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.838+0000 7f32ae7fc640 1 -- 172.21.15.133:0/1101638582 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4017d10 con 0x7f32b8102790 2024-09-11T23:35:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.839+0000 7f32bf998640 1 -- 172.21.15.133:0/1101638582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b81a7260 con 0x7f32b8102790 2024-09-11T23:35:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.840+0000 7f32bf998640 1 -- 172.21.15.133:0/1101638582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b8102240 con 0x7f32b8102790 2024-09-11T23:35:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.841+0000 7f32ae7fc640 1 -- 172.21.15.133:0/1101638582 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f32b4017070 con 0x7f32b8102790 2024-09-11T23:35:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.841+0000 7f32ae7fc640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f32a40a2c10 0x7f32a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.841+0000 7f32bd734640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f32a40a2c10 0x7f32a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.842+0000 7f32ae7fc640 1 -- 172.21.15.133:0/1101638582 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f32b40ce780 con 0x7f32b8102790 2024-09-11T23:35:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.842+0000 7f32bd734640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f32a40a2c10 0x7f32a40a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f32a80096c0 tx=0x7f32a8031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:26.844+0000 7f32ae7fc640 1 -- 172.21.15.133:0/1101638582 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32b409b790 con 0x7f32b8102790 2024-09-11T23:35:27.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.017+0000 7f32bf998640 1 -- 172.21.15.133:0/1101638582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32b8075a20 con 0x7f32b8102790 2024-09-11T23:35:27.017 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:26 smithi133 ceph-mon[31430]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-11T23:35:27.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.019+0000 7f32ae7fc640 1 -- 172.21.15.133:0/1101638582 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22677 (secure 0 0 0) 0x7f32b409b090 con 0x7f32b8102790 2024-09-11T23:35:27.019 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:35:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":81,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:35:08.840852+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 -- 172.21.15.133:0/1101638582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f32a40a2c10 msgr2=0x7f32a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f32a40a2c10 0x7f32a40a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f32a80096c0 tx=0x7f32a8031490 comp rx=0 tx=0).stop 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 -- 172.21.15.133:0/1101638582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b8102790 msgr2=0x7f32b8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b8102790 0x7f32b8074a30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32b400b4a0 tx=0x7f32b400b960 comp rx=0 tx=0).stop 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 -- 172.21.15.133:0/1101638582 shutdown_connections 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f32a40a2c10 0x7f32a40a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32b8106f10 0x7f32b8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b8102790 0x7f32b8074a30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 --2- 172.21.15.133:0/1101638582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b8101de0 0x7f32b80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 -- 172.21.15.133:0/1101638582 >> 172.21.15.133:0/1101638582 conn(0x7f32b80fd8c0 msgr2=0x7f32b80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 -- 172.21.15.133:0/1101638582 shutdown_connections 2024-09-11T23:35:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:27.021+0000 7f32bf998640 1 -- 172.21.15.133:0/1101638582 wait complete. 2024-09-11T23:35:27.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:26 smithi138 ceph-mon[35142]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-11T23:35:27.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:26 smithi178 ceph-mon[35882]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-11T23:35:27.725 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T23:35:27.725 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T23:35:27.726 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T23:35:27.726 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T23:35:27.726 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T23:35:27.727 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T23:35:27.727 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T23:35:27.727 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T23:35:27.728 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-11T23:35:27.729 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-11T23:35:27.729 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-11T23:35:27.729 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-11T23:35:27.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:27 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1101638582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:35:28.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:27 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1101638582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:35:28.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:27 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1101638582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:35:28.485 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:28.485 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:28.498 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:28.500 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:28.501 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:28.505 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:28.505 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:28.508 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:28.508 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:28.512 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:28.527 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:28.527 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:29.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:28 smithi133 ceph-mon[31430]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T23:35:29.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:28 smithi138 ceph-mon[35142]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T23:35:29.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:28 smithi178 ceph-mon[35882]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T23:35:29.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.478+0000 7fa78de69640 1 -- 172.21.15.133:0/3699814208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa788100eb0 msgr2=0x7fa7881067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:29.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.478+0000 7fa78de69640 1 --2- 172.21.15.133:0/3699814208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa788100eb0 0x7fa7881067a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009900 tx=0x7fa77c02a410 comp rx=0 tx=0).stop 2024-09-11T23:35:29.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.478+0000 7fa78de69640 1 -- 172.21.15.133:0/3699814208 shutdown_connections 2024-09-11T23:35:29.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.478+0000 7fa78de69640 1 --2- 172.21.15.133:0/3699814208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa788106ed0 0x7fa78810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.478+0000 7fa78de69640 1 --2- 172.21.15.133:0/3699814208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa788100eb0 0x7fa7881067a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.478+0000 7fa78de69640 1 --2- 172.21.15.133:0/3699814208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788100500 0x7fa7881008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.478+0000 7fa78de69640 1 -- 172.21.15.133:0/3699814208 >> 172.21.15.133:0/3699814208 conn(0x7fa7880fc5c0 msgr2=0x7fa7880fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.478+0000 7fa78de69640 1 -- 172.21.15.133:0/3699814208 shutdown_connections 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.478+0000 7fa78de69640 1 -- 172.21.15.133:0/3699814208 wait complete. 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.479+0000 7fa78de69640 1 Processor -- start 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.479+0000 7fa78de69640 1 -- start start 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.479+0000 7fa78de69640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788100500 0x7fa788198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.479+0000 7fa78de69640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa788100eb0 0x7fa788198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa78de69640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa788106ed0 0x7fa78819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa78de69640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa788111f80 con 0x7fa788100eb0 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa78de69640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa788111e00 con 0x7fa788106ed0 2024-09-11T23:35:29.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa78de69640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa788112100 con 0x7fa788100500 2024-09-11T23:35:29.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa786ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa788100eb0 0x7fa788198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:29.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa786ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa788100eb0 0x7fa788198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52426/0 (socket says 172.21.15.133:52426) 2024-09-11T23:35:29.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa786ffd640 1 -- 172.21.15.133:0/3072588939 learned_addr learned my addr 172.21.15.133:0/3072588939 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:29.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa786ffd640 1 -- 172.21.15.133:0/3072588939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788100500 msgr2=0x7fa788198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:29.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa7877fe640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788100500 0x7fa788198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:29.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa787fff640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa788106ed0 0x7fa78819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:29.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa786ffd640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788100500 0x7fa788198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:29.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa786ffd640 1 -- 172.21.15.133:0/3072588939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa788106ed0 msgr2=0x7fa78819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa786ffd640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa788106ed0 0x7fa78819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa786ffd640 1 -- 172.21.15.133:0/3072588939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78819d3f0 con 0x7fa788100eb0 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa7877fe640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788100500 0x7fa788198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa786ffd640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa788100eb0 0x7fa788198980 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa77c006010 tx=0x7fa77c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa784ff9640 1 -- 172.21.15.133:0/3072588939 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c03f030 con 0x7fa788100eb0 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa78de69640 1 -- 172.21.15.133:0/3072588939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78819d680 con 0x7fa788100eb0 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa784ff9640 1 -- 172.21.15.133:0/3072588939 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa77c003ce0 con 0x7fa788100eb0 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa784ff9640 1 -- 172.21.15.133:0/3072588939 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c035070 con 0x7fa788100eb0 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.480+0000 7fa78de69640 1 -- 172.21.15.133:0/3072588939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7881a4fc0 con 0x7fa788100eb0 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.481+0000 7fa78de69640 1 -- 172.21.15.133:0/3072588939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa788062980 con 0x7fa788100eb0 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.483+0000 7fa784ff9640 1 -- 172.21.15.133:0/3072588939 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa77c042030 con 0x7fa788100eb0 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.483+0000 7fa784ff9640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa7700a2c10 0x7fa7700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:29.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.483+0000 7fa7877fe640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa7700a2c10 0x7fa7700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.483+0000 7fa784ff9640 1 -- 172.21.15.133:0/3072588939 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fa77c0efde0 con 0x7fa788100eb0 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.484+0000 7fa784ff9640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854] conn(0x7fa7700ac6c0 0x7fa7700aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.484+0000 7fa784ff9640 1 -- 172.21.15.133:0/3072588939 --> [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa77c040e70 con 0x7fa7700ac6c0 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.484+0000 7fa784ff9640 1 -- 172.21.15.133:0/3072588939 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fa77c0f02c0 con 0x7fa788100eb0 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.484+0000 7fa7877fe640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa7700a2c10 0x7fa7700a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa778009b40 tx=0x7fa778005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.484+0000 7fa787fff640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854] conn(0x7fa7700ac6c0 0x7fa7700aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.484+0000 7fa787fff640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854] conn(0x7fa7700ac6c0 0x7fa7700aeb60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.485+0000 7fa784ff9640 1 -- 172.21.15.133:0/3072588939 <== osd.7 v2:172.21.15.138:6816/663031854 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa77c040e70 con 0x7fa7700ac6c0 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.499+0000 7fa78de69640 1 -- 172.21.15.133:0/3072588939 --> [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa788062d80 con 0x7fa7700ac6c0 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.499+0000 7fa784ff9640 1 -- 172.21.15.133:0/3072588939 <== osd.7 v2:172.21.15.138:6816/663031854 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa788062d80 con 0x7fa7700ac6c0 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.500+0000 7fa78de69640 1 -- 172.21.15.133:0/3072588939 >> [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854] conn(0x7fa7700ac6c0 msgr2=0x7fa7700aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.500+0000 7fa78de69640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854] conn(0x7fa7700ac6c0 0x7fa7700aeb60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.500+0000 7fa78de69640 1 -- 172.21.15.133:0/3072588939 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa7700a2c10 msgr2=0x7fa7700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:29.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.500+0000 7fa78de69640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa7700a2c10 0x7fa7700a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa778009b40 tx=0x7fa778005e50 comp rx=0 tx=0).stop 2024-09-11T23:35:29.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.500+0000 7fa78de69640 1 -- 172.21.15.133:0/3072588939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa788100eb0 msgr2=0x7fa788198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:29.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.500+0000 7fa78de69640 1 --2- 172.21.15.133:0/3072588939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa788100eb0 0x7fa788198980 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa77c006010 tx=0x7fa77c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:35:29.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.500+0000 7fa787fff640 1 -- 172.21.15.133:0/3072588939 reap_dead start 2024-09-11T23:35:29.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.500+0000 7fa78de69640 1 -- 172.21.15.133:0/3072588939 shutdown_connections 2024-09-11T23:35:29.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.500+0000 7fa78de69640 1 -- 172.21.15.133:0/3072588939 >> 172.21.15.133:0/3072588939 conn(0x7fa7880fc5c0 msgr2=0x7fa788105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:29.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.500+0000 7fa78de69640 1 -- 172.21.15.133:0/3072588939 shutdown_connections 2024-09-11T23:35:29.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:29.500+0000 7fa78de69640 1 -- 172.21.15.133:0/3072588939 wait complete. 2024-09-11T23:35:30.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.966+0000 7f6f97eb5640 1 -- 172.21.15.133:0/1630272556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f90101420 msgr2=0x7f6f9010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:30.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.966+0000 7f6f97eb5640 1 --2- 172.21.15.133:0/1630272556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f90101420 0x7f6f9010d950 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009930 tx=0x7f6f8002a420 comp rx=0 tx=0).stop 2024-09-11T23:35:30.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.966+0000 7f6f97eb5640 1 -- 172.21.15.133:0/1630272556 shutdown_connections 2024-09-11T23:35:30.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.966+0000 7f6f97eb5640 1 --2- 172.21.15.133:0/1630272556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f90101420 0x7f6f9010d950 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:30.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.966+0000 7f6f97eb5640 1 --2- 172.21.15.133:0/1630272556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f90100a80 0x7f6f90100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:30.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.966+0000 7f6f97eb5640 1 --2- 172.21.15.133:0/1630272556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f90106a80 0x7f6f90106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:30.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.966+0000 7f6f97eb5640 1 -- 172.21.15.133:0/1630272556 >> 172.21.15.133:0/1630272556 conn(0x7f6f900fc7a0 msgr2=0x7f6f900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:30.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.966+0000 7f6f97eb5640 1 -- 172.21.15.133:0/1630272556 shutdown_connections 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.966+0000 7f6f97eb5640 1 -- 172.21.15.133:0/1630272556 wait complete. 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.966+0000 7f6f97eb5640 1 Processor -- start 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f97eb5640 1 -- start start 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f97eb5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f90100a80 0x7f6f901972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f97eb5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f90101420 0x7f6f901977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f97eb5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f90106a80 0x7f6f9019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f97eb5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90110ed0 con 0x7f6f90106a80 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f97eb5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90110d50 con 0x7f6f90100a80 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f97eb5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90111050 con 0x7f6f90101420 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f96452640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f90106a80 0x7f6f9019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f96452640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f90106a80 0x7f6f9019bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52476/0 (socket says 172.21.15.133:52476) 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f96452640 1 -- 172.21.15.133:0/923964759 learned_addr learned my addr 172.21.15.133:0/923964759 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f95c51640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f90100a80 0x7f6f901972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:30.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f96452640 1 -- 172.21.15.133:0/923964759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f90101420 msgr2=0x7f6f901977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f96452640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f90101420 0x7f6f901977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f96452640 1 -- 172.21.15.133:0/923964759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f90100a80 msgr2=0x7f6f901972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f96452640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f90100a80 0x7f6f901972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f96452640 1 -- 172.21.15.133:0/923964759 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9019c260 con 0x7f6f90106a80 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f96452640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f90106a80 0x7f6f9019bb80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f80004770 tx=0x7f6f80003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f8effd640 1 -- 172.21.15.133:0/923964759 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8003f070 con 0x7f6f90106a80 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f97eb5640 1 -- 172.21.15.133:0/923964759 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9019c4f0 con 0x7f6f90106a80 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f8effd640 1 -- 172.21.15.133:0/923964759 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f80031c90 con 0x7f6f90106a80 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.967+0000 7f6f8effd640 1 -- 172.21.15.133:0/923964759 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80035030 con 0x7f6f90106a80 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.968+0000 7f6f97eb5640 1 -- 172.21.15.133:0/923964759 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f901a3e30 con 0x7f6f90106a80 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.968+0000 7f6f97eb5640 1 -- 172.21.15.133:0/923964759 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f900fd7d0 con 0x7f6f90106a80 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.970+0000 7f6f8effd640 1 -- 172.21.15.133:0/923964759 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6f80042070 con 0x7f6f90106a80 2024-09-11T23:35:30.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.970+0000 7f6f8effd640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6f780a2c10 0x7f6f780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:30.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.970+0000 7f6f95c51640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6f780a2c10 0x7f6f780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:30.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.970+0000 7f6f8effd640 1 -- 172.21.15.133:0/923964759 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f6f8003be00 con 0x7f6f90106a80 2024-09-11T23:35:30.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.971+0000 7f6f8effd640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573] conn(0x7f6f780ac6c0 0x7f6f780aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:30.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.971+0000 7f6f8effd640 1 -- 172.21.15.133:0/923964759 --> [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f80038e90 con 0x7f6f780ac6c0 2024-09-11T23:35:30.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.971+0000 7f6f8effd640 1 -- 172.21.15.133:0/923964759 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f6f800f00f0 con 0x7f6f90106a80 2024-09-11T23:35:30.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.971+0000 7f6f95450640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573] conn(0x7f6f780ac6c0 0x7f6f780aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:30.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.971+0000 7f6f95c51640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6f780a2c10 0x7f6f780a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f88005fd0 tx=0x7f6f8800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:30.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.971+0000 7f6f95450640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573] conn(0x7f6f780ac6c0 0x7f6f780aeb60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:30.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.972+0000 7f6f8effd640 1 -- 172.21.15.133:0/923964759 <== osd.6 v2:172.21.15.133:6818/3287486573 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f80038e90 con 0x7f6f780ac6c0 2024-09-11T23:35:30.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.985+0000 7f6f97eb5640 1 -- 172.21.15.133:0/923964759 --> [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6f901021c0 con 0x7f6f780ac6c0 2024-09-11T23:35:30.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.985+0000 7f6f8effd640 1 -- 172.21.15.133:0/923964759 <== osd.6 v2:172.21.15.133:6818/3287486573 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6f901021c0 con 0x7f6f780ac6c0 2024-09-11T23:35:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.986+0000 7f6f97eb5640 1 -- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573] conn(0x7f6f780ac6c0 msgr2=0x7f6f780aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.986+0000 7f6f97eb5640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573] conn(0x7f6f780ac6c0 0x7f6f780aeb60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.986+0000 7f6f97eb5640 1 -- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6f780a2c10 msgr2=0x7f6f780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.986+0000 7f6f97eb5640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6f780a2c10 0x7f6f780a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f88005fd0 tx=0x7f6f8800b390 comp rx=0 tx=0).stop 2024-09-11T23:35:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.986+0000 7f6f97eb5640 1 -- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f90106a80 msgr2=0x7f6f9019bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.986+0000 7f6f97eb5640 1 --2- 172.21.15.133:0/923964759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f90106a80 0x7f6f9019bb80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f80004770 tx=0x7f6f80003b90 comp rx=0 tx=0).stop 2024-09-11T23:35:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.986+0000 7f6f96452640 1 -- 172.21.15.133:0/923964759 reap_dead start 2024-09-11T23:35:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.986+0000 7f6f97eb5640 1 -- 172.21.15.133:0/923964759 shutdown_connections 2024-09-11T23:35:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.986+0000 7f6f97eb5640 1 -- 172.21.15.133:0/923964759 >> 172.21.15.133:0/923964759 conn(0x7f6f900fc7a0 msgr2=0x7f6f900fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.986+0000 7f6f97eb5640 1 -- 172.21.15.133:0/923964759 shutdown_connections 2024-09-11T23:35:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.986+0000 7f6f97eb5640 1 -- 172.21.15.133:0/923964759 wait complete. 2024-09-11T23:35:31.111 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:30 smithi133 ceph-mon[31430]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-11T23:35:31.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:30 smithi138 ceph-mon[35142]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-11T23:35:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.194+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/3092957964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac102ef0 msgr2=0x7f5eac10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.194+0000 7f5eb1b6e640 1 --2- 172.21.15.133:0/3092957964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac102ef0 0x7f5eac10da40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009900 tx=0x7f5e9c02a480 comp rx=0 tx=0).stop 2024-09-11T23:35:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.194+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/3092957964 shutdown_connections 2024-09-11T23:35:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.194+0000 7f5eb1b6e640 1 --2- 172.21.15.133:0/3092957964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac102ef0 0x7f5eac10da40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.194+0000 7f5eb1b6e640 1 --2- 172.21.15.133:0/3092957964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac102570 0x7f5eac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.194+0000 7f5eb1b6e640 1 --2- 172.21.15.133:0/3092957964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5eac100e20 0x7f5eac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.194+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/3092957964 >> 172.21.15.133:0/3092957964 conn(0x7f5eac0fc7d0 msgr2=0x7f5eac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.194+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/3092957964 shutdown_connections 2024-09-11T23:35:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.195+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/3092957964 wait complete. 2024-09-11T23:35:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.195+0000 7f5eb1b6e640 1 Processor -- start 2024-09-11T23:35:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.195+0000 7f5eb1b6e640 1 -- start start 2024-09-11T23:35:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eb1b6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac100e20 0x7f5eac1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eb1b6e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5eac102570 0x7f5eac197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eb1b6e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac102ef0 0x7f5eac19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eb1b6e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac110fa0 con 0x7f5eac100e20 2024-09-11T23:35:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eb1b6e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac110e20 con 0x7f5eac102ef0 2024-09-11T23:35:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eb1b6e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac111120 con 0x7f5eac102570 2024-09-11T23:35:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac100e20 0x7f5eac1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac100e20 0x7f5eac1973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52488/0 (socket says 172.21.15.133:52488) 2024-09-11T23:35:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eab7fe640 1 -- 172.21.15.133:0/1839642115 learned_addr learned my addr 172.21.15.133:0/1839642115 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eabfff640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac102ef0 0x7f5eac19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eab7fe640 1 -- 172.21.15.133:0/1839642115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5eac102570 msgr2=0x7f5eac197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eaaffd640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5eac102570 0x7f5eac197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eab7fe640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5eac102570 0x7f5eac197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eab7fe640 1 -- 172.21.15.133:0/1839642115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac102ef0 msgr2=0x7f5eac19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eab7fe640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac102ef0 0x7f5eac19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eab7fe640 1 -- 172.21.15.133:0/1839642115 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac19c390 con 0x7f5eac100e20 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eabfff640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac102ef0 0x7f5eac19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eab7fe640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac100e20 0x7f5eac1973e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400ba50 tx=0x7f5ea400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1839642115 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4019070 con 0x7f5eac100e20 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1839642115 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea400cbe0 con 0x7f5eac100e20 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1839642115 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea40076e0 con 0x7f5eac100e20 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/1839642115 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac19c680 con 0x7f5eac100e20 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.196+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/1839642115 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac1a3f60 con 0x7f5eac100e20 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.197+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/1839642115 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5eac107260 con 0x7f5eac100e20 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.198+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1839642115 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5ea401c070 con 0x7f5eac100e20 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.199+0000 7f5ea8ff9640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f5e980a2c10 0x7f5e980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.199+0000 7f5eaaffd640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f5e980a2c10 0x7f5e980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.199+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1839642115 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f5ea40ce770 con 0x7f5eac100e20 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.200+0000 7f5ea8ff9640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874] conn(0x7f5e980ac6c0 0x7f5e980aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.200+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1839642115 --> [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e980af230 con 0x7f5e980ac6c0 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.200+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1839642115 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5ea40d5050 con 0x7f5eac100e20 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.200+0000 7f5eaaffd640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f5e980a2c10 0x7f5e980a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0fec10 tx=0x7f5e94005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.200+0000 7f5eabfff640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874] conn(0x7f5e980ac6c0 0x7f5e980aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.200+0000 7f5eabfff640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874] conn(0x7f5e980ac6c0 0x7f5e980aeb60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.201+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1839642115 <== osd.3 v2:172.21.15.133:6810/1056435874 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e980af230 con 0x7f5e980ac6c0 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.215+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/1839642115 --> [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5eac1021b0 con 0x7f5e980ac6c0 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.215+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1839642115 <== osd.3 v2:172.21.15.133:6810/1056435874 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5eac1021b0 con 0x7f5e980ac6c0 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.216+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874] conn(0x7f5e980ac6c0 msgr2=0x7f5e980aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.216+0000 7f5eb1b6e640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874] conn(0x7f5e980ac6c0 0x7f5e980aeb60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.216+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f5e980a2c10 msgr2=0x7f5e980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.216+0000 7f5eb1b6e640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f5e980a2c10 0x7f5e980a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0fec10 tx=0x7f5e94005c60 comp rx=0 tx=0).stop 2024-09-11T23:35:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.216+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac100e20 msgr2=0x7f5eac1973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:31.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.216+0000 7f5eb1b6e640 1 --2- 172.21.15.133:0/1839642115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac100e20 0x7f5eac1973e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400ba50 tx=0x7f5ea400bf10 comp rx=0 tx=0).stop 2024-09-11T23:35:31.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.216+0000 7f5eabfff640 1 -- 172.21.15.133:0/1839642115 reap_dead start 2024-09-11T23:35:31.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.216+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/1839642115 shutdown_connections 2024-09-11T23:35:31.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.216+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/1839642115 >> 172.21.15.133:0/1839642115 conn(0x7f5eac0fc7d0 msgr2=0x7f5eac106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:31.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.216+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/1839642115 shutdown_connections 2024-09-11T23:35:31.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:31.216+0000 7f5eb1b6e640 1 -- 172.21.15.133:0/1839642115 wait complete. 2024-09-11T23:35:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:30 smithi178 ceph-mon[35882]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-11T23:35:32.010 INFO:teuthology.orchestra.run.smithi133.stdout:98784247867 2024-09-11T23:35:32.011 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T23:35:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.748+0000 7ffbcf285640 1 -- 172.21.15.133:0/3932329069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc8102570 msgr2=0x7ffbc81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.748+0000 7ffbcf285640 1 --2- 172.21.15.133:0/3932329069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc8102570 0x7ffbc81029b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009900 tx=0x7ffbbc02a480 comp rx=0 tx=0).stop 2024-09-11T23:35:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.748+0000 7ffbcf285640 1 -- 172.21.15.133:0/3932329069 shutdown_connections 2024-09-11T23:35:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.748+0000 7ffbcf285640 1 --2- 172.21.15.133:0/3932329069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc8102ef0 0x7ffbc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.748+0000 7ffbcf285640 1 --2- 172.21.15.133:0/3932329069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc8102570 0x7ffbc81029b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.748+0000 7ffbcf285640 1 --2- 172.21.15.133:0/3932329069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc8100e20 0x7ffbc8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.748+0000 7ffbcf285640 1 -- 172.21.15.133:0/3932329069 >> 172.21.15.133:0/3932329069 conn(0x7ffbc80fc7d0 msgr2=0x7ffbc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.748+0000 7ffbcf285640 1 -- 172.21.15.133:0/3932329069 shutdown_connections 2024-09-11T23:35:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.748+0000 7ffbcf285640 1 -- 172.21.15.133:0/3932329069 wait complete. 2024-09-11T23:35:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.748+0000 7ffbcf285640 1 Processor -- start 2024-09-11T23:35:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.748+0000 7ffbcf285640 1 -- start start 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcf285640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc8100e20 0x7ffbc8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcf285640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc8102570 0x7ffbc8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcf285640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc8102ef0 0x7ffbc819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcf285640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8111020 con 0x7ffbc8102ef0 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcf285640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8110ea0 con 0x7ffbc8102570 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcf285640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc81111a0 con 0x7ffbc8100e20 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcd822640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc8102ef0 0x7ffbc819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcd822640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc8102ef0 0x7ffbc819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52434/0 (socket says 172.21.15.133:52434) 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcd822640 1 -- 172.21.15.133:0/1867028287 learned_addr learned my addr 172.21.15.133:0/1867028287 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcd021640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc8100e20 0x7ffbc8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcc820640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc8102570 0x7ffbc8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:32.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcd822640 1 -- 172.21.15.133:0/1867028287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc8100e20 msgr2=0x7ffbc8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcd822640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc8100e20 0x7ffbc8198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcd822640 1 -- 172.21.15.133:0/1867028287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc8102570 msgr2=0x7ffbc8198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcd822640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc8102570 0x7ffbc8198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcd822640 1 -- 172.21.15.133:0/1867028287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc819d6c0 con 0x7ffbc8102ef0 2024-09-11T23:35:32.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcc820640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc8102570 0x7ffbc8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:32.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcd822640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc8102ef0 0x7ffbc819cfe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009fd0 tx=0x7ffbc000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:32.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbc67fc640 1 -- 172.21.15.133:0/1867028287 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0011830 con 0x7ffbc8102ef0 2024-09-11T23:35:32.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbc67fc640 1 -- 172.21.15.133:0/1867028287 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc0016070 con 0x7ffbc8102ef0 2024-09-11T23:35:32.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcf285640 1 -- 172.21.15.133:0/1867028287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc819d9b0 con 0x7ffbc8102ef0 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbc67fc640 1 -- 172.21.15.133:0/1867028287 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc000f2e0 con 0x7ffbc8102ef0 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.749+0000 7ffbcf285640 1 -- 172.21.15.133:0/1867028287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc81a5290 con 0x7ffbc8102ef0 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.750+0000 7ffbcf285640 1 -- 172.21.15.133:0/1867028287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffbc8062980 con 0x7ffbc8102ef0 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.751+0000 7ffbc67fc640 1 -- 172.21.15.133:0/1867028287 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ffbc00119d0 con 0x7ffbc8102ef0 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.752+0000 7ffbc67fc640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ffbb00a2c10 0x7ffbb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.752+0000 7ffbcd021640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ffbb00a2c10 0x7ffbb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.752+0000 7ffbc67fc640 1 -- 172.21.15.133:0/1867028287 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7ffbc00ce720 con 0x7ffbc8102ef0 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.752+0000 7ffbc67fc640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816] conn(0x7ffbb00ac6c0 0x7ffbb00aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.752+0000 7ffbc67fc640 1 -- 172.21.15.133:0/1867028287 --> [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffbb00af230 con 0x7ffbb00ac6c0 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.752+0000 7ffbc67fc640 1 -- 172.21.15.133:0/1867028287 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ffbc00d5050 con 0x7ffbc8102ef0 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.753+0000 7ffbcd021640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ffbb00a2c10 0x7ffbb00a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8007dc0 tx=0x7ffbb80072f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.753+0000 7ffbcc820640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816] conn(0x7ffbb00ac6c0 0x7ffbb00aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.753+0000 7ffbcc820640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816] conn(0x7ffbb00ac6c0 0x7ffbb00aeb60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.754+0000 7ffbc67fc640 1 -- 172.21.15.133:0/1867028287 <== osd.2 v2:172.21.15.178:6800/3161638816 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffbb00af230 con 0x7ffbb00ac6c0 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.767+0000 7ffbcf285640 1 -- 172.21.15.133:0/1867028287 --> [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffbc8062e00 con 0x7ffbb00ac6c0 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbc67fc640 1 -- 172.21.15.133:0/1867028287 <== osd.2 v2:172.21.15.178:6800/3161638816 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffbc8062e00 con 0x7ffbb00ac6c0 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbcf285640 1 -- 172.21.15.133:0/1867028287 >> [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816] conn(0x7ffbb00ac6c0 msgr2=0x7ffbb00aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbcf285640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816] conn(0x7ffbb00ac6c0 0x7ffbb00aeb60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbcf285640 1 -- 172.21.15.133:0/1867028287 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ffbb00a2c10 msgr2=0x7ffbb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbcf285640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ffbb00a2c10 0x7ffbb00a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8007dc0 tx=0x7ffbb80072f0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbcf285640 1 -- 172.21.15.133:0/1867028287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc8102ef0 msgr2=0x7ffbc819cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbcf285640 1 --2- 172.21.15.133:0/1867028287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc8102ef0 0x7ffbc819cfe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009fd0 tx=0x7ffbc000b8e0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbcd822640 1 -- 172.21.15.133:0/1867028287 reap_dead start 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbcf285640 1 -- 172.21.15.133:0/1867028287 shutdown_connections 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbcf285640 1 -- 172.21.15.133:0/1867028287 >> 172.21.15.133:0/1867028287 conn(0x7ffbc80fc7d0 msgr2=0x7ffbc806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbcf285640 1 -- 172.21.15.133:0/1867028287 shutdown_connections 2024-09-11T23:35:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.768+0000 7ffbcf285640 1 -- 172.21.15.133:0/1867028287 wait complete. 2024-09-11T23:35:32.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.859+0000 7f30dc069640 1 -- 172.21.15.133:0/3319099403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4101e10 msgr2=0x7f30d41021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.859+0000 7f30dc069640 1 --2- 172.21.15.133:0/3319099403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4101e10 0x7f30d41021f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009900 tx=0x7f30d002a410 comp rx=0 tx=0).stop 2024-09-11T23:35:32.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.859+0000 7f30dc069640 1 -- 172.21.15.133:0/3319099403 shutdown_connections 2024-09-11T23:35:32.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.859+0000 7f30dc069640 1 --2- 172.21.15.133:0/3319099403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30d4106f40 0x7f30d410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.859+0000 7f30dc069640 1 --2- 172.21.15.133:0/3319099403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d41027c0 0x7f30d4106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.859+0000 7f30dc069640 1 --2- 172.21.15.133:0/3319099403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4101e10 0x7f30d41021f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.859+0000 7f30dc069640 1 -- 172.21.15.133:0/3319099403 >> 172.21.15.133:0/3319099403 conn(0x7f30d40fd8d0 msgr2=0x7f30d40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:32.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.859+0000 7f30dc069640 1 -- 172.21.15.133:0/3319099403 shutdown_connections 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.859+0000 7f30dc069640 1 -- 172.21.15.133:0/3319099403 wait complete. 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.859+0000 7f30dc069640 1 Processor -- start 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.859+0000 7f30dc069640 1 -- start start 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30dc069640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4101e10 0x7f30d41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30dc069640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30d41027c0 0x7f30d4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30dc069640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d4106f40 0x7f30d419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30dc069640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4111fa0 con 0x7f30d4101e10 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30dc069640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4111e20 con 0x7f30d4106f40 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30dc069640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4112120 con 0x7f30d41027c0 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30d9e05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4101e10 0x7f30d41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30d9e05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4101e10 0x7f30d41983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52460/0 (socket says 172.21.15.133:52460) 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30d9e05640 1 -- 172.21.15.133:0/3198674512 learned_addr learned my addr 172.21.15.133:0/3198674512 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:32.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30da606640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d4106f40 0x7f30d419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:32.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30d9604640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30d41027c0 0x7f30d4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:32.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30d9e05640 1 -- 172.21.15.133:0/3198674512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30d41027c0 msgr2=0x7f30d4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30d9e05640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30d41027c0 0x7f30d4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30d9e05640 1 -- 172.21.15.133:0/3198674512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d4106f40 msgr2=0x7f30d419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30d9e05640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d4106f40 0x7f30d419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30d9e05640 1 -- 172.21.15.133:0/3198674512 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d419d3a0 con 0x7f30d4101e10 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30d9e05640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4101e10 0x7f30d41983f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f30d0006010 tx=0x7f30d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30caffd640 1 -- 172.21.15.133:0/3198674512 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d003f030 con 0x7f30d4101e10 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30caffd640 1 -- 172.21.15.133:0/3198674512 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d0003ce0 con 0x7f30d4101e10 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30dc069640 1 -- 172.21.15.133:0/3198674512 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d419d630 con 0x7f30d4101e10 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.860+0000 7f30caffd640 1 -- 172.21.15.133:0/3198674512 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0035070 con 0x7f30d4101e10 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.861+0000 7f30dc069640 1 -- 172.21.15.133:0/3198674512 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41a4f70 con 0x7f30d4101e10 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.861+0000 7f30dc069640 1 -- 172.21.15.133:0/3198674512 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30d4062980 con 0x7f30d4101e10 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.863+0000 7f30caffd640 1 -- 172.21.15.133:0/3198674512 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f30d0042030 con 0x7f30d4101e10 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.863+0000 7f30caffd640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f30c00a2c10 0x7f30c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.863+0000 7f30d9604640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f30c00a2c10 0x7f30c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:32.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.863+0000 7f30caffd640 1 -- 172.21.15.133:0/3198674512 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f30d00efdc0 con 0x7f30d4101e10 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.863+0000 7f30caffd640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877] conn(0x7f30c00ac6c0 0x7f30c00aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.863+0000 7f30caffd640 1 -- 172.21.15.133:0/3198674512 --> [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30d003be50 con 0x7f30c00ac6c0 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.863+0000 7f30caffd640 1 -- 172.21.15.133:0/3198674512 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f30d00f02a0 con 0x7f30d4101e10 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.864+0000 7f30da606640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877] conn(0x7f30c00ac6c0 0x7f30c00aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.864+0000 7f30d9604640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f30c00a2c10 0x7f30c00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f30d40ffe30 tx=0x7f30c400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.864+0000 7f30da606640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877] conn(0x7f30c00ac6c0 0x7f30c00aeb60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.864+0000 7f30caffd640 1 -- 172.21.15.133:0/3198674512 <== osd.0 v2:172.21.15.133:6802/901342877 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30d003be50 con 0x7f30c00ac6c0 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.878+0000 7f30dc069640 1 -- 172.21.15.133:0/3198674512 --> [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30d4062e00 con 0x7f30c00ac6c0 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.878+0000 7f30caffd640 1 -- 172.21.15.133:0/3198674512 <== osd.0 v2:172.21.15.133:6802/901342877 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f30d4062e00 con 0x7f30c00ac6c0 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.879+0000 7f30dc069640 1 -- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877] conn(0x7f30c00ac6c0 msgr2=0x7f30c00aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.879+0000 7f30dc069640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877] conn(0x7f30c00ac6c0 0x7f30c00aeb60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.879+0000 7f30dc069640 1 -- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f30c00a2c10 msgr2=0x7f30c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.879+0000 7f30dc069640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f30c00a2c10 0x7f30c00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f30d40ffe30 tx=0x7f30c400b390 comp rx=0 tx=0).stop 2024-09-11T23:35:32.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.879+0000 7f30dc069640 1 -- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4101e10 msgr2=0x7f30d41983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:32.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.879+0000 7f30dc069640 1 --2- 172.21.15.133:0/3198674512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4101e10 0x7f30d41983f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f30d0006010 tx=0x7f30d00042f0 comp rx=0 tx=0).stop 2024-09-11T23:35:32.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.879+0000 7f30da606640 1 -- 172.21.15.133:0/3198674512 reap_dead start 2024-09-11T23:35:32.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.879+0000 7f30dc069640 1 -- 172.21.15.133:0/3198674512 shutdown_connections 2024-09-11T23:35:32.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.879+0000 7f30dc069640 1 -- 172.21.15.133:0/3198674512 >> 172.21.15.133:0/3198674512 conn(0x7f30d40fd8d0 msgr2=0x7f30d40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:32.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.879+0000 7f30dc069640 1 -- 172.21.15.133:0/3198674512 shutdown_connections 2024-09-11T23:35:32.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:30.879+0000 7f30dc069640 1 -- 172.21.15.133:0/3198674512 wait complete. 2024-09-11T23:35:32.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:32 smithi133 ceph-mon[31430]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:35:33.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:32 smithi138 ceph-mon[35142]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:35:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:32 smithi178 ceph-mon[35882]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:35:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.356+0000 7f04dbe0d640 1 -- 172.21.15.133:0/659212301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04d4102570 msgr2=0x7f04d41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.356+0000 7f04dbe0d640 1 --2- 172.21.15.133:0/659212301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04d4102570 0x7f04d41029b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009900 tx=0x7f04c802a480 comp rx=0 tx=0).stop 2024-09-11T23:35:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.356+0000 7f04dbe0d640 1 -- 172.21.15.133:0/659212301 shutdown_connections 2024-09-11T23:35:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.356+0000 7f04dbe0d640 1 --2- 172.21.15.133:0/659212301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d4102ef0 0x7f04d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.356+0000 7f04dbe0d640 1 --2- 172.21.15.133:0/659212301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04d4102570 0x7f04d41029b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.356+0000 7f04dbe0d640 1 --2- 172.21.15.133:0/659212301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04d4100e20 0x7f04d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.356+0000 7f04dbe0d640 1 -- 172.21.15.133:0/659212301 >> 172.21.15.133:0/659212301 conn(0x7f04d40fc7d0 msgr2=0x7f04d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.356+0000 7f04dbe0d640 1 -- 172.21.15.133:0/659212301 shutdown_connections 2024-09-11T23:35:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.356+0000 7f04dbe0d640 1 -- 172.21.15.133:0/659212301 wait complete. 2024-09-11T23:35:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.356+0000 7f04dbe0d640 1 Processor -- start 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04dbe0d640 1 -- start start 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04dbe0d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d4100e20 0x7f04d40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04dbe0d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04d4102570 0x7f04d40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04dbe0d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04d4102ef0 0x7f04d40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04dbe0d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4111020 con 0x7f04d4102ef0 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04dbe0d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4110ea0 con 0x7f04d4100e20 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04dbe0d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41111a0 con 0x7f04d4102570 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04da3aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04d4102ef0 0x7f04d40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04da3aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04d4102ef0 0x7f04d40fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49638/0 (socket says 172.21.15.133:49638) 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04da3aa640 1 -- 172.21.15.133:0/578027522 learned_addr learned my addr 172.21.15.133:0/578027522 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04d93a8640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04d4102570 0x7f04d40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04d9ba9640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d4100e20 0x7f04d40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04da3aa640 1 -- 172.21.15.133:0/578027522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04d4102570 msgr2=0x7f04d40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04da3aa640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04d4102570 0x7f04d40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04da3aa640 1 -- 172.21.15.133:0/578027522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d4100e20 msgr2=0x7f04d40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04da3aa640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d4100e20 0x7f04d40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04da3aa640 1 -- 172.21.15.133:0/578027522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d41a6bb0 con 0x7f04d4102ef0 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04d93a8640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04d4102570 0x7f04d40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.357+0000 7f04da3aa640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04d4102ef0 0x7f04d40fd700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00b970 tx=0x7f04cc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.358+0000 7f04c6ffd640 1 -- 172.21.15.133:0/578027522 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc019070 con 0x7f04d4102ef0 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.358+0000 7f04dbe0d640 1 -- 172.21.15.133:0/578027522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d41a6ea0 con 0x7f04d4102ef0 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.358+0000 7f04c6ffd640 1 -- 172.21.15.133:0/578027522 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc00cb40 con 0x7f04d4102ef0 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.358+0000 7f04c6ffd640 1 -- 172.21.15.133:0/578027522 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc007650 con 0x7f04d4102ef0 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.358+0000 7f04dbe0d640 1 -- 172.21.15.133:0/578027522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d41a7360 con 0x7f04d4102ef0 2024-09-11T23:35:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.359+0000 7f04dbe0d640 1 -- 172.21.15.133:0/578027522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04d4062880 con 0x7f04d4102ef0 2024-09-11T23:35:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.360+0000 7f04c6ffd640 1 -- 172.21.15.133:0/578027522 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f04cc01c070 con 0x7f04d4102ef0 2024-09-11T23:35:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.360+0000 7f04c6ffd640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f04c00a2c10 0x7f04c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.361+0000 7f04d9ba9640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f04c00a2c10 0x7f04c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.361+0000 7f04c6ffd640 1 -- 172.21.15.133:0/578027522 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f04cc0cee20 con 0x7f04d4102ef0 2024-09-11T23:35:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.361+0000 7f04c6ffd640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034] conn(0x7f04c00ac6c0 0x7f04c00aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.361+0000 7f04c6ffd640 1 -- 172.21.15.133:0/578027522 --> [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04cc0047d0 con 0x7f04c00ac6c0 2024-09-11T23:35:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.361+0000 7f04c6ffd640 1 -- 172.21.15.133:0/578027522 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f04cc0cf300 con 0x7f04d4102ef0 2024-09-11T23:35:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.361+0000 7f04d9ba9640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f04c00a2c10 0x7f04c00a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009890 tx=0x7f04d0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.361+0000 7f04d93a8640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034] conn(0x7f04c00ac6c0 0x7f04c00aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.361+0000 7f04d93a8640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034] conn(0x7f04c00ac6c0 0x7f04c00aeb60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.362+0000 7f04c6ffd640 1 -- 172.21.15.133:0/578027522 <== osd.10 v2:172.21.15.138:6824/678369034 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04cc0047d0 con 0x7f04c00ac6c0 2024-09-11T23:35:33.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.376+0000 7f04dbe0d640 1 -- 172.21.15.133:0/578027522 --> [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04d4062d00 con 0x7f04c00ac6c0 2024-09-11T23:35:33.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.376+0000 7f04c6ffd640 1 -- 172.21.15.133:0/578027522 <== osd.10 v2:172.21.15.138:6824/678369034 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f04d4062d00 con 0x7f04c00ac6c0 2024-09-11T23:35:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.377+0000 7f04dbe0d640 1 -- 172.21.15.133:0/578027522 >> [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034] conn(0x7f04c00ac6c0 msgr2=0x7f04c00aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.377+0000 7f04dbe0d640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034] conn(0x7f04c00ac6c0 0x7f04c00aeb60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.377+0000 7f04dbe0d640 1 -- 172.21.15.133:0/578027522 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f04c00a2c10 msgr2=0x7f04c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.377+0000 7f04dbe0d640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f04c00a2c10 0x7f04c00a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009890 tx=0x7f04d0008040 comp rx=0 tx=0).stop 2024-09-11T23:35:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.377+0000 7f04dbe0d640 1 -- 172.21.15.133:0/578027522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04d4102ef0 msgr2=0x7f04d40fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.377+0000 7f04dbe0d640 1 --2- 172.21.15.133:0/578027522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04d4102ef0 0x7f04d40fd700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00b970 tx=0x7f04cc00be30 comp rx=0 tx=0).stop 2024-09-11T23:35:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.377+0000 7f04da3aa640 1 -- 172.21.15.133:0/578027522 reap_dead start 2024-09-11T23:35:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.377+0000 7f04dbe0d640 1 -- 172.21.15.133:0/578027522 shutdown_connections 2024-09-11T23:35:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.377+0000 7f04dbe0d640 1 -- 172.21.15.133:0/578027522 >> 172.21.15.133:0/578027522 conn(0x7f04d40fc7d0 msgr2=0x7f04d406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.377+0000 7f04dbe0d640 1 -- 172.21.15.133:0/578027522 shutdown_connections 2024-09-11T23:35:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:33.377+0000 7f04dbe0d640 1 -- 172.21.15.133:0/578027522 wait complete. 2024-09-11T23:35:34.670 INFO:teuthology.orchestra.run.smithi133.stdout:34359738443 2024-09-11T23:35:34.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T23:35:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.837+0000 7fd50bd44640 1 -- 172.21.15.133:0/869810949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5041082a0 msgr2=0x7fd50410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.837+0000 7fd50bd44640 1 --2- 172.21.15.133:0/869810949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5041082a0 0x7fd50410ec40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009cd0 tx=0x7fd4fc02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.837+0000 7fd50bd44640 1 -- 172.21.15.133:0/869810949 shutdown_connections 2024-09-11T23:35:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.837+0000 7fd50bd44640 1 --2- 172.21.15.133:0/869810949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5041082a0 0x7fd50410ec40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.837+0000 7fd50bd44640 1 --2- 172.21.15.133:0/869810949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5041036e0 0x7fd504107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.837+0000 7fd50bd44640 1 --2- 172.21.15.133:0/869810949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd504102d30 0x7fd504103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.837+0000 7fd50bd44640 1 -- 172.21.15.133:0/869810949 >> 172.21.15.133:0/869810949 conn(0x7fd5040fc7d0 msgr2=0x7fd5040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.838+0000 7fd50bd44640 1 -- 172.21.15.133:0/869810949 shutdown_connections 2024-09-11T23:35:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.838+0000 7fd50bd44640 1 -- 172.21.15.133:0/869810949 wait complete. 2024-09-11T23:35:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd50bd44640 1 Processor -- start 2024-09-11T23:35:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd50bd44640 1 -- start start 2024-09-11T23:35:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd50bd44640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd504102d30 0x7fd50419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd50bd44640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5041036e0 0x7fd50419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd50bd44640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5041082a0 0x7fd50419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd50bd44640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd5041120a0 con 0x7fd504102d30 2024-09-11T23:35:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd50bd44640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd504111f20 con 0x7fd5041082a0 2024-09-11T23:35:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd50bd44640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd504112220 con 0x7fd5041036e0 2024-09-11T23:35:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd509ae0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd504102d30 0x7fd50419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd509ae0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd504102d30 0x7fd50419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49656/0 (socket says 172.21.15.133:49656) 2024-09-11T23:35:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd509ae0640 1 -- 172.21.15.133:0/120804701 learned_addr learned my addr 172.21.15.133:0/120804701 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd5092df640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5041036e0 0x7fd50419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd50a2e1640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5041082a0 0x7fd50419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd509ae0640 1 -- 172.21.15.133:0/120804701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5041036e0 msgr2=0x7fd50419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd509ae0640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5041036e0 0x7fd50419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd509ae0640 1 -- 172.21.15.133:0/120804701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5041082a0 msgr2=0x7fd50419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd509ae0640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5041082a0 0x7fd50419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd509ae0640 1 -- 172.21.15.133:0/120804701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50419f6e0 con 0x7fd504102d30 2024-09-11T23:35:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.839+0000 7fd50a2e1640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5041082a0 0x7fd50419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.840+0000 7fd509ae0640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd504102d30 0x7fd50419a730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800c970 tx=0x7fd4f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.840+0000 7fd4f6ffd640 1 -- 172.21.15.133:0/120804701 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8007bb0 con 0x7fd504102d30 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.840+0000 7fd50bd44640 1 -- 172.21.15.133:0/120804701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50419f9d0 con 0x7fd504102d30 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.840+0000 7fd4f6ffd640 1 -- 172.21.15.133:0/120804701 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4f8007d50 con 0x7fd504102d30 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.840+0000 7fd4f6ffd640 1 -- 172.21.15.133:0/120804701 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800f450 con 0x7fd504102d30 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.840+0000 7fd50bd44640 1 -- 172.21.15.133:0/120804701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5041a72b0 con 0x7fd504102d30 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.841+0000 7fd50bd44640 1 -- 172.21.15.133:0/120804701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd504062980 con 0x7fd504102d30 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.842+0000 7fd4f6ffd640 1 -- 172.21.15.133:0/120804701 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd4f8004fa0 con 0x7fd504102d30 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.843+0000 7fd4f6ffd640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd4f00a2c10 0x7fd4f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.843+0000 7fd5092df640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd4f00a2c10 0x7fd4f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.843+0000 7fd4f6ffd640 1 -- 172.21.15.133:0/120804701 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fd4f80cf270 con 0x7fd504102d30 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.843+0000 7fd4f6ffd640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962] conn(0x7fd4f00ac6c0 0x7fd4f00aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.843+0000 7fd4f6ffd640 1 -- 172.21.15.133:0/120804701 --> [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd4f00af230 con 0x7fd4f00ac6c0 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.843+0000 7fd4f6ffd640 1 -- 172.21.15.133:0/120804701 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fd4f80cf750 con 0x7fd504102d30 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.843+0000 7fd5092df640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd4f00a2c10 0x7fd4f00a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd5040fdf50 tx=0x7fd50000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.843+0000 7fd50a2e1640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962] conn(0x7fd4f00ac6c0 0x7fd4f00aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.844+0000 7fd50a2e1640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962] conn(0x7fd4f00ac6c0 0x7fd4f00aeb60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.845+0000 7fd4f6ffd640 1 -- 172.21.15.133:0/120804701 <== osd.11 v2:172.21.15.178:6824/3213367962 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd4f00af230 con 0x7fd4f00ac6c0 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.858+0000 7fd50bd44640 1 -- 172.21.15.133:0/120804701 --> [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd504062e00 con 0x7fd4f00ac6c0 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.859+0000 7fd4f6ffd640 1 -- 172.21.15.133:0/120804701 <== osd.11 v2:172.21.15.178:6824/3213367962 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd504062e00 con 0x7fd4f00ac6c0 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.860+0000 7fd50bd44640 1 -- 172.21.15.133:0/120804701 >> [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962] conn(0x7fd4f00ac6c0 msgr2=0x7fd4f00aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.860+0000 7fd50bd44640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962] conn(0x7fd4f00ac6c0 0x7fd4f00aeb60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.860+0000 7fd50bd44640 1 -- 172.21.15.133:0/120804701 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd4f00a2c10 msgr2=0x7fd4f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.860+0000 7fd50bd44640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd4f00a2c10 0x7fd4f00a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd5040fdf50 tx=0x7fd50000a2d0 comp rx=0 tx=0).stop 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.860+0000 7fd50bd44640 1 -- 172.21.15.133:0/120804701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd504102d30 msgr2=0x7fd50419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.860+0000 7fd50bd44640 1 --2- 172.21.15.133:0/120804701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd504102d30 0x7fd50419a730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800c970 tx=0x7fd4f800ce30 comp rx=0 tx=0).stop 2024-09-11T23:35:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.860+0000 7fd50a2e1640 1 -- 172.21.15.133:0/120804701 reap_dead start 2024-09-11T23:35:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.860+0000 7fd50bd44640 1 -- 172.21.15.133:0/120804701 shutdown_connections 2024-09-11T23:35:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.860+0000 7fd50bd44640 1 -- 172.21.15.133:0/120804701 >> 172.21.15.133:0/120804701 conn(0x7fd5040fc7d0 msgr2=0x7fd504101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:34.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.860+0000 7fd50bd44640 1 -- 172.21.15.133:0/120804701 shutdown_connections 2024-09-11T23:35:34.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:34.860+0000 7fd50bd44640 1 -- 172.21.15.133:0/120804701 wait complete. 2024-09-11T23:35:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:34 smithi138 ceph-mon[35142]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:35:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:34 smithi133 ceph-mon[31430]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:35:35.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:34 smithi178 ceph-mon[35882]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:35:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.336+0000 7ff57f678640 1 -- 172.21.15.133:0/546392544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff578106a80 msgr2=0x7ff578106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.336+0000 7ff57f678640 1 --2- 172.21.15.133:0/546392544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff578106a80 0x7ff578106e60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff574009930 tx=0x7ff57402a420 comp rx=0 tx=0).stop 2024-09-11T23:35:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.336+0000 7ff57f678640 1 -- 172.21.15.133:0/546392544 shutdown_connections 2024-09-11T23:35:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.336+0000 7ff57f678640 1 --2- 172.21.15.133:0/546392544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff578101420 0x7ff57810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.336+0000 7ff57f678640 1 --2- 172.21.15.133:0/546392544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff578100a80 0x7ff578100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.336+0000 7ff57f678640 1 --2- 172.21.15.133:0/546392544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff578106a80 0x7ff578106e60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.336+0000 7ff57f678640 1 -- 172.21.15.133:0/546392544 >> 172.21.15.133:0/546392544 conn(0x7ff5780fc7a0 msgr2=0x7ff5780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.336+0000 7ff57f678640 1 -- 172.21.15.133:0/546392544 shutdown_connections 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.336+0000 7ff57f678640 1 -- 172.21.15.133:0/546392544 wait complete. 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57f678640 1 Processor -- start 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57f678640 1 -- start start 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57f678640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff578100a80 0x7ff5780fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57f678640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff578101420 0x7ff5780fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57f678640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff578106a80 0x7ff5780fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57f678640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff578110ea0 con 0x7ff578100a80 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57f678640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff578110d20 con 0x7ff578106a80 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57f678640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff578111020 con 0x7ff578101420 2024-09-11T23:35:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57d414640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff578100a80 0x7ff5780fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57d414640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff578100a80 0x7ff5780fce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49674/0 (socket says 172.21.15.133:49674) 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57d414640 1 -- 172.21.15.133:0/1893682746 learned_addr learned my addr 172.21.15.133:0/1893682746 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57dc15640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff578106a80 0x7ff5780fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57cc13640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff578101420 0x7ff5780fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57d414640 1 -- 172.21.15.133:0/1893682746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff578101420 msgr2=0x7ff5780fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57d414640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff578101420 0x7ff5780fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57d414640 1 -- 172.21.15.133:0/1893682746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff578106a80 msgr2=0x7ff5780fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57d414640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff578106a80 0x7ff5780fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57d414640 1 -- 172.21.15.133:0/1893682746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5781a6970 con 0x7ff578100a80 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.337+0000 7ff57cc13640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff578101420 0x7ff5780fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.338+0000 7ff57d414640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff578100a80 0x7ff5780fce70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff574009230 tx=0x7ff574004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:35.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.338+0000 7ff56e7fc640 1 -- 172.21.15.133:0/1893682746 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57403f030 con 0x7ff578100a80 2024-09-11T23:35:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.338+0000 7ff57f678640 1 -- 172.21.15.133:0/1893682746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5781a6c00 con 0x7ff578100a80 2024-09-11T23:35:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.338+0000 7ff56e7fc640 1 -- 172.21.15.133:0/1893682746 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff574003ad0 con 0x7ff578100a80 2024-09-11T23:35:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.338+0000 7ff56e7fc640 1 -- 172.21.15.133:0/1893682746 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574035070 con 0x7ff578100a80 2024-09-11T23:35:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.338+0000 7ff57f678640 1 -- 172.21.15.133:0/1893682746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5781a70e0 con 0x7ff578100a80 2024-09-11T23:35:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.339+0000 7ff57f678640 1 -- 172.21.15.133:0/1893682746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff5780628c0 con 0x7ff578100a80 2024-09-11T23:35:35.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.340+0000 7ff56e7fc640 1 -- 172.21.15.133:0/1893682746 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff574042030 con 0x7ff578100a80 2024-09-11T23:35:35.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.340+0000 7ff56e7fc640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ff5640a2c90 0x7ff5640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:35.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.341+0000 7ff57cc13640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ff5640a2c90 0x7ff5640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:35.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.341+0000 7ff56e7fc640 1 -- 172.21.15.133:0/1893682746 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7ff5740ef780 con 0x7ff578100a80 2024-09-11T23:35:35.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.341+0000 7ff56e7fc640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574] conn(0x7ff5640ac740 0x7ff5640aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:35.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.341+0000 7ff56e7fc640 1 -- 172.21.15.133:0/1893682746 --> [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff574049e50 con 0x7ff5640ac740 2024-09-11T23:35:35.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.341+0000 7ff56e7fc640 1 -- 172.21.15.133:0/1893682746 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ff5740efb70 con 0x7ff578100a80 2024-09-11T23:35:35.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.341+0000 7ff57dc15640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574] conn(0x7ff5640ac740 0x7ff5640aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:35.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.341+0000 7ff57cc13640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ff5640a2c90 0x7ff5640a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff5780fe900 tx=0x7ff568008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:35.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.341+0000 7ff57dc15640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574] conn(0x7ff5640ac740 0x7ff5640aebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:35.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.342+0000 7ff56e7fc640 1 -- 172.21.15.133:0/1893682746 <== osd.9 v2:172.21.15.133:6826/1405512574 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff574049e50 con 0x7ff5640ac740 2024-09-11T23:35:35.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.355+0000 7ff57f678640 1 -- 172.21.15.133:0/1893682746 --> [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff578062d40 con 0x7ff5640ac740 2024-09-11T23:35:35.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff56e7fc640 1 -- 172.21.15.133:0/1893682746 <== osd.9 v2:172.21.15.133:6826/1405512574 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff578062d40 con 0x7ff5640ac740 2024-09-11T23:35:35.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff57f678640 1 -- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574] conn(0x7ff5640ac740 msgr2=0x7ff5640aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:35.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff57f678640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574] conn(0x7ff5640ac740 0x7ff5640aebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:35.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff57f678640 1 -- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ff5640a2c90 msgr2=0x7ff5640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:35.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff57f678640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7ff5640a2c90 0x7ff5640a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff5780fe900 tx=0x7ff568008040 comp rx=0 tx=0).stop 2024-09-11T23:35:35.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff57f678640 1 -- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff578100a80 msgr2=0x7ff5780fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:35.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff57f678640 1 --2- 172.21.15.133:0/1893682746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff578100a80 0x7ff5780fce70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff574009230 tx=0x7ff574004270 comp rx=0 tx=0).stop 2024-09-11T23:35:35.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff57dc15640 1 -- 172.21.15.133:0/1893682746 reap_dead start 2024-09-11T23:35:35.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff57f678640 1 -- 172.21.15.133:0/1893682746 shutdown_connections 2024-09-11T23:35:35.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff57f678640 1 -- 172.21.15.133:0/1893682746 >> 172.21.15.133:0/1893682746 conn(0x7ff5780fc7a0 msgr2=0x7ff57806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:35.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff57f678640 1 -- 172.21.15.133:0/1893682746 shutdown_connections 2024-09-11T23:35:35.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.356+0000 7ff57f678640 1 -- 172.21.15.133:0/1893682746 wait complete. 2024-09-11T23:35:35.870 INFO:teuthology.orchestra.run.smithi133.stdout:197568495656 2024-09-11T23:35:35.870 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T23:35:36.591 INFO:teuthology.orchestra.run.smithi133.stdout:176093659180 2024-09-11T23:35:36.592 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T23:35:36.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.872+0000 7f0d12a4d640 1 -- 172.21.15.133:0/1038040814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d0c103650 msgr2=0x7f0d0c10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.872+0000 7f0d12a4d640 1 --2- 172.21.15.133:0/1038040814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d0c103650 0x7f0d0c10bb40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009900 tx=0x7f0cf402a480 comp rx=0 tx=0).stop 2024-09-11T23:35:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.872+0000 7f0d12a4d640 1 -- 172.21.15.133:0/1038040814 shutdown_connections 2024-09-11T23:35:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.872+0000 7f0d12a4d640 1 --2- 172.21.15.133:0/1038040814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d0c10c160 0x7f0d0c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.872+0000 7f0d12a4d640 1 --2- 172.21.15.133:0/1038040814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d0c103650 0x7f0d0c10bb40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.872+0000 7f0d12a4d640 1 --2- 172.21.15.133:0/1038040814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d0c102d30 0x7f0d0c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.872+0000 7f0d12a4d640 1 -- 172.21.15.133:0/1038040814 >> 172.21.15.133:0/1038040814 conn(0x7f0d0c0fc790 msgr2=0x7f0d0c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.872+0000 7f0d12a4d640 1 -- 172.21.15.133:0/1038040814 shutdown_connections 2024-09-11T23:35:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.873+0000 7f0d12a4d640 1 -- 172.21.15.133:0/1038040814 wait complete. 2024-09-11T23:35:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.873+0000 7f0d12a4d640 1 Processor -- start 2024-09-11T23:35:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.873+0000 7f0d12a4d640 1 -- start start 2024-09-11T23:35:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.873+0000 7f0d12a4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d0c102d30 0x7f0d0c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.873+0000 7f0d12a4d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d0c103650 0x7f0d0c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.873+0000 7f0d12a4d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d0c10c160 0x7f0d0c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.873+0000 7f0d12a4d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c111f60 con 0x7f0d0c102d30 2024-09-11T23:35:36.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.873+0000 7f0d12a4d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c111de0 con 0x7f0d0c10c160 2024-09-11T23:35:36.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.873+0000 7f0d12a4d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1120e0 con 0x7f0d0c103650 2024-09-11T23:35:36.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.873+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d0c102d30 0x7f0d0c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d0c102d30 0x7f0d0c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49704/0 (socket says 172.21.15.133:49704) 2024-09-11T23:35:36.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d0bfff640 1 -- 172.21.15.133:0/864160336 learned_addr learned my addr 172.21.15.133:0/864160336 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:36.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d03fff640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d0c103650 0x7f0d0c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d10fea640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d0c10c160 0x7f0d0c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d0bfff640 1 -- 172.21.15.133:0/864160336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d0c103650 msgr2=0x7f0d0c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d0bfff640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d0c103650 0x7f0d0c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d0bfff640 1 -- 172.21.15.133:0/864160336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d0c10c160 msgr2=0x7f0d0c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d0bfff640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d0c10c160 0x7f0d0c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d0bfff640 1 -- 172.21.15.133:0/864160336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c1a69f0 con 0x7f0d0c102d30 2024-09-11T23:35:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d0bfff640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d0c102d30 0x7f0d0c100990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00d950 tx=0x7f0cfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d09ffb640 1 -- 172.21.15.133:0/864160336 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc017070 con 0x7f0d0c102d30 2024-09-11T23:35:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d09ffb640 1 -- 172.21.15.133:0/864160336 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cfc009c30 con 0x7f0d0c102d30 2024-09-11T23:35:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d09ffb640 1 -- 172.21.15.133:0/864160336 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc00f6b0 con 0x7f0d0c102d30 2024-09-11T23:35:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d12a4d640 1 -- 172.21.15.133:0/864160336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0c1a6ce0 con 0x7f0d0c102d30 2024-09-11T23:35:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.874+0000 7f0d12a4d640 1 -- 172.21.15.133:0/864160336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0c1a71c0 con 0x7f0d0c102d30 2024-09-11T23:35:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.875+0000 7f0d12a4d640 1 -- 172.21.15.133:0/864160336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d0c0628c0 con 0x7f0d0c102d30 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.876+0000 7f0d09ffb640 1 -- 172.21.15.133:0/864160336 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0cfc0109d0 con 0x7f0d0c102d30 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.877+0000 7f0d09ffb640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0cf80a2c10 0x7f0cf80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.877+0000 7f0d03fff640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0cf80a2c10 0x7f0cf80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.877+0000 7f0d09ffb640 1 -- 172.21.15.133:0/864160336 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f0cfc0cf190 con 0x7f0d0c102d30 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.877+0000 7f0d09ffb640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734] conn(0x7f0cf80ac6c0 0x7f0cf80aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.877+0000 7f0d09ffb640 1 -- 172.21.15.133:0/864160336 --> [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0cfc016ec0 con 0x7f0cf80ac6c0 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.877+0000 7f0d09ffb640 1 -- 172.21.15.133:0/864160336 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc0cf670 con 0x7f0d0c102d30 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.878+0000 7f0d03fff640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0cf80a2c10 0x7f0cf80a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0fee60 tx=0x7f0cf4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.878+0000 7f0d10fea640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734] conn(0x7f0cf80ac6c0 0x7f0cf80aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.878+0000 7f0d10fea640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734] conn(0x7f0cf80ac6c0 0x7f0cf80aeb60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.879+0000 7f0d09ffb640 1 -- 172.21.15.133:0/864160336 <== osd.4 v2:172.21.15.138:6808/952086734 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0cfc016ec0 con 0x7f0cf80ac6c0 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.892+0000 7f0d12a4d640 1 -- 172.21.15.133:0/864160336 --> [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d0c062cc0 con 0x7f0cf80ac6c0 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d09ffb640 1 -- 172.21.15.133:0/864160336 <== osd.4 v2:172.21.15.138:6808/952086734 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0d0c062cc0 con 0x7f0cf80ac6c0 2024-09-11T23:35:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d12a4d640 1 -- 172.21.15.133:0/864160336 >> [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734] conn(0x7f0cf80ac6c0 msgr2=0x7f0cf80aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d12a4d640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734] conn(0x7f0cf80ac6c0 0x7f0cf80aeb60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d12a4d640 1 -- 172.21.15.133:0/864160336 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0cf80a2c10 msgr2=0x7f0cf80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d12a4d640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f0cf80a2c10 0x7f0cf80a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0fee60 tx=0x7f0cf4005a20 comp rx=0 tx=0).stop 2024-09-11T23:35:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d12a4d640 1 -- 172.21.15.133:0/864160336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d0c102d30 msgr2=0x7f0d0c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d12a4d640 1 --2- 172.21.15.133:0/864160336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d0c102d30 0x7f0d0c100990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00d950 tx=0x7f0cfc00de10 comp rx=0 tx=0).stop 2024-09-11T23:35:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d10fea640 1 -- 172.21.15.133:0/864160336 reap_dead start 2024-09-11T23:35:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d12a4d640 1 -- 172.21.15.133:0/864160336 shutdown_connections 2024-09-11T23:35:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d12a4d640 1 -- 172.21.15.133:0/864160336 >> 172.21.15.133:0/864160336 conn(0x7f0d0c0fc790 msgr2=0x7f0d0c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d12a4d640 1 -- 172.21.15.133:0/864160336 shutdown_connections 2024-09-11T23:35:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:36.893+0000 7f0d12a4d640 1 -- 172.21.15.133:0/864160336 wait complete. 2024-09-11T23:35:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.546+0000 7fc33cc68640 1 -- 172.21.15.133:0/326299632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc338101420 msgr2=0x7fc33810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.546+0000 7fc33cc68640 1 --2- 172.21.15.133:0/326299632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc338101420 0x7fc33810d950 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc330009cd0 tx=0x7fc33002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:35:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.546+0000 7fc33cc68640 1 -- 172.21.15.133:0/326299632 shutdown_connections 2024-09-11T23:35:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.546+0000 7fc33cc68640 1 --2- 172.21.15.133:0/326299632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc338101420 0x7fc33810d950 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.546+0000 7fc33cc68640 1 --2- 172.21.15.133:0/326299632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc338100a80 0x7fc338100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.546+0000 7fc33cc68640 1 --2- 172.21.15.133:0/326299632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc338106a80 0x7fc338106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.546+0000 7fc33cc68640 1 -- 172.21.15.133:0/326299632 >> 172.21.15.133:0/326299632 conn(0x7fc3380fc7a0 msgr2=0x7fc3380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.546+0000 7fc33cc68640 1 -- 172.21.15.133:0/326299632 shutdown_connections 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.546+0000 7fc33cc68640 1 -- 172.21.15.133:0/326299632 wait complete. 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc33cc68640 1 Processor -- start 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc33cc68640 1 -- start start 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc33cc68640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc338100a80 0x7fc3381983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc33cc68640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc338101420 0x7fc338198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc33cc68640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc338106a80 0x7fc33819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc33cc68640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc338111fa0 con 0x7fc338106a80 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc33cc68640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc338111e20 con 0x7fc338101420 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc33cc68640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc338112120 con 0x7fc338100a80 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc336d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc338106a80 0x7fc33819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc336d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc338106a80 0x7fc33819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49696/0 (socket says 172.21.15.133:49696) 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc336d9d640 1 -- 172.21.15.133:0/1603336178 learned_addr learned my addr 172.21.15.133:0/1603336178 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc33659c640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc338100a80 0x7fc3381983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:37.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.547+0000 7fc335d9b640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc338101420 0x7fc338198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:37.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.548+0000 7fc336d9d640 1 -- 172.21.15.133:0/1603336178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc338100a80 msgr2=0x7fc3381983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:37.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.548+0000 7fc336d9d640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc338100a80 0x7fc3381983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:37.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.548+0000 7fc336d9d640 1 -- 172.21.15.133:0/1603336178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc338101420 msgr2=0x7fc338198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:37.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.548+0000 7fc336d9d640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc338101420 0x7fc338198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:37.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.548+0000 7fc336d9d640 1 -- 172.21.15.133:0/1603336178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33819d370 con 0x7fc338106a80 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.548+0000 7fc336d9d640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc338106a80 0x7fc33819cc90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc330005cc0 tx=0x7fc330003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.548+0000 7fc3277fe640 1 -- 172.21.15.133:0/1603336178 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330004040 con 0x7fc338106a80 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.548+0000 7fc3277fe640 1 -- 172.21.15.133:0/1603336178 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3300046c0 con 0x7fc338106a80 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.548+0000 7fc33cc68640 1 -- 172.21.15.133:0/1603336178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33819d600 con 0x7fc338106a80 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.548+0000 7fc3277fe640 1 -- 172.21.15.133:0/1603336178 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330030e70 con 0x7fc338106a80 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.548+0000 7fc33cc68640 1 -- 172.21.15.133:0/1603336178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3381a4f40 con 0x7fc338106a80 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.549+0000 7fc33cc68640 1 -- 172.21.15.133:0/1603336178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc3380fd8f0 con 0x7fc338106a80 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.550+0000 7fc3277fe640 1 -- 172.21.15.133:0/1603336178 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc3300305a0 con 0x7fc338106a80 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.551+0000 7fc3277fe640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc3200a2c10 0x7fc3200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.551+0000 7fc33659c640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc3200a2c10 0x7fc3200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.551+0000 7fc3277fe640 1 -- 172.21.15.133:0/1603336178 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fc33002e030 con 0x7fc338106a80 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.551+0000 7fc3277fe640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700] conn(0x7fc3200ac6c0 0x7fc3200aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.551+0000 7fc3277fe640 1 -- 172.21.15.133:0/1603336178 --> [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3200af230 con 0x7fc3200ac6c0 2024-09-11T23:35:37.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.551+0000 7fc3277fe640 1 -- 172.21.15.133:0/1603336178 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc3300f08b0 con 0x7fc338106a80 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.551+0000 7fc33659c640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc3200a2c10 0x7fc3200a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc328005fd0 tx=0x7fc32800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.552+0000 7fc335d9b640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700] conn(0x7fc3200ac6c0 0x7fc3200aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.552+0000 7fc335d9b640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700] conn(0x7fc3200ac6c0 0x7fc3200aeb60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.553+0000 7fc3277fe640 1 -- 172.21.15.133:0/1603336178 <== osd.8 v2:172.21.15.178:6816/3469671700 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3200af230 con 0x7fc3200ac6c0 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.566+0000 7fc33cc68640 1 -- 172.21.15.133:0/1603336178 --> [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc3381021c0 con 0x7fc3200ac6c0 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.567+0000 7fc3277fe640 1 -- 172.21.15.133:0/1603336178 <== osd.8 v2:172.21.15.178:6816/3469671700 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc3381021c0 con 0x7fc3200ac6c0 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.567+0000 7fc33cc68640 1 -- 172.21.15.133:0/1603336178 >> [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700] conn(0x7fc3200ac6c0 msgr2=0x7fc3200aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.567+0000 7fc33cc68640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700] conn(0x7fc3200ac6c0 0x7fc3200aeb60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.567+0000 7fc33cc68640 1 -- 172.21.15.133:0/1603336178 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc3200a2c10 msgr2=0x7fc3200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.567+0000 7fc33cc68640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc3200a2c10 0x7fc3200a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc328005fd0 tx=0x7fc32800b390 comp rx=0 tx=0).stop 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.567+0000 7fc33cc68640 1 -- 172.21.15.133:0/1603336178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc338106a80 msgr2=0x7fc33819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.567+0000 7fc33cc68640 1 --2- 172.21.15.133:0/1603336178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc338106a80 0x7fc33819cc90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc330005cc0 tx=0x7fc330003970 comp rx=0 tx=0).stop 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.567+0000 7fc336d9d640 1 -- 172.21.15.133:0/1603336178 reap_dead start 2024-09-11T23:35:37.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.567+0000 7fc33cc68640 1 -- 172.21.15.133:0/1603336178 shutdown_connections 2024-09-11T23:35:37.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.567+0000 7fc33cc68640 1 -- 172.21.15.133:0/1603336178 >> 172.21.15.133:0/1603336178 conn(0x7fc3380fc7a0 msgr2=0x7fc3380fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:37.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.567+0000 7fc33cc68640 1 -- 172.21.15.133:0/1603336178 shutdown_connections 2024-09-11T23:35:37.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:35.568+0000 7fc33cc68640 1 -- 172.21.15.133:0/1603336178 wait complete. 2024-09-11T23:35:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:36 smithi138 ceph-mon[35142]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:35:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:36 smithi133 ceph-mon[31430]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:35:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:36 smithi178 ceph-mon[35882]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T23:35:38.429 INFO:teuthology.orchestra.run.smithi133.stdout:128849018936 2024-09-11T23:35:38.430 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T23:35:38.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.583+0000 7f301232d640 1 -- 172.21.15.133:0/2621742416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f300c1027a0 msgr2=0x7f300c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.583+0000 7f301232d640 1 --2- 172.21.15.133:0/2621742416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f300c1027a0 0x7f300c1067f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3000009900 tx=0x7f300002a410 comp rx=0 tx=0).stop 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.584+0000 7f301232d640 1 -- 172.21.15.133:0/2621742416 shutdown_connections 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.584+0000 7f301232d640 1 --2- 172.21.15.133:0/2621742416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f300c106f20 0x7f300c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.584+0000 7f301232d640 1 --2- 172.21.15.133:0/2621742416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f300c1027a0 0x7f300c1067f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.584+0000 7f301232d640 1 --2- 172.21.15.133:0/2621742416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f300c101df0 0x7f300c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.584+0000 7f301232d640 1 -- 172.21.15.133:0/2621742416 >> 172.21.15.133:0/2621742416 conn(0x7f300c0fd8f0 msgr2=0x7f300c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.584+0000 7f301232d640 1 -- 172.21.15.133:0/2621742416 shutdown_connections 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.584+0000 7f301232d640 1 -- 172.21.15.133:0/2621742416 wait complete. 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.584+0000 7f301232d640 1 Processor -- start 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.584+0000 7f301232d640 1 -- start start 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f301232d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f300c101df0 0x7f300c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:38.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f301232d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f300c1027a0 0x7f300c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f301232d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f300c106f20 0x7f300c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f301232d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f300c111f80 con 0x7f300c101df0 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f301232d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f300c111e00 con 0x7f300c106f20 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f301232d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f300c112100 con 0x7f300c1027a0 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f300bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f300c101df0 0x7f300c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f300bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f300c101df0 0x7f300c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49722/0 (socket says 172.21.15.133:49722) 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f300bfff640 1 -- 172.21.15.133:0/2108877682 learned_addr learned my addr 172.21.15.133:0/2108877682 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f300b7fe640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f300c1027a0 0x7f300c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f30108ca640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f300c106f20 0x7f300c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f300bfff640 1 -- 172.21.15.133:0/2108877682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f300c1027a0 msgr2=0x7f300c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f300bfff640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f300c1027a0 0x7f300c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f300bfff640 1 -- 172.21.15.133:0/2108877682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f300c106f20 msgr2=0x7f300c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:38.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f300bfff640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f300c106f20 0x7f300c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f300bfff640 1 -- 172.21.15.133:0/2108877682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c19d390 con 0x7f300c101df0 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f30108ca640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f300c106f20 0x7f300c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f300bfff640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f300c101df0 0x7f300c1983e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00c920 tx=0x7f2ffc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f30097fa640 1 -- 172.21.15.133:0/2108877682 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc007c70 con 0x7f300c101df0 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f301232d640 1 -- 172.21.15.133:0/2108877682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c19d680 con 0x7f300c101df0 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f30097fa640 1 -- 172.21.15.133:0/2108877682 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ffc007e10 con 0x7f300c101df0 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f30097fa640 1 -- 172.21.15.133:0/2108877682 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc018890 con 0x7f300c101df0 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.585+0000 7f301232d640 1 -- 172.21.15.133:0/2108877682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c1a4f60 con 0x7f300c101df0 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.586+0000 7f301232d640 1 -- 172.21.15.133:0/2108877682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f300c062980 con 0x7f300c101df0 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.588+0000 7f30097fa640 1 -- 172.21.15.133:0/2108877682 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2ffc018a30 con 0x7f300c101df0 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.589+0000 7f30097fa640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2ff40a2c10 0x7f2ff40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.589+0000 7f300b7fe640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2ff40a2c10 0x7f2ff40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.589+0000 7f30097fa640 1 -- 172.21.15.133:0/2108877682 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f2ffc0cf640 con 0x7f300c101df0 2024-09-11T23:35:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.589+0000 7f30097fa640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536] conn(0x7f2ff40ac6c0 0x7f2ff40aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:38.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.589+0000 7f30097fa640 1 -- 172.21.15.133:0/2108877682 --> [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ff40af230 con 0x7f2ff40ac6c0 2024-09-11T23:35:38.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.589+0000 7f30097fa640 1 -- 172.21.15.133:0/2108877682 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f2ffc0cfb20 con 0x7f300c101df0 2024-09-11T23:35:38.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.589+0000 7f300b7fe640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2ff40a2c10 0x7f2ff40a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3000009230 tx=0x7f30000058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:38.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.589+0000 7f30108ca640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536] conn(0x7f2ff40ac6c0 0x7f2ff40aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:38.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.590+0000 7f30108ca640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536] conn(0x7f2ff40ac6c0 0x7f2ff40aeb60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:38.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.591+0000 7f30097fa640 1 -- 172.21.15.133:0/2108877682 <== osd.5 v2:172.21.15.178:6808/1922026536 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ff40af230 con 0x7f2ff40ac6c0 2024-09-11T23:35:38.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.604+0000 7f301232d640 1 -- 172.21.15.133:0/2108877682 --> [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f300c062e00 con 0x7f2ff40ac6c0 2024-09-11T23:35:38.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.605+0000 7f30097fa640 1 -- 172.21.15.133:0/2108877682 <== osd.5 v2:172.21.15.178:6808/1922026536 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f300c062e00 con 0x7f2ff40ac6c0 2024-09-11T23:35:38.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.605+0000 7f301232d640 1 -- 172.21.15.133:0/2108877682 >> [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536] conn(0x7f2ff40ac6c0 msgr2=0x7f2ff40aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:38.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.605+0000 7f301232d640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536] conn(0x7f2ff40ac6c0 0x7f2ff40aeb60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:38.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.605+0000 7f301232d640 1 -- 172.21.15.133:0/2108877682 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2ff40a2c10 msgr2=0x7f2ff40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:38.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.605+0000 7f301232d640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2ff40a2c10 0x7f2ff40a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3000009230 tx=0x7f30000058d0 comp rx=0 tx=0).stop 2024-09-11T23:35:38.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.605+0000 7f301232d640 1 -- 172.21.15.133:0/2108877682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f300c101df0 msgr2=0x7f300c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:38.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.605+0000 7f301232d640 1 --2- 172.21.15.133:0/2108877682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f300c101df0 0x7f300c1983e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00c920 tx=0x7f2ffc00cde0 comp rx=0 tx=0).stop 2024-09-11T23:35:38.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.605+0000 7f30108ca640 1 -- 172.21.15.133:0/2108877682 reap_dead start 2024-09-11T23:35:38.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.605+0000 7f301232d640 1 -- 172.21.15.133:0/2108877682 shutdown_connections 2024-09-11T23:35:38.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.605+0000 7f301232d640 1 -- 172.21.15.133:0/2108877682 >> 172.21.15.133:0/2108877682 conn(0x7f300c0fd8f0 msgr2=0x7f300c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:38.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.606+0000 7f301232d640 1 -- 172.21.15.133:0/2108877682 shutdown_connections 2024-09-11T23:35:38.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:38.606+0000 7f301232d640 1 -- 172.21.15.133:0/2108877682 wait complete. 2024-09-11T23:35:39.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:38 smithi133 ceph-mon[31430]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T23:35:39.049 INFO:teuthology.orchestra.run.smithi133.stdout:73014444097 2024-09-11T23:35:39.049 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T23:35:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:38 smithi178 ceph-mon[35882]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T23:35:39.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:38 smithi138 ceph-mon[35142]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T23:35:39.801 INFO:teuthology.orchestra.run.smithi133.stdout:283467841561 2024-09-11T23:35:39.801 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-11T23:35:40.221 INFO:teuthology.orchestra.run.smithi133.stdout:313532612629 2024-09-11T23:35:40.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-11T23:35:40.775 INFO:teuthology.orchestra.run.smithi133.stdout:261993005087 2024-09-11T23:35:40.775 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-11T23:35:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.197+0000 7fe7b3568640 1 -- 172.21.15.133:0/3935262237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac102d30 msgr2=0x7fe7ac103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.197+0000 7fe7b3568640 1 --2- 172.21.15.133:0/3935262237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac102d30 0x7fe7ac103110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009930 tx=0x7fe7a002a440 comp rx=0 tx=0).stop 2024-09-11T23:35:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.197+0000 7fe7b3568640 1 -- 172.21.15.133:0/3935262237 shutdown_connections 2024-09-11T23:35:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.197+0000 7fe7b3568640 1 --2- 172.21.15.133:0/3935262237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7ac1082a0 0x7fe7ac10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.197+0000 7fe7b3568640 1 --2- 172.21.15.133:0/3935262237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7ac1036e0 0x7fe7ac107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.197+0000 7fe7b3568640 1 --2- 172.21.15.133:0/3935262237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac102d30 0x7fe7ac103110 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.197+0000 7fe7b3568640 1 -- 172.21.15.133:0/3935262237 >> 172.21.15.133:0/3935262237 conn(0x7fe7ac0fc7d0 msgr2=0x7fe7ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.198+0000 7fe7b3568640 1 -- 172.21.15.133:0/3935262237 shutdown_connections 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.198+0000 7fe7b3568640 1 -- 172.21.15.133:0/3935262237 wait complete. 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.198+0000 7fe7b3568640 1 Processor -- start 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.198+0000 7fe7b3568640 1 -- start start 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.198+0000 7fe7b3568640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7ac102d30 0x7fe7ac198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.198+0000 7fe7b3568640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac1036e0 0x7fe7ac198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.198+0000 7fe7b3568640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7ac1082a0 0x7fe7ac19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.198+0000 7fe7b3568640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac112160 con 0x7fe7ac1036e0 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.198+0000 7fe7b3568640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac111fe0 con 0x7fe7ac102d30 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.198+0000 7fe7b3568640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1122e0 con 0x7fe7ac1082a0 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.198+0000 7fe7b0b03640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac1036e0 0x7fe7ac198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b0b03640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac1036e0 0x7fe7ac198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49732/0 (socket says 172.21.15.133:49732) 2024-09-11T23:35:40.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b0b03640 1 -- 172.21.15.133:0/2937444516 learned_addr learned my addr 172.21.15.133:0/2937444516 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b1304640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7ac102d30 0x7fe7ac198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b1b05640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7ac1082a0 0x7fe7ac19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b0b03640 1 -- 172.21.15.133:0/2937444516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7ac1082a0 msgr2=0x7fe7ac19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b0b03640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7ac1082a0 0x7fe7ac19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b0b03640 1 -- 172.21.15.133:0/2937444516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7ac102d30 msgr2=0x7fe7ac198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b0b03640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7ac102d30 0x7fe7ac198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b0b03640 1 -- 172.21.15.133:0/2937444516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac19d6d0 con 0x7fe7ac1036e0 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b1b05640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7ac1082a0 0x7fe7ac19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b0b03640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac1036e0 0x7fe7ac198c60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8006fd0 tx=0x7fe7a800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe79e7fc640 1 -- 172.21.15.133:0/2937444516 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8011830 con 0x7fe7ac1036e0 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b3568640 1 -- 172.21.15.133:0/2937444516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac19d9c0 con 0x7fe7ac1036e0 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe79e7fc640 1 -- 172.21.15.133:0/2937444516 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a8016070 con 0x7fe7ac1036e0 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe79e7fc640 1 -- 172.21.15.133:0/2937444516 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800f2e0 con 0x7fe7ac1036e0 2024-09-11T23:35:40.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.199+0000 7fe7b3568640 1 -- 172.21.15.133:0/2937444516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac1a52a0 con 0x7fe7ac1036e0 2024-09-11T23:35:40.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.200+0000 7fe7b3568640 1 -- 172.21.15.133:0/2937444516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7ac062980 con 0x7fe7ac1036e0 2024-09-11T23:35:40.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.201+0000 7fe79e7fc640 1 -- 172.21.15.133:0/2937444516 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe7a80119d0 con 0x7fe7ac1036e0 2024-09-11T23:35:40.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.202+0000 7fe79e7fc640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe7980a2c10 0x7fe7980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:40.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.202+0000 7fe7b1304640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe7980a2c10 0x7fe7980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:40.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.202+0000 7fe79e7fc640 1 -- 172.21.15.133:0/2937444516 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fe7a80cf190 con 0x7fe7ac1036e0 2024-09-11T23:35:40.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.202+0000 7fe79e7fc640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926] conn(0x7fe7980ac6c0 0x7fe7980aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:40.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.202+0000 7fe79e7fc640 1 -- 172.21.15.133:0/2937444516 --> [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7a801ee50 con 0x7fe7980ac6c0 2024-09-11T23:35:40.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.202+0000 7fe79e7fc640 1 -- 172.21.15.133:0/2937444516 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fe7a80cf670 con 0x7fe7ac1036e0 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.202+0000 7fe7b1304640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe7980a2c10 0x7fe7980a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0000c00 tx=0x7fe7a0031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.203+0000 7fe7b1b05640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926] conn(0x7fe7980ac6c0 0x7fe7980aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.203+0000 7fe7b1b05640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926] conn(0x7fe7980ac6c0 0x7fe7980aeb60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.204+0000 7fe79e7fc640 1 -- 172.21.15.133:0/2937444516 <== osd.1 v2:172.21.15.138:6800/2975334926 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7a801ee50 con 0x7fe7980ac6c0 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.217+0000 7fe7b3568640 1 -- 172.21.15.133:0/2937444516 --> [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7ac062d80 con 0x7fe7980ac6c0 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.217+0000 7fe79e7fc640 1 -- 172.21.15.133:0/2937444516 <== osd.1 v2:172.21.15.138:6800/2975334926 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe7ac062d80 con 0x7fe7980ac6c0 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.218+0000 7fe7b3568640 1 -- 172.21.15.133:0/2937444516 >> [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926] conn(0x7fe7980ac6c0 msgr2=0x7fe7980aeb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.218+0000 7fe7b3568640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926] conn(0x7fe7980ac6c0 0x7fe7980aeb60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.218+0000 7fe7b3568640 1 -- 172.21.15.133:0/2937444516 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe7980a2c10 msgr2=0x7fe7980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.218+0000 7fe7b3568640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe7980a2c10 0x7fe7980a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0000c00 tx=0x7fe7a0031490 comp rx=0 tx=0).stop 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.218+0000 7fe7b3568640 1 -- 172.21.15.133:0/2937444516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac1036e0 msgr2=0x7fe7ac198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:40.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.218+0000 7fe7b3568640 1 --2- 172.21.15.133:0/2937444516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac1036e0 0x7fe7ac198c60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8006fd0 tx=0x7fe7a800b8e0 comp rx=0 tx=0).stop 2024-09-11T23:35:40.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.218+0000 7fe7b1b05640 1 -- 172.21.15.133:0/2937444516 reap_dead start 2024-09-11T23:35:40.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.218+0000 7fe7b3568640 1 -- 172.21.15.133:0/2937444516 shutdown_connections 2024-09-11T23:35:40.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.218+0000 7fe7b3568640 1 -- 172.21.15.133:0/2937444516 >> 172.21.15.133:0/2937444516 conn(0x7fe7ac0fc7d0 msgr2=0x7fe7ac105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:40.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.218+0000 7fe7b3568640 1 -- 172.21.15.133:0/2937444516 shutdown_connections 2024-09-11T23:35:40.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:39.218+0000 7fe7b3568640 1 -- 172.21.15.133:0/2937444516 wait complete. 2024-09-11T23:35:41.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:40 smithi133 ceph-mon[31430]: pgmap v297: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T23:35:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:40 smithi178 ceph-mon[35882]: pgmap v297: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T23:35:41.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:40 smithi138 ceph-mon[35142]: pgmap v297: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T23:35:41.953 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:41.991 INFO:teuthology.orchestra.run.smithi133.stdout:146028888115 2024-09-11T23:35:41.992 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T23:35:42.547 INFO:teuthology.orchestra.run.smithi133.stdout:51539607623 2024-09-11T23:35:42.547 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T23:35:43.028 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:43.031 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:43.031 INFO:teuthology.orchestra.run.smithi133.stdout:231928234021 2024-09-11T23:35:43.032 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-11T23:35:43.035 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:43.036 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:43.036 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:43.040 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:43.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:42 smithi133 ceph-mon[31430]: pgmap v298: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:35:43.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:42 smithi178 ceph-mon[35882]: pgmap v298: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:35:43.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:42 smithi138 ceph-mon[35142]: pgmap v298: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:35:43.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.789+0000 7f55a2225640 1 -- 172.21.15.133:0/3685365068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c106f10 msgr2=0x7f559c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:43.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.789+0000 7f55a2225640 1 --2- 172.21.15.133:0/3685365068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c106f10 0x7f559c10ea90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5594009900 tx=0x7f559402a410 comp rx=0 tx=0).stop 2024-09-11T23:35:43.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.789+0000 7f55a2225640 1 -- 172.21.15.133:0/3685365068 shutdown_connections 2024-09-11T23:35:43.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.789+0000 7f55a2225640 1 --2- 172.21.15.133:0/3685365068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c106f10 0x7f559c10ea90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.789+0000 7f55a2225640 1 --2- 172.21.15.133:0/3685365068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c102790 0x7f559c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.789+0000 7f55a2225640 1 --2- 172.21.15.133:0/3685365068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c101de0 0x7f559c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.789+0000 7f55a2225640 1 -- 172.21.15.133:0/3685365068 >> 172.21.15.133:0/3685365068 conn(0x7f559c0fd8c0 msgr2=0x7f559c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:43.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.789+0000 7f55a2225640 1 -- 172.21.15.133:0/3685365068 shutdown_connections 2024-09-11T23:35:43.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.789+0000 7f55a2225640 1 -- 172.21.15.133:0/3685365068 wait complete. 2024-09-11T23:35:43.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f55a2225640 1 Processor -- start 2024-09-11T23:35:43.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f55a2225640 1 -- start start 2024-09-11T23:35:43.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f55a2225640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c101de0 0x7f559c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:43.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f55a2225640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c102790 0x7f559c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:43.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f55a2225640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c106f10 0x7f559c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:43.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f55a2225640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f559c111f70 con 0x7f559c101de0 2024-09-11T23:35:43.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f55a2225640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f559c111df0 con 0x7f559c106f10 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f55a2225640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1120f0 con 0x7f559c102790 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f559b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c101de0 0x7f559c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f559b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c101de0 0x7f559c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60190/0 (socket says 172.21.15.133:60190) 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f559b7fe640 1 -- 172.21.15.133:0/597818018 learned_addr learned my addr 172.21.15.133:0/597818018 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f559bfff640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c106f10 0x7f559c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f559affd640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c102790 0x7f559c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f559b7fe640 1 -- 172.21.15.133:0/597818018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c102790 msgr2=0x7f559c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f559b7fe640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c102790 0x7f559c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f559b7fe640 1 -- 172.21.15.133:0/597818018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c106f10 msgr2=0x7f559c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f559b7fe640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c106f10 0x7f559c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f559b7fe640 1 -- 172.21.15.133:0/597818018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c19d340 con 0x7f559c101de0 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.790+0000 7f559bfff640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c106f10 0x7f559c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:43.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.791+0000 7f559b7fe640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c101de0 0x7f559c198390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f558c00cce0 tx=0x7f558c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.791+0000 7f5598ff9640 1 -- 172.21.15.133:0/597818018 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c004b20 con 0x7f559c101de0 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.791+0000 7f55a2225640 1 -- 172.21.15.133:0/597818018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c19d630 con 0x7f559c101de0 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.791+0000 7f5598ff9640 1 -- 172.21.15.133:0/597818018 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558c004cc0 con 0x7f559c101de0 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.791+0000 7f5598ff9640 1 -- 172.21.15.133:0/597818018 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c00f7d0 con 0x7f559c101de0 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.791+0000 7f55a2225640 1 -- 172.21.15.133:0/597818018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c1a4f10 con 0x7f559c101de0 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.792+0000 7f55a2225640 1 -- 172.21.15.133:0/597818018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559c1021c0 con 0x7f559c101de0 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.793+0000 7f5598ff9640 1 -- 172.21.15.133:0/597818018 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f558c00f970 con 0x7f559c101de0 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.794+0000 7f5598ff9640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f55840a2c90 0x7f55840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.794+0000 7f559affd640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f55840a2c90 0x7f55840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.794+0000 7f5598ff9640 1 -- 172.21.15.133:0/597818018 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f558c0cf4e0 con 0x7f559c101de0 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.794+0000 7f559affd640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f55840a2c90 0x7f55840a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f559c0fdce0 tx=0x7f5590005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:43.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.798+0000 7f5598ff9640 1 -- 172.21.15.133:0/597818018 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f558c014030 con 0x7f559c101de0 2024-09-11T23:35:43.972 INFO:teuthology.orchestra.run.smithi133.stdout:313532612631 2024-09-11T23:35:43.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.972+0000 7f55a2225640 1 -- 172.21.15.133:0/597818018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f559c1996f0 con 0x7f559c101de0 2024-09-11T23:35:43.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.972+0000 7f5598ff9640 1 -- 172.21.15.133:0/597818018 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f558c09c3e0 con 0x7f559c101de0 2024-09-11T23:35:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.974+0000 7f55a2225640 1 -- 172.21.15.133:0/597818018 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f55840a2c90 msgr2=0x7f55840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.974+0000 7f55a2225640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f55840a2c90 0x7f55840a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f559c0fdce0 tx=0x7f5590005be0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.974+0000 7f55a2225640 1 -- 172.21.15.133:0/597818018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c101de0 msgr2=0x7f559c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.974+0000 7f55a2225640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c101de0 0x7f559c198390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f558c00cce0 tx=0x7f558c00a620 comp rx=0 tx=0).stop 2024-09-11T23:35:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.974+0000 7f55a2225640 1 -- 172.21.15.133:0/597818018 shutdown_connections 2024-09-11T23:35:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.974+0000 7f55a2225640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f55840a2c90 0x7f55840a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.974+0000 7f55a2225640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c106f10 0x7f559c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.974+0000 7f55a2225640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c102790 0x7f559c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.974+0000 7f55a2225640 1 --2- 172.21.15.133:0/597818018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c101de0 0x7f559c198390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:43.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.974+0000 7f55a2225640 1 -- 172.21.15.133:0/597818018 >> 172.21.15.133:0/597818018 conn(0x7f559c0fd8c0 msgr2=0x7f559c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:43.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.974+0000 7f55a2225640 1 -- 172.21.15.133:0/597818018 shutdown_connections 2024-09-11T23:35:43.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:43.975+0000 7f55a2225640 1 -- 172.21.15.133:0/597818018 wait complete. 2024-09-11T23:35:44.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.651+0000 7f97ce865640 1 -- 172.21.15.133:0/654243223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c8101de0 msgr2=0x7f97c81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.651+0000 7f97ce865640 1 --2- 172.21.15.133:0/654243223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c8101de0 0x7f97c81021c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009900 tx=0x7f97b802a410 comp rx=0 tx=0).stop 2024-09-11T23:35:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.651+0000 7f97ce865640 1 -- 172.21.15.133:0/654243223 shutdown_connections 2024-09-11T23:35:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.651+0000 7f97ce865640 1 --2- 172.21.15.133:0/654243223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c8106f10 0x7f97c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.651+0000 7f97ce865640 1 --2- 172.21.15.133:0/654243223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8102790 0x7f97c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.651+0000 7f97ce865640 1 --2- 172.21.15.133:0/654243223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c8101de0 0x7f97c81021c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.651+0000 7f97ce865640 1 -- 172.21.15.133:0/654243223 >> 172.21.15.133:0/654243223 conn(0x7f97c80fd8c0 msgr2=0x7f97c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.651+0000 7f97ce865640 1 -- 172.21.15.133:0/654243223 shutdown_connections 2024-09-11T23:35:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.651+0000 7f97ce865640 1 -- 172.21.15.133:0/654243223 wait complete. 2024-09-11T23:35:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.652+0000 7f97ce865640 1 Processor -- start 2024-09-11T23:35:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.652+0000 7f97ce865640 1 -- start start 2024-09-11T23:35:44.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.652+0000 7f97ce865640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8101de0 0x7f97c80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.652+0000 7f97ce865640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c8102790 0x7f97c8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.652+0000 7f97ce865640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c8106f10 0x7f97c8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.652+0000 7f97ce865640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8111f90 con 0x7f97c8106f10 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.652+0000 7f97ce865640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8111e10 con 0x7f97c8101de0 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.652+0000 7f97ce865640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8112110 con 0x7f97c8102790 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97cce02640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c8106f10 0x7f97c8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97cce02640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c8106f10 0x7f97c8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60210/0 (socket says 172.21.15.133:60210) 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97cce02640 1 -- 172.21.15.133:0/3191080784 learned_addr learned my addr 172.21.15.133:0/3191080784 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97c7fff640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8101de0 0x7f97c80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97c77fe640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c8102790 0x7f97c8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97cce02640 1 -- 172.21.15.133:0/3191080784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c8102790 msgr2=0x7f97c8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97cce02640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c8102790 0x7f97c8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97cce02640 1 -- 172.21.15.133:0/3191080784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8101de0 msgr2=0x7f97c80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97cce02640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8101de0 0x7f97c80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97cce02640 1 -- 172.21.15.133:0/3191080784 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c81a6970 con 0x7f97c8106f10 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97cce02640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c8106f10 0x7f97c8074f70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f97c000b970 tx=0x7f97c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97c57fa640 1 -- 172.21.15.133:0/3191080784 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0019070 con 0x7f97c8106f10 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97c57fa640 1 -- 172.21.15.133:0/3191080784 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c000cb40 con 0x7f97c8106f10 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97ce865640 1 -- 172.21.15.133:0/3191080784 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c81a6c60 con 0x7f97c8106f10 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97c57fa640 1 -- 172.21.15.133:0/3191080784 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0007650 con 0x7f97c8106f10 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.653+0000 7f97ce865640 1 -- 172.21.15.133:0/3191080784 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c81a71a0 con 0x7f97c8106f10 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.654+0000 7f97ce865640 1 -- 172.21.15.133:0/3191080784 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c81021c0 con 0x7f97c8106f10 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.656+0000 7f97c57fa640 1 -- 172.21.15.133:0/3191080784 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f97c001c070 con 0x7f97c8106f10 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.657+0000 7f97c57fa640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f97b00a2c90 0x7f97b00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.657+0000 7f97c7fff640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f97b00a2c90 0x7f97b00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.657+0000 7f97c57fa640 1 -- 172.21.15.133:0/3191080784 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f97c00ce510 con 0x7f97c8106f10 2024-09-11T23:35:44.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.657+0000 7f97c7fff640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f97b00a2c90 0x7f97b00a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009230 tx=0x7f97b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:44.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.660+0000 7f97c57fa640 1 -- 172.21.15.133:0/3191080784 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97c009aed0 con 0x7f97c8106f10 2024-09-11T23:35:44.689 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:44.691 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:44.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.869+0000 7f97ce865640 1 -- 172.21.15.133:0/3191080784 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f97c802d7c0 con 0x7f97c8106f10 2024-09-11T23:35:44.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.869+0000 7f97c57fa640 1 -- 172.21.15.133:0/3191080784 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f97c009f1d0 con 0x7f97c8106f10 2024-09-11T23:35:44.869 INFO:teuthology.orchestra.run.smithi133.stdout:98784247869 2024-09-11T23:35:44.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 -- 172.21.15.133:0/3191080784 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f97b00a2c90 msgr2=0x7f97b00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:44.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f97b00a2c90 0x7f97b00a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009230 tx=0x7f97b80058d0 comp rx=0 tx=0).stop 2024-09-11T23:35:44.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 -- 172.21.15.133:0/3191080784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c8106f10 msgr2=0x7f97c8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:44.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c8106f10 0x7f97c8074f70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f97c000b970 tx=0x7f97c000be30 comp rx=0 tx=0).stop 2024-09-11T23:35:44.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 -- 172.21.15.133:0/3191080784 shutdown_connections 2024-09-11T23:35:44.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f97b00a2c90 0x7f97b00a5150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:44.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c8106f10 0x7f97c8074f70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:44.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c8102790 0x7f97c8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:44.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 --2- 172.21.15.133:0/3191080784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8101de0 0x7f97c80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:44.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 -- 172.21.15.133:0/3191080784 >> 172.21.15.133:0/3191080784 conn(0x7f97c80fd8c0 msgr2=0x7f97c80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:44.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 -- 172.21.15.133:0/3191080784 shutdown_connections 2024-09-11T23:35:44.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:44.871+0000 7f97ce865640 1 -- 172.21.15.133:0/3191080784 wait complete. 2024-09-11T23:35:45.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:44 smithi133 ceph-mon[31430]: pgmap v299: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:35:45.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:44 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/597818018' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:35:45.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:44 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3191080784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:35:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:44 smithi178 ceph-mon[35882]: pgmap v299: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:35:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:44 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/597818018' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:35:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:44 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3191080784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:35:45.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:44 smithi138 ceph-mon[35142]: pgmap v299: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:35:45.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:44 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/597818018' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:35:45.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:44 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3191080784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:35:45.777 INFO:tasks.cephadm.ceph_manager.ceph:need seq 98784247867 got 98784247869 for osd.3 2024-09-11T23:35:45.778 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:45.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.948+0000 7fe90549e640 1 -- 172.21.15.133:0/415661601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe900106a80 msgr2=0x7fe900106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:45.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.948+0000 7fe90549e640 1 --2- 172.21.15.133:0/415661601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe900106a80 0x7fe900106e60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009930 tx=0x7fe8f002a420 comp rx=0 tx=0).stop 2024-09-11T23:35:45.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.948+0000 7fe90549e640 1 -- 172.21.15.133:0/415661601 shutdown_connections 2024-09-11T23:35:45.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.948+0000 7fe90549e640 1 --2- 172.21.15.133:0/415661601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe900101420 0x7fe90010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:45.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.948+0000 7fe90549e640 1 --2- 172.21.15.133:0/415661601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe900100a80 0x7fe900100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:45.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.948+0000 7fe90549e640 1 --2- 172.21.15.133:0/415661601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe900106a80 0x7fe900106e60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.948+0000 7fe90549e640 1 -- 172.21.15.133:0/415661601 >> 172.21.15.133:0/415661601 conn(0x7fe9000fc7a0 msgr2=0x7fe9000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.948+0000 7fe90549e640 1 -- 172.21.15.133:0/415661601 shutdown_connections 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.948+0000 7fe90549e640 1 -- 172.21.15.133:0/415661601 wait complete. 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe90549e640 1 Processor -- start 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe90549e640 1 -- start start 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe90549e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe900100a80 0x7fe9001992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe90549e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe900101420 0x7fe9001997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe90549e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe900106a80 0x7fe9001a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe90549e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe900110ea0 con 0x7fe900100a80 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe90549e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe900110d20 con 0x7fe900101420 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe90549e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe900111020 con 0x7fe900106a80 2024-09-11T23:35:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe8feffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe900100a80 0x7fe9001992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe8feffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe900100a80 0x7fe9001992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60228/0 (socket says 172.21.15.133:60228) 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe8feffd640 1 -- 172.21.15.133:0/1414055899 learned_addr learned my addr 172.21.15.133:0/1414055899 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe8ff7fe640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe900106a80 0x7fe9001a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe8f7fff640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe900101420 0x7fe9001997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe8feffd640 1 -- 172.21.15.133:0/1414055899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe900106a80 msgr2=0x7fe9001a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe8feffd640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe900106a80 0x7fe9001a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe8feffd640 1 -- 172.21.15.133:0/1414055899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe900101420 msgr2=0x7fe9001997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe8feffd640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe900101420 0x7fe9001997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe8feffd640 1 -- 172.21.15.133:0/1414055899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9001a4870 con 0x7fe900100a80 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.949+0000 7fe8f7fff640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe900101420 0x7fe9001997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.950+0000 7fe8feffd640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe900100a80 0x7fe9001992b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009a50 tx=0x7fe8f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.950+0000 7fe8fcff9640 1 -- 172.21.15.133:0/1414055899 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f003f030 con 0x7fe900100a80 2024-09-11T23:35:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.950+0000 7fe8fcff9640 1 -- 172.21.15.133:0/1414055899 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f0003ce0 con 0x7fe900100a80 2024-09-11T23:35:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.950+0000 7fe90549e640 1 -- 172.21.15.133:0/1414055899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9001a4a40 con 0x7fe900100a80 2024-09-11T23:35:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.950+0000 7fe8fcff9640 1 -- 172.21.15.133:0/1414055899 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0035070 con 0x7fe900100a80 2024-09-11T23:35:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.950+0000 7fe90549e640 1 -- 172.21.15.133:0/1414055899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9001a4ef0 con 0x7fe900100a80 2024-09-11T23:35:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.951+0000 7fe90549e640 1 -- 172.21.15.133:0/1414055899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe900100ee0 con 0x7fe900100a80 2024-09-11T23:35:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.952+0000 7fe8fcff9640 1 -- 172.21.15.133:0/1414055899 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe8f0042030 con 0x7fe900100a80 2024-09-11T23:35:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.952+0000 7fe8fcff9640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8ec0a2c90 0x7fe8ec0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.953+0000 7fe8f7fff640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8ec0a2c90 0x7fe8ec0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.953+0000 7fe8fcff9640 1 -- 172.21.15.133:0/1414055899 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fe8f00ef630 con 0x7fe900100a80 2024-09-11T23:35:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.953+0000 7fe8f7fff640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8ec0a2c90 0x7fe8ec0a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe90006b2e0 tx=0x7fe8e800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:45.957+0000 7fe8fcff9640 1 -- 172.21.15.133:0/1414055899 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8f00bc530 con 0x7fe900100a80 2024-09-11T23:35:46.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.129+0000 7fe90549e640 1 -- 172.21.15.133:0/1414055899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe9001a50c0 con 0x7fe900100a80 2024-09-11T23:35:46.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.129+0000 7fe8fcff9640 1 -- 172.21.15.133:0/1414055899 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe8f00bbe30 con 0x7fe900100a80 2024-09-11T23:35:46.129 INFO:teuthology.orchestra.run.smithi133.stdout:73014444100 2024-09-11T23:35:46.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 -- 172.21.15.133:0/1414055899 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8ec0a2c90 msgr2=0x7fe8ec0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:46.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8ec0a2c90 0x7fe8ec0a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe90006b2e0 tx=0x7fe8e800a5c0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 -- 172.21.15.133:0/1414055899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe900100a80 msgr2=0x7fe9001992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:46.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe900100a80 0x7fe9001992b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009a50 tx=0x7fe8f0004270 comp rx=0 tx=0).stop 2024-09-11T23:35:46.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 -- 172.21.15.133:0/1414055899 shutdown_connections 2024-09-11T23:35:46.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8ec0a2c90 0x7fe8ec0a5150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe900106a80 0x7fe9001a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe900101420 0x7fe9001997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 --2- 172.21.15.133:0/1414055899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe900100a80 0x7fe9001992b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 -- 172.21.15.133:0/1414055899 >> 172.21.15.133:0/1414055899 conn(0x7fe9000fc7a0 msgr2=0x7fe9000fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:46.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 -- 172.21.15.133:0/1414055899 shutdown_connections 2024-09-11T23:35:46.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.131+0000 7fe90549e640 1 -- 172.21.15.133:0/1414055899 wait complete. 2024-09-11T23:35:46.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.664+0000 7fe75578e640 1 -- 172.21.15.133:0/1300755252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe750101420 msgr2=0x7fe75010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.664+0000 7fe75578e640 1 --2- 172.21.15.133:0/1300755252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe750101420 0x7fe75010d950 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe740009930 tx=0x7fe74002a420 comp rx=0 tx=0).stop 2024-09-11T23:35:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.664+0000 7fe75578e640 1 -- 172.21.15.133:0/1300755252 shutdown_connections 2024-09-11T23:35:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.664+0000 7fe75578e640 1 --2- 172.21.15.133:0/1300755252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe750101420 0x7fe75010d950 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.664+0000 7fe75578e640 1 --2- 172.21.15.133:0/1300755252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750100a80 0x7fe750100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.664+0000 7fe75578e640 1 --2- 172.21.15.133:0/1300755252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe750106a80 0x7fe750106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.664+0000 7fe75578e640 1 -- 172.21.15.133:0/1300755252 >> 172.21.15.133:0/1300755252 conn(0x7fe7500fc7a0 msgr2=0x7fe7500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.664+0000 7fe75578e640 1 -- 172.21.15.133:0/1300755252 shutdown_connections 2024-09-11T23:35:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.664+0000 7fe75578e640 1 -- 172.21.15.133:0/1300755252 wait complete. 2024-09-11T23:35:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.664+0000 7fe75578e640 1 Processor -- start 2024-09-11T23:35:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe75578e640 1 -- start start 2024-09-11T23:35:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe75578e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe750100a80 0x7fe750197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe75578e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe750101420 0x7fe750197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe75578e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750106a80 0x7fe75019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe75578e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe750110e80 con 0x7fe750100a80 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe75578e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe750110d00 con 0x7fe750101420 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe75578e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe750111000 con 0x7fe750106a80 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe74effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe750100a80 0x7fe750197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe74effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe750100a80 0x7fe750197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60242/0 (socket says 172.21.15.133:60242) 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe74effd640 1 -- 172.21.15.133:0/4284454044 learned_addr learned my addr 172.21.15.133:0/4284454044 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe74f7fe640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750106a80 0x7fe75019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe74e7fc640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe750101420 0x7fe750197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe74effd640 1 -- 172.21.15.133:0/4284454044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750106a80 msgr2=0x7fe75019bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe74effd640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750106a80 0x7fe75019bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe74effd640 1 -- 172.21.15.133:0/4284454044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe750101420 msgr2=0x7fe750197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe74effd640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe750101420 0x7fe750197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe74effd640 1 -- 172.21.15.133:0/4284454044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75019c2d0 con 0x7fe750100a80 2024-09-11T23:35:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.665+0000 7fe74effd640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe750100a80 0x7fe750197320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe74800db80 tx=0x7fe74800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.666+0000 7fe747fff640 1 -- 172.21.15.133:0/4284454044 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748017070 con 0x7fe750100a80 2024-09-11T23:35:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.666+0000 7fe747fff640 1 -- 172.21.15.133:0/4284454044 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe748003b70 con 0x7fe750100a80 2024-09-11T23:35:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.666+0000 7fe75578e640 1 -- 172.21.15.133:0/4284454044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75019c5c0 con 0x7fe750100a80 2024-09-11T23:35:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.666+0000 7fe747fff640 1 -- 172.21.15.133:0/4284454044 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480106c0 con 0x7fe750100a80 2024-09-11T23:35:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.666+0000 7fe75578e640 1 -- 172.21.15.133:0/4284454044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7501a3ea0 con 0x7fe750100a80 2024-09-11T23:35:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.667+0000 7fe75578e640 1 -- 172.21.15.133:0/4284454044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe750100ee0 con 0x7fe750100a80 2024-09-11T23:35:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.668+0000 7fe747fff640 1 -- 172.21.15.133:0/4284454044 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe748004020 con 0x7fe750100a80 2024-09-11T23:35:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.668+0000 7fe747fff640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe73c0a2c10 0x7fe73c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.668+0000 7fe74e7fc640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe73c0a2c10 0x7fe73c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.668+0000 7fe747fff640 1 -- 172.21.15.133:0/4284454044 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fe7480ceef0 con 0x7fe750100a80 2024-09-11T23:35:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.669+0000 7fe74e7fc640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe73c0a2c10 0x7fe73c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe738005fd0 tx=0x7fe738005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:46.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.673+0000 7fe747fff640 1 -- 172.21.15.133:0/4284454044 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe748020070 con 0x7fe750100a80 2024-09-11T23:35:46.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.844+0000 7fe75578e640 1 -- 172.21.15.133:0/4284454044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe7501a4070 con 0x7fe750100a80 2024-09-11T23:35:46.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.844+0000 7fe747fff640 1 -- 172.21.15.133:0/4284454044 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe74809bdf0 con 0x7fe750100a80 2024-09-11T23:35:46.844 INFO:teuthology.orchestra.run.smithi133.stdout:128849018938 2024-09-11T23:35:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.846+0000 7fe75578e640 1 -- 172.21.15.133:0/4284454044 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe73c0a2c10 msgr2=0x7fe73c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.846+0000 7fe75578e640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe73c0a2c10 0x7fe73c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe738005fd0 tx=0x7fe738005850 comp rx=0 tx=0).stop 2024-09-11T23:35:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.846+0000 7fe75578e640 1 -- 172.21.15.133:0/4284454044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe750100a80 msgr2=0x7fe750197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.846+0000 7fe75578e640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe750100a80 0x7fe750197320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe74800db80 tx=0x7fe74800b630 comp rx=0 tx=0).stop 2024-09-11T23:35:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.846+0000 7fe75578e640 1 -- 172.21.15.133:0/4284454044 shutdown_connections 2024-09-11T23:35:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.846+0000 7fe75578e640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe73c0a2c10 0x7fe73c0a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.846+0000 7fe75578e640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750106a80 0x7fe75019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.846+0000 7fe75578e640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe750101420 0x7fe750197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.846+0000 7fe75578e640 1 --2- 172.21.15.133:0/4284454044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe750100a80 0x7fe750197320 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.846+0000 7fe75578e640 1 -- 172.21.15.133:0/4284454044 >> 172.21.15.133:0/4284454044 conn(0x7fe7500fc7a0 msgr2=0x7fe7500fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.847+0000 7fe75578e640 1 -- 172.21.15.133:0/4284454044 shutdown_connections 2024-09-11T23:35:46.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:46.847+0000 7fe75578e640 1 -- 172.21.15.133:0/4284454044 wait complete. 2024-09-11T23:35:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:46 smithi133 ceph-mon[31430]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:35:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:46 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1414055899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:35:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:46 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/4284454044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:35:47.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:46 smithi178 ceph-mon[35882]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:35:47.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:46 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1414055899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:35:47.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:46 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/4284454044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:35:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.381+0000 7fbfb9272640 1 -- 172.21.15.133:0/4086270578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb4106fc0 msgr2=0x7fbfb410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.381+0000 7fbfb9272640 1 --2- 172.21.15.133:0/4086270578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb4106fc0 0x7fbfb410eab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009cd0 tx=0x7fbfac02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:35:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.381+0000 7fbfb9272640 1 -- 172.21.15.133:0/4086270578 shutdown_connections 2024-09-11T23:35:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.381+0000 7fbfb9272640 1 --2- 172.21.15.133:0/4086270578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb4106fc0 0x7fbfb410eab0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.381+0000 7fbfb9272640 1 --2- 172.21.15.133:0/4086270578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb41007f0 0x7fbfb4106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.381+0000 7fbfb9272640 1 --2- 172.21.15.133:0/4086270578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb40ffe40 0x7fbfb4100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.381+0000 7fbfb9272640 1 -- 172.21.15.133:0/4086270578 >> 172.21.15.133:0/4086270578 conn(0x7fbfb40fba90 msgr2=0x7fbfb40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.381+0000 7fbfb9272640 1 -- 172.21.15.133:0/4086270578 shutdown_connections 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.381+0000 7fbfb9272640 1 -- 172.21.15.133:0/4086270578 wait complete. 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.381+0000 7fbfb9272640 1 Processor -- start 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb9272640 1 -- start start 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb9272640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40ffe40 0x7fbfb4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb9272640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb41007f0 0x7fbfb41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb9272640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb4106fc0 0x7fbfb419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb9272640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4111f60 con 0x7fbfb40ffe40 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb9272640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4111de0 con 0x7fbfb41007f0 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb9272640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41120e0 con 0x7fbfb4106fc0 2024-09-11T23:35:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40ffe40 0x7fbfb4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40ffe40 0x7fbfb4198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60250/0 (socket says 172.21.15.133:60250) 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb2ffd640 1 -- 172.21.15.133:0/760458566 learned_addr learned my addr 172.21.15.133:0/760458566 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb27fc640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb41007f0 0x7fbfb41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb37fe640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb4106fc0 0x7fbfb419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb2ffd640 1 -- 172.21.15.133:0/760458566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb4106fc0 msgr2=0x7fbfb419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb2ffd640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb4106fc0 0x7fbfb419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb2ffd640 1 -- 172.21.15.133:0/760458566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb41007f0 msgr2=0x7fbfb41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb2ffd640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb41007f0 0x7fbfb41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb2ffd640 1 -- 172.21.15.133:0/760458566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb419d330 con 0x7fbfb40ffe40 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.382+0000 7fbfb2ffd640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40ffe40 0x7fbfb4198380 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400d950 tx=0x7fbfa400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.383+0000 7fbf9bfff640 1 -- 172.21.15.133:0/760458566 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4017070 con 0x7fbfb40ffe40 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.383+0000 7fbf9bfff640 1 -- 172.21.15.133:0/760458566 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa400de80 con 0x7fbfb40ffe40 2024-09-11T23:35:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.383+0000 7fbfb9272640 1 -- 172.21.15.133:0/760458566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb419d620 con 0x7fbfb40ffe40 2024-09-11T23:35:47.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.383+0000 7fbf9bfff640 1 -- 172.21.15.133:0/760458566 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa400f620 con 0x7fbfb40ffe40 2024-09-11T23:35:47.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.383+0000 7fbfb9272640 1 -- 172.21.15.133:0/760458566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb41a4f00 con 0x7fbfb40ffe40 2024-09-11T23:35:47.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.384+0000 7fbfb9272640 1 -- 172.21.15.133:0/760458566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb4100220 con 0x7fbfb40ffe40 2024-09-11T23:35:47.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.385+0000 7fbf9bfff640 1 -- 172.21.15.133:0/760458566 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbfa4009c30 con 0x7fbfb40ffe40 2024-09-11T23:35:47.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.385+0000 7fbf9bfff640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbf9c0a2c10 0x7fbf9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:47.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.385+0000 7fbfb27fc640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbf9c0a2c10 0x7fbf9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:47.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.386+0000 7fbf9bfff640 1 -- 172.21.15.133:0/760458566 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fbfa40ced40 con 0x7fbfb40ffe40 2024-09-11T23:35:47.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.386+0000 7fbfb27fc640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbf9c0a2c10 0x7fbf9c0a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40fbf20 tx=0x7fbfa800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:47.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.390+0000 7fbf9bfff640 1 -- 172.21.15.133:0/760458566 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbfa409bc40 con 0x7fbfb40ffe40 2024-09-11T23:35:47.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:46 smithi138 ceph-mon[35142]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:35:47.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:46 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1414055899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:35:47.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:46 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/4284454044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:35:47.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.562+0000 7fbfb9272640 1 -- 172.21.15.133:0/760458566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fbfb4109ae0 con 0x7fbfb40ffe40 2024-09-11T23:35:47.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.562+0000 7fbf9bfff640 1 -- 172.21.15.133:0/760458566 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fbfa409b540 con 0x7fbfb40ffe40 2024-09-11T23:35:47.562 INFO:teuthology.orchestra.run.smithi133.stdout:283467841564 2024-09-11T23:35:47.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 -- 172.21.15.133:0/760458566 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbf9c0a2c10 msgr2=0x7fbf9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:47.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbf9c0a2c10 0x7fbf9c0a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40fbf20 tx=0x7fbfa800a2d0 comp rx=0 tx=0).stop 2024-09-11T23:35:47.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 -- 172.21.15.133:0/760458566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40ffe40 msgr2=0x7fbfb4198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:47.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40ffe40 0x7fbfb4198380 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400d950 tx=0x7fbfa400dd50 comp rx=0 tx=0).stop 2024-09-11T23:35:47.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 -- 172.21.15.133:0/760458566 shutdown_connections 2024-09-11T23:35:47.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbf9c0a2c10 0x7fbf9c0a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:47.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb4106fc0 0x7fbfb419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:47.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb41007f0 0x7fbfb41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:47.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 --2- 172.21.15.133:0/760458566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40ffe40 0x7fbfb4198380 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:47.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 -- 172.21.15.133:0/760458566 >> 172.21.15.133:0/760458566 conn(0x7fbfb40fba90 msgr2=0x7fbfb40fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:47.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 -- 172.21.15.133:0/760458566 shutdown_connections 2024-09-11T23:35:47.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:47.565+0000 7fbfb9272640 1 -- 172.21.15.133:0/760458566 wait complete. 2024-09-11T23:35:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.076+0000 7fd31bb75640 1 -- 172.21.15.133:0/3699697263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314101420 msgr2=0x7fd31410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.076+0000 7fd31bb75640 1 --2- 172.21.15.133:0/3699697263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314101420 0x7fd31410d950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009cd0 tx=0x7fd30c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.076+0000 7fd31bb75640 1 -- 172.21.15.133:0/3699697263 shutdown_connections 2024-09-11T23:35:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.076+0000 7fd31bb75640 1 --2- 172.21.15.133:0/3699697263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314101420 0x7fd31410d950 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.076+0000 7fd31bb75640 1 --2- 172.21.15.133:0/3699697263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd314100a80 0x7fd314100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.076+0000 7fd31bb75640 1 --2- 172.21.15.133:0/3699697263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd314106a80 0x7fd314106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.076+0000 7fd31bb75640 1 -- 172.21.15.133:0/3699697263 >> 172.21.15.133:0/3699697263 conn(0x7fd3140fc7a0 msgr2=0x7fd3140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.076+0000 7fd31bb75640 1 -- 172.21.15.133:0/3699697263 shutdown_connections 2024-09-11T23:35:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.076+0000 7fd31bb75640 1 -- 172.21.15.133:0/3699697263 wait complete. 2024-09-11T23:35:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd31bb75640 1 Processor -- start 2024-09-11T23:35:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd31bb75640 1 -- start start 2024-09-11T23:35:48.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd31bb75640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314100a80 0x7fd3141985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:48.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd31bb75640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd314101420 0x7fd314198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:48.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd31bb75640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd314106a80 0x7fd31419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:48.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd31bb75640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd314112020 con 0x7fd314100a80 2024-09-11T23:35:48.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd31bb75640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd314111ea0 con 0x7fd314106a80 2024-09-11T23:35:48.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd31bb75640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141121a0 con 0x7fd314101420 2024-09-11T23:35:48.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd319911640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314100a80 0x7fd3141985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:48.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd319911640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314100a80 0x7fd3141985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60268/0 (socket says 172.21.15.133:60268) 2024-09-11T23:35:48.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd319911640 1 -- 172.21.15.133:0/2016067783 learned_addr learned my addr 172.21.15.133:0/2016067783 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:48.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd31a112640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd314106a80 0x7fd31419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd319110640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd314101420 0x7fd314198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd319911640 1 -- 172.21.15.133:0/2016067783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd314101420 msgr2=0x7fd314198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd319911640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd314101420 0x7fd314198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd319911640 1 -- 172.21.15.133:0/2016067783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd314106a80 msgr2=0x7fd31419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd319911640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd314106a80 0x7fd31419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd319911640 1 -- 172.21.15.133:0/2016067783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31419d550 con 0x7fd314100a80 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.077+0000 7fd31a112640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd314106a80 0x7fd31419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.078+0000 7fd319911640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314100a80 0x7fd3141985a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd30800b4a0 tx=0x7fd30800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.078+0000 7fd306ffd640 1 -- 172.21.15.133:0/2016067783 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308014070 con 0x7fd314100a80 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.078+0000 7fd306ffd640 1 -- 172.21.15.133:0/2016067783 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd308010af0 con 0x7fd314100a80 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.078+0000 7fd31bb75640 1 -- 172.21.15.133:0/2016067783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31419d840 con 0x7fd314100a80 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.078+0000 7fd306ffd640 1 -- 172.21.15.133:0/2016067783 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308017d10 con 0x7fd314100a80 2024-09-11T23:35:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.078+0000 7fd31bb75640 1 -- 172.21.15.133:0/2016067783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31410e310 con 0x7fd314100a80 2024-09-11T23:35:48.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.079+0000 7fd31bb75640 1 -- 172.21.15.133:0/2016067783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3141021c0 con 0x7fd314100a80 2024-09-11T23:35:48.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.080+0000 7fd306ffd640 1 -- 172.21.15.133:0/2016067783 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd308017070 con 0x7fd314100a80 2024-09-11T23:35:48.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.080+0000 7fd306ffd640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd3000a2c10 0x7fd3000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:48.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.081+0000 7fd319110640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd3000a2c10 0x7fd3000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:48.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.081+0000 7fd306ffd640 1 -- 172.21.15.133:0/2016067783 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fd3080ce850 con 0x7fd314100a80 2024-09-11T23:35:48.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.081+0000 7fd319110640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd3000a2c10 0x7fd3000a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd3140fedd0 tx=0x7fd310008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:48.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.084+0000 7fd306ffd640 1 -- 172.21.15.133:0/2016067783 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd30809b860 con 0x7fd314100a80 2024-09-11T23:35:48.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.256+0000 7fd31bb75640 1 -- 172.21.15.133:0/2016067783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd314109010 con 0x7fd314100a80 2024-09-11T23:35:48.256 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:47 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/760458566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:35:48.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.257+0000 7fd306ffd640 1 -- 172.21.15.133:0/2016067783 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd30809b160 con 0x7fd314100a80 2024-09-11T23:35:48.257 INFO:teuthology.orchestra.run.smithi133.stdout:261993005090 2024-09-11T23:35:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 -- 172.21.15.133:0/2016067783 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd3000a2c10 msgr2=0x7fd3000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd3000a2c10 0x7fd3000a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd3140fedd0 tx=0x7fd310008040 comp rx=0 tx=0).stop 2024-09-11T23:35:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 -- 172.21.15.133:0/2016067783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314100a80 msgr2=0x7fd3141985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314100a80 0x7fd3141985a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd30800b4a0 tx=0x7fd30800b960 comp rx=0 tx=0).stop 2024-09-11T23:35:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 -- 172.21.15.133:0/2016067783 shutdown_connections 2024-09-11T23:35:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd3000a2c10 0x7fd3000a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd314106a80 0x7fd31419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd314101420 0x7fd314198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 --2- 172.21.15.133:0/2016067783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314100a80 0x7fd3141985a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 -- 172.21.15.133:0/2016067783 >> 172.21.15.133:0/2016067783 conn(0x7fd3140fc7a0 msgr2=0x7fd3140fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 -- 172.21.15.133:0/2016067783 shutdown_connections 2024-09-11T23:35:48.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.259+0000 7fd31bb75640 1 -- 172.21.15.133:0/2016067783 wait complete. 2024-09-11T23:35:48.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:47 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/760458566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:35:48.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.267+0000 7f0940a31640 1 -- 172.21.15.133:0/1066338679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c103760 msgr2=0x7f093c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:48.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.267+0000 7f0940a31640 1 --2- 172.21.15.133:0/1066338679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c103760 0x7f093c107ba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f092c009930 tx=0x7f092c02a420 comp rx=0 tx=0).stop 2024-09-11T23:35:48.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.268+0000 7f0940a31640 1 -- 172.21.15.133:0/1066338679 shutdown_connections 2024-09-11T23:35:48.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.268+0000 7f0940a31640 1 --2- 172.21.15.133:0/1066338679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c1082e0 0x7f093c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.268+0000 7f0940a31640 1 --2- 172.21.15.133:0/1066338679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c103760 0x7f093c107ba0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.268+0000 7f0940a31640 1 --2- 172.21.15.133:0/1066338679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093c102db0 0x7f093c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.268+0000 7f0940a31640 1 -- 172.21.15.133:0/1066338679 >> 172.21.15.133:0/1066338679 conn(0x7f093c0fc7d0 msgr2=0x7f093c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:48.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.268+0000 7f0940a31640 1 -- 172.21.15.133:0/1066338679 shutdown_connections 2024-09-11T23:35:48.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.268+0000 7f0940a31640 1 -- 172.21.15.133:0/1066338679 wait complete. 2024-09-11T23:35:48.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.268+0000 7f0940a31640 1 Processor -- start 2024-09-11T23:35:48.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.268+0000 7f0940a31640 1 -- start start 2024-09-11T23:35:48.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0940a31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c102db0 0x7f093c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:48.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0940a31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c103760 0x7f093c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:48.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0940a31640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093c1082e0 0x7f093c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:48.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0940a31640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f093c112070 con 0x7f093c103760 2024-09-11T23:35:48.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0940a31640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f093c111ef0 con 0x7f093c102db0 2024-09-11T23:35:48.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0940a31640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f093c1121f0 con 0x7f093c1082e0 2024-09-11T23:35:48.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0939d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c103760 0x7f093c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:48.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0939d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c103760 0x7f093c198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60280/0 (socket says 172.21.15.133:60280) 2024-09-11T23:35:48.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0939d9b640 1 -- 172.21.15.133:0/1821077196 learned_addr learned my addr 172.21.15.133:0/1821077196 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:48.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0939d9b640 1 -- 172.21.15.133:0/1821077196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093c1082e0 msgr2=0x7f093c19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0939d9b640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093c1082e0 0x7f093c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0939d9b640 1 -- 172.21.15.133:0/1821077196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c102db0 msgr2=0x7f093c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0939d9b640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c102db0 0x7f093c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0939d9b640 1 -- 172.21.15.133:0/1821077196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c19d590 con 0x7f093c103760 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0939d9b640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c103760 0x7f093c198b20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f092c005940 tx=0x7f092c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f092b7fe640 1 -- 172.21.15.133:0/1821077196 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c03f070 con 0x7f093c103760 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f0940a31640 1 -- 172.21.15.133:0/1821077196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093c19d820 con 0x7f093c103760 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f092b7fe640 1 -- 172.21.15.133:0/1821077196 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f092c031c30 con 0x7f093c103760 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.269+0000 7f092b7fe640 1 -- 172.21.15.133:0/1821077196 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c035030 con 0x7f093c103760 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.270+0000 7f0940a31640 1 -- 172.21.15.133:0/1821077196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093c1a5160 con 0x7f093c103760 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.271+0000 7f0940a31640 1 -- 172.21.15.133:0/1821077196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f093c103190 con 0x7f093c103760 2024-09-11T23:35:48.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.272+0000 7f092b7fe640 1 -- 172.21.15.133:0/1821077196 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f092c042070 con 0x7f093c103760 2024-09-11T23:35:48.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.272+0000 7f092b7fe640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09240a2c10 0x7f09240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:48.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.272+0000 7f092b7fe640 1 -- 172.21.15.133:0/1821077196 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f092c0efa70 con 0x7f093c103760 2024-09-11T23:35:48.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.272+0000 7f093a59c640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09240a2c10 0x7f09240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:48.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.273+0000 7f093a59c640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09240a2c10 0x7f09240a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0930005fd0 tx=0x7f0930005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:48.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.277+0000 7f092b7fe640 1 -- 172.21.15.133:0/1821077196 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f092c031de0 con 0x7f093c103760 2024-09-11T23:35:48.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:47 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/760458566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:35:48.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.453+0000 7f0940a31640 1 -- 172.21.15.133:0/1821077196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f093c10a2c0 con 0x7f093c103760 2024-09-11T23:35:48.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.453+0000 7f092b7fe640 1 -- 172.21.15.133:0/1821077196 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f092c0bc870 con 0x7f093c103760 2024-09-11T23:35:48.453 INFO:teuthology.orchestra.run.smithi133.stdout:34359738447 2024-09-11T23:35:48.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 -- 172.21.15.133:0/1821077196 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09240a2c10 msgr2=0x7f09240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:48.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09240a2c10 0x7f09240a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0930005fd0 tx=0x7f0930005850 comp rx=0 tx=0).stop 2024-09-11T23:35:48.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 -- 172.21.15.133:0/1821077196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c103760 msgr2=0x7f093c198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:48.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c103760 0x7f093c198b20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f092c005940 tx=0x7f092c003c10 comp rx=0 tx=0).stop 2024-09-11T23:35:48.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 -- 172.21.15.133:0/1821077196 shutdown_connections 2024-09-11T23:35:48.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09240a2c10 0x7f09240a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093c1082e0 0x7f093c19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c103760 0x7f093c198b20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 --2- 172.21.15.133:0/1821077196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c102db0 0x7f093c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:48.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 -- 172.21.15.133:0/1821077196 >> 172.21.15.133:0/1821077196 conn(0x7f093c0fc7d0 msgr2=0x7f093c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:48.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 -- 172.21.15.133:0/1821077196 shutdown_connections 2024-09-11T23:35:48.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:48.455+0000 7f0940a31640 1 -- 172.21.15.133:0/1821077196 wait complete. 2024-09-11T23:35:49.067 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018936 got 128849018938 for osd.4 2024-09-11T23:35:49.067 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:49.272 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:48 smithi133 ceph-mon[31430]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T23:35:49.272 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:48 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2016067783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:35:49.273 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:48 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1821077196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:35:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:48 smithi178 ceph-mon[35882]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T23:35:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:48 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2016067783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:35:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:48 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1821077196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:35:49.349 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:48 smithi138 ceph-mon[35142]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T23:35:49.349 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:48 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2016067783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:35:49.349 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:48 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1821077196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:35:50.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:49 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:35:50.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:49 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1254885671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:35:50.179 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738447 for osd.0 2024-09-11T23:35:50.179 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:50.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:49 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:35:50.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:49 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1254885671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:35:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:49 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:35:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:49 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1254885671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:35:50.609 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005087 got 261993005090 for osd.9 2024-09-11T23:35:50.610 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:50 smithi178 ceph-mon[35882]: pgmap v302: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T23:35:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:50 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.297 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:50 smithi133 ceph-mon[31430]: pgmap v302: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T23:35:51.297 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.297 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.297 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.297 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:50 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.378+0000 7f64798e5640 1 -- 172.21.15.133:0/2107531243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64741027a0 msgr2=0x7f64741067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.378+0000 7f64798e5640 1 --2- 172.21.15.133:0/2107531243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64741027a0 0x7f64741067f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f645c009ca0 tx=0x7f645c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.378+0000 7f64798e5640 1 -- 172.21.15.133:0/2107531243 shutdown_connections 2024-09-11T23:35:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.378+0000 7f64798e5640 1 --2- 172.21.15.133:0/2107531243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6474106f20 0x7f647410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.378+0000 7f64798e5640 1 --2- 172.21.15.133:0/2107531243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64741027a0 0x7f64741067f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.378+0000 7f64798e5640 1 --2- 172.21.15.133:0/2107531243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6474101df0 0x7f64741021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.378+0000 7f64798e5640 1 -- 172.21.15.133:0/2107531243 >> 172.21.15.133:0/2107531243 conn(0x7f64740fd8b0 msgr2=0x7f64740ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.378+0000 7f64798e5640 1 -- 172.21.15.133:0/2107531243 shutdown_connections 2024-09-11T23:35:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.378+0000 7f64798e5640 1 -- 172.21.15.133:0/2107531243 wait complete. 2024-09-11T23:35:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.378+0000 7f64798e5640 1 Processor -- start 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64798e5640 1 -- start start 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64798e5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6474101df0 0x7f64740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64798e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64741027a0 0x7f6474074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64798e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6474106f20 0x7f6474074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64798e5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6474111f40 con 0x7f6474106f20 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64798e5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6474111dc0 con 0x7f64741027a0 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64798e5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64741120c0 con 0x7f6474101df0 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64737fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6474106f20 0x7f6474074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64737fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6474106f20 0x7f6474074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60312/0 (socket says 172.21.15.133:60312) 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64737fe640 1 -- 172.21.15.133:0/1318289989 learned_addr learned my addr 172.21.15.133:0/1318289989 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64727fc640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64741027a0 0x7f6474074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:51.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f6472ffd640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6474101df0 0x7f64740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64737fe640 1 -- 172.21.15.133:0/1318289989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6474101df0 msgr2=0x7f64740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64737fe640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6474101df0 0x7f64740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64737fe640 1 -- 172.21.15.133:0/1318289989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64741027a0 msgr2=0x7f6474074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64737fe640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64741027a0 0x7f6474074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64737fe640 1 -- 172.21.15.133:0/1318289989 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64741a6990 con 0x7f6474106f20 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64737fe640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6474106f20 0x7f6474074f70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f646c00e970 tx=0x7f646c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64788e3640 1 -- 172.21.15.133:0/1318289989 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c00ccc0 con 0x7f6474106f20 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.379+0000 7f64798e5640 1 -- 172.21.15.133:0/1318289989 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64741a6c80 con 0x7f6474106f20 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.380+0000 7f64788e3640 1 -- 172.21.15.133:0/1318289989 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f646c00ce60 con 0x7f6474106f20 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.380+0000 7f64798e5640 1 -- 172.21.15.133:0/1318289989 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64741a7190 con 0x7f6474106f20 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.380+0000 7f64788e3640 1 -- 172.21.15.133:0/1318289989 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c018630 con 0x7f6474106f20 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.380+0000 7f64798e5640 1 -- 172.21.15.133:0/1318289989 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64741021d0 con 0x7f6474106f20 2024-09-11T23:35:51.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.382+0000 7f64788e3640 1 -- 172.21.15.133:0/1318289989 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f646c0187d0 con 0x7f6474106f20 2024-09-11T23:35:51.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.382+0000 7f64788e3640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f64600a2c10 0x7f64600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:51.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.383+0000 7f6472ffd640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f64600a2c10 0x7f64600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:51.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.383+0000 7f64788e3640 1 -- 172.21.15.133:0/1318289989 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f646c0cfbf0 con 0x7f6474106f20 2024-09-11T23:35:51.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.383+0000 7f6472ffd640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f64600a2c10 0x7f64600a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6464009bd0 tx=0x7f6464005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:51.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.385+0000 7f64788e3640 1 -- 172.21.15.133:0/1318289989 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f646c09caf0 con 0x7f6474106f20 2024-09-11T23:35:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:50 smithi138 ceph-mon[35142]: pgmap v302: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T23:35:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:50 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:35:51.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.272+0000 7f9ef87c4640 1 -- 172.21.15.133:0/424841698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef0106a80 msgr2=0x7f9ef0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:51.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.272+0000 7f9ef87c4640 1 --2- 172.21.15.133:0/424841698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef0106a80 0x7f9ef0106e60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009930 tx=0x7f9eec02a420 comp rx=0 tx=0).stop 2024-09-11T23:35:51.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.272+0000 7f9ef87c4640 1 -- 172.21.15.133:0/424841698 shutdown_connections 2024-09-11T23:35:51.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.272+0000 7f9ef87c4640 1 --2- 172.21.15.133:0/424841698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0101420 0x7f9ef010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.272+0000 7f9ef87c4640 1 --2- 172.21.15.133:0/424841698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ef0100a80 0x7f9ef0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.272+0000 7f9ef87c4640 1 --2- 172.21.15.133:0/424841698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef0106a80 0x7f9ef0106e60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.272+0000 7f9ef87c4640 1 -- 172.21.15.133:0/424841698 >> 172.21.15.133:0/424841698 conn(0x7f9ef00fc7a0 msgr2=0x7f9ef00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.272+0000 7f9ef87c4640 1 -- 172.21.15.133:0/424841698 shutdown_connections 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.272+0000 7f9ef87c4640 1 -- 172.21.15.133:0/424841698 wait complete. 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef87c4640 1 Processor -- start 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef87c4640 1 -- start start 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef87c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef0100a80 0x7f9ef0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef87c4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0101420 0x7f9ef0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef87c4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ef0106a80 0x7f9ef019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef87c4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0110e80 con 0x7f9ef0100a80 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef87c4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0110d00 con 0x7f9ef0101420 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef87c4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0111000 con 0x7f9ef0106a80 2024-09-11T23:35:51.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef6560640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef0100a80 0x7f9ef0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:51.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef6560640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef0100a80 0x7f9ef0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60294/0 (socket says 172.21.15.133:60294) 2024-09-11T23:35:51.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef6560640 1 -- 172.21.15.133:0/1254885671 learned_addr learned my addr 172.21.15.133:0/1254885671 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:51.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef6d61640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ef0106a80 0x7f9ef019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:51.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.273+0000 7f9ef5d5f640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0101420 0x7f9ef0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:51.542 INFO:teuthology.orchestra.run.smithi133.stdout:197568495660 2024-09-11T23:35:51.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.274+0000 7f9ef6560640 1 -- 172.21.15.133:0/1254885671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ef0106a80 msgr2=0x7f9ef019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.274+0000 7f9ef6560640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ef0106a80 0x7f9ef019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.274+0000 7f9ef6560640 1 -- 172.21.15.133:0/1254885671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0101420 msgr2=0x7f9ef0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.274+0000 7f9ef6560640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0101420 0x7f9ef0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.274+0000 7f9ef6560640 1 -- 172.21.15.133:0/1254885671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef019d3c0 con 0x7f9ef0100a80 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.274+0000 7f9ef6560640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef0100a80 0x7f9ef0198410 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009230 tx=0x7f9eec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.274+0000 7f9ee37fe640 1 -- 172.21.15.133:0/1254885671 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec03f030 con 0x7f9ef0100a80 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.274+0000 7f9ee37fe640 1 -- 172.21.15.133:0/1254885671 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eec003ad0 con 0x7f9ef0100a80 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.274+0000 7f9ef87c4640 1 -- 172.21.15.133:0/1254885671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef019d650 con 0x7f9ef0100a80 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.274+0000 7f9ee37fe640 1 -- 172.21.15.133:0/1254885671 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec035070 con 0x7f9ef0100a80 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.274+0000 7f9ef87c4640 1 -- 172.21.15.133:0/1254885671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef01a4f90 con 0x7f9ef0100a80 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.275+0000 7f9ef87c4640 1 -- 172.21.15.133:0/1254885671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef01021c0 con 0x7f9ef0100a80 2024-09-11T23:35:51.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.277+0000 7f9ee37fe640 1 -- 172.21.15.133:0/1254885671 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9eec042030 con 0x7f9ef0100a80 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.277+0000 7f9ee37fe640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9edc0a2c10 0x7f9edc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.277+0000 7f9ef5d5f640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9edc0a2c10 0x7f9edc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.277+0000 7f9ee37fe640 1 -- 172.21.15.133:0/1254885671 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f9eec0ef950 con 0x7f9ef0100a80 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.278+0000 7f9ef5d5f640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9edc0a2c10 0x7f9edc0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0105180 tx=0x7f9ee400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.281+0000 7f9ee37fe640 1 -- 172.21.15.133:0/1254885671 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9eec0bc200 con 0x7f9ef0100a80 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.459+0000 7f9ef87c4640 1 -- 172.21.15.133:0/1254885671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9ef010a000 con 0x7f9ef0100a80 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.460+0000 7f9ee37fe640 1 -- 172.21.15.133:0/1254885671 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9eec0c0500 con 0x7f9ef0100a80 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 -- 172.21.15.133:0/1254885671 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9edc0a2c10 msgr2=0x7f9edc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9edc0a2c10 0x7f9edc0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0105180 tx=0x7f9ee400b390 comp rx=0 tx=0).stop 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 -- 172.21.15.133:0/1254885671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef0100a80 msgr2=0x7f9ef0198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef0100a80 0x7f9ef0198410 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009230 tx=0x7f9eec004270 comp rx=0 tx=0).stop 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 -- 172.21.15.133:0/1254885671 shutdown_connections 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f9edc0a2c10 0x7f9edc0a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ef0106a80 0x7f9ef019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0101420 0x7f9ef0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 --2- 172.21.15.133:0/1254885671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef0100a80 0x7f9ef0198410 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 -- 172.21.15.133:0/1254885671 >> 172.21.15.133:0/1254885671 conn(0x7f9ef00fc7a0 msgr2=0x7f9ef006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:51.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 -- 172.21.15.133:0/1254885671 shutdown_connections 2024-09-11T23:35:51.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:49.462+0000 7f9ef87c4640 1 -- 172.21.15.133:0/1254885671 wait complete. 2024-09-11T23:35:51.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.556+0000 7f64798e5640 1 -- 172.21.15.133:0/1318289989 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6474075a20 con 0x7f6474106f20 2024-09-11T23:35:51.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.557+0000 7f64788e3640 1 -- 172.21.15.133:0/1318289989 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6474075a20 con 0x7f6474106f20 2024-09-11T23:35:51.556 INFO:teuthology.orchestra.run.smithi133.stdout:176093659184 2024-09-11T23:35:51.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 -- 172.21.15.133:0/1318289989 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f64600a2c10 msgr2=0x7f64600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:51.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f64600a2c10 0x7f64600a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6464009bd0 tx=0x7f6464005f90 comp rx=0 tx=0).stop 2024-09-11T23:35:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 -- 172.21.15.133:0/1318289989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6474106f20 msgr2=0x7f6474074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6474106f20 0x7f6474074f70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f646c00e970 tx=0x7f646c00ed70 comp rx=0 tx=0).stop 2024-09-11T23:35:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 -- 172.21.15.133:0/1318289989 shutdown_connections 2024-09-11T23:35:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f64600a2c10 0x7f64600a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6474106f20 0x7f6474074f70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64741027a0 0x7f6474074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 --2- 172.21.15.133:0/1318289989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6474101df0 0x7f64740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 -- 172.21.15.133:0/1318289989 >> 172.21.15.133:0/1318289989 conn(0x7f64740fd8b0 msgr2=0x7f64740febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 -- 172.21.15.133:0/1318289989 shutdown_connections 2024-09-11T23:35:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:51.559+0000 7f64798e5640 1 -- 172.21.15.133:0/1318289989 wait complete. 2024-09-11T23:35:52.195 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:51 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1318289989' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:35:52.226 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495656 got 197568495660 for osd.7 2024-09-11T23:35:52.226 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:51 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1318289989' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:35:52.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:51 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1318289989' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:35:52.849 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:52.849 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:52.854 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:52.876 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659180 got 176093659184 for osd.6 2024-09-11T23:35:52.877 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:53.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:52 smithi133 ceph-mon[31430]: pgmap v303: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:35:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:52 smithi178 ceph-mon[35882]: pgmap v303: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:35:53.335 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444097 got 73014444100 for osd.2 2024-09-11T23:35:53.335 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:53.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:52 smithi138 ceph-mon[35142]: pgmap v303: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:35:54.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.979+0000 7f0748317640 1 -- 172.21.15.133:0/2682625215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0740101dd0 msgr2=0x7f07401021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:54.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.979+0000 7f0748317640 1 --2- 172.21.15.133:0/2682625215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0740101dd0 0x7f07401021b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f073c009ca0 tx=0x7f073c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.980+0000 7f0748317640 1 -- 172.21.15.133:0/2682625215 shutdown_connections 2024-09-11T23:35:54.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.980+0000 7f0748317640 1 --2- 172.21.15.133:0/2682625215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0740106f00 0x7f074010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.980+0000 7f0748317640 1 --2- 172.21.15.133:0/2682625215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0740102780 0x7f07401067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.980+0000 7f0748317640 1 --2- 172.21.15.133:0/2682625215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0740101dd0 0x7f07401021b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.980+0000 7f0748317640 1 -- 172.21.15.133:0/2682625215 >> 172.21.15.133:0/2682625215 conn(0x7f07400fd8b0 msgr2=0x7f07400ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.980+0000 7f0748317640 1 -- 172.21.15.133:0/2682625215 shutdown_connections 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.980+0000 7f0748317640 1 -- 172.21.15.133:0/2682625215 wait complete. 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.980+0000 7f0748317640 1 Processor -- start 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.980+0000 7f0748317640 1 -- start start 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.980+0000 7f0748317640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0740101dd0 0x7f07400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f0748317640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0740102780 0x7f0740074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f0748317640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0740106f00 0x7f0740074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f0748317640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0740111f60 con 0x7f0740106f00 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f0748317640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0740111de0 con 0x7f0740101dd0 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f0748317640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07401120e0 con 0x7f0740102780 2024-09-11T23:35:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07468b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0740106f00 0x7f0740074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07468b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0740106f00 0x7f0740074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48746/0 (socket says 172.21.15.133:48746) 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07468b4640 1 -- 172.21.15.133:0/2208372770 learned_addr learned my addr 172.21.15.133:0/2208372770 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07460b3640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0740101dd0 0x7f07400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07458b2640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0740102780 0x7f0740074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07468b4640 1 -- 172.21.15.133:0/2208372770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0740102780 msgr2=0x7f0740074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07468b4640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0740102780 0x7f0740074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07468b4640 1 -- 172.21.15.133:0/2208372770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0740101dd0 msgr2=0x7f07400744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07468b4640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0740101dd0 0x7f07400744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07468b4640 1 -- 172.21.15.133:0/2208372770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07401a6900 con 0x7f0740106f00 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07468b4640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0740106f00 0x7f0740074f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f073800e970 tx=0x7f073800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07377fe640 1 -- 172.21.15.133:0/2208372770 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073800ccc0 con 0x7f0740106f00 2024-09-11T23:35:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07377fe640 1 -- 172.21.15.133:0/2208372770 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f073800ce60 con 0x7f0740106f00 2024-09-11T23:35:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f0748317640 1 -- 172.21.15.133:0/2208372770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07401a6bf0 con 0x7f0740106f00 2024-09-11T23:35:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f07377fe640 1 -- 172.21.15.133:0/2208372770 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738010490 con 0x7f0740106f00 2024-09-11T23:35:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.981+0000 7f0748317640 1 -- 172.21.15.133:0/2208372770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07401a70e0 con 0x7f0740106f00 2024-09-11T23:35:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.982+0000 7f0748317640 1 -- 172.21.15.133:0/2208372770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07401021b0 con 0x7f0740106f00 2024-09-11T23:35:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.983+0000 7f07377fe640 1 -- 172.21.15.133:0/2208372770 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0738005020 con 0x7f0740106f00 2024-09-11T23:35:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.984+0000 7f07377fe640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f072c0a2c10 0x7f072c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.984+0000 7f07460b3640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f072c0a2c10 0x7f072c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.984+0000 7f07377fe640 1 -- 172.21.15.133:0/2208372770 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f07380cf530 con 0x7f0740106f00 2024-09-11T23:35:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.984+0000 7f07460b3640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f072c0a2c10 0x7f072c0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f073c006010 tx=0x7f073c009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:53.988+0000 7f07377fe640 1 -- 172.21.15.133:0/2208372770 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f073809c430 con 0x7f0740106f00 2024-09-11T23:35:54.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.159+0000 7f0748317640 1 -- 172.21.15.133:0/2208372770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f074010a0e0 con 0x7f0740106f00 2024-09-11T23:35:54.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.159+0000 7f07377fe640 1 -- 172.21.15.133:0/2208372770 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f073809bd30 con 0x7f0740106f00 2024-09-11T23:35:54.159 INFO:teuthology.orchestra.run.smithi133.stdout:51539607626 2024-09-11T23:35:54.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 -- 172.21.15.133:0/2208372770 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f072c0a2c10 msgr2=0x7f072c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:54.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f072c0a2c10 0x7f072c0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f073c006010 tx=0x7f073c009360 comp rx=0 tx=0).stop 2024-09-11T23:35:54.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 -- 172.21.15.133:0/2208372770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0740106f00 msgr2=0x7f0740074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:54.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0740106f00 0x7f0740074f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f073800e970 tx=0x7f073800ed70 comp rx=0 tx=0).stop 2024-09-11T23:35:54.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 -- 172.21.15.133:0/2208372770 shutdown_connections 2024-09-11T23:35:54.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f072c0a2c10 0x7f072c0a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0740106f00 0x7f0740074f70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0740102780 0x7f0740074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 --2- 172.21.15.133:0/2208372770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0740101dd0 0x7f07400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 -- 172.21.15.133:0/2208372770 >> 172.21.15.133:0/2208372770 conn(0x7f07400fd8b0 msgr2=0x7f07400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:54.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 -- 172.21.15.133:0/2208372770 shutdown_connections 2024-09-11T23:35:54.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.162+0000 7f0748317640 1 -- 172.21.15.133:0/2208372770 wait complete. 2024-09-11T23:35:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.223+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/1507932964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b8103760 msgr2=0x7fa6b8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.223+0000 7fa6bcb9d640 1 --2- 172.21.15.133:0/1507932964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b8103760 0x7fa6b8107ba0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009930 tx=0x7fa6a002a420 comp rx=0 tx=0).stop 2024-09-11T23:35:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.223+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/1507932964 shutdown_connections 2024-09-11T23:35:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.223+0000 7fa6bcb9d640 1 --2- 172.21.15.133:0/1507932964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6b81082e0 0x7fa6b810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.223+0000 7fa6bcb9d640 1 --2- 172.21.15.133:0/1507932964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b8103760 0x7fa6b8107ba0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.223+0000 7fa6bcb9d640 1 --2- 172.21.15.133:0/1507932964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b8102db0 0x7fa6b8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.223+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/1507932964 >> 172.21.15.133:0/1507932964 conn(0x7fa6b80fc7d0 msgr2=0x7fa6b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.223+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/1507932964 shutdown_connections 2024-09-11T23:35:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.223+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/1507932964 wait complete. 2024-09-11T23:35:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.224+0000 7fa6bcb9d640 1 Processor -- start 2024-09-11T23:35:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.224+0000 7fa6bcb9d640 1 -- start start 2024-09-11T23:35:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.224+0000 7fa6bcb9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b8102db0 0x7fa6b81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.224+0000 7fa6bcb9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6b8103760 0x7fa6b8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.224+0000 7fa6bcb9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b81082e0 0x7fa6b819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.224+0000 7fa6bcb9d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b8112050 con 0x7fa6b8102db0 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.224+0000 7fa6bcb9d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b8111ed0 con 0x7fa6b81082e0 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.224+0000 7fa6bcb9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b81121d0 con 0x7fa6b8103760 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b8102db0 0x7fa6b81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b8102db0 0x7fa6b81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48766/0 (socket says 172.21.15.133:48766) 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6b659c640 1 -- 172.21.15.133:0/3961925813 learned_addr learned my addr 172.21.15.133:0/3961925813 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6b5d9b640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6b8103760 0x7fa6b8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6b6d9d640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b81082e0 0x7fa6b819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6b659c640 1 -- 172.21.15.133:0/3961925813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6b8103760 msgr2=0x7fa6b8198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6b659c640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6b8103760 0x7fa6b8198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6b659c640 1 -- 172.21.15.133:0/3961925813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b81082e0 msgr2=0x7fa6b819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6b659c640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b81082e0 0x7fa6b819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6b659c640 1 -- 172.21.15.133:0/3961925813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b819d5a0 con 0x7fa6b8102db0 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6b659c640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b8102db0 0x7fa6b81985f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800cce0 tx=0x7fa6a800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6af7fe640 1 -- 172.21.15.133:0/3961925813 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8004b20 con 0x7fa6b8102db0 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6af7fe640 1 -- 172.21.15.133:0/3961925813 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a8004cc0 con 0x7fa6b8102db0 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6af7fe640 1 -- 172.21.15.133:0/3961925813 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800f7d0 con 0x7fa6b8102db0 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/3961925813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b819d890 con 0x7fa6b8102db0 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.225+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/3961925813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b81a5170 con 0x7fa6b8102db0 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.226+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/3961925813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b8103190 con 0x7fa6b8102db0 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.227+0000 7fa6af7fe640 1 -- 172.21.15.133:0/3961925813 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa6a800f970 con 0x7fa6b8102db0 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.229+0000 7fa6af7fe640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa6a40a2d10 0x7fa6a40a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.229+0000 7fa6b5d9b640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa6a40a2d10 0x7fa6a40a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.229+0000 7fa6af7fe640 1 -- 172.21.15.133:0/3961925813 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fa6a80cf310 con 0x7fa6b8102db0 2024-09-11T23:35:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.229+0000 7fa6b5d9b640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa6a40a2d10 0x7fa6a40a51d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009900 tx=0x7fa6a0005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.233+0000 7fa6af7fe640 1 -- 172.21.15.133:0/3961925813 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6a8014030 con 0x7fa6b8102db0 2024-09-11T23:35:54.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.403+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/3961925813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa6b810a2c0 con 0x7fa6b8102db0 2024-09-11T23:35:54.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.403+0000 7fa6af7fe640 1 -- 172.21.15.133:0/3961925813 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa6a809c110 con 0x7fa6b8102db0 2024-09-11T23:35:54.403 INFO:teuthology.orchestra.run.smithi133.stdout:231928234024 2024-09-11T23:35:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/3961925813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa6a40a2d10 msgr2=0x7fa6a40a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa6a40a2d10 0x7fa6a40a51d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009900 tx=0x7fa6a0005c00 comp rx=0 tx=0).stop 2024-09-11T23:35:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/3961925813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b8102db0 msgr2=0x7fa6b81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b8102db0 0x7fa6b81985f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800cce0 tx=0x7fa6a800a620 comp rx=0 tx=0).stop 2024-09-11T23:35:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/3961925813 shutdown_connections 2024-09-11T23:35:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa6a40a2d10 0x7fa6a40a51d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b81082e0 0x7fa6b819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6b8103760 0x7fa6b8198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 --2- 172.21.15.133:0/3961925813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b8102db0 0x7fa6b81985f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:54.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/3961925813 >> 172.21.15.133:0/3961925813 conn(0x7fa6b80fc7d0 msgr2=0x7fa6b8106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:54.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/3961925813 shutdown_connections 2024-09-11T23:35:54.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:54.405+0000 7fa6bcb9d640 1 -- 172.21.15.133:0/3961925813 wait complete. 2024-09-11T23:35:54.782 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841561 got 283467841564 for osd.10 2024-09-11T23:35:54.782 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:55.004 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:54 smithi133 ceph-mon[31430]: pgmap v304: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:35:55.004 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:54 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2208372770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:35:55.005 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:54 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3961925813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:35:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:54 smithi178 ceph-mon[35882]: pgmap v304: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:35:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:54 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2208372770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:35:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:54 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3961925813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:35:55.423 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612629 got 313532612631 for osd.11 2024-09-11T23:35:55.423 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:55.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:54 smithi138 ceph-mon[35142]: pgmap v304: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:35:55.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:54 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2208372770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:35:55.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:54 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3961925813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:35:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.643+0000 7f777937a640 1 -- 172.21.15.133:0/2685446152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7774102ef0 msgr2=0x7f777410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.643+0000 7f777937a640 1 --2- 172.21.15.133:0/2685446152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7774102ef0 0x7f777410da40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f776c009ca0 tx=0x7f776c02a950 comp rx=0 tx=0).stop 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.643+0000 7f777937a640 1 -- 172.21.15.133:0/2685446152 shutdown_connections 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.643+0000 7f777937a640 1 --2- 172.21.15.133:0/2685446152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7774102ef0 0x7f777410da40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.643+0000 7f777937a640 1 --2- 172.21.15.133:0/2685446152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7774102570 0x7f77741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.643+0000 7f777937a640 1 --2- 172.21.15.133:0/2685446152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7774100e20 0x7f7774101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.643+0000 7f777937a640 1 -- 172.21.15.133:0/2685446152 >> 172.21.15.133:0/2685446152 conn(0x7f77740fc7d0 msgr2=0x7f77740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.643+0000 7f777937a640 1 -- 172.21.15.133:0/2685446152 shutdown_connections 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.643+0000 7f777937a640 1 -- 172.21.15.133:0/2685446152 wait complete. 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f777937a640 1 Processor -- start 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f777937a640 1 -- start start 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f777937a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7774100e20 0x7f77741974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f777937a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7774102570 0x7f7774197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f777937a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7774102ef0 0x7f777419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f777937a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7774111020 con 0x7f7774102ef0 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f777937a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7774110ea0 con 0x7f7774100e20 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f777937a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77741111a0 con 0x7f7774102570 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f77737fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7774102ef0 0x7f777419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f77737fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7774102ef0 0x7f777419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48776/0 (socket says 172.21.15.133:48776) 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f77737fe640 1 -- 172.21.15.133:0/2879555813 learned_addr learned my addr 172.21.15.133:0/2879555813 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.644+0000 7f776bfff640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7774102570 0x7f7774197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f77737fe640 1 -- 172.21.15.133:0/2879555813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7774102570 msgr2=0x7f7774197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f77737fe640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7774102570 0x7f7774197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f7772ffd640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7774100e20 0x7f77741974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f77737fe640 1 -- 172.21.15.133:0/2879555813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7774100e20 msgr2=0x7f77741974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f77737fe640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7774100e20 0x7f77741974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f77737fe640 1 -- 172.21.15.133:0/2879555813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777419c490 con 0x7f7774102ef0 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f77737fe640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7774102ef0 0x7f777419bdb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f776c006010 tx=0x7f776c02f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f7770ff9640 1 -- 172.21.15.133:0/2879555813 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c034040 con 0x7f7774102ef0 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f777937a640 1 -- 172.21.15.133:0/2879555813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777419c720 con 0x7f7774102ef0 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f7770ff9640 1 -- 172.21.15.133:0/2879555813 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f776c003e00 con 0x7f7774102ef0 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f7770ff9640 1 -- 172.21.15.133:0/2879555813 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c045480 con 0x7f7774102ef0 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.645+0000 7f777937a640 1 -- 172.21.15.133:0/2879555813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77741a4060 con 0x7f7774102ef0 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.646+0000 7f777937a640 1 -- 172.21.15.133:0/2879555813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7774101280 con 0x7f7774102ef0 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.647+0000 7f7770ff9640 1 -- 172.21.15.133:0/2879555813 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f776c037020 con 0x7f7774102ef0 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.648+0000 7f7770ff9640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77600a2c10 0x7f77600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.648+0000 7f7772ffd640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77600a2c10 0x7f77600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.648+0000 7f7770ff9640 1 -- 172.21.15.133:0/2879555813 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f776c0f0190 con 0x7f7774102ef0 2024-09-11T23:35:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.648+0000 7f7772ffd640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77600a2c10 0x7f77600a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f776400a810 tx=0x7f7764008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:55.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.651+0000 7f7770ff9640 1 -- 172.21.15.133:0/2879555813 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f776c043380 con 0x7f7774102ef0 2024-09-11T23:35:55.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.819+0000 7f777937a640 1 -- 172.21.15.133:0/2879555813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7774109090 con 0x7f7774102ef0 2024-09-11T23:35:55.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.819+0000 7f7770ff9640 1 -- 172.21.15.133:0/2879555813 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f776c0bd230 con 0x7f7774102ef0 2024-09-11T23:35:55.820 INFO:teuthology.orchestra.run.smithi133.stdout:146028888118 2024-09-11T23:35:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 -- 172.21.15.133:0/2879555813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77600a2c10 msgr2=0x7f77600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77600a2c10 0x7f77600a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f776400a810 tx=0x7f7764008040 comp rx=0 tx=0).stop 2024-09-11T23:35:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 -- 172.21.15.133:0/2879555813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7774102ef0 msgr2=0x7f777419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7774102ef0 0x7f777419bdb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f776c006010 tx=0x7f776c02f6f0 comp rx=0 tx=0).stop 2024-09-11T23:35:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 -- 172.21.15.133:0/2879555813 shutdown_connections 2024-09-11T23:35:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f77600a2c10 0x7f77600a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7774102ef0 0x7f777419bdb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7774102570 0x7f7774197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 --2- 172.21.15.133:0/2879555813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7774100e20 0x7f77741974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 -- 172.21.15.133:0/2879555813 >> 172.21.15.133:0/2879555813 conn(0x7f77740fc7d0 msgr2=0x7f77740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 -- 172.21.15.133:0/2879555813 shutdown_connections 2024-09-11T23:35:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:55.821+0000 7f777937a640 1 -- 172.21.15.133:0/2879555813 wait complete. 2024-09-11T23:35:56.099 INFO:tasks.cephadm.ceph_manager.ceph:need seq 51539607623 got 51539607626 for osd.1 2024-09-11T23:35:56.099 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:56 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2879555813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:35:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:56 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2879555813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:35:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:56 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2879555813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:35:56.513 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234021 got 231928234024 for osd.8 2024-09-11T23:35:56.513 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:57.188 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:57 smithi133 ceph-mon[31430]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:35:57.215 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888115 got 146028888118 for osd.5 2024-09-11T23:35:57.215 DEBUG:teuthology.parallel:result is None 2024-09-11T23:35:57.215 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:35:57.216 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:35:57.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:57 smithi178 ceph-mon[35882]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:35:57.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:57 smithi138 ceph-mon[35142]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:35:57.987 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:35:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.465+0000 7f51b27f4640 1 -- 172.21.15.133:0/4015027757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac101df0 msgr2=0x7f51ac1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.465+0000 7f51b27f4640 1 --2- 172.21.15.133:0/4015027757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac101df0 0x7f51ac1021d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f519c009900 tx=0x7f519c02a410 comp rx=0 tx=0).stop 2024-09-11T23:35:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.465+0000 7f51b27f4640 1 -- 172.21.15.133:0/4015027757 shutdown_connections 2024-09-11T23:35:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.465+0000 7f51b27f4640 1 --2- 172.21.15.133:0/4015027757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51ac106f20 0x7f51ac10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.465+0000 7f51b27f4640 1 --2- 172.21.15.133:0/4015027757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51ac1027a0 0x7f51ac1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.465+0000 7f51b27f4640 1 --2- 172.21.15.133:0/4015027757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac101df0 0x7f51ac1021d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.465+0000 7f51b27f4640 1 -- 172.21.15.133:0/4015027757 >> 172.21.15.133:0/4015027757 conn(0x7f51ac0fd8b0 msgr2=0x7f51ac0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.465+0000 7f51b27f4640 1 -- 172.21.15.133:0/4015027757 shutdown_connections 2024-09-11T23:35:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.465+0000 7f51b27f4640 1 -- 172.21.15.133:0/4015027757 wait complete. 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.466+0000 7f51b27f4640 1 Processor -- start 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.466+0000 7f51b27f4640 1 -- start start 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.466+0000 7f51b27f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac101df0 0x7f51ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.466+0000 7f51b27f4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51ac1027a0 0x7f51ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.466+0000 7f51b27f4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51ac106f20 0x7f51ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.466+0000 7f51b27f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac111f80 con 0x7f51ac101df0 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.466+0000 7f51b27f4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac111e00 con 0x7f51ac106f20 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.466+0000 7f51b27f4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac112100 con 0x7f51ac1027a0 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.466+0000 7f51abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac101df0 0x7f51ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.466+0000 7f51abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac101df0 0x7f51ac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48790/0 (socket says 172.21.15.133:48790) 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.466+0000 7f51abfff640 1 -- 172.21.15.133:0/2945299732 learned_addr learned my addr 172.21.15.133:0/2945299732 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:35:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51ab7fe640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51ac1027a0 0x7f51ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51b0d91640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51ac106f20 0x7f51ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51abfff640 1 -- 172.21.15.133:0/2945299732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51ac1027a0 msgr2=0x7f51ac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51abfff640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51ac1027a0 0x7f51ac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51abfff640 1 -- 172.21.15.133:0/2945299732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51ac106f20 msgr2=0x7f51ac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51abfff640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51ac106f20 0x7f51ac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51abfff640 1 -- 172.21.15.133:0/2945299732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ac1a6920 con 0x7f51ac101df0 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51b0d91640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51ac106f20 0x7f51ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51abfff640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac101df0 0x7f51ac0744f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f519c006010 tx=0x7f519c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51a97fa640 1 -- 172.21.15.133:0/2945299732 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c03f030 con 0x7f51ac101df0 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51a97fa640 1 -- 172.21.15.133:0/2945299732 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f519c003ce0 con 0x7f51ac101df0 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51a97fa640 1 -- 172.21.15.133:0/2945299732 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c035070 con 0x7f51ac101df0 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51b27f4640 1 -- 172.21.15.133:0/2945299732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ac1a6bb0 con 0x7f51ac101df0 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.467+0000 7f51b27f4640 1 -- 172.21.15.133:0/2945299732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ac1a70e0 con 0x7f51ac101df0 2024-09-11T23:35:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.468+0000 7f51b27f4640 1 -- 172.21.15.133:0/2945299732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ac1021d0 con 0x7f51ac101df0 2024-09-11T23:35:58.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.469+0000 7f51a97fa640 1 -- 172.21.15.133:0/2945299732 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f519c042030 con 0x7f51ac101df0 2024-09-11T23:35:58.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.470+0000 7f51a97fa640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f51940a2c90 0x7f51940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:35:58.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.470+0000 7f51ab7fe640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f51940a2c90 0x7f51940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:35:58.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.470+0000 7f51a97fa640 1 -- 172.21.15.133:0/2945299732 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f519c0ef590 con 0x7f51ac101df0 2024-09-11T23:35:58.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.470+0000 7f51ab7fe640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f51940a2c90 0x7f51940a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51ac075f80 tx=0x7f51a000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:35:58.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.474+0000 7f51a97fa640 1 -- 172.21.15.133:0/2945299732 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f519c0bc550 con 0x7f51ac101df0 2024-09-11T23:35:58.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.659+0000 7f51b27f4640 1 -- 172.21.15.133:0/2945299732 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51ac10a170 con 0x7f51940a2c90 2024-09-11T23:35:58.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.667+0000 7f51a97fa640 1 -- 172.21.15.133:0/2945299732 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258760 (secure 0 0 0) 0x7f51a0003890 con 0x7f51940a2c90 2024-09-11T23:35:58.672 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:35:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T23:35:58.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.670+0000 7f51b27f4640 1 -- 172.21.15.133:0/2945299732 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f51940a2c90 msgr2=0x7f51940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:58.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.670+0000 7f51b27f4640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f51940a2c90 0x7f51940a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51ac075f80 tx=0x7f51a000b390 comp rx=0 tx=0).stop 2024-09-11T23:35:58.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.671+0000 7f51b27f4640 1 -- 172.21.15.133:0/2945299732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac101df0 msgr2=0x7f51ac0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:35:58.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.671+0000 7f51b27f4640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac101df0 0x7f51ac0744f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f519c006010 tx=0x7f519c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.671+0000 7f51b27f4640 1 -- 172.21.15.133:0/2945299732 shutdown_connections 2024-09-11T23:35:58.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.671+0000 7f51b27f4640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f51940a2c90 0x7f51940a5150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.671+0000 7f51b27f4640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51ac106f20 0x7f51ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.671+0000 7f51b27f4640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51ac1027a0 0x7f51ac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.671+0000 7f51b27f4640 1 --2- 172.21.15.133:0/2945299732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac101df0 0x7f51ac0744f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:35:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.671+0000 7f51b27f4640 1 -- 172.21.15.133:0/2945299732 >> 172.21.15.133:0/2945299732 conn(0x7f51ac0fd8b0 msgr2=0x7f51ac0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:35:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.671+0000 7f51b27f4640 1 -- 172.21.15.133:0/2945299732 shutdown_connections 2024-09-11T23:35:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:35:58.671+0000 7f51b27f4640 1 -- 172.21.15.133:0/2945299732 wait complete. 2024-09-11T23:35:59.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:35:59 smithi178 ceph-mon[35882]: pgmap v306: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:35:59.366 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:35:59 smithi133 ceph-mon[31430]: pgmap v306: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:35:59.404 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-11T23:35:57.860188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660120,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":693,"num_read_kb":470,"num_write":430,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357760,"kb_used_data":12648,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584064,"statfs":{"total":1151940427776,"available":1151574081536,"internally_reserved":0,"allocated":12951552,"data_stored":7178388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003058"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845014+0000","last_change":"2024-09-11T23:34:52.867694+0000","last_active":"2024-09-11T23:35:08.845014+0000","last_peered":"2024-09-11T23:35:08.845014+0000","last_clean":"2024-09-11T23:35:08.845014+0000","last_became_active":"2024-09-11T23:34:52.867368+0000","last_became_peered":"2024-09-11T23:34:52.867368+0000","last_unstale":"2024-09-11T23:35:08.845014+0000","last_undegraded":"2024-09-11T23:35:08.845014+0000","last_fullsized":"2024-09-11T23:35:08.845014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:46:07.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872241+0000","last_change":"2024-09-11T23:34:52.871881+0000","last_active":"2024-09-11T23:35:08.872241+0000","last_peered":"2024-09-11T23:35:08.872241+0000","last_clean":"2024-09-11T23:35:08.872241+0000","last_became_active":"2024-09-11T23:34:52.871609+0000","last_became_peered":"2024-09-11T23:34:52.871609+0000","last_unstale":"2024-09-11T23:35:08.872241+0000","last_undegraded":"2024-09-11T23:35:08.872241+0000","last_fullsized":"2024-09-11T23:35:08.872241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.938584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872054+0000","last_change":"2024-09-11T23:34:52.863931+0000","last_active":"2024-09-11T23:35:08.872054+0000","last_peered":"2024-09-11T23:35:08.872054+0000","last_clean":"2024-09-11T23:35:08.872054+0000","last_became_active":"2024-09-11T23:34:52.863637+0000","last_became_peered":"2024-09-11T23:34:52.863637+0000","last_unstale":"2024-09-11T23:35:08.872054+0000","last_undegraded":"2024-09-11T23:35:08.872054+0000","last_fullsized":"2024-09-11T23:35:08.872054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:44.118865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846293+0000","last_change":"2024-09-11T23:34:52.879619+0000","last_active":"2024-09-11T23:35:08.846293+0000","last_peered":"2024-09-11T23:35:08.846293+0000","last_clean":"2024-09-11T23:35:08.846293+0000","last_became_active":"2024-09-11T23:34:52.878695+0000","last_became_peered":"2024-09-11T23:34:52.878695+0000","last_unstale":"2024-09-11T23:35:08.846293+0000","last_undegraded":"2024-09-11T23:35:08.846293+0000","last_fullsized":"2024-09-11T23:35:08.846293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:30.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847368+0000","last_change":"2024-09-11T23:34:52.869363+0000","last_active":"2024-09-11T23:35:08.847368+0000","last_peered":"2024-09-11T23:35:08.847368+0000","last_clean":"2024-09-11T23:35:08.847368+0000","last_became_active":"2024-09-11T23:34:52.869027+0000","last_became_peered":"2024-09-11T23:34:52.869027+0000","last_unstale":"2024-09-11T23:35:08.847368+0000","last_undegraded":"2024-09-11T23:35:08.847368+0000","last_fullsized":"2024-09-11T23:35:08.847368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:15.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848328+0000","last_change":"2024-09-11T23:34:52.880665+0000","last_active":"2024-09-11T23:35:08.848328+0000","last_peered":"2024-09-11T23:35:08.848328+0000","last_clean":"2024-09-11T23:35:08.848328+0000","last_became_active":"2024-09-11T23:34:52.880172+0000","last_became_peered":"2024-09-11T23:34:52.880172+0000","last_unstale":"2024-09-11T23:35:08.848328+0000","last_undegraded":"2024-09-11T23:35:08.848328+0000","last_fullsized":"2024-09-11T23:35:08.848328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:12.025877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846502+0000","last_change":"2024-09-11T23:34:52.873897+0000","last_active":"2024-09-11T23:35:08.846502+0000","last_peered":"2024-09-11T23:35:08.846502+0000","last_clean":"2024-09-11T23:35:08.846502+0000","last_became_active":"2024-09-11T23:34:52.873219+0000","last_became_peered":"2024-09-11T23:34:52.873219+0000","last_unstale":"2024-09-11T23:35:08.846502+0000","last_undegraded":"2024-09-11T23:35:08.846502+0000","last_fullsized":"2024-09-11T23:35:08.846502+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:47.125682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847314+0000","last_change":"2024-09-11T23:34:52.870699+0000","last_active":"2024-09-11T23:35:08.847314+0000","last_peered":"2024-09-11T23:35:08.847314+0000","last_clean":"2024-09-11T23:35:08.847314+0000","last_became_active":"2024-09-11T23:34:52.870314+0000","last_became_peered":"2024-09-11T23:34:52.870314+0000","last_unstale":"2024-09-11T23:35:08.847314+0000","last_undegraded":"2024-09-11T23:35:08.847314+0000","last_fullsized":"2024-09-11T23:35:08.847314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:41.263159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.851159+0000","last_change":"2024-09-11T23:34:52.869700+0000","last_active":"2024-09-11T23:35:08.851159+0000","last_peered":"2024-09-11T23:35:08.851159+0000","last_clean":"2024-09-11T23:35:08.851159+0000","last_became_active":"2024-09-11T23:34:52.869398+0000","last_became_peered":"2024-09-11T23:34:52.869398+0000","last_unstale":"2024-09-11T23:35:08.851159+0000","last_undegraded":"2024-09-11T23:35:08.851159+0000","last_fullsized":"2024-09-11T23:35:08.851159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:49.776525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848589+0000","last_change":"2024-09-11T23:34:52.879999+0000","last_active":"2024-09-11T23:35:08.848589+0000","last_peered":"2024-09-11T23:35:08.848589+0000","last_clean":"2024-09-11T23:35:08.848589+0000","last_became_active":"2024-09-11T23:34:52.879646+0000","last_became_peered":"2024-09-11T23:34:52.879646+0000","last_unstale":"2024-09-11T23:35:08.848589+0000","last_undegraded":"2024-09-11T23:35:08.848589+0000","last_fullsized":"2024-09-11T23:35:08.848589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.058704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846881+0000","last_change":"2024-09-11T23:34:52.878740+0000","last_active":"2024-09-11T23:35:08.846881+0000","last_peered":"2024-09-11T23:35:08.846881+0000","last_clean":"2024-09-11T23:35:08.846881+0000","last_became_active":"2024-09-11T23:34:52.878120+0000","last_became_peered":"2024-09-11T23:34:52.878120+0000","last_unstale":"2024-09-11T23:35:08.846881+0000","last_undegraded":"2024-09-11T23:35:08.846881+0000","last_fullsized":"2024-09-11T23:35:08.846881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846051+0000","last_change":"2024-09-11T23:34:52.873409+0000","last_active":"2024-09-11T23:35:08.846051+0000","last_peered":"2024-09-11T23:35:08.846051+0000","last_clean":"2024-09-11T23:35:08.846051+0000","last_became_active":"2024-09-11T23:34:52.872959+0000","last_became_peered":"2024-09-11T23:34:52.872959+0000","last_unstale":"2024-09-11T23:35:08.846051+0000","last_undegraded":"2024-09-11T23:35:08.846051+0000","last_fullsized":"2024-09-11T23:35:08.846051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:38.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844266+0000","last_change":"2024-09-11T23:34:52.869164+0000","last_active":"2024-09-11T23:35:08.844266+0000","last_peered":"2024-09-11T23:35:08.844266+0000","last_clean":"2024-09-11T23:35:08.844266+0000","last_became_active":"2024-09-11T23:34:52.868847+0000","last_became_peered":"2024-09-11T23:34:52.868847+0000","last_unstale":"2024-09-11T23:35:08.844266+0000","last_undegraded":"2024-09-11T23:35:08.844266+0000","last_fullsized":"2024-09-11T23:35:08.844266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.764510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872027+0000","last_change":"2024-09-11T23:34:52.869494+0000","last_active":"2024-09-11T23:35:08.872027+0000","last_peered":"2024-09-11T23:35:08.872027+0000","last_clean":"2024-09-11T23:35:08.872027+0000","last_became_active":"2024-09-11T23:34:52.869178+0000","last_became_peered":"2024-09-11T23:34:52.869178+0000","last_unstale":"2024-09-11T23:35:08.872027+0000","last_undegraded":"2024-09-11T23:35:08.872027+0000","last_fullsized":"2024-09-11T23:35:08.872027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:23.431564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872033+0000","last_change":"2024-09-11T23:34:52.866269+0000","last_active":"2024-09-11T23:35:08.872033+0000","last_peered":"2024-09-11T23:35:08.872033+0000","last_clean":"2024-09-11T23:35:08.872033+0000","last_became_active":"2024-09-11T23:34:52.865980+0000","last_became_peered":"2024-09-11T23:34:52.865980+0000","last_unstale":"2024-09-11T23:35:08.872033+0000","last_undegraded":"2024-09-11T23:35:08.872033+0000","last_fullsized":"2024-09-11T23:35:08.872033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:48.492252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850466+0000","last_change":"2024-09-11T23:34:52.862192+0000","last_active":"2024-09-11T23:35:08.850466+0000","last_peered":"2024-09-11T23:35:08.850466+0000","last_clean":"2024-09-11T23:35:08.850466+0000","last_became_active":"2024-09-11T23:34:52.861841+0000","last_became_peered":"2024-09-11T23:34:52.861841+0000","last_unstale":"2024-09-11T23:35:08.850466+0000","last_undegraded":"2024-09-11T23:35:08.850466+0000","last_fullsized":"2024-09-11T23:35:08.850466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.258948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845097+0000","last_change":"2024-09-11T23:34:52.870984+0000","last_active":"2024-09-11T23:35:08.845097+0000","last_peered":"2024-09-11T23:35:08.845097+0000","last_clean":"2024-09-11T23:35:08.845097+0000","last_became_active":"2024-09-11T23:34:52.870435+0000","last_became_peered":"2024-09-11T23:34:52.870435+0000","last_unstale":"2024-09-11T23:35:08.845097+0000","last_undegraded":"2024-09-11T23:35:08.845097+0000","last_fullsized":"2024-09-11T23:35:08.845097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:34.391537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847466+0000","last_change":"2024-09-11T23:34:52.873176+0000","last_active":"2024-09-11T23:35:08.847466+0000","last_peered":"2024-09-11T23:35:08.847466+0000","last_clean":"2024-09-11T23:35:08.847466+0000","last_became_active":"2024-09-11T23:34:52.872862+0000","last_became_peered":"2024-09-11T23:34:52.872862+0000","last_unstale":"2024-09-11T23:35:08.847466+0000","last_undegraded":"2024-09-11T23:35:08.847466+0000","last_fullsized":"2024-09-11T23:35:08.847466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:38.058806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847715+0000","last_change":"2024-09-11T23:34:52.871346+0000","last_active":"2024-09-11T23:35:08.847715+0000","last_peered":"2024-09-11T23:35:08.847715+0000","last_clean":"2024-09-11T23:35:08.847715+0000","last_became_active":"2024-09-11T23:34:52.870981+0000","last_became_peered":"2024-09-11T23:34:52.870981+0000","last_unstale":"2024-09-11T23:35:08.847715+0000","last_undegraded":"2024-09-11T23:35:08.847715+0000","last_fullsized":"2024-09-11T23:35:08.847715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:56.973998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844737+0000","last_change":"2024-09-11T23:34:52.867058+0000","last_active":"2024-09-11T23:35:08.844737+0000","last_peered":"2024-09-11T23:35:08.844737+0000","last_clean":"2024-09-11T23:35:08.844737+0000","last_became_active":"2024-09-11T23:34:52.866687+0000","last_became_peered":"2024-09-11T23:34:52.866687+0000","last_unstale":"2024-09-11T23:35:08.844737+0000","last_undegraded":"2024-09-11T23:35:08.844737+0000","last_fullsized":"2024-09-11T23:35:08.844737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:09.019987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844347+0000","last_change":"2024-09-11T23:34:52.870882+0000","last_active":"2024-09-11T23:35:08.844347+0000","last_peered":"2024-09-11T23:35:08.844347+0000","last_clean":"2024-09-11T23:35:08.844347+0000","last_became_active":"2024-09-11T23:34:52.870667+0000","last_became_peered":"2024-09-11T23:34:52.870667+0000","last_unstale":"2024-09-11T23:35:08.844347+0000","last_undegraded":"2024-09-11T23:35:08.844347+0000","last_fullsized":"2024-09-11T23:35:08.844347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:08.386703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850359+0000","last_change":"2024-09-11T23:34:52.874070+0000","last_active":"2024-09-11T23:35:08.850359+0000","last_peered":"2024-09-11T23:35:08.850359+0000","last_clean":"2024-09-11T23:35:08.850359+0000","last_became_active":"2024-09-11T23:34:52.873772+0000","last_became_peered":"2024-09-11T23:34:52.873772+0000","last_unstale":"2024-09-11T23:35:08.850359+0000","last_undegraded":"2024-09-11T23:35:08.850359+0000","last_fullsized":"2024-09-11T23:35:08.850359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:49.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847766+0000","last_change":"2024-09-11T23:34:52.875502+0000","last_active":"2024-09-11T23:35:08.847766+0000","last_peered":"2024-09-11T23:35:08.847766+0000","last_clean":"2024-09-11T23:35:08.847766+0000","last_became_active":"2024-09-11T23:34:52.875193+0000","last_became_peered":"2024-09-11T23:34:52.875193+0000","last_unstale":"2024-09-11T23:35:08.847766+0000","last_undegraded":"2024-09-11T23:35:08.847766+0000","last_fullsized":"2024-09-11T23:35:08.847766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:37.213828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455450+0000","last_change":"2024-09-11T23:34:52.869015+0000","last_active":"2024-09-11T23:35:09.455450+0000","last_peered":"2024-09-11T23:35:09.455450+0000","last_clean":"2024-09-11T23:35:09.455450+0000","last_became_active":"2024-09-11T23:34:52.868731+0000","last_became_peered":"2024-09-11T23:34:52.868731+0000","last_unstale":"2024-09-11T23:35:09.455450+0000","last_undegraded":"2024-09-11T23:35:09.455450+0000","last_fullsized":"2024-09-11T23:35:09.455450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:12:15.993203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848557+0000","last_change":"2024-09-11T23:34:52.875555+0000","last_active":"2024-09-11T23:35:08.848557+0000","last_peered":"2024-09-11T23:35:08.848557+0000","last_clean":"2024-09-11T23:35:08.848557+0000","last_became_active":"2024-09-11T23:34:52.875219+0000","last_became_peered":"2024-09-11T23:34:52.875219+0000","last_unstale":"2024-09-11T23:35:08.848557+0000","last_undegraded":"2024-09-11T23:35:08.848557+0000","last_fullsized":"2024-09-11T23:35:08.848557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:16.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848205+0000","last_change":"2024-09-11T23:34:52.875629+0000","last_active":"2024-09-11T23:35:08.848205+0000","last_peered":"2024-09-11T23:35:08.848205+0000","last_clean":"2024-09-11T23:35:08.848205+0000","last_became_active":"2024-09-11T23:34:52.875302+0000","last_became_peered":"2024-09-11T23:34:52.875302+0000","last_unstale":"2024-09-11T23:35:08.848205+0000","last_undegraded":"2024-09-11T23:35:08.848205+0000","last_fullsized":"2024-09-11T23:35:08.848205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:12.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844886+0000","last_change":"2024-09-11T23:34:52.868613+0000","last_active":"2024-09-11T23:35:08.844886+0000","last_peered":"2024-09-11T23:35:08.844886+0000","last_clean":"2024-09-11T23:35:08.844886+0000","last_became_active":"2024-09-11T23:34:52.868184+0000","last_became_peered":"2024-09-11T23:34:52.868184+0000","last_unstale":"2024-09-11T23:35:08.844886+0000","last_undegraded":"2024-09-11T23:35:08.844886+0000","last_fullsized":"2024-09-11T23:35:08.844886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:25.990002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.873390+0000","last_change":"2024-09-11T23:35:08.873390+0000","last_active":"2024-09-11T23:35:08.873390+0000","last_peered":"2024-09-11T23:35:08.873390+0000","last_clean":"2024-09-11T23:35:08.873390+0000","last_became_active":"2024-09-11T23:35:08.873052+0000","last_became_peered":"2024-09-11T23:35:08.873052+0000","last_unstale":"2024-09-11T23:35:08.873390+0000","last_undegraded":"2024-09-11T23:35:08.873390+0000","last_fullsized":"2024-09-11T23:35:08.873390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.937951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872829+0000","last_change":"2024-09-11T23:34:52.873604+0000","last_active":"2024-09-11T23:35:08.872829+0000","last_peered":"2024-09-11T23:35:08.872829+0000","last_clean":"2024-09-11T23:35:08.872829+0000","last_became_active":"2024-09-11T23:34:52.873220+0000","last_became_peered":"2024-09-11T23:34:52.873220+0000","last_unstale":"2024-09-11T23:35:08.872829+0000","last_undegraded":"2024-09-11T23:35:08.872829+0000","last_fullsized":"2024-09-11T23:35:08.872829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:38.480630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847599+0000","last_change":"2024-09-11T23:34:52.872902+0000","last_active":"2024-09-11T23:35:08.847599+0000","last_peered":"2024-09-11T23:35:08.847599+0000","last_clean":"2024-09-11T23:35:08.847599+0000","last_became_active":"2024-09-11T23:34:52.872571+0000","last_became_peered":"2024-09-11T23:34:52.872571+0000","last_unstale":"2024-09-11T23:35:08.847599+0000","last_undegraded":"2024-09-11T23:35:08.847599+0000","last_fullsized":"2024-09-11T23:35:08.847599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:21.844995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.457056+0000","last_change":"2024-09-11T23:35:09.457056+0000","last_active":"2024-09-11T23:35:09.457056+0000","last_peered":"2024-09-11T23:35:09.457056+0000","last_clean":"2024-09-11T23:35:09.457056+0000","last_became_active":"2024-09-11T23:35:09.456657+0000","last_became_peered":"2024-09-11T23:35:09.456657+0000","last_unstale":"2024-09-11T23:35:09.457056+0000","last_undegraded":"2024-09-11T23:35:09.457056+0000","last_fullsized":"2024-09-11T23:35:09.457056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:12:56.121280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844633+0000","last_change":"2024-09-11T23:34:52.875312+0000","last_active":"2024-09-11T23:35:08.844633+0000","last_peered":"2024-09-11T23:35:08.844633+0000","last_clean":"2024-09-11T23:35:08.844633+0000","last_became_active":"2024-09-11T23:34:52.874873+0000","last_became_peered":"2024-09-11T23:34:52.874873+0000","last_unstale":"2024-09-11T23:35:08.844633+0000","last_undegraded":"2024-09-11T23:35:08.844633+0000","last_fullsized":"2024-09-11T23:35:08.844633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:18.894713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847244+0000","last_change":"2024-09-11T23:34:52.862266+0000","last_active":"2024-09-11T23:35:08.847244+0000","last_peered":"2024-09-11T23:35:08.847244+0000","last_clean":"2024-09-11T23:35:08.847244+0000","last_became_active":"2024-09-11T23:34:52.861923+0000","last_became_peered":"2024-09-11T23:34:52.861923+0000","last_unstale":"2024-09-11T23:35:08.847244+0000","last_undegraded":"2024-09-11T23:35:08.847244+0000","last_fullsized":"2024-09-11T23:35:08.847244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:23.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":82,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:54.517127+0000","last_change":"2024-09-11T23:34:50.854020+0000","last_active":"2024-09-11T23:35:54.517127+0000","last_peered":"2024-09-11T23:35:54.517127+0000","last_clean":"2024-09-11T23:35:54.517127+0000","last_became_active":"2024-09-11T23:34:50.853683+0000","last_became_peered":"2024-09-11T23:34:50.853683+0000","last_unstale":"2024-09-11T23:35:54.517127+0000","last_undegraded":"2024-09-11T23:35:54.517127+0000","last_fullsized":"2024-09-11T23:35:54.517127+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:18.725495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455234+0000","last_change":"2024-09-11T23:34:52.874016+0000","last_active":"2024-09-11T23:35:09.455234+0000","last_peered":"2024-09-11T23:35:09.455234+0000","last_clean":"2024-09-11T23:35:09.455234+0000","last_became_active":"2024-09-11T23:34:52.873675+0000","last_became_peered":"2024-09-11T23:34:52.873675+0000","last_unstale":"2024-09-11T23:35:09.455234+0000","last_undegraded":"2024-09-11T23:35:09.455234+0000","last_fullsized":"2024-09-11T23:35:09.455234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:41:28.955548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848555+0000","last_change":"2024-09-11T23:34:50.855391+0000","last_active":"2024-09-11T23:35:08.848555+0000","last_peered":"2024-09-11T23:35:08.848555+0000","last_clean":"2024-09-11T23:35:08.848555+0000","last_became_active":"2024-09-11T23:34:50.855052+0000","last_became_peered":"2024-09-11T23:34:50.855052+0000","last_unstale":"2024-09-11T23:35:08.848555+0000","last_undegraded":"2024-09-11T23:35:08.848555+0000","last_fullsized":"2024-09-11T23:35:08.848555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.972239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844031+0000","last_change":"2024-09-11T23:34:50.848985+0000","last_active":"2024-09-11T23:35:08.844031+0000","last_peered":"2024-09-11T23:35:08.844031+0000","last_clean":"2024-09-11T23:35:08.844031+0000","last_became_active":"2024-09-11T23:34:50.848614+0000","last_became_peered":"2024-09-11T23:34:50.848614+0000","last_unstale":"2024-09-11T23:35:08.844031+0000","last_undegraded":"2024-09-11T23:35:08.844031+0000","last_fullsized":"2024-09-11T23:35:08.844031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:43:04.723136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847368+0000","last_change":"2024-09-11T23:34:52.866933+0000","last_active":"2024-09-11T23:35:08.847368+0000","last_peered":"2024-09-11T23:35:08.847368+0000","last_clean":"2024-09-11T23:35:08.847368+0000","last_became_active":"2024-09-11T23:34:52.866559+0000","last_became_peered":"2024-09-11T23:34:52.866559+0000","last_unstale":"2024-09-11T23:35:08.847368+0000","last_undegraded":"2024-09-11T23:35:08.847368+0000","last_fullsized":"2024-09-11T23:35:08.847368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:27.745118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456265+0000","last_change":"2024-09-11T23:34:50.849787+0000","last_active":"2024-09-11T23:35:09.456265+0000","last_peered":"2024-09-11T23:35:09.456265+0000","last_clean":"2024-09-11T23:35:09.456265+0000","last_became_active":"2024-09-11T23:34:50.849368+0000","last_became_peered":"2024-09-11T23:34:50.849368+0000","last_unstale":"2024-09-11T23:35:09.456265+0000","last_undegraded":"2024-09-11T23:35:09.456265+0000","last_fullsized":"2024-09-11T23:35:09.456265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:26.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847291+0000","last_change":"2024-09-11T23:34:52.870832+0000","last_active":"2024-09-11T23:35:08.847291+0000","last_peered":"2024-09-11T23:35:08.847291+0000","last_clean":"2024-09-11T23:35:08.847291+0000","last_became_active":"2024-09-11T23:34:52.870474+0000","last_became_peered":"2024-09-11T23:34:52.870474+0000","last_unstale":"2024-09-11T23:35:08.847291+0000","last_undegraded":"2024-09-11T23:35:08.847291+0000","last_fullsized":"2024-09-11T23:35:08.847291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:30.196114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":126,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:53.514937+0000","last_change":"2024-09-11T23:34:50.859074+0000","last_active":"2024-09-11T23:35:53.514937+0000","last_peered":"2024-09-11T23:35:53.514937+0000","last_clean":"2024-09-11T23:35:53.514937+0000","last_became_active":"2024-09-11T23:34:50.858664+0000","last_became_peered":"2024-09-11T23:34:50.858664+0000","last_unstale":"2024-09-11T23:35:53.514937+0000","last_undegraded":"2024-09-11T23:35:53.514937+0000","last_fullsized":"2024-09-11T23:35:53.514937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:04.077813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850853+0000","last_change":"2024-09-11T23:34:52.875901+0000","last_active":"2024-09-11T23:35:08.850853+0000","last_peered":"2024-09-11T23:35:08.850853+0000","last_clean":"2024-09-11T23:35:08.850853+0000","last_became_active":"2024-09-11T23:34:52.875568+0000","last_became_peered":"2024-09-11T23:34:52.875568+0000","last_unstale":"2024-09-11T23:35:08.850853+0000","last_undegraded":"2024-09-11T23:35:08.850853+0000","last_fullsized":"2024-09-11T23:35:08.850853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:54.173548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846938+0000","last_change":"2024-09-11T23:34:50.854957+0000","last_active":"2024-09-11T23:35:08.846938+0000","last_peered":"2024-09-11T23:35:08.846938+0000","last_clean":"2024-09-11T23:35:08.846938+0000","last_became_active":"2024-09-11T23:34:50.854626+0000","last_became_peered":"2024-09-11T23:34:50.854626+0000","last_unstale":"2024-09-11T23:35:08.846938+0000","last_undegraded":"2024-09-11T23:35:08.846938+0000","last_fullsized":"2024-09-11T23:35:08.846938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:53.738966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872227+0000","last_change":"2024-09-11T23:34:52.868850+0000","last_active":"2024-09-11T23:35:08.872227+0000","last_peered":"2024-09-11T23:35:08.872227+0000","last_clean":"2024-09-11T23:35:08.872227+0000","last_became_active":"2024-09-11T23:34:52.868536+0000","last_became_peered":"2024-09-11T23:34:52.868536+0000","last_unstale":"2024-09-11T23:35:08.872227+0000","last_undegraded":"2024-09-11T23:35:08.872227+0000","last_fullsized":"2024-09-11T23:35:08.872227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:11.746821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456837+0000","last_change":"2024-09-11T23:34:50.924819+0000","last_active":"2024-09-11T23:35:09.456837+0000","last_peered":"2024-09-11T23:35:09.456837+0000","last_clean":"2024-09-11T23:35:09.456837+0000","last_became_active":"2024-09-11T23:34:50.924283+0000","last_became_peered":"2024-09-11T23:34:50.924283+0000","last_unstale":"2024-09-11T23:35:09.456837+0000","last_undegraded":"2024-09-11T23:35:09.456837+0000","last_fullsized":"2024-09-11T23:35:09.456837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:41.345233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850516+0000","last_change":"2024-09-11T23:34:52.867113+0000","last_active":"2024-09-11T23:35:08.850516+0000","last_peered":"2024-09-11T23:35:08.850516+0000","last_clean":"2024-09-11T23:35:08.850516+0000","last_became_active":"2024-09-11T23:34:52.866125+0000","last_became_peered":"2024-09-11T23:34:52.866125+0000","last_unstale":"2024-09-11T23:35:08.850516+0000","last_undegraded":"2024-09-11T23:35:08.850516+0000","last_fullsized":"2024-09-11T23:35:08.850516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:18:55.615079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":166,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:53.515072+0000","last_change":"2024-09-11T23:34:50.857272+0000","last_active":"2024-09-11T23:35:53.515072+0000","last_peered":"2024-09-11T23:35:53.515072+0000","last_clean":"2024-09-11T23:35:53.515072+0000","last_became_active":"2024-09-11T23:34:50.855701+0000","last_became_peered":"2024-09-11T23:34:50.855701+0000","last_unstale":"2024-09-11T23:35:53.515072+0000","last_undegraded":"2024-09-11T23:35:53.515072+0000","last_fullsized":"2024-09-11T23:35:53.515072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:49.997617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62310,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":116,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848168+0000","last_change":"2024-09-11T23:34:52.876928+0000","last_active":"2024-09-11T23:35:08.848168+0000","last_peered":"2024-09-11T23:35:08.848168+0000","last_clean":"2024-09-11T23:35:08.848168+0000","last_became_active":"2024-09-11T23:34:52.876578+0000","last_became_peered":"2024-09-11T23:34:52.876578+0000","last_unstale":"2024-09-11T23:35:08.848168+0000","last_undegraded":"2024-09-11T23:35:08.848168+0000","last_fullsized":"2024-09-11T23:35:08.848168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:07.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.870264+0000","last_change":"2024-09-11T23:35:08.869275+0000","last_active":"2024-09-11T23:35:08.870264+0000","last_peered":"2024-09-11T23:35:08.870264+0000","last_clean":"2024-09-11T23:35:08.870264+0000","last_became_active":"2024-09-11T23:35:08.850112+0000","last_became_peered":"2024-09-11T23:35:08.850112+0000","last_unstale":"2024-09-11T23:35:08.870264+0000","last_undegraded":"2024-09-11T23:35:08.870264+0000","last_fullsized":"2024-09-11T23:35:08.870264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:36.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844084+0000","last_change":"2024-09-11T23:34:52.871970+0000","last_active":"2024-09-11T23:35:08.844084+0000","last_peered":"2024-09-11T23:35:08.844084+0000","last_clean":"2024-09-11T23:35:08.844084+0000","last_became_active":"2024-09-11T23:34:52.871488+0000","last_became_peered":"2024-09-11T23:34:52.871488+0000","last_unstale":"2024-09-11T23:35:08.844084+0000","last_undegraded":"2024-09-11T23:35:08.844084+0000","last_fullsized":"2024-09-11T23:35:08.844084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:08.266431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456386+0000","last_change":"2024-09-11T23:34:50.854023+0000","last_active":"2024-09-11T23:35:09.456386+0000","last_peered":"2024-09-11T23:35:09.456386+0000","last_clean":"2024-09-11T23:35:09.456386+0000","last_became_active":"2024-09-11T23:34:50.853661+0000","last_became_peered":"2024-09-11T23:34:50.853661+0000","last_unstale":"2024-09-11T23:35:09.456386+0000","last_undegraded":"2024-09-11T23:35:09.456386+0000","last_fullsized":"2024-09-11T23:35:09.456386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:25.281976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.849648+0000","last_change":"2024-09-11T23:35:08.849648+0000","last_active":"2024-09-11T23:35:08.849648+0000","last_peered":"2024-09-11T23:35:08.849648+0000","last_clean":"2024-09-11T23:35:08.849648+0000","last_became_active":"2024-09-11T23:35:08.849246+0000","last_became_peered":"2024-09-11T23:35:08.849246+0000","last_unstale":"2024-09-11T23:35:08.849648+0000","last_undegraded":"2024-09-11T23:35:08.849648+0000","last_fullsized":"2024-09-11T23:35:08.849648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:17.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":85,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:55.538651+0000","last_change":"2024-09-11T23:34:50.853855+0000","last_active":"2024-09-11T23:35:55.538651+0000","last_peered":"2024-09-11T23:35:55.538651+0000","last_clean":"2024-09-11T23:35:55.538651+0000","last_became_active":"2024-09-11T23:34:50.853493+0000","last_became_peered":"2024-09-11T23:34:50.853493+0000","last_unstale":"2024-09-11T23:35:55.538651+0000","last_undegraded":"2024-09-11T23:35:55.538651+0000","last_fullsized":"2024-09-11T23:35:55.538651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:23.262472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846505+0000","last_change":"2024-09-11T23:34:52.875835+0000","last_active":"2024-09-11T23:35:08.846505+0000","last_peered":"2024-09-11T23:35:08.846505+0000","last_clean":"2024-09-11T23:35:08.846505+0000","last_became_active":"2024-09-11T23:34:52.875525+0000","last_became_peered":"2024-09-11T23:34:52.875525+0000","last_unstale":"2024-09-11T23:35:08.846505+0000","last_undegraded":"2024-09-11T23:35:08.846505+0000","last_fullsized":"2024-09-11T23:35:08.846505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:41.031816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847151+0000","last_change":"2024-09-11T23:34:50.924803+0000","last_active":"2024-09-11T23:35:08.847151+0000","last_peered":"2024-09-11T23:35:08.847151+0000","last_clean":"2024-09-11T23:35:08.847151+0000","last_became_active":"2024-09-11T23:34:50.924217+0000","last_became_peered":"2024-09-11T23:34:50.924217+0000","last_unstale":"2024-09-11T23:35:08.847151+0000","last_undegraded":"2024-09-11T23:35:08.847151+0000","last_fullsized":"2024-09-11T23:35:08.847151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:45.790592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.852839+0000","last_change":"2024-09-11T23:35:08.851856+0000","last_active":"2024-09-11T23:35:08.852839+0000","last_peered":"2024-09-11T23:35:08.852839+0000","last_clean":"2024-09-11T23:35:08.852839+0000","last_became_active":"2024-09-11T23:35:08.851490+0000","last_became_peered":"2024-09-11T23:35:08.851490+0000","last_unstale":"2024-09-11T23:35:08.852839+0000","last_undegraded":"2024-09-11T23:35:08.852839+0000","last_fullsized":"2024-09-11T23:35:08.852839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:46.269022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847890+0000","last_change":"2024-09-11T23:34:50.848385+0000","last_active":"2024-09-11T23:35:08.847890+0000","last_peered":"2024-09-11T23:35:08.847890+0000","last_clean":"2024-09-11T23:35:08.847890+0000","last_became_active":"2024-09-11T23:34:50.847808+0000","last_became_peered":"2024-09-11T23:34:50.847808+0000","last_unstale":"2024-09-11T23:35:08.847890+0000","last_undegraded":"2024-09-11T23:35:08.847890+0000","last_fullsized":"2024-09-11T23:35:08.847890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:38.796622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844817+0000","last_change":"2024-09-11T23:34:52.867341+0000","last_active":"2024-09-11T23:35:08.844817+0000","last_peered":"2024-09-11T23:35:08.844817+0000","last_clean":"2024-09-11T23:35:08.844817+0000","last_became_active":"2024-09-11T23:34:52.866161+0000","last_became_peered":"2024-09-11T23:34:52.866161+0000","last_unstale":"2024-09-11T23:35:08.844817+0000","last_undegraded":"2024-09-11T23:35:08.844817+0000","last_fullsized":"2024-09-11T23:35:08.844817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:27.913705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844372+0000","last_change":"2024-09-11T23:34:50.851466+0000","last_active":"2024-09-11T23:35:08.844372+0000","last_peered":"2024-09-11T23:35:08.844372+0000","last_clean":"2024-09-11T23:35:08.844372+0000","last_became_active":"2024-09-11T23:34:50.851191+0000","last_became_peered":"2024-09-11T23:34:50.851191+0000","last_unstale":"2024-09-11T23:35:08.844372+0000","last_undegraded":"2024-09-11T23:35:08.844372+0000","last_fullsized":"2024-09-11T23:35:08.844372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:00.328658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847008+0000","last_change":"2024-09-11T23:34:52.872290+0000","last_active":"2024-09-11T23:35:08.847008+0000","last_peered":"2024-09-11T23:35:08.847008+0000","last_clean":"2024-09-11T23:35:08.847008+0000","last_became_active":"2024-09-11T23:34:52.871935+0000","last_became_peered":"2024-09-11T23:34:52.871935+0000","last_unstale":"2024-09-11T23:35:08.847008+0000","last_undegraded":"2024-09-11T23:35:08.847008+0000","last_fullsized":"2024-09-11T23:35:08.847008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:08.217548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456729+0000","last_change":"2024-09-11T23:34:50.857763+0000","last_active":"2024-09-11T23:35:09.456729+0000","last_peered":"2024-09-11T23:35:09.456729+0000","last_clean":"2024-09-11T23:35:09.456729+0000","last_became_active":"2024-09-11T23:34:50.857427+0000","last_became_peered":"2024-09-11T23:34:50.857427+0000","last_unstale":"2024-09-11T23:35:09.456729+0000","last_undegraded":"2024-09-11T23:35:09.456729+0000","last_fullsized":"2024-09-11T23:35:09.456729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:59.703580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846908+0000","last_change":"2024-09-11T23:34:52.872426+0000","last_active":"2024-09-11T23:35:08.846908+0000","last_peered":"2024-09-11T23:35:08.846908+0000","last_clean":"2024-09-11T23:35:08.846908+0000","last_became_active":"2024-09-11T23:34:52.872090+0000","last_became_peered":"2024-09-11T23:34:52.872090+0000","last_unstale":"2024-09-11T23:35:08.846908+0000","last_undegraded":"2024-09-11T23:35:08.846908+0000","last_fullsized":"2024-09-11T23:35:08.846908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:04:00.963963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'6","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:55.538350+0000","last_change":"2024-09-11T23:34:50.924792+0000","last_active":"2024-09-11T23:35:55.538350+0000","last_peered":"2024-09-11T23:35:55.538350+0000","last_clean":"2024-09-11T23:35:55.538350+0000","last_became_active":"2024-09-11T23:34:50.924444+0000","last_became_peered":"2024-09-11T23:34:50.924444+0000","last_unstale":"2024-09-11T23:35:55.538350+0000","last_undegraded":"2024-09-11T23:35:55.538350+0000","last_fullsized":"2024-09-11T23:35:55.538350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:12.855314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846881+0000","last_change":"2024-09-11T23:34:52.872079+0000","last_active":"2024-09-11T23:35:08.846881+0000","last_peered":"2024-09-11T23:35:08.846881+0000","last_clean":"2024-09-11T23:35:08.846881+0000","last_became_active":"2024-09-11T23:34:52.871755+0000","last_became_peered":"2024-09-11T23:34:52.871755+0000","last_unstale":"2024-09-11T23:35:08.846881+0000","last_undegraded":"2024-09-11T23:35:08.846881+0000","last_fullsized":"2024-09-11T23:35:08.846881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:05.990832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":344,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846729+0000","last_change":"2024-09-11T23:33:42.256521+0000","last_active":"2024-09-11T23:35:08.846729+0000","last_peered":"2024-09-11T23:35:08.846729+0000","last_clean":"2024-09-11T23:35:08.846729+0000","last_became_active":"2024-09-11T23:33:42.256041+0000","last_became_peered":"2024-09-11T23:33:42.256041+0000","last_unstale":"2024-09-11T23:35:08.846729+0000","last_undegraded":"2024-09-11T23:35:08.846729+0000","last_fullsized":"2024-09-11T23:35:08.846729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_clean_scrub_stamp":"2024-09-11T23:30:22.061515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:21.391839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845844+0000","last_change":"2024-09-11T23:34:50.854087+0000","last_active":"2024-09-11T23:35:08.845844+0000","last_peered":"2024-09-11T23:35:08.845844+0000","last_clean":"2024-09-11T23:35:08.845844+0000","last_became_active":"2024-09-11T23:34:50.852346+0000","last_became_peered":"2024-09-11T23:34:50.852346+0000","last_unstale":"2024-09-11T23:35:08.845844+0000","last_undegraded":"2024-09-11T23:35:08.845844+0000","last_fullsized":"2024-09-11T23:35:08.845844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.287119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847645+0000","last_change":"2024-09-11T23:34:52.869198+0000","last_active":"2024-09-11T23:35:08.847645+0000","last_peered":"2024-09-11T23:35:08.847645+0000","last_clean":"2024-09-11T23:35:08.847645+0000","last_became_active":"2024-09-11T23:34:52.868621+0000","last_became_peered":"2024-09-11T23:34:52.868621+0000","last_unstale":"2024-09-11T23:35:08.847645+0000","last_undegraded":"2024-09-11T23:35:08.847645+0000","last_fullsized":"2024-09-11T23:35:08.847645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:17.430654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846715+0000","last_change":"2024-09-11T23:34:50.852752+0000","last_active":"2024-09-11T23:35:08.846715+0000","last_peered":"2024-09-11T23:35:08.846715+0000","last_clean":"2024-09-11T23:35:08.846715+0000","last_became_active":"2024-09-11T23:34:50.851895+0000","last_became_peered":"2024-09-11T23:34:50.851895+0000","last_unstale":"2024-09-11T23:35:08.846715+0000","last_undegraded":"2024-09-11T23:35:08.846715+0000","last_fullsized":"2024-09-11T23:35:08.846715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:46.894432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844043+0000","last_change":"2024-09-11T23:34:52.866191+0000","last_active":"2024-09-11T23:35:08.844043+0000","last_peered":"2024-09-11T23:35:08.844043+0000","last_clean":"2024-09-11T23:35:08.844043+0000","last_became_active":"2024-09-11T23:34:52.865858+0000","last_became_peered":"2024-09-11T23:34:52.865858+0000","last_unstale":"2024-09-11T23:35:08.844043+0000","last_undegraded":"2024-09-11T23:35:08.844043+0000","last_fullsized":"2024-09-11T23:35:08.844043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:21.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":82,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:55.538109+0000","last_change":"2024-09-11T23:34:50.924125+0000","last_active":"2024-09-11T23:35:55.538109+0000","last_peered":"2024-09-11T23:35:55.538109+0000","last_clean":"2024-09-11T23:35:55.538109+0000","last_became_active":"2024-09-11T23:34:50.923780+0000","last_became_peered":"2024-09-11T23:34:50.923780+0000","last_unstale":"2024-09-11T23:35:55.538109+0000","last_undegraded":"2024-09-11T23:35:55.538109+0000","last_fullsized":"2024-09-11T23:35:55.538109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:39.774827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455096+0000","last_change":"2024-09-11T23:34:52.868364+0000","last_active":"2024-09-11T23:35:09.455096+0000","last_peered":"2024-09-11T23:35:09.455096+0000","last_clean":"2024-09-11T23:35:09.455096+0000","last_became_active":"2024-09-11T23:34:52.867913+0000","last_became_peered":"2024-09-11T23:34:52.867913+0000","last_unstale":"2024-09-11T23:35:09.455096+0000","last_undegraded":"2024-09-11T23:35:09.455096+0000","last_fullsized":"2024-09-11T23:35:09.455096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.571825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:54.523178+0000","last_change":"2024-09-11T23:34:50.852421+0000","last_active":"2024-09-11T23:35:54.523178+0000","last_peered":"2024-09-11T23:35:54.523178+0000","last_clean":"2024-09-11T23:35:54.523178+0000","last_became_active":"2024-09-11T23:34:50.851898+0000","last_became_peered":"2024-09-11T23:34:50.851898+0000","last_unstale":"2024-09-11T23:35:54.523178+0000","last_undegraded":"2024-09-11T23:35:54.523178+0000","last_fullsized":"2024-09-11T23:35:54.523178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.661276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848710+0000","last_change":"2024-09-11T23:34:52.879995+0000","last_active":"2024-09-11T23:35:08.848710+0000","last_peered":"2024-09-11T23:35:08.848710+0000","last_clean":"2024-09-11T23:35:08.848710+0000","last_became_active":"2024-09-11T23:34:52.879678+0000","last_became_peered":"2024-09-11T23:34:52.879678+0000","last_unstale":"2024-09-11T23:35:08.848710+0000","last_undegraded":"2024-09-11T23:35:08.848710+0000","last_fullsized":"2024-09-11T23:35:08.848710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:47.456674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847399+0000","last_change":"2024-09-11T23:34:50.924957+0000","last_active":"2024-09-11T23:35:08.847399+0000","last_peered":"2024-09-11T23:35:08.847399+0000","last_clean":"2024-09-11T23:35:08.847399+0000","last_became_active":"2024-09-11T23:34:50.924500+0000","last_became_peered":"2024-09-11T23:34:50.924500+0000","last_unstale":"2024-09-11T23:35:08.847399+0000","last_undegraded":"2024-09-11T23:35:08.847399+0000","last_fullsized":"2024-09-11T23:35:08.847399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:39.268842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848946+0000","last_change":"2024-09-11T23:34:52.867337+0000","last_active":"2024-09-11T23:35:08.848946+0000","last_peered":"2024-09-11T23:35:08.848946+0000","last_clean":"2024-09-11T23:35:08.848946+0000","last_became_active":"2024-09-11T23:34:52.866799+0000","last_became_peered":"2024-09-11T23:34:52.866799+0000","last_unstale":"2024-09-11T23:35:08.848946+0000","last_undegraded":"2024-09-11T23:35:08.848946+0000","last_fullsized":"2024-09-11T23:35:08.848946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:38.177135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848411+0000","last_change":"2024-09-11T23:34:50.925156+0000","last_active":"2024-09-11T23:35:08.848411+0000","last_peered":"2024-09-11T23:35:08.848411+0000","last_clean":"2024-09-11T23:35:08.848411+0000","last_became_active":"2024-09-11T23:34:50.924282+0000","last_became_peered":"2024-09-11T23:34:50.924282+0000","last_unstale":"2024-09-11T23:35:08.848411+0000","last_undegraded":"2024-09-11T23:35:08.848411+0000","last_fullsized":"2024-09-11T23:35:08.848411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:47.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844337+0000","last_change":"2024-09-11T23:34:52.864993+0000","last_active":"2024-09-11T23:35:08.844337+0000","last_peered":"2024-09-11T23:35:08.844337+0000","last_clean":"2024-09-11T23:35:08.844337+0000","last_became_active":"2024-09-11T23:34:52.864267+0000","last_became_peered":"2024-09-11T23:34:52.864267+0000","last_unstale":"2024-09-11T23:35:08.844337+0000","last_undegraded":"2024-09-11T23:35:08.844337+0000","last_fullsized":"2024-09-11T23:35:08.844337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:29.912068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:17.449902+0000","last_change":"2024-09-11T23:34:50.849864+0000","last_active":"2024-09-11T23:35:17.449902+0000","last_peered":"2024-09-11T23:35:17.449902+0000","last_clean":"2024-09-11T23:35:17.449902+0000","last_became_active":"2024-09-11T23:34:50.849577+0000","last_became_peered":"2024-09-11T23:34:50.849577+0000","last_unstale":"2024-09-11T23:35:17.449902+0000","last_undegraded":"2024-09-11T23:35:17.449902+0000","last_fullsized":"2024-09-11T23:35:17.449902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:06.845000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848578+0000","last_change":"2024-09-11T23:34:52.872028+0000","last_active":"2024-09-11T23:35:08.848578+0000","last_peered":"2024-09-11T23:35:08.848578+0000","last_clean":"2024-09-11T23:35:08.848578+0000","last_became_active":"2024-09-11T23:34:52.871723+0000","last_became_peered":"2024-09-11T23:34:52.871723+0000","last_unstale":"2024-09-11T23:35:08.848578+0000","last_undegraded":"2024-09-11T23:35:08.848578+0000","last_fullsized":"2024-09-11T23:35:08.848578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:44.656218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846269+0000","last_change":"2024-09-11T23:34:50.860314+0000","last_active":"2024-09-11T23:35:08.846269+0000","last_peered":"2024-09-11T23:35:08.846269+0000","last_clean":"2024-09-11T23:35:08.846269+0000","last_became_active":"2024-09-11T23:34:50.859953+0000","last_became_peered":"2024-09-11T23:34:50.859953+0000","last_unstale":"2024-09-11T23:35:08.846269+0000","last_undegraded":"2024-09-11T23:35:08.846269+0000","last_fullsized":"2024-09-11T23:35:08.846269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:46.705864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872092+0000","last_change":"2024-09-11T23:34:52.867190+0000","last_active":"2024-09-11T23:35:08.872092+0000","last_peered":"2024-09-11T23:35:08.872092+0000","last_clean":"2024-09-11T23:35:08.872092+0000","last_became_active":"2024-09-11T23:34:52.866829+0000","last_became_peered":"2024-09-11T23:34:52.866829+0000","last_unstale":"2024-09-11T23:35:08.872092+0000","last_undegraded":"2024-09-11T23:35:08.872092+0000","last_fullsized":"2024-09-11T23:35:08.872092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.752006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850343+0000","last_change":"2024-09-11T23:34:50.924566+0000","last_active":"2024-09-11T23:35:08.850343+0000","last_peered":"2024-09-11T23:35:08.850343+0000","last_clean":"2024-09-11T23:35:08.850343+0000","last_became_active":"2024-09-11T23:34:50.924284+0000","last_became_peered":"2024-09-11T23:34:50.924284+0000","last_unstale":"2024-09-11T23:35:08.850343+0000","last_undegraded":"2024-09-11T23:35:08.850343+0000","last_fullsized":"2024-09-11T23:35:08.850343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:54.246154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847042+0000","last_change":"2024-09-11T23:34:52.869302+0000","last_active":"2024-09-11T23:35:08.847042+0000","last_peered":"2024-09-11T23:35:08.847042+0000","last_clean":"2024-09-11T23:35:08.847042+0000","last_became_active":"2024-09-11T23:34:52.868960+0000","last_became_peered":"2024-09-11T23:34:52.868960+0000","last_unstale":"2024-09-11T23:35:08.847042+0000","last_undegraded":"2024-09-11T23:35:08.847042+0000","last_fullsized":"2024-09-11T23:35:08.847042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.561712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847953+0000","last_change":"2024-09-11T23:34:50.854573+0000","last_active":"2024-09-11T23:35:08.847953+0000","last_peered":"2024-09-11T23:35:08.847953+0000","last_clean":"2024-09-11T23:35:08.847953+0000","last_became_active":"2024-09-11T23:34:50.854165+0000","last_became_peered":"2024-09-11T23:34:50.854165+0000","last_unstale":"2024-09-11T23:35:08.847953+0000","last_undegraded":"2024-09-11T23:35:08.847953+0000","last_fullsized":"2024-09-11T23:35:08.847953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:57:31.589495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844752+0000","last_change":"2024-09-11T23:34:52.875987+0000","last_active":"2024-09-11T23:35:08.844752+0000","last_peered":"2024-09-11T23:35:08.844752+0000","last_clean":"2024-09-11T23:35:08.844752+0000","last_became_active":"2024-09-11T23:34:52.875662+0000","last_became_peered":"2024-09-11T23:34:52.875662+0000","last_unstale":"2024-09-11T23:35:08.844752+0000","last_undegraded":"2024-09-11T23:35:08.844752+0000","last_fullsized":"2024-09-11T23:35:08.844752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:16.514938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847766+0000","last_change":"2024-09-11T23:34:50.850726+0000","last_active":"2024-09-11T23:35:08.847766+0000","last_peered":"2024-09-11T23:35:08.847766+0000","last_clean":"2024-09-11T23:35:08.847766+0000","last_became_active":"2024-09-11T23:34:50.850381+0000","last_became_peered":"2024-09-11T23:34:50.850381+0000","last_unstale":"2024-09-11T23:35:08.847766+0000","last_undegraded":"2024-09-11T23:35:08.847766+0000","last_fullsized":"2024-09-11T23:35:08.847766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:50.431293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847564+0000","last_change":"2024-09-11T23:34:52.870652+0000","last_active":"2024-09-11T23:35:08.847564+0000","last_peered":"2024-09-11T23:35:08.847564+0000","last_clean":"2024-09-11T23:35:08.847564+0000","last_became_active":"2024-09-11T23:34:52.870303+0000","last_became_peered":"2024-09-11T23:34:52.870303+0000","last_unstale":"2024-09-11T23:35:08.847564+0000","last_undegraded":"2024-09-11T23:35:08.847564+0000","last_fullsized":"2024-09-11T23:35:08.847564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:17.670981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.855045+0000","last_change":"2024-09-11T23:35:08.854490+0000","last_active":"2024-09-11T23:35:08.855045+0000","last_peered":"2024-09-11T23:35:08.855045+0000","last_clean":"2024-09-11T23:35:08.855045+0000","last_became_active":"2024-09-11T23:35:08.849820+0000","last_became_peered":"2024-09-11T23:35:08.849820+0000","last_unstale":"2024-09-11T23:35:08.855045+0000","last_undegraded":"2024-09-11T23:35:08.855045+0000","last_fullsized":"2024-09-11T23:35:08.855045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:09.509867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846681+0000","last_change":"2024-09-11T23:34:52.869695+0000","last_active":"2024-09-11T23:35:08.846681+0000","last_peered":"2024-09-11T23:35:08.846681+0000","last_clean":"2024-09-11T23:35:08.846681+0000","last_became_active":"2024-09-11T23:34:52.869367+0000","last_became_peered":"2024-09-11T23:34:52.869367+0000","last_unstale":"2024-09-11T23:35:08.846681+0000","last_undegraded":"2024-09-11T23:35:08.846681+0000","last_fullsized":"2024-09-11T23:35:08.846681+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:16.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455461+0000","last_change":"2024-09-11T23:34:52.872778+0000","last_active":"2024-09-11T23:35:09.455461+0000","last_peered":"2024-09-11T23:35:09.455461+0000","last_clean":"2024-09-11T23:35:09.455461+0000","last_became_active":"2024-09-11T23:34:52.872451+0000","last_became_peered":"2024-09-11T23:34:52.872451+0000","last_unstale":"2024-09-11T23:35:09.455461+0000","last_undegraded":"2024-09-11T23:35:09.455461+0000","last_fullsized":"2024-09-11T23:35:09.455461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:51.769927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847892+0000","last_change":"2024-09-11T23:34:50.849675+0000","last_active":"2024-09-11T23:35:08.847892+0000","last_peered":"2024-09-11T23:35:08.847892+0000","last_clean":"2024-09-11T23:35:08.847892+0000","last_became_active":"2024-09-11T23:34:50.849228+0000","last_became_peered":"2024-09-11T23:34:50.849228+0000","last_unstale":"2024-09-11T23:35:08.847892+0000","last_undegraded":"2024-09-11T23:35:08.847892+0000","last_fullsized":"2024-09-11T23:35:08.847892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:50.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850319+0000","last_change":"2024-09-11T23:34:52.867739+0000","last_active":"2024-09-11T23:35:08.850319+0000","last_peered":"2024-09-11T23:35:08.850319+0000","last_clean":"2024-09-11T23:35:08.850319+0000","last_became_active":"2024-09-11T23:34:52.866613+0000","last_became_peered":"2024-09-11T23:34:52.866613+0000","last_unstale":"2024-09-11T23:35:08.850319+0000","last_undegraded":"2024-09-11T23:35:08.850319+0000","last_fullsized":"2024-09-11T23:35:08.850319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:49.348386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848269+0000","last_change":"2024-09-11T23:34:50.851401+0000","last_active":"2024-09-11T23:35:08.848269+0000","last_peered":"2024-09-11T23:35:08.848269+0000","last_clean":"2024-09-11T23:35:08.848269+0000","last_became_active":"2024-09-11T23:34:50.851068+0000","last_became_peered":"2024-09-11T23:34:50.851068+0000","last_unstale":"2024-09-11T23:35:08.848269+0000","last_undegraded":"2024-09-11T23:35:08.848269+0000","last_fullsized":"2024-09-11T23:35:08.848269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:09.673864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872365+0000","last_change":"2024-09-11T23:34:52.865216+0000","last_active":"2024-09-11T23:35:08.872365+0000","last_peered":"2024-09-11T23:35:08.872365+0000","last_clean":"2024-09-11T23:35:08.872365+0000","last_became_active":"2024-09-11T23:34:52.864917+0000","last_became_peered":"2024-09-11T23:34:52.864917+0000","last_unstale":"2024-09-11T23:35:08.872365+0000","last_undegraded":"2024-09-11T23:35:08.872365+0000","last_fullsized":"2024-09-11T23:35:08.872365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:36.469793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848560+0000","last_change":"2024-09-11T23:34:50.924291+0000","last_active":"2024-09-11T23:35:08.848560+0000","last_peered":"2024-09-11T23:35:08.848560+0000","last_clean":"2024-09-11T23:35:08.848560+0000","last_became_active":"2024-09-11T23:34:50.923892+0000","last_became_peered":"2024-09-11T23:34:50.923892+0000","last_unstale":"2024-09-11T23:35:08.848560+0000","last_undegraded":"2024-09-11T23:35:08.848560+0000","last_fullsized":"2024-09-11T23:35:08.848560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:52.266778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848164+0000","last_change":"2024-09-11T23:34:52.875995+0000","last_active":"2024-09-11T23:35:08.848164+0000","last_peered":"2024-09-11T23:35:08.848164+0000","last_clean":"2024-09-11T23:35:08.848164+0000","last_became_active":"2024-09-11T23:34:52.875654+0000","last_became_peered":"2024-09-11T23:34:52.875654+0000","last_unstale":"2024-09-11T23:35:08.848164+0000","last_undegraded":"2024-09-11T23:35:08.848164+0000","last_fullsized":"2024-09-11T23:35:08.848164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:19.218291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":87,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:54.522570+0000","last_change":"2024-09-11T23:34:50.853881+0000","last_active":"2024-09-11T23:35:54.522570+0000","last_peered":"2024-09-11T23:35:54.522570+0000","last_clean":"2024-09-11T23:35:54.522570+0000","last_became_active":"2024-09-11T23:34:50.853576+0000","last_became_peered":"2024-09-11T23:34:50.853576+0000","last_unstale":"2024-09-11T23:35:54.522570+0000","last_undegraded":"2024-09-11T23:35:54.522570+0000","last_fullsized":"2024-09-11T23:35:54.522570+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:38.550587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69518,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":557,"num_read_kb":351,"num_write":247,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":208554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612634,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":434822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841566,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1597440,"data_stored":1088093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005092,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":434291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":434669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495662,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659186,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":901120,"data_stored":434008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888119,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1026474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018941,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":434262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247872,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":433833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444103,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":497776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607627,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":497231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":921600,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:35:59.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:35:59.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:35:59 smithi138 ceph-mon[35142]: pgmap v306: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:35:59.679 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:00.111 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:00 smithi133 ceph-mon[31430]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:00.111 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:36:00.111 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:36:00.111 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:00.112 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:00.112 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:00 smithi133 ceph-mon[31430]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:36:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.110+0000 7fd894aaa640 1 -- 172.21.15.133:0/3466100177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890106f10 msgr2=0x7fd89010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.110+0000 7fd894aaa640 1 --2- 172.21.15.133:0/3466100177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890106f10 0x7fd89010ea90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd888009900 tx=0x7fd88802a410 comp rx=0 tx=0).stop 2024-09-11T23:36:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.110+0000 7fd894aaa640 1 -- 172.21.15.133:0/3466100177 shutdown_connections 2024-09-11T23:36:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.110+0000 7fd894aaa640 1 --2- 172.21.15.133:0/3466100177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890106f10 0x7fd89010ea90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.110+0000 7fd894aaa640 1 --2- 172.21.15.133:0/3466100177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd890102790 0x7fd8901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.110+0000 7fd894aaa640 1 --2- 172.21.15.133:0/3466100177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd890101de0 0x7fd8901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.110+0000 7fd894aaa640 1 -- 172.21.15.133:0/3466100177 >> 172.21.15.133:0/3466100177 conn(0x7fd8900fd8c0 msgr2=0x7fd8900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.110+0000 7fd894aaa640 1 -- 172.21.15.133:0/3466100177 shutdown_connections 2024-09-11T23:36:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.110+0000 7fd894aaa640 1 -- 172.21.15.133:0/3466100177 wait complete. 2024-09-11T23:36:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.111+0000 7fd894aaa640 1 Processor -- start 2024-09-11T23:36:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.111+0000 7fd894aaa640 1 -- start start 2024-09-11T23:36:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.111+0000 7fd894aaa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd890101de0 0x7fd8900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.111+0000 7fd894aaa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd890102790 0x7fd890074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.111+0000 7fd894aaa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890106f10 0x7fd890074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.111+0000 7fd894aaa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd890111f70 con 0x7fd890106f10 2024-09-11T23:36:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.111+0000 7fd894aaa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd890111df0 con 0x7fd890101de0 2024-09-11T23:36:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.111+0000 7fd894aaa640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901120f0 con 0x7fd890102790 2024-09-11T23:36:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890106f10 0x7fd890074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890106f10 0x7fd890074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48820/0 (socket says 172.21.15.133:48820) 2024-09-11T23:36:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88ed9d640 1 -- 172.21.15.133:0/3678874527 learned_addr learned my addr 172.21.15.133:0/3678874527 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88e59c640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd890101de0 0x7fd8900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88dd9b640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd890102790 0x7fd890074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88ed9d640 1 -- 172.21.15.133:0/3678874527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd890102790 msgr2=0x7fd890074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88ed9d640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd890102790 0x7fd890074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88ed9d640 1 -- 172.21.15.133:0/3678874527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd890101de0 msgr2=0x7fd8900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88ed9d640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd890101de0 0x7fd8900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88ed9d640 1 -- 172.21.15.133:0/3678874527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8901a68b0 con 0x7fd890106f10 2024-09-11T23:36:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88e59c640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd890101de0 0x7fd8900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd88ed9d640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890106f10 0x7fd890074f70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd888005ab0 tx=0x7fd888004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd87f7fe640 1 -- 172.21.15.133:0/3678874527 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88803f030 con 0x7fd890106f10 2024-09-11T23:36:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd894aaa640 1 -- 172.21.15.133:0/3678874527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8901a6b40 con 0x7fd890106f10 2024-09-11T23:36:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd87f7fe640 1 -- 172.21.15.133:0/3678874527 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd888003ad0 con 0x7fd890106f10 2024-09-11T23:36:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd87f7fe640 1 -- 172.21.15.133:0/3678874527 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888035070 con 0x7fd890106f10 2024-09-11T23:36:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.112+0000 7fd894aaa640 1 -- 172.21.15.133:0/3678874527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8901a7070 con 0x7fd890106f10 2024-09-11T23:36:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.113+0000 7fd894aaa640 1 -- 172.21.15.133:0/3678874527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8901021c0 con 0x7fd890106f10 2024-09-11T23:36:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.114+0000 7fd87f7fe640 1 -- 172.21.15.133:0/3678874527 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd888042030 con 0x7fd890106f10 2024-09-11T23:36:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.115+0000 7fd87f7fe640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd8780a2c10 0x7fd8780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.115+0000 7fd88e59c640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd8780a2c10 0x7fd8780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.115+0000 7fd87f7fe640 1 -- 172.21.15.133:0/3678874527 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fd8880efdb0 con 0x7fd890106f10 2024-09-11T23:36:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.116+0000 7fd88e59c640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd8780a2c10 0x7fd8780a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd880009c20 tx=0x7fd880005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.118+0000 7fd87f7fe640 1 -- 172.21.15.133:0/3678874527 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8880bccb0 con 0x7fd890106f10 2024-09-11T23:36:00.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.281+0000 7fd894aaa640 1 -- 172.21.15.133:0/3678874527 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd890062630 con 0x7fd8780a2c10 2024-09-11T23:36:00.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.287+0000 7fd87f7fe640 1 -- 172.21.15.133:0/3678874527 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258760 (secure 0 0 0) 0x7fd890062630 con 0x7fd8780a2c10 2024-09-11T23:36:00.290 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:00.291 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T23:36:00.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.290+0000 7fd894aaa640 1 -- 172.21.15.133:0/3678874527 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd8780a2c10 msgr2=0x7fd8780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:00.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.290+0000 7fd894aaa640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd8780a2c10 0x7fd8780a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd880009c20 tx=0x7fd880005f90 comp rx=0 tx=0).stop 2024-09-11T23:36:00.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.290+0000 7fd894aaa640 1 -- 172.21.15.133:0/3678874527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890106f10 msgr2=0x7fd890074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:00.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.290+0000 7fd894aaa640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890106f10 0x7fd890074f70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd888005ab0 tx=0x7fd888004270 comp rx=0 tx=0).stop 2024-09-11T23:36:00.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.291+0000 7fd894aaa640 1 -- 172.21.15.133:0/3678874527 shutdown_connections 2024-09-11T23:36:00.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.291+0000 7fd894aaa640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fd8780a2c10 0x7fd8780a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.291+0000 7fd894aaa640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890106f10 0x7fd890074f70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.291+0000 7fd894aaa640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd890102790 0x7fd890074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.291+0000 7fd894aaa640 1 --2- 172.21.15.133:0/3678874527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd890101de0 0x7fd8900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:00.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.291+0000 7fd894aaa640 1 -- 172.21.15.133:0/3678874527 >> 172.21.15.133:0/3678874527 conn(0x7fd8900fd8c0 msgr2=0x7fd8900feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:00.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.291+0000 7fd894aaa640 1 -- 172.21.15.133:0/3678874527 shutdown_connections 2024-09-11T23:36:00.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:00.291+0000 7fd894aaa640 1 -- 172.21.15.133:0/3678874527 wait complete. 2024-09-11T23:36:00.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:00 smithi178 ceph-mon[35882]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:00.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:36:00.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:36:00.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:00.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:00.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:00 smithi178 ceph-mon[35882]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:36:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:00 smithi138 ceph-mon[35142]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:36:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:36:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:00.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:00 smithi138 ceph-mon[35142]: from='mgr.14150 172.21.15.133:0/3533435496' entity='mgr.x' 2024-09-11T23:36:00.953 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-11T23:35:59.860900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660120,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":713,"num_read_kb":485,"num_write":430,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357760,"kb_used_data":12648,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584064,"statfs":{"total":1151940427776,"available":1151574081536,"internally_reserved":0,"allocated":12951552,"data_stored":7178388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003306"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845014+0000","last_change":"2024-09-11T23:34:52.867694+0000","last_active":"2024-09-11T23:35:08.845014+0000","last_peered":"2024-09-11T23:35:08.845014+0000","last_clean":"2024-09-11T23:35:08.845014+0000","last_became_active":"2024-09-11T23:34:52.867368+0000","last_became_peered":"2024-09-11T23:34:52.867368+0000","last_unstale":"2024-09-11T23:35:08.845014+0000","last_undegraded":"2024-09-11T23:35:08.845014+0000","last_fullsized":"2024-09-11T23:35:08.845014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:46:07.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872241+0000","last_change":"2024-09-11T23:34:52.871881+0000","last_active":"2024-09-11T23:35:08.872241+0000","last_peered":"2024-09-11T23:35:08.872241+0000","last_clean":"2024-09-11T23:35:08.872241+0000","last_became_active":"2024-09-11T23:34:52.871609+0000","last_became_peered":"2024-09-11T23:34:52.871609+0000","last_unstale":"2024-09-11T23:35:08.872241+0000","last_undegraded":"2024-09-11T23:35:08.872241+0000","last_fullsized":"2024-09-11T23:35:08.872241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.938584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872054+0000","last_change":"2024-09-11T23:34:52.863931+0000","last_active":"2024-09-11T23:35:08.872054+0000","last_peered":"2024-09-11T23:35:08.872054+0000","last_clean":"2024-09-11T23:35:08.872054+0000","last_became_active":"2024-09-11T23:34:52.863637+0000","last_became_peered":"2024-09-11T23:34:52.863637+0000","last_unstale":"2024-09-11T23:35:08.872054+0000","last_undegraded":"2024-09-11T23:35:08.872054+0000","last_fullsized":"2024-09-11T23:35:08.872054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:44.118865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846293+0000","last_change":"2024-09-11T23:34:52.879619+0000","last_active":"2024-09-11T23:35:08.846293+0000","last_peered":"2024-09-11T23:35:08.846293+0000","last_clean":"2024-09-11T23:35:08.846293+0000","last_became_active":"2024-09-11T23:34:52.878695+0000","last_became_peered":"2024-09-11T23:34:52.878695+0000","last_unstale":"2024-09-11T23:35:08.846293+0000","last_undegraded":"2024-09-11T23:35:08.846293+0000","last_fullsized":"2024-09-11T23:35:08.846293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:30.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847368+0000","last_change":"2024-09-11T23:34:52.869363+0000","last_active":"2024-09-11T23:35:08.847368+0000","last_peered":"2024-09-11T23:35:08.847368+0000","last_clean":"2024-09-11T23:35:08.847368+0000","last_became_active":"2024-09-11T23:34:52.869027+0000","last_became_peered":"2024-09-11T23:34:52.869027+0000","last_unstale":"2024-09-11T23:35:08.847368+0000","last_undegraded":"2024-09-11T23:35:08.847368+0000","last_fullsized":"2024-09-11T23:35:08.847368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:15.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848328+0000","last_change":"2024-09-11T23:34:52.880665+0000","last_active":"2024-09-11T23:35:08.848328+0000","last_peered":"2024-09-11T23:35:08.848328+0000","last_clean":"2024-09-11T23:35:08.848328+0000","last_became_active":"2024-09-11T23:34:52.880172+0000","last_became_peered":"2024-09-11T23:34:52.880172+0000","last_unstale":"2024-09-11T23:35:08.848328+0000","last_undegraded":"2024-09-11T23:35:08.848328+0000","last_fullsized":"2024-09-11T23:35:08.848328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:12.025877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846502+0000","last_change":"2024-09-11T23:34:52.873897+0000","last_active":"2024-09-11T23:35:08.846502+0000","last_peered":"2024-09-11T23:35:08.846502+0000","last_clean":"2024-09-11T23:35:08.846502+0000","last_became_active":"2024-09-11T23:34:52.873219+0000","last_became_peered":"2024-09-11T23:34:52.873219+0000","last_unstale":"2024-09-11T23:35:08.846502+0000","last_undegraded":"2024-09-11T23:35:08.846502+0000","last_fullsized":"2024-09-11T23:35:08.846502+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:47.125682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847314+0000","last_change":"2024-09-11T23:34:52.870699+0000","last_active":"2024-09-11T23:35:08.847314+0000","last_peered":"2024-09-11T23:35:08.847314+0000","last_clean":"2024-09-11T23:35:08.847314+0000","last_became_active":"2024-09-11T23:34:52.870314+0000","last_became_peered":"2024-09-11T23:34:52.870314+0000","last_unstale":"2024-09-11T23:35:08.847314+0000","last_undegraded":"2024-09-11T23:35:08.847314+0000","last_fullsized":"2024-09-11T23:35:08.847314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:41.263159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.851159+0000","last_change":"2024-09-11T23:34:52.869700+0000","last_active":"2024-09-11T23:35:08.851159+0000","last_peered":"2024-09-11T23:35:08.851159+0000","last_clean":"2024-09-11T23:35:08.851159+0000","last_became_active":"2024-09-11T23:34:52.869398+0000","last_became_peered":"2024-09-11T23:34:52.869398+0000","last_unstale":"2024-09-11T23:35:08.851159+0000","last_undegraded":"2024-09-11T23:35:08.851159+0000","last_fullsized":"2024-09-11T23:35:08.851159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:49.776525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848589+0000","last_change":"2024-09-11T23:34:52.879999+0000","last_active":"2024-09-11T23:35:08.848589+0000","last_peered":"2024-09-11T23:35:08.848589+0000","last_clean":"2024-09-11T23:35:08.848589+0000","last_became_active":"2024-09-11T23:34:52.879646+0000","last_became_peered":"2024-09-11T23:34:52.879646+0000","last_unstale":"2024-09-11T23:35:08.848589+0000","last_undegraded":"2024-09-11T23:35:08.848589+0000","last_fullsized":"2024-09-11T23:35:08.848589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.058704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846881+0000","last_change":"2024-09-11T23:34:52.878740+0000","last_active":"2024-09-11T23:35:08.846881+0000","last_peered":"2024-09-11T23:35:08.846881+0000","last_clean":"2024-09-11T23:35:08.846881+0000","last_became_active":"2024-09-11T23:34:52.878120+0000","last_became_peered":"2024-09-11T23:34:52.878120+0000","last_unstale":"2024-09-11T23:35:08.846881+0000","last_undegraded":"2024-09-11T23:35:08.846881+0000","last_fullsized":"2024-09-11T23:35:08.846881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846051+0000","last_change":"2024-09-11T23:34:52.873409+0000","last_active":"2024-09-11T23:35:08.846051+0000","last_peered":"2024-09-11T23:35:08.846051+0000","last_clean":"2024-09-11T23:35:08.846051+0000","last_became_active":"2024-09-11T23:34:52.872959+0000","last_became_peered":"2024-09-11T23:34:52.872959+0000","last_unstale":"2024-09-11T23:35:08.846051+0000","last_undegraded":"2024-09-11T23:35:08.846051+0000","last_fullsized":"2024-09-11T23:35:08.846051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:38.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844266+0000","last_change":"2024-09-11T23:34:52.869164+0000","last_active":"2024-09-11T23:35:08.844266+0000","last_peered":"2024-09-11T23:35:08.844266+0000","last_clean":"2024-09-11T23:35:08.844266+0000","last_became_active":"2024-09-11T23:34:52.868847+0000","last_became_peered":"2024-09-11T23:34:52.868847+0000","last_unstale":"2024-09-11T23:35:08.844266+0000","last_undegraded":"2024-09-11T23:35:08.844266+0000","last_fullsized":"2024-09-11T23:35:08.844266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.764510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872027+0000","last_change":"2024-09-11T23:34:52.869494+0000","last_active":"2024-09-11T23:35:08.872027+0000","last_peered":"2024-09-11T23:35:08.872027+0000","last_clean":"2024-09-11T23:35:08.872027+0000","last_became_active":"2024-09-11T23:34:52.869178+0000","last_became_peered":"2024-09-11T23:34:52.869178+0000","last_unstale":"2024-09-11T23:35:08.872027+0000","last_undegraded":"2024-09-11T23:35:08.872027+0000","last_fullsized":"2024-09-11T23:35:08.872027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:23.431564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872033+0000","last_change":"2024-09-11T23:34:52.866269+0000","last_active":"2024-09-11T23:35:08.872033+0000","last_peered":"2024-09-11T23:35:08.872033+0000","last_clean":"2024-09-11T23:35:08.872033+0000","last_became_active":"2024-09-11T23:34:52.865980+0000","last_became_peered":"2024-09-11T23:34:52.865980+0000","last_unstale":"2024-09-11T23:35:08.872033+0000","last_undegraded":"2024-09-11T23:35:08.872033+0000","last_fullsized":"2024-09-11T23:35:08.872033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:48.492252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850466+0000","last_change":"2024-09-11T23:34:52.862192+0000","last_active":"2024-09-11T23:35:08.850466+0000","last_peered":"2024-09-11T23:35:08.850466+0000","last_clean":"2024-09-11T23:35:08.850466+0000","last_became_active":"2024-09-11T23:34:52.861841+0000","last_became_peered":"2024-09-11T23:34:52.861841+0000","last_unstale":"2024-09-11T23:35:08.850466+0000","last_undegraded":"2024-09-11T23:35:08.850466+0000","last_fullsized":"2024-09-11T23:35:08.850466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.258948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845097+0000","last_change":"2024-09-11T23:34:52.870984+0000","last_active":"2024-09-11T23:35:08.845097+0000","last_peered":"2024-09-11T23:35:08.845097+0000","last_clean":"2024-09-11T23:35:08.845097+0000","last_became_active":"2024-09-11T23:34:52.870435+0000","last_became_peered":"2024-09-11T23:34:52.870435+0000","last_unstale":"2024-09-11T23:35:08.845097+0000","last_undegraded":"2024-09-11T23:35:08.845097+0000","last_fullsized":"2024-09-11T23:35:08.845097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:34.391537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847466+0000","last_change":"2024-09-11T23:34:52.873176+0000","last_active":"2024-09-11T23:35:08.847466+0000","last_peered":"2024-09-11T23:35:08.847466+0000","last_clean":"2024-09-11T23:35:08.847466+0000","last_became_active":"2024-09-11T23:34:52.872862+0000","last_became_peered":"2024-09-11T23:34:52.872862+0000","last_unstale":"2024-09-11T23:35:08.847466+0000","last_undegraded":"2024-09-11T23:35:08.847466+0000","last_fullsized":"2024-09-11T23:35:08.847466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:38.058806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847715+0000","last_change":"2024-09-11T23:34:52.871346+0000","last_active":"2024-09-11T23:35:08.847715+0000","last_peered":"2024-09-11T23:35:08.847715+0000","last_clean":"2024-09-11T23:35:08.847715+0000","last_became_active":"2024-09-11T23:34:52.870981+0000","last_became_peered":"2024-09-11T23:34:52.870981+0000","last_unstale":"2024-09-11T23:35:08.847715+0000","last_undegraded":"2024-09-11T23:35:08.847715+0000","last_fullsized":"2024-09-11T23:35:08.847715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:56.973998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844737+0000","last_change":"2024-09-11T23:34:52.867058+0000","last_active":"2024-09-11T23:35:08.844737+0000","last_peered":"2024-09-11T23:35:08.844737+0000","last_clean":"2024-09-11T23:35:08.844737+0000","last_became_active":"2024-09-11T23:34:52.866687+0000","last_became_peered":"2024-09-11T23:34:52.866687+0000","last_unstale":"2024-09-11T23:35:08.844737+0000","last_undegraded":"2024-09-11T23:35:08.844737+0000","last_fullsized":"2024-09-11T23:35:08.844737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:09.019987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844347+0000","last_change":"2024-09-11T23:34:52.870882+0000","last_active":"2024-09-11T23:35:08.844347+0000","last_peered":"2024-09-11T23:35:08.844347+0000","last_clean":"2024-09-11T23:35:08.844347+0000","last_became_active":"2024-09-11T23:34:52.870667+0000","last_became_peered":"2024-09-11T23:34:52.870667+0000","last_unstale":"2024-09-11T23:35:08.844347+0000","last_undegraded":"2024-09-11T23:35:08.844347+0000","last_fullsized":"2024-09-11T23:35:08.844347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:08.386703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850359+0000","last_change":"2024-09-11T23:34:52.874070+0000","last_active":"2024-09-11T23:35:08.850359+0000","last_peered":"2024-09-11T23:35:08.850359+0000","last_clean":"2024-09-11T23:35:08.850359+0000","last_became_active":"2024-09-11T23:34:52.873772+0000","last_became_peered":"2024-09-11T23:34:52.873772+0000","last_unstale":"2024-09-11T23:35:08.850359+0000","last_undegraded":"2024-09-11T23:35:08.850359+0000","last_fullsized":"2024-09-11T23:35:08.850359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:49.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847766+0000","last_change":"2024-09-11T23:34:52.875502+0000","last_active":"2024-09-11T23:35:08.847766+0000","last_peered":"2024-09-11T23:35:08.847766+0000","last_clean":"2024-09-11T23:35:08.847766+0000","last_became_active":"2024-09-11T23:34:52.875193+0000","last_became_peered":"2024-09-11T23:34:52.875193+0000","last_unstale":"2024-09-11T23:35:08.847766+0000","last_undegraded":"2024-09-11T23:35:08.847766+0000","last_fullsized":"2024-09-11T23:35:08.847766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:37.213828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455450+0000","last_change":"2024-09-11T23:34:52.869015+0000","last_active":"2024-09-11T23:35:09.455450+0000","last_peered":"2024-09-11T23:35:09.455450+0000","last_clean":"2024-09-11T23:35:09.455450+0000","last_became_active":"2024-09-11T23:34:52.868731+0000","last_became_peered":"2024-09-11T23:34:52.868731+0000","last_unstale":"2024-09-11T23:35:09.455450+0000","last_undegraded":"2024-09-11T23:35:09.455450+0000","last_fullsized":"2024-09-11T23:35:09.455450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:12:15.993203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848557+0000","last_change":"2024-09-11T23:34:52.875555+0000","last_active":"2024-09-11T23:35:08.848557+0000","last_peered":"2024-09-11T23:35:08.848557+0000","last_clean":"2024-09-11T23:35:08.848557+0000","last_became_active":"2024-09-11T23:34:52.875219+0000","last_became_peered":"2024-09-11T23:34:52.875219+0000","last_unstale":"2024-09-11T23:35:08.848557+0000","last_undegraded":"2024-09-11T23:35:08.848557+0000","last_fullsized":"2024-09-11T23:35:08.848557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:16.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848205+0000","last_change":"2024-09-11T23:34:52.875629+0000","last_active":"2024-09-11T23:35:08.848205+0000","last_peered":"2024-09-11T23:35:08.848205+0000","last_clean":"2024-09-11T23:35:08.848205+0000","last_became_active":"2024-09-11T23:34:52.875302+0000","last_became_peered":"2024-09-11T23:34:52.875302+0000","last_unstale":"2024-09-11T23:35:08.848205+0000","last_undegraded":"2024-09-11T23:35:08.848205+0000","last_fullsized":"2024-09-11T23:35:08.848205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:12.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844886+0000","last_change":"2024-09-11T23:34:52.868613+0000","last_active":"2024-09-11T23:35:08.844886+0000","last_peered":"2024-09-11T23:35:08.844886+0000","last_clean":"2024-09-11T23:35:08.844886+0000","last_became_active":"2024-09-11T23:34:52.868184+0000","last_became_peered":"2024-09-11T23:34:52.868184+0000","last_unstale":"2024-09-11T23:35:08.844886+0000","last_undegraded":"2024-09-11T23:35:08.844886+0000","last_fullsized":"2024-09-11T23:35:08.844886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:25.990002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.873390+0000","last_change":"2024-09-11T23:35:08.873390+0000","last_active":"2024-09-11T23:35:08.873390+0000","last_peered":"2024-09-11T23:35:08.873390+0000","last_clean":"2024-09-11T23:35:08.873390+0000","last_became_active":"2024-09-11T23:35:08.873052+0000","last_became_peered":"2024-09-11T23:35:08.873052+0000","last_unstale":"2024-09-11T23:35:08.873390+0000","last_undegraded":"2024-09-11T23:35:08.873390+0000","last_fullsized":"2024-09-11T23:35:08.873390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.937951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872829+0000","last_change":"2024-09-11T23:34:52.873604+0000","last_active":"2024-09-11T23:35:08.872829+0000","last_peered":"2024-09-11T23:35:08.872829+0000","last_clean":"2024-09-11T23:35:08.872829+0000","last_became_active":"2024-09-11T23:34:52.873220+0000","last_became_peered":"2024-09-11T23:34:52.873220+0000","last_unstale":"2024-09-11T23:35:08.872829+0000","last_undegraded":"2024-09-11T23:35:08.872829+0000","last_fullsized":"2024-09-11T23:35:08.872829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:38.480630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847599+0000","last_change":"2024-09-11T23:34:52.872902+0000","last_active":"2024-09-11T23:35:08.847599+0000","last_peered":"2024-09-11T23:35:08.847599+0000","last_clean":"2024-09-11T23:35:08.847599+0000","last_became_active":"2024-09-11T23:34:52.872571+0000","last_became_peered":"2024-09-11T23:34:52.872571+0000","last_unstale":"2024-09-11T23:35:08.847599+0000","last_undegraded":"2024-09-11T23:35:08.847599+0000","last_fullsized":"2024-09-11T23:35:08.847599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:21.844995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.457056+0000","last_change":"2024-09-11T23:35:09.457056+0000","last_active":"2024-09-11T23:35:09.457056+0000","last_peered":"2024-09-11T23:35:09.457056+0000","last_clean":"2024-09-11T23:35:09.457056+0000","last_became_active":"2024-09-11T23:35:09.456657+0000","last_became_peered":"2024-09-11T23:35:09.456657+0000","last_unstale":"2024-09-11T23:35:09.457056+0000","last_undegraded":"2024-09-11T23:35:09.457056+0000","last_fullsized":"2024-09-11T23:35:09.457056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:12:56.121280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844633+0000","last_change":"2024-09-11T23:34:52.875312+0000","last_active":"2024-09-11T23:35:08.844633+0000","last_peered":"2024-09-11T23:35:08.844633+0000","last_clean":"2024-09-11T23:35:08.844633+0000","last_became_active":"2024-09-11T23:34:52.874873+0000","last_became_peered":"2024-09-11T23:34:52.874873+0000","last_unstale":"2024-09-11T23:35:08.844633+0000","last_undegraded":"2024-09-11T23:35:08.844633+0000","last_fullsized":"2024-09-11T23:35:08.844633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:18.894713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847244+0000","last_change":"2024-09-11T23:34:52.862266+0000","last_active":"2024-09-11T23:35:08.847244+0000","last_peered":"2024-09-11T23:35:08.847244+0000","last_clean":"2024-09-11T23:35:08.847244+0000","last_became_active":"2024-09-11T23:34:52.861923+0000","last_became_peered":"2024-09-11T23:34:52.861923+0000","last_unstale":"2024-09-11T23:35:08.847244+0000","last_undegraded":"2024-09-11T23:35:08.847244+0000","last_fullsized":"2024-09-11T23:35:08.847244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:23.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":82,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:54.517127+0000","last_change":"2024-09-11T23:34:50.854020+0000","last_active":"2024-09-11T23:35:54.517127+0000","last_peered":"2024-09-11T23:35:54.517127+0000","last_clean":"2024-09-11T23:35:54.517127+0000","last_became_active":"2024-09-11T23:34:50.853683+0000","last_became_peered":"2024-09-11T23:34:50.853683+0000","last_unstale":"2024-09-11T23:35:54.517127+0000","last_undegraded":"2024-09-11T23:35:54.517127+0000","last_fullsized":"2024-09-11T23:35:54.517127+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:18.725495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455234+0000","last_change":"2024-09-11T23:34:52.874016+0000","last_active":"2024-09-11T23:35:09.455234+0000","last_peered":"2024-09-11T23:35:09.455234+0000","last_clean":"2024-09-11T23:35:09.455234+0000","last_became_active":"2024-09-11T23:34:52.873675+0000","last_became_peered":"2024-09-11T23:34:52.873675+0000","last_unstale":"2024-09-11T23:35:09.455234+0000","last_undegraded":"2024-09-11T23:35:09.455234+0000","last_fullsized":"2024-09-11T23:35:09.455234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:41:28.955548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848555+0000","last_change":"2024-09-11T23:34:50.855391+0000","last_active":"2024-09-11T23:35:08.848555+0000","last_peered":"2024-09-11T23:35:08.848555+0000","last_clean":"2024-09-11T23:35:08.848555+0000","last_became_active":"2024-09-11T23:34:50.855052+0000","last_became_peered":"2024-09-11T23:34:50.855052+0000","last_unstale":"2024-09-11T23:35:08.848555+0000","last_undegraded":"2024-09-11T23:35:08.848555+0000","last_fullsized":"2024-09-11T23:35:08.848555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.972239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844031+0000","last_change":"2024-09-11T23:34:50.848985+0000","last_active":"2024-09-11T23:35:08.844031+0000","last_peered":"2024-09-11T23:35:08.844031+0000","last_clean":"2024-09-11T23:35:08.844031+0000","last_became_active":"2024-09-11T23:34:50.848614+0000","last_became_peered":"2024-09-11T23:34:50.848614+0000","last_unstale":"2024-09-11T23:35:08.844031+0000","last_undegraded":"2024-09-11T23:35:08.844031+0000","last_fullsized":"2024-09-11T23:35:08.844031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:43:04.723136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847368+0000","last_change":"2024-09-11T23:34:52.866933+0000","last_active":"2024-09-11T23:35:08.847368+0000","last_peered":"2024-09-11T23:35:08.847368+0000","last_clean":"2024-09-11T23:35:08.847368+0000","last_became_active":"2024-09-11T23:34:52.866559+0000","last_became_peered":"2024-09-11T23:34:52.866559+0000","last_unstale":"2024-09-11T23:35:08.847368+0000","last_undegraded":"2024-09-11T23:35:08.847368+0000","last_fullsized":"2024-09-11T23:35:08.847368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:27.745118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456265+0000","last_change":"2024-09-11T23:34:50.849787+0000","last_active":"2024-09-11T23:35:09.456265+0000","last_peered":"2024-09-11T23:35:09.456265+0000","last_clean":"2024-09-11T23:35:09.456265+0000","last_became_active":"2024-09-11T23:34:50.849368+0000","last_became_peered":"2024-09-11T23:34:50.849368+0000","last_unstale":"2024-09-11T23:35:09.456265+0000","last_undegraded":"2024-09-11T23:35:09.456265+0000","last_fullsized":"2024-09-11T23:35:09.456265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:26.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847291+0000","last_change":"2024-09-11T23:34:52.870832+0000","last_active":"2024-09-11T23:35:08.847291+0000","last_peered":"2024-09-11T23:35:08.847291+0000","last_clean":"2024-09-11T23:35:08.847291+0000","last_became_active":"2024-09-11T23:34:52.870474+0000","last_became_peered":"2024-09-11T23:34:52.870474+0000","last_unstale":"2024-09-11T23:35:08.847291+0000","last_undegraded":"2024-09-11T23:35:08.847291+0000","last_fullsized":"2024-09-11T23:35:08.847291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:30.196114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":136,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:58.522407+0000","last_change":"2024-09-11T23:34:50.859074+0000","last_active":"2024-09-11T23:35:58.522407+0000","last_peered":"2024-09-11T23:35:58.522407+0000","last_clean":"2024-09-11T23:35:58.522407+0000","last_became_active":"2024-09-11T23:34:50.858664+0000","last_became_peered":"2024-09-11T23:34:50.858664+0000","last_unstale":"2024-09-11T23:35:58.522407+0000","last_undegraded":"2024-09-11T23:35:58.522407+0000","last_fullsized":"2024-09-11T23:35:58.522407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:04.077813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850853+0000","last_change":"2024-09-11T23:34:52.875901+0000","last_active":"2024-09-11T23:35:08.850853+0000","last_peered":"2024-09-11T23:35:08.850853+0000","last_clean":"2024-09-11T23:35:08.850853+0000","last_became_active":"2024-09-11T23:34:52.875568+0000","last_became_peered":"2024-09-11T23:34:52.875568+0000","last_unstale":"2024-09-11T23:35:08.850853+0000","last_undegraded":"2024-09-11T23:35:08.850853+0000","last_fullsized":"2024-09-11T23:35:08.850853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:54.173548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846938+0000","last_change":"2024-09-11T23:34:50.854957+0000","last_active":"2024-09-11T23:35:08.846938+0000","last_peered":"2024-09-11T23:35:08.846938+0000","last_clean":"2024-09-11T23:35:08.846938+0000","last_became_active":"2024-09-11T23:34:50.854626+0000","last_became_peered":"2024-09-11T23:34:50.854626+0000","last_unstale":"2024-09-11T23:35:08.846938+0000","last_undegraded":"2024-09-11T23:35:08.846938+0000","last_fullsized":"2024-09-11T23:35:08.846938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:53.738966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872227+0000","last_change":"2024-09-11T23:34:52.868850+0000","last_active":"2024-09-11T23:35:08.872227+0000","last_peered":"2024-09-11T23:35:08.872227+0000","last_clean":"2024-09-11T23:35:08.872227+0000","last_became_active":"2024-09-11T23:34:52.868536+0000","last_became_peered":"2024-09-11T23:34:52.868536+0000","last_unstale":"2024-09-11T23:35:08.872227+0000","last_undegraded":"2024-09-11T23:35:08.872227+0000","last_fullsized":"2024-09-11T23:35:08.872227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:11.746821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456837+0000","last_change":"2024-09-11T23:34:50.924819+0000","last_active":"2024-09-11T23:35:09.456837+0000","last_peered":"2024-09-11T23:35:09.456837+0000","last_clean":"2024-09-11T23:35:09.456837+0000","last_became_active":"2024-09-11T23:34:50.924283+0000","last_became_peered":"2024-09-11T23:34:50.924283+0000","last_unstale":"2024-09-11T23:35:09.456837+0000","last_undegraded":"2024-09-11T23:35:09.456837+0000","last_fullsized":"2024-09-11T23:35:09.456837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:41.345233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850516+0000","last_change":"2024-09-11T23:34:52.867113+0000","last_active":"2024-09-11T23:35:08.850516+0000","last_peered":"2024-09-11T23:35:08.850516+0000","last_clean":"2024-09-11T23:35:08.850516+0000","last_became_active":"2024-09-11T23:34:52.866125+0000","last_became_peered":"2024-09-11T23:34:52.866125+0000","last_unstale":"2024-09-11T23:35:08.850516+0000","last_undegraded":"2024-09-11T23:35:08.850516+0000","last_fullsized":"2024-09-11T23:35:08.850516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:18:55.615079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":176,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:58.522375+0000","last_change":"2024-09-11T23:34:50.857272+0000","last_active":"2024-09-11T23:35:58.522375+0000","last_peered":"2024-09-11T23:35:58.522375+0000","last_clean":"2024-09-11T23:35:58.522375+0000","last_became_active":"2024-09-11T23:34:50.855701+0000","last_became_peered":"2024-09-11T23:34:50.855701+0000","last_unstale":"2024-09-11T23:35:58.522375+0000","last_undegraded":"2024-09-11T23:35:58.522375+0000","last_fullsized":"2024-09-11T23:35:58.522375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:49.997617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62310,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":122,"num_read_kb":121,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848168+0000","last_change":"2024-09-11T23:34:52.876928+0000","last_active":"2024-09-11T23:35:08.848168+0000","last_peered":"2024-09-11T23:35:08.848168+0000","last_clean":"2024-09-11T23:35:08.848168+0000","last_became_active":"2024-09-11T23:34:52.876578+0000","last_became_peered":"2024-09-11T23:34:52.876578+0000","last_unstale":"2024-09-11T23:35:08.848168+0000","last_undegraded":"2024-09-11T23:35:08.848168+0000","last_fullsized":"2024-09-11T23:35:08.848168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:07.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.870264+0000","last_change":"2024-09-11T23:35:08.869275+0000","last_active":"2024-09-11T23:35:08.870264+0000","last_peered":"2024-09-11T23:35:08.870264+0000","last_clean":"2024-09-11T23:35:08.870264+0000","last_became_active":"2024-09-11T23:35:08.850112+0000","last_became_peered":"2024-09-11T23:35:08.850112+0000","last_unstale":"2024-09-11T23:35:08.870264+0000","last_undegraded":"2024-09-11T23:35:08.870264+0000","last_fullsized":"2024-09-11T23:35:08.870264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:36.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844084+0000","last_change":"2024-09-11T23:34:52.871970+0000","last_active":"2024-09-11T23:35:08.844084+0000","last_peered":"2024-09-11T23:35:08.844084+0000","last_clean":"2024-09-11T23:35:08.844084+0000","last_became_active":"2024-09-11T23:34:52.871488+0000","last_became_peered":"2024-09-11T23:34:52.871488+0000","last_unstale":"2024-09-11T23:35:08.844084+0000","last_undegraded":"2024-09-11T23:35:08.844084+0000","last_fullsized":"2024-09-11T23:35:08.844084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:08.266431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456386+0000","last_change":"2024-09-11T23:34:50.854023+0000","last_active":"2024-09-11T23:35:09.456386+0000","last_peered":"2024-09-11T23:35:09.456386+0000","last_clean":"2024-09-11T23:35:09.456386+0000","last_became_active":"2024-09-11T23:34:50.853661+0000","last_became_peered":"2024-09-11T23:34:50.853661+0000","last_unstale":"2024-09-11T23:35:09.456386+0000","last_undegraded":"2024-09-11T23:35:09.456386+0000","last_fullsized":"2024-09-11T23:35:09.456386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:25.281976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.849648+0000","last_change":"2024-09-11T23:35:08.849648+0000","last_active":"2024-09-11T23:35:08.849648+0000","last_peered":"2024-09-11T23:35:08.849648+0000","last_clean":"2024-09-11T23:35:08.849648+0000","last_became_active":"2024-09-11T23:35:08.849246+0000","last_became_peered":"2024-09-11T23:35:08.849246+0000","last_unstale":"2024-09-11T23:35:08.849648+0000","last_undegraded":"2024-09-11T23:35:08.849648+0000","last_fullsized":"2024-09-11T23:35:08.849648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:17.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":85,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:55.538651+0000","last_change":"2024-09-11T23:34:50.853855+0000","last_active":"2024-09-11T23:35:55.538651+0000","last_peered":"2024-09-11T23:35:55.538651+0000","last_clean":"2024-09-11T23:35:55.538651+0000","last_became_active":"2024-09-11T23:34:50.853493+0000","last_became_peered":"2024-09-11T23:34:50.853493+0000","last_unstale":"2024-09-11T23:35:55.538651+0000","last_undegraded":"2024-09-11T23:35:55.538651+0000","last_fullsized":"2024-09-11T23:35:55.538651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:23.262472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846505+0000","last_change":"2024-09-11T23:34:52.875835+0000","last_active":"2024-09-11T23:35:08.846505+0000","last_peered":"2024-09-11T23:35:08.846505+0000","last_clean":"2024-09-11T23:35:08.846505+0000","last_became_active":"2024-09-11T23:34:52.875525+0000","last_became_peered":"2024-09-11T23:34:52.875525+0000","last_unstale":"2024-09-11T23:35:08.846505+0000","last_undegraded":"2024-09-11T23:35:08.846505+0000","last_fullsized":"2024-09-11T23:35:08.846505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:41.031816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847151+0000","last_change":"2024-09-11T23:34:50.924803+0000","last_active":"2024-09-11T23:35:08.847151+0000","last_peered":"2024-09-11T23:35:08.847151+0000","last_clean":"2024-09-11T23:35:08.847151+0000","last_became_active":"2024-09-11T23:34:50.924217+0000","last_became_peered":"2024-09-11T23:34:50.924217+0000","last_unstale":"2024-09-11T23:35:08.847151+0000","last_undegraded":"2024-09-11T23:35:08.847151+0000","last_fullsized":"2024-09-11T23:35:08.847151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:45.790592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.852839+0000","last_change":"2024-09-11T23:35:08.851856+0000","last_active":"2024-09-11T23:35:08.852839+0000","last_peered":"2024-09-11T23:35:08.852839+0000","last_clean":"2024-09-11T23:35:08.852839+0000","last_became_active":"2024-09-11T23:35:08.851490+0000","last_became_peered":"2024-09-11T23:35:08.851490+0000","last_unstale":"2024-09-11T23:35:08.852839+0000","last_undegraded":"2024-09-11T23:35:08.852839+0000","last_fullsized":"2024-09-11T23:35:08.852839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:46.269022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847890+0000","last_change":"2024-09-11T23:34:50.848385+0000","last_active":"2024-09-11T23:35:08.847890+0000","last_peered":"2024-09-11T23:35:08.847890+0000","last_clean":"2024-09-11T23:35:08.847890+0000","last_became_active":"2024-09-11T23:34:50.847808+0000","last_became_peered":"2024-09-11T23:34:50.847808+0000","last_unstale":"2024-09-11T23:35:08.847890+0000","last_undegraded":"2024-09-11T23:35:08.847890+0000","last_fullsized":"2024-09-11T23:35:08.847890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:38.796622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844817+0000","last_change":"2024-09-11T23:34:52.867341+0000","last_active":"2024-09-11T23:35:08.844817+0000","last_peered":"2024-09-11T23:35:08.844817+0000","last_clean":"2024-09-11T23:35:08.844817+0000","last_became_active":"2024-09-11T23:34:52.866161+0000","last_became_peered":"2024-09-11T23:34:52.866161+0000","last_unstale":"2024-09-11T23:35:08.844817+0000","last_undegraded":"2024-09-11T23:35:08.844817+0000","last_fullsized":"2024-09-11T23:35:08.844817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:27.913705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844372+0000","last_change":"2024-09-11T23:34:50.851466+0000","last_active":"2024-09-11T23:35:08.844372+0000","last_peered":"2024-09-11T23:35:08.844372+0000","last_clean":"2024-09-11T23:35:08.844372+0000","last_became_active":"2024-09-11T23:34:50.851191+0000","last_became_peered":"2024-09-11T23:34:50.851191+0000","last_unstale":"2024-09-11T23:35:08.844372+0000","last_undegraded":"2024-09-11T23:35:08.844372+0000","last_fullsized":"2024-09-11T23:35:08.844372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:00.328658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847008+0000","last_change":"2024-09-11T23:34:52.872290+0000","last_active":"2024-09-11T23:35:08.847008+0000","last_peered":"2024-09-11T23:35:08.847008+0000","last_clean":"2024-09-11T23:35:08.847008+0000","last_became_active":"2024-09-11T23:34:52.871935+0000","last_became_peered":"2024-09-11T23:34:52.871935+0000","last_unstale":"2024-09-11T23:35:08.847008+0000","last_undegraded":"2024-09-11T23:35:08.847008+0000","last_fullsized":"2024-09-11T23:35:08.847008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:08.217548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.456729+0000","last_change":"2024-09-11T23:34:50.857763+0000","last_active":"2024-09-11T23:35:09.456729+0000","last_peered":"2024-09-11T23:35:09.456729+0000","last_clean":"2024-09-11T23:35:09.456729+0000","last_became_active":"2024-09-11T23:34:50.857427+0000","last_became_peered":"2024-09-11T23:34:50.857427+0000","last_unstale":"2024-09-11T23:35:09.456729+0000","last_undegraded":"2024-09-11T23:35:09.456729+0000","last_fullsized":"2024-09-11T23:35:09.456729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:59.703580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846908+0000","last_change":"2024-09-11T23:34:52.872426+0000","last_active":"2024-09-11T23:35:08.846908+0000","last_peered":"2024-09-11T23:35:08.846908+0000","last_clean":"2024-09-11T23:35:08.846908+0000","last_became_active":"2024-09-11T23:34:52.872090+0000","last_became_peered":"2024-09-11T23:34:52.872090+0000","last_unstale":"2024-09-11T23:35:08.846908+0000","last_undegraded":"2024-09-11T23:35:08.846908+0000","last_fullsized":"2024-09-11T23:35:08.846908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:04:00.963963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'6","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:55.538350+0000","last_change":"2024-09-11T23:34:50.924792+0000","last_active":"2024-09-11T23:35:55.538350+0000","last_peered":"2024-09-11T23:35:55.538350+0000","last_clean":"2024-09-11T23:35:55.538350+0000","last_became_active":"2024-09-11T23:34:50.924444+0000","last_became_peered":"2024-09-11T23:34:50.924444+0000","last_unstale":"2024-09-11T23:35:55.538350+0000","last_undegraded":"2024-09-11T23:35:55.538350+0000","last_fullsized":"2024-09-11T23:35:55.538350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:12.855314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846881+0000","last_change":"2024-09-11T23:34:52.872079+0000","last_active":"2024-09-11T23:35:08.846881+0000","last_peered":"2024-09-11T23:35:08.846881+0000","last_clean":"2024-09-11T23:35:08.846881+0000","last_became_active":"2024-09-11T23:34:52.871755+0000","last_became_peered":"2024-09-11T23:34:52.871755+0000","last_unstale":"2024-09-11T23:35:08.846881+0000","last_undegraded":"2024-09-11T23:35:08.846881+0000","last_fullsized":"2024-09-11T23:35:08.846881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:05.990832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":344,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846729+0000","last_change":"2024-09-11T23:33:42.256521+0000","last_active":"2024-09-11T23:35:08.846729+0000","last_peered":"2024-09-11T23:35:08.846729+0000","last_clean":"2024-09-11T23:35:08.846729+0000","last_became_active":"2024-09-11T23:33:42.256041+0000","last_became_peered":"2024-09-11T23:33:42.256041+0000","last_unstale":"2024-09-11T23:35:08.846729+0000","last_undegraded":"2024-09-11T23:35:08.846729+0000","last_fullsized":"2024-09-11T23:35:08.846729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_clean_scrub_stamp":"2024-09-11T23:30:22.061515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:21.391839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.845844+0000","last_change":"2024-09-11T23:34:50.854087+0000","last_active":"2024-09-11T23:35:08.845844+0000","last_peered":"2024-09-11T23:35:08.845844+0000","last_clean":"2024-09-11T23:35:08.845844+0000","last_became_active":"2024-09-11T23:34:50.852346+0000","last_became_peered":"2024-09-11T23:34:50.852346+0000","last_unstale":"2024-09-11T23:35:08.845844+0000","last_undegraded":"2024-09-11T23:35:08.845844+0000","last_fullsized":"2024-09-11T23:35:08.845844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.287119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847645+0000","last_change":"2024-09-11T23:34:52.869198+0000","last_active":"2024-09-11T23:35:08.847645+0000","last_peered":"2024-09-11T23:35:08.847645+0000","last_clean":"2024-09-11T23:35:08.847645+0000","last_became_active":"2024-09-11T23:34:52.868621+0000","last_became_peered":"2024-09-11T23:34:52.868621+0000","last_unstale":"2024-09-11T23:35:08.847645+0000","last_undegraded":"2024-09-11T23:35:08.847645+0000","last_fullsized":"2024-09-11T23:35:08.847645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:17.430654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846715+0000","last_change":"2024-09-11T23:34:50.852752+0000","last_active":"2024-09-11T23:35:08.846715+0000","last_peered":"2024-09-11T23:35:08.846715+0000","last_clean":"2024-09-11T23:35:08.846715+0000","last_became_active":"2024-09-11T23:34:50.851895+0000","last_became_peered":"2024-09-11T23:34:50.851895+0000","last_unstale":"2024-09-11T23:35:08.846715+0000","last_undegraded":"2024-09-11T23:35:08.846715+0000","last_fullsized":"2024-09-11T23:35:08.846715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:46.894432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844043+0000","last_change":"2024-09-11T23:34:52.866191+0000","last_active":"2024-09-11T23:35:08.844043+0000","last_peered":"2024-09-11T23:35:08.844043+0000","last_clean":"2024-09-11T23:35:08.844043+0000","last_became_active":"2024-09-11T23:34:52.865858+0000","last_became_peered":"2024-09-11T23:34:52.865858+0000","last_unstale":"2024-09-11T23:35:08.844043+0000","last_undegraded":"2024-09-11T23:35:08.844043+0000","last_fullsized":"2024-09-11T23:35:08.844043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:21.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":82,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:55.538109+0000","last_change":"2024-09-11T23:34:50.924125+0000","last_active":"2024-09-11T23:35:55.538109+0000","last_peered":"2024-09-11T23:35:55.538109+0000","last_clean":"2024-09-11T23:35:55.538109+0000","last_became_active":"2024-09-11T23:34:50.923780+0000","last_became_peered":"2024-09-11T23:34:50.923780+0000","last_unstale":"2024-09-11T23:35:55.538109+0000","last_undegraded":"2024-09-11T23:35:55.538109+0000","last_fullsized":"2024-09-11T23:35:55.538109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:39.774827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455096+0000","last_change":"2024-09-11T23:34:52.868364+0000","last_active":"2024-09-11T23:35:09.455096+0000","last_peered":"2024-09-11T23:35:09.455096+0000","last_clean":"2024-09-11T23:35:09.455096+0000","last_became_active":"2024-09-11T23:34:52.867913+0000","last_became_peered":"2024-09-11T23:34:52.867913+0000","last_unstale":"2024-09-11T23:35:09.455096+0000","last_undegraded":"2024-09-11T23:35:09.455096+0000","last_fullsized":"2024-09-11T23:35:09.455096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.571825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:54.523178+0000","last_change":"2024-09-11T23:34:50.852421+0000","last_active":"2024-09-11T23:35:54.523178+0000","last_peered":"2024-09-11T23:35:54.523178+0000","last_clean":"2024-09-11T23:35:54.523178+0000","last_became_active":"2024-09-11T23:34:50.851898+0000","last_became_peered":"2024-09-11T23:34:50.851898+0000","last_unstale":"2024-09-11T23:35:54.523178+0000","last_undegraded":"2024-09-11T23:35:54.523178+0000","last_fullsized":"2024-09-11T23:35:54.523178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.661276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848710+0000","last_change":"2024-09-11T23:34:52.879995+0000","last_active":"2024-09-11T23:35:08.848710+0000","last_peered":"2024-09-11T23:35:08.848710+0000","last_clean":"2024-09-11T23:35:08.848710+0000","last_became_active":"2024-09-11T23:34:52.879678+0000","last_became_peered":"2024-09-11T23:34:52.879678+0000","last_unstale":"2024-09-11T23:35:08.848710+0000","last_undegraded":"2024-09-11T23:35:08.848710+0000","last_fullsized":"2024-09-11T23:35:08.848710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:47.456674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847399+0000","last_change":"2024-09-11T23:34:50.924957+0000","last_active":"2024-09-11T23:35:08.847399+0000","last_peered":"2024-09-11T23:35:08.847399+0000","last_clean":"2024-09-11T23:35:08.847399+0000","last_became_active":"2024-09-11T23:34:50.924500+0000","last_became_peered":"2024-09-11T23:34:50.924500+0000","last_unstale":"2024-09-11T23:35:08.847399+0000","last_undegraded":"2024-09-11T23:35:08.847399+0000","last_fullsized":"2024-09-11T23:35:08.847399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:39.268842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848946+0000","last_change":"2024-09-11T23:34:52.867337+0000","last_active":"2024-09-11T23:35:08.848946+0000","last_peered":"2024-09-11T23:35:08.848946+0000","last_clean":"2024-09-11T23:35:08.848946+0000","last_became_active":"2024-09-11T23:34:52.866799+0000","last_became_peered":"2024-09-11T23:34:52.866799+0000","last_unstale":"2024-09-11T23:35:08.848946+0000","last_undegraded":"2024-09-11T23:35:08.848946+0000","last_fullsized":"2024-09-11T23:35:08.848946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:38.177135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848411+0000","last_change":"2024-09-11T23:34:50.925156+0000","last_active":"2024-09-11T23:35:08.848411+0000","last_peered":"2024-09-11T23:35:08.848411+0000","last_clean":"2024-09-11T23:35:08.848411+0000","last_became_active":"2024-09-11T23:34:50.924282+0000","last_became_peered":"2024-09-11T23:34:50.924282+0000","last_unstale":"2024-09-11T23:35:08.848411+0000","last_undegraded":"2024-09-11T23:35:08.848411+0000","last_fullsized":"2024-09-11T23:35:08.848411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:47.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844337+0000","last_change":"2024-09-11T23:34:52.864993+0000","last_active":"2024-09-11T23:35:08.844337+0000","last_peered":"2024-09-11T23:35:08.844337+0000","last_clean":"2024-09-11T23:35:08.844337+0000","last_became_active":"2024-09-11T23:34:52.864267+0000","last_became_peered":"2024-09-11T23:34:52.864267+0000","last_unstale":"2024-09-11T23:35:08.844337+0000","last_undegraded":"2024-09-11T23:35:08.844337+0000","last_fullsized":"2024-09-11T23:35:08.844337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:29.912068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:17.449902+0000","last_change":"2024-09-11T23:34:50.849864+0000","last_active":"2024-09-11T23:35:17.449902+0000","last_peered":"2024-09-11T23:35:17.449902+0000","last_clean":"2024-09-11T23:35:17.449902+0000","last_became_active":"2024-09-11T23:34:50.849577+0000","last_became_peered":"2024-09-11T23:34:50.849577+0000","last_unstale":"2024-09-11T23:35:17.449902+0000","last_undegraded":"2024-09-11T23:35:17.449902+0000","last_fullsized":"2024-09-11T23:35:17.449902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:06.845000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848578+0000","last_change":"2024-09-11T23:34:52.872028+0000","last_active":"2024-09-11T23:35:08.848578+0000","last_peered":"2024-09-11T23:35:08.848578+0000","last_clean":"2024-09-11T23:35:08.848578+0000","last_became_active":"2024-09-11T23:34:52.871723+0000","last_became_peered":"2024-09-11T23:34:52.871723+0000","last_unstale":"2024-09-11T23:35:08.848578+0000","last_undegraded":"2024-09-11T23:35:08.848578+0000","last_fullsized":"2024-09-11T23:35:08.848578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:44.656218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846269+0000","last_change":"2024-09-11T23:34:50.860314+0000","last_active":"2024-09-11T23:35:08.846269+0000","last_peered":"2024-09-11T23:35:08.846269+0000","last_clean":"2024-09-11T23:35:08.846269+0000","last_became_active":"2024-09-11T23:34:50.859953+0000","last_became_peered":"2024-09-11T23:34:50.859953+0000","last_unstale":"2024-09-11T23:35:08.846269+0000","last_undegraded":"2024-09-11T23:35:08.846269+0000","last_fullsized":"2024-09-11T23:35:08.846269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:46.705864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872092+0000","last_change":"2024-09-11T23:34:52.867190+0000","last_active":"2024-09-11T23:35:08.872092+0000","last_peered":"2024-09-11T23:35:08.872092+0000","last_clean":"2024-09-11T23:35:08.872092+0000","last_became_active":"2024-09-11T23:34:52.866829+0000","last_became_peered":"2024-09-11T23:34:52.866829+0000","last_unstale":"2024-09-11T23:35:08.872092+0000","last_undegraded":"2024-09-11T23:35:08.872092+0000","last_fullsized":"2024-09-11T23:35:08.872092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.752006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850343+0000","last_change":"2024-09-11T23:34:50.924566+0000","last_active":"2024-09-11T23:35:08.850343+0000","last_peered":"2024-09-11T23:35:08.850343+0000","last_clean":"2024-09-11T23:35:08.850343+0000","last_became_active":"2024-09-11T23:34:50.924284+0000","last_became_peered":"2024-09-11T23:34:50.924284+0000","last_unstale":"2024-09-11T23:35:08.850343+0000","last_undegraded":"2024-09-11T23:35:08.850343+0000","last_fullsized":"2024-09-11T23:35:08.850343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:54.246154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847042+0000","last_change":"2024-09-11T23:34:52.869302+0000","last_active":"2024-09-11T23:35:08.847042+0000","last_peered":"2024-09-11T23:35:08.847042+0000","last_clean":"2024-09-11T23:35:08.847042+0000","last_became_active":"2024-09-11T23:34:52.868960+0000","last_became_peered":"2024-09-11T23:34:52.868960+0000","last_unstale":"2024-09-11T23:35:08.847042+0000","last_undegraded":"2024-09-11T23:35:08.847042+0000","last_fullsized":"2024-09-11T23:35:08.847042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.561712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847953+0000","last_change":"2024-09-11T23:34:50.854573+0000","last_active":"2024-09-11T23:35:08.847953+0000","last_peered":"2024-09-11T23:35:08.847953+0000","last_clean":"2024-09-11T23:35:08.847953+0000","last_became_active":"2024-09-11T23:34:50.854165+0000","last_became_peered":"2024-09-11T23:34:50.854165+0000","last_unstale":"2024-09-11T23:35:08.847953+0000","last_undegraded":"2024-09-11T23:35:08.847953+0000","last_fullsized":"2024-09-11T23:35:08.847953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:57:31.589495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.844752+0000","last_change":"2024-09-11T23:34:52.875987+0000","last_active":"2024-09-11T23:35:08.844752+0000","last_peered":"2024-09-11T23:35:08.844752+0000","last_clean":"2024-09-11T23:35:08.844752+0000","last_became_active":"2024-09-11T23:34:52.875662+0000","last_became_peered":"2024-09-11T23:34:52.875662+0000","last_unstale":"2024-09-11T23:35:08.844752+0000","last_undegraded":"2024-09-11T23:35:08.844752+0000","last_fullsized":"2024-09-11T23:35:08.844752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:16.514938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847766+0000","last_change":"2024-09-11T23:34:50.850726+0000","last_active":"2024-09-11T23:35:08.847766+0000","last_peered":"2024-09-11T23:35:08.847766+0000","last_clean":"2024-09-11T23:35:08.847766+0000","last_became_active":"2024-09-11T23:34:50.850381+0000","last_became_peered":"2024-09-11T23:34:50.850381+0000","last_unstale":"2024-09-11T23:35:08.847766+0000","last_undegraded":"2024-09-11T23:35:08.847766+0000","last_fullsized":"2024-09-11T23:35:08.847766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:50.431293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847564+0000","last_change":"2024-09-11T23:34:52.870652+0000","last_active":"2024-09-11T23:35:08.847564+0000","last_peered":"2024-09-11T23:35:08.847564+0000","last_clean":"2024-09-11T23:35:08.847564+0000","last_became_active":"2024-09-11T23:34:52.870303+0000","last_became_peered":"2024-09-11T23:34:52.870303+0000","last_unstale":"2024-09-11T23:35:08.847564+0000","last_undegraded":"2024-09-11T23:35:08.847564+0000","last_fullsized":"2024-09-11T23:35:08.847564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:17.670981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.855045+0000","last_change":"2024-09-11T23:35:08.854490+0000","last_active":"2024-09-11T23:35:08.855045+0000","last_peered":"2024-09-11T23:35:08.855045+0000","last_clean":"2024-09-11T23:35:08.855045+0000","last_became_active":"2024-09-11T23:35:08.849820+0000","last_became_peered":"2024-09-11T23:35:08.849820+0000","last_unstale":"2024-09-11T23:35:08.855045+0000","last_undegraded":"2024-09-11T23:35:08.855045+0000","last_fullsized":"2024-09-11T23:35:08.855045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:09.509867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.846681+0000","last_change":"2024-09-11T23:34:52.869695+0000","last_active":"2024-09-11T23:35:08.846681+0000","last_peered":"2024-09-11T23:35:08.846681+0000","last_clean":"2024-09-11T23:35:08.846681+0000","last_became_active":"2024-09-11T23:34:52.869367+0000","last_became_peered":"2024-09-11T23:34:52.869367+0000","last_unstale":"2024-09-11T23:35:08.846681+0000","last_undegraded":"2024-09-11T23:35:08.846681+0000","last_fullsized":"2024-09-11T23:35:08.846681+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:16.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:09.455461+0000","last_change":"2024-09-11T23:34:52.872778+0000","last_active":"2024-09-11T23:35:09.455461+0000","last_peered":"2024-09-11T23:35:09.455461+0000","last_clean":"2024-09-11T23:35:09.455461+0000","last_became_active":"2024-09-11T23:34:52.872451+0000","last_became_peered":"2024-09-11T23:34:52.872451+0000","last_unstale":"2024-09-11T23:35:09.455461+0000","last_undegraded":"2024-09-11T23:35:09.455461+0000","last_fullsized":"2024-09-11T23:35:09.455461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:51.769927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.847892+0000","last_change":"2024-09-11T23:34:50.849675+0000","last_active":"2024-09-11T23:35:08.847892+0000","last_peered":"2024-09-11T23:35:08.847892+0000","last_clean":"2024-09-11T23:35:08.847892+0000","last_became_active":"2024-09-11T23:34:50.849228+0000","last_became_peered":"2024-09-11T23:34:50.849228+0000","last_unstale":"2024-09-11T23:35:08.847892+0000","last_undegraded":"2024-09-11T23:35:08.847892+0000","last_fullsized":"2024-09-11T23:35:08.847892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:50.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.850319+0000","last_change":"2024-09-11T23:34:52.867739+0000","last_active":"2024-09-11T23:35:08.850319+0000","last_peered":"2024-09-11T23:35:08.850319+0000","last_clean":"2024-09-11T23:35:08.850319+0000","last_became_active":"2024-09-11T23:34:52.866613+0000","last_became_peered":"2024-09-11T23:34:52.866613+0000","last_unstale":"2024-09-11T23:35:08.850319+0000","last_undegraded":"2024-09-11T23:35:08.850319+0000","last_fullsized":"2024-09-11T23:35:08.850319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:49.348386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848269+0000","last_change":"2024-09-11T23:34:50.851401+0000","last_active":"2024-09-11T23:35:08.848269+0000","last_peered":"2024-09-11T23:35:08.848269+0000","last_clean":"2024-09-11T23:35:08.848269+0000","last_became_active":"2024-09-11T23:34:50.851068+0000","last_became_peered":"2024-09-11T23:34:50.851068+0000","last_unstale":"2024-09-11T23:35:08.848269+0000","last_undegraded":"2024-09-11T23:35:08.848269+0000","last_fullsized":"2024-09-11T23:35:08.848269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:09.673864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.872365+0000","last_change":"2024-09-11T23:34:52.865216+0000","last_active":"2024-09-11T23:35:08.872365+0000","last_peered":"2024-09-11T23:35:08.872365+0000","last_clean":"2024-09-11T23:35:08.872365+0000","last_became_active":"2024-09-11T23:34:52.864917+0000","last_became_peered":"2024-09-11T23:34:52.864917+0000","last_unstale":"2024-09-11T23:35:08.872365+0000","last_undegraded":"2024-09-11T23:35:08.872365+0000","last_fullsized":"2024-09-11T23:35:08.872365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:36.469793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848560+0000","last_change":"2024-09-11T23:34:50.924291+0000","last_active":"2024-09-11T23:35:08.848560+0000","last_peered":"2024-09-11T23:35:08.848560+0000","last_clean":"2024-09-11T23:35:08.848560+0000","last_became_active":"2024-09-11T23:34:50.923892+0000","last_became_peered":"2024-09-11T23:34:50.923892+0000","last_unstale":"2024-09-11T23:35:08.848560+0000","last_undegraded":"2024-09-11T23:35:08.848560+0000","last_fullsized":"2024-09-11T23:35:08.848560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:52.266778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:08.848164+0000","last_change":"2024-09-11T23:34:52.875995+0000","last_active":"2024-09-11T23:35:08.848164+0000","last_peered":"2024-09-11T23:35:08.848164+0000","last_clean":"2024-09-11T23:35:08.848164+0000","last_became_active":"2024-09-11T23:34:52.875654+0000","last_became_peered":"2024-09-11T23:34:52.875654+0000","last_unstale":"2024-09-11T23:35:08.848164+0000","last_undegraded":"2024-09-11T23:35:08.848164+0000","last_fullsized":"2024-09-11T23:35:08.848164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:19.218291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":87,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T23:35:54.522570+0000","last_change":"2024-09-11T23:34:50.853881+0000","last_active":"2024-09-11T23:35:54.522570+0000","last_peered":"2024-09-11T23:35:54.522570+0000","last_clean":"2024-09-11T23:35:54.522570+0000","last_became_active":"2024-09-11T23:34:50.853576+0000","last_became_peered":"2024-09-11T23:34:50.853576+0000","last_unstale":"2024-09-11T23:35:54.522570+0000","last_undegraded":"2024-09-11T23:35:54.522570+0000","last_fullsized":"2024-09-11T23:35:54.522570+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:38.550587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69518,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":577,"num_read_kb":366,"num_write":247,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":208554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612634,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":434822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841567,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1597440,"data_stored":1088093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005092,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":434291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":434669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495662,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1511424,"data_stored":1027189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659186,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":901120,"data_stored":434008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888119,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1026474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018941,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":434262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247872,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":433833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444103,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":497776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607627,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":497231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":921600,"data_stored":435740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:36:00.957 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:36:00.957 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T23:36:00.957 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T23:36:00.957 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T23:36:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:01.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:01 smithi133 ceph-mon[31430]: pgmap v307: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:01.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:01 smithi178 ceph-mon[35882]: pgmap v307: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:01.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:01 smithi138 ceph-mon[35142]: pgmap v307: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:01.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.666+0000 7f7a7d296640 1 -- 172.21.15.133:0/3152499738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a78102d30 msgr2=0x7f7a78103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:01.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.666+0000 7f7a7d296640 1 --2- 172.21.15.133:0/3152499738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a78102d30 0x7f7a78103110 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009930 tx=0x7f7a6802a440 comp rx=0 tx=0).stop 2024-09-11T23:36:01.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.666+0000 7f7a7d296640 1 -- 172.21.15.133:0/3152499738 shutdown_connections 2024-09-11T23:36:01.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.666+0000 7f7a7d296640 1 --2- 172.21.15.133:0/3152499738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a781082a0 0x7f7a7810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.666+0000 7f7a7d296640 1 --2- 172.21.15.133:0/3152499738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a781036e0 0x7f7a78107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.666+0000 7f7a7d296640 1 --2- 172.21.15.133:0/3152499738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a78102d30 0x7f7a78103110 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.666+0000 7f7a7d296640 1 -- 172.21.15.133:0/3152499738 >> 172.21.15.133:0/3152499738 conn(0x7f7a780fc7d0 msgr2=0x7f7a780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.666+0000 7f7a7d296640 1 -- 172.21.15.133:0/3152499738 shutdown_connections 2024-09-11T23:36:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.666+0000 7f7a7d296640 1 -- 172.21.15.133:0/3152499738 wait complete. 2024-09-11T23:36:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.666+0000 7f7a7d296640 1 Processor -- start 2024-09-11T23:36:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a7d296640 1 -- start start 2024-09-11T23:36:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a7d296640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a78102d30 0x7f7a78198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a7d296640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a781036e0 0x7f7a78198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a7d296640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a781082a0 0x7f7a7819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a7d296640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7a781120e0 con 0x7f7a781036e0 2024-09-11T23:36:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a7d296640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a78111f60 con 0x7f7a781082a0 2024-09-11T23:36:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a7d296640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a78112260 con 0x7f7a78102d30 2024-09-11T23:36:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a6e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a781036e0 0x7f7a78198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a6e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a781036e0 0x7f7a78198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48850/0 (socket says 172.21.15.133:48850) 2024-09-11T23:36:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a6e7fc640 1 -- 172.21.15.133:0/3736701413 learned_addr learned my addr 172.21.15.133:0/3736701413 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a76ffd640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a78102d30 0x7f7a78198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a777fe640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a781082a0 0x7f7a7819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a6e7fc640 1 -- 172.21.15.133:0/3736701413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a78102d30 msgr2=0x7f7a78198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a6e7fc640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a78102d30 0x7f7a78198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a6e7fc640 1 -- 172.21.15.133:0/3736701413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a781082a0 msgr2=0x7f7a7819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a6e7fc640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a781082a0 0x7f7a7819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a6e7fc640 1 -- 172.21.15.133:0/3736701413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7819d5d0 con 0x7f7a781036e0 2024-09-11T23:36:01.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a777fe640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a781082a0 0x7f7a7819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:01.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.667+0000 7f7a6e7fc640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a781036e0 0x7f7a78198b60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7a60006fd0 tx=0x7f7a6000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:01.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.668+0000 7f7a74ff9640 1 -- 172.21.15.133:0/3736701413 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60011830 con 0x7f7a781036e0 2024-09-11T23:36:01.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.668+0000 7f7a74ff9640 1 -- 172.21.15.133:0/3736701413 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a60016070 con 0x7f7a781036e0 2024-09-11T23:36:01.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.668+0000 7f7a7d296640 1 -- 172.21.15.133:0/3736701413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7819d8c0 con 0x7f7a781036e0 2024-09-11T23:36:01.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.668+0000 7f7a74ff9640 1 -- 172.21.15.133:0/3736701413 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000f2e0 con 0x7f7a781036e0 2024-09-11T23:36:01.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.668+0000 7f7a7d296640 1 -- 172.21.15.133:0/3736701413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a781a51a0 con 0x7f7a781036e0 2024-09-11T23:36:01.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.669+0000 7f7a7d296640 1 -- 172.21.15.133:0/3736701413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7806a850 con 0x7f7a781036e0 2024-09-11T23:36:01.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.670+0000 7f7a74ff9640 1 -- 172.21.15.133:0/3736701413 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7a6001b460 con 0x7f7a781036e0 2024-09-11T23:36:01.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.671+0000 7f7a74ff9640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7a640a2c90 0x7f7a640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:01.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.671+0000 7f7a76ffd640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7a640a2c90 0x7f7a640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:01.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.671+0000 7f7a74ff9640 1 -- 172.21.15.133:0/3736701413 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f7a600cec90 con 0x7f7a781036e0 2024-09-11T23:36:01.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.671+0000 7f7a76ffd640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7a640a2c90 0x7f7a640a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009900 tx=0x7f7a680058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:01.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.674+0000 7f7a74ff9640 1 -- 172.21.15.133:0/3736701413 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a6009bca0 con 0x7f7a781036e0 2024-09-11T23:36:01.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.875+0000 7f7a7d296640 1 -- 172.21.15.133:0/3736701413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7a78103110 con 0x7f7a781036e0 2024-09-11T23:36:01.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.876+0000 7f7a74ff9640 1 -- 172.21.15.133:0/3736701413 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f7a78103110 con 0x7f7a781036e0 2024-09-11T23:36:01.876 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:01.876 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T23:36:01.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 -- 172.21.15.133:0/3736701413 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7a640a2c90 msgr2=0x7f7a640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:01.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7a640a2c90 0x7f7a640a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009900 tx=0x7f7a680058d0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 -- 172.21.15.133:0/3736701413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a781036e0 msgr2=0x7f7a78198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:01.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a781036e0 0x7f7a78198b60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7a60006fd0 tx=0x7f7a6000b8e0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 -- 172.21.15.133:0/3736701413 shutdown_connections 2024-09-11T23:36:01.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7a640a2c90 0x7f7a640a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a781082a0 0x7f7a7819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a781036e0 0x7f7a78198b60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 --2- 172.21.15.133:0/3736701413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a78102d30 0x7f7a78198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:01.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 -- 172.21.15.133:0/3736701413 >> 172.21.15.133:0/3736701413 conn(0x7f7a780fc7d0 msgr2=0x7f7a78105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:01.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 -- 172.21.15.133:0/3736701413 shutdown_connections 2024-09-11T23:36:01.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:01.878+0000 7f7a7d296640 1 -- 172.21.15.133:0/3736701413 wait complete. 2024-09-11T23:36:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:02 smithi133 ceph-mon[31430]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:02.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:02 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3736701413' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:36:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:02 smithi178 ceph-mon[35882]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:02.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:02 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3736701413' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:36:02.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:02 smithi138 ceph-mon[35142]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:02 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3736701413' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:36:02.555 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T23:36:02.555 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:36:02.812 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:03.187 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:03 smithi133 ceph-mon[31430]: pgmap v308: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:03 smithi178 ceph-mon[35882]: pgmap v308: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.270+0000 7fc5cecbe640 1 -- 172.21.15.133:0/943332059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5c8100e20 msgr2=0x7fc5c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.270+0000 7fc5cecbe640 1 --2- 172.21.15.133:0/943332059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5c8100e20 0x7fc5c8101200 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009900 tx=0x7fc5bc02a480 comp rx=0 tx=0).stop 2024-09-11T23:36:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.270+0000 7fc5cecbe640 1 -- 172.21.15.133:0/943332059 shutdown_connections 2024-09-11T23:36:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.270+0000 7fc5cecbe640 1 --2- 172.21.15.133:0/943332059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c8102ef0 0x7fc5c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.270+0000 7fc5cecbe640 1 --2- 172.21.15.133:0/943332059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c8102570 0x7fc5c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.270+0000 7fc5cecbe640 1 --2- 172.21.15.133:0/943332059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5c8100e20 0x7fc5c8101200 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.270+0000 7fc5cecbe640 1 -- 172.21.15.133:0/943332059 >> 172.21.15.133:0/943332059 conn(0x7fc5c80fc7d0 msgr2=0x7fc5c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.270+0000 7fc5cecbe640 1 -- 172.21.15.133:0/943332059 shutdown_connections 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.270+0000 7fc5cecbe640 1 -- 172.21.15.133:0/943332059 wait complete. 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5cecbe640 1 Processor -- start 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5cecbe640 1 -- start start 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5cecbe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c8100e20 0x7fc5c8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5cecbe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5c8102570 0x7fc5c81998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5cecbe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c8102ef0 0x7fc5c81a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5cecbe640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8110f80 con 0x7fc5c8102570 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5cecbe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8110e00 con 0x7fc5c8100e20 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5cecbe640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8111100 con 0x7fc5c8102ef0 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5c7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5c8102570 0x7fc5c81998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5c7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5c8102570 0x7fc5c81998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59368/0 (socket says 172.21.15.133:59368) 2024-09-11T23:36:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5c7fff640 1 -- 172.21.15.133:0/141606253 learned_addr learned my addr 172.21.15.133:0/141606253 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.271+0000 7fc5cca5a640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c8100e20 0x7fc5c8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5c7fff640 1 -- 172.21.15.133:0/141606253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c8102ef0 msgr2=0x7fc5c81a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5cd25b640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c8102ef0 0x7fc5c81a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5c7fff640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c8102ef0 0x7fc5c81a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5c7fff640 1 -- 172.21.15.133:0/141606253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c8100e20 msgr2=0x7fc5c8199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5c7fff640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c8100e20 0x7fc5c8199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5c7fff640 1 -- 172.21.15.133:0/141606253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c81a4950 con 0x7fc5c8102570 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5cd25b640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c8102ef0 0x7fc5c81a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5c7fff640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5c8102570 0x7fc5c81998d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400e9f0 tx=0x7fc5b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5c5ffb640 1 -- 172.21.15.133:0/141606253 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400cd80 con 0x7fc5c8102570 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5c5ffb640 1 -- 172.21.15.133:0/141606253 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b4005500 con 0x7fc5c8102570 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5cecbe640 1 -- 172.21.15.133:0/141606253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c81a4b20 con 0x7fc5c8102570 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5c5ffb640 1 -- 172.21.15.133:0/141606253 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4010690 con 0x7fc5c8102570 2024-09-11T23:36:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.272+0000 7fc5cecbe640 1 -- 172.21.15.133:0/141606253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c81a5010 con 0x7fc5c8102570 2024-09-11T23:36:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.273+0000 7fc5cecbe640 1 -- 172.21.15.133:0/141606253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c8101200 con 0x7fc5c8102570 2024-09-11T23:36:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.274+0000 7fc5c5ffb640 1 -- 172.21.15.133:0/141606253 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc5b40046c0 con 0x7fc5c8102570 2024-09-11T23:36:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.275+0000 7fc5c5ffb640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5b80a2c10 0x7fc5b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.275+0000 7fc5c5ffb640 1 -- 172.21.15.133:0/141606253 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fc5b40cf5a0 con 0x7fc5c8102570 2024-09-11T23:36:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.275+0000 7fc5cca5a640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5b80a2c10 0x7fc5b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.275+0000 7fc5cca5a640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5b80a2c10 0x7fc5b80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009230 tx=0x7fc5bc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.279+0000 7fc5c5ffb640 1 -- 172.21.15.133:0/141606253 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5b409c4a0 con 0x7fc5c8102570 2024-09-11T23:36:03.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:03 smithi138 ceph-mon[35142]: pgmap v308: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.491+0000 7fc5cecbe640 1 -- 172.21.15.133:0/141606253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5c802d7c0 con 0x7fc5c8102570 2024-09-11T23:36:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.491+0000 7fc5c5ffb640 1 -- 172.21.15.133:0/141606253 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fc5b409bda0 con 0x7fc5c8102570 2024-09-11T23:36:03.492 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:03.492 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"btime":"2024-09-11T23:35:05:451373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:35:05.451342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24472,24464,24419,24401]},"id":1}]} 2024-09-11T23:36:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 -- 172.21.15.133:0/141606253 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5b80a2c10 msgr2=0x7fc5b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5b80a2c10 0x7fc5b80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009230 tx=0x7fc5bc0058d0 comp rx=0 tx=0).stop 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 -- 172.21.15.133:0/141606253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5c8102570 msgr2=0x7fc5c81998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5c8102570 0x7fc5c81998d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400e9f0 tx=0x7fc5b400edf0 comp rx=0 tx=0).stop 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 -- 172.21.15.133:0/141606253 shutdown_connections 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc5b80a2c10 0x7fc5b80a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5c8102ef0 0x7fc5c81a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5c8102570 0x7fc5c81998d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 --2- 172.21.15.133:0/141606253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c8100e20 0x7fc5c8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 -- 172.21.15.133:0/141606253 >> 172.21.15.133:0/141606253 conn(0x7fc5c80fc7d0 msgr2=0x7fc5c80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 -- 172.21.15.133:0/141606253 shutdown_connections 2024-09-11T23:36:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:03.494+0000 7fc5cecbe640 1 -- 172.21.15.133:0/141606253 wait complete. 2024-09-11T23:36:04.108 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:04 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/141606253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:04.136 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:36:04.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:04 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/141606253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:04.386 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:04.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:04 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/141606253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:04.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.887+0000 7f837d58a640 1 -- 172.21.15.133:0/3820691749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8378100a80 msgr2=0x7f8378100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:04.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.887+0000 7f837d58a640 1 --2- 172.21.15.133:0/3820691749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8378100a80 0x7f8378100ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8360009930 tx=0x7f836002a420 comp rx=0 tx=0).stop 2024-09-11T23:36:04.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.887+0000 7f837d58a640 1 -- 172.21.15.133:0/3820691749 shutdown_connections 2024-09-11T23:36:04.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.887+0000 7f837d58a640 1 --2- 172.21.15.133:0/3820691749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378101420 0x7f837810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.887+0000 7f837d58a640 1 --2- 172.21.15.133:0/3820691749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8378100a80 0x7f8378100ee0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.887+0000 7f837d58a640 1 --2- 172.21.15.133:0/3820691749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8378106a80 0x7f8378106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.887+0000 7f837d58a640 1 -- 172.21.15.133:0/3820691749 >> 172.21.15.133:0/3820691749 conn(0x7f83780fc7a0 msgr2=0x7f83780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:04.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.887+0000 7f837d58a640 1 -- 172.21.15.133:0/3820691749 shutdown_connections 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.887+0000 7f837d58a640 1 -- 172.21.15.133:0/3820691749 wait complete. 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.887+0000 7f837d58a640 1 Processor -- start 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.887+0000 7f837d58a640 1 -- start start 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f837d58a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378100a80 0x7f8378199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f837d58a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8378101420 0x7f8378199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f837d58a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8378106a80 0x7f83781a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f837d58a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8378110e40 con 0x7f8378101420 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f837d58a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8378110cc0 con 0x7f8378100a80 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f837d58a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8378110fc0 con 0x7f8378106a80 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f83767fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8378101420 0x7f8378199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f83767fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8378101420 0x7f8378199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59392/0 (socket says 172.21.15.133:59392) 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f83767fc640 1 -- 172.21.15.133:0/1978615994 learned_addr learned my addr 172.21.15.133:0/1978615994 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f83777fe640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8378106a80 0x7f83781a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f8376ffd640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378100a80 0x7f8378199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f83767fc640 1 -- 172.21.15.133:0/1978615994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8378106a80 msgr2=0x7f83781a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f83767fc640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8378106a80 0x7f83781a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f83767fc640 1 -- 172.21.15.133:0/1978615994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378100a80 msgr2=0x7f8378199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f83767fc640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378100a80 0x7f8378199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f83767fc640 1 -- 172.21.15.133:0/1978615994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83781a48d0 con 0x7f8378101420 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f83767fc640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8378101420 0x7f8378199850 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8360009230 tx=0x7f8360004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f836ffff640 1 -- 172.21.15.133:0/1978615994 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836003f030 con 0x7f8378101420 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f837d58a640 1 -- 172.21.15.133:0/1978615994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83781a4aa0 con 0x7f8378101420 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f836ffff640 1 -- 172.21.15.133:0/1978615994 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8360003ad0 con 0x7f8378101420 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f836ffff640 1 -- 172.21.15.133:0/1978615994 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360035070 con 0x7f8378101420 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.888+0000 7f837d58a640 1 -- 172.21.15.133:0/1978615994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83781a4f50 con 0x7f8378101420 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.889+0000 7f837d58a640 1 -- 172.21.15.133:0/1978615994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8378100ee0 con 0x7f8378101420 2024-09-11T23:36:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.890+0000 7f836ffff640 1 -- 172.21.15.133:0/1978615994 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8360042030 con 0x7f8378101420 2024-09-11T23:36:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.891+0000 7f836ffff640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f83640a2c10 0x7f83640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.891+0000 7f8376ffd640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f83640a2c10 0x7f83640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.891+0000 7f836ffff640 1 -- 172.21.15.133:0/1978615994 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f83600ef5c0 con 0x7f8378101420 2024-09-11T23:36:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.891+0000 7f8376ffd640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f83640a2c10 0x7f83640a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f83680099e0 tx=0x7f8368005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:04.895+0000 7f836ffff640 1 -- 172.21.15.133:0/1978615994 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83600bc4c0 con 0x7f8378101420 2024-09-11T23:36:05.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.097+0000 7f837d58a640 1 -- 172.21.15.133:0/1978615994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8378109010 con 0x7f8378101420 2024-09-11T23:36:05.098 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:05 smithi133 ceph-mon[31430]: pgmap v309: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:05.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.098+0000 7f836ffff640 1 -- 172.21.15.133:0/1978615994 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f83600bbdc0 con 0x7f8378101420 2024-09-11T23:36:05.098 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:05.099 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"btime":"2024-09-11T23:35:05:451373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:35:05.451342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24472,24464,24419,24401]},"id":1}]} 2024-09-11T23:36:05.099 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-09-11T23:36:05.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.100+0000 7f837d58a640 1 -- 172.21.15.133:0/1978615994 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f83640a2c10 msgr2=0x7f83640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:05.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.100+0000 7f837d58a640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f83640a2c10 0x7f83640a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f83680099e0 tx=0x7f8368005ed0 comp rx=0 tx=0).stop 2024-09-11T23:36:05.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.100+0000 7f837d58a640 1 -- 172.21.15.133:0/1978615994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8378101420 msgr2=0x7f8378199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:05.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.100+0000 7f837d58a640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8378101420 0x7f8378199850 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8360009230 tx=0x7f8360004270 comp rx=0 tx=0).stop 2024-09-11T23:36:05.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.100+0000 7f837d58a640 1 -- 172.21.15.133:0/1978615994 shutdown_connections 2024-09-11T23:36:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.100+0000 7f837d58a640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f83640a2c10 0x7f83640a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.100+0000 7f837d58a640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8378106a80 0x7f83781a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.100+0000 7f837d58a640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8378101420 0x7f8378199850 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.100+0000 7f837d58a640 1 --2- 172.21.15.133:0/1978615994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378100a80 0x7f8378199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.100+0000 7f837d58a640 1 -- 172.21.15.133:0/1978615994 >> 172.21.15.133:0/1978615994 conn(0x7f83780fc7a0 msgr2=0x7f83780fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.101+0000 7f837d58a640 1 -- 172.21.15.133:0/1978615994 shutdown_connections 2024-09-11T23:36:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:05.101+0000 7f837d58a640 1 -- 172.21.15.133:0/1978615994 wait complete. 2024-09-11T23:36:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:05 smithi178 ceph-mon[35882]: pgmap v309: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:05.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:05 smithi138 ceph-mon[35142]: pgmap v309: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:05.789 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:36:06.049 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:06.171 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:06 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1978615994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:06.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:06 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1978615994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:06.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:06 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1978615994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:06.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.500+0000 7fc710a3d640 1 -- 172.21.15.133:0/3627439215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc70c102ef0 msgr2=0x7fc70c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:06.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.500+0000 7fc710a3d640 1 --2- 172.21.15.133:0/3627439215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc70c102ef0 0x7fc70c10da40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009900 tx=0x7fc6fc02a480 comp rx=0 tx=0).stop 2024-09-11T23:36:06.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.500+0000 7fc710a3d640 1 -- 172.21.15.133:0/3627439215 shutdown_connections 2024-09-11T23:36:06.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.500+0000 7fc710a3d640 1 --2- 172.21.15.133:0/3627439215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc70c102ef0 0x7fc70c10da40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:06.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.500+0000 7fc710a3d640 1 --2- 172.21.15.133:0/3627439215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc70c102570 0x7fc70c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:06.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.500+0000 7fc710a3d640 1 --2- 172.21.15.133:0/3627439215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc70c100e20 0x7fc70c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:06.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.500+0000 7fc710a3d640 1 -- 172.21.15.133:0/3627439215 >> 172.21.15.133:0/3627439215 conn(0x7fc70c0fc7d0 msgr2=0x7fc70c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:06.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.500+0000 7fc710a3d640 1 -- 172.21.15.133:0/3627439215 shutdown_connections 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.500+0000 7fc710a3d640 1 -- 172.21.15.133:0/3627439215 wait complete. 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc710a3d640 1 Processor -- start 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc710a3d640 1 -- start start 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc710a3d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc70c100e20 0x7fc70c197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc710a3d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc70c102570 0x7fc70c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc710a3d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc70c102ef0 0x7fc70c19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc710a3d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c110fc0 con 0x7fc70c102ef0 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc710a3d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c110e40 con 0x7fc70c100e20 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc710a3d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c111140 con 0x7fc70c102570 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc70ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc70c102ef0 0x7fc70c19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc70ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc70c102ef0 0x7fc70c19bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59408/0 (socket says 172.21.15.133:59408) 2024-09-11T23:36:06.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc70ad9d640 1 -- 172.21.15.133:0/3181099449 learned_addr learned my addr 172.21.15.133:0/3181099449 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc709d9b640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc70c102570 0x7fc70c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.501+0000 7fc70a59c640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc70c100e20 0x7fc70c197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.502+0000 7fc70ad9d640 1 -- 172.21.15.133:0/3181099449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc70c102570 msgr2=0x7fc70c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.502+0000 7fc70ad9d640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc70c102570 0x7fc70c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.502+0000 7fc70ad9d640 1 -- 172.21.15.133:0/3181099449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc70c100e20 msgr2=0x7fc70c197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.502+0000 7fc70ad9d640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc70c100e20 0x7fc70c197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.502+0000 7fc70ad9d640 1 -- 172.21.15.133:0/3181099449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c19c3b0 con 0x7fc70c102ef0 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.502+0000 7fc70ad9d640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc70c102ef0 0x7fc70c19bcd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc004770 tx=0x7fc6fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.502+0000 7fc7037fe640 1 -- 172.21.15.133:0/3181099449 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc03f070 con 0x7fc70c102ef0 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.502+0000 7fc710a3d640 1 -- 172.21.15.133:0/3181099449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c19c640 con 0x7fc70c102ef0 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.502+0000 7fc7037fe640 1 -- 172.21.15.133:0/3181099449 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6fc031c30 con 0x7fc70c102ef0 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.502+0000 7fc7037fe640 1 -- 172.21.15.133:0/3181099449 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc035030 con 0x7fc70c102ef0 2024-09-11T23:36:06.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.502+0000 7fc710a3d640 1 -- 172.21.15.133:0/3181099449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c1a3f80 con 0x7fc70c102ef0 2024-09-11T23:36:06.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.503+0000 7fc710a3d640 1 -- 172.21.15.133:0/3181099449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c101200 con 0x7fc70c102ef0 2024-09-11T23:36:06.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.504+0000 7fc7037fe640 1 -- 172.21.15.133:0/3181099449 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc6fc042070 con 0x7fc70c102ef0 2024-09-11T23:36:06.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.505+0000 7fc7037fe640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6f80a2c10 0x7fc6f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:06.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.505+0000 7fc70a59c640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6f80a2c10 0x7fc6f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:06.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.505+0000 7fc7037fe640 1 -- 172.21.15.133:0/3181099449 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fc6fc0efc60 con 0x7fc70c102ef0 2024-09-11T23:36:06.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.505+0000 7fc70a59c640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6f80a2c10 0x7fc6f80a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc704005fd0 tx=0x7fc704009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:06.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.509+0000 7fc7037fe640 1 -- 172.21.15.133:0/3181099449 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6fc0bcb60 con 0x7fc70c102ef0 2024-09-11T23:36:06.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.678+0000 7fc710a3d640 1 -- 172.21.15.133:0/3181099449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc70c1a4150 con 0x7fc70c102ef0 2024-09-11T23:36:06.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.680+0000 7fc7037fe640 1 -- 172.21.15.133:0/3181099449 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22677 (secure 0 0 0) 0x7fc6fc0bc460 con 0x7fc70c102ef0 2024-09-11T23:36:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:06.679 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":81,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:35:08.840852+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:36:06.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 -- 172.21.15.133:0/3181099449 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6f80a2c10 msgr2=0x7fc6f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:06.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6f80a2c10 0x7fc6f80a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc704005fd0 tx=0x7fc704009870 comp rx=0 tx=0).stop 2024-09-11T23:36:06.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 -- 172.21.15.133:0/3181099449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc70c102ef0 msgr2=0x7fc70c19bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:06.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc70c102ef0 0x7fc70c19bcd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc004770 tx=0x7fc6fc003c10 comp rx=0 tx=0).stop 2024-09-11T23:36:06.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 -- 172.21.15.133:0/3181099449 shutdown_connections 2024-09-11T23:36:06.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc6f80a2c10 0x7fc6f80a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:06.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc70c102ef0 0x7fc70c19bcd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:06.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc70c102570 0x7fc70c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:06.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 --2- 172.21.15.133:0/3181099449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc70c100e20 0x7fc70c197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:06.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 -- 172.21.15.133:0/3181099449 >> 172.21.15.133:0/3181099449 conn(0x7fc70c0fc7d0 msgr2=0x7fc70c106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:06.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 -- 172.21.15.133:0/3181099449 shutdown_connections 2024-09-11T23:36:06.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:06.682+0000 7fc710a3d640 1 -- 172.21.15.133:0/3181099449 wait complete. 2024-09-11T23:36:07.280 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:07 smithi133 ceph-mon[31430]: pgmap v310: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:07.281 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:07 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3181099449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:07.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:07 smithi178 ceph-mon[35882]: pgmap v310: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:07.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:07 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3181099449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:07.309 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:36:07.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:07 smithi138 ceph-mon[35142]: pgmap v310: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:07.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:07 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3181099449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:07.556 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:08.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.980+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/998702921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc08106a80 msgr2=0x7fbc08106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.980+0000 7fbc0fd6e640 1 --2- 172.21.15.133:0/998702921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc08106a80 0x7fbc08106e60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009930 tx=0x7fbc0402a420 comp rx=0 tx=0).stop 2024-09-11T23:36:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.980+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/998702921 shutdown_connections 2024-09-11T23:36:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.980+0000 7fbc0fd6e640 1 --2- 172.21.15.133:0/998702921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc08101420 0x7fbc0810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.980+0000 7fbc0fd6e640 1 --2- 172.21.15.133:0/998702921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc08100a80 0x7fbc08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.980+0000 7fbc0fd6e640 1 --2- 172.21.15.133:0/998702921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc08106a80 0x7fbc08106e60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.980+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/998702921 >> 172.21.15.133:0/998702921 conn(0x7fbc080fc7a0 msgr2=0x7fbc080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/998702921 shutdown_connections 2024-09-11T23:36:08.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/998702921 wait complete. 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0fd6e640 1 Processor -- start 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0fd6e640 1 -- start start 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0fd6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc08100a80 0x7fbc08198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0fd6e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc08101420 0x7fbc08198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0fd6e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc08106a80 0x7fbc0819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0fd6e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbc08110ea0 con 0x7fbc08100a80 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0fd6e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc08110d20 con 0x7fbc08106a80 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0fd6e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc08111020 con 0x7fbc08101420 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0db0a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc08100a80 0x7fbc08198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0db0a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc08100a80 0x7fbc08198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59426/0 (socket says 172.21.15.133:59426) 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.981+0000 7fbc0db0a640 1 -- 172.21.15.133:0/1810914116 learned_addr learned my addr 172.21.15.133:0/1810914116 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:08.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbc0e30b640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc08106a80 0x7fbc0819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbc0d309640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc08101420 0x7fbc08198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbc0db0a640 1 -- 172.21.15.133:0/1810914116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc08101420 msgr2=0x7fbc08198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbc0db0a640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc08101420 0x7fbc08198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbc0db0a640 1 -- 172.21.15.133:0/1810914116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc08106a80 msgr2=0x7fbc0819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbc0db0a640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc08106a80 0x7fbc0819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbc0db0a640 1 -- 172.21.15.133:0/1810914116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0819d3c0 con 0x7fbc08100a80 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbc0d309640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc08101420 0x7fbc08198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbc0db0a640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc08100a80 0x7fbc08198410 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009a50 tx=0x7fbc04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbbfeffd640 1 -- 172.21.15.133:0/1810914116 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0403f030 con 0x7fbc08100a80 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbbfeffd640 1 -- 172.21.15.133:0/1810914116 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc04003ce0 con 0x7fbc08100a80 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/1810914116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0819d650 con 0x7fbc08100a80 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbbfeffd640 1 -- 172.21.15.133:0/1810914116 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04035070 con 0x7fbc08100a80 2024-09-11T23:36:08.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.982+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/1810914116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc081a4f90 con 0x7fbc08100a80 2024-09-11T23:36:08.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.983+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/1810914116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc081021c0 con 0x7fbc08100a80 2024-09-11T23:36:08.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.985+0000 7fbbfeffd640 1 -- 172.21.15.133:0/1810914116 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbc04042030 con 0x7fbc08100a80 2024-09-11T23:36:08.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.985+0000 7fbbfeffd640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf40a2c10 0x7fbbf40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:08.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.985+0000 7fbc0d309640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf40a2c10 0x7fbbf40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:08.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.985+0000 7fbbfeffd640 1 -- 172.21.15.133:0/1810914116 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fbc040efac0 con 0x7fbc08100a80 2024-09-11T23:36:08.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.986+0000 7fbc0d309640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf40a2c10 0x7fbbf40a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbc0806b7e0 tx=0x7fbbf800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:08.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:07.988+0000 7fbbfeffd640 1 -- 172.21.15.133:0/1810914116 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc040bc9c0 con 0x7fbc08100a80 2024-09-11T23:36:08.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.190+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/1810914116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc08100ee0 con 0x7fbc08100a80 2024-09-11T23:36:08.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.191+0000 7fbbfeffd640 1 -- 172.21.15.133:0/1810914116 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fbc08100ee0 con 0x7fbc08100a80 2024-09-11T23:36:08.191 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:08.191 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"btime":"2024-09-11T23:35:05:451373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:35:05.451342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24472,24464,24419,24401]},"id":1}]} 2024-09-11T23:36:08.191 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-09-11T23:36:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/1810914116 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf40a2c10 msgr2=0x7fbbf40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf40a2c10 0x7fbbf40a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbc0806b7e0 tx=0x7fbbf800a5c0 comp rx=0 tx=0).stop 2024-09-11T23:36:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/1810914116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc08100a80 msgr2=0x7fbc08198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc08100a80 0x7fbc08198410 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009a50 tx=0x7fbc04004270 comp rx=0 tx=0).stop 2024-09-11T23:36:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/1810914116 shutdown_connections 2024-09-11T23:36:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf40a2c10 0x7fbbf40a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc08106a80 0x7fbc0819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc08101420 0x7fbc08198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 --2- 172.21.15.133:0/1810914116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc08100a80 0x7fbc08198410 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/1810914116 >> 172.21.15.133:0/1810914116 conn(0x7fbc080fc7a0 msgr2=0x7fbc08104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/1810914116 shutdown_connections 2024-09-11T23:36:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:08.193+0000 7fbc0fd6e640 1 -- 172.21.15.133:0/1810914116 wait complete. 2024-09-11T23:36:08.912 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:34:53.462890+0000', 'modified': '2024-09-11T23:35:05.451342+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24481, 'mds_1': 24472, 'mds_2': 24464, 'mds_3': 24419, 'mds_4': 24401}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6839/570951791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 570951791}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 570951791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6841/1294883246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6840', 'nonce': 1294883246}, {'type': 'v1', 'addr': '172.21.15.133:6841', 'nonce': 1294883246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.138:6833/1969946111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1969946111}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1969946111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.138:6835/137089847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 137089847}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 137089847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/3569562128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 3569562128}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 3569562128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6837/1115908278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 1115908278}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 1115908278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/1681049778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 1681049778}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 1681049778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.138:6839/2407895128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 2407895128}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 2407895128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.178:6837/1040997001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 1040997001}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 1040997001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.178:6839/3794667907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 3794667907}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 3794667907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24481, 'qdb_cluster': [24481, 24472, 24464, 24419, 24401]} 2024-09-11T23:36:08.912 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T23:36:08.912 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-11T23:36:09.158 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:09.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:09 smithi133 ceph-mon[31430]: pgmap v311: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:09.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:09 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1810914116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:09 smithi178 ceph-mon[35882]: pgmap v311: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:09.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:09 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1810914116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:09.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:09 smithi138 ceph-mon[35142]: pgmap v311: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:09.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:09 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1810914116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:09.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.680+0000 7f151dfcc640 1 -- 172.21.15.133:0/141501152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1518102de0 msgr2=0x7f15181031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.680+0000 7f151dfcc640 1 --2- 172.21.15.133:0/141501152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1518102de0 0x7f15181031c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1508009900 tx=0x7f150802a480 comp rx=0 tx=0).stop 2024-09-11T23:36:09.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.680+0000 7f151dfcc640 1 -- 172.21.15.133:0/141501152 shutdown_connections 2024-09-11T23:36:09.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.680+0000 7f151dfcc640 1 --2- 172.21.15.133:0/141501152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f151810c180 0x7f151810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.680+0000 7f151dfcc640 1 --2- 172.21.15.133:0/141501152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1518103700 0x7f151810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.680+0000 7f151dfcc640 1 --2- 172.21.15.133:0/141501152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1518102de0 0x7f15181031c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.680+0000 7f151dfcc640 1 -- 172.21.15.133:0/141501152 >> 172.21.15.133:0/141501152 conn(0x7f15180fc7a0 msgr2=0x7f15180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.680+0000 7f151dfcc640 1 -- 172.21.15.133:0/141501152 shutdown_connections 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.680+0000 7f151dfcc640 1 -- 172.21.15.133:0/141501152 wait complete. 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f151dfcc640 1 Processor -- start 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f151dfcc640 1 -- start start 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f151dfcc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1518102de0 0x7f1518198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f151dfcc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1518103700 0x7f15181989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f151dfcc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f151810c180 0x7f151819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f151dfcc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1518111fb0 con 0x7f1518102de0 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f151dfcc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1518111e30 con 0x7f151810c180 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f151dfcc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1518112130 con 0x7f1518103700 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f15177fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1518102de0 0x7f1518198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f15177fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1518102de0 0x7f1518198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59454/0 (socket says 172.21.15.133:59454) 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f15177fe640 1 -- 172.21.15.133:0/78850722 learned_addr learned my addr 172.21.15.133:0/78850722 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f1517fff640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f151810c180 0x7f151819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f150ffff640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1518103700 0x7f15181989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f15177fe640 1 -- 172.21.15.133:0/78850722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1518103700 msgr2=0x7f15181989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f15177fe640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1518103700 0x7f15181989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f15177fe640 1 -- 172.21.15.133:0/78850722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f151810c180 msgr2=0x7f151819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f15177fe640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f151810c180 0x7f151819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f15177fe640 1 -- 172.21.15.133:0/78850722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151819d420 con 0x7f1518102de0 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f1517fff640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f151810c180 0x7f151819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.681+0000 7f150ffff640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1518103700 0x7f15181989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.682+0000 7f15177fe640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1518102de0 0x7f1518198470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1508006010 tx=0x7f15080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.682+0000 7f15157fa640 1 -- 172.21.15.133:0/78850722 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150803f030 con 0x7f1518102de0 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.682+0000 7f151dfcc640 1 -- 172.21.15.133:0/78850722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151819d6b0 con 0x7f1518102de0 2024-09-11T23:36:09.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.682+0000 7f15157fa640 1 -- 172.21.15.133:0/78850722 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1508003ce0 con 0x7f1518102de0 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.682+0000 7f15157fa640 1 -- 172.21.15.133:0/78850722 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508035070 con 0x7f1518102de0 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.682+0000 7f151dfcc640 1 -- 172.21.15.133:0/78850722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15181a4ff0 con 0x7f1518102de0 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.684+0000 7f15157fa640 1 -- 172.21.15.133:0/78850722 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1508042030 con 0x7f1518102de0 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.684+0000 7f15203a5740 1 Processor -- start 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.684+0000 7f15203a5740 1 -- start start 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.684+0000 7f15157fa640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f15040a2c10 0x7f15040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f1517fff640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f15040a2c10 0x7f15040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15203a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc54de0 0x55a67fc551c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15203a5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc5fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15203a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc5ffc0 0x55a67fc623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15157fa640 1 -- 172.21.15.133:0/78850722 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f15080ef940 con 0x7f1518102de0 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15203a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a67fbb7090 con 0x55a67fc54de0 2024-09-11T23:36:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15203a5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a67fb5ea70 con 0x55a67fc55700 2024-09-11T23:36:09.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15203a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a67fbb3050 con 0x55a67fc5ffc0 2024-09-11T23:36:09.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15177fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc54de0 0x55a67fc551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15177fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc54de0 0x55a67fc551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59470/0 (socket says 172.21.15.133:59470) 2024-09-11T23:36:09.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15177fe640 1 -- 172.21.15.133:0/440273569 learned_addr learned my addr 172.21.15.133:0/440273569 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f150ffff640 1 --2- 172.21.15.133:0/440273569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc5fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f1517fff640 1 --2- 172.21.15.133:0/440273569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc5ffc0 0x55a67fc623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15177fe640 1 -- 172.21.15.133:0/440273569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc5ffc0 msgr2=0x55a67fc623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15177fe640 1 --2- 172.21.15.133:0/440273569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc5ffc0 0x55a67fc623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f1517fff640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f15040a2c10 0x7f15040a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1510007ea0 tx=0x7f151000c690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15177fe640 1 -- 172.21.15.133:0/440273569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 msgr2=0x55a67fc5fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15177fe640 1 --2- 172.21.15.133:0/440273569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc5fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15177fe640 1 -- 172.21.15.133:0/440273569 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67fb6b130 con 0x55a67fc54de0 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f150ffff640 1 --2- 172.21.15.133:0/440273569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc5fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f15177fe640 1 --2- 172.21.15.133:0/440273569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc54de0 0x55a67fc551c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f15080ed290 tx=0x7f15080bbcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2272748106b456ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.685+0000 7f150dffb640 1 -- 172.21.15.133:0/440273569 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508049040 con 0x55a67fc54de0 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.686+0000 7f150dffb640 1 -- 172.21.15.133:0/440273569 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150803b580 con 0x55a67fc54de0 2024-09-11T23:36:09.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.686+0000 7f150dffb640 1 -- 172.21.15.133:0/440273569 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080c4d70 con 0x55a67fc54de0 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.686+0000 7f15203a5740 1 -- 172.21.15.133:0/440273569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc54de0 msgr2=0x55a67fc551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.686+0000 7f15203a5740 1 --2- 172.21.15.133:0/440273569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc54de0 0x55a67fc551c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f15080ed290 tx=0x7f15080bbcc0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.686+0000 7f15203a5740 1 -- 172.21.15.133:0/440273569 shutdown_connections 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.686+0000 7f15203a5740 1 --2- 172.21.15.133:0/440273569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc5ffc0 0x55a67fc623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.686+0000 7f15203a5740 1 --2- 172.21.15.133:0/440273569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc5fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.686+0000 7f15203a5740 1 --2- 172.21.15.133:0/440273569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc54de0 0x55a67fc551c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.686+0000 7f15203a5740 1 -- 172.21.15.133:0/440273569 >> 172.21.15.133:0/440273569 conn(0x55a67fc51370 msgr2=0x55a67fc51750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.686+0000 7f15203a5740 1 -- 172.21.15.133:0/440273569 shutdown_connections 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.686+0000 7f15203a5740 1 -- 172.21.15.133:0/440273569 wait complete. 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f15203a5740 1 -- start start 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f15203a5740 1 Processor -- start 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f15203a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc5f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f15203a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc55700 0x55a67fc5cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f15203a5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc5ffc0 0x55a67fc5d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f15203a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a67fc62950 con 0x55a67fc55700 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f15203a5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a67fb71f20 con 0x55a67fc5ffc0 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f15203a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a67fc62ad0 con 0x55a67fc54de0 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f15203a5740 10 client.0 _ll_register_callbacks cb 0x55a67fb3b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f150ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc55700 0x55a67fc5cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f150ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc55700 0x55a67fc5cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59486/0 (socket says 172.21.15.133:59486) 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f150ffff640 1 -- 172.21.15.133:0/3059241340 learned_addr learned my addr 172.21.15.133:0/3059241340 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f15177fe640 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc5f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.687+0000 7f1517fff640 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc5ffc0 0x55a67fc5d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f150ffff640 1 -- 172.21.15.133:0/3059241340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 msgr2=0x55a67fc5f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f150ffff640 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc5f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f150ffff640 1 -- 172.21.15.133:0/3059241340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc5ffc0 msgr2=0x55a67fc5d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f150ffff640 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc5ffc0 0x55a67fc5d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f150ffff640 1 -- 172.21.15.133:0/3059241340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67f9282b0 con 0x55a67fc55700 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f150ffff640 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc55700 0x55a67fc5cb70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f15000106a0 tx=0x7f1500010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785acd17413412fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f14f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f14f6ffd640 1 -- 172.21.15.133:0/3059241340 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500010ea0 con 0x55a67fc55700 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f15203a5740 1 -- 172.21.15.133:0/3059241340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a67fbc4600 con 0x55a67fc55700 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f14f6ffd640 1 -- 172.21.15.133:0/3059241340 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f1500004220 con 0x55a67fc55700 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f14f6ffd640 1 -- 172.21.15.133:0/3059241340 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1500004420 con 0x55a67fc55700 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f15203a5740 10 client.15003 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f15203a5740 1 -- 172.21.15.133:0/3059241340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a67fc6e210 con 0x55a67fc55700 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f14f6ffd640 1 -- 172.21.15.133:0/3059241340 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a67fbc4600 con 0x55a67fc55700 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.688+0000 7f14f6ffd640 1 -- 172.21.15.133:0/3059241340 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f150000f040 con 0x55a67fc55700 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.689+0000 7f15203a5740 10 client.15003 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.689+0000 7f15203a5740 10 client.15003 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-11T23:36:09.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.689+0000 7f15203a5740 20 client.15003 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.689+0000 7f15203a5740 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a67fc708d0 0x55a67fc71d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.689+0000 7f15203a5740 4 client.15003 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.689+0000 7f15203a5740 1 -- 172.21.15.133:0/3059241340 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a67fbbeb10 con 0x55a67fc708d0 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.689+0000 7f15177fe640 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a67fc708d0 0x55a67fc71d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.689+0000 7f15177fe640 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a67fc708d0 0x55a67fc71d40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=af54b4b306f695c7 server_cookie=3665788fa6070313 in_seq=0 out_seq=0 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.689+0000 7f14f6ffd640 10 client.15003 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f14f6ffd640 1 -- 172.21.15.133:0/3059241340 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a67fbbeb10 con 0x55a67fc708d0 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f14f6ffd640 10 client.15003 handle_command_reply: tid=0 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f15203a5740 1 client.15003 shutdown 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f15203a5740 1 -- 172.21.15.133:0/3059241340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc55700 msgr2=0x55a67fc5cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f15203a5740 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc55700 0x55a67fc5cb70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f15000106a0 tx=0x7f1500010b60 comp rx=0 tx=0).stop 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f15203a5740 1 -- 172.21.15.133:0/3059241340 shutdown_connections 2024-09-11T23:36:09.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f15203a5740 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a67fc708d0 0x55a67fc71d40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f15203a5740 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc5ffc0 0x55a67fc5d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f15203a5740 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc55700 0x55a67fc5cb70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f15203a5740 1 --2- 172.21.15.133:0/3059241340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc5f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f14f6ffd640 0 client.15003 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:36:09.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f15203a5740 1 -- 172.21.15.133:0/3059241340 >> 172.21.15.133:0/3059241340 conn(0x55a67fc51370 msgr2=0x55a67fc4c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f15203a5740 1 -- 172.21.15.133:0/3059241340 shutdown_connections 2024-09-11T23:36:09.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.690+0000 7f15203a5740 1 -- 172.21.15.133:0/3059241340 wait complete. 2024-09-11T23:36:09.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.691+0000 7f15203a5740 20 client.15003 trim_cache size 0 max 16384 2024-09-11T23:36:09.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f15203a5740 1 Processor -- start 2024-09-11T23:36:09.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f15203a5740 1 -- start start 2024-09-11T23:36:09.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f15203a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc6b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f15203a5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc51750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f15203a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc58130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f15203a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a67fc70080 con 0x55a67fc5ffc0 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f15203a5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a67fc70750 con 0x55a67fc55700 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f15203a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a67fc70e20 con 0x55a67fc54de0 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f1517fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc58130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f1517fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc58130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59498/0 (socket says 172.21.15.133:59498) 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f1517fff640 1 -- 172.21.15.133:0/249643319 learned_addr learned my addr 172.21.15.133:0/249643319 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f15177fe640 1 --2- 172.21.15.133:0/249643319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc6b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f1517fff640 1 -- 172.21.15.133:0/249643319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 msgr2=0x55a67fc6b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f1517fff640 1 --2- 172.21.15.133:0/249643319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc6b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f1517fff640 1 -- 172.21.15.133:0/249643319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 msgr2=0x55a67fc51750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f1517fff640 1 --2- 172.21.15.133:0/249643319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc51750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f1517fff640 1 -- 172.21.15.133:0/249643319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67fc54c10 con 0x55a67fc5ffc0 2024-09-11T23:36:09.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f15177fe640 1 --2- 172.21.15.133:0/249643319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc6b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:09.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.708+0000 7f1517fff640 1 --2- 172.21.15.133:0/249643319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc58130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1510011790 tx=0x7f1510014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59f60f86eb3e869 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f150d7fa640 1 -- 172.21.15.133:0/249643319 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151001f890 con 0x55a67fc5ffc0 2024-09-11T23:36:09.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f150d7fa640 1 -- 172.21.15.133:0/249643319 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1510017030 con 0x55a67fc5ffc0 2024-09-11T23:36:09.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f150d7fa640 1 -- 172.21.15.133:0/249643319 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151001e3e0 con 0x55a67fc5ffc0 2024-09-11T23:36:09.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f15203a5740 1 -- 172.21.15.133:0/249643319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 msgr2=0x55a67fc58130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f15203a5740 1 --2- 172.21.15.133:0/249643319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc58130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1510011790 tx=0x7f1510014000 comp rx=0 tx=0).stop 2024-09-11T23:36:09.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f15203a5740 1 -- 172.21.15.133:0/249643319 shutdown_connections 2024-09-11T23:36:09.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f15203a5740 1 --2- 172.21.15.133:0/249643319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc58130 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f15203a5740 1 --2- 172.21.15.133:0/249643319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc51750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f15203a5740 1 --2- 172.21.15.133:0/249643319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc6b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f15203a5740 1 -- 172.21.15.133:0/249643319 >> 172.21.15.133:0/249643319 conn(0x55a67fc51370 msgr2=0x55a67fc57a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f15203a5740 1 -- 172.21.15.133:0/249643319 shutdown_connections 2024-09-11T23:36:09.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.709+0000 7f15203a5740 1 -- 172.21.15.133:0/249643319 wait complete. 2024-09-11T23:36:09.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f15203a5740 1 -- start start 2024-09-11T23:36:09.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f15203a5740 1 Processor -- start 2024-09-11T23:36:09.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f15203a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc7a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f15203a5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc73b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f15203a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f15203a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a67fc71bc0 con 0x55a67fc5ffc0 2024-09-11T23:36:09.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f15203a5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a67fc714f0 con 0x55a67fc55700 2024-09-11T23:36:09.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f15203a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a67fc72290 con 0x55a67fc54de0 2024-09-11T23:36:09.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f15203a5740 10 client.0 _ll_register_callbacks cb 0x7f1500009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:09.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f1517fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f1517fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc740c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59500/0 (socket says 172.21.15.133:59500) 2024-09-11T23:36:09.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f1517fff640 1 -- 172.21.15.133:0/4146316856 learned_addr learned my addr 172.21.15.133:0/4146316856 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:09.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f15177fe640 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc7a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.710+0000 7f150ffff640 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc73b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f1517fff640 1 -- 172.21.15.133:0/4146316856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 msgr2=0x55a67fc7a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f1517fff640 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc7a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f1517fff640 1 -- 172.21.15.133:0/4146316856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 msgr2=0x55a67fc73b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f1517fff640 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc73b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f1517fff640 1 -- 172.21.15.133:0/4146316856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67fc5dbb0 con 0x55a67fc5ffc0 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f1517fff640 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc740c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f151000fab0 tx=0x7f151000f620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8483dc19cb7449 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f14f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f14f6ffd640 1 -- 172.21.15.133:0/4146316856 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510036060 con 0x55a67fc5ffc0 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f14f6ffd640 1 -- 172.21.15.133:0/4146316856 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f151001f8a0 con 0x55a67fc5ffc0 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f15203a5740 1 -- 172.21.15.133:0/4146316856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1510017030 con 0x55a67fc5ffc0 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f14f6ffd640 1 -- 172.21.15.133:0/4146316856 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f151001faa0 con 0x55a67fc5ffc0 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f15203a5740 10 client.15009 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f15203a5740 1 -- 172.21.15.133:0/4146316856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a67fc790d0 con 0x55a67fc5ffc0 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f14f6ffd640 1 -- 172.21.15.133:0/4146316856 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1510017030 con 0x55a67fc5ffc0 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f14f6ffd640 1 -- 172.21.15.133:0/4146316856 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f151001c050 con 0x55a67fc5ffc0 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.711+0000 7f15203a5740 10 client.15009 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f15203a5740 10 client.15009 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f15203a5740 20 client.15009 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f15203a5740 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a67fcdf600 0x55a67fcde310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f15203a5740 4 client.15009 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f15203a5740 1 -- 172.21.15.133:0/4146316856 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "status"}) -- 0x55a67fb3b0e0 con 0x55a67fcdf600 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f15177fe640 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a67fcdf600 0x55a67fcde310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f15177fe640 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a67fcdf600 0x55a67fcde310 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c2bef02080156325 server_cookie=1169f16f3f2c2c3f in_seq=0 out_seq=0 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f14f6ffd640 10 client.15009 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f14f6ffd640 1 -- 172.21.15.133:0/4146316856 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a67fb3b0e0 con 0x55a67fcdf600 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f14f6ffd640 10 client.15009 handle_command_reply: tid=0 2024-09-11T23:36:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f15203a5740 1 client.15009 shutdown 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f15203a5740 1 -- 172.21.15.133:0/4146316856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 msgr2=0x55a67fc740c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.712+0000 7f15203a5740 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc740c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f151000fab0 tx=0x7f151000f620 comp rx=0 tx=0).stop 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f15203a5740 1 -- 172.21.15.133:0/4146316856 shutdown_connections 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f15203a5740 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a67fcdf600 0x55a67fcde310 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f15203a5740 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a67fc5ffc0 0x55a67fc740c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f15203a5740 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67fc55700 0x55a67fc73b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f15203a5740 1 --2- 172.21.15.133:0/4146316856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a67fc54de0 0x55a67fc7a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f15203a5740 1 -- 172.21.15.133:0/4146316856 >> 172.21.15.133:0/4146316856 conn(0x55a67fc51370 msgr2=0x55a67fc4c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f14f6ffd640 0 client.15009 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f15203a5740 1 -- 172.21.15.133:0/4146316856 shutdown_connections 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f15203a5740 1 -- 172.21.15.133:0/4146316856 wait complete. 2024-09-11T23:36:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f15203a5740 20 client.15009 trim_cache size 0 max 16384 2024-09-11T23:36:09.729 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:09.729 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:36:09.729 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 4, 2024-09-11T23:36:09.729 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24401, 2024-09-11T23:36:09.729 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:36:09.729 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:36:09.729 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:36:09.729 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 65.264556691999999, 2024-09-11T23:36:09.729 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 27, 2024-09-11T23:36:09.729 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 81, 2024-09-11T23:36:09.729 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 79, 2024-09-11T23:36:09.730 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 114.40175654399999 2024-09-11T23:36:09.730 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:09.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f151dfcc640 1 -- 172.21.15.133:0/78850722 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f15040a2c10 msgr2=0x7f15040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f151dfcc640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f15040a2c10 0x7f15040a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1510007ea0 tx=0x7f151000c690 comp rx=0 tx=0).stop 2024-09-11T23:36:09.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f151dfcc640 1 -- 172.21.15.133:0/78850722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1518102de0 msgr2=0x7f1518198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:09.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f151dfcc640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1518102de0 0x7f1518198470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1508006010 tx=0x7f15080042f0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f151dfcc640 1 -- 172.21.15.133:0/78850722 shutdown_connections 2024-09-11T23:36:09.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f151dfcc640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f15040a2c10 0x7f15040a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f151dfcc640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f151810c180 0x7f151819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f151dfcc640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1518103700 0x7f15181989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f151dfcc640 1 --2- 172.21.15.133:0/78850722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1518102de0 0x7f1518198470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:09.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.713+0000 7f151dfcc640 1 -- 172.21.15.133:0/78850722 >> 172.21.15.133:0/78850722 conn(0x7f15180fc7a0 msgr2=0x7f15180fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:09.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.714+0000 7f151dfcc640 1 -- 172.21.15.133:0/78850722 shutdown_connections 2024-09-11T23:36:09.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:09.714+0000 7f151dfcc640 1 -- 172.21.15.133:0/78850722 wait complete. 2024-09-11T23:36:10.442 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T23:36:10.688 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:11.108 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:11 smithi133 ceph-mon[31430]: pgmap v312: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:11.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.190+0000 7f50099a9640 1 -- 172.21.15.133:0/3948519227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5004106a80 msgr2=0x7f5004106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:11.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.190+0000 7f50099a9640 1 --2- 172.21.15.133:0/3948519227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5004106a80 0x7f5004106e60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009930 tx=0x7f4ff402a420 comp rx=0 tx=0).stop 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.190+0000 7f50099a9640 1 -- 172.21.15.133:0/3948519227 shutdown_connections 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.190+0000 7f50099a9640 1 --2- 172.21.15.133:0/3948519227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5004101420 0x7f500410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.190+0000 7f50099a9640 1 --2- 172.21.15.133:0/3948519227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5004100a80 0x7f5004100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.190+0000 7f50099a9640 1 --2- 172.21.15.133:0/3948519227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5004106a80 0x7f5004106e60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.190+0000 7f50099a9640 1 -- 172.21.15.133:0/3948519227 >> 172.21.15.133:0/3948519227 conn(0x7f50040fc7a0 msgr2=0x7f50040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.190+0000 7f50099a9640 1 -- 172.21.15.133:0/3948519227 shutdown_connections 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.190+0000 7f50099a9640 1 -- 172.21.15.133:0/3948519227 wait complete. 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.190+0000 7f50099a9640 1 Processor -- start 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f50099a9640 1 -- start start 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f50099a9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5004100a80 0x7f50041992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f50099a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5004101420 0x7f5004199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f50099a9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5004106a80 0x7f50041a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f50099a9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5004110ea0 con 0x7f5004100a80 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f50099a9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5004110d20 con 0x7f5004101420 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f50099a9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5004111020 con 0x7f5004106a80 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f5002ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5004100a80 0x7f50041992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f5002ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5004100a80 0x7f50041992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59518/0 (socket says 172.21.15.133:59518) 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f5002ffd640 1 -- 172.21.15.133:0/1477892348 learned_addr learned my addr 172.21.15.133:0/1477892348 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f50027fc640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5004101420 0x7f5004199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f50037fe640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5004106a80 0x7f50041a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f5002ffd640 1 -- 172.21.15.133:0/1477892348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5004106a80 msgr2=0x7f50041a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f5002ffd640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5004106a80 0x7f50041a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f5002ffd640 1 -- 172.21.15.133:0/1477892348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5004101420 msgr2=0x7f5004199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f5002ffd640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5004101420 0x7f5004199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f5002ffd640 1 -- 172.21.15.133:0/1477892348 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50041a4890 con 0x7f5004100a80 2024-09-11T23:36:11.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f50037fe640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5004106a80 0x7f50041a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.191+0000 7f5002ffd640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5004100a80 0x7f50041992d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009230 tx=0x7f4ff4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.192+0000 7f50089a7640 1 -- 172.21.15.133:0/1477892348 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff403f030 con 0x7f5004100a80 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.192+0000 7f50089a7640 1 -- 172.21.15.133:0/1477892348 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff4003ad0 con 0x7f5004100a80 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.192+0000 7f50099a9640 1 -- 172.21.15.133:0/1477892348 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50041a4a60 con 0x7f5004100a80 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.192+0000 7f50089a7640 1 -- 172.21.15.133:0/1477892348 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4035070 con 0x7f5004100a80 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.192+0000 7f50099a9640 1 -- 172.21.15.133:0/1477892348 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50041a4f10 con 0x7f5004100a80 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.194+0000 7f50089a7640 1 -- 172.21.15.133:0/1477892348 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4ff4042030 con 0x7f5004100a80 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f500bd82740 1 Processor -- start 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f500bd82740 1 -- start start 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f500bd82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d4848d0 0x56036d484cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f500bd82740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4851f0 0x56036d493a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f500bd82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d493fc0 0x56036d4963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f500bd82740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56036d3eb090 con 0x56036d4851f0 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f500bd82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56036d392a70 con 0x56036d4848d0 2024-09-11T23:36:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f500bd82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56036d3e7050 con 0x56036d493fc0 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f5002ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4851f0 0x56036d493a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f5002ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4851f0 0x56036d493a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59524/0 (socket says 172.21.15.133:59524) 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f5002ffd640 1 -- 172.21.15.133:0/979497174 learned_addr learned my addr 172.21.15.133:0/979497174 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f50027fc640 1 --2- 172.21.15.133:0/979497174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d493fc0 0x56036d4963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f50037fe640 1 --2- 172.21.15.133:0/979497174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d4848d0 0x56036d484cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f50089a7640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4fec0a2c10 0x7f4fec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f5002ffd640 1 -- 172.21.15.133:0/979497174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d493fc0 msgr2=0x56036d4963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f5002ffd640 1 --2- 172.21.15.133:0/979497174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d493fc0 0x56036d4963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f5002ffd640 1 -- 172.21.15.133:0/979497174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d4848d0 msgr2=0x56036d484cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f5002ffd640 1 --2- 172.21.15.133:0/979497174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d4848d0 0x56036d484cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.195+0000 7f5002ffd640 1 -- 172.21.15.133:0/979497174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56036d39f130 con 0x56036d4851f0 2024-09-11T23:36:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.196+0000 7f50037fe640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4fec0a2c10 0x7f4fec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.196+0000 7f5002ffd640 1 --2- 172.21.15.133:0/979497174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4851f0 0x56036d493a80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40ecfb0 tx=0x7f4ff40b7930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb0973a7e92bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.196+0000 7f50089a7640 1 -- 172.21.15.133:0/1477892348 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f4ff40ef5c0 con 0x7f5004100a80 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.196+0000 7f4fe9ffb640 1 -- 172.21.15.133:0/979497174 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40bb900 con 0x56036d4851f0 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.196+0000 7f4fe9ffb640 1 -- 172.21.15.133:0/979497174 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff40e5e70 con 0x56036d4851f0 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.196+0000 7f4fe9ffb640 1 -- 172.21.15.133:0/979497174 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40bc020 con 0x56036d4851f0 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.196+0000 7f500bd82740 1 -- 172.21.15.133:0/979497174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4851f0 msgr2=0x56036d493a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.196+0000 7f50037fe640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4fec0a2c10 0x7f4fec0a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00f4f0 tx=0x7f4ffc004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.196+0000 7f500bd82740 1 --2- 172.21.15.133:0/979497174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4851f0 0x56036d493a80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40ecfb0 tx=0x7f4ff40b7930 comp rx=0 tx=0).stop 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 -- 172.21.15.133:0/979497174 shutdown_connections 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 --2- 172.21.15.133:0/979497174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d493fc0 0x56036d4963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 --2- 172.21.15.133:0/979497174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4851f0 0x56036d493a80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 --2- 172.21.15.133:0/979497174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d4848d0 0x56036d484cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 -- 172.21.15.133:0/979497174 >> 172.21.15.133:0/979497174 conn(0x56036d480f60 msgr2=0x56036d481340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:11.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 -- 172.21.15.133:0/979497174 shutdown_connections 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 -- 172.21.15.133:0/979497174 wait complete. 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 -- start start 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 Processor -- start 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d493850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d4851f0 0x56036d490ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d493fc0 0x56036d4910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56036d4969b0 con 0x56036d4848d0 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56036d3a5f20 con 0x56036d4851f0 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56036d496b30 con 0x56036d493fc0 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f500bd82740 10 client.0 _ll_register_callbacks cb 0x56036d36f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f5002ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d493850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.197+0000 7f5002ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d493850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59532/0 (socket says 172.21.15.133:59532) 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f5002ffd640 1 -- 172.21.15.133:0/3183813213 learned_addr learned my addr 172.21.15.133:0/3183813213 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:11.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f50027fc640 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d4851f0 0x56036d490ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f50037fe640 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d493fc0 0x56036d4910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f5002ffd640 1 -- 172.21.15.133:0/3183813213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d493fc0 msgr2=0x56036d4910e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f5002ffd640 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d493fc0 0x56036d4910e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f5002ffd640 1 -- 172.21.15.133:0/3183813213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d4851f0 msgr2=0x56036d490ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f5002ffd640 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d4851f0 0x56036d490ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f5002ffd640 1 -- 172.21.15.133:0/3183813213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56036d15c2b0 con 0x56036d4848d0 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f5002ffd640 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d493850 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40bbac0 tx=0x7f4ff40ed030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50dd252689f0ca2c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f4fe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/3183813213 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4049050 con 0x56036d4848d0 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/3183813213 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f4ff40cffa0 con 0x56036d4848d0 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f500bd82740 1 -- 172.21.15.133:0/3183813213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56036d3f8600 con 0x56036d4848d0 2024-09-11T23:36:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/3183813213 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ff40bb720 con 0x56036d4848d0 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f500bd82740 10 client.15021 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f500bd82740 1 -- 172.21.15.133:0/3183813213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56036d4969b0 con 0x56036d4848d0 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.198+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/3183813213 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56036d3f8600 con 0x56036d4848d0 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.199+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/3183813213 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4ff40ec130 con 0x56036d4848d0 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.199+0000 7f500bd82740 10 client.15021 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.199+0000 7f500bd82740 10 client.15021 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.199+0000 7f500bd82740 20 client.15021 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.199+0000 7f500bd82740 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56036d4a4650 0x56036d4a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.199+0000 7f500bd82740 4 client.15021 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.199+0000 7f500bd82740 1 -- 172.21.15.133:0/3183813213 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56036d3f2b10 con 0x56036d4a4650 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.199+0000 7f50027fc640 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56036d4a4650 0x56036d4a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.199+0000 7f50027fc640 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56036d4a4650 0x56036d4a5ac0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e4911a8352866f74 server_cookie=d478233d64c612b5 in_seq=0 out_seq=0 2024-09-11T23:36:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.199+0000 7f4fe2ffd640 10 client.15021 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:36:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/3183813213 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56036d3f2b10 con 0x56036d4a4650 2024-09-11T23:36:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f4fe2ffd640 10 client.15021 handle_command_reply: tid=0 2024-09-11T23:36:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f500bd82740 1 client.15021 shutdown 2024-09-11T23:36:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f500bd82740 1 -- 172.21.15.133:0/3183813213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 msgr2=0x56036d493850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f500bd82740 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d493850 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40bbac0 tx=0x7f4ff40ed030 comp rx=0 tx=0).stop 2024-09-11T23:36:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f500bd82740 1 -- 172.21.15.133:0/3183813213 shutdown_connections 2024-09-11T23:36:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f500bd82740 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56036d4a4650 0x56036d4a5ac0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f500bd82740 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d493fc0 0x56036d4910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f500bd82740 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d4851f0 0x56036d490ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f4fe2ffd640 0 client.15021 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:36:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f500bd82740 1 --2- 172.21.15.133:0/3183813213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d493850 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.200+0000 7f500bd82740 1 -- 172.21.15.133:0/3183813213 >> 172.21.15.133:0/3183813213 conn(0x56036d480f60 msgr2=0x56036d486cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.201+0000 7f500bd82740 1 -- 172.21.15.133:0/3183813213 shutdown_connections 2024-09-11T23:36:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.201+0000 7f500bd82740 1 -- 172.21.15.133:0/3183813213 wait complete. 2024-09-11T23:36:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.201+0000 7f500bd82740 20 client.15021 trim_cache size 0 max 16384 2024-09-11T23:36:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f500bd82740 1 Processor -- start 2024-09-11T23:36:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f500bd82740 1 -- start start 2024-09-11T23:36:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f500bd82740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f500bd82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 0x56036d48c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f500bd82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d493fc0 0x56036d48de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f500bd82740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56036d4a3e00 con 0x56036d4848d0 2024-09-11T23:36:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f500bd82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56036d4a44d0 con 0x56036d493fc0 2024-09-11T23:36:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f500bd82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56036d4a4ba0 con 0x56036d4851f0 2024-09-11T23:36:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f5002ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f5002ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59534/0 (socket says 172.21.15.133:59534) 2024-09-11T23:36:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f5002ffd640 1 -- 172.21.15.133:0/1230135281 learned_addr learned my addr 172.21.15.133:0/1230135281 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.219+0000 7f50027fc640 1 --2- 172.21.15.133:0/1230135281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 0x56036d48c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.220+0000 7f50037fe640 1 --2- 172.21.15.133:0/1230135281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d493fc0 0x56036d48de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.220+0000 7f5002ffd640 1 -- 172.21.15.133:0/1230135281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 msgr2=0x56036d48c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.220+0000 7f5002ffd640 1 --2- 172.21.15.133:0/1230135281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 0x56036d48c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.220+0000 7f5002ffd640 1 -- 172.21.15.133:0/1230135281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d493fc0 msgr2=0x56036d48de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.220+0000 7f5002ffd640 1 --2- 172.21.15.133:0/1230135281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d493fc0 0x56036d48de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.220+0000 7f5002ffd640 1 -- 172.21.15.133:0/1230135281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56036d493bc0 con 0x56036d4848d0 2024-09-11T23:36:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.220+0000 7f50027fc640 1 --2- 172.21.15.133:0/1230135281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 0x56036d48c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.220+0000 7f5002ffd640 1 --2- 172.21.15.133:0/1230135281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a3370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40ba070 tx=0x7f4ff40ba2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd113952b7537e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.220+0000 7f4fe97fa640 1 -- 172.21.15.133:0/1230135281 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4049070 con 0x56036d4848d0 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.220+0000 7f4fe97fa640 1 -- 172.21.15.133:0/1230135281 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff40ef3b0 con 0x56036d4848d0 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.220+0000 7f4fe97fa640 1 -- 172.21.15.133:0/1230135281 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40cff20 con 0x56036d4848d0 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.221+0000 7f500bd82740 1 -- 172.21.15.133:0/1230135281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 msgr2=0x56036d4a3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.221+0000 7f500bd82740 1 --2- 172.21.15.133:0/1230135281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a3370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40ba070 tx=0x7f4ff40ba2d0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.221+0000 7f500bd82740 1 -- 172.21.15.133:0/1230135281 shutdown_connections 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.221+0000 7f500bd82740 1 --2- 172.21.15.133:0/1230135281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d493fc0 0x56036d48de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.221+0000 7f500bd82740 1 --2- 172.21.15.133:0/1230135281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 0x56036d48c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.221+0000 7f500bd82740 1 --2- 172.21.15.133:0/1230135281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a3370 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.221+0000 7f500bd82740 1 -- 172.21.15.133:0/1230135281 >> 172.21.15.133:0/1230135281 conn(0x56036d480f60 msgr2=0x56036d48ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.221+0000 7f500bd82740 1 -- 172.21.15.133:0/1230135281 shutdown_connections 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.221+0000 7f500bd82740 1 -- 172.21.15.133:0/1230135281 wait complete. 2024-09-11T23:36:11.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.221+0000 7f500bd82740 1 -- start start 2024-09-11T23:36:11.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.221+0000 7f500bd82740 1 Processor -- start 2024-09-11T23:36:11.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f500bd82740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f500bd82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 0x56036d4a7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f500bd82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d493fc0 0x56036d50cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f500bd82740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56036d4a5940 con 0x56036d4848d0 2024-09-11T23:36:11.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f500bd82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56036d4a5270 con 0x56036d493fc0 2024-09-11T23:36:11.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f500bd82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56036d4a6010 con 0x56036d4851f0 2024-09-11T23:36:11.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f500bd82740 10 client.0 _ll_register_callbacks cb 0x7f4ffc0075e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:11.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f5002ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f5002ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59544/0 (socket says 172.21.15.133:59544) 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f5002ffd640 1 -- 172.21.15.133:0/1690992014 learned_addr learned my addr 172.21.15.133:0/1690992014 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f50027fc640 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 0x56036d4a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f50037fe640 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d493fc0 0x56036d50cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f5002ffd640 1 -- 172.21.15.133:0/1690992014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 msgr2=0x56036d4a7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f5002ffd640 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 0x56036d4a7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f5002ffd640 1 -- 172.21.15.133:0/1690992014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d493fc0 msgr2=0x56036d50cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f5002ffd640 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d493fc0 0x56036d50cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f5002ffd640 1 -- 172.21.15.133:0/1690992014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56036d4a7bd0 con 0x56036d4848d0 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f50027fc640 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 0x56036d4a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f50037fe640 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d493fc0 0x56036d50cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f5002ffd640 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a6dc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4032c00 tx=0x7f4ff40bbdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c157f077e5d4d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:11.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f4fe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/1690992014 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4049070 con 0x56036d4848d0 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/1690992014 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f4ff40cff20 con 0x56036d4848d0 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f500bd82740 1 -- 172.21.15.133:0/1690992014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff40ef3b0 con 0x56036d4848d0 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/1690992014 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ff40bb900 con 0x56036d4848d0 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.222+0000 7f500bd82740 10 client.15027 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f500bd82740 1 -- 172.21.15.133:0/1690992014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56036d4a7bd0 con 0x56036d4848d0 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/1690992014 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff40ef3b0 con 0x56036d4848d0 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/1690992014 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4ff40ec830 con 0x56036d4848d0 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f500bd82740 10 client.15027 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f500bd82740 10 client.15027 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f500bd82740 20 client.15027 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f500bd82740 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56036d512770 0x56036d511480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f500bd82740 4 client.15027 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f500bd82740 1 -- 172.21.15.133:0/1690992014 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "status"}) -- 0x56036d36f0e0 con 0x56036d512770 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f50027fc640 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56036d512770 0x56036d511480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f50027fc640 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56036d512770 0x56036d511480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=911b83b2065a87d7 server_cookie=a1234a952360808c in_seq=0 out_seq=0 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.223+0000 7f4fe2ffd640 10 client.15027 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f4fe2ffd640 1 -- 172.21.15.133:0/1690992014 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56036d36f0e0 con 0x56036d512770 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f4fe2ffd640 10 client.15027 handle_command_reply: tid=0 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 1 client.15027 shutdown 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 1 -- 172.21.15.133:0/1690992014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 msgr2=0x56036d4a6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a6dc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4032c00 tx=0x7f4ff40bbdc0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 1 -- 172.21.15.133:0/1690992014 shutdown_connections 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56036d512770 0x56036d511480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56036d493fc0 0x56036d50cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036d4851f0 0x56036d4a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f4fe2ffd640 0 client.15027 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:36:11.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 1 --2- 172.21.15.133:0/1690992014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56036d4848d0 0x56036d4a6dc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 1 -- 172.21.15.133:0/1690992014 >> 172.21.15.133:0/1690992014 conn(0x56036d480f60 msgr2=0x56036d48f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:11.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 1 -- 172.21.15.133:0/1690992014 shutdown_connections 2024-09-11T23:36:11.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 1 -- 172.21.15.133:0/1690992014 wait complete. 2024-09-11T23:36:11.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.224+0000 7f500bd82740 20 client.15027 trim_cache size 0 max 16384 2024-09-11T23:36:11.242 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:11.242 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:36:11.242 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 3, 2024-09-11T23:36:11.243 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24419, 2024-09-11T23:36:11.243 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:36:11.243 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:36:11.243 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:36:11.243 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 67.781236907999997, 2024-09-11T23:36:11.243 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 27, 2024-09-11T23:36:11.243 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 81, 2024-09-11T23:36:11.243 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 79, 2024-09-11T23:36:11.243 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 106.651540833 2024-09-11T23:36:11.243 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:11.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 -- 172.21.15.133:0/1477892348 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4fec0a2c10 msgr2=0x7f4fec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:11.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4fec0a2c10 0x7f4fec0a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00f4f0 tx=0x7f4ffc004b50 comp rx=0 tx=0).stop 2024-09-11T23:36:11.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 -- 172.21.15.133:0/1477892348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5004100a80 msgr2=0x7f50041992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:11.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5004100a80 0x7f50041992d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009230 tx=0x7f4ff4004270 comp rx=0 tx=0).stop 2024-09-11T23:36:11.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 -- 172.21.15.133:0/1477892348 shutdown_connections 2024-09-11T23:36:11.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4fec0a2c10 0x7f4fec0a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5004106a80 0x7f50041a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5004101420 0x7f5004199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 --2- 172.21.15.133:0/1477892348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5004100a80 0x7f50041992d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:11.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 -- 172.21.15.133:0/1477892348 >> 172.21.15.133:0/1477892348 conn(0x7f50040fc7a0 msgr2=0x7f50040fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:11.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 -- 172.21.15.133:0/1477892348 shutdown_connections 2024-09-11T23:36:11.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:11.225+0000 7f50099a9640 1 -- 172.21.15.133:0/1477892348 wait complete. 2024-09-11T23:36:11.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:11 smithi138 ceph-mon[35142]: pgmap v312: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:11.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:11 smithi178 ceph-mon[35882]: pgmap v312: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:11.906 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:36:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.637+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1038981698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c24100b30 msgr2=0x7f4c24100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.637+0000 7f4c2bca3640 1 --2- 172.21.15.133:0/1038981698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c24100b30 0x7f4c24100f10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009ca0 tx=0x7f4c2002a940 comp rx=0 tx=0).stop 2024-09-11T23:36:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.637+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1038981698 shutdown_connections 2024-09-11T23:36:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.637+0000 7f4c2bca3640 1 --2- 172.21.15.133:0/1038981698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c24105f40 0x7f4c241063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.637+0000 7f4c2bca3640 1 --2- 172.21.15.133:0/1038981698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c24101450 0x7f4c24105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.637+0000 7f4c2bca3640 1 --2- 172.21.15.133:0/1038981698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c24100b30 0x7f4c24100f10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.637+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1038981698 >> 172.21.15.133:0/1038981698 conn(0x7f4c240fc7a0 msgr2=0x7f4c240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.637+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1038981698 shutdown_connections 2024-09-11T23:36:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.637+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1038981698 wait complete. 2024-09-11T23:36:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.637+0000 7f4c2bca3640 1 Processor -- start 2024-09-11T23:36:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.637+0000 7f4c2bca3640 1 -- start start 2024-09-11T23:36:12.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c2bca3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c24100b30 0x7f4c241a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c2bca3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c24101450 0x7f4c241a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c2bca3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c24105f40 0x7f4c2419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c2bca3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241185d0 con 0x7f4c24100b30 2024-09-11T23:36:12.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c2bca3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24118450 con 0x7f4c24105f40 2024-09-11T23:36:12.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c2bca3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24118750 con 0x7f4c24101450 2024-09-11T23:36:12.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c29a3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c24100b30 0x7f4c241a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c29a3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c24100b30 0x7f4c241a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48208/0 (socket says 172.21.15.133:48208) 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c29a3f640 1 -- 172.21.15.133:0/1369219104 learned_addr learned my addr 172.21.15.133:0/1369219104 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c2a240640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c24105f40 0x7f4c2419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c29a3f640 1 -- 172.21.15.133:0/1369219104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c24101450 msgr2=0x7f4c241a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c24101450 0x7f4c241a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c29a3f640 1 -- 172.21.15.133:0/1369219104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c24105f40 msgr2=0x7f4c2419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c24105f40 0x7f4c2419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c29a3f640 1 -- 172.21.15.133:0/1369219104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2419efc0 con 0x7f4c24100b30 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c2a240640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c24105f40 0x7f4c2419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c24100b30 0x7f4c241a46d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4c20005cc0 tx=0x7f4c2002f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c1affd640 1 -- 172.21.15.133:0/1369219104 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20034040 con 0x7f4c24100b30 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c1affd640 1 -- 172.21.15.133:0/1369219104 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c20003e50 con 0x7f4c24100b30 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1369219104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2419f250 con 0x7f4c24100b30 2024-09-11T23:36:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c1affd640 1 -- 172.21.15.133:0/1369219104 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20039430 con 0x7f4c24100b30 2024-09-11T23:36:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.638+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1369219104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c241ab5a0 con 0x7f4c24100b30 2024-09-11T23:36:12.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c1affd640 1 -- 172.21.15.133:0/1369219104 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4c20037020 con 0x7f4c24100b30 2024-09-11T23:36:12.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2e07c740 1 Processor -- start 2024-09-11T23:36:12.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2e07c740 1 -- start start 2024-09-11T23:36:12.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22479d0 0x5589c2247db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22482f0 0x5589c2252670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c2254fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5589c21ae090 con 0x5589c22479d0 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589c2155a70 con 0x5589c22482f0 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589c21aa050 con 0x5589c2252bb0 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2a240640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22479d0 0x5589c2247db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2a240640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22479d0 0x5589c2247db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48210/0 (socket says 172.21.15.133:48210) 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2a240640 1 -- 172.21.15.133:0/4046200651 learned_addr learned my addr 172.21.15.133:0/4046200651 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c2923e640 1 --2- 172.21.15.133:0/4046200651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c2254fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.641+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/4046200651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22482f0 0x5589c2252670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c1affd640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c100a2c90 0x7f4c100a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c2a240640 1 -- 172.21.15.133:0/4046200651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 msgr2=0x5589c2254fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c2a240640 1 --2- 172.21.15.133:0/4046200651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c2254fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c2a240640 1 -- 172.21.15.133:0/4046200651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22482f0 msgr2=0x5589c2252670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c2a240640 1 --2- 172.21.15.133:0/4046200651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22482f0 0x5589c2252670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c2a240640 1 -- 172.21.15.133:0/4046200651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589c2162130 con 0x5589c22479d0 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/4046200651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22482f0 0x5589c2252670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c2a240640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c100a2c90 0x7f4c100a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c1affd640 1 -- 172.21.15.133:0/1369219104 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f4c200f0260 con 0x7f4c24100b30 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c2a240640 1 --2- 172.21.15.133:0/4046200651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22479d0 0x5589c2247db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00ab10 tx=0x7f4c1c00c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc82fc0cc4f5a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c0ffff640 1 -- 172.21.15.133:0/4046200651 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c013610 con 0x5589c22479d0 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c2a240640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c100a2c90 0x7f4c100a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c01e000 tx=0x7f4c1c01b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c0ffff640 1 -- 172.21.15.133:0/4046200651 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c1c013d50 con 0x5589c22479d0 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.642+0000 7f4c0ffff640 1 -- 172.21.15.133:0/4046200651 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c01cc60 con 0x5589c22479d0 2024-09-11T23:36:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.643+0000 7f4c2e07c740 1 -- 172.21.15.133:0/4046200651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22479d0 msgr2=0x5589c2247db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.643+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/4046200651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22479d0 0x5589c2247db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00ab10 tx=0x7f4c1c00c150 comp rx=0 tx=0).stop 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.643+0000 7f4c2e07c740 1 -- 172.21.15.133:0/4046200651 shutdown_connections 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.643+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/4046200651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c2254fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.643+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/4046200651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22482f0 0x5589c2252670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.643+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/4046200651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22479d0 0x5589c2247db0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.643+0000 7f4c2e07c740 1 -- 172.21.15.133:0/4046200651 >> 172.21.15.133:0/4046200651 conn(0x5589c2243f60 msgr2=0x5589c2244340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.643+0000 7f4c2e07c740 1 -- 172.21.15.133:0/4046200651 shutdown_connections 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.643+0000 7f4c2e07c740 1 -- 172.21.15.133:0/4046200651 wait complete. 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.643+0000 7f4c2e07c740 1 -- start start 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.643+0000 7f4c2e07c740 1 Processor -- start 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22479d0 0x5589c22524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c224fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5589c2255780 con 0x5589c22482f0 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589c2168f20 con 0x5589c22479d0 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589c2255900 con 0x5589c2252bb0 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2e07c740 10 client.0 _ll_register_callbacks cb 0x5589c2132030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2923e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2923e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48220/0 (socket says 172.21.15.133:48220) 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2923e640 1 -- 172.21.15.133:0/114670364 learned_addr learned my addr 172.21.15.133:0/114670364 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22479d0 0x5589c22524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2a240640 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c224fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2923e640 1 -- 172.21.15.133:0/114670364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 msgr2=0x5589c224fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2923e640 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c224fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2923e640 1 -- 172.21.15.133:0/114670364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22479d0 msgr2=0x5589c22524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2923e640 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22479d0 0x5589c22524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2923e640 1 -- 172.21.15.133:0/114670364 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589c1f1f2b0 con 0x5589c22482f0 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22479d0 0x5589c22524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c2923e640 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224f830 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400e600 tx=0x7f4c1400eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beabd7970d01a911 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.644+0000 7f4c0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/114670364 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14004220 con 0x5589c22482f0 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/114670364 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f4c140043c0 con 0x5589c22482f0 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c2e07c740 1 -- 172.21.15.133:0/114670364 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c1c013d50 con 0x5589c22482f0 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/114670364 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c1400a7f0 con 0x5589c22482f0 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c2e07c740 10 client.15039 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c2e07c740 1 -- 172.21.15.133:0/114670364 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589c2261190 con 0x5589c22482f0 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/114670364 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c1c013d50 con 0x5589c22482f0 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/114670364 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4c14010430 con 0x5589c22482f0 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c2e07c740 10 client.15039 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c2e07c740 10 client.15039 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:36:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c2e07c740 20 client.15039 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5589c2263880 0x5589c2264cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c2e07c740 4 client.15039 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.645+0000 7f4c2e07c740 1 -- 172.21.15.133:0/114670364 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589c21bb600 con 0x5589c2263880 2024-09-11T23:36:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.646+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5589c2263880 0x5589c2264cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.646+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5589c2263880 0x5589c2264cf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30276da0987f8696 server_cookie=929489abe165319e in_seq=0 out_seq=0 2024-09-11T23:36:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.646+0000 7f4c0d7fa640 10 client.15039 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:36:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/114670364 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589c21bb600 con 0x5589c2263880 2024-09-11T23:36:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c0d7fa640 10 client.15039 handle_command_reply: tid=0 2024-09-11T23:36:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 1 client.15039 shutdown 2024-09-11T23:36:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 1 -- 172.21.15.133:0/114670364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 msgr2=0x5589c224f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224f830 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400e600 tx=0x7f4c1400eac0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 1 -- 172.21.15.133:0/114670364 shutdown_connections 2024-09-11T23:36:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5589c2263880 0x5589c2264cf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c224fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224f830 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c0d7fa640 0 client.15039 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:36:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/114670364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22479d0 0x5589c22524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 1 -- 172.21.15.133:0/114670364 >> 172.21.15.133:0/114670364 conn(0x5589c2243f60 msgr2=0x5589c2249c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 1 -- 172.21.15.133:0/114670364 shutdown_connections 2024-09-11T23:36:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 1 -- 172.21.15.133:0/114670364 wait complete. 2024-09-11T23:36:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.647+0000 7f4c2e07c740 20 client.15039 trim_cache size 0 max 16384 2024-09-11T23:36:12.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.664+0000 7f4c2e07c740 1 Processor -- start 2024-09-11T23:36:12.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.664+0000 7f4c2e07c740 1 -- start start 2024-09-11T23:36:12.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c22479d0 0x5589c2249470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c22499b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c2252bb0 0x5589c2249ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5589c2263030 con 0x5589c22482f0 2024-09-11T23:36:12.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589c2263700 con 0x5589c2252bb0 2024-09-11T23:36:12.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589c2263dd0 con 0x5589c22479d0 2024-09-11T23:36:12.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2923e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c22499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2923e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c22499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48228/0 (socket says 172.21.15.133:48228) 2024-09-11T23:36:12.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2923e640 1 -- 172.21.15.133:0/2095500358 learned_addr learned my addr 172.21.15.133:0/2095500358 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:12.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/2095500358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c22479d0 0x5589c2249470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2a240640 1 --2- 172.21.15.133:0/2095500358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c2252bb0 0x5589c2249ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2923e640 1 -- 172.21.15.133:0/2095500358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c22479d0 msgr2=0x5589c2249470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2923e640 1 --2- 172.21.15.133:0/2095500358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c22479d0 0x5589c2249470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2923e640 1 -- 172.21.15.133:0/2095500358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c2252bb0 msgr2=0x5589c2249ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2923e640 1 --2- 172.21.15.133:0/2095500358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c2252bb0 0x5589c2249ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2923e640 1 -- 172.21.15.133:0/2095500358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589c2255310 con 0x5589c22482f0 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2a240640 1 --2- 172.21.15.133:0/2095500358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c2252bb0 0x5589c2249ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c2923e640 1 --2- 172.21.15.133:0/2095500358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c22499b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4c14006010 tx=0x7f4c14000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559e86f5dee44388 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c0f7fe640 1 -- 172.21.15.133:0/2095500358 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14010630 con 0x5589c22482f0 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c0f7fe640 1 -- 172.21.15.133:0/2095500358 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c1400a990 con 0x5589c22482f0 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.665+0000 7f4c0f7fe640 1 -- 172.21.15.133:0/2095500358 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14003850 con 0x5589c22482f0 2024-09-11T23:36:12.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.666+0000 7f4c2e07c740 1 -- 172.21.15.133:0/2095500358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 msgr2=0x5589c22499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:12.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.666+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/2095500358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c22499b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4c14006010 tx=0x7f4c14000bc0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.666+0000 7f4c2e07c740 1 -- 172.21.15.133:0/2095500358 shutdown_connections 2024-09-11T23:36:12.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.666+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/2095500358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c2252bb0 0x5589c2249ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.666+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/2095500358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c22499b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.666+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/2095500358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c22479d0 0x5589c2249470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.666+0000 7f4c2e07c740 1 -- 172.21.15.133:0/2095500358 >> 172.21.15.133:0/2095500358 conn(0x5589c2243f60 msgr2=0x5589c2262650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:12.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.666+0000 7f4c2e07c740 1 -- 172.21.15.133:0/2095500358 shutdown_connections 2024-09-11T23:36:12.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.666+0000 7f4c2e07c740 1 -- 172.21.15.133:0/2095500358 wait complete. 2024-09-11T23:36:12.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.666+0000 7f4c2e07c740 1 -- start start 2024-09-11T23:36:12.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.666+0000 7f4c2e07c740 1 Processor -- start 2024-09-11T23:36:12.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22479d0 0x5589c224c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2e07c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c224d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5589c2264b70 con 0x5589c22482f0 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589c22644a0 con 0x5589c22479d0 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2e07c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589c2265240 con 0x5589c2252bb0 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2e07c740 10 client.0 _ll_register_callbacks cb 0x7f4c2406b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2923e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2923e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48236/0 (socket says 172.21.15.133:48236) 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2923e640 1 -- 172.21.15.133:0/3934872185 learned_addr learned my addr 172.21.15.133:0/3934872185 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22479d0 0x5589c224c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2a240640 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c224d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2923e640 1 -- 172.21.15.133:0/3934872185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 msgr2=0x5589c224d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2923e640 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c224d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2923e640 1 -- 172.21.15.133:0/3934872185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22479d0 msgr2=0x5589c224c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2923e640 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22479d0 0x5589c224c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2923e640 1 -- 172.21.15.133:0/3934872185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589c226b620 con 0x5589c22482f0 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2a240640 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c224d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c2923e640 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224ce90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4c14001eb0 tx=0x7f4c1400add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff57e504c99d9a80 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.667+0000 7f4c0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/3934872185 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14005220 con 0x5589c22482f0 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/3934872185 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f4c140053c0 con 0x5589c22482f0 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c2e07c740 1 -- 172.21.15.133:0/3934872185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c1400a990 con 0x5589c22482f0 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/3934872185 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c14010ac0 con 0x5589c22482f0 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c2e07c740 10 client.15045 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c2e07c740 1 -- 172.21.15.133:0/3934872185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589c22ce280 con 0x5589c22482f0 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/3934872185 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c1400a990 con 0x5589c22482f0 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/3934872185 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4c1402c070 con 0x5589c22482f0 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c2e07c740 10 client.15045 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:12.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c2e07c740 10 client.15045 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c2e07c740 20 client.15045 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5589c22d2ab0 0x5589c22d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c2e07c740 4 client.15045 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c2e07c740 1 -- 172.21.15.133:0/3934872185 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "status"}) -- 0x5589c21b5b10 con 0x5589c22d2ab0 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.668+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5589c22d2ab0 0x5589c22d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c0d7fa640 10 client.15045 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c29a3f640 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5589c22d2ab0 0x5589c22d17c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b87d61497d09862d server_cookie=37a0dd5365c270ff in_seq=0 out_seq=0 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c0d7fa640 1 -- 172.21.15.133:0/3934872185 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5589c21b5b10 con 0x5589c22d2ab0 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c0d7fa640 10 client.15045 handle_command_reply: tid=0 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 1 client.15045 shutdown 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 1 -- 172.21.15.133:0/3934872185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 msgr2=0x5589c224ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224ce90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4c14001eb0 tx=0x7f4c1400add0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 1 -- 172.21.15.133:0/3934872185 shutdown_connections 2024-09-11T23:36:12.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5589c22d2ab0 0x5589c22d17c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c2252bb0 0x5589c224d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589c22482f0 0x5589c224ce90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 1 --2- 172.21.15.133:0/3934872185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589c22479d0 0x5589c224c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c0d7fa640 0 client.15045 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:36:12.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 1 -- 172.21.15.133:0/3934872185 >> 172.21.15.133:0/3934872185 conn(0x5589c2243f60 msgr2=0x5589c2249470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:12.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 1 -- 172.21.15.133:0/3934872185 shutdown_connections 2024-09-11T23:36:12.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 1 -- 172.21.15.133:0/3934872185 wait complete. 2024-09-11T23:36:12.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.669+0000 7f4c2e07c740 20 client.15045 trim_cache size 0 max 16384 2024-09-11T23:36:12.675 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 2, 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24464, 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 70.185821337999997, 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 27, 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 81, 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 79, 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 89.654875988000001 2024-09-11T23:36:12.676 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:12.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1369219104 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c100a2c90 msgr2=0x7f4c100a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:12.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c100a2c90 0x7f4c100a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c01e000 tx=0x7f4c1c01b000 comp rx=0 tx=0).stop 2024-09-11T23:36:12.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1369219104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c24100b30 msgr2=0x7f4c241a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:12.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c24100b30 0x7f4c241a46d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4c20005cc0 tx=0x7f4c2002f880 comp rx=0 tx=0).stop 2024-09-11T23:36:12.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1369219104 shutdown_connections 2024-09-11T23:36:12.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c100a2c90 0x7f4c100a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c24105f40 0x7f4c2419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c24101450 0x7f4c241a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 --2- 172.21.15.133:0/1369219104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c24100b30 0x7f4c241a46d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:12.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1369219104 >> 172.21.15.133:0/1369219104 conn(0x7f4c240fc7a0 msgr2=0x7f4c241038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:12.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1369219104 shutdown_connections 2024-09-11T23:36:12.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:12.670+0000 7f4c2bca3640 1 -- 172.21.15.133:0/1369219104 wait complete. 2024-09-11T23:36:13.306 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:13 smithi133 ceph-mon[31430]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:13.374 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T23:36:13.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:13 smithi138 ceph-mon[35142]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:13.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:13 smithi178 ceph-mon[35882]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:13.638 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.114+0000 7f18f4d86640 1 -- 172.21.15.133:0/3258685076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f0102790 msgr2=0x7f18f01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.114+0000 7f18f4d86640 1 --2- 172.21.15.133:0/3258685076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f0102790 0x7f18f01067e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009900 tx=0x7f18e002a410 comp rx=0 tx=0).stop 2024-09-11T23:36:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.114+0000 7f18f4d86640 1 -- 172.21.15.133:0/3258685076 shutdown_connections 2024-09-11T23:36:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.114+0000 7f18f4d86640 1 --2- 172.21.15.133:0/3258685076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f0106f10 0x7f18f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.114+0000 7f18f4d86640 1 --2- 172.21.15.133:0/3258685076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f0102790 0x7f18f01067e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.114+0000 7f18f4d86640 1 --2- 172.21.15.133:0/3258685076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18f0101de0 0x7f18f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.114+0000 7f18f4d86640 1 -- 172.21.15.133:0/3258685076 >> 172.21.15.133:0/3258685076 conn(0x7f18f00fd8c0 msgr2=0x7f18f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.114+0000 7f18f4d86640 1 -- 172.21.15.133:0/3258685076 shutdown_connections 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.114+0000 7f18f4d86640 1 -- 172.21.15.133:0/3258685076 wait complete. 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18f4d86640 1 Processor -- start 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18f4d86640 1 -- start start 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18f4d86640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f0101de0 0x7f18f0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18f4d86640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f0102790 0x7f18f0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18f4d86640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18f0106f10 0x7f18f019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18f4d86640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0111fb0 con 0x7f18f0102790 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18f4d86640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0111e30 con 0x7f18f0101de0 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18f4d86640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0112130 con 0x7f18f0106f10 2024-09-11T23:36:14.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18edd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f0102790 0x7f18f0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18edd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f0102790 0x7f18f0198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48242/0 (socket says 172.21.15.133:48242) 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18edd9b640 1 -- 172.21.15.133:0/1108768087 learned_addr learned my addr 172.21.15.133:0/1108768087 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18ee59c640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f0101de0 0x7f18f0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18eed9d640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18f0106f10 0x7f18f019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18edd9b640 1 -- 172.21.15.133:0/1108768087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18f0106f10 msgr2=0x7f18f019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18edd9b640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18f0106f10 0x7f18f019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18edd9b640 1 -- 172.21.15.133:0/1108768087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f0101de0 msgr2=0x7f18f0198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18edd9b640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f0101de0 0x7f18f0198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18edd9b640 1 -- 172.21.15.133:0/1108768087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f019d4e0 con 0x7f18f0102790 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.115+0000 7f18eed9d640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18f0106f10 0x7f18f019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.116+0000 7f18edd9b640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f0102790 0x7f18f0198a70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005ab0 tx=0x7f18e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.116+0000 7f18e77fe640 1 -- 172.21.15.133:0/1108768087 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e003f030 con 0x7f18f0102790 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.116+0000 7f18f4d86640 1 -- 172.21.15.133:0/1108768087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f019d770 con 0x7f18f0102790 2024-09-11T23:36:14.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.116+0000 7f18e77fe640 1 -- 172.21.15.133:0/1108768087 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e0003ce0 con 0x7f18f0102790 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.116+0000 7f18e77fe640 1 -- 172.21.15.133:0/1108768087 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0035070 con 0x7f18f0102790 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.116+0000 7f18f4d86640 1 -- 172.21.15.133:0/1108768087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f01a50b0 con 0x7f18f0102790 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.118+0000 7f18e77fe640 1 -- 172.21.15.133:0/1108768087 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f18e0042030 con 0x7f18f0102790 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.118+0000 7f18f715f740 1 Processor -- start 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18f715f740 1 -- start start 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18f715f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18f715f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850c12f0 0x5637850cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18f715f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850cbbb0 0x5637850cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18f715f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563785027090 con 0x5637850c09d0 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18f715f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563784fcea70 con 0x5637850cbbb0 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18f715f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563785023050 con 0x5637850c12f0 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18eed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18eed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850c0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48258/0 (socket says 172.21.15.133:48258) 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18eed9d640 1 -- 172.21.15.133:0/4168809158 learned_addr learned my addr 172.21.15.133:0/4168809158 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18e77fe640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f18dc0a2c10 0x7f18dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18edd9b640 1 --2- 172.21.15.133:0/4168809158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850cbbb0 0x5637850cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18ee59c640 1 --2- 172.21.15.133:0/4168809158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850c12f0 0x5637850cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18eed9d640 1 -- 172.21.15.133:0/4168809158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850c12f0 msgr2=0x5637850cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18eed9d640 1 --2- 172.21.15.133:0/4168809158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850c12f0 0x5637850cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18eed9d640 1 -- 172.21.15.133:0/4168809158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850cbbb0 msgr2=0x5637850cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18eed9d640 1 --2- 172.21.15.133:0/4168809158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850cbbb0 0x5637850cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18eed9d640 1 -- 172.21.15.133:0/4168809158 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563784fdb130 con 0x5637850c09d0 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.119+0000 7f18e77fe640 1 -- 172.21.15.133:0/1108768087 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f18e00ef6a0 con 0x7f18f0102790 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.120+0000 7f18eed9d640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f18dc0a2c10 0x7f18dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.120+0000 7f18eed9d640 1 --2- 172.21.15.133:0/4168809158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850c0db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f18e800e450 tx=0x7f18e800e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31129dc2b61f264 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.120+0000 7f18e4ff9640 1 -- 172.21.15.133:0/4168809158 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8012e60 con 0x5637850c09d0 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.120+0000 7f18e4ff9640 1 -- 172.21.15.133:0/4168809158 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e801ce30 con 0x5637850c09d0 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.120+0000 7f18e4ff9640 1 -- 172.21.15.133:0/4168809158 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e801b620 con 0x5637850c09d0 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 -- 172.21.15.133:0/4168809158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 msgr2=0x5637850c0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 --2- 172.21.15.133:0/4168809158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850c0db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f18e800e450 tx=0x7f18e800e910 comp rx=0 tx=0).stop 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18eed9d640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f18dc0a2c10 0x7f18dc0a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18e801f000 tx=0x7f18e8020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 -- 172.21.15.133:0/4168809158 shutdown_connections 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 --2- 172.21.15.133:0/4168809158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850cbbb0 0x5637850cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 --2- 172.21.15.133:0/4168809158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850c12f0 0x5637850cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 --2- 172.21.15.133:0/4168809158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850c0db0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 -- 172.21.15.133:0/4168809158 >> 172.21.15.133:0/4168809158 conn(0x5637850bcf60 msgr2=0x5637850bd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 -- 172.21.15.133:0/4168809158 shutdown_connections 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 -- 172.21.15.133:0/4168809158 wait complete. 2024-09-11T23:36:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 -- start start 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 Processor -- start 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c12f0 0x5637850c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850cbbb0 0x5637850c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5637850ce780 con 0x5637850c09d0 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563784fe1f20 con 0x5637850c12f0 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5637850ce900 con 0x5637850cbbb0 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18f715f740 10 client.0 _ll_register_callbacks cb 0x563784fab030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18ee59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.121+0000 7f18ee59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48272/0 (socket says 172.21.15.133:48272) 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18ee59c640 1 -- 172.21.15.133:0/2003454754 learned_addr learned my addr 172.21.15.133:0/2003454754 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18ee59c640 1 -- 172.21.15.133:0/2003454754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850cbbb0 msgr2=0x5637850c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18ee59c640 1 --2- 172.21.15.133:0/2003454754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850cbbb0 0x5637850c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18ee59c640 1 -- 172.21.15.133:0/2003454754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c12f0 msgr2=0x5637850c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18ee59c640 1 --2- 172.21.15.133:0/2003454754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c12f0 0x5637850c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18ee59c640 1 -- 172.21.15.133:0/2003454754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563784d982b0 con 0x5637850c09d0 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18ee59c640 1 --2- 172.21.15.133:0/2003454754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850cb450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f18d80104a0 tx=0x7f18d8010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd3e90ce385d6b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2003454754 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8018070 con 0x5637850c09d0 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2003454754 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f18d8014800 con 0x5637850c09d0 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18f715f740 1 -- 172.21.15.133:0/2003454754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e801ce30 con 0x5637850c09d0 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2003454754 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18d8024050 con 0x5637850c09d0 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18f715f740 10 client.15057 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18f715f740 1 -- 172.21.15.133:0/2003454754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637850da100 con 0x5637850c09d0 2024-09-11T23:36:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.122+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2003454754 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e801ce30 con 0x5637850c09d0 2024-09-11T23:36:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.123+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2003454754 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f18d801cca0 con 0x5637850c09d0 2024-09-11T23:36:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.123+0000 7f18f715f740 10 client.15057 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.123+0000 7f18f715f740 10 client.15057 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:36:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.123+0000 7f18f715f740 20 client.15057 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.123+0000 7f18f715f740 1 --2- 172.21.15.133:0/2003454754 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5637850dc7c0 0x5637850ddc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.123+0000 7f18f715f740 4 client.15057 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.123+0000 7f18f715f740 1 -- 172.21.15.133:0/2003454754 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563785034600 con 0x5637850dc7c0 2024-09-11T23:36:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.123+0000 7f18edd9b640 1 --2- 172.21.15.133:0/2003454754 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5637850dc7c0 0x5637850ddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.123+0000 7f18edd9b640 1 --2- 172.21.15.133:0/2003454754 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5637850dc7c0 0x5637850ddc30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ac72afa8891ef36 server_cookie=89dfa013c990991a in_seq=0 out_seq=0 2024-09-11T23:36:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.123+0000 7f18d5ffb640 10 client.15057 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:36:14.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.124+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2003454754 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563785034600 con 0x5637850dc7c0 2024-09-11T23:36:14.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.124+0000 7f18d5ffb640 10 client.15057 handle_command_reply: tid=0 2024-09-11T23:36:14.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.124+0000 7f18f715f740 1 client.15057 shutdown 2024-09-11T23:36:14.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.124+0000 7f18f715f740 1 -- 172.21.15.133:0/2003454754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 msgr2=0x5637850cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:14.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.125+0000 7f18f715f740 1 --2- 172.21.15.133:0/2003454754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850cb450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f18d80104a0 tx=0x7f18d8010960 comp rx=0 tx=0).stop 2024-09-11T23:36:14.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.125+0000 7f18f715f740 1 -- 172.21.15.133:0/2003454754 shutdown_connections 2024-09-11T23:36:14.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.125+0000 7f18f715f740 1 --2- 172.21.15.133:0/2003454754 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5637850dc7c0 0x5637850ddc30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.125+0000 7f18f715f740 1 --2- 172.21.15.133:0/2003454754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850cbbb0 0x5637850c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.125+0000 7f18f715f740 1 --2- 172.21.15.133:0/2003454754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c12f0 0x5637850c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.125+0000 7f18f715f740 1 --2- 172.21.15.133:0/2003454754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c09d0 0x5637850cb450 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.125+0000 7f18f715f740 1 -- 172.21.15.133:0/2003454754 >> 172.21.15.133:0/2003454754 conn(0x5637850bcf60 msgr2=0x5637850c2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.125+0000 7f18d5ffb640 0 client.15057 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:36:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.125+0000 7f18f715f740 1 -- 172.21.15.133:0/2003454754 shutdown_connections 2024-09-11T23:36:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.125+0000 7f18f715f740 1 -- 172.21.15.133:0/2003454754 wait complete. 2024-09-11T23:36:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.125+0000 7f18f715f740 20 client.15057 trim_cache size 0 max 16384 2024-09-11T23:36:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.142+0000 7f18f715f740 1 Processor -- start 2024-09-11T23:36:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.142+0000 7f18f715f740 1 -- start start 2024-09-11T23:36:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18f715f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c09d0 0x5637850dc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18f715f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850c12f0 0x5637850c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18f715f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850cbbb0 0x5637850c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18f715f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5637850dbf70 con 0x5637850cbbb0 2024-09-11T23:36:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18f715f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5637850dc640 con 0x5637850c09d0 2024-09-11T23:36:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18f715f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5637850dcd10 con 0x5637850c12f0 2024-09-11T23:36:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18eed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850cbbb0 0x5637850c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18eed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850cbbb0 0x5637850c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48280/0 (socket says 172.21.15.133:48280) 2024-09-11T23:36:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18eed9d640 1 -- 172.21.15.133:0/1239846513 learned_addr learned my addr 172.21.15.133:0/1239846513 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18ee59c640 1 --2- 172.21.15.133:0/1239846513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c09d0 0x5637850dc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18edd9b640 1 --2- 172.21.15.133:0/1239846513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850c12f0 0x5637850c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18eed9d640 1 -- 172.21.15.133:0/1239846513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850c12f0 msgr2=0x5637850c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18eed9d640 1 --2- 172.21.15.133:0/1239846513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850c12f0 0x5637850c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18eed9d640 1 -- 172.21.15.133:0/1239846513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c09d0 msgr2=0x5637850dc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18eed9d640 1 --2- 172.21.15.133:0/1239846513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c09d0 0x5637850dc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18eed9d640 1 -- 172.21.15.133:0/1239846513 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637850cb7c0 con 0x5637850cbbb0 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18ee59c640 1 --2- 172.21.15.133:0/1239846513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c09d0 0x5637850dc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18eed9d640 1 --2- 172.21.15.133:0/1239846513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850cbbb0 0x5637850c5f40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f18e800aaf0 tx=0x7f18e8012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce564713cb89062 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18d7fff640 1 -- 172.21.15.133:0/1239846513 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8032020 con 0x5637850cbbb0 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18d7fff640 1 -- 172.21.15.133:0/1239846513 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e800a7b0 con 0x5637850cbbb0 2024-09-11T23:36:14.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.143+0000 7f18d7fff640 1 -- 172.21.15.133:0/1239846513 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e80258d0 con 0x5637850cbbb0 2024-09-11T23:36:14.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.144+0000 7f18f715f740 1 -- 172.21.15.133:0/1239846513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850cbbb0 msgr2=0x5637850c5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:14.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.144+0000 7f18f715f740 1 --2- 172.21.15.133:0/1239846513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850cbbb0 0x5637850c5f40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f18e800aaf0 tx=0x7f18e8012510 comp rx=0 tx=0).stop 2024-09-11T23:36:14.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.144+0000 7f18f715f740 1 -- 172.21.15.133:0/1239846513 shutdown_connections 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.144+0000 7f18f715f740 1 --2- 172.21.15.133:0/1239846513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850cbbb0 0x5637850c5f40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.144+0000 7f18f715f740 1 --2- 172.21.15.133:0/1239846513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850c12f0 0x5637850c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.144+0000 7f18f715f740 1 --2- 172.21.15.133:0/1239846513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c09d0 0x5637850dc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.144+0000 7f18f715f740 1 -- 172.21.15.133:0/1239846513 >> 172.21.15.133:0/1239846513 conn(0x5637850bcf60 msgr2=0x5637850c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.144+0000 7f18f715f740 1 -- 172.21.15.133:0/1239846513 shutdown_connections 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.144+0000 7f18f715f740 1 -- 172.21.15.133:0/1239846513 wait complete. 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18f715f740 1 -- start start 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18f715f740 1 Processor -- start 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18f715f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c09d0 0x5637850dee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18f715f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c12f0 0x5637850df370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18f715f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850cbbb0 0x5637850df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18f715f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5637850ddab0 con 0x5637850c12f0 2024-09-11T23:36:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18f715f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5637850dd3e0 con 0x5637850c09d0 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18f715f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5637850de180 con 0x5637850cbbb0 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18f715f740 10 client.0 _ll_register_callbacks cb 0x7f18d8009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18edd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c12f0 0x5637850df370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18edd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c12f0 0x5637850df370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48284/0 (socket says 172.21.15.133:48284) 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18edd9b640 1 -- 172.21.15.133:0/2434668373 learned_addr learned my addr 172.21.15.133:0/2434668373 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18ee59c640 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c09d0 0x5637850dee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18edd9b640 1 -- 172.21.15.133:0/2434668373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850cbbb0 msgr2=0x5637850df8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18edd9b640 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850cbbb0 0x5637850df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18edd9b640 1 -- 172.21.15.133:0/2434668373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c09d0 msgr2=0x5637850dee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18edd9b640 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c09d0 0x5637850dee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18edd9b640 1 -- 172.21.15.133:0/2434668373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563785144ec0 con 0x5637850c12f0 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18edd9b640 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c12f0 0x5637850df370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f18e00bbe30 tx=0x7f18e00b8140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767f2ccdd4b21f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.145+0000 7f18d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2434668373 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0049040 con 0x5637850c12f0 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2434668373 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f18e003bd00 con 0x5637850c12f0 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18f715f740 1 -- 172.21.15.133:0/2434668373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e800a7b0 con 0x5637850c12f0 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2434668373 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18e00bb800 con 0x5637850c12f0 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18f715f740 10 client.15063 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18f715f740 1 -- 172.21.15.133:0/2434668373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637851460c0 con 0x5637850c12f0 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2434668373 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e800a7b0 con 0x5637850c12f0 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2434668373 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f18e00b8510 con 0x5637850c12f0 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18f715f740 10 client.15063 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18f715f740 10 client.15063 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18f715f740 20 client.15063 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18f715f740 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56378514acd0 0x5637851499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18f715f740 4 client.15063 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:36:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18f715f740 1 -- 172.21.15.133:0/2434668373 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "status"}) -- 0x56378502eb10 con 0x56378514acd0 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.146+0000 7f18ee59c640 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56378514acd0 0x5637851499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18ee59c640 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56378514acd0 0x5637851499e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47512e35a81e0bb7 server_cookie=16bcb775a46e8170 in_seq=0 out_seq=0 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18d5ffb640 10 client.15063 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18d5ffb640 1 -- 172.21.15.133:0/2434668373 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56378502eb10 con 0x56378514acd0 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18d5ffb640 10 client.15063 handle_command_reply: tid=0 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 1 client.15063 shutdown 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 1 -- 172.21.15.133:0/2434668373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c12f0 msgr2=0x5637850df370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c12f0 0x5637850df370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f18e00bbe30 tx=0x7f18e00b8140 comp rx=0 tx=0).stop 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 1 -- 172.21.15.133:0/2434668373 shutdown_connections 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56378514acd0 0x5637851499e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637850cbbb0 0x5637850df8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18d5ffb640 0 client.15063 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:36:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637850c12f0 0x5637850df370 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 1 --2- 172.21.15.133:0/2434668373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637850c09d0 0x5637850dee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 1 -- 172.21.15.133:0/2434668373 >> 172.21.15.133:0/2434668373 conn(0x5637850bcf60 msgr2=0x5637850c3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:14.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 1 -- 172.21.15.133:0/2434668373 shutdown_connections 2024-09-11T23:36:14.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 1 -- 172.21.15.133:0/2434668373 wait complete. 2024-09-11T23:36:14.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.147+0000 7f18f715f740 20 client.15063 trim_cache size 0 max 16384 2024-09-11T23:36:14.164 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:14.164 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:36:14.164 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 1, 2024-09-11T23:36:14.164 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24472, 2024-09-11T23:36:14.164 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:36:14.165 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:36:14.165 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:36:14.165 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 72.678231069000006, 2024-09-11T23:36:14.165 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 27, 2024-09-11T23:36:14.165 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 81, 2024-09-11T23:36:14.165 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 79, 2024-09-11T23:36:14.165 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 104.435437581 2024-09-11T23:36:14.165 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 -- 172.21.15.133:0/1108768087 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f18dc0a2c10 msgr2=0x7f18dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f18dc0a2c10 0x7f18dc0a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18e801f000 tx=0x7f18e8020000 comp rx=0 tx=0).stop 2024-09-11T23:36:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 -- 172.21.15.133:0/1108768087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f0102790 msgr2=0x7f18f0198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f0102790 0x7f18f0198a70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005ab0 tx=0x7f18e00042f0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 -- 172.21.15.133:0/1108768087 shutdown_connections 2024-09-11T23:36:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f18dc0a2c10 0x7f18dc0a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18f0106f10 0x7f18f019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f0102790 0x7f18f0198a70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 --2- 172.21.15.133:0/1108768087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f0101de0 0x7f18f0198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 -- 172.21.15.133:0/1108768087 >> 172.21.15.133:0/1108768087 conn(0x7f18f00fd8c0 msgr2=0x7f18f00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 -- 172.21.15.133:0/1108768087 shutdown_connections 2024-09-11T23:36:14.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:14.148+0000 7f18f4d86640 1 -- 172.21.15.133:0/1108768087 wait complete. 2024-09-11T23:36:14.916 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:36:15.160 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:15.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:15 smithi133 ceph-mon[31430]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:15.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:15 smithi138 ceph-mon[35142]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:15.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:15 smithi178 ceph-mon[35882]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:15.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.614+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/1534005211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64102790 msgr2=0x7fdc641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:15.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.614+0000 7fdc6bcf3640 1 --2- 172.21.15.133:0/1534005211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64102790 0x7fdc641067e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009900 tx=0x7fdc5802a410 comp rx=0 tx=0).stop 2024-09-11T23:36:15.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.614+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/1534005211 shutdown_connections 2024-09-11T23:36:15.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.614+0000 7fdc6bcf3640 1 --2- 172.21.15.133:0/1534005211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc64106f10 0x7fdc6410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.614+0000 7fdc6bcf3640 1 --2- 172.21.15.133:0/1534005211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64102790 0x7fdc641067e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.614+0000 7fdc6bcf3640 1 --2- 172.21.15.133:0/1534005211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64101de0 0x7fdc641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.614+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/1534005211 >> 172.21.15.133:0/1534005211 conn(0x7fdc640fd8c0 msgr2=0x7fdc640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:15.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.614+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/1534005211 shutdown_connections 2024-09-11T23:36:15.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.614+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/1534005211 wait complete. 2024-09-11T23:36:15.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc6bcf3640 1 Processor -- start 2024-09-11T23:36:15.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc6bcf3640 1 -- start start 2024-09-11T23:36:15.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc6bcf3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64101de0 0x7fdc640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc6bcf3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64102790 0x7fdc64074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc6bcf3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc64106f10 0x7fdc64074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc6bcf3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64112010 con 0x7fdc64102790 2024-09-11T23:36:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc6bcf3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64111e90 con 0x7fdc64101de0 2024-09-11T23:36:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc6bcf3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64112190 con 0x7fdc64106f10 2024-09-11T23:36:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc6928e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64102790 0x7fdc64074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc6928e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64102790 0x7fdc64074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48298/0 (socket says 172.21.15.133:48298) 2024-09-11T23:36:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc6928e640 1 -- 172.21.15.133:0/419412758 learned_addr learned my addr 172.21.15.133:0/419412758 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.615+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64101de0 0x7fdc640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc6a290640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc64106f10 0x7fdc64074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc6928e640 1 -- 172.21.15.133:0/419412758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc64106f10 msgr2=0x7fdc64074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc6928e640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc64106f10 0x7fdc64074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc6928e640 1 -- 172.21.15.133:0/419412758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64101de0 msgr2=0x7fdc640744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:15.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc6928e640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64101de0 0x7fdc640744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc6928e640 1 -- 172.21.15.133:0/419412758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc641a6a70 con 0x7fdc64102790 2024-09-11T23:36:15.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64101de0 0x7fdc640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:15.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc6928e640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64102790 0x7fdc64074a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdc58005ab0 tx=0x7fdc580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:15.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc62ffd640 1 -- 172.21.15.133:0/419412758 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5803f030 con 0x7fdc64102790 2024-09-11T23:36:15.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/419412758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc641a6d00 con 0x7fdc64102790 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc62ffd640 1 -- 172.21.15.133:0/419412758 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc58003ce0 con 0x7fdc64102790 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc62ffd640 1 -- 172.21.15.133:0/419412758 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58035070 con 0x7fdc64102790 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.616+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/419412758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc641a71d0 con 0x7fdc64102790 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.618+0000 7fdc62ffd640 1 -- 172.21.15.133:0/419412758 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdc58042030 con 0x7fdc64102790 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.618+0000 7fdc6e0cc740 1 Processor -- start 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6e0cc740 1 -- start start 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357579d0 0x55fc35757db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc35762670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc35762bb0 0x55fc35764fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fc356be090 con 0x55fc357579d0 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc35665a70 con 0x55fc357582f0 2024-09-11T23:36:15.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fc356ba050 con 0x55fc35762bb0 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc62ffd640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc4c0a2c10 0x7fdc4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6a290640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357579d0 0x55fc35757db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6a290640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357579d0 0x55fc35757db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48310/0 (socket says 172.21.15.133:48310) 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6a290640 1 -- 172.21.15.133:0/1223406610 learned_addr learned my addr 172.21.15.133:0/1223406610 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6a290640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc4c0a2c10 0x7fdc4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/1223406610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc35762670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc62ffd640 1 -- 172.21.15.133:0/419412758 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fdc580efa90 con 0x7fdc64102790 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6928e640 1 --2- 172.21.15.133:0/1223406610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc35762bb0 0x55fc35764fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6a290640 1 -- 172.21.15.133:0/1223406610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc35762bb0 msgr2=0x55fc35764fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6a290640 1 --2- 172.21.15.133:0/1223406610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc35762bb0 0x55fc35764fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6a290640 1 -- 172.21.15.133:0/1223406610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 msgr2=0x55fc35762670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6a290640 1 --2- 172.21.15.133:0/1223406610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc35762670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6a290640 1 -- 172.21.15.133:0/1223406610 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc35672130 con 0x55fc357579d0 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc6928e640 1 --2- 172.21.15.133:0/1223406610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc35762bb0 0x55fc35764fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.619+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/1223406610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc35762670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.620+0000 7fdc6a290640 1 --2- 172.21.15.133:0/1223406610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357579d0 0x55fc35757db0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c015700 tx=0x7fdc5c015bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bceb19bcedc3c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.620+0000 7fdc6a290640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc4c0a2c10 0x7fdc4c0a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c006fd0 tx=0x7fdc5c019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.620+0000 7fdc4bfff640 1 -- 172.21.15.133:0/1223406610 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c00f040 con 0x55fc357579d0 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.620+0000 7fdc4bfff640 1 -- 172.21.15.133:0/1223406610 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c024070 con 0x55fc357579d0 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.620+0000 7fdc4bfff640 1 -- 172.21.15.133:0/1223406610 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c01f5f0 con 0x55fc357579d0 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.621+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1223406610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357579d0 msgr2=0x55fc35757db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.621+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/1223406610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357579d0 0x55fc35757db0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c015700 tx=0x7fdc5c015bc0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.621+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1223406610 shutdown_connections 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.621+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/1223406610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc35762bb0 0x55fc35764fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.621+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/1223406610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc35762670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.621+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/1223406610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357579d0 0x55fc35757db0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.621+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1223406610 >> 172.21.15.133:0/1223406610 conn(0x55fc35753f60 msgr2=0x55fc35754340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.621+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1223406610 shutdown_connections 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.621+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1223406610 wait complete. 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.621+0000 7fdc6e0cc740 1 -- start start 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.621+0000 7fdc6e0cc740 1 Processor -- start 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357579d0 0x55fc357624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357582f0 0x55fc3575f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc35762bb0 0x55fc3575fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fc35765780 con 0x55fc357582f0 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc35678f20 con 0x55fc357579d0 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fc35765900 con 0x55fc35762bb0 2024-09-11T23:36:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6e0cc740 10 client.0 _ll_register_callbacks cb 0x55fc35642030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:15.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6928e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357582f0 0x55fc3575f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6928e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357582f0 0x55fc3575f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48324/0 (socket says 172.21.15.133:48324) 2024-09-11T23:36:15.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6928e640 1 -- 172.21.15.133:0/3250892420 learned_addr learned my addr 172.21.15.133:0/3250892420 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:15.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357579d0 0x55fc357624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6928e640 1 -- 172.21.15.133:0/3250892420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc35762bb0 msgr2=0x55fc3575fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:15.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6928e640 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc35762bb0 0x55fc3575fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6928e640 1 -- 172.21.15.133:0/3250892420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357579d0 msgr2=0x55fc357624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:15.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6928e640 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357579d0 0x55fc357624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6928e640 1 -- 172.21.15.133:0/3250892420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc3542f2b0 con 0x55fc357582f0 2024-09-11T23:36:15.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6928e640 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357582f0 0x55fc3575f800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdc580bbf90 tx=0x7fdc580b7ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9df8bb5face server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:15.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc497fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc497fa640 1 -- 172.21.15.133:0/3250892420 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58049040 con 0x55fc357582f0 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc497fa640 1 -- 172.21.15.133:0/3250892420 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fdc580d5190 con 0x55fc357582f0 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/3250892420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5c024070 con 0x55fc357582f0 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc497fa640 1 -- 172.21.15.133:0/3250892420 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc580bbbf0 con 0x55fc357582f0 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6e0cc740 10 client.15075 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.622+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/3250892420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc357710c0 con 0x55fc357582f0 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.623+0000 7fdc497fa640 1 -- 172.21.15.133:0/3250892420 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c024070 con 0x55fc357582f0 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.623+0000 7fdc497fa640 1 -- 172.21.15.133:0/3250892420 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdc5803b460 con 0x55fc357582f0 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.623+0000 7fdc6e0cc740 10 client.15075 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.623+0000 7fdc6e0cc740 10 client.15075 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.623+0000 7fdc6e0cc740 20 client.15075 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.623+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fc35773750 0x55fc35774bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.623+0000 7fdc6e0cc740 4 client.15075 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:15.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.623+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/3250892420 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc356cb600 con 0x55fc35773750 2024-09-11T23:36:15.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.623+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fc35773750 0x55fc35774bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.624+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fc35773750 0x55fc35774bc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795544043475bb81 server_cookie=5ebea0356f1717bf in_seq=0 out_seq=0 2024-09-11T23:36:15.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.624+0000 7fdc497fa640 10 client.15075 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:15.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.624+0000 7fdc497fa640 1 -- 172.21.15.133:0/3250892420 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc356cb600 con 0x55fc35773750 2024-09-11T23:36:15.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.624+0000 7fdc497fa640 10 client.15075 handle_command_reply: tid=0 2024-09-11T23:36:15.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.624+0000 7fdc6e0cc740 1 client.15075 shutdown 2024-09-11T23:36:15.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.624+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/3250892420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357582f0 msgr2=0x55fc3575f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:15.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.624+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357582f0 0x55fc3575f800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdc580bbf90 tx=0x7fdc580b7ed0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.625+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/3250892420 shutdown_connections 2024-09-11T23:36:15.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.625+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fc35773750 0x55fc35774bc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.625+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc35762bb0 0x55fc3575fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.625+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc357582f0 0x55fc3575f800 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.625+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/3250892420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357579d0 0x55fc357624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.625+0000 7fdc497fa640 0 client.15075 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:15.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.625+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/3250892420 >> 172.21.15.133:0/3250892420 conn(0x55fc35753f60 msgr2=0x55fc35759c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:15.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.625+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/3250892420 shutdown_connections 2024-09-11T23:36:15.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.625+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/3250892420 wait complete. 2024-09-11T23:36:15.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.625+0000 7fdc6e0cc740 20 client.15075 trim_cache size 0 max 16384 2024-09-11T23:36:15.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.642+0000 7fdc6e0cc740 1 Processor -- start 2024-09-11T23:36:15.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.642+0000 7fdc6e0cc740 1 -- start start 2024-09-11T23:36:15.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.642+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc357579d0 0x55fc35773e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.642+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc35760f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.642+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc35761490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.642+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fc35772f00 con 0x55fc35762bb0 2024-09-11T23:36:15.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.642+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc357735d0 con 0x55fc357582f0 2024-09-11T23:36:15.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.642+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fc35773ca0 con 0x55fc357579d0 2024-09-11T23:36:15.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc6a290640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc35761490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc6a290640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc35761490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48338/0 (socket says 172.21.15.133:48338) 2024-09-11T23:36:15.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc6a290640 1 -- 172.21.15.133:0/265928784 learned_addr learned my addr 172.21.15.133:0/265928784 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:15.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc6a290640 1 -- 172.21.15.133:0/265928784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc357579d0 msgr2=0x55fc35773e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:15.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/265928784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc357579d0 0x55fc35773e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc6a290640 1 --2- 172.21.15.133:0/265928784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc357579d0 0x55fc35773e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc6928e640 1 --2- 172.21.15.133:0/265928784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc35760f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc6a290640 1 -- 172.21.15.133:0/265928784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 msgr2=0x55fc35760f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:15.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc6a290640 1 --2- 172.21.15.133:0/265928784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc35760f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc6a290640 1 -- 172.21.15.133:0/265928784 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc357651d0 con 0x55fc35762bb0 2024-09-11T23:36:15.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc6a290640 1 --2- 172.21.15.133:0/265928784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc35761490 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c006b20 tx=0x7fdc5c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91d8bf2812a4642 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:15.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc4b7fe640 1 -- 172.21.15.133:0/265928784 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c02c020 con 0x55fc35762bb0 2024-09-11T23:36:15.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc4b7fe640 1 -- 172.21.15.133:0/265928784 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c024040 con 0x55fc35762bb0 2024-09-11T23:36:15.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.643+0000 7fdc4b7fe640 1 -- 172.21.15.133:0/265928784 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c01f650 con 0x55fc35762bb0 2024-09-11T23:36:15.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.644+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/265928784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 msgr2=0x55fc35761490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:15.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.644+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/265928784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc35761490 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c006b20 tx=0x7fdc5c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.644+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/265928784 shutdown_connections 2024-09-11T23:36:15.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.644+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/265928784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc35761490 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.644+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/265928784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc35760f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.644+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/265928784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc357579d0 0x55fc35773e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.644+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/265928784 >> 172.21.15.133:0/265928784 conn(0x55fc35753f60 msgr2=0x55fc3575e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:15.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.644+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/265928784 shutdown_connections 2024-09-11T23:36:15.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.644+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/265928784 wait complete. 2024-09-11T23:36:15.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.644+0000 7fdc6e0cc740 1 -- start start 2024-09-11T23:36:15.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.644+0000 7fdc6e0cc740 1 Processor -- start 2024-09-11T23:36:15.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc357579d0 0x55fc3575c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc3575c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6e0cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc3577bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fc35774a40 con 0x55fc35762bb0 2024-09-11T23:36:15.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc35774370 con 0x55fc357582f0 2024-09-11T23:36:15.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6e0cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fc35775110 con 0x55fc357579d0 2024-09-11T23:36:15.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6e0cc740 10 client.0 _ll_register_callbacks cb 0x7fdc58009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:15.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6a290640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc3577bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6a290640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc3577bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48340/0 (socket says 172.21.15.133:48340) 2024-09-11T23:36:15.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6a290640 1 -- 172.21.15.133:0/1535033125 learned_addr learned my addr 172.21.15.133:0/1535033125 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:15.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6928e640 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc3575c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc357579d0 0x55fc3575c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6a290640 1 -- 172.21.15.133:0/1535033125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc357579d0 msgr2=0x55fc3575c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:15.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6a290640 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc357579d0 0x55fc3575c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6a290640 1 -- 172.21.15.133:0/1535033125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 msgr2=0x55fc3575c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:15.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6a290640 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc3575c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6a290640 1 -- 172.21.15.133:0/1535033125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc3577c300 con 0x55fc35762bb0 2024-09-11T23:36:15.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc357579d0 0x55fc3575c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:15.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc6a290640 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc3577bdc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c006b80 tx=0x7fdc5c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab13c69e125d0058 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:15.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc497fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:15.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.645+0000 7fdc497fa640 1 -- 172.21.15.133:0/1535033125 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c02c030 con 0x55fc35762bb0 2024-09-11T23:36:15.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc497fa640 1 -- 172.21.15.133:0/1535033125 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fdc5c01f860 con 0x55fc35762bb0 2024-09-11T23:36:15.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1535033125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5c024040 con 0x55fc35762bb0 2024-09-11T23:36:15.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc497fa640 1 -- 172.21.15.133:0/1535033125 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc5c037070 con 0x55fc35762bb0 2024-09-11T23:36:15.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc6e0cc740 10 client.15081 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:15.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1535033125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc3577d600 con 0x55fc35762bb0 2024-09-11T23:36:15.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc497fa640 1 -- 172.21.15.133:0/1535033125 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c024040 con 0x55fc35762bb0 2024-09-11T23:36:15.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc497fa640 1 -- 172.21.15.133:0/1535033125 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdc5c006120 con 0x55fc35762bb0 2024-09-11T23:36:15.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc6e0cc740 10 client.15081 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:15.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc6e0cc740 10 client.15081 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:36:15.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc6e0cc740 20 client.15081 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:15.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fc357e1ba0 0x55fc357e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:15.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc6e0cc740 4 client.15081 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:36:15.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1535033125 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "status"}) -- 0x55fc356c5b10 con 0x55fc357e1ba0 2024-09-11T23:36:15.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.646+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fc357e1ba0 0x55fc357e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:15.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc69a8f640 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fc357e1ba0 0x55fc357e08b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffaa6cac9ce3c965 server_cookie=427e85acf6fb75d0 in_seq=0 out_seq=0 2024-09-11T23:36:15.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc497fa640 10 client.15081 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:15.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc497fa640 1 -- 172.21.15.133:0/1535033125 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fc356c5b10 con 0x55fc357e1ba0 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc497fa640 10 client.15081 handle_command_reply: tid=0 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 1 client.15081 shutdown 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1535033125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 msgr2=0x55fc3577bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc3577bdc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c006b80 tx=0x7fdc5c0047d0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1535033125 shutdown_connections 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fc357e1ba0 0x55fc357e08b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc35762bb0 0x55fc3577bdc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc357582f0 0x55fc3575c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc497fa640 0 client.15081 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 1 --2- 172.21.15.133:0/1535033125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc357579d0 0x55fc3575c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1535033125 >> 172.21.15.133:0/1535033125 conn(0x55fc35753f60 msgr2=0x55fc35753000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:15.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1535033125 shutdown_connections 2024-09-11T23:36:15.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 1 -- 172.21.15.133:0/1535033125 wait complete. 2024-09-11T23:36:15.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.647+0000 7fdc6e0cc740 20 client.15081 trim_cache size 0 max 16384 2024-09-11T23:36:15.674 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:15.674 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:36:15.674 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-09-11T23:36:15.674 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24481, 2024-09-11T23:36:15.674 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:36:15.674 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:36:15.674 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:36:15.674 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 81.206682110000003, 2024-09-11T23:36:15.675 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 27, 2024-09-11T23:36:15.675 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 81, 2024-09-11T23:36:15.675 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 79, 2024-09-11T23:36:15.675 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 100.883241238 2024-09-11T23:36:15.675 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/419412758 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc4c0a2c10 msgr2=0x7fdc4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc4c0a2c10 0x7fdc4c0a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c006fd0 tx=0x7fdc5c019000 comp rx=0 tx=0).stop 2024-09-11T23:36:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/419412758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64102790 msgr2=0x7fdc64074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64102790 0x7fdc64074a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdc58005ab0 tx=0x7fdc580042f0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/419412758 shutdown_connections 2024-09-11T23:36:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fdc4c0a2c10 0x7fdc4c0a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc64106f10 0x7fdc64074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64102790 0x7fdc64074a30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 --2- 172.21.15.133:0/419412758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64101de0 0x7fdc640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:15.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/419412758 >> 172.21.15.133:0/419412758 conn(0x7fdc640fd8c0 msgr2=0x7fdc640fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:15.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/419412758 shutdown_connections 2024-09-11T23:36:15.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:15.648+0000 7fdc6bcf3640 1 -- 172.21.15.133:0/419412758 wait complete. 2024-09-11T23:36:16.364 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T23:36:16.598 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T23:36:17.524 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T23:36:17.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T23:36:17.525 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:17 smithi138 ceph-mon[35142]: pgmap v315: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:17.526 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:17 smithi133 ceph-mon[31430]: pgmap v315: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:17.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:17 smithi178 ceph-mon[35882]: pgmap v315: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:17.774 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:18.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.263+0000 7f0954fd4640 1 -- 172.21.15.133:0/1465086440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0950100b10 msgr2=0x7f0950100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:18.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.263+0000 7f0954fd4640 1 --2- 172.21.15.133:0/1465086440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0950100b10 0x7f0950100f10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0940009900 tx=0x7f094002a410 comp rx=0 tx=0).stop 2024-09-11T23:36:18.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.263+0000 7f0954fd4640 1 -- 172.21.15.133:0/1465086440 shutdown_connections 2024-09-11T23:36:18.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.263+0000 7f0954fd4640 1 --2- 172.21.15.133:0/1465086440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09501026d0 0x7f0950108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:18.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.263+0000 7f0954fd4640 1 --2- 172.21.15.133:0/1465086440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0950101d10 0x7f0950102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:18.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.263+0000 7f0954fd4640 1 --2- 172.21.15.133:0/1465086440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0950100b10 0x7f0950100f10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:18.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.263+0000 7f0954fd4640 1 -- 172.21.15.133:0/1465086440 >> 172.21.15.133:0/1465086440 conn(0x7f09500fc2a0 msgr2=0x7f09500fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:18.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.263+0000 7f0954fd4640 1 -- 172.21.15.133:0/1465086440 shutdown_connections 2024-09-11T23:36:18.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.263+0000 7f0954fd4640 1 -- 172.21.15.133:0/1465086440 wait complete. 2024-09-11T23:36:18.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.263+0000 7f0954fd4640 1 Processor -- start 2024-09-11T23:36:18.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.263+0000 7f0954fd4640 1 -- start start 2024-09-11T23:36:18.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f0954fd4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0950100b10 0x7f0950196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:18.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f0954fd4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0950101d10 0x7f09501973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:18.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f0954fd4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09501026d0 0x7f095019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:18.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f0954fd4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f095010c7f0 con 0x7f0950100b10 2024-09-11T23:36:18.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f0954fd4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f095010c670 con 0x7f09501026d0 2024-09-11T23:36:18.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f0954fd4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f095010c970 con 0x7f0950101d10 2024-09-11T23:36:18.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0950100b10 0x7f0950196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:18.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0950100b10 0x7f0950196e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48358/0 (socket says 172.21.15.133:48358) 2024-09-11T23:36:18.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094e59c640 1 -- 172.21.15.133:0/748162276 learned_addr learned my addr 172.21.15.133:0/748162276 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:18.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094dd9b640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0950101d10 0x7f09501973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:18.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094ed9d640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09501026d0 0x7f095019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:18.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094e59c640 1 -- 172.21.15.133:0/748162276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0950101d10 msgr2=0x7f09501973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:18.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094e59c640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0950101d10 0x7f09501973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:18.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094e59c640 1 -- 172.21.15.133:0/748162276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09501026d0 msgr2=0x7f095019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:18.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094e59c640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09501026d0 0x7f095019e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:18.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094e59c640 1 -- 172.21.15.133:0/748162276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095019eaa0 con 0x7f0950100b10 2024-09-11T23:36:18.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094dd9b640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0950101d10 0x7f09501973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:18.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094ed9d640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09501026d0 0x7f095019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:18.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f094e59c640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0950100b10 0x7f0950196e70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0940031610 tx=0x7f09400043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:18.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f093f7fe640 1 -- 172.21.15.133:0/748162276 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094003f030 con 0x7f0950100b10 2024-09-11T23:36:18.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f093f7fe640 1 -- 172.21.15.133:0/748162276 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0940003ce0 con 0x7f0950100b10 2024-09-11T23:36:18.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f093f7fe640 1 -- 172.21.15.133:0/748162276 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0940035070 con 0x7f0950100b10 2024-09-11T23:36:18.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f0954fd4640 1 -- 172.21.15.133:0/748162276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095019ed30 con 0x7f0950100b10 2024-09-11T23:36:18.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.264+0000 7f0954fd4640 1 -- 172.21.15.133:0/748162276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095019f230 con 0x7f0950100b10 2024-09-11T23:36:18.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.266+0000 7f0954fd4640 1 -- 172.21.15.133:0/748162276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0950107f60 con 0x7f0950100b10 2024-09-11T23:36:18.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.267+0000 7f093f7fe640 1 -- 172.21.15.133:0/748162276 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0940042030 con 0x7f0950100b10 2024-09-11T23:36:18.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.267+0000 7f093f7fe640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09380a2cc0 0x7f09380a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:18.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.267+0000 7f094dd9b640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09380a2cc0 0x7f09380a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:18.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.267+0000 7f093f7fe640 1 -- 172.21.15.133:0/748162276 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f09400ef650 con 0x7f0950100b10 2024-09-11T23:36:18.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.268+0000 7f094dd9b640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09380a2cc0 0x7f09380a5180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0950198390 tx=0x7f0944009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:18.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.271+0000 7f093f7fe640 1 -- 172.21.15.133:0/748162276 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09400bc550 con 0x7f0950100b10 2024-09-11T23:36:18.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.453+0000 7f0954fd4640 1 -- 172.21.15.133:0/748162276 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f0950062630 con 0x7f09380a2cc0 2024-09-11T23:36:18.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.454+0000 7f093f7fe640 1 -- 172.21.15.133:0/748162276 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0950062630 con 0x7f09380a2cc0 2024-09-11T23:36:18.454 INFO:teuthology.orchestra.run.smithi133.stdout:Backend: cephadm 2024-09-11T23:36:18.454 INFO:teuthology.orchestra.run.smithi133.stdout:Available: Yes 2024-09-11T23:36:18.454 INFO:teuthology.orchestra.run.smithi133.stdout:Paused: No 2024-09-11T23:36:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.456+0000 7f0954fd4640 1 -- 172.21.15.133:0/748162276 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09380a2cc0 msgr2=0x7f09380a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.456+0000 7f0954fd4640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09380a2cc0 0x7f09380a5180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0950198390 tx=0x7f0944009520 comp rx=0 tx=0).stop 2024-09-11T23:36:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.456+0000 7f0954fd4640 1 -- 172.21.15.133:0/748162276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0950100b10 msgr2=0x7f0950196e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.456+0000 7f0954fd4640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0950100b10 0x7f0950196e70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0940031610 tx=0x7f09400043b0 comp rx=0 tx=0).stop 2024-09-11T23:36:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.457+0000 7f0954fd4640 1 -- 172.21.15.133:0/748162276 shutdown_connections 2024-09-11T23:36:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.457+0000 7f0954fd4640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f09380a2cc0 0x7f09380a5180 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.457+0000 7f0954fd4640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09501026d0 0x7f095019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.457+0000 7f0954fd4640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0950101d10 0x7f09501973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.457+0000 7f0954fd4640 1 --2- 172.21.15.133:0/748162276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0950100b10 0x7f0950196e70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.457+0000 7f0954fd4640 1 -- 172.21.15.133:0/748162276 >> 172.21.15.133:0/748162276 conn(0x7f09500fc2a0 msgr2=0x7f0950105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.457+0000 7f0954fd4640 1 -- 172.21.15.133:0/748162276 shutdown_connections 2024-09-11T23:36:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:18.457+0000 7f0954fd4640 1 -- 172.21.15.133:0/748162276 wait complete. 2024-09-11T23:36:19.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:19 smithi133 ceph-mon[31430]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:19.315 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T23:36:19.342 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:19 smithi178 ceph-mon[35882]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:19.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:19 smithi138 ceph-mon[35142]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:19.563 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:20.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.977+0000 7f3a33e39640 1 -- 172.21.15.133:0/2033873084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a2c0ffed0 msgr2=0x7f3a2c1002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:20.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.977+0000 7f3a33e39640 1 --2- 172.21.15.133:0/2033873084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a2c0ffed0 0x7f3a2c1002d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009930 tx=0x7f3a2802a420 comp rx=0 tx=0).stop 2024-09-11T23:36:20.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.977+0000 7f3a33e39640 1 -- 172.21.15.133:0/2033873084 shutdown_connections 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.977+0000 7f3a33e39640 1 --2- 172.21.15.133:0/2033873084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2c103870 0x7f3a2c10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.977+0000 7f3a33e39640 1 --2- 172.21.15.133:0/2033873084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a2c102eb0 0x7f3a2c103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.977+0000 7f3a33e39640 1 --2- 172.21.15.133:0/2033873084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a2c0ffed0 0x7f3a2c1002d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.977+0000 7f3a33e39640 1 -- 172.21.15.133:0/2033873084 >> 172.21.15.133:0/2033873084 conn(0x7f3a2c0fba40 msgr2=0x7f3a2c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.977+0000 7f3a33e39640 1 -- 172.21.15.133:0/2033873084 shutdown_connections 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.977+0000 7f3a33e39640 1 -- 172.21.15.133:0/2033873084 wait complete. 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a33e39640 1 Processor -- start 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a33e39640 1 -- start start 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a33e39640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a2c0ffed0 0x7f3a2c102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a33e39640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2c102eb0 0x7f3a2c100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a33e39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a2c103870 0x7f3a2c1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:20.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a33e39640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c10d890 con 0x7f3a2c103870 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a33e39640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c10d710 con 0x7f3a2c102eb0 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a33e39640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c10da10 con 0x7f3a2c0ffed0 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a323d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a2c103870 0x7f3a2c1013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a323d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a2c103870 0x7f3a2c1013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48376/0 (socket says 172.21.15.133:48376) 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a323d6640 1 -- 172.21.15.133:0/2923050316 learned_addr learned my addr 172.21.15.133:0/2923050316 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a313d4640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2c102eb0 0x7f3a2c100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a31bd5640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a2c0ffed0 0x7f3a2c102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a323d6640 1 -- 172.21.15.133:0/2923050316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a2c0ffed0 msgr2=0x7f3a2c102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a323d6640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a2c0ffed0 0x7f3a2c102810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a323d6640 1 -- 172.21.15.133:0/2923050316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2c102eb0 msgr2=0x7f3a2c100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a323d6640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2c102eb0 0x7f3a2c100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:20.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.978+0000 7f3a323d6640 1 -- 172.21.15.133:0/2923050316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2c101c60 con 0x7f3a2c103870 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.979+0000 7f3a323d6640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a2c103870 0x7f3a2c1013a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400eaa0 tx=0x7f3a2400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.979+0000 7f3a22ffd640 1 -- 172.21.15.133:0/2923050316 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2400cdb0 con 0x7f3a2c103870 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.979+0000 7f3a22ffd640 1 -- 172.21.15.133:0/2923050316 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a24005020 con 0x7f3a2c103870 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.979+0000 7f3a33e39640 1 -- 172.21.15.133:0/2923050316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2c1a2140 con 0x7f3a2c103870 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.979+0000 7f3a22ffd640 1 -- 172.21.15.133:0/2923050316 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a240116b0 con 0x7f3a2c103870 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.979+0000 7f3a33e39640 1 -- 172.21.15.133:0/2923050316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2c1a2600 con 0x7f3a2c103870 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.980+0000 7f3a33e39640 1 -- 172.21.15.133:0/2923050316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a2c109150 con 0x7f3a2c103870 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.981+0000 7f3a22ffd640 1 -- 172.21.15.133:0/2923050316 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3a24005560 con 0x7f3a2c103870 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.981+0000 7f3a22ffd640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a180a2c10 0x7f3a180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.982+0000 7f3a31bd5640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a180a2c10 0x7f3a180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.982+0000 7f3a22ffd640 1 -- 172.21.15.133:0/2923050316 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f3a240cf350 con 0x7f3a2c103870 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.982+0000 7f3a31bd5640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a180a2c10 0x7f3a180a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a28031610 tx=0x7f3a28000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:20.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:19.986+0000 7f3a22ffd640 1 -- 172.21.15.133:0/2923050316 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a24016030 con 0x7f3a2c103870 2024-09-11T23:36:20.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.163+0000 7f3a33e39640 1 -- 172.21.15.133:0/2923050316 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3a2c062630 con 0x7f3a180a2c10 2024-09-11T23:36:20.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.175+0000 7f3a22ffd640 1 -- 172.21.15.133:0/2923050316 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f3a2c062630 con 0x7f3a180a2c10 2024-09-11T23:36:20.175 INFO:teuthology.orchestra.run.smithi133.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T23:36:20.175 INFO:teuthology.orchestra.run.smithi133.stdout:mds.a smithi133 running (100s) 20s ago 99s 16.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 f47ac7b17b43 2024-09-11T23:36:20.175 INFO:teuthology.orchestra.run.smithi133.stdout:mds.b smithi138 running (97s) 29s ago 97s 17.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 dfb088c5ff20 2024-09-11T23:36:20.175 INFO:teuthology.orchestra.run.smithi133.stdout:mds.c smithi178 running (110s) 29s ago 110s 17.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 0440d39b47ca 2024-09-11T23:36:20.175 INFO:teuthology.orchestra.run.smithi133.stdout:mds.d smithi133 running (118s) 20s ago 117s 15.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 bca343033f22 2024-09-11T23:36:20.175 INFO:teuthology.orchestra.run.smithi133.stdout:mds.e smithi138 running (115s) 29s ago 115s 20.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 4a6d2a34381e 2024-09-11T23:36:20.175 INFO:teuthology.orchestra.run.smithi133.stdout:mds.f smithi178 running (113s) 29s ago 112s 19.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 486f09cfb721 2024-09-11T23:36:20.175 INFO:teuthology.orchestra.run.smithi133.stdout:mds.g smithi133 running (103s) 20s ago 102s 17.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 0fa68f80ac2e 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:mds.h smithi138 running (108s) 29s ago 107s 16.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 f93610347ef8 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:mds.i smithi178 running (105s) 29s ago 105s 28.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 ac19066487aa 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:mds.j smithi133 running (2m) 20s ago 2m 16.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 34093403e5e3 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:mds.k smithi138 running (2m) 29s ago 2m 17.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 53cbe519fe35 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:mds.l smithi178 running (2m) 29s ago 2m 17.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 d605213c2822 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.x smithi133 *:9283,8765 running (8m) 20s ago 8m 509M - 19.1.1-298-gb53b6d3e 54467abc55f8 85e3425cd9c0 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.y smithi138 *:8443,8765 running (7m) 29s ago 7m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b7d14b3bbac 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.z smithi178 *:8443,8765 running (7m) 29s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 44877262ab92 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:mon.a smithi133 running (8m) 20s ago 8m 50.8M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 573b8e5cf426 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:mon.b smithi138 running (7m) 29s ago 7m 39.1M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 645e84e2dcdb 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:mon.c smithi178 running (7m) 29s ago 7m 42.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 e0fe9a71a8b5 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:osd.0 smithi133 running (6m) 20s ago 6m 56.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 f490ce00575e 2024-09-11T23:36:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:osd.1 smithi138 running (6m) 29s ago 6m 51.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 dc20fe753528 2024-09-11T23:36:20.177 INFO:teuthology.orchestra.run.smithi133.stdout:osd.2 smithi178 running (6m) 29s ago 6m 49.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 aead7c577c2f 2024-09-11T23:36:20.177 INFO:teuthology.orchestra.run.smithi133.stdout:osd.3 smithi133 running (5m) 20s ago 5m 46.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 bf7577837d59 2024-09-11T23:36:20.177 INFO:teuthology.orchestra.run.smithi133.stdout:osd.4 smithi138 running (5m) 29s ago 5m 53.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 25dc8ea4f57a 2024-09-11T23:36:20.177 INFO:teuthology.orchestra.run.smithi133.stdout:osd.5 smithi178 running (4m) 29s ago 4m 47.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 1baf5e1f3f23 2024-09-11T23:36:20.177 INFO:teuthology.orchestra.run.smithi133.stdout:osd.6 smithi133 running (4m) 20s ago 4m 44.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 60edf2443a35 2024-09-11T23:36:20.177 INFO:teuthology.orchestra.run.smithi133.stdout:osd.7 smithi138 running (4m) 29s ago 4m 54.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 c1d761cfd9fc 2024-09-11T23:36:20.177 INFO:teuthology.orchestra.run.smithi133.stdout:osd.8 smithi178 running (3m) 29s ago 3m 45.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 6274f2da20b7 2024-09-11T23:36:20.177 INFO:teuthology.orchestra.run.smithi133.stdout:osd.9 smithi133 running (3m) 20s ago 3m 50.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 fabafb439d44 2024-09-11T23:36:20.177 INFO:teuthology.orchestra.run.smithi133.stdout:osd.10 smithi138 running (2m) 29s ago 2m 47.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 01642201aab5 2024-09-11T23:36:20.177 INFO:teuthology.orchestra.run.smithi133.stdout:osd.11 smithi178 running (2m) 29s ago 2m 43.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 182e022c208c 2024-09-11T23:36:20.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.177+0000 7f3a33e39640 1 -- 172.21.15.133:0/2923050316 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a180a2c10 msgr2=0x7f3a180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:20.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.177+0000 7f3a33e39640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a180a2c10 0x7f3a180a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a28031610 tx=0x7f3a28000bc0 comp rx=0 tx=0).stop 2024-09-11T23:36:20.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.177+0000 7f3a33e39640 1 -- 172.21.15.133:0/2923050316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a2c103870 msgr2=0x7f3a2c1013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:20.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.177+0000 7f3a33e39640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a2c103870 0x7f3a2c1013a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400eaa0 tx=0x7f3a2400ef60 comp rx=0 tx=0).stop 2024-09-11T23:36:20.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.177+0000 7f3a33e39640 1 -- 172.21.15.133:0/2923050316 shutdown_connections 2024-09-11T23:36:20.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.177+0000 7f3a33e39640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f3a180a2c10 0x7f3a180a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:20.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.177+0000 7f3a33e39640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a2c103870 0x7f3a2c1013a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:20.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.177+0000 7f3a33e39640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2c102eb0 0x7f3a2c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:20.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.177+0000 7f3a33e39640 1 --2- 172.21.15.133:0/2923050316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a2c0ffed0 0x7f3a2c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:20.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.177+0000 7f3a33e39640 1 -- 172.21.15.133:0/2923050316 >> 172.21.15.133:0/2923050316 conn(0x7f3a2c0fba40 msgr2=0x7f3a2c106320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:20.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.178+0000 7f3a33e39640 1 -- 172.21.15.133:0/2923050316 shutdown_connections 2024-09-11T23:36:20.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:20.178+0000 7f3a33e39640 1 -- 172.21.15.133:0/2923050316 wait complete. 2024-09-11T23:36:20.358 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:20 smithi178 ceph-mon[35882]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:20.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:20 smithi138 ceph-mon[35142]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:20.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:20 smithi133 ceph-mon[31430]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:20.838 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T23:36:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:21.206 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:21 smithi133 ceph-mon[31430]: pgmap v317: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:21.206 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:21 smithi133 ceph-mon[31430]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:21.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:21 smithi138 ceph-mon[35142]: pgmap v317: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:21.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:21 smithi138 ceph-mon[35142]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:21.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:21 smithi178 ceph-mon[35882]: pgmap v317: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:21.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:21 smithi178 ceph-mon[35882]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:21.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.551+0000 7f7bffcb8640 1 -- 172.21.15.133:0/921093070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bf8107900 msgr2=0x7f7bf8109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:21.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.551+0000 7f7bffcb8640 1 --2- 172.21.15.133:0/921093070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bf8107900 0x7f7bf8109d50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009ca0 tx=0x7f7bf002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.551+0000 7f7bffcb8640 1 -- 172.21.15.133:0/921093070 shutdown_connections 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.551+0000 7f7bffcb8640 1 --2- 172.21.15.133:0/921093070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bf8107900 0x7f7bf8109d50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.551+0000 7f7bffcb8640 1 --2- 172.21.15.133:0/921093070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf8104fd0 0x7f7bf81073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.551+0000 7f7bffcb8640 1 --2- 172.21.15.133:0/921093070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf81026a0 0x7f7bf8104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.551+0000 7f7bffcb8640 1 -- 172.21.15.133:0/921093070 >> 172.21.15.133:0/921093070 conn(0x7f7bf80fc2a0 msgr2=0x7f7bf80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.551+0000 7f7bffcb8640 1 -- 172.21.15.133:0/921093070 shutdown_connections 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.551+0000 7f7bffcb8640 1 -- 172.21.15.133:0/921093070 wait complete. 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.551+0000 7f7bffcb8640 1 Processor -- start 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bffcb8640 1 -- start start 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bffcb8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf81026a0 0x7f7bf8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bffcb8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8104fd0 0x7f7bf8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bffcb8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bf8107900 0x7f7bf819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bffcb8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf810d8a0 con 0x7f7bf8107900 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bffcb8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf810d720 con 0x7f7bf8104fd0 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bffcb8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf810da20 con 0x7f7bf81026a0 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bfe255640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bf8107900 0x7f7bf819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bfe255640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bf8107900 0x7f7bf819f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48394/0 (socket says 172.21.15.133:48394) 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bfe255640 1 -- 172.21.15.133:0/2471615574 learned_addr learned my addr 172.21.15.133:0/2471615574 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bfda54640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf81026a0 0x7f7bf8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bfd253640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8104fd0 0x7f7bf8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bfe255640 1 -- 172.21.15.133:0/2471615574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf81026a0 msgr2=0x7f7bf8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bfe255640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf81026a0 0x7f7bf8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bfe255640 1 -- 172.21.15.133:0/2471615574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8104fd0 msgr2=0x7f7bf8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bfe255640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8104fd0 0x7f7bf8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bfe255640 1 -- 172.21.15.133:0/2471615574 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf819fa70 con 0x7f7bf8107900 2024-09-11T23:36:21.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bfe255640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bf8107900 0x7f7bf819f400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0005e30 tx=0x7f7bf0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:21.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7beeffd640 1 -- 172.21.15.133:0/2471615574 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0003b70 con 0x7f7bf8107900 2024-09-11T23:36:21.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7beeffd640 1 -- 172.21.15.133:0/2471615574 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf0003d10 con 0x7f7bf8107900 2024-09-11T23:36:21.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7bffcb8640 1 -- 172.21.15.133:0/2471615574 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf819fd00 con 0x7f7bf8107900 2024-09-11T23:36:21.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.552+0000 7f7beeffd640 1 -- 172.21.15.133:0/2471615574 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf002fd30 con 0x7f7bf8107900 2024-09-11T23:36:21.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.553+0000 7f7bffcb8640 1 -- 172.21.15.133:0/2471615574 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf81a01e0 con 0x7f7bf8107900 2024-09-11T23:36:21.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.553+0000 7f7bffcb8640 1 -- 172.21.15.133:0/2471615574 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf80fdd90 con 0x7f7bf8107900 2024-09-11T23:36:21.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.555+0000 7f7beeffd640 1 -- 172.21.15.133:0/2471615574 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7bf0004050 con 0x7f7bf8107900 2024-09-11T23:36:21.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.555+0000 7f7beeffd640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7be40a2c40 0x7f7be40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:21.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.556+0000 7f7bfda54640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7be40a2c40 0x7f7be40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:21.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.556+0000 7f7beeffd640 1 -- 172.21.15.133:0/2471615574 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f7bf00f0210 con 0x7f7bf8107900 2024-09-11T23:36:21.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.556+0000 7f7bfda54640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7be40a2c40 0x7f7be40a5100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7be800abe0 tx=0x7f7be800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:21.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.560+0000 7f7beeffd640 1 -- 172.21.15.133:0/2471615574 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7bf00bd340 con 0x7f7bf8107900 2024-09-11T23:36:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.734+0000 7f7bffcb8640 1 -- 172.21.15.133:0/2471615574 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7bf8062630 con 0x7f7be40a2c40 2024-09-11T23:36:21.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.740+0000 7f7beeffd640 1 -- 172.21.15.133:0/2471615574 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7bf8062630 con 0x7f7be40a2c40 2024-09-11T23:36:21.740 INFO:teuthology.orchestra.run.smithi133.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T23:36:21.740 INFO:teuthology.orchestra.run.smithi133.stdout:mds.all 12/12 31s ago 2m smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-11T23:36:21.740 INFO:teuthology.orchestra.run.smithi133.stdout:mgr 3/3 31s ago 7m smithi133=x;smithi138=y;smithi178=z;count:3 2024-09-11T23:36:21.740 INFO:teuthology.orchestra.run.smithi133.stdout:mon 3/3 31s ago 7m smithi133:172.21.15.133=a;smithi138:172.21.15.138=b;smithi178:172.21.15.178=c;count:3 2024-09-11T23:36:21.741 INFO:teuthology.orchestra.run.smithi133.stdout:osd 12 31s ago - 2024-09-11T23:36:21.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.742+0000 7f7bffcb8640 1 -- 172.21.15.133:0/2471615574 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7be40a2c40 msgr2=0x7f7be40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:21.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.742+0000 7f7bffcb8640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7be40a2c40 0x7f7be40a5100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7be800abe0 tx=0x7f7be800a2d0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.742+0000 7f7bffcb8640 1 -- 172.21.15.133:0/2471615574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bf8107900 msgr2=0x7f7bf819f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.743+0000 7f7bffcb8640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bf8107900 0x7f7bf819f400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0005e30 tx=0x7f7bf0031ee0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.743+0000 7f7bffcb8640 1 -- 172.21.15.133:0/2471615574 shutdown_connections 2024-09-11T23:36:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.743+0000 7f7bffcb8640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7be40a2c40 0x7f7be40a5100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.743+0000 7f7bffcb8640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bf8107900 0x7f7bf819f400 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.743+0000 7f7bffcb8640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8104fd0 0x7f7bf8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.743+0000 7f7bffcb8640 1 --2- 172.21.15.133:0/2471615574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf81026a0 0x7f7bf8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.743+0000 7f7bffcb8640 1 -- 172.21.15.133:0/2471615574 >> 172.21.15.133:0/2471615574 conn(0x7f7bf80fc2a0 msgr2=0x7f7bf8102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.743+0000 7f7bffcb8640 1 -- 172.21.15.133:0/2471615574 shutdown_connections 2024-09-11T23:36:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:21.743+0000 7f7bffcb8640 1 -- 172.21.15.133:0/2471615574 wait complete. 2024-09-11T23:36:22.434 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:22 smithi133 ceph-mon[31430]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:22.434 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:22 smithi133 ceph-mon[31430]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:22.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:22 smithi138 ceph-mon[35142]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:22.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:22 smithi138 ceph-mon[35142]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:22.463 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T23:36:22.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:22 smithi178 ceph-mon[35882]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:22.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:22 smithi178 ceph-mon[35882]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:22.704 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:23.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.169+0000 7f6a434aa640 1 -- 172.21.15.133:0/3989504945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a3c102e00 msgr2=0x7f6a3c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:23.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.169+0000 7f6a434aa640 1 --2- 172.21.15.133:0/3989504945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a3c102e00 0x7f6a3c103280 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009900 tx=0x7f6a3802a410 comp rx=0 tx=0).stop 2024-09-11T23:36:23.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.169+0000 7f6a434aa640 1 -- 172.21.15.133:0/3989504945 shutdown_connections 2024-09-11T23:36:23.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.169+0000 7f6a434aa640 1 --2- 172.21.15.133:0/3989504945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a3c1037c0 0x7f6a3c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.169+0000 7f6a434aa640 1 --2- 172.21.15.133:0/3989504945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a3c102e00 0x7f6a3c103280 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.169+0000 7f6a434aa640 1 --2- 172.21.15.133:0/3989504945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c101c00 0x7f6a3c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.169+0000 7f6a434aa640 1 -- 172.21.15.133:0/3989504945 >> 172.21.15.133:0/3989504945 conn(0x7f6a3c0fd390 msgr2=0x7f6a3c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.169+0000 7f6a434aa640 1 -- 172.21.15.133:0/3989504945 shutdown_connections 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.169+0000 7f6a434aa640 1 -- 172.21.15.133:0/3989504945 wait complete. 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.169+0000 7f6a434aa640 1 Processor -- start 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.169+0000 7f6a434aa640 1 -- start start 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a434aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a3c101c00 0x7f6a3c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a434aa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a3c102e00 0x7f6a3c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a434aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c1037c0 0x7f6a3c19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a434aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c10d8e0 con 0x7f6a3c101c00 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a434aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c10d760 con 0x7f6a3c1037c0 2024-09-11T23:36:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a434aa640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c10da60 con 0x7f6a3c102e00 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a41246640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a3c101c00 0x7f6a3c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a41246640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a3c101c00 0x7f6a3c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43804/0 (socket says 172.21.15.133:43804) 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a41246640 1 -- 172.21.15.133:0/2359198366 learned_addr learned my addr 172.21.15.133:0/2359198366 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a41a47640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c1037c0 0x7f6a3c19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a41246640 1 -- 172.21.15.133:0/2359198366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a3c102e00 msgr2=0x7f6a3c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a41246640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a3c102e00 0x7f6a3c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a41246640 1 -- 172.21.15.133:0/2359198366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c1037c0 msgr2=0x7f6a3c19f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a41246640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c1037c0 0x7f6a3c19f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a41246640 1 -- 172.21.15.133:0/2359198366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3c19fb00 con 0x7f6a3c101c00 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a41a47640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c1037c0 0x7f6a3c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a41246640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a3c101c00 0x7f6a3c197ed0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c970 tx=0x7f6a2c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a327fc640 1 -- 172.21.15.133:0/2359198366 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c007c10 con 0x7f6a3c101c00 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a434aa640 1 -- 172.21.15.133:0/2359198366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3c19fdf0 con 0x7f6a3c101c00 2024-09-11T23:36:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a327fc640 1 -- 172.21.15.133:0/2359198366 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2c007db0 con 0x7f6a3c101c00 2024-09-11T23:36:23.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a327fc640 1 -- 172.21.15.133:0/2359198366 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00f940 con 0x7f6a3c101c00 2024-09-11T23:36:23.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.170+0000 7f6a434aa640 1 -- 172.21.15.133:0/2359198366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3c1a0320 con 0x7f6a3c101c00 2024-09-11T23:36:23.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.172+0000 7f6a434aa640 1 -- 172.21.15.133:0/2359198366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a3c108fb0 con 0x7f6a3c101c00 2024-09-11T23:36:23.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.174+0000 7f6a327fc640 1 -- 172.21.15.133:0/2359198366 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6a2c00fb00 con 0x7f6a3c101c00 2024-09-11T23:36:23.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.174+0000 7f6a327fc640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6a280a2c40 0x7f6a280a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:23.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.174+0000 7f6a40a45640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6a280a2c40 0x7f6a280a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:23.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.174+0000 7f6a327fc640 1 -- 172.21.15.133:0/2359198366 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f6a2c0cf750 con 0x7f6a3c101c00 2024-09-11T23:36:23.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.175+0000 7f6a40a45640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6a280a2c40 0x7f6a280a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6a380096c0 tx=0x7f6a38031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:23.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.177+0000 7f6a327fc640 1 -- 172.21.15.133:0/2359198366 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a2c09c770 con 0x7f6a3c101c00 2024-09-11T23:36:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.349+0000 7f6a434aa640 1 -- 172.21.15.133:0/2359198366 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f6a3c062630 con 0x7f6a280a2c40 2024-09-11T23:36:23.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.351+0000 7f6a327fc640 1 -- 172.21.15.133:0/2359198366 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f6a3c062630 con 0x7f6a280a2c40 2024-09-11T23:36:23.351 INFO:teuthology.orchestra.run.smithi133.stdout:HOST ADDR LABELS STATUS 2024-09-11T23:36:23.351 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133 172.21.15.133 2024-09-11T23:36:23.351 INFO:teuthology.orchestra.run.smithi133.stdout:smithi138 172.21.15.138 2024-09-11T23:36:23.351 INFO:teuthology.orchestra.run.smithi133.stdout:smithi178 172.21.15.178 2024-09-11T23:36:23.351 INFO:teuthology.orchestra.run.smithi133.stdout:3 hosts in cluster 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 -- 172.21.15.133:0/2359198366 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6a280a2c40 msgr2=0x7f6a280a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6a280a2c40 0x7f6a280a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6a380096c0 tx=0x7f6a38031400 comp rx=0 tx=0).stop 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 -- 172.21.15.133:0/2359198366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a3c101c00 msgr2=0x7f6a3c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a3c101c00 0x7f6a3c197ed0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c970 tx=0x7f6a2c00ce30 comp rx=0 tx=0).stop 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 -- 172.21.15.133:0/2359198366 shutdown_connections 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f6a280a2c40 0x7f6a280a5100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c1037c0 0x7f6a3c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a3c102e00 0x7f6a3c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 --2- 172.21.15.133:0/2359198366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a3c101c00 0x7f6a3c197ed0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 -- 172.21.15.133:0/2359198366 >> 172.21.15.133:0/2359198366 conn(0x7f6a3c0fd390 msgr2=0x7f6a3c106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 -- 172.21.15.133:0/2359198366 shutdown_connections 2024-09-11T23:36:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:23.353+0000 7f6a434aa640 1 -- 172.21.15.133:0/2359198366 wait complete. 2024-09-11T23:36:24.062 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T23:36:24.309 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.735+0000 7fe872861640 1 -- 172.21.15.133:0/695404980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe86c100b10 msgr2=0x7fe86c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.735+0000 7fe872861640 1 --2- 172.21.15.133:0/695404980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe86c100b10 0x7fe86c100f10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009900 tx=0x7fe85c02a410 comp rx=0 tx=0).stop 2024-09-11T23:36:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 -- 172.21.15.133:0/695404980 shutdown_connections 2024-09-11T23:36:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 --2- 172.21.15.133:0/695404980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c1026d0 0x7fe86c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 --2- 172.21.15.133:0/695404980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe86c101d10 0x7fe86c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 --2- 172.21.15.133:0/695404980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe86c100b10 0x7fe86c100f10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 -- 172.21.15.133:0/695404980 >> 172.21.15.133:0/695404980 conn(0x7fe86c0fc2a0 msgr2=0x7fe86c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:24.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 -- 172.21.15.133:0/695404980 shutdown_connections 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 -- 172.21.15.133:0/695404980 wait complete. 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 Processor -- start 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 -- start start 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe86c100b10 0x7fe86c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe86c101d10 0x7fe86c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.736+0000 7fe872861640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c1026d0 0x7fe86c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe872861640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c10c770 con 0x7fe86c100b10 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe872861640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c10c5f0 con 0x7fe86c101d10 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe872861640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c10c8f0 con 0x7fe86c1026d0 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe86bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe86c100b10 0x7fe86c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe86bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe86c100b10 0x7fe86c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43832/0 (socket says 172.21.15.133:43832) 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe86bfff640 1 -- 172.21.15.133:0/2509297582 learned_addr learned my addr 172.21.15.133:0/2509297582 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:24.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe870dfe640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c1026d0 0x7fe86c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe86bfff640 1 -- 172.21.15.133:0/2509297582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c1026d0 msgr2=0x7fe86c19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe86bfff640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c1026d0 0x7fe86c19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe86bfff640 1 -- 172.21.15.133:0/2509297582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe86c101d10 msgr2=0x7fe86c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe86bfff640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe86c101d10 0x7fe86c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe86bfff640 1 -- 172.21.15.133:0/2509297582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c19fb20 con 0x7fe86c100b10 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe870dfe640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c1026d0 0x7fe86c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe86bfff640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe86c100b10 0x7fe86c197ef0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe85c031610 tx=0x7fe85c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe8697fa640 1 -- 172.21.15.133:0/2509297582 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c03f030 con 0x7fe86c100b10 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe872861640 1 -- 172.21.15.133:0/2509297582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86c19fdb0 con 0x7fe86c100b10 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe8697fa640 1 -- 172.21.15.133:0/2509297582 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe85c003ce0 con 0x7fe86c100b10 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe8697fa640 1 -- 172.21.15.133:0/2509297582 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c035070 con 0x7fe86c100b10 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.737+0000 7fe872861640 1 -- 172.21.15.133:0/2509297582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86c1a0230 con 0x7fe86c100b10 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.738+0000 7fe872861640 1 -- 172.21.15.133:0/2509297582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe86c1a0460 con 0x7fe86c100b10 2024-09-11T23:36:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.739+0000 7fe8697fa640 1 -- 172.21.15.133:0/2509297582 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe85c042030 con 0x7fe86c100b10 2024-09-11T23:36:24.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.740+0000 7fe8697fa640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8540a2c40 0x7fe8540a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:24.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.740+0000 7fe86b7fe640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8540a2c40 0x7fe8540a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:24.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.740+0000 7fe8697fa640 1 -- 172.21.15.133:0/2509297582 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fe85c0ef7f0 con 0x7fe86c100b10 2024-09-11T23:36:24.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.740+0000 7fe86b7fe640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8540a2c40 0x7fe8540a5100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe86c199410 tx=0x7fe860005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:24.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.744+0000 7fe8697fa640 1 -- 172.21.15.133:0/2509297582 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe85c0bc6f0 con 0x7fe86c100b10 2024-09-11T23:36:24.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.917+0000 7fe872861640 1 -- 172.21.15.133:0/2509297582 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe86c06af50 con 0x7fe8540a2c40 2024-09-11T23:36:24.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.919+0000 7fe8697fa640 1 -- 172.21.15.133:0/2509297582 <== mgr.14150 v2:172.21.15.133:6800/3324898762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe86c06af50 con 0x7fe8540a2c40 2024-09-11T23:36:24.919 INFO:teuthology.orchestra.run.smithi133.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T23:36:24.919 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401AQ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-11T23:36:24.919 INFO:teuthology.orchestra.run.smithi133.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T23:36:24.919 INFO:teuthology.orchestra.run.smithi133.stdout:smithi178 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130029480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-11T23:36:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.921+0000 7fe872861640 1 -- 172.21.15.133:0/2509297582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8540a2c40 msgr2=0x7fe8540a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.921+0000 7fe872861640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8540a2c40 0x7fe8540a5100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe86c199410 tx=0x7fe860005c70 comp rx=0 tx=0).stop 2024-09-11T23:36:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.922+0000 7fe872861640 1 -- 172.21.15.133:0/2509297582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe86c100b10 msgr2=0x7fe86c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.922+0000 7fe872861640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe86c100b10 0x7fe86c197ef0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe85c031610 tx=0x7fe85c0043b0 comp rx=0 tx=0).stop 2024-09-11T23:36:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.922+0000 7fe872861640 1 -- 172.21.15.133:0/2509297582 shutdown_connections 2024-09-11T23:36:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.922+0000 7fe872861640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe8540a2c40 0x7fe8540a5100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.922+0000 7fe872861640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe86c1026d0 0x7fe86c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.922+0000 7fe872861640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe86c101d10 0x7fe86c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.922+0000 7fe872861640 1 --2- 172.21.15.133:0/2509297582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe86c100b10 0x7fe86c197ef0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.922+0000 7fe872861640 1 -- 172.21.15.133:0/2509297582 >> 172.21.15.133:0/2509297582 conn(0x7fe86c0fc2a0 msgr2=0x7fe86c06a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.922+0000 7fe872861640 1 -- 172.21.15.133:0/2509297582 shutdown_connections 2024-09-11T23:36:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:24.922+0000 7fe872861640 1 -- 172.21.15.133:0/2509297582 wait complete. 2024-09-11T23:36:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:24 smithi138 ceph-mon[35142]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:24 smithi138 ceph-mon[35142]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:24 smithi133 ceph-mon[31430]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:24 smithi133 ceph-mon[31430]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:25.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:24 smithi178 ceph-mon[35882]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:25.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:24 smithi178 ceph-mon[35882]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:25.669 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T23:36:25.678 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T23:36:25.679 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-11T23:36:25.926 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:26.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:25 smithi133 ceph-mon[31430]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:26.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:25 smithi138 ceph-mon[35142]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:26.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:25 smithi178 ceph-mon[35882]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:26.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.365+0000 7fc9681da640 1 -- 172.21.15.133:0/652944957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc960101750 msgr2=0x7fc960101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:26.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.365+0000 7fc9681da640 1 --2- 172.21.15.133:0/652944957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc960101750 0x7fc960101c00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc958009ca0 tx=0x7fc95802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:36:26.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.365+0000 7fc9681da640 1 -- 172.21.15.133:0/652944957 shutdown_connections 2024-09-11T23:36:26.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.365+0000 7fc9681da640 1 --2- 172.21.15.133:0/652944957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc960101750 0x7fc960101c00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:26.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.365+0000 7fc9681da640 1 --2- 172.21.15.133:0/652944957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960100db0 0x7fc960101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:26.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.365+0000 7fc9681da640 1 --2- 172.21.15.133:0/652944957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc960100470 0x7fc960100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.365+0000 7fc9681da640 1 -- 172.21.15.133:0/652944957 >> 172.21.15.133:0/652944957 conn(0x7fc9600fc280 msgr2=0x7fc9600fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.365+0000 7fc9681da640 1 -- 172.21.15.133:0/652944957 shutdown_connections 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.365+0000 7fc9681da640 1 -- 172.21.15.133:0/652944957 wait complete. 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc9681da640 1 Processor -- start 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc9681da640 1 -- start start 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc9681da640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc960100470 0x7fc96019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc9681da640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960100db0 0x7fc96019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc9681da640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc960101750 0x7fc9601a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc9681da640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc96010fa60 con 0x7fc960100470 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc9681da640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc96010f8e0 con 0x7fc960100db0 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc9681da640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc96010fbe0 con 0x7fc960101750 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc965f76640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc960100470 0x7fc96019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:26.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc965f76640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc960100470 0x7fc96019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43850/0 (socket says 172.21.15.133:43850) 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc965f76640 1 -- 172.21.15.133:0/3195128395 learned_addr learned my addr 172.21.15.133:0/3195128395 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc966777640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc960101750 0x7fc9601a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc965f76640 1 -- 172.21.15.133:0/3195128395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc960101750 msgr2=0x7fc9601a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.366+0000 7fc965f76640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc960101750 0x7fc9601a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.367+0000 7fc965f76640 1 -- 172.21.15.133:0/3195128395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960100db0 msgr2=0x7fc96019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.367+0000 7fc965775640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960100db0 0x7fc96019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.367+0000 7fc965f76640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960100db0 0x7fc96019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.367+0000 7fc965f76640 1 -- 172.21.15.133:0/3195128395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9601a1da0 con 0x7fc960100470 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.367+0000 7fc966777640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc960101750 0x7fc9601a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.367+0000 7fc965f76640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc960100470 0x7fc96019a170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc95400d770 tx=0x7fc95400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.367+0000 7fc952ffd640 1 -- 172.21.15.133:0/3195128395 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954017070 con 0x7fc960100470 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.367+0000 7fc952ffd640 1 -- 172.21.15.133:0/3195128395 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc954005500 con 0x7fc960100470 2024-09-11T23:36:26.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.367+0000 7fc9681da640 1 -- 172.21.15.133:0/3195128395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9601a2090 con 0x7fc960100470 2024-09-11T23:36:26.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.367+0000 7fc952ffd640 1 -- 172.21.15.133:0/3195128395 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954004e00 con 0x7fc960100470 2024-09-11T23:36:26.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.367+0000 7fc9681da640 1 -- 172.21.15.133:0/3195128395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9601a2580 con 0x7fc960100470 2024-09-11T23:36:26.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.368+0000 7fc9681da640 1 -- 172.21.15.133:0/3195128395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96010b3c0 con 0x7fc960100470 2024-09-11T23:36:26.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.369+0000 7fc952ffd640 1 -- 172.21.15.133:0/3195128395 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc954004fa0 con 0x7fc960100470 2024-09-11T23:36:26.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.370+0000 7fc952ffd640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc94c0a2c40 0x7fc94c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:26.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.370+0000 7fc965775640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc94c0a2c40 0x7fc94c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:26.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.370+0000 7fc952ffd640 1 -- 172.21.15.133:0/3195128395 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fc9540cf0b0 con 0x7fc960100470 2024-09-11T23:36:26.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.370+0000 7fc965775640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc94c0a2c40 0x7fc94c0a5100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc95c005fd0 tx=0x7fc95c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:26.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.374+0000 7fc952ffd640 1 -- 172.21.15.133:0/3195128395 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc95409beb0 con 0x7fc960100470 2024-09-11T23:36:26.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.577+0000 7fc9681da640 1 -- 172.21.15.133:0/3195128395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc96002d7c0 con 0x7fc960100470 2024-09-11T23:36:26.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.578+0000 7fc952ffd640 1 -- 172.21.15.133:0/3195128395 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3142 (secure 0 0 0) 0x7fc95409b7b0 con 0x7fc960100470 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:e27 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:btime 2024-09-11T23:35:05:451373+0000 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:legacy client fscid: 1 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:Filesystem 'cephfs' (1) 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:fs_name cephfs 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 27 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:created 2024-09-11T23:34:53.462890+0000 2024-09-11T23:36:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:modified 2024-09-11T23:35:05.451342+0000 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:tableserver 0 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:root 0 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:session_timeout 300 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:session_autoclose 300 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:max_file_size 1099511627776 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:max_xattr_size 65536 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:required_client_features {} 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:last_failure 0 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:last_failure_osd_epoch 0 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:max_mds 5 2024-09-11T23:36:26.579 INFO:teuthology.orchestra.run.smithi133.stdout:in 0,1,2,3,4 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:up {0=24481,1=24472,2=24464,3=24419,4=24401} 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:failed 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:damaged 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:stopped 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:data_pools [3] 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:metadata_pool 2 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:inline_data disabled 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:balancer 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:bal_rank_mask -1 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:standby_count_wanted 0 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:qdb_cluster leader: 24481 members: 24481,24472,24464,24419,24401 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.i{0:24481} state up:active seq 6 addr [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.f{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.178:6834/1681049778,v1:172.21.15.178:6835/1681049778] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c{1:24472} state up:active seq 9 addr [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.h{1:24440} state up:standby-replay seq 1 addr [v2:172.21.15.138:6836/1115908278,v1:172.21.15.138:6837/1115908278] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b{2:24464} state up:active seq 6 addr [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.l{2:24430} state up:standby-replay seq 1 addr [v2:172.21.15.178:6832/3569562128,v1:172.21.15.178:6833/3569562128] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.e{3:24419} state up:active seq 11 addr [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a{3:14643} state up:standby-replay seq 1 addr [v2:172.21.15.133:6840/1294883246,v1:172.21.15.133:6841/1294883246] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.k{4:24401} state up:active seq 14 addr [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.g{4:14637} state up:standby-replay seq 1 addr [v2:172.21.15.133:6838/570951791,v1:172.21.15.133:6839/570951791] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout:Standby daemons: 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.j{-1:14571} state up:standby seq 1 addr [v2:172.21.15.133:6834/1603982530,v1:172.21.15.133:6835/1603982530] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.581 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d{-1:14583} state up:standby seq 1 addr [v2:172.21.15.133:6836/3098733497,v1:172.21.15.133:6837/3098733497] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:36:26.582 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-09-11T23:36:26.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 -- 172.21.15.133:0/3195128395 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc94c0a2c40 msgr2=0x7fc94c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:26.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc94c0a2c40 0x7fc94c0a5100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc95c005fd0 tx=0x7fc95c005850 comp rx=0 tx=0).stop 2024-09-11T23:36:26.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 -- 172.21.15.133:0/3195128395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc960100470 msgr2=0x7fc96019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:26.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc960100470 0x7fc96019a170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc95400d770 tx=0x7fc95400dc30 comp rx=0 tx=0).stop 2024-09-11T23:36:26.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 -- 172.21.15.133:0/3195128395 shutdown_connections 2024-09-11T23:36:26.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fc94c0a2c40 0x7fc94c0a5100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:26.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc960101750 0x7fc9601a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:26.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960100db0 0x7fc96019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:26.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 --2- 172.21.15.133:0/3195128395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc960100470 0x7fc96019a170 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:26.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 -- 172.21.15.133:0/3195128395 >> 172.21.15.133:0/3195128395 conn(0x7fc9600fc280 msgr2=0x7fc9600fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:26.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 -- 172.21.15.133:0/3195128395 shutdown_connections 2024-09-11T23:36:26.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:26.580+0000 7fc9681da640 1 -- 172.21.15.133:0/3195128395 wait complete. 2024-09-11T23:36:26.935 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:26 smithi133 ceph-mon[31430]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:26 smithi138 ceph-mon[35142]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:26 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3195128395' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:36:27.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:26 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3195128395' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:36:27.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:26 smithi178 ceph-mon[35882]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:27.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:26 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3195128395' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:36:27.322 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-11T23:36:27.570 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.031+0000 7fe9564d7640 1 -- 172.21.15.133:0/3979793773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe950100b10 msgr2=0x7fe950100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.031+0000 7fe9564d7640 1 --2- 172.21.15.133:0/3979793773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe950100b10 0x7fe950100f10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe940009900 tx=0x7fe94002a410 comp rx=0 tx=0).stop 2024-09-11T23:36:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.031+0000 7fe9564d7640 1 -- 172.21.15.133:0/3979793773 shutdown_connections 2024-09-11T23:36:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.031+0000 7fe9564d7640 1 --2- 172.21.15.133:0/3979793773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9501026d0 0x7fe950108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.031+0000 7fe9564d7640 1 --2- 172.21.15.133:0/3979793773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe950101d10 0x7fe950102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.031+0000 7fe9564d7640 1 --2- 172.21.15.133:0/3979793773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe950100b10 0x7fe950100f10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.031+0000 7fe9564d7640 1 -- 172.21.15.133:0/3979793773 >> 172.21.15.133:0/3979793773 conn(0x7fe9500fc2a0 msgr2=0x7fe9500fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.031+0000 7fe9564d7640 1 -- 172.21.15.133:0/3979793773 shutdown_connections 2024-09-11T23:36:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.031+0000 7fe9564d7640 1 -- 172.21.15.133:0/3979793773 wait complete. 2024-09-11T23:36:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.031+0000 7fe9564d7640 1 Processor -- start 2024-09-11T23:36:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe9564d7640 1 -- start start 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe9564d7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe950100b10 0x7fe95006b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe9564d7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe950101d10 0x7fe95019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe9564d7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9501026d0 0x7fe95019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe9564d7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe95010c790 con 0x7fe950101d10 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe9564d7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe95010c610 con 0x7fe9501026d0 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe9564d7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe95010c910 con 0x7fe950100b10 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe94f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe950101d10 0x7fe95019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe94f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe950101d10 0x7fe95019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43864/0 (socket says 172.21.15.133:43864) 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe94f7fe640 1 -- 172.21.15.133:0/155927404 learned_addr learned my addr 172.21.15.133:0/155927404 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe94ffff640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe950100b10 0x7fe95006b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe954a74640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9501026d0 0x7fe95019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe94f7fe640 1 -- 172.21.15.133:0/155927404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe950100b10 msgr2=0x7fe95006b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe94f7fe640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe950100b10 0x7fe95006b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe94f7fe640 1 -- 172.21.15.133:0/155927404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9501026d0 msgr2=0x7fe95019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe94f7fe640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9501026d0 0x7fe95019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe94f7fe640 1 -- 172.21.15.133:0/155927404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95019faf0 con 0x7fe950101d10 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.032+0000 7fe954a74640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9501026d0 0x7fe95019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.033+0000 7fe94f7fe640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe950101d10 0x7fe95019cbe0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe94400c970 tx=0x7fe94400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.033+0000 7fe94d7fa640 1 -- 172.21.15.133:0/155927404 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944007bb0 con 0x7fe950101d10 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.033+0000 7fe9564d7640 1 -- 172.21.15.133:0/155927404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95019fde0 con 0x7fe950101d10 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.033+0000 7fe94d7fa640 1 -- 172.21.15.133:0/155927404 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe944007d50 con 0x7fe950101d10 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.033+0000 7fe94d7fa640 1 -- 172.21.15.133:0/155927404 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94400f450 con 0x7fe950101d10 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.033+0000 7fe9564d7640 1 -- 172.21.15.133:0/155927404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9501a0290 con 0x7fe950101d10 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.034+0000 7fe9564d7640 1 -- 172.21.15.133:0/155927404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe950107f60 con 0x7fe950101d10 2024-09-11T23:36:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.035+0000 7fe94d7fa640 1 -- 172.21.15.133:0/155927404 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe944005020 con 0x7fe950101d10 2024-09-11T23:36:28.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.036+0000 7fe94d7fa640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe9380a2c10 0x7fe9380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:28.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.036+0000 7fe94ffff640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe9380a2c10 0x7fe9380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:28.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.036+0000 7fe94d7fa640 1 -- 172.21.15.133:0/155927404 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fe9440cef70 con 0x7fe950101d10 2024-09-11T23:36:28.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.036+0000 7fe94ffff640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe9380a2c10 0x7fe9380a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9400314e0 tx=0x7fe940000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:28.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.040+0000 7fe94d7fa640 1 -- 172.21.15.133:0/155927404 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe944014030 con 0x7fe950101d10 2024-09-11T23:36:28.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.209+0000 7fe9564d7640 1 -- 172.21.15.133:0/155927404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe95002d970 con 0x7fe950101d10 2024-09-11T23:36:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.210+0000 7fe94d7fa640 1 -- 172.21.15.133:0/155927404 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) ==== 56+0+5254 (secure 0 0 0) 0x7fe94409be70 con 0x7fe950101d10 2024-09-11T23:36:28.210 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 81 2024-09-11T23:36:28.210 INFO:teuthology.orchestra.run.smithi133.stdout:fsid 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:36:28.210 INFO:teuthology.orchestra.run.smithi133.stdout:created 2024-09-11T23:27:21.704064+0000 2024-09-11T23:36:28.210 INFO:teuthology.orchestra.run.smithi133.stdout:modified 2024-09-11T23:35:08.840852+0000 2024-09-11T23:36:28.210 INFO:teuthology.orchestra.run.smithi133.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:crush_version 26 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:full_ratio 0.95 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:backfillfull_ratio 0.9 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:nearfull_ratio 0.85 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:require_min_compat_client luminous 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:min_compat_client luminous 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:require_osd_release squid 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:stretch_mode_enabled false 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-11T23:36:28.211 INFO:teuthology.orchestra.run.smithi133.stdout:max_osd 12 2024-09-11T23:36:28.212 INFO:teuthology.orchestra.run.smithi133.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6802/901342877,v1:172.21.15.133:6803/901342877] [v2:172.21.15.133:6804/901342877,v1:172.21.15.133:6805/901342877] exists,up 0ba73339-5c3f-4621-85af-3c3addd0583c 2024-09-11T23:36:28.212 INFO:teuthology.orchestra.run.smithi133.stdout:osd.1 up in weight 1 up_from 12 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6800/2975334926,v1:172.21.15.138:6801/2975334926] [v2:172.21.15.138:6802/2975334926,v1:172.21.15.138:6803/2975334926] exists,up 19075246-d736-48fd-b96d-68395df7eb8b 2024-09-11T23:36:28.212 INFO:teuthology.orchestra.run.smithi133.stdout:osd.2 up in weight 1 up_from 17 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6800/3161638816,v1:172.21.15.178:6801/3161638816] [v2:172.21.15.178:6802/3161638816,v1:172.21.15.178:6803/3161638816] exists,up 3ae0f14e-2263-4fd2-9848-7b23b8e2638c 2024-09-11T23:36:28.212 INFO:teuthology.orchestra.run.smithi133.stdout:osd.3 up in weight 1 up_from 23 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6810/1056435874,v1:172.21.15.133:6811/1056435874] [v2:172.21.15.133:6812/1056435874,v1:172.21.15.133:6813/1056435874] exists,up e2392e1a-af87-4af1-a362-f17ce49444bb 2024-09-11T23:36:28.212 INFO:teuthology.orchestra.run.smithi133.stdout:osd.4 up in weight 1 up_from 30 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6808/952086734,v1:172.21.15.138:6809/952086734] [v2:172.21.15.138:6810/952086734,v1:172.21.15.138:6811/952086734] exists,up d5414771-6d7e-4ea1-9b0a-50175189b9a9 2024-09-11T23:36:28.212 INFO:teuthology.orchestra.run.smithi133.stdout:osd.5 up in weight 1 up_from 34 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6808/1922026536,v1:172.21.15.178:6809/1922026536] [v2:172.21.15.178:6810/1922026536,v1:172.21.15.178:6811/1922026536] exists,up 8d1bd3f9-6661-405e-ad49-71be51a3034f 2024-09-11T23:36:28.212 INFO:teuthology.orchestra.run.smithi133.stdout:osd.6 up in weight 1 up_from 41 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6818/3287486573,v1:172.21.15.133:6819/3287486573] [v2:172.21.15.133:6820/3287486573,v1:172.21.15.133:6821/3287486573] exists,up 92c712b8-c5b7-4097-b108-1df572e8f155 2024-09-11T23:36:28.212 INFO:teuthology.orchestra.run.smithi133.stdout:osd.7 up in weight 1 up_from 46 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6816/663031854,v1:172.21.15.138:6817/663031854] [v2:172.21.15.138:6818/663031854,v1:172.21.15.138:6819/663031854] exists,up 46533b61-85a2-4c27-8fb0-140a2e34f03c 2024-09-11T23:36:28.212 INFO:teuthology.orchestra.run.smithi133.stdout:osd.8 up in weight 1 up_from 54 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6816/3469671700,v1:172.21.15.178:6817/3469671700] [v2:172.21.15.178:6818/3469671700,v1:172.21.15.178:6819/3469671700] exists,up 9e16c39c-28cf-427c-b40a-73cffe154908 2024-09-11T23:36:28.212 INFO:teuthology.orchestra.run.smithi133.stdout:osd.9 up in weight 1 up_from 61 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6826/1405512574,v1:172.21.15.133:6827/1405512574] [v2:172.21.15.133:6828/1405512574,v1:172.21.15.133:6829/1405512574] exists,up 1436e241-5439-49db-a07a-8b80f4499da2 2024-09-11T23:36:28.212 INFO:teuthology.orchestra.run.smithi133.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6824/678369034,v1:172.21.15.138:6825/678369034] [v2:172.21.15.138:6826/678369034,v1:172.21.15.138:6827/678369034] exists,up 46e606d4-591e-415d-8aae-06d311761798 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:osd.11 up in weight 1 up_from 73 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6824/3213367962,v1:172.21.15.178:6825/3213367962] [v2:172.21.15.178:6826/3213367962,v1:172.21.15.178:6827/3213367962] exists,up 652ee9fd-003e-4bc5-80db-25e7880754d3 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 2.b [8,11] 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 2.13 [2,11] 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.11 [5,9] 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.13 [4,9] 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.21 [4,8] 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.24 [4,9] 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/3985386459 expires 2024-09-12T23:28:07.759499+0000 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6800/2152407978 expires 2024-09-12T23:28:07.759499+0000 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/2629770686 expires 2024-09-12T23:28:07.759499+0000 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/294728041 expires 2024-09-12T23:28:07.759499+0000 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/1011550148 expires 2024-09-12T23:27:45.045954+0000 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6801/2152407978 expires 2024-09-12T23:28:07.759499+0000 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/67600571 expires 2024-09-12T23:27:45.045954+0000 2024-09-11T23:36:28.213 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/3318533253 expires 2024-09-12T23:27:45.045954+0000 2024-09-11T23:36:28.214 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6801/2855546700 expires 2024-09-12T23:27:45.045954+0000 2024-09-11T23:36:28.214 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6800/2855546700 expires 2024-09-12T23:27:45.045954+0000 2024-09-11T23:36:28.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.212+0000 7fe9564d7640 1 -- 172.21.15.133:0/155927404 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe9380a2c10 msgr2=0x7fe9380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.212+0000 7fe9564d7640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe9380a2c10 0x7fe9380a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9400314e0 tx=0x7fe940000bc0 comp rx=0 tx=0).stop 2024-09-11T23:36:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.212+0000 7fe9564d7640 1 -- 172.21.15.133:0/155927404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe950101d10 msgr2=0x7fe95019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.212+0000 7fe9564d7640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe950101d10 0x7fe95019cbe0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe94400c970 tx=0x7fe94400ce30 comp rx=0 tx=0).stop 2024-09-11T23:36:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.213+0000 7fe9564d7640 1 -- 172.21.15.133:0/155927404 shutdown_connections 2024-09-11T23:36:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.213+0000 7fe9564d7640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fe9380a2c10 0x7fe9380a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.213+0000 7fe9564d7640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9501026d0 0x7fe95019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.213+0000 7fe9564d7640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe950101d10 0x7fe95019cbe0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.213+0000 7fe9564d7640 1 --2- 172.21.15.133:0/155927404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe950100b10 0x7fe95006b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.213+0000 7fe9564d7640 1 -- 172.21.15.133:0/155927404 >> 172.21.15.133:0/155927404 conn(0x7fe9500fc2a0 msgr2=0x7fe950105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.213+0000 7fe9564d7640 1 -- 172.21.15.133:0/155927404 shutdown_connections 2024-09-11T23:36:28.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:28.213+0000 7fe9564d7640 1 -- 172.21.15.133:0/155927404 wait complete. 2024-09-11T23:36:28.909 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-11T23:36:28.919 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:36:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:28 smithi138 ceph-mon[35142]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:28 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/155927404' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:36:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:28 smithi133 ceph-mon[31430]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:28 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/155927404' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:36:29.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:28 smithi178 ceph-mon[35882]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:29.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:28 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/155927404' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:36:29.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.681+0000 7fbc0712b640 1 -- 172.21.15.133:0/249708563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc00100e20 msgr2=0x7fbc00101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:29.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.681+0000 7fbc0712b640 1 --2- 172.21.15.133:0/249708563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc00100e20 0x7fbc00101200 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009900 tx=0x7fbbf402a480 comp rx=0 tx=0).stop 2024-09-11T23:36:29.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.681+0000 7fbc0712b640 1 -- 172.21.15.133:0/249708563 shutdown_connections 2024-09-11T23:36:29.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.681+0000 7fbc0712b640 1 --2- 172.21.15.133:0/249708563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc00102ef0 0x7fbc0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.681+0000 7fbc0712b640 1 --2- 172.21.15.133:0/249708563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc00102570 0x7fbc001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.681+0000 7fbc0712b640 1 --2- 172.21.15.133:0/249708563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc00100e20 0x7fbc00101200 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.681+0000 7fbc0712b640 1 -- 172.21.15.133:0/249708563 >> 172.21.15.133:0/249708563 conn(0x7fbc000fc7d0 msgr2=0x7fbc000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.681+0000 7fbc0712b640 1 -- 172.21.15.133:0/249708563 shutdown_connections 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.681+0000 7fbc0712b640 1 -- 172.21.15.133:0/249708563 wait complete. 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.681+0000 7fbc0712b640 1 Processor -- start 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.681+0000 7fbc0712b640 1 -- start start 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbc0712b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc00100e20 0x7fbc000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbc0712b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc00102570 0x7fbc000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbc0712b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc00102ef0 0x7fbc000fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbc0712b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00110fa0 con 0x7fbc00102570 2024-09-11T23:36:29.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbc0712b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00110e20 con 0x7fbc00100e20 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbc0712b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00111120 con 0x7fbc00102ef0 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc00102570 0x7fbc000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc00102570 0x7fbc000fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43884/0 (socket says 172.21.15.133:43884) 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfffff640 1 -- 172.21.15.133:0/3627664908 learned_addr learned my addr 172.21.15.133:0/3627664908 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfffff640 1 -- 172.21.15.133:0/3627664908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc00102ef0 msgr2=0x7fbc000fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfffff640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc00102ef0 0x7fbc000fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfffff640 1 -- 172.21.15.133:0/3627664908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc00100e20 msgr2=0x7fbc000fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfffff640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc00100e20 0x7fbc000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfffff640 1 -- 172.21.15.133:0/3627664908 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc001a6980 con 0x7fbc00102570 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfffff640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc00102570 0x7fbc000fd1c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c920 tx=0x7fbbec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfdffb640 1 -- 172.21.15.133:0/3627664908 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec007c70 con 0x7fbc00102570 2024-09-11T23:36:29.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfdffb640 1 -- 172.21.15.133:0/3627664908 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbec007e10 con 0x7fbc00102570 2024-09-11T23:36:29.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbc0712b640 1 -- 172.21.15.133:0/3627664908 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc001a6c70 con 0x7fbc00102570 2024-09-11T23:36:29.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbbfdffb640 1 -- 172.21.15.133:0/3627664908 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec018890 con 0x7fbc00102570 2024-09-11T23:36:29.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.682+0000 7fbc0712b640 1 -- 172.21.15.133:0/3627664908 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc001a71b0 con 0x7fbc00102570 2024-09-11T23:36:29.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.684+0000 7fbc0712b640 1 -- 172.21.15.133:0/3627664908 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc00101200 con 0x7fbc00102570 2024-09-11T23:36:29.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.685+0000 7fbbfdffb640 1 -- 172.21.15.133:0/3627664908 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbbec018a70 con 0x7fbc00102570 2024-09-11T23:36:29.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.685+0000 7fbbfdffb640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf00a2c10 0x7fbbf00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:29.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.685+0000 7fbc04ec7640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf00a2c10 0x7fbbf00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:29.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.686+0000 7fbbfdffb640 1 -- 172.21.15.133:0/3627664908 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fbbec0cf2d0 con 0x7fbc00102570 2024-09-11T23:36:29.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.686+0000 7fbc04ec7640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf00a2c10 0x7fbbf00a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40096c0 tx=0x7fbbf4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:29.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.690+0000 7fbbfdffb640 1 -- 172.21.15.133:0/3627664908 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbbec014030 con 0x7fbc00102570 2024-09-11T23:36:29.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.890+0000 7fbc0712b640 1 -- 172.21.15.133:0/3627664908 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc0010a0a0 con 0x7fbc00102570 2024-09-11T23:36:29.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.891+0000 7fbbfdffb640 1 -- 172.21.15.133:0/3627664908 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fbbec00de20 con 0x7fbc00102570 2024-09-11T23:36:29.891 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:29.891 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"btime":"2024-09-11T23:35:05:451373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:35:05.451342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24472,24464,24419,24401]},"id":1}]} 2024-09-11T23:36:29.891 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-09-11T23:36:29.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.893+0000 7fbc0712b640 1 -- 172.21.15.133:0/3627664908 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf00a2c10 msgr2=0x7fbbf00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:29.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.893+0000 7fbc0712b640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf00a2c10 0x7fbbf00a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40096c0 tx=0x7fbbf4000bc0 comp rx=0 tx=0).stop 2024-09-11T23:36:29.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.893+0000 7fbc0712b640 1 -- 172.21.15.133:0/3627664908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc00102570 msgr2=0x7fbc000fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:29.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.893+0000 7fbc0712b640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc00102570 0x7fbc000fd1c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c920 tx=0x7fbbec00cde0 comp rx=0 tx=0).stop 2024-09-11T23:36:29.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.893+0000 7fbc0712b640 1 -- 172.21.15.133:0/3627664908 shutdown_connections 2024-09-11T23:36:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.893+0000 7fbc0712b640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fbbf00a2c10 0x7fbbf00a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.893+0000 7fbc0712b640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc00102ef0 0x7fbc000fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.893+0000 7fbc0712b640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc00102570 0x7fbc000fd1c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.893+0000 7fbc0712b640 1 --2- 172.21.15.133:0/3627664908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc00100e20 0x7fbc000fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.894+0000 7fbc0712b640 1 -- 172.21.15.133:0/3627664908 >> 172.21.15.133:0/3627664908 conn(0x7fbc000fc7d0 msgr2=0x7fbc0006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.894+0000 7fbc0712b640 1 -- 172.21.15.133:0/3627664908 shutdown_connections 2024-09-11T23:36:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:29.894+0000 7fbc0712b640 1 -- 172.21.15.133:0/3627664908 wait complete. 2024-09-11T23:36:30.196 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:29 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3627664908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:30.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:29 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3627664908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:30.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:29 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3627664908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:30.491 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:36:30.738 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:31.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:30 smithi133 ceph-mon[31430]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:31.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:30 smithi138 ceph-mon[35142]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:31.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.213+0000 7f8e577c7640 1 -- 172.21.15.133:0/3160828016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e50101de0 msgr2=0x7f8e501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.213+0000 7f8e577c7640 1 --2- 172.21.15.133:0/3160828016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e50101de0 0x7f8e501021c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009900 tx=0x7f8e4c02a410 comp rx=0 tx=0).stop 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.213+0000 7f8e577c7640 1 -- 172.21.15.133:0/3160828016 shutdown_connections 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.213+0000 7f8e577c7640 1 --2- 172.21.15.133:0/3160828016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e50106f10 0x7f8e5010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.213+0000 7f8e577c7640 1 --2- 172.21.15.133:0/3160828016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e50102790 0x7f8e501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.213+0000 7f8e577c7640 1 --2- 172.21.15.133:0/3160828016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e50101de0 0x7f8e501021c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.213+0000 7f8e577c7640 1 -- 172.21.15.133:0/3160828016 >> 172.21.15.133:0/3160828016 conn(0x7f8e500fd8c0 msgr2=0x7f8e500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.213+0000 7f8e577c7640 1 -- 172.21.15.133:0/3160828016 shutdown_connections 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.213+0000 7f8e577c7640 1 -- 172.21.15.133:0/3160828016 wait complete. 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e577c7640 1 Processor -- start 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e577c7640 1 -- start start 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e577c7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e50101de0 0x7f8e50198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e577c7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e50102790 0x7f8e50198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e577c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e50106f10 0x7f8e5019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e577c7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50111f70 con 0x7f8e50106f10 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e577c7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50111df0 con 0x7f8e50101de0 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e577c7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501120f0 con 0x7f8e50102790 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e55d64640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e50106f10 0x7f8e5019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e55d64640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e50106f10 0x7f8e5019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43908/0 (socket says 172.21.15.133:43908) 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e55d64640 1 -- 172.21.15.133:0/1396826274 learned_addr learned my addr 172.21.15.133:0/1396826274 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e55563640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e50101de0 0x7f8e50198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e54d62640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e50102790 0x7f8e50198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e55d64640 1 -- 172.21.15.133:0/1396826274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e50102790 msgr2=0x7f8e50198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e55d64640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e50102790 0x7f8e50198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:31.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e55d64640 1 -- 172.21.15.133:0/1396826274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e50101de0 msgr2=0x7f8e50198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e55d64640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e50101de0 0x7f8e50198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e55d64640 1 -- 172.21.15.133:0/1396826274 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5019d3e0 con 0x7f8e50106f10 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.214+0000 7f8e54d62640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e50102790 0x7f8e50198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.215+0000 7f8e55d64640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e50106f10 0x7f8e5019cd00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800cad0 tx=0x7f8e4800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.215+0000 7f8e467fc640 1 -- 172.21.15.133:0/1396826274 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4800ae20 con 0x7f8e50106f10 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.215+0000 7f8e467fc640 1 -- 172.21.15.133:0/1396826274 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e48005020 con 0x7f8e50106f10 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.215+0000 7f8e577c7640 1 -- 172.21.15.133:0/1396826274 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5019d6d0 con 0x7f8e50106f10 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.215+0000 7f8e467fc640 1 -- 172.21.15.133:0/1396826274 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4800f6f0 con 0x7f8e50106f10 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.215+0000 7f8e577c7640 1 -- 172.21.15.133:0/1396826274 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e501a4fb0 con 0x7f8e50106f10 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.216+0000 7f8e577c7640 1 -- 172.21.15.133:0/1396826274 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e501021c0 con 0x7f8e50106f10 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.217+0000 7f8e467fc640 1 -- 172.21.15.133:0/1396826274 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8e480051d0 con 0x7f8e50106f10 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.217+0000 7f8e467fc640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8e3c0a2c10 0x7f8e3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.218+0000 7f8e55563640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8e3c0a2c10 0x7f8e3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:31.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.218+0000 7f8e467fc640 1 -- 172.21.15.133:0/1396826274 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f8e480cf3b0 con 0x7f8e50106f10 2024-09-11T23:36:31.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.218+0000 7f8e55563640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8e3c0a2c10 0x7f8e3c0a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009230 tx=0x7f8e4c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:31.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.222+0000 7f8e467fc640 1 -- 172.21.15.133:0/1396826274 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e48014030 con 0x7f8e50106f10 2024-09-11T23:36:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:30 smithi178 ceph-mon[35882]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-11T23:36:31.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.421+0000 7f8e577c7640 1 -- 172.21.15.133:0/1396826274 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e50109b60 con 0x7f8e50106f10 2024-09-11T23:36:31.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.422+0000 7f8e467fc640 1 -- 172.21.15.133:0/1396826274 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f8e4809c2b0 con 0x7f8e50106f10 2024-09-11T23:36:31.421 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:31.421 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"btime":"2024-09-11T23:35:05:451373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:35:05.451342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24481,24472,24464,24419,24401]},"id":1}]} 2024-09-11T23:36:31.422 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-09-11T23:36:31.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 -- 172.21.15.133:0/1396826274 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8e3c0a2c10 msgr2=0x7f8e3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:31.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8e3c0a2c10 0x7f8e3c0a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009230 tx=0x7f8e4c0058d0 comp rx=0 tx=0).stop 2024-09-11T23:36:31.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 -- 172.21.15.133:0/1396826274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e50106f10 msgr2=0x7f8e5019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:31.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e50106f10 0x7f8e5019cd00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800cad0 tx=0x7f8e4800cf90 comp rx=0 tx=0).stop 2024-09-11T23:36:31.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 -- 172.21.15.133:0/1396826274 shutdown_connections 2024-09-11T23:36:31.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f8e3c0a2c10 0x7f8e3c0a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:31.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e50106f10 0x7f8e5019cd00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:31.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e50102790 0x7f8e50198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:31.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 --2- 172.21.15.133:0/1396826274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e50101de0 0x7f8e50198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:31.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 -- 172.21.15.133:0/1396826274 >> 172.21.15.133:0/1396826274 conn(0x7f8e500fd8c0 msgr2=0x7f8e500fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:31.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 -- 172.21.15.133:0/1396826274 shutdown_connections 2024-09-11T23:36:31.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:31.424+0000 7f8e577c7640 1 -- 172.21.15.133:0/1396826274 wait complete. 2024-09-11T23:36:31.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:31 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1396826274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:32.118 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:36:32.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:31 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1396826274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:32.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:31 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1396826274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:32.368 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.832+0000 7fa33b407640 1 -- 172.21.15.133:0/2624413760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa334106f50 msgr2=0x7fa33410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.832+0000 7fa33b407640 1 --2- 172.21.15.133:0/2624413760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa334106f50 0x7fa33410eaf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009cd0 tx=0x7fa32c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.832+0000 7fa33b407640 1 -- 172.21.15.133:0/2624413760 shutdown_connections 2024-09-11T23:36:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.832+0000 7fa33b407640 1 --2- 172.21.15.133:0/2624413760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa334106f50 0x7fa33410eaf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.832+0000 7fa33b407640 1 --2- 172.21.15.133:0/2624413760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa334100ca0 0x7fa334106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.832+0000 7fa33b407640 1 --2- 172.21.15.133:0/2624413760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3341002f0 0x7fa3341006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.832+0000 7fa33b407640 1 -- 172.21.15.133:0/2624413760 >> 172.21.15.133:0/2624413760 conn(0x7fa334077210 msgr2=0x7fa3340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.832+0000 7fa33b407640 1 -- 172.21.15.133:0/2624413760 shutdown_connections 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.832+0000 7fa33b407640 1 -- 172.21.15.133:0/2624413760 wait complete. 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.833+0000 7fa33b407640 1 Processor -- start 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.833+0000 7fa33b407640 1 -- start start 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.833+0000 7fa33b407640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3341002f0 0x7fa3341985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.833+0000 7fa33b407640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa334100ca0 0x7fa334198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.833+0000 7fa33b407640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa334106f50 0x7fa33419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.833+0000 7fa33b407640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa334112080 con 0x7fa334100ca0 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.833+0000 7fa33b407640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa334111f00 con 0x7fa3341002f0 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.833+0000 7fa33b407640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa334112200 con 0x7fa334106f50 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa3389a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa334100ca0 0x7fa334198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa3389a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa334100ca0 0x7fa334198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39080/0 (socket says 172.21.15.133:39080) 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa3389a2640 1 -- 172.21.15.133:0/877550630 learned_addr learned my addr 172.21.15.133:0/877550630 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa3391a3640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3341002f0 0x7fa3341985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa3399a4640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa334106f50 0x7fa33419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa3389a2640 1 -- 172.21.15.133:0/877550630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa334106f50 msgr2=0x7fa33419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa3389a2640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa334106f50 0x7fa33419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa3389a2640 1 -- 172.21.15.133:0/877550630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3341002f0 msgr2=0x7fa3341985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa3389a2640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3341002f0 0x7fa3341985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa3389a2640 1 -- 172.21.15.133:0/877550630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33419d590 con 0x7fa334100ca0 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa3389a2640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa334100ca0 0x7fa334198b20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa32400b450 tx=0x7fa32400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa32a7fc640 1 -- 172.21.15.133:0/877550630 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324015020 con 0x7fa334100ca0 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa33b407640 1 -- 172.21.15.133:0/877550630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33419d880 con 0x7fa334100ca0 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa32a7fc640 1 -- 172.21.15.133:0/877550630 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa324011d70 con 0x7fa334100ca0 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa32a7fc640 1 -- 172.21.15.133:0/877550630 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32400f340 con 0x7fa334100ca0 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.834+0000 7fa33b407640 1 -- 172.21.15.133:0/877550630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3341a5160 con 0x7fa334100ca0 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.835+0000 7fa33b407640 1 -- 172.21.15.133:0/877550630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa334100750 con 0x7fa334100ca0 2024-09-11T23:36:32.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.837+0000 7fa32a7fc640 1 -- 172.21.15.133:0/877550630 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa324018020 con 0x7fa334100ca0 2024-09-11T23:36:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.837+0000 7fa32a7fc640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa3200a2c10 0x7fa3200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.837+0000 7fa3391a3640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa3200a2c10 0x7fa3200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.837+0000 7fa32a7fc640 1 -- 172.21.15.133:0/877550630 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fa3240cf050 con 0x7fa334100ca0 2024-09-11T23:36:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.838+0000 7fa3391a3640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa3200a2c10 0x7fa3200a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa330009890 tx=0x7fa330008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:32.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:32.840+0000 7fa32a7fc640 1 -- 172.21.15.133:0/877550630 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa32401f020 con 0x7fa334100ca0 2024-09-11T23:36:33.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.012+0000 7fa33b407640 1 -- 172.21.15.133:0/877550630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa334199940 con 0x7fa334100ca0 2024-09-11T23:36:33.012 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:32 smithi133 ceph-mon[31430]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:33.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.014+0000 7fa32a7fc640 1 -- 172.21.15.133:0/877550630 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22677 (secure 0 0 0) 0x7fa334199940 con 0x7fa334100ca0 2024-09-11T23:36:33.013 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:33.014 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":81,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:35:08.840852+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:36:33.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 -- 172.21.15.133:0/877550630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa3200a2c10 msgr2=0x7fa3200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:33.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa3200a2c10 0x7fa3200a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa330009890 tx=0x7fa330008040 comp rx=0 tx=0).stop 2024-09-11T23:36:33.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 -- 172.21.15.133:0/877550630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa334100ca0 msgr2=0x7fa334198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:33.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa334100ca0 0x7fa334198b20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa32400b450 tx=0x7fa32400b910 comp rx=0 tx=0).stop 2024-09-11T23:36:33.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 -- 172.21.15.133:0/877550630 shutdown_connections 2024-09-11T23:36:33.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fa3200a2c10 0x7fa3200a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:33.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa334106f50 0x7fa33419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:33.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa334100ca0 0x7fa334198b20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:33.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 --2- 172.21.15.133:0/877550630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3341002f0 0x7fa3341985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:33.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 -- 172.21.15.133:0/877550630 >> 172.21.15.133:0/877550630 conn(0x7fa334077210 msgr2=0x7fa3341051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:33.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 -- 172.21.15.133:0/877550630 shutdown_connections 2024-09-11T23:36:33.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:33.016+0000 7fa33b407640 1 -- 172.21.15.133:0/877550630 wait complete. 2024-09-11T23:36:33.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:32 smithi138 ceph-mon[35142]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:32 smithi178 ceph-mon[35882]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:33.843 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:34:53.462890+0000', 'modified': '2024-09-11T23:35:05.451342+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24481, 'mds_1': 24472, 'mds_2': 24464, 'mds_3': 24419, 'mds_4': 24401}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6839/570951791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 570951791}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 570951791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6841/1294883246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6840', 'nonce': 1294883246}, {'type': 'v1', 'addr': '172.21.15.133:6841', 'nonce': 1294883246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.138:6833/1969946111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1969946111}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1969946111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.138:6835/137089847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 137089847}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 137089847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/3569562128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 3569562128}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 3569562128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6837/1115908278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 1115908278}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 1115908278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/1681049778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 1681049778}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 1681049778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.138:6839/2407895128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 2407895128}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 2407895128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.178:6837/1040997001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 1040997001}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 1040997001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.178:6839/3794667907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 3794667907}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 3794667907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24481, 'qdb_cluster': [24481, 24472, 24464, 24419, 24401]} 2024-09-11T23:36:33.844 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T23:36:33.844 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-11T23:36:34.094 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:34.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:33 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/877550630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:34.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:33 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/877550630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:34.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:33 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/877550630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.613+0000 7f23918af640 1 -- 172.21.15.133:0/1156549286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c100a80 msgr2=0x7f238c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.613+0000 7f23918af640 1 --2- 172.21.15.133:0/1156549286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c100a80 0x7f238c100ee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2380009930 tx=0x7f238002a420 comp rx=0 tx=0).stop 2024-09-11T23:36:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.613+0000 7f23918af640 1 -- 172.21.15.133:0/1156549286 shutdown_connections 2024-09-11T23:36:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.613+0000 7f23918af640 1 --2- 172.21.15.133:0/1156549286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f238c101420 0x7f238c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.613+0000 7f23918af640 1 --2- 172.21.15.133:0/1156549286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c100a80 0x7f238c100ee0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.613+0000 7f23918af640 1 --2- 172.21.15.133:0/1156549286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f238c106a80 0x7f238c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.614+0000 7f23918af640 1 -- 172.21.15.133:0/1156549286 >> 172.21.15.133:0/1156549286 conn(0x7f238c0fc7a0 msgr2=0x7f238c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.614+0000 7f23918af640 1 -- 172.21.15.133:0/1156549286 shutdown_connections 2024-09-11T23:36:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.614+0000 7f23918af640 1 -- 172.21.15.133:0/1156549286 wait complete. 2024-09-11T23:36:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.614+0000 7f23918af640 1 Processor -- start 2024-09-11T23:36:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.614+0000 7f23918af640 1 -- start start 2024-09-11T23:36:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.614+0000 7f23918af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f238c100a80 0x7f238c1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.614+0000 7f23918af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c101420 0x7f238c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.614+0000 7f23918af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f238c106a80 0x7f238c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.614+0000 7f23918af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f238c110f20 con 0x7f238c101420 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f23918af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f238c110da0 con 0x7f238c106a80 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f23918af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f238c1110a0 con 0x7f238c100a80 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c101420 0x7f238c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c101420 0x7f238c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39098/0 (socket says 172.21.15.133:39098) 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238a7fc640 1 -- 172.21.15.133:0/652815972 learned_addr learned my addr 172.21.15.133:0/652815972 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238b7fe640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f238c106a80 0x7f238c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238affd640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f238c100a80 0x7f238c1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238a7fc640 1 -- 172.21.15.133:0/652815972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f238c100a80 msgr2=0x7f238c1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238a7fc640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f238c100a80 0x7f238c1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238a7fc640 1 -- 172.21.15.133:0/652815972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f238c106a80 msgr2=0x7f238c19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238a7fc640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f238c106a80 0x7f238c19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238a7fc640 1 -- 172.21.15.133:0/652815972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c19c4a0 con 0x7f238c101420 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238affd640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f238c100a80 0x7f238c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f238a7fc640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c101420 0x7f238c197a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2380009a50 tx=0x7f2380004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f23908ad640 1 -- 172.21.15.133:0/652815972 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238003f030 con 0x7f238c101420 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f23908ad640 1 -- 172.21.15.133:0/652815972 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2380003ce0 con 0x7f238c101420 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f23918af640 1 -- 172.21.15.133:0/652815972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238c19c730 con 0x7f238c101420 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f23908ad640 1 -- 172.21.15.133:0/652815972 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380035070 con 0x7f238c101420 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.615+0000 7f23918af640 1 -- 172.21.15.133:0/652815972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238c1a4070 con 0x7f238c101420 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f23908ad640 1 -- 172.21.15.133:0/652815972 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2380042030 con 0x7f238c101420 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f2393c88740 1 Processor -- start 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f2393c88740 1 -- start start 2024-09-11T23:36:34.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f2393c88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f5f850 0x55c529f5fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f2393c88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f6c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f2393c88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 0x55c529f6f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f2393c88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c529ec6090 con 0x55c529f60170 2024-09-11T23:36:34.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f2393c88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c529e6da70 con 0x55c529f5f850 2024-09-11T23:36:34.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f2393c88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c529ec2050 con 0x55c529f6cdc0 2024-09-11T23:36:34.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f23908ad640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f23740a2c10 0x7f23740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f238affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f6c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f238affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f6c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39100/0 (socket says 172.21.15.133:39100) 2024-09-11T23:36:34.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.618+0000 7f238affd640 1 -- 172.21.15.133:0/2835455051 learned_addr learned my addr 172.21.15.133:0/2835455051 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:34.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f238b7fe640 1 --2- 172.21.15.133:0/2835455051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f5f850 0x55c529f5fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f238a7fc640 1 --2- 172.21.15.133:0/2835455051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 0x55c529f6f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f238b7fe640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f23740a2c10 0x7f23740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f23908ad640 1 -- 172.21.15.133:0/652815972 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f23800ef720 con 0x7f238c101420 2024-09-11T23:36:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f238affd640 1 -- 172.21.15.133:0/2835455051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 msgr2=0x55c529f6f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f238affd640 1 --2- 172.21.15.133:0/2835455051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 0x55c529f6f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f238affd640 1 -- 172.21.15.133:0/2835455051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f5f850 msgr2=0x55c529f5fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f238affd640 1 --2- 172.21.15.133:0/2835455051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f5f850 0x55c529f5fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f238affd640 1 -- 172.21.15.133:0/2835455051 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c529e7a130 con 0x55c529f60170 2024-09-11T23:36:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f238a7fc640 1 --2- 172.21.15.133:0/2835455051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 0x55c529f6f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f238affd640 1 --2- 172.21.15.133:0/2835455051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f6c880 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f237c00aa10 tx=0x7f237c00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb225292a82fe4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f238b7fe640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f23740a2c10 0x7f23740a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f23840070b0 tx=0x7f23840093d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f2371ffb640 1 -- 172.21.15.133:0/2835455051 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c015070 con 0x55c529f60170 2024-09-11T23:36:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f2371ffb640 1 -- 172.21.15.133:0/2835455051 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f237c00cc60 con 0x55c529f60170 2024-09-11T23:36:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.619+0000 7f2371ffb640 1 -- 172.21.15.133:0/2835455051 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c018690 con 0x55c529f60170 2024-09-11T23:36:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.620+0000 7f2393c88740 1 -- 172.21.15.133:0/2835455051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 msgr2=0x55c529f6c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.620+0000 7f2393c88740 1 --2- 172.21.15.133:0/2835455051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f6c880 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f237c00aa10 tx=0x7f237c00aed0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.620+0000 7f2393c88740 1 -- 172.21.15.133:0/2835455051 shutdown_connections 2024-09-11T23:36:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.620+0000 7f2393c88740 1 --2- 172.21.15.133:0/2835455051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 0x55c529f6f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.620+0000 7f2393c88740 1 --2- 172.21.15.133:0/2835455051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f6c880 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.620+0000 7f2393c88740 1 --2- 172.21.15.133:0/2835455051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f5f850 0x55c529f5fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.620+0000 7f2393c88740 1 -- 172.21.15.133:0/2835455051 >> 172.21.15.133:0/2835455051 conn(0x55c529f5bf60 msgr2=0x55c529f5c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:34.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.620+0000 7f2393c88740 1 -- 172.21.15.133:0/2835455051 shutdown_connections 2024-09-11T23:36:34.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.620+0000 7f2393c88740 1 -- 172.21.15.133:0/2835455051 wait complete. 2024-09-11T23:36:34.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.620+0000 7f2393c88740 1 -- start start 2024-09-11T23:36:34.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.620+0000 7f2393c88740 1 Processor -- start 2024-09-11T23:36:34.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f2393c88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f6c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f2393c88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 0x55c529f69980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f2393c88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f6cdc0 0x55c529f69ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f2393c88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c529f6f9f0 con 0x55c529f5f850 2024-09-11T23:36:34.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f2393c88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c529e80f20 con 0x55c529f6cdc0 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f2393c88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c529f6fb70 con 0x55c529f60170 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f2393c88740 10 client.0 _ll_register_callbacks cb 0x55c529e4a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f238affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f6c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f238affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f6c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39104/0 (socket says 172.21.15.133:39104) 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f238affd640 1 -- 172.21.15.133:0/2771766208 learned_addr learned my addr 172.21.15.133:0/2771766208 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f238a7fc640 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 0x55c529f69980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f238affd640 1 -- 172.21.15.133:0/2771766208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 msgr2=0x55c529f69980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f238affd640 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 0x55c529f69980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f238affd640 1 -- 172.21.15.133:0/2771766208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f6cdc0 msgr2=0x55c529f69ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f238affd640 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f6cdc0 0x55c529f69ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f238affd640 1 -- 172.21.15.133:0/2771766208 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c529c372b0 con 0x55c529f5f850 2024-09-11T23:36:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f238a7fc640 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 0x55c529f69980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f238affd640 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f6c630 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f237c00c520 tx=0x7f237c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a8a37b9669ccf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f236affd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f236affd640 1 -- 172.21.15.133:0/2771766208 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c015040 con 0x55c529f5f850 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f236affd640 1 -- 172.21.15.133:0/2771766208 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f237c01a660 con 0x55c529f5f850 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.621+0000 7f2393c88740 1 -- 172.21.15.133:0/2771766208 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f237c00cc60 con 0x55c529f5f850 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f236affd640 1 -- 172.21.15.133:0/2771766208 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f237c00ce10 con 0x55c529f5f850 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f2393c88740 10 client.15153 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f2393c88740 1 -- 172.21.15.133:0/2771766208 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c529f7b320 con 0x55c529f5f850 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f236affd640 1 -- 172.21.15.133:0/2771766208 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f237c00cc60 con 0x55c529f5f850 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f236affd640 1 -- 172.21.15.133:0/2771766208 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f237c025930 con 0x55c529f5f850 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f2393c88740 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f2393c88740 10 client.15153 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f2393c88740 20 client.15153 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f2393c88740 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55c529f7d590 0x55c529f7ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f2393c88740 4 client.15153 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f2393c88740 1 -- 172.21.15.133:0/2771766208 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c529ed3600 con 0x55c529f7d590 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.622+0000 7f238a7fc640 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55c529f7d590 0x55c529f7ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.623+0000 7f236affd640 10 client.15153 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.623+0000 7f238a7fc640 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55c529f7d590 0x55c529f7ea00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad56365a38be8301 server_cookie=d31ab6a2b1315b80 in_seq=0 out_seq=0 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.623+0000 7f236affd640 1 -- 172.21.15.133:0/2771766208 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c529ed3600 con 0x55c529f7d590 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.623+0000 7f236affd640 10 client.15153 handle_command_reply: tid=0 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.623+0000 7f2393c88740 1 client.15153 shutdown 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f2393c88740 1 -- 172.21.15.133:0/2771766208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 msgr2=0x55c529f6c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f2393c88740 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f6c630 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f237c00c520 tx=0x7f237c011ba0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f2393c88740 1 -- 172.21.15.133:0/2771766208 shutdown_connections 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f2393c88740 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55c529f7d590 0x55c529f7ea00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f2393c88740 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f6cdc0 0x55c529f69ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f2393c88740 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 0x55c529f69980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f2393c88740 1 --2- 172.21.15.133:0/2771766208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f6c630 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f236affd640 0 client.15153 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:36:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f2393c88740 1 -- 172.21.15.133:0/2771766208 >> 172.21.15.133:0/2771766208 conn(0x55c529f5bf60 msgr2=0x55c529f61c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f2393c88740 1 -- 172.21.15.133:0/2771766208 shutdown_connections 2024-09-11T23:36:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f2393c88740 1 -- 172.21.15.133:0/2771766208 wait complete. 2024-09-11T23:36:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.624+0000 7f2393c88740 20 client.15153 trim_cache size 0 max 16384 2024-09-11T23:36:34.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.641+0000 7f2393c88740 1 Processor -- start 2024-09-11T23:36:34.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.641+0000 7f2393c88740 1 -- start start 2024-09-11T23:36:34.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.641+0000 7f2393c88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f7ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.641+0000 7f2393c88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 0x55c529f66be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.641+0000 7f2393c88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f6cdc0 0x55c529f67120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.641+0000 7f2393c88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c529f7cd40 con 0x55c529f5f850 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.641+0000 7f2393c88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c529f7d410 con 0x55c529f6cdc0 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.641+0000 7f2393c88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c529f7dae0 con 0x55c529f60170 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.641+0000 7f238affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f7ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.641+0000 7f238affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f7ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39108/0 (socket says 172.21.15.133:39108) 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.641+0000 7f238affd640 1 -- 172.21.15.133:0/948146528 learned_addr learned my addr 172.21.15.133:0/948146528 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f238a7fc640 1 --2- 172.21.15.133:0/948146528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 0x55c529f66be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f238b7fe640 1 --2- 172.21.15.133:0/948146528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f6cdc0 0x55c529f67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f238affd640 1 -- 172.21.15.133:0/948146528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 msgr2=0x55c529f66be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f238affd640 1 --2- 172.21.15.133:0/948146528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 0x55c529f66be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f238affd640 1 -- 172.21.15.133:0/948146528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f6cdc0 msgr2=0x55c529f67120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f238affd640 1 --2- 172.21.15.133:0/948146528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f6cdc0 0x55c529f67120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f238affd640 1 -- 172.21.15.133:0/948146528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c529f6c9a0 con 0x55c529f5f850 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f238b7fe640 1 --2- 172.21.15.133:0/948146528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f6cdc0 0x55c529f67120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f238a7fc640 1 --2- 172.21.15.133:0/948146528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 0x55c529f66be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f238affd640 1 --2- 172.21.15.133:0/948146528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f7ea00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f237c025f10 tx=0x7f237c00a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b204f4b3feca6ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f23717fa640 1 -- 172.21.15.133:0/948146528 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c015050 con 0x55c529f5f850 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f23717fa640 1 -- 172.21.15.133:0/948146528 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f237c00a5e0 con 0x55c529f5f850 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.642+0000 7f23717fa640 1 -- 172.21.15.133:0/948146528 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c0309a0 con 0x55c529f5f850 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 -- 172.21.15.133:0/948146528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 msgr2=0x55c529f7ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 --2- 172.21.15.133:0/948146528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f7ea00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f237c025f10 tx=0x7f237c00a210 comp rx=0 tx=0).stop 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 -- 172.21.15.133:0/948146528 shutdown_connections 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 --2- 172.21.15.133:0/948146528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f6cdc0 0x55c529f67120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 --2- 172.21.15.133:0/948146528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f60170 0x55c529f66be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 --2- 172.21.15.133:0/948146528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f5f850 0x55c529f7ea00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 -- 172.21.15.133:0/948146528 >> 172.21.15.133:0/948146528 conn(0x55c529f5bf60 msgr2=0x55c529f65040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 -- 172.21.15.133:0/948146528 shutdown_connections 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 -- 172.21.15.133:0/948146528 wait complete. 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 -- start start 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 Processor -- start 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.643+0000 7f2393c88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f5f850 0x55c529f87370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f2393c88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f80770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f2393c88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 0x55c529f80cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f2393c88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c529f7e880 con 0x55c529f60170 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f2393c88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c529f7e1b0 con 0x55c529f5f850 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f2393c88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c529f7ef50 con 0x55c529f6cdc0 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f2393c88740 10 client.0 _ll_register_callbacks cb 0x7f2384009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f80770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f80770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39116/0 (socket says 172.21.15.133:39116) 2024-09-11T23:36:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238a7fc640 1 -- 172.21.15.133:0/571942432 learned_addr learned my addr 172.21.15.133:0/571942432 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238b7fe640 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 0x55c529f80cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238affd640 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f5f850 0x55c529f87370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238a7fc640 1 -- 172.21.15.133:0/571942432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 msgr2=0x55c529f80cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238a7fc640 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 0x55c529f80cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238a7fc640 1 -- 172.21.15.133:0/571942432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f5f850 msgr2=0x55c529f87370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238a7fc640 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f5f850 0x55c529f87370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238a7fc640 1 -- 172.21.15.133:0/571942432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c529f814b0 con 0x55c529f60170 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238b7fe640 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 0x55c529f80cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238affd640 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f5f850 0x55c529f87370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f238a7fc640 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f80770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2380009900 tx=0x7f238008bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cea9a0968c82cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f236affd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f236affd640 1 -- 172.21.15.133:0/571942432 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380049040 con 0x55c529f60170 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f236affd640 1 -- 172.21.15.133:0/571942432 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f238003b460 con 0x55c529f60170 2024-09-11T23:36:34.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f2393c88740 1 -- 172.21.15.133:0/571942432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f237c00a5e0 con 0x55c529f60170 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f236affd640 1 -- 172.21.15.133:0/571942432 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f23800bb880 con 0x55c529f60170 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f2393c88740 10 client.15159 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.644+0000 7f2393c88740 1 -- 172.21.15.133:0/571942432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c529fe7ca0 con 0x55c529f60170 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.645+0000 7f236affd640 1 -- 172.21.15.133:0/571942432 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f237c00a5e0 con 0x55c529f60170 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.645+0000 7f236affd640 1 -- 172.21.15.133:0/571942432 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f23800bc180 con 0x55c529f60170 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.645+0000 7f2393c88740 10 client.15159 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.645+0000 7f2393c88740 10 client.15159 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.645+0000 7f2393c88740 20 client.15159 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.645+0000 7f2393c88740 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55c529fec800 0x55c529feb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.645+0000 7f2393c88740 4 client.15159 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.645+0000 7f2393c88740 1 -- 172.21.15.133:0/571942432 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "status"}) -- 0x55c529ecdb10 con 0x55c529fec800 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.645+0000 7f238affd640 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55c529fec800 0x55c529feb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.645+0000 7f236affd640 10 client.15159 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:36:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.645+0000 7f238affd640 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55c529fec800 0x55c529feb510 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6de95713ad03df1f server_cookie=659cd43a1ff0b73d in_seq=0 out_seq=0 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f236affd640 1 -- 172.21.15.133:0/571942432 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c529ecdb10 con 0x55c529fec800 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f236affd640 10 client.15159 handle_command_reply: tid=0 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 1 client.15159 shutdown 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 1 -- 172.21.15.133:0/571942432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 msgr2=0x55c529f80770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f80770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2380009900 tx=0x7f238008bdd0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 1 -- 172.21.15.133:0/571942432 shutdown_connections 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55c529fec800 0x55c529feb510 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f236affd640 0 client.15159 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c529f6cdc0 0x55c529f80cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c529f60170 0x55c529f80770 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 1 --2- 172.21.15.133:0/571942432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c529f5f850 0x55c529f87370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 1 -- 172.21.15.133:0/571942432 >> 172.21.15.133:0/571942432 conn(0x55c529f5bf60 msgr2=0x55c529f5a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 1 -- 172.21.15.133:0/571942432 shutdown_connections 2024-09-11T23:36:34.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 1 -- 172.21.15.133:0/571942432 wait complete. 2024-09-11T23:36:34.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f2393c88740 20 client.15159 trim_cache size 0 max 16384 2024-09-11T23:36:34.653 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:34.653 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:36:34.653 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 4, 2024-09-11T23:36:34.653 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24401, 2024-09-11T23:36:34.653 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:36:34.653 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:36:34.653 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:36:34.653 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 90.197798262000006, 2024-09-11T23:36:34.653 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 27, 2024-09-11T23:36:34.654 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 81, 2024-09-11T23:36:34.654 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 79, 2024-09-11T23:36:34.654 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 139.33499429599999 2024-09-11T23:36:34.654 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:34.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f23918af640 1 -- 172.21.15.133:0/652815972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f23740a2c10 msgr2=0x7f23740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:34.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f23918af640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f23740a2c10 0x7f23740a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f23840070b0 tx=0x7f23840093d0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f23918af640 1 -- 172.21.15.133:0/652815972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c101420 msgr2=0x7f238c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:34.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f23918af640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c101420 0x7f238c197a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2380009a50 tx=0x7f2380004270 comp rx=0 tx=0).stop 2024-09-11T23:36:34.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f23918af640 1 -- 172.21.15.133:0/652815972 shutdown_connections 2024-09-11T23:36:34.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f23918af640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f23740a2c10 0x7f23740a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f23918af640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f238c106a80 0x7f238c19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f23918af640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c101420 0x7f238c197a30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f23918af640 1 --2- 172.21.15.133:0/652815972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f238c100a80 0x7f238c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:34.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.646+0000 7f23918af640 1 -- 172.21.15.133:0/652815972 >> 172.21.15.133:0/652815972 conn(0x7f238c0fc7a0 msgr2=0x7f238c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:34.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.647+0000 7f23918af640 1 -- 172.21.15.133:0/652815972 shutdown_connections 2024-09-11T23:36:34.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:34.647+0000 7f23918af640 1 -- 172.21.15.133:0/652815972 wait complete. 2024-09-11T23:36:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:34 smithi133 ceph-mon[31430]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:35.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:34 smithi178 ceph-mon[35882]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:35.426 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T23:36:35.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:34 smithi138 ceph-mon[35142]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.165+0000 7fb0629d3640 1 -- 172.21.15.133:0/168783367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb05c101420 msgr2=0x7fb05c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.165+0000 7fb0629d3640 1 --2- 172.21.15.133:0/168783367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb05c101420 0x7fb05c10d950 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb054009cd0 tx=0x7fb05402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.165+0000 7fb0629d3640 1 -- 172.21.15.133:0/168783367 shutdown_connections 2024-09-11T23:36:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.165+0000 7fb0629d3640 1 --2- 172.21.15.133:0/168783367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb05c101420 0x7fb05c10d950 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.165+0000 7fb0629d3640 1 --2- 172.21.15.133:0/168783367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb05c100a80 0x7fb05c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.165+0000 7fb0629d3640 1 --2- 172.21.15.133:0/168783367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb05c106a80 0x7fb05c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.165+0000 7fb0629d3640 1 -- 172.21.15.133:0/168783367 >> 172.21.15.133:0/168783367 conn(0x7fb05c0fc7a0 msgr2=0x7fb05c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.165+0000 7fb0629d3640 1 -- 172.21.15.133:0/168783367 shutdown_connections 2024-09-11T23:36:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.165+0000 7fb0629d3640 1 -- 172.21.15.133:0/168783367 wait complete. 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb0629d3640 1 Processor -- start 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb0629d3640 1 -- start start 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb0629d3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb05c100a80 0x7fb05c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb0629d3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb05c101420 0x7fb05c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb0629d3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb05c106a80 0x7fb05c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb0629d3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c110f20 con 0x7fb05c101420 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb0629d3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c110da0 con 0x7fb05c100a80 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb0629d3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c1110a0 con 0x7fb05c106a80 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb053fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb05c101420 0x7fb05c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb053fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb05c101420 0x7fb05c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39134/0 (socket says 172.21.15.133:39134) 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb053fff640 1 -- 172.21.15.133:0/1819228582 learned_addr learned my addr 172.21.15.133:0/1819228582 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb060f70640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb05c106a80 0x7fb05c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb05bfff640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb05c100a80 0x7fb05c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb053fff640 1 -- 172.21.15.133:0/1819228582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb05c106a80 msgr2=0x7fb05c19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb053fff640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb05c106a80 0x7fb05c19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb053fff640 1 -- 172.21.15.133:0/1819228582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb05c100a80 msgr2=0x7fb05c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb053fff640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb05c100a80 0x7fb05c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb053fff640 1 -- 172.21.15.133:0/1819228582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c19d530 con 0x7fb05c101420 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb05bfff640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb05c100a80 0x7fb05c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb053fff640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb05c101420 0x7fb05c198ac0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb04400b4a0 tx=0x7fb04400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.166+0000 7fb059ffb640 1 -- 172.21.15.133:0/1819228582 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044014070 con 0x7fb05c101420 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.167+0000 7fb059ffb640 1 -- 172.21.15.133:0/1819228582 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb044010af0 con 0x7fb05c101420 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.167+0000 7fb059ffb640 1 -- 172.21.15.133:0/1819228582 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044017d10 con 0x7fb05c101420 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.167+0000 7fb0629d3640 1 -- 172.21.15.133:0/1819228582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05c19d820 con 0x7fb05c101420 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.167+0000 7fb0629d3640 1 -- 172.21.15.133:0/1819228582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb05c1a5100 con 0x7fb05c101420 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.169+0000 7fb059ffb640 1 -- 172.21.15.133:0/1819228582 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb044017070 con 0x7fb05c101420 2024-09-11T23:36:36.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.169+0000 7fb064dac740 1 Processor -- start 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.169+0000 7fb064dac740 1 -- start start 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.169+0000 7fb064dac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f623db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.169+0000 7fb064dac740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f62e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb064dac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f630fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb064dac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562b5f58a090 con 0x562b5f6239d0 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb064dac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562b5f531a70 con 0x562b5f62ebb0 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb064dac740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562b5f586050 con 0x562b5f6242f0 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb060f70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f623db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb059ffb640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb0480a2c10 0x7fb0480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb060f70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f623db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39148/0 (socket says 172.21.15.133:39148) 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb060f70640 1 -- 172.21.15.133:0/2715105281 learned_addr learned my addr 172.21.15.133:0/2715105281 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb060f70640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb0480a2c10 0x7fb0480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb05bfff640 1 --2- 172.21.15.133:0/2715105281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f62e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb059ffb640 1 -- 172.21.15.133:0/1819228582 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fb0440ce4b0 con 0x7fb05c101420 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb060f70640 1 -- 172.21.15.133:0/2715105281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 msgr2=0x562b5f62e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb060f70640 1 --2- 172.21.15.133:0/2715105281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f62e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb060f70640 1 -- 172.21.15.133:0/2715105281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 msgr2=0x562b5f630fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb053fff640 1 --2- 172.21.15.133:0/2715105281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f630fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb060f70640 1 --2- 172.21.15.133:0/2715105281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f630fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb060f70640 1 -- 172.21.15.133:0/2715105281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b5f53e130 con 0x562b5f6239d0 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb053fff640 1 --2- 172.21.15.133:0/2715105281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f630fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb060f70640 1 --2- 172.21.15.133:0/2715105281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f623db0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb054005fe0 tx=0x7fb054031da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8cf06a35c46c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb060f70640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb0480a2c10 0x7fb0480a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb054035000 tx=0x7fb054036000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb0527fc640 1 -- 172.21.15.133:0/2715105281 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05404d070 con 0x562b5f6239d0 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb0527fc640 1 -- 172.21.15.133:0/2715105281 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb054036070 con 0x562b5f6239d0 2024-09-11T23:36:36.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.170+0000 7fb0527fc640 1 -- 172.21.15.133:0/2715105281 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054030b90 con 0x562b5f6239d0 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.171+0000 7fb064dac740 1 -- 172.21.15.133:0/2715105281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 msgr2=0x562b5f623db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.171+0000 7fb064dac740 1 --2- 172.21.15.133:0/2715105281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f623db0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb054005fe0 tx=0x7fb054031da0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.171+0000 7fb064dac740 1 -- 172.21.15.133:0/2715105281 shutdown_connections 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.171+0000 7fb064dac740 1 --2- 172.21.15.133:0/2715105281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f630fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.171+0000 7fb064dac740 1 --2- 172.21.15.133:0/2715105281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f62e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.171+0000 7fb064dac740 1 --2- 172.21.15.133:0/2715105281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f623db0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.171+0000 7fb064dac740 1 -- 172.21.15.133:0/2715105281 >> 172.21.15.133:0/2715105281 conn(0x562b5f61ff60 msgr2=0x562b5f620340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.171+0000 7fb064dac740 1 -- 172.21.15.133:0/2715105281 shutdown_connections 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.171+0000 7fb064dac740 1 -- 172.21.15.133:0/2715105281 wait complete. 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb064dac740 1 -- start start 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb064dac740 1 Processor -- start 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb064dac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f6239d0 0x562b5f62e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb064dac740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f62b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb064dac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f62ebb0 0x562b5f62bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb064dac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562b5f635b90 con 0x562b5f62ebb0 2024-09-11T23:36:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb064dac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562b5f544f20 con 0x562b5f6239d0 2024-09-11T23:36:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb064dac740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562b5f635d10 con 0x562b5f6242f0 2024-09-11T23:36:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb064dac740 10 client.0 _ll_register_callbacks cb 0x562b5f50e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb060f70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f62ebb0 0x562b5f62bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb060f70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f62ebb0 0x562b5f62bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39150/0 (socket says 172.21.15.133:39150) 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb060f70640 1 -- 172.21.15.133:0/979517440 learned_addr learned my addr 172.21.15.133:0/979517440 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb05bfff640 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f6239d0 0x562b5f62e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb053fff640 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f62b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb060f70640 1 -- 172.21.15.133:0/979517440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 msgr2=0x562b5f62b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb060f70640 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f62b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb060f70640 1 -- 172.21.15.133:0/979517440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f6239d0 msgr2=0x562b5f62e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb060f70640 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f6239d0 0x562b5f62e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb060f70640 1 -- 172.21.15.133:0/979517440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b5f2fb2b0 con 0x562b5f62ebb0 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb05bfff640 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f6239d0 0x562b5f62e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.172+0000 7fb053fff640 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f62b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb060f70640 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f62ebb0 0x562b5f62bd40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb0540093e0 tx=0x7fb0540314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae21b35994afa70 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb03b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb03b7fe640 1 -- 172.21.15.133:0/979517440 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05404d040 con 0x562b5f62ebb0 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb03b7fe640 1 -- 172.21.15.133:0/979517440 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fb054030b20 con 0x562b5f62ebb0 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb064dac740 1 -- 172.21.15.133:0/979517440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb054036070 con 0x562b5f62ebb0 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb03b7fe640 1 -- 172.21.15.133:0/979517440 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb054004440 con 0x562b5f62ebb0 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb064dac740 10 client.15171 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb064dac740 1 -- 172.21.15.133:0/979517440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b5f641470 con 0x562b5f62ebb0 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb03b7fe640 1 -- 172.21.15.133:0/979517440 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb054036070 con 0x562b5f62ebb0 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb03b7fe640 1 -- 172.21.15.133:0/979517440 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb05405b020 con 0x562b5f62ebb0 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb064dac740 10 client.15171 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb064dac740 10 client.15171 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb064dac740 20 client.15171 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb064dac740 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562b5f6435f0 0x562b5f644a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb064dac740 4 client.15171 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.173+0000 7fb064dac740 1 -- 172.21.15.133:0/979517440 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b5f597600 con 0x562b5f6435f0 2024-09-11T23:36:36.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.174+0000 7fb05bfff640 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562b5f6435f0 0x562b5f644a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.174+0000 7fb05bfff640 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562b5f6435f0 0x562b5f644a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d193edd74793cae server_cookie=fa29b9aa2a15bd87 in_seq=0 out_seq=0 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.174+0000 7fb03b7fe640 10 client.15171 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb03b7fe640 1 -- 172.21.15.133:0/979517440 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b5f597600 con 0x562b5f6435f0 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb03b7fe640 10 client.15171 handle_command_reply: tid=0 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb064dac740 1 client.15171 shutdown 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb064dac740 1 -- 172.21.15.133:0/979517440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f62ebb0 msgr2=0x562b5f62bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb064dac740 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f62ebb0 0x562b5f62bd40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb0540093e0 tx=0x7fb0540314b0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb064dac740 1 -- 172.21.15.133:0/979517440 shutdown_connections 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb064dac740 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562b5f6435f0 0x562b5f644a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb064dac740 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f62ebb0 0x562b5f62bd40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb064dac740 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f62b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb03b7fe640 0 client.15171 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb064dac740 1 --2- 172.21.15.133:0/979517440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f6239d0 0x562b5f62e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb064dac740 1 -- 172.21.15.133:0/979517440 >> 172.21.15.133:0/979517440 conn(0x562b5f61ff60 msgr2=0x562b5f625c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:36.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb064dac740 1 -- 172.21.15.133:0/979517440 shutdown_connections 2024-09-11T23:36:36.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.175+0000 7fb064dac740 1 -- 172.21.15.133:0/979517440 wait complete. 2024-09-11T23:36:36.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.176+0000 7fb064dac740 20 client.15171 trim_cache size 0 max 16384 2024-09-11T23:36:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.193+0000 7fb064dac740 1 Processor -- start 2024-09-11T23:36:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.193+0000 7fb064dac740 1 -- start start 2024-09-11T23:36:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.193+0000 7fb064dac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f643cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.193+0000 7fb064dac740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f6427c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.193+0000 7fb064dac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f62d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.193+0000 7fb064dac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562b5f642da0 con 0x562b5f6239d0 2024-09-11T23:36:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.193+0000 7fb064dac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562b5f643470 con 0x562b5f62ebb0 2024-09-11T23:36:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.193+0000 7fb064dac740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562b5f643b40 con 0x562b5f6242f0 2024-09-11T23:36:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb05bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f643cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb05bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f643cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39156/0 (socket says 172.21.15.133:39156) 2024-09-11T23:36:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb05bfff640 1 -- 172.21.15.133:0/3296807420 learned_addr learned my addr 172.21.15.133:0/3296807420 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb05bfff640 1 -- 172.21.15.133:0/3296807420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 msgr2=0x562b5f6427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb05bfff640 1 --2- 172.21.15.133:0/3296807420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f6427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb05bfff640 1 -- 172.21.15.133:0/3296807420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 msgr2=0x562b5f62d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb05bfff640 1 --2- 172.21.15.133:0/3296807420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f62d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb05bfff640 1 -- 172.21.15.133:0/3296807420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b5f629d50 con 0x562b5f6239d0 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb05bfff640 1 --2- 172.21.15.133:0/3296807420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f643cc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0070f0 tx=0x7fb04c007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab2702aa6fbe179 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb051ffb640 1 -- 172.21.15.133:0/3296807420 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c016b20 con 0x562b5f6239d0 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb051ffb640 1 -- 172.21.15.133:0/3296807420 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c016cc0 con 0x562b5f6239d0 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.194+0000 7fb051ffb640 1 -- 172.21.15.133:0/3296807420 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c014680 con 0x562b5f6239d0 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.195+0000 7fb064dac740 1 -- 172.21.15.133:0/3296807420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 msgr2=0x562b5f643cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.195+0000 7fb064dac740 1 --2- 172.21.15.133:0/3296807420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f643cc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0070f0 tx=0x7fb04c007830 comp rx=0 tx=0).stop 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.195+0000 7fb064dac740 1 -- 172.21.15.133:0/3296807420 shutdown_connections 2024-09-11T23:36:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.195+0000 7fb064dac740 1 --2- 172.21.15.133:0/3296807420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f62d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.195+0000 7fb064dac740 1 --2- 172.21.15.133:0/3296807420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f6427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.195+0000 7fb064dac740 1 --2- 172.21.15.133:0/3296807420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f643cc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.195+0000 7fb064dac740 1 -- 172.21.15.133:0/3296807420 >> 172.21.15.133:0/3296807420 conn(0x562b5f61ff60 msgr2=0x562b5f62cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.195+0000 7fb064dac740 1 -- 172.21.15.133:0/3296807420 shutdown_connections 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.195+0000 7fb064dac740 1 -- 172.21.15.133:0/3296807420 wait complete. 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.195+0000 7fb064dac740 1 -- start start 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb064dac740 1 Processor -- start 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb064dac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f6483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb064dac740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f648900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb064dac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f6a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb064dac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562b5f6448e0 con 0x562b5f6239d0 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb064dac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562b5f644210 con 0x562b5f62ebb0 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb064dac740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562b5f644fb0 con 0x562b5f6242f0 2024-09-11T23:36:36.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb064dac740 10 client.0 _ll_register_callbacks cb 0x7fb04c0071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb05bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f6483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb05bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f6483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39158/0 (socket says 172.21.15.133:39158) 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb05bfff640 1 -- 172.21.15.133:0/532625356 learned_addr learned my addr 172.21.15.133:0/532625356 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb053fff640 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f648900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb060f70640 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f6a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb05bfff640 1 -- 172.21.15.133:0/532625356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 msgr2=0x562b5f648900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb05bfff640 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f648900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb05bfff640 1 -- 172.21.15.133:0/532625356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 msgr2=0x562b5f6a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb05bfff640 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f6a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb05bfff640 1 -- 172.21.15.133:0/532625356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b5f6a63a0 con 0x562b5f6239d0 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb060f70640 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f6a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb053fff640 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f648900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb05bfff640 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f6483c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb04c004700 tx=0x7fb04c016b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b358b4683bf864dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb03b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.196+0000 7fb03b7fe640 1 -- 172.21.15.133:0/532625356 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c024770 con 0x562b5f6239d0 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb03b7fe640 1 -- 172.21.15.133:0/532625356 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fb04c024910 con 0x562b5f6239d0 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb064dac740 1 -- 172.21.15.133:0/532625356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb04c016cc0 con 0x562b5f6239d0 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb03b7fe640 1 -- 172.21.15.133:0/532625356 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb04c02d450 con 0x562b5f6239d0 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb064dac740 10 client.15177 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb064dac740 1 -- 172.21.15.133:0/532625356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b5f6ad720 con 0x562b5f6239d0 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb03b7fe640 1 -- 172.21.15.133:0/532625356 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c016cc0 con 0x562b5f6239d0 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb03b7fe640 1 -- 172.21.15.133:0/532625356 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb04c038730 con 0x562b5f6239d0 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb064dac740 10 client.15177 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb064dac740 10 client.15177 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb064dac740 20 client.15177 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb064dac740 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562b5f6435f0 0x562b5f6b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb064dac740 4 client.15177 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:36:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.197+0000 7fb064dac740 1 -- 172.21.15.133:0/532625356 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "status"}) -- 0x562b5f591b10 con 0x562b5f6435f0 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb053fff640 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562b5f6435f0 0x562b5f6b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb03b7fe640 10 client.15177 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb053fff640 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562b5f6435f0 0x562b5f6b0b20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=28f813c749d539d8 server_cookie=4ff6eaea77973fe8 in_seq=0 out_seq=0 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb03b7fe640 1 -- 172.21.15.133:0/532625356 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562b5f591b10 con 0x562b5f6435f0 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb03b7fe640 10 client.15177 handle_command_reply: tid=0 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb064dac740 1 client.15177 shutdown 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb064dac740 1 -- 172.21.15.133:0/532625356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 msgr2=0x562b5f6483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb064dac740 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f6483c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb04c004700 tx=0x7fb04c016b20 comp rx=0 tx=0).stop 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb064dac740 1 -- 172.21.15.133:0/532625356 shutdown_connections 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb064dac740 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562b5f6435f0 0x562b5f6b0b20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb064dac740 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b5f62ebb0 0x562b5f6a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb064dac740 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562b5f6242f0 0x562b5f648900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb064dac740 1 --2- 172.21.15.133:0/532625356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b5f6239d0 0x562b5f6483c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb064dac740 1 -- 172.21.15.133:0/532625356 >> 172.21.15.133:0/532625356 conn(0x562b5f61ff60 msgr2=0x562b5f62d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:36.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb03b7fe640 0 client.15177 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:36:36.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.198+0000 7fb064dac740 1 -- 172.21.15.133:0/532625356 shutdown_connections 2024-09-11T23:36:36.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb064dac740 1 -- 172.21.15.133:0/532625356 wait complete. 2024-09-11T23:36:36.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb064dac740 20 client.15177 trim_cache size 0 max 16384 2024-09-11T23:36:36.205 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:36.205 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:36:36.205 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 3, 2024-09-11T23:36:36.206 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24419, 2024-09-11T23:36:36.206 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:36:36.206 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:36:36.206 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:36:36.206 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 92.755766159999993, 2024-09-11T23:36:36.206 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 27, 2024-09-11T23:36:36.206 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 81, 2024-09-11T23:36:36.206 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 79, 2024-09-11T23:36:36.206 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 131.62607275299999 2024-09-11T23:36:36.206 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:36.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 -- 172.21.15.133:0/1819228582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb0480a2c10 msgr2=0x7fb0480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb0480a2c10 0x7fb0480a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb054035000 tx=0x7fb054036000 comp rx=0 tx=0).stop 2024-09-11T23:36:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 -- 172.21.15.133:0/1819228582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb05c101420 msgr2=0x7fb05c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb05c101420 0x7fb05c198ac0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb04400b4a0 tx=0x7fb04400b960 comp rx=0 tx=0).stop 2024-09-11T23:36:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 -- 172.21.15.133:0/1819228582 shutdown_connections 2024-09-11T23:36:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb0480a2c10 0x7fb0480a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb05c106a80 0x7fb05c19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb05c101420 0x7fb05c198ac0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 --2- 172.21.15.133:0/1819228582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb05c100a80 0x7fb05c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 -- 172.21.15.133:0/1819228582 >> 172.21.15.133:0/1819228582 conn(0x7fb05c0fc7a0 msgr2=0x7fb05c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 -- 172.21.15.133:0/1819228582 shutdown_connections 2024-09-11T23:36:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:36.199+0000 7fb0629d3640 1 -- 172.21.15.133:0/1819228582 wait complete. 2024-09-11T23:36:36.959 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:36:37.205 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:36 smithi133 ceph-mon[31430]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:37.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:36 smithi178 ceph-mon[35882]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:37.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:36 smithi138 ceph-mon[35142]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T23:36:37.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.673+0000 7fddf013d640 1 -- 172.21.15.133:0/2988285076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde81008d0 msgr2=0x7fdde8100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:37.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.673+0000 7fddf013d640 1 --2- 172.21.15.133:0/2988285076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde81008d0 0x7fdde8100cb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009900 tx=0x7fdde402a410 comp rx=0 tx=0).stop 2024-09-11T23:36:37.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.673+0000 7fddf013d640 1 -- 172.21.15.133:0/2988285076 shutdown_connections 2024-09-11T23:36:37.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.673+0000 7fddf013d640 1 --2- 172.21.15.133:0/2988285076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde810e480 0x7fdde8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.673+0000 7fddf013d640 1 --2- 172.21.15.133:0/2988285076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8101280 0x7fdde810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.673+0000 7fddf013d640 1 --2- 172.21.15.133:0/2988285076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde81008d0 0x7fdde8100cb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.673+0000 7fddf013d640 1 -- 172.21.15.133:0/2988285076 >> 172.21.15.133:0/2988285076 conn(0x7fdde80fc7a0 msgr2=0x7fdde80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:37.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.673+0000 7fddf013d640 1 -- 172.21.15.133:0/2988285076 shutdown_connections 2024-09-11T23:36:37.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.673+0000 7fddf013d640 1 -- 172.21.15.133:0/2988285076 wait complete. 2024-09-11T23:36:37.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddf013d640 1 Processor -- start 2024-09-11T23:36:37.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddf013d640 1 -- start start 2024-09-11T23:36:37.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddf013d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde81008d0 0x7fdde819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddf013d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8101280 0x7fdde819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddf013d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde810e480 0x7fdde819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddf013d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8114200 con 0x7fdde810e480 2024-09-11T23:36:37.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddf013d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8114080 con 0x7fdde8101280 2024-09-11T23:36:37.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddf013d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8114380 con 0x7fdde81008d0 2024-09-11T23:36:37.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddee6da640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde810e480 0x7fdde819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddee6da640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde810e480 0x7fdde819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39176/0 (socket says 172.21.15.133:39176) 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddee6da640 1 -- 172.21.15.133:0/1228154925 learned_addr learned my addr 172.21.15.133:0/1228154925 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddeded9640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde81008d0 0x7fdde819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fdded6d8640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8101280 0x7fdde819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddee6da640 1 -- 172.21.15.133:0/1228154925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde81008d0 msgr2=0x7fdde819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddee6da640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde81008d0 0x7fdde819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddee6da640 1 -- 172.21.15.133:0/1228154925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8101280 msgr2=0x7fdde819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddee6da640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8101280 0x7fdde819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.674+0000 7fddee6da640 1 -- 172.21.15.133:0/1228154925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde819f6f0 con 0x7fdde810e480 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.675+0000 7fdded6d8640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8101280 0x7fdde819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.675+0000 7fddeded9640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde81008d0 0x7fdde819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.675+0000 7fddee6da640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde810e480 0x7fdde819f010 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdde000cab0 tx=0x7fdde000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.675+0000 7fdddeffd640 1 -- 172.21.15.133:0/1228154925 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0007e10 con 0x7fdde810e480 2024-09-11T23:36:37.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.675+0000 7fddf013d640 1 -- 172.21.15.133:0/1228154925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde819f9e0 con 0x7fdde810e480 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.675+0000 7fdddeffd640 1 -- 172.21.15.133:0/1228154925 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde000f930 con 0x7fdde810e480 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.675+0000 7fdddeffd640 1 -- 172.21.15.133:0/1228154925 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0018b80 con 0x7fdde810e480 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.675+0000 7fddf013d640 1 -- 172.21.15.133:0/1228154925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde81a72c0 con 0x7fdde810e480 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.677+0000 7fdddeffd640 1 -- 172.21.15.133:0/1228154925 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdde000fae0 con 0x7fdde810e480 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.677+0000 7fddf2516740 1 Processor -- start 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.677+0000 7fddf2516740 1 -- start start 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddf2516740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c3a9d0 0x559dd3c3adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddf2516740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c45670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddf2516740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c45bb0 0x559dd3c47fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddf2516740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559dd3ba1090 con 0x559dd3c3b2f0 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddf2516740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559dd3b48a70 con 0x559dd3c3a9d0 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddf2516740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559dd3b9d050 con 0x559dd3c45bb0 2024-09-11T23:36:37.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fdddeffd640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fddd40a2c10 0x7fddd40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddeded9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c45670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddeded9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c45670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39184/0 (socket says 172.21.15.133:39184) 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddeded9640 1 -- 172.21.15.133:0/3413819286 learned_addr learned my addr 172.21.15.133:0/3413819286 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddee6da640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fddd40a2c10 0x7fddd40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fdded6d8640 1 --2- 172.21.15.133:0/3413819286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c45bb0 0x559dd3c47fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fdddeffd640 1 -- 172.21.15.133:0/1228154925 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fdde00cf460 con 0x7fdde810e480 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fdded6d8640 1 --2- 172.21.15.133:0/3413819286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c3a9d0 0x559dd3c3adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddeded9640 1 -- 172.21.15.133:0/3413819286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c45bb0 msgr2=0x559dd3c47fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddeded9640 1 --2- 172.21.15.133:0/3413819286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c45bb0 0x559dd3c47fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddeded9640 1 -- 172.21.15.133:0/3413819286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c3a9d0 msgr2=0x559dd3c3adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddeded9640 1 --2- 172.21.15.133:0/3413819286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c3a9d0 0x559dd3c3adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddeded9640 1 -- 172.21.15.133:0/3413819286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd3b55130 con 0x559dd3c3b2f0 2024-09-11T23:36:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddee6da640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fddd40a2c10 0x7fddd40a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdde0098630 tx=0x7fdde006dc70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.678+0000 7fddeded9640 1 --2- 172.21.15.133:0/3413819286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c45670 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009230 tx=0x7fdde4004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e31dda39eedaa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.679+0000 7fddd3fff640 1 -- 172.21.15.133:0/3413819286 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4035030 con 0x559dd3c3b2f0 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.679+0000 7fddd3fff640 1 -- 172.21.15.133:0/3413819286 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde4005520 con 0x559dd3c3b2f0 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.679+0000 7fddd3fff640 1 -- 172.21.15.133:0/3413819286 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4003810 con 0x559dd3c3b2f0 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.679+0000 7fddf2516740 1 -- 172.21.15.133:0/3413819286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 msgr2=0x559dd3c45670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.679+0000 7fddf2516740 1 --2- 172.21.15.133:0/3413819286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c45670 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009230 tx=0x7fdde4004200 comp rx=0 tx=0).stop 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.679+0000 7fddf2516740 1 -- 172.21.15.133:0/3413819286 shutdown_connections 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.679+0000 7fddf2516740 1 --2- 172.21.15.133:0/3413819286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c45bb0 0x559dd3c47fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.679+0000 7fddf2516740 1 --2- 172.21.15.133:0/3413819286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c45670 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.679+0000 7fddf2516740 1 --2- 172.21.15.133:0/3413819286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c3a9d0 0x559dd3c3adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.679+0000 7fddf2516740 1 -- 172.21.15.133:0/3413819286 >> 172.21.15.133:0/3413819286 conn(0x559dd3c36f60 msgr2=0x559dd3c37340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.679+0000 7fddf2516740 1 -- 172.21.15.133:0/3413819286 shutdown_connections 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fddf2516740 1 -- 172.21.15.133:0/3413819286 wait complete. 2024-09-11T23:36:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fddf2516740 1 -- start start 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fddf2516740 1 Processor -- start 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fddf2516740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c45470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fddf2516740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3b2f0 0x559dd3c427c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fddf2516740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3c42d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fddf2516740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559dd3c48780 con 0x559dd3c3a9d0 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fddf2516740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559dd3b5bf20 con 0x559dd3c45bb0 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fddf2516740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559dd3c48900 con 0x559dd3c3b2f0 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fddf2516740 10 client.0 _ll_register_callbacks cb 0x559dd3b25030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fdded6d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c45470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fdded6d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c45470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39190/0 (socket says 172.21.15.133:39190) 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.680+0000 7fdded6d8640 1 -- 172.21.15.133:0/910453900 learned_addr learned my addr 172.21.15.133:0/910453900 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fdded6d8640 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3c42d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fddeded9640 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3b2f0 0x559dd3c427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fdded6d8640 1 -- 172.21.15.133:0/910453900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3b2f0 msgr2=0x559dd3c427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fdded6d8640 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3b2f0 0x559dd3c427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fdded6d8640 1 -- 172.21.15.133:0/910453900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 msgr2=0x559dd3c42d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fdded6d8640 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3c42d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fdded6d8640 1 -- 172.21.15.133:0/910453900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd39122b0 con 0x559dd3c3a9d0 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fdded6d8640 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c45470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdde810ed10 tx=0x7fddd8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e422724612729661 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fddd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fddd17fa640 1 -- 172.21.15.133:0/910453900 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8011bb0 con 0x559dd3c3a9d0 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fddd17fa640 1 -- 172.21.15.133:0/910453900 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fddd8011d50 con 0x559dd3c3a9d0 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fddf2516740 1 -- 172.21.15.133:0/910453900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde4005520 con 0x559dd3c3a9d0 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fddd17fa640 1 -- 172.21.15.133:0/910453900 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fddd8025050 con 0x559dd3c3a9d0 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fddf2516740 10 client.15189 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fddf2516740 1 -- 172.21.15.133:0/910453900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559dd3c54130 con 0x559dd3c3a9d0 2024-09-11T23:36:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.681+0000 7fddd17fa640 1 -- 172.21.15.133:0/910453900 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde4005520 con 0x559dd3c3a9d0 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.682+0000 7fddd17fa640 1 -- 172.21.15.133:0/910453900 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fddd8004c50 con 0x559dd3c3a9d0 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.682+0000 7fddf2516740 10 client.15189 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.682+0000 7fddf2516740 10 client.15189 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.682+0000 7fddf2516740 20 client.15189 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.682+0000 7fddf2516740 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559dd3c567c0 0x559dd3c57c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.682+0000 7fddf2516740 4 client.15189 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.682+0000 7fddf2516740 1 -- 172.21.15.133:0/910453900 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dd3bae600 con 0x559dd3c567c0 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.682+0000 7fddeded9640 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559dd3c567c0 0x559dd3c57c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.682+0000 7fddeded9640 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559dd3c567c0 0x559dd3c57c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a80932ed47ba35d server_cookie=8d805bb6960b42f1 in_seq=0 out_seq=0 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.682+0000 7fddd17fa640 10 client.15189 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddd17fa640 1 -- 172.21.15.133:0/910453900 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559dd3bae600 con 0x559dd3c567c0 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddd17fa640 10 client.15189 handle_command_reply: tid=0 2024-09-11T23:36:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 1 client.15189 shutdown 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 1 -- 172.21.15.133:0/910453900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 msgr2=0x559dd3c45470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c45470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdde810ed10 tx=0x7fddd8015040 comp rx=0 tx=0).stop 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 1 -- 172.21.15.133:0/910453900 shutdown_connections 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559dd3c567c0 0x559dd3c57c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3c42d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddd17fa640 0 client.15189 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3b2f0 0x559dd3c427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 1 --2- 172.21.15.133:0/910453900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c45470 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 1 -- 172.21.15.133:0/910453900 >> 172.21.15.133:0/910453900 conn(0x559dd3c36f60 msgr2=0x559dd3c3cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 1 -- 172.21.15.133:0/910453900 shutdown_connections 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 1 -- 172.21.15.133:0/910453900 wait complete. 2024-09-11T23:36:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.683+0000 7fddf2516740 20 client.15189 trim_cache size 0 max 16384 2024-09-11T23:36:37.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.701+0000 7fddf2516740 1 Processor -- start 2024-09-11T23:36:37.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.701+0000 7fddf2516740 1 -- start start 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.701+0000 7fddf2516740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3a9d0 0x559dd3c560f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.701+0000 7fddf2516740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c57c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.701+0000 7fddf2516740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3c5ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.701+0000 7fddf2516740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559dd3c55f70 con 0x559dd3c3b2f0 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.701+0000 7fddf2516740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559dd3c56640 con 0x559dd3c45bb0 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.701+0000 7fddf2516740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559dd3c56d10 con 0x559dd3c3a9d0 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.701+0000 7fddeded9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c57c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fddeded9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c57c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39202/0 (socket says 172.21.15.133:39202) 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fddeded9640 1 -- 172.21.15.133:0/1353677860 learned_addr learned my addr 172.21.15.133:0/1353677860 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fdded6d8640 1 --2- 172.21.15.133:0/1353677860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3a9d0 0x559dd3c560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fddeded9640 1 -- 172.21.15.133:0/1353677860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3a9d0 msgr2=0x559dd3c560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fddeded9640 1 --2- 172.21.15.133:0/1353677860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3a9d0 0x559dd3c560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fdded6d8640 1 --2- 172.21.15.133:0/1353677860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3c5ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fddeded9640 1 -- 172.21.15.133:0/1353677860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 msgr2=0x559dd3c5ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fddeded9640 1 --2- 172.21.15.133:0/1353677860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3c5ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fddeded9640 1 -- 172.21.15.133:0/1353677860 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd3c457e0 con 0x559dd3c3b2f0 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fdded6d8640 1 --2- 172.21.15.133:0/1353677860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3c5ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fddeded9640 1 --2- 172.21.15.133:0/1353677860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c57c30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdde4006010 tx=0x7fdde4004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e5ff0b1a00c188 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fddd37fe640 1 -- 172.21.15.133:0/1353677860 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4035040 con 0x559dd3c3b2f0 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fddd37fe640 1 -- 172.21.15.133:0/1353677860 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde4031c90 con 0x559dd3c3b2f0 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.702+0000 7fddd37fe640 1 -- 172.21.15.133:0/1353677860 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4005230 con 0x559dd3c3b2f0 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.703+0000 7fddf2516740 1 -- 172.21.15.133:0/1353677860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 msgr2=0x559dd3c57c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.703+0000 7fddf2516740 1 --2- 172.21.15.133:0/1353677860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c57c30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdde4006010 tx=0x7fdde4004c90 comp rx=0 tx=0).stop 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.703+0000 7fddf2516740 1 -- 172.21.15.133:0/1353677860 shutdown_connections 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.703+0000 7fddf2516740 1 --2- 172.21.15.133:0/1353677860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3c5ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.703+0000 7fddf2516740 1 --2- 172.21.15.133:0/1353677860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3b2f0 0x559dd3c57c30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.703+0000 7fddf2516740 1 --2- 172.21.15.133:0/1353677860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3a9d0 0x559dd3c560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.703+0000 7fddf2516740 1 -- 172.21.15.133:0/1353677860 >> 172.21.15.133:0/1353677860 conn(0x559dd3c36f60 msgr2=0x559dd3c41090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.703+0000 7fddf2516740 1 -- 172.21.15.133:0/1353677860 shutdown_connections 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.703+0000 7fddf2516740 1 -- 172.21.15.133:0/1353677860 wait complete. 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.703+0000 7fddf2516740 1 -- start start 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.703+0000 7fddf2516740 1 Processor -- start 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fddf2516740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c5b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fddf2516740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3b2f0 0x559dd3c5b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fddf2516740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3cbeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fddf2516740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559dd3c57ab0 con 0x559dd3c3a9d0 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fddf2516740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559dd3c573e0 con 0x559dd3c45bb0 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fddf2516740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559dd3c58180 con 0x559dd3c3b2f0 2024-09-11T23:36:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fddf2516740 10 client.0 _ll_register_callbacks cb 0x7fddd800d970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fdded6d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c5b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fdded6d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c5b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39206/0 (socket says 172.21.15.133:39206) 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fdded6d8640 1 -- 172.21.15.133:0/2763311352 learned_addr learned my addr 172.21.15.133:0/2763311352 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fddeded9640 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3b2f0 0x559dd3c5b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fdded6d8640 1 -- 172.21.15.133:0/2763311352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3b2f0 msgr2=0x559dd3c5b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fdded6d8640 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3b2f0 0x559dd3c5b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fdded6d8640 1 -- 172.21.15.133:0/2763311352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 msgr2=0x559dd3cbeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fdded6d8640 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3cbeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fdded6d8640 1 -- 172.21.15.133:0/2763311352 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd3cbf400 con 0x559dd3c3a9d0 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fdded6d8640 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c5b040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fddd8015010 tx=0x7fddd8013690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859aeaa064da06d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.704+0000 7fddd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddd17fa640 1 -- 172.21.15.133:0/2763311352 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd802d070 con 0x559dd3c3a9d0 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddd17fa640 1 -- 172.21.15.133:0/2763311352 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fddd8011bb0 con 0x559dd3c3a9d0 2024-09-11T23:36:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddf2516740 1 -- 172.21.15.133:0/2763311352 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde4031c90 con 0x559dd3c3a9d0 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddd17fa640 1 -- 172.21.15.133:0/2763311352 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fddd800d4e0 con 0x559dd3c3a9d0 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddf2516740 10 client.15195 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddf2516740 1 -- 172.21.15.133:0/2763311352 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559dd3cc04a0 con 0x559dd3c3a9d0 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddd17fa640 1 -- 172.21.15.133:0/2763311352 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde4031c90 con 0x559dd3c3a9d0 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddd17fa640 1 -- 172.21.15.133:0/2763311352 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fddd800fba0 con 0x559dd3c3a9d0 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddf2516740 10 client.15195 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddf2516740 10 client.15195 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddf2516740 20 client.15195 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddf2516740 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559dd3cc4fd0 0x559dd3cc3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddf2516740 4 client.15195 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.705+0000 7fddf2516740 1 -- 172.21.15.133:0/2763311352 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "status"}) -- 0x559dd3ba8b10 con 0x559dd3cc4fd0 2024-09-11T23:36:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddeded9640 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559dd3cc4fd0 0x559dd3cc3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddeded9640 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559dd3cc4fd0 0x559dd3cc3ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f76e883b15c8d26 server_cookie=ec500a06c447fde9 in_seq=0 out_seq=0 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddd17fa640 10 client.15195 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddd17fa640 1 -- 172.21.15.133:0/2763311352 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x559dd3ba8b10 con 0x559dd3cc4fd0 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddd17fa640 10 client.15195 handle_command_reply: tid=0 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddf2516740 1 client.15195 shutdown 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddf2516740 1 -- 172.21.15.133:0/2763311352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 msgr2=0x559dd3c5b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddf2516740 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c5b040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fddd8015010 tx=0x7fddd8013690 comp rx=0 tx=0).stop 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddf2516740 1 -- 172.21.15.133:0/2763311352 shutdown_connections 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddf2516740 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559dd3cc4fd0 0x559dd3cc3ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddf2516740 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dd3c45bb0 0x559dd3cbeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddf2516740 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd3c3b2f0 0x559dd3c5b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddf2516740 1 --2- 172.21.15.133:0/2763311352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559dd3c3a9d0 0x559dd3c5b040 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddd17fa640 0 client.15195 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:36:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddf2516740 1 -- 172.21.15.133:0/2763311352 >> 172.21.15.133:0/2763311352 conn(0x559dd3c36f60 msgr2=0x559dd3c41010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:37.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddf2516740 1 -- 172.21.15.133:0/2763311352 shutdown_connections 2024-09-11T23:36:37.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.706+0000 7fddf2516740 1 -- 172.21.15.133:0/2763311352 wait complete. 2024-09-11T23:36:37.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf2516740 20 client.15195 trim_cache size 0 max 16384 2024-09-11T23:36:37.714 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:37.714 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:36:37.714 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 2, 2024-09-11T23:36:37.714 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24464, 2024-09-11T23:36:37.714 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:36:37.714 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:36:37.714 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:36:37.715 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 95.222930418000004, 2024-09-11T23:36:37.715 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 27, 2024-09-11T23:36:37.715 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 81, 2024-09-11T23:36:37.715 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 79, 2024-09-11T23:36:37.715 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 114.69198448 2024-09-11T23:36:37.715 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:37.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 -- 172.21.15.133:0/1228154925 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fddd40a2c10 msgr2=0x7fddd40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fddd40a2c10 0x7fddd40a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdde0098630 tx=0x7fdde006dc70 comp rx=0 tx=0).stop 2024-09-11T23:36:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 -- 172.21.15.133:0/1228154925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde810e480 msgr2=0x7fdde819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde810e480 0x7fdde819f010 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdde000cab0 tx=0x7fdde000cf70 comp rx=0 tx=0).stop 2024-09-11T23:36:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 -- 172.21.15.133:0/1228154925 shutdown_connections 2024-09-11T23:36:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fddd40a2c10 0x7fddd40a50d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde810e480 0x7fdde819f010 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde8101280 0x7fdde819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 --2- 172.21.15.133:0/1228154925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdde81008d0 0x7fdde819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 -- 172.21.15.133:0/1228154925 >> 172.21.15.133:0/1228154925 conn(0x7fdde80fc7a0 msgr2=0x7fdde80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 -- 172.21.15.133:0/1228154925 shutdown_connections 2024-09-11T23:36:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:37.707+0000 7fddf013d640 1 -- 172.21.15.133:0/1228154925 wait complete. 2024-09-11T23:36:38.414 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T23:36:38.662 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:39.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:38 smithi133 ceph-mon[31430]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:39.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.116+0000 7f2dccb14640 1 -- 172.21.15.133:0/3667354831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc8100a80 msgr2=0x7f2dc8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:39.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.116+0000 7f2dccb14640 1 --2- 172.21.15.133:0/3667354831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc8100a80 0x7f2dc8100ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009930 tx=0x7f2db802a420 comp rx=0 tx=0).stop 2024-09-11T23:36:39.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.116+0000 7f2dccb14640 1 -- 172.21.15.133:0/3667354831 shutdown_connections 2024-09-11T23:36:39.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.116+0000 7f2dccb14640 1 --2- 172.21.15.133:0/3667354831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc8101420 0x7f2dc810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.116+0000 7f2dccb14640 1 --2- 172.21.15.133:0/3667354831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc8100a80 0x7f2dc8100ee0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.116+0000 7f2dccb14640 1 --2- 172.21.15.133:0/3667354831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dc8106a80 0x7f2dc8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.116+0000 7f2dccb14640 1 -- 172.21.15.133:0/3667354831 >> 172.21.15.133:0/3667354831 conn(0x7f2dc80fc7a0 msgr2=0x7f2dc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:39.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.116+0000 7f2dccb14640 1 -- 172.21.15.133:0/3667354831 shutdown_connections 2024-09-11T23:36:39.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.116+0000 7f2dccb14640 1 -- 172.21.15.133:0/3667354831 wait complete. 2024-09-11T23:36:39.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dccb14640 1 Processor -- start 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dccb14640 1 -- start start 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dccb14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc8100a80 0x7f2dc81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dccb14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc8101420 0x7f2dc8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dccb14640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dc8106a80 0x7f2dc819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dccb14640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc8110ea0 con 0x7f2dc8101420 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dccb14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc8110d20 con 0x7f2dc8100a80 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dccb14640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc8111020 con 0x7f2dc8106a80 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dc5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc8101420 0x7f2dc8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dc5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc8101420 0x7f2dc8198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39232/0 (socket says 172.21.15.133:39232) 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dc5d9b640 1 -- 172.21.15.133:0/3562343259 learned_addr learned my addr 172.21.15.133:0/3562343259 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dc659c640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc8100a80 0x7f2dc81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dc6d9d640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dc8106a80 0x7f2dc819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dc5d9b640 1 -- 172.21.15.133:0/3562343259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dc8106a80 msgr2=0x7f2dc819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dc8106a80 0x7f2dc819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dc5d9b640 1 -- 172.21.15.133:0/3562343259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc8100a80 msgr2=0x7f2dc81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc8100a80 0x7f2dc81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dc5d9b640 1 -- 172.21.15.133:0/3562343259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc819d480 con 0x7f2dc8101420 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.117+0000 7f2dc659c640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc8100a80 0x7f2dc81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.118+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc8101420 0x7f2dc8198a10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009a50 tx=0x7f2db8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.118+0000 7f2dbf7fe640 1 -- 172.21.15.133:0/3562343259 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db803f030 con 0x7f2dc8101420 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.118+0000 7f2dccb14640 1 -- 172.21.15.133:0/3562343259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc819d710 con 0x7f2dc8101420 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.118+0000 7f2dbf7fe640 1 -- 172.21.15.133:0/3562343259 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db8003ce0 con 0x7f2dc8101420 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.118+0000 7f2dbf7fe640 1 -- 172.21.15.133:0/3562343259 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8035070 con 0x7f2dc8101420 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.118+0000 7f2dccb14640 1 -- 172.21.15.133:0/3562343259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc81a5050 con 0x7f2dc8101420 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.120+0000 7f2dbf7fe640 1 -- 172.21.15.133:0/3562343259 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2db8042030 con 0x7f2dc8101420 2024-09-11T23:36:39.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.120+0000 7f2dceeed740 1 Processor -- start 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dceeed740 1 -- start start 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193bd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dceeed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564b19319090 con 0x564b193b29d0 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dceeed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b192c0a70 con 0x564b193bdbb0 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dceeed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564b19315050 con 0x564b193b32f0 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dbf7fe640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2db40a2b80 0x7f2db40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dc6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dc6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39242/0 (socket says 172.21.15.133:39242) 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dc6d9d640 1 -- 172.21.15.133:0/2302176354 learned_addr learned my addr 172.21.15.133:0/2302176354 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:39.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dc6d9d640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2db40a2b80 0x7f2db40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dbf7fe640 1 -- 172.21.15.133:0/3562343259 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f2db80ef610 con 0x7f2dc8101420 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/2302176354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dc659c640 1 --2- 172.21.15.133:0/2302176354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193bd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dc6d9d640 1 -- 172.21.15.133:0/2302176354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 msgr2=0x564b193bd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dc6d9d640 1 --2- 172.21.15.133:0/2302176354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193bd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dc6d9d640 1 -- 172.21.15.133:0/2302176354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 msgr2=0x564b193bffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.121+0000 7f2dc6d9d640 1 --2- 172.21.15.133:0/2302176354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193bffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.122+0000 7f2dc6d9d640 1 -- 172.21.15.133:0/2302176354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b192cd130 con 0x564b193b29d0 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.122+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/2302176354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193bffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.122+0000 7f2dc6d9d640 1 --2- 172.21.15.133:0/2302176354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193b2db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0013c90 tx=0x7f2dc0015710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae00db6cbcab43f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.122+0000 7f2dc6d9d640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2db40a2b80 0x7f2db40a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0015c10 tx=0x7f2dc001a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.122+0000 7f2dbcff9640 1 -- 172.21.15.133:0/2302176354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0024070 con 0x564b193b29d0 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.122+0000 7f2dbcff9640 1 -- 172.21.15.133:0/2302176354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc0017cb0 con 0x564b193b29d0 2024-09-11T23:36:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.122+0000 7f2dbcff9640 1 -- 172.21.15.133:0/2302176354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00163b0 con 0x564b193b29d0 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 -- 172.21.15.133:0/2302176354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 msgr2=0x564b193b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 --2- 172.21.15.133:0/2302176354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193b2db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0013c90 tx=0x7f2dc0015710 comp rx=0 tx=0).stop 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 -- 172.21.15.133:0/2302176354 shutdown_connections 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 --2- 172.21.15.133:0/2302176354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193bffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 --2- 172.21.15.133:0/2302176354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193bd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 --2- 172.21.15.133:0/2302176354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193b2db0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 -- 172.21.15.133:0/2302176354 >> 172.21.15.133:0/2302176354 conn(0x564b193aef60 msgr2=0x564b193af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 -- 172.21.15.133:0/2302176354 shutdown_connections 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 -- 172.21.15.133:0/2302176354 wait complete. 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 -- start start 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 Processor -- start 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193b32f0 0x564b193ba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193bdbb0 0x564b193badb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564b193c0780 con 0x564b193b29d0 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b192d3f20 con 0x564b193b32f0 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.123+0000 7f2dceeed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564b193c0900 con 0x564b193bdbb0 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dceeed740 10 client.0 _ll_register_callbacks cb 0x564b1929d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dc659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dc659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39256/0 (socket says 172.21.15.133:39256) 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dc659c640 1 -- 172.21.15.133:0/2534728511 learned_addr learned my addr 172.21.15.133:0/2534728511 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dc6d9d640 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193bdbb0 0x564b193badb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dc659c640 1 -- 172.21.15.133:0/2534728511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193bdbb0 msgr2=0x564b193badb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dc659c640 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193bdbb0 0x564b193badb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193b32f0 0x564b193ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dc659c640 1 -- 172.21.15.133:0/2534728511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193b32f0 msgr2=0x564b193ba850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:39.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dc659c640 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193b32f0 0x564b193ba850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dc659c640 1 -- 172.21.15.133:0/2534728511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b1908a2b0 con 0x564b193b29d0 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dc659c640 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bd500 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2db0010840 tx=0x7f2db0010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba55ff355e76f24a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dadffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dadffb640 1 -- 172.21.15.133:0/2534728511 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db000eda0 con 0x564b193b29d0 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dadffb640 1 -- 172.21.15.133:0/2534728511 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f2db0005020 con 0x564b193b29d0 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dceeed740 1 -- 172.21.15.133:0/2534728511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc0017cb0 con 0x564b193b29d0 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dadffb640 1 -- 172.21.15.133:0/2534728511 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2db0005260 con 0x564b193b29d0 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dceeed740 10 client.15207 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dceeed740 1 -- 172.21.15.133:0/2534728511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b193cc170 con 0x564b193b29d0 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.124+0000 7f2dadffb640 1 -- 172.21.15.133:0/2534728511 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc0017cb0 con 0x564b193b29d0 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.125+0000 7f2dadffb640 1 -- 172.21.15.133:0/2534728511 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2db0018550 con 0x564b193b29d0 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.125+0000 7f2dceeed740 10 client.15207 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:39.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.125+0000 7f2dceeed740 10 client.15207 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:36:39.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.125+0000 7f2dceeed740 20 client.15207 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:39.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.125+0000 7f2dceeed740 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x564b193ce820 0x564b193cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.125+0000 7f2dceeed740 4 client.15207 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:39.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.125+0000 7f2dceeed740 1 -- 172.21.15.133:0/2534728511 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b19326600 con 0x564b193ce820 2024-09-11T23:36:39.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.125+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x564b193ce820 0x564b193cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.125+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x564b193ce820 0x564b193cfc90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3668d83a60f52b85 server_cookie=c7c2f51a81cac68e in_seq=0 out_seq=0 2024-09-11T23:36:39.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.125+0000 7f2dadffb640 10 client.15207 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:36:39.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dadffb640 1 -- 172.21.15.133:0/2534728511 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b19326600 con 0x564b193ce820 2024-09-11T23:36:39.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dadffb640 10 client.15207 handle_command_reply: tid=0 2024-09-11T23:36:39.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 1 client.15207 shutdown 2024-09-11T23:36:39.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 1 -- 172.21.15.133:0/2534728511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 msgr2=0x564b193bd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bd500 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2db0010840 tx=0x7f2db0010d00 comp rx=0 tx=0).stop 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 1 -- 172.21.15.133:0/2534728511 shutdown_connections 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x564b193ce820 0x564b193cfc90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193bdbb0 0x564b193badb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dadffb640 0 client.15207 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193b32f0 0x564b193ba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 1 --2- 172.21.15.133:0/2534728511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bd500 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 1 -- 172.21.15.133:0/2534728511 >> 172.21.15.133:0/2534728511 conn(0x564b193aef60 msgr2=0x564b193b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 1 -- 172.21.15.133:0/2534728511 shutdown_connections 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 1 -- 172.21.15.133:0/2534728511 wait complete. 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.126+0000 7f2dceeed740 20 client.15207 trim_cache size 0 max 16384 2024-09-11T23:36:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dceeed740 1 Processor -- start 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dceeed740 1 -- start start 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193bc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dceeed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564b193cdfd0 con 0x564b193b29d0 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dceeed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b193ce6a0 con 0x564b193bdbb0 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dceeed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564b193ced70 con 0x564b193b32f0 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39264/0 (socket says 172.21.15.133:39264) 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc659c640 1 -- 172.21.15.133:0/4170100933 learned_addr learned my addr 172.21.15.133:0/4170100933 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc659c640 1 -- 172.21.15.133:0/4170100933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 msgr2=0x564b193bbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/4170100933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc6d9d640 1 --2- 172.21.15.133:0/4170100933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193bc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc659c640 1 --2- 172.21.15.133:0/4170100933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193bbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc659c640 1 -- 172.21.15.133:0/4170100933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 msgr2=0x564b193bc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc659c640 1 --2- 172.21.15.133:0/4170100933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193bc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc659c640 1 -- 172.21.15.133:0/4170100933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b193c0890 con 0x564b193b29d0 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc6d9d640 1 --2- 172.21.15.133:0/4170100933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193bc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.144+0000 7f2dc659c640 1 --2- 172.21.15.133:0/4170100933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bada0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2db0010810 tx=0x7f2db0010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcac2640a1592083 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2daffff640 1 -- 172.21.15.133:0/4170100933 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0031020 con 0x564b193b29d0 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2daffff640 1 -- 172.21.15.133:0/4170100933 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db0010020 con 0x564b193b29d0 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2daffff640 1 -- 172.21.15.133:0/4170100933 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0027040 con 0x564b193b29d0 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2dceeed740 1 -- 172.21.15.133:0/4170100933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 msgr2=0x564b193bada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2dceeed740 1 --2- 172.21.15.133:0/4170100933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bada0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2db0010810 tx=0x7f2db0010520 comp rx=0 tx=0).stop 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2dceeed740 1 -- 172.21.15.133:0/4170100933 shutdown_connections 2024-09-11T23:36:39.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2dceeed740 1 --2- 172.21.15.133:0/4170100933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193bc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2dceeed740 1 --2- 172.21.15.133:0/4170100933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193bbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2dceeed740 1 --2- 172.21.15.133:0/4170100933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193bada0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2dceeed740 1 -- 172.21.15.133:0/4170100933 >> 172.21.15.133:0/4170100933 conn(0x564b193aef60 msgr2=0x564b193b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2dceeed740 1 -- 172.21.15.133:0/4170100933 shutdown_connections 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.145+0000 7f2dceeed740 1 -- 172.21.15.133:0/4170100933 wait complete. 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dceeed740 1 -- start start 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dceeed740 1 Processor -- start 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193d84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dceeed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dceeed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564b193cfb10 con 0x564b193b29d0 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dceeed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b193cf440 con 0x564b193bdbb0 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dceeed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564b193d01e0 con 0x564b193b32f0 2024-09-11T23:36:39.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dceeed740 10 client.0 _ll_register_callbacks cb 0x7f2db000b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dc659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193d84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dc659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193d84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39280/0 (socket says 172.21.15.133:39280) 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dc659c640 1 -- 172.21.15.133:0/1714674859 learned_addr learned my addr 172.21.15.133:0/1714674859 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dc6d9d640 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dc659c640 1 -- 172.21.15.133:0/1714674859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 msgr2=0x564b193d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dc659c640 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dc659c640 1 -- 172.21.15.133:0/1714674859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 msgr2=0x564b193d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dc659c640 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.146+0000 7f2dc659c640 1 -- 172.21.15.133:0/1714674859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b193d23d0 con 0x564b193b29d0 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dc659c640 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193d84d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2db000bd40 tx=0x7f2db0018a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da026ace415dfb4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dadffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dadffb640 1 -- 172.21.15.133:0/1714674859 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db003a020 con 0x564b193b29d0 2024-09-11T23:36:39.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dadffb640 1 -- 172.21.15.133:0/1714674859 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f2db0018630 con 0x564b193b29d0 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dceeed740 1 -- 172.21.15.133:0/1714674859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db0010020 con 0x564b193b29d0 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dadffb640 1 -- 172.21.15.133:0/1714674859 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2db0027030 con 0x564b193b29d0 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dceeed740 10 client.15213 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dceeed740 1 -- 172.21.15.133:0/1714674859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b193d6d90 con 0x564b193b29d0 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dadffb640 1 -- 172.21.15.133:0/1714674859 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db0010020 con 0x564b193b29d0 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dadffb640 1 -- 172.21.15.133:0/1714674859 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2db002a070 con 0x564b193b29d0 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dceeed740 10 client.15213 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dceeed740 10 client.15213 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dceeed740 20 client.15213 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dceeed740 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x564b1943d5f0 0x564b1943c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dceeed740 4 client.15213 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.147+0000 7f2dceeed740 1 -- 172.21.15.133:0/1714674859 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "status"}) -- 0x564b19320b10 con 0x564b1943d5f0 2024-09-11T23:36:39.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x564b1943d5f0 0x564b1943c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:39.168 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:39.168 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:36:39.168 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 1, 2024-09-11T23:36:39.168 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24472, 2024-09-11T23:36:39.168 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:36:39.168 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:36:39.168 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:36:39.168 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 97.679300788999996, 2024-09-11T23:36:39.168 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 27, 2024-09-11T23:36:39.169 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 81, 2024-09-11T23:36:39.169 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 79, 2024-09-11T23:36:39.169 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 129.436506205 2024-09-11T23:36:39.169 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:39.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dc5d9b640 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x564b1943d5f0 0x564b1943c300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6230abb7c9f23681 server_cookie=c0799b965dc81147 in_seq=0 out_seq=0 2024-09-11T23:36:39.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dadffb640 10 client.15213 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:36:39.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dadffb640 1 -- 172.21.15.133:0/1714674859 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564b19320b10 con 0x564b1943d5f0 2024-09-11T23:36:39.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dadffb640 10 client.15213 handle_command_reply: tid=0 2024-09-11T23:36:39.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 1 client.15213 shutdown 2024-09-11T23:36:39.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 1 -- 172.21.15.133:0/1714674859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 msgr2=0x564b193d84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193d84d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2db000bd40 tx=0x7f2db0018a40 comp rx=0 tx=0).stop 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 1 -- 172.21.15.133:0/1714674859 shutdown_connections 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x564b1943d5f0 0x564b1943c300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b193bdbb0 0x564b193d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564b193b32f0 0x564b193d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 1 --2- 172.21.15.133:0/1714674859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b193b29d0 0x564b193d84d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dadffb640 0 client.15213 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 1 -- 172.21.15.133:0/1714674859 >> 172.21.15.133:0/1714674859 conn(0x564b193aef60 msgr2=0x564b193ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 1 -- 172.21.15.133:0/1714674859 shutdown_connections 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 1 -- 172.21.15.133:0/1714674859 wait complete. 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.148+0000 7f2dceeed740 20 client.15213 trim_cache size 0 max 16384 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 -- 172.21.15.133:0/3562343259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2db40a2b80 msgr2=0x7f2db40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2db40a2b80 0x7f2db40a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0015c10 tx=0x7f2dc001a000 comp rx=0 tx=0).stop 2024-09-11T23:36:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 -- 172.21.15.133:0/3562343259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc8101420 msgr2=0x7f2dc8198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:39.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc8101420 0x7f2dc8198a10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009a50 tx=0x7f2db8004270 comp rx=0 tx=0).stop 2024-09-11T23:36:39.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 -- 172.21.15.133:0/3562343259 shutdown_connections 2024-09-11T23:36:39.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f2db40a2b80 0x7f2db40a5040 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dc8106a80 0x7f2dc819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc8101420 0x7f2dc8198a10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 --2- 172.21.15.133:0/3562343259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc8100a80 0x7f2dc81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:39.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 -- 172.21.15.133:0/3562343259 >> 172.21.15.133:0/3562343259 conn(0x7f2dc80fc7a0 msgr2=0x7f2dc806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:39.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 -- 172.21.15.133:0/3562343259 shutdown_connections 2024-09-11T23:36:39.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:39.149+0000 7f2dccb14640 1 -- 172.21.15.133:0/3562343259 wait complete. 2024-09-11T23:36:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:38 smithi178 ceph-mon[35882]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:39.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:38 smithi138 ceph-mon[35142]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:36:39.870 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:36:40.117 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:40.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:39 smithi133 ceph-mon[31430]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-09-11T23:36:40.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:39 smithi178 ceph-mon[35882]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-09-11T23:36:40.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:39 smithi138 ceph-mon[35142]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-09-11T23:36:40.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 -- 172.21.15.133:0/821584314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be8101df0 msgr2=0x7f7be81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 --2- 172.21.15.133:0/821584314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be8101df0 0x7f7be81021d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc009900 tx=0x7f7bdc02a410 comp rx=0 tx=0).stop 2024-09-11T23:36:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 -- 172.21.15.133:0/821584314 shutdown_connections 2024-09-11T23:36:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 --2- 172.21.15.133:0/821584314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7be8106f20 0x7f7be810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 --2- 172.21.15.133:0/821584314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7be81027a0 0x7f7be81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 --2- 172.21.15.133:0/821584314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be8101df0 0x7f7be81021d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 -- 172.21.15.133:0/821584314 >> 172.21.15.133:0/821584314 conn(0x7f7be80fd8b0 msgr2=0x7f7be80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 -- 172.21.15.133:0/821584314 shutdown_connections 2024-09-11T23:36:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 -- 172.21.15.133:0/821584314 wait complete. 2024-09-11T23:36:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 Processor -- start 2024-09-11T23:36:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 -- start start 2024-09-11T23:36:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be8101df0 0x7f7be8198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7be81027a0 0x7f7be8198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7be8106f20 0x7f7be819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.623+0000 7f7bef2aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7be8112020 con 0x7f7be8101df0 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bef2aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7be8111ea0 con 0x7f7be8106f20 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bef2aa640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7be81121a0 con 0x7f7be81027a0 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bed046640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be8101df0 0x7f7be8198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bed046640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be8101df0 0x7f7be8198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39292/0 (socket says 172.21.15.133:39292) 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bed046640 1 -- 172.21.15.133:0/4173161630 learned_addr learned my addr 172.21.15.133:0/4173161630 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bed046640 1 -- 172.21.15.133:0/4173161630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7be81027a0 msgr2=0x7f7be8198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bed046640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7be81027a0 0x7f7be8198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bed046640 1 -- 172.21.15.133:0/4173161630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7be8106f20 msgr2=0x7f7be819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bed046640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7be8106f20 0x7f7be819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bed046640 1 -- 172.21.15.133:0/4173161630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be819d630 con 0x7f7be8101df0 2024-09-11T23:36:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bed046640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be8101df0 0x7f7be8198680 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc005ab0 tx=0x7f7bdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:40.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bda7fc640 1 -- 172.21.15.133:0/4173161630 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc03f030 con 0x7f7be8101df0 2024-09-11T23:36:40.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bef2aa640 1 -- 172.21.15.133:0/4173161630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be819d8c0 con 0x7f7be8101df0 2024-09-11T23:36:40.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bda7fc640 1 -- 172.21.15.133:0/4173161630 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bdc003ce0 con 0x7f7be8101df0 2024-09-11T23:36:40.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bda7fc640 1 -- 172.21.15.133:0/4173161630 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc035070 con 0x7f7be8101df0 2024-09-11T23:36:40.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.624+0000 7f7bef2aa640 1 -- 172.21.15.133:0/4173161630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be81a5200 con 0x7f7be8101df0 2024-09-11T23:36:40.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.627+0000 7f7bda7fc640 1 -- 172.21.15.133:0/4173161630 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7bdc042030 con 0x7f7be8101df0 2024-09-11T23:36:40.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bf1683740 1 Processor -- start 2024-09-11T23:36:40.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bf1683740 1 -- start start 2024-09-11T23:36:40.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352dda99d0 0x56352dda9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bda7fc640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7bd40a2b80 0x7f7bd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddaa2f0 0x56352ddb4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddb4bb0 0x56352ddb6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bf1683740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56352dd10090 con 0x56352ddaa2f0 2024-09-11T23:36:40.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bf1683740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56352dcb7a70 con 0x56352ddb4bb0 2024-09-11T23:36:40.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bf1683740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56352dd0c050 con 0x56352dda99d0 2024-09-11T23:36:40.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bec845640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddaa2f0 0x56352ddb4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bed046640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7bd40a2b80 0x7f7bd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bec845640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddaa2f0 0x56352ddb4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39300/0 (socket says 172.21.15.133:39300) 2024-09-11T23:36:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bec845640 1 -- 172.21.15.133:0/4045091703 learned_addr learned my addr 172.21.15.133:0/4045091703 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bed847640 1 --2- 172.21.15.133:0/4045091703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddb4bb0 0x56352ddb6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bed847640 1 --2- 172.21.15.133:0/4045091703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352dda99d0 0x56352dda9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.628+0000 7f7bda7fc640 1 -- 172.21.15.133:0/4173161630 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f7bdc0ef940 con 0x7f7be8101df0 2024-09-11T23:36:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.629+0000 7f7bec845640 1 -- 172.21.15.133:0/4045091703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352dda99d0 msgr2=0x56352dda9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.629+0000 7f7bec845640 1 --2- 172.21.15.133:0/4045091703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352dda99d0 0x56352dda9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.629+0000 7f7bec845640 1 -- 172.21.15.133:0/4045091703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddb4bb0 msgr2=0x56352ddb6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.629+0000 7f7bec845640 1 --2- 172.21.15.133:0/4045091703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddb4bb0 0x56352ddb6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.629+0000 7f7bec845640 1 -- 172.21.15.133:0/4045091703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56352dcc4130 con 0x56352ddaa2f0 2024-09-11T23:36:40.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.629+0000 7f7bed847640 1 --2- 172.21.15.133:0/4045091703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddb4bb0 0x56352ddb6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:40.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.629+0000 7f7bed046640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7bd40a2b80 0x7f7bd40a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0bc1d0 tx=0x7f7bdc08c0f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:40.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.629+0000 7f7bec845640 1 --2- 172.21.15.133:0/4045091703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddaa2f0 0x56352ddb4670 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7be40097f0 tx=0x7f7be4009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a134c07697c2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:40.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.629+0000 7f7bd37fe640 1 -- 172.21.15.133:0/4045091703 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4015020 con 0x56352ddaa2f0 2024-09-11T23:36:40.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.629+0000 7f7bd37fe640 1 -- 172.21.15.133:0/4045091703 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be4011420 con 0x56352ddaa2f0 2024-09-11T23:36:40.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.629+0000 7f7bd37fe640 1 -- 172.21.15.133:0/4045091703 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4010580 con 0x56352ddaa2f0 2024-09-11T23:36:40.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.630+0000 7f7bf1683740 1 -- 172.21.15.133:0/4045091703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddaa2f0 msgr2=0x56352ddb4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:40.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.630+0000 7f7bf1683740 1 --2- 172.21.15.133:0/4045091703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddaa2f0 0x56352ddb4670 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7be40097f0 tx=0x7f7be4009cb0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.630+0000 7f7bf1683740 1 -- 172.21.15.133:0/4045091703 shutdown_connections 2024-09-11T23:36:40.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.630+0000 7f7bf1683740 1 --2- 172.21.15.133:0/4045091703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddb4bb0 0x56352ddb6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.630+0000 7f7bf1683740 1 --2- 172.21.15.133:0/4045091703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddaa2f0 0x56352ddb4670 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.630+0000 7f7bf1683740 1 --2- 172.21.15.133:0/4045091703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352dda99d0 0x56352dda9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.630+0000 7f7bf1683740 1 -- 172.21.15.133:0/4045091703 >> 172.21.15.133:0/4045091703 conn(0x56352dda5f60 msgr2=0x56352dda6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:40.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.630+0000 7f7bf1683740 1 -- 172.21.15.133:0/4045091703 shutdown_connections 2024-09-11T23:36:40.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.630+0000 7f7bf1683740 1 -- 172.21.15.133:0/4045091703 wait complete. 2024-09-11T23:36:40.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.630+0000 7f7bf1683740 1 -- start start 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.630+0000 7f7bf1683740 1 Processor -- start 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddb43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddaa2f0 0x56352ddb1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddb4bb0 0x56352ddb1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bf1683740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56352ddb7780 con 0x56352dda99d0 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bf1683740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56352dccaf20 con 0x56352ddaa2f0 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bf1683740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56352ddb7900 con 0x56352ddb4bb0 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bf1683740 10 client.0 _ll_register_callbacks cb 0x56352dc94030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bed847640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddb43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bed847640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddb43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39314/0 (socket says 172.21.15.133:39314) 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bed847640 1 -- 172.21.15.133:0/519931808 learned_addr learned my addr 172.21.15.133:0/519931808 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bec845640 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddaa2f0 0x56352ddb1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bed847640 1 -- 172.21.15.133:0/519931808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddb4bb0 msgr2=0x56352ddb1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bed847640 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddb4bb0 0x56352ddb1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bed847640 1 -- 172.21.15.133:0/519931808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddaa2f0 msgr2=0x56352ddb1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bed847640 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddaa2f0 0x56352ddb1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bed847640 1 -- 172.21.15.133:0/519931808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56352da812b0 con 0x56352dda99d0 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bec845640 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddaa2f0 0x56352ddb1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bed847640 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddb43f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7be0019fc0 tx=0x7f7be000f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b576d4024529fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/519931808 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0012040 con 0x56352dda99d0 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/519931808 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f7be0017ae0 con 0x56352dda99d0 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.631+0000 7f7bf1683740 1 -- 172.21.15.133:0/519931808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be4011420 con 0x56352dda99d0 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/519931808 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7be000fe60 con 0x56352dda99d0 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bf1683740 10 client.15225 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bf1683740 1 -- 172.21.15.133:0/519931808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56352ddc3040 con 0x56352dda99d0 2024-09-11T23:36:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/519931808 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be4011420 con 0x56352dda99d0 2024-09-11T23:36:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/519931808 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7be0007430 con 0x56352dda99d0 2024-09-11T23:36:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bf1683740 10 client.15225 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bf1683740 10 client.15225 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:36:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bf1683740 20 client.15225 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bf1683740 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56352ddc56f0 0x56352ddc6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bf1683740 4 client.15225 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bf1683740 1 -- 172.21.15.133:0/519931808 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56352dd1d600 con 0x56352ddc56f0 2024-09-11T23:36:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.632+0000 7f7bec845640 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56352ddc56f0 0x56352ddc6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.633+0000 7f7bec845640 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56352ddc56f0 0x56352ddc6b60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7edcb6319d3e198 server_cookie=c548cebfc8db1cb5 in_seq=0 out_seq=0 2024-09-11T23:36:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.633+0000 7f7bd0ff9640 10 client.15225 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.633+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/519931808 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56352dd1d600 con 0x56352ddc56f0 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.633+0000 7f7bd0ff9640 10 client.15225 handle_command_reply: tid=0 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.633+0000 7f7bf1683740 1 client.15225 shutdown 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bf1683740 1 -- 172.21.15.133:0/519931808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 msgr2=0x56352ddb43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bf1683740 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddb43f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7be0019fc0 tx=0x7f7be000f5e0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bf1683740 1 -- 172.21.15.133:0/519931808 shutdown_connections 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bf1683740 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56352ddc56f0 0x56352ddc6b60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bf1683740 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddb4bb0 0x56352ddb1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bf1683740 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddaa2f0 0x56352ddb1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bd0ff9640 0 client.15225 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bf1683740 1 --2- 172.21.15.133:0/519931808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddb43f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bf1683740 1 -- 172.21.15.133:0/519931808 >> 172.21.15.133:0/519931808 conn(0x56352dda5f60 msgr2=0x56352ddabc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:40.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bf1683740 1 -- 172.21.15.133:0/519931808 shutdown_connections 2024-09-11T23:36:40.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bf1683740 1 -- 172.21.15.133:0/519931808 wait complete. 2024-09-11T23:36:40.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.634+0000 7f7bf1683740 20 client.15225 trim_cache size 0 max 16384 2024-09-11T23:36:40.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.651+0000 7f7bf1683740 1 Processor -- start 2024-09-11T23:36:40.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bf1683740 1 -- start start 2024-09-11T23:36:40.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352dda99d0 0x56352ddc5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddaa2f0 0x56352ddb1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddb4bb0 0x56352ddb0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bf1683740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56352ddc4ea0 con 0x56352ddb4bb0 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bf1683740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56352ddc5570 con 0x56352dda99d0 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bf1683740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56352ddc5c40 con 0x56352ddaa2f0 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bed847640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddb4bb0 0x56352ddb0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bed847640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddb4bb0 0x56352ddb0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39324/0 (socket says 172.21.15.133:39324) 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bed847640 1 -- 172.21.15.133:0/2568324403 learned_addr learned my addr 172.21.15.133:0/2568324403 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bed847640 1 --2- 172.21.15.133:0/2568324403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352dda99d0 0x56352ddc5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bec845640 1 --2- 172.21.15.133:0/2568324403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddaa2f0 0x56352ddb1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bed847640 1 -- 172.21.15.133:0/2568324403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddaa2f0 msgr2=0x56352ddb1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bed847640 1 --2- 172.21.15.133:0/2568324403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddaa2f0 0x56352ddb1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bed847640 1 -- 172.21.15.133:0/2568324403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352dda99d0 msgr2=0x56352ddc5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bed847640 1 --2- 172.21.15.133:0/2568324403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352dda99d0 0x56352ddc5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bed847640 1 -- 172.21.15.133:0/2568324403 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56352ddaa120 con 0x56352ddb4bb0 2024-09-11T23:36:40.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.652+0000 7f7bed847640 1 --2- 172.21.15.133:0/2568324403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddb4bb0 0x56352ddb0010 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7be0001630 tx=0x7f7be0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54c366167de63d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:40.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bd2ffd640 1 -- 172.21.15.133:0/2568324403 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0036070 con 0x56352ddb4bb0 2024-09-11T23:36:40.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bd2ffd640 1 -- 172.21.15.133:0/2568324403 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be001e070 con 0x56352ddb4bb0 2024-09-11T23:36:40.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bd2ffd640 1 -- 172.21.15.133:0/2568324403 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be00225d0 con 0x56352ddb4bb0 2024-09-11T23:36:40.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bf1683740 1 -- 172.21.15.133:0/2568324403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddb4bb0 msgr2=0x56352ddb0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bf1683740 1 --2- 172.21.15.133:0/2568324403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddb4bb0 0x56352ddb0010 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7be0001630 tx=0x7f7be0000f40 comp rx=0 tx=0).stop 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bf1683740 1 -- 172.21.15.133:0/2568324403 shutdown_connections 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bf1683740 1 --2- 172.21.15.133:0/2568324403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352ddb4bb0 0x56352ddb0010 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bf1683740 1 --2- 172.21.15.133:0/2568324403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddaa2f0 0x56352ddb1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bf1683740 1 --2- 172.21.15.133:0/2568324403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352dda99d0 0x56352ddc5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bf1683740 1 -- 172.21.15.133:0/2568324403 >> 172.21.15.133:0/2568324403 conn(0x56352dda5f60 msgr2=0x56352dda6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bf1683740 1 -- 172.21.15.133:0/2568324403 shutdown_connections 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.653+0000 7f7bf1683740 1 -- 172.21.15.133:0/2568324403 wait complete. 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bf1683740 1 -- start start 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bf1683740 1 Processor -- start 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddca490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddaa2f0 0x56352ddca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bf1683740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddb4bb0 0x56352de27e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bf1683740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56352ddc69e0 con 0x56352dda99d0 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bf1683740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56352ddc6310 con 0x56352ddb4bb0 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bf1683740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56352ddc70b0 con 0x56352ddaa2f0 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bed847640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bf1683740 10 client.0 _ll_register_callbacks cb 0x7f7be0002140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bed847640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39334/0 (socket says 172.21.15.133:39334) 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bed847640 1 -- 172.21.15.133:0/1930661472 learned_addr learned my addr 172.21.15.133:0/1930661472 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.654+0000 7f7bec845640 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddaa2f0 0x56352ddca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bed847640 1 -- 172.21.15.133:0/1930661472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddaa2f0 msgr2=0x56352ddca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bed847640 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddaa2f0 0x56352ddca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bed847640 1 -- 172.21.15.133:0/1930661472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddb4bb0 msgr2=0x56352de27e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bed847640 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddb4bb0 0x56352de27e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bed847640 1 -- 172.21.15.133:0/1930661472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56352de284a0 con 0x56352dda99d0 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bed847640 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddca490 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7be000b040 tx=0x7f7be0019cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f04fa5a8390a64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/1930661472 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0036030 con 0x56352dda99d0 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/1930661472 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f7be0007160 con 0x56352dda99d0 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bf1683740 1 -- 172.21.15.133:0/1930661472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be001e070 con 0x56352dda99d0 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/1930661472 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7be00011c0 con 0x56352dda99d0 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bf1683740 10 client.15231 fetch_fsmap learned FSMap version 27 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bf1683740 1 -- 172.21.15.133:0/1930661472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56352de2f820 con 0x56352dda99d0 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/1930661472 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be001e070 con 0x56352dda99d0 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/1930661472 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7be0017b40 con 0x56352dda99d0 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bf1683740 10 client.15231 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bf1683740 10 client.15231 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:36:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bf1683740 20 client.15231 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bf1683740 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56352de33fa0 0x56352de32cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bf1683740 4 client.15231 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.655+0000 7f7bf1683740 1 -- 172.21.15.133:0/1930661472 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "status"}) -- 0x56352dd17b10 con 0x56352de33fa0 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bec845640 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56352de33fa0 0x56352de32cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bd0ff9640 10 client.15231 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bec845640 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56352de33fa0 0x56352de32cb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b50df2827b7344 server_cookie=f92c7e1d4b695ec1 in_seq=0 out_seq=0 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bd0ff9640 1 -- 172.21.15.133:0/1930661472 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56352dd17b10 con 0x56352de33fa0 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bd0ff9640 10 client.15231 handle_command_reply: tid=0 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bf1683740 1 client.15231 shutdown 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bf1683740 1 -- 172.21.15.133:0/1930661472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 msgr2=0x56352ddca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bf1683740 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddca490 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7be000b040 tx=0x7f7be0019cd0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bf1683740 1 -- 172.21.15.133:0/1930661472 shutdown_connections 2024-09-11T23:36:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bf1683740 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56352de33fa0 0x56352de32cb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bf1683740 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352ddb4bb0 0x56352de27e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bf1683740 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56352ddaa2f0 0x56352ddca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bd0ff9640 0 client.15231 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bf1683740 1 --2- 172.21.15.133:0/1930661472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352dda99d0 0x56352ddca490 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.656+0000 7f7bf1683740 1 -- 172.21.15.133:0/1930661472 >> 172.21.15.133:0/1930661472 conn(0x56352dda5f60 msgr2=0x56352ddafea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bf1683740 1 -- 172.21.15.133:0/1930661472 shutdown_connections 2024-09-11T23:36:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bf1683740 1 -- 172.21.15.133:0/1930661472 wait complete. 2024-09-11T23:36:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bf1683740 20 client.15231 trim_cache size 0 max 16384 2024-09-11T23:36:40.662 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:40.662 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:36:40.662 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-09-11T23:36:40.663 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24481, 2024-09-11T23:36:40.663 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:36:40.663 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:36:40.663 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:36:40.663 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 106.215991868, 2024-09-11T23:36:40.663 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 27, 2024-09-11T23:36:40.663 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 81, 2024-09-11T23:36:40.663 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 79, 2024-09-11T23:36:40.663 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 125.892553252 2024-09-11T23:36:40.663 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 -- 172.21.15.133:0/4173161630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7bd40a2b80 msgr2=0x7f7bd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7bd40a2b80 0x7f7bd40a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0bc1d0 tx=0x7f7bdc08c0f0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 -- 172.21.15.133:0/4173161630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be8101df0 msgr2=0x7f7be8198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be8101df0 0x7f7be8198680 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc005ab0 tx=0x7f7bdc0042f0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 -- 172.21.15.133:0/4173161630 shutdown_connections 2024-09-11T23:36:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f7bd40a2b80 0x7f7bd40a5040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7be8106f20 0x7f7be819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7be81027a0 0x7f7be8198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 --2- 172.21.15.133:0/4173161630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be8101df0 0x7f7be8198680 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 -- 172.21.15.133:0/4173161630 >> 172.21.15.133:0/4173161630 conn(0x7f7be80fd8b0 msgr2=0x7f7be80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:40.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 -- 172.21.15.133:0/4173161630 shutdown_connections 2024-09-11T23:36:40.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:40.657+0000 7f7bef2aa640 1 -- 172.21.15.133:0/4173161630 wait complete. 2024-09-11T23:36:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:40 smithi178 ceph-mon[35882]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-11T23:36:41.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:40 smithi133 ceph-mon[31430]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-11T23:36:41.348 INFO:teuthology.run_tasks:Running task sequential... 2024-09-11T23:36:41.358 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-11T23:36:41.358 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-11T23:36:41.359 INFO:teuthology.task.print:Enabling mgr modules 2024-09-11T23:36:41.359 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-11T23:36:41.360 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:36:41.360 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T23:36:41.360 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-11T23:36:41.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:40 smithi138 ceph-mon[35142]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-11T23:36:41.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.462+0000 7f4cb894b640 1 -- 172.21.15.133:0/2554600735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb41013d0 msgr2=0x7f4cb4101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.462+0000 7f4cb894b640 1 --2- 172.21.15.133:0/2554600735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb41013d0 0x7f4cb4101850 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009ca0 tx=0x7f4ca802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:36:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.462+0000 7f4cb894b640 1 -- 172.21.15.133:0/2554600735 shutdown_connections 2024-09-11T23:36:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.462+0000 7f4cb894b640 1 --2- 172.21.15.133:0/2554600735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb4101d90 0x7f4cb4108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.462+0000 7f4cb894b640 1 --2- 172.21.15.133:0/2554600735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb41013d0 0x7f4cb4101850 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.462+0000 7f4cb894b640 1 --2- 172.21.15.133:0/2554600735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cb41001e0 0x7f4cb41005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.462+0000 7f4cb894b640 1 -- 172.21.15.133:0/2554600735 >> 172.21.15.133:0/2554600735 conn(0x7f4cb40fb980 msgr2=0x7f4cb40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.462+0000 7f4cb894b640 1 -- 172.21.15.133:0/2554600735 shutdown_connections 2024-09-11T23:36:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.462+0000 7f4cb894b640 1 -- 172.21.15.133:0/2554600735 wait complete. 2024-09-11T23:36:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb894b640 1 Processor -- start 2024-09-11T23:36:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb894b640 1 -- start start 2024-09-11T23:36:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb894b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb41001e0 0x7f4cb4196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb894b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cb41013d0 0x7f4cb4197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb894b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb4101d90 0x7f4cb419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb894b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb410bee0 con 0x7f4cb4101d90 2024-09-11T23:36:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb894b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb410bd60 con 0x7f4cb41001e0 2024-09-11T23:36:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb894b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb410c060 con 0x7f4cb41013d0 2024-09-11T23:36:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb4101d90 0x7f4cb419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb4101d90 0x7f4cb419e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39354/0 (socket says 172.21.15.133:39354) 2024-09-11T23:36:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb2d9d640 1 -- 172.21.15.133:0/343825972 learned_addr learned my addr 172.21.15.133:0/343825972 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb1d9b640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cb41013d0 0x7f4cb4197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb259c640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb41001e0 0x7f4cb4196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb2d9d640 1 -- 172.21.15.133:0/343825972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cb41013d0 msgr2=0x7f4cb4197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb2d9d640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cb41013d0 0x7f4cb4197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.463+0000 7f4cb2d9d640 1 -- 172.21.15.133:0/343825972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb41001e0 msgr2=0x7f4cb4196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.464+0000 7f4cb2d9d640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb41001e0 0x7f4cb4196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.464+0000 7f4cb2d9d640 1 -- 172.21.15.133:0/343825972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb419e9e0 con 0x7f4cb4101d90 2024-09-11T23:36:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.464+0000 7f4cb2d9d640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb4101d90 0x7f4cb419e370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009fd0 tx=0x7f4ca400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:41.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.464+0000 7f4ca37fe640 1 -- 172.21.15.133:0/343825972 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4011830 con 0x7f4cb4101d90 2024-09-11T23:36:41.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.464+0000 7f4ca37fe640 1 -- 172.21.15.133:0/343825972 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ca4016070 con 0x7f4cb4101d90 2024-09-11T23:36:41.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.464+0000 7f4cb894b640 1 -- 172.21.15.133:0/343825972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb419ecd0 con 0x7f4cb4101d90 2024-09-11T23:36:41.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.464+0000 7f4ca37fe640 1 -- 172.21.15.133:0/343825972 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca400f2e0 con 0x7f4cb4101d90 2024-09-11T23:36:41.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.464+0000 7f4cb894b640 1 -- 172.21.15.133:0/343825972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb419f1d0 con 0x7f4cb4101d90 2024-09-11T23:36:41.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.465+0000 7f4cb894b640 1 -- 172.21.15.133:0/343825972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb4107890 con 0x7f4cb4101d90 2024-09-11T23:36:41.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.466+0000 7f4ca37fe640 1 -- 172.21.15.133:0/343825972 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4ca40119d0 con 0x7f4cb4101d90 2024-09-11T23:36:41.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.466+0000 7f4ca37fe640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c9c0a2c30 0x7f4c9c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:41.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.467+0000 7f4cb259c640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c9c0a2c30 0x7f4c9c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:41.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.467+0000 7f4ca37fe640 1 -- 172.21.15.133:0/343825972 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7f4ca40ce610 con 0x7f4cb4101d90 2024-09-11T23:36:41.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.467+0000 7f4cb259c640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c9c0a2c30 0x7f4c9c0a50f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4cac005fd0 tx=0x7f4cac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:41.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.471+0000 7f4ca37fe640 1 -- 172.21.15.133:0/343825972 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ca409b610 con 0x7f4cb4101d90 2024-09-11T23:36:41.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.687+0000 7f4cb894b640 1 -- 172.21.15.133:0/343825972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f4cb402d9c0 con 0x7f4cb4101d90 2024-09-11T23:36:41.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.997+0000 7f4ca37fe640 1 -- 172.21.15.133:0/343825972 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f4ca409af10 con 0x7f4cb4101d90 2024-09-11T23:36:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.999+0000 7f4cb894b640 1 -- 172.21.15.133:0/343825972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c9c0a2c30 msgr2=0x7f4c9c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.999+0000 7f4cb894b640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c9c0a2c30 0x7f4c9c0a50f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4cac005fd0 tx=0x7f4cac005850 comp rx=0 tx=0).stop 2024-09-11T23:36:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.999+0000 7f4cb894b640 1 -- 172.21.15.133:0/343825972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb4101d90 msgr2=0x7f4cb419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.999+0000 7f4cb894b640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb4101d90 0x7f4cb419e370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009fd0 tx=0x7f4ca400b8e0 comp rx=0 tx=0).stop 2024-09-11T23:36:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:41.999+0000 7f4cb894b640 1 -- 172.21.15.133:0/343825972 shutdown_connections 2024-09-11T23:36:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.000+0000 7f4cb894b640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7f4c9c0a2c30 0x7f4c9c0a50f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.000+0000 7f4cb894b640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb4101d90 0x7f4cb419e370 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.000+0000 7f4cb894b640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cb41013d0 0x7f4cb4197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.000+0000 7f4cb894b640 1 --2- 172.21.15.133:0/343825972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb41001e0 0x7f4cb4196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.000+0000 7f4cb894b640 1 -- 172.21.15.133:0/343825972 >> 172.21.15.133:0/343825972 conn(0x7f4cb40fb980 msgr2=0x7f4cb4104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:42.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.000+0000 7f4cb894b640 1 -- 172.21.15.133:0/343825972 shutdown_connections 2024-09-11T23:36:42.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.000+0000 7f4cb894b640 1 -- 172.21.15.133:0/343825972 wait complete. 2024-09-11T23:36:42.018 INFO:teuthology.run_tasks:Running task kclient... 2024-09-11T23:36:42.043 INFO:tasks.kclient:Mounting kernel clients... 2024-09-11T23:36:42.043 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-11T23:36:42.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:36:42.044 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-09-11T23:36:42.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:36:42.060 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-09-11T23:36:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:Cannot find device "ceph-brx" 2024-09-11T23:36:42.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:36:42.130 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-11T23:36:42.130 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T23:36:42.131 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T23:36:42.131 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T23:36:42.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:41 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/343825972' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:36:42.290 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: ignoring --setuser ceph since I am not root 2024-09-11T23:36:42.290 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: ignoring --setgroup ceph since I am not root 2024-09-11T23:36:42.290 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:42 smithi178 ceph-mgr[37694]: -- 172.21.15.178:0/3509452574 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556162f8b0e0 con 0x556162f69000 2024-09-11T23:36:42.290 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:42.149+0000 7f07f309a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:36:42.290 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:42.205+0000 7f07f309a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:36:42.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:41 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/343825972' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:36:42.357 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:42 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: ignoring --setuser ceph since I am not root 2024-09-11T23:36:42.357 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:42 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: ignoring --setgroup ceph since I am not root 2024-09-11T23:36:42.357 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:42 smithi138 ceph-mgr[36517]: -- 172.21.15.138:0/2056046634 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5631ef8a30e0 con 0x5631ef880800 2024-09-11T23:36:42.357 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:42 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:42.152+0000 7fedcb7c0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:36:42.357 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:42 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:42.209+0000 7fedcb7c0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:36:42.358 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:42 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:42.357+0000 7fedcb7c0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:36:42.371 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:41 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/343825972' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:36:42.372 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: ignoring --setuser ceph since I am not root 2024-09-11T23:36:42.372 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: ignoring --setgroup ceph since I am not root 2024-09-11T23:36:42.372 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:42.155+0000 7f05d0c0c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:36:42.372 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:42.215+0000 7f05d0c0c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:36:42.372 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:42.370+0000 7f05d0c0c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:36:42.425 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:42.698 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:42 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:42.423+0000 7fedcb7c0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:36:42.698 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:42 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:42.571+0000 7fedcb7c0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:36:42.703 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:42.352+0000 7f07f309a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:36:42.703 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:42.417+0000 7f07f309a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:36:42.703 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:42.562+0000 7f07f309a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:36:42.706 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:42.445+0000 7f05d0c0c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:36:42.706 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:42.598+0000 7f05d0c0c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:36:42.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.916+0000 7fb773327640 1 -- 172.21.15.133:0/3111096212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76c101420 msgr2=0x7fb76c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:42.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.916+0000 7fb773327640 1 --2- 172.21.15.133:0/3111096212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76c101420 0x7fb76c10d950 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb764009cd0 tx=0x7fb76402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:36:42.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.916+0000 7fb773327640 1 -- 172.21.15.133:0/3111096212 shutdown_connections 2024-09-11T23:36:42.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.916+0000 7fb773327640 1 --2- 172.21.15.133:0/3111096212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76c101420 0x7fb76c10d950 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:42.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.916+0000 7fb773327640 1 --2- 172.21.15.133:0/3111096212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb76c100a80 0x7fb76c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:42.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.916+0000 7fb773327640 1 --2- 172.21.15.133:0/3111096212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb76c106a80 0x7fb76c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:42.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.916+0000 7fb773327640 1 -- 172.21.15.133:0/3111096212 >> 172.21.15.133:0/3111096212 conn(0x7fb76c0fc7c0 msgr2=0x7fb76c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:42.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.916+0000 7fb773327640 1 -- 172.21.15.133:0/3111096212 shutdown_connections 2024-09-11T23:36:42.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.916+0000 7fb773327640 1 -- 172.21.15.133:0/3111096212 wait complete. 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.917+0000 7fb773327640 1 Processor -- start 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.917+0000 7fb773327640 1 -- start start 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.917+0000 7fb773327640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76c100a80 0x7fb76c197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.917+0000 7fb773327640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb76c101420 0x7fb76c1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.917+0000 7fb773327640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb76c106a80 0x7fb76c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.917+0000 7fb773327640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c110e80 con 0x7fb76c100a80 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.917+0000 7fb773327640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c110d00 con 0x7fb76c106a80 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.917+0000 7fb773327640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c111000 con 0x7fb76c101420 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7710c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76c100a80 0x7fb76c197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7710c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76c100a80 0x7fb76c197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60528/0 (socket says 172.21.15.133:60528) 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7710c3640 1 -- 172.21.15.133:0/709238128 learned_addr learned my addr 172.21.15.133:0/709238128 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7708c2640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb76c101420 0x7fb76c1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7718c4640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb76c106a80 0x7fb76c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7710c3640 1 -- 172.21.15.133:0/709238128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb76c101420 msgr2=0x7fb76c1978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7710c3640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb76c101420 0x7fb76c1978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7710c3640 1 -- 172.21.15.133:0/709238128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb76c106a80 msgr2=0x7fb76c19bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7710c3640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb76c106a80 0x7fb76c19bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7710c3640 1 -- 172.21.15.133:0/709238128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c19c310 con 0x7fb76c100a80 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7710c3640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76c100a80 0x7fb76c197360 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e9f0 tx=0x7fb76800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7627fc640 1 -- 172.21.15.133:0/709238128 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76800cd80 con 0x7fb76c100a80 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7627fc640 1 -- 172.21.15.133:0/709238128 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb768010910 con 0x7fb76c100a80 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb773327640 1 -- 172.21.15.133:0/709238128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76c19c600 con 0x7fb76c100a80 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb7627fc640 1 -- 172.21.15.133:0/709238128 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768018990 con 0x7fb76c100a80 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.918+0000 7fb773327640 1 -- 172.21.15.133:0/709238128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76c06a790 con 0x7fb76c100a80 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.920+0000 7fb773327640 1 -- 172.21.15.133:0/709238128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76c1021c0 con 0x7fb76c100a80 2024-09-11T23:36:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.921+0000 7fb7627fc640 1 -- 172.21.15.133:0/709238128 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7fb768010ac0 con 0x7fb76c100a80 2024-09-11T23:36:42.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.921+0000 7fb7627fc640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb7580a2bd0 0x7fb7580a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:42.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.921+0000 7fb7708c2640 1 -- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb7580a2bd0 msgr2=0x7fb7580a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3324898762 2024-09-11T23:36:42.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.921+0000 7fb7627fc640 1 -- 172.21.15.133:0/709238128 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fb7680cf290 con 0x7fb76c100a80 2024-09-11T23:36:42.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.921+0000 7fb7708c2640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb7580a2bd0 0x7fb7580a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:36:42.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:42.926+0000 7fb7627fc640 1 -- 172.21.15.133:0/709238128 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb76809bc40 con 0x7fb76c100a80 2024-09-11T23:36:42.959 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:42.744+0000 7f05d0c0c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:36:42.997 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:42.702+0000 7f07f309a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:36:42.998 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:42 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:42.712+0000 7fedcb7c0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:36:43.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.122+0000 7fb7708c2640 1 -- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb7580a2bd0 msgr2=0x7fb7580a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3324898762 2024-09-11T23:36:43.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.122+0000 7fb7708c2640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb7580a2bd0 0x7fb7580a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:36:43.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.136+0000 7fb773327640 1 -- 172.21.15.133:0/709238128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb76c06b1b0 con 0x7fb76c100a80 2024-09-11T23:36:43.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.136+0000 7fb7627fc640 1 -- 172.21.15.133:0/709238128 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fb76809ff40 con 0x7fb76c100a80 2024-09-11T23:36:43.136 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T23:36:43.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.139+0000 7fb773327640 1 -- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb7580a2bd0 msgr2=0x7fb7580a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:36:43.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.139+0000 7fb773327640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb7580a2bd0 0x7fb7580a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:43.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.139+0000 7fb773327640 1 -- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76c100a80 msgr2=0x7fb76c197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:43.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.139+0000 7fb773327640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76c100a80 0x7fb76c197360 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e9f0 tx=0x7fb76800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:36:43.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.140+0000 7fb773327640 1 -- 172.21.15.133:0/709238128 shutdown_connections 2024-09-11T23:36:43.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.140+0000 7fb773327640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fb7580a2bd0 0x7fb7580a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:43.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.140+0000 7fb773327640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb76c106a80 0x7fb76c19bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:43.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.140+0000 7fb773327640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb76c101420 0x7fb76c1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:43.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.140+0000 7fb773327640 1 --2- 172.21.15.133:0/709238128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76c100a80 0x7fb76c197360 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:43.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.140+0000 7fb773327640 1 -- 172.21.15.133:0/709238128 >> 172.21.15.133:0/709238128 conn(0x7fb76c0fc7c0 msgr2=0x7fb76c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:43.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.140+0000 7fb773327640 1 -- 172.21.15.133:0/709238128 shutdown_connections 2024-09-11T23:36:43.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:43.140+0000 7fb773327640 1 -- 172.21.15.133:0/709238128 wait complete. 2024-09-11T23:36:43.257 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:43.014+0000 7f07f309a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:36:43.258 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:43.073+0000 7f07f309a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:36:43.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:42 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/343825972' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:36:43.258 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:42 smithi178 ceph-mon[35882]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-11T23:36:43.274 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:42 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/343825972' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:36:43.275 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:42 smithi138 ceph-mon[35142]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-11T23:36:43.275 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:43 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:43.027+0000 7fedcb7c0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:36:43.275 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:43 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:43.088+0000 7fedcb7c0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:36:43.330 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:42 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/343825972' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:36:43.330 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:42 smithi133 ceph-mon[31430]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-11T23:36:43.330 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:43.071+0000 7f05d0c0c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:36:43.330 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:43.134+0000 7f05d0c0c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:36:43.539 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:43.257+0000 7f07f309a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:36:43.697 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:43 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:43.274+0000 7fedcb7c0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:36:43.709 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:43.330+0000 7f05d0c0c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:36:43.926 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T23:36:43.927 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T23:36:43.927 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-09-11T23:36:43.927 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T23:36:43.927 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T23:36:43.927 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T23:36:43.927 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T23:36:43.927 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T23:36:43.927 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T23:36:43.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:36:43.927 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-09-11T23:36:43.944 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:36:43.945 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:36:43.945 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T23:36:43.945 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:36:43.945 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-11T23:36:43.945 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42222sec preferred_lft 42222sec 2024-09-11T23:36:43.945 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link noprefixroute 2024-09-11T23:36:43.945 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:36:43.945 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T23:36:43.945 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:36:43.946 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-11T23:36:43.946 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2024-09-11T23:36:43.946 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2024-09-11T23:36:43.946 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2024-09-11T23:36:43.946 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T23:36:43.946 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-11T23:36:44.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:36:44.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:36:44.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:36:44.151 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T23:36:44.161 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:43 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/709238128' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:36:44.176 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-09-11T23:36:44.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:36:44.179 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-09-11T23:36:44.253 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-09-11T23:36:44.253 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T23:36:44.253 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T23:36:44.253 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T23:36:44.253 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T23:36:44.254 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:36:44.254 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-11T23:36:44.254 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T23:36:44.255 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T23:36:44.255 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T23:36:44.255 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-11T23:36:44.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:44 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/709238128' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:36:44.290 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:44.136+0000 7f07f309a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:36:44.290 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:44.204+0000 7f07f309a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:36:44.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:36:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:44 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/709238128' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:36:44.448 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:44 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:44.155+0000 7fedcb7c0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:36:44.448 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:44 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:44.222+0000 7fedcb7c0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:36:44.459 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:44 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:44.256+0000 7f05d0c0c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:36:44.459 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:44 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:44.326+0000 7f05d0c0c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:36:44.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:36:44.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:36:44.564 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-09-11T23:36:44.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:36:44.580 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-09-11T23:36:44.635 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:36:44.635 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-11T23:36:44.635 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T23:36:44.635 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-11T23:36:44.636 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-11T23:36:44.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:36:44.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:36:44.755 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-11T23:36:44.755 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:36:44.755 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-11T23:36:44.755 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-11T23:36:44.756 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T23:36:44.756 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-11T23:36:44.756 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-11T23:36:44.756 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-11T23:36:44.756 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-11T23:36:44.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:36:44.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:36:44.881 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:36:44.882 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-11T23:36:44.882 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-09-11T23:36:44.882 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T23:36:44.882 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-11T23:36:44.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:36:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:36:44.966 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T23:36:44.966 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T23:36:44.982 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T23:36:44.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T23:36:44.983 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T23:36:45.038 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-11T23:36:45.038 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-11T23:36:45.038 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/53fd07fa-2161-401a-8fee-11858bbc78a1 2024-09-11T23:36:45.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:36:45.038 DEBUG:teuthology.orchestra.run.smithi133:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/53fd07fa-2161-401a-8fee-11858bbc78a1 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-11T23:36:45.135 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-11T23:36:45.136 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-11T23:36:45.136 INFO:teuthology.orchestra.run.smithi133.stdout:invalid new device string format 2024-09-11T23:36:45.175 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:44.916+0000 7f07f309a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:36:45.175 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:45.032+0000 7f07f309a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:36:45.197 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:44 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:44.938+0000 7fedcb7c0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:36:45.198 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:45.055+0000 7fedcb7c0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:36:45.278 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-11T23:36:45.278 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-11T23:36:45.278 INFO:teuthology.orchestra.run.smithi133.stdout:invalid new device string format 2024-09-11T23:36:45.279 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: resolved to: "172.21.15.133:3300,172.21.15.138:3300,172.21.15.178:3300" 2024-09-11T23:36:45.279 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.133:3300,172.21.15.138:3300,172.21.15.178:3300:/volumes/qa/sv_1/53fd07fa-2161-401a-8fee-11858bbc78a1 2024-09-11T23:36:45.279 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=42321b88-7095-11ef-bcea-c7b262605968" will pass to kernel 2024-09-11T23:36:45.279 INFO:teuthology.orchestra.run.smithi133.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-11T23:36:45.279 INFO:teuthology.orchestra.run.smithi133.stdout: You just mounted a file system that supports labels which does not 2024-09-11T23:36:45.279 INFO:teuthology.orchestra.run.smithi133.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-11T23:36:45.279 INFO:teuthology.orchestra.run.smithi133.stdout: applications will generate AVC messages and not be allowed access to 2024-09-11T23:36:45.280 INFO:teuthology.orchestra.run.smithi133.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-11T23:36:45.281 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T23:36:45.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:36:45.281 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T23:36:45.311 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:36:45.363 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:45.089+0000 7f05d0c0c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:36:45.363 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:45.213+0000 7f05d0c0c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:36:45.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:36:45.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:36:45.409 INFO:teuthology.orchestra.run.smithi133.stdout:client24826 2024-09-11T23:36:45.410 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:36:45.410 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:36:45.455 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:45.175+0000 7f07f309a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:36:45.455 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:45.235+0000 7f07f309a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:36:45.455 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:45.386+0000 7f07f309a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:36:45.455 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:45.455+0000 7f07f309a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:36:45.467 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:36:45.478 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:45.198+0000 7fedcb7c0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:36:45.478 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:45.259+0000 7fedcb7c0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:36:45.478 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:45.410+0000 7fedcb7c0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:36:45.478 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:45.478+0000 7fedcb7c0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:36:45.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:36:45.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:36:45.565 INFO:teuthology.orchestra.run.smithi133.stdout:client24826 2024-09-11T23:36:45.565 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:36:45.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:36:45.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:36:45.623 INFO:teuthology.orchestra.run.smithi133.stdout:client24826 2024-09-11T23:36:45.624 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/42321b88-7095-11ef-bcea-c7b262605968.client24826/status') 2024-09-11T23:36:45.669 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:45.364+0000 7f05d0c0c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:36:45.669 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:45.429+0000 7f05d0c0c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:36:45.669 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:45.596+0000 7f05d0c0c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:36:45.670 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:45.669+0000 7f05d0c0c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:36:45.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:36:45.682 INFO:teuthology.orchestra.run.smithi133.stdout:instance: client.24826 (3)172.21.15.133:0/3542003278 2024-09-11T23:36:45.683 INFO:teuthology.orchestra.run.smithi133.stdout:blocklisted: false 2024-09-11T23:36:45.683 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records in 2024-09-11T23:36:45.683 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records out 2024-09-11T23:36:45.683 INFO:teuthology.orchestra.run.smithi133.stderr:72 bytes copied, 2.7648e-05 s, 2.6 MB/s 2024-09-11T23:36:45.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:36:45.690 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T23:36:45.701 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:36:45.701 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T23:36:45.702 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-11T23:36:45.742 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:45.516+0000 7f07f309a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:36:45.742 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:45.676+0000 7f07f309a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:36:45.742 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:45.742+0000 7f07f309a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:36:45.765 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:45.539+0000 7fedcb7c0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:36:45.765 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:45.699+0000 7fedcb7c0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:36:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.806+0000 7fcae597f640 1 -- 172.21.15.133:0/4249213882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae0100a50 msgr2=0x7fcae0100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.806+0000 7fcae597f640 1 --2- 172.21.15.133:0/4249213882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae0100a50 0x7fcae0100e50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009900 tx=0x7fcad002a410 comp rx=0 tx=0).stop 2024-09-11T23:36:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.806+0000 7fcae597f640 1 -- 172.21.15.133:0/4249213882 shutdown_connections 2024-09-11T23:36:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.806+0000 7fcae597f640 1 --2- 172.21.15.133:0/4249213882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcae0102600 0x7fcae0108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.806+0000 7fcae597f640 1 --2- 172.21.15.133:0/4249213882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcae0101c40 0x7fcae01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.806+0000 7fcae597f640 1 --2- 172.21.15.133:0/4249213882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae0100a50 0x7fcae0100e50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.806+0000 7fcae597f640 1 -- 172.21.15.133:0/4249213882 >> 172.21.15.133:0/4249213882 conn(0x7fcae00fc1b0 msgr2=0x7fcae00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.806+0000 7fcae597f640 1 -- 172.21.15.133:0/4249213882 shutdown_connections 2024-09-11T23:36:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.806+0000 7fcae597f640 1 -- 172.21.15.133:0/4249213882 wait complete. 2024-09-11T23:36:45.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcae597f640 1 Processor -- start 2024-09-11T23:36:45.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcae597f640 1 -- start start 2024-09-11T23:36:45.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcae597f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcae0100a50 0x7fcae0196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:45.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcae597f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae0101c40 0x7fcae0197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:45.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcae597f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcae0102600 0x7fcae019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:45.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcae597f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcae010c6b0 con 0x7fcae0101c40 2024-09-11T23:36:45.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcae597f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcae010c530 con 0x7fcae0100a50 2024-09-11T23:36:45.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcae597f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcae010c830 con 0x7fcae0102600 2024-09-11T23:36:45.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcade7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae0101c40 0x7fcae0197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:45.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcade7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae0101c40 0x7fcae0197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60554/0 (socket says 172.21.15.133:60554) 2024-09-11T23:36:45.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcadeffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcae0100a50 0x7fcae0196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcade7fc640 1 -- 172.21.15.133:0/252118739 learned_addr learned my addr 172.21.15.133:0/252118739 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.807+0000 7fcadf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcae0102600 0x7fcae019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcadeffd640 1 -- 172.21.15.133:0/252118739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcae0102600 msgr2=0x7fcae019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcadeffd640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcae0102600 0x7fcae019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcadeffd640 1 -- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae0101c40 msgr2=0x7fcae0197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcadeffd640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae0101c40 0x7fcae0197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcadeffd640 1 -- 172.21.15.133:0/252118739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae019e900 con 0x7fcae0100a50 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcadf7fe640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcae0102600 0x7fcae019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcade7fc640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae0101c40 0x7fcae0197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcadeffd640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcae0100a50 0x7fcae0196cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009230 tx=0x7fcad0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcae497d640 1 -- 172.21.15.133:0/252118739 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad003f070 con 0x7fcae0100a50 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcae597f640 1 -- 172.21.15.133:0/252118739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae006a2a0 con 0x7fcae0100a50 2024-09-11T23:36:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcae597f640 1 -- 172.21.15.133:0/252118739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae006a7b0 con 0x7fcae0100a50 2024-09-11T23:36:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcae497d640 1 -- 172.21.15.133:0/252118739 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad0003cd0 con 0x7fcae0100a50 2024-09-11T23:36:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.808+0000 7fcae497d640 1 -- 172.21.15.133:0/252118739 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0048060 con 0x7fcae0100a50 2024-09-11T23:36:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.809+0000 7fcae597f640 1 -- 172.21.15.133:0/252118739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae010ce70 con 0x7fcae0100a50 2024-09-11T23:36:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.810+0000 7fcae497d640 1 -- 172.21.15.133:0/252118739 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7fcad003b460 con 0x7fcae0100a50 2024-09-11T23:36:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.811+0000 7fcae497d640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcacc0a2c80 0x7fcacc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.811+0000 7fcade7fc640 1 -- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcacc0a2c80 msgr2=0x7fcacc0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3324898762 2024-09-11T23:36:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.811+0000 7fcae497d640 1 -- 172.21.15.133:0/252118739 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fcad00ef470 con 0x7fcae0100a50 2024-09-11T23:36:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.811+0000 7fcade7fc640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcacc0a2c80 0x7fcacc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:36:45.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:45.819+0000 7fcae497d640 1 -- 172.21.15.133:0/252118739 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcad00bc470 con 0x7fcae0100a50 2024-09-11T23:36:45.959 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:45.734+0000 7f05d0c0c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:36:45.959 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:45.902+0000 7f05d0c0c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:36:46.008 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:45.808+0000 7f07f309a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:36:46.008 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:45.891+0000 7f07f309a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:36:46.008 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:45.950+0000 7f07f309a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:36:46.008 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:46 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:46.008+0000 7f07f309a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:36:46.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.012+0000 7fcade7fc640 1 -- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcacc0a2c80 msgr2=0x7fcacc0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3324898762 2024-09-11T23:36:46.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.012+0000 7fcade7fc640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcacc0a2c80 0x7fcacc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:36:46.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.029+0000 7fcae597f640 1 -- 172.21.15.133:0/252118739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fcae0107d40 con 0x7fcae0100a50 2024-09-11T23:36:46.035 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:45.765+0000 7fedcb7c0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:36:46.035 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:45.836+0000 7fedcb7c0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:36:46.035 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:45.917+0000 7fedcb7c0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:36:46.035 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:45.976+0000 7fedcb7c0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:36:46.035 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:46 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:46.035+0000 7fedcb7c0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:36:46.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.087+0000 7fcae497d640 1 -- 172.21.15.133:0/252118739 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7fcad00bbd70 con 0x7fcae0100a50 2024-09-11T23:36:46.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 -- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcacc0a2c80 msgr2=0x7fcacc0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:36:46.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcacc0a2c80 0x7fcacc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 -- 172.21.15.133:0/252118739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcae0100a50 msgr2=0x7fcae0196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcae0100a50 0x7fcae0196cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009230 tx=0x7fcad0004270 comp rx=0 tx=0).stop 2024-09-11T23:36:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 -- 172.21.15.133:0/252118739 shutdown_connections 2024-09-11T23:36:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fcacc0a2c80 0x7fcacc0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcae0102600 0x7fcae019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae0101c40 0x7fcae0197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 --2- 172.21.15.133:0/252118739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcae0100a50 0x7fcae0196cf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 -- 172.21.15.133:0/252118739 >> 172.21.15.133:0/252118739 conn(0x7fcae00fc1b0 msgr2=0x7fcae0104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 -- 172.21.15.133:0/252118739 shutdown_connections 2024-09-11T23:36:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.089+0000 7fcae597f640 1 -- 172.21.15.133:0/252118739 wait complete. 2024-09-11T23:36:46.100 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-11T23:36:46.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.207+0000 7fda4e5d6640 1 -- 172.21.15.133:0/2172425265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda48107880 msgr2=0x7fda48109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:46.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.207+0000 7fda4e5d6640 1 --2- 172.21.15.133:0/2172425265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda48107880 0x7fda48109c90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fda40009ca0 tx=0x7fda4002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:36:46.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.207+0000 7fda4e5d6640 1 -- 172.21.15.133:0/2172425265 shutdown_connections 2024-09-11T23:36:46.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.207+0000 7fda4e5d6640 1 --2- 172.21.15.133:0/2172425265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda48107880 0x7fda48109c90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:46.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.207+0000 7fda4e5d6640 1 --2- 172.21.15.133:0/2172425265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda48104f50 0x7fda48107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:46.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.207+0000 7fda4e5d6640 1 --2- 172.21.15.133:0/2172425265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48102620 0x7fda48104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:46.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.208+0000 7fda4e5d6640 1 -- 172.21.15.133:0/2172425265 >> 172.21.15.133:0/2172425265 conn(0x7fda480fc1e0 msgr2=0x7fda480fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:46.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.208+0000 7fda4e5d6640 1 -- 172.21.15.133:0/2172425265 shutdown_connections 2024-09-11T23:36:46.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.208+0000 7fda4e5d6640 1 -- 172.21.15.133:0/2172425265 wait complete. 2024-09-11T23:36:46.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.208+0000 7fda4e5d6640 1 Processor -- start 2024-09-11T23:36:46.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.208+0000 7fda4e5d6640 1 -- start start 2024-09-11T23:36:46.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.208+0000 7fda4e5d6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda48102620 0x7fda48197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:46.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.208+0000 7fda4e5d6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48104f50 0x7fda48198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:46.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.208+0000 7fda4e5d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda48107880 0x7fda4819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:46.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4e5d6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda4810d7e0 con 0x7fda48107880 2024-09-11T23:36:46.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4e5d6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda4810d660 con 0x7fda48102620 2024-09-11T23:36:46.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4e5d6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda4810d960 con 0x7fda48104f50 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4cb73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda48107880 0x7fda4819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4cb73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda48107880 0x7fda4819f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60570/0 (socket says 172.21.15.133:60570) 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4cb73640 1 -- 172.21.15.133:0/3416168152 learned_addr learned my addr 172.21.15.133:0/3416168152 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda47fff640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda48102620 0x7fda48197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda477fe640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48104f50 0x7fda48198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4cb73640 1 -- 172.21.15.133:0/3416168152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48104f50 msgr2=0x7fda48198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4cb73640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48104f50 0x7fda48198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4cb73640 1 -- 172.21.15.133:0/3416168152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda48102620 msgr2=0x7fda48197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4cb73640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda48102620 0x7fda48197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4cb73640 1 -- 172.21.15.133:0/3416168152 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4819fa50 con 0x7fda48107880 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda47fff640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda48102620 0x7fda48197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4cb73640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda48107880 0x7fda4819f3e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fda40006010 tx=0x7fda40004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:46.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda457fa640 1 -- 172.21.15.133:0/3416168152 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40003b70 con 0x7fda48107880 2024-09-11T23:36:46.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4e5d6640 1 -- 172.21.15.133:0/3416168152 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4819fce0 con 0x7fda48107880 2024-09-11T23:36:46.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda457fa640 1 -- 172.21.15.133:0/3416168152 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda40003d10 con 0x7fda48107880 2024-09-11T23:36:46.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda457fa640 1 -- 172.21.15.133:0/3416168152 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4002fc40 con 0x7fda48107880 2024-09-11T23:36:46.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.209+0000 7fda4e5d6640 1 -- 172.21.15.133:0/3416168152 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda481a0220 con 0x7fda48107880 2024-09-11T23:36:46.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.210+0000 7fda4e5d6640 1 -- 172.21.15.133:0/3416168152 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda480fdcd0 con 0x7fda48107880 2024-09-11T23:36:46.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.212+0000 7fda457fa640 1 -- 172.21.15.133:0/3416168152 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7fda40004050 con 0x7fda48107880 2024-09-11T23:36:46.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.213+0000 7fda457fa640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fda300a2c80 0x7fda300a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:46.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.213+0000 7fda47fff640 1 -- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fda300a2c80 msgr2=0x7fda300a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3324898762 2024-09-11T23:36:46.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.213+0000 7fda457fa640 1 -- 172.21.15.133:0/3416168152 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fda400f07c0 con 0x7fda48107880 2024-09-11T23:36:46.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.213+0000 7fda47fff640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fda300a2c80 0x7fda300a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:36:46.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.216+0000 7fda457fa640 1 -- 172.21.15.133:0/3416168152 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda400bd8e0 con 0x7fda48107880 2024-09-11T23:36:46.255 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:46 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/252118739' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:36:46.255 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:46 smithi133 ceph-mon[31430]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:36:46.255 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:45.970+0000 7f05d0c0c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:36:46.255 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:46 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:46.039+0000 7f05d0c0c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:36:46.255 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:46 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:46.126+0000 7f05d0c0c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:36:46.255 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:46 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:46.190+0000 7f05d0c0c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:36:46.255 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:46 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:46.255+0000 7f05d0c0c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:36:46.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:46 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/252118739' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:36:46.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:46 smithi178 ceph-mon[35882]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:36:46.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.414+0000 7fda47fff640 1 -- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fda300a2c80 msgr2=0x7fda300a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3324898762 2024-09-11T23:36:46.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.414+0000 7fda47fff640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fda300a2c80 0x7fda300a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:36:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.415+0000 7fda4e5d6640 1 -- 172.21.15.133:0/3416168152 --> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fda480626d0 con 0x7fda300a2c80 2024-09-11T23:36:46.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:46 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/252118739' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:36:46.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:46 smithi138 ceph-mon[35142]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:36:46.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.815+0000 7fda47fff640 1 -- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fda300a2c80 msgr2=0x7fda300a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3324898762 2024-09-11T23:36:46.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:46.815+0000 7fda47fff640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fda300a2c80 0x7fda300a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T23:36:47.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:47.062+0000 7fda457fa640 1 -- 172.21.15.133:0/3416168152 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7fda400b9850 con 0x7fda48107880 2024-09-11T23:36:47.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:47.063+0000 7fda457fa640 1 -- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fda300a2c80 msgr2=0x7fda300a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:36:47.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:47.063+0000 7fda457fa640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/3324898762,v1:172.21.15.133:6801/3324898762] conn(0x7fda300a2c80 0x7fda300a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:47.090 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:36:46 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z[37690]: 2024-09-11T23:36:46.795+0000 7f07f309a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:36:47.091 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:36:46 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y[36513]: 2024-09-11T23:36:46.832+0000 7fedcb7c0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:36:47.318 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:47.049+0000 7f05d0c0c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:36:47.318 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:47.285+0000 7f0538559640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:47.318 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:47.285+0000 7f0538559640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:47.319 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:47.285+0000 7f0538559640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:47.319 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:47.285+0000 7f0538559640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:47.319 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:47.285+0000 7f0538559640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:47.319 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:47.286+0000 7f053b55f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:47.319 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:47.286+0000 7f053b55f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:47.319 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:47.286+0000 7f053b55f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:47.319 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:47.286+0000 7f053b55f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:47.319 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:47.286+0000 7f053b55f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:47.319 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: Standby manager daemon z restarted 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: Standby manager daemon z started 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: Standby manager daemon y restarted 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: Standby manager daemon y started 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:36:47.320 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: Active manager daemon x restarted 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: Activating manager daemon x 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: mgrmap e20: x(active, starting, since 0.0120575s), standbys: y, z 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:36:47.321 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:36:47.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:36:47.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:36:47.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:36:47.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:36:47.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:36:47.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:36:47.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:36:47.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:36:47.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:36:47.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:36:47.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:36:47.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:47 smithi133 ceph-mon[31430]: Manager daemon x is now available 2024-09-11T23:36:47.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-11T23:36:47.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:47.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: Standby manager daemon z restarted 2024-09-11T23:36:47.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: Standby manager daemon z started 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: Standby manager daemon y restarted 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: Standby manager daemon y started 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: Active manager daemon x restarted 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: Activating manager daemon x 2024-09-11T23:36:47.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: mgrmap e20: x(active, starting, since 0.0120575s), standbys: y, z 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:36:47.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:36:47.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:36:47.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:36:47.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:36:47.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:36:47.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:36:47.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:36:47.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:36:47.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:47 smithi138 ceph-mon[35142]: Manager daemon x is now available 2024-09-11T23:36:47.422 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-11T23:36:47.422 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:36:47.422 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: Standby manager daemon z restarted 2024-09-11T23:36:47.422 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: Standby manager daemon z started 2024-09-11T23:36:47.422 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.178:0/2945975533' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: Standby manager daemon y restarted 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: Standby manager daemon y started 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.? 172.21.15.138:0/1713588069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: Active manager daemon x restarted 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: Activating manager daemon x 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: mgrmap e20: x(active, starting, since 0.0120575s), standbys: y, z 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:36:47.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:36:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:36:47.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:36:47.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:36:47.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:36:47.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:36:47.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:47 smithi178 ceph-mon[35882]: Manager daemon x is now available 2024-09-11T23:36:48.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.067+0000 7fda457fa640 1 -- 172.21.15.133:0/3416168152 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fda40004340 con 0x7fda48107880 2024-09-11T23:36:48.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.068+0000 7fda457fa640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fda30051920 0x7fda30051d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:48.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.068+0000 7fda457fa640 1 -- 172.21.15.133:0/3416168152 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fda40033e70 con 0x7fda30051920 2024-09-11T23:36:48.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.069+0000 7fda47fff640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fda30051920 0x7fda30051d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:48.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.071+0000 7fda47fff640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fda30051920 0x7fda30051d00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fda380095f0 tx=0x7fda38005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:48.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.089+0000 7fda457fa640 1 -- 172.21.15.133:0/3416168152 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fda40033e70 con 0x7fda30051920 2024-09-11T23:36:48.338 INFO:teuthology.orchestra.run.smithi133.stdout:{} 2024-09-11T23:36:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.092+0000 7fda4e5d6640 1 -- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fda30051920 msgr2=0x7fda30051d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.092+0000 7fda4e5d6640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fda30051920 0x7fda30051d00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fda380095f0 tx=0x7fda38005f90 comp rx=0 tx=0).stop 2024-09-11T23:36:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.092+0000 7fda4e5d6640 1 -- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda48107880 msgr2=0x7fda4819f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.092+0000 7fda4e5d6640 1 --2- 172.21.15.133:0/3416168152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda48107880 0x7fda4819f3e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fda40006010 tx=0x7fda40004ab0 comp rx=0 tx=0).stop 2024-09-11T23:36:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.092+0000 7fda4cb73640 1 -- 172.21.15.133:0/3416168152 reap_dead start 2024-09-11T23:36:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.092+0000 7fda4e5d6640 1 -- 172.21.15.133:0/3416168152 shutdown_connections 2024-09-11T23:36:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.092+0000 7fda4e5d6640 1 -- 172.21.15.133:0/3416168152 >> 172.21.15.133:0/3416168152 conn(0x7fda480fc1e0 msgr2=0x7fda48102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.092+0000 7fda4e5d6640 1 -- 172.21.15.133:0/3416168152 shutdown_connections 2024-09-11T23:36:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:48.092+0000 7fda4e5d6640 1 -- 172.21.15.133:0/3416168152 wait complete. 2024-09-11T23:36:48.341 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-11T23:36:48.752 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-11T23:36:48.933 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T23:36:48.933 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:48.933 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:36:48.934 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:36:48.934 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-mon[31430]: mgrmap e21: x(active, since 1.01904s), standbys: y, z 2024-09-11T23:36:48.934 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:48.077+0000 7f0535598640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:48.934 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:48.077+0000 7f0535598640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:48.934 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:48.077+0000 7f0535598640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:48.077+0000 7f0535598640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:48.077+0000 7f0535598640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:48.414+0000 7f051a423640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:36:48] ENGINE Error in HTTPServer.serve 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: Traceback (most recent call last): 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self._connections.run(self.expiration_interval) 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self._run(expiration_interval) 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:36:48.935 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:36:48.936 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:36:48.936 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: s = self.context.wrap_socket( 2024-09-11T23:36:48.936 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:36:48.936 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: return self.sslsocket_class._create( 2024-09-11T23:36:48.936 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:36:48.936 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self.do_handshake() 2024-09-11T23:36:48.937 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:36:48.937 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: self._sslobj.do_handshake() 2024-09-11T23:36:48.937 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:36:48.937 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:36:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:36:48.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:48 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:48.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:48 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:36:48.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:48 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:36:48.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:48 smithi178 ceph-mon[35882]: mgrmap e21: x(active, since 1.01904s), standbys: y, z 2024-09-11T23:36:48.939 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:48 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:48.939 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:48 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:36:48.939 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:48 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:36:48.939 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:48 smithi138 ceph-mon[35142]: mgrmap e21: x(active, since 1.01904s), standbys: y, z 2024-09-11T23:36:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.056+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/3396479563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6b81025b0 msgr2=0x7ff6b81049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.056+0000 7ff6bd8ac640 1 --2- 172.21.15.133:0/3396479563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6b81025b0 0x7ff6b81049a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8009860 tx=0x7ff6a802a340 comp rx=0 tx=0).stop 2024-09-11T23:36:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.056+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/3396479563 shutdown_connections 2024-09-11T23:36:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.056+0000 7ff6bd8ac640 1 --2- 172.21.15.133:0/3396479563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8107810 0x7ff6b8109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.056+0000 7ff6bd8ac640 1 --2- 172.21.15.133:0/3396479563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b8104ee0 0x7ff6b81072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.056+0000 7ff6bd8ac640 1 --2- 172.21.15.133:0/3396479563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6b81025b0 0x7ff6b81049a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.056+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/3396479563 >> 172.21.15.133:0/3396479563 conn(0x7ff6b80fc1b0 msgr2=0x7ff6b80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.056+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/3396479563 shutdown_connections 2024-09-11T23:36:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.056+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/3396479563 wait complete. 2024-09-11T23:36:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6bd8ac640 1 Processor -- start 2024-09-11T23:36:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6bd8ac640 1 -- start start 2024-09-11T23:36:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6bd8ac640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6b81025b0 0x7ff6b8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6bd8ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b8104ee0 0x7ff6b81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6bd8ac640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8107810 0x7ff6b819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6bd8ac640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b810d820 con 0x7ff6b8104ee0 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6bd8ac640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b810d6a0 con 0x7ff6b8107810 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6bd8ac640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b810d9a0 con 0x7ff6b81025b0 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6b67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b8104ee0 0x7ff6b81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6b6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6b81025b0 0x7ff6b8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6b67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b8104ee0 0x7ff6b81983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60752/0 (socket says 172.21.15.133:60752) 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6b67fc640 1 -- 172.21.15.133:0/467726487 learned_addr learned my addr 172.21.15.133:0/467726487 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6b77fe640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8107810 0x7ff6b819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6b6ffd640 1 -- 172.21.15.133:0/467726487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8107810 msgr2=0x7ff6b819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6b6ffd640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8107810 0x7ff6b819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6b6ffd640 1 -- 172.21.15.133:0/467726487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b8104ee0 msgr2=0x7ff6b81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.057+0000 7ff6b6ffd640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b8104ee0 0x7ff6b81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.058+0000 7ff6b6ffd640 1 -- 172.21.15.133:0/467726487 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b819fad0 con 0x7ff6b81025b0 2024-09-11T23:36:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.058+0000 7ff6b6ffd640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6b81025b0 0x7ff6b8197eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8035040 tx=0x7ff6a8033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.058+0000 7ff6bc8aa640 1 -- 172.21.15.133:0/467726487 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8037040 con 0x7ff6b81025b0 2024-09-11T23:36:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.058+0000 7ff6bc8aa640 1 -- 172.21.15.133:0/467726487 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a8032770 con 0x7ff6b81025b0 2024-09-11T23:36:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.058+0000 7ff6bc8aa640 1 -- 172.21.15.133:0/467726487 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8005520 con 0x7ff6b81025b0 2024-09-11T23:36:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.058+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/467726487 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b819fd60 con 0x7ff6b81025b0 2024-09-11T23:36:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.058+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/467726487 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b81a01a0 con 0x7ff6b81025b0 2024-09-11T23:36:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.059+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/467726487 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b80fdca0 con 0x7ff6b81025b0 2024-09-11T23:36:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.063+0000 7ff6bc8aa640 1 -- 172.21.15.133:0/467726487 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7ff6a8042020 con 0x7ff6b81025b0 2024-09-11T23:36:49.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.064+0000 7ff6bc8aa640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6a00a2a10 0x7ff6a00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.064+0000 7ff6bc8aa640 1 -- 172.21.15.133:0/467726487 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff6a80ee990 con 0x7ff6b81025b0 2024-09-11T23:36:49.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.064+0000 7ff6b67fc640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6a00a2a10 0x7ff6a00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.064+0000 7ff6b67fc640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6a00a2a10 0x7ff6a00a4ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0096c0 tx=0x7ff6ac009260 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:49.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.066+0000 7ff6bc8aa640 1 -- 172.21.15.133:0/467726487 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6a8049020 con 0x7ff6b81025b0 2024-09-11T23:36:49.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.252+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/467726487 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6b802d990 con 0x7ff6b81025b0 2024-09-11T23:36:49.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.254+0000 7ff6bc8aa640 1 -- 172.21.15.133:0/467726487 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+23060 (secure 0 0 0) 0x7ff6a80bbb30 con 0x7ff6b81025b0 2024-09-11T23:36:49.255 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:49.256 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":82,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:36:47.051856+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/3324898762":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3596945316":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6801/3324898762":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/1658494809":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3947064111":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/802317131":"2024-09-12T23:36:47.051817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:36:49.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/467726487 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6a00a2a10 msgr2=0x7ff6a00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6a00a2a10 0x7ff6a00a4ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0096c0 tx=0x7ff6ac009260 comp rx=0 tx=0).stop 2024-09-11T23:36:49.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/467726487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6b81025b0 msgr2=0x7ff6b8197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6b81025b0 0x7ff6b8197eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8035040 tx=0x7ff6a8033d90 comp rx=0 tx=0).stop 2024-09-11T23:36:49.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/467726487 shutdown_connections 2024-09-11T23:36:49.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6a00a2a10 0x7ff6a00a4ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8107810 0x7ff6b819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b8104ee0 0x7ff6b81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 --2- 172.21.15.133:0/467726487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6b81025b0 0x7ff6b8197eb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/467726487 >> 172.21.15.133:0/467726487 conn(0x7ff6b80fc1b0 msgr2=0x7ff6b8102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:49.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/467726487 shutdown_connections 2024-09-11T23:36:49.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.256+0000 7ff6bd8ac640 1 -- 172.21.15.133:0/467726487 wait complete. 2024-09-11T23:36:49.269 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:30:22.061012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T23:34:49.162653+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T23:34:51.723611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:36:49.269 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T23:36:49.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.376+0000 7fec50274640 1 -- 172.21.15.133:0/2883862164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec48107860 msgr2=0x7fec48109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.376+0000 7fec50274640 1 --2- 172.21.15.133:0/2883862164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec48107860 0x7fec48109c50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fec40009ca0 tx=0x7fec4002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.376+0000 7fec50274640 1 -- 172.21.15.133:0/2883862164 shutdown_connections 2024-09-11T23:36:49.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.376+0000 7fec50274640 1 --2- 172.21.15.133:0/2883862164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec48107860 0x7fec48109c50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.376+0000 7fec50274640 1 --2- 172.21.15.133:0/2883862164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec48104f30 0x7fec48107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.376+0000 7fec50274640 1 --2- 172.21.15.133:0/2883862164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec48102600 0x7fec481049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.376+0000 7fec50274640 1 -- 172.21.15.133:0/2883862164 >> 172.21.15.133:0/2883862164 conn(0x7fec480fc140 msgr2=0x7fec480fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:49.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.376+0000 7fec50274640 1 -- 172.21.15.133:0/2883862164 shutdown_connections 2024-09-11T23:36:49.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.376+0000 7fec50274640 1 -- 172.21.15.133:0/2883862164 wait complete. 2024-09-11T23:36:49.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.376+0000 7fec50274640 1 Processor -- start 2024-09-11T23:36:49.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.376+0000 7fec50274640 1 -- start start 2024-09-11T23:36:49.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec50274640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec48102600 0x7fec48197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec50274640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec48104f30 0x7fec481983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec50274640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec48107860 0x7fec4819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec50274640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fec4810d7c0 con 0x7fec48104f30 2024-09-11T23:36:49.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec50274640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec4810d640 con 0x7fec48107860 2024-09-11T23:36:49.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec50274640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec4810d940 con 0x7fec48102600 2024-09-11T23:36:49.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4d80f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec48104f30 0x7fec481983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4d80f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec48104f30 0x7fec481983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60790/0 (socket says 172.21.15.133:60790) 2024-09-11T23:36:49.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4d80f640 1 -- 172.21.15.133:0/609084798 learned_addr learned my addr 172.21.15.133:0/609084798 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:49.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4e010640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec48102600 0x7fec48197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4e811640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec48107860 0x7fec4819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4d80f640 1 -- 172.21.15.133:0/609084798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec48102600 msgr2=0x7fec48197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4d80f640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec48102600 0x7fec48197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4d80f640 1 -- 172.21.15.133:0/609084798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec48107860 msgr2=0x7fec4819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4d80f640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec48107860 0x7fec4819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4d80f640 1 -- 172.21.15.133:0/609084798 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4819fac0 con 0x7fec48104f30 2024-09-11T23:36:49.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4e811640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec48107860 0x7fec4819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:49.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec4d80f640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec48104f30 0x7fec481983e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fec38005fd0 tx=0x7fec3800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:49.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.377+0000 7fec3f7fe640 1 -- 172.21.15.133:0/609084798 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38017070 con 0x7fec48104f30 2024-09-11T23:36:49.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.378+0000 7fec50274640 1 -- 172.21.15.133:0/609084798 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4819fdb0 con 0x7fec48104f30 2024-09-11T23:36:49.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.378+0000 7fec3f7fe640 1 -- 172.21.15.133:0/609084798 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec38009c30 con 0x7fec48104f30 2024-09-11T23:36:49.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.378+0000 7fec3f7fe640 1 -- 172.21.15.133:0/609084798 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38010460 con 0x7fec48104f30 2024-09-11T23:36:49.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.378+0000 7fec50274640 1 -- 172.21.15.133:0/609084798 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec481a0280 con 0x7fec48104f30 2024-09-11T23:36:49.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.379+0000 7fec50274640 1 -- 172.21.15.133:0/609084798 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec480fdc30 con 0x7fec48104f30 2024-09-11T23:36:49.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.380+0000 7fec3f7fe640 1 -- 172.21.15.133:0/609084798 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fec38003e70 con 0x7fec48104f30 2024-09-11T23:36:49.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.381+0000 7fec3f7fe640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec340a2990 0x7fec340a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.381+0000 7fec4e010640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec340a2990 0x7fec340a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.381+0000 7fec3f7fe640 1 -- 172.21.15.133:0/609084798 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fec380ce670 con 0x7fec48104f30 2024-09-11T23:36:49.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.381+0000 7fec4e010640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec340a2990 0x7fec340a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fec44009bd0 tx=0x7fec44005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:49.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.390+0000 7fec3f7fe640 1 -- 172.21.15.133:0/609084798 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec38020070 con 0x7fec48104f30 2024-09-11T23:36:49.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.566+0000 7fec50274640 1 -- 172.21.15.133:0/609084798 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec4802d7c0 con 0x7fec48104f30 2024-09-11T23:36:49.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.567+0000 7fec3f7fe640 1 -- 172.21.15.133:0/609084798 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fec3809b810 con 0x7fec48104f30 2024-09-11T23:36:49.568 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-09-11T23:36:49.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 -- 172.21.15.133:0/609084798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec340a2990 msgr2=0x7fec340a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec340a2990 0x7fec340a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fec44009bd0 tx=0x7fec44005f90 comp rx=0 tx=0).stop 2024-09-11T23:36:49.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 -- 172.21.15.133:0/609084798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec48104f30 msgr2=0x7fec481983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec48104f30 0x7fec481983e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fec38005fd0 tx=0x7fec3800dde0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 -- 172.21.15.133:0/609084798 shutdown_connections 2024-09-11T23:36:49.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec340a2990 0x7fec340a4e50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec48107860 0x7fec4819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec48104f30 0x7fec481983e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 --2- 172.21.15.133:0/609084798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec48102600 0x7fec48197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 -- 172.21.15.133:0/609084798 >> 172.21.15.133:0/609084798 conn(0x7fec480fc140 msgr2=0x7fec48102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:49.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 -- 172.21.15.133:0/609084798 shutdown_connections 2024-09-11T23:36:49.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.569+0000 7fec50274640 1 -- 172.21.15.133:0/609084798 wait complete. 2024-09-11T23:36:49.581 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T23:36:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.684+0000 7f10adffd640 1 -- 172.21.15.133:0/505794312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a8100a00 msgr2=0x7f10a8100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.684+0000 7f10adffd640 1 --2- 172.21.15.133:0/505794312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a8100a00 0x7f10a8100e00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1098009ca0 tx=0x7f109802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.684+0000 7f10adffd640 1 -- 172.21.15.133:0/505794312 shutdown_connections 2024-09-11T23:36:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.684+0000 7f10adffd640 1 --2- 172.21.15.133:0/505794312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a81025b0 0x7f10a8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.684+0000 7f10adffd640 1 --2- 172.21.15.133:0/505794312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a8101bf0 0x7f10a8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.684+0000 7f10adffd640 1 --2- 172.21.15.133:0/505794312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a8100a00 0x7f10a8100e00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.684+0000 7f10adffd640 1 -- 172.21.15.133:0/505794312 >> 172.21.15.133:0/505794312 conn(0x7f10a80fc140 msgr2=0x7f10a80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.684+0000 7f10adffd640 1 -- 172.21.15.133:0/505794312 shutdown_connections 2024-09-11T23:36:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.684+0000 7f10adffd640 1 -- 172.21.15.133:0/505794312 wait complete. 2024-09-11T23:36:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10adffd640 1 Processor -- start 2024-09-11T23:36:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10adffd640 1 -- start start 2024-09-11T23:36:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10adffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a8100a00 0x7f10a8104990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10adffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a8101bf0 0x7f10a8102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10adffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a81025b0 0x7f10a8103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10adffd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f10a810c660 con 0x7f10a8101bf0 2024-09-11T23:36:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10adffd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10a810c4e0 con 0x7f10a81025b0 2024-09-11T23:36:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10adffd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f10a810c7e0 con 0x7f10a8100a00 2024-09-11T23:36:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10a6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a8101bf0 0x7f10a8102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10a6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a8101bf0 0x7f10a8102fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60806/0 (socket says 172.21.15.133:60806) 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10a6ffd640 1 -- 172.21.15.133:0/2802785171 learned_addr learned my addr 172.21.15.133:0/2802785171 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10a77fe640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a8100a00 0x7f10a8104990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10a7fff640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a81025b0 0x7f10a8103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10a6ffd640 1 -- 172.21.15.133:0/2802785171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a8100a00 msgr2=0x7f10a8104990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10a6ffd640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a8100a00 0x7f10a8104990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10a6ffd640 1 -- 172.21.15.133:0/2802785171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a81025b0 msgr2=0x7f10a8103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10a6ffd640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a81025b0 0x7f10a8103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.685+0000 7f10a6ffd640 1 -- 172.21.15.133:0/2802785171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a8103da0 con 0x7f10a8101bf0 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.686+0000 7f10a6ffd640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a8101bf0 0x7f10a8102fd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f109000d950 tx=0x7f109000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.686+0000 7f10a4ff9640 1 -- 172.21.15.133:0/2802785171 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090017070 con 0x7f10a8101bf0 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.686+0000 7f10a4ff9640 1 -- 172.21.15.133:0/2802785171 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1090009c30 con 0x7f10a8101bf0 2024-09-11T23:36:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.686+0000 7f10adffd640 1 -- 172.21.15.133:0/2802785171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a81a1fc0 con 0x7f10a8101bf0 2024-09-11T23:36:49.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.686+0000 7f10a4ff9640 1 -- 172.21.15.133:0/2802785171 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109000f6b0 con 0x7f10a8101bf0 2024-09-11T23:36:49.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.686+0000 7f10adffd640 1 -- 172.21.15.133:0/2802785171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a81a2420 con 0x7f10a8101bf0 2024-09-11T23:36:49.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.687+0000 7f10adffd640 1 -- 172.21.15.133:0/2802785171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a81a2710 con 0x7f10a8101bf0 2024-09-11T23:36:49.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.688+0000 7f10a4ff9640 1 -- 172.21.15.133:0/2802785171 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f10900109d0 con 0x7f10a8101bf0 2024-09-11T23:36:49.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.689+0000 7f10a4ff9640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f10940a2990 0x7f10940a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:49.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.689+0000 7f10a4ff9640 1 -- 172.21.15.133:0/2802785171 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f10900ce7d0 con 0x7f10a8101bf0 2024-09-11T23:36:49.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.689+0000 7f10a77fe640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f10940a2990 0x7f10940a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:49.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.689+0000 7f10a77fe640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f10940a2990 0x7f10940a4e50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1098006010 tx=0x7f1098000f40 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.692+0000 7f10a4ff9640 1 -- 172.21.15.133:0/2802785171 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1090009de0 con 0x7f10a8101bf0 2024-09-11T23:36:49.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.891+0000 7f10adffd640 1 -- 172.21.15.133:0/2802785171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f10a81a28e0 con 0x7f10a8101bf0 2024-09-11T23:36:49.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.891+0000 7f10a4ff9640 1 -- 172.21.15.133:0/2802785171 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f109009b320 con 0x7f10a8101bf0 2024-09-11T23:36:49.892 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-09-11T23:36:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 -- 172.21.15.133:0/2802785171 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f10940a2990 msgr2=0x7f10940a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f10940a2990 0x7f10940a4e50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1098006010 tx=0x7f1098000f40 comp rx=0 tx=0).stop 2024-09-11T23:36:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 -- 172.21.15.133:0/2802785171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a8101bf0 msgr2=0x7f10a8102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a8101bf0 0x7f10a8102fd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f109000d950 tx=0x7f109000de10 comp rx=0 tx=0).stop 2024-09-11T23:36:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 -- 172.21.15.133:0/2802785171 shutdown_connections 2024-09-11T23:36:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f10940a2990 0x7f10940a4e50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a81025b0 0x7f10a8103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a8101bf0 0x7f10a8102fd0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 --2- 172.21.15.133:0/2802785171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a8100a00 0x7f10a8104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 -- 172.21.15.133:0/2802785171 >> 172.21.15.133:0/2802785171 conn(0x7f10a80fc140 msgr2=0x7f10a8069330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 -- 172.21.15.133:0/2802785171 shutdown_connections 2024-09-11T23:36:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:49.894+0000 7f10adffd640 1 -- 172.21.15.133:0/2802785171 wait complete. 2024-09-11T23:36:49.905 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T23:36:50.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.012+0000 7f20ff6af640 1 -- 172.21.15.133:0/440033173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20f8102600 msgr2=0x7f20f81049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:50.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.012+0000 7f20ff6af640 1 --2- 172.21.15.133:0/440033173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20f8102600 0x7f20f81049f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009900 tx=0x7f20f402a410 comp rx=0 tx=0).stop 2024-09-11T23:36:50.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.012+0000 7f20ff6af640 1 -- 172.21.15.133:0/440033173 shutdown_connections 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.012+0000 7f20ff6af640 1 --2- 172.21.15.133:0/440033173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8107860 0x7f20f8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.012+0000 7f20ff6af640 1 --2- 172.21.15.133:0/440033173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8104f30 0x7f20f8107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.012+0000 7f20ff6af640 1 --2- 172.21.15.133:0/440033173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20f8102600 0x7f20f81049f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.012+0000 7f20ff6af640 1 -- 172.21.15.133:0/440033173 >> 172.21.15.133:0/440033173 conn(0x7f20f80fc180 msgr2=0x7f20f80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.012+0000 7f20ff6af640 1 -- 172.21.15.133:0/440033173 shutdown_connections 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.012+0000 7f20ff6af640 1 -- 172.21.15.133:0/440033173 wait complete. 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.012+0000 7f20ff6af640 1 Processor -- start 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.013+0000 7f20ff6af640 1 -- start start 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.013+0000 7f20ff6af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20f8102600 0x7f20f8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.013+0000 7f20ff6af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8104f30 0x7f20f81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.013+0000 7f20ff6af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8107860 0x7f20f819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.013+0000 7f20ff6af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f20f810d840 con 0x7f20f8104f30 2024-09-11T23:36:50.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.013+0000 7f20ff6af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20f810d6c0 con 0x7f20f8107860 2024-09-11T23:36:50.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.013+0000 7f20ff6af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20f810d9c0 con 0x7f20f8102600 2024-09-11T23:36:50.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.013+0000 7f20fdc4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8107860 0x7f20f819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:50.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.013+0000 7f20fdc4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8107860 0x7f20f819f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53972/0 (socket says 172.21.15.133:53972) 2024-09-11T23:36:50.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.013+0000 7f20fdc4c640 1 -- 172.21.15.133:0/1418885183 learned_addr learned my addr 172.21.15.133:0/1418885183 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:50.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.013+0000 7f20fd44b640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20f8102600 0x7f20f8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:50.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.014+0000 7f20fcc4a640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8104f30 0x7f20f81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:50.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.014+0000 7f20fcc4a640 1 -- 172.21.15.133:0/1418885183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20f8102600 msgr2=0x7f20f8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:50.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.014+0000 7f20fcc4a640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20f8102600 0x7f20f8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:50.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.014+0000 7f20fcc4a640 1 -- 172.21.15.133:0/1418885183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8107860 msgr2=0x7f20f819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:50.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.014+0000 7f20fcc4a640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8107860 0x7f20f819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:50.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.014+0000 7f20fcc4a640 1 -- 172.21.15.133:0/1418885183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f819fac0 con 0x7f20f8104f30 2024-09-11T23:36:50.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.018+0000 7f20fcc4a640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8104f30 0x7f20f81983e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f20e800d6a0 tx=0x7f20e800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:50.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.024+0000 7f20ee7fc640 1 -- 172.21.15.133:0/1418885183 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8018070 con 0x7f20f8104f30 2024-09-11T23:36:50.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.024+0000 7f20ff6af640 1 -- 172.21.15.133:0/1418885183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f8100450 con 0x7f20f8104f30 2024-09-11T23:36:50.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.024+0000 7f20ee7fc640 1 -- 172.21.15.133:0/1418885183 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e8009820 con 0x7f20f8104f30 2024-09-11T23:36:50.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.024+0000 7f20ee7fc640 1 -- 172.21.15.133:0/1418885183 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8004cd0 con 0x7f20f8104f30 2024-09-11T23:36:50.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.025+0000 7f20ff6af640 1 -- 172.21.15.133:0/1418885183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f81009a0 con 0x7f20f8104f30 2024-09-11T23:36:50.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.026+0000 7f20ff6af640 1 -- 172.21.15.133:0/1418885183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f80fdc70 con 0x7f20f8104f30 2024-09-11T23:36:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.027+0000 7f20ee7fc640 1 -- 172.21.15.133:0/1418885183 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 138005+0+0 (secure 0 0 0) 0x7f20e80099d0 con 0x7f20f8104f30 2024-09-11T23:36:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.028+0000 7f20ee7fc640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f20e40a2d90 0x7f20e40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.028+0000 7f20fd44b640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f20e40a2d90 0x7f20e40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.028+0000 7f20ee7fc640 1 -- 172.21.15.133:0/1418885183 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f20e80d3060 con 0x7f20f8104f30 2024-09-11T23:36:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.028+0000 7f20fd44b640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f20e40a2d90 0x7f20e40a5250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20f4006010 tx=0x7f20f4000bc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.032+0000 7f20ee7fc640 1 -- 172.21.15.133:0/1418885183 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20e809fc90 con 0x7f20f8104f30 2024-09-11T23:36:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: [11/Sep/2024:23:36:48] ENGINE Bus STARTING 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: [11/Sep/2024:23:36:48] ENGINE Serving on http://172.21.15.133:8765 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: [11/Sep/2024:23:36:48] ENGINE Serving on https://172.21.15.133:7150 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: [11/Sep/2024:23:36:48] ENGINE Bus STARTED 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: [11/Sep/2024:23:36:48] ENGINE Error in HTTPServer.serve 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout: Traceback (most recent call last): 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout: self._run(expiration_interval) 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:36:50.210 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout: s = self.context.wrap_socket( 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout: return self.sslsocket_class._create( 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout: self.do_handshake() 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout: self._sslobj.do_handshake() 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/467726487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/609084798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:49 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:50 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:50 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:50 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:50 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:50.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:50 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2802785171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:36:50.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.222+0000 7f20ff6af640 1 -- 172.21.15.133:0/1418885183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f20f802da10 con 0x7f20f8104f30 2024-09-11T23:36:50.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.222+0000 7f20ee7fc640 1 -- 172.21.15.133:0/1418885183 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f20e809f590 con 0x7f20f8104f30 2024-09-11T23:36:50.223 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-09-11T23:36:50.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 -- 172.21.15.133:0/1418885183 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f20e40a2d90 msgr2=0x7f20e40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:50.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f20e40a2d90 0x7f20e40a5250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20f4006010 tx=0x7f20f4000bc0 comp rx=0 tx=0).stop 2024-09-11T23:36:50.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 -- 172.21.15.133:0/1418885183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8104f30 msgr2=0x7f20f81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:50.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8104f30 0x7f20f81983e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f20e800d6a0 tx=0x7f20e800db60 comp rx=0 tx=0).stop 2024-09-11T23:36:50.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 -- 172.21.15.133:0/1418885183 shutdown_connections 2024-09-11T23:36:50.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f20e40a2d90 0x7f20e40a5250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:50.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20f8107860 0x7f20f819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:50.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8104f30 0x7f20f81983e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:50.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 --2- 172.21.15.133:0/1418885183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20f8102600 0x7f20f8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:50.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 -- 172.21.15.133:0/1418885183 >> 172.21.15.133:0/1418885183 conn(0x7f20f80fc180 msgr2=0x7f20f81030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:50.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 -- 172.21.15.133:0/1418885183 shutdown_connections 2024-09-11T23:36:50.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:50.225+0000 7f20ff6af640 1 -- 172.21.15.133:0/1418885183 wait complete. 2024-09-11T23:36:50.237 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-11T23:36:50.238 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:36:50.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: [11/Sep/2024:23:36:48] ENGINE Bus STARTING 2024-09-11T23:36:50.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: [11/Sep/2024:23:36:48] ENGINE Serving on http://172.21.15.133:8765 2024-09-11T23:36:50.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: [11/Sep/2024:23:36:48] ENGINE Serving on https://172.21.15.133:7150 2024-09-11T23:36:50.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: [11/Sep/2024:23:36:48] ENGINE Bus STARTED 2024-09-11T23:36:50.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: [11/Sep/2024:23:36:48] ENGINE Error in HTTPServer.serve 2024-09-11T23:36:50.291 INFO:journalctl@ceph.mon.c.smithi178.stdout: Traceback (most recent call last): 2024-09-11T23:36:50.291 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:36:50.291 INFO:journalctl@ceph.mon.c.smithi178.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:36:50.291 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:36:50.291 INFO:journalctl@ceph.mon.c.smithi178.stdout: self._run(expiration_interval) 2024-09-11T23:36:50.291 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:36:50.291 INFO:journalctl@ceph.mon.c.smithi178.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:36:50.291 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:36:50.291 INFO:journalctl@ceph.mon.c.smithi178.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:36:50.292 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:36:50.292 INFO:journalctl@ceph.mon.c.smithi178.stdout: s = self.context.wrap_socket( 2024-09-11T23:36:50.292 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:36:50.292 INFO:journalctl@ceph.mon.c.smithi178.stdout: return self.sslsocket_class._create( 2024-09-11T23:36:50.292 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:36:50.292 INFO:journalctl@ceph.mon.c.smithi178.stdout: self.do_handshake() 2024-09-11T23:36:50.292 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:36:50.292 INFO:journalctl@ceph.mon.c.smithi178.stdout: self._sslobj.do_handshake() 2024-09-11T23:36:50.292 INFO:journalctl@ceph.mon.c.smithi178.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:36:50.292 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/467726487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:50.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/609084798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:36:50.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:50.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:50.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:49 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2802785171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:36:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: [11/Sep/2024:23:36:48] ENGINE Bus STARTING 2024-09-11T23:36:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: [11/Sep/2024:23:36:48] ENGINE Serving on http://172.21.15.133:8765 2024-09-11T23:36:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: [11/Sep/2024:23:36:48] ENGINE Serving on https://172.21.15.133:7150 2024-09-11T23:36:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: [11/Sep/2024:23:36:48] ENGINE Bus STARTED 2024-09-11T23:36:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: [11/Sep/2024:23:36:48] ENGINE Error in HTTPServer.serve 2024-09-11T23:36:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout: Traceback (most recent call last): 2024-09-11T23:36:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:36:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:36:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: self._run(expiration_interval) 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: s = self.context.wrap_socket( 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: return self.sslsocket_class._create( 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: self.do_handshake() 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:36:50.449 INFO:journalctl@ceph.mon.b.smithi138.stdout: self._sslobj.do_handshake() 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/467726487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/609084798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:36:50.451 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:50.452 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:50.452 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:49 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2802785171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:36:50.554 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:51.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.135+0000 7f003a877640 1 -- 172.21.15.133:0/75980955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0034100a80 msgr2=0x7f0034100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.135+0000 7f003a877640 1 --2- 172.21.15.133:0/75980955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0034100a80 0x7f0034100ee0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0028009930 tx=0x7f002802a420 comp rx=0 tx=0).stop 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.135+0000 7f003a877640 1 -- 172.21.15.133:0/75980955 shutdown_connections 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.135+0000 7f003a877640 1 --2- 172.21.15.133:0/75980955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0034101420 0x7f003410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.135+0000 7f003a877640 1 --2- 172.21.15.133:0/75980955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0034100a80 0x7f0034100ee0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.135+0000 7f003a877640 1 --2- 172.21.15.133:0/75980955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0034106a80 0x7f0034106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.135+0000 7f003a877640 1 -- 172.21.15.133:0/75980955 >> 172.21.15.133:0/75980955 conn(0x7f00340fc7a0 msgr2=0x7f00340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.135+0000 7f003a877640 1 -- 172.21.15.133:0/75980955 shutdown_connections 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.135+0000 7f003a877640 1 -- 172.21.15.133:0/75980955 wait complete. 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f003a877640 1 Processor -- start 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f003a877640 1 -- start start 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f003a877640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0034100a80 0x7f0034199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f003a877640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0034101420 0x7f0034199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f003a877640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0034106a80 0x7f00341a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f003a877640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0034110e80 con 0x7f0034106a80 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f003a877640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0034110d00 con 0x7f0034100a80 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f003a877640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0034111000 con 0x7f0034101420 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f0038e14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0034106a80 0x7f00341a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f0038e14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0034106a80 0x7f00341a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60834/0 (socket says 172.21.15.133:60834) 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f0033fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0034100a80 0x7f0034199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f0038e14640 1 -- 172.21.15.133:0/3361486778 learned_addr learned my addr 172.21.15.133:0/3361486778 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.136+0000 7f00337fe640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0034101420 0x7f0034199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.137+0000 7f0038e14640 1 -- 172.21.15.133:0/3361486778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0034101420 msgr2=0x7f0034199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.137+0000 7f0038e14640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0034101420 0x7f0034199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.137+0000 7f0038e14640 1 -- 172.21.15.133:0/3361486778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0034100a80 msgr2=0x7f0034199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.137+0000 7f0038e14640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0034100a80 0x7f0034199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.137+0000 7f0038e14640 1 -- 172.21.15.133:0/3361486778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00341a48d0 con 0x7f0034106a80 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.137+0000 7f0038e14640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0034106a80 0x7f00341a4390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f002c00eaa0 tx=0x7f002c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.137+0000 7f00317fa640 1 -- 172.21.15.133:0/3361486778 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c00cdb0 con 0x7f0034106a80 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.137+0000 7f00317fa640 1 -- 172.21.15.133:0/3361486778 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f002c005020 con 0x7f0034106a80 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.137+0000 7f003a877640 1 -- 172.21.15.133:0/3361486778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00341a4aa0 con 0x7f0034106a80 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.137+0000 7f00317fa640 1 -- 172.21.15.133:0/3361486778 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0116b0 con 0x7f0034106a80 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.137+0000 7f003a877640 1 -- 172.21.15.133:0/3361486778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00341a4f60 con 0x7f0034106a80 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.138+0000 7f003a877640 1 -- 172.21.15.133:0/3361486778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0034100ee0 con 0x7f0034106a80 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.139+0000 7f00317fa640 1 -- 172.21.15.133:0/3361486778 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 138005+0+0 (secure 0 0 0) 0x7f002c005560 con 0x7f0034106a80 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.140+0000 7f00317fa640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f001c0a2e00 0x7f001c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.140+0000 7f0033fff640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f001c0a2e00 0x7f001c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.140+0000 7f00317fa640 1 -- 172.21.15.133:0/3361486778 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f002c0cfa70 con 0x7f0034106a80 2024-09-11T23:36:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.141+0000 7f0033fff640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f001c0a2e00 0x7f001c0a52c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0024009b50 tx=0x7f0024005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:51.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.143+0000 7f00317fa640 1 -- 172.21.15.133:0/3361486778 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f002c016030 con 0x7f0034106a80 2024-09-11T23:36:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:36:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi138 to 176.0M 2024-09-11T23:36:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: Unable to set osd_memory_target on smithi138 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:36:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T23:36:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T23:36:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-11T23:36:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: Unable to set osd_memory_target on smithi133 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-11T23:36:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-11T23:36:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1418885183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:51.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:51 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.343+0000 7f003a877640 1 -- 172.21.15.133:0/3361486778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0034108ff0 con 0x7f0034106a80 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi138 to 176.0M 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: Unable to set osd_memory_target on smithi138 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: Unable to set osd_memory_target on smithi133 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1418885183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:36:51.345 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.346 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.346 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.346 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.346 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.346 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.346 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.346 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:51.346 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:51.346 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:51.346 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:51 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.344+0000 7f00317fa640 1 -- 172.21.15.133:0/3361486778 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f002c09bfc0 con 0x7f0034106a80 2024-09-11T23:36:51.347 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:51.347 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:36:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:36:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.346+0000 7f003a877640 1 -- 172.21.15.133:0/3361486778 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f001c0a2e00 msgr2=0x7f001c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.346+0000 7f003a877640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f001c0a2e00 0x7f001c0a52c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0024009b50 tx=0x7f0024005f90 comp rx=0 tx=0).stop 2024-09-11T23:36:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.346+0000 7f003a877640 1 -- 172.21.15.133:0/3361486778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0034106a80 msgr2=0x7f00341a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.346+0000 7f003a877640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0034106a80 0x7f00341a4390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f002c00eaa0 tx=0x7f002c00ef60 comp rx=0 tx=0).stop 2024-09-11T23:36:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.347+0000 7f003a877640 1 -- 172.21.15.133:0/3361486778 shutdown_connections 2024-09-11T23:36:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.347+0000 7f003a877640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f001c0a2e00 0x7f001c0a52c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.347+0000 7f003a877640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0034106a80 0x7f00341a4390 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.347+0000 7f003a877640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0034101420 0x7f0034199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.347+0000 7f003a877640 1 --2- 172.21.15.133:0/3361486778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0034100a80 0x7f0034199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.347+0000 7f003a877640 1 -- 172.21.15.133:0/3361486778 >> 172.21.15.133:0/3361486778 conn(0x7f00340fc7a0 msgr2=0x7f00340fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.347+0000 7f003a877640 1 -- 172.21.15.133:0/3361486778 shutdown_connections 2024-09-11T23:36:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:51.347+0000 7f003a877640 1 -- 172.21.15.133:0/3361486778 wait complete. 2024-09-11T23:36:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:36:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi138 to 176.0M 2024-09-11T23:36:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: Unable to set osd_memory_target on smithi138 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:36:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T23:36:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T23:36:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-11T23:36:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: Unable to set osd_memory_target on smithi133 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-11T23:36:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-11T23:36:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-11T23:36:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-11T23:36:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1418885183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:51.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:51 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:52.163 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-11T23:36:52.164 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-11T23:36:52.164 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:36:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.272+0000 7fb555465640 1 -- 172.21.15.133:0/696473814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb55010c080 msgr2=0x7fb55010e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.272+0000 7fb555465640 1 --2- 172.21.15.133:0/696473814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb55010c080 0x7fb55010e540 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb548009ca0 tx=0x7fb54802a950 comp rx=0 tx=0).stop 2024-09-11T23:36:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.272+0000 7fb555465640 1 -- 172.21.15.133:0/696473814 shutdown_connections 2024-09-11T23:36:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.272+0000 7fb555465640 1 --2- 172.21.15.133:0/696473814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb55010c080 0x7fb55010e540 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.272+0000 7fb555465640 1 --2- 172.21.15.133:0/696473814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb550103580 0x7fb55010ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.272+0000 7fb555465640 1 --2- 172.21.15.133:0/696473814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb550102c60 0x7fb550103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.272+0000 7fb555465640 1 -- 172.21.15.133:0/696473814 >> 172.21.15.133:0/696473814 conn(0x7fb5500fc6d0 msgr2=0x7fb5500feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.272+0000 7fb555465640 1 -- 172.21.15.133:0/696473814 shutdown_connections 2024-09-11T23:36:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.272+0000 7fb555465640 1 -- 172.21.15.133:0/696473814 wait complete. 2024-09-11T23:36:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.272+0000 7fb555465640 1 Processor -- start 2024-09-11T23:36:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb555465640 1 -- start start 2024-09-11T23:36:52.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb555465640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb550102c60 0x7fb5501008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb555465640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb550103580 0x7fb550100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb555465640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb55010c080 0x7fb550101330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb555465640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb550111ea0 con 0x7fb550103580 2024-09-11T23:36:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb555465640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb550111d20 con 0x7fb55010c080 2024-09-11T23:36:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb555465640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb550112020 con 0x7fb550102c60 2024-09-11T23:36:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb5467fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb550103580 0x7fb550100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb5467fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb550103580 0x7fb550100df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33118/0 (socket says 172.21.15.133:33118) 2024-09-11T23:36:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb5467fc640 1 -- 172.21.15.133:0/1873181962 learned_addr learned my addr 172.21.15.133:0/1873181962 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb54effd640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb550102c60 0x7fb5501008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb54f7fe640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb55010c080 0x7fb550101330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb5467fc640 1 -- 172.21.15.133:0/1873181962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb550102c60 msgr2=0x7fb5501008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb5467fc640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb550102c60 0x7fb5501008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb5467fc640 1 -- 172.21.15.133:0/1873181962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb55010c080 msgr2=0x7fb550101330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb5467fc640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb55010c080 0x7fb550101330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb5467fc640 1 -- 172.21.15.133:0/1873181962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5501a6880 con 0x7fb550103580 2024-09-11T23:36:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb54f7fe640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb55010c080 0x7fb550101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.273+0000 7fb5467fc640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb550103580 0x7fb550100df0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb53800e9f0 tx=0x7fb53800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.274+0000 7fb54cff9640 1 -- 172.21.15.133:0/1873181962 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53800cd80 con 0x7fb550103580 2024-09-11T23:36:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.274+0000 7fb54cff9640 1 -- 172.21.15.133:0/1873181962 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb538010910 con 0x7fb550103580 2024-09-11T23:36:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.274+0000 7fb555465640 1 -- 172.21.15.133:0/1873181962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5501a6b70 con 0x7fb550103580 2024-09-11T23:36:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.274+0000 7fb555465640 1 -- 172.21.15.133:0/1873181962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5501a70c0 con 0x7fb550103580 2024-09-11T23:36:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.274+0000 7fb54cff9640 1 -- 172.21.15.133:0/1873181962 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538026060 con 0x7fb550103580 2024-09-11T23:36:52.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.275+0000 7fb555465640 1 -- 172.21.15.133:0/1873181962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb550103040 con 0x7fb550103580 2024-09-11T23:36:52.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.276+0000 7fb54cff9640 1 -- 172.21.15.133:0/1873181962 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fb538010ac0 con 0x7fb550103580 2024-09-11T23:36:52.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.277+0000 7fb54cff9640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb53c0a2e20 0x7fb53c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.277+0000 7fb54effd640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb53c0a2e20 0x7fb53c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.277+0000 7fb54cff9640 1 -- 172.21.15.133:0/1873181962 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fb5380d00a0 con 0x7fb550103580 2024-09-11T23:36:52.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.277+0000 7fb54effd640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb53c0a2e20 0x7fb53c0a52e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb54000bbe0 tx=0x7fb54000b390 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:52.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.280+0000 7fb54cff9640 1 -- 172.21.15.133:0/1873181962 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb53809cae0 con 0x7fb550103580 2024-09-11T23:36:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:52 smithi178 ceph-mon[35882]: Updating smithi133:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:36:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:52 smithi178 ceph-mon[35882]: Updating smithi138:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:36:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:52 smithi178 ceph-mon[35882]: Updating smithi178:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:36:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:52 smithi178 ceph-mon[35882]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:52 smithi178 ceph-mon[35882]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:52 smithi178 ceph-mon[35882]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:52 smithi178 ceph-mon[35882]: Updating smithi133:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:36:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:52 smithi178 ceph-mon[35882]: Updating smithi178:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:36:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:52 smithi178 ceph-mon[35882]: Updating smithi138:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:36:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:52 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3361486778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:52.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.442+0000 7fb555465640 1 -- 172.21.15.133:0/1873181962 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb550107150 con 0x7fb53c0a2e20 2024-09-11T23:36:52.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.448+0000 7fb54cff9640 1 -- 172.21.15.133:0/1873181962 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257810 (secure 0 0 0) 0x7fb540010c30 con 0x7fb53c0a2e20 2024-09-11T23:36:52.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:52 smithi138 ceph-mon[35142]: Updating smithi133:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:36:52.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:52 smithi138 ceph-mon[35142]: Updating smithi138:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:36:52.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:52 smithi138 ceph-mon[35142]: Updating smithi178:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:36:52.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:52 smithi138 ceph-mon[35142]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:52.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:52 smithi138 ceph-mon[35142]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:52.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:52 smithi138 ceph-mon[35142]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:52.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:52 smithi138 ceph-mon[35142]: Updating smithi133:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:36:52.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:52 smithi138 ceph-mon[35142]: Updating smithi178:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:36:52.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:52 smithi138 ceph-mon[35142]: Updating smithi138:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:36:52.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:52 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3361486778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:52.453 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:52.455 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T23:36:52.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 -- 172.21.15.133:0/1873181962 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb53c0a2e20 msgr2=0x7fb53c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb53c0a2e20 0x7fb53c0a52e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb54000bbe0 tx=0x7fb54000b390 comp rx=0 tx=0).stop 2024-09-11T23:36:52.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 -- 172.21.15.133:0/1873181962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb550103580 msgr2=0x7fb550100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb550103580 0x7fb550100df0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb53800e9f0 tx=0x7fb53800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 -- 172.21.15.133:0/1873181962 shutdown_connections 2024-09-11T23:36:52.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb53c0a2e20 0x7fb53c0a52e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb55010c080 0x7fb550101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb550103580 0x7fb550100df0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 --2- 172.21.15.133:0/1873181962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb550102c60 0x7fb5501008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 -- 172.21.15.133:0/1873181962 >> 172.21.15.133:0/1873181962 conn(0x7fb5500fc6d0 msgr2=0x7fb5500fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:52.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 -- 172.21.15.133:0/1873181962 shutdown_connections 2024-09-11T23:36:52.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.452+0000 7fb555465640 1 -- 172.21.15.133:0/1873181962 wait complete. 2024-09-11T23:36:52.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:52 smithi133 ceph-mon[31430]: Updating smithi133:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:36:52.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:52 smithi133 ceph-mon[31430]: Updating smithi138:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:36:52.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:52 smithi133 ceph-mon[31430]: Updating smithi178:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:36:52.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:52 smithi133 ceph-mon[31430]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:52.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:52 smithi133 ceph-mon[31430]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:52.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:52 smithi133 ceph-mon[31430]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:52.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:52 smithi133 ceph-mon[31430]: Updating smithi133:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:36:52.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:52 smithi133 ceph-mon[31430]: Updating smithi178:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:36:52.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:52 smithi133 ceph-mon[31430]: Updating smithi138:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:36:52.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:52 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3361486778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:52.462 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-11T23:36:51.069176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188048,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1318,"num_read_kb":822,"num_write":564,"num_write_kb":4404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359512,"kb_used_data":14388,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582312,"statfs":{"total":1151940427776,"available":1151572287488,"internally_reserved":0,"allocated":14733312,"data_stored":8867028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.989107"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327838+0000","last_change":"2024-09-11T23:34:52.867694+0000","last_active":"2024-09-11T23:36:47.327838+0000","last_peered":"2024-09-11T23:36:47.327838+0000","last_clean":"2024-09-11T23:36:47.327838+0000","last_became_active":"2024-09-11T23:34:52.867368+0000","last_became_peered":"2024-09-11T23:34:52.867368+0000","last_unstale":"2024-09-11T23:36:47.327838+0000","last_undegraded":"2024-09-11T23:36:47.327838+0000","last_fullsized":"2024-09-11T23:36:47.327838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:46:07.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327981+0000","last_change":"2024-09-11T23:34:52.871881+0000","last_active":"2024-09-11T23:36:47.327981+0000","last_peered":"2024-09-11T23:36:47.327981+0000","last_clean":"2024-09-11T23:36:47.327981+0000","last_became_active":"2024-09-11T23:34:52.871609+0000","last_became_peered":"2024-09-11T23:34:52.871609+0000","last_unstale":"2024-09-11T23:36:47.327981+0000","last_undegraded":"2024-09-11T23:36:47.327981+0000","last_fullsized":"2024-09-11T23:36:47.327981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.938584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327720+0000","last_change":"2024-09-11T23:34:52.863931+0000","last_active":"2024-09-11T23:36:47.327720+0000","last_peered":"2024-09-11T23:36:47.327720+0000","last_clean":"2024-09-11T23:36:47.327720+0000","last_became_active":"2024-09-11T23:34:52.863637+0000","last_became_peered":"2024-09-11T23:34:52.863637+0000","last_unstale":"2024-09-11T23:36:47.327720+0000","last_undegraded":"2024-09-11T23:36:47.327720+0000","last_fullsized":"2024-09-11T23:36:47.327720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:44.118865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292412+0000","last_change":"2024-09-11T23:34:52.879619+0000","last_active":"2024-09-11T23:36:47.292412+0000","last_peered":"2024-09-11T23:36:47.292412+0000","last_clean":"2024-09-11T23:36:47.292412+0000","last_became_active":"2024-09-11T23:34:52.878695+0000","last_became_peered":"2024-09-11T23:34:52.878695+0000","last_unstale":"2024-09-11T23:36:47.292412+0000","last_undegraded":"2024-09-11T23:36:47.292412+0000","last_fullsized":"2024-09-11T23:36:47.292412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:30.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145593+0000","last_change":"2024-09-11T23:34:52.869363+0000","last_active":"2024-09-11T23:36:47.145593+0000","last_peered":"2024-09-11T23:36:47.145593+0000","last_clean":"2024-09-11T23:36:47.145593+0000","last_became_active":"2024-09-11T23:34:52.869027+0000","last_became_peered":"2024-09-11T23:34:52.869027+0000","last_unstale":"2024-09-11T23:36:47.145593+0000","last_undegraded":"2024-09-11T23:36:47.145593+0000","last_fullsized":"2024-09-11T23:36:47.145593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:15.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156014+0000","last_change":"2024-09-11T23:34:52.880665+0000","last_active":"2024-09-11T23:36:47.156014+0000","last_peered":"2024-09-11T23:36:47.156014+0000","last_clean":"2024-09-11T23:36:47.156014+0000","last_became_active":"2024-09-11T23:34:52.880172+0000","last_became_peered":"2024-09-11T23:34:52.880172+0000","last_unstale":"2024-09-11T23:36:47.156014+0000","last_undegraded":"2024-09-11T23:36:47.156014+0000","last_fullsized":"2024-09-11T23:36:47.156014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:12.025877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152489+0000","last_change":"2024-09-11T23:34:52.873897+0000","last_active":"2024-09-11T23:36:47.152489+0000","last_peered":"2024-09-11T23:36:47.152489+0000","last_clean":"2024-09-11T23:36:47.152489+0000","last_became_active":"2024-09-11T23:34:52.873219+0000","last_became_peered":"2024-09-11T23:34:52.873219+0000","last_unstale":"2024-09-11T23:36:47.152489+0000","last_undegraded":"2024-09-11T23:36:47.152489+0000","last_fullsized":"2024-09-11T23:36:47.152489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:47.125682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145794+0000","last_change":"2024-09-11T23:34:52.870699+0000","last_active":"2024-09-11T23:36:47.145794+0000","last_peered":"2024-09-11T23:36:47.145794+0000","last_clean":"2024-09-11T23:36:47.145794+0000","last_became_active":"2024-09-11T23:34:52.870314+0000","last_became_peered":"2024-09-11T23:34:52.870314+0000","last_unstale":"2024-09-11T23:36:47.145794+0000","last_undegraded":"2024-09-11T23:36:47.145794+0000","last_fullsized":"2024-09-11T23:36:47.145794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:41.263159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058666+0000","last_change":"2024-09-11T23:34:52.869700+0000","last_active":"2024-09-11T23:36:47.058666+0000","last_peered":"2024-09-11T23:36:47.058666+0000","last_clean":"2024-09-11T23:36:47.058666+0000","last_became_active":"2024-09-11T23:34:52.869398+0000","last_became_peered":"2024-09-11T23:34:52.869398+0000","last_unstale":"2024-09-11T23:36:47.058666+0000","last_undegraded":"2024-09-11T23:36:47.058666+0000","last_fullsized":"2024-09-11T23:36:47.058666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:49.776525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156161+0000","last_change":"2024-09-11T23:34:52.879999+0000","last_active":"2024-09-11T23:36:47.156161+0000","last_peered":"2024-09-11T23:36:47.156161+0000","last_clean":"2024-09-11T23:36:47.156161+0000","last_became_active":"2024-09-11T23:34:52.879646+0000","last_became_peered":"2024-09-11T23:34:52.879646+0000","last_unstale":"2024-09-11T23:36:47.156161+0000","last_undegraded":"2024-09-11T23:36:47.156161+0000","last_fullsized":"2024-09-11T23:36:47.156161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.058704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152665+0000","last_change":"2024-09-11T23:34:52.878740+0000","last_active":"2024-09-11T23:36:47.152665+0000","last_peered":"2024-09-11T23:36:47.152665+0000","last_clean":"2024-09-11T23:36:47.152665+0000","last_became_active":"2024-09-11T23:34:52.878120+0000","last_became_peered":"2024-09-11T23:34:52.878120+0000","last_unstale":"2024-09-11T23:36:47.152665+0000","last_undegraded":"2024-09-11T23:36:47.152665+0000","last_fullsized":"2024-09-11T23:36:47.152665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292447+0000","last_change":"2024-09-11T23:34:52.873409+0000","last_active":"2024-09-11T23:36:47.292447+0000","last_peered":"2024-09-11T23:36:47.292447+0000","last_clean":"2024-09-11T23:36:47.292447+0000","last_became_active":"2024-09-11T23:34:52.872959+0000","last_became_peered":"2024-09-11T23:34:52.872959+0000","last_unstale":"2024-09-11T23:36:47.292447+0000","last_undegraded":"2024-09-11T23:36:47.292447+0000","last_fullsized":"2024-09-11T23:36:47.292447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:38.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057362+0000","last_change":"2024-09-11T23:34:52.869164+0000","last_active":"2024-09-11T23:36:47.057362+0000","last_peered":"2024-09-11T23:36:47.057362+0000","last_clean":"2024-09-11T23:36:47.057362+0000","last_became_active":"2024-09-11T23:34:52.868847+0000","last_became_peered":"2024-09-11T23:34:52.868847+0000","last_unstale":"2024-09-11T23:36:47.057362+0000","last_undegraded":"2024-09-11T23:36:47.057362+0000","last_fullsized":"2024-09-11T23:36:47.057362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.764510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327562+0000","last_change":"2024-09-11T23:34:52.869494+0000","last_active":"2024-09-11T23:36:47.327562+0000","last_peered":"2024-09-11T23:36:47.327562+0000","last_clean":"2024-09-11T23:36:47.327562+0000","last_became_active":"2024-09-11T23:34:52.869178+0000","last_became_peered":"2024-09-11T23:34:52.869178+0000","last_unstale":"2024-09-11T23:36:47.327562+0000","last_undegraded":"2024-09-11T23:36:47.327562+0000","last_fullsized":"2024-09-11T23:36:47.327562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:23.431564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327741+0000","last_change":"2024-09-11T23:34:52.866269+0000","last_active":"2024-09-11T23:36:47.327741+0000","last_peered":"2024-09-11T23:36:47.327741+0000","last_clean":"2024-09-11T23:36:47.327741+0000","last_became_active":"2024-09-11T23:34:52.865980+0000","last_became_peered":"2024-09-11T23:34:52.865980+0000","last_unstale":"2024-09-11T23:36:47.327741+0000","last_undegraded":"2024-09-11T23:36:47.327741+0000","last_fullsized":"2024-09-11T23:36:47.327741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:48.492252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058174+0000","last_change":"2024-09-11T23:34:52.862192+0000","last_active":"2024-09-11T23:36:47.058174+0000","last_peered":"2024-09-11T23:36:47.058174+0000","last_clean":"2024-09-11T23:36:47.058174+0000","last_became_active":"2024-09-11T23:34:52.861841+0000","last_became_peered":"2024-09-11T23:34:52.861841+0000","last_unstale":"2024-09-11T23:36:47.058174+0000","last_undegraded":"2024-09-11T23:36:47.058174+0000","last_fullsized":"2024-09-11T23:36:47.058174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.258948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327836+0000","last_change":"2024-09-11T23:34:52.870984+0000","last_active":"2024-09-11T23:36:47.327836+0000","last_peered":"2024-09-11T23:36:47.327836+0000","last_clean":"2024-09-11T23:36:47.327836+0000","last_became_active":"2024-09-11T23:34:52.870435+0000","last_became_peered":"2024-09-11T23:34:52.870435+0000","last_unstale":"2024-09-11T23:36:47.327836+0000","last_undegraded":"2024-09-11T23:36:47.327836+0000","last_fullsized":"2024-09-11T23:36:47.327836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:34.391537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143297+0000","last_change":"2024-09-11T23:34:52.873176+0000","last_active":"2024-09-11T23:36:47.143297+0000","last_peered":"2024-09-11T23:36:47.143297+0000","last_clean":"2024-09-11T23:36:47.143297+0000","last_became_active":"2024-09-11T23:34:52.872862+0000","last_became_peered":"2024-09-11T23:34:52.872862+0000","last_unstale":"2024-09-11T23:36:47.143297+0000","last_undegraded":"2024-09-11T23:36:47.143297+0000","last_fullsized":"2024-09-11T23:36:47.143297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:38.058806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058317+0000","last_change":"2024-09-11T23:34:52.871346+0000","last_active":"2024-09-11T23:36:47.058317+0000","last_peered":"2024-09-11T23:36:47.058317+0000","last_clean":"2024-09-11T23:36:47.058317+0000","last_became_active":"2024-09-11T23:34:52.870981+0000","last_became_peered":"2024-09-11T23:34:52.870981+0000","last_unstale":"2024-09-11T23:36:47.058317+0000","last_undegraded":"2024-09-11T23:36:47.058317+0000","last_fullsized":"2024-09-11T23:36:47.058317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:56.973998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327601+0000","last_change":"2024-09-11T23:34:52.867058+0000","last_active":"2024-09-11T23:36:47.327601+0000","last_peered":"2024-09-11T23:36:47.327601+0000","last_clean":"2024-09-11T23:36:47.327601+0000","last_became_active":"2024-09-11T23:34:52.866687+0000","last_became_peered":"2024-09-11T23:34:52.866687+0000","last_unstale":"2024-09-11T23:36:47.327601+0000","last_undegraded":"2024-09-11T23:36:47.327601+0000","last_fullsized":"2024-09-11T23:36:47.327601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:09.019987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057475+0000","last_change":"2024-09-11T23:34:52.870882+0000","last_active":"2024-09-11T23:36:47.057475+0000","last_peered":"2024-09-11T23:36:47.057475+0000","last_clean":"2024-09-11T23:36:47.057475+0000","last_became_active":"2024-09-11T23:34:52.870667+0000","last_became_peered":"2024-09-11T23:34:52.870667+0000","last_unstale":"2024-09-11T23:36:47.057475+0000","last_undegraded":"2024-09-11T23:36:47.057475+0000","last_fullsized":"2024-09-11T23:36:47.057475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:08.386703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058304+0000","last_change":"2024-09-11T23:34:52.874070+0000","last_active":"2024-09-11T23:36:47.058304+0000","last_peered":"2024-09-11T23:36:47.058304+0000","last_clean":"2024-09-11T23:36:47.058304+0000","last_became_active":"2024-09-11T23:34:52.873772+0000","last_became_peered":"2024-09-11T23:34:52.873772+0000","last_unstale":"2024-09-11T23:36:47.058304+0000","last_undegraded":"2024-09-11T23:36:47.058304+0000","last_fullsized":"2024-09-11T23:36:47.058304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:49.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146028+0000","last_change":"2024-09-11T23:34:52.875502+0000","last_active":"2024-09-11T23:36:47.146028+0000","last_peered":"2024-09-11T23:36:47.146028+0000","last_clean":"2024-09-11T23:36:47.146028+0000","last_became_active":"2024-09-11T23:34:52.875193+0000","last_became_peered":"2024-09-11T23:34:52.875193+0000","last_unstale":"2024-09-11T23:36:47.146028+0000","last_undegraded":"2024-09-11T23:36:47.146028+0000","last_fullsized":"2024-09-11T23:36:47.146028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:37.213828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153497+0000","last_change":"2024-09-11T23:34:52.869015+0000","last_active":"2024-09-11T23:36:47.153497+0000","last_peered":"2024-09-11T23:36:47.153497+0000","last_clean":"2024-09-11T23:36:47.153497+0000","last_became_active":"2024-09-11T23:34:52.868731+0000","last_became_peered":"2024-09-11T23:34:52.868731+0000","last_unstale":"2024-09-11T23:36:47.153497+0000","last_undegraded":"2024-09-11T23:36:47.153497+0000","last_fullsized":"2024-09-11T23:36:47.153497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:12:15.993203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147366+0000","last_change":"2024-09-11T23:34:52.875555+0000","last_active":"2024-09-11T23:36:47.147366+0000","last_peered":"2024-09-11T23:36:47.147366+0000","last_clean":"2024-09-11T23:36:47.147366+0000","last_became_active":"2024-09-11T23:34:52.875219+0000","last_became_peered":"2024-09-11T23:34:52.875219+0000","last_unstale":"2024-09-11T23:36:47.147366+0000","last_undegraded":"2024-09-11T23:36:47.147366+0000","last_fullsized":"2024-09-11T23:36:47.147366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:16.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058480+0000","last_change":"2024-09-11T23:34:52.875629+0000","last_active":"2024-09-11T23:36:47.058480+0000","last_peered":"2024-09-11T23:36:47.058480+0000","last_clean":"2024-09-11T23:36:47.058480+0000","last_became_active":"2024-09-11T23:34:52.875302+0000","last_became_peered":"2024-09-11T23:34:52.875302+0000","last_unstale":"2024-09-11T23:36:47.058480+0000","last_undegraded":"2024-09-11T23:36:47.058480+0000","last_fullsized":"2024-09-11T23:36:47.058480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:12.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327744+0000","last_change":"2024-09-11T23:34:52.868613+0000","last_active":"2024-09-11T23:36:47.327744+0000","last_peered":"2024-09-11T23:36:47.327744+0000","last_clean":"2024-09-11T23:36:47.327744+0000","last_became_active":"2024-09-11T23:34:52.868184+0000","last_became_peered":"2024-09-11T23:34:52.868184+0000","last_unstale":"2024-09-11T23:36:47.327744+0000","last_undegraded":"2024-09-11T23:36:47.327744+0000","last_fullsized":"2024-09-11T23:36:47.327744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:25.990002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057237+0000","last_change":"2024-09-11T23:35:08.873390+0000","last_active":"2024-09-11T23:36:47.057237+0000","last_peered":"2024-09-11T23:36:47.057237+0000","last_clean":"2024-09-11T23:36:47.057237+0000","last_became_active":"2024-09-11T23:35:08.873052+0000","last_became_peered":"2024-09-11T23:35:08.873052+0000","last_unstale":"2024-09-11T23:36:47.057237+0000","last_undegraded":"2024-09-11T23:36:47.057237+0000","last_fullsized":"2024-09-11T23:36:47.057237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.937951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327689+0000","last_change":"2024-09-11T23:34:52.873604+0000","last_active":"2024-09-11T23:36:47.327689+0000","last_peered":"2024-09-11T23:36:47.327689+0000","last_clean":"2024-09-11T23:36:47.327689+0000","last_became_active":"2024-09-11T23:34:52.873220+0000","last_became_peered":"2024-09-11T23:34:52.873220+0000","last_unstale":"2024-09-11T23:36:47.327689+0000","last_undegraded":"2024-09-11T23:36:47.327689+0000","last_fullsized":"2024-09-11T23:36:47.327689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:38.480630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058025+0000","last_change":"2024-09-11T23:34:52.872902+0000","last_active":"2024-09-11T23:36:47.058025+0000","last_peered":"2024-09-11T23:36:47.058025+0000","last_clean":"2024-09-11T23:36:47.058025+0000","last_became_active":"2024-09-11T23:34:52.872571+0000","last_became_peered":"2024-09-11T23:34:52.872571+0000","last_unstale":"2024-09-11T23:36:47.058025+0000","last_undegraded":"2024-09-11T23:36:47.058025+0000","last_fullsized":"2024-09-11T23:36:47.058025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:21.844995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058115+0000","last_change":"2024-09-11T23:35:09.457056+0000","last_active":"2024-09-11T23:36:47.058115+0000","last_peered":"2024-09-11T23:36:47.058115+0000","last_clean":"2024-09-11T23:36:47.058115+0000","last_became_active":"2024-09-11T23:35:09.456657+0000","last_became_peered":"2024-09-11T23:35:09.456657+0000","last_unstale":"2024-09-11T23:36:47.058115+0000","last_undegraded":"2024-09-11T23:36:47.058115+0000","last_fullsized":"2024-09-11T23:36:47.058115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:12:56.121280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327466+0000","last_change":"2024-09-11T23:34:52.875312+0000","last_active":"2024-09-11T23:36:47.327466+0000","last_peered":"2024-09-11T23:36:47.327466+0000","last_clean":"2024-09-11T23:36:47.327466+0000","last_became_active":"2024-09-11T23:34:52.874873+0000","last_became_peered":"2024-09-11T23:34:52.874873+0000","last_unstale":"2024-09-11T23:36:47.327466+0000","last_undegraded":"2024-09-11T23:36:47.327466+0000","last_fullsized":"2024-09-11T23:36:47.327466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:18.894713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145537+0000","last_change":"2024-09-11T23:34:52.862266+0000","last_active":"2024-09-11T23:36:47.145537+0000","last_peered":"2024-09-11T23:36:47.145537+0000","last_clean":"2024-09-11T23:36:47.145537+0000","last_became_active":"2024-09-11T23:34:52.861923+0000","last_became_peered":"2024-09-11T23:34:52.861923+0000","last_unstale":"2024-09-11T23:36:47.145537+0000","last_undegraded":"2024-09-11T23:36:47.145537+0000","last_fullsized":"2024-09-11T23:36:47.145537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:23.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":138,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.591379+0000","last_change":"2024-09-11T23:34:50.854020+0000","last_active":"2024-09-11T23:36:47.591379+0000","last_peered":"2024-09-11T23:36:47.591379+0000","last_clean":"2024-09-11T23:36:47.591379+0000","last_became_active":"2024-09-11T23:34:50.853683+0000","last_became_peered":"2024-09-11T23:34:50.853683+0000","last_unstale":"2024-09-11T23:36:47.591379+0000","last_undegraded":"2024-09-11T23:36:47.591379+0000","last_fullsized":"2024-09-11T23:36:47.591379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:18.725495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153611+0000","last_change":"2024-09-11T23:34:52.874016+0000","last_active":"2024-09-11T23:36:47.153611+0000","last_peered":"2024-09-11T23:36:47.153611+0000","last_clean":"2024-09-11T23:36:47.153611+0000","last_became_active":"2024-09-11T23:34:52.873675+0000","last_became_peered":"2024-09-11T23:34:52.873675+0000","last_unstale":"2024-09-11T23:36:47.153611+0000","last_undegraded":"2024-09-11T23:36:47.153611+0000","last_fullsized":"2024-09-11T23:36:47.153611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:41:28.955548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156082+0000","last_change":"2024-09-11T23:34:50.855391+0000","last_active":"2024-09-11T23:36:47.156082+0000","last_peered":"2024-09-11T23:36:47.156082+0000","last_clean":"2024-09-11T23:36:47.156082+0000","last_became_active":"2024-09-11T23:34:50.855052+0000","last_became_peered":"2024-09-11T23:34:50.855052+0000","last_unstale":"2024-09-11T23:36:47.156082+0000","last_undegraded":"2024-09-11T23:36:47.156082+0000","last_fullsized":"2024-09-11T23:36:47.156082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.972239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057065+0000","last_change":"2024-09-11T23:34:50.848985+0000","last_active":"2024-09-11T23:36:47.057065+0000","last_peered":"2024-09-11T23:36:47.057065+0000","last_clean":"2024-09-11T23:36:47.057065+0000","last_became_active":"2024-09-11T23:34:50.848614+0000","last_became_peered":"2024-09-11T23:34:50.848614+0000","last_unstale":"2024-09-11T23:36:47.057065+0000","last_undegraded":"2024-09-11T23:36:47.057065+0000","last_fullsized":"2024-09-11T23:36:47.057065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:43:04.723136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146018+0000","last_change":"2024-09-11T23:34:52.866933+0000","last_active":"2024-09-11T23:36:47.146018+0000","last_peered":"2024-09-11T23:36:47.146018+0000","last_clean":"2024-09-11T23:36:47.146018+0000","last_became_active":"2024-09-11T23:34:52.866559+0000","last_became_peered":"2024-09-11T23:34:52.866559+0000","last_unstale":"2024-09-11T23:36:47.146018+0000","last_undegraded":"2024-09-11T23:36:47.146018+0000","last_fullsized":"2024-09-11T23:36:47.146018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:27.745118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153745+0000","last_change":"2024-09-11T23:34:50.849787+0000","last_active":"2024-09-11T23:36:47.153745+0000","last_peered":"2024-09-11T23:36:47.153745+0000","last_clean":"2024-09-11T23:36:47.153745+0000","last_became_active":"2024-09-11T23:34:50.849368+0000","last_became_peered":"2024-09-11T23:34:50.849368+0000","last_unstale":"2024-09-11T23:36:47.153745+0000","last_undegraded":"2024-09-11T23:36:47.153745+0000","last_fullsized":"2024-09-11T23:36:47.153745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:26.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142558+0000","last_change":"2024-09-11T23:34:52.870832+0000","last_active":"2024-09-11T23:36:47.142558+0000","last_peered":"2024-09-11T23:36:47.142558+0000","last_clean":"2024-09-11T23:36:47.142558+0000","last_became_active":"2024-09-11T23:34:52.870474+0000","last_became_peered":"2024-09-11T23:34:52.870474+0000","last_unstale":"2024-09-11T23:36:47.142558+0000","last_undegraded":"2024-09-11T23:36:47.142558+0000","last_fullsized":"2024-09-11T23:36:47.142558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:30.196114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":238,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.590875+0000","last_change":"2024-09-11T23:34:50.859074+0000","last_active":"2024-09-11T23:36:47.590875+0000","last_peered":"2024-09-11T23:36:47.590875+0000","last_clean":"2024-09-11T23:36:47.590875+0000","last_became_active":"2024-09-11T23:34:50.858664+0000","last_became_peered":"2024-09-11T23:34:50.858664+0000","last_unstale":"2024-09-11T23:36:47.590875+0000","last_undegraded":"2024-09-11T23:36:47.590875+0000","last_fullsized":"2024-09-11T23:36:47.590875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:04.077813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058577+0000","last_change":"2024-09-11T23:34:52.875901+0000","last_active":"2024-09-11T23:36:47.058577+0000","last_peered":"2024-09-11T23:36:47.058577+0000","last_clean":"2024-09-11T23:36:47.058577+0000","last_became_active":"2024-09-11T23:34:52.875568+0000","last_became_peered":"2024-09-11T23:34:52.875568+0000","last_unstale":"2024-09-11T23:36:47.058577+0000","last_undegraded":"2024-09-11T23:36:47.058577+0000","last_fullsized":"2024-09-11T23:36:47.058577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:54.173548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152813+0000","last_change":"2024-09-11T23:34:50.854957+0000","last_active":"2024-09-11T23:36:47.152813+0000","last_peered":"2024-09-11T23:36:47.152813+0000","last_clean":"2024-09-11T23:36:47.152813+0000","last_became_active":"2024-09-11T23:34:50.854626+0000","last_became_peered":"2024-09-11T23:34:50.854626+0000","last_unstale":"2024-09-11T23:36:47.152813+0000","last_undegraded":"2024-09-11T23:36:47.152813+0000","last_fullsized":"2024-09-11T23:36:47.152813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:53.738966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328149+0000","last_change":"2024-09-11T23:34:52.868850+0000","last_active":"2024-09-11T23:36:47.328149+0000","last_peered":"2024-09-11T23:36:47.328149+0000","last_clean":"2024-09-11T23:36:47.328149+0000","last_became_active":"2024-09-11T23:34:52.868536+0000","last_became_peered":"2024-09-11T23:34:52.868536+0000","last_unstale":"2024-09-11T23:36:47.328149+0000","last_undegraded":"2024-09-11T23:36:47.328149+0000","last_fullsized":"2024-09-11T23:36:47.328149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:11.746821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153829+0000","last_change":"2024-09-11T23:34:50.924819+0000","last_active":"2024-09-11T23:36:47.153829+0000","last_peered":"2024-09-11T23:36:47.153829+0000","last_clean":"2024-09-11T23:36:47.153829+0000","last_became_active":"2024-09-11T23:34:50.924283+0000","last_became_peered":"2024-09-11T23:34:50.924283+0000","last_unstale":"2024-09-11T23:36:47.153829+0000","last_undegraded":"2024-09-11T23:36:47.153829+0000","last_fullsized":"2024-09-11T23:36:47.153829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:41.345233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058417+0000","last_change":"2024-09-11T23:34:52.867113+0000","last_active":"2024-09-11T23:36:47.058417+0000","last_peered":"2024-09-11T23:36:47.058417+0000","last_clean":"2024-09-11T23:36:47.058417+0000","last_became_active":"2024-09-11T23:34:52.866125+0000","last_became_peered":"2024-09-11T23:34:52.866125+0000","last_unstale":"2024-09-11T23:36:47.058417+0000","last_undegraded":"2024-09-11T23:36:47.058417+0000","last_fullsized":"2024-09-11T23:36:47.058417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:18:55.615079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'36","reported_seq":282,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.590874+0000","last_change":"2024-09-11T23:34:50.857272+0000","last_active":"2024-09-11T23:36:47.590874+0000","last_peered":"2024-09-11T23:36:47.590874+0000","last_clean":"2024-09-11T23:36:47.590874+0000","last_became_active":"2024-09-11T23:34:50.855701+0000","last_became_peered":"2024-09-11T23:34:50.855701+0000","last_unstale":"2024-09-11T23:36:47.590874+0000","last_undegraded":"2024-09-11T23:36:47.590874+0000","last_fullsized":"2024-09-11T23:36:47.590874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:49.997617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62867,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":222,"num_read_kb":172,"num_write":38,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145783+0000","last_change":"2024-09-11T23:34:52.876928+0000","last_active":"2024-09-11T23:36:47.145783+0000","last_peered":"2024-09-11T23:36:47.145783+0000","last_clean":"2024-09-11T23:36:47.145783+0000","last_became_active":"2024-09-11T23:34:52.876578+0000","last_became_peered":"2024-09-11T23:34:52.876578+0000","last_unstale":"2024-09-11T23:36:47.145783+0000","last_undegraded":"2024-09-11T23:36:47.145783+0000","last_fullsized":"2024-09-11T23:36:47.145783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:07.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327515+0000","last_change":"2024-09-11T23:35:08.869275+0000","last_active":"2024-09-11T23:36:47.327515+0000","last_peered":"2024-09-11T23:36:47.327515+0000","last_clean":"2024-09-11T23:36:47.327515+0000","last_became_active":"2024-09-11T23:35:08.850112+0000","last_became_peered":"2024-09-11T23:35:08.850112+0000","last_unstale":"2024-09-11T23:36:47.327515+0000","last_undegraded":"2024-09-11T23:36:47.327515+0000","last_fullsized":"2024-09-11T23:36:47.327515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:36.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057105+0000","last_change":"2024-09-11T23:34:52.871970+0000","last_active":"2024-09-11T23:36:47.057105+0000","last_peered":"2024-09-11T23:36:47.057105+0000","last_clean":"2024-09-11T23:36:47.057105+0000","last_became_active":"2024-09-11T23:34:52.871488+0000","last_became_peered":"2024-09-11T23:34:52.871488+0000","last_unstale":"2024-09-11T23:36:47.057105+0000","last_undegraded":"2024-09-11T23:36:47.057105+0000","last_fullsized":"2024-09-11T23:36:47.057105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:08.266431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153627+0000","last_change":"2024-09-11T23:34:50.854023+0000","last_active":"2024-09-11T23:36:47.153627+0000","last_peered":"2024-09-11T23:36:47.153627+0000","last_clean":"2024-09-11T23:36:47.153627+0000","last_became_active":"2024-09-11T23:34:50.853661+0000","last_became_peered":"2024-09-11T23:34:50.853661+0000","last_unstale":"2024-09-11T23:36:47.153627+0000","last_undegraded":"2024-09-11T23:36:47.153627+0000","last_fullsized":"2024-09-11T23:36:47.153627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:25.281976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152646+0000","last_change":"2024-09-11T23:35:08.849648+0000","last_active":"2024-09-11T23:36:47.152646+0000","last_peered":"2024-09-11T23:36:47.152646+0000","last_clean":"2024-09-11T23:36:47.152646+0000","last_became_active":"2024-09-11T23:35:08.849246+0000","last_became_peered":"2024-09-11T23:35:08.849246+0000","last_unstale":"2024-09-11T23:36:47.152646+0000","last_undegraded":"2024-09-11T23:36:47.152646+0000","last_fullsized":"2024-09-11T23:36:47.152646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:17.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'10","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.609150+0000","last_change":"2024-09-11T23:34:50.853855+0000","last_active":"2024-09-11T23:36:47.609150+0000","last_peered":"2024-09-11T23:36:47.609150+0000","last_clean":"2024-09-11T23:36:47.609150+0000","last_became_active":"2024-09-11T23:34:50.853493+0000","last_became_peered":"2024-09-11T23:34:50.853493+0000","last_unstale":"2024-09-11T23:36:47.609150+0000","last_undegraded":"2024-09-11T23:36:47.609150+0000","last_fullsized":"2024-09-11T23:36:47.609150+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:23.262472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":7,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152450+0000","last_change":"2024-09-11T23:34:52.875835+0000","last_active":"2024-09-11T23:36:47.152450+0000","last_peered":"2024-09-11T23:36:47.152450+0000","last_clean":"2024-09-11T23:36:47.152450+0000","last_became_active":"2024-09-11T23:34:52.875525+0000","last_became_peered":"2024-09-11T23:34:52.875525+0000","last_unstale":"2024-09-11T23:36:47.152450+0000","last_undegraded":"2024-09-11T23:36:47.152450+0000","last_fullsized":"2024-09-11T23:36:47.152450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:41.031816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145473+0000","last_change":"2024-09-11T23:34:50.924803+0000","last_active":"2024-09-11T23:36:47.145473+0000","last_peered":"2024-09-11T23:36:47.145473+0000","last_clean":"2024-09-11T23:36:47.145473+0000","last_became_active":"2024-09-11T23:34:50.924217+0000","last_became_peered":"2024-09-11T23:34:50.924217+0000","last_unstale":"2024-09-11T23:36:47.145473+0000","last_undegraded":"2024-09-11T23:36:47.145473+0000","last_fullsized":"2024-09-11T23:36:47.145473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:45.790592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142408+0000","last_change":"2024-09-11T23:35:08.851856+0000","last_active":"2024-09-11T23:36:47.142408+0000","last_peered":"2024-09-11T23:36:47.142408+0000","last_clean":"2024-09-11T23:36:47.142408+0000","last_became_active":"2024-09-11T23:35:08.851490+0000","last_became_peered":"2024-09-11T23:35:08.851490+0000","last_unstale":"2024-09-11T23:36:47.142408+0000","last_undegraded":"2024-09-11T23:36:47.142408+0000","last_fullsized":"2024-09-11T23:36:47.142408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:46.269022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"81'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145816+0000","last_change":"2024-09-11T23:34:50.848385+0000","last_active":"2024-09-11T23:36:47.145816+0000","last_peered":"2024-09-11T23:36:47.145816+0000","last_clean":"2024-09-11T23:36:47.145816+0000","last_became_active":"2024-09-11T23:34:50.847808+0000","last_became_peered":"2024-09-11T23:34:50.847808+0000","last_unstale":"2024-09-11T23:36:47.145816+0000","last_undegraded":"2024-09-11T23:36:47.145816+0000","last_fullsized":"2024-09-11T23:36:47.145816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:38.796622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327734+0000","last_change":"2024-09-11T23:34:52.867341+0000","last_active":"2024-09-11T23:36:47.327734+0000","last_peered":"2024-09-11T23:36:47.327734+0000","last_clean":"2024-09-11T23:36:47.327734+0000","last_became_active":"2024-09-11T23:34:52.866161+0000","last_became_peered":"2024-09-11T23:34:52.866161+0000","last_unstale":"2024-09-11T23:36:47.327734+0000","last_undegraded":"2024-09-11T23:36:47.327734+0000","last_fullsized":"2024-09-11T23:36:47.327734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:27.913705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057317+0000","last_change":"2024-09-11T23:34:50.851466+0000","last_active":"2024-09-11T23:36:47.057317+0000","last_peered":"2024-09-11T23:36:47.057317+0000","last_clean":"2024-09-11T23:36:47.057317+0000","last_became_active":"2024-09-11T23:34:50.851191+0000","last_became_peered":"2024-09-11T23:34:50.851191+0000","last_unstale":"2024-09-11T23:36:47.057317+0000","last_undegraded":"2024-09-11T23:36:47.057317+0000","last_fullsized":"2024-09-11T23:36:47.057317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:00.328658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152878+0000","last_change":"2024-09-11T23:34:52.872290+0000","last_active":"2024-09-11T23:36:47.152878+0000","last_peered":"2024-09-11T23:36:47.152878+0000","last_clean":"2024-09-11T23:36:47.152878+0000","last_became_active":"2024-09-11T23:34:52.871935+0000","last_became_peered":"2024-09-11T23:34:52.871935+0000","last_unstale":"2024-09-11T23:36:47.152878+0000","last_undegraded":"2024-09-11T23:36:47.152878+0000","last_fullsized":"2024-09-11T23:36:47.152878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:08.217548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'41","reported_seq":73,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.248156+0000","last_change":"2024-09-11T23:34:50.857763+0000","last_active":"2024-09-11T23:36:47.248156+0000","last_peered":"2024-09-11T23:36:47.248156+0000","last_clean":"2024-09-11T23:36:47.248156+0000","last_became_active":"2024-09-11T23:34:50.857427+0000","last_became_peered":"2024-09-11T23:34:50.857427+0000","last_unstale":"2024-09-11T23:36:47.248156+0000","last_undegraded":"2024-09-11T23:36:47.248156+0000","last_fullsized":"2024-09-11T23:36:47.248156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:59.703580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152618+0000","last_change":"2024-09-11T23:34:52.872426+0000","last_active":"2024-09-11T23:36:47.152618+0000","last_peered":"2024-09-11T23:36:47.152618+0000","last_clean":"2024-09-11T23:36:47.152618+0000","last_became_active":"2024-09-11T23:34:52.872090+0000","last_became_peered":"2024-09-11T23:34:52.872090+0000","last_unstale":"2024-09-11T23:36:47.152618+0000","last_undegraded":"2024-09-11T23:36:47.152618+0000","last_fullsized":"2024-09-11T23:36:47.152618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:04:00.963963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'7","reported_seq":138,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058454+0000","last_change":"2024-09-11T23:34:50.924792+0000","last_active":"2024-09-11T23:36:47.058454+0000","last_peered":"2024-09-11T23:36:47.058454+0000","last_clean":"2024-09-11T23:36:47.058454+0000","last_became_active":"2024-09-11T23:34:50.924444+0000","last_became_peered":"2024-09-11T23:34:50.924444+0000","last_unstale":"2024-09-11T23:36:47.058454+0000","last_undegraded":"2024-09-11T23:36:47.058454+0000","last_fullsized":"2024-09-11T23:36:47.058454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:12.855314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152787+0000","last_change":"2024-09-11T23:34:52.872079+0000","last_active":"2024-09-11T23:36:47.152787+0000","last_peered":"2024-09-11T23:36:47.152787+0000","last_clean":"2024-09-11T23:36:47.152787+0000","last_became_active":"2024-09-11T23:34:52.871755+0000","last_became_peered":"2024-09-11T23:34:52.871755+0000","last_unstale":"2024-09-11T23:36:47.152787+0000","last_undegraded":"2024-09-11T23:36:47.152787+0000","last_fullsized":"2024-09-11T23:36:47.152787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:05.990832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":347,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152603+0000","last_change":"2024-09-11T23:33:42.256521+0000","last_active":"2024-09-11T23:36:47.152603+0000","last_peered":"2024-09-11T23:36:47.152603+0000","last_clean":"2024-09-11T23:36:47.152603+0000","last_became_active":"2024-09-11T23:33:42.256041+0000","last_became_peered":"2024-09-11T23:33:42.256041+0000","last_unstale":"2024-09-11T23:36:47.152603+0000","last_undegraded":"2024-09-11T23:36:47.152603+0000","last_fullsized":"2024-09-11T23:36:47.152603+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_clean_scrub_stamp":"2024-09-11T23:30:22.061515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:21.391839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292288+0000","last_change":"2024-09-11T23:34:50.854087+0000","last_active":"2024-09-11T23:36:47.292288+0000","last_peered":"2024-09-11T23:36:47.292288+0000","last_clean":"2024-09-11T23:36:47.292288+0000","last_became_active":"2024-09-11T23:34:50.852346+0000","last_became_peered":"2024-09-11T23:34:50.852346+0000","last_unstale":"2024-09-11T23:36:47.292288+0000","last_undegraded":"2024-09-11T23:36:47.292288+0000","last_fullsized":"2024-09-11T23:36:47.292288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.287119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146988+0000","last_change":"2024-09-11T23:34:52.869198+0000","last_active":"2024-09-11T23:36:47.146988+0000","last_peered":"2024-09-11T23:36:47.146988+0000","last_clean":"2024-09-11T23:36:47.146988+0000","last_became_active":"2024-09-11T23:34:52.868621+0000","last_became_peered":"2024-09-11T23:34:52.868621+0000","last_unstale":"2024-09-11T23:36:47.146988+0000","last_undegraded":"2024-09-11T23:36:47.146988+0000","last_fullsized":"2024-09-11T23:36:47.146988+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:17.430654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152604+0000","last_change":"2024-09-11T23:34:50.852752+0000","last_active":"2024-09-11T23:36:47.152604+0000","last_peered":"2024-09-11T23:36:47.152604+0000","last_clean":"2024-09-11T23:36:47.152604+0000","last_became_active":"2024-09-11T23:34:50.851895+0000","last_became_peered":"2024-09-11T23:34:50.851895+0000","last_unstale":"2024-09-11T23:36:47.152604+0000","last_undegraded":"2024-09-11T23:36:47.152604+0000","last_fullsized":"2024-09-11T23:36:47.152604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:46.894432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057056+0000","last_change":"2024-09-11T23:34:52.866191+0000","last_active":"2024-09-11T23:36:47.057056+0000","last_peered":"2024-09-11T23:36:47.057056+0000","last_clean":"2024-09-11T23:36:47.057056+0000","last_became_active":"2024-09-11T23:34:52.865858+0000","last_became_peered":"2024-09-11T23:34:52.865858+0000","last_unstale":"2024-09-11T23:36:47.057056+0000","last_undegraded":"2024-09-11T23:36:47.057056+0000","last_fullsized":"2024-09-11T23:36:47.057056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:21.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":136,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057262+0000","last_change":"2024-09-11T23:34:50.924125+0000","last_active":"2024-09-11T23:36:47.057262+0000","last_peered":"2024-09-11T23:36:47.057262+0000","last_clean":"2024-09-11T23:36:47.057262+0000","last_became_active":"2024-09-11T23:34:50.923780+0000","last_became_peered":"2024-09-11T23:34:50.923780+0000","last_unstale":"2024-09-11T23:36:47.057262+0000","last_undegraded":"2024-09-11T23:36:47.057262+0000","last_fullsized":"2024-09-11T23:36:47.057262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:39.774827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153672+0000","last_change":"2024-09-11T23:34:52.868364+0000","last_active":"2024-09-11T23:36:47.153672+0000","last_peered":"2024-09-11T23:36:47.153672+0000","last_clean":"2024-09-11T23:36:47.153672+0000","last_became_active":"2024-09-11T23:34:52.867913+0000","last_became_peered":"2024-09-11T23:34:52.867913+0000","last_unstale":"2024-09-11T23:36:47.153672+0000","last_undegraded":"2024-09-11T23:36:47.153672+0000","last_fullsized":"2024-09-11T23:36:47.153672+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.571825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":136,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.602771+0000","last_change":"2024-09-11T23:34:50.852421+0000","last_active":"2024-09-11T23:36:47.602771+0000","last_peered":"2024-09-11T23:36:47.602771+0000","last_clean":"2024-09-11T23:36:47.602771+0000","last_became_active":"2024-09-11T23:34:50.851898+0000","last_became_peered":"2024-09-11T23:34:50.851898+0000","last_unstale":"2024-09-11T23:36:47.602771+0000","last_undegraded":"2024-09-11T23:36:47.602771+0000","last_fullsized":"2024-09-11T23:36:47.602771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.661276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156359+0000","last_change":"2024-09-11T23:34:52.879995+0000","last_active":"2024-09-11T23:36:47.156359+0000","last_peered":"2024-09-11T23:36:47.156359+0000","last_clean":"2024-09-11T23:36:47.156359+0000","last_became_active":"2024-09-11T23:34:52.879678+0000","last_became_peered":"2024-09-11T23:34:52.879678+0000","last_unstale":"2024-09-11T23:36:47.156359+0000","last_undegraded":"2024-09-11T23:36:47.156359+0000","last_fullsized":"2024-09-11T23:36:47.156359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:47.456674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143075+0000","last_change":"2024-09-11T23:34:50.924957+0000","last_active":"2024-09-11T23:36:47.143075+0000","last_peered":"2024-09-11T23:36:47.143075+0000","last_clean":"2024-09-11T23:36:47.143075+0000","last_became_active":"2024-09-11T23:34:50.924500+0000","last_became_peered":"2024-09-11T23:34:50.924500+0000","last_unstale":"2024-09-11T23:36:47.143075+0000","last_undegraded":"2024-09-11T23:36:47.143075+0000","last_fullsized":"2024-09-11T23:36:47.143075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:39.268842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156159+0000","last_change":"2024-09-11T23:34:52.867337+0000","last_active":"2024-09-11T23:36:47.156159+0000","last_peered":"2024-09-11T23:36:47.156159+0000","last_clean":"2024-09-11T23:36:47.156159+0000","last_became_active":"2024-09-11T23:34:52.866799+0000","last_became_peered":"2024-09-11T23:34:52.866799+0000","last_unstale":"2024-09-11T23:36:47.156159+0000","last_undegraded":"2024-09-11T23:36:47.156159+0000","last_fullsized":"2024-09-11T23:36:47.156159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:38.177135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145920+0000","last_change":"2024-09-11T23:34:50.925156+0000","last_active":"2024-09-11T23:36:47.145920+0000","last_peered":"2024-09-11T23:36:47.145920+0000","last_clean":"2024-09-11T23:36:47.145920+0000","last_became_active":"2024-09-11T23:34:50.924282+0000","last_became_peered":"2024-09-11T23:34:50.924282+0000","last_unstale":"2024-09-11T23:36:47.145920+0000","last_undegraded":"2024-09-11T23:36:47.145920+0000","last_fullsized":"2024-09-11T23:36:47.145920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:47.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057264+0000","last_change":"2024-09-11T23:34:52.864993+0000","last_active":"2024-09-11T23:36:47.057264+0000","last_peered":"2024-09-11T23:36:47.057264+0000","last_clean":"2024-09-11T23:36:47.057264+0000","last_became_active":"2024-09-11T23:34:52.864267+0000","last_became_peered":"2024-09-11T23:34:52.864267+0000","last_unstale":"2024-09-11T23:36:47.057264+0000","last_undegraded":"2024-09-11T23:36:47.057264+0000","last_fullsized":"2024-09-11T23:36:47.057264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:29.912068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'52","reported_seq":99,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.250482+0000","last_change":"2024-09-11T23:34:50.849864+0000","last_active":"2024-09-11T23:36:47.250482+0000","last_peered":"2024-09-11T23:36:47.250482+0000","last_clean":"2024-09-11T23:36:47.250482+0000","last_became_active":"2024-09-11T23:34:50.849577+0000","last_became_peered":"2024-09-11T23:34:50.849577+0000","last_unstale":"2024-09-11T23:36:47.250482+0000","last_undegraded":"2024-09-11T23:36:47.250482+0000","last_fullsized":"2024-09-11T23:36:47.250482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:06.845000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147479+0000","last_change":"2024-09-11T23:34:52.872028+0000","last_active":"2024-09-11T23:36:47.147479+0000","last_peered":"2024-09-11T23:36:47.147479+0000","last_clean":"2024-09-11T23:36:47.147479+0000","last_became_active":"2024-09-11T23:34:52.871723+0000","last_became_peered":"2024-09-11T23:34:52.871723+0000","last_unstale":"2024-09-11T23:36:47.147479+0000","last_undegraded":"2024-09-11T23:36:47.147479+0000","last_fullsized":"2024-09-11T23:36:47.147479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:44.656218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292619+0000","last_change":"2024-09-11T23:34:50.860314+0000","last_active":"2024-09-11T23:36:47.292619+0000","last_peered":"2024-09-11T23:36:47.292619+0000","last_clean":"2024-09-11T23:36:47.292619+0000","last_became_active":"2024-09-11T23:34:50.859953+0000","last_became_peered":"2024-09-11T23:34:50.859953+0000","last_unstale":"2024-09-11T23:36:47.292619+0000","last_undegraded":"2024-09-11T23:36:47.292619+0000","last_fullsized":"2024-09-11T23:36:47.292619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:46.705864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328008+0000","last_change":"2024-09-11T23:34:52.867190+0000","last_active":"2024-09-11T23:36:47.328008+0000","last_peered":"2024-09-11T23:36:47.328008+0000","last_clean":"2024-09-11T23:36:47.328008+0000","last_became_active":"2024-09-11T23:34:52.866829+0000","last_became_peered":"2024-09-11T23:34:52.866829+0000","last_unstale":"2024-09-11T23:36:47.328008+0000","last_undegraded":"2024-09-11T23:36:47.328008+0000","last_fullsized":"2024-09-11T23:36:47.328008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.752006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058275+0000","last_change":"2024-09-11T23:34:50.924566+0000","last_active":"2024-09-11T23:36:47.058275+0000","last_peered":"2024-09-11T23:36:47.058275+0000","last_clean":"2024-09-11T23:36:47.058275+0000","last_became_active":"2024-09-11T23:34:50.924284+0000","last_became_peered":"2024-09-11T23:34:50.924284+0000","last_unstale":"2024-09-11T23:36:47.058275+0000","last_undegraded":"2024-09-11T23:36:47.058275+0000","last_fullsized":"2024-09-11T23:36:47.058275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:54.246154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145487+0000","last_change":"2024-09-11T23:34:52.869302+0000","last_active":"2024-09-11T23:36:47.145487+0000","last_peered":"2024-09-11T23:36:47.145487+0000","last_clean":"2024-09-11T23:36:47.145487+0000","last_became_active":"2024-09-11T23:34:52.868960+0000","last_became_peered":"2024-09-11T23:34:52.868960+0000","last_unstale":"2024-09-11T23:36:47.145487+0000","last_undegraded":"2024-09-11T23:36:47.145487+0000","last_fullsized":"2024-09-11T23:36:47.145487+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.561712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146271+0000","last_change":"2024-09-11T23:34:50.854573+0000","last_active":"2024-09-11T23:36:47.146271+0000","last_peered":"2024-09-11T23:36:47.146271+0000","last_clean":"2024-09-11T23:36:47.146271+0000","last_became_active":"2024-09-11T23:34:50.854165+0000","last_became_peered":"2024-09-11T23:34:50.854165+0000","last_unstale":"2024-09-11T23:36:47.146271+0000","last_undegraded":"2024-09-11T23:36:47.146271+0000","last_fullsized":"2024-09-11T23:36:47.146271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:57:31.589495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327542+0000","last_change":"2024-09-11T23:34:52.875987+0000","last_active":"2024-09-11T23:36:47.327542+0000","last_peered":"2024-09-11T23:36:47.327542+0000","last_clean":"2024-09-11T23:36:47.327542+0000","last_became_active":"2024-09-11T23:34:52.875662+0000","last_became_peered":"2024-09-11T23:34:52.875662+0000","last_unstale":"2024-09-11T23:36:47.327542+0000","last_undegraded":"2024-09-11T23:36:47.327542+0000","last_fullsized":"2024-09-11T23:36:47.327542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:16.514938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058234+0000","last_change":"2024-09-11T23:34:50.850726+0000","last_active":"2024-09-11T23:36:47.058234+0000","last_peered":"2024-09-11T23:36:47.058234+0000","last_clean":"2024-09-11T23:36:47.058234+0000","last_became_active":"2024-09-11T23:34:50.850381+0000","last_became_peered":"2024-09-11T23:34:50.850381+0000","last_unstale":"2024-09-11T23:36:47.058234+0000","last_undegraded":"2024-09-11T23:36:47.058234+0000","last_fullsized":"2024-09-11T23:36:47.058234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:50.431293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146377+0000","last_change":"2024-09-11T23:34:52.870652+0000","last_active":"2024-09-11T23:36:47.146377+0000","last_peered":"2024-09-11T23:36:47.146377+0000","last_clean":"2024-09-11T23:36:47.146377+0000","last_became_active":"2024-09-11T23:34:52.870303+0000","last_became_peered":"2024-09-11T23:34:52.870303+0000","last_unstale":"2024-09-11T23:36:47.146377+0000","last_undegraded":"2024-09-11T23:36:47.146377+0000","last_fullsized":"2024-09-11T23:36:47.146377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:17.670981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327509+0000","last_change":"2024-09-11T23:35:08.854490+0000","last_active":"2024-09-11T23:36:47.327509+0000","last_peered":"2024-09-11T23:36:47.327509+0000","last_clean":"2024-09-11T23:36:47.327509+0000","last_became_active":"2024-09-11T23:35:08.849820+0000","last_became_peered":"2024-09-11T23:35:08.849820+0000","last_unstale":"2024-09-11T23:36:47.327509+0000","last_undegraded":"2024-09-11T23:36:47.327509+0000","last_fullsized":"2024-09-11T23:36:47.327509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:09.509867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152580+0000","last_change":"2024-09-11T23:34:52.869695+0000","last_active":"2024-09-11T23:36:47.152580+0000","last_peered":"2024-09-11T23:36:47.152580+0000","last_clean":"2024-09-11T23:36:47.152580+0000","last_became_active":"2024-09-11T23:34:52.869367+0000","last_became_peered":"2024-09-11T23:34:52.869367+0000","last_unstale":"2024-09-11T23:36:47.152580+0000","last_undegraded":"2024-09-11T23:36:47.152580+0000","last_fullsized":"2024-09-11T23:36:47.152580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:16.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153822+0000","last_change":"2024-09-11T23:34:52.872778+0000","last_active":"2024-09-11T23:36:47.153822+0000","last_peered":"2024-09-11T23:36:47.153822+0000","last_clean":"2024-09-11T23:36:47.153822+0000","last_became_active":"2024-09-11T23:34:52.872451+0000","last_became_peered":"2024-09-11T23:34:52.872451+0000","last_unstale":"2024-09-11T23:36:47.153822+0000","last_undegraded":"2024-09-11T23:36:47.153822+0000","last_fullsized":"2024-09-11T23:36:47.153822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:51.769927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145825+0000","last_change":"2024-09-11T23:34:50.849675+0000","last_active":"2024-09-11T23:36:47.145825+0000","last_peered":"2024-09-11T23:36:47.145825+0000","last_clean":"2024-09-11T23:36:47.145825+0000","last_became_active":"2024-09-11T23:34:50.849228+0000","last_became_peered":"2024-09-11T23:34:50.849228+0000","last_unstale":"2024-09-11T23:36:47.145825+0000","last_undegraded":"2024-09-11T23:36:47.145825+0000","last_fullsized":"2024-09-11T23:36:47.145825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:50.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058399+0000","last_change":"2024-09-11T23:34:52.867739+0000","last_active":"2024-09-11T23:36:47.058399+0000","last_peered":"2024-09-11T23:36:47.058399+0000","last_clean":"2024-09-11T23:36:47.058399+0000","last_became_active":"2024-09-11T23:34:52.866613+0000","last_became_peered":"2024-09-11T23:34:52.866613+0000","last_unstale":"2024-09-11T23:36:47.058399+0000","last_undegraded":"2024-09-11T23:36:47.058399+0000","last_fullsized":"2024-09-11T23:36:47.058399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:49.348386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'13","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145845+0000","last_change":"2024-09-11T23:34:50.851401+0000","last_active":"2024-09-11T23:36:47.145845+0000","last_peered":"2024-09-11T23:36:47.145845+0000","last_clean":"2024-09-11T23:36:47.145845+0000","last_became_active":"2024-09-11T23:34:50.851068+0000","last_became_peered":"2024-09-11T23:34:50.851068+0000","last_unstale":"2024-09-11T23:36:47.145845+0000","last_undegraded":"2024-09-11T23:36:47.145845+0000","last_fullsized":"2024-09-11T23:36:47.145845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:09.673864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328128+0000","last_change":"2024-09-11T23:34:52.865216+0000","last_active":"2024-09-11T23:36:47.328128+0000","last_peered":"2024-09-11T23:36:47.328128+0000","last_clean":"2024-09-11T23:36:47.328128+0000","last_became_active":"2024-09-11T23:34:52.864917+0000","last_became_peered":"2024-09-11T23:34:52.864917+0000","last_unstale":"2024-09-11T23:36:47.328128+0000","last_undegraded":"2024-09-11T23:36:47.328128+0000","last_fullsized":"2024-09-11T23:36:47.328128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:36.469793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147616+0000","last_change":"2024-09-11T23:34:50.924291+0000","last_active":"2024-09-11T23:36:47.147616+0000","last_peered":"2024-09-11T23:36:47.147616+0000","last_clean":"2024-09-11T23:36:47.147616+0000","last_became_active":"2024-09-11T23:34:50.923892+0000","last_became_peered":"2024-09-11T23:34:50.923892+0000","last_unstale":"2024-09-11T23:36:47.147616+0000","last_undegraded":"2024-09-11T23:36:47.147616+0000","last_fullsized":"2024-09-11T23:36:47.147616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:52.266778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147387+0000","last_change":"2024-09-11T23:34:52.875995+0000","last_active":"2024-09-11T23:36:47.147387+0000","last_peered":"2024-09-11T23:36:47.147387+0000","last_clean":"2024-09-11T23:36:47.147387+0000","last_became_active":"2024-09-11T23:34:52.875654+0000","last_became_peered":"2024-09-11T23:34:52.875654+0000","last_unstale":"2024-09-11T23:36:47.147387+0000","last_undegraded":"2024-09-11T23:36:47.147387+0000","last_fullsized":"2024-09-11T23:36:47.147387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:19.218291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292143+0000","last_change":"2024-09-11T23:34:50.853881+0000","last_active":"2024-09-11T23:36:47.292143+0000","last_peered":"2024-09-11T23:36:47.292143+0000","last_clean":"2024-09-11T23:36:47.292143+0000","last_became_active":"2024-09-11T23:34:50.853576+0000","last_became_peered":"2024-09-11T23:34:50.853576+0000","last_unstale":"2024-09-11T23:36:47.292143+0000","last_undegraded":"2024-09-11T23:36:47.292143+0000","last_fullsized":"2024-09-11T23:36:47.292143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:38.550587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597446,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1182,"num_read_kb":703,"num_write":381,"num_write_kb":1464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1792338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612644,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841576,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1748992,"data_stored":1230354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005102,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32484,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712668,"statfs":{"total":95995035648,"available":95961772032,"internally_reserved":0,"allocated":1327104,"data_stored":836376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28320,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716832,"statfs":{"total":95995035648,"available":95966035968,"internally_reserved":0,"allocated":1323008,"data_stored":836754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495672,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1527808,"data_stored":1038472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659195,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":442877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888129,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1167336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018951,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":444366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1462272,"data_stored":967518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444113,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":507736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607637,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":506657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":444750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":197831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:36:52.467 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:36:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.602+0000 7f38835b9640 1 -- 172.21.15.133:0/4187578570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f387c100a80 msgr2=0x7f387c100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.602+0000 7f38835b9640 1 --2- 172.21.15.133:0/4187578570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f387c100a80 0x7f387c100e60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3878009900 tx=0x7f387802a480 comp rx=0 tx=0).stop 2024-09-11T23:36:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.602+0000 7f38835b9640 1 -- 172.21.15.133:0/4187578570 shutdown_connections 2024-09-11T23:36:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.602+0000 7f38835b9640 1 --2- 172.21.15.133:0/4187578570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f387c105eb0 0x7f387c106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.602+0000 7f38835b9640 1 --2- 172.21.15.133:0/4187578570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f387c1013a0 0x7f387c105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.602+0000 7f38835b9640 1 --2- 172.21.15.133:0/4187578570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f387c100a80 0x7f387c100e60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.602+0000 7f38835b9640 1 -- 172.21.15.133:0/4187578570 >> 172.21.15.133:0/4187578570 conn(0x7f387c0fc6a0 msgr2=0x7f387c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.603+0000 7f38835b9640 1 -- 172.21.15.133:0/4187578570 shutdown_connections 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.603+0000 7f38835b9640 1 -- 172.21.15.133:0/4187578570 wait complete. 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.603+0000 7f38835b9640 1 Processor -- start 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.603+0000 7f38835b9640 1 -- start start 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.603+0000 7f38835b9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f387c100a80 0x7f387c1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.603+0000 7f38835b9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f387c1013a0 0x7f387c1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.603+0000 7f38835b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f387c105eb0 0x7f387c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.603+0000 7f38835b9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f387c1184c0 con 0x7f387c105eb0 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.603+0000 7f38835b9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f387c118340 con 0x7f387c1013a0 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.603+0000 7f38835b9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f387c118640 con 0x7f387c100a80 2024-09-11T23:36:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.603+0000 7f3881b56640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f387c105eb0 0x7f387c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f3881b56640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f387c105eb0 0x7f387c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33142/0 (socket says 172.21.15.133:33142) 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f3881b56640 1 -- 172.21.15.133:0/3247384722 learned_addr learned my addr 172.21.15.133:0/3247384722 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f3881355640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f387c100a80 0x7f387c1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f3880b54640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f387c1013a0 0x7f387c1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f3881b56640 1 -- 172.21.15.133:0/3247384722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f387c100a80 msgr2=0x7f387c1a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f3881b56640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f387c100a80 0x7f387c1a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f3881b56640 1 -- 172.21.15.133:0/3247384722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f387c1013a0 msgr2=0x7f387c1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f3881b56640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f387c1013a0 0x7f387c1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f3881b56640 1 -- 172.21.15.133:0/3247384722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c19eff0 con 0x7f387c105eb0 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f3880b54640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f387c1013a0 0x7f387c1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f3881b56640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f387c105eb0 0x7f387c19e870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f387400cad0 tx=0x7f387400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f38727fc640 1 -- 172.21.15.133:0/3247384722 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387400ae20 con 0x7f387c105eb0 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f38727fc640 1 -- 172.21.15.133:0/3247384722 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3874005020 con 0x7f387c105eb0 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f38835b9640 1 -- 172.21.15.133:0/3247384722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387c19f2e0 con 0x7f387c105eb0 2024-09-11T23:36:52.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f38727fc640 1 -- 172.21.15.133:0/3247384722 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387400f6f0 con 0x7f387c105eb0 2024-09-11T23:36:52.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.604+0000 7f38835b9640 1 -- 172.21.15.133:0/3247384722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387c1ab580 con 0x7f387c105eb0 2024-09-11T23:36:52.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.605+0000 7f38835b9640 1 -- 172.21.15.133:0/3247384722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387c105860 con 0x7f387c105eb0 2024-09-11T23:36:52.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.606+0000 7f38727fc640 1 -- 172.21.15.133:0/3247384722 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f38740051d0 con 0x7f387c105eb0 2024-09-11T23:36:52.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.607+0000 7f38727fc640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38680a2e20 0x7f38680a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.607+0000 7f3881355640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38680a2e20 0x7f38680a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.607+0000 7f38727fc640 1 -- 172.21.15.133:0/3247384722 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f38740cfa40 con 0x7f387c105eb0 2024-09-11T23:36:52.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.607+0000 7f3881355640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38680a2e20 0x7f38680a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f387c19f9b0 tx=0x7f38780058d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:52.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.610+0000 7f38727fc640 1 -- 172.21.15.133:0/3247384722 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3874014030 con 0x7f387c105eb0 2024-09-11T23:36:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.771+0000 7f38835b9640 1 -- 172.21.15.133:0/3247384722 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f387c0fde50 con 0x7f38680a2e20 2024-09-11T23:36:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.778+0000 7f38727fc640 1 -- 172.21.15.133:0/3247384722 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257810 (secure 0 0 0) 0x7f387c0fde50 con 0x7f38680a2e20 2024-09-11T23:36:52.780 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T23:36:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.780+0000 7f38835b9640 1 -- 172.21.15.133:0/3247384722 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38680a2e20 msgr2=0x7f38680a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.780+0000 7f38835b9640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38680a2e20 0x7f38680a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f387c19f9b0 tx=0x7f38780058d0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.780+0000 7f38835b9640 1 -- 172.21.15.133:0/3247384722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f387c105eb0 msgr2=0x7f387c19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.780+0000 7f38835b9640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f387c105eb0 0x7f387c19e870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f387400cad0 tx=0x7f387400cf90 comp rx=0 tx=0).stop 2024-09-11T23:36:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.780+0000 7f38835b9640 1 -- 172.21.15.133:0/3247384722 shutdown_connections 2024-09-11T23:36:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.780+0000 7f38835b9640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38680a2e20 0x7f38680a52e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.780+0000 7f38835b9640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f387c105eb0 0x7f387c19e870 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.780+0000 7f38835b9640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f387c1013a0 0x7f387c1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.780+0000 7f38835b9640 1 --2- 172.21.15.133:0/3247384722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f387c100a80 0x7f387c1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.781+0000 7f38835b9640 1 -- 172.21.15.133:0/3247384722 >> 172.21.15.133:0/3247384722 conn(0x7f387c0fc6a0 msgr2=0x7f387c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.781+0000 7f38835b9640 1 -- 172.21.15.133:0/3247384722 shutdown_connections 2024-09-11T23:36:52.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.781+0000 7f38835b9640 1 -- 172.21.15.133:0/3247384722 wait complete. 2024-09-11T23:36:52.791 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-11T23:36:51.069176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188048,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1318,"num_read_kb":822,"num_write":564,"num_write_kb":4404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359512,"kb_used_data":14388,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582312,"statfs":{"total":1151940427776,"available":1151572287488,"internally_reserved":0,"allocated":14733312,"data_stored":8867028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.989107"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327838+0000","last_change":"2024-09-11T23:34:52.867694+0000","last_active":"2024-09-11T23:36:47.327838+0000","last_peered":"2024-09-11T23:36:47.327838+0000","last_clean":"2024-09-11T23:36:47.327838+0000","last_became_active":"2024-09-11T23:34:52.867368+0000","last_became_peered":"2024-09-11T23:34:52.867368+0000","last_unstale":"2024-09-11T23:36:47.327838+0000","last_undegraded":"2024-09-11T23:36:47.327838+0000","last_fullsized":"2024-09-11T23:36:47.327838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:46:07.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327981+0000","last_change":"2024-09-11T23:34:52.871881+0000","last_active":"2024-09-11T23:36:47.327981+0000","last_peered":"2024-09-11T23:36:47.327981+0000","last_clean":"2024-09-11T23:36:47.327981+0000","last_became_active":"2024-09-11T23:34:52.871609+0000","last_became_peered":"2024-09-11T23:34:52.871609+0000","last_unstale":"2024-09-11T23:36:47.327981+0000","last_undegraded":"2024-09-11T23:36:47.327981+0000","last_fullsized":"2024-09-11T23:36:47.327981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.938584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327720+0000","last_change":"2024-09-11T23:34:52.863931+0000","last_active":"2024-09-11T23:36:47.327720+0000","last_peered":"2024-09-11T23:36:47.327720+0000","last_clean":"2024-09-11T23:36:47.327720+0000","last_became_active":"2024-09-11T23:34:52.863637+0000","last_became_peered":"2024-09-11T23:34:52.863637+0000","last_unstale":"2024-09-11T23:36:47.327720+0000","last_undegraded":"2024-09-11T23:36:47.327720+0000","last_fullsized":"2024-09-11T23:36:47.327720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:44.118865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292412+0000","last_change":"2024-09-11T23:34:52.879619+0000","last_active":"2024-09-11T23:36:47.292412+0000","last_peered":"2024-09-11T23:36:47.292412+0000","last_clean":"2024-09-11T23:36:47.292412+0000","last_became_active":"2024-09-11T23:34:52.878695+0000","last_became_peered":"2024-09-11T23:34:52.878695+0000","last_unstale":"2024-09-11T23:36:47.292412+0000","last_undegraded":"2024-09-11T23:36:47.292412+0000","last_fullsized":"2024-09-11T23:36:47.292412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:30.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145593+0000","last_change":"2024-09-11T23:34:52.869363+0000","last_active":"2024-09-11T23:36:47.145593+0000","last_peered":"2024-09-11T23:36:47.145593+0000","last_clean":"2024-09-11T23:36:47.145593+0000","last_became_active":"2024-09-11T23:34:52.869027+0000","last_became_peered":"2024-09-11T23:34:52.869027+0000","last_unstale":"2024-09-11T23:36:47.145593+0000","last_undegraded":"2024-09-11T23:36:47.145593+0000","last_fullsized":"2024-09-11T23:36:47.145593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:15.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156014+0000","last_change":"2024-09-11T23:34:52.880665+0000","last_active":"2024-09-11T23:36:47.156014+0000","last_peered":"2024-09-11T23:36:47.156014+0000","last_clean":"2024-09-11T23:36:47.156014+0000","last_became_active":"2024-09-11T23:34:52.880172+0000","last_became_peered":"2024-09-11T23:34:52.880172+0000","last_unstale":"2024-09-11T23:36:47.156014+0000","last_undegraded":"2024-09-11T23:36:47.156014+0000","last_fullsized":"2024-09-11T23:36:47.156014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:12.025877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152489+0000","last_change":"2024-09-11T23:34:52.873897+0000","last_active":"2024-09-11T23:36:47.152489+0000","last_peered":"2024-09-11T23:36:47.152489+0000","last_clean":"2024-09-11T23:36:47.152489+0000","last_became_active":"2024-09-11T23:34:52.873219+0000","last_became_peered":"2024-09-11T23:34:52.873219+0000","last_unstale":"2024-09-11T23:36:47.152489+0000","last_undegraded":"2024-09-11T23:36:47.152489+0000","last_fullsized":"2024-09-11T23:36:47.152489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:47.125682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145794+0000","last_change":"2024-09-11T23:34:52.870699+0000","last_active":"2024-09-11T23:36:47.145794+0000","last_peered":"2024-09-11T23:36:47.145794+0000","last_clean":"2024-09-11T23:36:47.145794+0000","last_became_active":"2024-09-11T23:34:52.870314+0000","last_became_peered":"2024-09-11T23:34:52.870314+0000","last_unstale":"2024-09-11T23:36:47.145794+0000","last_undegraded":"2024-09-11T23:36:47.145794+0000","last_fullsized":"2024-09-11T23:36:47.145794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:41.263159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058666+0000","last_change":"2024-09-11T23:34:52.869700+0000","last_active":"2024-09-11T23:36:47.058666+0000","last_peered":"2024-09-11T23:36:47.058666+0000","last_clean":"2024-09-11T23:36:47.058666+0000","last_became_active":"2024-09-11T23:34:52.869398+0000","last_became_peered":"2024-09-11T23:34:52.869398+0000","last_unstale":"2024-09-11T23:36:47.058666+0000","last_undegraded":"2024-09-11T23:36:47.058666+0000","last_fullsized":"2024-09-11T23:36:47.058666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:49.776525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156161+0000","last_change":"2024-09-11T23:34:52.879999+0000","last_active":"2024-09-11T23:36:47.156161+0000","last_peered":"2024-09-11T23:36:47.156161+0000","last_clean":"2024-09-11T23:36:47.156161+0000","last_became_active":"2024-09-11T23:34:52.879646+0000","last_became_peered":"2024-09-11T23:34:52.879646+0000","last_unstale":"2024-09-11T23:36:47.156161+0000","last_undegraded":"2024-09-11T23:36:47.156161+0000","last_fullsized":"2024-09-11T23:36:47.156161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.058704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152665+0000","last_change":"2024-09-11T23:34:52.878740+0000","last_active":"2024-09-11T23:36:47.152665+0000","last_peered":"2024-09-11T23:36:47.152665+0000","last_clean":"2024-09-11T23:36:47.152665+0000","last_became_active":"2024-09-11T23:34:52.878120+0000","last_became_peered":"2024-09-11T23:34:52.878120+0000","last_unstale":"2024-09-11T23:36:47.152665+0000","last_undegraded":"2024-09-11T23:36:47.152665+0000","last_fullsized":"2024-09-11T23:36:47.152665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292447+0000","last_change":"2024-09-11T23:34:52.873409+0000","last_active":"2024-09-11T23:36:47.292447+0000","last_peered":"2024-09-11T23:36:47.292447+0000","last_clean":"2024-09-11T23:36:47.292447+0000","last_became_active":"2024-09-11T23:34:52.872959+0000","last_became_peered":"2024-09-11T23:34:52.872959+0000","last_unstale":"2024-09-11T23:36:47.292447+0000","last_undegraded":"2024-09-11T23:36:47.292447+0000","last_fullsized":"2024-09-11T23:36:47.292447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:38.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057362+0000","last_change":"2024-09-11T23:34:52.869164+0000","last_active":"2024-09-11T23:36:47.057362+0000","last_peered":"2024-09-11T23:36:47.057362+0000","last_clean":"2024-09-11T23:36:47.057362+0000","last_became_active":"2024-09-11T23:34:52.868847+0000","last_became_peered":"2024-09-11T23:34:52.868847+0000","last_unstale":"2024-09-11T23:36:47.057362+0000","last_undegraded":"2024-09-11T23:36:47.057362+0000","last_fullsized":"2024-09-11T23:36:47.057362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.764510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327562+0000","last_change":"2024-09-11T23:34:52.869494+0000","last_active":"2024-09-11T23:36:47.327562+0000","last_peered":"2024-09-11T23:36:47.327562+0000","last_clean":"2024-09-11T23:36:47.327562+0000","last_became_active":"2024-09-11T23:34:52.869178+0000","last_became_peered":"2024-09-11T23:34:52.869178+0000","last_unstale":"2024-09-11T23:36:47.327562+0000","last_undegraded":"2024-09-11T23:36:47.327562+0000","last_fullsized":"2024-09-11T23:36:47.327562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:23.431564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327741+0000","last_change":"2024-09-11T23:34:52.866269+0000","last_active":"2024-09-11T23:36:47.327741+0000","last_peered":"2024-09-11T23:36:47.327741+0000","last_clean":"2024-09-11T23:36:47.327741+0000","last_became_active":"2024-09-11T23:34:52.865980+0000","last_became_peered":"2024-09-11T23:34:52.865980+0000","last_unstale":"2024-09-11T23:36:47.327741+0000","last_undegraded":"2024-09-11T23:36:47.327741+0000","last_fullsized":"2024-09-11T23:36:47.327741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:48.492252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058174+0000","last_change":"2024-09-11T23:34:52.862192+0000","last_active":"2024-09-11T23:36:47.058174+0000","last_peered":"2024-09-11T23:36:47.058174+0000","last_clean":"2024-09-11T23:36:47.058174+0000","last_became_active":"2024-09-11T23:34:52.861841+0000","last_became_peered":"2024-09-11T23:34:52.861841+0000","last_unstale":"2024-09-11T23:36:47.058174+0000","last_undegraded":"2024-09-11T23:36:47.058174+0000","last_fullsized":"2024-09-11T23:36:47.058174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.258948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327836+0000","last_change":"2024-09-11T23:34:52.870984+0000","last_active":"2024-09-11T23:36:47.327836+0000","last_peered":"2024-09-11T23:36:47.327836+0000","last_clean":"2024-09-11T23:36:47.327836+0000","last_became_active":"2024-09-11T23:34:52.870435+0000","last_became_peered":"2024-09-11T23:34:52.870435+0000","last_unstale":"2024-09-11T23:36:47.327836+0000","last_undegraded":"2024-09-11T23:36:47.327836+0000","last_fullsized":"2024-09-11T23:36:47.327836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:34.391537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143297+0000","last_change":"2024-09-11T23:34:52.873176+0000","last_active":"2024-09-11T23:36:47.143297+0000","last_peered":"2024-09-11T23:36:47.143297+0000","last_clean":"2024-09-11T23:36:47.143297+0000","last_became_active":"2024-09-11T23:34:52.872862+0000","last_became_peered":"2024-09-11T23:34:52.872862+0000","last_unstale":"2024-09-11T23:36:47.143297+0000","last_undegraded":"2024-09-11T23:36:47.143297+0000","last_fullsized":"2024-09-11T23:36:47.143297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:38.058806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058317+0000","last_change":"2024-09-11T23:34:52.871346+0000","last_active":"2024-09-11T23:36:47.058317+0000","last_peered":"2024-09-11T23:36:47.058317+0000","last_clean":"2024-09-11T23:36:47.058317+0000","last_became_active":"2024-09-11T23:34:52.870981+0000","last_became_peered":"2024-09-11T23:34:52.870981+0000","last_unstale":"2024-09-11T23:36:47.058317+0000","last_undegraded":"2024-09-11T23:36:47.058317+0000","last_fullsized":"2024-09-11T23:36:47.058317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:56.973998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327601+0000","last_change":"2024-09-11T23:34:52.867058+0000","last_active":"2024-09-11T23:36:47.327601+0000","last_peered":"2024-09-11T23:36:47.327601+0000","last_clean":"2024-09-11T23:36:47.327601+0000","last_became_active":"2024-09-11T23:34:52.866687+0000","last_became_peered":"2024-09-11T23:34:52.866687+0000","last_unstale":"2024-09-11T23:36:47.327601+0000","last_undegraded":"2024-09-11T23:36:47.327601+0000","last_fullsized":"2024-09-11T23:36:47.327601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:09.019987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057475+0000","last_change":"2024-09-11T23:34:52.870882+0000","last_active":"2024-09-11T23:36:47.057475+0000","last_peered":"2024-09-11T23:36:47.057475+0000","last_clean":"2024-09-11T23:36:47.057475+0000","last_became_active":"2024-09-11T23:34:52.870667+0000","last_became_peered":"2024-09-11T23:34:52.870667+0000","last_unstale":"2024-09-11T23:36:47.057475+0000","last_undegraded":"2024-09-11T23:36:47.057475+0000","last_fullsized":"2024-09-11T23:36:47.057475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:08.386703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058304+0000","last_change":"2024-09-11T23:34:52.874070+0000","last_active":"2024-09-11T23:36:47.058304+0000","last_peered":"2024-09-11T23:36:47.058304+0000","last_clean":"2024-09-11T23:36:47.058304+0000","last_became_active":"2024-09-11T23:34:52.873772+0000","last_became_peered":"2024-09-11T23:34:52.873772+0000","last_unstale":"2024-09-11T23:36:47.058304+0000","last_undegraded":"2024-09-11T23:36:47.058304+0000","last_fullsized":"2024-09-11T23:36:47.058304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:49.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146028+0000","last_change":"2024-09-11T23:34:52.875502+0000","last_active":"2024-09-11T23:36:47.146028+0000","last_peered":"2024-09-11T23:36:47.146028+0000","last_clean":"2024-09-11T23:36:47.146028+0000","last_became_active":"2024-09-11T23:34:52.875193+0000","last_became_peered":"2024-09-11T23:34:52.875193+0000","last_unstale":"2024-09-11T23:36:47.146028+0000","last_undegraded":"2024-09-11T23:36:47.146028+0000","last_fullsized":"2024-09-11T23:36:47.146028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:37.213828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153497+0000","last_change":"2024-09-11T23:34:52.869015+0000","last_active":"2024-09-11T23:36:47.153497+0000","last_peered":"2024-09-11T23:36:47.153497+0000","last_clean":"2024-09-11T23:36:47.153497+0000","last_became_active":"2024-09-11T23:34:52.868731+0000","last_became_peered":"2024-09-11T23:34:52.868731+0000","last_unstale":"2024-09-11T23:36:47.153497+0000","last_undegraded":"2024-09-11T23:36:47.153497+0000","last_fullsized":"2024-09-11T23:36:47.153497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:12:15.993203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147366+0000","last_change":"2024-09-11T23:34:52.875555+0000","last_active":"2024-09-11T23:36:47.147366+0000","last_peered":"2024-09-11T23:36:47.147366+0000","last_clean":"2024-09-11T23:36:47.147366+0000","last_became_active":"2024-09-11T23:34:52.875219+0000","last_became_peered":"2024-09-11T23:34:52.875219+0000","last_unstale":"2024-09-11T23:36:47.147366+0000","last_undegraded":"2024-09-11T23:36:47.147366+0000","last_fullsized":"2024-09-11T23:36:47.147366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:16.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058480+0000","last_change":"2024-09-11T23:34:52.875629+0000","last_active":"2024-09-11T23:36:47.058480+0000","last_peered":"2024-09-11T23:36:47.058480+0000","last_clean":"2024-09-11T23:36:47.058480+0000","last_became_active":"2024-09-11T23:34:52.875302+0000","last_became_peered":"2024-09-11T23:34:52.875302+0000","last_unstale":"2024-09-11T23:36:47.058480+0000","last_undegraded":"2024-09-11T23:36:47.058480+0000","last_fullsized":"2024-09-11T23:36:47.058480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:12.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327744+0000","last_change":"2024-09-11T23:34:52.868613+0000","last_active":"2024-09-11T23:36:47.327744+0000","last_peered":"2024-09-11T23:36:47.327744+0000","last_clean":"2024-09-11T23:36:47.327744+0000","last_became_active":"2024-09-11T23:34:52.868184+0000","last_became_peered":"2024-09-11T23:34:52.868184+0000","last_unstale":"2024-09-11T23:36:47.327744+0000","last_undegraded":"2024-09-11T23:36:47.327744+0000","last_fullsized":"2024-09-11T23:36:47.327744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:25.990002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057237+0000","last_change":"2024-09-11T23:35:08.873390+0000","last_active":"2024-09-11T23:36:47.057237+0000","last_peered":"2024-09-11T23:36:47.057237+0000","last_clean":"2024-09-11T23:36:47.057237+0000","last_became_active":"2024-09-11T23:35:08.873052+0000","last_became_peered":"2024-09-11T23:35:08.873052+0000","last_unstale":"2024-09-11T23:36:47.057237+0000","last_undegraded":"2024-09-11T23:36:47.057237+0000","last_fullsized":"2024-09-11T23:36:47.057237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.937951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327689+0000","last_change":"2024-09-11T23:34:52.873604+0000","last_active":"2024-09-11T23:36:47.327689+0000","last_peered":"2024-09-11T23:36:47.327689+0000","last_clean":"2024-09-11T23:36:47.327689+0000","last_became_active":"2024-09-11T23:34:52.873220+0000","last_became_peered":"2024-09-11T23:34:52.873220+0000","last_unstale":"2024-09-11T23:36:47.327689+0000","last_undegraded":"2024-09-11T23:36:47.327689+0000","last_fullsized":"2024-09-11T23:36:47.327689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:38.480630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058025+0000","last_change":"2024-09-11T23:34:52.872902+0000","last_active":"2024-09-11T23:36:47.058025+0000","last_peered":"2024-09-11T23:36:47.058025+0000","last_clean":"2024-09-11T23:36:47.058025+0000","last_became_active":"2024-09-11T23:34:52.872571+0000","last_became_peered":"2024-09-11T23:34:52.872571+0000","last_unstale":"2024-09-11T23:36:47.058025+0000","last_undegraded":"2024-09-11T23:36:47.058025+0000","last_fullsized":"2024-09-11T23:36:47.058025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:21.844995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058115+0000","last_change":"2024-09-11T23:35:09.457056+0000","last_active":"2024-09-11T23:36:47.058115+0000","last_peered":"2024-09-11T23:36:47.058115+0000","last_clean":"2024-09-11T23:36:47.058115+0000","last_became_active":"2024-09-11T23:35:09.456657+0000","last_became_peered":"2024-09-11T23:35:09.456657+0000","last_unstale":"2024-09-11T23:36:47.058115+0000","last_undegraded":"2024-09-11T23:36:47.058115+0000","last_fullsized":"2024-09-11T23:36:47.058115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:12:56.121280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327466+0000","last_change":"2024-09-11T23:34:52.875312+0000","last_active":"2024-09-11T23:36:47.327466+0000","last_peered":"2024-09-11T23:36:47.327466+0000","last_clean":"2024-09-11T23:36:47.327466+0000","last_became_active":"2024-09-11T23:34:52.874873+0000","last_became_peered":"2024-09-11T23:34:52.874873+0000","last_unstale":"2024-09-11T23:36:47.327466+0000","last_undegraded":"2024-09-11T23:36:47.327466+0000","last_fullsized":"2024-09-11T23:36:47.327466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:18.894713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145537+0000","last_change":"2024-09-11T23:34:52.862266+0000","last_active":"2024-09-11T23:36:47.145537+0000","last_peered":"2024-09-11T23:36:47.145537+0000","last_clean":"2024-09-11T23:36:47.145537+0000","last_became_active":"2024-09-11T23:34:52.861923+0000","last_became_peered":"2024-09-11T23:34:52.861923+0000","last_unstale":"2024-09-11T23:36:47.145537+0000","last_undegraded":"2024-09-11T23:36:47.145537+0000","last_fullsized":"2024-09-11T23:36:47.145537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:23.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":138,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.591379+0000","last_change":"2024-09-11T23:34:50.854020+0000","last_active":"2024-09-11T23:36:47.591379+0000","last_peered":"2024-09-11T23:36:47.591379+0000","last_clean":"2024-09-11T23:36:47.591379+0000","last_became_active":"2024-09-11T23:34:50.853683+0000","last_became_peered":"2024-09-11T23:34:50.853683+0000","last_unstale":"2024-09-11T23:36:47.591379+0000","last_undegraded":"2024-09-11T23:36:47.591379+0000","last_fullsized":"2024-09-11T23:36:47.591379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:18.725495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153611+0000","last_change":"2024-09-11T23:34:52.874016+0000","last_active":"2024-09-11T23:36:47.153611+0000","last_peered":"2024-09-11T23:36:47.153611+0000","last_clean":"2024-09-11T23:36:47.153611+0000","last_became_active":"2024-09-11T23:34:52.873675+0000","last_became_peered":"2024-09-11T23:34:52.873675+0000","last_unstale":"2024-09-11T23:36:47.153611+0000","last_undegraded":"2024-09-11T23:36:47.153611+0000","last_fullsized":"2024-09-11T23:36:47.153611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:41:28.955548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156082+0000","last_change":"2024-09-11T23:34:50.855391+0000","last_active":"2024-09-11T23:36:47.156082+0000","last_peered":"2024-09-11T23:36:47.156082+0000","last_clean":"2024-09-11T23:36:47.156082+0000","last_became_active":"2024-09-11T23:34:50.855052+0000","last_became_peered":"2024-09-11T23:34:50.855052+0000","last_unstale":"2024-09-11T23:36:47.156082+0000","last_undegraded":"2024-09-11T23:36:47.156082+0000","last_fullsized":"2024-09-11T23:36:47.156082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.972239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057065+0000","last_change":"2024-09-11T23:34:50.848985+0000","last_active":"2024-09-11T23:36:47.057065+0000","last_peered":"2024-09-11T23:36:47.057065+0000","last_clean":"2024-09-11T23:36:47.057065+0000","last_became_active":"2024-09-11T23:34:50.848614+0000","last_became_peered":"2024-09-11T23:34:50.848614+0000","last_unstale":"2024-09-11T23:36:47.057065+0000","last_undegraded":"2024-09-11T23:36:47.057065+0000","last_fullsized":"2024-09-11T23:36:47.057065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:43:04.723136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146018+0000","last_change":"2024-09-11T23:34:52.866933+0000","last_active":"2024-09-11T23:36:47.146018+0000","last_peered":"2024-09-11T23:36:47.146018+0000","last_clean":"2024-09-11T23:36:47.146018+0000","last_became_active":"2024-09-11T23:34:52.866559+0000","last_became_peered":"2024-09-11T23:34:52.866559+0000","last_unstale":"2024-09-11T23:36:47.146018+0000","last_undegraded":"2024-09-11T23:36:47.146018+0000","last_fullsized":"2024-09-11T23:36:47.146018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:27.745118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153745+0000","last_change":"2024-09-11T23:34:50.849787+0000","last_active":"2024-09-11T23:36:47.153745+0000","last_peered":"2024-09-11T23:36:47.153745+0000","last_clean":"2024-09-11T23:36:47.153745+0000","last_became_active":"2024-09-11T23:34:50.849368+0000","last_became_peered":"2024-09-11T23:34:50.849368+0000","last_unstale":"2024-09-11T23:36:47.153745+0000","last_undegraded":"2024-09-11T23:36:47.153745+0000","last_fullsized":"2024-09-11T23:36:47.153745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:26.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142558+0000","last_change":"2024-09-11T23:34:52.870832+0000","last_active":"2024-09-11T23:36:47.142558+0000","last_peered":"2024-09-11T23:36:47.142558+0000","last_clean":"2024-09-11T23:36:47.142558+0000","last_became_active":"2024-09-11T23:34:52.870474+0000","last_became_peered":"2024-09-11T23:34:52.870474+0000","last_unstale":"2024-09-11T23:36:47.142558+0000","last_undegraded":"2024-09-11T23:36:47.142558+0000","last_fullsized":"2024-09-11T23:36:47.142558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:30.196114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":238,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.590875+0000","last_change":"2024-09-11T23:34:50.859074+0000","last_active":"2024-09-11T23:36:47.590875+0000","last_peered":"2024-09-11T23:36:47.590875+0000","last_clean":"2024-09-11T23:36:47.590875+0000","last_became_active":"2024-09-11T23:34:50.858664+0000","last_became_peered":"2024-09-11T23:34:50.858664+0000","last_unstale":"2024-09-11T23:36:47.590875+0000","last_undegraded":"2024-09-11T23:36:47.590875+0000","last_fullsized":"2024-09-11T23:36:47.590875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:04.077813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058577+0000","last_change":"2024-09-11T23:34:52.875901+0000","last_active":"2024-09-11T23:36:47.058577+0000","last_peered":"2024-09-11T23:36:47.058577+0000","last_clean":"2024-09-11T23:36:47.058577+0000","last_became_active":"2024-09-11T23:34:52.875568+0000","last_became_peered":"2024-09-11T23:34:52.875568+0000","last_unstale":"2024-09-11T23:36:47.058577+0000","last_undegraded":"2024-09-11T23:36:47.058577+0000","last_fullsized":"2024-09-11T23:36:47.058577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:54.173548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152813+0000","last_change":"2024-09-11T23:34:50.854957+0000","last_active":"2024-09-11T23:36:47.152813+0000","last_peered":"2024-09-11T23:36:47.152813+0000","last_clean":"2024-09-11T23:36:47.152813+0000","last_became_active":"2024-09-11T23:34:50.854626+0000","last_became_peered":"2024-09-11T23:34:50.854626+0000","last_unstale":"2024-09-11T23:36:47.152813+0000","last_undegraded":"2024-09-11T23:36:47.152813+0000","last_fullsized":"2024-09-11T23:36:47.152813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:53.738966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328149+0000","last_change":"2024-09-11T23:34:52.868850+0000","last_active":"2024-09-11T23:36:47.328149+0000","last_peered":"2024-09-11T23:36:47.328149+0000","last_clean":"2024-09-11T23:36:47.328149+0000","last_became_active":"2024-09-11T23:34:52.868536+0000","last_became_peered":"2024-09-11T23:34:52.868536+0000","last_unstale":"2024-09-11T23:36:47.328149+0000","last_undegraded":"2024-09-11T23:36:47.328149+0000","last_fullsized":"2024-09-11T23:36:47.328149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:11.746821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153829+0000","last_change":"2024-09-11T23:34:50.924819+0000","last_active":"2024-09-11T23:36:47.153829+0000","last_peered":"2024-09-11T23:36:47.153829+0000","last_clean":"2024-09-11T23:36:47.153829+0000","last_became_active":"2024-09-11T23:34:50.924283+0000","last_became_peered":"2024-09-11T23:34:50.924283+0000","last_unstale":"2024-09-11T23:36:47.153829+0000","last_undegraded":"2024-09-11T23:36:47.153829+0000","last_fullsized":"2024-09-11T23:36:47.153829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:41.345233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058417+0000","last_change":"2024-09-11T23:34:52.867113+0000","last_active":"2024-09-11T23:36:47.058417+0000","last_peered":"2024-09-11T23:36:47.058417+0000","last_clean":"2024-09-11T23:36:47.058417+0000","last_became_active":"2024-09-11T23:34:52.866125+0000","last_became_peered":"2024-09-11T23:34:52.866125+0000","last_unstale":"2024-09-11T23:36:47.058417+0000","last_undegraded":"2024-09-11T23:36:47.058417+0000","last_fullsized":"2024-09-11T23:36:47.058417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:18:55.615079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'36","reported_seq":282,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.590874+0000","last_change":"2024-09-11T23:34:50.857272+0000","last_active":"2024-09-11T23:36:47.590874+0000","last_peered":"2024-09-11T23:36:47.590874+0000","last_clean":"2024-09-11T23:36:47.590874+0000","last_became_active":"2024-09-11T23:34:50.855701+0000","last_became_peered":"2024-09-11T23:34:50.855701+0000","last_unstale":"2024-09-11T23:36:47.590874+0000","last_undegraded":"2024-09-11T23:36:47.590874+0000","last_fullsized":"2024-09-11T23:36:47.590874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:49.997617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62867,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":222,"num_read_kb":172,"num_write":38,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145783+0000","last_change":"2024-09-11T23:34:52.876928+0000","last_active":"2024-09-11T23:36:47.145783+0000","last_peered":"2024-09-11T23:36:47.145783+0000","last_clean":"2024-09-11T23:36:47.145783+0000","last_became_active":"2024-09-11T23:34:52.876578+0000","last_became_peered":"2024-09-11T23:34:52.876578+0000","last_unstale":"2024-09-11T23:36:47.145783+0000","last_undegraded":"2024-09-11T23:36:47.145783+0000","last_fullsized":"2024-09-11T23:36:47.145783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:07.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327515+0000","last_change":"2024-09-11T23:35:08.869275+0000","last_active":"2024-09-11T23:36:47.327515+0000","last_peered":"2024-09-11T23:36:47.327515+0000","last_clean":"2024-09-11T23:36:47.327515+0000","last_became_active":"2024-09-11T23:35:08.850112+0000","last_became_peered":"2024-09-11T23:35:08.850112+0000","last_unstale":"2024-09-11T23:36:47.327515+0000","last_undegraded":"2024-09-11T23:36:47.327515+0000","last_fullsized":"2024-09-11T23:36:47.327515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:36.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057105+0000","last_change":"2024-09-11T23:34:52.871970+0000","last_active":"2024-09-11T23:36:47.057105+0000","last_peered":"2024-09-11T23:36:47.057105+0000","last_clean":"2024-09-11T23:36:47.057105+0000","last_became_active":"2024-09-11T23:34:52.871488+0000","last_became_peered":"2024-09-11T23:34:52.871488+0000","last_unstale":"2024-09-11T23:36:47.057105+0000","last_undegraded":"2024-09-11T23:36:47.057105+0000","last_fullsized":"2024-09-11T23:36:47.057105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:08.266431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153627+0000","last_change":"2024-09-11T23:34:50.854023+0000","last_active":"2024-09-11T23:36:47.153627+0000","last_peered":"2024-09-11T23:36:47.153627+0000","last_clean":"2024-09-11T23:36:47.153627+0000","last_became_active":"2024-09-11T23:34:50.853661+0000","last_became_peered":"2024-09-11T23:34:50.853661+0000","last_unstale":"2024-09-11T23:36:47.153627+0000","last_undegraded":"2024-09-11T23:36:47.153627+0000","last_fullsized":"2024-09-11T23:36:47.153627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:25.281976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152646+0000","last_change":"2024-09-11T23:35:08.849648+0000","last_active":"2024-09-11T23:36:47.152646+0000","last_peered":"2024-09-11T23:36:47.152646+0000","last_clean":"2024-09-11T23:36:47.152646+0000","last_became_active":"2024-09-11T23:35:08.849246+0000","last_became_peered":"2024-09-11T23:35:08.849246+0000","last_unstale":"2024-09-11T23:36:47.152646+0000","last_undegraded":"2024-09-11T23:36:47.152646+0000","last_fullsized":"2024-09-11T23:36:47.152646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:17.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'10","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.609150+0000","last_change":"2024-09-11T23:34:50.853855+0000","last_active":"2024-09-11T23:36:47.609150+0000","last_peered":"2024-09-11T23:36:47.609150+0000","last_clean":"2024-09-11T23:36:47.609150+0000","last_became_active":"2024-09-11T23:34:50.853493+0000","last_became_peered":"2024-09-11T23:34:50.853493+0000","last_unstale":"2024-09-11T23:36:47.609150+0000","last_undegraded":"2024-09-11T23:36:47.609150+0000","last_fullsized":"2024-09-11T23:36:47.609150+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:23.262472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":7,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152450+0000","last_change":"2024-09-11T23:34:52.875835+0000","last_active":"2024-09-11T23:36:47.152450+0000","last_peered":"2024-09-11T23:36:47.152450+0000","last_clean":"2024-09-11T23:36:47.152450+0000","last_became_active":"2024-09-11T23:34:52.875525+0000","last_became_peered":"2024-09-11T23:34:52.875525+0000","last_unstale":"2024-09-11T23:36:47.152450+0000","last_undegraded":"2024-09-11T23:36:47.152450+0000","last_fullsized":"2024-09-11T23:36:47.152450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:41.031816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145473+0000","last_change":"2024-09-11T23:34:50.924803+0000","last_active":"2024-09-11T23:36:47.145473+0000","last_peered":"2024-09-11T23:36:47.145473+0000","last_clean":"2024-09-11T23:36:47.145473+0000","last_became_active":"2024-09-11T23:34:50.924217+0000","last_became_peered":"2024-09-11T23:34:50.924217+0000","last_unstale":"2024-09-11T23:36:47.145473+0000","last_undegraded":"2024-09-11T23:36:47.145473+0000","last_fullsized":"2024-09-11T23:36:47.145473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:45.790592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142408+0000","last_change":"2024-09-11T23:35:08.851856+0000","last_active":"2024-09-11T23:36:47.142408+0000","last_peered":"2024-09-11T23:36:47.142408+0000","last_clean":"2024-09-11T23:36:47.142408+0000","last_became_active":"2024-09-11T23:35:08.851490+0000","last_became_peered":"2024-09-11T23:35:08.851490+0000","last_unstale":"2024-09-11T23:36:47.142408+0000","last_undegraded":"2024-09-11T23:36:47.142408+0000","last_fullsized":"2024-09-11T23:36:47.142408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:46.269022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"81'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145816+0000","last_change":"2024-09-11T23:34:50.848385+0000","last_active":"2024-09-11T23:36:47.145816+0000","last_peered":"2024-09-11T23:36:47.145816+0000","last_clean":"2024-09-11T23:36:47.145816+0000","last_became_active":"2024-09-11T23:34:50.847808+0000","last_became_peered":"2024-09-11T23:34:50.847808+0000","last_unstale":"2024-09-11T23:36:47.145816+0000","last_undegraded":"2024-09-11T23:36:47.145816+0000","last_fullsized":"2024-09-11T23:36:47.145816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:38.796622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327734+0000","last_change":"2024-09-11T23:34:52.867341+0000","last_active":"2024-09-11T23:36:47.327734+0000","last_peered":"2024-09-11T23:36:47.327734+0000","last_clean":"2024-09-11T23:36:47.327734+0000","last_became_active":"2024-09-11T23:34:52.866161+0000","last_became_peered":"2024-09-11T23:34:52.866161+0000","last_unstale":"2024-09-11T23:36:47.327734+0000","last_undegraded":"2024-09-11T23:36:47.327734+0000","last_fullsized":"2024-09-11T23:36:47.327734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:27.913705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057317+0000","last_change":"2024-09-11T23:34:50.851466+0000","last_active":"2024-09-11T23:36:47.057317+0000","last_peered":"2024-09-11T23:36:47.057317+0000","last_clean":"2024-09-11T23:36:47.057317+0000","last_became_active":"2024-09-11T23:34:50.851191+0000","last_became_peered":"2024-09-11T23:34:50.851191+0000","last_unstale":"2024-09-11T23:36:47.057317+0000","last_undegraded":"2024-09-11T23:36:47.057317+0000","last_fullsized":"2024-09-11T23:36:47.057317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:00.328658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152878+0000","last_change":"2024-09-11T23:34:52.872290+0000","last_active":"2024-09-11T23:36:47.152878+0000","last_peered":"2024-09-11T23:36:47.152878+0000","last_clean":"2024-09-11T23:36:47.152878+0000","last_became_active":"2024-09-11T23:34:52.871935+0000","last_became_peered":"2024-09-11T23:34:52.871935+0000","last_unstale":"2024-09-11T23:36:47.152878+0000","last_undegraded":"2024-09-11T23:36:47.152878+0000","last_fullsized":"2024-09-11T23:36:47.152878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:08.217548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'41","reported_seq":73,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.248156+0000","last_change":"2024-09-11T23:34:50.857763+0000","last_active":"2024-09-11T23:36:47.248156+0000","last_peered":"2024-09-11T23:36:47.248156+0000","last_clean":"2024-09-11T23:36:47.248156+0000","last_became_active":"2024-09-11T23:34:50.857427+0000","last_became_peered":"2024-09-11T23:34:50.857427+0000","last_unstale":"2024-09-11T23:36:47.248156+0000","last_undegraded":"2024-09-11T23:36:47.248156+0000","last_fullsized":"2024-09-11T23:36:47.248156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:59.703580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152618+0000","last_change":"2024-09-11T23:34:52.872426+0000","last_active":"2024-09-11T23:36:47.152618+0000","last_peered":"2024-09-11T23:36:47.152618+0000","last_clean":"2024-09-11T23:36:47.152618+0000","last_became_active":"2024-09-11T23:34:52.872090+0000","last_became_peered":"2024-09-11T23:34:52.872090+0000","last_unstale":"2024-09-11T23:36:47.152618+0000","last_undegraded":"2024-09-11T23:36:47.152618+0000","last_fullsized":"2024-09-11T23:36:47.152618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:04:00.963963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'7","reported_seq":138,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058454+0000","last_change":"2024-09-11T23:34:50.924792+0000","last_active":"2024-09-11T23:36:47.058454+0000","last_peered":"2024-09-11T23:36:47.058454+0000","last_clean":"2024-09-11T23:36:47.058454+0000","last_became_active":"2024-09-11T23:34:50.924444+0000","last_became_peered":"2024-09-11T23:34:50.924444+0000","last_unstale":"2024-09-11T23:36:47.058454+0000","last_undegraded":"2024-09-11T23:36:47.058454+0000","last_fullsized":"2024-09-11T23:36:47.058454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:12.855314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152787+0000","last_change":"2024-09-11T23:34:52.872079+0000","last_active":"2024-09-11T23:36:47.152787+0000","last_peered":"2024-09-11T23:36:47.152787+0000","last_clean":"2024-09-11T23:36:47.152787+0000","last_became_active":"2024-09-11T23:34:52.871755+0000","last_became_peered":"2024-09-11T23:34:52.871755+0000","last_unstale":"2024-09-11T23:36:47.152787+0000","last_undegraded":"2024-09-11T23:36:47.152787+0000","last_fullsized":"2024-09-11T23:36:47.152787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:05.990832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":347,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152603+0000","last_change":"2024-09-11T23:33:42.256521+0000","last_active":"2024-09-11T23:36:47.152603+0000","last_peered":"2024-09-11T23:36:47.152603+0000","last_clean":"2024-09-11T23:36:47.152603+0000","last_became_active":"2024-09-11T23:33:42.256041+0000","last_became_peered":"2024-09-11T23:33:42.256041+0000","last_unstale":"2024-09-11T23:36:47.152603+0000","last_undegraded":"2024-09-11T23:36:47.152603+0000","last_fullsized":"2024-09-11T23:36:47.152603+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_clean_scrub_stamp":"2024-09-11T23:30:22.061515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:21.391839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292288+0000","last_change":"2024-09-11T23:34:50.854087+0000","last_active":"2024-09-11T23:36:47.292288+0000","last_peered":"2024-09-11T23:36:47.292288+0000","last_clean":"2024-09-11T23:36:47.292288+0000","last_became_active":"2024-09-11T23:34:50.852346+0000","last_became_peered":"2024-09-11T23:34:50.852346+0000","last_unstale":"2024-09-11T23:36:47.292288+0000","last_undegraded":"2024-09-11T23:36:47.292288+0000","last_fullsized":"2024-09-11T23:36:47.292288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.287119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146988+0000","last_change":"2024-09-11T23:34:52.869198+0000","last_active":"2024-09-11T23:36:47.146988+0000","last_peered":"2024-09-11T23:36:47.146988+0000","last_clean":"2024-09-11T23:36:47.146988+0000","last_became_active":"2024-09-11T23:34:52.868621+0000","last_became_peered":"2024-09-11T23:34:52.868621+0000","last_unstale":"2024-09-11T23:36:47.146988+0000","last_undegraded":"2024-09-11T23:36:47.146988+0000","last_fullsized":"2024-09-11T23:36:47.146988+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:17.430654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152604+0000","last_change":"2024-09-11T23:34:50.852752+0000","last_active":"2024-09-11T23:36:47.152604+0000","last_peered":"2024-09-11T23:36:47.152604+0000","last_clean":"2024-09-11T23:36:47.152604+0000","last_became_active":"2024-09-11T23:34:50.851895+0000","last_became_peered":"2024-09-11T23:34:50.851895+0000","last_unstale":"2024-09-11T23:36:47.152604+0000","last_undegraded":"2024-09-11T23:36:47.152604+0000","last_fullsized":"2024-09-11T23:36:47.152604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:46.894432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057056+0000","last_change":"2024-09-11T23:34:52.866191+0000","last_active":"2024-09-11T23:36:47.057056+0000","last_peered":"2024-09-11T23:36:47.057056+0000","last_clean":"2024-09-11T23:36:47.057056+0000","last_became_active":"2024-09-11T23:34:52.865858+0000","last_became_peered":"2024-09-11T23:34:52.865858+0000","last_unstale":"2024-09-11T23:36:47.057056+0000","last_undegraded":"2024-09-11T23:36:47.057056+0000","last_fullsized":"2024-09-11T23:36:47.057056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:21.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":136,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057262+0000","last_change":"2024-09-11T23:34:50.924125+0000","last_active":"2024-09-11T23:36:47.057262+0000","last_peered":"2024-09-11T23:36:47.057262+0000","last_clean":"2024-09-11T23:36:47.057262+0000","last_became_active":"2024-09-11T23:34:50.923780+0000","last_became_peered":"2024-09-11T23:34:50.923780+0000","last_unstale":"2024-09-11T23:36:47.057262+0000","last_undegraded":"2024-09-11T23:36:47.057262+0000","last_fullsized":"2024-09-11T23:36:47.057262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:39.774827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153672+0000","last_change":"2024-09-11T23:34:52.868364+0000","last_active":"2024-09-11T23:36:47.153672+0000","last_peered":"2024-09-11T23:36:47.153672+0000","last_clean":"2024-09-11T23:36:47.153672+0000","last_became_active":"2024-09-11T23:34:52.867913+0000","last_became_peered":"2024-09-11T23:34:52.867913+0000","last_unstale":"2024-09-11T23:36:47.153672+0000","last_undegraded":"2024-09-11T23:36:47.153672+0000","last_fullsized":"2024-09-11T23:36:47.153672+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.571825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":136,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.602771+0000","last_change":"2024-09-11T23:34:50.852421+0000","last_active":"2024-09-11T23:36:47.602771+0000","last_peered":"2024-09-11T23:36:47.602771+0000","last_clean":"2024-09-11T23:36:47.602771+0000","last_became_active":"2024-09-11T23:34:50.851898+0000","last_became_peered":"2024-09-11T23:34:50.851898+0000","last_unstale":"2024-09-11T23:36:47.602771+0000","last_undegraded":"2024-09-11T23:36:47.602771+0000","last_fullsized":"2024-09-11T23:36:47.602771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.661276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156359+0000","last_change":"2024-09-11T23:34:52.879995+0000","last_active":"2024-09-11T23:36:47.156359+0000","last_peered":"2024-09-11T23:36:47.156359+0000","last_clean":"2024-09-11T23:36:47.156359+0000","last_became_active":"2024-09-11T23:34:52.879678+0000","last_became_peered":"2024-09-11T23:34:52.879678+0000","last_unstale":"2024-09-11T23:36:47.156359+0000","last_undegraded":"2024-09-11T23:36:47.156359+0000","last_fullsized":"2024-09-11T23:36:47.156359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:47.456674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143075+0000","last_change":"2024-09-11T23:34:50.924957+0000","last_active":"2024-09-11T23:36:47.143075+0000","last_peered":"2024-09-11T23:36:47.143075+0000","last_clean":"2024-09-11T23:36:47.143075+0000","last_became_active":"2024-09-11T23:34:50.924500+0000","last_became_peered":"2024-09-11T23:34:50.924500+0000","last_unstale":"2024-09-11T23:36:47.143075+0000","last_undegraded":"2024-09-11T23:36:47.143075+0000","last_fullsized":"2024-09-11T23:36:47.143075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:39.268842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156159+0000","last_change":"2024-09-11T23:34:52.867337+0000","last_active":"2024-09-11T23:36:47.156159+0000","last_peered":"2024-09-11T23:36:47.156159+0000","last_clean":"2024-09-11T23:36:47.156159+0000","last_became_active":"2024-09-11T23:34:52.866799+0000","last_became_peered":"2024-09-11T23:34:52.866799+0000","last_unstale":"2024-09-11T23:36:47.156159+0000","last_undegraded":"2024-09-11T23:36:47.156159+0000","last_fullsized":"2024-09-11T23:36:47.156159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:38.177135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145920+0000","last_change":"2024-09-11T23:34:50.925156+0000","last_active":"2024-09-11T23:36:47.145920+0000","last_peered":"2024-09-11T23:36:47.145920+0000","last_clean":"2024-09-11T23:36:47.145920+0000","last_became_active":"2024-09-11T23:34:50.924282+0000","last_became_peered":"2024-09-11T23:34:50.924282+0000","last_unstale":"2024-09-11T23:36:47.145920+0000","last_undegraded":"2024-09-11T23:36:47.145920+0000","last_fullsized":"2024-09-11T23:36:47.145920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:47.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057264+0000","last_change":"2024-09-11T23:34:52.864993+0000","last_active":"2024-09-11T23:36:47.057264+0000","last_peered":"2024-09-11T23:36:47.057264+0000","last_clean":"2024-09-11T23:36:47.057264+0000","last_became_active":"2024-09-11T23:34:52.864267+0000","last_became_peered":"2024-09-11T23:34:52.864267+0000","last_unstale":"2024-09-11T23:36:47.057264+0000","last_undegraded":"2024-09-11T23:36:47.057264+0000","last_fullsized":"2024-09-11T23:36:47.057264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:29.912068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'52","reported_seq":99,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.250482+0000","last_change":"2024-09-11T23:34:50.849864+0000","last_active":"2024-09-11T23:36:47.250482+0000","last_peered":"2024-09-11T23:36:47.250482+0000","last_clean":"2024-09-11T23:36:47.250482+0000","last_became_active":"2024-09-11T23:34:50.849577+0000","last_became_peered":"2024-09-11T23:34:50.849577+0000","last_unstale":"2024-09-11T23:36:47.250482+0000","last_undegraded":"2024-09-11T23:36:47.250482+0000","last_fullsized":"2024-09-11T23:36:47.250482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:06.845000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147479+0000","last_change":"2024-09-11T23:34:52.872028+0000","last_active":"2024-09-11T23:36:47.147479+0000","last_peered":"2024-09-11T23:36:47.147479+0000","last_clean":"2024-09-11T23:36:47.147479+0000","last_became_active":"2024-09-11T23:34:52.871723+0000","last_became_peered":"2024-09-11T23:34:52.871723+0000","last_unstale":"2024-09-11T23:36:47.147479+0000","last_undegraded":"2024-09-11T23:36:47.147479+0000","last_fullsized":"2024-09-11T23:36:47.147479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:44.656218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292619+0000","last_change":"2024-09-11T23:34:50.860314+0000","last_active":"2024-09-11T23:36:47.292619+0000","last_peered":"2024-09-11T23:36:47.292619+0000","last_clean":"2024-09-11T23:36:47.292619+0000","last_became_active":"2024-09-11T23:34:50.859953+0000","last_became_peered":"2024-09-11T23:34:50.859953+0000","last_unstale":"2024-09-11T23:36:47.292619+0000","last_undegraded":"2024-09-11T23:36:47.292619+0000","last_fullsized":"2024-09-11T23:36:47.292619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:46.705864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328008+0000","last_change":"2024-09-11T23:34:52.867190+0000","last_active":"2024-09-11T23:36:47.328008+0000","last_peered":"2024-09-11T23:36:47.328008+0000","last_clean":"2024-09-11T23:36:47.328008+0000","last_became_active":"2024-09-11T23:34:52.866829+0000","last_became_peered":"2024-09-11T23:34:52.866829+0000","last_unstale":"2024-09-11T23:36:47.328008+0000","last_undegraded":"2024-09-11T23:36:47.328008+0000","last_fullsized":"2024-09-11T23:36:47.328008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.752006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058275+0000","last_change":"2024-09-11T23:34:50.924566+0000","last_active":"2024-09-11T23:36:47.058275+0000","last_peered":"2024-09-11T23:36:47.058275+0000","last_clean":"2024-09-11T23:36:47.058275+0000","last_became_active":"2024-09-11T23:34:50.924284+0000","last_became_peered":"2024-09-11T23:34:50.924284+0000","last_unstale":"2024-09-11T23:36:47.058275+0000","last_undegraded":"2024-09-11T23:36:47.058275+0000","last_fullsized":"2024-09-11T23:36:47.058275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:54.246154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145487+0000","last_change":"2024-09-11T23:34:52.869302+0000","last_active":"2024-09-11T23:36:47.145487+0000","last_peered":"2024-09-11T23:36:47.145487+0000","last_clean":"2024-09-11T23:36:47.145487+0000","last_became_active":"2024-09-11T23:34:52.868960+0000","last_became_peered":"2024-09-11T23:34:52.868960+0000","last_unstale":"2024-09-11T23:36:47.145487+0000","last_undegraded":"2024-09-11T23:36:47.145487+0000","last_fullsized":"2024-09-11T23:36:47.145487+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.561712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146271+0000","last_change":"2024-09-11T23:34:50.854573+0000","last_active":"2024-09-11T23:36:47.146271+0000","last_peered":"2024-09-11T23:36:47.146271+0000","last_clean":"2024-09-11T23:36:47.146271+0000","last_became_active":"2024-09-11T23:34:50.854165+0000","last_became_peered":"2024-09-11T23:34:50.854165+0000","last_unstale":"2024-09-11T23:36:47.146271+0000","last_undegraded":"2024-09-11T23:36:47.146271+0000","last_fullsized":"2024-09-11T23:36:47.146271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:57:31.589495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327542+0000","last_change":"2024-09-11T23:34:52.875987+0000","last_active":"2024-09-11T23:36:47.327542+0000","last_peered":"2024-09-11T23:36:47.327542+0000","last_clean":"2024-09-11T23:36:47.327542+0000","last_became_active":"2024-09-11T23:34:52.875662+0000","last_became_peered":"2024-09-11T23:34:52.875662+0000","last_unstale":"2024-09-11T23:36:47.327542+0000","last_undegraded":"2024-09-11T23:36:47.327542+0000","last_fullsized":"2024-09-11T23:36:47.327542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:16.514938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058234+0000","last_change":"2024-09-11T23:34:50.850726+0000","last_active":"2024-09-11T23:36:47.058234+0000","last_peered":"2024-09-11T23:36:47.058234+0000","last_clean":"2024-09-11T23:36:47.058234+0000","last_became_active":"2024-09-11T23:34:50.850381+0000","last_became_peered":"2024-09-11T23:34:50.850381+0000","last_unstale":"2024-09-11T23:36:47.058234+0000","last_undegraded":"2024-09-11T23:36:47.058234+0000","last_fullsized":"2024-09-11T23:36:47.058234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:50.431293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146377+0000","last_change":"2024-09-11T23:34:52.870652+0000","last_active":"2024-09-11T23:36:47.146377+0000","last_peered":"2024-09-11T23:36:47.146377+0000","last_clean":"2024-09-11T23:36:47.146377+0000","last_became_active":"2024-09-11T23:34:52.870303+0000","last_became_peered":"2024-09-11T23:34:52.870303+0000","last_unstale":"2024-09-11T23:36:47.146377+0000","last_undegraded":"2024-09-11T23:36:47.146377+0000","last_fullsized":"2024-09-11T23:36:47.146377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:17.670981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327509+0000","last_change":"2024-09-11T23:35:08.854490+0000","last_active":"2024-09-11T23:36:47.327509+0000","last_peered":"2024-09-11T23:36:47.327509+0000","last_clean":"2024-09-11T23:36:47.327509+0000","last_became_active":"2024-09-11T23:35:08.849820+0000","last_became_peered":"2024-09-11T23:35:08.849820+0000","last_unstale":"2024-09-11T23:36:47.327509+0000","last_undegraded":"2024-09-11T23:36:47.327509+0000","last_fullsized":"2024-09-11T23:36:47.327509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:09.509867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152580+0000","last_change":"2024-09-11T23:34:52.869695+0000","last_active":"2024-09-11T23:36:47.152580+0000","last_peered":"2024-09-11T23:36:47.152580+0000","last_clean":"2024-09-11T23:36:47.152580+0000","last_became_active":"2024-09-11T23:34:52.869367+0000","last_became_peered":"2024-09-11T23:34:52.869367+0000","last_unstale":"2024-09-11T23:36:47.152580+0000","last_undegraded":"2024-09-11T23:36:47.152580+0000","last_fullsized":"2024-09-11T23:36:47.152580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:16.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153822+0000","last_change":"2024-09-11T23:34:52.872778+0000","last_active":"2024-09-11T23:36:47.153822+0000","last_peered":"2024-09-11T23:36:47.153822+0000","last_clean":"2024-09-11T23:36:47.153822+0000","last_became_active":"2024-09-11T23:34:52.872451+0000","last_became_peered":"2024-09-11T23:34:52.872451+0000","last_unstale":"2024-09-11T23:36:47.153822+0000","last_undegraded":"2024-09-11T23:36:47.153822+0000","last_fullsized":"2024-09-11T23:36:47.153822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:51.769927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145825+0000","last_change":"2024-09-11T23:34:50.849675+0000","last_active":"2024-09-11T23:36:47.145825+0000","last_peered":"2024-09-11T23:36:47.145825+0000","last_clean":"2024-09-11T23:36:47.145825+0000","last_became_active":"2024-09-11T23:34:50.849228+0000","last_became_peered":"2024-09-11T23:34:50.849228+0000","last_unstale":"2024-09-11T23:36:47.145825+0000","last_undegraded":"2024-09-11T23:36:47.145825+0000","last_fullsized":"2024-09-11T23:36:47.145825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:50.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058399+0000","last_change":"2024-09-11T23:34:52.867739+0000","last_active":"2024-09-11T23:36:47.058399+0000","last_peered":"2024-09-11T23:36:47.058399+0000","last_clean":"2024-09-11T23:36:47.058399+0000","last_became_active":"2024-09-11T23:34:52.866613+0000","last_became_peered":"2024-09-11T23:34:52.866613+0000","last_unstale":"2024-09-11T23:36:47.058399+0000","last_undegraded":"2024-09-11T23:36:47.058399+0000","last_fullsized":"2024-09-11T23:36:47.058399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:49.348386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'13","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145845+0000","last_change":"2024-09-11T23:34:50.851401+0000","last_active":"2024-09-11T23:36:47.145845+0000","last_peered":"2024-09-11T23:36:47.145845+0000","last_clean":"2024-09-11T23:36:47.145845+0000","last_became_active":"2024-09-11T23:34:50.851068+0000","last_became_peered":"2024-09-11T23:34:50.851068+0000","last_unstale":"2024-09-11T23:36:47.145845+0000","last_undegraded":"2024-09-11T23:36:47.145845+0000","last_fullsized":"2024-09-11T23:36:47.145845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:09.673864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328128+0000","last_change":"2024-09-11T23:34:52.865216+0000","last_active":"2024-09-11T23:36:47.328128+0000","last_peered":"2024-09-11T23:36:47.328128+0000","last_clean":"2024-09-11T23:36:47.328128+0000","last_became_active":"2024-09-11T23:34:52.864917+0000","last_became_peered":"2024-09-11T23:34:52.864917+0000","last_unstale":"2024-09-11T23:36:47.328128+0000","last_undegraded":"2024-09-11T23:36:47.328128+0000","last_fullsized":"2024-09-11T23:36:47.328128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:36.469793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147616+0000","last_change":"2024-09-11T23:34:50.924291+0000","last_active":"2024-09-11T23:36:47.147616+0000","last_peered":"2024-09-11T23:36:47.147616+0000","last_clean":"2024-09-11T23:36:47.147616+0000","last_became_active":"2024-09-11T23:34:50.923892+0000","last_became_peered":"2024-09-11T23:34:50.923892+0000","last_unstale":"2024-09-11T23:36:47.147616+0000","last_undegraded":"2024-09-11T23:36:47.147616+0000","last_fullsized":"2024-09-11T23:36:47.147616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:52.266778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147387+0000","last_change":"2024-09-11T23:34:52.875995+0000","last_active":"2024-09-11T23:36:47.147387+0000","last_peered":"2024-09-11T23:36:47.147387+0000","last_clean":"2024-09-11T23:36:47.147387+0000","last_became_active":"2024-09-11T23:34:52.875654+0000","last_became_peered":"2024-09-11T23:34:52.875654+0000","last_unstale":"2024-09-11T23:36:47.147387+0000","last_undegraded":"2024-09-11T23:36:47.147387+0000","last_fullsized":"2024-09-11T23:36:47.147387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:19.218291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292143+0000","last_change":"2024-09-11T23:34:50.853881+0000","last_active":"2024-09-11T23:36:47.292143+0000","last_peered":"2024-09-11T23:36:47.292143+0000","last_clean":"2024-09-11T23:36:47.292143+0000","last_became_active":"2024-09-11T23:34:50.853576+0000","last_became_peered":"2024-09-11T23:34:50.853576+0000","last_unstale":"2024-09-11T23:36:47.292143+0000","last_undegraded":"2024-09-11T23:36:47.292143+0000","last_fullsized":"2024-09-11T23:36:47.292143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:38.550587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597446,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1182,"num_read_kb":703,"num_write":381,"num_write_kb":1464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1792338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612644,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841576,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1748992,"data_stored":1230354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005102,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32484,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712668,"statfs":{"total":95995035648,"available":95961772032,"internally_reserved":0,"allocated":1327104,"data_stored":836376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28320,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716832,"statfs":{"total":95995035648,"available":95966035968,"internally_reserved":0,"allocated":1323008,"data_stored":836754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495672,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1527808,"data_stored":1038472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659195,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":442877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888129,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1167336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018951,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":444366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1462272,"data_stored":967518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444113,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":507736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607637,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":506657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":444750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":197831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:36:52.794 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-11T23:36:52.795 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:36:52.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.934+0000 7f8324cdd640 1 -- 172.21.15.133:0/725327542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f832010c080 msgr2=0x7f832010e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.934+0000 7f8324cdd640 1 --2- 172.21.15.133:0/725327542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f832010c080 0x7f832010e540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8318009900 tx=0x7f831802a480 comp rx=0 tx=0).stop 2024-09-11T23:36:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.934+0000 7f8324cdd640 1 -- 172.21.15.133:0/725327542 shutdown_connections 2024-09-11T23:36:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.934+0000 7f8324cdd640 1 --2- 172.21.15.133:0/725327542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f832010c080 0x7f832010e540 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.934+0000 7f8324cdd640 1 --2- 172.21.15.133:0/725327542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8320103580 0x7f832010ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.934+0000 7f8324cdd640 1 --2- 172.21.15.133:0/725327542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8320102c60 0x7f8320103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.934+0000 7f8324cdd640 1 -- 172.21.15.133:0/725327542 >> 172.21.15.133:0/725327542 conn(0x7f83200fc6d0 msgr2=0x7f83200feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.934+0000 7f8324cdd640 1 -- 172.21.15.133:0/725327542 shutdown_connections 2024-09-11T23:36:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.934+0000 7f8324cdd640 1 -- 172.21.15.133:0/725327542 wait complete. 2024-09-11T23:36:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f8324cdd640 1 Processor -- start 2024-09-11T23:36:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f8324cdd640 1 -- start start 2024-09-11T23:36:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f8324cdd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320102c60 0x7f8320198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f8324cdd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8320103580 0x7f8320198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f8324cdd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832010c080 0x7f832019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f8324cdd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8320111ea0 con 0x7f8320102c60 2024-09-11T23:36:52.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f8324cdd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8320111d20 con 0x7f832010c080 2024-09-11T23:36:52.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f8324cdd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8320112020 con 0x7f8320103580 2024-09-11T23:36:52.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f831e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320102c60 0x7f8320198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f831e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320102c60 0x7f8320198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33166/0 (socket says 172.21.15.133:33166) 2024-09-11T23:36:52.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f831e59c640 1 -- 172.21.15.133:0/3960657899 learned_addr learned my addr 172.21.15.133:0/3960657899 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:52.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f831dd9b640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8320103580 0x7f8320198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f831ed9d640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832010c080 0x7f832019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f831e59c640 1 -- 172.21.15.133:0/3960657899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8320103580 msgr2=0x7f8320198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f831e59c640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8320103580 0x7f8320198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f831e59c640 1 -- 172.21.15.133:0/3960657899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832010c080 msgr2=0x7f832019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:52.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f831e59c640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832010c080 0x7f832019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:52.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.935+0000 7f831e59c640 1 -- 172.21.15.133:0/3960657899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832019d340 con 0x7f8320102c60 2024-09-11T23:36:52.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.936+0000 7f831e59c640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320102c60 0x7f8320198350 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f831400ea80 tx=0x7f831400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:52.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.936+0000 7f830f7fe640 1 -- 172.21.15.133:0/3960657899 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831400cd80 con 0x7f8320102c60 2024-09-11T23:36:52.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.936+0000 7f8324cdd640 1 -- 172.21.15.133:0/3960657899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832019d630 con 0x7f8320102c60 2024-09-11T23:36:52.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.936+0000 7f830f7fe640 1 -- 172.21.15.133:0/3960657899 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8314005020 con 0x7f8320102c60 2024-09-11T23:36:52.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.936+0000 7f830f7fe640 1 -- 172.21.15.133:0/3960657899 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140116d0 con 0x7f8320102c60 2024-09-11T23:36:52.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.936+0000 7f8324cdd640 1 -- 172.21.15.133:0/3960657899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83201a4ec0 con 0x7f8320102c60 2024-09-11T23:36:52.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.937+0000 7f8324cdd640 1 -- 172.21.15.133:0/3960657899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8320103040 con 0x7f8320102c60 2024-09-11T23:36:52.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.938+0000 7f830f7fe640 1 -- 172.21.15.133:0/3960657899 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f8314005560 con 0x7f8320102c60 2024-09-11T23:36:52.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.938+0000 7f830f7fe640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f83080a2e20 0x7f83080a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:52.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.939+0000 7f830f7fe640 1 -- 172.21.15.133:0/3960657899 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f83140cfca0 con 0x7f8320102c60 2024-09-11T23:36:52.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.939+0000 7f831dd9b640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f83080a2e20 0x7f83080a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:52.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.939+0000 7f831dd9b640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f83080a2e20 0x7f83080a52e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f83200fec10 tx=0x7f831000a2d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:52.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:52.943+0000 7f830f7fe640 1 -- 172.21.15.133:0/3960657899 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8314016030 con 0x7f8320102c60 2024-09-11T23:36:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.108+0000 7f8324cdd640 1 -- 172.21.15.133:0/3960657899 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83200fe3b0 con 0x7f83080a2e20 2024-09-11T23:36:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.114+0000 7f830f7fe640 1 -- 172.21.15.133:0/3960657899 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257786 (secure 0 0 0) 0x7f83200fe3b0 con 0x7f83080a2e20 2024-09-11T23:36:53.117 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T23:36:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 -- 172.21.15.133:0/3960657899 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f83080a2e20 msgr2=0x7f83080a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f83080a2e20 0x7f83080a52e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f83200fec10 tx=0x7f831000a2d0 comp rx=0 tx=0).stop 2024-09-11T23:36:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 -- 172.21.15.133:0/3960657899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320102c60 msgr2=0x7f8320198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:53.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320102c60 0x7f8320198350 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f831400ea80 tx=0x7f831400ee80 comp rx=0 tx=0).stop 2024-09-11T23:36:53.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 -- 172.21.15.133:0/3960657899 shutdown_connections 2024-09-11T23:36:53.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f83080a2e20 0x7f83080a52e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:53.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832010c080 0x7f832019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:53.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8320103580 0x7f8320198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:53.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 --2- 172.21.15.133:0/3960657899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320102c60 0x7f8320198350 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:53.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 -- 172.21.15.133:0/3960657899 >> 172.21.15.133:0/3960657899 conn(0x7f83200fc6d0 msgr2=0x7f83200fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:53.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 -- 172.21.15.133:0/3960657899 shutdown_connections 2024-09-11T23:36:53.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.117+0000 7f8324cdd640 1 -- 172.21.15.133:0/3960657899 wait complete. 2024-09-11T23:36:53.127 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:53 smithi133 ceph-mon[31430]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:36:53.127 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:53 smithi133 ceph-mon[31430]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-11T23:36:53.128 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-09-11T23:36:53.069466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188048,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1323,"num_read_kb":827,"num_write":564,"num_write_kb":4404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359512,"kb_used_data":14388,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582312,"statfs":{"total":1151940427776,"available":1151572287488,"internally_reserved":0,"allocated":14733312,"data_stored":8868207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.989396"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327838+0000","last_change":"2024-09-11T23:34:52.867694+0000","last_active":"2024-09-11T23:36:47.327838+0000","last_peered":"2024-09-11T23:36:47.327838+0000","last_clean":"2024-09-11T23:36:47.327838+0000","last_became_active":"2024-09-11T23:34:52.867368+0000","last_became_peered":"2024-09-11T23:34:52.867368+0000","last_unstale":"2024-09-11T23:36:47.327838+0000","last_undegraded":"2024-09-11T23:36:47.327838+0000","last_fullsized":"2024-09-11T23:36:47.327838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:46:07.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327981+0000","last_change":"2024-09-11T23:34:52.871881+0000","last_active":"2024-09-11T23:36:47.327981+0000","last_peered":"2024-09-11T23:36:47.327981+0000","last_clean":"2024-09-11T23:36:47.327981+0000","last_became_active":"2024-09-11T23:34:52.871609+0000","last_became_peered":"2024-09-11T23:34:52.871609+0000","last_unstale":"2024-09-11T23:36:47.327981+0000","last_undegraded":"2024-09-11T23:36:47.327981+0000","last_fullsized":"2024-09-11T23:36:47.327981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.938584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327774+0000","last_change":"2024-09-11T23:34:52.863931+0000","last_active":"2024-09-11T23:36:47.327774+0000","last_peered":"2024-09-11T23:36:47.327774+0000","last_clean":"2024-09-11T23:36:47.327774+0000","last_became_active":"2024-09-11T23:34:52.863637+0000","last_became_peered":"2024-09-11T23:34:52.863637+0000","last_unstale":"2024-09-11T23:36:47.327774+0000","last_undegraded":"2024-09-11T23:36:47.327774+0000","last_fullsized":"2024-09-11T23:36:47.327774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:44.118865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292412+0000","last_change":"2024-09-11T23:34:52.879619+0000","last_active":"2024-09-11T23:36:47.292412+0000","last_peered":"2024-09-11T23:36:47.292412+0000","last_clean":"2024-09-11T23:36:47.292412+0000","last_became_active":"2024-09-11T23:34:52.878695+0000","last_became_peered":"2024-09-11T23:34:52.878695+0000","last_unstale":"2024-09-11T23:36:47.292412+0000","last_undegraded":"2024-09-11T23:36:47.292412+0000","last_fullsized":"2024-09-11T23:36:47.292412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:30.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145593+0000","last_change":"2024-09-11T23:34:52.869363+0000","last_active":"2024-09-11T23:36:47.145593+0000","last_peered":"2024-09-11T23:36:47.145593+0000","last_clean":"2024-09-11T23:36:47.145593+0000","last_became_active":"2024-09-11T23:34:52.869027+0000","last_became_peered":"2024-09-11T23:34:52.869027+0000","last_unstale":"2024-09-11T23:36:47.145593+0000","last_undegraded":"2024-09-11T23:36:47.145593+0000","last_fullsized":"2024-09-11T23:36:47.145593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:15.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156014+0000","last_change":"2024-09-11T23:34:52.880665+0000","last_active":"2024-09-11T23:36:47.156014+0000","last_peered":"2024-09-11T23:36:47.156014+0000","last_clean":"2024-09-11T23:36:47.156014+0000","last_became_active":"2024-09-11T23:34:52.880172+0000","last_became_peered":"2024-09-11T23:34:52.880172+0000","last_unstale":"2024-09-11T23:36:47.156014+0000","last_undegraded":"2024-09-11T23:36:47.156014+0000","last_fullsized":"2024-09-11T23:36:47.156014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:12.025877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152489+0000","last_change":"2024-09-11T23:34:52.873897+0000","last_active":"2024-09-11T23:36:47.152489+0000","last_peered":"2024-09-11T23:36:47.152489+0000","last_clean":"2024-09-11T23:36:47.152489+0000","last_became_active":"2024-09-11T23:34:52.873219+0000","last_became_peered":"2024-09-11T23:34:52.873219+0000","last_unstale":"2024-09-11T23:36:47.152489+0000","last_undegraded":"2024-09-11T23:36:47.152489+0000","last_fullsized":"2024-09-11T23:36:47.152489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:47.125682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145794+0000","last_change":"2024-09-11T23:34:52.870699+0000","last_active":"2024-09-11T23:36:47.145794+0000","last_peered":"2024-09-11T23:36:47.145794+0000","last_clean":"2024-09-11T23:36:47.145794+0000","last_became_active":"2024-09-11T23:34:52.870314+0000","last_became_peered":"2024-09-11T23:34:52.870314+0000","last_unstale":"2024-09-11T23:36:47.145794+0000","last_undegraded":"2024-09-11T23:36:47.145794+0000","last_fullsized":"2024-09-11T23:36:47.145794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:41.263159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058666+0000","last_change":"2024-09-11T23:34:52.869700+0000","last_active":"2024-09-11T23:36:47.058666+0000","last_peered":"2024-09-11T23:36:47.058666+0000","last_clean":"2024-09-11T23:36:47.058666+0000","last_became_active":"2024-09-11T23:34:52.869398+0000","last_became_peered":"2024-09-11T23:34:52.869398+0000","last_unstale":"2024-09-11T23:36:47.058666+0000","last_undegraded":"2024-09-11T23:36:47.058666+0000","last_fullsized":"2024-09-11T23:36:47.058666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:49.776525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156161+0000","last_change":"2024-09-11T23:34:52.879999+0000","last_active":"2024-09-11T23:36:47.156161+0000","last_peered":"2024-09-11T23:36:47.156161+0000","last_clean":"2024-09-11T23:36:47.156161+0000","last_became_active":"2024-09-11T23:34:52.879646+0000","last_became_peered":"2024-09-11T23:34:52.879646+0000","last_unstale":"2024-09-11T23:36:47.156161+0000","last_undegraded":"2024-09-11T23:36:47.156161+0000","last_fullsized":"2024-09-11T23:36:47.156161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.058704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152665+0000","last_change":"2024-09-11T23:34:52.878740+0000","last_active":"2024-09-11T23:36:47.152665+0000","last_peered":"2024-09-11T23:36:47.152665+0000","last_clean":"2024-09-11T23:36:47.152665+0000","last_became_active":"2024-09-11T23:34:52.878120+0000","last_became_peered":"2024-09-11T23:34:52.878120+0000","last_unstale":"2024-09-11T23:36:47.152665+0000","last_undegraded":"2024-09-11T23:36:47.152665+0000","last_fullsized":"2024-09-11T23:36:47.152665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292447+0000","last_change":"2024-09-11T23:34:52.873409+0000","last_active":"2024-09-11T23:36:47.292447+0000","last_peered":"2024-09-11T23:36:47.292447+0000","last_clean":"2024-09-11T23:36:47.292447+0000","last_became_active":"2024-09-11T23:34:52.872959+0000","last_became_peered":"2024-09-11T23:34:52.872959+0000","last_unstale":"2024-09-11T23:36:47.292447+0000","last_undegraded":"2024-09-11T23:36:47.292447+0000","last_fullsized":"2024-09-11T23:36:47.292447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:38.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057362+0000","last_change":"2024-09-11T23:34:52.869164+0000","last_active":"2024-09-11T23:36:47.057362+0000","last_peered":"2024-09-11T23:36:47.057362+0000","last_clean":"2024-09-11T23:36:47.057362+0000","last_became_active":"2024-09-11T23:34:52.868847+0000","last_became_peered":"2024-09-11T23:34:52.868847+0000","last_unstale":"2024-09-11T23:36:47.057362+0000","last_undegraded":"2024-09-11T23:36:47.057362+0000","last_fullsized":"2024-09-11T23:36:47.057362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.764510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327562+0000","last_change":"2024-09-11T23:34:52.869494+0000","last_active":"2024-09-11T23:36:47.327562+0000","last_peered":"2024-09-11T23:36:47.327562+0000","last_clean":"2024-09-11T23:36:47.327562+0000","last_became_active":"2024-09-11T23:34:52.869178+0000","last_became_peered":"2024-09-11T23:34:52.869178+0000","last_unstale":"2024-09-11T23:36:47.327562+0000","last_undegraded":"2024-09-11T23:36:47.327562+0000","last_fullsized":"2024-09-11T23:36:47.327562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:23.431564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327741+0000","last_change":"2024-09-11T23:34:52.866269+0000","last_active":"2024-09-11T23:36:47.327741+0000","last_peered":"2024-09-11T23:36:47.327741+0000","last_clean":"2024-09-11T23:36:47.327741+0000","last_became_active":"2024-09-11T23:34:52.865980+0000","last_became_peered":"2024-09-11T23:34:52.865980+0000","last_unstale":"2024-09-11T23:36:47.327741+0000","last_undegraded":"2024-09-11T23:36:47.327741+0000","last_fullsized":"2024-09-11T23:36:47.327741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:48.492252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058174+0000","last_change":"2024-09-11T23:34:52.862192+0000","last_active":"2024-09-11T23:36:47.058174+0000","last_peered":"2024-09-11T23:36:47.058174+0000","last_clean":"2024-09-11T23:36:47.058174+0000","last_became_active":"2024-09-11T23:34:52.861841+0000","last_became_peered":"2024-09-11T23:34:52.861841+0000","last_unstale":"2024-09-11T23:36:47.058174+0000","last_undegraded":"2024-09-11T23:36:47.058174+0000","last_fullsized":"2024-09-11T23:36:47.058174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.258948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327836+0000","last_change":"2024-09-11T23:34:52.870984+0000","last_active":"2024-09-11T23:36:47.327836+0000","last_peered":"2024-09-11T23:36:47.327836+0000","last_clean":"2024-09-11T23:36:47.327836+0000","last_became_active":"2024-09-11T23:34:52.870435+0000","last_became_peered":"2024-09-11T23:34:52.870435+0000","last_unstale":"2024-09-11T23:36:47.327836+0000","last_undegraded":"2024-09-11T23:36:47.327836+0000","last_fullsized":"2024-09-11T23:36:47.327836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:34.391537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143297+0000","last_change":"2024-09-11T23:34:52.873176+0000","last_active":"2024-09-11T23:36:47.143297+0000","last_peered":"2024-09-11T23:36:47.143297+0000","last_clean":"2024-09-11T23:36:47.143297+0000","last_became_active":"2024-09-11T23:34:52.872862+0000","last_became_peered":"2024-09-11T23:34:52.872862+0000","last_unstale":"2024-09-11T23:36:47.143297+0000","last_undegraded":"2024-09-11T23:36:47.143297+0000","last_fullsized":"2024-09-11T23:36:47.143297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:38.058806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058317+0000","last_change":"2024-09-11T23:34:52.871346+0000","last_active":"2024-09-11T23:36:47.058317+0000","last_peered":"2024-09-11T23:36:47.058317+0000","last_clean":"2024-09-11T23:36:47.058317+0000","last_became_active":"2024-09-11T23:34:52.870981+0000","last_became_peered":"2024-09-11T23:34:52.870981+0000","last_unstale":"2024-09-11T23:36:47.058317+0000","last_undegraded":"2024-09-11T23:36:47.058317+0000","last_fullsized":"2024-09-11T23:36:47.058317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:56.973998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327601+0000","last_change":"2024-09-11T23:34:52.867058+0000","last_active":"2024-09-11T23:36:47.327601+0000","last_peered":"2024-09-11T23:36:47.327601+0000","last_clean":"2024-09-11T23:36:47.327601+0000","last_became_active":"2024-09-11T23:34:52.866687+0000","last_became_peered":"2024-09-11T23:34:52.866687+0000","last_unstale":"2024-09-11T23:36:47.327601+0000","last_undegraded":"2024-09-11T23:36:47.327601+0000","last_fullsized":"2024-09-11T23:36:47.327601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:09.019987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057475+0000","last_change":"2024-09-11T23:34:52.870882+0000","last_active":"2024-09-11T23:36:47.057475+0000","last_peered":"2024-09-11T23:36:47.057475+0000","last_clean":"2024-09-11T23:36:47.057475+0000","last_became_active":"2024-09-11T23:34:52.870667+0000","last_became_peered":"2024-09-11T23:34:52.870667+0000","last_unstale":"2024-09-11T23:36:47.057475+0000","last_undegraded":"2024-09-11T23:36:47.057475+0000","last_fullsized":"2024-09-11T23:36:47.057475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:08.386703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058304+0000","last_change":"2024-09-11T23:34:52.874070+0000","last_active":"2024-09-11T23:36:47.058304+0000","last_peered":"2024-09-11T23:36:47.058304+0000","last_clean":"2024-09-11T23:36:47.058304+0000","last_became_active":"2024-09-11T23:34:52.873772+0000","last_became_peered":"2024-09-11T23:34:52.873772+0000","last_unstale":"2024-09-11T23:36:47.058304+0000","last_undegraded":"2024-09-11T23:36:47.058304+0000","last_fullsized":"2024-09-11T23:36:47.058304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:49.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146028+0000","last_change":"2024-09-11T23:34:52.875502+0000","last_active":"2024-09-11T23:36:47.146028+0000","last_peered":"2024-09-11T23:36:47.146028+0000","last_clean":"2024-09-11T23:36:47.146028+0000","last_became_active":"2024-09-11T23:34:52.875193+0000","last_became_peered":"2024-09-11T23:34:52.875193+0000","last_unstale":"2024-09-11T23:36:47.146028+0000","last_undegraded":"2024-09-11T23:36:47.146028+0000","last_fullsized":"2024-09-11T23:36:47.146028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:37.213828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153497+0000","last_change":"2024-09-11T23:34:52.869015+0000","last_active":"2024-09-11T23:36:47.153497+0000","last_peered":"2024-09-11T23:36:47.153497+0000","last_clean":"2024-09-11T23:36:47.153497+0000","last_became_active":"2024-09-11T23:34:52.868731+0000","last_became_peered":"2024-09-11T23:34:52.868731+0000","last_unstale":"2024-09-11T23:36:47.153497+0000","last_undegraded":"2024-09-11T23:36:47.153497+0000","last_fullsized":"2024-09-11T23:36:47.153497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:12:15.993203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147366+0000","last_change":"2024-09-11T23:34:52.875555+0000","last_active":"2024-09-11T23:36:47.147366+0000","last_peered":"2024-09-11T23:36:47.147366+0000","last_clean":"2024-09-11T23:36:47.147366+0000","last_became_active":"2024-09-11T23:34:52.875219+0000","last_became_peered":"2024-09-11T23:34:52.875219+0000","last_unstale":"2024-09-11T23:36:47.147366+0000","last_undegraded":"2024-09-11T23:36:47.147366+0000","last_fullsized":"2024-09-11T23:36:47.147366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:16.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058480+0000","last_change":"2024-09-11T23:34:52.875629+0000","last_active":"2024-09-11T23:36:47.058480+0000","last_peered":"2024-09-11T23:36:47.058480+0000","last_clean":"2024-09-11T23:36:47.058480+0000","last_became_active":"2024-09-11T23:34:52.875302+0000","last_became_peered":"2024-09-11T23:34:52.875302+0000","last_unstale":"2024-09-11T23:36:47.058480+0000","last_undegraded":"2024-09-11T23:36:47.058480+0000","last_fullsized":"2024-09-11T23:36:47.058480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:12.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327744+0000","last_change":"2024-09-11T23:34:52.868613+0000","last_active":"2024-09-11T23:36:47.327744+0000","last_peered":"2024-09-11T23:36:47.327744+0000","last_clean":"2024-09-11T23:36:47.327744+0000","last_became_active":"2024-09-11T23:34:52.868184+0000","last_became_peered":"2024-09-11T23:34:52.868184+0000","last_unstale":"2024-09-11T23:36:47.327744+0000","last_undegraded":"2024-09-11T23:36:47.327744+0000","last_fullsized":"2024-09-11T23:36:47.327744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:25.990002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057237+0000","last_change":"2024-09-11T23:35:08.873390+0000","last_active":"2024-09-11T23:36:47.057237+0000","last_peered":"2024-09-11T23:36:47.057237+0000","last_clean":"2024-09-11T23:36:47.057237+0000","last_became_active":"2024-09-11T23:35:08.873052+0000","last_became_peered":"2024-09-11T23:35:08.873052+0000","last_unstale":"2024-09-11T23:36:47.057237+0000","last_undegraded":"2024-09-11T23:36:47.057237+0000","last_fullsized":"2024-09-11T23:36:47.057237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.937951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327689+0000","last_change":"2024-09-11T23:34:52.873604+0000","last_active":"2024-09-11T23:36:47.327689+0000","last_peered":"2024-09-11T23:36:47.327689+0000","last_clean":"2024-09-11T23:36:47.327689+0000","last_became_active":"2024-09-11T23:34:52.873220+0000","last_became_peered":"2024-09-11T23:34:52.873220+0000","last_unstale":"2024-09-11T23:36:47.327689+0000","last_undegraded":"2024-09-11T23:36:47.327689+0000","last_fullsized":"2024-09-11T23:36:47.327689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:38.480630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058025+0000","last_change":"2024-09-11T23:34:52.872902+0000","last_active":"2024-09-11T23:36:47.058025+0000","last_peered":"2024-09-11T23:36:47.058025+0000","last_clean":"2024-09-11T23:36:47.058025+0000","last_became_active":"2024-09-11T23:34:52.872571+0000","last_became_peered":"2024-09-11T23:34:52.872571+0000","last_unstale":"2024-09-11T23:36:47.058025+0000","last_undegraded":"2024-09-11T23:36:47.058025+0000","last_fullsized":"2024-09-11T23:36:47.058025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:21.844995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058115+0000","last_change":"2024-09-11T23:35:09.457056+0000","last_active":"2024-09-11T23:36:47.058115+0000","last_peered":"2024-09-11T23:36:47.058115+0000","last_clean":"2024-09-11T23:36:47.058115+0000","last_became_active":"2024-09-11T23:35:09.456657+0000","last_became_peered":"2024-09-11T23:35:09.456657+0000","last_unstale":"2024-09-11T23:36:47.058115+0000","last_undegraded":"2024-09-11T23:36:47.058115+0000","last_fullsized":"2024-09-11T23:36:47.058115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:12:56.121280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327466+0000","last_change":"2024-09-11T23:34:52.875312+0000","last_active":"2024-09-11T23:36:47.327466+0000","last_peered":"2024-09-11T23:36:47.327466+0000","last_clean":"2024-09-11T23:36:47.327466+0000","last_became_active":"2024-09-11T23:34:52.874873+0000","last_became_peered":"2024-09-11T23:34:52.874873+0000","last_unstale":"2024-09-11T23:36:47.327466+0000","last_undegraded":"2024-09-11T23:36:47.327466+0000","last_fullsized":"2024-09-11T23:36:47.327466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:18.894713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145537+0000","last_change":"2024-09-11T23:34:52.862266+0000","last_active":"2024-09-11T23:36:47.145537+0000","last_peered":"2024-09-11T23:36:47.145537+0000","last_clean":"2024-09-11T23:36:47.145537+0000","last_became_active":"2024-09-11T23:34:52.861923+0000","last_became_peered":"2024-09-11T23:34:52.861923+0000","last_unstale":"2024-09-11T23:36:47.145537+0000","last_undegraded":"2024-09-11T23:36:47.145537+0000","last_fullsized":"2024-09-11T23:36:47.145537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:23.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":138,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.591379+0000","last_change":"2024-09-11T23:34:50.854020+0000","last_active":"2024-09-11T23:36:47.591379+0000","last_peered":"2024-09-11T23:36:47.591379+0000","last_clean":"2024-09-11T23:36:47.591379+0000","last_became_active":"2024-09-11T23:34:50.853683+0000","last_became_peered":"2024-09-11T23:34:50.853683+0000","last_unstale":"2024-09-11T23:36:47.591379+0000","last_undegraded":"2024-09-11T23:36:47.591379+0000","last_fullsized":"2024-09-11T23:36:47.591379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:18.725495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153611+0000","last_change":"2024-09-11T23:34:52.874016+0000","last_active":"2024-09-11T23:36:47.153611+0000","last_peered":"2024-09-11T23:36:47.153611+0000","last_clean":"2024-09-11T23:36:47.153611+0000","last_became_active":"2024-09-11T23:34:52.873675+0000","last_became_peered":"2024-09-11T23:34:52.873675+0000","last_unstale":"2024-09-11T23:36:47.153611+0000","last_undegraded":"2024-09-11T23:36:47.153611+0000","last_fullsized":"2024-09-11T23:36:47.153611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:41:28.955548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156082+0000","last_change":"2024-09-11T23:34:50.855391+0000","last_active":"2024-09-11T23:36:47.156082+0000","last_peered":"2024-09-11T23:36:47.156082+0000","last_clean":"2024-09-11T23:36:47.156082+0000","last_became_active":"2024-09-11T23:34:50.855052+0000","last_became_peered":"2024-09-11T23:34:50.855052+0000","last_unstale":"2024-09-11T23:36:47.156082+0000","last_undegraded":"2024-09-11T23:36:47.156082+0000","last_fullsized":"2024-09-11T23:36:47.156082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.972239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057065+0000","last_change":"2024-09-11T23:34:50.848985+0000","last_active":"2024-09-11T23:36:47.057065+0000","last_peered":"2024-09-11T23:36:47.057065+0000","last_clean":"2024-09-11T23:36:47.057065+0000","last_became_active":"2024-09-11T23:34:50.848614+0000","last_became_peered":"2024-09-11T23:34:50.848614+0000","last_unstale":"2024-09-11T23:36:47.057065+0000","last_undegraded":"2024-09-11T23:36:47.057065+0000","last_fullsized":"2024-09-11T23:36:47.057065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:43:04.723136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146018+0000","last_change":"2024-09-11T23:34:52.866933+0000","last_active":"2024-09-11T23:36:47.146018+0000","last_peered":"2024-09-11T23:36:47.146018+0000","last_clean":"2024-09-11T23:36:47.146018+0000","last_became_active":"2024-09-11T23:34:52.866559+0000","last_became_peered":"2024-09-11T23:34:52.866559+0000","last_unstale":"2024-09-11T23:36:47.146018+0000","last_undegraded":"2024-09-11T23:36:47.146018+0000","last_fullsized":"2024-09-11T23:36:47.146018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:27.745118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153745+0000","last_change":"2024-09-11T23:34:50.849787+0000","last_active":"2024-09-11T23:36:47.153745+0000","last_peered":"2024-09-11T23:36:47.153745+0000","last_clean":"2024-09-11T23:36:47.153745+0000","last_became_active":"2024-09-11T23:34:50.849368+0000","last_became_peered":"2024-09-11T23:34:50.849368+0000","last_unstale":"2024-09-11T23:36:47.153745+0000","last_undegraded":"2024-09-11T23:36:47.153745+0000","last_fullsized":"2024-09-11T23:36:47.153745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:26.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142558+0000","last_change":"2024-09-11T23:34:52.870832+0000","last_active":"2024-09-11T23:36:47.142558+0000","last_peered":"2024-09-11T23:36:47.142558+0000","last_clean":"2024-09-11T23:36:47.142558+0000","last_became_active":"2024-09-11T23:34:52.870474+0000","last_became_peered":"2024-09-11T23:34:52.870474+0000","last_unstale":"2024-09-11T23:36:47.142558+0000","last_undegraded":"2024-09-11T23:36:47.142558+0000","last_fullsized":"2024-09-11T23:36:47.142558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:30.196114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":238,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.590875+0000","last_change":"2024-09-11T23:34:50.859074+0000","last_active":"2024-09-11T23:36:47.590875+0000","last_peered":"2024-09-11T23:36:47.590875+0000","last_clean":"2024-09-11T23:36:47.590875+0000","last_became_active":"2024-09-11T23:34:50.858664+0000","last_became_peered":"2024-09-11T23:34:50.858664+0000","last_unstale":"2024-09-11T23:36:47.590875+0000","last_undegraded":"2024-09-11T23:36:47.590875+0000","last_fullsized":"2024-09-11T23:36:47.590875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:04.077813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058577+0000","last_change":"2024-09-11T23:34:52.875901+0000","last_active":"2024-09-11T23:36:47.058577+0000","last_peered":"2024-09-11T23:36:47.058577+0000","last_clean":"2024-09-11T23:36:47.058577+0000","last_became_active":"2024-09-11T23:34:52.875568+0000","last_became_peered":"2024-09-11T23:34:52.875568+0000","last_unstale":"2024-09-11T23:36:47.058577+0000","last_undegraded":"2024-09-11T23:36:47.058577+0000","last_fullsized":"2024-09-11T23:36:47.058577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:54.173548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152813+0000","last_change":"2024-09-11T23:34:50.854957+0000","last_active":"2024-09-11T23:36:47.152813+0000","last_peered":"2024-09-11T23:36:47.152813+0000","last_clean":"2024-09-11T23:36:47.152813+0000","last_became_active":"2024-09-11T23:34:50.854626+0000","last_became_peered":"2024-09-11T23:34:50.854626+0000","last_unstale":"2024-09-11T23:36:47.152813+0000","last_undegraded":"2024-09-11T23:36:47.152813+0000","last_fullsized":"2024-09-11T23:36:47.152813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:53.738966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328149+0000","last_change":"2024-09-11T23:34:52.868850+0000","last_active":"2024-09-11T23:36:47.328149+0000","last_peered":"2024-09-11T23:36:47.328149+0000","last_clean":"2024-09-11T23:36:47.328149+0000","last_became_active":"2024-09-11T23:34:52.868536+0000","last_became_peered":"2024-09-11T23:34:52.868536+0000","last_unstale":"2024-09-11T23:36:47.328149+0000","last_undegraded":"2024-09-11T23:36:47.328149+0000","last_fullsized":"2024-09-11T23:36:47.328149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:11.746821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153829+0000","last_change":"2024-09-11T23:34:50.924819+0000","last_active":"2024-09-11T23:36:47.153829+0000","last_peered":"2024-09-11T23:36:47.153829+0000","last_clean":"2024-09-11T23:36:47.153829+0000","last_became_active":"2024-09-11T23:34:50.924283+0000","last_became_peered":"2024-09-11T23:34:50.924283+0000","last_unstale":"2024-09-11T23:36:47.153829+0000","last_undegraded":"2024-09-11T23:36:47.153829+0000","last_fullsized":"2024-09-11T23:36:47.153829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:41.345233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058417+0000","last_change":"2024-09-11T23:34:52.867113+0000","last_active":"2024-09-11T23:36:47.058417+0000","last_peered":"2024-09-11T23:36:47.058417+0000","last_clean":"2024-09-11T23:36:47.058417+0000","last_became_active":"2024-09-11T23:34:52.866125+0000","last_became_peered":"2024-09-11T23:34:52.866125+0000","last_unstale":"2024-09-11T23:36:47.058417+0000","last_undegraded":"2024-09-11T23:36:47.058417+0000","last_fullsized":"2024-09-11T23:36:47.058417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:18:55.615079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'36","reported_seq":282,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.590874+0000","last_change":"2024-09-11T23:34:50.857272+0000","last_active":"2024-09-11T23:36:47.590874+0000","last_peered":"2024-09-11T23:36:47.590874+0000","last_clean":"2024-09-11T23:36:47.590874+0000","last_became_active":"2024-09-11T23:34:50.855701+0000","last_became_peered":"2024-09-11T23:34:50.855701+0000","last_unstale":"2024-09-11T23:36:47.590874+0000","last_undegraded":"2024-09-11T23:36:47.590874+0000","last_fullsized":"2024-09-11T23:36:47.590874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:49.997617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62867,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":222,"num_read_kb":172,"num_write":38,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145783+0000","last_change":"2024-09-11T23:34:52.876928+0000","last_active":"2024-09-11T23:36:47.145783+0000","last_peered":"2024-09-11T23:36:47.145783+0000","last_clean":"2024-09-11T23:36:47.145783+0000","last_became_active":"2024-09-11T23:34:52.876578+0000","last_became_peered":"2024-09-11T23:34:52.876578+0000","last_unstale":"2024-09-11T23:36:47.145783+0000","last_undegraded":"2024-09-11T23:36:47.145783+0000","last_fullsized":"2024-09-11T23:36:47.145783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:07.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327515+0000","last_change":"2024-09-11T23:35:08.869275+0000","last_active":"2024-09-11T23:36:47.327515+0000","last_peered":"2024-09-11T23:36:47.327515+0000","last_clean":"2024-09-11T23:36:47.327515+0000","last_became_active":"2024-09-11T23:35:08.850112+0000","last_became_peered":"2024-09-11T23:35:08.850112+0000","last_unstale":"2024-09-11T23:36:47.327515+0000","last_undegraded":"2024-09-11T23:36:47.327515+0000","last_fullsized":"2024-09-11T23:36:47.327515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:36.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057105+0000","last_change":"2024-09-11T23:34:52.871970+0000","last_active":"2024-09-11T23:36:47.057105+0000","last_peered":"2024-09-11T23:36:47.057105+0000","last_clean":"2024-09-11T23:36:47.057105+0000","last_became_active":"2024-09-11T23:34:52.871488+0000","last_became_peered":"2024-09-11T23:34:52.871488+0000","last_unstale":"2024-09-11T23:36:47.057105+0000","last_undegraded":"2024-09-11T23:36:47.057105+0000","last_fullsized":"2024-09-11T23:36:47.057105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:08.266431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153627+0000","last_change":"2024-09-11T23:34:50.854023+0000","last_active":"2024-09-11T23:36:47.153627+0000","last_peered":"2024-09-11T23:36:47.153627+0000","last_clean":"2024-09-11T23:36:47.153627+0000","last_became_active":"2024-09-11T23:34:50.853661+0000","last_became_peered":"2024-09-11T23:34:50.853661+0000","last_unstale":"2024-09-11T23:36:47.153627+0000","last_undegraded":"2024-09-11T23:36:47.153627+0000","last_fullsized":"2024-09-11T23:36:47.153627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:25.281976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152646+0000","last_change":"2024-09-11T23:35:08.849648+0000","last_active":"2024-09-11T23:36:47.152646+0000","last_peered":"2024-09-11T23:36:47.152646+0000","last_clean":"2024-09-11T23:36:47.152646+0000","last_became_active":"2024-09-11T23:35:08.849246+0000","last_became_peered":"2024-09-11T23:35:08.849246+0000","last_unstale":"2024-09-11T23:36:47.152646+0000","last_undegraded":"2024-09-11T23:36:47.152646+0000","last_fullsized":"2024-09-11T23:36:47.152646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:17.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'10","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.609150+0000","last_change":"2024-09-11T23:34:50.853855+0000","last_active":"2024-09-11T23:36:47.609150+0000","last_peered":"2024-09-11T23:36:47.609150+0000","last_clean":"2024-09-11T23:36:47.609150+0000","last_became_active":"2024-09-11T23:34:50.853493+0000","last_became_peered":"2024-09-11T23:34:50.853493+0000","last_unstale":"2024-09-11T23:36:47.609150+0000","last_undegraded":"2024-09-11T23:36:47.609150+0000","last_fullsized":"2024-09-11T23:36:47.609150+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:23.262472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":7,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152450+0000","last_change":"2024-09-11T23:34:52.875835+0000","last_active":"2024-09-11T23:36:47.152450+0000","last_peered":"2024-09-11T23:36:47.152450+0000","last_clean":"2024-09-11T23:36:47.152450+0000","last_became_active":"2024-09-11T23:34:52.875525+0000","last_became_peered":"2024-09-11T23:34:52.875525+0000","last_unstale":"2024-09-11T23:36:47.152450+0000","last_undegraded":"2024-09-11T23:36:47.152450+0000","last_fullsized":"2024-09-11T23:36:47.152450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:41.031816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145473+0000","last_change":"2024-09-11T23:34:50.924803+0000","last_active":"2024-09-11T23:36:47.145473+0000","last_peered":"2024-09-11T23:36:47.145473+0000","last_clean":"2024-09-11T23:36:47.145473+0000","last_became_active":"2024-09-11T23:34:50.924217+0000","last_became_peered":"2024-09-11T23:34:50.924217+0000","last_unstale":"2024-09-11T23:36:47.145473+0000","last_undegraded":"2024-09-11T23:36:47.145473+0000","last_fullsized":"2024-09-11T23:36:47.145473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:45.790592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142408+0000","last_change":"2024-09-11T23:35:08.851856+0000","last_active":"2024-09-11T23:36:47.142408+0000","last_peered":"2024-09-11T23:36:47.142408+0000","last_clean":"2024-09-11T23:36:47.142408+0000","last_became_active":"2024-09-11T23:35:08.851490+0000","last_became_peered":"2024-09-11T23:35:08.851490+0000","last_unstale":"2024-09-11T23:36:47.142408+0000","last_undegraded":"2024-09-11T23:36:47.142408+0000","last_fullsized":"2024-09-11T23:36:47.142408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:46.269022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"81'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145816+0000","last_change":"2024-09-11T23:34:50.848385+0000","last_active":"2024-09-11T23:36:47.145816+0000","last_peered":"2024-09-11T23:36:47.145816+0000","last_clean":"2024-09-11T23:36:47.145816+0000","last_became_active":"2024-09-11T23:34:50.847808+0000","last_became_peered":"2024-09-11T23:34:50.847808+0000","last_unstale":"2024-09-11T23:36:47.145816+0000","last_undegraded":"2024-09-11T23:36:47.145816+0000","last_fullsized":"2024-09-11T23:36:47.145816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:38.796622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327734+0000","last_change":"2024-09-11T23:34:52.867341+0000","last_active":"2024-09-11T23:36:47.327734+0000","last_peered":"2024-09-11T23:36:47.327734+0000","last_clean":"2024-09-11T23:36:47.327734+0000","last_became_active":"2024-09-11T23:34:52.866161+0000","last_became_peered":"2024-09-11T23:34:52.866161+0000","last_unstale":"2024-09-11T23:36:47.327734+0000","last_undegraded":"2024-09-11T23:36:47.327734+0000","last_fullsized":"2024-09-11T23:36:47.327734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:27.913705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057317+0000","last_change":"2024-09-11T23:34:50.851466+0000","last_active":"2024-09-11T23:36:47.057317+0000","last_peered":"2024-09-11T23:36:47.057317+0000","last_clean":"2024-09-11T23:36:47.057317+0000","last_became_active":"2024-09-11T23:34:50.851191+0000","last_became_peered":"2024-09-11T23:34:50.851191+0000","last_unstale":"2024-09-11T23:36:47.057317+0000","last_undegraded":"2024-09-11T23:36:47.057317+0000","last_fullsized":"2024-09-11T23:36:47.057317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:00.328658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152878+0000","last_change":"2024-09-11T23:34:52.872290+0000","last_active":"2024-09-11T23:36:47.152878+0000","last_peered":"2024-09-11T23:36:47.152878+0000","last_clean":"2024-09-11T23:36:47.152878+0000","last_became_active":"2024-09-11T23:34:52.871935+0000","last_became_peered":"2024-09-11T23:34:52.871935+0000","last_unstale":"2024-09-11T23:36:47.152878+0000","last_undegraded":"2024-09-11T23:36:47.152878+0000","last_fullsized":"2024-09-11T23:36:47.152878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:08.217548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'41","reported_seq":73,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.248156+0000","last_change":"2024-09-11T23:34:50.857763+0000","last_active":"2024-09-11T23:36:47.248156+0000","last_peered":"2024-09-11T23:36:47.248156+0000","last_clean":"2024-09-11T23:36:47.248156+0000","last_became_active":"2024-09-11T23:34:50.857427+0000","last_became_peered":"2024-09-11T23:34:50.857427+0000","last_unstale":"2024-09-11T23:36:47.248156+0000","last_undegraded":"2024-09-11T23:36:47.248156+0000","last_fullsized":"2024-09-11T23:36:47.248156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:59.703580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152618+0000","last_change":"2024-09-11T23:34:52.872426+0000","last_active":"2024-09-11T23:36:47.152618+0000","last_peered":"2024-09-11T23:36:47.152618+0000","last_clean":"2024-09-11T23:36:47.152618+0000","last_became_active":"2024-09-11T23:34:52.872090+0000","last_became_peered":"2024-09-11T23:34:52.872090+0000","last_unstale":"2024-09-11T23:36:47.152618+0000","last_undegraded":"2024-09-11T23:36:47.152618+0000","last_fullsized":"2024-09-11T23:36:47.152618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:04:00.963963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'7","reported_seq":138,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058454+0000","last_change":"2024-09-11T23:34:50.924792+0000","last_active":"2024-09-11T23:36:47.058454+0000","last_peered":"2024-09-11T23:36:47.058454+0000","last_clean":"2024-09-11T23:36:47.058454+0000","last_became_active":"2024-09-11T23:34:50.924444+0000","last_became_peered":"2024-09-11T23:34:50.924444+0000","last_unstale":"2024-09-11T23:36:47.058454+0000","last_undegraded":"2024-09-11T23:36:47.058454+0000","last_fullsized":"2024-09-11T23:36:47.058454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:12.855314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152787+0000","last_change":"2024-09-11T23:34:52.872079+0000","last_active":"2024-09-11T23:36:47.152787+0000","last_peered":"2024-09-11T23:36:47.152787+0000","last_clean":"2024-09-11T23:36:47.152787+0000","last_became_active":"2024-09-11T23:34:52.871755+0000","last_became_peered":"2024-09-11T23:34:52.871755+0000","last_unstale":"2024-09-11T23:36:47.152787+0000","last_undegraded":"2024-09-11T23:36:47.152787+0000","last_fullsized":"2024-09-11T23:36:47.152787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:05.990832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":347,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152603+0000","last_change":"2024-09-11T23:33:42.256521+0000","last_active":"2024-09-11T23:36:47.152603+0000","last_peered":"2024-09-11T23:36:47.152603+0000","last_clean":"2024-09-11T23:36:47.152603+0000","last_became_active":"2024-09-11T23:33:42.256041+0000","last_became_peered":"2024-09-11T23:33:42.256041+0000","last_unstale":"2024-09-11T23:36:47.152603+0000","last_undegraded":"2024-09-11T23:36:47.152603+0000","last_fullsized":"2024-09-11T23:36:47.152603+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_clean_scrub_stamp":"2024-09-11T23:30:22.061515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:21.391839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292288+0000","last_change":"2024-09-11T23:34:50.854087+0000","last_active":"2024-09-11T23:36:47.292288+0000","last_peered":"2024-09-11T23:36:47.292288+0000","last_clean":"2024-09-11T23:36:47.292288+0000","last_became_active":"2024-09-11T23:34:50.852346+0000","last_became_peered":"2024-09-11T23:34:50.852346+0000","last_unstale":"2024-09-11T23:36:47.292288+0000","last_undegraded":"2024-09-11T23:36:47.292288+0000","last_fullsized":"2024-09-11T23:36:47.292288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.287119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146988+0000","last_change":"2024-09-11T23:34:52.869198+0000","last_active":"2024-09-11T23:36:47.146988+0000","last_peered":"2024-09-11T23:36:47.146988+0000","last_clean":"2024-09-11T23:36:47.146988+0000","last_became_active":"2024-09-11T23:34:52.868621+0000","last_became_peered":"2024-09-11T23:34:52.868621+0000","last_unstale":"2024-09-11T23:36:47.146988+0000","last_undegraded":"2024-09-11T23:36:47.146988+0000","last_fullsized":"2024-09-11T23:36:47.146988+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:17.430654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152604+0000","last_change":"2024-09-11T23:34:50.852752+0000","last_active":"2024-09-11T23:36:47.152604+0000","last_peered":"2024-09-11T23:36:47.152604+0000","last_clean":"2024-09-11T23:36:47.152604+0000","last_became_active":"2024-09-11T23:34:50.851895+0000","last_became_peered":"2024-09-11T23:34:50.851895+0000","last_unstale":"2024-09-11T23:36:47.152604+0000","last_undegraded":"2024-09-11T23:36:47.152604+0000","last_fullsized":"2024-09-11T23:36:47.152604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:46.894432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057125+0000","last_change":"2024-09-11T23:34:52.866191+0000","last_active":"2024-09-11T23:36:47.057125+0000","last_peered":"2024-09-11T23:36:47.057125+0000","last_clean":"2024-09-11T23:36:47.057125+0000","last_became_active":"2024-09-11T23:34:52.865858+0000","last_became_peered":"2024-09-11T23:34:52.865858+0000","last_unstale":"2024-09-11T23:36:47.057125+0000","last_undegraded":"2024-09-11T23:36:47.057125+0000","last_fullsized":"2024-09-11T23:36:47.057125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:21.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:52.615164+0000","last_change":"2024-09-11T23:34:50.924125+0000","last_active":"2024-09-11T23:36:52.615164+0000","last_peered":"2024-09-11T23:36:52.615164+0000","last_clean":"2024-09-11T23:36:52.615164+0000","last_became_active":"2024-09-11T23:34:50.923780+0000","last_became_peered":"2024-09-11T23:34:50.923780+0000","last_unstale":"2024-09-11T23:36:52.615164+0000","last_undegraded":"2024-09-11T23:36:52.615164+0000","last_fullsized":"2024-09-11T23:36:52.615164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:39.774827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153672+0000","last_change":"2024-09-11T23:34:52.868364+0000","last_active":"2024-09-11T23:36:47.153672+0000","last_peered":"2024-09-11T23:36:47.153672+0000","last_clean":"2024-09-11T23:36:47.153672+0000","last_became_active":"2024-09-11T23:34:52.867913+0000","last_became_peered":"2024-09-11T23:34:52.867913+0000","last_unstale":"2024-09-11T23:36:47.153672+0000","last_undegraded":"2024-09-11T23:36:47.153672+0000","last_fullsized":"2024-09-11T23:36:47.153672+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.571825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":136,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.602771+0000","last_change":"2024-09-11T23:34:50.852421+0000","last_active":"2024-09-11T23:36:47.602771+0000","last_peered":"2024-09-11T23:36:47.602771+0000","last_clean":"2024-09-11T23:36:47.602771+0000","last_became_active":"2024-09-11T23:34:50.851898+0000","last_became_peered":"2024-09-11T23:34:50.851898+0000","last_unstale":"2024-09-11T23:36:47.602771+0000","last_undegraded":"2024-09-11T23:36:47.602771+0000","last_fullsized":"2024-09-11T23:36:47.602771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.661276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156359+0000","last_change":"2024-09-11T23:34:52.879995+0000","last_active":"2024-09-11T23:36:47.156359+0000","last_peered":"2024-09-11T23:36:47.156359+0000","last_clean":"2024-09-11T23:36:47.156359+0000","last_became_active":"2024-09-11T23:34:52.879678+0000","last_became_peered":"2024-09-11T23:34:52.879678+0000","last_unstale":"2024-09-11T23:36:47.156359+0000","last_undegraded":"2024-09-11T23:36:47.156359+0000","last_fullsized":"2024-09-11T23:36:47.156359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:47.456674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143075+0000","last_change":"2024-09-11T23:34:50.924957+0000","last_active":"2024-09-11T23:36:47.143075+0000","last_peered":"2024-09-11T23:36:47.143075+0000","last_clean":"2024-09-11T23:36:47.143075+0000","last_became_active":"2024-09-11T23:34:50.924500+0000","last_became_peered":"2024-09-11T23:34:50.924500+0000","last_unstale":"2024-09-11T23:36:47.143075+0000","last_undegraded":"2024-09-11T23:36:47.143075+0000","last_fullsized":"2024-09-11T23:36:47.143075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:39.268842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156159+0000","last_change":"2024-09-11T23:34:52.867337+0000","last_active":"2024-09-11T23:36:47.156159+0000","last_peered":"2024-09-11T23:36:47.156159+0000","last_clean":"2024-09-11T23:36:47.156159+0000","last_became_active":"2024-09-11T23:34:52.866799+0000","last_became_peered":"2024-09-11T23:34:52.866799+0000","last_unstale":"2024-09-11T23:36:47.156159+0000","last_undegraded":"2024-09-11T23:36:47.156159+0000","last_fullsized":"2024-09-11T23:36:47.156159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:38.177135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145920+0000","last_change":"2024-09-11T23:34:50.925156+0000","last_active":"2024-09-11T23:36:47.145920+0000","last_peered":"2024-09-11T23:36:47.145920+0000","last_clean":"2024-09-11T23:36:47.145920+0000","last_became_active":"2024-09-11T23:34:50.924282+0000","last_became_peered":"2024-09-11T23:34:50.924282+0000","last_unstale":"2024-09-11T23:36:47.145920+0000","last_undegraded":"2024-09-11T23:36:47.145920+0000","last_fullsized":"2024-09-11T23:36:47.145920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:47.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057264+0000","last_change":"2024-09-11T23:34:52.864993+0000","last_active":"2024-09-11T23:36:47.057264+0000","last_peered":"2024-09-11T23:36:47.057264+0000","last_clean":"2024-09-11T23:36:47.057264+0000","last_became_active":"2024-09-11T23:34:52.864267+0000","last_became_peered":"2024-09-11T23:34:52.864267+0000","last_unstale":"2024-09-11T23:36:47.057264+0000","last_undegraded":"2024-09-11T23:36:47.057264+0000","last_fullsized":"2024-09-11T23:36:47.057264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:29.912068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'52","reported_seq":99,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.250482+0000","last_change":"2024-09-11T23:34:50.849864+0000","last_active":"2024-09-11T23:36:47.250482+0000","last_peered":"2024-09-11T23:36:47.250482+0000","last_clean":"2024-09-11T23:36:47.250482+0000","last_became_active":"2024-09-11T23:34:50.849577+0000","last_became_peered":"2024-09-11T23:34:50.849577+0000","last_unstale":"2024-09-11T23:36:47.250482+0000","last_undegraded":"2024-09-11T23:36:47.250482+0000","last_fullsized":"2024-09-11T23:36:47.250482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:06.845000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147479+0000","last_change":"2024-09-11T23:34:52.872028+0000","last_active":"2024-09-11T23:36:47.147479+0000","last_peered":"2024-09-11T23:36:47.147479+0000","last_clean":"2024-09-11T23:36:47.147479+0000","last_became_active":"2024-09-11T23:34:52.871723+0000","last_became_peered":"2024-09-11T23:34:52.871723+0000","last_unstale":"2024-09-11T23:36:47.147479+0000","last_undegraded":"2024-09-11T23:36:47.147479+0000","last_fullsized":"2024-09-11T23:36:47.147479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:44.656218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292619+0000","last_change":"2024-09-11T23:34:50.860314+0000","last_active":"2024-09-11T23:36:47.292619+0000","last_peered":"2024-09-11T23:36:47.292619+0000","last_clean":"2024-09-11T23:36:47.292619+0000","last_became_active":"2024-09-11T23:34:50.859953+0000","last_became_peered":"2024-09-11T23:34:50.859953+0000","last_unstale":"2024-09-11T23:36:47.292619+0000","last_undegraded":"2024-09-11T23:36:47.292619+0000","last_fullsized":"2024-09-11T23:36:47.292619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:46.705864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328008+0000","last_change":"2024-09-11T23:34:52.867190+0000","last_active":"2024-09-11T23:36:47.328008+0000","last_peered":"2024-09-11T23:36:47.328008+0000","last_clean":"2024-09-11T23:36:47.328008+0000","last_became_active":"2024-09-11T23:34:52.866829+0000","last_became_peered":"2024-09-11T23:34:52.866829+0000","last_unstale":"2024-09-11T23:36:47.328008+0000","last_undegraded":"2024-09-11T23:36:47.328008+0000","last_fullsized":"2024-09-11T23:36:47.328008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.752006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058275+0000","last_change":"2024-09-11T23:34:50.924566+0000","last_active":"2024-09-11T23:36:47.058275+0000","last_peered":"2024-09-11T23:36:47.058275+0000","last_clean":"2024-09-11T23:36:47.058275+0000","last_became_active":"2024-09-11T23:34:50.924284+0000","last_became_peered":"2024-09-11T23:34:50.924284+0000","last_unstale":"2024-09-11T23:36:47.058275+0000","last_undegraded":"2024-09-11T23:36:47.058275+0000","last_fullsized":"2024-09-11T23:36:47.058275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:54.246154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145487+0000","last_change":"2024-09-11T23:34:52.869302+0000","last_active":"2024-09-11T23:36:47.145487+0000","last_peered":"2024-09-11T23:36:47.145487+0000","last_clean":"2024-09-11T23:36:47.145487+0000","last_became_active":"2024-09-11T23:34:52.868960+0000","last_became_peered":"2024-09-11T23:34:52.868960+0000","last_unstale":"2024-09-11T23:36:47.145487+0000","last_undegraded":"2024-09-11T23:36:47.145487+0000","last_fullsized":"2024-09-11T23:36:47.145487+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.561712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146271+0000","last_change":"2024-09-11T23:34:50.854573+0000","last_active":"2024-09-11T23:36:47.146271+0000","last_peered":"2024-09-11T23:36:47.146271+0000","last_clean":"2024-09-11T23:36:47.146271+0000","last_became_active":"2024-09-11T23:34:50.854165+0000","last_became_peered":"2024-09-11T23:34:50.854165+0000","last_unstale":"2024-09-11T23:36:47.146271+0000","last_undegraded":"2024-09-11T23:36:47.146271+0000","last_fullsized":"2024-09-11T23:36:47.146271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:57:31.589495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327542+0000","last_change":"2024-09-11T23:34:52.875987+0000","last_active":"2024-09-11T23:36:47.327542+0000","last_peered":"2024-09-11T23:36:47.327542+0000","last_clean":"2024-09-11T23:36:47.327542+0000","last_became_active":"2024-09-11T23:34:52.875662+0000","last_became_peered":"2024-09-11T23:34:52.875662+0000","last_unstale":"2024-09-11T23:36:47.327542+0000","last_undegraded":"2024-09-11T23:36:47.327542+0000","last_fullsized":"2024-09-11T23:36:47.327542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:16.514938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058234+0000","last_change":"2024-09-11T23:34:50.850726+0000","last_active":"2024-09-11T23:36:47.058234+0000","last_peered":"2024-09-11T23:36:47.058234+0000","last_clean":"2024-09-11T23:36:47.058234+0000","last_became_active":"2024-09-11T23:34:50.850381+0000","last_became_peered":"2024-09-11T23:34:50.850381+0000","last_unstale":"2024-09-11T23:36:47.058234+0000","last_undegraded":"2024-09-11T23:36:47.058234+0000","last_fullsized":"2024-09-11T23:36:47.058234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:50.431293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146377+0000","last_change":"2024-09-11T23:34:52.870652+0000","last_active":"2024-09-11T23:36:47.146377+0000","last_peered":"2024-09-11T23:36:47.146377+0000","last_clean":"2024-09-11T23:36:47.146377+0000","last_became_active":"2024-09-11T23:34:52.870303+0000","last_became_peered":"2024-09-11T23:34:52.870303+0000","last_unstale":"2024-09-11T23:36:47.146377+0000","last_undegraded":"2024-09-11T23:36:47.146377+0000","last_fullsized":"2024-09-11T23:36:47.146377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:17.670981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327509+0000","last_change":"2024-09-11T23:35:08.854490+0000","last_active":"2024-09-11T23:36:47.327509+0000","last_peered":"2024-09-11T23:36:47.327509+0000","last_clean":"2024-09-11T23:36:47.327509+0000","last_became_active":"2024-09-11T23:35:08.849820+0000","last_became_peered":"2024-09-11T23:35:08.849820+0000","last_unstale":"2024-09-11T23:36:47.327509+0000","last_undegraded":"2024-09-11T23:36:47.327509+0000","last_fullsized":"2024-09-11T23:36:47.327509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:09.509867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152580+0000","last_change":"2024-09-11T23:34:52.869695+0000","last_active":"2024-09-11T23:36:47.152580+0000","last_peered":"2024-09-11T23:36:47.152580+0000","last_clean":"2024-09-11T23:36:47.152580+0000","last_became_active":"2024-09-11T23:34:52.869367+0000","last_became_peered":"2024-09-11T23:34:52.869367+0000","last_unstale":"2024-09-11T23:36:47.152580+0000","last_undegraded":"2024-09-11T23:36:47.152580+0000","last_fullsized":"2024-09-11T23:36:47.152580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:16.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153822+0000","last_change":"2024-09-11T23:34:52.872778+0000","last_active":"2024-09-11T23:36:47.153822+0000","last_peered":"2024-09-11T23:36:47.153822+0000","last_clean":"2024-09-11T23:36:47.153822+0000","last_became_active":"2024-09-11T23:34:52.872451+0000","last_became_peered":"2024-09-11T23:34:52.872451+0000","last_unstale":"2024-09-11T23:36:47.153822+0000","last_undegraded":"2024-09-11T23:36:47.153822+0000","last_fullsized":"2024-09-11T23:36:47.153822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:51.769927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145825+0000","last_change":"2024-09-11T23:34:50.849675+0000","last_active":"2024-09-11T23:36:47.145825+0000","last_peered":"2024-09-11T23:36:47.145825+0000","last_clean":"2024-09-11T23:36:47.145825+0000","last_became_active":"2024-09-11T23:34:50.849228+0000","last_became_peered":"2024-09-11T23:34:50.849228+0000","last_unstale":"2024-09-11T23:36:47.145825+0000","last_undegraded":"2024-09-11T23:36:47.145825+0000","last_fullsized":"2024-09-11T23:36:47.145825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:50.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058399+0000","last_change":"2024-09-11T23:34:52.867739+0000","last_active":"2024-09-11T23:36:47.058399+0000","last_peered":"2024-09-11T23:36:47.058399+0000","last_clean":"2024-09-11T23:36:47.058399+0000","last_became_active":"2024-09-11T23:34:52.866613+0000","last_became_peered":"2024-09-11T23:34:52.866613+0000","last_unstale":"2024-09-11T23:36:47.058399+0000","last_undegraded":"2024-09-11T23:36:47.058399+0000","last_fullsized":"2024-09-11T23:36:47.058399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:49.348386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'13","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145845+0000","last_change":"2024-09-11T23:34:50.851401+0000","last_active":"2024-09-11T23:36:47.145845+0000","last_peered":"2024-09-11T23:36:47.145845+0000","last_clean":"2024-09-11T23:36:47.145845+0000","last_became_active":"2024-09-11T23:34:50.851068+0000","last_became_peered":"2024-09-11T23:34:50.851068+0000","last_unstale":"2024-09-11T23:36:47.145845+0000","last_undegraded":"2024-09-11T23:36:47.145845+0000","last_fullsized":"2024-09-11T23:36:47.145845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:09.673864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328160+0000","last_change":"2024-09-11T23:34:52.865216+0000","last_active":"2024-09-11T23:36:47.328160+0000","last_peered":"2024-09-11T23:36:47.328160+0000","last_clean":"2024-09-11T23:36:47.328160+0000","last_became_active":"2024-09-11T23:34:52.864917+0000","last_became_peered":"2024-09-11T23:34:52.864917+0000","last_unstale":"2024-09-11T23:36:47.328160+0000","last_undegraded":"2024-09-11T23:36:47.328160+0000","last_fullsized":"2024-09-11T23:36:47.328160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:36.469793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147661+0000","last_change":"2024-09-11T23:34:50.924291+0000","last_active":"2024-09-11T23:36:47.147661+0000","last_peered":"2024-09-11T23:36:47.147661+0000","last_clean":"2024-09-11T23:36:47.147661+0000","last_became_active":"2024-09-11T23:34:50.923892+0000","last_became_peered":"2024-09-11T23:34:50.923892+0000","last_unstale":"2024-09-11T23:36:47.147661+0000","last_undegraded":"2024-09-11T23:36:47.147661+0000","last_fullsized":"2024-09-11T23:36:47.147661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:52.266778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147387+0000","last_change":"2024-09-11T23:34:52.875995+0000","last_active":"2024-09-11T23:36:47.147387+0000","last_peered":"2024-09-11T23:36:47.147387+0000","last_clean":"2024-09-11T23:36:47.147387+0000","last_became_active":"2024-09-11T23:34:52.875654+0000","last_became_peered":"2024-09-11T23:34:52.875654+0000","last_unstale":"2024-09-11T23:36:47.147387+0000","last_undegraded":"2024-09-11T23:36:47.147387+0000","last_fullsized":"2024-09-11T23:36:47.147387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:19.218291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292143+0000","last_change":"2024-09-11T23:34:50.853881+0000","last_active":"2024-09-11T23:36:47.292143+0000","last_peered":"2024-09-11T23:36:47.292143+0000","last_clean":"2024-09-11T23:36:47.292143+0000","last_became_active":"2024-09-11T23:34:50.853576+0000","last_became_peered":"2024-09-11T23:34:50.853576+0000","last_unstale":"2024-09-11T23:36:47.292143+0000","last_undegraded":"2024-09-11T23:36:47.292143+0000","last_fullsized":"2024-09-11T23:36:47.292143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:38.550587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597446,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1187,"num_read_kb":708,"num_write":381,"num_write_kb":1464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1792731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612644,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841576,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1748992,"data_stored":1230354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005102,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32484,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712668,"statfs":{"total":95995035648,"available":95961772032,"internally_reserved":0,"allocated":1327104,"data_stored":836376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28320,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716832,"statfs":{"total":95995035648,"available":95966035968,"internally_reserved":0,"allocated":1323008,"data_stored":836754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495672,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1527808,"data_stored":1038472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659196,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":444056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888129,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1167336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018951,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":444366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1462272,"data_stored":967518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444113,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":507736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607637,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":506657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":444750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":197831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:36:53.133 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:36:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:53 smithi178 ceph-mon[35882]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:36:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:53 smithi178 ceph-mon[35882]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-11T23:36:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:53.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:53 smithi138 ceph-mon[35142]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:36:53.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:53 smithi138 ceph-mon[35142]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-11T23:36:53.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.880+0000 7f6610364640 1 -- 172.21.15.133:0/1700574493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6608101df0 msgr2=0x7f66081021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:53.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.880+0000 7f6610364640 1 --2- 172.21.15.133:0/1700574493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6608101df0 0x7f66081021d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009ca0 tx=0x7f65f802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:36:53.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.880+0000 7f6610364640 1 -- 172.21.15.133:0/1700574493 shutdown_connections 2024-09-11T23:36:53.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.880+0000 7f6610364640 1 --2- 172.21.15.133:0/1700574493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6608106f20 0x7f660810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:53.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.880+0000 7f6610364640 1 --2- 172.21.15.133:0/1700574493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66081027a0 0x7f66081067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:53.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.880+0000 7f6610364640 1 --2- 172.21.15.133:0/1700574493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6608101df0 0x7f66081021d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:53.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.880+0000 7f6610364640 1 -- 172.21.15.133:0/1700574493 >> 172.21.15.133:0/1700574493 conn(0x7f66080fd8f0 msgr2=0x7f66080ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:53.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.880+0000 7f6610364640 1 -- 172.21.15.133:0/1700574493 shutdown_connections 2024-09-11T23:36:53.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.880+0000 7f6610364640 1 -- 172.21.15.133:0/1700574493 wait complete. 2024-09-11T23:36:53.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.880+0000 7f6610364640 1 Processor -- start 2024-09-11T23:36:53.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.880+0000 7f6610364640 1 -- start start 2024-09-11T23:36:53.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f6610364640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6608101df0 0x7f66081984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:53.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f6610364640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66081027a0 0x7f6608198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:53.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f6610364640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6608106f20 0x7f660819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:53.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f6610364640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6608111fc0 con 0x7f6608101df0 2024-09-11T23:36:53.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f6610364640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6608111e40 con 0x7f6608106f20 2024-09-11T23:36:53.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f6610364640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6608112140 con 0x7f66081027a0 2024-09-11T23:36:53.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660e100640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6608101df0 0x7f66081984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660e100640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6608101df0 0x7f66081984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33184/0 (socket says 172.21.15.133:33184) 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660e100640 1 -- 172.21.15.133:0/690497448 learned_addr learned my addr 172.21.15.133:0/690497448 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660d8ff640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66081027a0 0x7f6608198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660e901640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6608106f20 0x7f660819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660e100640 1 -- 172.21.15.133:0/690497448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66081027a0 msgr2=0x7f6608198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660e100640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66081027a0 0x7f6608198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660e100640 1 -- 172.21.15.133:0/690497448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6608106f20 msgr2=0x7f660819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660e100640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6608106f20 0x7f660819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660e100640 1 -- 172.21.15.133:0/690497448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660819d490 con 0x7f6608101df0 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660d8ff640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66081027a0 0x7f6608198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f660e100640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6608101df0 0x7f66081984e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f65f802ade0 tx=0x7f65f80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f66077fe640 1 -- 172.21.15.133:0/690497448 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f802fa20 con 0x7f6608101df0 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f66077fe640 1 -- 172.21.15.133:0/690497448 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65f802fbc0 con 0x7f6608101df0 2024-09-11T23:36:53.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f6610364640 1 -- 172.21.15.133:0/690497448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660819d720 con 0x7f6608101df0 2024-09-11T23:36:53.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f66077fe640 1 -- 172.21.15.133:0/690497448 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f80467b0 con 0x7f6608101df0 2024-09-11T23:36:53.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.881+0000 7f6610364640 1 -- 172.21.15.133:0/690497448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66081a5060 con 0x7f6608101df0 2024-09-11T23:36:53.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.882+0000 7f6610364640 1 -- 172.21.15.133:0/690497448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66081021d0 con 0x7f6608101df0 2024-09-11T23:36:53.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.884+0000 7f66077fe640 1 -- 172.21.15.133:0/690497448 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f65f802fd70 con 0x7f6608101df0 2024-09-11T23:36:53.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.884+0000 7f66077fe640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65f00a2e10 0x7f65f00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:53.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.884+0000 7f66077fe640 1 -- 172.21.15.133:0/690497448 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f65f80f0df0 con 0x7f6608101df0 2024-09-11T23:36:53.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.885+0000 7f660d8ff640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65f00a2e10 0x7f65f00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:53.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.885+0000 7f660d8ff640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65f00a2e10 0x7f65f00a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f65fc006fd0 tx=0x7f65fc008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:53.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:53.888+0000 7f66077fe640 1 -- 172.21.15.133:0/690497448 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65f80453e0 con 0x7f6608101df0 2024-09-11T23:36:54.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.091+0000 7f6610364640 1 -- 172.21.15.133:0/690497448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f660810a100 con 0x7f6608101df0 2024-09-11T23:36:54.091 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:54 smithi133 ceph-mon[31430]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:54.091 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:54 smithi133 ceph-mon[31430]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:54.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.091+0000 7f66077fe640 1 -- 172.21.15.133:0/690497448 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f65f80bd9d0 con 0x7f6608101df0 2024-09-11T23:36:54.091 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:54.092 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:36:54.092 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 -- 172.21.15.133:0/690497448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65f00a2e10 msgr2=0x7f65f00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65f00a2e10 0x7f65f00a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f65fc006fd0 tx=0x7f65fc008040 comp rx=0 tx=0).stop 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 -- 172.21.15.133:0/690497448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6608101df0 msgr2=0x7f66081984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6608101df0 0x7f66081984e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f65f802ade0 tx=0x7f65f80319f0 comp rx=0 tx=0).stop 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 -- 172.21.15.133:0/690497448 shutdown_connections 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65f00a2e10 0x7f65f00a52d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6608106f20 0x7f660819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66081027a0 0x7f6608198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 --2- 172.21.15.133:0/690497448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6608101df0 0x7f66081984e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 -- 172.21.15.133:0/690497448 >> 172.21.15.133:0/690497448 conn(0x7f66080fd8f0 msgr2=0x7f66080fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 -- 172.21.15.133:0/690497448 shutdown_connections 2024-09-11T23:36:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:54.094+0000 7f6610364640 1 -- 172.21.15.133:0/690497448 wait complete. 2024-09-11T23:36:54.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:54 smithi178 ceph-mon[35882]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:54.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:54 smithi178 ceph-mon[35882]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:54.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:54 smithi138 ceph-mon[35142]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:54.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:54 smithi138 ceph-mon[35142]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:55.033 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:36:55.283 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:55.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:55 smithi178 ceph-mon[35882]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T23:36:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:55 smithi178 ceph-mon[35882]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:55 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/690497448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:55.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:55 smithi133 ceph-mon[31430]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T23:36:55.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:55 smithi133 ceph-mon[31430]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:55.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:55 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/690497448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:55.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:55 smithi138 ceph-mon[35142]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T23:36:55.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:55 smithi138 ceph-mon[35142]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:36:55.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:55 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/690497448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:36:55.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.746+0000 7f5b3f20f640 1 -- 172.21.15.133:0/1612230781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3810c170 msgr2=0x7f5b3810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:55.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.746+0000 7f5b3f20f640 1 --2- 172.21.15.133:0/1612230781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3810c170 0x7f5b3810e610 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009ca0 tx=0x7f5b3002a950 comp rx=0 tx=0).stop 2024-09-11T23:36:55.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.746+0000 7f5b3f20f640 1 -- 172.21.15.133:0/1612230781 shutdown_connections 2024-09-11T23:36:55.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.746+0000 7f5b3f20f640 1 --2- 172.21.15.133:0/1612230781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3810c170 0x7f5b3810e610 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:55.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.746+0000 7f5b3f20f640 1 --2- 172.21.15.133:0/1612230781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38103660 0x7f5b3810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:55.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.746+0000 7f5b3f20f640 1 --2- 172.21.15.133:0/1612230781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b38102d40 0x7f5b38103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:55.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.746+0000 7f5b3f20f640 1 -- 172.21.15.133:0/1612230781 >> 172.21.15.133:0/1612230781 conn(0x7f5b380fc7a0 msgr2=0x7f5b380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:55.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.746+0000 7f5b3f20f640 1 -- 172.21.15.133:0/1612230781 shutdown_connections 2024-09-11T23:36:55.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.746+0000 7f5b3f20f640 1 -- 172.21.15.133:0/1612230781 wait complete. 2024-09-11T23:36:55.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3f20f640 1 Processor -- start 2024-09-11T23:36:55.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3f20f640 1 -- start start 2024-09-11T23:36:55.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3f20f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38102d40 0x7f5b3819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:55.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3f20f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b38103660 0x7f5b3819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3f20f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3810c170 0x7f5b3819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3f20f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b38111f30 con 0x7f5b3810c170 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3f20f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b38111db0 con 0x7f5b38102d40 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3f20f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381120b0 con 0x7f5b38103660 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3d7ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3810c170 0x7f5b3819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3d7ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3810c170 0x7f5b3819ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33200/0 (socket says 172.21.15.133:33200) 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3d7ac640 1 -- 172.21.15.133:0/3046412813 learned_addr learned my addr 172.21.15.133:0/3046412813 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3cfab640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38102d40 0x7f5b3819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b37fff640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b38103660 0x7f5b3819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3d7ac640 1 -- 172.21.15.133:0/3046412813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b38103660 msgr2=0x7f5b3819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3d7ac640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b38103660 0x7f5b3819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3d7ac640 1 -- 172.21.15.133:0/3046412813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38102d40 msgr2=0x7f5b3819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3d7ac640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38102d40 0x7f5b3819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3d7ac640 1 -- 172.21.15.133:0/3046412813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3819f530 con 0x7f5b3810c170 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.747+0000 7f5b3cfab640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38102d40 0x7f5b3819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.748+0000 7f5b3d7ac640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3810c170 0x7f5b3819ee50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b3002ae70 tx=0x7f5b3002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.748+0000 7f5b35ffb640 1 -- 172.21.15.133:0/3046412813 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30034040 con 0x7f5b3810c170 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.748+0000 7f5b3f20f640 1 -- 172.21.15.133:0/3046412813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3819f7c0 con 0x7f5b3810c170 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.748+0000 7f5b35ffb640 1 -- 172.21.15.133:0/3046412813 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b30030920 con 0x7f5b3810c170 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.748+0000 7f5b35ffb640 1 -- 172.21.15.133:0/3046412813 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30045480 con 0x7f5b3810c170 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.748+0000 7f5b3f20f640 1 -- 172.21.15.133:0/3046412813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b381a7100 con 0x7f5b3810c170 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.749+0000 7f5b3f20f640 1 -- 172.21.15.133:0/3046412813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b38103120 con 0x7f5b3810c170 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.750+0000 7f5b35ffb640 1 -- 172.21.15.133:0/3046412813 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f5b30037020 con 0x7f5b3810c170 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.751+0000 7f5b35ffb640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b280a2e10 0x7f5b280a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.751+0000 7f5b3cfab640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b280a2e10 0x7f5b280a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:55.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.751+0000 7f5b35ffb640 1 -- 172.21.15.133:0/3046412813 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f5b300f0570 con 0x7f5b3810c170 2024-09-11T23:36:55.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.751+0000 7f5b3cfab640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b280a2e10 0x7f5b280a52d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00abe0 tx=0x7f5b2c00a5c0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:55.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.755+0000 7f5b35ffb640 1 -- 172.21.15.133:0/3046412813 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b300bd0c0 con 0x7f5b3810c170 2024-09-11T23:36:55.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.923+0000 7f5b3f20f640 1 -- 172.21.15.133:0/3046412813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b381a7330 con 0x7f5b3810c170 2024-09-11T23:36:55.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.925+0000 7f5b35ffb640 1 -- 172.21.15.133:0/3046412813 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+23060 (secure 0 0 0) 0x7f5b300bc9c0 con 0x7f5b3810c170 2024-09-11T23:36:55.925 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:36:55.925 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":82,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:36:47.051856+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/3324898762":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3596945316":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6801/3324898762":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/1658494809":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3947064111":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/802317131":"2024-09-12T23:36:47.051817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:36:55.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.927+0000 7f5b3f20f640 1 -- 172.21.15.133:0/3046412813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b280a2e10 msgr2=0x7f5b280a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:55.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.927+0000 7f5b3f20f640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b280a2e10 0x7f5b280a52d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00abe0 tx=0x7f5b2c00a5c0 comp rx=0 tx=0).stop 2024-09-11T23:36:55.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.927+0000 7f5b3f20f640 1 -- 172.21.15.133:0/3046412813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3810c170 msgr2=0x7f5b3819ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:55.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.927+0000 7f5b3f20f640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3810c170 0x7f5b3819ee50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b3002ae70 tx=0x7f5b3002f5b0 comp rx=0 tx=0).stop 2024-09-11T23:36:55.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.927+0000 7f5b3f20f640 1 -- 172.21.15.133:0/3046412813 shutdown_connections 2024-09-11T23:36:55.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.927+0000 7f5b3f20f640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b280a2e10 0x7f5b280a52d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:55.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.927+0000 7f5b3f20f640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3810c170 0x7f5b3819ee50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:55.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.927+0000 7f5b3f20f640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b38103660 0x7f5b3819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:55.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.927+0000 7f5b3f20f640 1 --2- 172.21.15.133:0/3046412813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b38102d40 0x7f5b3819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:55.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.927+0000 7f5b3f20f640 1 -- 172.21.15.133:0/3046412813 >> 172.21.15.133:0/3046412813 conn(0x7f5b380fc7a0 msgr2=0x7f5b3810d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:55.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.927+0000 7f5b3f20f640 1 -- 172.21.15.133:0/3046412813 shutdown_connections 2024-09-11T23:36:55.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:55.928+0000 7f5b3f20f640 1 -- 172.21.15.133:0/3046412813 wait complete. 2024-09-11T23:36:56.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:56 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3046412813' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:56 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3046412813' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:56.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:56 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3046412813' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:36:56.750 DEBUG:tasks.fwd_scrub:Yielding 2024-09-11T23:36:56.750 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T23:36:56.760 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:36:56.760 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T23:36:56.760 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-11T23:36:56.760 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-11T23:36:56.761 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:36:56.761 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:36:56.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.866+0000 7f1027523640 1 -- 172.21.15.133:0/3673091988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10201009b0 msgr2=0x7f1020100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:56.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.866+0000 7f1027523640 1 --2- 172.21.15.133:0/3673091988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10201009b0 0x7f1020100e10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1010009930 tx=0x7f101002a420 comp rx=0 tx=0).stop 2024-09-11T23:36:56.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.866+0000 7f1027523640 1 -- 172.21.15.133:0/3673091988 shutdown_connections 2024-09-11T23:36:56.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.866+0000 7f1027523640 1 --2- 172.21.15.133:0/3673091988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1020101350 0x7f102010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:56.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.866+0000 7f1027523640 1 --2- 172.21.15.133:0/3673091988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10201009b0 0x7f1020100e10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:56.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.866+0000 7f1027523640 1 --2- 172.21.15.133:0/3673091988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10201069c0 0x7f1020106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:56.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.866+0000 7f1027523640 1 -- 172.21.15.133:0/3673091988 >> 172.21.15.133:0/3673091988 conn(0x7f1020077990 msgr2=0x7f10200fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:56.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.866+0000 7f1027523640 1 -- 172.21.15.133:0/3673091988 shutdown_connections 2024-09-11T23:36:56.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.866+0000 7f1027523640 1 -- 172.21.15.133:0/3673091988 wait complete. 2024-09-11T23:36:56.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f1027523640 1 Processor -- start 2024-09-11T23:36:56.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f1027523640 1 -- start start 2024-09-11T23:36:56.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f1027523640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10201009b0 0x7f1020198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:56.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f1027523640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1020101350 0x7f1020198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:56.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f1027523640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10201069c0 0x7f102019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:56.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f1027523640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1020110e00 con 0x7f10201009b0 2024-09-11T23:36:56.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f1027523640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1020110c80 con 0x7f10201069c0 2024-09-11T23:36:56.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f1027523640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1020110f80 con 0x7f1020101350 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f10252bf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10201009b0 0x7f1020198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f10252bf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10201009b0 0x7f1020198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33230/0 (socket says 172.21.15.133:33230) 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f10252bf640 1 -- 172.21.15.133:0/3673604259 learned_addr learned my addr 172.21.15.133:0/3673604259 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f1025ac0640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10201069c0 0x7f102019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.867+0000 7f1024abe640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1020101350 0x7f1020198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.868+0000 7f10252bf640 1 -- 172.21.15.133:0/3673604259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1020101350 msgr2=0x7f1020198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.868+0000 7f10252bf640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1020101350 0x7f1020198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.868+0000 7f10252bf640 1 -- 172.21.15.133:0/3673604259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10201069c0 msgr2=0x7f102019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.868+0000 7f10252bf640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10201069c0 0x7f102019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.868+0000 7f10252bf640 1 -- 172.21.15.133:0/3673604259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102019d320 con 0x7f10201009b0 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.868+0000 7f10252bf640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10201009b0 0x7f1020198330 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f101c00d950 tx=0x7f101c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.868+0000 7f10167fc640 1 -- 172.21.15.133:0/3673604259 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c017070 con 0x7f10201009b0 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.868+0000 7f10167fc640 1 -- 172.21.15.133:0/3673604259 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f101c009c30 con 0x7f10201009b0 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.868+0000 7f1027523640 1 -- 172.21.15.133:0/3673604259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102019d610 con 0x7f10201009b0 2024-09-11T23:36:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.868+0000 7f10167fc640 1 -- 172.21.15.133:0/3673604259 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c00f6b0 con 0x7f10201009b0 2024-09-11T23:36:56.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.868+0000 7f1027523640 1 -- 172.21.15.133:0/3673604259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10201a4ea0 con 0x7f10201009b0 2024-09-11T23:36:56.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.869+0000 7f1027523640 1 -- 172.21.15.133:0/3673604259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1020100e10 con 0x7f10201009b0 2024-09-11T23:36:56.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.871+0000 7f10167fc640 1 -- 172.21.15.133:0/3673604259 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f101c0109d0 con 0x7f10201009b0 2024-09-11T23:36:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.871+0000 7f10167fc640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f100c0a2ec0 0x7f100c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.872+0000 7f1024abe640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f100c0a2ec0 0x7f100c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.872+0000 7f10167fc640 1 -- 172.21.15.133:0/3673604259 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f101c0cf2d0 con 0x7f10201009b0 2024-09-11T23:36:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.872+0000 7f1024abe640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f100c0a2ec0 0x7f100c0a5380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1010000c00 tx=0x7f1010005a20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:56.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:56.875+0000 7f10167fc640 1 -- 172.21.15.133:0/3673604259 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f101c009de0 con 0x7f10201009b0 2024-09-11T23:36:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:57.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.058+0000 7f1027523640 1 -- 172.21.15.133:0/3673604259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f10201a5140 con 0x7f10201009b0 2024-09-11T23:36:57.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.063+0000 7f10167fc640 1 -- 172.21.15.133:0/3673604259 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v27)=0 v27) ==== 129+0+0 (secure 0 0 0) 0x7f101c09bd10 con 0x7f10201009b0 2024-09-11T23:36:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 -- 172.21.15.133:0/3673604259 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f100c0a2ec0 msgr2=0x7f100c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f100c0a2ec0 0x7f100c0a5380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1010000c00 tx=0x7f1010005a20 comp rx=0 tx=0).stop 2024-09-11T23:36:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 -- 172.21.15.133:0/3673604259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10201009b0 msgr2=0x7f1020198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10201009b0 0x7f1020198330 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f101c00d950 tx=0x7f101c00de10 comp rx=0 tx=0).stop 2024-09-11T23:36:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 -- 172.21.15.133:0/3673604259 shutdown_connections 2024-09-11T23:36:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f100c0a2ec0 0x7f100c0a5380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10201069c0 0x7f102019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1020101350 0x7f1020198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 --2- 172.21.15.133:0/3673604259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10201009b0 0x7f1020198330 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 -- 172.21.15.133:0/3673604259 >> 172.21.15.133:0/3673604259 conn(0x7f1020077990 msgr2=0x7f1020069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 -- 172.21.15.133:0/3673604259 shutdown_connections 2024-09-11T23:36:57.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.066+0000 7f1027523640 1 -- 172.21.15.133:0/3673604259 wait complete. 2024-09-11T23:36:57.077 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-11T23:36:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.183+0000 7fea2575b640 1 -- 172.21.15.133:0/2424004155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea201009d0 msgr2=0x7fea20100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.183+0000 7fea2575b640 1 --2- 172.21.15.133:0/2424004155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea201009d0 0x7fea20100e30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fea08009930 tx=0x7fea0802a420 comp rx=0 tx=0).stop 2024-09-11T23:36:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.183+0000 7fea2575b640 1 -- 172.21.15.133:0/2424004155 shutdown_connections 2024-09-11T23:36:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.183+0000 7fea2575b640 1 --2- 172.21.15.133:0/2424004155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea20101370 0x7fea2010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.183+0000 7fea2575b640 1 --2- 172.21.15.133:0/2424004155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea201009d0 0x7fea20100e30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.183+0000 7fea2575b640 1 --2- 172.21.15.133:0/2424004155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea201069e0 0x7fea20106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.183+0000 7fea2575b640 1 -- 172.21.15.133:0/2424004155 >> 172.21.15.133:0/2424004155 conn(0x7fea200fc6b0 msgr2=0x7fea200fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.183+0000 7fea2575b640 1 -- 172.21.15.133:0/2424004155 shutdown_connections 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.183+0000 7fea2575b640 1 -- 172.21.15.133:0/2424004155 wait complete. 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea2575b640 1 Processor -- start 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea2575b640 1 -- start start 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea2575b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea201009d0 0x7fea20199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea2575b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea20101370 0x7fea20199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea2575b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea201069e0 0x7fea201a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea2575b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea20110e20 con 0x7fea20101370 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea2575b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fea20110ca0 con 0x7fea201069e0 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea2575b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea20110fa0 con 0x7fea201009d0 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea167fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea20101370 0x7fea20199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea167fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea20101370 0x7fea20199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33244/0 (socket says 172.21.15.133:33244) 2024-09-11T23:36:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea167fc640 1 -- 172.21.15.133:0/3116404880 learned_addr learned my addr 172.21.15.133:0/3116404880 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.184+0000 7fea1effd640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea201009d0 0x7fea20199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea1f7fe640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea201069e0 0x7fea201a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea167fc640 1 -- 172.21.15.133:0/3116404880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea201009d0 msgr2=0x7fea20199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea167fc640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea201009d0 0x7fea20199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea167fc640 1 -- 172.21.15.133:0/3116404880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea201069e0 msgr2=0x7fea201a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea167fc640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea201069e0 0x7fea201a41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea167fc640 1 -- 172.21.15.133:0/3116404880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea201a4710 con 0x7fea20101370 2024-09-11T23:36:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea1f7fe640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea201069e0 0x7fea201a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea167fc640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea20101370 0x7fea20199690 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fea08009230 tx=0x7fea08004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea1cff9640 1 -- 172.21.15.133:0/3116404880 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0803f030 con 0x7fea20101370 2024-09-11T23:36:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea1cff9640 1 -- 172.21.15.133:0/3116404880 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea08003ad0 con 0x7fea20101370 2024-09-11T23:36:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea2575b640 1 -- 172.21.15.133:0/3116404880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea201a48e0 con 0x7fea20101370 2024-09-11T23:36:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea1cff9640 1 -- 172.21.15.133:0/3116404880 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08035070 con 0x7fea20101370 2024-09-11T23:36:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.185+0000 7fea2575b640 1 -- 172.21.15.133:0/3116404880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea201a4d40 con 0x7fea20101370 2024-09-11T23:36:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.186+0000 7fea2575b640 1 -- 172.21.15.133:0/3116404880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea20100e30 con 0x7fea20101370 2024-09-11T23:36:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.187+0000 7fea1cff9640 1 -- 172.21.15.133:0/3116404880 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fea08042030 con 0x7fea20101370 2024-09-11T23:36:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.188+0000 7fea1cff9640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea0c0a2ec0 0x7fea0c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.188+0000 7fea1effd640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea0c0a2ec0 0x7fea0c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.188+0000 7fea1cff9640 1 -- 172.21.15.133:0/3116404880 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fea080efc00 con 0x7fea20101370 2024-09-11T23:36:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.188+0000 7fea1effd640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea0c0a2ec0 0x7fea0c0a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fea10009b50 tx=0x7fea10005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.192+0000 7fea1cff9640 1 -- 172.21.15.133:0/3116404880 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea080f6080 con 0x7fea20101370 2024-09-11T23:36:57.288 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:57 smithi133 ceph-mon[31430]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.7 KiB/s wr, 26 op/s 2024-09-11T23:36:57.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:57 smithi178 ceph-mon[35882]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.7 KiB/s wr, 26 op/s 2024-09-11T23:36:57.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.372+0000 7fea2575b640 1 -- 172.21.15.133:0/3116404880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fea201a4fa0 con 0x7fea20101370 2024-09-11T23:36:57.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.376+0000 7fea1cff9640 1 -- 172.21.15.133:0/3116404880 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fea080bc710 con 0x7fea20101370 2024-09-11T23:36:57.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 -- 172.21.15.133:0/3116404880 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea0c0a2ec0 msgr2=0x7fea0c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea0c0a2ec0 0x7fea0c0a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fea10009b50 tx=0x7fea10005f90 comp rx=0 tx=0).stop 2024-09-11T23:36:57.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 -- 172.21.15.133:0/3116404880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea20101370 msgr2=0x7fea20199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea20101370 0x7fea20199690 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fea08009230 tx=0x7fea08004270 comp rx=0 tx=0).stop 2024-09-11T23:36:57.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 -- 172.21.15.133:0/3116404880 shutdown_connections 2024-09-11T23:36:57.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea0c0a2ec0 0x7fea0c0a5380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea201069e0 0x7fea201a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea20101370 0x7fea20199690 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 --2- 172.21.15.133:0/3116404880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea201009d0 0x7fea20199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 -- 172.21.15.133:0/3116404880 >> 172.21.15.133:0/3116404880 conn(0x7fea200fc6b0 msgr2=0x7fea200fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 -- 172.21.15.133:0/3116404880 shutdown_connections 2024-09-11T23:36:57.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.379+0000 7fea2575b640 1 -- 172.21.15.133:0/3116404880 wait complete. 2024-09-11T23:36:57.390 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-11T23:36:57.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:57 smithi138 ceph-mon[35142]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.7 KiB/s wr, 26 op/s 2024-09-11T23:36:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f587cb91640 1 -- 172.21.15.133:0/1848027959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58781069e0 msgr2=0x7f5878106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f587cb91640 1 --2- 172.21.15.133:0/1848027959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58781069e0 0x7f5878106dc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5868009890 tx=0x7f586802a380 comp rx=0 tx=0).stop 2024-09-11T23:36:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f587cb91640 1 -- 172.21.15.133:0/1848027959 shutdown_connections 2024-09-11T23:36:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f587cb91640 1 --2- 172.21.15.133:0/1848027959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5878101370 0x7f587810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f587cb91640 1 --2- 172.21.15.133:0/1848027959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58781009d0 0x7f5878100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f587cb91640 1 --2- 172.21.15.133:0/1848027959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58781069e0 0x7f5878106dc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f587cb91640 1 -- 172.21.15.133:0/1848027959 >> 172.21.15.133:0/1848027959 conn(0x7f58780fc6b0 msgr2=0x7f58780fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f587cb91640 1 -- 172.21.15.133:0/1848027959 shutdown_connections 2024-09-11T23:36:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f587cb91640 1 -- 172.21.15.133:0/1848027959 wait complete. 2024-09-11T23:36:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.498+0000 7f587cb91640 1 Processor -- start 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.498+0000 7f587cb91640 1 -- start start 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f587cb91640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58781009d0 0x7f58781991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f587cb91640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5878101370 0x7f5878199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f587cb91640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58781069e0 0x7f58781a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f587cb91640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5878110e20 con 0x7f5878101370 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f587cb91640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5878110ca0 con 0x7f58781069e0 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f587cb91640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5878110fa0 con 0x7f58781009d0 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f587659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58781009d0 0x7f58781991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f5875d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5878101370 0x7f5878199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f5875d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5878101370 0x7f5878199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33308/0 (socket says 172.21.15.133:33308) 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f5875d9b640 1 -- 172.21.15.133:0/2253155216 learned_addr learned my addr 172.21.15.133:0/2253155216 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f5876d9d640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58781069e0 0x7f58781a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f5875d9b640 1 -- 172.21.15.133:0/2253155216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58781009d0 msgr2=0x7f58781991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f5875d9b640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58781009d0 0x7f58781991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f5875d9b640 1 -- 172.21.15.133:0/2253155216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58781069e0 msgr2=0x7f58781a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f5875d9b640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58781069e0 0x7f58781a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f5875d9b640 1 -- 172.21.15.133:0/2253155216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58781a4790 con 0x7f5878101370 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f5876d9d640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58781069e0 0x7f58781a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.500+0000 7f5875d9b640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5878101370 0x7f5878199710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f586c00e9c0 tx=0x7f586c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.500+0000 7f58677fe640 1 -- 172.21.15.133:0/2253155216 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c00cde0 con 0x7f5878101370 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.500+0000 7f58677fe640 1 -- 172.21.15.133:0/2253155216 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f586c0035f0 con 0x7f5878101370 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.500+0000 7f587cb91640 1 -- 172.21.15.133:0/2253155216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5878196fc0 con 0x7f5878101370 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.500+0000 7f58677fe640 1 -- 172.21.15.133:0/2253155216 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c010690 con 0x7f5878101370 2024-09-11T23:36:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.500+0000 7f587cb91640 1 -- 172.21.15.133:0/2253155216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5878197500 con 0x7f5878101370 2024-09-11T23:36:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.490+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2123242982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c4108300 msgr2=0x7f55c410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.490+0000 7f55cb9e1640 1 --2- 172.21.15.133:0/2123242982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c4108300 0x7f55c410ec70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009930 tx=0x7f55b402a420 comp rx=0 tx=0).stop 2024-09-11T23:36:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.490+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2123242982 shutdown_connections 2024-09-11T23:36:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.490+0000 7f55cb9e1640 1 --2- 172.21.15.133:0/2123242982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c4108300 0x7f55c410ec70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.490+0000 7f55cb9e1640 1 --2- 172.21.15.133:0/2123242982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c4103780 0x7f55c4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.490+0000 7f55cb9e1640 1 --2- 172.21.15.133:0/2123242982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c4102dd0 0x7f55c41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.490+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2123242982 >> 172.21.15.133:0/2123242982 conn(0x7f55c40fc7d0 msgr2=0x7f55c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.490+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2123242982 shutdown_connections 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.490+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2123242982 wait complete. 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.490+0000 7f55cb9e1640 1 Processor -- start 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.490+0000 7f55cb9e1640 1 -- start start 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55cb9e1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c4102dd0 0x7f55c4100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55cb9e1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c4103780 0x7f55c4100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55cb9e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c4108300 0x7f55c41014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55cb9e1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41120d0 con 0x7f55c4103780 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55cb9e1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4111f50 con 0x7f55c4102dd0 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55cb9e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4112250 con 0x7f55c4108300 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c8f7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c4103780 0x7f55c4100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c8f7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c4103780 0x7f55c4100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33270/0 (socket says 172.21.15.133:33270) 2024-09-11T23:36:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c8f7c640 1 -- 172.21.15.133:0/2626836101 learned_addr learned my addr 172.21.15.133:0/2626836101 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c9f7e640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c4108300 0x7f55c41014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c977d640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c4102dd0 0x7f55c4100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c8f7c640 1 -- 172.21.15.133:0/2626836101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c4108300 msgr2=0x7f55c41014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c4108300 0x7f55c41014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c8f7c640 1 -- 172.21.15.133:0/2626836101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c4102dd0 msgr2=0x7f55c4100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c4102dd0 0x7f55c4100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c8f7c640 1 -- 172.21.15.133:0/2626836101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c41a6a50 con 0x7f55c4103780 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c4103780 0x7f55c4100f70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f55b800d950 tx=0x7f55b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c27fc640 1 -- 172.21.15.133:0/2626836101 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8017070 con 0x7f55c4103780 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c27fc640 1 -- 172.21.15.133:0/2626836101 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b8009c30 con 0x7f55c4103780 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2626836101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c41a6d40 con 0x7f55c4103780 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55c27fc640 1 -- 172.21.15.133:0/2626836101 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80047a0 con 0x7f55c4103780 2024-09-11T23:36:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.491+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2626836101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41a7220 con 0x7f55c4103780 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.494+0000 7f55c27fc640 1 -- 172.21.15.133:0/2626836101 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f55b8004020 con 0x7f55c4103780 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55cddba740 1 Processor -- start 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55cddba740 1 -- start start 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55cddba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f80496b60 0x559f80496f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55cddba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804a1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55c27fc640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55ac0a2e10 0x7f55ac0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55cddba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f804a1d40 0x559f804a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55cddba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559f803fc280 con 0x559f80497480 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55cddba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f803a3b10 con 0x559f804a1d40 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55cddba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559f803f8240 con 0x559f80496b60 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55c8f7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55c977d640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55ac0a2e10 0x7f55ac0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55c9f7e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f804a1d40 0x559f804a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55c9f7e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f804a1d40 0x559f804a4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:36196/0 (socket says 172.21.15.133:36196) 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55c9f7e640 1 -- 172.21.15.133:0/3548185074 learned_addr learned my addr 172.21.15.133:0/3548185074 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55c9f7e640 1 --2- 172.21.15.133:0/3548185074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f80496b60 0x559f80496f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.495+0000 7f55c27fc640 1 -- 172.21.15.133:0/2626836101 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f55b80cf1c0 con 0x7f55c4103780 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.496+0000 7f55c8f7c640 1 -- 172.21.15.133:0/3548185074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f80496b60 msgr2=0x559f80496f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.496+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/3548185074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f80496b60 0x559f80496f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.496+0000 7f55c8f7c640 1 -- 172.21.15.133:0/3548185074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f804a1d40 msgr2=0x559f804a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.496+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/3548185074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f804a1d40 0x559f804a4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.496+0000 7f55c8f7c640 1 -- 172.21.15.133:0/3548185074 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f803b01f0 con 0x559f80497480 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.496+0000 7f55c9f7e640 1 --2- 172.21.15.133:0/3548185074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f804a1d40 0x559f804a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.496+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/3548185074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804a1800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f55b80cc800 tx=0x7f55b806b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f40a90f1f353549 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.496+0000 7f55c977d640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55ac0a2e10 0x7f55ac0a52d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005fd0 tx=0x7f55bc009500 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.496+0000 7f55ab7fe640 1 -- 172.21.15.133:0/3548185074 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80cf020 con 0x559f80497480 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.496+0000 7f55ab7fe640 1 -- 172.21.15.133:0/3548185074 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b809bc00 con 0x559f80497480 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.496+0000 7f55ab7fe640 1 -- 172.21.15.133:0/3548185074 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80998a0 con 0x559f80497480 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f55cddba740 1 -- 172.21.15.133:0/3548185074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 msgr2=0x559f804a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f55cddba740 1 --2- 172.21.15.133:0/3548185074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804a1800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f55b80cc800 tx=0x7f55b806b360 comp rx=0 tx=0).stop 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f55cddba740 1 -- 172.21.15.133:0/3548185074 shutdown_connections 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f55cddba740 1 --2- 172.21.15.133:0/3548185074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f804a1d40 0x559f804a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f55cddba740 1 --2- 172.21.15.133:0/3548185074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804a1800 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f55cddba740 1 --2- 172.21.15.133:0/3548185074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f80496b60 0x559f80496f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f55cddba740 1 -- 172.21.15.133:0/3548185074 >> 172.21.15.133:0/3548185074 conn(0x559f804930f0 msgr2=0x559f804934d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f55cddba740 1 -- 172.21.15.133:0/3548185074 shutdown_connections 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.497+0000 7f55cddba740 1 -- 172.21.15.133:0/3548185074 wait complete. 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.498+0000 7f55cddba740 1 -- start start 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.498+0000 7f55cddba740 1 Processor -- start 2024-09-11T23:36:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.498+0000 7f55cddba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80496b60 0x559f8049ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.498+0000 7f55cddba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f80497480 0x559f8049f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.498+0000 7f55cddba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f804a1d40 0x559f8049f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.498+0000 7f55cddba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559f804a48b0 con 0x559f80496b60 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.498+0000 7f55c977d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80496b60 0x559f8049ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55cddba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f803b6f10 con 0x559f80497480 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55cddba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559f804a4a30 con 0x559f804a1d40 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55c977d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80496b60 0x559f8049ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33292/0 (socket says 172.21.15.133:33292) 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55c977d640 1 -- 172.21.15.133:0/3679279408 learned_addr learned my addr 172.21.15.133:0/3679279408 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55cddba740 10 client.0 _ll_register_callbacks cb 0x559f8037fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55c9f7e640 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f804a1d40 0x559f8049f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f80497480 0x559f8049f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55c977d640 1 -- 172.21.15.133:0/3679279408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f804a1d40 msgr2=0x559f8049f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55c977d640 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f804a1d40 0x559f8049f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55c977d640 1 -- 172.21.15.133:0/3679279408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f80497480 msgr2=0x559f8049f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55c977d640 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f80497480 0x559f8049f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55c977d640 1 -- 172.21.15.133:0/3679279408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8016d6a0 con 0x559f80496b60 2024-09-11T23:36:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55c977d640 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80496b60 0x559f8049ec90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00ebb0 tx=0x7f55bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ea69a4a1d20b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.499+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3679279408 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc01db90 con 0x559f80496b60 2024-09-11T23:36:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.500+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3679279408 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f55bc01dd30 con 0x559f80496b60 2024-09-11T23:36:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.500+0000 7f55cddba740 1 -- 172.21.15.133:0/3679279408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b809bc00 con 0x559f80496b60 2024-09-11T23:36:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.500+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3679279408 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55bc003970 con 0x559f80496b60 2024-09-11T23:36:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.500+0000 7f55cddba740 10 client.15408 fetch_fsmap learned FSMap version 28 2024-09-11T23:36:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.500+0000 7f55cddba740 1 -- 172.21.15.133:0/3679279408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f804b0590 con 0x559f80496b60 2024-09-11T23:36:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.501+0000 7f587cb91640 1 -- 172.21.15.133:0/2253155216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5878100e30 con 0x7f5878101370 2024-09-11T23:36:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.502+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3679279408 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b809bc00 con 0x559f80496b60 2024-09-11T23:36:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.502+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3679279408 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f55bc030470 con 0x559f80496b60 2024-09-11T23:36:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.502+0000 7f55cddba740 10 client.15408 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:36:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.502+0000 7f55cddba740 10 client.15408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:36:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.502+0000 7f55cddba740 20 client.15408 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.502+0000 7f55cddba740 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559f804b19a0 0x559f804b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.502+0000 7f55cddba740 4 client.15408 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.502+0000 7f55cddba740 1 -- 172.21.15.133:0/3679279408 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f804097f0 con 0x559f804b19a0 2024-09-11T23:36:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.502+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559f804b19a0 0x559f804b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.503+0000 7f55a8ff9640 10 client.15408 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.503+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559f804b19a0 0x559f804b3e40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e2bfb41da52d1d server_cookie=d8d6ceac99dcad6 in_seq=0 out_seq=0 2024-09-11T23:36:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.503+0000 7f58677fe640 1 -- 172.21.15.133:0/2253155216 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f586c0037a0 con 0x7f5878101370 2024-09-11T23:36:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.503+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3679279408 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f804097f0 con 0x559f804b19a0 2024-09-11T23:36:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.503+0000 7f55a8ff9640 10 client.15408 handle_command_reply: tid=0 2024-09-11T23:36:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.503+0000 7f55cddba740 1 client.15408 shutdown 2024-09-11T23:36:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f58677fe640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58600a2f40 0x7f58600a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f55cddba740 1 -- 172.21.15.133:0/3679279408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80496b60 msgr2=0x559f8049ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f55cddba740 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80496b60 0x559f8049ec90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00ebb0 tx=0x7f55bc004ab0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f587659c640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58600a2f40 0x7f58600a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f58677fe640 1 -- 172.21.15.133:0/2253155216 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f586c0cfab0 con 0x7f5878101370 2024-09-11T23:36:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f55cddba740 1 -- 172.21.15.133:0/3679279408 shutdown_connections 2024-09-11T23:36:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f55cddba740 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559f804b19a0 0x559f804b3e40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f55a8ff9640 0 client.15408 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f55cddba740 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f804a1d40 0x559f8049f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f55cddba740 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f80497480 0x559f8049f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f55cddba740 1 --2- 172.21.15.133:0/3679279408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80496b60 0x559f8049ec90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f55cddba740 1 -- 172.21.15.133:0/3679279408 >> 172.21.15.133:0/3679279408 conn(0x559f804930f0 msgr2=0x559f80498e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f55cddba740 1 -- 172.21.15.133:0/3679279408 shutdown_connections 2024-09-11T23:36:57.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f587659c640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58600a2f40 0x7f58600a5400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f58680097b0 tx=0x7f58680058d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.504+0000 7f55cddba740 1 -- 172.21.15.133:0/3679279408 wait complete. 2024-09-11T23:36:57.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.505+0000 7f55cddba740 20 client.15408 trim_cache size 0 max 16384 2024-09-11T23:36:57.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.507+0000 7f58677fe640 1 -- 172.21.15.133:0/2253155216 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f586c09c580 con 0x7f5878101370 2024-09-11T23:36:57.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.523+0000 7f55cddba740 1 Processor -- start 2024-09-11T23:36:57.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.523+0000 7f55cddba740 1 -- start start 2024-09-11T23:36:57.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55cddba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f80496b60 0x559f804995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55cddba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55cddba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f804a1d40 0x559f804b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55cddba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41ab630 con 0x559f80497480 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55cddba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41ad170 con 0x559f80496b60 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55cddba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41abd00 con 0x559f804a1d40 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55c8f7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55c8f7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33318/0 (socket says 172.21.15.133:33318) 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55c8f7c640 1 -- 172.21.15.133:0/988115750 learned_addr learned my addr 172.21.15.133:0/988115750 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55c9f7e640 1 --2- 172.21.15.133:0/988115750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f804a1d40 0x559f804b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55c977d640 1 --2- 172.21.15.133:0/988115750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f80496b60 0x559f804995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55c8f7c640 1 -- 172.21.15.133:0/988115750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f804a1d40 msgr2=0x559f804b9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/988115750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f804a1d40 0x559f804b9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55c8f7c640 1 -- 172.21.15.133:0/988115750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f80496b60 msgr2=0x559f804995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/988115750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f80496b60 0x559f804995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55c8f7c640 1 -- 172.21.15.133:0/988115750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f804a1560 con 0x559f80497480 2024-09-11T23:36:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.524+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/988115750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804b9760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f55b806d490 tx=0x7f55b806d4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea475f923e8d18a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.525+0000 7f55aaffd640 1 -- 172.21.15.133:0/988115750 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80cf020 con 0x559f80497480 2024-09-11T23:36:57.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.525+0000 7f55aaffd640 1 -- 172.21.15.133:0/988115750 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b8009de0 con 0x559f80497480 2024-09-11T23:36:57.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.525+0000 7f55aaffd640 1 -- 172.21.15.133:0/988115750 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b806a070 con 0x559f80497480 2024-09-11T23:36:57.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.525+0000 7f55cddba740 1 -- 172.21.15.133:0/988115750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 msgr2=0x559f804b9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.525+0000 7f55cddba740 1 --2- 172.21.15.133:0/988115750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804b9760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f55b806d490 tx=0x7f55b806d4c0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.525+0000 7f55cddba740 1 -- 172.21.15.133:0/988115750 shutdown_connections 2024-09-11T23:36:57.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.525+0000 7f55cddba740 1 --2- 172.21.15.133:0/988115750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f804a1d40 0x559f804b9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.525+0000 7f55cddba740 1 --2- 172.21.15.133:0/988115750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804b9760 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.525+0000 7f55cddba740 1 --2- 172.21.15.133:0/988115750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f80496b60 0x559f804995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.526+0000 7f55cddba740 1 -- 172.21.15.133:0/988115750 >> 172.21.15.133:0/988115750 conn(0x559f804930f0 msgr2=0x559f804a0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.526+0000 7f55cddba740 1 -- 172.21.15.133:0/988115750 shutdown_connections 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.526+0000 7f55cddba740 1 -- 172.21.15.133:0/988115750 wait complete. 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.526+0000 7f55cddba740 1 -- start start 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.526+0000 7f55cddba740 1 Processor -- start 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.526+0000 7f55cddba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f80496b60 0x559f804a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55cddba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55cddba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f804a1d40 0x559f804a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55cddba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41acaa0 con 0x559f80497480 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55cddba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41ac3d0 con 0x559f804a1d40 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55cddba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41a9630 con 0x559f80496b60 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55cddba740 10 client.0 _ll_register_callbacks cb 0x7f55bc00e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55c8f7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55c8f7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33330/0 (socket says 172.21.15.133:33330) 2024-09-11T23:36:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55c8f7c640 1 -- 172.21.15.133:0/3912431141 learned_addr learned my addr 172.21.15.133:0/3912431141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55c8f7c640 1 -- 172.21.15.133:0/3912431141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f80496b60 msgr2=0x559f804a63b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/3912431141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f80496b60 0x559f804a63b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55c8f7c640 1 -- 172.21.15.133:0/3912431141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f804a1d40 msgr2=0x559f804a9a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/3912431141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f804a1d40 0x559f804a9a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55c8f7c640 1 -- 172.21.15.133:0/3912431141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f804aa110 con 0x559f80497480 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55c8f7c640 1 --2- 172.21.15.133:0/3912431141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804a68f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f55b80ccf80 tx=0x7f55b8005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97eacbcb1061538 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3912431141 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80cf020 con 0x559f80497480 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3912431141 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f55b8099860 con 0x559f80497480 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.527+0000 7f55cddba740 1 -- 172.21.15.133:0/3912431141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b8009de0 con 0x559f80497480 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3912431141 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55b806a050 con 0x559f80497480 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55cddba740 10 client.15417 fetch_fsmap learned FSMap version 28 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55cddba740 1 -- 172.21.15.133:0/3912431141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f8051d2b0 con 0x559f80497480 2024-09-11T23:36:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3912431141 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b8009de0 con 0x559f80497480 2024-09-11T23:36:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3912431141 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f55b8099ac0 con 0x559f80497480 2024-09-11T23:36:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55cddba740 10 client.15417 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:36:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55cddba740 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:36:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55cddba740 20 client.15417 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55cddba740 1 --2- 172.21.15.133:0/3912431141 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559f804b19a0 0x559f8051f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55cddba740 4 client.15417 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:36:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55cddba740 1 -- 172.21.15.133:0/3912431141 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f80403d00 con 0x559f804b19a0 2024-09-11T23:36:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.528+0000 7f55c977d640 1 --2- 172.21.15.133:0/3912431141 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559f804b19a0 0x559f8051f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.529+0000 7f55a8ff9640 10 client.15417 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.529+0000 7f55c977d640 1 --2- 172.21.15.133:0/3912431141 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559f804b19a0 0x559f8051f4e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e3479f0b59bc26 server_cookie=849bb61a03358d6 in_seq=0 out_seq=0 2024-09-11T23:36:57.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55a8ff9640 1 -- 172.21.15.133:0/3912431141 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f80403d00 con 0x559f804b19a0 2024-09-11T23:36:57.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55a8ff9640 10 client.15417 handle_command_reply: tid=0 2024-09-11T23:36:57.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 1 client.15417 shutdown 2024-09-11T23:36:57.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 1 -- 172.21.15.133:0/3912431141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 msgr2=0x559f804a68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 1 --2- 172.21.15.133:0/3912431141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804a68f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f55b80ccf80 tx=0x7f55b8005da0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 1 -- 172.21.15.133:0/3912431141 shutdown_connections 2024-09-11T23:36:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 1 --2- 172.21.15.133:0/3912431141 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559f804b19a0 0x559f8051f4e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 1 --2- 172.21.15.133:0/3912431141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f804a1d40 0x559f804a9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 1 --2- 172.21.15.133:0/3912431141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f80497480 0x559f804a68f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 1 --2- 172.21.15.133:0/3912431141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f80496b60 0x559f804a63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 1 -- 172.21.15.133:0/3912431141 >> 172.21.15.133:0/3912431141 conn(0x559f804930f0 msgr2=0x559f8049ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55a8ff9640 0 client.15417 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 1 -- 172.21.15.133:0/3912431141 shutdown_connections 2024-09-11T23:36:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 1 -- 172.21.15.133:0/3912431141 wait complete. 2024-09-11T23:36:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.530+0000 7f55cddba740 20 client.15417 trim_cache size 0 max 16384 2024-09-11T23:36:57.533 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:57.534 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:36:57.534 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "13415344-3fbf-464e-8ad1-a94bf5179db4", 2024-09-11T23:36:57.534 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:36:57.534 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:57.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2626836101 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55ac0a2e10 msgr2=0x7f55ac0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55ac0a2e10 0x7f55ac0a52d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005fd0 tx=0x7f55bc009500 comp rx=0 tx=0).stop 2024-09-11T23:36:57.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2626836101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c4103780 msgr2=0x7f55c4100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c4103780 0x7f55c4100f70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f55b800d950 tx=0x7f55b800de10 comp rx=0 tx=0).stop 2024-09-11T23:36:57.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2626836101 shutdown_connections 2024-09-11T23:36:57.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55ac0a2e10 0x7f55ac0a52d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c4108300 0x7f55c41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c4103780 0x7f55c4100f70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 --2- 172.21.15.133:0/2626836101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c4102dd0 0x7f55c4100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2626836101 >> 172.21.15.133:0/2626836101 conn(0x7f55c40fc7d0 msgr2=0x7f55c4105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2626836101 shutdown_connections 2024-09-11T23:36:57.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.531+0000 7f55cb9e1640 1 -- 172.21.15.133:0/2626836101 wait complete. 2024-09-11T23:36:57.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.692+0000 7f587cb91640 1 -- 172.21.15.133:0/2253155216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f587806a6b0 con 0x7f5878101370 2024-09-11T23:36:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.696+0000 7f58677fe640 1 -- 172.21.15.133:0/2253155216 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f587806a6b0 con 0x7f5878101370 2024-09-11T23:36:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 -- 172.21.15.133:0/2253155216 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58600a2f40 msgr2=0x7f58600a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58600a2f40 0x7f58600a5400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f58680097b0 tx=0x7f58680058d0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 -- 172.21.15.133:0/2253155216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5878101370 msgr2=0x7f5878199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5878101370 0x7f5878199710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f586c00e9c0 tx=0x7f586c00ee80 comp rx=0 tx=0).stop 2024-09-11T23:36:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 -- 172.21.15.133:0/2253155216 shutdown_connections 2024-09-11T23:36:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58600a2f40 0x7f58600a5400 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58781069e0 0x7f58781a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5878101370 0x7f5878199710 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 --2- 172.21.15.133:0/2253155216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58781009d0 0x7f58781991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 -- 172.21.15.133:0/2253155216 >> 172.21.15.133:0/2253155216 conn(0x7f58780fc6b0 msgr2=0x7f58780fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 -- 172.21.15.133:0/2253155216 shutdown_connections 2024-09-11T23:36:57.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.698+0000 7f587cb91640 1 -- 172.21.15.133:0/2253155216 wait complete. 2024-09-11T23:36:57.711 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-11T23:36:57.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.817+0000 7fc90bcdd640 1 -- 172.21.15.133:0/938143469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc904106a00 msgr2=0x7fc904106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.817+0000 7fc90bcdd640 1 --2- 172.21.15.133:0/938143469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc904106a00 0x7fc904106de0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc900009930 tx=0x7fc90002a420 comp rx=0 tx=0).stop 2024-09-11T23:36:57.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.817+0000 7fc90bcdd640 1 -- 172.21.15.133:0/938143469 shutdown_connections 2024-09-11T23:36:57.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.817+0000 7fc90bcdd640 1 --2- 172.21.15.133:0/938143469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc904101390 0x7fc90410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.817+0000 7fc90bcdd640 1 --2- 172.21.15.133:0/938143469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9041009f0 0x7fc904100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.817+0000 7fc90bcdd640 1 --2- 172.21.15.133:0/938143469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc904106a00 0x7fc904106de0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.817+0000 7fc90bcdd640 1 -- 172.21.15.133:0/938143469 >> 172.21.15.133:0/938143469 conn(0x7fc9040fc6b0 msgr2=0x7fc9040fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:57.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.817+0000 7fc90bcdd640 1 -- 172.21.15.133:0/938143469 shutdown_connections 2024-09-11T23:36:57.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.817+0000 7fc90bcdd640 1 -- 172.21.15.133:0/938143469 wait complete. 2024-09-11T23:36:57.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc90bcdd640 1 Processor -- start 2024-09-11T23:36:57.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc90bcdd640 1 -- start start 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc90bcdd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9041009f0 0x7fc904199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc90bcdd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc904101390 0x7fc904199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc90bcdd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc904106a00 0x7fc9041a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc90bcdd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc904110e40 con 0x7fc904101390 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc90bcdd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc904110cc0 con 0x7fc9041009f0 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc90bcdd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc904110fc0 con 0x7fc904106a00 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc909278640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc904101390 0x7fc904199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc909278640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc904101390 0x7fc904199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33352/0 (socket says 172.21.15.133:33352) 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc909278640 1 -- 172.21.15.133:0/866092706 learned_addr learned my addr 172.21.15.133:0/866092706 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc90a27a640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc904106a00 0x7fc9041a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc909a79640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9041009f0 0x7fc904199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc909278640 1 -- 172.21.15.133:0/866092706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc904106a00 msgr2=0x7fc9041a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc909278640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc904106a00 0x7fc9041a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc909278640 1 -- 172.21.15.133:0/866092706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9041009f0 msgr2=0x7fc904199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc909278640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9041009f0 0x7fc904199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.818+0000 7fc909278640 1 -- 172.21.15.133:0/866092706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9041a47f0 con 0x7fc904101390 2024-09-11T23:36:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.819+0000 7fc909278640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc904101390 0x7fc904199770 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400e9f0 tx=0x7fc8f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.819+0000 7fc8faffd640 1 -- 172.21.15.133:0/866092706 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f400cd80 con 0x7fc904101390 2024-09-11T23:36:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.819+0000 7fc8faffd640 1 -- 172.21.15.133:0/866092706 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4005500 con 0x7fc904101390 2024-09-11T23:36:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.819+0000 7fc90bcdd640 1 -- 172.21.15.133:0/866092706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9041a49c0 con 0x7fc904101390 2024-09-11T23:36:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.819+0000 7fc8faffd640 1 -- 172.21.15.133:0/866092706 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4010690 con 0x7fc904101390 2024-09-11T23:36:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.819+0000 7fc90bcdd640 1 -- 172.21.15.133:0/866092706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9041a4e90 con 0x7fc904101390 2024-09-11T23:36:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.820+0000 7fc90bcdd640 1 -- 172.21.15.133:0/866092706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc904100e50 con 0x7fc904101390 2024-09-11T23:36:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.821+0000 7fc8faffd640 1 -- 172.21.15.133:0/866092706 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fc8f40046c0 con 0x7fc904101390 2024-09-11T23:36:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.822+0000 7fc8faffd640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc8f00a2ec0 0x7fc8f00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.822+0000 7fc8faffd640 1 -- 172.21.15.133:0/866092706 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc8f40cfb10 con 0x7fc904101390 2024-09-11T23:36:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.822+0000 7fc909a79640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc8f00a2ec0 0x7fc8f00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.822+0000 7fc909a79640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc8f00a2ec0 0x7fc8f00a5380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc900009900 tx=0x7fc9000058d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:57.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:57.826+0000 7fc8faffd640 1 -- 172.21.15.133:0/866092706 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8f409c4f0 con 0x7fc904101390 2024-09-11T23:36:58.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.016+0000 7fc90bcdd640 1 -- 172.21.15.133:0/866092706 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc904062630 con 0x7fc8f00a2ec0 2024-09-11T23:36:58.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.079+0000 7fc8faffd640 1 -- 172.21.15.133:0/866092706 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc904062630 con 0x7fc8f00a2ec0 2024-09-11T23:36:58.079 INFO:teuthology.orchestra.run.smithi133.stdout:Schedule set for path / 2024-09-11T23:36:58.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 -- 172.21.15.133:0/866092706 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc8f00a2ec0 msgr2=0x7fc8f00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc8f00a2ec0 0x7fc8f00a5380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc900009900 tx=0x7fc9000058d0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 -- 172.21.15.133:0/866092706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc904101390 msgr2=0x7fc904199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc904101390 0x7fc904199770 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400e9f0 tx=0x7fc8f400edf0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 -- 172.21.15.133:0/866092706 shutdown_connections 2024-09-11T23:36:58.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc8f00a2ec0 0x7fc8f00a5380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc904106a00 0x7fc9041a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc904101390 0x7fc904199770 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 --2- 172.21.15.133:0/866092706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9041009f0 0x7fc904199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 -- 172.21.15.133:0/866092706 >> 172.21.15.133:0/866092706 conn(0x7fc9040fc6b0 msgr2=0x7fc9040fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:58.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 -- 172.21.15.133:0/866092706 shutdown_connections 2024-09-11T23:36:58.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.081+0000 7fc90bcdd640 1 -- 172.21.15.133:0/866092706 wait complete. 2024-09-11T23:36:58.092 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-11T23:36:58.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.195+0000 7ff4f2178640 1 -- 172.21.15.133:0/3832691010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4ec102460 msgr2=0x7ff4ec10e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.195+0000 7ff4f2178640 1 --2- 172.21.15.133:0/3832691010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4ec102460 0x7ff4ec10e990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4009930 tx=0x7ff4e402a420 comp rx=0 tx=0).stop 2024-09-11T23:36:58.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.195+0000 7ff4f2178640 1 -- 172.21.15.133:0/3832691010 shutdown_connections 2024-09-11T23:36:58.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.195+0000 7ff4f2178640 1 --2- 172.21.15.133:0/3832691010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4ec102460 0x7ff4ec10e990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.195+0000 7ff4f2178640 1 --2- 172.21.15.133:0/3832691010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4ec101ac0 0x7ff4ec101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.195+0000 7ff4f2178640 1 --2- 172.21.15.133:0/3832691010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec107ad0 0x7ff4ec107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.195+0000 7ff4f2178640 1 -- 172.21.15.133:0/3832691010 >> 172.21.15.133:0/3832691010 conn(0x7ff4ec0fd7e0 msgr2=0x7ff4ec0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.195+0000 7ff4f2178640 1 -- 172.21.15.133:0/3832691010 shutdown_connections 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.195+0000 7ff4f2178640 1 -- 172.21.15.133:0/3832691010 wait complete. 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.196+0000 7ff4f2178640 1 Processor -- start 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.196+0000 7ff4f2178640 1 -- start start 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.196+0000 7ff4f2178640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4ec101ac0 0x7ff4ec198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.196+0000 7ff4f2178640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec102460 0x7ff4ec198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.196+0000 7ff4f2178640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4ec107ad0 0x7ff4ec19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.196+0000 7ff4f2178640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec111f60 con 0x7ff4ec101ac0 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.196+0000 7ff4f2178640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec111de0 con 0x7ff4ec107ad0 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.196+0000 7ff4f2178640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec1120e0 con 0x7ff4ec102460 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4eb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4ec101ac0 0x7ff4ec198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4eb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4ec101ac0 0x7ff4ec198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33364/0 (socket says 172.21.15.133:33364) 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4eb7fe640 1 -- 172.21.15.133:0/3140986957 learned_addr learned my addr 172.21.15.133:0/3140986957 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4eaffd640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec102460 0x7ff4ec198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4eb7fe640 1 -- 172.21.15.133:0/3140986957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec102460 msgr2=0x7ff4ec198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4eb7fe640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec102460 0x7ff4ec198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4eb7fe640 1 -- 172.21.15.133:0/3140986957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4ec107ad0 msgr2=0x7ff4ec19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4ebfff640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4ec107ad0 0x7ff4ec19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4eb7fe640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4ec107ad0 0x7ff4ec19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4eb7fe640 1 -- 172.21.15.133:0/3140986957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec19d340 con 0x7ff4ec101ac0 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4eb7fe640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4ec101ac0 0x7ff4ec198350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00ebc0 tx=0x7ff4dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4e8ff9640 1 -- 172.21.15.133:0/3140986957 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc010630 con 0x7ff4ec101ac0 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4e8ff9640 1 -- 172.21.15.133:0/3140986957 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc010cb0 con 0x7ff4ec101ac0 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4f2178640 1 -- 172.21.15.133:0/3140986957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ec19d630 con 0x7ff4ec101ac0 2024-09-11T23:36:58.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4e8ff9640 1 -- 172.21.15.133:0/3140986957 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc018bd0 con 0x7ff4ec101ac0 2024-09-11T23:36:58.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.197+0000 7ff4f2178640 1 -- 172.21.15.133:0/3140986957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ec1a4ec0 con 0x7ff4ec101ac0 2024-09-11T23:36:58.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.198+0000 7ff4f2178640 1 -- 172.21.15.133:0/3140986957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ec103200 con 0x7ff4ec101ac0 2024-09-11T23:36:58.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.200+0000 7ff4e8ff9640 1 -- 172.21.15.133:0/3140986957 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7ff4dc0107d0 con 0x7ff4ec101ac0 2024-09-11T23:36:58.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.200+0000 7ff4e8ff9640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4d40a2f40 0x7ff4d40a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.200+0000 7ff4eaffd640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4d40a2f40 0x7ff4d40a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.200+0000 7ff4e8ff9640 1 -- 172.21.15.133:0/3140986957 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff4dc0cfe60 con 0x7ff4ec101ac0 2024-09-11T23:36:58.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.201+0000 7ff4eaffd640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4d40a2f40 0x7ff4d40a5400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0ffc90 tx=0x7ff4e000b410 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:58.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.203+0000 7ff4e8ff9640 1 -- 172.21.15.133:0/3140986957 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4dc09c8a0 con 0x7ff4ec101ac0 2024-09-11T23:36:58.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3673604259' entity='client.admin' 2024-09-11T23:36:58.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:58.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:58.322 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3116404880' entity='client.admin' 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2253155216' entity='client.admin' 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:58.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:58.324 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:58.349 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:36:58.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.398+0000 7ff4f2178640 1 -- 172.21.15.133:0/3140986957 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7ff4ec0fe4e0 con 0x7ff4d40a2f40 2024-09-11T23:36:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3673604259' entity='client.admin' 2024-09-11T23:36:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3116404880' entity='client.admin' 2024-09-11T23:36:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:36:58.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:36:58.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:36:58.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:36:58.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2253155216' entity='client.admin' 2024-09-11T23:36:58.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:58.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:58.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:58.449 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:58.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.458+0000 7ff4e8ff9640 1 -- 172.21.15.133:0/3140986957 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7ff4ec0fe4e0 con 0x7ff4d40a2f40 2024-09-11T23:36:58.457 INFO:teuthology.orchestra.run.smithi133.stdout:Retention added to path / 2024-09-11T23:36:58.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 -- 172.21.15.133:0/3140986957 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4d40a2f40 msgr2=0x7ff4d40a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4d40a2f40 0x7ff4d40a5400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0ffc90 tx=0x7ff4e000b410 comp rx=0 tx=0).stop 2024-09-11T23:36:58.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 -- 172.21.15.133:0/3140986957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4ec101ac0 msgr2=0x7ff4ec198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4ec101ac0 0x7ff4ec198350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00ebc0 tx=0x7ff4dc00c6a0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 -- 172.21.15.133:0/3140986957 shutdown_connections 2024-09-11T23:36:58.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4d40a2f40 0x7ff4d40a5400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4ec107ad0 0x7ff4ec19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec102460 0x7ff4ec198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 --2- 172.21.15.133:0/3140986957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4ec101ac0 0x7ff4ec198350 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 -- 172.21.15.133:0/3140986957 >> 172.21.15.133:0/3140986957 conn(0x7ff4ec0fd7e0 msgr2=0x7ff4ec0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:58.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 -- 172.21.15.133:0/3140986957 shutdown_connections 2024-09-11T23:36:58.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.460+0000 7ff4f2178640 1 -- 172.21.15.133:0/3140986957 wait complete. 2024-09-11T23:36:58.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T23:36:58.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3673604259' entity='client.admin' 2024-09-11T23:36:58.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:58.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:58.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:58.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3116404880' entity='client.admin' 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2253155216' entity='client.admin' 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:36:58.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:36:58.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.576+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2979901874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb81035a0 msgr2=0x7f8eb810ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.576+0000 7f8ebf73a640 1 --2- 172.21.15.133:0/2979901874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb81035a0 0x7f8eb810ba80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009900 tx=0x7f8eb402a480 comp rx=0 tx=0).stop 2024-09-11T23:36:58.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.576+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2979901874 shutdown_connections 2024-09-11T23:36:58.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.576+0000 7f8ebf73a640 1 --2- 172.21.15.133:0/2979901874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8eb810c0d0 0x7f8eb810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.576+0000 7f8ebf73a640 1 --2- 172.21.15.133:0/2979901874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb81035a0 0x7f8eb810ba80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.576+0000 7f8ebf73a640 1 --2- 172.21.15.133:0/2979901874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eb8102c80 0x7f8eb8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.576+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2979901874 >> 172.21.15.133:0/2979901874 conn(0x7f8eb8077990 msgr2=0x7f8eb80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.576+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2979901874 shutdown_connections 2024-09-11T23:36:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.576+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2979901874 wait complete. 2024-09-11T23:36:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebf73a640 1 Processor -- start 2024-09-11T23:36:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebf73a640 1 -- start start 2024-09-11T23:36:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebf73a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eb8102c80 0x7f8eb8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebf73a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb81035a0 0x7f8eb81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebf73a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8eb810c0d0 0x7f8eb819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebf73a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8111eb0 con 0x7f8eb81035a0 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebf73a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8111d30 con 0x7f8eb8102c80 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebf73a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8112030 con 0x7f8eb810c0d0 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebccd5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb81035a0 0x7f8eb81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebccd5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb81035a0 0x7f8eb81988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33370/0 (socket says 172.21.15.133:33370) 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebccd5640 1 -- 172.21.15.133:0/2891666921 learned_addr learned my addr 172.21.15.133:0/2891666921 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.577+0000 7f8ebdcd7640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8eb810c0d0 0x7f8eb819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8ebd4d6640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eb8102c80 0x7f8eb8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8ebccd5640 1 -- 172.21.15.133:0/2891666921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8eb810c0d0 msgr2=0x7f8eb819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8ebccd5640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8eb810c0d0 0x7f8eb819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8ebccd5640 1 -- 172.21.15.133:0/2891666921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eb8102c80 msgr2=0x7f8eb8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8ebccd5640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eb8102c80 0x7f8eb8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8ebccd5640 1 -- 172.21.15.133:0/2891666921 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb819d350 con 0x7f8eb81035a0 2024-09-11T23:36:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8ebccd5640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb81035a0 0x7f8eb81988a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4006010 tx=0x7f8eb4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8eae7fc640 1 -- 172.21.15.133:0/2891666921 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb403f030 con 0x7f8eb81035a0 2024-09-11T23:36:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8eae7fc640 1 -- 172.21.15.133:0/2891666921 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb4004440 con 0x7f8eb81035a0 2024-09-11T23:36:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2891666921 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb819d5e0 con 0x7f8eb81035a0 2024-09-11T23:36:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8eae7fc640 1 -- 172.21.15.133:0/2891666921 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4035070 con 0x7f8eb81035a0 2024-09-11T23:36:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.578+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2891666921 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb81a4ed0 con 0x7f8eb81035a0 2024-09-11T23:36:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.580+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2891666921 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb806a180 con 0x7f8eb81035a0 2024-09-11T23:36:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.580+0000 7f8eae7fc640 1 -- 172.21.15.133:0/2891666921 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f8eb4042030 con 0x7f8eb81035a0 2024-09-11T23:36:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.581+0000 7f8eae7fc640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ea40a2ec0 0x7f8ea40a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.581+0000 7f8ebd4d6640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ea40a2ec0 0x7f8ea40a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.581+0000 7f8eae7fc640 1 -- 172.21.15.133:0/2891666921 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8eb40ef900 con 0x7f8eb81035a0 2024-09-11T23:36:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.582+0000 7f8ebd4d6640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ea40a2ec0 0x7f8ea40a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009bd0 tx=0x7f8ea8005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:58.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.587+0000 7f8eae7fc640 1 -- 172.21.15.133:0/2891666921 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8eb40bbdc0 con 0x7f8eb81035a0 2024-09-11T23:36:58.613 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:36:58.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.784+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2891666921 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f8eb80fd650 con 0x7f8ea40a2ec0 2024-09-11T23:36:58.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.790+0000 7f8eae7fc640 1 -- 172.21.15.133:0/2891666921 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f8eb80fd650 con 0x7f8ea40a2ec0 2024-09-11T23:36:58.790 INFO:teuthology.orchestra.run.smithi133.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T23:36:58", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-11T23:36:58.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.792+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2891666921 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ea40a2ec0 msgr2=0x7f8ea40a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.792+0000 7f8ebf73a640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ea40a2ec0 0x7f8ea40a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009bd0 tx=0x7f8ea8005f90 comp rx=0 tx=0).stop 2024-09-11T23:36:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.792+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2891666921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb81035a0 msgr2=0x7f8eb81988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.792+0000 7f8ebf73a640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb81035a0 0x7f8eb81988a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4006010 tx=0x7f8eb4004040 comp rx=0 tx=0).stop 2024-09-11T23:36:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.792+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2891666921 shutdown_connections 2024-09-11T23:36:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.793+0000 7f8ebf73a640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ea40a2ec0 0x7f8ea40a5380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.793+0000 7f8ebf73a640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8eb810c0d0 0x7f8eb819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.793+0000 7f8ebf73a640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb81035a0 0x7f8eb81988a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.793+0000 7f8ebf73a640 1 --2- 172.21.15.133:0/2891666921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eb8102c80 0x7f8eb8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.793+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2891666921 >> 172.21.15.133:0/2891666921 conn(0x7f8eb8077990 msgr2=0x7f8eb80fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.793+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2891666921 shutdown_connections 2024-09-11T23:36:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.793+0000 7f8ebf73a640 1 -- 172.21.15.133:0/2891666921 wait complete. 2024-09-11T23:36:58.803 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T23:36:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.908+0000 7ff6a2731640 1 -- 172.21.15.133:0/201898553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c101350 msgr2=0x7ff69c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.909+0000 7ff6a2731640 1 --2- 172.21.15.133:0/201898553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c101350 0x7ff69c10d880 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff694009cd0 tx=0x7ff69402a910 comp rx=0 tx=0).stop 2024-09-11T23:36:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.909+0000 7ff6a2731640 1 -- 172.21.15.133:0/201898553 shutdown_connections 2024-09-11T23:36:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.909+0000 7ff6a2731640 1 --2- 172.21.15.133:0/201898553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c101350 0x7ff69c10d880 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.909+0000 7ff6a2731640 1 --2- 172.21.15.133:0/201898553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff69c1009b0 0x7ff69c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.909+0000 7ff6a2731640 1 --2- 172.21.15.133:0/201898553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff69c1069c0 0x7ff69c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.909+0000 7ff6a2731640 1 -- 172.21.15.133:0/201898553 >> 172.21.15.133:0/201898553 conn(0x7ff69c077990 msgr2=0x7ff69c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:58.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.909+0000 7ff6a2731640 1 -- 172.21.15.133:0/201898553 shutdown_connections 2024-09-11T23:36:58.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.909+0000 7ff6a2731640 1 -- 172.21.15.133:0/201898553 wait complete. 2024-09-11T23:36:58.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.909+0000 7ff6a2731640 1 Processor -- start 2024-09-11T23:36:58.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.909+0000 7ff6a2731640 1 -- start start 2024-09-11T23:36:58.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6a2731640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff69c1009b0 0x7ff69c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6a2731640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c101350 0x7ff69c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6a2731640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff69c1069c0 0x7ff69c19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6a2731640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c110e20 con 0x7ff69c101350 2024-09-11T23:36:58.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6a2731640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c110ca0 con 0x7ff69c1069c0 2024-09-11T23:36:58.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6a2731640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c110fa0 con 0x7ff69c1009b0 2024-09-11T23:36:58.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c101350 0x7ff69c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c101350 0x7ff69c197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33378/0 (socket says 172.21.15.133:33378) 2024-09-11T23:36:58.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff69b7fe640 1 -- 172.21.15.133:0/3080931817 learned_addr learned my addr 172.21.15.133:0/3080931817 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:58.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6a0cce640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff69c1069c0 0x7ff69c19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff69bfff640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff69c1009b0 0x7ff69c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff69b7fe640 1 -- 172.21.15.133:0/3080931817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff69c1009b0 msgr2=0x7ff69c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff69b7fe640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff69c1009b0 0x7ff69c1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff69b7fe640 1 -- 172.21.15.133:0/3080931817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff69c1069c0 msgr2=0x7ff69c19bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:58.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff69b7fe640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff69c1069c0 0x7ff69c19bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:58.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff69b7fe640 1 -- 172.21.15.133:0/3080931817 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c19c2d0 con 0x7ff69c101350 2024-09-11T23:36:58.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff69b7fe640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c101350 0x7ff69c197820 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e9f0 tx=0x7ff69000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:58.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6997fa640 1 -- 172.21.15.133:0/3080931817 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69000cd80 con 0x7ff69c101350 2024-09-11T23:36:58.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6997fa640 1 -- 172.21.15.133:0/3080931817 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff690005500 con 0x7ff69c101350 2024-09-11T23:36:58.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6997fa640 1 -- 172.21.15.133:0/3080931817 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690010690 con 0x7ff69c101350 2024-09-11T23:36:58.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6a2731640 1 -- 172.21.15.133:0/3080931817 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c19c5c0 con 0x7ff69c101350 2024-09-11T23:36:58.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.910+0000 7ff6a2731640 1 -- 172.21.15.133:0/3080931817 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c1a3e50 con 0x7ff69c101350 2024-09-11T23:36:58.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.911+0000 7ff6a2731640 1 -- 172.21.15.133:0/3080931817 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69c100e10 con 0x7ff69c101350 2024-09-11T23:36:58.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.913+0000 7ff6997fa640 1 -- 172.21.15.133:0/3080931817 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7ff6900046c0 con 0x7ff69c101350 2024-09-11T23:36:58.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.913+0000 7ff6997fa640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6840a2ec0 0x7ff6840a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:58.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.913+0000 7ff69bfff640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6840a2ec0 0x7ff6840a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:58.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.913+0000 7ff6997fa640 1 -- 172.21.15.133:0/3080931817 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff6900cfdf0 con 0x7ff69c101350 2024-09-11T23:36:58.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.914+0000 7ff69bfff640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6840a2ec0 0x7ff6840a5380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006fd0 tx=0x7ff68c008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:58.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:58.916+0000 7ff6997fa640 1 -- 172.21.15.133:0/3080931817 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6900541b0 con 0x7ff69c101350 2024-09-11T23:36:59.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.051+0000 7faa95f70640 1 -- 172.21.15.133:0/1674008132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa90100e20 msgr2=0x7faa90101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:59.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.051+0000 7faa95f70640 1 --2- 172.21.15.133:0/1674008132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa90100e20 0x7faa90101200 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faa80009900 tx=0x7faa8002a480 comp rx=0 tx=0).stop 2024-09-11T23:36:59.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.051+0000 7faa95f70640 1 -- 172.21.15.133:0/1674008132 shutdown_connections 2024-09-11T23:36:59.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.051+0000 7faa95f70640 1 --2- 172.21.15.133:0/1674008132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa90102ef0 0x7faa9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.051+0000 7faa95f70640 1 --2- 172.21.15.133:0/1674008132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa90102570 0x7faa901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.051+0000 7faa95f70640 1 --2- 172.21.15.133:0/1674008132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa90100e20 0x7faa90101200 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.051+0000 7faa95f70640 1 -- 172.21.15.133:0/1674008132 >> 172.21.15.133:0/1674008132 conn(0x7faa900fc7d0 msgr2=0x7faa900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:59.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.051+0000 7faa95f70640 1 -- 172.21.15.133:0/1674008132 shutdown_connections 2024-09-11T23:36:59.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.051+0000 7faa95f70640 1 -- 172.21.15.133:0/1674008132 wait complete. 2024-09-11T23:36:59.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.052+0000 7faa95f70640 1 Processor -- start 2024-09-11T23:36:59.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.052+0000 7faa95f70640 1 -- start start 2024-09-11T23:36:59.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.052+0000 7faa95f70640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa90100e20 0x7faa90103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.052+0000 7faa95f70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa90102570 0x7faa90103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.052+0000 7faa95f70640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa90102ef0 0x7faa901041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.052+0000 7faa95f70640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faa90110f80 con 0x7faa90102570 2024-09-11T23:36:59.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.052+0000 7faa95f70640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa90110e00 con 0x7faa90102ef0 2024-09-11T23:36:59.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.052+0000 7faa95f70640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faa90111100 con 0x7faa90100e20 2024-09-11T23:36:59.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.052+0000 7faa8effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa90102570 0x7faa90103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa90102570 0x7faa90103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33404/0 (socket says 172.21.15.133:33404) 2024-09-11T23:36:59.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8effd640 1 -- 172.21.15.133:0/1390986276 learned_addr learned my addr 172.21.15.133:0/1390986276 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:59.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8f7fe640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa90100e20 0x7faa90103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8ffff640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa90102ef0 0x7faa901041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8effd640 1 -- 172.21.15.133:0/1390986276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa90100e20 msgr2=0x7faa90103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:59.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8effd640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa90100e20 0x7faa90103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8effd640 1 -- 172.21.15.133:0/1390986276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa90102ef0 msgr2=0x7faa901041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:59.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8effd640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa90102ef0 0x7faa901041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8effd640 1 -- 172.21.15.133:0/1390986276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa901a69e0 con 0x7faa90102570 2024-09-11T23:36:59.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8f7fe640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa90100e20 0x7faa90103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:59.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8effd640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa90102570 0x7faa90103c90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faa8400d770 tx=0x7faa8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:59.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8cff9640 1 -- 172.21.15.133:0/1390986276 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84017070 con 0x7faa90102570 2024-09-11T23:36:59.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa8cff9640 1 -- 172.21.15.133:0/1390986276 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa84005500 con 0x7faa90102570 2024-09-11T23:36:59.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.053+0000 7faa95f70640 1 -- 172.21.15.133:0/1390986276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa901a6cd0 con 0x7faa90102570 2024-09-11T23:36:59.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.054+0000 7faa8cff9640 1 -- 172.21.15.133:0/1390986276 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84004e00 con 0x7faa90102570 2024-09-11T23:36:59.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.054+0000 7faa95f70640 1 -- 172.21.15.133:0/1390986276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa901a71c0 con 0x7faa90102570 2024-09-11T23:36:59.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.056+0000 7faa8cff9640 1 -- 172.21.15.133:0/1390986276 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7faa84004fa0 con 0x7faa90102570 2024-09-11T23:36:59.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa98349740 1 Processor -- start 2024-09-11T23:36:59.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa98349740 1 -- start start 2024-09-11T23:36:59.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa98349740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed7124a0 0x5556ed712880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa98349740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed712dc0 0x5556ed71d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa98349740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed71d680 0x5556ed71fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa98349740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5556ed6782e0 con 0x5556ed71d680 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa98349740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5556ed61fce0 con 0x5556ed712dc0 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa98349740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5556ed6742a0 con 0x5556ed7124a0 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa8effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed71d680 0x5556ed71fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa8effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed71d680 0x5556ed71fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33408/0 (socket says 172.21.15.133:33408) 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa8effd640 1 -- 172.21.15.133:0/681922089 learned_addr learned my addr 172.21.15.133:0/681922089 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa8cff9640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faa780a2e10 0x7faa780a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa8f7fe640 1 --2- 172.21.15.133:0/681922089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed712dc0 0x5556ed71d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa8ffff640 1 --2- 172.21.15.133:0/681922089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed7124a0 0x5556ed712880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.057+0000 7faa8ffff640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faa780a2e10 0x7faa780a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa8ffff640 1 -- 172.21.15.133:0/681922089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed712dc0 msgr2=0x5556ed71d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa8ffff640 1 --2- 172.21.15.133:0/681922089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed712dc0 0x5556ed71d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa8ffff640 1 -- 172.21.15.133:0/681922089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed71d680 msgr2=0x5556ed71fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa8ffff640 1 --2- 172.21.15.133:0/681922089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed71d680 0x5556ed71fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa8cff9640 1 -- 172.21.15.133:0/1390986276 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7faa840cf060 con 0x7faa90102570 2024-09-11T23:36:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa8ffff640 1 -- 172.21.15.133:0/681922089 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ed604440 con 0x5556ed7124a0 2024-09-11T23:36:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa8effd640 1 --2- 172.21.15.133:0/681922089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed71d680 0x5556ed71fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:36:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa8ffff640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faa780a2e10 0x7faa780a52d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa88010c40 tx=0x7faa8800c000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa8ffff640 1 --2- 172.21.15.133:0/681922089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed7124a0 0x5556ed712880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa88005fd0 tx=0x7faa88010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6906c3d4522bcb41 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa75ffb640 1 -- 172.21.15.133:0/681922089 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88024070 con 0x5556ed7124a0 2024-09-11T23:36:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa75ffb640 1 -- 172.21.15.133:0/681922089 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa88009c40 con 0x5556ed7124a0 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.058+0000 7faa75ffb640 1 -- 172.21.15.133:0/681922089 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88005280 con 0x5556ed7124a0 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.059+0000 7faa98349740 1 -- 172.21.15.133:0/681922089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed7124a0 msgr2=0x5556ed712880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.059+0000 7faa98349740 1 --2- 172.21.15.133:0/681922089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed7124a0 0x5556ed712880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa88005fd0 tx=0x7faa88010740 comp rx=0 tx=0).stop 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.059+0000 7faa98349740 1 -- 172.21.15.133:0/681922089 shutdown_connections 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.059+0000 7faa98349740 1 --2- 172.21.15.133:0/681922089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed71d680 0x5556ed71fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.059+0000 7faa98349740 1 --2- 172.21.15.133:0/681922089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed712dc0 0x5556ed71d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.059+0000 7faa98349740 1 --2- 172.21.15.133:0/681922089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed7124a0 0x5556ed712880 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.059+0000 7faa98349740 1 -- 172.21.15.133:0/681922089 >> 172.21.15.133:0/681922089 conn(0x5556ed70ea30 msgr2=0x5556ed70ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.059+0000 7faa98349740 1 -- 172.21.15.133:0/681922089 shutdown_connections 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.059+0000 7faa98349740 1 -- 172.21.15.133:0/681922089 wait complete. 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.059+0000 7faa98349740 1 -- start start 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa98349740 1 Processor -- start 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa98349740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed71d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa98349740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed712dc0 0x5556ed71a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa98349740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed71d680 0x5556ed71a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa98349740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5556ed720250 con 0x5556ed7124a0 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa98349740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5556ed633050 con 0x5556ed712dc0 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa98349740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5556ed7203d0 con 0x5556ed71d680 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed71d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed71d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33418/0 (socket says 172.21.15.133:33418) 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa8f7fe640 1 -- 172.21.15.133:0/4279351898 learned_addr learned my addr 172.21.15.133:0/4279351898 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa98349740 10 client.0 _ll_register_callbacks cb 0x5556ed673160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa8ffff640 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed71d680 0x5556ed71a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa8effd640 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed712dc0 0x5556ed71a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa8f7fe640 1 -- 172.21.15.133:0/4279351898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed71d680 msgr2=0x5556ed71a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa8f7fe640 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed71d680 0x5556ed71a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa8f7fe640 1 -- 172.21.15.133:0/4279351898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed712dc0 msgr2=0x5556ed71a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:59.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa8f7fe640 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed712dc0 0x5556ed71a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.060+0000 7faa8f7fe640 1 -- 172.21.15.133:0/4279351898 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ed3e93c0 con 0x5556ed7124a0 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa8f7fe640 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed71d020 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7faa800314e0 tx=0x7faa80003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1165587a63929668 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa6effd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa6effd640 1 -- 172.21.15.133:0/4279351898 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80035070 con 0x5556ed7124a0 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa6effd640 1 -- 172.21.15.133:0/4279351898 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7faa80003ba0 con 0x5556ed7124a0 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa98349740 1 -- 172.21.15.133:0/4279351898 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa88009c40 con 0x5556ed7124a0 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa6effd640 1 -- 172.21.15.133:0/4279351898 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa80003da0 con 0x5556ed7124a0 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa98349740 10 client.15453 fetch_fsmap learned FSMap version 28 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa98349740 1 -- 172.21.15.133:0/4279351898 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556ed72bc90 con 0x5556ed7124a0 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa6effd640 1 -- 172.21.15.133:0/4279351898 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa88009c40 con 0x5556ed7124a0 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa6effd640 1 -- 172.21.15.133:0/4279351898 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faa8004e020 con 0x5556ed7124a0 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa98349740 10 client.15453 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:36:59.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.061+0000 7faa98349740 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:36:59.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.062+0000 7faa98349740 20 client.15453 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:59.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.062+0000 7faa98349740 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ed72d0a0 0x5556ed72f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.062+0000 7faa98349740 4 client.15453 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:36:59.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.062+0000 7faa98349740 1 -- 172.21.15.133:0/4279351898 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556ed685850 con 0x5556ed72d0a0 2024-09-11T23:36:59.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.062+0000 7faa8effd640 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ed72d0a0 0x5556ed72f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.062+0000 7faa6effd640 10 client.15453 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:59.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.062+0000 7faa8effd640 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ed72d0a0 0x5556ed72f490 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff108a8531b7f18 server_cookie=9b6df35e315733a in_seq=0 out_seq=0 2024-09-11T23:36:59.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa6effd640 1 -- 172.21.15.133:0/4279351898 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556ed685850 con 0x5556ed72d0a0 2024-09-11T23:36:59.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa6effd640 10 client.15453 handle_command_reply: tid=0 2024-09-11T23:36:59.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa98349740 1 client.15453 shutdown 2024-09-11T23:36:59.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa98349740 1 -- 172.21.15.133:0/4279351898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 msgr2=0x5556ed71d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:59.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa98349740 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed71d020 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7faa800314e0 tx=0x7faa80003820 comp rx=0 tx=0).stop 2024-09-11T23:36:59.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa98349740 1 -- 172.21.15.133:0/4279351898 shutdown_connections 2024-09-11T23:36:59.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa98349740 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ed72d0a0 0x5556ed72f490 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa98349740 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed71d680 0x5556ed71a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa98349740 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed712dc0 0x5556ed71a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa98349740 1 --2- 172.21.15.133:0/4279351898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed71d020 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa6effd640 0 client.15453 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:59.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa98349740 1 -- 172.21.15.133:0/4279351898 >> 172.21.15.133:0/4279351898 conn(0x5556ed70ea30 msgr2=0x5556ed714760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:59.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa98349740 1 -- 172.21.15.133:0/4279351898 shutdown_connections 2024-09-11T23:36:59.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.063+0000 7faa98349740 1 -- 172.21.15.133:0/4279351898 wait complete. 2024-09-11T23:36:59.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.064+0000 7faa98349740 20 client.15453 trim_cache size 0 max 16384 2024-09-11T23:36:59.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa98349740 1 Processor -- start 2024-09-11T23:36:59.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa98349740 1 -- start start 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa98349740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed7124a0 0x5556ed734d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa98349740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed712dc0 0x5556ed735280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa98349740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed71d680 0x5556ed717530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa98349740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faa901af8b0 con 0x5556ed712dc0 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa98349740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa901b13f0 con 0x5556ed71d680 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa98349740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faa901aff80 con 0x5556ed7124a0 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa8effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed712dc0 0x5556ed735280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa8effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed712dc0 0x5556ed735280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33426/0 (socket says 172.21.15.133:33426) 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa8effd640 1 -- 172.21.15.133:0/3180238639 learned_addr learned my addr 172.21.15.133:0/3180238639 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa8ffff640 1 --2- 172.21.15.133:0/3180238639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed71d680 0x5556ed717530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.081+0000 7faa8f7fe640 1 --2- 172.21.15.133:0/3180238639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed7124a0 0x5556ed734d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.082+0000 7faa8effd640 1 -- 172.21.15.133:0/3180238639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed7124a0 msgr2=0x5556ed734d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.082+0000 7faa8effd640 1 --2- 172.21.15.133:0/3180238639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed7124a0 0x5556ed734d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.082+0000 7faa8effd640 1 -- 172.21.15.133:0/3180238639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed71d680 msgr2=0x5556ed717530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:59.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.082+0000 7faa8effd640 1 --2- 172.21.15.133:0/3180238639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed71d680 0x5556ed717530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.082+0000 7faa8effd640 1 -- 172.21.15.133:0/3180238639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ed71abe0 con 0x5556ed712dc0 2024-09-11T23:36:59.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.082+0000 7faa8effd640 1 --2- 172.21.15.133:0/3180238639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed712dc0 0x5556ed735280 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7faa84097e20 tx=0x7faa840bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a841e5945cc6c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:59.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.082+0000 7faa757fa640 1 -- 172.21.15.133:0/3180238639 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84083070 con 0x5556ed712dc0 2024-09-11T23:36:59.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.082+0000 7faa757fa640 1 -- 172.21.15.133:0/3180238639 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa840046c0 con 0x5556ed712dc0 2024-09-11T23:36:59.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.082+0000 7faa757fa640 1 -- 172.21.15.133:0/3180238639 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84020040 con 0x5556ed712dc0 2024-09-11T23:36:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.083+0000 7faa98349740 1 -- 172.21.15.133:0/3180238639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed712dc0 msgr2=0x5556ed735280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.083+0000 7faa98349740 1 --2- 172.21.15.133:0/3180238639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed712dc0 0x5556ed735280 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7faa84097e20 tx=0x7faa840bb040 comp rx=0 tx=0).stop 2024-09-11T23:36:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.083+0000 7faa98349740 1 -- 172.21.15.133:0/3180238639 shutdown_connections 2024-09-11T23:36:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.083+0000 7faa98349740 1 --2- 172.21.15.133:0/3180238639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed71d680 0x5556ed717530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.083+0000 7faa98349740 1 --2- 172.21.15.133:0/3180238639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed712dc0 0x5556ed735280 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.083+0000 7faa98349740 1 --2- 172.21.15.133:0/3180238639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed7124a0 0x5556ed734d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.083+0000 7faa98349740 1 -- 172.21.15.133:0/3180238639 >> 172.21.15.133:0/3180238639 conn(0x5556ed70ea30 msgr2=0x5556ed70db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.083+0000 7faa98349740 1 -- 172.21.15.133:0/3180238639 shutdown_connections 2024-09-11T23:36:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.083+0000 7faa98349740 1 -- 172.21.15.133:0/3180238639 wait complete. 2024-09-11T23:36:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.083+0000 7faa98349740 1 -- start start 2024-09-11T23:36:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.083+0000 7faa98349740 1 Processor -- start 2024-09-11T23:36:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa98349740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed723450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa98349740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed712dc0 0x5556ed723990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa98349740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed71d680 0x5556ed723ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa98349740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faa901b0d20 con 0x5556ed7124a0 2024-09-11T23:36:59.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa98349740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa901b0650 con 0x5556ed71d680 2024-09-11T23:36:59.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa98349740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faa901ad890 con 0x5556ed712dc0 2024-09-11T23:36:59.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa98349740 10 client.0 _ll_register_callbacks cb 0x7faa80009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:36:59.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed723450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa8f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed723450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33438/0 (socket says 172.21.15.133:33438) 2024-09-11T23:36:59.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa8f7fe640 1 -- 172.21.15.133:0/4080679478 learned_addr learned my addr 172.21.15.133:0/4080679478 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:36:59.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa8effd640 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed712dc0 0x5556ed723990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa8ffff640 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed71d680 0x5556ed723ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa8f7fe640 1 -- 172.21.15.133:0/4080679478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed712dc0 msgr2=0x5556ed723990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa8f7fe640 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed712dc0 0x5556ed723990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa8f7fe640 1 -- 172.21.15.133:0/4080679478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed71d680 msgr2=0x5556ed723ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa8f7fe640 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed71d680 0x5556ed723ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa8f7fe640 1 -- 172.21.15.133:0/4080679478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ed796da0 con 0x5556ed7124a0 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa8f7fe640 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed723450 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faa8002a9a0 tx=0x7faa80050c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16805527acac6e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa6effd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.084+0000 7faa6effd640 1 -- 172.21.15.133:0/4080679478 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80055060 con 0x5556ed7124a0 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa6effd640 1 -- 172.21.15.133:0/4080679478 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7faa80003ba0 con 0x5556ed7124a0 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa98349740 1 -- 172.21.15.133:0/4080679478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556ed67fd60 con 0x5556ed7124a0 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa6effd640 1 -- 172.21.15.133:0/4080679478 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa80035070 con 0x5556ed7124a0 2024-09-11T23:36:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa98349740 10 client.15459 fetch_fsmap learned FSMap version 28 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa98349740 1 -- 172.21.15.133:0/4080679478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556ed7980f0 con 0x5556ed7124a0 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa6effd640 1 -- 172.21.15.133:0/4080679478 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5556ed67fd60 con 0x5556ed7124a0 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa6effd640 1 -- 172.21.15.133:0/4080679478 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faa80058020 con 0x5556ed7124a0 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa98349740 10 client.15459 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa98349740 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa98349740 20 client.15459 populate_metadata read hostname 'smithi133' 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa98349740 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ed798840 0x5556ed79ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa98349740 4 client.15459 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa98349740 1 -- 172.21.15.133:0/4080679478 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5556ed5fc210 con 0x5556ed798840 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.085+0000 7faa8effd640 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ed798840 0x5556ed79ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa6effd640 10 client.15459 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa8effd640 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ed798840 0x5556ed79ac60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf5ff25d6ac4b905 server_cookie=61486b29adc6e287 in_seq=0 out_seq=0 2024-09-11T23:36:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa6effd640 1 -- 172.21.15.133:0/4080679478 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5556ed5fc210 con 0x5556ed798840 2024-09-11T23:36:59.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa6effd640 10 client.15459 handle_command_reply: tid=0 2024-09-11T23:36:59.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa98349740 1 client.15459 shutdown 2024-09-11T23:36:59.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa98349740 1 -- 172.21.15.133:0/4080679478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 msgr2=0x5556ed723450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:59.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa98349740 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed723450 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faa8002a9a0 tx=0x7faa80050c60 comp rx=0 tx=0).stop 2024-09-11T23:36:59.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa98349740 1 -- 172.21.15.133:0/4080679478 shutdown_connections 2024-09-11T23:36:59.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa98349740 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ed798840 0x5556ed79ac60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa6effd640 0 client.15459 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:36:59.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa98349740 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ed71d680 0x5556ed723ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa98349740 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ed712dc0 0x5556ed723990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa98349740 1 --2- 172.21.15.133:0/4080679478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ed7124a0 0x5556ed723450 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa98349740 1 -- 172.21.15.133:0/4080679478 >> 172.21.15.133:0/4080679478 conn(0x5556ed70ea30 msgr2=0x5556ed71bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:59.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa98349740 1 -- 172.21.15.133:0/4080679478 shutdown_connections 2024-09-11T23:36:59.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.086+0000 7faa98349740 1 -- 172.21.15.133:0/4080679478 wait complete. 2024-09-11T23:36:59.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa98349740 20 client.15459 trim_cache size 0 max 16384 2024-09-11T23:36:59.094 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:36:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:36:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-09-11T23:36:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "13415344-3fbf-464e-8ad1-a94bf5179db4": { 2024-09-11T23:36:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-09-11T23:36:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "13415344-3fbf-464e-8ad1-a94bf5179db4", 2024-09-11T23:36:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2024-09-11T23:36:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:36:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:36:59.095 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:36:59.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 -- 172.21.15.133:0/1390986276 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faa780a2e10 msgr2=0x7faa780a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:59.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faa780a2e10 0x7faa780a52d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa88010c40 tx=0x7faa8800c000 comp rx=0 tx=0).stop 2024-09-11T23:36:59.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 -- 172.21.15.133:0/1390986276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa90102570 msgr2=0x7faa90103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:59.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa90102570 0x7faa90103c90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faa8400d770 tx=0x7faa8400dc30 comp rx=0 tx=0).stop 2024-09-11T23:36:59.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 -- 172.21.15.133:0/1390986276 shutdown_connections 2024-09-11T23:36:59.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faa780a2e10 0x7faa780a52d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa90102ef0 0x7faa901041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa90102570 0x7faa90103c90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 --2- 172.21.15.133:0/1390986276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa90100e20 0x7faa90103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 -- 172.21.15.133:0/1390986276 >> 172.21.15.133:0/1390986276 conn(0x7faa900fc7d0 msgr2=0x7faa9006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:59.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 -- 172.21.15.133:0/1390986276 shutdown_connections 2024-09-11T23:36:59.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.087+0000 7faa95f70640 1 -- 172.21.15.133:0/1390986276 wait complete. 2024-09-11T23:36:59.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.118+0000 7ff6a2731640 1 -- 172.21.15.133:0/3080931817 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff69c103f20 con 0x7ff6840a2ec0 2024-09-11T23:36:59.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.125+0000 7ff6997fa640 1 -- 172.21.15.133:0/3080931817 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff69c103f20 con 0x7ff6840a2ec0 2024-09-11T23:36:59.125 INFO:teuthology.orchestra.run.smithi133.stdout:/ 1m 6m3h 2024-09-11T23:36:59.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.127+0000 7ff6a2731640 1 -- 172.21.15.133:0/3080931817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6840a2ec0 msgr2=0x7ff6840a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:59.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.127+0000 7ff6a2731640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6840a2ec0 0x7ff6840a5380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006fd0 tx=0x7ff68c008040 comp rx=0 tx=0).stop 2024-09-11T23:36:59.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.127+0000 7ff6a2731640 1 -- 172.21.15.133:0/3080931817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c101350 msgr2=0x7ff69c197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:36:59.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.127+0000 7ff6a2731640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c101350 0x7ff69c197820 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e9f0 tx=0x7ff69000edf0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.127+0000 7ff6a2731640 1 -- 172.21.15.133:0/3080931817 shutdown_connections 2024-09-11T23:36:59.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.127+0000 7ff6a2731640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6840a2ec0 0x7ff6840a5380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.127+0000 7ff6a2731640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff69c1069c0 0x7ff69c19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.127+0000 7ff6a2731640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c101350 0x7ff69c197820 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.127+0000 7ff6a2731640 1 --2- 172.21.15.133:0/3080931817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff69c1009b0 0x7ff69c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:36:59.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.127+0000 7ff6a2731640 1 -- 172.21.15.133:0/3080931817 >> 172.21.15.133:0/3080931817 conn(0x7ff69c077990 msgr2=0x7ff69c103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:36:59.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.128+0000 7ff6a2731640 1 -- 172.21.15.133:0/3080931817 shutdown_connections 2024-09-11T23:36:59.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:36:59.128+0000 7ff6a2731640 1 -- 172.21.15.133:0/3080931817 wait complete. 2024-09-11T23:36:59.138 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-11T23:36:59.165 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-11T23:36:59.183 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-11T23:36:59.201 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T23:36:59.202 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-11T23:36:59.207 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-11T23:36:59.220 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:36:59.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:36:59.438 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:59 smithi133 ceph-mon[31430]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.3 KiB/s wr, 20 op/s 2024-09-11T23:36:59.438 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:36:59 smithi133 ceph-mon[31430]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:59.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:59 smithi138 ceph-mon[35142]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.3 KiB/s wr, 20 op/s 2024-09-11T23:36:59.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:36:59 smithi138 ceph-mon[35142]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:59.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:59 smithi178 ceph-mon[35882]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.3 KiB/s wr, 20 op/s 2024-09-11T23:36:59.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:36:59 smithi178 ceph-mon[35882]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:36:59.798 INFO:tasks.cephfs.filesystem:scrub status for tag:13415344-3fbf-464e-8ad1-a94bf5179db4 - {'path': '/', 'tag': '13415344-3fbf-464e-8ad1-a94bf5179db4', 'options': 'recursive,force'} 2024-09-11T23:36:59.900 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:00.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:00 smithi133 ceph-mon[31430]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:37:00.323 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:00 smithi133 ceph-mon[31430]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:37:00.324 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:00 smithi133 ceph-mon[31430]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 35 op/s 2024-09-11T23:37:00.324 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:00 smithi133 ceph-mon[31430]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:37:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.404+0000 7fbc17a80640 1 -- 172.21.15.133:0/2080417642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc10101220 msgr2=0x7fbc10101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.404+0000 7fbc17a80640 1 --2- 172.21.15.133:0/2080417642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc10101220 0x7fbc10101660 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009900 tx=0x7fbc0002a410 comp rx=0 tx=0).stop 2024-09-11T23:37:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.404+0000 7fbc17a80640 1 -- 172.21.15.133:0/2080417642 shutdown_connections 2024-09-11T23:37:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.404+0000 7fbc17a80640 1 --2- 172.21.15.133:0/2080417642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc10101ba0 0x7fbc1010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.404+0000 7fbc17a80640 1 --2- 172.21.15.133:0/2080417642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc10101220 0x7fbc10101660 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.404+0000 7fbc17a80640 1 --2- 172.21.15.133:0/2080417642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc10105a70 0x7fbc10105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.404+0000 7fbc17a80640 1 -- 172.21.15.133:0/2080417642 >> 172.21.15.133:0/2080417642 conn(0x7fbc100fbee0 msgr2=0x7fbc100fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.404+0000 7fbc17a80640 1 -- 172.21.15.133:0/2080417642 shutdown_connections 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.404+0000 7fbc17a80640 1 -- 172.21.15.133:0/2080417642 wait complete. 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc17a80640 1 Processor -- start 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc17a80640 1 -- start start 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc17a80640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc10101220 0x7fbc10198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc17a80640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc10101ba0 0x7fbc10198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc17a80640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc10105a70 0x7fbc1019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc17a80640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10112070 con 0x7fbc10101220 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc17a80640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10111ef0 con 0x7fbc10101ba0 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc17a80640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101121f0 con 0x7fbc10105a70 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc1581c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc10101220 0x7fbc10198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc1581c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc10101220 0x7fbc10198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33452/0 (socket says 172.21.15.133:33452) 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc1581c640 1 -- 172.21.15.133:0/2563188302 learned_addr learned my addr 172.21.15.133:0/2563188302 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc1501b640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc10101ba0 0x7fbc10198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc1581c640 1 -- 172.21.15.133:0/2563188302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc10105a70 msgr2=0x7fbc1019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc1601d640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc10105a70 0x7fbc1019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc1581c640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc10105a70 0x7fbc1019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc1581c640 1 -- 172.21.15.133:0/2563188302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc10101ba0 msgr2=0x7fbc10198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc1581c640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc10101ba0 0x7fbc10198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc1581c640 1 -- 172.21.15.133:0/2563188302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1019d400 con 0x7fbc10101220 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.405+0000 7fbc1581c640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc10101220 0x7fbc10198450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c920 tx=0x7fbc0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.406+0000 7fbc06ffd640 1 -- 172.21.15.133:0/2563188302 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c007c70 con 0x7fbc10101220 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.406+0000 7fbc17a80640 1 -- 172.21.15.133:0/2563188302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1019d6f0 con 0x7fbc10101220 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.406+0000 7fbc06ffd640 1 -- 172.21.15.133:0/2563188302 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc0c007e10 con 0x7fbc10101220 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.406+0000 7fbc06ffd640 1 -- 172.21.15.133:0/2563188302 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c018890 con 0x7fbc10101220 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.406+0000 7fbc17a80640 1 -- 172.21.15.133:0/2563188302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc101a4fd0 con 0x7fbc10101220 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.407+0000 7fbc17a80640 1 -- 172.21.15.133:0/2563188302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc10101660 con 0x7fbc10101220 2024-09-11T23:37:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.408+0000 7fbc06ffd640 1 -- 172.21.15.133:0/2563188302 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fbc0c018a70 con 0x7fbc10101220 2024-09-11T23:37:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.408+0000 7fbc06ffd640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbbfc0a2e10 0x7fbbfc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.408+0000 7fbc1501b640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbbfc0a2e10 0x7fbbfc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.408+0000 7fbc06ffd640 1 -- 172.21.15.133:0/2563188302 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fbc0c0cf890 con 0x7fbc10101220 2024-09-11T23:37:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.409+0000 7fbc1501b640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbbfc0a2e10 0x7fbbfc0a52d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc100fe430 tx=0x7fbc00005a20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.412+0000 7fbc06ffd640 1 -- 172.21.15.133:0/2563188302 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc0c014030 con 0x7fbc10101220 2024-09-11T23:37:00.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:00 smithi138 ceph-mon[35142]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:37:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:00 smithi138 ceph-mon[35142]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:37:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:00 smithi138 ceph-mon[35142]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 35 op/s 2024-09-11T23:37:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:00 smithi138 ceph-mon[35142]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:37:00.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:00 smithi178 ceph-mon[35882]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:37:00.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:00 smithi178 ceph-mon[35882]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:37:00.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:00 smithi178 ceph-mon[35882]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 60 KiB/s wr, 35 op/s 2024-09-11T23:37:00.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:00 smithi178 ceph-mon[35882]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:37:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.592+0000 7fbc17a80640 1 -- 172.21.15.133:0/2563188302 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc100fccc0 con 0x7fbbfc0a2e10 2024-09-11T23:37:00.592 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:37:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:37:00.467+0000 7f05102cf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:37:00.592 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:37:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:37:00.467+0000 7f05102cf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:37:00.592 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:37:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:37:00.467+0000 7f05102cf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:37:00.592 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:37:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:37:00.467+0000 7f05102cf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:37:00.593 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:37:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:37:00.467+0000 7f05102cf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:37:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.599+0000 7fbc06ffd640 1 -- 172.21.15.133:0/2563188302 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257809 (secure 0 0 0) 0x7fbc100fccc0 con 0x7fbbfc0a2e10 2024-09-11T23:37:00.601 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:37:00.603 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T23:37:00.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 -- 172.21.15.133:0/2563188302 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbbfc0a2e10 msgr2=0x7fbbfc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:00.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbbfc0a2e10 0x7fbbfc0a52d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc100fe430 tx=0x7fbc00005a20 comp rx=0 tx=0).stop 2024-09-11T23:37:00.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 -- 172.21.15.133:0/2563188302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc10101220 msgr2=0x7fbc10198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:00.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc10101220 0x7fbc10198450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c920 tx=0x7fbc0c00cde0 comp rx=0 tx=0).stop 2024-09-11T23:37:00.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 -- 172.21.15.133:0/2563188302 shutdown_connections 2024-09-11T23:37:00.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbbfc0a2e10 0x7fbbfc0a52d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:00.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc10105a70 0x7fbc1019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:00.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc10101ba0 0x7fbc10198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:00.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 --2- 172.21.15.133:0/2563188302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc10101220 0x7fbc10198450 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:00.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 -- 172.21.15.133:0/2563188302 >> 172.21.15.133:0/2563188302 conn(0x7fbc100fbee0 msgr2=0x7fbc100fc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:00.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 -- 172.21.15.133:0/2563188302 shutdown_connections 2024-09-11T23:37:00.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:00.603+0000 7fbc17a80640 1 -- 172.21.15.133:0/2563188302 wait complete. 2024-09-11T23:37:01.297 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:01 smithi133 ceph-mon[31430]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:37:01.327 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-11T23:36:59.070764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1392693,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1622,"num_read_kb":1183,"num_write":650,"num_write_kb":5068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1001,"ondisk_log_size":1001,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359520,"kb_used_data":14400,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582304,"statfs":{"total":1151940427776,"available":1151572279296,"internally_reserved":0,"allocated":14745600,"data_stored":8892639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":204645,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":361,"num_write":86,"num_write_kb":664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.990694"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327868+0000","last_change":"2024-09-11T23:34:52.867694+0000","last_active":"2024-09-11T23:36:47.327868+0000","last_peered":"2024-09-11T23:36:47.327868+0000","last_clean":"2024-09-11T23:36:47.327868+0000","last_became_active":"2024-09-11T23:34:52.867368+0000","last_became_peered":"2024-09-11T23:34:52.867368+0000","last_unstale":"2024-09-11T23:36:47.327868+0000","last_undegraded":"2024-09-11T23:36:47.327868+0000","last_fullsized":"2024-09-11T23:36:47.327868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:46:07.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328024+0000","last_change":"2024-09-11T23:34:52.871881+0000","last_active":"2024-09-11T23:36:47.328024+0000","last_peered":"2024-09-11T23:36:47.328024+0000","last_clean":"2024-09-11T23:36:47.328024+0000","last_became_active":"2024-09-11T23:34:52.871609+0000","last_became_peered":"2024-09-11T23:34:52.871609+0000","last_unstale":"2024-09-11T23:36:47.328024+0000","last_undegraded":"2024-09-11T23:36:47.328024+0000","last_fullsized":"2024-09-11T23:36:47.328024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.938584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327774+0000","last_change":"2024-09-11T23:34:52.863931+0000","last_active":"2024-09-11T23:36:47.327774+0000","last_peered":"2024-09-11T23:36:47.327774+0000","last_clean":"2024-09-11T23:36:47.327774+0000","last_became_active":"2024-09-11T23:34:52.863637+0000","last_became_peered":"2024-09-11T23:34:52.863637+0000","last_unstale":"2024-09-11T23:36:47.327774+0000","last_undegraded":"2024-09-11T23:36:47.327774+0000","last_fullsized":"2024-09-11T23:36:47.327774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:44.118865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292453+0000","last_change":"2024-09-11T23:34:52.879619+0000","last_active":"2024-09-11T23:36:47.292453+0000","last_peered":"2024-09-11T23:36:47.292453+0000","last_clean":"2024-09-11T23:36:47.292453+0000","last_became_active":"2024-09-11T23:34:52.878695+0000","last_became_peered":"2024-09-11T23:34:52.878695+0000","last_unstale":"2024-09-11T23:36:47.292453+0000","last_undegraded":"2024-09-11T23:36:47.292453+0000","last_fullsized":"2024-09-11T23:36:47.292453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:30.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145593+0000","last_change":"2024-09-11T23:34:52.869363+0000","last_active":"2024-09-11T23:36:47.145593+0000","last_peered":"2024-09-11T23:36:47.145593+0000","last_clean":"2024-09-11T23:36:47.145593+0000","last_became_active":"2024-09-11T23:34:52.869027+0000","last_became_peered":"2024-09-11T23:34:52.869027+0000","last_unstale":"2024-09-11T23:36:47.145593+0000","last_undegraded":"2024-09-11T23:36:47.145593+0000","last_fullsized":"2024-09-11T23:36:47.145593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:15.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156014+0000","last_change":"2024-09-11T23:34:52.880665+0000","last_active":"2024-09-11T23:36:47.156014+0000","last_peered":"2024-09-11T23:36:47.156014+0000","last_clean":"2024-09-11T23:36:47.156014+0000","last_became_active":"2024-09-11T23:34:52.880172+0000","last_became_peered":"2024-09-11T23:34:52.880172+0000","last_unstale":"2024-09-11T23:36:47.156014+0000","last_undegraded":"2024-09-11T23:36:47.156014+0000","last_fullsized":"2024-09-11T23:36:47.156014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:12.025877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152489+0000","last_change":"2024-09-11T23:34:52.873897+0000","last_active":"2024-09-11T23:36:47.152489+0000","last_peered":"2024-09-11T23:36:47.152489+0000","last_clean":"2024-09-11T23:36:47.152489+0000","last_became_active":"2024-09-11T23:34:52.873219+0000","last_became_peered":"2024-09-11T23:34:52.873219+0000","last_unstale":"2024-09-11T23:36:47.152489+0000","last_undegraded":"2024-09-11T23:36:47.152489+0000","last_fullsized":"2024-09-11T23:36:47.152489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:47.125682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145794+0000","last_change":"2024-09-11T23:34:52.870699+0000","last_active":"2024-09-11T23:36:47.145794+0000","last_peered":"2024-09-11T23:36:47.145794+0000","last_clean":"2024-09-11T23:36:47.145794+0000","last_became_active":"2024-09-11T23:34:52.870314+0000","last_became_peered":"2024-09-11T23:34:52.870314+0000","last_unstale":"2024-09-11T23:36:47.145794+0000","last_undegraded":"2024-09-11T23:36:47.145794+0000","last_fullsized":"2024-09-11T23:36:47.145794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:41.263159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058666+0000","last_change":"2024-09-11T23:34:52.869700+0000","last_active":"2024-09-11T23:36:47.058666+0000","last_peered":"2024-09-11T23:36:47.058666+0000","last_clean":"2024-09-11T23:36:47.058666+0000","last_became_active":"2024-09-11T23:34:52.869398+0000","last_became_peered":"2024-09-11T23:34:52.869398+0000","last_unstale":"2024-09-11T23:36:47.058666+0000","last_undegraded":"2024-09-11T23:36:47.058666+0000","last_fullsized":"2024-09-11T23:36:47.058666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:49.776525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156161+0000","last_change":"2024-09-11T23:34:52.879999+0000","last_active":"2024-09-11T23:36:47.156161+0000","last_peered":"2024-09-11T23:36:47.156161+0000","last_clean":"2024-09-11T23:36:47.156161+0000","last_became_active":"2024-09-11T23:34:52.879646+0000","last_became_peered":"2024-09-11T23:34:52.879646+0000","last_unstale":"2024-09-11T23:36:47.156161+0000","last_undegraded":"2024-09-11T23:36:47.156161+0000","last_fullsized":"2024-09-11T23:36:47.156161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.058704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152665+0000","last_change":"2024-09-11T23:34:52.878740+0000","last_active":"2024-09-11T23:36:47.152665+0000","last_peered":"2024-09-11T23:36:47.152665+0000","last_clean":"2024-09-11T23:36:47.152665+0000","last_became_active":"2024-09-11T23:34:52.878120+0000","last_became_peered":"2024-09-11T23:34:52.878120+0000","last_unstale":"2024-09-11T23:36:47.152665+0000","last_undegraded":"2024-09-11T23:36:47.152665+0000","last_fullsized":"2024-09-11T23:36:47.152665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292447+0000","last_change":"2024-09-11T23:34:52.873409+0000","last_active":"2024-09-11T23:36:47.292447+0000","last_peered":"2024-09-11T23:36:47.292447+0000","last_clean":"2024-09-11T23:36:47.292447+0000","last_became_active":"2024-09-11T23:34:52.872959+0000","last_became_peered":"2024-09-11T23:34:52.872959+0000","last_unstale":"2024-09-11T23:36:47.292447+0000","last_undegraded":"2024-09-11T23:36:47.292447+0000","last_fullsized":"2024-09-11T23:36:47.292447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:38.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057362+0000","last_change":"2024-09-11T23:34:52.869164+0000","last_active":"2024-09-11T23:36:47.057362+0000","last_peered":"2024-09-11T23:36:47.057362+0000","last_clean":"2024-09-11T23:36:47.057362+0000","last_became_active":"2024-09-11T23:34:52.868847+0000","last_became_peered":"2024-09-11T23:34:52.868847+0000","last_unstale":"2024-09-11T23:36:47.057362+0000","last_undegraded":"2024-09-11T23:36:47.057362+0000","last_fullsized":"2024-09-11T23:36:47.057362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.764510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327562+0000","last_change":"2024-09-11T23:34:52.869494+0000","last_active":"2024-09-11T23:36:47.327562+0000","last_peered":"2024-09-11T23:36:47.327562+0000","last_clean":"2024-09-11T23:36:47.327562+0000","last_became_active":"2024-09-11T23:34:52.869178+0000","last_became_peered":"2024-09-11T23:34:52.869178+0000","last_unstale":"2024-09-11T23:36:47.327562+0000","last_undegraded":"2024-09-11T23:36:47.327562+0000","last_fullsized":"2024-09-11T23:36:47.327562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:23.431564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327741+0000","last_change":"2024-09-11T23:34:52.866269+0000","last_active":"2024-09-11T23:36:47.327741+0000","last_peered":"2024-09-11T23:36:47.327741+0000","last_clean":"2024-09-11T23:36:47.327741+0000","last_became_active":"2024-09-11T23:34:52.865980+0000","last_became_peered":"2024-09-11T23:34:52.865980+0000","last_unstale":"2024-09-11T23:36:47.327741+0000","last_undegraded":"2024-09-11T23:36:47.327741+0000","last_fullsized":"2024-09-11T23:36:47.327741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:48.492252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058174+0000","last_change":"2024-09-11T23:34:52.862192+0000","last_active":"2024-09-11T23:36:47.058174+0000","last_peered":"2024-09-11T23:36:47.058174+0000","last_clean":"2024-09-11T23:36:47.058174+0000","last_became_active":"2024-09-11T23:34:52.861841+0000","last_became_peered":"2024-09-11T23:34:52.861841+0000","last_unstale":"2024-09-11T23:36:47.058174+0000","last_undegraded":"2024-09-11T23:36:47.058174+0000","last_fullsized":"2024-09-11T23:36:47.058174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.258948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327836+0000","last_change":"2024-09-11T23:34:52.870984+0000","last_active":"2024-09-11T23:36:47.327836+0000","last_peered":"2024-09-11T23:36:47.327836+0000","last_clean":"2024-09-11T23:36:47.327836+0000","last_became_active":"2024-09-11T23:34:52.870435+0000","last_became_peered":"2024-09-11T23:34:52.870435+0000","last_unstale":"2024-09-11T23:36:47.327836+0000","last_undegraded":"2024-09-11T23:36:47.327836+0000","last_fullsized":"2024-09-11T23:36:47.327836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:34.391537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143297+0000","last_change":"2024-09-11T23:34:52.873176+0000","last_active":"2024-09-11T23:36:47.143297+0000","last_peered":"2024-09-11T23:36:47.143297+0000","last_clean":"2024-09-11T23:36:47.143297+0000","last_became_active":"2024-09-11T23:34:52.872862+0000","last_became_peered":"2024-09-11T23:34:52.872862+0000","last_unstale":"2024-09-11T23:36:47.143297+0000","last_undegraded":"2024-09-11T23:36:47.143297+0000","last_fullsized":"2024-09-11T23:36:47.143297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:38.058806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058317+0000","last_change":"2024-09-11T23:34:52.871346+0000","last_active":"2024-09-11T23:36:47.058317+0000","last_peered":"2024-09-11T23:36:47.058317+0000","last_clean":"2024-09-11T23:36:47.058317+0000","last_became_active":"2024-09-11T23:34:52.870981+0000","last_became_peered":"2024-09-11T23:34:52.870981+0000","last_unstale":"2024-09-11T23:36:47.058317+0000","last_undegraded":"2024-09-11T23:36:47.058317+0000","last_fullsized":"2024-09-11T23:36:47.058317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:56.973998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327601+0000","last_change":"2024-09-11T23:34:52.867058+0000","last_active":"2024-09-11T23:36:47.327601+0000","last_peered":"2024-09-11T23:36:47.327601+0000","last_clean":"2024-09-11T23:36:47.327601+0000","last_became_active":"2024-09-11T23:34:52.866687+0000","last_became_peered":"2024-09-11T23:34:52.866687+0000","last_unstale":"2024-09-11T23:36:47.327601+0000","last_undegraded":"2024-09-11T23:36:47.327601+0000","last_fullsized":"2024-09-11T23:36:47.327601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:09.019987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057475+0000","last_change":"2024-09-11T23:34:52.870882+0000","last_active":"2024-09-11T23:36:47.057475+0000","last_peered":"2024-09-11T23:36:47.057475+0000","last_clean":"2024-09-11T23:36:47.057475+0000","last_became_active":"2024-09-11T23:34:52.870667+0000","last_became_peered":"2024-09-11T23:34:52.870667+0000","last_unstale":"2024-09-11T23:36:47.057475+0000","last_undegraded":"2024-09-11T23:36:47.057475+0000","last_fullsized":"2024-09-11T23:36:47.057475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:08.386703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058304+0000","last_change":"2024-09-11T23:34:52.874070+0000","last_active":"2024-09-11T23:36:47.058304+0000","last_peered":"2024-09-11T23:36:47.058304+0000","last_clean":"2024-09-11T23:36:47.058304+0000","last_became_active":"2024-09-11T23:34:52.873772+0000","last_became_peered":"2024-09-11T23:34:52.873772+0000","last_unstale":"2024-09-11T23:36:47.058304+0000","last_undegraded":"2024-09-11T23:36:47.058304+0000","last_fullsized":"2024-09-11T23:36:47.058304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:49.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146028+0000","last_change":"2024-09-11T23:34:52.875502+0000","last_active":"2024-09-11T23:36:47.146028+0000","last_peered":"2024-09-11T23:36:47.146028+0000","last_clean":"2024-09-11T23:36:47.146028+0000","last_became_active":"2024-09-11T23:34:52.875193+0000","last_became_peered":"2024-09-11T23:34:52.875193+0000","last_unstale":"2024-09-11T23:36:47.146028+0000","last_undegraded":"2024-09-11T23:36:47.146028+0000","last_fullsized":"2024-09-11T23:36:47.146028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:37.213828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153497+0000","last_change":"2024-09-11T23:34:52.869015+0000","last_active":"2024-09-11T23:36:47.153497+0000","last_peered":"2024-09-11T23:36:47.153497+0000","last_clean":"2024-09-11T23:36:47.153497+0000","last_became_active":"2024-09-11T23:34:52.868731+0000","last_became_peered":"2024-09-11T23:34:52.868731+0000","last_unstale":"2024-09-11T23:36:47.153497+0000","last_undegraded":"2024-09-11T23:36:47.153497+0000","last_fullsized":"2024-09-11T23:36:47.153497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:12:15.993203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147366+0000","last_change":"2024-09-11T23:34:52.875555+0000","last_active":"2024-09-11T23:36:47.147366+0000","last_peered":"2024-09-11T23:36:47.147366+0000","last_clean":"2024-09-11T23:36:47.147366+0000","last_became_active":"2024-09-11T23:34:52.875219+0000","last_became_peered":"2024-09-11T23:34:52.875219+0000","last_unstale":"2024-09-11T23:36:47.147366+0000","last_undegraded":"2024-09-11T23:36:47.147366+0000","last_fullsized":"2024-09-11T23:36:47.147366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:16.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058480+0000","last_change":"2024-09-11T23:34:52.875629+0000","last_active":"2024-09-11T23:36:47.058480+0000","last_peered":"2024-09-11T23:36:47.058480+0000","last_clean":"2024-09-11T23:36:47.058480+0000","last_became_active":"2024-09-11T23:34:52.875302+0000","last_became_peered":"2024-09-11T23:34:52.875302+0000","last_unstale":"2024-09-11T23:36:47.058480+0000","last_undegraded":"2024-09-11T23:36:47.058480+0000","last_fullsized":"2024-09-11T23:36:47.058480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:12.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327744+0000","last_change":"2024-09-11T23:34:52.868613+0000","last_active":"2024-09-11T23:36:47.327744+0000","last_peered":"2024-09-11T23:36:47.327744+0000","last_clean":"2024-09-11T23:36:47.327744+0000","last_became_active":"2024-09-11T23:34:52.868184+0000","last_became_peered":"2024-09-11T23:34:52.868184+0000","last_unstale":"2024-09-11T23:36:47.327744+0000","last_undegraded":"2024-09-11T23:36:47.327744+0000","last_fullsized":"2024-09-11T23:36:47.327744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:25.990002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057412+0000","last_change":"2024-09-11T23:35:08.873390+0000","last_active":"2024-09-11T23:36:47.057412+0000","last_peered":"2024-09-11T23:36:47.057412+0000","last_clean":"2024-09-11T23:36:47.057412+0000","last_became_active":"2024-09-11T23:35:08.873052+0000","last_became_peered":"2024-09-11T23:35:08.873052+0000","last_unstale":"2024-09-11T23:36:47.057412+0000","last_undegraded":"2024-09-11T23:36:47.057412+0000","last_fullsized":"2024-09-11T23:36:47.057412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.937951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327734+0000","last_change":"2024-09-11T23:34:52.873604+0000","last_active":"2024-09-11T23:36:47.327734+0000","last_peered":"2024-09-11T23:36:47.327734+0000","last_clean":"2024-09-11T23:36:47.327734+0000","last_became_active":"2024-09-11T23:34:52.873220+0000","last_became_peered":"2024-09-11T23:34:52.873220+0000","last_unstale":"2024-09-11T23:36:47.327734+0000","last_undegraded":"2024-09-11T23:36:47.327734+0000","last_fullsized":"2024-09-11T23:36:47.327734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:38.480630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058083+0000","last_change":"2024-09-11T23:34:52.872902+0000","last_active":"2024-09-11T23:36:47.058083+0000","last_peered":"2024-09-11T23:36:47.058083+0000","last_clean":"2024-09-11T23:36:47.058083+0000","last_became_active":"2024-09-11T23:34:52.872571+0000","last_became_peered":"2024-09-11T23:34:52.872571+0000","last_unstale":"2024-09-11T23:36:47.058083+0000","last_undegraded":"2024-09-11T23:36:47.058083+0000","last_fullsized":"2024-09-11T23:36:47.058083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:21.844995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058162+0000","last_change":"2024-09-11T23:35:09.457056+0000","last_active":"2024-09-11T23:36:47.058162+0000","last_peered":"2024-09-11T23:36:47.058162+0000","last_clean":"2024-09-11T23:36:47.058162+0000","last_became_active":"2024-09-11T23:35:09.456657+0000","last_became_peered":"2024-09-11T23:35:09.456657+0000","last_unstale":"2024-09-11T23:36:47.058162+0000","last_undegraded":"2024-09-11T23:36:47.058162+0000","last_fullsized":"2024-09-11T23:36:47.058162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:12:56.121280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327534+0000","last_change":"2024-09-11T23:34:52.875312+0000","last_active":"2024-09-11T23:36:47.327534+0000","last_peered":"2024-09-11T23:36:47.327534+0000","last_clean":"2024-09-11T23:36:47.327534+0000","last_became_active":"2024-09-11T23:34:52.874873+0000","last_became_peered":"2024-09-11T23:34:52.874873+0000","last_unstale":"2024-09-11T23:36:47.327534+0000","last_undegraded":"2024-09-11T23:36:47.327534+0000","last_fullsized":"2024-09-11T23:36:47.327534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:18.894713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145537+0000","last_change":"2024-09-11T23:34:52.862266+0000","last_active":"2024-09-11T23:36:47.145537+0000","last_peered":"2024-09-11T23:36:47.145537+0000","last_clean":"2024-09-11T23:36:47.145537+0000","last_became_active":"2024-09-11T23:34:52.861923+0000","last_became_peered":"2024-09-11T23:34:52.861923+0000","last_unstale":"2024-09-11T23:36:47.145537+0000","last_undegraded":"2024-09-11T23:36:47.145537+0000","last_fullsized":"2024-09-11T23:36:47.145537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:23.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":148,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.603783+0000","last_change":"2024-09-11T23:34:50.854020+0000","last_active":"2024-09-11T23:36:56.603783+0000","last_peered":"2024-09-11T23:36:56.603783+0000","last_clean":"2024-09-11T23:36:56.603783+0000","last_became_active":"2024-09-11T23:34:50.853683+0000","last_became_peered":"2024-09-11T23:34:50.853683+0000","last_unstale":"2024-09-11T23:36:56.603783+0000","last_undegraded":"2024-09-11T23:36:56.603783+0000","last_fullsized":"2024-09-11T23:36:56.603783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:18.725495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153611+0000","last_change":"2024-09-11T23:34:52.874016+0000","last_active":"2024-09-11T23:36:47.153611+0000","last_peered":"2024-09-11T23:36:47.153611+0000","last_clean":"2024-09-11T23:36:47.153611+0000","last_became_active":"2024-09-11T23:34:52.873675+0000","last_became_peered":"2024-09-11T23:34:52.873675+0000","last_unstale":"2024-09-11T23:36:47.153611+0000","last_undegraded":"2024-09-11T23:36:47.153611+0000","last_fullsized":"2024-09-11T23:36:47.153611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:41:28.955548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156082+0000","last_change":"2024-09-11T23:34:50.855391+0000","last_active":"2024-09-11T23:36:47.156082+0000","last_peered":"2024-09-11T23:36:47.156082+0000","last_clean":"2024-09-11T23:36:47.156082+0000","last_became_active":"2024-09-11T23:34:50.855052+0000","last_became_peered":"2024-09-11T23:34:50.855052+0000","last_unstale":"2024-09-11T23:36:47.156082+0000","last_undegraded":"2024-09-11T23:36:47.156082+0000","last_fullsized":"2024-09-11T23:36:47.156082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.972239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057065+0000","last_change":"2024-09-11T23:34:50.848985+0000","last_active":"2024-09-11T23:36:47.057065+0000","last_peered":"2024-09-11T23:36:47.057065+0000","last_clean":"2024-09-11T23:36:47.057065+0000","last_became_active":"2024-09-11T23:34:50.848614+0000","last_became_peered":"2024-09-11T23:34:50.848614+0000","last_unstale":"2024-09-11T23:36:47.057065+0000","last_undegraded":"2024-09-11T23:36:47.057065+0000","last_fullsized":"2024-09-11T23:36:47.057065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:43:04.723136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146018+0000","last_change":"2024-09-11T23:34:52.866933+0000","last_active":"2024-09-11T23:36:47.146018+0000","last_peered":"2024-09-11T23:36:47.146018+0000","last_clean":"2024-09-11T23:36:47.146018+0000","last_became_active":"2024-09-11T23:34:52.866559+0000","last_became_peered":"2024-09-11T23:34:52.866559+0000","last_unstale":"2024-09-11T23:36:47.146018+0000","last_undegraded":"2024-09-11T23:36:47.146018+0000","last_fullsized":"2024-09-11T23:36:47.146018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:27.745118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153745+0000","last_change":"2024-09-11T23:34:50.849787+0000","last_active":"2024-09-11T23:36:47.153745+0000","last_peered":"2024-09-11T23:36:47.153745+0000","last_clean":"2024-09-11T23:36:47.153745+0000","last_became_active":"2024-09-11T23:34:50.849368+0000","last_became_peered":"2024-09-11T23:34:50.849368+0000","last_unstale":"2024-09-11T23:36:47.153745+0000","last_undegraded":"2024-09-11T23:36:47.153745+0000","last_fullsized":"2024-09-11T23:36:47.153745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:26.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142558+0000","last_change":"2024-09-11T23:34:52.870832+0000","last_active":"2024-09-11T23:36:47.142558+0000","last_peered":"2024-09-11T23:36:47.142558+0000","last_clean":"2024-09-11T23:36:47.142558+0000","last_became_active":"2024-09-11T23:34:52.870474+0000","last_became_peered":"2024-09-11T23:34:52.870474+0000","last_unstale":"2024-09-11T23:36:47.142558+0000","last_undegraded":"2024-09-11T23:36:47.142558+0000","last_fullsized":"2024-09-11T23:36:47.142558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:30.196114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":258,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:57.604585+0000","last_change":"2024-09-11T23:34:50.859074+0000","last_active":"2024-09-11T23:36:57.604585+0000","last_peered":"2024-09-11T23:36:57.604585+0000","last_clean":"2024-09-11T23:36:57.604585+0000","last_became_active":"2024-09-11T23:34:50.858664+0000","last_became_peered":"2024-09-11T23:34:50.858664+0000","last_unstale":"2024-09-11T23:36:57.604585+0000","last_undegraded":"2024-09-11T23:36:57.604585+0000","last_fullsized":"2024-09-11T23:36:57.604585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:04.077813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058577+0000","last_change":"2024-09-11T23:34:52.875901+0000","last_active":"2024-09-11T23:36:47.058577+0000","last_peered":"2024-09-11T23:36:47.058577+0000","last_clean":"2024-09-11T23:36:47.058577+0000","last_became_active":"2024-09-11T23:34:52.875568+0000","last_became_peered":"2024-09-11T23:34:52.875568+0000","last_unstale":"2024-09-11T23:36:47.058577+0000","last_undegraded":"2024-09-11T23:36:47.058577+0000","last_fullsized":"2024-09-11T23:36:47.058577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:54.173548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152813+0000","last_change":"2024-09-11T23:34:50.854957+0000","last_active":"2024-09-11T23:36:47.152813+0000","last_peered":"2024-09-11T23:36:47.152813+0000","last_clean":"2024-09-11T23:36:47.152813+0000","last_became_active":"2024-09-11T23:34:50.854626+0000","last_became_peered":"2024-09-11T23:34:50.854626+0000","last_unstale":"2024-09-11T23:36:47.152813+0000","last_undegraded":"2024-09-11T23:36:47.152813+0000","last_fullsized":"2024-09-11T23:36:47.152813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:53.738966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328149+0000","last_change":"2024-09-11T23:34:52.868850+0000","last_active":"2024-09-11T23:36:47.328149+0000","last_peered":"2024-09-11T23:36:47.328149+0000","last_clean":"2024-09-11T23:36:47.328149+0000","last_became_active":"2024-09-11T23:34:52.868536+0000","last_became_peered":"2024-09-11T23:34:52.868536+0000","last_unstale":"2024-09-11T23:36:47.328149+0000","last_undegraded":"2024-09-11T23:36:47.328149+0000","last_fullsized":"2024-09-11T23:36:47.328149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:11.746821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153829+0000","last_change":"2024-09-11T23:34:50.924819+0000","last_active":"2024-09-11T23:36:47.153829+0000","last_peered":"2024-09-11T23:36:47.153829+0000","last_clean":"2024-09-11T23:36:47.153829+0000","last_became_active":"2024-09-11T23:34:50.924283+0000","last_became_peered":"2024-09-11T23:34:50.924283+0000","last_unstale":"2024-09-11T23:36:47.153829+0000","last_undegraded":"2024-09-11T23:36:47.153829+0000","last_fullsized":"2024-09-11T23:36:47.153829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:41.345233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058417+0000","last_change":"2024-09-11T23:34:52.867113+0000","last_active":"2024-09-11T23:36:47.058417+0000","last_peered":"2024-09-11T23:36:47.058417+0000","last_clean":"2024-09-11T23:36:47.058417+0000","last_became_active":"2024-09-11T23:34:52.866125+0000","last_became_peered":"2024-09-11T23:34:52.866125+0000","last_unstale":"2024-09-11T23:36:47.058417+0000","last_undegraded":"2024-09-11T23:36:47.058417+0000","last_fullsized":"2024-09-11T23:36:47.058417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:18:55.615079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'38","reported_seq":307,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:57.604631+0000","last_change":"2024-09-11T23:34:50.857272+0000","last_active":"2024-09-11T23:36:57.604631+0000","last_peered":"2024-09-11T23:36:57.604631+0000","last_clean":"2024-09-11T23:36:57.604631+0000","last_became_active":"2024-09-11T23:34:50.855701+0000","last_became_peered":"2024-09-11T23:34:50.855701+0000","last_unstale":"2024-09-11T23:36:57.604631+0000","last_undegraded":"2024-09-11T23:36:57.604631+0000","last_fullsized":"2024-09-11T23:36:57.604631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:49.997617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70094,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":245,"num_read_kb":191,"num_write":40,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145783+0000","last_change":"2024-09-11T23:34:52.876928+0000","last_active":"2024-09-11T23:36:47.145783+0000","last_peered":"2024-09-11T23:36:47.145783+0000","last_clean":"2024-09-11T23:36:47.145783+0000","last_became_active":"2024-09-11T23:34:52.876578+0000","last_became_peered":"2024-09-11T23:34:52.876578+0000","last_unstale":"2024-09-11T23:36:47.145783+0000","last_undegraded":"2024-09-11T23:36:47.145783+0000","last_fullsized":"2024-09-11T23:36:47.145783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:07.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327515+0000","last_change":"2024-09-11T23:35:08.869275+0000","last_active":"2024-09-11T23:36:47.327515+0000","last_peered":"2024-09-11T23:36:47.327515+0000","last_clean":"2024-09-11T23:36:47.327515+0000","last_became_active":"2024-09-11T23:35:08.850112+0000","last_became_peered":"2024-09-11T23:35:08.850112+0000","last_unstale":"2024-09-11T23:36:47.327515+0000","last_undegraded":"2024-09-11T23:36:47.327515+0000","last_fullsized":"2024-09-11T23:36:47.327515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:36.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057105+0000","last_change":"2024-09-11T23:34:52.871970+0000","last_active":"2024-09-11T23:36:47.057105+0000","last_peered":"2024-09-11T23:36:47.057105+0000","last_clean":"2024-09-11T23:36:47.057105+0000","last_became_active":"2024-09-11T23:34:52.871488+0000","last_became_peered":"2024-09-11T23:34:52.871488+0000","last_unstale":"2024-09-11T23:36:47.057105+0000","last_undegraded":"2024-09-11T23:36:47.057105+0000","last_fullsized":"2024-09-11T23:36:47.057105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:08.266431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153627+0000","last_change":"2024-09-11T23:34:50.854023+0000","last_active":"2024-09-11T23:36:47.153627+0000","last_peered":"2024-09-11T23:36:47.153627+0000","last_clean":"2024-09-11T23:36:47.153627+0000","last_became_active":"2024-09-11T23:34:50.853661+0000","last_became_peered":"2024-09-11T23:34:50.853661+0000","last_unstale":"2024-09-11T23:36:47.153627+0000","last_undegraded":"2024-09-11T23:36:47.153627+0000","last_fullsized":"2024-09-11T23:36:47.153627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:25.281976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152646+0000","last_change":"2024-09-11T23:35:08.849648+0000","last_active":"2024-09-11T23:36:47.152646+0000","last_peered":"2024-09-11T23:36:47.152646+0000","last_clean":"2024-09-11T23:36:47.152646+0000","last_became_active":"2024-09-11T23:35:08.849246+0000","last_became_peered":"2024-09-11T23:35:08.849246+0000","last_unstale":"2024-09-11T23:36:47.152646+0000","last_undegraded":"2024-09-11T23:36:47.152646+0000","last_fullsized":"2024-09-11T23:36:47.152646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:17.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'11","reported_seq":157,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.621527+0000","last_change":"2024-09-11T23:34:50.853855+0000","last_active":"2024-09-11T23:36:56.621527+0000","last_peered":"2024-09-11T23:36:56.621527+0000","last_clean":"2024-09-11T23:36:56.621527+0000","last_became_active":"2024-09-11T23:34:50.853493+0000","last_became_peered":"2024-09-11T23:34:50.853493+0000","last_unstale":"2024-09-11T23:36:56.621527+0000","last_undegraded":"2024-09-11T23:36:56.621527+0000","last_fullsized":"2024-09-11T23:36:56.621527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:23.262472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2609,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":8,"num_write":11,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152450+0000","last_change":"2024-09-11T23:34:52.875835+0000","last_active":"2024-09-11T23:36:47.152450+0000","last_peered":"2024-09-11T23:36:47.152450+0000","last_clean":"2024-09-11T23:36:47.152450+0000","last_became_active":"2024-09-11T23:34:52.875525+0000","last_became_peered":"2024-09-11T23:34:52.875525+0000","last_unstale":"2024-09-11T23:36:47.152450+0000","last_undegraded":"2024-09-11T23:36:47.152450+0000","last_fullsized":"2024-09-11T23:36:47.152450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:41.031816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145473+0000","last_change":"2024-09-11T23:34:50.924803+0000","last_active":"2024-09-11T23:36:47.145473+0000","last_peered":"2024-09-11T23:36:47.145473+0000","last_clean":"2024-09-11T23:36:47.145473+0000","last_became_active":"2024-09-11T23:34:50.924217+0000","last_became_peered":"2024-09-11T23:34:50.924217+0000","last_unstale":"2024-09-11T23:36:47.145473+0000","last_undegraded":"2024-09-11T23:36:47.145473+0000","last_fullsized":"2024-09-11T23:36:47.145473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:45.790592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142408+0000","last_change":"2024-09-11T23:35:08.851856+0000","last_active":"2024-09-11T23:36:47.142408+0000","last_peered":"2024-09-11T23:36:47.142408+0000","last_clean":"2024-09-11T23:36:47.142408+0000","last_became_active":"2024-09-11T23:35:08.851490+0000","last_became_peered":"2024-09-11T23:35:08.851490+0000","last_unstale":"2024-09-11T23:36:47.142408+0000","last_undegraded":"2024-09-11T23:36:47.142408+0000","last_fullsized":"2024-09-11T23:36:47.142408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:46.269022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145857+0000","last_change":"2024-09-11T23:34:50.848385+0000","last_active":"2024-09-11T23:36:47.145857+0000","last_peered":"2024-09-11T23:36:47.145857+0000","last_clean":"2024-09-11T23:36:47.145857+0000","last_became_active":"2024-09-11T23:34:50.847808+0000","last_became_peered":"2024-09-11T23:34:50.847808+0000","last_unstale":"2024-09-11T23:36:47.145857+0000","last_undegraded":"2024-09-11T23:36:47.145857+0000","last_fullsized":"2024-09-11T23:36:47.145857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:38.796622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327734+0000","last_change":"2024-09-11T23:34:52.867341+0000","last_active":"2024-09-11T23:36:47.327734+0000","last_peered":"2024-09-11T23:36:47.327734+0000","last_clean":"2024-09-11T23:36:47.327734+0000","last_became_active":"2024-09-11T23:34:52.866161+0000","last_became_peered":"2024-09-11T23:34:52.866161+0000","last_unstale":"2024-09-11T23:36:47.327734+0000","last_undegraded":"2024-09-11T23:36:47.327734+0000","last_fullsized":"2024-09-11T23:36:47.327734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:27.913705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057317+0000","last_change":"2024-09-11T23:34:50.851466+0000","last_active":"2024-09-11T23:36:47.057317+0000","last_peered":"2024-09-11T23:36:47.057317+0000","last_clean":"2024-09-11T23:36:47.057317+0000","last_became_active":"2024-09-11T23:34:50.851191+0000","last_became_peered":"2024-09-11T23:34:50.851191+0000","last_unstale":"2024-09-11T23:36:47.057317+0000","last_undegraded":"2024-09-11T23:36:47.057317+0000","last_fullsized":"2024-09-11T23:36:47.057317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:00.328658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152878+0000","last_change":"2024-09-11T23:34:52.872290+0000","last_active":"2024-09-11T23:36:47.152878+0000","last_peered":"2024-09-11T23:36:47.152878+0000","last_clean":"2024-09-11T23:36:47.152878+0000","last_became_active":"2024-09-11T23:34:52.871935+0000","last_became_peered":"2024-09-11T23:34:52.871935+0000","last_unstale":"2024-09-11T23:36:47.152878+0000","last_undegraded":"2024-09-11T23:36:47.152878+0000","last_fullsized":"2024-09-11T23:36:47.152878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:08.217548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'59","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:58.069867+0000","last_change":"2024-09-11T23:34:50.857763+0000","last_active":"2024-09-11T23:36:58.069867+0000","last_peered":"2024-09-11T23:36:58.069867+0000","last_clean":"2024-09-11T23:36:58.069867+0000","last_became_active":"2024-09-11T23:34:50.857427+0000","last_became_peered":"2024-09-11T23:34:50.857427+0000","last_unstale":"2024-09-11T23:36:58.069867+0000","last_undegraded":"2024-09-11T23:36:58.069867+0000","last_fullsized":"2024-09-11T23:36:58.069867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:59.703580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":19,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152618+0000","last_change":"2024-09-11T23:34:52.872426+0000","last_active":"2024-09-11T23:36:47.152618+0000","last_peered":"2024-09-11T23:36:47.152618+0000","last_clean":"2024-09-11T23:36:47.152618+0000","last_became_active":"2024-09-11T23:34:52.872090+0000","last_became_peered":"2024-09-11T23:34:52.872090+0000","last_unstale":"2024-09-11T23:36:47.152618+0000","last_undegraded":"2024-09-11T23:36:47.152618+0000","last_fullsized":"2024-09-11T23:36:47.152618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:04:00.963963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'7","reported_seq":148,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.620272+0000","last_change":"2024-09-11T23:34:50.924792+0000","last_active":"2024-09-11T23:36:56.620272+0000","last_peered":"2024-09-11T23:36:56.620272+0000","last_clean":"2024-09-11T23:36:56.620272+0000","last_became_active":"2024-09-11T23:34:50.924444+0000","last_became_peered":"2024-09-11T23:34:50.924444+0000","last_unstale":"2024-09-11T23:36:56.620272+0000","last_undegraded":"2024-09-11T23:36:56.620272+0000","last_fullsized":"2024-09-11T23:36:56.620272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:12.855314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152787+0000","last_change":"2024-09-11T23:34:52.872079+0000","last_active":"2024-09-11T23:36:47.152787+0000","last_peered":"2024-09-11T23:36:47.152787+0000","last_clean":"2024-09-11T23:36:47.152787+0000","last_became_active":"2024-09-11T23:34:52.871755+0000","last_became_peered":"2024-09-11T23:34:52.871755+0000","last_unstale":"2024-09-11T23:36:47.152787+0000","last_undegraded":"2024-09-11T23:36:47.152787+0000","last_fullsized":"2024-09-11T23:36:47.152787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:05.990832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"82'148","reported_seq":391,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:49.148435+0000","last_change":"2024-09-11T23:33:42.256521+0000","last_active":"2024-09-11T23:36:49.148435+0000","last_peered":"2024-09-11T23:36:49.148435+0000","last_clean":"2024-09-11T23:36:49.148435+0000","last_became_active":"2024-09-11T23:33:42.256041+0000","last_became_peered":"2024-09-11T23:33:42.256041+0000","last_unstale":"2024-09-11T23:36:49.148435+0000","last_undegraded":"2024-09-11T23:36:49.148435+0000","last_fullsized":"2024-09-11T23:36:49.148435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_clean_scrub_stamp":"2024-09-11T23:30:22.061515+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:21.391839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292343+0000","last_change":"2024-09-11T23:34:50.854087+0000","last_active":"2024-09-11T23:36:47.292343+0000","last_peered":"2024-09-11T23:36:47.292343+0000","last_clean":"2024-09-11T23:36:47.292343+0000","last_became_active":"2024-09-11T23:34:50.852346+0000","last_became_peered":"2024-09-11T23:34:50.852346+0000","last_unstale":"2024-09-11T23:36:47.292343+0000","last_undegraded":"2024-09-11T23:36:47.292343+0000","last_fullsized":"2024-09-11T23:36:47.292343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.287119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147041+0000","last_change":"2024-09-11T23:34:52.869198+0000","last_active":"2024-09-11T23:36:47.147041+0000","last_peered":"2024-09-11T23:36:47.147041+0000","last_clean":"2024-09-11T23:36:47.147041+0000","last_became_active":"2024-09-11T23:34:52.868621+0000","last_became_peered":"2024-09-11T23:34:52.868621+0000","last_unstale":"2024-09-11T23:36:47.147041+0000","last_undegraded":"2024-09-11T23:36:47.147041+0000","last_fullsized":"2024-09-11T23:36:47.147041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:17.430654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152654+0000","last_change":"2024-09-11T23:34:50.852752+0000","last_active":"2024-09-11T23:36:47.152654+0000","last_peered":"2024-09-11T23:36:47.152654+0000","last_clean":"2024-09-11T23:36:47.152654+0000","last_became_active":"2024-09-11T23:34:50.851895+0000","last_became_peered":"2024-09-11T23:34:50.851895+0000","last_unstale":"2024-09-11T23:36:47.152654+0000","last_undegraded":"2024-09-11T23:36:47.152654+0000","last_fullsized":"2024-09-11T23:36:47.152654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:46.894432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057125+0000","last_change":"2024-09-11T23:34:52.866191+0000","last_active":"2024-09-11T23:36:47.057125+0000","last_peered":"2024-09-11T23:36:47.057125+0000","last_clean":"2024-09-11T23:36:47.057125+0000","last_became_active":"2024-09-11T23:34:52.865858+0000","last_became_peered":"2024-09-11T23:34:52.865858+0000","last_unstale":"2024-09-11T23:36:47.057125+0000","last_undegraded":"2024-09-11T23:36:47.057125+0000","last_fullsized":"2024-09-11T23:36:47.057125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:21.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.620820+0000","last_change":"2024-09-11T23:34:50.924125+0000","last_active":"2024-09-11T23:36:56.620820+0000","last_peered":"2024-09-11T23:36:56.620820+0000","last_clean":"2024-09-11T23:36:56.620820+0000","last_became_active":"2024-09-11T23:34:50.923780+0000","last_became_peered":"2024-09-11T23:34:50.923780+0000","last_unstale":"2024-09-11T23:36:56.620820+0000","last_undegraded":"2024-09-11T23:36:56.620820+0000","last_fullsized":"2024-09-11T23:36:56.620820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:39.774827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153717+0000","last_change":"2024-09-11T23:34:52.868364+0000","last_active":"2024-09-11T23:36:47.153717+0000","last_peered":"2024-09-11T23:36:47.153717+0000","last_clean":"2024-09-11T23:36:47.153717+0000","last_became_active":"2024-09-11T23:34:52.867913+0000","last_became_peered":"2024-09-11T23:34:52.867913+0000","last_unstale":"2024-09-11T23:36:47.153717+0000","last_undegraded":"2024-09-11T23:36:47.153717+0000","last_fullsized":"2024-09-11T23:36:47.153717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.571825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'8","reported_seq":149,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.615174+0000","last_change":"2024-09-11T23:34:50.852421+0000","last_active":"2024-09-11T23:36:56.615174+0000","last_peered":"2024-09-11T23:36:56.615174+0000","last_clean":"2024-09-11T23:36:56.615174+0000","last_became_active":"2024-09-11T23:34:50.851898+0000","last_became_peered":"2024-09-11T23:34:50.851898+0000","last_unstale":"2024-09-11T23:36:56.615174+0000","last_undegraded":"2024-09-11T23:36:56.615174+0000","last_fullsized":"2024-09-11T23:36:56.615174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.661276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1767,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156409+0000","last_change":"2024-09-11T23:34:52.879995+0000","last_active":"2024-09-11T23:36:47.156409+0000","last_peered":"2024-09-11T23:36:47.156409+0000","last_clean":"2024-09-11T23:36:47.156409+0000","last_became_active":"2024-09-11T23:34:52.879678+0000","last_became_peered":"2024-09-11T23:34:52.879678+0000","last_unstale":"2024-09-11T23:36:47.156409+0000","last_undegraded":"2024-09-11T23:36:47.156409+0000","last_fullsized":"2024-09-11T23:36:47.156409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:47.456674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143130+0000","last_change":"2024-09-11T23:34:50.924957+0000","last_active":"2024-09-11T23:36:47.143130+0000","last_peered":"2024-09-11T23:36:47.143130+0000","last_clean":"2024-09-11T23:36:47.143130+0000","last_became_active":"2024-09-11T23:34:50.924500+0000","last_became_peered":"2024-09-11T23:34:50.924500+0000","last_unstale":"2024-09-11T23:36:47.143130+0000","last_undegraded":"2024-09-11T23:36:47.143130+0000","last_fullsized":"2024-09-11T23:36:47.143130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:39.268842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156159+0000","last_change":"2024-09-11T23:34:52.867337+0000","last_active":"2024-09-11T23:36:47.156159+0000","last_peered":"2024-09-11T23:36:47.156159+0000","last_clean":"2024-09-11T23:36:47.156159+0000","last_became_active":"2024-09-11T23:34:52.866799+0000","last_became_peered":"2024-09-11T23:34:52.866799+0000","last_unstale":"2024-09-11T23:36:47.156159+0000","last_undegraded":"2024-09-11T23:36:47.156159+0000","last_fullsized":"2024-09-11T23:36:47.156159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:38.177135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145946+0000","last_change":"2024-09-11T23:34:50.925156+0000","last_active":"2024-09-11T23:36:47.145946+0000","last_peered":"2024-09-11T23:36:47.145946+0000","last_clean":"2024-09-11T23:36:47.145946+0000","last_became_active":"2024-09-11T23:34:50.924282+0000","last_became_peered":"2024-09-11T23:34:50.924282+0000","last_unstale":"2024-09-11T23:36:47.145946+0000","last_undegraded":"2024-09-11T23:36:47.145946+0000","last_fullsized":"2024-09-11T23:36:47.145946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:47.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057324+0000","last_change":"2024-09-11T23:34:52.864993+0000","last_active":"2024-09-11T23:36:47.057324+0000","last_peered":"2024-09-11T23:36:47.057324+0000","last_clean":"2024-09-11T23:36:47.057324+0000","last_became_active":"2024-09-11T23:34:52.864267+0000","last_became_peered":"2024-09-11T23:34:52.864267+0000","last_unstale":"2024-09-11T23:36:47.057324+0000","last_undegraded":"2024-09-11T23:36:47.057324+0000","last_fullsized":"2024-09-11T23:36:47.057324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:29.912068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'75","reported_seq":150,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:58.074331+0000","last_change":"2024-09-11T23:34:50.849864+0000","last_active":"2024-09-11T23:36:58.074331+0000","last_peered":"2024-09-11T23:36:58.074331+0000","last_clean":"2024-09-11T23:36:58.074331+0000","last_became_active":"2024-09-11T23:34:50.849577+0000","last_became_peered":"2024-09-11T23:34:50.849577+0000","last_unstale":"2024-09-11T23:36:58.074331+0000","last_undegraded":"2024-09-11T23:36:58.074331+0000","last_fullsized":"2024-09-11T23:36:58.074331+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:06.845000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":170,"num_read_kb":149,"num_write":133,"num_write_kb":1159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147479+0000","last_change":"2024-09-11T23:34:52.872028+0000","last_active":"2024-09-11T23:36:47.147479+0000","last_peered":"2024-09-11T23:36:47.147479+0000","last_clean":"2024-09-11T23:36:47.147479+0000","last_became_active":"2024-09-11T23:34:52.871723+0000","last_became_peered":"2024-09-11T23:34:52.871723+0000","last_unstale":"2024-09-11T23:36:47.147479+0000","last_undegraded":"2024-09-11T23:36:47.147479+0000","last_fullsized":"2024-09-11T23:36:47.147479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:44.656218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292619+0000","last_change":"2024-09-11T23:34:50.860314+0000","last_active":"2024-09-11T23:36:47.292619+0000","last_peered":"2024-09-11T23:36:47.292619+0000","last_clean":"2024-09-11T23:36:47.292619+0000","last_became_active":"2024-09-11T23:34:50.859953+0000","last_became_peered":"2024-09-11T23:34:50.859953+0000","last_unstale":"2024-09-11T23:36:47.292619+0000","last_undegraded":"2024-09-11T23:36:47.292619+0000","last_fullsized":"2024-09-11T23:36:47.292619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:46.705864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328058+0000","last_change":"2024-09-11T23:34:52.867190+0000","last_active":"2024-09-11T23:36:47.328058+0000","last_peered":"2024-09-11T23:36:47.328058+0000","last_clean":"2024-09-11T23:36:47.328058+0000","last_became_active":"2024-09-11T23:34:52.866829+0000","last_became_peered":"2024-09-11T23:34:52.866829+0000","last_unstale":"2024-09-11T23:36:47.328058+0000","last_undegraded":"2024-09-11T23:36:47.328058+0000","last_fullsized":"2024-09-11T23:36:47.328058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.752006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058275+0000","last_change":"2024-09-11T23:34:50.924566+0000","last_active":"2024-09-11T23:36:47.058275+0000","last_peered":"2024-09-11T23:36:47.058275+0000","last_clean":"2024-09-11T23:36:47.058275+0000","last_became_active":"2024-09-11T23:34:50.924284+0000","last_became_peered":"2024-09-11T23:34:50.924284+0000","last_unstale":"2024-09-11T23:36:47.058275+0000","last_undegraded":"2024-09-11T23:36:47.058275+0000","last_fullsized":"2024-09-11T23:36:47.058275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:54.246154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145487+0000","last_change":"2024-09-11T23:34:52.869302+0000","last_active":"2024-09-11T23:36:47.145487+0000","last_peered":"2024-09-11T23:36:47.145487+0000","last_clean":"2024-09-11T23:36:47.145487+0000","last_became_active":"2024-09-11T23:34:52.868960+0000","last_became_peered":"2024-09-11T23:34:52.868960+0000","last_unstale":"2024-09-11T23:36:47.145487+0000","last_undegraded":"2024-09-11T23:36:47.145487+0000","last_fullsized":"2024-09-11T23:36:47.145487+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.561712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146271+0000","last_change":"2024-09-11T23:34:50.854573+0000","last_active":"2024-09-11T23:36:47.146271+0000","last_peered":"2024-09-11T23:36:47.146271+0000","last_clean":"2024-09-11T23:36:47.146271+0000","last_became_active":"2024-09-11T23:34:50.854165+0000","last_became_peered":"2024-09-11T23:34:50.854165+0000","last_unstale":"2024-09-11T23:36:47.146271+0000","last_undegraded":"2024-09-11T23:36:47.146271+0000","last_fullsized":"2024-09-11T23:36:47.146271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:57:31.589495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327542+0000","last_change":"2024-09-11T23:34:52.875987+0000","last_active":"2024-09-11T23:36:47.327542+0000","last_peered":"2024-09-11T23:36:47.327542+0000","last_clean":"2024-09-11T23:36:47.327542+0000","last_became_active":"2024-09-11T23:34:52.875662+0000","last_became_peered":"2024-09-11T23:34:52.875662+0000","last_unstale":"2024-09-11T23:36:47.327542+0000","last_undegraded":"2024-09-11T23:36:47.327542+0000","last_fullsized":"2024-09-11T23:36:47.327542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:16.514938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058234+0000","last_change":"2024-09-11T23:34:50.850726+0000","last_active":"2024-09-11T23:36:47.058234+0000","last_peered":"2024-09-11T23:36:47.058234+0000","last_clean":"2024-09-11T23:36:47.058234+0000","last_became_active":"2024-09-11T23:34:50.850381+0000","last_became_peered":"2024-09-11T23:34:50.850381+0000","last_unstale":"2024-09-11T23:36:47.058234+0000","last_undegraded":"2024-09-11T23:36:47.058234+0000","last_fullsized":"2024-09-11T23:36:47.058234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:50.431293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146377+0000","last_change":"2024-09-11T23:34:52.870652+0000","last_active":"2024-09-11T23:36:47.146377+0000","last_peered":"2024-09-11T23:36:47.146377+0000","last_clean":"2024-09-11T23:36:47.146377+0000","last_became_active":"2024-09-11T23:34:52.870303+0000","last_became_peered":"2024-09-11T23:34:52.870303+0000","last_unstale":"2024-09-11T23:36:47.146377+0000","last_undegraded":"2024-09-11T23:36:47.146377+0000","last_fullsized":"2024-09-11T23:36:47.146377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:17.670981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327509+0000","last_change":"2024-09-11T23:35:08.854490+0000","last_active":"2024-09-11T23:36:47.327509+0000","last_peered":"2024-09-11T23:36:47.327509+0000","last_clean":"2024-09-11T23:36:47.327509+0000","last_became_active":"2024-09-11T23:35:08.849820+0000","last_became_peered":"2024-09-11T23:35:08.849820+0000","last_unstale":"2024-09-11T23:36:47.327509+0000","last_undegraded":"2024-09-11T23:36:47.327509+0000","last_fullsized":"2024-09-11T23:36:47.327509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:09.509867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152580+0000","last_change":"2024-09-11T23:34:52.869695+0000","last_active":"2024-09-11T23:36:47.152580+0000","last_peered":"2024-09-11T23:36:47.152580+0000","last_clean":"2024-09-11T23:36:47.152580+0000","last_became_active":"2024-09-11T23:34:52.869367+0000","last_became_peered":"2024-09-11T23:34:52.869367+0000","last_unstale":"2024-09-11T23:36:47.152580+0000","last_undegraded":"2024-09-11T23:36:47.152580+0000","last_fullsized":"2024-09-11T23:36:47.152580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:16.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153822+0000","last_change":"2024-09-11T23:34:52.872778+0000","last_active":"2024-09-11T23:36:47.153822+0000","last_peered":"2024-09-11T23:36:47.153822+0000","last_clean":"2024-09-11T23:36:47.153822+0000","last_became_active":"2024-09-11T23:34:52.872451+0000","last_became_peered":"2024-09-11T23:34:52.872451+0000","last_unstale":"2024-09-11T23:36:47.153822+0000","last_undegraded":"2024-09-11T23:36:47.153822+0000","last_fullsized":"2024-09-11T23:36:47.153822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:51.769927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145825+0000","last_change":"2024-09-11T23:34:50.849675+0000","last_active":"2024-09-11T23:36:47.145825+0000","last_peered":"2024-09-11T23:36:47.145825+0000","last_clean":"2024-09-11T23:36:47.145825+0000","last_became_active":"2024-09-11T23:34:50.849228+0000","last_became_peered":"2024-09-11T23:34:50.849228+0000","last_unstale":"2024-09-11T23:36:47.145825+0000","last_undegraded":"2024-09-11T23:36:47.145825+0000","last_fullsized":"2024-09-11T23:36:47.145825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:50.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058399+0000","last_change":"2024-09-11T23:34:52.867739+0000","last_active":"2024-09-11T23:36:47.058399+0000","last_peered":"2024-09-11T23:36:47.058399+0000","last_clean":"2024-09-11T23:36:47.058399+0000","last_became_active":"2024-09-11T23:34:52.866613+0000","last_became_peered":"2024-09-11T23:34:52.866613+0000","last_unstale":"2024-09-11T23:36:47.058399+0000","last_undegraded":"2024-09-11T23:36:47.058399+0000","last_fullsized":"2024-09-11T23:36:47.058399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:49.348386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'13","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145845+0000","last_change":"2024-09-11T23:34:50.851401+0000","last_active":"2024-09-11T23:36:47.145845+0000","last_peered":"2024-09-11T23:36:47.145845+0000","last_clean":"2024-09-11T23:36:47.145845+0000","last_became_active":"2024-09-11T23:34:50.851068+0000","last_became_peered":"2024-09-11T23:34:50.851068+0000","last_unstale":"2024-09-11T23:36:47.145845+0000","last_undegraded":"2024-09-11T23:36:47.145845+0000","last_fullsized":"2024-09-11T23:36:47.145845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:09.673864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328160+0000","last_change":"2024-09-11T23:34:52.865216+0000","last_active":"2024-09-11T23:36:47.328160+0000","last_peered":"2024-09-11T23:36:47.328160+0000","last_clean":"2024-09-11T23:36:47.328160+0000","last_became_active":"2024-09-11T23:34:52.864917+0000","last_became_peered":"2024-09-11T23:34:52.864917+0000","last_unstale":"2024-09-11T23:36:47.328160+0000","last_undegraded":"2024-09-11T23:36:47.328160+0000","last_fullsized":"2024-09-11T23:36:47.328160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:36.469793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147661+0000","last_change":"2024-09-11T23:34:50.924291+0000","last_active":"2024-09-11T23:36:47.147661+0000","last_peered":"2024-09-11T23:36:47.147661+0000","last_clean":"2024-09-11T23:36:47.147661+0000","last_became_active":"2024-09-11T23:34:50.923892+0000","last_became_peered":"2024-09-11T23:34:50.923892+0000","last_unstale":"2024-09-11T23:36:47.147661+0000","last_undegraded":"2024-09-11T23:36:47.147661+0000","last_fullsized":"2024-09-11T23:36:47.147661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:52.266778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147431+0000","last_change":"2024-09-11T23:34:52.875995+0000","last_active":"2024-09-11T23:36:47.147431+0000","last_peered":"2024-09-11T23:36:47.147431+0000","last_clean":"2024-09-11T23:36:47.147431+0000","last_became_active":"2024-09-11T23:34:52.875654+0000","last_became_peered":"2024-09-11T23:34:52.875654+0000","last_unstale":"2024-09-11T23:36:47.147431+0000","last_undegraded":"2024-09-11T23:36:47.147431+0000","last_fullsized":"2024-09-11T23:36:47.147431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:19.218291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":154,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:57.530808+0000","last_change":"2024-09-11T23:34:50.853881+0000","last_active":"2024-09-11T23:36:57.530808+0000","last_peered":"2024-09-11T23:36:57.530808+0000","last_clean":"2024-09-11T23:36:57.530808+0000","last_became_active":"2024-09-11T23:34:50.853576+0000","last_became_peered":"2024-09-11T23:34:50.853576+0000","last_unstale":"2024-09-11T23:36:57.530808+0000","last_undegraded":"2024-09-11T23:36:57.530808+0000","last_fullsized":"2024-09-11T23:36:57.530808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:38.550587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":802091,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1385,"num_read_kb":853,"num_write":447,"num_write_kb":2128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2406273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612646,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1753088,"data_stored":1237974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005104,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32484,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712668,"statfs":{"total":95995035648,"available":95961772032,"internally_reserved":0,"allocated":1327104,"data_stored":836376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234037,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28316,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716836,"statfs":{"total":95995035648,"available":95966040064,"internally_reserved":0,"allocated":1323008,"data_stored":836754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495674,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1527808,"data_stored":1039258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":444056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888131,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1167336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018953,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":444759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247884,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1462272,"data_stored":967518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444115,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":514963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607639,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":515063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":444750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":73025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":402083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:37:01.331 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:37:01.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:01 smithi138 ceph-mon[35142]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:37:01.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:01 smithi178 ceph-mon[35882]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:37:01.577 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:02.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.023+0000 7f4c089dd640 1 -- 172.21.15.133:0/2498226578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c04100e20 msgr2=0x7f4c04101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:02.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.023+0000 7f4c089dd640 1 --2- 172.21.15.133:0/2498226578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c04100e20 0x7f4c04101200 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009ca0 tx=0x7f4bf402a950 comp rx=0 tx=0).stop 2024-09-11T23:37:02.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.023+0000 7f4c089dd640 1 -- 172.21.15.133:0/2498226578 shutdown_connections 2024-09-11T23:37:02.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.023+0000 7f4c089dd640 1 --2- 172.21.15.133:0/2498226578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c04102ef0 0x7f4c0410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:02.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.023+0000 7f4c089dd640 1 --2- 172.21.15.133:0/2498226578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c04102570 0x7f4c041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:02.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.023+0000 7f4c089dd640 1 --2- 172.21.15.133:0/2498226578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c04100e20 0x7f4c04101200 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:02.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.023+0000 7f4c089dd640 1 -- 172.21.15.133:0/2498226578 >> 172.21.15.133:0/2498226578 conn(0x7f4c040fc7d0 msgr2=0x7f4c040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:02.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.023+0000 7f4c089dd640 1 -- 172.21.15.133:0/2498226578 shutdown_connections 2024-09-11T23:37:02.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.023+0000 7f4c089dd640 1 -- 172.21.15.133:0/2498226578 wait complete. 2024-09-11T23:37:02.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c089dd640 1 Processor -- start 2024-09-11T23:37:02.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c089dd640 1 -- start start 2024-09-11T23:37:02.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c089dd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c04100e20 0x7f4c0419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:02.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c089dd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c04102570 0x7f4c0419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:02.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c089dd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c04102ef0 0x7f4c0419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:02.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c089dd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041120a0 con 0x7f4c04102570 2024-09-11T23:37:02.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c089dd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c04111f20 con 0x7f4c04100e20 2024-09-11T23:37:02.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c089dd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c04112220 con 0x7f4c04102ef0 2024-09-11T23:37:02.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c01d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c04102570 0x7f4c0419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:02.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c01d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c04102570 0x7f4c0419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33484/0 (socket says 172.21.15.133:33484) 2024-09-11T23:37:02.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c01d9b640 1 -- 172.21.15.133:0/4025833191 learned_addr learned my addr 172.21.15.133:0/4025833191 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:02.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c02d9d640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c04102ef0 0x7f4c0419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:02.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c0259c640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c04100e20 0x7f4c0419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:02.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c01d9b640 1 -- 172.21.15.133:0/4025833191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c04102ef0 msgr2=0x7f4c0419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:02.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c01d9b640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c04102ef0 0x7f4c0419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:02.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c01d9b640 1 -- 172.21.15.133:0/4025833191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c04100e20 msgr2=0x7f4c0419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:02.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c01d9b640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c04100e20 0x7f4c0419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:02.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.024+0000 7f4c01d9b640 1 -- 172.21.15.133:0/4025833191 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0419f6b0 con 0x7f4c04102570 2024-09-11T23:37:02.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.025+0000 7f4c01d9b640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c04102570 0x7f4c0419ac40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800d770 tx=0x7f4bf800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:02.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.025+0000 7f4bf37fe640 1 -- 172.21.15.133:0/4025833191 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8017070 con 0x7f4c04102570 2024-09-11T23:37:02.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.025+0000 7f4bf37fe640 1 -- 172.21.15.133:0/4025833191 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bf8005500 con 0x7f4c04102570 2024-09-11T23:37:02.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.025+0000 7f4bf37fe640 1 -- 172.21.15.133:0/4025833191 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8004e00 con 0x7f4c04102570 2024-09-11T23:37:02.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.025+0000 7f4c089dd640 1 -- 172.21.15.133:0/4025833191 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0419f9a0 con 0x7f4c04102570 2024-09-11T23:37:02.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.025+0000 7f4c089dd640 1 -- 172.21.15.133:0/4025833191 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c041a7280 con 0x7f4c04102570 2024-09-11T23:37:02.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.026+0000 7f4c089dd640 1 -- 172.21.15.133:0/4025833191 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c04101200 con 0x7f4c04102570 2024-09-11T23:37:02.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.027+0000 7f4bf37fe640 1 -- 172.21.15.133:0/4025833191 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f4bf8004fa0 con 0x7f4c04102570 2024-09-11T23:37:02.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.028+0000 7f4bf37fe640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4bec0a2f20 0x7f4bec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:02.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.028+0000 7f4c0259c640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4bec0a2f20 0x7f4bec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:02.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.028+0000 7f4bf37fe640 1 -- 172.21.15.133:0/4025833191 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f4bf80cefe0 con 0x7f4c04102570 2024-09-11T23:37:02.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.028+0000 7f4c0259c640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4bec0a2f20 0x7f4bec0a53e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4004ad0 tx=0x7f4bf4004a60 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:02.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.032+0000 7f4bf37fe640 1 -- 172.21.15.133:0/4025833191 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bf809b3e0 con 0x7f4c04102570 2024-09-11T23:37:02.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.197+0000 7f4c089dd640 1 -- 172.21.15.133:0/4025833191 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c040fe580 con 0x7f4bec0a2f20 2024-09-11T23:37:02.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.203+0000 7f4bf37fe640 1 -- 172.21.15.133:0/4025833191 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257810 (secure 0 0 0) 0x7f4c040fe580 con 0x7f4bec0a2f20 2024-09-11T23:37:02.205 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:37:02.206 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T23:37:02.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 -- 172.21.15.133:0/4025833191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4bec0a2f20 msgr2=0x7f4bec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4bec0a2f20 0x7f4bec0a53e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4004ad0 tx=0x7f4bf4004a60 comp rx=0 tx=0).stop 2024-09-11T23:37:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 -- 172.21.15.133:0/4025833191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c04102570 msgr2=0x7f4c0419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c04102570 0x7f4c0419ac40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800d770 tx=0x7f4bf800dc30 comp rx=0 tx=0).stop 2024-09-11T23:37:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 -- 172.21.15.133:0/4025833191 shutdown_connections 2024-09-11T23:37:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4bec0a2f20 0x7f4bec0a53e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c04102ef0 0x7f4c0419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c04102570 0x7f4c0419ac40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 --2- 172.21.15.133:0/4025833191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c04100e20 0x7f4c0419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 -- 172.21.15.133:0/4025833191 >> 172.21.15.133:0/4025833191 conn(0x7f4c040fc7d0 msgr2=0x7f4c040fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 -- 172.21.15.133:0/4025833191 shutdown_connections 2024-09-11T23:37:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:02.206+0000 7f4c089dd640 1 -- 172.21.15.133:0/4025833191 wait complete. 2024-09-11T23:37:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:02 smithi138 ceph-mon[35142]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-09-11T23:37:02.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:02 smithi138 ceph-mon[35142]: mgrmap e24: x(active, since 14s), standbys: y, z 2024-09-11T23:37:02.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:02 smithi133 ceph-mon[31430]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-09-11T23:37:02.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:02 smithi133 ceph-mon[31430]: mgrmap e24: x(active, since 14s), standbys: y, z 2024-09-11T23:37:02.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:02 smithi178 ceph-mon[35882]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-09-11T23:37:02.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:02 smithi178 ceph-mon[35882]: mgrmap e24: x(active, since 14s), standbys: y, z 2024-09-11T23:37:02.894 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-11T23:37:01.071441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1392693,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1622,"num_read_kb":1183,"num_write":650,"num_write_kb":5068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1001,"ondisk_log_size":1001,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359520,"kb_used_data":14400,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582304,"statfs":{"total":1151940427776,"available":1151572279296,"internally_reserved":0,"allocated":14745600,"data_stored":8892639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":204645,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":361,"num_write":86,"num_write_kb":664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002672"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327868+0000","last_change":"2024-09-11T23:34:52.867694+0000","last_active":"2024-09-11T23:36:47.327868+0000","last_peered":"2024-09-11T23:36:47.327868+0000","last_clean":"2024-09-11T23:36:47.327868+0000","last_became_active":"2024-09-11T23:34:52.867368+0000","last_became_peered":"2024-09-11T23:34:52.867368+0000","last_unstale":"2024-09-11T23:36:47.327868+0000","last_undegraded":"2024-09-11T23:36:47.327868+0000","last_fullsized":"2024-09-11T23:36:47.327868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:46:07.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328024+0000","last_change":"2024-09-11T23:34:52.871881+0000","last_active":"2024-09-11T23:36:47.328024+0000","last_peered":"2024-09-11T23:36:47.328024+0000","last_clean":"2024-09-11T23:36:47.328024+0000","last_became_active":"2024-09-11T23:34:52.871609+0000","last_became_peered":"2024-09-11T23:34:52.871609+0000","last_unstale":"2024-09-11T23:36:47.328024+0000","last_undegraded":"2024-09-11T23:36:47.328024+0000","last_fullsized":"2024-09-11T23:36:47.328024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.938584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327774+0000","last_change":"2024-09-11T23:34:52.863931+0000","last_active":"2024-09-11T23:36:47.327774+0000","last_peered":"2024-09-11T23:36:47.327774+0000","last_clean":"2024-09-11T23:36:47.327774+0000","last_became_active":"2024-09-11T23:34:52.863637+0000","last_became_peered":"2024-09-11T23:34:52.863637+0000","last_unstale":"2024-09-11T23:36:47.327774+0000","last_undegraded":"2024-09-11T23:36:47.327774+0000","last_fullsized":"2024-09-11T23:36:47.327774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:44.118865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292453+0000","last_change":"2024-09-11T23:34:52.879619+0000","last_active":"2024-09-11T23:36:47.292453+0000","last_peered":"2024-09-11T23:36:47.292453+0000","last_clean":"2024-09-11T23:36:47.292453+0000","last_became_active":"2024-09-11T23:34:52.878695+0000","last_became_peered":"2024-09-11T23:34:52.878695+0000","last_unstale":"2024-09-11T23:36:47.292453+0000","last_undegraded":"2024-09-11T23:36:47.292453+0000","last_fullsized":"2024-09-11T23:36:47.292453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:30.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145593+0000","last_change":"2024-09-11T23:34:52.869363+0000","last_active":"2024-09-11T23:36:47.145593+0000","last_peered":"2024-09-11T23:36:47.145593+0000","last_clean":"2024-09-11T23:36:47.145593+0000","last_became_active":"2024-09-11T23:34:52.869027+0000","last_became_peered":"2024-09-11T23:34:52.869027+0000","last_unstale":"2024-09-11T23:36:47.145593+0000","last_undegraded":"2024-09-11T23:36:47.145593+0000","last_fullsized":"2024-09-11T23:36:47.145593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:15.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156014+0000","last_change":"2024-09-11T23:34:52.880665+0000","last_active":"2024-09-11T23:36:47.156014+0000","last_peered":"2024-09-11T23:36:47.156014+0000","last_clean":"2024-09-11T23:36:47.156014+0000","last_became_active":"2024-09-11T23:34:52.880172+0000","last_became_peered":"2024-09-11T23:34:52.880172+0000","last_unstale":"2024-09-11T23:36:47.156014+0000","last_undegraded":"2024-09-11T23:36:47.156014+0000","last_fullsized":"2024-09-11T23:36:47.156014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:12.025877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152489+0000","last_change":"2024-09-11T23:34:52.873897+0000","last_active":"2024-09-11T23:36:47.152489+0000","last_peered":"2024-09-11T23:36:47.152489+0000","last_clean":"2024-09-11T23:36:47.152489+0000","last_became_active":"2024-09-11T23:34:52.873219+0000","last_became_peered":"2024-09-11T23:34:52.873219+0000","last_unstale":"2024-09-11T23:36:47.152489+0000","last_undegraded":"2024-09-11T23:36:47.152489+0000","last_fullsized":"2024-09-11T23:36:47.152489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:47.125682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145794+0000","last_change":"2024-09-11T23:34:52.870699+0000","last_active":"2024-09-11T23:36:47.145794+0000","last_peered":"2024-09-11T23:36:47.145794+0000","last_clean":"2024-09-11T23:36:47.145794+0000","last_became_active":"2024-09-11T23:34:52.870314+0000","last_became_peered":"2024-09-11T23:34:52.870314+0000","last_unstale":"2024-09-11T23:36:47.145794+0000","last_undegraded":"2024-09-11T23:36:47.145794+0000","last_fullsized":"2024-09-11T23:36:47.145794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:41.263159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058666+0000","last_change":"2024-09-11T23:34:52.869700+0000","last_active":"2024-09-11T23:36:47.058666+0000","last_peered":"2024-09-11T23:36:47.058666+0000","last_clean":"2024-09-11T23:36:47.058666+0000","last_became_active":"2024-09-11T23:34:52.869398+0000","last_became_peered":"2024-09-11T23:34:52.869398+0000","last_unstale":"2024-09-11T23:36:47.058666+0000","last_undegraded":"2024-09-11T23:36:47.058666+0000","last_fullsized":"2024-09-11T23:36:47.058666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:49.776525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156161+0000","last_change":"2024-09-11T23:34:52.879999+0000","last_active":"2024-09-11T23:36:47.156161+0000","last_peered":"2024-09-11T23:36:47.156161+0000","last_clean":"2024-09-11T23:36:47.156161+0000","last_became_active":"2024-09-11T23:34:52.879646+0000","last_became_peered":"2024-09-11T23:34:52.879646+0000","last_unstale":"2024-09-11T23:36:47.156161+0000","last_undegraded":"2024-09-11T23:36:47.156161+0000","last_fullsized":"2024-09-11T23:36:47.156161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.058704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152665+0000","last_change":"2024-09-11T23:34:52.878740+0000","last_active":"2024-09-11T23:36:47.152665+0000","last_peered":"2024-09-11T23:36:47.152665+0000","last_clean":"2024-09-11T23:36:47.152665+0000","last_became_active":"2024-09-11T23:34:52.878120+0000","last_became_peered":"2024-09-11T23:34:52.878120+0000","last_unstale":"2024-09-11T23:36:47.152665+0000","last_undegraded":"2024-09-11T23:36:47.152665+0000","last_fullsized":"2024-09-11T23:36:47.152665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292447+0000","last_change":"2024-09-11T23:34:52.873409+0000","last_active":"2024-09-11T23:36:47.292447+0000","last_peered":"2024-09-11T23:36:47.292447+0000","last_clean":"2024-09-11T23:36:47.292447+0000","last_became_active":"2024-09-11T23:34:52.872959+0000","last_became_peered":"2024-09-11T23:34:52.872959+0000","last_unstale":"2024-09-11T23:36:47.292447+0000","last_undegraded":"2024-09-11T23:36:47.292447+0000","last_fullsized":"2024-09-11T23:36:47.292447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:38.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057362+0000","last_change":"2024-09-11T23:34:52.869164+0000","last_active":"2024-09-11T23:36:47.057362+0000","last_peered":"2024-09-11T23:36:47.057362+0000","last_clean":"2024-09-11T23:36:47.057362+0000","last_became_active":"2024-09-11T23:34:52.868847+0000","last_became_peered":"2024-09-11T23:34:52.868847+0000","last_unstale":"2024-09-11T23:36:47.057362+0000","last_undegraded":"2024-09-11T23:36:47.057362+0000","last_fullsized":"2024-09-11T23:36:47.057362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.764510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327562+0000","last_change":"2024-09-11T23:34:52.869494+0000","last_active":"2024-09-11T23:36:47.327562+0000","last_peered":"2024-09-11T23:36:47.327562+0000","last_clean":"2024-09-11T23:36:47.327562+0000","last_became_active":"2024-09-11T23:34:52.869178+0000","last_became_peered":"2024-09-11T23:34:52.869178+0000","last_unstale":"2024-09-11T23:36:47.327562+0000","last_undegraded":"2024-09-11T23:36:47.327562+0000","last_fullsized":"2024-09-11T23:36:47.327562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:23.431564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327741+0000","last_change":"2024-09-11T23:34:52.866269+0000","last_active":"2024-09-11T23:36:47.327741+0000","last_peered":"2024-09-11T23:36:47.327741+0000","last_clean":"2024-09-11T23:36:47.327741+0000","last_became_active":"2024-09-11T23:34:52.865980+0000","last_became_peered":"2024-09-11T23:34:52.865980+0000","last_unstale":"2024-09-11T23:36:47.327741+0000","last_undegraded":"2024-09-11T23:36:47.327741+0000","last_fullsized":"2024-09-11T23:36:47.327741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:48.492252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058174+0000","last_change":"2024-09-11T23:34:52.862192+0000","last_active":"2024-09-11T23:36:47.058174+0000","last_peered":"2024-09-11T23:36:47.058174+0000","last_clean":"2024-09-11T23:36:47.058174+0000","last_became_active":"2024-09-11T23:34:52.861841+0000","last_became_peered":"2024-09-11T23:34:52.861841+0000","last_unstale":"2024-09-11T23:36:47.058174+0000","last_undegraded":"2024-09-11T23:36:47.058174+0000","last_fullsized":"2024-09-11T23:36:47.058174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.258948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327836+0000","last_change":"2024-09-11T23:34:52.870984+0000","last_active":"2024-09-11T23:36:47.327836+0000","last_peered":"2024-09-11T23:36:47.327836+0000","last_clean":"2024-09-11T23:36:47.327836+0000","last_became_active":"2024-09-11T23:34:52.870435+0000","last_became_peered":"2024-09-11T23:34:52.870435+0000","last_unstale":"2024-09-11T23:36:47.327836+0000","last_undegraded":"2024-09-11T23:36:47.327836+0000","last_fullsized":"2024-09-11T23:36:47.327836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:34.391537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143297+0000","last_change":"2024-09-11T23:34:52.873176+0000","last_active":"2024-09-11T23:36:47.143297+0000","last_peered":"2024-09-11T23:36:47.143297+0000","last_clean":"2024-09-11T23:36:47.143297+0000","last_became_active":"2024-09-11T23:34:52.872862+0000","last_became_peered":"2024-09-11T23:34:52.872862+0000","last_unstale":"2024-09-11T23:36:47.143297+0000","last_undegraded":"2024-09-11T23:36:47.143297+0000","last_fullsized":"2024-09-11T23:36:47.143297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:38.058806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058317+0000","last_change":"2024-09-11T23:34:52.871346+0000","last_active":"2024-09-11T23:36:47.058317+0000","last_peered":"2024-09-11T23:36:47.058317+0000","last_clean":"2024-09-11T23:36:47.058317+0000","last_became_active":"2024-09-11T23:34:52.870981+0000","last_became_peered":"2024-09-11T23:34:52.870981+0000","last_unstale":"2024-09-11T23:36:47.058317+0000","last_undegraded":"2024-09-11T23:36:47.058317+0000","last_fullsized":"2024-09-11T23:36:47.058317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:56.973998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327601+0000","last_change":"2024-09-11T23:34:52.867058+0000","last_active":"2024-09-11T23:36:47.327601+0000","last_peered":"2024-09-11T23:36:47.327601+0000","last_clean":"2024-09-11T23:36:47.327601+0000","last_became_active":"2024-09-11T23:34:52.866687+0000","last_became_peered":"2024-09-11T23:34:52.866687+0000","last_unstale":"2024-09-11T23:36:47.327601+0000","last_undegraded":"2024-09-11T23:36:47.327601+0000","last_fullsized":"2024-09-11T23:36:47.327601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:09.019987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057475+0000","last_change":"2024-09-11T23:34:52.870882+0000","last_active":"2024-09-11T23:36:47.057475+0000","last_peered":"2024-09-11T23:36:47.057475+0000","last_clean":"2024-09-11T23:36:47.057475+0000","last_became_active":"2024-09-11T23:34:52.870667+0000","last_became_peered":"2024-09-11T23:34:52.870667+0000","last_unstale":"2024-09-11T23:36:47.057475+0000","last_undegraded":"2024-09-11T23:36:47.057475+0000","last_fullsized":"2024-09-11T23:36:47.057475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:08.386703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058304+0000","last_change":"2024-09-11T23:34:52.874070+0000","last_active":"2024-09-11T23:36:47.058304+0000","last_peered":"2024-09-11T23:36:47.058304+0000","last_clean":"2024-09-11T23:36:47.058304+0000","last_became_active":"2024-09-11T23:34:52.873772+0000","last_became_peered":"2024-09-11T23:34:52.873772+0000","last_unstale":"2024-09-11T23:36:47.058304+0000","last_undegraded":"2024-09-11T23:36:47.058304+0000","last_fullsized":"2024-09-11T23:36:47.058304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:49.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146028+0000","last_change":"2024-09-11T23:34:52.875502+0000","last_active":"2024-09-11T23:36:47.146028+0000","last_peered":"2024-09-11T23:36:47.146028+0000","last_clean":"2024-09-11T23:36:47.146028+0000","last_became_active":"2024-09-11T23:34:52.875193+0000","last_became_peered":"2024-09-11T23:34:52.875193+0000","last_unstale":"2024-09-11T23:36:47.146028+0000","last_undegraded":"2024-09-11T23:36:47.146028+0000","last_fullsized":"2024-09-11T23:36:47.146028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:37.213828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153497+0000","last_change":"2024-09-11T23:34:52.869015+0000","last_active":"2024-09-11T23:36:47.153497+0000","last_peered":"2024-09-11T23:36:47.153497+0000","last_clean":"2024-09-11T23:36:47.153497+0000","last_became_active":"2024-09-11T23:34:52.868731+0000","last_became_peered":"2024-09-11T23:34:52.868731+0000","last_unstale":"2024-09-11T23:36:47.153497+0000","last_undegraded":"2024-09-11T23:36:47.153497+0000","last_fullsized":"2024-09-11T23:36:47.153497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:12:15.993203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147366+0000","last_change":"2024-09-11T23:34:52.875555+0000","last_active":"2024-09-11T23:36:47.147366+0000","last_peered":"2024-09-11T23:36:47.147366+0000","last_clean":"2024-09-11T23:36:47.147366+0000","last_became_active":"2024-09-11T23:34:52.875219+0000","last_became_peered":"2024-09-11T23:34:52.875219+0000","last_unstale":"2024-09-11T23:36:47.147366+0000","last_undegraded":"2024-09-11T23:36:47.147366+0000","last_fullsized":"2024-09-11T23:36:47.147366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:16.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058480+0000","last_change":"2024-09-11T23:34:52.875629+0000","last_active":"2024-09-11T23:36:47.058480+0000","last_peered":"2024-09-11T23:36:47.058480+0000","last_clean":"2024-09-11T23:36:47.058480+0000","last_became_active":"2024-09-11T23:34:52.875302+0000","last_became_peered":"2024-09-11T23:34:52.875302+0000","last_unstale":"2024-09-11T23:36:47.058480+0000","last_undegraded":"2024-09-11T23:36:47.058480+0000","last_fullsized":"2024-09-11T23:36:47.058480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:12.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327744+0000","last_change":"2024-09-11T23:34:52.868613+0000","last_active":"2024-09-11T23:36:47.327744+0000","last_peered":"2024-09-11T23:36:47.327744+0000","last_clean":"2024-09-11T23:36:47.327744+0000","last_became_active":"2024-09-11T23:34:52.868184+0000","last_became_peered":"2024-09-11T23:34:52.868184+0000","last_unstale":"2024-09-11T23:36:47.327744+0000","last_undegraded":"2024-09-11T23:36:47.327744+0000","last_fullsized":"2024-09-11T23:36:47.327744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:25.990002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057412+0000","last_change":"2024-09-11T23:35:08.873390+0000","last_active":"2024-09-11T23:36:47.057412+0000","last_peered":"2024-09-11T23:36:47.057412+0000","last_clean":"2024-09-11T23:36:47.057412+0000","last_became_active":"2024-09-11T23:35:08.873052+0000","last_became_peered":"2024-09-11T23:35:08.873052+0000","last_unstale":"2024-09-11T23:36:47.057412+0000","last_undegraded":"2024-09-11T23:36:47.057412+0000","last_fullsized":"2024-09-11T23:36:47.057412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.937951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327734+0000","last_change":"2024-09-11T23:34:52.873604+0000","last_active":"2024-09-11T23:36:47.327734+0000","last_peered":"2024-09-11T23:36:47.327734+0000","last_clean":"2024-09-11T23:36:47.327734+0000","last_became_active":"2024-09-11T23:34:52.873220+0000","last_became_peered":"2024-09-11T23:34:52.873220+0000","last_unstale":"2024-09-11T23:36:47.327734+0000","last_undegraded":"2024-09-11T23:36:47.327734+0000","last_fullsized":"2024-09-11T23:36:47.327734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:38.480630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058083+0000","last_change":"2024-09-11T23:34:52.872902+0000","last_active":"2024-09-11T23:36:47.058083+0000","last_peered":"2024-09-11T23:36:47.058083+0000","last_clean":"2024-09-11T23:36:47.058083+0000","last_became_active":"2024-09-11T23:34:52.872571+0000","last_became_peered":"2024-09-11T23:34:52.872571+0000","last_unstale":"2024-09-11T23:36:47.058083+0000","last_undegraded":"2024-09-11T23:36:47.058083+0000","last_fullsized":"2024-09-11T23:36:47.058083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:21.844995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058162+0000","last_change":"2024-09-11T23:35:09.457056+0000","last_active":"2024-09-11T23:36:47.058162+0000","last_peered":"2024-09-11T23:36:47.058162+0000","last_clean":"2024-09-11T23:36:47.058162+0000","last_became_active":"2024-09-11T23:35:09.456657+0000","last_became_peered":"2024-09-11T23:35:09.456657+0000","last_unstale":"2024-09-11T23:36:47.058162+0000","last_undegraded":"2024-09-11T23:36:47.058162+0000","last_fullsized":"2024-09-11T23:36:47.058162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:12:56.121280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327534+0000","last_change":"2024-09-11T23:34:52.875312+0000","last_active":"2024-09-11T23:36:47.327534+0000","last_peered":"2024-09-11T23:36:47.327534+0000","last_clean":"2024-09-11T23:36:47.327534+0000","last_became_active":"2024-09-11T23:34:52.874873+0000","last_became_peered":"2024-09-11T23:34:52.874873+0000","last_unstale":"2024-09-11T23:36:47.327534+0000","last_undegraded":"2024-09-11T23:36:47.327534+0000","last_fullsized":"2024-09-11T23:36:47.327534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:18.894713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145537+0000","last_change":"2024-09-11T23:34:52.862266+0000","last_active":"2024-09-11T23:36:47.145537+0000","last_peered":"2024-09-11T23:36:47.145537+0000","last_clean":"2024-09-11T23:36:47.145537+0000","last_became_active":"2024-09-11T23:34:52.861923+0000","last_became_peered":"2024-09-11T23:34:52.861923+0000","last_unstale":"2024-09-11T23:36:47.145537+0000","last_undegraded":"2024-09-11T23:36:47.145537+0000","last_fullsized":"2024-09-11T23:36:47.145537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:23.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":148,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.603783+0000","last_change":"2024-09-11T23:34:50.854020+0000","last_active":"2024-09-11T23:36:56.603783+0000","last_peered":"2024-09-11T23:36:56.603783+0000","last_clean":"2024-09-11T23:36:56.603783+0000","last_became_active":"2024-09-11T23:34:50.853683+0000","last_became_peered":"2024-09-11T23:34:50.853683+0000","last_unstale":"2024-09-11T23:36:56.603783+0000","last_undegraded":"2024-09-11T23:36:56.603783+0000","last_fullsized":"2024-09-11T23:36:56.603783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:18.725495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153611+0000","last_change":"2024-09-11T23:34:52.874016+0000","last_active":"2024-09-11T23:36:47.153611+0000","last_peered":"2024-09-11T23:36:47.153611+0000","last_clean":"2024-09-11T23:36:47.153611+0000","last_became_active":"2024-09-11T23:34:52.873675+0000","last_became_peered":"2024-09-11T23:34:52.873675+0000","last_unstale":"2024-09-11T23:36:47.153611+0000","last_undegraded":"2024-09-11T23:36:47.153611+0000","last_fullsized":"2024-09-11T23:36:47.153611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:41:28.955548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156082+0000","last_change":"2024-09-11T23:34:50.855391+0000","last_active":"2024-09-11T23:36:47.156082+0000","last_peered":"2024-09-11T23:36:47.156082+0000","last_clean":"2024-09-11T23:36:47.156082+0000","last_became_active":"2024-09-11T23:34:50.855052+0000","last_became_peered":"2024-09-11T23:34:50.855052+0000","last_unstale":"2024-09-11T23:36:47.156082+0000","last_undegraded":"2024-09-11T23:36:47.156082+0000","last_fullsized":"2024-09-11T23:36:47.156082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.972239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057065+0000","last_change":"2024-09-11T23:34:50.848985+0000","last_active":"2024-09-11T23:36:47.057065+0000","last_peered":"2024-09-11T23:36:47.057065+0000","last_clean":"2024-09-11T23:36:47.057065+0000","last_became_active":"2024-09-11T23:34:50.848614+0000","last_became_peered":"2024-09-11T23:34:50.848614+0000","last_unstale":"2024-09-11T23:36:47.057065+0000","last_undegraded":"2024-09-11T23:36:47.057065+0000","last_fullsized":"2024-09-11T23:36:47.057065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:43:04.723136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146018+0000","last_change":"2024-09-11T23:34:52.866933+0000","last_active":"2024-09-11T23:36:47.146018+0000","last_peered":"2024-09-11T23:36:47.146018+0000","last_clean":"2024-09-11T23:36:47.146018+0000","last_became_active":"2024-09-11T23:34:52.866559+0000","last_became_peered":"2024-09-11T23:34:52.866559+0000","last_unstale":"2024-09-11T23:36:47.146018+0000","last_undegraded":"2024-09-11T23:36:47.146018+0000","last_fullsized":"2024-09-11T23:36:47.146018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:27.745118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153745+0000","last_change":"2024-09-11T23:34:50.849787+0000","last_active":"2024-09-11T23:36:47.153745+0000","last_peered":"2024-09-11T23:36:47.153745+0000","last_clean":"2024-09-11T23:36:47.153745+0000","last_became_active":"2024-09-11T23:34:50.849368+0000","last_became_peered":"2024-09-11T23:34:50.849368+0000","last_unstale":"2024-09-11T23:36:47.153745+0000","last_undegraded":"2024-09-11T23:36:47.153745+0000","last_fullsized":"2024-09-11T23:36:47.153745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:26.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142558+0000","last_change":"2024-09-11T23:34:52.870832+0000","last_active":"2024-09-11T23:36:47.142558+0000","last_peered":"2024-09-11T23:36:47.142558+0000","last_clean":"2024-09-11T23:36:47.142558+0000","last_became_active":"2024-09-11T23:34:52.870474+0000","last_became_peered":"2024-09-11T23:34:52.870474+0000","last_unstale":"2024-09-11T23:36:47.142558+0000","last_undegraded":"2024-09-11T23:36:47.142558+0000","last_fullsized":"2024-09-11T23:36:47.142558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:30.196114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":258,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:57.604585+0000","last_change":"2024-09-11T23:34:50.859074+0000","last_active":"2024-09-11T23:36:57.604585+0000","last_peered":"2024-09-11T23:36:57.604585+0000","last_clean":"2024-09-11T23:36:57.604585+0000","last_became_active":"2024-09-11T23:34:50.858664+0000","last_became_peered":"2024-09-11T23:34:50.858664+0000","last_unstale":"2024-09-11T23:36:57.604585+0000","last_undegraded":"2024-09-11T23:36:57.604585+0000","last_fullsized":"2024-09-11T23:36:57.604585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:04.077813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058577+0000","last_change":"2024-09-11T23:34:52.875901+0000","last_active":"2024-09-11T23:36:47.058577+0000","last_peered":"2024-09-11T23:36:47.058577+0000","last_clean":"2024-09-11T23:36:47.058577+0000","last_became_active":"2024-09-11T23:34:52.875568+0000","last_became_peered":"2024-09-11T23:34:52.875568+0000","last_unstale":"2024-09-11T23:36:47.058577+0000","last_undegraded":"2024-09-11T23:36:47.058577+0000","last_fullsized":"2024-09-11T23:36:47.058577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:54.173548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152813+0000","last_change":"2024-09-11T23:34:50.854957+0000","last_active":"2024-09-11T23:36:47.152813+0000","last_peered":"2024-09-11T23:36:47.152813+0000","last_clean":"2024-09-11T23:36:47.152813+0000","last_became_active":"2024-09-11T23:34:50.854626+0000","last_became_peered":"2024-09-11T23:34:50.854626+0000","last_unstale":"2024-09-11T23:36:47.152813+0000","last_undegraded":"2024-09-11T23:36:47.152813+0000","last_fullsized":"2024-09-11T23:36:47.152813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:53.738966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328149+0000","last_change":"2024-09-11T23:34:52.868850+0000","last_active":"2024-09-11T23:36:47.328149+0000","last_peered":"2024-09-11T23:36:47.328149+0000","last_clean":"2024-09-11T23:36:47.328149+0000","last_became_active":"2024-09-11T23:34:52.868536+0000","last_became_peered":"2024-09-11T23:34:52.868536+0000","last_unstale":"2024-09-11T23:36:47.328149+0000","last_undegraded":"2024-09-11T23:36:47.328149+0000","last_fullsized":"2024-09-11T23:36:47.328149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:11.746821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153829+0000","last_change":"2024-09-11T23:34:50.924819+0000","last_active":"2024-09-11T23:36:47.153829+0000","last_peered":"2024-09-11T23:36:47.153829+0000","last_clean":"2024-09-11T23:36:47.153829+0000","last_became_active":"2024-09-11T23:34:50.924283+0000","last_became_peered":"2024-09-11T23:34:50.924283+0000","last_unstale":"2024-09-11T23:36:47.153829+0000","last_undegraded":"2024-09-11T23:36:47.153829+0000","last_fullsized":"2024-09-11T23:36:47.153829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:41.345233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058417+0000","last_change":"2024-09-11T23:34:52.867113+0000","last_active":"2024-09-11T23:36:47.058417+0000","last_peered":"2024-09-11T23:36:47.058417+0000","last_clean":"2024-09-11T23:36:47.058417+0000","last_became_active":"2024-09-11T23:34:52.866125+0000","last_became_peered":"2024-09-11T23:34:52.866125+0000","last_unstale":"2024-09-11T23:36:47.058417+0000","last_undegraded":"2024-09-11T23:36:47.058417+0000","last_fullsized":"2024-09-11T23:36:47.058417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:18:55.615079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'38","reported_seq":307,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:57.604631+0000","last_change":"2024-09-11T23:34:50.857272+0000","last_active":"2024-09-11T23:36:57.604631+0000","last_peered":"2024-09-11T23:36:57.604631+0000","last_clean":"2024-09-11T23:36:57.604631+0000","last_became_active":"2024-09-11T23:34:50.855701+0000","last_became_peered":"2024-09-11T23:34:50.855701+0000","last_unstale":"2024-09-11T23:36:57.604631+0000","last_undegraded":"2024-09-11T23:36:57.604631+0000","last_fullsized":"2024-09-11T23:36:57.604631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:49.997617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70094,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":245,"num_read_kb":191,"num_write":40,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145783+0000","last_change":"2024-09-11T23:34:52.876928+0000","last_active":"2024-09-11T23:36:47.145783+0000","last_peered":"2024-09-11T23:36:47.145783+0000","last_clean":"2024-09-11T23:36:47.145783+0000","last_became_active":"2024-09-11T23:34:52.876578+0000","last_became_peered":"2024-09-11T23:34:52.876578+0000","last_unstale":"2024-09-11T23:36:47.145783+0000","last_undegraded":"2024-09-11T23:36:47.145783+0000","last_fullsized":"2024-09-11T23:36:47.145783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:07.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327515+0000","last_change":"2024-09-11T23:35:08.869275+0000","last_active":"2024-09-11T23:36:47.327515+0000","last_peered":"2024-09-11T23:36:47.327515+0000","last_clean":"2024-09-11T23:36:47.327515+0000","last_became_active":"2024-09-11T23:35:08.850112+0000","last_became_peered":"2024-09-11T23:35:08.850112+0000","last_unstale":"2024-09-11T23:36:47.327515+0000","last_undegraded":"2024-09-11T23:36:47.327515+0000","last_fullsized":"2024-09-11T23:36:47.327515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:36.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057105+0000","last_change":"2024-09-11T23:34:52.871970+0000","last_active":"2024-09-11T23:36:47.057105+0000","last_peered":"2024-09-11T23:36:47.057105+0000","last_clean":"2024-09-11T23:36:47.057105+0000","last_became_active":"2024-09-11T23:34:52.871488+0000","last_became_peered":"2024-09-11T23:34:52.871488+0000","last_unstale":"2024-09-11T23:36:47.057105+0000","last_undegraded":"2024-09-11T23:36:47.057105+0000","last_fullsized":"2024-09-11T23:36:47.057105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:08.266431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153627+0000","last_change":"2024-09-11T23:34:50.854023+0000","last_active":"2024-09-11T23:36:47.153627+0000","last_peered":"2024-09-11T23:36:47.153627+0000","last_clean":"2024-09-11T23:36:47.153627+0000","last_became_active":"2024-09-11T23:34:50.853661+0000","last_became_peered":"2024-09-11T23:34:50.853661+0000","last_unstale":"2024-09-11T23:36:47.153627+0000","last_undegraded":"2024-09-11T23:36:47.153627+0000","last_fullsized":"2024-09-11T23:36:47.153627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:25.281976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152646+0000","last_change":"2024-09-11T23:35:08.849648+0000","last_active":"2024-09-11T23:36:47.152646+0000","last_peered":"2024-09-11T23:36:47.152646+0000","last_clean":"2024-09-11T23:36:47.152646+0000","last_became_active":"2024-09-11T23:35:08.849246+0000","last_became_peered":"2024-09-11T23:35:08.849246+0000","last_unstale":"2024-09-11T23:36:47.152646+0000","last_undegraded":"2024-09-11T23:36:47.152646+0000","last_fullsized":"2024-09-11T23:36:47.152646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:17.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'11","reported_seq":157,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.621527+0000","last_change":"2024-09-11T23:34:50.853855+0000","last_active":"2024-09-11T23:36:56.621527+0000","last_peered":"2024-09-11T23:36:56.621527+0000","last_clean":"2024-09-11T23:36:56.621527+0000","last_became_active":"2024-09-11T23:34:50.853493+0000","last_became_peered":"2024-09-11T23:34:50.853493+0000","last_unstale":"2024-09-11T23:36:56.621527+0000","last_undegraded":"2024-09-11T23:36:56.621527+0000","last_fullsized":"2024-09-11T23:36:56.621527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:23.262472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2609,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":8,"num_write":11,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152450+0000","last_change":"2024-09-11T23:34:52.875835+0000","last_active":"2024-09-11T23:36:47.152450+0000","last_peered":"2024-09-11T23:36:47.152450+0000","last_clean":"2024-09-11T23:36:47.152450+0000","last_became_active":"2024-09-11T23:34:52.875525+0000","last_became_peered":"2024-09-11T23:34:52.875525+0000","last_unstale":"2024-09-11T23:36:47.152450+0000","last_undegraded":"2024-09-11T23:36:47.152450+0000","last_fullsized":"2024-09-11T23:36:47.152450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:41.031816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145473+0000","last_change":"2024-09-11T23:34:50.924803+0000","last_active":"2024-09-11T23:36:47.145473+0000","last_peered":"2024-09-11T23:36:47.145473+0000","last_clean":"2024-09-11T23:36:47.145473+0000","last_became_active":"2024-09-11T23:34:50.924217+0000","last_became_peered":"2024-09-11T23:34:50.924217+0000","last_unstale":"2024-09-11T23:36:47.145473+0000","last_undegraded":"2024-09-11T23:36:47.145473+0000","last_fullsized":"2024-09-11T23:36:47.145473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:45.790592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142408+0000","last_change":"2024-09-11T23:35:08.851856+0000","last_active":"2024-09-11T23:36:47.142408+0000","last_peered":"2024-09-11T23:36:47.142408+0000","last_clean":"2024-09-11T23:36:47.142408+0000","last_became_active":"2024-09-11T23:35:08.851490+0000","last_became_peered":"2024-09-11T23:35:08.851490+0000","last_unstale":"2024-09-11T23:36:47.142408+0000","last_undegraded":"2024-09-11T23:36:47.142408+0000","last_fullsized":"2024-09-11T23:36:47.142408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:46.269022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145857+0000","last_change":"2024-09-11T23:34:50.848385+0000","last_active":"2024-09-11T23:36:47.145857+0000","last_peered":"2024-09-11T23:36:47.145857+0000","last_clean":"2024-09-11T23:36:47.145857+0000","last_became_active":"2024-09-11T23:34:50.847808+0000","last_became_peered":"2024-09-11T23:34:50.847808+0000","last_unstale":"2024-09-11T23:36:47.145857+0000","last_undegraded":"2024-09-11T23:36:47.145857+0000","last_fullsized":"2024-09-11T23:36:47.145857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:38.796622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327734+0000","last_change":"2024-09-11T23:34:52.867341+0000","last_active":"2024-09-11T23:36:47.327734+0000","last_peered":"2024-09-11T23:36:47.327734+0000","last_clean":"2024-09-11T23:36:47.327734+0000","last_became_active":"2024-09-11T23:34:52.866161+0000","last_became_peered":"2024-09-11T23:34:52.866161+0000","last_unstale":"2024-09-11T23:36:47.327734+0000","last_undegraded":"2024-09-11T23:36:47.327734+0000","last_fullsized":"2024-09-11T23:36:47.327734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:27.913705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057317+0000","last_change":"2024-09-11T23:34:50.851466+0000","last_active":"2024-09-11T23:36:47.057317+0000","last_peered":"2024-09-11T23:36:47.057317+0000","last_clean":"2024-09-11T23:36:47.057317+0000","last_became_active":"2024-09-11T23:34:50.851191+0000","last_became_peered":"2024-09-11T23:34:50.851191+0000","last_unstale":"2024-09-11T23:36:47.057317+0000","last_undegraded":"2024-09-11T23:36:47.057317+0000","last_fullsized":"2024-09-11T23:36:47.057317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:00.328658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152878+0000","last_change":"2024-09-11T23:34:52.872290+0000","last_active":"2024-09-11T23:36:47.152878+0000","last_peered":"2024-09-11T23:36:47.152878+0000","last_clean":"2024-09-11T23:36:47.152878+0000","last_became_active":"2024-09-11T23:34:52.871935+0000","last_became_peered":"2024-09-11T23:34:52.871935+0000","last_unstale":"2024-09-11T23:36:47.152878+0000","last_undegraded":"2024-09-11T23:36:47.152878+0000","last_fullsized":"2024-09-11T23:36:47.152878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:08.217548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'59","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:58.069867+0000","last_change":"2024-09-11T23:34:50.857763+0000","last_active":"2024-09-11T23:36:58.069867+0000","last_peered":"2024-09-11T23:36:58.069867+0000","last_clean":"2024-09-11T23:36:58.069867+0000","last_became_active":"2024-09-11T23:34:50.857427+0000","last_became_peered":"2024-09-11T23:34:50.857427+0000","last_unstale":"2024-09-11T23:36:58.069867+0000","last_undegraded":"2024-09-11T23:36:58.069867+0000","last_fullsized":"2024-09-11T23:36:58.069867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:59.703580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":19,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152618+0000","last_change":"2024-09-11T23:34:52.872426+0000","last_active":"2024-09-11T23:36:47.152618+0000","last_peered":"2024-09-11T23:36:47.152618+0000","last_clean":"2024-09-11T23:36:47.152618+0000","last_became_active":"2024-09-11T23:34:52.872090+0000","last_became_peered":"2024-09-11T23:34:52.872090+0000","last_unstale":"2024-09-11T23:36:47.152618+0000","last_undegraded":"2024-09-11T23:36:47.152618+0000","last_fullsized":"2024-09-11T23:36:47.152618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:04:00.963963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'7","reported_seq":148,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.620272+0000","last_change":"2024-09-11T23:34:50.924792+0000","last_active":"2024-09-11T23:36:56.620272+0000","last_peered":"2024-09-11T23:36:56.620272+0000","last_clean":"2024-09-11T23:36:56.620272+0000","last_became_active":"2024-09-11T23:34:50.924444+0000","last_became_peered":"2024-09-11T23:34:50.924444+0000","last_unstale":"2024-09-11T23:36:56.620272+0000","last_undegraded":"2024-09-11T23:36:56.620272+0000","last_fullsized":"2024-09-11T23:36:56.620272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:12.855314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152787+0000","last_change":"2024-09-11T23:34:52.872079+0000","last_active":"2024-09-11T23:36:47.152787+0000","last_peered":"2024-09-11T23:36:47.152787+0000","last_clean":"2024-09-11T23:36:47.152787+0000","last_became_active":"2024-09-11T23:34:52.871755+0000","last_became_peered":"2024-09-11T23:34:52.871755+0000","last_unstale":"2024-09-11T23:36:47.152787+0000","last_undegraded":"2024-09-11T23:36:47.152787+0000","last_fullsized":"2024-09-11T23:36:47.152787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:05.990832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"82'148","reported_seq":391,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:49.148435+0000","last_change":"2024-09-11T23:33:42.256521+0000","last_active":"2024-09-11T23:36:49.148435+0000","last_peered":"2024-09-11T23:36:49.148435+0000","last_clean":"2024-09-11T23:36:49.148435+0000","last_became_active":"2024-09-11T23:33:42.256041+0000","last_became_peered":"2024-09-11T23:33:42.256041+0000","last_unstale":"2024-09-11T23:36:49.148435+0000","last_undegraded":"2024-09-11T23:36:49.148435+0000","last_fullsized":"2024-09-11T23:36:49.148435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_clean_scrub_stamp":"2024-09-11T23:30:22.061515+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:21.391839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292343+0000","last_change":"2024-09-11T23:34:50.854087+0000","last_active":"2024-09-11T23:36:47.292343+0000","last_peered":"2024-09-11T23:36:47.292343+0000","last_clean":"2024-09-11T23:36:47.292343+0000","last_became_active":"2024-09-11T23:34:50.852346+0000","last_became_peered":"2024-09-11T23:34:50.852346+0000","last_unstale":"2024-09-11T23:36:47.292343+0000","last_undegraded":"2024-09-11T23:36:47.292343+0000","last_fullsized":"2024-09-11T23:36:47.292343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.287119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147041+0000","last_change":"2024-09-11T23:34:52.869198+0000","last_active":"2024-09-11T23:36:47.147041+0000","last_peered":"2024-09-11T23:36:47.147041+0000","last_clean":"2024-09-11T23:36:47.147041+0000","last_became_active":"2024-09-11T23:34:52.868621+0000","last_became_peered":"2024-09-11T23:34:52.868621+0000","last_unstale":"2024-09-11T23:36:47.147041+0000","last_undegraded":"2024-09-11T23:36:47.147041+0000","last_fullsized":"2024-09-11T23:36:47.147041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:17.430654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152654+0000","last_change":"2024-09-11T23:34:50.852752+0000","last_active":"2024-09-11T23:36:47.152654+0000","last_peered":"2024-09-11T23:36:47.152654+0000","last_clean":"2024-09-11T23:36:47.152654+0000","last_became_active":"2024-09-11T23:34:50.851895+0000","last_became_peered":"2024-09-11T23:34:50.851895+0000","last_unstale":"2024-09-11T23:36:47.152654+0000","last_undegraded":"2024-09-11T23:36:47.152654+0000","last_fullsized":"2024-09-11T23:36:47.152654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:46.894432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057125+0000","last_change":"2024-09-11T23:34:52.866191+0000","last_active":"2024-09-11T23:36:47.057125+0000","last_peered":"2024-09-11T23:36:47.057125+0000","last_clean":"2024-09-11T23:36:47.057125+0000","last_became_active":"2024-09-11T23:34:52.865858+0000","last_became_peered":"2024-09-11T23:34:52.865858+0000","last_unstale":"2024-09-11T23:36:47.057125+0000","last_undegraded":"2024-09-11T23:36:47.057125+0000","last_fullsized":"2024-09-11T23:36:47.057125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:21.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.620820+0000","last_change":"2024-09-11T23:34:50.924125+0000","last_active":"2024-09-11T23:36:56.620820+0000","last_peered":"2024-09-11T23:36:56.620820+0000","last_clean":"2024-09-11T23:36:56.620820+0000","last_became_active":"2024-09-11T23:34:50.923780+0000","last_became_peered":"2024-09-11T23:34:50.923780+0000","last_unstale":"2024-09-11T23:36:56.620820+0000","last_undegraded":"2024-09-11T23:36:56.620820+0000","last_fullsized":"2024-09-11T23:36:56.620820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:39.774827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153717+0000","last_change":"2024-09-11T23:34:52.868364+0000","last_active":"2024-09-11T23:36:47.153717+0000","last_peered":"2024-09-11T23:36:47.153717+0000","last_clean":"2024-09-11T23:36:47.153717+0000","last_became_active":"2024-09-11T23:34:52.867913+0000","last_became_peered":"2024-09-11T23:34:52.867913+0000","last_unstale":"2024-09-11T23:36:47.153717+0000","last_undegraded":"2024-09-11T23:36:47.153717+0000","last_fullsized":"2024-09-11T23:36:47.153717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.571825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'8","reported_seq":149,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.615174+0000","last_change":"2024-09-11T23:34:50.852421+0000","last_active":"2024-09-11T23:36:56.615174+0000","last_peered":"2024-09-11T23:36:56.615174+0000","last_clean":"2024-09-11T23:36:56.615174+0000","last_became_active":"2024-09-11T23:34:50.851898+0000","last_became_peered":"2024-09-11T23:34:50.851898+0000","last_unstale":"2024-09-11T23:36:56.615174+0000","last_undegraded":"2024-09-11T23:36:56.615174+0000","last_fullsized":"2024-09-11T23:36:56.615174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.661276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1767,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156409+0000","last_change":"2024-09-11T23:34:52.879995+0000","last_active":"2024-09-11T23:36:47.156409+0000","last_peered":"2024-09-11T23:36:47.156409+0000","last_clean":"2024-09-11T23:36:47.156409+0000","last_became_active":"2024-09-11T23:34:52.879678+0000","last_became_peered":"2024-09-11T23:34:52.879678+0000","last_unstale":"2024-09-11T23:36:47.156409+0000","last_undegraded":"2024-09-11T23:36:47.156409+0000","last_fullsized":"2024-09-11T23:36:47.156409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:47.456674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143130+0000","last_change":"2024-09-11T23:34:50.924957+0000","last_active":"2024-09-11T23:36:47.143130+0000","last_peered":"2024-09-11T23:36:47.143130+0000","last_clean":"2024-09-11T23:36:47.143130+0000","last_became_active":"2024-09-11T23:34:50.924500+0000","last_became_peered":"2024-09-11T23:34:50.924500+0000","last_unstale":"2024-09-11T23:36:47.143130+0000","last_undegraded":"2024-09-11T23:36:47.143130+0000","last_fullsized":"2024-09-11T23:36:47.143130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:39.268842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156159+0000","last_change":"2024-09-11T23:34:52.867337+0000","last_active":"2024-09-11T23:36:47.156159+0000","last_peered":"2024-09-11T23:36:47.156159+0000","last_clean":"2024-09-11T23:36:47.156159+0000","last_became_active":"2024-09-11T23:34:52.866799+0000","last_became_peered":"2024-09-11T23:34:52.866799+0000","last_unstale":"2024-09-11T23:36:47.156159+0000","last_undegraded":"2024-09-11T23:36:47.156159+0000","last_fullsized":"2024-09-11T23:36:47.156159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:38.177135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145946+0000","last_change":"2024-09-11T23:34:50.925156+0000","last_active":"2024-09-11T23:36:47.145946+0000","last_peered":"2024-09-11T23:36:47.145946+0000","last_clean":"2024-09-11T23:36:47.145946+0000","last_became_active":"2024-09-11T23:34:50.924282+0000","last_became_peered":"2024-09-11T23:34:50.924282+0000","last_unstale":"2024-09-11T23:36:47.145946+0000","last_undegraded":"2024-09-11T23:36:47.145946+0000","last_fullsized":"2024-09-11T23:36:47.145946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:47.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057324+0000","last_change":"2024-09-11T23:34:52.864993+0000","last_active":"2024-09-11T23:36:47.057324+0000","last_peered":"2024-09-11T23:36:47.057324+0000","last_clean":"2024-09-11T23:36:47.057324+0000","last_became_active":"2024-09-11T23:34:52.864267+0000","last_became_peered":"2024-09-11T23:34:52.864267+0000","last_unstale":"2024-09-11T23:36:47.057324+0000","last_undegraded":"2024-09-11T23:36:47.057324+0000","last_fullsized":"2024-09-11T23:36:47.057324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:29.912068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'75","reported_seq":150,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:58.074331+0000","last_change":"2024-09-11T23:34:50.849864+0000","last_active":"2024-09-11T23:36:58.074331+0000","last_peered":"2024-09-11T23:36:58.074331+0000","last_clean":"2024-09-11T23:36:58.074331+0000","last_became_active":"2024-09-11T23:34:50.849577+0000","last_became_peered":"2024-09-11T23:34:50.849577+0000","last_unstale":"2024-09-11T23:36:58.074331+0000","last_undegraded":"2024-09-11T23:36:58.074331+0000","last_fullsized":"2024-09-11T23:36:58.074331+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:06.845000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":170,"num_read_kb":149,"num_write":133,"num_write_kb":1159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147479+0000","last_change":"2024-09-11T23:34:52.872028+0000","last_active":"2024-09-11T23:36:47.147479+0000","last_peered":"2024-09-11T23:36:47.147479+0000","last_clean":"2024-09-11T23:36:47.147479+0000","last_became_active":"2024-09-11T23:34:52.871723+0000","last_became_peered":"2024-09-11T23:34:52.871723+0000","last_unstale":"2024-09-11T23:36:47.147479+0000","last_undegraded":"2024-09-11T23:36:47.147479+0000","last_fullsized":"2024-09-11T23:36:47.147479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:44.656218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292619+0000","last_change":"2024-09-11T23:34:50.860314+0000","last_active":"2024-09-11T23:36:47.292619+0000","last_peered":"2024-09-11T23:36:47.292619+0000","last_clean":"2024-09-11T23:36:47.292619+0000","last_became_active":"2024-09-11T23:34:50.859953+0000","last_became_peered":"2024-09-11T23:34:50.859953+0000","last_unstale":"2024-09-11T23:36:47.292619+0000","last_undegraded":"2024-09-11T23:36:47.292619+0000","last_fullsized":"2024-09-11T23:36:47.292619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:46.705864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328058+0000","last_change":"2024-09-11T23:34:52.867190+0000","last_active":"2024-09-11T23:36:47.328058+0000","last_peered":"2024-09-11T23:36:47.328058+0000","last_clean":"2024-09-11T23:36:47.328058+0000","last_became_active":"2024-09-11T23:34:52.866829+0000","last_became_peered":"2024-09-11T23:34:52.866829+0000","last_unstale":"2024-09-11T23:36:47.328058+0000","last_undegraded":"2024-09-11T23:36:47.328058+0000","last_fullsized":"2024-09-11T23:36:47.328058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.752006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058275+0000","last_change":"2024-09-11T23:34:50.924566+0000","last_active":"2024-09-11T23:36:47.058275+0000","last_peered":"2024-09-11T23:36:47.058275+0000","last_clean":"2024-09-11T23:36:47.058275+0000","last_became_active":"2024-09-11T23:34:50.924284+0000","last_became_peered":"2024-09-11T23:34:50.924284+0000","last_unstale":"2024-09-11T23:36:47.058275+0000","last_undegraded":"2024-09-11T23:36:47.058275+0000","last_fullsized":"2024-09-11T23:36:47.058275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:54.246154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145487+0000","last_change":"2024-09-11T23:34:52.869302+0000","last_active":"2024-09-11T23:36:47.145487+0000","last_peered":"2024-09-11T23:36:47.145487+0000","last_clean":"2024-09-11T23:36:47.145487+0000","last_became_active":"2024-09-11T23:34:52.868960+0000","last_became_peered":"2024-09-11T23:34:52.868960+0000","last_unstale":"2024-09-11T23:36:47.145487+0000","last_undegraded":"2024-09-11T23:36:47.145487+0000","last_fullsized":"2024-09-11T23:36:47.145487+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.561712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146271+0000","last_change":"2024-09-11T23:34:50.854573+0000","last_active":"2024-09-11T23:36:47.146271+0000","last_peered":"2024-09-11T23:36:47.146271+0000","last_clean":"2024-09-11T23:36:47.146271+0000","last_became_active":"2024-09-11T23:34:50.854165+0000","last_became_peered":"2024-09-11T23:34:50.854165+0000","last_unstale":"2024-09-11T23:36:47.146271+0000","last_undegraded":"2024-09-11T23:36:47.146271+0000","last_fullsized":"2024-09-11T23:36:47.146271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:57:31.589495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327542+0000","last_change":"2024-09-11T23:34:52.875987+0000","last_active":"2024-09-11T23:36:47.327542+0000","last_peered":"2024-09-11T23:36:47.327542+0000","last_clean":"2024-09-11T23:36:47.327542+0000","last_became_active":"2024-09-11T23:34:52.875662+0000","last_became_peered":"2024-09-11T23:34:52.875662+0000","last_unstale":"2024-09-11T23:36:47.327542+0000","last_undegraded":"2024-09-11T23:36:47.327542+0000","last_fullsized":"2024-09-11T23:36:47.327542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:16.514938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058234+0000","last_change":"2024-09-11T23:34:50.850726+0000","last_active":"2024-09-11T23:36:47.058234+0000","last_peered":"2024-09-11T23:36:47.058234+0000","last_clean":"2024-09-11T23:36:47.058234+0000","last_became_active":"2024-09-11T23:34:50.850381+0000","last_became_peered":"2024-09-11T23:34:50.850381+0000","last_unstale":"2024-09-11T23:36:47.058234+0000","last_undegraded":"2024-09-11T23:36:47.058234+0000","last_fullsized":"2024-09-11T23:36:47.058234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:50.431293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146377+0000","last_change":"2024-09-11T23:34:52.870652+0000","last_active":"2024-09-11T23:36:47.146377+0000","last_peered":"2024-09-11T23:36:47.146377+0000","last_clean":"2024-09-11T23:36:47.146377+0000","last_became_active":"2024-09-11T23:34:52.870303+0000","last_became_peered":"2024-09-11T23:34:52.870303+0000","last_unstale":"2024-09-11T23:36:47.146377+0000","last_undegraded":"2024-09-11T23:36:47.146377+0000","last_fullsized":"2024-09-11T23:36:47.146377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:17.670981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327509+0000","last_change":"2024-09-11T23:35:08.854490+0000","last_active":"2024-09-11T23:36:47.327509+0000","last_peered":"2024-09-11T23:36:47.327509+0000","last_clean":"2024-09-11T23:36:47.327509+0000","last_became_active":"2024-09-11T23:35:08.849820+0000","last_became_peered":"2024-09-11T23:35:08.849820+0000","last_unstale":"2024-09-11T23:36:47.327509+0000","last_undegraded":"2024-09-11T23:36:47.327509+0000","last_fullsized":"2024-09-11T23:36:47.327509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:09.509867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152580+0000","last_change":"2024-09-11T23:34:52.869695+0000","last_active":"2024-09-11T23:36:47.152580+0000","last_peered":"2024-09-11T23:36:47.152580+0000","last_clean":"2024-09-11T23:36:47.152580+0000","last_became_active":"2024-09-11T23:34:52.869367+0000","last_became_peered":"2024-09-11T23:34:52.869367+0000","last_unstale":"2024-09-11T23:36:47.152580+0000","last_undegraded":"2024-09-11T23:36:47.152580+0000","last_fullsized":"2024-09-11T23:36:47.152580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:16.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153822+0000","last_change":"2024-09-11T23:34:52.872778+0000","last_active":"2024-09-11T23:36:47.153822+0000","last_peered":"2024-09-11T23:36:47.153822+0000","last_clean":"2024-09-11T23:36:47.153822+0000","last_became_active":"2024-09-11T23:34:52.872451+0000","last_became_peered":"2024-09-11T23:34:52.872451+0000","last_unstale":"2024-09-11T23:36:47.153822+0000","last_undegraded":"2024-09-11T23:36:47.153822+0000","last_fullsized":"2024-09-11T23:36:47.153822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:51.769927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145825+0000","last_change":"2024-09-11T23:34:50.849675+0000","last_active":"2024-09-11T23:36:47.145825+0000","last_peered":"2024-09-11T23:36:47.145825+0000","last_clean":"2024-09-11T23:36:47.145825+0000","last_became_active":"2024-09-11T23:34:50.849228+0000","last_became_peered":"2024-09-11T23:34:50.849228+0000","last_unstale":"2024-09-11T23:36:47.145825+0000","last_undegraded":"2024-09-11T23:36:47.145825+0000","last_fullsized":"2024-09-11T23:36:47.145825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:50.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058399+0000","last_change":"2024-09-11T23:34:52.867739+0000","last_active":"2024-09-11T23:36:47.058399+0000","last_peered":"2024-09-11T23:36:47.058399+0000","last_clean":"2024-09-11T23:36:47.058399+0000","last_became_active":"2024-09-11T23:34:52.866613+0000","last_became_peered":"2024-09-11T23:34:52.866613+0000","last_unstale":"2024-09-11T23:36:47.058399+0000","last_undegraded":"2024-09-11T23:36:47.058399+0000","last_fullsized":"2024-09-11T23:36:47.058399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:49.348386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'13","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145845+0000","last_change":"2024-09-11T23:34:50.851401+0000","last_active":"2024-09-11T23:36:47.145845+0000","last_peered":"2024-09-11T23:36:47.145845+0000","last_clean":"2024-09-11T23:36:47.145845+0000","last_became_active":"2024-09-11T23:34:50.851068+0000","last_became_peered":"2024-09-11T23:34:50.851068+0000","last_unstale":"2024-09-11T23:36:47.145845+0000","last_undegraded":"2024-09-11T23:36:47.145845+0000","last_fullsized":"2024-09-11T23:36:47.145845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:09.673864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328160+0000","last_change":"2024-09-11T23:34:52.865216+0000","last_active":"2024-09-11T23:36:47.328160+0000","last_peered":"2024-09-11T23:36:47.328160+0000","last_clean":"2024-09-11T23:36:47.328160+0000","last_became_active":"2024-09-11T23:34:52.864917+0000","last_became_peered":"2024-09-11T23:34:52.864917+0000","last_unstale":"2024-09-11T23:36:47.328160+0000","last_undegraded":"2024-09-11T23:36:47.328160+0000","last_fullsized":"2024-09-11T23:36:47.328160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:36.469793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147661+0000","last_change":"2024-09-11T23:34:50.924291+0000","last_active":"2024-09-11T23:36:47.147661+0000","last_peered":"2024-09-11T23:36:47.147661+0000","last_clean":"2024-09-11T23:36:47.147661+0000","last_became_active":"2024-09-11T23:34:50.923892+0000","last_became_peered":"2024-09-11T23:34:50.923892+0000","last_unstale":"2024-09-11T23:36:47.147661+0000","last_undegraded":"2024-09-11T23:36:47.147661+0000","last_fullsized":"2024-09-11T23:36:47.147661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:52.266778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147431+0000","last_change":"2024-09-11T23:34:52.875995+0000","last_active":"2024-09-11T23:36:47.147431+0000","last_peered":"2024-09-11T23:36:47.147431+0000","last_clean":"2024-09-11T23:36:47.147431+0000","last_became_active":"2024-09-11T23:34:52.875654+0000","last_became_peered":"2024-09-11T23:34:52.875654+0000","last_unstale":"2024-09-11T23:36:47.147431+0000","last_undegraded":"2024-09-11T23:36:47.147431+0000","last_fullsized":"2024-09-11T23:36:47.147431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:19.218291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":154,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:57.530808+0000","last_change":"2024-09-11T23:34:50.853881+0000","last_active":"2024-09-11T23:36:57.530808+0000","last_peered":"2024-09-11T23:36:57.530808+0000","last_clean":"2024-09-11T23:36:57.530808+0000","last_became_active":"2024-09-11T23:34:50.853576+0000","last_became_peered":"2024-09-11T23:34:50.853576+0000","last_unstale":"2024-09-11T23:36:57.530808+0000","last_undegraded":"2024-09-11T23:36:57.530808+0000","last_fullsized":"2024-09-11T23:36:57.530808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:38.550587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":802091,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1385,"num_read_kb":853,"num_write":447,"num_write_kb":2128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2406273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612646,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1753088,"data_stored":1237974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005104,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32484,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712668,"statfs":{"total":95995035648,"available":95961772032,"internally_reserved":0,"allocated":1327104,"data_stored":836376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234037,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28316,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716836,"statfs":{"total":95995035648,"available":95966040064,"internally_reserved":0,"allocated":1323008,"data_stored":836754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495674,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1527808,"data_stored":1039258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":444056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888131,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1167336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018953,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":444759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247884,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1462272,"data_stored":967518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444115,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":514963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607639,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":515063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":444750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":73025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":402083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:37:02.898 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:37:02.898 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:37:03.156 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:03.418 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:03 smithi133 ceph-mon[31430]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:37:03.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:03 smithi138 ceph-mon[35142]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:37:03.543 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:03 smithi178 ceph-mon[35882]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:37:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.640+0000 7f6285a06640 1 -- 172.21.15.133:0/1712921995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6280102d40 msgr2=0x7f6280103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.640+0000 7f6285a06640 1 --2- 172.21.15.133:0/1712921995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6280102d40 0x7f6280103120 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6270009900 tx=0x7f627002a480 comp rx=0 tx=0).stop 2024-09-11T23:37:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.641+0000 7f6285a06640 1 -- 172.21.15.133:0/1712921995 shutdown_connections 2024-09-11T23:37:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.641+0000 7f6285a06640 1 --2- 172.21.15.133:0/1712921995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f628010c170 0x7f628010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.641+0000 7f6285a06640 1 --2- 172.21.15.133:0/1712921995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6280103660 0x7f628010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.641+0000 7f6285a06640 1 --2- 172.21.15.133:0/1712921995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6280102d40 0x7f6280103120 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.641+0000 7f6285a06640 1 -- 172.21.15.133:0/1712921995 >> 172.21.15.133:0/1712921995 conn(0x7f62800fc7a0 msgr2=0x7f62800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.641+0000 7f6285a06640 1 -- 172.21.15.133:0/1712921995 shutdown_connections 2024-09-11T23:37:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.641+0000 7f6285a06640 1 -- 172.21.15.133:0/1712921995 wait complete. 2024-09-11T23:37:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.641+0000 7f6285a06640 1 Processor -- start 2024-09-11T23:37:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.641+0000 7f6285a06640 1 -- start start 2024-09-11T23:37:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f6285a06640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6280102d40 0x7f6280198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f6285a06640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6280103660 0x7f62801989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f6285a06640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f628010c170 0x7f628019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f6285a06640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6280111f70 con 0x7f6280102d40 2024-09-11T23:37:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f6285a06640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6280111df0 con 0x7f628010c170 2024-09-11T23:37:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f6285a06640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f62801120f0 con 0x7f6280103660 2024-09-11T23:37:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f627effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6280102d40 0x7f6280198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f627effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6280102d40 0x7f6280198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60348/0 (socket says 172.21.15.133:60348) 2024-09-11T23:37:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f627effd640 1 -- 172.21.15.133:0/1827549754 learned_addr learned my addr 172.21.15.133:0/1827549754 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f627f7fe640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f628010c170 0x7f628019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f62767fc640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6280103660 0x7f62801989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f627effd640 1 -- 172.21.15.133:0/1827549754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6280103660 msgr2=0x7f62801989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f627effd640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6280103660 0x7f62801989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f627effd640 1 -- 172.21.15.133:0/1827549754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f628010c170 msgr2=0x7f628019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f627effd640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f628010c170 0x7f628019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f627effd640 1 -- 172.21.15.133:0/1827549754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628019d420 con 0x7f6280102d40 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f627effd640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6280102d40 0x7f6280198470 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6270006010 tx=0x7f62700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f6284a04640 1 -- 172.21.15.133:0/1827549754 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627003f030 con 0x7f6280102d40 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.642+0000 7f6285a06640 1 -- 172.21.15.133:0/1827549754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628019d6b0 con 0x7f6280102d40 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.643+0000 7f6284a04640 1 -- 172.21.15.133:0/1827549754 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6270003ce0 con 0x7f6280102d40 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.643+0000 7f6284a04640 1 -- 172.21.15.133:0/1827549754 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270035070 con 0x7f6280102d40 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.643+0000 7f6285a06640 1 -- 172.21.15.133:0/1827549754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62801a4ff0 con 0x7f6280102d40 2024-09-11T23:37:03.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.644+0000 7f6285a06640 1 -- 172.21.15.133:0/1827549754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6280103120 con 0x7f6280102d40 2024-09-11T23:37:03.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.645+0000 7f6284a04640 1 -- 172.21.15.133:0/1827549754 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f6270042030 con 0x7f6280102d40 2024-09-11T23:37:03.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.646+0000 7f6284a04640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f626c0a2ea0 0x7f626c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:03.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.646+0000 7f62767fc640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f626c0a2ea0 0x7f626c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:03.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.646+0000 7f6284a04640 1 -- 172.21.15.133:0/1827549754 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f62700efc00 con 0x7f6280102d40 2024-09-11T23:37:03.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.646+0000 7f62767fc640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f626c0a2ea0 0x7f626c0a5360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f62680058c0 tx=0x7f6268005850 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:03.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.650+0000 7f6284a04640 1 -- 172.21.15.133:0/1827549754 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62700bc710 con 0x7f6280102d40 2024-09-11T23:37:03.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.818+0000 7f6285a06640 1 -- 172.21.15.133:0/1827549754 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6280062630 con 0x7f626c0a2ea0 2024-09-11T23:37:03.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.824+0000 7f6284a04640 1 -- 172.21.15.133:0/1827549754 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257811 (secure 0 0 0) 0x7f6280062630 con 0x7f626c0a2ea0 2024-09-11T23:37:03.824 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:37:03.830 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T23:37:03.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.827+0000 7f6285a06640 1 -- 172.21.15.133:0/1827549754 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f626c0a2ea0 msgr2=0x7f626c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:03.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.827+0000 7f6285a06640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f626c0a2ea0 0x7f626c0a5360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f62680058c0 tx=0x7f6268005850 comp rx=0 tx=0).stop 2024-09-11T23:37:03.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.827+0000 7f6285a06640 1 -- 172.21.15.133:0/1827549754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6280102d40 msgr2=0x7f6280198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:03.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.827+0000 7f6285a06640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6280102d40 0x7f6280198470 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6270006010 tx=0x7f62700042f0 comp rx=0 tx=0).stop 2024-09-11T23:37:03.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.827+0000 7f6285a06640 1 -- 172.21.15.133:0/1827549754 shutdown_connections 2024-09-11T23:37:03.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.827+0000 7f6285a06640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f626c0a2ea0 0x7f626c0a5360 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:03.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.827+0000 7f6285a06640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f628010c170 0x7f628019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:03.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.827+0000 7f6285a06640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6280103660 0x7f62801989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:03.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.827+0000 7f6285a06640 1 --2- 172.21.15.133:0/1827549754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6280102d40 0x7f6280198470 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:03.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.827+0000 7f6285a06640 1 -- 172.21.15.133:0/1827549754 >> 172.21.15.133:0/1827549754 conn(0x7f62800fc7a0 msgr2=0x7f62800fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:03.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.828+0000 7f6285a06640 1 -- 172.21.15.133:0/1827549754 shutdown_connections 2024-09-11T23:37:03.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:03.828+0000 7f6285a06640 1 -- 172.21.15.133:0/1827549754 wait complete. 2024-09-11T23:37:04.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:04 smithi138 ceph-mon[35142]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-09-11T23:37:04.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:04 smithi138 ceph-mon[35142]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:37:04.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:04 smithi133 ceph-mon[31430]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-09-11T23:37:04.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:04 smithi133 ceph-mon[31430]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:37:04.517 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-11T23:37:03.071782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1392693,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1627,"num_read_kb":1188,"num_write":651,"num_write_kb":5069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1002,"ondisk_log_size":1002,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359708,"kb_used_data":14596,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582116,"statfs":{"total":1151940427776,"available":1151572086784,"internally_reserved":0,"allocated":14946304,"data_stored":9090711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":204645,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":309,"num_read_kb":366,"num_write":87,"num_write_kb":665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002605"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327868+0000","last_change":"2024-09-11T23:34:52.867694+0000","last_active":"2024-09-11T23:36:47.327868+0000","last_peered":"2024-09-11T23:36:47.327868+0000","last_clean":"2024-09-11T23:36:47.327868+0000","last_became_active":"2024-09-11T23:34:52.867368+0000","last_became_peered":"2024-09-11T23:34:52.867368+0000","last_unstale":"2024-09-11T23:36:47.327868+0000","last_undegraded":"2024-09-11T23:36:47.327868+0000","last_fullsized":"2024-09-11T23:36:47.327868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:46:07.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328024+0000","last_change":"2024-09-11T23:34:52.871881+0000","last_active":"2024-09-11T23:36:47.328024+0000","last_peered":"2024-09-11T23:36:47.328024+0000","last_clean":"2024-09-11T23:36:47.328024+0000","last_became_active":"2024-09-11T23:34:52.871609+0000","last_became_peered":"2024-09-11T23:34:52.871609+0000","last_unstale":"2024-09-11T23:36:47.328024+0000","last_undegraded":"2024-09-11T23:36:47.328024+0000","last_fullsized":"2024-09-11T23:36:47.328024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.938584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327774+0000","last_change":"2024-09-11T23:34:52.863931+0000","last_active":"2024-09-11T23:36:47.327774+0000","last_peered":"2024-09-11T23:36:47.327774+0000","last_clean":"2024-09-11T23:36:47.327774+0000","last_became_active":"2024-09-11T23:34:52.863637+0000","last_became_peered":"2024-09-11T23:34:52.863637+0000","last_unstale":"2024-09-11T23:36:47.327774+0000","last_undegraded":"2024-09-11T23:36:47.327774+0000","last_fullsized":"2024-09-11T23:36:47.327774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:44.118865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292453+0000","last_change":"2024-09-11T23:34:52.879619+0000","last_active":"2024-09-11T23:36:47.292453+0000","last_peered":"2024-09-11T23:36:47.292453+0000","last_clean":"2024-09-11T23:36:47.292453+0000","last_became_active":"2024-09-11T23:34:52.878695+0000","last_became_peered":"2024-09-11T23:34:52.878695+0000","last_unstale":"2024-09-11T23:36:47.292453+0000","last_undegraded":"2024-09-11T23:36:47.292453+0000","last_fullsized":"2024-09-11T23:36:47.292453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:30.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145593+0000","last_change":"2024-09-11T23:34:52.869363+0000","last_active":"2024-09-11T23:36:47.145593+0000","last_peered":"2024-09-11T23:36:47.145593+0000","last_clean":"2024-09-11T23:36:47.145593+0000","last_became_active":"2024-09-11T23:34:52.869027+0000","last_became_peered":"2024-09-11T23:34:52.869027+0000","last_unstale":"2024-09-11T23:36:47.145593+0000","last_undegraded":"2024-09-11T23:36:47.145593+0000","last_fullsized":"2024-09-11T23:36:47.145593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:15.049324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156014+0000","last_change":"2024-09-11T23:34:52.880665+0000","last_active":"2024-09-11T23:36:47.156014+0000","last_peered":"2024-09-11T23:36:47.156014+0000","last_clean":"2024-09-11T23:36:47.156014+0000","last_became_active":"2024-09-11T23:34:52.880172+0000","last_became_peered":"2024-09-11T23:34:52.880172+0000","last_unstale":"2024-09-11T23:36:47.156014+0000","last_undegraded":"2024-09-11T23:36:47.156014+0000","last_fullsized":"2024-09-11T23:36:47.156014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:12.025877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152489+0000","last_change":"2024-09-11T23:34:52.873897+0000","last_active":"2024-09-11T23:36:47.152489+0000","last_peered":"2024-09-11T23:36:47.152489+0000","last_clean":"2024-09-11T23:36:47.152489+0000","last_became_active":"2024-09-11T23:34:52.873219+0000","last_became_peered":"2024-09-11T23:34:52.873219+0000","last_unstale":"2024-09-11T23:36:47.152489+0000","last_undegraded":"2024-09-11T23:36:47.152489+0000","last_fullsized":"2024-09-11T23:36:47.152489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:47.125682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145794+0000","last_change":"2024-09-11T23:34:52.870699+0000","last_active":"2024-09-11T23:36:47.145794+0000","last_peered":"2024-09-11T23:36:47.145794+0000","last_clean":"2024-09-11T23:36:47.145794+0000","last_became_active":"2024-09-11T23:34:52.870314+0000","last_became_peered":"2024-09-11T23:34:52.870314+0000","last_unstale":"2024-09-11T23:36:47.145794+0000","last_undegraded":"2024-09-11T23:36:47.145794+0000","last_fullsized":"2024-09-11T23:36:47.145794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:41.263159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058666+0000","last_change":"2024-09-11T23:34:52.869700+0000","last_active":"2024-09-11T23:36:47.058666+0000","last_peered":"2024-09-11T23:36:47.058666+0000","last_clean":"2024-09-11T23:36:47.058666+0000","last_became_active":"2024-09-11T23:34:52.869398+0000","last_became_peered":"2024-09-11T23:34:52.869398+0000","last_unstale":"2024-09-11T23:36:47.058666+0000","last_undegraded":"2024-09-11T23:36:47.058666+0000","last_fullsized":"2024-09-11T23:36:47.058666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:49.776525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156161+0000","last_change":"2024-09-11T23:34:52.879999+0000","last_active":"2024-09-11T23:36:47.156161+0000","last_peered":"2024-09-11T23:36:47.156161+0000","last_clean":"2024-09-11T23:36:47.156161+0000","last_became_active":"2024-09-11T23:34:52.879646+0000","last_became_peered":"2024-09-11T23:34:52.879646+0000","last_unstale":"2024-09-11T23:36:47.156161+0000","last_undegraded":"2024-09-11T23:36:47.156161+0000","last_fullsized":"2024-09-11T23:36:47.156161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.058704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152665+0000","last_change":"2024-09-11T23:34:52.878740+0000","last_active":"2024-09-11T23:36:47.152665+0000","last_peered":"2024-09-11T23:36:47.152665+0000","last_clean":"2024-09-11T23:36:47.152665+0000","last_became_active":"2024-09-11T23:34:52.878120+0000","last_became_peered":"2024-09-11T23:34:52.878120+0000","last_unstale":"2024-09-11T23:36:47.152665+0000","last_undegraded":"2024-09-11T23:36:47.152665+0000","last_fullsized":"2024-09-11T23:36:47.152665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292447+0000","last_change":"2024-09-11T23:34:52.873409+0000","last_active":"2024-09-11T23:36:47.292447+0000","last_peered":"2024-09-11T23:36:47.292447+0000","last_clean":"2024-09-11T23:36:47.292447+0000","last_became_active":"2024-09-11T23:34:52.872959+0000","last_became_peered":"2024-09-11T23:34:52.872959+0000","last_unstale":"2024-09-11T23:36:47.292447+0000","last_undegraded":"2024-09-11T23:36:47.292447+0000","last_fullsized":"2024-09-11T23:36:47.292447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:38.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057362+0000","last_change":"2024-09-11T23:34:52.869164+0000","last_active":"2024-09-11T23:36:47.057362+0000","last_peered":"2024-09-11T23:36:47.057362+0000","last_clean":"2024-09-11T23:36:47.057362+0000","last_became_active":"2024-09-11T23:34:52.868847+0000","last_became_peered":"2024-09-11T23:34:52.868847+0000","last_unstale":"2024-09-11T23:36:47.057362+0000","last_undegraded":"2024-09-11T23:36:47.057362+0000","last_fullsized":"2024-09-11T23:36:47.057362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.764510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327562+0000","last_change":"2024-09-11T23:34:52.869494+0000","last_active":"2024-09-11T23:36:47.327562+0000","last_peered":"2024-09-11T23:36:47.327562+0000","last_clean":"2024-09-11T23:36:47.327562+0000","last_became_active":"2024-09-11T23:34:52.869178+0000","last_became_peered":"2024-09-11T23:34:52.869178+0000","last_unstale":"2024-09-11T23:36:47.327562+0000","last_undegraded":"2024-09-11T23:36:47.327562+0000","last_fullsized":"2024-09-11T23:36:47.327562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:01:23.431564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327741+0000","last_change":"2024-09-11T23:34:52.866269+0000","last_active":"2024-09-11T23:36:47.327741+0000","last_peered":"2024-09-11T23:36:47.327741+0000","last_clean":"2024-09-11T23:36:47.327741+0000","last_became_active":"2024-09-11T23:34:52.865980+0000","last_became_peered":"2024-09-11T23:34:52.865980+0000","last_unstale":"2024-09-11T23:36:47.327741+0000","last_undegraded":"2024-09-11T23:36:47.327741+0000","last_fullsized":"2024-09-11T23:36:47.327741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:48.492252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058174+0000","last_change":"2024-09-11T23:34:52.862192+0000","last_active":"2024-09-11T23:36:47.058174+0000","last_peered":"2024-09-11T23:36:47.058174+0000","last_clean":"2024-09-11T23:36:47.058174+0000","last_became_active":"2024-09-11T23:34:52.861841+0000","last_became_peered":"2024-09-11T23:34:52.861841+0000","last_unstale":"2024-09-11T23:36:47.058174+0000","last_undegraded":"2024-09-11T23:36:47.058174+0000","last_fullsized":"2024-09-11T23:36:47.058174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.258948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327836+0000","last_change":"2024-09-11T23:34:52.870984+0000","last_active":"2024-09-11T23:36:47.327836+0000","last_peered":"2024-09-11T23:36:47.327836+0000","last_clean":"2024-09-11T23:36:47.327836+0000","last_became_active":"2024-09-11T23:34:52.870435+0000","last_became_peered":"2024-09-11T23:34:52.870435+0000","last_unstale":"2024-09-11T23:36:47.327836+0000","last_undegraded":"2024-09-11T23:36:47.327836+0000","last_fullsized":"2024-09-11T23:36:47.327836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:25:34.391537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143297+0000","last_change":"2024-09-11T23:34:52.873176+0000","last_active":"2024-09-11T23:36:47.143297+0000","last_peered":"2024-09-11T23:36:47.143297+0000","last_clean":"2024-09-11T23:36:47.143297+0000","last_became_active":"2024-09-11T23:34:52.872862+0000","last_became_peered":"2024-09-11T23:34:52.872862+0000","last_unstale":"2024-09-11T23:36:47.143297+0000","last_undegraded":"2024-09-11T23:36:47.143297+0000","last_fullsized":"2024-09-11T23:36:47.143297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:38.058806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058317+0000","last_change":"2024-09-11T23:34:52.871346+0000","last_active":"2024-09-11T23:36:47.058317+0000","last_peered":"2024-09-11T23:36:47.058317+0000","last_clean":"2024-09-11T23:36:47.058317+0000","last_became_active":"2024-09-11T23:34:52.870981+0000","last_became_peered":"2024-09-11T23:34:52.870981+0000","last_unstale":"2024-09-11T23:36:47.058317+0000","last_undegraded":"2024-09-11T23:36:47.058317+0000","last_fullsized":"2024-09-11T23:36:47.058317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:56.973998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327601+0000","last_change":"2024-09-11T23:34:52.867058+0000","last_active":"2024-09-11T23:36:47.327601+0000","last_peered":"2024-09-11T23:36:47.327601+0000","last_clean":"2024-09-11T23:36:47.327601+0000","last_became_active":"2024-09-11T23:34:52.866687+0000","last_became_peered":"2024-09-11T23:34:52.866687+0000","last_unstale":"2024-09-11T23:36:47.327601+0000","last_undegraded":"2024-09-11T23:36:47.327601+0000","last_fullsized":"2024-09-11T23:36:47.327601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:09.019987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057475+0000","last_change":"2024-09-11T23:34:52.870882+0000","last_active":"2024-09-11T23:36:47.057475+0000","last_peered":"2024-09-11T23:36:47.057475+0000","last_clean":"2024-09-11T23:36:47.057475+0000","last_became_active":"2024-09-11T23:34:52.870667+0000","last_became_peered":"2024-09-11T23:34:52.870667+0000","last_unstale":"2024-09-11T23:36:47.057475+0000","last_undegraded":"2024-09-11T23:36:47.057475+0000","last_fullsized":"2024-09-11T23:36:47.057475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:08.386703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058304+0000","last_change":"2024-09-11T23:34:52.874070+0000","last_active":"2024-09-11T23:36:47.058304+0000","last_peered":"2024-09-11T23:36:47.058304+0000","last_clean":"2024-09-11T23:36:47.058304+0000","last_became_active":"2024-09-11T23:34:52.873772+0000","last_became_peered":"2024-09-11T23:34:52.873772+0000","last_unstale":"2024-09-11T23:36:47.058304+0000","last_undegraded":"2024-09-11T23:36:47.058304+0000","last_fullsized":"2024-09-11T23:36:47.058304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:49.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146028+0000","last_change":"2024-09-11T23:34:52.875502+0000","last_active":"2024-09-11T23:36:47.146028+0000","last_peered":"2024-09-11T23:36:47.146028+0000","last_clean":"2024-09-11T23:36:47.146028+0000","last_became_active":"2024-09-11T23:34:52.875193+0000","last_became_peered":"2024-09-11T23:34:52.875193+0000","last_unstale":"2024-09-11T23:36:47.146028+0000","last_undegraded":"2024-09-11T23:36:47.146028+0000","last_fullsized":"2024-09-11T23:36:47.146028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:37.213828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153497+0000","last_change":"2024-09-11T23:34:52.869015+0000","last_active":"2024-09-11T23:36:47.153497+0000","last_peered":"2024-09-11T23:36:47.153497+0000","last_clean":"2024-09-11T23:36:47.153497+0000","last_became_active":"2024-09-11T23:34:52.868731+0000","last_became_peered":"2024-09-11T23:34:52.868731+0000","last_unstale":"2024-09-11T23:36:47.153497+0000","last_undegraded":"2024-09-11T23:36:47.153497+0000","last_fullsized":"2024-09-11T23:36:47.153497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:12:15.993203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147366+0000","last_change":"2024-09-11T23:34:52.875555+0000","last_active":"2024-09-11T23:36:47.147366+0000","last_peered":"2024-09-11T23:36:47.147366+0000","last_clean":"2024-09-11T23:36:47.147366+0000","last_became_active":"2024-09-11T23:34:52.875219+0000","last_became_peered":"2024-09-11T23:34:52.875219+0000","last_unstale":"2024-09-11T23:36:47.147366+0000","last_undegraded":"2024-09-11T23:36:47.147366+0000","last_fullsized":"2024-09-11T23:36:47.147366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:16.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058480+0000","last_change":"2024-09-11T23:34:52.875629+0000","last_active":"2024-09-11T23:36:47.058480+0000","last_peered":"2024-09-11T23:36:47.058480+0000","last_clean":"2024-09-11T23:36:47.058480+0000","last_became_active":"2024-09-11T23:34:52.875302+0000","last_became_peered":"2024-09-11T23:34:52.875302+0000","last_unstale":"2024-09-11T23:36:47.058480+0000","last_undegraded":"2024-09-11T23:36:47.058480+0000","last_fullsized":"2024-09-11T23:36:47.058480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:12.341959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327744+0000","last_change":"2024-09-11T23:34:52.868613+0000","last_active":"2024-09-11T23:36:47.327744+0000","last_peered":"2024-09-11T23:36:47.327744+0000","last_clean":"2024-09-11T23:36:47.327744+0000","last_became_active":"2024-09-11T23:34:52.868184+0000","last_became_peered":"2024-09-11T23:34:52.868184+0000","last_unstale":"2024-09-11T23:36:47.327744+0000","last_undegraded":"2024-09-11T23:36:47.327744+0000","last_fullsized":"2024-09-11T23:36:47.327744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:25.990002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057412+0000","last_change":"2024-09-11T23:35:08.873390+0000","last_active":"2024-09-11T23:36:47.057412+0000","last_peered":"2024-09-11T23:36:47.057412+0000","last_clean":"2024-09-11T23:36:47.057412+0000","last_became_active":"2024-09-11T23:35:08.873052+0000","last_became_peered":"2024-09-11T23:35:08.873052+0000","last_unstale":"2024-09-11T23:36:47.057412+0000","last_undegraded":"2024-09-11T23:36:47.057412+0000","last_fullsized":"2024-09-11T23:36:47.057412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:42.937951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327734+0000","last_change":"2024-09-11T23:34:52.873604+0000","last_active":"2024-09-11T23:36:47.327734+0000","last_peered":"2024-09-11T23:36:47.327734+0000","last_clean":"2024-09-11T23:36:47.327734+0000","last_became_active":"2024-09-11T23:34:52.873220+0000","last_became_peered":"2024-09-11T23:34:52.873220+0000","last_unstale":"2024-09-11T23:36:47.327734+0000","last_undegraded":"2024-09-11T23:36:47.327734+0000","last_fullsized":"2024-09-11T23:36:47.327734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:38.480630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058083+0000","last_change":"2024-09-11T23:34:52.872902+0000","last_active":"2024-09-11T23:36:47.058083+0000","last_peered":"2024-09-11T23:36:47.058083+0000","last_clean":"2024-09-11T23:36:47.058083+0000","last_became_active":"2024-09-11T23:34:52.872571+0000","last_became_peered":"2024-09-11T23:34:52.872571+0000","last_unstale":"2024-09-11T23:36:47.058083+0000","last_undegraded":"2024-09-11T23:36:47.058083+0000","last_fullsized":"2024-09-11T23:36:47.058083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:21.844995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058162+0000","last_change":"2024-09-11T23:35:09.457056+0000","last_active":"2024-09-11T23:36:47.058162+0000","last_peered":"2024-09-11T23:36:47.058162+0000","last_clean":"2024-09-11T23:36:47.058162+0000","last_became_active":"2024-09-11T23:35:09.456657+0000","last_became_peered":"2024-09-11T23:35:09.456657+0000","last_unstale":"2024-09-11T23:36:47.058162+0000","last_undegraded":"2024-09-11T23:36:47.058162+0000","last_fullsized":"2024-09-11T23:36:47.058162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:12:56.121280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327534+0000","last_change":"2024-09-11T23:34:52.875312+0000","last_active":"2024-09-11T23:36:47.327534+0000","last_peered":"2024-09-11T23:36:47.327534+0000","last_clean":"2024-09-11T23:36:47.327534+0000","last_became_active":"2024-09-11T23:34:52.874873+0000","last_became_peered":"2024-09-11T23:34:52.874873+0000","last_unstale":"2024-09-11T23:36:47.327534+0000","last_undegraded":"2024-09-11T23:36:47.327534+0000","last_fullsized":"2024-09-11T23:36:47.327534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:47:18.894713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145537+0000","last_change":"2024-09-11T23:34:52.862266+0000","last_active":"2024-09-11T23:36:47.145537+0000","last_peered":"2024-09-11T23:36:47.145537+0000","last_clean":"2024-09-11T23:36:47.145537+0000","last_became_active":"2024-09-11T23:34:52.861923+0000","last_became_peered":"2024-09-11T23:34:52.861923+0000","last_unstale":"2024-09-11T23:36:47.145537+0000","last_undegraded":"2024-09-11T23:36:47.145537+0000","last_fullsized":"2024-09-11T23:36:47.145537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:23.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":148,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.603783+0000","last_change":"2024-09-11T23:34:50.854020+0000","last_active":"2024-09-11T23:36:56.603783+0000","last_peered":"2024-09-11T23:36:56.603783+0000","last_clean":"2024-09-11T23:36:56.603783+0000","last_became_active":"2024-09-11T23:34:50.853683+0000","last_became_peered":"2024-09-11T23:34:50.853683+0000","last_unstale":"2024-09-11T23:36:56.603783+0000","last_undegraded":"2024-09-11T23:36:56.603783+0000","last_fullsized":"2024-09-11T23:36:56.603783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:18.725495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153611+0000","last_change":"2024-09-11T23:34:52.874016+0000","last_active":"2024-09-11T23:36:47.153611+0000","last_peered":"2024-09-11T23:36:47.153611+0000","last_clean":"2024-09-11T23:36:47.153611+0000","last_became_active":"2024-09-11T23:34:52.873675+0000","last_became_peered":"2024-09-11T23:34:52.873675+0000","last_unstale":"2024-09-11T23:36:47.153611+0000","last_undegraded":"2024-09-11T23:36:47.153611+0000","last_fullsized":"2024-09-11T23:36:47.153611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:41:28.955548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156082+0000","last_change":"2024-09-11T23:34:50.855391+0000","last_active":"2024-09-11T23:36:47.156082+0000","last_peered":"2024-09-11T23:36:47.156082+0000","last_clean":"2024-09-11T23:36:47.156082+0000","last_became_active":"2024-09-11T23:34:50.855052+0000","last_became_peered":"2024-09-11T23:34:50.855052+0000","last_unstale":"2024-09-11T23:36:47.156082+0000","last_undegraded":"2024-09-11T23:36:47.156082+0000","last_fullsized":"2024-09-11T23:36:47.156082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.972239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057065+0000","last_change":"2024-09-11T23:34:50.848985+0000","last_active":"2024-09-11T23:36:47.057065+0000","last_peered":"2024-09-11T23:36:47.057065+0000","last_clean":"2024-09-11T23:36:47.057065+0000","last_became_active":"2024-09-11T23:34:50.848614+0000","last_became_peered":"2024-09-11T23:34:50.848614+0000","last_unstale":"2024-09-11T23:36:47.057065+0000","last_undegraded":"2024-09-11T23:36:47.057065+0000","last_fullsized":"2024-09-11T23:36:47.057065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:43:04.723136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146018+0000","last_change":"2024-09-11T23:34:52.866933+0000","last_active":"2024-09-11T23:36:47.146018+0000","last_peered":"2024-09-11T23:36:47.146018+0000","last_clean":"2024-09-11T23:36:47.146018+0000","last_became_active":"2024-09-11T23:34:52.866559+0000","last_became_peered":"2024-09-11T23:34:52.866559+0000","last_unstale":"2024-09-11T23:36:47.146018+0000","last_undegraded":"2024-09-11T23:36:47.146018+0000","last_fullsized":"2024-09-11T23:36:47.146018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:27.745118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153745+0000","last_change":"2024-09-11T23:34:50.849787+0000","last_active":"2024-09-11T23:36:47.153745+0000","last_peered":"2024-09-11T23:36:47.153745+0000","last_clean":"2024-09-11T23:36:47.153745+0000","last_became_active":"2024-09-11T23:34:50.849368+0000","last_became_peered":"2024-09-11T23:34:50.849368+0000","last_unstale":"2024-09-11T23:36:47.153745+0000","last_undegraded":"2024-09-11T23:36:47.153745+0000","last_fullsized":"2024-09-11T23:36:47.153745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:26.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142558+0000","last_change":"2024-09-11T23:34:52.870832+0000","last_active":"2024-09-11T23:36:47.142558+0000","last_peered":"2024-09-11T23:36:47.142558+0000","last_clean":"2024-09-11T23:36:47.142558+0000","last_became_active":"2024-09-11T23:34:52.870474+0000","last_became_peered":"2024-09-11T23:34:52.870474+0000","last_unstale":"2024-09-11T23:36:47.142558+0000","last_undegraded":"2024-09-11T23:36:47.142558+0000","last_fullsized":"2024-09-11T23:36:47.142558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:30.196114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":258,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:57.604585+0000","last_change":"2024-09-11T23:34:50.859074+0000","last_active":"2024-09-11T23:36:57.604585+0000","last_peered":"2024-09-11T23:36:57.604585+0000","last_clean":"2024-09-11T23:36:57.604585+0000","last_became_active":"2024-09-11T23:34:50.858664+0000","last_became_peered":"2024-09-11T23:34:50.858664+0000","last_unstale":"2024-09-11T23:36:57.604585+0000","last_undegraded":"2024-09-11T23:36:57.604585+0000","last_fullsized":"2024-09-11T23:36:57.604585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:04.077813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058577+0000","last_change":"2024-09-11T23:34:52.875901+0000","last_active":"2024-09-11T23:36:47.058577+0000","last_peered":"2024-09-11T23:36:47.058577+0000","last_clean":"2024-09-11T23:36:47.058577+0000","last_became_active":"2024-09-11T23:34:52.875568+0000","last_became_peered":"2024-09-11T23:34:52.875568+0000","last_unstale":"2024-09-11T23:36:47.058577+0000","last_undegraded":"2024-09-11T23:36:47.058577+0000","last_fullsized":"2024-09-11T23:36:47.058577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:54.173548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152813+0000","last_change":"2024-09-11T23:34:50.854957+0000","last_active":"2024-09-11T23:36:47.152813+0000","last_peered":"2024-09-11T23:36:47.152813+0000","last_clean":"2024-09-11T23:36:47.152813+0000","last_became_active":"2024-09-11T23:34:50.854626+0000","last_became_peered":"2024-09-11T23:34:50.854626+0000","last_unstale":"2024-09-11T23:36:47.152813+0000","last_undegraded":"2024-09-11T23:36:47.152813+0000","last_fullsized":"2024-09-11T23:36:47.152813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:53.738966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328149+0000","last_change":"2024-09-11T23:34:52.868850+0000","last_active":"2024-09-11T23:36:47.328149+0000","last_peered":"2024-09-11T23:36:47.328149+0000","last_clean":"2024-09-11T23:36:47.328149+0000","last_became_active":"2024-09-11T23:34:52.868536+0000","last_became_peered":"2024-09-11T23:34:52.868536+0000","last_unstale":"2024-09-11T23:36:47.328149+0000","last_undegraded":"2024-09-11T23:36:47.328149+0000","last_fullsized":"2024-09-11T23:36:47.328149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:11.746821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153829+0000","last_change":"2024-09-11T23:34:50.924819+0000","last_active":"2024-09-11T23:36:47.153829+0000","last_peered":"2024-09-11T23:36:47.153829+0000","last_clean":"2024-09-11T23:36:47.153829+0000","last_became_active":"2024-09-11T23:34:50.924283+0000","last_became_peered":"2024-09-11T23:34:50.924283+0000","last_unstale":"2024-09-11T23:36:47.153829+0000","last_undegraded":"2024-09-11T23:36:47.153829+0000","last_fullsized":"2024-09-11T23:36:47.153829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:41.345233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058417+0000","last_change":"2024-09-11T23:34:52.867113+0000","last_active":"2024-09-11T23:36:47.058417+0000","last_peered":"2024-09-11T23:36:47.058417+0000","last_clean":"2024-09-11T23:36:47.058417+0000","last_became_active":"2024-09-11T23:34:52.866125+0000","last_became_peered":"2024-09-11T23:34:52.866125+0000","last_unstale":"2024-09-11T23:36:47.058417+0000","last_undegraded":"2024-09-11T23:36:47.058417+0000","last_fullsized":"2024-09-11T23:36:47.058417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:18:55.615079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'38","reported_seq":307,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:57.604631+0000","last_change":"2024-09-11T23:34:50.857272+0000","last_active":"2024-09-11T23:36:57.604631+0000","last_peered":"2024-09-11T23:36:57.604631+0000","last_clean":"2024-09-11T23:36:57.604631+0000","last_became_active":"2024-09-11T23:34:50.855701+0000","last_became_peered":"2024-09-11T23:34:50.855701+0000","last_unstale":"2024-09-11T23:36:57.604631+0000","last_undegraded":"2024-09-11T23:36:57.604631+0000","last_fullsized":"2024-09-11T23:36:57.604631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:49.997617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70094,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":245,"num_read_kb":191,"num_write":40,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145783+0000","last_change":"2024-09-11T23:34:52.876928+0000","last_active":"2024-09-11T23:36:47.145783+0000","last_peered":"2024-09-11T23:36:47.145783+0000","last_clean":"2024-09-11T23:36:47.145783+0000","last_became_active":"2024-09-11T23:34:52.876578+0000","last_became_peered":"2024-09-11T23:34:52.876578+0000","last_unstale":"2024-09-11T23:36:47.145783+0000","last_undegraded":"2024-09-11T23:36:47.145783+0000","last_fullsized":"2024-09-11T23:36:47.145783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:07.427667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327515+0000","last_change":"2024-09-11T23:35:08.869275+0000","last_active":"2024-09-11T23:36:47.327515+0000","last_peered":"2024-09-11T23:36:47.327515+0000","last_clean":"2024-09-11T23:36:47.327515+0000","last_became_active":"2024-09-11T23:35:08.850112+0000","last_became_peered":"2024-09-11T23:35:08.850112+0000","last_unstale":"2024-09-11T23:36:47.327515+0000","last_undegraded":"2024-09-11T23:36:47.327515+0000","last_fullsized":"2024-09-11T23:36:47.327515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:36.348202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057105+0000","last_change":"2024-09-11T23:34:52.871970+0000","last_active":"2024-09-11T23:36:47.057105+0000","last_peered":"2024-09-11T23:36:47.057105+0000","last_clean":"2024-09-11T23:36:47.057105+0000","last_became_active":"2024-09-11T23:34:52.871488+0000","last_became_peered":"2024-09-11T23:34:52.871488+0000","last_unstale":"2024-09-11T23:36:47.057105+0000","last_undegraded":"2024-09-11T23:36:47.057105+0000","last_fullsized":"2024-09-11T23:36:47.057105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:08.266431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153627+0000","last_change":"2024-09-11T23:34:50.854023+0000","last_active":"2024-09-11T23:36:47.153627+0000","last_peered":"2024-09-11T23:36:47.153627+0000","last_clean":"2024-09-11T23:36:47.153627+0000","last_became_active":"2024-09-11T23:34:50.853661+0000","last_became_peered":"2024-09-11T23:34:50.853661+0000","last_unstale":"2024-09-11T23:36:47.153627+0000","last_undegraded":"2024-09-11T23:36:47.153627+0000","last_fullsized":"2024-09-11T23:36:47.153627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:25.281976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152646+0000","last_change":"2024-09-11T23:35:08.849648+0000","last_active":"2024-09-11T23:36:47.152646+0000","last_peered":"2024-09-11T23:36:47.152646+0000","last_clean":"2024-09-11T23:36:47.152646+0000","last_became_active":"2024-09-11T23:35:08.849246+0000","last_became_peered":"2024-09-11T23:35:08.849246+0000","last_unstale":"2024-09-11T23:36:47.152646+0000","last_undegraded":"2024-09-11T23:36:47.152646+0000","last_fullsized":"2024-09-11T23:36:47.152646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:17.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'11","reported_seq":157,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.621527+0000","last_change":"2024-09-11T23:34:50.853855+0000","last_active":"2024-09-11T23:36:56.621527+0000","last_peered":"2024-09-11T23:36:56.621527+0000","last_clean":"2024-09-11T23:36:56.621527+0000","last_became_active":"2024-09-11T23:34:50.853493+0000","last_became_peered":"2024-09-11T23:34:50.853493+0000","last_unstale":"2024-09-11T23:36:56.621527+0000","last_undegraded":"2024-09-11T23:36:56.621527+0000","last_fullsized":"2024-09-11T23:36:56.621527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:23.262472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2609,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":8,"num_write":11,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152450+0000","last_change":"2024-09-11T23:34:52.875835+0000","last_active":"2024-09-11T23:36:47.152450+0000","last_peered":"2024-09-11T23:36:47.152450+0000","last_clean":"2024-09-11T23:36:47.152450+0000","last_became_active":"2024-09-11T23:34:52.875525+0000","last_became_peered":"2024-09-11T23:34:52.875525+0000","last_unstale":"2024-09-11T23:36:47.152450+0000","last_undegraded":"2024-09-11T23:36:47.152450+0000","last_fullsized":"2024-09-11T23:36:47.152450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:41.031816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145473+0000","last_change":"2024-09-11T23:34:50.924803+0000","last_active":"2024-09-11T23:36:47.145473+0000","last_peered":"2024-09-11T23:36:47.145473+0000","last_clean":"2024-09-11T23:36:47.145473+0000","last_became_active":"2024-09-11T23:34:50.924217+0000","last_became_peered":"2024-09-11T23:34:50.924217+0000","last_unstale":"2024-09-11T23:36:47.145473+0000","last_undegraded":"2024-09-11T23:36:47.145473+0000","last_fullsized":"2024-09-11T23:36:47.145473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:45.790592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.142408+0000","last_change":"2024-09-11T23:35:08.851856+0000","last_active":"2024-09-11T23:36:47.142408+0000","last_peered":"2024-09-11T23:36:47.142408+0000","last_clean":"2024-09-11T23:36:47.142408+0000","last_became_active":"2024-09-11T23:35:08.851490+0000","last_became_peered":"2024-09-11T23:35:08.851490+0000","last_unstale":"2024-09-11T23:36:47.142408+0000","last_undegraded":"2024-09-11T23:36:47.142408+0000","last_fullsized":"2024-09-11T23:36:47.142408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:46.269022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145857+0000","last_change":"2024-09-11T23:34:50.848385+0000","last_active":"2024-09-11T23:36:47.145857+0000","last_peered":"2024-09-11T23:36:47.145857+0000","last_clean":"2024-09-11T23:36:47.145857+0000","last_became_active":"2024-09-11T23:34:50.847808+0000","last_became_peered":"2024-09-11T23:34:50.847808+0000","last_unstale":"2024-09-11T23:36:47.145857+0000","last_undegraded":"2024-09-11T23:36:47.145857+0000","last_fullsized":"2024-09-11T23:36:47.145857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:38.796622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327734+0000","last_change":"2024-09-11T23:34:52.867341+0000","last_active":"2024-09-11T23:36:47.327734+0000","last_peered":"2024-09-11T23:36:47.327734+0000","last_clean":"2024-09-11T23:36:47.327734+0000","last_became_active":"2024-09-11T23:34:52.866161+0000","last_became_peered":"2024-09-11T23:34:52.866161+0000","last_unstale":"2024-09-11T23:36:47.327734+0000","last_undegraded":"2024-09-11T23:36:47.327734+0000","last_fullsized":"2024-09-11T23:36:47.327734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:27.913705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057317+0000","last_change":"2024-09-11T23:34:50.851466+0000","last_active":"2024-09-11T23:36:47.057317+0000","last_peered":"2024-09-11T23:36:47.057317+0000","last_clean":"2024-09-11T23:36:47.057317+0000","last_became_active":"2024-09-11T23:34:50.851191+0000","last_became_peered":"2024-09-11T23:34:50.851191+0000","last_unstale":"2024-09-11T23:36:47.057317+0000","last_undegraded":"2024-09-11T23:36:47.057317+0000","last_fullsized":"2024-09-11T23:36:47.057317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:00.328658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152878+0000","last_change":"2024-09-11T23:34:52.872290+0000","last_active":"2024-09-11T23:36:47.152878+0000","last_peered":"2024-09-11T23:36:47.152878+0000","last_clean":"2024-09-11T23:36:47.152878+0000","last_became_active":"2024-09-11T23:34:52.871935+0000","last_became_peered":"2024-09-11T23:34:52.871935+0000","last_unstale":"2024-09-11T23:36:47.152878+0000","last_undegraded":"2024-09-11T23:36:47.152878+0000","last_fullsized":"2024-09-11T23:36:47.152878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:08.217548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'59","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:58.069867+0000","last_change":"2024-09-11T23:34:50.857763+0000","last_active":"2024-09-11T23:36:58.069867+0000","last_peered":"2024-09-11T23:36:58.069867+0000","last_clean":"2024-09-11T23:36:58.069867+0000","last_became_active":"2024-09-11T23:34:50.857427+0000","last_became_peered":"2024-09-11T23:34:50.857427+0000","last_unstale":"2024-09-11T23:36:58.069867+0000","last_undegraded":"2024-09-11T23:36:58.069867+0000","last_fullsized":"2024-09-11T23:36:58.069867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:59.703580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":19,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152618+0000","last_change":"2024-09-11T23:34:52.872426+0000","last_active":"2024-09-11T23:36:47.152618+0000","last_peered":"2024-09-11T23:36:47.152618+0000","last_clean":"2024-09-11T23:36:47.152618+0000","last_became_active":"2024-09-11T23:34:52.872090+0000","last_became_peered":"2024-09-11T23:34:52.872090+0000","last_unstale":"2024-09-11T23:36:47.152618+0000","last_undegraded":"2024-09-11T23:36:47.152618+0000","last_fullsized":"2024-09-11T23:36:47.152618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:04:00.963963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'7","reported_seq":148,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.620272+0000","last_change":"2024-09-11T23:34:50.924792+0000","last_active":"2024-09-11T23:36:56.620272+0000","last_peered":"2024-09-11T23:36:56.620272+0000","last_clean":"2024-09-11T23:36:56.620272+0000","last_became_active":"2024-09-11T23:34:50.924444+0000","last_became_peered":"2024-09-11T23:34:50.924444+0000","last_unstale":"2024-09-11T23:36:56.620272+0000","last_undegraded":"2024-09-11T23:36:56.620272+0000","last_fullsized":"2024-09-11T23:36:56.620272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:12.855314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152787+0000","last_change":"2024-09-11T23:34:52.872079+0000","last_active":"2024-09-11T23:36:47.152787+0000","last_peered":"2024-09-11T23:36:47.152787+0000","last_clean":"2024-09-11T23:36:47.152787+0000","last_became_active":"2024-09-11T23:34:52.871755+0000","last_became_peered":"2024-09-11T23:34:52.871755+0000","last_unstale":"2024-09-11T23:36:47.152787+0000","last_undegraded":"2024-09-11T23:36:47.152787+0000","last_fullsized":"2024-09-11T23:36:47.152787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:05.990832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"82'148","reported_seq":391,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:49.148435+0000","last_change":"2024-09-11T23:33:42.256521+0000","last_active":"2024-09-11T23:36:49.148435+0000","last_peered":"2024-09-11T23:36:49.148435+0000","last_clean":"2024-09-11T23:36:49.148435+0000","last_became_active":"2024-09-11T23:33:42.256041+0000","last_became_peered":"2024-09-11T23:33:42.256041+0000","last_unstale":"2024-09-11T23:36:49.148435+0000","last_undegraded":"2024-09-11T23:36:49.148435+0000","last_fullsized":"2024-09-11T23:36:49.148435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:30:22.061515+0000","last_clean_scrub_stamp":"2024-09-11T23:30:22.061515+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:21.391839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292343+0000","last_change":"2024-09-11T23:34:50.854087+0000","last_active":"2024-09-11T23:36:47.292343+0000","last_peered":"2024-09-11T23:36:47.292343+0000","last_clean":"2024-09-11T23:36:47.292343+0000","last_became_active":"2024-09-11T23:34:50.852346+0000","last_became_peered":"2024-09-11T23:34:50.852346+0000","last_unstale":"2024-09-11T23:36:47.292343+0000","last_undegraded":"2024-09-11T23:36:47.292343+0000","last_fullsized":"2024-09-11T23:36:47.292343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.287119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147041+0000","last_change":"2024-09-11T23:34:52.869198+0000","last_active":"2024-09-11T23:36:47.147041+0000","last_peered":"2024-09-11T23:36:47.147041+0000","last_clean":"2024-09-11T23:36:47.147041+0000","last_became_active":"2024-09-11T23:34:52.868621+0000","last_became_peered":"2024-09-11T23:34:52.868621+0000","last_unstale":"2024-09-11T23:36:47.147041+0000","last_undegraded":"2024-09-11T23:36:47.147041+0000","last_fullsized":"2024-09-11T23:36:47.147041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:17.430654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152654+0000","last_change":"2024-09-11T23:34:50.852752+0000","last_active":"2024-09-11T23:36:47.152654+0000","last_peered":"2024-09-11T23:36:47.152654+0000","last_clean":"2024-09-11T23:36:47.152654+0000","last_became_active":"2024-09-11T23:34:50.851895+0000","last_became_peered":"2024-09-11T23:34:50.851895+0000","last_unstale":"2024-09-11T23:36:47.152654+0000","last_undegraded":"2024-09-11T23:36:47.152654+0000","last_fullsized":"2024-09-11T23:36:47.152654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:46.894432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057125+0000","last_change":"2024-09-11T23:34:52.866191+0000","last_active":"2024-09-11T23:36:47.057125+0000","last_peered":"2024-09-11T23:36:47.057125+0000","last_clean":"2024-09-11T23:36:47.057125+0000","last_became_active":"2024-09-11T23:34:52.865858+0000","last_became_peered":"2024-09-11T23:34:52.865858+0000","last_unstale":"2024-09-11T23:36:47.057125+0000","last_undegraded":"2024-09-11T23:36:47.057125+0000","last_fullsized":"2024-09-11T23:36:47.057125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:21.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'7","reported_seq":153,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:37:01.627963+0000","last_change":"2024-09-11T23:34:50.924125+0000","last_active":"2024-09-11T23:37:01.627963+0000","last_peered":"2024-09-11T23:37:01.627963+0000","last_clean":"2024-09-11T23:37:01.627963+0000","last_became_active":"2024-09-11T23:34:50.923780+0000","last_became_peered":"2024-09-11T23:34:50.923780+0000","last_unstale":"2024-09-11T23:37:01.627963+0000","last_undegraded":"2024-09-11T23:37:01.627963+0000","last_fullsized":"2024-09-11T23:37:01.627963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:39.774827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153717+0000","last_change":"2024-09-11T23:34:52.868364+0000","last_active":"2024-09-11T23:36:47.153717+0000","last_peered":"2024-09-11T23:36:47.153717+0000","last_clean":"2024-09-11T23:36:47.153717+0000","last_became_active":"2024-09-11T23:34:52.867913+0000","last_became_peered":"2024-09-11T23:34:52.867913+0000","last_unstale":"2024-09-11T23:36:47.153717+0000","last_undegraded":"2024-09-11T23:36:47.153717+0000","last_fullsized":"2024-09-11T23:36:47.153717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.571825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'8","reported_seq":149,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:56.615174+0000","last_change":"2024-09-11T23:34:50.852421+0000","last_active":"2024-09-11T23:36:56.615174+0000","last_peered":"2024-09-11T23:36:56.615174+0000","last_clean":"2024-09-11T23:36:56.615174+0000","last_became_active":"2024-09-11T23:34:50.851898+0000","last_became_peered":"2024-09-11T23:34:50.851898+0000","last_unstale":"2024-09-11T23:36:56.615174+0000","last_undegraded":"2024-09-11T23:36:56.615174+0000","last_fullsized":"2024-09-11T23:36:56.615174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.661276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1767,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156409+0000","last_change":"2024-09-11T23:34:52.879995+0000","last_active":"2024-09-11T23:36:47.156409+0000","last_peered":"2024-09-11T23:36:47.156409+0000","last_clean":"2024-09-11T23:36:47.156409+0000","last_became_active":"2024-09-11T23:34:52.879678+0000","last_became_peered":"2024-09-11T23:34:52.879678+0000","last_unstale":"2024-09-11T23:36:47.156409+0000","last_undegraded":"2024-09-11T23:36:47.156409+0000","last_fullsized":"2024-09-11T23:36:47.156409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:47.456674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.143130+0000","last_change":"2024-09-11T23:34:50.924957+0000","last_active":"2024-09-11T23:36:47.143130+0000","last_peered":"2024-09-11T23:36:47.143130+0000","last_clean":"2024-09-11T23:36:47.143130+0000","last_became_active":"2024-09-11T23:34:50.924500+0000","last_became_peered":"2024-09-11T23:34:50.924500+0000","last_unstale":"2024-09-11T23:36:47.143130+0000","last_undegraded":"2024-09-11T23:36:47.143130+0000","last_fullsized":"2024-09-11T23:36:47.143130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:40:39.268842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.156159+0000","last_change":"2024-09-11T23:34:52.867337+0000","last_active":"2024-09-11T23:36:47.156159+0000","last_peered":"2024-09-11T23:36:47.156159+0000","last_clean":"2024-09-11T23:36:47.156159+0000","last_became_active":"2024-09-11T23:34:52.866799+0000","last_became_peered":"2024-09-11T23:34:52.866799+0000","last_unstale":"2024-09-11T23:36:47.156159+0000","last_undegraded":"2024-09-11T23:36:47.156159+0000","last_fullsized":"2024-09-11T23:36:47.156159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:38.177135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145946+0000","last_change":"2024-09-11T23:34:50.925156+0000","last_active":"2024-09-11T23:36:47.145946+0000","last_peered":"2024-09-11T23:36:47.145946+0000","last_clean":"2024-09-11T23:36:47.145946+0000","last_became_active":"2024-09-11T23:34:50.924282+0000","last_became_peered":"2024-09-11T23:34:50.924282+0000","last_unstale":"2024-09-11T23:36:47.145946+0000","last_undegraded":"2024-09-11T23:36:47.145946+0000","last_fullsized":"2024-09-11T23:36:47.145946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:47.836902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.057324+0000","last_change":"2024-09-11T23:34:52.864993+0000","last_active":"2024-09-11T23:36:47.057324+0000","last_peered":"2024-09-11T23:36:47.057324+0000","last_clean":"2024-09-11T23:36:47.057324+0000","last_became_active":"2024-09-11T23:34:52.864267+0000","last_became_peered":"2024-09-11T23:34:52.864267+0000","last_unstale":"2024-09-11T23:36:47.057324+0000","last_undegraded":"2024-09-11T23:36:47.057324+0000","last_fullsized":"2024-09-11T23:36:47.057324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:29.912068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'75","reported_seq":150,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:58.074331+0000","last_change":"2024-09-11T23:34:50.849864+0000","last_active":"2024-09-11T23:36:58.074331+0000","last_peered":"2024-09-11T23:36:58.074331+0000","last_clean":"2024-09-11T23:36:58.074331+0000","last_became_active":"2024-09-11T23:34:50.849577+0000","last_became_peered":"2024-09-11T23:34:50.849577+0000","last_unstale":"2024-09-11T23:36:58.074331+0000","last_undegraded":"2024-09-11T23:36:58.074331+0000","last_fullsized":"2024-09-11T23:36:58.074331+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:06.845000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":170,"num_read_kb":149,"num_write":133,"num_write_kb":1159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147524+0000","last_change":"2024-09-11T23:34:52.872028+0000","last_active":"2024-09-11T23:36:47.147524+0000","last_peered":"2024-09-11T23:36:47.147524+0000","last_clean":"2024-09-11T23:36:47.147524+0000","last_became_active":"2024-09-11T23:34:52.871723+0000","last_became_peered":"2024-09-11T23:34:52.871723+0000","last_unstale":"2024-09-11T23:36:47.147524+0000","last_undegraded":"2024-09-11T23:36:47.147524+0000","last_fullsized":"2024-09-11T23:36:47.147524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:44.656218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.292619+0000","last_change":"2024-09-11T23:34:50.860314+0000","last_active":"2024-09-11T23:36:47.292619+0000","last_peered":"2024-09-11T23:36:47.292619+0000","last_clean":"2024-09-11T23:36:47.292619+0000","last_became_active":"2024-09-11T23:34:50.859953+0000","last_became_peered":"2024-09-11T23:34:50.859953+0000","last_unstale":"2024-09-11T23:36:47.292619+0000","last_undegraded":"2024-09-11T23:36:47.292619+0000","last_fullsized":"2024-09-11T23:36:47.292619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:46.705864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328058+0000","last_change":"2024-09-11T23:34:52.867190+0000","last_active":"2024-09-11T23:36:47.328058+0000","last_peered":"2024-09-11T23:36:47.328058+0000","last_clean":"2024-09-11T23:36:47.328058+0000","last_became_active":"2024-09-11T23:34:52.866829+0000","last_became_peered":"2024-09-11T23:34:52.866829+0000","last_unstale":"2024-09-11T23:36:47.328058+0000","last_undegraded":"2024-09-11T23:36:47.328058+0000","last_fullsized":"2024-09-11T23:36:47.328058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.752006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058330+0000","last_change":"2024-09-11T23:34:50.924566+0000","last_active":"2024-09-11T23:36:47.058330+0000","last_peered":"2024-09-11T23:36:47.058330+0000","last_clean":"2024-09-11T23:36:47.058330+0000","last_became_active":"2024-09-11T23:34:50.924284+0000","last_became_peered":"2024-09-11T23:34:50.924284+0000","last_unstale":"2024-09-11T23:36:47.058330+0000","last_undegraded":"2024-09-11T23:36:47.058330+0000","last_fullsized":"2024-09-11T23:36:47.058330+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:54.246154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145487+0000","last_change":"2024-09-11T23:34:52.869302+0000","last_active":"2024-09-11T23:36:47.145487+0000","last_peered":"2024-09-11T23:36:47.145487+0000","last_clean":"2024-09-11T23:36:47.145487+0000","last_became_active":"2024-09-11T23:34:52.868960+0000","last_became_peered":"2024-09-11T23:34:52.868960+0000","last_unstale":"2024-09-11T23:36:47.145487+0000","last_undegraded":"2024-09-11T23:36:47.145487+0000","last_fullsized":"2024-09-11T23:36:47.145487+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.561712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146316+0000","last_change":"2024-09-11T23:34:50.854573+0000","last_active":"2024-09-11T23:36:47.146316+0000","last_peered":"2024-09-11T23:36:47.146316+0000","last_clean":"2024-09-11T23:36:47.146316+0000","last_became_active":"2024-09-11T23:34:50.854165+0000","last_became_peered":"2024-09-11T23:34:50.854165+0000","last_unstale":"2024-09-11T23:36:47.146316+0000","last_undegraded":"2024-09-11T23:36:47.146316+0000","last_fullsized":"2024-09-11T23:36:47.146316+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:57:31.589495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327542+0000","last_change":"2024-09-11T23:34:52.875987+0000","last_active":"2024-09-11T23:36:47.327542+0000","last_peered":"2024-09-11T23:36:47.327542+0000","last_clean":"2024-09-11T23:36:47.327542+0000","last_became_active":"2024-09-11T23:34:52.875662+0000","last_became_peered":"2024-09-11T23:34:52.875662+0000","last_unstale":"2024-09-11T23:36:47.327542+0000","last_undegraded":"2024-09-11T23:36:47.327542+0000","last_fullsized":"2024-09-11T23:36:47.327542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:16.514938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058234+0000","last_change":"2024-09-11T23:34:50.850726+0000","last_active":"2024-09-11T23:36:47.058234+0000","last_peered":"2024-09-11T23:36:47.058234+0000","last_clean":"2024-09-11T23:36:47.058234+0000","last_became_active":"2024-09-11T23:34:50.850381+0000","last_became_peered":"2024-09-11T23:34:50.850381+0000","last_unstale":"2024-09-11T23:36:47.058234+0000","last_undegraded":"2024-09-11T23:36:47.058234+0000","last_fullsized":"2024-09-11T23:36:47.058234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:50.431293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.146377+0000","last_change":"2024-09-11T23:34:52.870652+0000","last_active":"2024-09-11T23:36:47.146377+0000","last_peered":"2024-09-11T23:36:47.146377+0000","last_clean":"2024-09-11T23:36:47.146377+0000","last_became_active":"2024-09-11T23:34:52.870303+0000","last_became_peered":"2024-09-11T23:34:52.870303+0000","last_unstale":"2024-09-11T23:36:47.146377+0000","last_undegraded":"2024-09-11T23:36:47.146377+0000","last_fullsized":"2024-09-11T23:36:47.146377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:17.670981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.327509+0000","last_change":"2024-09-11T23:35:08.854490+0000","last_active":"2024-09-11T23:36:47.327509+0000","last_peered":"2024-09-11T23:36:47.327509+0000","last_clean":"2024-09-11T23:36:47.327509+0000","last_became_active":"2024-09-11T23:35:08.849820+0000","last_became_peered":"2024-09-11T23:35:08.849820+0000","last_unstale":"2024-09-11T23:36:47.327509+0000","last_undegraded":"2024-09-11T23:36:47.327509+0000","last_fullsized":"2024-09-11T23:36:47.327509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:09.509867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.152580+0000","last_change":"2024-09-11T23:34:52.869695+0000","last_active":"2024-09-11T23:36:47.152580+0000","last_peered":"2024-09-11T23:36:47.152580+0000","last_clean":"2024-09-11T23:36:47.152580+0000","last_became_active":"2024-09-11T23:34:52.869367+0000","last_became_peered":"2024-09-11T23:34:52.869367+0000","last_unstale":"2024-09-11T23:36:47.152580+0000","last_undegraded":"2024-09-11T23:36:47.152580+0000","last_fullsized":"2024-09-11T23:36:47.152580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:16.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.153822+0000","last_change":"2024-09-11T23:34:52.872778+0000","last_active":"2024-09-11T23:36:47.153822+0000","last_peered":"2024-09-11T23:36:47.153822+0000","last_clean":"2024-09-11T23:36:47.153822+0000","last_became_active":"2024-09-11T23:34:52.872451+0000","last_became_peered":"2024-09-11T23:34:52.872451+0000","last_unstale":"2024-09-11T23:36:47.153822+0000","last_undegraded":"2024-09-11T23:36:47.153822+0000","last_fullsized":"2024-09-11T23:36:47.153822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:51.769927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145825+0000","last_change":"2024-09-11T23:34:50.849675+0000","last_active":"2024-09-11T23:36:47.145825+0000","last_peered":"2024-09-11T23:36:47.145825+0000","last_clean":"2024-09-11T23:36:47.145825+0000","last_became_active":"2024-09-11T23:34:50.849228+0000","last_became_peered":"2024-09-11T23:34:50.849228+0000","last_unstale":"2024-09-11T23:36:47.145825+0000","last_undegraded":"2024-09-11T23:36:47.145825+0000","last_fullsized":"2024-09-11T23:36:47.145825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:50.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.058399+0000","last_change":"2024-09-11T23:34:52.867739+0000","last_active":"2024-09-11T23:36:47.058399+0000","last_peered":"2024-09-11T23:36:47.058399+0000","last_clean":"2024-09-11T23:36:47.058399+0000","last_became_active":"2024-09-11T23:34:52.866613+0000","last_became_peered":"2024-09-11T23:34:52.866613+0000","last_unstale":"2024-09-11T23:36:47.058399+0000","last_undegraded":"2024-09-11T23:36:47.058399+0000","last_fullsized":"2024-09-11T23:36:47.058399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:49.348386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'13","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.145845+0000","last_change":"2024-09-11T23:34:50.851401+0000","last_active":"2024-09-11T23:36:47.145845+0000","last_peered":"2024-09-11T23:36:47.145845+0000","last_clean":"2024-09-11T23:36:47.145845+0000","last_became_active":"2024-09-11T23:34:50.851068+0000","last_became_peered":"2024-09-11T23:34:50.851068+0000","last_unstale":"2024-09-11T23:36:47.145845+0000","last_undegraded":"2024-09-11T23:36:47.145845+0000","last_fullsized":"2024-09-11T23:36:47.145845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:09.673864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.328160+0000","last_change":"2024-09-11T23:34:52.865216+0000","last_active":"2024-09-11T23:36:47.328160+0000","last_peered":"2024-09-11T23:36:47.328160+0000","last_clean":"2024-09-11T23:36:47.328160+0000","last_became_active":"2024-09-11T23:34:52.864917+0000","last_became_peered":"2024-09-11T23:34:52.864917+0000","last_unstale":"2024-09-11T23:36:47.328160+0000","last_undegraded":"2024-09-11T23:36:47.328160+0000","last_fullsized":"2024-09-11T23:36:47.328160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:36.469793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147661+0000","last_change":"2024-09-11T23:34:50.924291+0000","last_active":"2024-09-11T23:36:47.147661+0000","last_peered":"2024-09-11T23:36:47.147661+0000","last_clean":"2024-09-11T23:36:47.147661+0000","last_became_active":"2024-09-11T23:34:50.923892+0000","last_became_peered":"2024-09-11T23:34:50.923892+0000","last_unstale":"2024-09-11T23:36:47.147661+0000","last_undegraded":"2024-09-11T23:36:47.147661+0000","last_fullsized":"2024-09-11T23:36:47.147661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:52.266778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:47.147431+0000","last_change":"2024-09-11T23:34:52.875995+0000","last_active":"2024-09-11T23:36:47.147431+0000","last_peered":"2024-09-11T23:36:47.147431+0000","last_clean":"2024-09-11T23:36:47.147431+0000","last_became_active":"2024-09-11T23:34:52.875654+0000","last_became_peered":"2024-09-11T23:34:52.875654+0000","last_unstale":"2024-09-11T23:36:47.147431+0000","last_undegraded":"2024-09-11T23:36:47.147431+0000","last_fullsized":"2024-09-11T23:36:47.147431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:51.840502+0000","last_clean_scrub_stamp":"2024-09-11T23:34:51.840502+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:30:19.218291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":154,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:36:57.530808+0000","last_change":"2024-09-11T23:34:50.853881+0000","last_active":"2024-09-11T23:36:57.530808+0000","last_peered":"2024-09-11T23:36:57.530808+0000","last_clean":"2024-09-11T23:36:57.530808+0000","last_became_active":"2024-09-11T23:34:50.853576+0000","last_became_peered":"2024-09-11T23:34:50.853576+0000","last_unstale":"2024-09-11T23:36:57.530808+0000","last_undegraded":"2024-09-11T23:36:57.530808+0000","last_fullsized":"2024-09-11T23:36:57.530808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:34:49.828837+0000","last_clean_scrub_stamp":"2024-09-11T23:34:49.828837+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:38.550587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":802091,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1390,"num_read_kb":858,"num_write":448,"num_write_kb":2129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2768896,"data_stored":2407713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612646,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1753088,"data_stored":1237974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005104,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32484,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712668,"statfs":{"total":95995035648,"available":95961772032,"internally_reserved":0,"allocated":1327104,"data_stored":836376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234037,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28316,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716836,"statfs":{"total":95995035648,"available":95966040064,"internally_reserved":0,"allocated":1323008,"data_stored":836754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495674,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1527808,"data_stored":1039258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":444056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888131,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1167336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018953,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":444759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1164150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444115,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":514963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607640,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":516503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":444750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":402083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:37:04.521 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:37:04.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:04 smithi178 ceph-mon[35882]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-09-11T23:37:04.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:04 smithi178 ceph-mon[35882]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:37:04.769 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:05.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:05 smithi133 ceph-mon[31430]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 99 KiB/s wr, 70 op/s 2024-09-11T23:37:05.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.213+0000 7f990d92f640 1 -- 172.21.15.133:0/2240332236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9908102570 msgr2=0x7f99081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.213+0000 7f990d92f640 1 --2- 172.21.15.133:0/2240332236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9908102570 0x7f99081029b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009900 tx=0x7f98f002a480 comp rx=0 tx=0).stop 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.213+0000 7f990d92f640 1 -- 172.21.15.133:0/2240332236 shutdown_connections 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.213+0000 7f990d92f640 1 --2- 172.21.15.133:0/2240332236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9908102ef0 0x7f990810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.213+0000 7f990d92f640 1 --2- 172.21.15.133:0/2240332236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9908102570 0x7f99081029b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.213+0000 7f990d92f640 1 --2- 172.21.15.133:0/2240332236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9908100e20 0x7f9908101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.213+0000 7f990d92f640 1 -- 172.21.15.133:0/2240332236 >> 172.21.15.133:0/2240332236 conn(0x7f99080fc7d0 msgr2=0x7f99080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.213+0000 7f990d92f640 1 -- 172.21.15.133:0/2240332236 shutdown_connections 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.213+0000 7f990d92f640 1 -- 172.21.15.133:0/2240332236 wait complete. 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.214+0000 7f990d92f640 1 Processor -- start 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.214+0000 7f990d92f640 1 -- start start 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.214+0000 7f990d92f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9908100e20 0x7f99080fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.214+0000 7f990d92f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9908102570 0x7f99080fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.214+0000 7f990d92f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9908102ef0 0x7f99080fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.214+0000 7f990d92f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9908110fa0 con 0x7f9908100e20 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.214+0000 7f990d92f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9908110e20 con 0x7f9908102570 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.214+0000 7f990d92f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9908111120 con 0x7f9908102ef0 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.214+0000 7f9906ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9908100e20 0x7f99080fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f9906ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9908100e20 0x7f99080fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60366/0 (socket says 172.21.15.133:60366) 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f9906ffd640 1 -- 172.21.15.133:0/3640072132 learned_addr learned my addr 172.21.15.133:0/3640072132 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f98fe7fc640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9908102570 0x7f99080fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f9906ffd640 1 -- 172.21.15.133:0/3640072132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9908102ef0 msgr2=0x7f99080fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f9906ffd640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9908102ef0 0x7f99080fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f9906ffd640 1 -- 172.21.15.133:0/3640072132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9908102570 msgr2=0x7f99080fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f9906ffd640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9908102570 0x7f99080fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f9906ffd640 1 -- 172.21.15.133:0/3640072132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99081a6a80 con 0x7f9908100e20 2024-09-11T23:37:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f98fe7fc640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9908102570 0x7f99080fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f9906ffd640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9908100e20 0x7f99080fcc80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f98f800b4a0 tx=0x7f98f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f990c92d640 1 -- 172.21.15.133:0/3640072132 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8014070 con 0x7f9908100e20 2024-09-11T23:37:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f990d92f640 1 -- 172.21.15.133:0/3640072132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99081a6d70 con 0x7f9908100e20 2024-09-11T23:37:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f990c92d640 1 -- 172.21.15.133:0/3640072132 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f8010af0 con 0x7f9908100e20 2024-09-11T23:37:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f990c92d640 1 -- 172.21.15.133:0/3640072132 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8017d10 con 0x7f9908100e20 2024-09-11T23:37:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.215+0000 7f990d92f640 1 -- 172.21.15.133:0/3640072132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99081a7250 con 0x7f9908100e20 2024-09-11T23:37:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.216+0000 7f990d92f640 1 -- 172.21.15.133:0/3640072132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9908101200 con 0x7f9908100e20 2024-09-11T23:37:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.217+0000 7f990c92d640 1 -- 172.21.15.133:0/3640072132 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f98f8017070 con 0x7f9908100e20 2024-09-11T23:37:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.218+0000 7f990c92d640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f98f40a2ea0 0x7f98f40a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.218+0000 7f98fe7fc640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f98f40a2ea0 0x7f98f40a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:05.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.218+0000 7f990c92d640 1 -- 172.21.15.133:0/3640072132 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f98f80ceea0 con 0x7f9908100e20 2024-09-11T23:37:05.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.219+0000 7f98fe7fc640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f98f40a2ea0 0x7f98f40a5360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000c00 tx=0x7f98f00041e0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:05.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.221+0000 7f990c92d640 1 -- 172.21.15.133:0/3640072132 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98f809b920 con 0x7f9908100e20 2024-09-11T23:37:05.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.422+0000 7f990d92f640 1 -- 172.21.15.133:0/3640072132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99080fe1b0 con 0x7f9908100e20 2024-09-11T23:37:05.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.423+0000 7f990c92d640 1 -- 172.21.15.133:0/3640072132 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f98f809b220 con 0x7f9908100e20 2024-09-11T23:37:05.422 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:37:05.423 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:37:05.423 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:37:05.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 -- 172.21.15.133:0/3640072132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f98f40a2ea0 msgr2=0x7f98f40a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f98f40a2ea0 0x7f98f40a5360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000c00 tx=0x7f98f00041e0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 -- 172.21.15.133:0/3640072132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9908100e20 msgr2=0x7f99080fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:05.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9908100e20 0x7f99080fcc80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f98f800b4a0 tx=0x7f98f800b960 comp rx=0 tx=0).stop 2024-09-11T23:37:05.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 -- 172.21.15.133:0/3640072132 shutdown_connections 2024-09-11T23:37:05.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f98f40a2ea0 0x7f98f40a5360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9908102ef0 0x7f99080fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9908102570 0x7f99080fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 --2- 172.21.15.133:0/3640072132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9908100e20 0x7f99080fcc80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:05.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 -- 172.21.15.133:0/3640072132 >> 172.21.15.133:0/3640072132 conn(0x7f99080fc7d0 msgr2=0x7f990806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:05.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 -- 172.21.15.133:0/3640072132 shutdown_connections 2024-09-11T23:37:05.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:05.425+0000 7f990d92f640 1 -- 172.21.15.133:0/3640072132 wait complete. 2024-09-11T23:37:05.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:05 smithi138 ceph-mon[35142]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 99 KiB/s wr, 70 op/s 2024-09-11T23:37:05.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:05 smithi178 ceph-mon[35882]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 99 KiB/s wr, 70 op/s 2024-09-11T23:37:06.100 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:37:06.345 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:06.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:06 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3640072132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:06.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:06 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3640072132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:06.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:06 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3640072132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:06.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.819+0000 7f91bf0f7640 1 -- 172.21.15.133:0/1414926166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b810c170 msgr2=0x7f91b810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.819+0000 7f91bf0f7640 1 --2- 172.21.15.133:0/1414926166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b810c170 0x7f91b810e610 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009ca0 tx=0x7f91b002a950 comp rx=0 tx=0).stop 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.819+0000 7f91bf0f7640 1 -- 172.21.15.133:0/1414926166 shutdown_connections 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.819+0000 7f91bf0f7640 1 --2- 172.21.15.133:0/1414926166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b810c170 0x7f91b810e610 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.819+0000 7f91bf0f7640 1 --2- 172.21.15.133:0/1414926166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b8103660 0x7f91b810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.819+0000 7f91bf0f7640 1 --2- 172.21.15.133:0/1414926166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b8102d40 0x7f91b8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.819+0000 7f91bf0f7640 1 -- 172.21.15.133:0/1414926166 >> 172.21.15.133:0/1414926166 conn(0x7f91b80fc7a0 msgr2=0x7f91b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.819+0000 7f91bf0f7640 1 -- 172.21.15.133:0/1414926166 shutdown_connections 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.819+0000 7f91bf0f7640 1 -- 172.21.15.133:0/1414926166 wait complete. 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91bf0f7640 1 Processor -- start 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91bf0f7640 1 -- start start 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91bf0f7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b8102d40 0x7f91b81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:06.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91bf0f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b8103660 0x7f91b8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:06.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91bf0f7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b810c170 0x7f91b819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:06.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91bf0f7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8111f50 con 0x7f91b8103660 2024-09-11T23:37:06.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91bf0f7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8111dd0 con 0x7f91b810c170 2024-09-11T23:37:06.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91bf0f7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91b81120d0 con 0x7f91b8102d40 2024-09-11T23:37:06.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91b7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b8103660 0x7f91b8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:06.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91b7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b8103660 0x7f91b8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60372/0 (socket says 172.21.15.133:60372) 2024-09-11T23:37:06.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91b7fff640 1 -- 172.21.15.133:0/4266104788 learned_addr learned my addr 172.21.15.133:0/4266104788 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:06.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91bd694640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b810c170 0x7f91b819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:06.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91bce93640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b8102d40 0x7f91b81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:06.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91b7fff640 1 -- 172.21.15.133:0/4266104788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b8102d40 msgr2=0x7f91b81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91b7fff640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b8102d40 0x7f91b81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91b7fff640 1 -- 172.21.15.133:0/4266104788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b810c170 msgr2=0x7f91b819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91b7fff640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b810c170 0x7f91b819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91b7fff640 1 -- 172.21.15.133:0/4266104788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b819d380 con 0x7f91b8103660 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.820+0000 7f91bce93640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b8102d40 0x7f91b81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.821+0000 7f91b7fff640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b8103660 0x7f91b8198910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f91a400d750 tx=0x7f91a400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.821+0000 7f91b5ffb640 1 -- 172.21.15.133:0/4266104788 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4017070 con 0x7f91b8103660 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.821+0000 7f91bf0f7640 1 -- 172.21.15.133:0/4266104788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b819d670 con 0x7f91b8103660 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.821+0000 7f91b5ffb640 1 -- 172.21.15.133:0/4266104788 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a4005020 con 0x7f91b8103660 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.821+0000 7f91b5ffb640 1 -- 172.21.15.133:0/4266104788 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4004d00 con 0x7f91b8103660 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.821+0000 7f91bf0f7640 1 -- 172.21.15.133:0/4266104788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b81a4f50 con 0x7f91b8103660 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.822+0000 7f91bf0f7640 1 -- 172.21.15.133:0/4266104788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b8103120 con 0x7f91b8103660 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.823+0000 7f91b5ffb640 1 -- 172.21.15.133:0/4266104788 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f91a4009a30 con 0x7f91b8103660 2024-09-11T23:37:06.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.824+0000 7f91b5ffb640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f91a80a2ea0 0x7f91a80a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:06.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.824+0000 7f91bce93640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f91a80a2ea0 0x7f91a80a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:06.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.824+0000 7f91b5ffb640 1 -- 172.21.15.133:0/4266104788 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f91a40cf080 con 0x7f91b8103660 2024-09-11T23:37:06.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.824+0000 7f91bce93640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f91a80a2ea0 0x7f91a80a5360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0058c0 tx=0x7f91ac005850 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:06.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:06.828+0000 7f91b5ffb640 1 -- 172.21.15.133:0/4266104788 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91a409bad0 con 0x7f91b8103660 2024-09-11T23:37:07.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.031+0000 7f91bf0f7640 1 -- 172.21.15.133:0/4266104788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91b81a5120 con 0x7f91b8103660 2024-09-11T23:37:07.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.031+0000 7f91b5ffb640 1 -- 172.21.15.133:0/4266104788 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f91a409b3d0 con 0x7f91b8103660 2024-09-11T23:37:07.031 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:37:07.032 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:37:07.032 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:37:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 -- 172.21.15.133:0/4266104788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f91a80a2ea0 msgr2=0x7f91a80a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f91a80a2ea0 0x7f91a80a5360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0058c0 tx=0x7f91ac005850 comp rx=0 tx=0).stop 2024-09-11T23:37:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 -- 172.21.15.133:0/4266104788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b8103660 msgr2=0x7f91b8198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b8103660 0x7f91b8198910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f91a400d750 tx=0x7f91a400dc10 comp rx=0 tx=0).stop 2024-09-11T23:37:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 -- 172.21.15.133:0/4266104788 shutdown_connections 2024-09-11T23:37:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f91a80a2ea0 0x7f91a80a5360 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b810c170 0x7f91b819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b8103660 0x7f91b8198910 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 --2- 172.21.15.133:0/4266104788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b8102d40 0x7f91b81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 -- 172.21.15.133:0/4266104788 >> 172.21.15.133:0/4266104788 conn(0x7f91b80fc7a0 msgr2=0x7f91b80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 -- 172.21.15.133:0/4266104788 shutdown_connections 2024-09-11T23:37:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:07.034+0000 7f91bf0f7640 1 -- 172.21.15.133:0/4266104788 wait complete. 2024-09-11T23:37:07.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:07 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/4266104788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:07.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:07 smithi133 ceph-mon[31430]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 98 KiB/s wr, 55 op/s 2024-09-11T23:37:07.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:07 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/4266104788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:07.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:07 smithi138 ceph-mon[35142]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 98 KiB/s wr, 55 op/s 2024-09-11T23:37:07.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:07 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/4266104788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:07.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:07 smithi178 ceph-mon[35882]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 98 KiB/s wr, 55 op/s 2024-09-11T23:37:07.726 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:37:07.974 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.470+0000 7f2db57cc640 1 -- 172.21.15.133:0/1720937446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db0105560 msgr2=0x7f2db010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.470+0000 7f2db57cc640 1 --2- 172.21.15.133:0/1720937446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db0105560 0x7f2db010d0e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009ca0 tx=0x7f2da802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:37:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.470+0000 7f2db57cc640 1 -- 172.21.15.133:0/1720937446 shutdown_connections 2024-09-11T23:37:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.470+0000 7f2db57cc640 1 --2- 172.21.15.133:0/1720937446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db0105560 0x7f2db010d0e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.470+0000 7f2db57cc640 1 --2- 172.21.15.133:0/1720937446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db0100de0 0x7f2db0104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.470+0000 7f2db57cc640 1 --2- 172.21.15.133:0/1720937446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0100430 0x7f2db0100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:08.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.470+0000 7f2db57cc640 1 -- 172.21.15.133:0/1720937446 >> 172.21.15.133:0/1720937446 conn(0x7f2db0077230 msgr2=0x7f2db00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.470+0000 7f2db57cc640 1 -- 172.21.15.133:0/1720937446 shutdown_connections 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.471+0000 7f2db57cc640 1 -- 172.21.15.133:0/1720937446 wait complete. 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.471+0000 7f2db57cc640 1 Processor -- start 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.471+0000 7f2db57cc640 1 -- start start 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.471+0000 7f2db57cc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0100430 0x7f2db019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.471+0000 7f2db57cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db0100de0 0x7f2db019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.471+0000 7f2db57cc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db0105560 0x7f2db019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.471+0000 7f2db57cc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2db0111f40 con 0x7f2db0100de0 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.471+0000 7f2db57cc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2db0111dc0 con 0x7f2db0100430 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.471+0000 7f2db57cc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2db01120c0 con 0x7f2db0105560 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2dae7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db0100de0 0x7f2db019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2dae7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db0100de0 0x7f2db019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60394/0 (socket says 172.21.15.133:60394) 2024-09-11T23:37:08.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2dae7fc640 1 -- 172.21.15.133:0/2352593315 learned_addr learned my addr 172.21.15.133:0/2352593315 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2daeffd640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0100430 0x7f2db019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2daf7fe640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db0105560 0x7f2db019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2dae7fc640 1 -- 172.21.15.133:0/2352593315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db0105560 msgr2=0x7f2db019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2dae7fc640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db0105560 0x7f2db019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2dae7fc640 1 -- 172.21.15.133:0/2352593315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0100430 msgr2=0x7f2db019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2dae7fc640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0100430 0x7f2db019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2dae7fc640 1 -- 172.21.15.133:0/2352593315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db019f580 con 0x7f2db0100de0 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2dae7fc640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db0100de0 0x7f2db019ab10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2da400d950 tx=0x7f2da400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2d97fff640 1 -- 172.21.15.133:0/2352593315 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da4017070 con 0x7f2db0100de0 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2d97fff640 1 -- 172.21.15.133:0/2352593315 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da4009c30 con 0x7f2db0100de0 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2d97fff640 1 -- 172.21.15.133:0/2352593315 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da400f6b0 con 0x7f2db0100de0 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2db57cc640 1 -- 172.21.15.133:0/2352593315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db019f870 con 0x7f2db0100de0 2024-09-11T23:37:08.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.472+0000 7f2db57cc640 1 -- 172.21.15.133:0/2352593315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db01a7150 con 0x7f2db0100de0 2024-09-11T23:37:08.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.473+0000 7f2db57cc640 1 -- 172.21.15.133:0/2352593315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db0100810 con 0x7f2db0100de0 2024-09-11T23:37:08.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.475+0000 7f2d97fff640 1 -- 172.21.15.133:0/2352593315 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f2da40109d0 con 0x7f2db0100de0 2024-09-11T23:37:08.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.476+0000 7f2d97fff640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d980a2ea0 0x7f2d980a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:08.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.476+0000 7f2daeffd640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d980a2ea0 0x7f2d980a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:08.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.476+0000 7f2d97fff640 1 -- 172.21.15.133:0/2352593315 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f2da40cf190 con 0x7f2db0100de0 2024-09-11T23:37:08.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.476+0000 7f2daeffd640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d980a2ea0 0x7f2d980a5360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2da0005d00 tx=0x7f2da000b410 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:08.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.479+0000 7f2d97fff640 1 -- 172.21.15.133:0/2352593315 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2da4009de0 con 0x7f2db0100de0 2024-09-11T23:37:08.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.647+0000 7f2db57cc640 1 -- 172.21.15.133:0/2352593315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2db01a7480 con 0x7f2db0100de0 2024-09-11T23:37:08.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.649+0000 7f2d97fff640 1 -- 172.21.15.133:0/2352593315 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+23060 (secure 0 0 0) 0x7f2da409b5d0 con 0x7f2db0100de0 2024-09-11T23:37:08.649 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:37:08.649 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":82,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:36:47.051856+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/3324898762":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3596945316":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6801/3324898762":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/1658494809":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3947064111":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/802317131":"2024-09-12T23:36:47.051817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:37:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.651+0000 7f2db57cc640 1 -- 172.21.15.133:0/2352593315 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d980a2ea0 msgr2=0x7f2d980a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.651+0000 7f2db57cc640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d980a2ea0 0x7f2d980a5360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2da0005d00 tx=0x7f2da000b410 comp rx=0 tx=0).stop 2024-09-11T23:37:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.651+0000 7f2db57cc640 1 -- 172.21.15.133:0/2352593315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db0100de0 msgr2=0x7f2db019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.651+0000 7f2db57cc640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db0100de0 0x7f2db019ab10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2da400d950 tx=0x7f2da400de10 comp rx=0 tx=0).stop 2024-09-11T23:37:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.652+0000 7f2db57cc640 1 -- 172.21.15.133:0/2352593315 shutdown_connections 2024-09-11T23:37:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.652+0000 7f2db57cc640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d980a2ea0 0x7f2d980a5360 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.652+0000 7f2db57cc640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db0105560 0x7f2db019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.652+0000 7f2db57cc640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db0100de0 0x7f2db019ab10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.652+0000 7f2db57cc640 1 --2- 172.21.15.133:0/2352593315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2db0100430 0x7f2db019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.652+0000 7f2db57cc640 1 -- 172.21.15.133:0/2352593315 >> 172.21.15.133:0/2352593315 conn(0x7f2db0077230 msgr2=0x7f2db0103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.652+0000 7f2db57cc640 1 -- 172.21.15.133:0/2352593315 shutdown_connections 2024-09-11T23:37:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:08.652+0000 7f2db57cc640 1 -- 172.21.15.133:0/2352593315 wait complete. 2024-09-11T23:37:09.339 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-11T23:37:09.586 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:09.707 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:09 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2352593315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:37:09.707 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:09 smithi133 ceph-mon[31430]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-09-11T23:37:09.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:09 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2352593315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:37:09.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:09 smithi178 ceph-mon[35882]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-09-11T23:37:09.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:09 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2352593315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:37:09.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:09 smithi138 ceph-mon[35142]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-09-11T23:37:10.697 INFO:teuthology.orchestra.run.smithi133.stdout:{"run_dir":"/var/run/ceph"} 2024-09-11T23:37:10.697 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-11T23:37:10.697 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-11T23:37:10.697 DEBUG:tasks.quiescer:Yielding 2024-09-11T23:37:10.698 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T23:37:10.713 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T23:37:10.713 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T23:37:10.714 INFO:tasks.workunit:timeout=3h 2024-09-11T23:37:10.714 INFO:tasks.workunit:cleanup=True 2024-09-11T23:37:10.714 DEBUG:teuthology.orchestra.run.smithi133:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T23:37:10.714 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:37:10.730 INFO:teuthology.orchestra.run.smithi133.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-11T23:37:10.731 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-11T23:37:10.731 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-11T23:37:10.731 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:37:10.731 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-11T23:37:10.731 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-11 23:35:11.959991550 +0000 2024-09-11T23:37:10.731 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-11 23:35:11.959991550 +0000 2024-09-11T23:37:10.731 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-11 23:36:45.306040703 +0000 2024-09-11T23:37:10.732 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-11 23:35:11.959991550 +0000 2024-09-11T23:37:10.732 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-11T23:37:10.732 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-11T23:37:10.735 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:10 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:37:10.768 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T23:37:10.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:10 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:37:10.826 INFO:tasks.workunit.client.0.smithi133.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T23:37:10.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:10 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:37:10.969 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:11.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.444+0000 7efe4424c640 1 -- 172.21.15.133:0/2415246217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe3c102ef0 msgr2=0x7efe3c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:11.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.444+0000 7efe4424c640 1 --2- 172.21.15.133:0/2415246217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe3c102ef0 0x7efe3c10da40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efe34009ca0 tx=0x7efe3402a950 comp rx=0 tx=0).stop 2024-09-11T23:37:11.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.444+0000 7efe4424c640 1 -- 172.21.15.133:0/2415246217 shutdown_connections 2024-09-11T23:37:11.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.444+0000 7efe4424c640 1 --2- 172.21.15.133:0/2415246217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe3c102ef0 0x7efe3c10da40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:11.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.444+0000 7efe4424c640 1 --2- 172.21.15.133:0/2415246217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c102570 0x7efe3c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:11.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.444+0000 7efe4424c640 1 --2- 172.21.15.133:0/2415246217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c100e20 0x7efe3c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:11.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.444+0000 7efe4424c640 1 -- 172.21.15.133:0/2415246217 >> 172.21.15.133:0/2415246217 conn(0x7efe3c0fc7d0 msgr2=0x7efe3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:11.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.445+0000 7efe4424c640 1 -- 172.21.15.133:0/2415246217 shutdown_connections 2024-09-11T23:37:11.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.445+0000 7efe4424c640 1 -- 172.21.15.133:0/2415246217 wait complete. 2024-09-11T23:37:11.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.445+0000 7efe4424c640 1 Processor -- start 2024-09-11T23:37:11.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.445+0000 7efe4424c640 1 -- start start 2024-09-11T23:37:11.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe4424c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c100e20 0x7efe3c1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:11.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe4424c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c102570 0x7efe3c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:11.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe4424c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe3c102ef0 0x7efe3c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:11.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe4424c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c10fe80 con 0x7efe3c102ef0 2024-09-11T23:37:11.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe4424c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c10fd00 con 0x7efe3c100e20 2024-09-11T23:37:11.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe4424c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c110000 con 0x7efe3c102570 2024-09-11T23:37:11.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe427e9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe3c102ef0 0x7efe3c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:11.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe417e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c102570 0x7efe3c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:11.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe427e9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe3c102ef0 0x7efe3c19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60420/0 (socket says 172.21.15.133:60420) 2024-09-11T23:37:11.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe427e9640 1 -- 172.21.15.133:0/3527328448 learned_addr learned my addr 172.21.15.133:0/3527328448 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:11.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe41fe8640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c100e20 0x7efe3c1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:11.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe427e9640 1 -- 172.21.15.133:0/3527328448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c102570 msgr2=0x7efe3c196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:11.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe427e9640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c102570 0x7efe3c196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:11.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe427e9640 1 -- 172.21.15.133:0/3527328448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c100e20 msgr2=0x7efe3c1961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:11.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe427e9640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c100e20 0x7efe3c1961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:11.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.446+0000 7efe427e9640 1 -- 172.21.15.133:0/3527328448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c19b190 con 0x7efe3c102ef0 2024-09-11T23:37:11.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.447+0000 7efe427e9640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe3c102ef0 0x7efe3c19aab0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efe3402ae70 tx=0x7efe3402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:11.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.447+0000 7efe32ffd640 1 -- 172.21.15.133:0/3527328448 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34034040 con 0x7efe3c102ef0 2024-09-11T23:37:11.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.447+0000 7efe4424c640 1 -- 172.21.15.133:0/3527328448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3c19b420 con 0x7efe3c102ef0 2024-09-11T23:37:11.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.447+0000 7efe32ffd640 1 -- 172.21.15.133:0/3527328448 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe34030920 con 0x7efe3c102ef0 2024-09-11T23:37:11.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.447+0000 7efe32ffd640 1 -- 172.21.15.133:0/3527328448 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe340047f0 con 0x7efe3c102ef0 2024-09-11T23:37:11.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.447+0000 7efe4424c640 1 -- 172.21.15.133:0/3527328448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3c1a2d60 con 0x7efe3c102ef0 2024-09-11T23:37:11.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.448+0000 7efe4424c640 1 -- 172.21.15.133:0/3527328448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c1021b0 con 0x7efe3c102ef0 2024-09-11T23:37:11.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.449+0000 7efe32ffd640 1 -- 172.21.15.133:0/3527328448 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7efe34037020 con 0x7efe3c102ef0 2024-09-11T23:37:11.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.450+0000 7efe32ffd640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efe280a2ea0 0x7efe280a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:11.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.450+0000 7efe32ffd640 1 -- 172.21.15.133:0/3527328448 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7efe340f04c0 con 0x7efe3c102ef0 2024-09-11T23:37:11.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.450+0000 7efe41fe8640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efe280a2ea0 0x7efe280a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:11.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.451+0000 7efe41fe8640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efe280a2ea0 0x7efe280a5360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efe38004630 tx=0x7efe3800b4d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:11.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.454+0000 7efe32ffd640 1 -- 172.21.15.133:0/3527328448 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe340bd020 con 0x7efe3c102ef0 2024-09-11T23:37:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.659+0000 7efe4424c640 1 -- 172.21.15.133:0/3527328448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe3c1a2f30 con 0x7efe3c102ef0 2024-09-11T23:37:11.659 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:11 smithi133 ceph-mon[31430]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 KiB/s wr, 42 op/s 2024-09-11T23:37:11.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.660+0000 7efe32ffd640 1 -- 172.21.15.133:0/3527328448 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7efe340bc920 con 0x7efe3c102ef0 2024-09-11T23:37:11.660 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:37:11.660 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:37:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:37:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 -- 172.21.15.133:0/3527328448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efe280a2ea0 msgr2=0x7efe280a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efe280a2ea0 0x7efe280a5360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efe38004630 tx=0x7efe3800b4d0 comp rx=0 tx=0).stop 2024-09-11T23:37:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 -- 172.21.15.133:0/3527328448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe3c102ef0 msgr2=0x7efe3c19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe3c102ef0 0x7efe3c19aab0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efe3402ae70 tx=0x7efe3402f5d0 comp rx=0 tx=0).stop 2024-09-11T23:37:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 -- 172.21.15.133:0/3527328448 shutdown_connections 2024-09-11T23:37:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efe280a2ea0 0x7efe280a5360 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe3c102ef0 0x7efe3c19aab0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe3c102570 0x7efe3c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 --2- 172.21.15.133:0/3527328448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c100e20 0x7efe3c1961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 -- 172.21.15.133:0/3527328448 >> 172.21.15.133:0/3527328448 conn(0x7efe3c0fc7d0 msgr2=0x7efe3c106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 -- 172.21.15.133:0/3527328448 shutdown_connections 2024-09-11T23:37:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:11.662+0000 7efe4424c640 1 -- 172.21.15.133:0/3527328448 wait complete. 2024-09-11T23:37:11.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:11 smithi178 ceph-mon[35882]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 KiB/s wr, 42 op/s 2024-09-11T23:37:11.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:11 smithi138 ceph-mon[35142]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 KiB/s wr, 42 op/s 2024-09-11T23:37:12.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:34:53.462890+0000', 'modified': '2024-09-11T23:36:46.031976+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24481, 'mds_1': 24472, 'mds_2': 24464, 'mds_3': 24419, 'mds_4': 24401}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6839/570951791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 570951791}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 570951791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6841/1294883246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6840', 'nonce': 1294883246}, {'type': 'v1', 'addr': '172.21.15.133:6841', 'nonce': 1294883246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.138:6833/1969946111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1969946111}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1969946111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.138:6835/137089847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 137089847}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 137089847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/3569562128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 3569562128}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 3569562128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6837/1115908278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 1115908278}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 1115908278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/1681049778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 1681049778}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 1681049778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.138:6839/2407895128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 2407895128}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 2407895128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.178:6837/1040997001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 1040997001}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 1040997001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.178:6839/3794667907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 3794667907}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 3794667907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24481, 'qdb_cluster': [24401, 24419, 24464, 24472, 24481]} 2024-09-11T23:37:12.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T23:37:12.324 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-11T23:37:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.340+0000 7fe67db78640 1 -- 172.21.15.133:0/746705594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe678101350 msgr2=0x7fe67810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.340+0000 7fe67db78640 1 --2- 172.21.15.133:0/746705594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe678101350 0x7fe67810d880 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe670009cd0 tx=0x7fe67002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.340+0000 7fe67db78640 1 -- 172.21.15.133:0/746705594 shutdown_connections 2024-09-11T23:37:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.340+0000 7fe67db78640 1 --2- 172.21.15.133:0/746705594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe678101350 0x7fe67810d880 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.340+0000 7fe67db78640 1 --2- 172.21.15.133:0/746705594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6781009b0 0x7fe678100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.340+0000 7fe67db78640 1 --2- 172.21.15.133:0/746705594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6781069c0 0x7fe678106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.340+0000 7fe67db78640 1 -- 172.21.15.133:0/746705594 >> 172.21.15.133:0/746705594 conn(0x7fe678077990 msgr2=0x7fe6780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.340+0000 7fe67db78640 1 -- 172.21.15.133:0/746705594 shutdown_connections 2024-09-11T23:37:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.340+0000 7fe67db78640 1 -- 172.21.15.133:0/746705594 wait complete. 2024-09-11T23:37:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe67db78640 1 Processor -- start 2024-09-11T23:37:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe67db78640 1 -- start start 2024-09-11T23:37:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe67db78640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6781009b0 0x7fe6781991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe67db78640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe678101350 0x7fe678199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe67db78640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6781069c0 0x7fe6781a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe67db78640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe678110e00 con 0x7fe678101350 2024-09-11T23:37:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe67db78640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe678110c80 con 0x7fe6781009b0 2024-09-11T23:37:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe67db78640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe678110f80 con 0x7fe6781069c0 2024-09-11T23:37:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe66ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe678101350 0x7fe678199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe66ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe678101350 0x7fe678199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60698/0 (socket says 172.21.15.133:60698) 2024-09-11T23:37:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe66ffff640 1 -- 172.21.15.133:0/3018442322 learned_addr learned my addr 172.21.15.133:0/3018442322 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe6777fe640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6781009b0 0x7fe6781991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe677fff640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6781069c0 0x7fe6781a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe66ffff640 1 -- 172.21.15.133:0/3018442322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6781069c0 msgr2=0x7fe6781a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe66ffff640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6781069c0 0x7fe6781a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe66ffff640 1 -- 172.21.15.133:0/3018442322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6781009b0 msgr2=0x7fe6781991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe66ffff640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6781009b0 0x7fe6781991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe66ffff640 1 -- 172.21.15.133:0/3018442322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6781a4790 con 0x7fe678101350 2024-09-11T23:37:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.341+0000 7fe677fff640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6781069c0 0x7fe6781a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.342+0000 7fe66ffff640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe678101350 0x7fe678199710 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe66000d950 tx=0x7fe66000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.342+0000 7fe6757fa640 1 -- 172.21.15.133:0/3018442322 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660017070 con 0x7fe678101350 2024-09-11T23:37:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.342+0000 7fe6757fa640 1 -- 172.21.15.133:0/3018442322 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe66000de80 con 0x7fe678101350 2024-09-11T23:37:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.342+0000 7fe67db78640 1 -- 172.21.15.133:0/3018442322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6781a4960 con 0x7fe678101350 2024-09-11T23:37:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.342+0000 7fe6757fa640 1 -- 172.21.15.133:0/3018442322 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66000f620 con 0x7fe678101350 2024-09-11T23:37:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.342+0000 7fe67db78640 1 -- 172.21.15.133:0/3018442322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6781a4e80 con 0x7fe678101350 2024-09-11T23:37:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.344+0000 7fe6757fa640 1 -- 172.21.15.133:0/3018442322 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fe660009c30 con 0x7fe678101350 2024-09-11T23:37:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe6809ee740 1 Processor -- start 2024-09-11T23:37:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe6809ee740 1 -- start start 2024-09-11T23:37:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5289b0 0x55c67c528d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5292d0 0x55c67c537b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c53a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe6809ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c67c48d2f0 con 0x55c67c5289b0 2024-09-11T23:37:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe6809ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c67c435270 con 0x55c67c5380c0 2024-09-11T23:37:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe6809ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c67c4892b0 con 0x55c67c5292d0 2024-09-11T23:37:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe677fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5289b0 0x55c67c528d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe677fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5289b0 0x55c67c528d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60700/0 (socket says 172.21.15.133:60700) 2024-09-11T23:37:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe677fff640 1 -- 172.21.15.133:0/4250801774 learned_addr learned my addr 172.21.15.133:0/4250801774 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe6777fe640 1 --2- 172.21.15.133:0/4250801774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5292d0 0x55c67c537b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe66ffff640 1 --2- 172.21.15.133:0/4250801774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c53a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe677fff640 1 -- 172.21.15.133:0/4250801774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5292d0 msgr2=0x55c67c537b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe677fff640 1 --2- 172.21.15.133:0/4250801774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5292d0 0x55c67c537b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe6757fa640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6640a2f50 0x7fe6640a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe677fff640 1 -- 172.21.15.133:0/4250801774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 msgr2=0x55c67c53a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe677fff640 1 --2- 172.21.15.133:0/4250801774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c53a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.345+0000 7fe677fff640 1 -- 172.21.15.133:0/4250801774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67c434340 con 0x55c67c5289b0 2024-09-11T23:37:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.346+0000 7fe6757fa640 1 -- 172.21.15.133:0/3018442322 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe6600cf330 con 0x7fe678101350 2024-09-11T23:37:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.346+0000 7fe677fff640 1 --2- 172.21.15.133:0/4250801774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5289b0 0x55c67c528d90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe670005e30 tx=0x7fe670009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7caded4b0ab5c0d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.346+0000 7fe677fff640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6640a2f50 0x7fe6640a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.346+0000 7fe66dffb640 1 -- 172.21.15.133:0/4250801774 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670004240 con 0x55c67c5289b0 2024-09-11T23:37:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.346+0000 7fe66dffb640 1 -- 172.21.15.133:0/4250801774 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe670036040 con 0x55c67c5289b0 2024-09-11T23:37:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.346+0000 7fe66dffb640 1 -- 172.21.15.133:0/4250801774 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670030e10 con 0x55c67c5289b0 2024-09-11T23:37:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe6809ee740 1 -- 172.21.15.133:0/4250801774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5289b0 msgr2=0x55c67c528d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe677fff640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6640a2f50 0x7fe6640a5410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe670048000 tx=0x7fe67004b000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe6809ee740 1 --2- 172.21.15.133:0/4250801774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5289b0 0x55c67c528d90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe670005e30 tx=0x7fe670009a20 comp rx=0 tx=0).stop 2024-09-11T23:37:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe6809ee740 1 -- 172.21.15.133:0/4250801774 shutdown_connections 2024-09-11T23:37:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe6809ee740 1 --2- 172.21.15.133:0/4250801774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c53a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe6809ee740 1 --2- 172.21.15.133:0/4250801774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5292d0 0x55c67c537b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe6809ee740 1 --2- 172.21.15.133:0/4250801774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5289b0 0x55c67c528d90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe6809ee740 1 -- 172.21.15.133:0/4250801774 >> 172.21.15.133:0/4250801774 conn(0x55c67c525060 msgr2=0x55c67c525440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe6809ee740 1 -- 172.21.15.133:0/4250801774 shutdown_connections 2024-09-11T23:37:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe6809ee740 1 -- 172.21.15.133:0/4250801774 wait complete. 2024-09-11T23:37:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe6809ee740 1 -- start start 2024-09-11T23:37:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.347+0000 7fe6809ee740 1 Processor -- start 2024-09-11T23:37:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5289b0 0x55c67c535010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c535550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c535a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe6809ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c67c53ac30 con 0x55c67c5292d0 2024-09-11T23:37:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe6809ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c67c416290 con 0x55c67c5380c0 2024-09-11T23:37:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe6809ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c67c53adb0 con 0x55c67c5289b0 2024-09-11T23:37:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe6809ee740 10 client.0 _ll_register_callbacks cb 0x55c67c447140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe66ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c535550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe66ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c535550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60706/0 (socket says 172.21.15.133:60706) 2024-09-11T23:37:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe66ffff640 1 -- 172.21.15.133:0/1543676268 learned_addr learned my addr 172.21.15.133:0/1543676268 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe677fff640 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c535a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe6777fe640 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5289b0 0x55c67c535010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe66ffff640 1 -- 172.21.15.133:0/1543676268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5289b0 msgr2=0x55c67c535010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe66ffff640 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5289b0 0x55c67c535010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe66ffff640 1 -- 172.21.15.133:0/1543676268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 msgr2=0x55c67c535a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe66ffff640 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c535a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe66ffff640 1 -- 172.21.15.133:0/1543676268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67c1a0210 con 0x55c67c5292d0 2024-09-11T23:37:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe66ffff640 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c535550 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe660009610 tx=0x7fe66009afe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d10d2f4a3034e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe656ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe656ffd640 1 -- 172.21.15.133:0/1543676268 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660020050 con 0x55c67c5292d0 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe656ffd640 1 -- 172.21.15.133:0/1543676268 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe660084dc0 con 0x55c67c5292d0 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.348+0000 7fe6809ee740 1 -- 172.21.15.133:0/1543676268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe670036040 con 0x55c67c5292d0 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe656ffd640 1 -- 172.21.15.133:0/1543676268 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe66009fa30 con 0x55c67c5292d0 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe6809ee740 10 client.15519 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe6809ee740 1 -- 172.21.15.133:0/1543676268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c67c26c620 con 0x55c67c5292d0 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe656ffd640 1 -- 172.21.15.133:0/1543676268 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe670036040 con 0x55c67c5292d0 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe656ffd640 1 -- 172.21.15.133:0/1543676268 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe66009aad0 con 0x55c67c5292d0 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe6809ee740 10 client.15519 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe6809ee740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe6809ee740 20 client.15519 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe6809ee740 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c67c547fd0 0x55c67c54a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe6809ee740 4 client.15519 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe6809ee740 1 -- 172.21.15.133:0/1543676268 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c67c49aa20 con 0x55c67c547fd0 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.349+0000 7fe6777fe640 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c67c547fd0 0x55c67c54a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.350+0000 7fe6777fe640 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c67c547fd0 0x55c67c54a3c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f233ecb8952ec90 server_cookie=abbc06a15bc14c88 in_seq=0 out_seq=0 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.350+0000 7fe656ffd640 10 client.15519 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.350+0000 7fe656ffd640 1 -- 172.21.15.133:0/1543676268 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c67c49aa20 con 0x55c67c547fd0 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.350+0000 7fe656ffd640 10 client.15519 handle_command_reply: tid=0 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 1 client.15519 shutdown 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 1 -- 172.21.15.133:0/1543676268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 msgr2=0x55c67c535550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c535550 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe660009610 tx=0x7fe66009afe0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 1 -- 172.21.15.133:0/1543676268 shutdown_connections 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c67c547fd0 0x55c67c54a3c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c535a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c535550 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 1 --2- 172.21.15.133:0/1543676268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5289b0 0x55c67c535010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe656ffd640 0 client.15519 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 1 -- 172.21.15.133:0/1543676268 >> 172.21.15.133:0/1543676268 conn(0x55c67c525060 msgr2=0x55c67c52adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 1 -- 172.21.15.133:0/1543676268 shutdown_connections 2024-09-11T23:37:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 1 -- 172.21.15.133:0/1543676268 wait complete. 2024-09-11T23:37:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.351+0000 7fe6809ee740 20 client.15519 trim_cache size 0 max 16384 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.366+0000 7fe6809ee740 1 Processor -- start 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.366+0000 7fe6809ee740 1 -- start start 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.366+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5289b0 0x55c67c54fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c550170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c532250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe6809ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781ab440 con 0x55c67c5292d0 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe6809ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781acf80 con 0x55c67c5380c0 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe6809ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781abb10 con 0x55c67c5289b0 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c550170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c550170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60708/0 (socket says 172.21.15.133:60708) 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66ffff640 1 -- 172.21.15.133:0/2724065720 learned_addr learned my addr 172.21.15.133:0/2724065720 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe6777fe640 1 --2- 172.21.15.133:0/2724065720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5289b0 0x55c67c54fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe677fff640 1 --2- 172.21.15.133:0/2724065720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c532250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66ffff640 1 -- 172.21.15.133:0/2724065720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5289b0 msgr2=0x55c67c54fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66ffff640 1 --2- 172.21.15.133:0/2724065720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5289b0 0x55c67c54fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66ffff640 1 -- 172.21.15.133:0/2724065720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 msgr2=0x55c67c532250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66ffff640 1 --2- 172.21.15.133:0/2724065720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c532250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66ffff640 1 -- 172.21.15.133:0/2724065720 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67c54a9a0 con 0x55c67c5292d0 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe6777fe640 1 --2- 172.21.15.133:0/2724065720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5289b0 0x55c67c54fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe677fff640 1 --2- 172.21.15.133:0/2724065720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c532250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66ffff640 1 --2- 172.21.15.133:0/2724065720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c550170 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe66009aea0 tx=0x7fe66009b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a270c247a80572fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66d7fa640 1 -- 172.21.15.133:0/2724065720 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660020070 con 0x55c67c5292d0 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66d7fa640 1 -- 172.21.15.133:0/2724065720 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe660015020 con 0x55c67c5292d0 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.367+0000 7fe66d7fa640 1 -- 172.21.15.133:0/2724065720 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660084bc0 con 0x55c67c5292d0 2024-09-11T23:37:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.368+0000 7fe6809ee740 1 -- 172.21.15.133:0/2724065720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 msgr2=0x55c67c550170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.368+0000 7fe6809ee740 1 --2- 172.21.15.133:0/2724065720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c550170 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe66009aea0 tx=0x7fe66009b110 comp rx=0 tx=0).stop 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.368+0000 7fe6809ee740 1 -- 172.21.15.133:0/2724065720 shutdown_connections 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.368+0000 7fe6809ee740 1 --2- 172.21.15.133:0/2724065720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5380c0 0x55c67c532250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.368+0000 7fe6809ee740 1 --2- 172.21.15.133:0/2724065720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5292d0 0x55c67c550170 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.368+0000 7fe6809ee740 1 --2- 172.21.15.133:0/2724065720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5289b0 0x55c67c54fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.368+0000 7fe6809ee740 1 -- 172.21.15.133:0/2724065720 >> 172.21.15.133:0/2724065720 conn(0x55c67c525060 msgr2=0x55c67c537090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.368+0000 7fe6809ee740 1 -- 172.21.15.133:0/2724065720 shutdown_connections 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.368+0000 7fe6809ee740 1 -- 172.21.15.133:0/2724065720 wait complete. 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.368+0000 7fe6809ee740 1 -- start start 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe6809ee740 1 Processor -- start 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5289b0 0x55c67c53c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5292d0 0x55c67c53c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe6809ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5380c0 0x55c67c5b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe6809ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781ac8b0 con 0x55c67c5380c0 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe6809ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781ac1e0 con 0x55c67c5289b0 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe6809ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781a9460 con 0x55c67c5292d0 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe6809ee740 10 client.0 _ll_register_callbacks cb 0x7fe66009b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe677fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5380c0 0x55c67c5b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe677fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5380c0 0x55c67c5b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60726/0 (socket says 172.21.15.133:60726) 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe677fff640 1 -- 172.21.15.133:0/2379534749 learned_addr learned my addr 172.21.15.133:0/2379534749 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe66ffff640 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5292d0 0x55c67c53c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe6777fe640 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5289b0 0x55c67c53c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe677fff640 1 -- 172.21.15.133:0/2379534749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5292d0 msgr2=0x55c67c53c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe677fff640 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5292d0 0x55c67c53c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe677fff640 1 -- 172.21.15.133:0/2379534749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5289b0 msgr2=0x55c67c53c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe677fff640 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5289b0 0x55c67c53c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.369+0000 7fe677fff640 1 -- 172.21.15.133:0/2379534749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67c52fd50 con 0x55c67c5380c0 2024-09-11T23:37:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe677fff640 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5380c0 0x55c67c5b1c00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe670005e30 tx=0x7fe670031710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8f6a536ededd34 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe656ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe656ffd640 1 -- 172.21.15.133:0/2379534749 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670004b40 con 0x55c67c5380c0 2024-09-11T23:37:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe6809ee740 1 -- 172.21.15.133:0/2379534749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe660015020 con 0x55c67c5380c0 2024-09-11T23:37:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe656ffd640 1 -- 172.21.15.133:0/2379534749 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe670004ce0 con 0x55c67c5380c0 2024-09-11T23:37:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe656ffd640 1 -- 172.21.15.133:0/2379534749 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6700309e0 con 0x55c67c5380c0 2024-09-11T23:37:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe6809ee740 10 client.15525 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe6809ee740 1 -- 172.21.15.133:0/2379534749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c67c537ba0 con 0x55c67c5380c0 2024-09-11T23:37:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe656ffd640 1 -- 172.21.15.133:0/2379534749 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe660015020 con 0x55c67c5380c0 2024-09-11T23:37:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe656ffd640 1 -- 172.21.15.133:0/2379534749 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe67006c020 con 0x55c67c5380c0 2024-09-11T23:37:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe6809ee740 10 client.15525 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe6809ee740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe6809ee740 20 client.15525 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe6809ee740 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c67c5b3150 0x55c67c5b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe6809ee740 4 client.15525 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:37:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.370+0000 7fe6809ee740 1 -- 172.21.15.133:0/2379534749 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c67c494d70 con 0x55c67c5b3150 2024-09-11T23:37:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.371+0000 7fe6777fe640 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c67c5b3150 0x55c67c5b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.371+0000 7fe656ffd640 10 client.15525 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.371+0000 7fe6777fe640 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c67c5b3150 0x55c67c5b5540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f49f9558df9183c9 server_cookie=38eac7fe9835c8bf in_seq=0 out_seq=0 2024-09-11T23:37:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe656ffd640 1 -- 172.21.15.133:0/2379534749 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c67c494d70 con 0x55c67c5b3150 2024-09-11T23:37:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe656ffd640 10 client.15525 handle_command_reply: tid=0 2024-09-11T23:37:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 1 client.15525 shutdown 2024-09-11T23:37:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 1 -- 172.21.15.133:0/2379534749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5380c0 msgr2=0x55c67c5b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5380c0 0x55c67c5b1c00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe670005e30 tx=0x7fe670031710 comp rx=0 tx=0).stop 2024-09-11T23:37:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 1 -- 172.21.15.133:0/2379534749 shutdown_connections 2024-09-11T23:37:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c67c5b3150 0x55c67c5b5540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c67c5380c0 0x55c67c5b1c00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67c5292d0 0x55c67c53c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 1 --2- 172.21.15.133:0/2379534749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c67c5289b0 0x55c67c53c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe656ffd640 0 client.15525 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 1 -- 172.21.15.133:0/2379534749 >> 172.21.15.133:0/2379534749 conn(0x55c67c525060 msgr2=0x55c67c5239a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 1 -- 172.21.15.133:0/2379534749 shutdown_connections 2024-09-11T23:37:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 1 -- 172.21.15.133:0/2379534749 wait complete. 2024-09-11T23:37:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.382+0000 7fe6809ee740 20 client.15525 trim_cache size 0 max 16384 2024-09-11T23:37:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:37:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:37:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:37:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:37:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 -- 172.21.15.133:0/3018442322 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6640a2f50 msgr2=0x7fe6640a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6640a2f50 0x7fe6640a5410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe670048000 tx=0x7fe67004b000 comp rx=0 tx=0).stop 2024-09-11T23:37:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 -- 172.21.15.133:0/3018442322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe678101350 msgr2=0x7fe678199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe678101350 0x7fe678199710 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe66000d950 tx=0x7fe66000dd50 comp rx=0 tx=0).stop 2024-09-11T23:37:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 -- 172.21.15.133:0/3018442322 shutdown_connections 2024-09-11T23:37:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6640a2f50 0x7fe6640a5410 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6781069c0 0x7fe6781a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe678101350 0x7fe678199710 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 --2- 172.21.15.133:0/3018442322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6781009b0 0x7fe6781991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 -- 172.21.15.133:0/3018442322 >> 172.21.15.133:0/3018442322 conn(0x7fe678077990 msgr2=0x7fe6780fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 -- 172.21.15.133:0/3018442322 shutdown_connections 2024-09-11T23:37:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:12.383+0000 7fe67db78640 1 -- 172.21.15.133:0/3018442322 wait complete. 2024-09-11T23:37:12.588 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:12.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:12 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3527328448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:12.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:12 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3527328448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:12.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:12 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3527328448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:13.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.051+0000 7f1da0f6f640 1 -- 172.21.15.133:0/2577753675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c106f10 msgr2=0x7f1d9c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:13.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.051+0000 7f1da0f6f640 1 --2- 172.21.15.133:0/2577753675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c106f10 0x7f1d9c10ea90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009ca0 tx=0x7f1d9402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.051+0000 7f1da0f6f640 1 -- 172.21.15.133:0/2577753675 shutdown_connections 2024-09-11T23:37:13.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.051+0000 7f1da0f6f640 1 --2- 172.21.15.133:0/2577753675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c106f10 0x7f1d9c10ea90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.051+0000 7f1da0f6f640 1 --2- 172.21.15.133:0/2577753675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c102790 0x7f1d9c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.051+0000 7f1da0f6f640 1 --2- 172.21.15.133:0/2577753675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c101de0 0x7f1d9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.051+0000 7f1da0f6f640 1 -- 172.21.15.133:0/2577753675 >> 172.21.15.133:0/2577753675 conn(0x7f1d9c0fd8c0 msgr2=0x7f1d9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:13.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.051+0000 7f1da0f6f640 1 -- 172.21.15.133:0/2577753675 shutdown_connections 2024-09-11T23:37:13.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.051+0000 7f1da0f6f640 1 -- 172.21.15.133:0/2577753675 wait complete. 2024-09-11T23:37:13.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.051+0000 7f1da0f6f640 1 Processor -- start 2024-09-11T23:37:13.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1da0f6f640 1 -- start start 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1da0f6f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c101de0 0x7f1d9c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1da0f6f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c102790 0x7f1d9c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1da0f6f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c106f10 0x7f1d9c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1da0f6f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c111f70 con 0x7f1d9c102790 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1da0f6f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c111df0 con 0x7f1d9c101de0 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1da0f6f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c1120f0 con 0x7f1d9c106f10 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d99d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c102790 0x7f1d9c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d99d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c102790 0x7f1d9c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60742/0 (socket says 172.21.15.133:60742) 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d99d9b640 1 -- 172.21.15.133:0/4073086720 learned_addr learned my addr 172.21.15.133:0/4073086720 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d9a59c640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c101de0 0x7f1d9c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d9ad9d640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c106f10 0x7f1d9c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d99d9b640 1 -- 172.21.15.133:0/4073086720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c106f10 msgr2=0x7f1d9c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c106f10 0x7f1d9c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d99d9b640 1 -- 172.21.15.133:0/4073086720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c101de0 msgr2=0x7f1d9c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c101de0 0x7f1d9c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d99d9b640 1 -- 172.21.15.133:0/4073086720 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c19d380 con 0x7f1d9c102790 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d9ad9d640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c106f10 0x7f1d9c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c102790 0x7f1d9c198910 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400d950 tx=0x7f1d8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d937fe640 1 -- 172.21.15.133:0/4073086720 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84017070 con 0x7f1d9c102790 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d937fe640 1 -- 172.21.15.133:0/4073086720 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d84009c30 con 0x7f1d9c102790 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1da0f6f640 1 -- 172.21.15.133:0/4073086720 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c19d670 con 0x7f1d9c102790 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.052+0000 7f1d937fe640 1 -- 172.21.15.133:0/4073086720 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400f6b0 con 0x7f1d9c102790 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.053+0000 7f1da0f6f640 1 -- 172.21.15.133:0/4073086720 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c1a4f50 con 0x7f1d9c102790 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.055+0000 7f1d937fe640 1 -- 172.21.15.133:0/4073086720 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f1d840109d0 con 0x7f1d9c102790 2024-09-11T23:37:13.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.055+0000 7f1da3348740 1 Processor -- start 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.055+0000 7f1da3348740 1 -- start start 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.055+0000 7f1da3348740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0203e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.055+0000 7f1da3348740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e12f0 0x55e0203eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.055+0000 7f1da3348740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.055+0000 7f1da3348740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e020347090 con 0x55e0203e09d0 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.055+0000 7f1da3348740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0202eea70 con 0x55e0203ebbb0 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.055+0000 7f1da3348740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e020343050 con 0x55e0203e12f0 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.055+0000 7f1d937fe640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1d880a2ea0 0x7f1d880a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0203e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0203e0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60746/0 (socket says 172.21.15.133:60746) 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9ad9d640 1 -- 172.21.15.133:0/2994262946 learned_addr learned my addr 172.21.15.133:0/2994262946 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9ad9d640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1d880a2ea0 0x7f1d880a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/2994262946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9a59c640 1 --2- 172.21.15.133:0/2994262946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e12f0 0x55e0203eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d937fe640 1 -- 172.21.15.133:0/4073086720 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f1d840cf210 con 0x7f1d9c102790 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9ad9d640 1 -- 172.21.15.133:0/2994262946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e12f0 msgr2=0x55e0203eb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9ad9d640 1 --2- 172.21.15.133:0/2994262946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e12f0 0x55e0203eb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9ad9d640 1 -- 172.21.15.133:0/2994262946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 msgr2=0x55e0203edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9ad9d640 1 --2- 172.21.15.133:0/2994262946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203edfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9ad9d640 1 -- 172.21.15.133:0/2994262946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0202fb130 con 0x55e0203e09d0 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/2994262946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9ad9d640 1 --2- 172.21.15.133:0/2994262946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0203e0db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006010 tx=0x7f1d94031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8169300e3c015e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d9ad9d640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1d880a2ea0 0x7f1d880a5360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009bf0 tx=0x7f1d94047000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d90ff9640 1 -- 172.21.15.133:0/2994262946 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9402fdb0 con 0x55e0203e09d0 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d90ff9640 1 -- 172.21.15.133:0/2994262946 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d94036040 con 0x55e0203e09d0 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.056+0000 7f1d90ff9640 1 -- 172.21.15.133:0/2994262946 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94049690 con 0x55e0203e09d0 2024-09-11T23:37:13.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.057+0000 7f1da3348740 1 -- 172.21.15.133:0/2994262946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 msgr2=0x55e0203e0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.057+0000 7f1da3348740 1 --2- 172.21.15.133:0/2994262946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0203e0db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006010 tx=0x7f1d94031870 comp rx=0 tx=0).stop 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.057+0000 7f1da3348740 1 -- 172.21.15.133:0/2994262946 shutdown_connections 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.057+0000 7f1da3348740 1 --2- 172.21.15.133:0/2994262946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.057+0000 7f1da3348740 1 --2- 172.21.15.133:0/2994262946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e12f0 0x55e0203eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.057+0000 7f1da3348740 1 --2- 172.21.15.133:0/2994262946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0203e0db0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.057+0000 7f1da3348740 1 -- 172.21.15.133:0/2994262946 >> 172.21.15.133:0/2994262946 conn(0x55e0203dcf60 msgr2=0x55e0203dd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.057+0000 7f1da3348740 1 -- 172.21.15.133:0/2994262946 shutdown_connections 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.057+0000 7f1da3348740 1 -- 172.21.15.133:0/2994262946 wait complete. 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1da3348740 1 -- start start 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1da3348740 1 Processor -- start 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1da3348740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e09d0 0x55e0203eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1da3348740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1da3348740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1da3348740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e0203ee780 con 0x55e0203e12f0 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1da3348740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e020301f20 con 0x55e0203ebbb0 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1da3348740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0203ee900 con 0x55e0203e09d0 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1da3348740 10 client.0 _ll_register_callbacks cb 0x55e0202cb030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1d99d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1d99d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60754/0 (socket says 172.21.15.133:60754) 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1d99d9b640 1 -- 172.21.15.133:0/2108218019 learned_addr learned my addr 172.21.15.133:0/2108218019 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1d9ad9d640 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1d9a59c640 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e09d0 0x55e0203eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1d99d9b640 1 -- 172.21.15.133:0/2108218019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e09d0 msgr2=0x55e0203eb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e09d0 0x55e0203eb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1d99d9b640 1 -- 172.21.15.133:0/2108218019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 msgr2=0x55e0203e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.058+0000 7f1d99d9b640 1 -- 172.21.15.133:0/2108218019 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0200b82b0 con 0x55e0203e12f0 2024-09-11T23:37:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.059+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e8800 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1d84010fd0 tx=0x7f1d84010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342338de98490498 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.059+0000 7f1d81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.059+0000 7f1d81ffb640 1 -- 172.21.15.133:0/2108218019 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84020040 con 0x55e0203e12f0 2024-09-11T23:37:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.059+0000 7f1d81ffb640 1 -- 172.21.15.133:0/2108218019 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f1d84097c10 con 0x55e0203e12f0 2024-09-11T23:37:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.059+0000 7f1da3348740 1 -- 172.21.15.133:0/2108218019 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d94036040 con 0x55e0203e12f0 2024-09-11T23:37:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.059+0000 7f1d81ffb640 1 -- 172.21.15.133:0/2108218019 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1d8409b5a0 con 0x55e0203e12f0 2024-09-11T23:37:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.059+0000 7f1da3348740 10 client.15537 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.059+0000 7f1da3348740 1 -- 172.21.15.133:0/2108218019 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e0203fa060 con 0x55e0203e12f0 2024-09-11T23:37:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.059+0000 7f1d81ffb640 1 -- 172.21.15.133:0/2108218019 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d94036040 con 0x55e0203e12f0 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.060+0000 7f1d81ffb640 1 -- 172.21.15.133:0/2108218019 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1d8409a9f0 con 0x55e0203e12f0 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.060+0000 7f1da3348740 10 client.15537 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.060+0000 7f1da3348740 10 client.15537 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.060+0000 7f1da3348740 20 client.15537 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.060+0000 7f1da3348740 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e0203fc670 0x55e0203fdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.060+0000 7f1da3348740 4 client.15537 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.060+0000 7f1da3348740 1 -- 172.21.15.133:0/2108218019 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e020354600 con 0x55e0203fc670 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.060+0000 7f1d9a59c640 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e0203fc670 0x55e0203fdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.060+0000 7f1d81ffb640 10 client.15537 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.060+0000 7f1d9a59c640 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e0203fc670 0x55e0203fdae0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=917f74e247aa11c3 server_cookie=6627dd396dbc1027 in_seq=0 out_seq=0 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1d81ffb640 1 -- 172.21.15.133:0/2108218019 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e020354600 con 0x55e0203fc670 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1d81ffb640 10 client.15537 handle_command_reply: tid=0 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 1 client.15537 shutdown 2024-09-11T23:37:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 1 -- 172.21.15.133:0/2108218019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 msgr2=0x55e0203e8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e8800 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1d84010fd0 tx=0x7f1d84010d70 comp rx=0 tx=0).stop 2024-09-11T23:37:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 1 -- 172.21.15.133:0/2108218019 shutdown_connections 2024-09-11T23:37:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e0203fc670 0x55e0203fdae0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e8800 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 1 --2- 172.21.15.133:0/2108218019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e09d0 0x55e0203eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1d81ffb640 0 client.15537 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 1 -- 172.21.15.133:0/2108218019 >> 172.21.15.133:0/2108218019 conn(0x55e0203dcf60 msgr2=0x55e0203e2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 1 -- 172.21.15.133:0/2108218019 shutdown_connections 2024-09-11T23:37:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 1 -- 172.21.15.133:0/2108218019 wait complete. 2024-09-11T23:37:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.061+0000 7f1da3348740 20 client.15537 trim_cache size 0 max 16384 2024-09-11T23:37:13.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1da3348740 1 Processor -- start 2024-09-11T23:37:13.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1da3348740 1 -- start start 2024-09-11T23:37:13.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1da3348740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e09d0 0x55e0203e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1da3348740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1da3348740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1da3348740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e0203fbe20 con 0x55e0203e12f0 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1da3348740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0203fc4f0 con 0x55e0203ebbb0 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1da3348740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0203fcbc0 con 0x55e0203e09d0 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1d99d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1d99d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60762/0 (socket says 172.21.15.133:60762) 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1d99d9b640 1 -- 172.21.15.133:0/3609921086 learned_addr learned my addr 172.21.15.133:0/3609921086 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1d9ad9d640 1 --2- 172.21.15.133:0/3609921086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203e5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1d9a59c640 1 --2- 172.21.15.133:0/3609921086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e09d0 0x55e0203e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1d99d9b640 1 -- 172.21.15.133:0/3609921086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e09d0 msgr2=0x55e0203e80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/3609921086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e09d0 0x55e0203e80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1d99d9b640 1 -- 172.21.15.133:0/3609921086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 msgr2=0x55e0203e5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/3609921086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203e5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.081+0000 7f1d99d9b640 1 -- 172.21.15.133:0/3609921086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0203ddff0 con 0x55e0203e12f0 2024-09-11T23:37:13.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.082+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/3609921086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e5a80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1d84097f70 tx=0x7f1d8409b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a40d093c1bc59 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:13.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.082+0000 7f1d83fff640 1 -- 172.21.15.133:0/3609921086 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84020070 con 0x55e0203e12f0 2024-09-11T23:37:13.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.082+0000 7f1d83fff640 1 -- 172.21.15.133:0/3609921086 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d8409f950 con 0x55e0203e12f0 2024-09-11T23:37:13.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.082+0000 7f1d83fff640 1 -- 172.21.15.133:0/3609921086 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8409a9f0 con 0x55e0203e12f0 2024-09-11T23:37:13.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.082+0000 7f1da3348740 1 -- 172.21.15.133:0/3609921086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 msgr2=0x55e0203e5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:13.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.082+0000 7f1da3348740 1 --2- 172.21.15.133:0/3609921086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e5a80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1d84097f70 tx=0x7f1d8409b030 comp rx=0 tx=0).stop 2024-09-11T23:37:13.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.082+0000 7f1da3348740 1 -- 172.21.15.133:0/3609921086 shutdown_connections 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.082+0000 7f1da3348740 1 --2- 172.21.15.133:0/3609921086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e0203e5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 --2- 172.21.15.133:0/3609921086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e12f0 0x55e0203e5a80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 --2- 172.21.15.133:0/3609921086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e09d0 0x55e0203e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 -- 172.21.15.133:0/3609921086 >> 172.21.15.133:0/3609921086 conn(0x55e0203dcf60 msgr2=0x55e0203e87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 -- 172.21.15.133:0/3609921086 shutdown_connections 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 -- 172.21.15.133:0/3609921086 wait complete. 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 -- start start 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 Processor -- start 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0204063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e12f0 0x55e0204016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e020401c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e0203fd960 con 0x55e0203e09d0 2024-09-11T23:37:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0203fd290 con 0x55e0203ebbb0 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0203fe030 con 0x55e0203e12f0 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1da3348740 10 client.0 _ll_register_callbacks cb 0x7f1d8400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1d9a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0204063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1d9a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0204063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60764/0 (socket says 172.21.15.133:60764) 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1d9a59c640 1 -- 172.21.15.133:0/420883380 learned_addr learned my addr 172.21.15.133:0/420883380 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1d9ad9d640 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e020401c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.083+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e12f0 0x55e0204016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d9a59c640 1 -- 172.21.15.133:0/420883380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e12f0 msgr2=0x55e0204016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d9a59c640 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e12f0 0x55e0204016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d9a59c640 1 -- 172.21.15.133:0/420883380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 msgr2=0x55e020401c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d9a59c640 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e020401c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d9a59c640 1 -- 172.21.15.133:0/420883380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e020402310 con 0x55e0203e09d0 2024-09-11T23:37:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e12f0 0x55e0204016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d9a59c640 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0204063b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x55e0203fe320 tx=0x7f1d8c00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5edbeae9f5921a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d81ffb640 1 -- 172.21.15.133:0/420883380 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c011640 con 0x55e0203e09d0 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d81ffb640 1 -- 172.21.15.133:0/420883380 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f1d8c0117e0 con 0x55e0203e09d0 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1da3348740 1 -- 172.21.15.133:0/420883380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d8409f950 con 0x55e0203e09d0 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d81ffb640 1 -- 172.21.15.133:0/420883380 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1d8c0119e0 con 0x55e0203e09d0 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1da3348740 10 client.15543 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1da3348740 1 -- 172.21.15.133:0/420883380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e020466ca0 con 0x55e0203e09d0 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.084+0000 7f1d81ffb640 1 -- 172.21.15.133:0/420883380 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d8409f950 con 0x55e0203e09d0 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.085+0000 7f1d81ffb640 1 -- 172.21.15.133:0/420883380 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1d8c01d3f0 con 0x55e0203e09d0 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.085+0000 7f1da3348740 10 client.15543 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.085+0000 7f1da3348740 10 client.15543 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-11T23:37:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.085+0000 7f1da3348740 20 client.15543 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.085+0000 7f1da3348740 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e02046b8a0 0x55e02046a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.085+0000 7f1da3348740 4 client.15543 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.085+0000 7f1da3348740 1 -- 172.21.15.133:0/420883380 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "status"}) -- 0x55e02034eb10 con 0x55e02046b8a0 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.085+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e02046b8a0 0x55e02046a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.085+0000 7f1d99d9b640 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e02046b8a0 0x55e02046a5b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=513a2350bae1c0af server_cookie=d38354b3d852017b in_seq=0 out_seq=0 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.085+0000 7f1d81ffb640 10 client.15543 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1d81ffb640 1 -- 172.21.15.133:0/420883380 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e02034eb10 con 0x55e02046b8a0 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1d81ffb640 10 client.15543 handle_command_reply: tid=0 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 1 client.15543 shutdown 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 1 -- 172.21.15.133:0/420883380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 msgr2=0x55e0204063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0204063b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x55e0203fe320 tx=0x7f1d8c00beb0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 1 -- 172.21.15.133:0/420883380 shutdown_connections 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e02046b8a0 0x55e02046a5b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0203ebbb0 0x55e020401c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0203e12f0 0x55e0204016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 1 --2- 172.21.15.133:0/420883380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0203e09d0 0x55e0204063b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 1 -- 172.21.15.133:0/420883380 >> 172.21.15.133:0/420883380 conn(0x55e0203dcf60 msgr2=0x55e0203db800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1d81ffb640 0 client.15543 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 1 -- 172.21.15.133:0/420883380 shutdown_connections 2024-09-11T23:37:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 1 -- 172.21.15.133:0/420883380 wait complete. 2024-09-11T23:37:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da3348740 20 client.15543 trim_cache size 0 max 16384 2024-09-11T23:37:13.091 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:37:13.091 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:37:13.091 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 4, 2024-09-11T23:37:13.092 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24401, 2024-09-11T23:37:13.092 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:37:13.092 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:37:13.092 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:37:13.092 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 128.637804297, 2024-09-11T23:37:13.092 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 28, 2024-09-11T23:37:13.092 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 82, 2024-09-11T23:37:13.092 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T23:37:13.092 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 177.775000178 2024-09-11T23:37:13.092 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:37:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da0f6f640 1 -- 172.21.15.133:0/4073086720 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1d880a2ea0 msgr2=0x7f1d880a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da0f6f640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1d880a2ea0 0x7f1d880a5360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009bf0 tx=0x7f1d94047000 comp rx=0 tx=0).stop 2024-09-11T23:37:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da0f6f640 1 -- 172.21.15.133:0/4073086720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c102790 msgr2=0x7f1d9c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da0f6f640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c102790 0x7f1d9c198910 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400d950 tx=0x7f1d8400de10 comp rx=0 tx=0).stop 2024-09-11T23:37:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da0f6f640 1 -- 172.21.15.133:0/4073086720 shutdown_connections 2024-09-11T23:37:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da0f6f640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1d880a2ea0 0x7f1d880a5360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da0f6f640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c106f10 0x7f1d9c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da0f6f640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c102790 0x7f1d9c198910 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.086+0000 7f1da0f6f640 1 --2- 172.21.15.133:0/4073086720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c101de0 0x7f1d9c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.087+0000 7f1da0f6f640 1 -- 172.21.15.133:0/4073086720 >> 172.21.15.133:0/4073086720 conn(0x7f1d9c0fd8c0 msgr2=0x7f1d9c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.087+0000 7f1da0f6f640 1 -- 172.21.15.133:0/4073086720 shutdown_connections 2024-09-11T23:37:13.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:13.087+0000 7f1da0f6f640 1 -- 172.21.15.133:0/4073086720 wait complete. 2024-09-11T23:37:13.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:13 smithi178 ceph-mon[35882]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 43 op/s 2024-09-11T23:37:13.817 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:13 smithi133 ceph-mon[31430]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 43 op/s 2024-09-11T23:37:13.845 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T23:37:13.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:13 smithi138 ceph-mon[35142]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 43 op/s 2024-09-11T23:37:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.594+0000 7f8124b51640 1 -- 172.21.15.133:0/1933107124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8120102ef0 msgr2=0x7f812010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.594+0000 7f8124b51640 1 --2- 172.21.15.133:0/1933107124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8120102ef0 0x7f812010da40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8118009ca0 tx=0x7f811802a950 comp rx=0 tx=0).stop 2024-09-11T23:37:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.594+0000 7f8124b51640 1 -- 172.21.15.133:0/1933107124 shutdown_connections 2024-09-11T23:37:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.594+0000 7f8124b51640 1 --2- 172.21.15.133:0/1933107124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8120102ef0 0x7f812010da40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.594+0000 7f8124b51640 1 --2- 172.21.15.133:0/1933107124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8120102570 0x7f81201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.594+0000 7f8124b51640 1 --2- 172.21.15.133:0/1933107124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8120100e20 0x7f8120101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.594+0000 7f8124b51640 1 -- 172.21.15.133:0/1933107124 >> 172.21.15.133:0/1933107124 conn(0x7f81200fc7d0 msgr2=0x7f81200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.594+0000 7f8124b51640 1 -- 172.21.15.133:0/1933107124 shutdown_connections 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.594+0000 7f8124b51640 1 -- 172.21.15.133:0/1933107124 wait complete. 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f8124b51640 1 Processor -- start 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f8124b51640 1 -- start start 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f8124b51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8120100e20 0x7f8120197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f8124b51640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8120102570 0x7f8120197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f8124b51640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8120102ef0 0x7f812019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f8124b51640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8120110f80 con 0x7f8120102ef0 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f8124b51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8120110e00 con 0x7f8120100e20 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f8124b51640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8120111100 con 0x7f8120102570 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8120102ef0 0x7f812019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8120102ef0 0x7f812019bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60782/0 (socket says 172.21.15.133:60782) 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811ed9d640 1 -- 172.21.15.133:0/2327346330 learned_addr learned my addr 172.21.15.133:0/2327346330 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811e59c640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8120100e20 0x7f8120197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811dd9b640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8120102570 0x7f8120197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811ed9d640 1 -- 172.21.15.133:0/2327346330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8120102570 msgr2=0x7f8120197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811ed9d640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8120102570 0x7f8120197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811ed9d640 1 -- 172.21.15.133:0/2327346330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8120100e20 msgr2=0x7f8120197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811ed9d640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8120100e20 0x7f8120197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811ed9d640 1 -- 172.21.15.133:0/2327346330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812019c3b0 con 0x7f8120102ef0 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811dd9b640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8120102570 0x7f8120197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811e59c640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8120100e20 0x7f8120197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.595+0000 7f811ed9d640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8120102ef0 0x7f812019bcd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f811802ae70 tx=0x7f811802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.596+0000 7f810f7fe640 1 -- 172.21.15.133:0/2327346330 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118034040 con 0x7f8120102ef0 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.596+0000 7f8124b51640 1 -- 172.21.15.133:0/2327346330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812019c640 con 0x7f8120102ef0 2024-09-11T23:37:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.596+0000 7f810f7fe640 1 -- 172.21.15.133:0/2327346330 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8118030920 con 0x7f8120102ef0 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.596+0000 7f810f7fe640 1 -- 172.21.15.133:0/2327346330 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118045480 con 0x7f8120102ef0 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.596+0000 7f8124b51640 1 -- 172.21.15.133:0/2327346330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81201a3f80 con 0x7f8120102ef0 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.598+0000 7f810f7fe640 1 -- 172.21.15.133:0/2327346330 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f8118037020 con 0x7f8120102ef0 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.598+0000 7f8126f2a740 1 Processor -- start 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.598+0000 7f8126f2a740 1 -- start start 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a5f9d0 0x5649f2a5fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a6a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a6cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f8126f2a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5649f29c6090 con 0x5649f2a6abb0 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f8126f2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5649f296da70 con 0x5649f2a5f9d0 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f8126f2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5649f29c2050 con 0x5649f2a602f0 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f810f7fe640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f81080a2ea0 0x7f81080a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a6cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a6cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60794/0 (socket says 172.21.15.133:60794) 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811ed9d640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f81080a2ea0 0x7f81080a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811dd9b640 1 -- 172.21.15.133:0/1617824712 learned_addr learned my addr 172.21.15.133:0/1617824712 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f810f7fe640 1 -- 172.21.15.133:0/2327346330 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f81180f04c0 con 0x7f8120102ef0 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811e59c640 1 --2- 172.21.15.133:0/1617824712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a6a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811dd9b640 1 --2- 172.21.15.133:0/1617824712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a5f9d0 0x5649f2a5fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811dd9b640 1 -- 172.21.15.133:0/1617824712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 msgr2=0x5649f2a6a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811dd9b640 1 --2- 172.21.15.133:0/1617824712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a6a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811dd9b640 1 -- 172.21.15.133:0/1617824712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a5f9d0 msgr2=0x5649f2a5fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811dd9b640 1 --2- 172.21.15.133:0/1617824712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a5f9d0 0x5649f2a5fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811dd9b640 1 -- 172.21.15.133:0/1617824712 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f297a130 con 0x5649f2a6abb0 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811ed9d640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f81080a2ea0 0x7f81080a5360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f81180edbb0 tx=0x7f81180a7040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.599+0000 7f811dd9b640 1 --2- 172.21.15.133:0/1617824712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a6cfa0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f81140136a0 tx=0x7f811400c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8790d1ca4e1f3a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f810cff9640 1 -- 172.21.15.133:0/1617824712 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114021070 con 0x5649f2a6abb0 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f810cff9640 1 -- 172.21.15.133:0/1617824712 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8114011cb0 con 0x5649f2a6abb0 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f810cff9640 1 -- 172.21.15.133:0/1617824712 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811401cde0 con 0x5649f2a6abb0 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f8126f2a740 1 -- 172.21.15.133:0/1617824712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 msgr2=0x5649f2a6cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f8126f2a740 1 --2- 172.21.15.133:0/1617824712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a6cfa0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f81140136a0 tx=0x7f811400c6d0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f8126f2a740 1 -- 172.21.15.133:0/1617824712 shutdown_connections 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f8126f2a740 1 --2- 172.21.15.133:0/1617824712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a6cfa0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f8126f2a740 1 --2- 172.21.15.133:0/1617824712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a6a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f8126f2a740 1 --2- 172.21.15.133:0/1617824712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a5f9d0 0x5649f2a5fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f8126f2a740 1 -- 172.21.15.133:0/1617824712 >> 172.21.15.133:0/1617824712 conn(0x5649f2a5bf60 msgr2=0x5649f2a5c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f8126f2a740 1 -- 172.21.15.133:0/1617824712 shutdown_connections 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.600+0000 7f8126f2a740 1 -- 172.21.15.133:0/1617824712 wait complete. 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f8126f2a740 1 -- start start 2024-09-11T23:37:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f8126f2a740 1 Processor -- start 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a6a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a67840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a6abb0 0x5649f2a67d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f8126f2a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5649f2a6d780 con 0x5649f2a5f9d0 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f8126f2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5649f2980f20 con 0x5649f2a6abb0 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f8126f2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5649f2a6d900 con 0x5649f2a602f0 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f8126f2a740 10 client.0 _ll_register_callbacks cb 0x5649f294a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f811dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a6a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f811dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a6a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60808/0 (socket says 172.21.15.133:60808) 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f811dd9b640 1 -- 172.21.15.133:0/2646161362 learned_addr learned my addr 172.21.15.133:0/2646161362 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f811e59c640 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a67840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f811dd9b640 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a6abb0 0x5649f2a67d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f811dd9b640 1 -- 172.21.15.133:0/2646161362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 msgr2=0x5649f2a67840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.601+0000 7f811dd9b640 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a67840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f811dd9b640 1 -- 172.21.15.133:0/2646161362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a6abb0 msgr2=0x5649f2a67d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f811dd9b640 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a6abb0 0x5649f2a67d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f811dd9b640 1 -- 172.21.15.133:0/2646161362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f27372b0 con 0x5649f2a5f9d0 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f811dd9b640 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a6a4f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8114005d80 tx=0x7f8114005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611b080966ee9973 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8105ffb640 1 -- 172.21.15.133:0/2646161362 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114021040 con 0x5649f2a5f9d0 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8105ffb640 1 -- 172.21.15.133:0/2646161362 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f811401cd60 con 0x5649f2a5f9d0 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8126f2a740 1 -- 172.21.15.133:0/2646161362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8114011cb0 con 0x5649f2a5f9d0 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8105ffb640 1 -- 172.21.15.133:0/2646161362 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8114011e60 con 0x5649f2a5f9d0 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8126f2a740 10 client.15555 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8126f2a740 1 -- 172.21.15.133:0/2646161362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649f2a79130 con 0x5649f2a5f9d0 2024-09-11T23:37:14.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8105ffb640 1 -- 172.21.15.133:0/2646161362 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8114011cb0 con 0x5649f2a5f9d0 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8105ffb640 1 -- 172.21.15.133:0/2646161362 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8114018050 con 0x5649f2a5f9d0 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8126f2a740 10 client.15555 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8126f2a740 10 client.15555 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8126f2a740 20 client.15555 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5649f2a7b590 0x5649f2a7ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8126f2a740 4 client.15555 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.602+0000 7f8126f2a740 1 -- 172.21.15.133:0/2646161362 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649f29d3600 con 0x5649f2a7b590 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.603+0000 7f811e59c640 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5649f2a7b590 0x5649f2a7ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.603+0000 7f8105ffb640 10 client.15555 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.603+0000 7f811e59c640 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5649f2a7b590 0x5649f2a7ca00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b15b326b6f2c58b0 server_cookie=77ee4af09b3c7e48 in_seq=0 out_seq=0 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8105ffb640 1 -- 172.21.15.133:0/2646161362 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649f29d3600 con 0x5649f2a7b590 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8105ffb640 10 client.15555 handle_command_reply: tid=0 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 1 client.15555 shutdown 2024-09-11T23:37:14.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 1 -- 172.21.15.133:0/2646161362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 msgr2=0x5649f2a6a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a6a4f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8114005d80 tx=0x7f8114005d10 comp rx=0 tx=0).stop 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 1 -- 172.21.15.133:0/2646161362 shutdown_connections 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5649f2a7b590 0x5649f2a7ca00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a6abb0 0x5649f2a67d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a67840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8105ffb640 0 client.15555 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2646161362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a6a4f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 1 -- 172.21.15.133:0/2646161362 >> 172.21.15.133:0/2646161362 conn(0x5649f2a5bf60 msgr2=0x5649f2a61c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 1 -- 172.21.15.133:0/2646161362 shutdown_connections 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 1 -- 172.21.15.133:0/2646161362 wait complete. 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.604+0000 7f8126f2a740 20 client.15555 trim_cache size 0 max 16384 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.621+0000 7f8126f2a740 1 Processor -- start 2024-09-11T23:37:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f8126f2a740 1 -- start start 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a5f9d0 0x5649f2a56ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a64ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a65020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f8126f2a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5649f2a7ad40 con 0x5649f2a6abb0 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f8126f2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5649f2a7b410 con 0x5649f2a5f9d0 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f8126f2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5649f2a7bae0 con 0x5649f2a602f0 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f811dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a65020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f811dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a65020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60820/0 (socket says 172.21.15.133:60820) 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f811dd9b640 1 -- 172.21.15.133:0/530948126 learned_addr learned my addr 172.21.15.133:0/530948126 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f811e59c640 1 --2- 172.21.15.133:0/530948126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a64ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f811dd9b640 1 --2- 172.21.15.133:0/530948126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a5f9d0 0x5649f2a56ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f811dd9b640 1 -- 172.21.15.133:0/530948126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 msgr2=0x5649f2a64ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f811dd9b640 1 --2- 172.21.15.133:0/530948126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a64ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f811dd9b640 1 -- 172.21.15.133:0/530948126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a5f9d0 msgr2=0x5649f2a56ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:14.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f811dd9b640 1 --2- 172.21.15.133:0/530948126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a5f9d0 0x5649f2a56ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f811dd9b640 1 -- 172.21.15.133:0/530948126 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f2a6d1d0 con 0x5649f2a6abb0 2024-09-11T23:37:14.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.622+0000 7f811dd9b640 1 --2- 172.21.15.133:0/530948126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a65020 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8114013680 tx=0x7f811401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98030f5981f891d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:14.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8107fff640 1 -- 172.21.15.133:0/530948126 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114021070 con 0x5649f2a6abb0 2024-09-11T23:37:14.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8107fff640 1 -- 172.21.15.133:0/530948126 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f811403a040 con 0x5649f2a6abb0 2024-09-11T23:37:14.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8107fff640 1 -- 172.21.15.133:0/530948126 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114039070 con 0x5649f2a6abb0 2024-09-11T23:37:14.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8126f2a740 1 -- 172.21.15.133:0/530948126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 msgr2=0x5649f2a65020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:14.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8126f2a740 1 --2- 172.21.15.133:0/530948126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a65020 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8114013680 tx=0x7f811401e040 comp rx=0 tx=0).stop 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8126f2a740 1 -- 172.21.15.133:0/530948126 shutdown_connections 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8126f2a740 1 --2- 172.21.15.133:0/530948126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a6abb0 0x5649f2a65020 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8126f2a740 1 --2- 172.21.15.133:0/530948126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a64ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8126f2a740 1 --2- 172.21.15.133:0/530948126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a5f9d0 0x5649f2a56ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8126f2a740 1 -- 172.21.15.133:0/530948126 >> 172.21.15.133:0/530948126 conn(0x5649f2a5bf60 msgr2=0x5649f2a7a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8126f2a740 1 -- 172.21.15.133:0/530948126 shutdown_connections 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.623+0000 7f8126f2a740 1 -- 172.21.15.133:0/530948126 wait complete. 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f8126f2a740 1 -- start start 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f8126f2a740 1 Processor -- start 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a63b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a64050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f8126f2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a6abb0 0x5649f2a64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f8126f2a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5649f2a7c880 con 0x5649f2a5f9d0 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f8126f2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5649f2a7c1b0 con 0x5649f2a6abb0 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f8126f2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5649f2a7cf50 con 0x5649f2a602f0 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f8126f2a740 10 client.0 _ll_register_callbacks cb 0x7f811401f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f811dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a63b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f811dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a63b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60824/0 (socket says 172.21.15.133:60824) 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f811dd9b640 1 -- 172.21.15.133:0/2275202616 learned_addr learned my addr 172.21.15.133:0/2275202616 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.624+0000 7f811e59c640 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a64050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f811dd9b640 1 -- 172.21.15.133:0/2275202616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 msgr2=0x5649f2a64050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f811dd9b640 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a64050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f811dd9b640 1 -- 172.21.15.133:0/2275202616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a6abb0 msgr2=0x5649f2a64590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f811dd9b640 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a6abb0 0x5649f2a64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f811dd9b640 1 -- 172.21.15.133:0/2275202616 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f2a83db0 con 0x5649f2a5f9d0 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f811dd9b640 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a63b10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f811400c5f0 tx=0x7f8114011450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff45e846901d44a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:14.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f8105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f8105ffb640 1 -- 172.21.15.133:0/2275202616 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114021040 con 0x5649f2a5f9d0 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f8105ffb640 1 -- 172.21.15.133:0/2275202616 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8114011710 con 0x5649f2a5f9d0 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f8126f2a740 1 -- 172.21.15.133:0/2275202616 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f811403a040 con 0x5649f2a5f9d0 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f8105ffb640 1 -- 172.21.15.133:0/2275202616 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8114053050 con 0x5649f2a5f9d0 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f8126f2a740 10 client.15561 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f8126f2a740 1 -- 172.21.15.133:0/2275202616 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649f2a850b0 con 0x5649f2a5f9d0 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f8105ffb640 1 -- 172.21.15.133:0/2275202616 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f811403a040 con 0x5649f2a5f9d0 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f8105ffb640 1 -- 172.21.15.133:0/2275202616 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8114018040 con 0x5649f2a5f9d0 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.625+0000 7f8126f2a740 10 client.15561 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.626+0000 7f8126f2a740 10 client.15561 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.626+0000 7f8126f2a740 20 client.15561 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.626+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5649f2ae9b90 0x5649f2ae88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.626+0000 7f8126f2a740 4 client.15561 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:37:14.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.626+0000 7f8126f2a740 1 -- 172.21.15.133:0/2275202616 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "status"}) -- 0x5649f29cdb10 con 0x5649f2ae9b90 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.626+0000 7f811e59c640 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5649f2ae9b90 0x5649f2ae88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.626+0000 7f811e59c640 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5649f2ae9b90 0x5649f2ae88a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=65315e9b83b09b7e server_cookie=11ac9cd3f0262510 in_seq=0 out_seq=0 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.626+0000 7f8105ffb640 10 client.15561 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.626+0000 7f8105ffb640 1 -- 172.21.15.133:0/2275202616 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5649f29cdb10 con 0x5649f2ae9b90 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.626+0000 7f8105ffb640 10 client.15561 handle_command_reply: tid=0 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.626+0000 7f8126f2a740 1 client.15561 shutdown 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8126f2a740 1 -- 172.21.15.133:0/2275202616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 msgr2=0x5649f2a63b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a63b10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f811400c5f0 tx=0x7f8114011450 comp rx=0 tx=0).stop 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8126f2a740 1 -- 172.21.15.133:0/2275202616 shutdown_connections 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5649f2ae9b90 0x5649f2ae88a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649f2a6abb0 0x5649f2a64590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f2a602f0 0x5649f2a64050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8126f2a740 1 --2- 172.21.15.133:0/2275202616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649f2a5f9d0 0x5649f2a63b10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8126f2a740 1 -- 172.21.15.133:0/2275202616 >> 172.21.15.133:0/2275202616 conn(0x5649f2a5bf60 msgr2=0x5649f2a56ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:14.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8105ffb640 0 client.15561 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:14.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8126f2a740 1 -- 172.21.15.133:0/2275202616 shutdown_connections 2024-09-11T23:37:14.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8126f2a740 1 -- 172.21.15.133:0/2275202616 wait complete. 2024-09-11T23:37:14.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8126f2a740 20 client.15561 trim_cache size 0 max 16384 2024-09-11T23:37:14.641 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:37:14.641 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:37:14.641 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 3, 2024-09-11T23:37:14.641 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24419, 2024-09-11T23:37:14.641 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:37:14.641 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:37:14.641 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:37:14.642 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 131.18400469700001, 2024-09-11T23:37:14.642 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 28, 2024-09-11T23:37:14.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 82, 2024-09-11T23:37:14.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T23:37:14.642 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 170.054307558 2024-09-11T23:37:14.642 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:37:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 -- 172.21.15.133:0/2327346330 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f81080a2ea0 msgr2=0x7f81080a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f81080a2ea0 0x7f81080a5360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f81180edbb0 tx=0x7f81180a7040 comp rx=0 tx=0).stop 2024-09-11T23:37:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 -- 172.21.15.133:0/2327346330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8120102ef0 msgr2=0x7f812019bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8120102ef0 0x7f812019bcd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f811802ae70 tx=0x7f811802f5d0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 -- 172.21.15.133:0/2327346330 shutdown_connections 2024-09-11T23:37:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f81080a2ea0 0x7f81080a5360 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8120102ef0 0x7f812019bcd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8120102570 0x7f8120197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 --2- 172.21.15.133:0/2327346330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8120100e20 0x7f8120197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 -- 172.21.15.133:0/2327346330 >> 172.21.15.133:0/2327346330 conn(0x7f81200fc7d0 msgr2=0x7f81201068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 -- 172.21.15.133:0/2327346330 shutdown_connections 2024-09-11T23:37:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:14.627+0000 7f8124b51640 1 -- 172.21.15.133:0/2327346330 wait complete. 2024-09-11T23:37:15.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:15 smithi133 ceph-mon[31430]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 KiB/s wr, 51 op/s 2024-09-11T23:37:15.368 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:37:15.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:15 smithi138 ceph-mon[35142]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 KiB/s wr, 51 op/s 2024-09-11T23:37:15.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:15 smithi178 ceph-mon[35882]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 KiB/s wr, 51 op/s 2024-09-11T23:37:15.623 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:16.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.100+0000 7fa9f3d06640 1 -- 172.21.15.133:0/650101555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec10e450 msgr2=0x7fa9ec110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:16.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.100+0000 7fa9f3d06640 1 --2- 172.21.15.133:0/650101555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec10e450 0x7fa9ec110840 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009ca0 tx=0x7fa9e402a950 comp rx=0 tx=0).stop 2024-09-11T23:37:16.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.100+0000 7fa9f3d06640 1 -- 172.21.15.133:0/650101555 shutdown_connections 2024-09-11T23:37:16.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.100+0000 7fa9f3d06640 1 --2- 172.21.15.133:0/650101555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec10e450 0x7fa9ec110840 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.100+0000 7fa9f3d06640 1 --2- 172.21.15.133:0/650101555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec074e30 0x7fa9ec10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.100+0000 7fa9f3d06640 1 --2- 172.21.15.133:0/650101555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9ec074510 0x7fa9ec0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.100+0000 7fa9f3d06640 1 -- 172.21.15.133:0/650101555 >> 172.21.15.133:0/650101555 conn(0x7fa9ec0fd8f0 msgr2=0x7fa9ec0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:16.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f3d06640 1 -- 172.21.15.133:0/650101555 shutdown_connections 2024-09-11T23:37:16.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f3d06640 1 -- 172.21.15.133:0/650101555 wait complete. 2024-09-11T23:37:16.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f3d06640 1 Processor -- start 2024-09-11T23:37:16.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f3d06640 1 -- start start 2024-09-11T23:37:16.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f3d06640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9ec074510 0x7fa9ec19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f3d06640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec074e30 0x7fa9ec19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f3d06640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec10e450 0x7fa9ec19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f3d06640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec114170 con 0x7fa9ec074e30 2024-09-11T23:37:16.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f3d06640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec113ff0 con 0x7fa9ec10e450 2024-09-11T23:37:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f3d06640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec1142f0 con 0x7fa9ec074510 2024-09-11T23:37:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f12a1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec074e30 0x7fa9ec19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f12a1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec074e30 0x7fa9ec19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60840/0 (socket says 172.21.15.133:60840) 2024-09-11T23:37:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.101+0000 7fa9f12a1640 1 -- 172.21.15.133:0/3513347380 learned_addr learned my addr 172.21.15.133:0/3513347380 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9ec074510 0x7fa9ec19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9f22a3640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec10e450 0x7fa9ec19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9f12a1640 1 -- 172.21.15.133:0/3513347380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9ec074510 msgr2=0x7fa9ec19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9f12a1640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9ec074510 0x7fa9ec19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9f12a1640 1 -- 172.21.15.133:0/3513347380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec10e450 msgr2=0x7fa9ec19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9f12a1640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec10e450 0x7fa9ec19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9f12a1640 1 -- 172.21.15.133:0/3513347380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec19f600 con 0x7fa9ec074e30 2024-09-11T23:37:16.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9ec074510 0x7fa9ec19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:16.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9f12a1640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec074e30 0x7fa9ec19ab90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c890 tx=0x7fa9dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:16.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9e2ffd640 1 -- 172.21.15.133:0/3513347380 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc004220 con 0x7fa9ec074e30 2024-09-11T23:37:16.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9e2ffd640 1 -- 172.21.15.133:0/3513347380 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9dc0043c0 con 0x7fa9ec074e30 2024-09-11T23:37:16.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9f3d06640 1 -- 172.21.15.133:0/3513347380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ec19f8f0 con 0x7fa9ec074e30 2024-09-11T23:37:16.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9e2ffd640 1 -- 172.21.15.133:0/3513347380 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0049e0 con 0x7fa9ec074e30 2024-09-11T23:37:16.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.102+0000 7fa9f3d06640 1 -- 172.21.15.133:0/3513347380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ec1a71d0 con 0x7fa9ec074e30 2024-09-11T23:37:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.104+0000 7fa9e2ffd640 1 -- 172.21.15.133:0/3513347380 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fa9dc0079b0 con 0x7fa9ec074e30 2024-09-11T23:37:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f60df740 1 Processor -- start 2024-09-11T23:37:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f60df740 1 -- start start 2024-09-11T23:37:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 0x559f5016edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f50179670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f50179bb0 0x559f5017bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f60df740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559f500d5090 con 0x559f5016e9d0 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f60df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f5007ca70 con 0x559f50179bb0 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f60df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559f500d1050 con 0x559f5016f2f0 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f22a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 0x559f5016edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f22a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 0x559f5016edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60848/0 (socket says 172.21.15.133:60848) 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f22a3640 1 -- 172.21.15.133:0/3130421388 learned_addr learned my addr 172.21.15.133:0/3130421388 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f1aa2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f50179670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9e2ffd640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9d80a2ea0 0x7fa9d80a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.105+0000 7fa9f12a1640 1 --2- 172.21.15.133:0/3130421388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f50179bb0 0x559f5017bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9f22a3640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9d80a2ea0 0x7fa9d80a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9f1aa2640 1 -- 172.21.15.133:0/3130421388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f50179bb0 msgr2=0x559f5017bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/3130421388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f50179bb0 0x559f5017bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9f1aa2640 1 -- 172.21.15.133:0/3130421388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 msgr2=0x559f5016edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/3130421388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 0x559f5016edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9f1aa2640 1 -- 172.21.15.133:0/3130421388 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f50089130 con 0x559f5016f2f0 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9f12a1640 1 --2- 172.21.15.133:0/3130421388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f50179bb0 0x559f5017bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9e2ffd640 1 -- 172.21.15.133:0/3513347380 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa9dc0cf6c0 con 0x7fa9ec074e30 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/3130421388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f50179670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800bc80 tx=0x7fa9e800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8ebb4e0f1b42680 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9f22a3640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9d80a2ea0 0x7fa9d80a5360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4030040 tx=0x7fa9e402a950 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9d7fff640 1 -- 172.21.15.133:0/3130421388 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8015070 con 0x559f5016f2f0 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9d7fff640 1 -- 172.21.15.133:0/3130421388 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e8011920 con 0x559f5016f2f0 2024-09-11T23:37:16.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.106+0000 7fa9d7fff640 1 -- 172.21.15.133:0/3130421388 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80108e0 con 0x559f5016f2f0 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.107+0000 7fa9f60df740 1 -- 172.21.15.133:0/3130421388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 msgr2=0x559f50179670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.107+0000 7fa9f60df740 1 --2- 172.21.15.133:0/3130421388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f50179670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800bc80 tx=0x7fa9e800f730 comp rx=0 tx=0).stop 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.107+0000 7fa9f60df740 1 -- 172.21.15.133:0/3130421388 shutdown_connections 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.107+0000 7fa9f60df740 1 --2- 172.21.15.133:0/3130421388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f50179bb0 0x559f5017bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.107+0000 7fa9f60df740 1 --2- 172.21.15.133:0/3130421388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f50179670 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.107+0000 7fa9f60df740 1 --2- 172.21.15.133:0/3130421388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 0x559f5016edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.107+0000 7fa9f60df740 1 -- 172.21.15.133:0/3130421388 >> 172.21.15.133:0/3130421388 conn(0x559f5016af60 msgr2=0x559f5016b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.107+0000 7fa9f60df740 1 -- 172.21.15.133:0/3130421388 shutdown_connections 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.107+0000 7fa9f60df740 1 -- 172.21.15.133:0/3130421388 wait complete. 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.107+0000 7fa9f60df740 1 -- start start 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f60df740 1 Processor -- start 2024-09-11T23:37:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f5016e9d0 0x559f501793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f50176700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f60df740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559f5017c780 con 0x559f50179bb0 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f60df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f5008ff20 con 0x559f5016e9d0 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f60df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559f5017c900 con 0x559f5016f2f0 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f60df740 10 client.0 _ll_register_callbacks cb 0x559f50059030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f22a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f22a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60864/0 (socket says 172.21.15.133:60864) 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f22a3640 1 -- 172.21.15.133:0/374316231 learned_addr learned my addr 172.21.15.133:0/374316231 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f5016e9d0 0x559f501793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f12a1640 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f50176700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f22a3640 1 -- 172.21.15.133:0/374316231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 msgr2=0x559f50176700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f22a3640 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f50176700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f22a3640 1 -- 172.21.15.133:0/374316231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f5016e9d0 msgr2=0x559f501793b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f22a3640 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f5016e9d0 0x559f501793b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.108+0000 7fa9f22a3640 1 -- 172.21.15.133:0/374316231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f4fe462b0 con 0x559f50179bb0 2024-09-11T23:37:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9f22a3640 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4006010 tx=0x7fa9e40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d0bd8629566bb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9d57fa640 1 -- 172.21.15.133:0/374316231 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4043050 con 0x559f50179bb0 2024-09-11T23:37:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9d57fa640 1 -- 172.21.15.133:0/374316231 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa9e4004510 con 0x559f50179bb0 2024-09-11T23:37:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9f60df740 1 -- 172.21.15.133:0/374316231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e8011920 con 0x559f50179bb0 2024-09-11T23:37:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9d57fa640 1 -- 172.21.15.133:0/374316231 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa9e4034040 con 0x559f50179bb0 2024-09-11T23:37:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9f60df740 10 client.15573 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9f60df740 1 -- 172.21.15.133:0/374316231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f50188010 con 0x559f50179bb0 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9d57fa640 1 -- 172.21.15.133:0/374316231 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e8011920 con 0x559f50179bb0 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9d57fa640 1 -- 172.21.15.133:0/374316231 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa9e405b050 con 0x559f50179bb0 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9f60df740 10 client.15573 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.109+0000 7fa9f60df740 10 client.15573 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.110+0000 7fa9f60df740 20 client.15573 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.110+0000 7fa9f60df740 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559f5018a5f0 0x559f5018ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.110+0000 7fa9f60df740 4 client.15573 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.110+0000 7fa9f60df740 1 -- 172.21.15.133:0/374316231 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f500e2600 con 0x559f5018a5f0 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.110+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559f5018a5f0 0x559f5018ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.110+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559f5018a5f0 0x559f5018ba60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e8af03d6d1a51da server_cookie=d06bab2357b6090a in_seq=0 out_seq=0 2024-09-11T23:37:16.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.110+0000 7fa9d57fa640 10 client.15573 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:16.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9d57fa640 1 -- 172.21.15.133:0/374316231 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f500e2600 con 0x559f5018a5f0 2024-09-11T23:37:16.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9d57fa640 10 client.15573 handle_command_reply: tid=0 2024-09-11T23:37:16.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 1 client.15573 shutdown 2024-09-11T23:37:16.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 1 -- 172.21.15.133:0/374316231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 msgr2=0x559f50176c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:16.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4006010 tx=0x7fa9e40095c0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 1 -- 172.21.15.133:0/374316231 shutdown_connections 2024-09-11T23:37:16.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559f5018a5f0 0x559f5018ba60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9d57fa640 0 client.15573 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:16.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f50176700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 1 --2- 172.21.15.133:0/374316231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f5016e9d0 0x559f501793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 1 -- 172.21.15.133:0/374316231 >> 172.21.15.133:0/374316231 conn(0x559f5016af60 msgr2=0x559f50170c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:16.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 1 -- 172.21.15.133:0/374316231 shutdown_connections 2024-09-11T23:37:16.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 1 -- 172.21.15.133:0/374316231 wait complete. 2024-09-11T23:37:16.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.111+0000 7fa9f60df740 20 client.15573 trim_cache size 0 max 16384 2024-09-11T23:37:16.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f60df740 1 Processor -- start 2024-09-11T23:37:16.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f60df740 1 -- start start 2024-09-11T23:37:16.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016e9d0 0x559f5018ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f5016f2f0 0x559f501766c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f60df740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559f50189da0 con 0x559f50179bb0 2024-09-11T23:37:16.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f60df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f5018a470 con 0x559f5016f2f0 2024-09-11T23:37:16.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f60df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559f5018ab40 con 0x559f5016e9d0 2024-09-11T23:37:16.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f22a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f22a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60868/0 (socket says 172.21.15.133:60868) 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f22a3640 1 -- 172.21.15.133:0/1228318059 learned_addr learned my addr 172.21.15.133:0/1228318059 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/1228318059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016e9d0 0x559f5018ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f12a1640 1 --2- 172.21.15.133:0/1228318059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f5016f2f0 0x559f501766c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f22a3640 1 -- 172.21.15.133:0/1228318059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016e9d0 msgr2=0x559f5018ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f22a3640 1 --2- 172.21.15.133:0/1228318059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016e9d0 0x559f5018ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f22a3640 1 -- 172.21.15.133:0/1228318059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f5016f2f0 msgr2=0x559f501766c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f22a3640 1 --2- 172.21.15.133:0/1228318059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f5016f2f0 0x559f501766c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f22a3640 1 -- 172.21.15.133:0/1228318059 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5016ad90 con 0x559f50179bb0 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9f22a3640 1 --2- 172.21.15.133:0/1228318059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa9e402ae70 tx=0x7fa9e4005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28218712c1d02f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.129+0000 7fa9d77fe640 1 -- 172.21.15.133:0/1228318059 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4043070 con 0x559f50179bb0 2024-09-11T23:37:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9d77fe640 1 -- 172.21.15.133:0/1228318059 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e404f070 con 0x559f50179bb0 2024-09-11T23:37:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9d77fe640 1 -- 172.21.15.133:0/1228318059 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e404b6a0 con 0x559f50179bb0 2024-09-11T23:37:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9f60df740 1 -- 172.21.15.133:0/1228318059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 msgr2=0x559f50176c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9f60df740 1 --2- 172.21.15.133:0/1228318059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa9e402ae70 tx=0x7fa9e4005cc0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9f60df740 1 -- 172.21.15.133:0/1228318059 shutdown_connections 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9f60df740 1 --2- 172.21.15.133:0/1228318059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f50179bb0 0x559f50176c00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9f60df740 1 --2- 172.21.15.133:0/1228318059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f5016f2f0 0x559f501766c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9f60df740 1 --2- 172.21.15.133:0/1228318059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016e9d0 0x559f5018ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9f60df740 1 -- 172.21.15.133:0/1228318059 >> 172.21.15.133:0/1228318059 conn(0x559f5016af60 msgr2=0x559f50175c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9f60df740 1 -- 172.21.15.133:0/1228318059 shutdown_connections 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9f60df740 1 -- 172.21.15.133:0/1228318059 wait complete. 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.130+0000 7fa9f60df740 1 -- start start 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f60df740 1 Processor -- start 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 0x559f5018f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f5018f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f60df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f50179bb0 0x559f501f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f60df740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559f5018b8e0 con 0x559f5016e9d0 2024-09-11T23:37:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f60df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f5018b210 con 0x559f50179bb0 2024-09-11T23:37:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f60df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559f5018bfb0 con 0x559f5016f2f0 2024-09-11T23:37:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f60df740 10 client.0 _ll_register_callbacks cb 0x7fa9e402f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f1aa2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 0x559f5018f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f12a1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f5018f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f1aa2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 0x559f5018f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60882/0 (socket says 172.21.15.133:60882) 2024-09-11T23:37:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f1aa2640 1 -- 172.21.15.133:0/322548454 learned_addr learned my addr 172.21.15.133:0/322548454 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f22a3640 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f50179bb0 0x559f501f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f1aa2640 1 -- 172.21.15.133:0/322548454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 msgr2=0x559f5018f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f5018f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f1aa2640 1 -- 172.21.15.133:0/322548454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f50179bb0 msgr2=0x559f501f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f50179bb0 0x559f501f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.131+0000 7fa9f1aa2640 1 -- 172.21.15.133:0/322548454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f501f3230 con 0x559f5016e9d0 2024-09-11T23:37:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9f1aa2640 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 0x559f5018f180 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80058c0 tx=0x7fa9e80104e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95523940bd133480 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9d57fa640 1 -- 172.21.15.133:0/322548454 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8015040 con 0x559f5016e9d0 2024-09-11T23:37:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9d57fa640 1 -- 172.21.15.133:0/322548454 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa9e8005450 con 0x559f5016e9d0 2024-09-11T23:37:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9f60df740 1 -- 172.21.15.133:0/322548454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e404f070 con 0x559f5016e9d0 2024-09-11T23:37:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9d57fa640 1 -- 172.21.15.133:0/322548454 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa9e8005650 con 0x559f5016e9d0 2024-09-11T23:37:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9f60df740 10 client.15579 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9f60df740 1 -- 172.21.15.133:0/322548454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f501f44e0 con 0x559f5016e9d0 2024-09-11T23:37:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9d57fa640 1 -- 172.21.15.133:0/322548454 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e404f070 con 0x559f5016e9d0 2024-09-11T23:37:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9d57fa640 1 -- 172.21.15.133:0/322548454 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa9e8037020 con 0x559f5016e9d0 2024-09-11T23:37:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9f60df740 10 client.15579 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9f60df740 10 client.15579 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:37:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9f60df740 20 client.15579 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9f60df740 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559f501f8d20 0x559f501f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9f60df740 4 client.15579 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:37:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.132+0000 7fa9f60df740 1 -- 172.21.15.133:0/322548454 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "status"}) -- 0x559f500dcb10 con 0x559f501f8d20 2024-09-11T23:37:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f12a1640 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559f501f8d20 0x559f501f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f12a1640 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559f501f8d20 0x559f501f7a30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b35f30e38fa7452 server_cookie=69b29f117509a439 in_seq=0 out_seq=0 2024-09-11T23:37:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9d57fa640 10 client.15579 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9d57fa640 1 -- 172.21.15.133:0/322548454 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x559f500dcb10 con 0x559f501f8d20 2024-09-11T23:37:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9d57fa640 10 client.15579 handle_command_reply: tid=0 2024-09-11T23:37:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f60df740 1 client.15579 shutdown 2024-09-11T23:37:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f60df740 1 -- 172.21.15.133:0/322548454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 msgr2=0x559f5018f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f60df740 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 0x559f5018f180 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80058c0 tx=0x7fa9e80104e0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f60df740 1 -- 172.21.15.133:0/322548454 shutdown_connections 2024-09-11T23:37:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f60df740 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x559f501f8d20 0x559f501f7a30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f60df740 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f50179bb0 0x559f501f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9d57fa640 0 client.15579 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f60df740 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f5016f2f0 0x559f5018f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f60df740 1 --2- 172.21.15.133:0/322548454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f5016e9d0 0x559f5018f180 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f60df740 1 -- 172.21.15.133:0/322548454 >> 172.21.15.133:0/322548454 conn(0x559f5016af60 msgr2=0x559f501766c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:16.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f60df740 1 -- 172.21.15.133:0/322548454 shutdown_connections 2024-09-11T23:37:16.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.133+0000 7fa9f60df740 1 -- 172.21.15.133:0/322548454 wait complete. 2024-09-11T23:37:16.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f60df740 20 client.15579 trim_cache size 0 max 16384 2024-09-11T23:37:16.144 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:37:16.144 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:37:16.144 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 2, 2024-09-11T23:37:16.144 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24464, 2024-09-11T23:37:16.144 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:37:16.144 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:37:16.144 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:37:16.144 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 133.649927069, 2024-09-11T23:37:16.145 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 28, 2024-09-11T23:37:16.145 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 82, 2024-09-11T23:37:16.145 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T23:37:16.145 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 153.118981039 2024-09-11T23:37:16.145 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:37:16.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 -- 172.21.15.133:0/3513347380 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9d80a2ea0 msgr2=0x7fa9d80a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:16.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9d80a2ea0 0x7fa9d80a5360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4030040 tx=0x7fa9e402a950 comp rx=0 tx=0).stop 2024-09-11T23:37:16.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 -- 172.21.15.133:0/3513347380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec074e30 msgr2=0x7fa9ec19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:16.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec074e30 0x7fa9ec19ab90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c890 tx=0x7fa9dc00cd50 comp rx=0 tx=0).stop 2024-09-11T23:37:16.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 -- 172.21.15.133:0/3513347380 shutdown_connections 2024-09-11T23:37:16.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9d80a2ea0 0x7fa9d80a5360 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec10e450 0x7fa9ec19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec074e30 0x7fa9ec19ab90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 --2- 172.21.15.133:0/3513347380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9ec074510 0x7fa9ec19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 -- 172.21.15.133:0/3513347380 >> 172.21.15.133:0/3513347380 conn(0x7fa9ec0fd8f0 msgr2=0x7fa9ec0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 -- 172.21.15.133:0/3513347380 shutdown_connections 2024-09-11T23:37:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:16.134+0000 7fa9f3d06640 1 -- 172.21.15.133:0/3513347380 wait complete. 2024-09-11T23:37:16.868 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T23:37:17.115 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:17.390 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:17 smithi133 ceph-mon[31430]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2024-09-11T23:37:17.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:17 smithi138 ceph-mon[35142]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2024-09-11T23:37:17.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:17 smithi178 ceph-mon[35882]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2024-09-11T23:37:17.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.624+0000 7f0422e3f640 1 -- 172.21.15.133:0/3113817273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c102ef0 msgr2=0x7f041c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:17.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.624+0000 7f0422e3f640 1 --2- 172.21.15.133:0/3113817273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c102ef0 0x7f041c10da40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0414009900 tx=0x7f041402a480 comp rx=0 tx=0).stop 2024-09-11T23:37:17.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.624+0000 7f0422e3f640 1 -- 172.21.15.133:0/3113817273 shutdown_connections 2024-09-11T23:37:17.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.624+0000 7f0422e3f640 1 --2- 172.21.15.133:0/3113817273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c102ef0 0x7f041c10da40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.624+0000 7f0422e3f640 1 --2- 172.21.15.133:0/3113817273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c102570 0x7f041c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.624+0000 7f0422e3f640 1 --2- 172.21.15.133:0/3113817273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c100e20 0x7f041c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.624+0000 7f0422e3f640 1 -- 172.21.15.133:0/3113817273 >> 172.21.15.133:0/3113817273 conn(0x7f041c0fc7d0 msgr2=0x7f041c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.624+0000 7f0422e3f640 1 -- 172.21.15.133:0/3113817273 shutdown_connections 2024-09-11T23:37:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.624+0000 7f0422e3f640 1 -- 172.21.15.133:0/3113817273 wait complete. 2024-09-11T23:37:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.624+0000 7f0422e3f640 1 Processor -- start 2024-09-11T23:37:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.624+0000 7f0422e3f640 1 -- start start 2024-09-11T23:37:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0422e3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c100e20 0x7f041c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0422e3f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102570 0x7f041c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0422e3f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c102ef0 0x7f041c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0422e3f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1120a0 con 0x7f041c100e20 2024-09-11T23:37:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0422e3f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f041c111f20 con 0x7f041c102570 2024-09-11T23:37:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0422e3f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f041c112220 con 0x7f041c102ef0 2024-09-11T23:37:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0420bdb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c100e20 0x7f041c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0420bdb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c100e20 0x7f041c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60904/0 (socket says 172.21.15.133:60904) 2024-09-11T23:37:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0420bdb640 1 -- 172.21.15.133:0/820648656 learned_addr learned my addr 172.21.15.133:0/820648656 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f04213dc640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c102ef0 0x7f041c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f041bfff640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102570 0x7f041c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0420bdb640 1 -- 172.21.15.133:0/820648656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c102ef0 msgr2=0x7f041c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0420bdb640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c102ef0 0x7f041c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0420bdb640 1 -- 172.21.15.133:0/820648656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102570 msgr2=0x7f041c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0420bdb640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102570 0x7f041c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0420bdb640 1 -- 172.21.15.133:0/820648656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c19d3d0 con 0x7f041c100e20 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f04213dc640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c102ef0 0x7f041c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f041bfff640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102570 0x7f041c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0420bdb640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c100e20 0x7f041c198420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c890 tx=0x7f040c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.625+0000 7f0419ffb640 1 -- 172.21.15.133:0/820648656 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c004220 con 0x7f041c100e20 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.626+0000 7f0419ffb640 1 -- 172.21.15.133:0/820648656 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f040c0043c0 con 0x7f041c100e20 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.626+0000 7f0422e3f640 1 -- 172.21.15.133:0/820648656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c19d6c0 con 0x7f041c100e20 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.626+0000 7f0419ffb640 1 -- 172.21.15.133:0/820648656 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0049e0 con 0x7f041c100e20 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.626+0000 7f0422e3f640 1 -- 172.21.15.133:0/820648656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c1a4fa0 con 0x7f041c100e20 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f0419ffb640 1 -- 172.21.15.133:0/820648656 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f040c0079b0 con 0x7f041c100e20 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f0425218740 1 Processor -- start 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f0425218740 1 -- start start 2024-09-11T23:37:17.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f0425218740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0ed4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f0425218740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0edf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f0425218740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0ee1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f0425218740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557af0e3b090 con 0x557af0edfbb0 2024-09-11T23:37:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f0425218740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557af0de2a70 con 0x557af0ed52f0 2024-09-11T23:37:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f0425218740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557af0e37050 con 0x557af0ed49d0 2024-09-11T23:37:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f0419ffb640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04040a2ea0 0x7f04040a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f041bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0ee1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f041bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0ee1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60906/0 (socket says 172.21.15.133:60906) 2024-09-11T23:37:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f041bfff640 1 -- 172.21.15.133:0/2345505462 learned_addr learned my addr 172.21.15.133:0/2345505462 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f0420bdb640 1 --2- 172.21.15.133:0/2345505462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0edf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f04213dc640 1 --2- 172.21.15.133:0/2345505462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0ed4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.629+0000 7f04213dc640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04040a2ea0 0x7f04040a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.630+0000 7f0419ffb640 1 -- 172.21.15.133:0/820648656 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f040c0cf8b0 con 0x7f041c100e20 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.630+0000 7f041bfff640 1 -- 172.21.15.133:0/2345505462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 msgr2=0x557af0ed4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.630+0000 7f041bfff640 1 --2- 172.21.15.133:0/2345505462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0ed4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.630+0000 7f041bfff640 1 -- 172.21.15.133:0/2345505462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 msgr2=0x557af0edf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.630+0000 7f041bfff640 1 --2- 172.21.15.133:0/2345505462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0edf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.630+0000 7f041bfff640 1 -- 172.21.15.133:0/2345505462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557af0def130 con 0x557af0edfbb0 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.630+0000 7f041bfff640 1 --2- 172.21.15.133:0/2345505462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0ee1fa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0410008ee0 tx=0x7f041000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f079713ca81079ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.630+0000 7f04213dc640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04040a2ea0 0x7f04040a5360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f041402a9a0 tx=0x7f0414005fd0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.630+0000 7f0402ffd640 1 -- 172.21.15.133:0/2345505462 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041000f830 con 0x557af0edfbb0 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.630+0000 7f0402ffd640 1 -- 172.21.15.133:0/2345505462 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0410018480 con 0x557af0edfbb0 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.630+0000 7f0402ffd640 1 -- 172.21.15.133:0/2345505462 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041000e5f0 con 0x557af0edfbb0 2024-09-11T23:37:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.631+0000 7f0425218740 1 -- 172.21.15.133:0/2345505462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 msgr2=0x557af0ee1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.631+0000 7f0425218740 1 --2- 172.21.15.133:0/2345505462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0ee1fa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0410008ee0 tx=0x7f041000d710 comp rx=0 tx=0).stop 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.631+0000 7f0425218740 1 -- 172.21.15.133:0/2345505462 shutdown_connections 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.631+0000 7f0425218740 1 --2- 172.21.15.133:0/2345505462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0ee1fa0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.631+0000 7f0425218740 1 --2- 172.21.15.133:0/2345505462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0edf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.631+0000 7f0425218740 1 --2- 172.21.15.133:0/2345505462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0ed4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.631+0000 7f0425218740 1 -- 172.21.15.133:0/2345505462 >> 172.21.15.133:0/2345505462 conn(0x557af0ed0f60 msgr2=0x557af0ed1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.631+0000 7f0425218740 1 -- 172.21.15.133:0/2345505462 shutdown_connections 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.631+0000 7f0425218740 1 -- 172.21.15.133:0/2345505462 wait complete. 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.631+0000 7f0425218740 1 -- start start 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.631+0000 7f0425218740 1 Processor -- start 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f0425218740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0edf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f0425218740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0edc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f0425218740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f0425218740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557af0ee2780 con 0x557af0edfbb0 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f0425218740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557af0df5f20 con 0x557af0ed52f0 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f0425218740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557af0ee2900 con 0x557af0ed49d0 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f0425218740 10 client.0 _ll_register_callbacks cb 0x557af0dbf030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f04213dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f04213dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60920/0 (socket says 172.21.15.133:60920) 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f04213dc640 1 -- 172.21.15.133:0/124348532 learned_addr learned my addr 172.21.15.133:0/124348532 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f041bfff640 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0edc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f0420bdb640 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0edf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f04213dc640 1 -- 172.21.15.133:0/124348532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 msgr2=0x557af0edf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f04213dc640 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0edf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f04213dc640 1 -- 172.21.15.133:0/124348532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 msgr2=0x557af0edc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f04213dc640 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0edc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f04213dc640 1 -- 172.21.15.133:0/124348532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557af0bac2b0 con 0x557af0edfbb0 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f04213dc640 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edcc90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0414003f00 tx=0x7f0414009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8d462be1c08349 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f03fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f03fbfff640 1 -- 172.21.15.133:0/124348532 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414035040 con 0x557af0edfbb0 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f0425218740 1 -- 172.21.15.133:0/124348532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0410018480 con 0x557af0edfbb0 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.632+0000 7f03fbfff640 1 -- 172.21.15.133:0/124348532 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0414031dd0 con 0x557af0edfbb0 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f03fbfff640 1 -- 172.21.15.133:0/124348532 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0414059050 con 0x557af0edfbb0 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f0425218740 10 client.15591 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f0425218740 1 -- 172.21.15.133:0/124348532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557af0eee050 con 0x557af0edfbb0 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f03fbfff640 1 -- 172.21.15.133:0/124348532 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0410018480 con 0x557af0edfbb0 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f03fbfff640 1 -- 172.21.15.133:0/124348532 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f041403a030 con 0x557af0edfbb0 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f0425218740 10 client.15591 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f0425218740 10 client.15591 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f0425218740 20 client.15591 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f0425218740 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557af0ef0700 0x557af0ef1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f0425218740 4 client.15591 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f0425218740 1 -- 172.21.15.133:0/124348532 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557af0e48600 con 0x557af0ef0700 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.633+0000 7f0420bdb640 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557af0ef0700 0x557af0ef1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.634+0000 7f0420bdb640 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557af0ef0700 0x557af0ef1b70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5905dedfbcf4bf0 server_cookie=885b8c0c4322f6e1 in_seq=0 out_seq=0 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.634+0000 7f03fbfff640 10 client.15591 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f03fbfff640 1 -- 172.21.15.133:0/124348532 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557af0e48600 con 0x557af0ef0700 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f03fbfff640 10 client.15591 handle_command_reply: tid=0 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 1 client.15591 shutdown 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 1 -- 172.21.15.133:0/124348532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 msgr2=0x557af0edcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edcc90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0414003f00 tx=0x7f0414009260 comp rx=0 tx=0).stop 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 1 -- 172.21.15.133:0/124348532 shutdown_connections 2024-09-11T23:37:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557af0ef0700 0x557af0ef1b70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edcc90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0edc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f03fbfff640 0 client.15591 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 1 --2- 172.21.15.133:0/124348532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0edf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 1 -- 172.21.15.133:0/124348532 >> 172.21.15.133:0/124348532 conn(0x557af0ed0f60 msgr2=0x557af0ed6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 1 -- 172.21.15.133:0/124348532 shutdown_connections 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 1 -- 172.21.15.133:0/124348532 wait complete. 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.635+0000 7f0425218740 20 client.15591 trim_cache size 0 max 16384 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.653+0000 7f0425218740 1 Processor -- start 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.653+0000 7f0425218740 1 -- start start 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.653+0000 7f0425218740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed49d0 0x557af0edbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.653+0000 7f0425218740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed52f0 0x557af0edb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.653+0000 7f0425218740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.653+0000 7f0425218740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557af0eefeb0 con 0x557af0edfbb0 2024-09-11T23:37:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.653+0000 7f0425218740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557af0ef0580 con 0x557af0ed49d0 2024-09-11T23:37:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.653+0000 7f0425218740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557af0ef0c50 con 0x557af0ed52f0 2024-09-11T23:37:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.653+0000 7f04213dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f041bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed52f0 0x557af0edb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f04213dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60928/0 (socket says 172.21.15.133:60928) 2024-09-11T23:37:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f04213dc640 1 -- 172.21.15.133:0/1783987849 learned_addr learned my addr 172.21.15.133:0/1783987849 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f0420bdb640 1 --2- 172.21.15.133:0/1783987849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed49d0 0x557af0edbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f04213dc640 1 -- 172.21.15.133:0/1783987849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed52f0 msgr2=0x557af0edb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f04213dc640 1 --2- 172.21.15.133:0/1783987849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed52f0 0x557af0edb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f04213dc640 1 -- 172.21.15.133:0/1783987849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed49d0 msgr2=0x557af0edbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f04213dc640 1 --2- 172.21.15.133:0/1783987849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed49d0 0x557af0edbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f04213dc640 1 -- 172.21.15.133:0/1783987849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557af0ed20f0 con 0x557af0edfbb0 2024-09-11T23:37:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f0420bdb640 1 --2- 172.21.15.133:0/1783987849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed49d0 0x557af0edbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f04213dc640 1 --2- 172.21.15.133:0/1783987849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edb560 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0414003ed0 tx=0x7f0414009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c6171d13193da4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f04027fc640 1 -- 172.21.15.133:0/1783987849 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414062070 con 0x557af0edfbb0 2024-09-11T23:37:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f04027fc640 1 -- 172.21.15.133:0/1783987849 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0414050070 con 0x557af0edfbb0 2024-09-11T23:37:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.654+0000 7f04027fc640 1 -- 172.21.15.133:0/1783987849 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414035040 con 0x557af0edfbb0 2024-09-11T23:37:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.655+0000 7f0425218740 1 -- 172.21.15.133:0/1783987849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 msgr2=0x557af0edb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.655+0000 7f0425218740 1 --2- 172.21.15.133:0/1783987849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edb560 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0414003ed0 tx=0x7f0414009520 comp rx=0 tx=0).stop 2024-09-11T23:37:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.655+0000 7f0425218740 1 -- 172.21.15.133:0/1783987849 shutdown_connections 2024-09-11T23:37:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.655+0000 7f0425218740 1 --2- 172.21.15.133:0/1783987849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0edb560 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.655+0000 7f0425218740 1 --2- 172.21.15.133:0/1783987849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed52f0 0x557af0edb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.655+0000 7f0425218740 1 --2- 172.21.15.133:0/1783987849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed49d0 0x557af0edbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.655+0000 7f0425218740 1 -- 172.21.15.133:0/1783987849 >> 172.21.15.133:0/1783987849 conn(0x557af0ed0f60 msgr2=0x557af0ed1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.655+0000 7f0425218740 1 -- 172.21.15.133:0/1783987849 shutdown_connections 2024-09-11T23:37:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.655+0000 7f0425218740 1 -- 172.21.15.133:0/1783987849 wait complete. 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f0425218740 1 -- start start 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f0425218740 1 Processor -- start 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f0425218740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0ef5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f0425218740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0ef58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f0425218740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0f58e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f0425218740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557af0ef19f0 con 0x557af0edfbb0 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f0425218740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557af0ef1320 con 0x557af0ed52f0 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f0425218740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557af0ef20c0 con 0x557af0ed49d0 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f0425218740 10 client.0 _ll_register_callbacks cb 0x7f0414003b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f04213dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0f58e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f04213dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0f58e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60938/0 (socket says 172.21.15.133:60938) 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f04213dc640 1 -- 172.21.15.133:0/2510093398 learned_addr learned my addr 172.21.15.133:0/2510093398 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f041bfff640 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0ef58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.656+0000 7f0420bdb640 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0ef5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f04213dc640 1 -- 172.21.15.133:0/2510093398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 msgr2=0x557af0ef5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f04213dc640 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0ef5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f04213dc640 1 -- 172.21.15.133:0/2510093398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 msgr2=0x557af0ef58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f04213dc640 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0ef58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f04213dc640 1 -- 172.21.15.133:0/2510093398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557af0f593f0 con 0x557af0edfbb0 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f041bfff640 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0ef58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f04213dc640 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0f58e00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0414005110 tx=0x7f0414031bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45be6f2c1bf0f4db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f03fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f03fbfff640 1 -- 172.21.15.133:0/2510093398 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414062070 con 0x557af0edfbb0 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f03fbfff640 1 -- 172.21.15.133:0/2510093398 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f041403e850 con 0x557af0edfbb0 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f0425218740 1 -- 172.21.15.133:0/2510093398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0414050070 con 0x557af0edfbb0 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f03fbfff640 1 -- 172.21.15.133:0/2510093398 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0414035070 con 0x557af0edfbb0 2024-09-11T23:37:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f0425218740 10 client.15597 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f0425218740 1 -- 172.21.15.133:0/2510093398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557af0f5a6b0 con 0x557af0edfbb0 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.657+0000 7f03fbfff640 1 -- 172.21.15.133:0/2510093398 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0414050070 con 0x557af0edfbb0 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.658+0000 7f03fbfff640 1 -- 172.21.15.133:0/2510093398 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0414004d00 con 0x557af0edfbb0 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.658+0000 7f0425218740 10 client.15597 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.658+0000 7f0425218740 10 client.15597 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.658+0000 7f0425218740 20 client.15597 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.658+0000 7f0425218740 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557af0f5eff0 0x557af0f5dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.658+0000 7f0425218740 4 client.15597 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.658+0000 7f0425218740 1 -- 172.21.15.133:0/2510093398 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "status"}) -- 0x557af0e42b10 con 0x557af0f5eff0 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.658+0000 7f0420bdb640 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557af0f5eff0 0x557af0f5dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0420bdb640 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557af0f5eff0 0x557af0f5dd00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27263494ddf3dbe1 server_cookie=e5394d499520b102 in_seq=0 out_seq=0 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f03fbfff640 10 client.15597 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f03fbfff640 1 -- 172.21.15.133:0/2510093398 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557af0e42b10 con 0x557af0f5eff0 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f03fbfff640 10 client.15597 handle_command_reply: tid=0 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 1 client.15597 shutdown 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 1 -- 172.21.15.133:0/2510093398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 msgr2=0x557af0f58e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0f58e00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0414005110 tx=0x7f0414031bd0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 1 -- 172.21.15.133:0/2510093398 shutdown_connections 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557af0f5eff0 0x557af0f5dd00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557af0edfbb0 0x557af0f58e00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557af0ed52f0 0x557af0ef58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 1 --2- 172.21.15.133:0/2510093398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557af0ed49d0 0x557af0ef5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f03fbfff640 0 client.15597 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 1 -- 172.21.15.133:0/2510093398 >> 172.21.15.133:0/2510093398 conn(0x557af0ed0f60 msgr2=0x557af0edaf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 1 -- 172.21.15.133:0/2510093398 shutdown_connections 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 1 -- 172.21.15.133:0/2510093398 wait complete. 2024-09-11T23:37:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.659+0000 7f0425218740 20 client.15597 trim_cache size 0 max 16384 2024-09-11T23:37:17.675 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:37:17.675 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:37:17.675 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 1, 2024-09-11T23:37:17.675 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24472, 2024-09-11T23:37:17.675 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:37:17.675 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:37:17.675 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:37:17.675 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 136.190142272, 2024-09-11T23:37:17.675 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 28, 2024-09-11T23:37:17.675 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 82, 2024-09-11T23:37:17.675 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T23:37:17.676 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 167.947348275 2024-09-11T23:37:17.676 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:37:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 -- 172.21.15.133:0/820648656 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04040a2ea0 msgr2=0x7f04040a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04040a2ea0 0x7f04040a5360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f041402a9a0 tx=0x7f0414005fd0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 -- 172.21.15.133:0/820648656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c100e20 msgr2=0x7f041c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c100e20 0x7f041c198420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c890 tx=0x7f040c00cd50 comp rx=0 tx=0).stop 2024-09-11T23:37:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 -- 172.21.15.133:0/820648656 shutdown_connections 2024-09-11T23:37:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04040a2ea0 0x7f04040a5360 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c102ef0 0x7f041c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102570 0x7f041c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 --2- 172.21.15.133:0/820648656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c100e20 0x7f041c198420 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 -- 172.21.15.133:0/820648656 >> 172.21.15.133:0/820648656 conn(0x7f041c0fc7d0 msgr2=0x7f041c1067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 -- 172.21.15.133:0/820648656 shutdown_connections 2024-09-11T23:37:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:17.660+0000 7f0422e3f640 1 -- 172.21.15.133:0/820648656 wait complete. 2024-09-11T23:37:18.374 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:37:18.626 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.090+0000 7fb2d7fff640 1 -- 172.21.15.133:0/3881075249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d810e260 msgr2=0x7fb2d8110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.090+0000 7fb2d7fff640 1 --2- 172.21.15.133:0/3881075249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d810e260 0x7fb2d8110650 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009ca0 tx=0x7fb2d002a950 comp rx=0 tx=0).stop 2024-09-11T23:37:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.090+0000 7fb2d7fff640 1 -- 172.21.15.133:0/3881075249 shutdown_connections 2024-09-11T23:37:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.090+0000 7fb2d7fff640 1 --2- 172.21.15.133:0/3881075249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d810e260 0x7fb2d8110650 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.090+0000 7fb2d7fff640 1 --2- 172.21.15.133:0/3881075249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2d8074e30 0x7fb2d810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.090+0000 7fb2d7fff640 1 --2- 172.21.15.133:0/3881075249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2d8074510 0x7fb2d80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.090+0000 7fb2d7fff640 1 -- 172.21.15.133:0/3881075249 >> 172.21.15.133:0/3881075249 conn(0x7fb2d80fd680 msgr2=0x7fb2d80ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.090+0000 7fb2d7fff640 1 -- 172.21.15.133:0/3881075249 shutdown_connections 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.090+0000 7fb2d7fff640 1 -- 172.21.15.133:0/3881075249 wait complete. 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.091+0000 7fb2d7fff640 1 Processor -- start 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.091+0000 7fb2d7fff640 1 -- start start 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.091+0000 7fb2d7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d8074510 0x7fb2d819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.091+0000 7fb2d7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2d8074e30 0x7fb2d819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.091+0000 7fb2d7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2d810e260 0x7fb2d819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.091+0000 7fb2d7fff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8113f90 con 0x7fb2d8074510 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.091+0000 7fb2d7fff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8113e10 con 0x7fb2d810e260 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.091+0000 7fb2d7fff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8114110 con 0x7fb2d8074e30 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d8074510 0x7fb2d819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d8074510 0x7fb2d819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60942/0 (socket says 172.21.15.133:60942) 2024-09-11T23:37:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d6ffd640 1 -- 172.21.15.133:0/2739282437 learned_addr learned my addr 172.21.15.133:0/2739282437 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d77fe640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2d810e260 0x7fb2d819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d67fc640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2d8074e30 0x7fb2d819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d6ffd640 1 -- 172.21.15.133:0/2739282437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2d8074e30 msgr2=0x7fb2d819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2d8074e30 0x7fb2d819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d6ffd640 1 -- 172.21.15.133:0/2739282437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2d810e260 msgr2=0x7fb2d819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2d810e260 0x7fb2d819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d6ffd640 1 -- 172.21.15.133:0/2739282437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d819f3f0 con 0x7fb2d8074510 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d8074510 0x7fb2d819a440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800d770 tx=0x7fb2c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2bffff640 1 -- 172.21.15.133:0/2739282437 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8017070 con 0x7fb2d8074510 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d7fff640 1 -- 172.21.15.133:0/2739282437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d819f6e0 con 0x7fb2d8074510 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2bffff640 1 -- 172.21.15.133:0/2739282437 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c8005500 con 0x7fb2d8074510 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2bffff640 1 -- 172.21.15.133:0/2739282437 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8004e00 con 0x7fb2d8074510 2024-09-11T23:37:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.092+0000 7fb2d7fff640 1 -- 172.21.15.133:0/2739282437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d81a6fc0 con 0x7fb2d8074510 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2bffff640 1 -- 172.21.15.133:0/2739282437 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fb2c8004fa0 con 0x7fb2d8074510 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2dfb0a740 1 Processor -- start 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2dfb0a740 1 -- start start 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dcbffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcc04f0 0x55723dcca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dccd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55723dc26090 con 0x55723dcbfbd0 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55723dbcda70 con 0x55723dcc04f0 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55723dc22050 con 0x55723dccadb0 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dcbffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dcbffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60958/0 (socket says 172.21.15.133:60958) 2024-09-11T23:37:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2d77fe640 1 -- 172.21.15.133:0/240666486 learned_addr learned my addr 172.21.15.133:0/240666486 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:19.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2d67fc640 1 --2- 172.21.15.133:0/240666486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dccd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/240666486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcc04f0 0x55723dcca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2bffff640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2c00a2ea0 0x7fb2c00a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2d77fe640 1 -- 172.21.15.133:0/240666486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 msgr2=0x55723dccd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2d77fe640 1 --2- 172.21.15.133:0/240666486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dccd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2d77fe640 1 -- 172.21.15.133:0/240666486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcc04f0 msgr2=0x55723dcca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2d77fe640 1 --2- 172.21.15.133:0/240666486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcc04f0 0x55723dcca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.095+0000 7fb2d77fe640 1 -- 172.21.15.133:0/240666486 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55723dbda130 con 0x55723dcbfbd0 2024-09-11T23:37:19.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.096+0000 7fb2bffff640 1 -- 172.21.15.133:0/2739282437 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fb2c80cf050 con 0x7fb2d8074510 2024-09-11T23:37:19.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.096+0000 7fb2d77fe640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2c00a2ea0 0x7fb2c00a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.096+0000 7fb2d77fe640 1 --2- 172.21.15.133:0/240666486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dcbffb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80754f0 tx=0x7fb2d00098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eafd7c5da411f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:19.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.096+0000 7fb2bd7fa640 1 -- 172.21.15.133:0/240666486 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0034040 con 0x55723dcbfbd0 2024-09-11T23:37:19.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.096+0000 7fb2bd7fa640 1 -- 172.21.15.133:0/240666486 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d0003b30 con 0x55723dcbfbd0 2024-09-11T23:37:19.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.096+0000 7fb2bd7fa640 1 -- 172.21.15.133:0/240666486 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0030640 con 0x55723dcbfbd0 2024-09-11T23:37:19.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/240666486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 msgr2=0x55723dcbffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/240666486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dcbffb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80754f0 tx=0x7fb2d00098d0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2d77fe640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2c00a2ea0 0x7fb2c00a5360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2d004a000 tx=0x7fb2d004d000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:19.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/240666486 shutdown_connections 2024-09-11T23:37:19.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/240666486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dccd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/240666486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcc04f0 0x55723dcca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/240666486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dcbffb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/240666486 >> 172.21.15.133:0/240666486 conn(0x55723dcbc160 msgr2=0x55723dcbc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:19.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/240666486 shutdown_connections 2024-09-11T23:37:19.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/240666486 wait complete. 2024-09-11T23:37:19.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2dfb0a740 1 -- start start 2024-09-11T23:37:19.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.097+0000 7fb2dfb0a740 1 Processor -- start 2024-09-11T23:37:19.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcbfbd0 0x55723dcca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcc7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dcc7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55723dccd980 con 0x55723dcc04f0 2024-09-11T23:37:19.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55723dbe0f20 con 0x55723dcbfbd0 2024-09-11T23:37:19.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55723dccdb00 con 0x55723dccadb0 2024-09-11T23:37:19.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2dfb0a740 10 client.0 _ll_register_callbacks cb 0x55723dbaa030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcc7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcc7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60974/0 (socket says 172.21.15.133:60974) 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d67fc640 1 -- 172.21.15.133:0/3060536097 learned_addr learned my addr 172.21.15.133:0/3060536097 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d77fe640 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dcc7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcbfbd0 0x55723dcca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d67fc640 1 -- 172.21.15.133:0/3060536097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 msgr2=0x55723dcc7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d67fc640 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dcc7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d67fc640 1 -- 172.21.15.133:0/3060536097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcbfbd0 msgr2=0x55723dcca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d67fc640 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcbfbd0 0x55723dcca5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d67fc640 1 -- 172.21.15.133:0/3060536097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55723d9972b0 con 0x55723dcc04f0 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcbfbd0 0x55723dcca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2d67fc640 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcc7930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc011760 tx=0x7fb2cc011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12d484ac317b78d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2baff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2baff5640 1 -- 172.21.15.133:0/3060536097 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc00fe10 con 0x55723dcc04f0 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2baff5640 1 -- 172.21.15.133:0/3060536097 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fb2cc005020 con 0x55723dcc04f0 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3060536097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d0003b30 con 0x55723dcc04f0 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2baff5640 1 -- 172.21.15.133:0/3060536097 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2cc005220 con 0x55723dcc04f0 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2dfb0a740 10 client.15609 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.098+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3060536097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55723dcd9290 con 0x55723dcc04f0 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.099+0000 7fb2baff5640 1 -- 172.21.15.133:0/3060536097 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d0003b30 con 0x55723dcc04f0 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.099+0000 7fb2baff5640 1 -- 172.21.15.133:0/3060536097 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb2cc021030 con 0x55723dcc04f0 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.099+0000 7fb2dfb0a740 10 client.15609 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.099+0000 7fb2dfb0a740 10 client.15609 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.099+0000 7fb2dfb0a740 20 client.15609 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.099+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55723dcdb970 0x55723dcdcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.099+0000 7fb2dfb0a740 4 client.15609 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.099+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3060536097 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55723dc33600 con 0x55723dcdb970 2024-09-11T23:37:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.099+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55723dcdb970 0x55723dcdcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55723dcdb970 0x55723dcdcde0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87e482dab705f998 server_cookie=68b372a11789392e in_seq=0 out_seq=0 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2baff5640 10 client.15609 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2baff5640 1 -- 172.21.15.133:0/3060536097 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55723dc33600 con 0x55723dcdb970 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2baff5640 10 client.15609 handle_command_reply: tid=0 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2dfb0a740 1 client.15609 shutdown 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3060536097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 msgr2=0x55723dcc7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcc7930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc011760 tx=0x7fb2cc011c20 comp rx=0 tx=0).stop 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3060536097 shutdown_connections 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55723dcdb970 0x55723dcdcde0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dcc7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcc7930 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/3060536097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcbfbd0 0x55723dcca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2baff5640 0 client.15609 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.100+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3060536097 >> 172.21.15.133:0/3060536097 conn(0x55723dcbc160 msgr2=0x55723dcc1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.101+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3060536097 shutdown_connections 2024-09-11T23:37:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.101+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3060536097 wait complete. 2024-09-11T23:37:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.101+0000 7fb2dfb0a740 20 client.15609 trim_cache size 0 max 16384 2024-09-11T23:37:19.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2dfb0a740 1 Processor -- start 2024-09-11T23:37:19.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2dfb0a740 1 -- start start 2024-09-11T23:37:19.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcbfbd0 0x55723dcdcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcbc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dce50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55723dcdb120 con 0x55723dcc04f0 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55723dcdb7f0 con 0x55723dcbfbd0 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55723dcdbec0 con 0x55723dccadb0 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2d67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcbc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2d67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcbc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60980/0 (socket says 172.21.15.133:60980) 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2d67fc640 1 -- 172.21.15.133:0/3585314384 learned_addr learned my addr 172.21.15.133:0/3585314384 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2d77fe640 1 --2- 172.21.15.133:0/3585314384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dce50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/3585314384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcbfbd0 0x55723dcdcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2d67fc640 1 -- 172.21.15.133:0/3585314384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 msgr2=0x55723dce50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2d67fc640 1 --2- 172.21.15.133:0/3585314384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dce50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2d67fc640 1 -- 172.21.15.133:0/3585314384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcbfbd0 msgr2=0x55723dcdcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2d67fc640 1 --2- 172.21.15.133:0/3585314384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcbfbd0 0x55723dcdcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.118+0000 7fb2d67fc640 1 -- 172.21.15.133:0/3585314384 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55723dcc0320 con 0x55723dcc04f0 2024-09-11T23:37:19.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2d67fc640 1 --2- 172.21.15.133:0/3585314384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcbc540 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0195b0 tx=0x7fb2cc019470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1bd48b67d1a1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:19.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2bcff9640 1 -- 172.21.15.133:0/3585314384 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc030070 con 0x55723dcc04f0 2024-09-11T23:37:19.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2bcff9640 1 -- 172.21.15.133:0/3585314384 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2cc0197a0 con 0x55723dcc04f0 2024-09-11T23:37:19.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2bcff9640 1 -- 172.21.15.133:0/3585314384 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc004da0 con 0x55723dcc04f0 2024-09-11T23:37:19.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3585314384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 msgr2=0x55723dcbc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/3585314384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcbc540 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0195b0 tx=0x7fb2cc019470 comp rx=0 tx=0).stop 2024-09-11T23:37:19.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3585314384 shutdown_connections 2024-09-11T23:37:19.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/3585314384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dce50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/3585314384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcc04f0 0x55723dcbc540 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/3585314384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcbfbd0 0x55723dcdcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3585314384 >> 172.21.15.133:0/3585314384 conn(0x55723dcbc160 msgr2=0x55723dcc6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:19.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.119+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3585314384 shutdown_connections 2024-09-11T23:37:19.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/3585314384 wait complete. 2024-09-11T23:37:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2dfb0a740 1 -- start start 2024-09-11T23:37:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2dfb0a740 1 Processor -- start 2024-09-11T23:37:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dce5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcc04f0 0x55723dce0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2dfb0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dce0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55723dcdcc60 con 0x55723dcbfbd0 2024-09-11T23:37:19.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55723dcdc590 con 0x55723dcc04f0 2024-09-11T23:37:19.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2dfb0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55723dcdd330 con 0x55723dccadb0 2024-09-11T23:37:19.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2dfb0a740 10 client.0 _ll_register_callbacks cb 0x7fb2cc00ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:19.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dce5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dce5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60992/0 (socket says 172.21.15.133:60992) 2024-09-11T23:37:19.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2d6ffd640 1 -- 172.21.15.133:0/436201031 learned_addr learned my addr 172.21.15.133:0/436201031 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:19.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2d67fc640 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcc04f0 0x55723dce0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2d77fe640 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dce0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2d6ffd640 1 -- 172.21.15.133:0/436201031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 msgr2=0x55723dce0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dce0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2d6ffd640 1 -- 172.21.15.133:0/436201031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcc04f0 msgr2=0x55723dce0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:19.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcc04f0 0x55723dce0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.120+0000 7fb2d6ffd640 1 -- 172.21.15.133:0/436201031 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55723dce12f0 con 0x55723dcbfbd0 2024-09-11T23:37:19.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2d6ffd640 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dce5540 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009610 tx=0x7fb2c80048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ec8d05008e23ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:19.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2baff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:19.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2baff5640 1 -- 172.21.15.133:0/436201031 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8020040 con 0x55723dcbfbd0 2024-09-11T23:37:19.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2baff5640 1 -- 172.21.15.133:0/436201031 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fb2c8009cc0 con 0x55723dcbfbd0 2024-09-11T23:37:19.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/436201031 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2cc0197a0 con 0x55723dcbfbd0 2024-09-11T23:37:19.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2baff5640 1 -- 172.21.15.133:0/436201031 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2c80056b0 con 0x55723dcbfbd0 2024-09-11T23:37:19.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2dfb0a740 10 client.15615 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:19.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/436201031 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55723dd45e30 con 0x55723dcbfbd0 2024-09-11T23:37:19.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2baff5640 1 -- 172.21.15.133:0/436201031 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2cc0197a0 con 0x55723dcbfbd0 2024-09-11T23:37:19.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2baff5640 1 -- 172.21.15.133:0/436201031 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb2c8097ab0 con 0x55723dcbfbd0 2024-09-11T23:37:19.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2dfb0a740 10 client.15615 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:19.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2dfb0a740 10 client.15615 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:37:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2dfb0a740 20 client.15615 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55723dd4a9d0 0x55723dd496e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2dfb0a740 4 client.15615 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:37:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.121+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/436201031 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "status"}) -- 0x55723dc2db10 con 0x55723dd4a9d0 2024-09-11T23:37:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2d67fc640 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55723dd4a9d0 0x55723dd496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2baff5640 10 client.15615 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2d67fc640 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55723dd4a9d0 0x55723dd496e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87857a1ffa1603ce server_cookie=631f5a80326156eb in_seq=0 out_seq=0 2024-09-11T23:37:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2baff5640 1 -- 172.21.15.133:0/436201031 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55723dc2db10 con 0x55723dd4a9d0 2024-09-11T23:37:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2baff5640 10 client.15615 handle_command_reply: tid=0 2024-09-11T23:37:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2dfb0a740 1 client.15615 shutdown 2024-09-11T23:37:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/436201031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 msgr2=0x55723dce5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dce5540 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009610 tx=0x7fb2c80048a0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/436201031 shutdown_connections 2024-09-11T23:37:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55723dd4a9d0 0x55723dd496e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55723dccadb0 0x55723dce0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55723dcc04f0 0x55723dce0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2baff5640 0 client.15615 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2dfb0a740 1 --2- 172.21.15.133:0/436201031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55723dcbfbd0 0x55723dce5540 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/436201031 >> 172.21.15.133:0/436201031 conn(0x55723dcbc160 msgr2=0x55723dcd5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.122+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/436201031 shutdown_connections 2024-09-11T23:37:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2dfb0a740 1 -- 172.21.15.133:0/436201031 wait complete. 2024-09-11T23:37:19.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2dfb0a740 20 client.15615 trim_cache size 0 max 16384 2024-09-11T23:37:19.139 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:37:19.139 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "42321b88-7095-11ef-bcea-c7b262605968", 2024-09-11T23:37:19.139 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-09-11T23:37:19.139 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24481, 2024-09-11T23:37:19.139 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-11T23:37:19.140 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-11T23:37:19.140 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-11T23:37:19.140 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 144.68200850700001, 2024-09-11T23:37:19.140 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 28, 2024-09-11T23:37:19.140 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 82, 2024-09-11T23:37:19.140 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T23:37:19.140 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 164.358567685 2024-09-11T23:37:19.140 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:37:19.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 -- 172.21.15.133:0/2739282437 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2c00a2ea0 msgr2=0x7fb2c00a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2c00a2ea0 0x7fb2c00a5360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2d004a000 tx=0x7fb2d004d000 comp rx=0 tx=0).stop 2024-09-11T23:37:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 -- 172.21.15.133:0/2739282437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d8074510 msgr2=0x7fb2d819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d8074510 0x7fb2d819a440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800d770 tx=0x7fb2c800dc30 comp rx=0 tx=0).stop 2024-09-11T23:37:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 -- 172.21.15.133:0/2739282437 shutdown_connections 2024-09-11T23:37:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2c00a2ea0 0x7fb2c00a5360 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2d810e260 0x7fb2d819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2d8074e30 0x7fb2d819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 --2- 172.21.15.133:0/2739282437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d8074510 0x7fb2d819a440 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 -- 172.21.15.133:0/2739282437 >> 172.21.15.133:0/2739282437 conn(0x7fb2d80fd680 msgr2=0x7fb2d80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 -- 172.21.15.133:0/2739282437 shutdown_connections 2024-09-11T23:37:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:19.123+0000 7fb2d7fff640 1 -- 172.21.15.133:0/2739282437 wait complete. 2024-09-11T23:37:19.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:19 smithi138 ceph-mon[35142]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 17 op/s 2024-09-11T23:37:19.956 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:19 smithi178 ceph-mon[35882]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 17 op/s 2024-09-11T23:37:19.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:19 smithi133 ceph-mon[31430]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 17 op/s 2024-09-11T23:37:19.957 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-11T23:37:21.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:21 smithi138 ceph-mon[35142]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T23:37:21.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:21 smithi133 ceph-mon[31430]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T23:37:21.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:21 smithi178 ceph-mon[35882]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T23:37:23.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:23 smithi138 ceph-mon[35142]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-11T23:37:23.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:23 smithi133 ceph-mon[31430]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-11T23:37:23.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:23 smithi178 ceph-mon[35882]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-11T23:37:25.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:25 smithi138 ceph-mon[35142]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 16 op/s 2024-09-11T23:37:25.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:25 smithi133 ceph-mon[31430]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 16 op/s 2024-09-11T23:37:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.480+0000 7fe729966640 1 -- 172.21.15.133:0/2970585155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe724102d20 msgr2=0x7fe724103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.480+0000 7fe729966640 1 --2- 172.21.15.133:0/2970585155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe724102d20 0x7fe724103100 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe714009900 tx=0x7fe71402a480 comp rx=0 tx=0).stop 2024-09-11T23:37:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.480+0000 7fe729966640 1 -- 172.21.15.133:0/2970585155 shutdown_connections 2024-09-11T23:37:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.480+0000 7fe729966640 1 --2- 172.21.15.133:0/2970585155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe72410c0f0 0x7fe72410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.480+0000 7fe729966640 1 --2- 172.21.15.133:0/2970585155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe724103640 0x7fe72410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.480+0000 7fe729966640 1 --2- 172.21.15.133:0/2970585155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe724102d20 0x7fe724103100 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.481+0000 7fe729966640 1 -- 172.21.15.133:0/2970585155 >> 172.21.15.133:0/2970585155 conn(0x7fe7240fc6b0 msgr2=0x7fe7240fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.481+0000 7fe729966640 1 -- 172.21.15.133:0/2970585155 shutdown_connections 2024-09-11T23:37:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.481+0000 7fe729966640 1 -- 172.21.15.133:0/2970585155 wait complete. 2024-09-11T23:37:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.481+0000 7fe729966640 1 Processor -- start 2024-09-11T23:37:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.481+0000 7fe729966640 1 -- start start 2024-09-11T23:37:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe729966640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe724102d20 0x7fe724198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe729966640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe724103640 0x7fe7241988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe729966640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe72410c0f0 0x7fe72419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe729966640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe724111ef0 con 0x7fe724103640 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe729966640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe724111d70 con 0x7fe72410c0f0 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe729966640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe724112070 con 0x7fe724102d20 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe7227fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe724103640 0x7fe7241988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe7227fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe724103640 0x7fe7241988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47522/0 (socket says 172.21.15.133:47522) 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe7227fc640 1 -- 172.21.15.133:0/2376652673 learned_addr learned my addr 172.21.15.133:0/2376652673 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe722ffd640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe724102d20 0x7fe724198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe7237fe640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe72410c0f0 0x7fe72419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe7227fc640 1 -- 172.21.15.133:0/2376652673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe724102d20 msgr2=0x7fe724198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe7227fc640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe724102d20 0x7fe724198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe7227fc640 1 -- 172.21.15.133:0/2376652673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe72410c0f0 msgr2=0x7fe72419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe7227fc640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe72410c0f0 0x7fe72419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe7227fc640 1 -- 172.21.15.133:0/2376652673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72419d350 con 0x7fe724103640 2024-09-11T23:37:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe7237fe640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe72410c0f0 0x7fe72419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe7227fc640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe724103640 0x7fe7241988a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe71800d950 tx=0x7fe71800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe728964640 1 -- 172.21.15.133:0/2376652673 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718017030 con 0x7fe724103640 2024-09-11T23:37:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe728964640 1 -- 172.21.15.133:0/2376652673 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe718004020 con 0x7fe724103640 2024-09-11T23:37:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.482+0000 7fe729966640 1 -- 172.21.15.133:0/2376652673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72419d640 con 0x7fe724103640 2024-09-11T23:37:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.483+0000 7fe728964640 1 -- 172.21.15.133:0/2376652673 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7180047a0 con 0x7fe724103640 2024-09-11T23:37:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.483+0000 7fe729966640 1 -- 172.21.15.133:0/2376652673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7241a4ed0 con 0x7fe724103640 2024-09-11T23:37:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe728964640 1 -- 172.21.15.133:0/2376652673 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fe7180041d0 con 0x7fe724103640 2024-09-11T23:37:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe72c7df740 1 Processor -- start 2024-09-11T23:37:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe72c7df740 1 -- start start 2024-09-11T23:37:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc65db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc74b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc774b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe72c7df740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56349dbca2f0 con 0x56349dc750c0 2024-09-11T23:37:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe72c7df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56349db72270 con 0x56349dc659d0 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe72c7df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56349dbc62b0 con 0x56349dc662f0 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe7227fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe7227fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc774b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47530/0 (socket says 172.21.15.133:47530) 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe7227fc640 1 -- 172.21.15.133:0/2198806961 learned_addr learned my addr 172.21.15.133:0/2198806961 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe7237fe640 1 --2- 172.21.15.133:0/2198806961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc65db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe722ffd640 1 --2- 172.21.15.133:0/2198806961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc74b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.486+0000 7fe728964640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe70c0a2f50 0x7fe70c0a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe7227fc640 1 -- 172.21.15.133:0/2198806961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 msgr2=0x56349dc74b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe7227fc640 1 --2- 172.21.15.133:0/2198806961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc74b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe7227fc640 1 -- 172.21.15.133:0/2198806961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 msgr2=0x56349dc65db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe7227fc640 1 --2- 172.21.15.133:0/2198806961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc65db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe728964640 1 -- 172.21.15.133:0/2376652673 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe7180cf0a0 con 0x7fe724103640 2024-09-11T23:37:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe7227fc640 1 -- 172.21.15.133:0/2198806961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349db71340 con 0x56349dc750c0 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe7237fe640 1 --2- 172.21.15.133:0/2198806961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc65db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe722ffd640 1 --2- 172.21.15.133:0/2198806961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc74b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe7237fe640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe70c0a2f50 0x7fe70c0a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe7227fc640 1 --2- 172.21.15.133:0/2198806961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc774b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe7180cc760 tx=0x7fe718000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80a099135ceb078 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe709ffb640 1 -- 172.21.15.133:0/2198806961 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71809af30 con 0x56349dc750c0 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe709ffb640 1 -- 172.21.15.133:0/2198806961 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe71809baf0 con 0x56349dc750c0 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.487+0000 7fe709ffb640 1 -- 172.21.15.133:0/2198806961 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71809ad50 con 0x56349dc750c0 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.488+0000 7fe72c7df740 1 -- 172.21.15.133:0/2198806961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 msgr2=0x56349dc774b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.488+0000 7fe7237fe640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe70c0a2f50 0x7fe70c0a5410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009370 tx=0x7fe71c004a40 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.488+0000 7fe72c7df740 1 --2- 172.21.15.133:0/2198806961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc774b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe7180cc760 tx=0x7fe718000bc0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.488+0000 7fe72c7df740 1 -- 172.21.15.133:0/2198806961 shutdown_connections 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.488+0000 7fe72c7df740 1 --2- 172.21.15.133:0/2198806961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc774b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.488+0000 7fe72c7df740 1 --2- 172.21.15.133:0/2198806961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc74b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.488+0000 7fe72c7df740 1 --2- 172.21.15.133:0/2198806961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc65db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.488+0000 7fe72c7df740 1 -- 172.21.15.133:0/2198806961 >> 172.21.15.133:0/2198806961 conn(0x56349dc62060 msgr2=0x56349dc62440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.488+0000 7fe72c7df740 1 -- 172.21.15.133:0/2198806961 shutdown_connections 2024-09-11T23:37:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.488+0000 7fe72c7df740 1 -- 172.21.15.133:0/2198806961 wait complete. 2024-09-11T23:37:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.488+0000 7fe72c7df740 1 -- start start 2024-09-11T23:37:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe72c7df740 1 Processor -- start 2024-09-11T23:37:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc71fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc72500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc72a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe72c7df740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56349dc77c30 con 0x56349dc750c0 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe72c7df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56349db53290 con 0x56349dc659d0 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe72c7df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56349dc77db0 con 0x56349dc662f0 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe72c7df740 10 client.0 _ll_register_callbacks cb 0x56349db84140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe7237fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc72a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe7237fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc72a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47540/0 (socket says 172.21.15.133:47540) 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe7237fe640 1 -- 172.21.15.133:0/3471011327 learned_addr learned my addr 172.21.15.133:0/3471011327 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe7227fc640 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc72500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe722ffd640 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc71fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe7237fe640 1 -- 172.21.15.133:0/3471011327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 msgr2=0x56349dc72500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe7237fe640 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc72500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe7237fe640 1 -- 172.21.15.133:0/3471011327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 msgr2=0x56349dc71fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe7237fe640 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc71fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.489+0000 7fe7237fe640 1 -- 172.21.15.133:0/3471011327 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349d8dd210 con 0x56349dc750c0 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe7237fe640 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc72a40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0039c0 tx=0x7fe71c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2219ff3e29212833 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe702ffd640 1 -- 172.21.15.133:0/3471011327 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c012800 con 0x56349dc750c0 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe72c7df740 1 -- 172.21.15.133:0/3471011327 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe71809baf0 con 0x56349dc750c0 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe702ffd640 1 -- 172.21.15.133:0/3471011327 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe71c0129a0 con 0x56349dc750c0 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe702ffd640 1 -- 172.21.15.133:0/3471011327 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe71c012ba0 con 0x56349dc750c0 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe72c7df740 10 client.15627 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe72c7df740 1 -- 172.21.15.133:0/3471011327 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56349d9a9620 con 0x56349dc750c0 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe702ffd640 1 -- 172.21.15.133:0/3471011327 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe71809baf0 con 0x56349dc750c0 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe702ffd640 1 -- 172.21.15.133:0/3471011327 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe71c00f040 con 0x56349dc750c0 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe72c7df740 10 client.15627 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe72c7df740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe72c7df740 20 client.15627 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe72c7df740 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56349dc85030 0x56349dc87460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe72c7df740 4 client.15627 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.490+0000 7fe72c7df740 1 -- 172.21.15.133:0/3471011327 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56349dbd7a20 con 0x56349dc85030 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.491+0000 7fe722ffd640 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56349dc85030 0x56349dc87460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.491+0000 7fe722ffd640 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56349dc85030 0x56349dc87460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f999cbabdb547a server_cookie=492a72e5df123a3 in_seq=0 out_seq=0 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.491+0000 7fe702ffd640 10 client.15627 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe702ffd640 1 -- 172.21.15.133:0/3471011327 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56349dbd7a20 con 0x56349dc85030 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe702ffd640 10 client.15627 handle_command_reply: tid=0 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 1 client.15627 shutdown 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 1 -- 172.21.15.133:0/3471011327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 msgr2=0x56349dc72a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc72a40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0039c0 tx=0x7fe71c004ab0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 1 -- 172.21.15.133:0/3471011327 shutdown_connections 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56349dc85030 0x56349dc87460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc72a40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc72500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe702ffd640 0 client.15627 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 1 --2- 172.21.15.133:0/3471011327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc71fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 1 -- 172.21.15.133:0/3471011327 >> 172.21.15.133:0/3471011327 conn(0x56349dc62060 msgr2=0x56349dc67dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 1 -- 172.21.15.133:0/3471011327 shutdown_connections 2024-09-11T23:37:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 1 -- 172.21.15.133:0/3471011327 wait complete. 2024-09-11T23:37:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.492+0000 7fe72c7df740 20 client.15627 trim_cache size 0 max 16384 2024-09-11T23:37:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.506+0000 7fe72c7df740 1 Processor -- start 2024-09-11T23:37:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe72c7df740 1 -- start start 2024-09-11T23:37:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc73f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc8cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc8d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe72c7df740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241ab570 con 0x56349dc750c0 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe72c7df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241ad0b0 con 0x56349dc659d0 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe72c7df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241abc40 con 0x56349dc662f0 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7237fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc8d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7237fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc8d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47542/0 (socket says 172.21.15.133:47542) 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7237fe640 1 -- 172.21.15.133:0/2127640216 learned_addr learned my addr 172.21.15.133:0/2127640216 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7227fc640 1 --2- 172.21.15.133:0/2127640216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc8cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7237fe640 1 -- 172.21.15.133:0/2127640216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 msgr2=0x56349dc8cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7237fe640 1 --2- 172.21.15.133:0/2127640216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc8cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7237fe640 1 -- 172.21.15.133:0/2127640216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 msgr2=0x56349dc73f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7237fe640 1 --2- 172.21.15.133:0/2127640216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc73f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7237fe640 1 -- 172.21.15.133:0/2127640216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349dc6cd50 con 0x56349dc750c0 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7237fe640 1 --2- 172.21.15.133:0/2127640216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc8d300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe71c004a10 tx=0x7fe71c009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9c928c1841ba2c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7097fa640 1 -- 172.21.15.133:0/2127640216 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c012800 con 0x56349dc750c0 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.507+0000 7fe7097fa640 1 -- 172.21.15.133:0/2127640216 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe71c024070 con 0x56349dc750c0 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.508+0000 7fe7097fa640 1 -- 172.21.15.133:0/2127640216 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c01f690 con 0x56349dc750c0 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.508+0000 7fe72c7df740 1 -- 172.21.15.133:0/2127640216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 msgr2=0x56349dc8d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.508+0000 7fe72c7df740 1 --2- 172.21.15.133:0/2127640216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc8d300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe71c004a10 tx=0x7fe71c009c60 comp rx=0 tx=0).stop 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.508+0000 7fe72c7df740 1 -- 172.21.15.133:0/2127640216 shutdown_connections 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.508+0000 7fe72c7df740 1 --2- 172.21.15.133:0/2127640216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc750c0 0x56349dc8d300 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.508+0000 7fe72c7df740 1 --2- 172.21.15.133:0/2127640216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc8cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.508+0000 7fe72c7df740 1 --2- 172.21.15.133:0/2127640216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc659d0 0x56349dc73f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.508+0000 7fe72c7df740 1 -- 172.21.15.133:0/2127640216 >> 172.21.15.133:0/2127640216 conn(0x56349dc62060 msgr2=0x56349dc73820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.508+0000 7fe72c7df740 1 -- 172.21.15.133:0/2127640216 shutdown_connections 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.508+0000 7fe72c7df740 1 -- 172.21.15.133:0/2127640216 wait complete. 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.508+0000 7fe72c7df740 1 -- start start 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe72c7df740 1 Processor -- start 2024-09-11T23:37:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc659d0 0x56349dc78fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe72c7df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc750c0 0x56349dc79a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe72c7df740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241ac9e0 con 0x56349dc659d0 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe72c7df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241ac310 con 0x56349dc750c0 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe72c7df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241a9570 con 0x56349dc662f0 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe72c7df740 10 client.0 _ll_register_callbacks cb 0x7fe71c009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe722ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc659d0 0x56349dc78fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe722ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc659d0 0x56349dc78fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47544/0 (socket says 172.21.15.133:47544) 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe722ffd640 1 -- 172.21.15.133:0/842153242 learned_addr learned my addr 172.21.15.133:0/842153242 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe7227fc640 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe7237fe640 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc750c0 0x56349dc79a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe722ffd640 1 -- 172.21.15.133:0/842153242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 msgr2=0x56349dc794e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe722ffd640 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc794e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe722ffd640 1 -- 172.21.15.133:0/842153242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc750c0 msgr2=0x56349dc79a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe722ffd640 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc750c0 0x56349dc79a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe722ffd640 1 -- 172.21.15.133:0/842153242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349dceefe0 con 0x56349dc659d0 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe7237fe640 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc750c0 0x56349dc79a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.509+0000 7fe7227fc640 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe722ffd640 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc659d0 0x56349dc78fa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe714009230 tx=0x7fe7140316a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c2491121903b9d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe702ffd640 1 -- 172.21.15.133:0/842153242 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714035040 con 0x56349dc659d0 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe702ffd640 1 -- 172.21.15.133:0/842153242 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe7140319e0 con 0x56349dc659d0 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe72c7df740 1 -- 172.21.15.133:0/842153242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe71c024070 con 0x56349dc659d0 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe702ffd640 1 -- 172.21.15.133:0/842153242 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe714004040 con 0x56349dc659d0 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe72c7df740 10 client.15633 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe72c7df740 1 -- 172.21.15.133:0/842153242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56349dcf02d0 con 0x56349dc659d0 2024-09-11T23:37:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe702ffd640 1 -- 172.21.15.133:0/842153242 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe71c024070 con 0x56349dc659d0 2024-09-11T23:37:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe702ffd640 1 -- 172.21.15.133:0/842153242 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe714004a00 con 0x56349dc659d0 2024-09-11T23:37:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe72c7df740 10 client.15633 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe72c7df740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe72c7df740 20 client.15633 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe72c7df740 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56349dcf09d0 0x56349dcf2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe72c7df740 4 client.15633 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:37:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.510+0000 7fe72c7df740 1 -- 172.21.15.133:0/842153242 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56349dbd1d70 con 0x56349dcf09d0 2024-09-11T23:37:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.511+0000 7fe7227fc640 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56349dcf09d0 0x56349dcf2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.511+0000 7fe702ffd640 10 client.15633 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.511+0000 7fe7227fc640 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56349dcf09d0 0x56349dcf2df0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e49902f260cbf6c server_cookie=de55019e446a07dd in_seq=0 out_seq=0 2024-09-11T23:37:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.514+0000 7fe702ffd640 1 -- 172.21.15.133:0/842153242 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56349dbd1d70 con 0x56349dcf09d0 2024-09-11T23:37:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.514+0000 7fe702ffd640 10 client.15633 handle_command_reply: tid=0 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.514+0000 7fe72c7df740 1 client.15633 shutdown 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.514+0000 7fe72c7df740 1 -- 172.21.15.133:0/842153242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc659d0 msgr2=0x56349dc78fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.514+0000 7fe72c7df740 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc659d0 0x56349dc78fa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe714009230 tx=0x7fe7140316a0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe72c7df740 1 -- 172.21.15.133:0/842153242 shutdown_connections 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe72c7df740 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56349dcf09d0 0x56349dcf2df0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe72c7df740 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56349dc750c0 0x56349dc79a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe72c7df740 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56349dc662f0 0x56349dc794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe702ffd640 0 client.15633 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe72c7df740 1 --2- 172.21.15.133:0/842153242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56349dc659d0 0x56349dc78fa0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe72c7df740 1 -- 172.21.15.133:0/842153242 >> 172.21.15.133:0/842153242 conn(0x56349dc62060 msgr2=0x56349dc727a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe72c7df740 1 -- 172.21.15.133:0/842153242 shutdown_connections 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe72c7df740 1 -- 172.21.15.133:0/842153242 wait complete. 2024-09-11T23:37:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe72c7df740 20 client.15633 trim_cache size 0 max 16384 2024-09-11T23:37:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:37:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:37:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:37:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:37:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 -- 172.21.15.133:0/2376652673 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe70c0a2f50 msgr2=0x7fe70c0a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe70c0a2f50 0x7fe70c0a5410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009370 tx=0x7fe71c004a40 comp rx=0 tx=0).stop 2024-09-11T23:37:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 -- 172.21.15.133:0/2376652673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe724103640 msgr2=0x7fe7241988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe724103640 0x7fe7241988a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe71800d950 tx=0x7fe71800de10 comp rx=0 tx=0).stop 2024-09-11T23:37:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 -- 172.21.15.133:0/2376652673 shutdown_connections 2024-09-11T23:37:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe70c0a2f50 0x7fe70c0a5410 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe72410c0f0 0x7fe72419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe724103640 0x7fe7241988a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 --2- 172.21.15.133:0/2376652673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe724102d20 0x7fe724198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 -- 172.21.15.133:0/2376652673 >> 172.21.15.133:0/2376652673 conn(0x7fe7240fc6b0 msgr2=0x7fe7240fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 -- 172.21.15.133:0/2376652673 shutdown_connections 2024-09-11T23:37:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:25.515+0000 7fe729966640 1 -- 172.21.15.133:0/2376652673 wait complete. 2024-09-11T23:37:25.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:25 smithi178 ceph-mon[35882]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 16 op/s 2024-09-11T23:37:27.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:27 smithi138 ceph-mon[35142]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:37:27.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:27 smithi133 ceph-mon[31430]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:37:27.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:27 smithi178 ceph-mon[35882]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:37:29.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:29 smithi138 ceph-mon[35142]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T23:37:29.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:29 smithi133 ceph-mon[31430]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T23:37:29.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:29 smithi178 ceph-mon[35882]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T23:37:29.799 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:37:30.063 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:30.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.594+0000 7fcab2838640 1 -- 172.21.15.133:0/3800414521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaac102790 msgr2=0x7fcaac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:30.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.594+0000 7fcab2838640 1 --2- 172.21.15.133:0/3800414521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaac102790 0x7fcaac1067e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009900 tx=0x7fcaa002a3f0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.594+0000 7fcab2838640 1 -- 172.21.15.133:0/3800414521 shutdown_connections 2024-09-11T23:37:30.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.594+0000 7fcab2838640 1 --2- 172.21.15.133:0/3800414521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaac106f10 0x7fcaac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.594+0000 7fcab2838640 1 --2- 172.21.15.133:0/3800414521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaac102790 0x7fcaac1067e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.594+0000 7fcab2838640 1 --2- 172.21.15.133:0/3800414521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaac101de0 0x7fcaac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.594+0000 7fcab2838640 1 -- 172.21.15.133:0/3800414521 >> 172.21.15.133:0/3800414521 conn(0x7fcaac0fd8c0 msgr2=0x7fcaac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:30.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.594+0000 7fcab2838640 1 -- 172.21.15.133:0/3800414521 shutdown_connections 2024-09-11T23:37:30.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.594+0000 7fcab2838640 1 -- 172.21.15.133:0/3800414521 wait complete. 2024-09-11T23:37:30.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcab2838640 1 Processor -- start 2024-09-11T23:37:30.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcab2838640 1 -- start start 2024-09-11T23:37:30.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcab2838640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaac101de0 0x7fcaac1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcab2838640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaac102790 0x7fcaac198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcab2838640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaac106f10 0x7fcaac19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcab2838640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac111f90 con 0x7fcaac102790 2024-09-11T23:37:30.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcab2838640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac111e10 con 0x7fcaac106f10 2024-09-11T23:37:30.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcab2838640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac112110 con 0x7fcaac101de0 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcaab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaac102790 0x7fcaac198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcaab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaac102790 0x7fcaac198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47560/0 (socket says 172.21.15.133:47560) 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcaab7fe640 1 -- 172.21.15.133:0/2894962381 learned_addr learned my addr 172.21.15.133:0/2894962381 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcab0dd5640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaac106f10 0x7fcaac19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.595+0000 7fcaabfff640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaac101de0 0x7fcaac1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.596+0000 7fcaab7fe640 1 -- 172.21.15.133:0/2894962381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaac101de0 msgr2=0x7fcaac1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.596+0000 7fcaab7fe640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaac101de0 0x7fcaac1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.596+0000 7fcaab7fe640 1 -- 172.21.15.133:0/2894962381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaac106f10 msgr2=0x7fcaac19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.596+0000 7fcaab7fe640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaac106f10 0x7fcaac19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.596+0000 7fcaab7fe640 1 -- 172.21.15.133:0/2894962381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaac19d3a0 con 0x7fcaac102790 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.596+0000 7fcaab7fe640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaac102790 0x7fcaac198930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0005ab0 tx=0x7fcaa00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.596+0000 7fcaa97fa640 1 -- 172.21.15.133:0/2894962381 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa003f030 con 0x7fcaac102790 2024-09-11T23:37:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.596+0000 7fcaa97fa640 1 -- 172.21.15.133:0/2894962381 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa0003ce0 con 0x7fcaac102790 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.596+0000 7fcaa97fa640 1 -- 172.21.15.133:0/2894962381 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0035070 con 0x7fcaac102790 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.596+0000 7fcab2838640 1 -- 172.21.15.133:0/2894962381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaac19d630 con 0x7fcaac102790 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.596+0000 7fcab2838640 1 -- 172.21.15.133:0/2894962381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaac19dac0 con 0x7fcaac102790 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.598+0000 7fcaa97fa640 1 -- 172.21.15.133:0/2894962381 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fcaa0042030 con 0x7fcaac102790 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcab4c11740 1 Processor -- start 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcab4c11740 1 -- start start 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcaa97fa640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fca940a2ea0 0x7fca940a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x56201239d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 0x5620123a8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcab4c11740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5620123032e0 con 0x56201239d4a0 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcab4c11740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5620122aace0 con 0x5620123a8680 2024-09-11T23:37:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcab4c11740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5620122ff2a0 con 0x56201239ddc0 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcab0dd5640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fca940a2ea0 0x7fca940a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcaa97fa640 1 -- 172.21.15.133:0/2894962381 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fcaa00efb90 con 0x7fcaac102790 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcaabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x56201239d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcaabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x56201239d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47566/0 (socket says 172.21.15.133:47566) 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcaabfff640 1 -- 172.21.15.133:0/694613731 learned_addr learned my addr 172.21.15.133:0/694613731 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.599+0000 7fcab0dd5640 1 --2- 172.21.15.133:0/694613731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.600+0000 7fcaabfff640 1 -- 172.21.15.133:0/694613731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 msgr2=0x5620123a8140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.600+0000 7fcaabfff640 1 --2- 172.21.15.133:0/694613731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 0x5620123a8140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.600+0000 7fcaabfff640 1 -- 172.21.15.133:0/694613731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 msgr2=0x5620123aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.600+0000 7fcaabfff640 1 --2- 172.21.15.133:0/694613731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.600+0000 7fcaabfff640 1 -- 172.21.15.133:0/694613731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56201228f440 con 0x56201239d4a0 2024-09-11T23:37:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.600+0000 7fcab0dd5640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fca940a2ea0 0x7fca940a5360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa400aba0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.600+0000 7fcaabfff640 1 --2- 172.21.15.133:0/694613731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x56201239d880 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00ad00 tx=0x7fca9c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0ba9d167481b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.600+0000 7fca927fc640 1 -- 172.21.15.133:0/694613731 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c01d070 con 0x56201239d4a0 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.600+0000 7fca927fc640 1 -- 172.21.15.133:0/694613731 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9c018b20 con 0x56201239d4a0 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.600+0000 7fca927fc640 1 -- 172.21.15.133:0/694613731 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c010bc0 con 0x56201239d4a0 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.601+0000 7fcab4c11740 1 -- 172.21.15.133:0/694613731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 msgr2=0x56201239d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.601+0000 7fcab4c11740 1 --2- 172.21.15.133:0/694613731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x56201239d880 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00ad00 tx=0x7fca9c00f790 comp rx=0 tx=0).stop 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.601+0000 7fcab4c11740 1 -- 172.21.15.133:0/694613731 shutdown_connections 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.601+0000 7fcab4c11740 1 --2- 172.21.15.133:0/694613731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.601+0000 7fcab4c11740 1 --2- 172.21.15.133:0/694613731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 0x5620123a8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.601+0000 7fcab4c11740 1 --2- 172.21.15.133:0/694613731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x56201239d880 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.601+0000 7fcab4c11740 1 -- 172.21.15.133:0/694613731 >> 172.21.15.133:0/694613731 conn(0x562012399a30 msgr2=0x562012399e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.601+0000 7fcab4c11740 1 -- 172.21.15.133:0/694613731 shutdown_connections 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.601+0000 7fcab4c11740 1 -- 172.21.15.133:0/694613731 wait complete. 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.601+0000 7fcab4c11740 1 -- start start 2024-09-11T23:37:30.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.601+0000 7fcab4c11740 1 Processor -- start 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123a8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 0x5620123a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123a5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcab4c11740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5620123ab250 con 0x56201239d4a0 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcab4c11740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5620122be050 con 0x5620123a8680 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcab4c11740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5620123ab3d0 con 0x56201239ddc0 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcaabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcab4c11740 10 client.0 _ll_register_callbacks cb 0x5620122fe160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcaabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123a8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47574/0 (socket says 172.21.15.133:47574) 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcaabfff640 1 -- 172.21.15.133:0/2604685589 learned_addr learned my addr 172.21.15.133:0/2604685589 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcaab7fe640 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 0x5620123a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcab0dd5640 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcaabfff640 1 -- 172.21.15.133:0/2604685589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 msgr2=0x5620123a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcaabfff640 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 0x5620123a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcaabfff640 1 -- 172.21.15.133:0/2604685589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 msgr2=0x5620123a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcaabfff640 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcaabfff640 1 -- 172.21.15.133:0/2604685589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620120743c0 con 0x56201239d4a0 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fcaabfff640 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123a8080 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fca9c010700 tx=0x7fca9c019f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8a691e27d07314 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fca8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.602+0000 7fca8b7fe640 1 -- 172.21.15.133:0/2604685589 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c01d040 con 0x56201239d4a0 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fca8b7fe640 1 -- 172.21.15.133:0/2604685589 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fca9c018640 con 0x56201239d4a0 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fcab4c11740 1 -- 172.21.15.133:0/2604685589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9c018b20 con 0x56201239d4a0 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fca8b7fe640 1 -- 172.21.15.133:0/2604685589 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fca9c031050 con 0x56201239d4a0 2024-09-11T23:37:30.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fcab4c11740 10 client.15645 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fcab4c11740 1 -- 172.21.15.133:0/2604685589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5620123b6c90 con 0x56201239d4a0 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fca8b7fe640 1 -- 172.21.15.133:0/2604685589 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9c018b20 con 0x56201239d4a0 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fca8b7fe640 1 -- 172.21.15.133:0/2604685589 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fca9c020030 con 0x56201239d4a0 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fcab4c11740 10 client.15645 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fcab4c11740 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fcab4c11740 20 client.15645 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fcab4c11740 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5620123b80f0 0x5620123ba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fcab4c11740 4 client.15645 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fcab4c11740 1 -- 172.21.15.133:0/2604685589 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562012310850 con 0x5620123b80f0 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.603+0000 7fcaab7fe640 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5620123b80f0 0x5620123ba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.604+0000 7fca8b7fe640 10 client.15645 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.604+0000 7fcaab7fe640 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5620123b80f0 0x5620123ba4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f117ce415f70ecdf server_cookie=370788954312b776 in_seq=0 out_seq=0 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.604+0000 7fca8b7fe640 1 -- 172.21.15.133:0/2604685589 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562012310850 con 0x5620123b80f0 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.604+0000 7fca8b7fe640 10 client.15645 handle_command_reply: tid=0 2024-09-11T23:37:30.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 1 client.15645 shutdown 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 1 -- 172.21.15.133:0/2604685589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 msgr2=0x5620123a8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123a8080 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fca9c010700 tx=0x7fca9c019f10 comp rx=0 tx=0).stop 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 1 -- 172.21.15.133:0/2604685589 shutdown_connections 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5620123b80f0 0x5620123ba4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 0x5620123a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 1 --2- 172.21.15.133:0/2604685589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123a8080 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fca8b7fe640 0 client.15645 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 1 -- 172.21.15.133:0/2604685589 >> 172.21.15.133:0/2604685589 conn(0x562012399a30 msgr2=0x56201239f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 1 -- 172.21.15.133:0/2604685589 shutdown_connections 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 1 -- 172.21.15.133:0/2604685589 wait complete. 2024-09-11T23:37:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.605+0000 7fcab4c11740 20 client.15645 trim_cache size 0 max 16384 2024-09-11T23:37:30.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.622+0000 7fcab4c11740 1 Processor -- start 2024-09-11T23:37:30.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.622+0000 7fcab4c11740 1 -- start start 2024-09-11T23:37:30.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 0x5620123bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcab4c11740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1af930 con 0x56201239d4a0 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcab4c11740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1b1470 con 0x5620123a8680 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcab4c11740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1b0000 con 0x56201239ddc0 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcaabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcaabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47590/0 (socket says 172.21.15.133:47590) 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcaabfff640 1 -- 172.21.15.133:0/2966633458 learned_addr learned my addr 172.21.15.133:0/2966633458 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcaab7fe640 1 --2- 172.21.15.133:0/2966633458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 0x5620123bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcab0dd5640 1 --2- 172.21.15.133:0/2966633458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcaabfff640 1 -- 172.21.15.133:0/2966633458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 msgr2=0x5620123bfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcaabfff640 1 --2- 172.21.15.133:0/2966633458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 0x5620123bfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcaabfff640 1 -- 172.21.15.133:0/2966633458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 msgr2=0x5620123c0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcaabfff640 1 --2- 172.21.15.133:0/2966633458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123c0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcaabfff640 1 -- 172.21.15.133:0/2966633458 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56201239dbf0 con 0x56201239d4a0 2024-09-11T23:37:30.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fcaabfff640 1 --2- 172.21.15.133:0/2966633458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123b44b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000c00 tx=0x7fca9c010550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecdc63013e7314e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:30.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.623+0000 7fca91ffb640 1 -- 172.21.15.133:0/2966633458 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c03a070 con 0x56201239d4a0 2024-09-11T23:37:30.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.624+0000 7fca91ffb640 1 -- 172.21.15.133:0/2966633458 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9c028070 con 0x56201239d4a0 2024-09-11T23:37:30.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.624+0000 7fca91ffb640 1 -- 172.21.15.133:0/2966633458 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c01d040 con 0x56201239d4a0 2024-09-11T23:37:30.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.624+0000 7fcab4c11740 1 -- 172.21.15.133:0/2966633458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 msgr2=0x5620123b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:30.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.624+0000 7fcab4c11740 1 --2- 172.21.15.133:0/2966633458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123b44b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000c00 tx=0x7fca9c010550 comp rx=0 tx=0).stop 2024-09-11T23:37:30.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.624+0000 7fcab4c11740 1 -- 172.21.15.133:0/2966633458 shutdown_connections 2024-09-11T23:37:30.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.624+0000 7fcab4c11740 1 --2- 172.21.15.133:0/2966633458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620123a8680 0x5620123c0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.624+0000 7fcab4c11740 1 --2- 172.21.15.133:0/2966633458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56201239ddc0 0x5620123bfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.624+0000 7fcab4c11740 1 --2- 172.21.15.133:0/2966633458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239d4a0 0x5620123b44b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.624+0000 7fcab4c11740 1 -- 172.21.15.133:0/2966633458 >> 172.21.15.133:0/2966633458 conn(0x562012399a30 msgr2=0x562012398b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:30.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.624+0000 7fcab4c11740 1 -- 172.21.15.133:0/2966633458 shutdown_connections 2024-09-11T23:37:30.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.624+0000 7fcab4c11740 1 -- 172.21.15.133:0/2966633458 wait complete. 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.625+0000 7fcab4c11740 1 -- start start 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.625+0000 7fcab4c11740 1 Processor -- start 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.625+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56201239d4a0 0x5620123b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.625+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239ddc0 0x5620123adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.625+0000 7fcab4c11740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620123a8680 0x5620123ae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.625+0000 7fcab4c11740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1b0da0 con 0x56201239ddc0 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.625+0000 7fcab4c11740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1b06d0 con 0x56201239d4a0 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.625+0000 7fcab4c11740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1ad910 con 0x5620123a8680 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.625+0000 7fcab4c11740 10 client.0 _ll_register_callbacks cb 0x7fca9c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcaab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239ddc0 0x5620123adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcaab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239ddc0 0x5620123adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47594/0 (socket says 172.21.15.133:47594) 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcaab7fe640 1 -- 172.21.15.133:0/1716680624 learned_addr learned my addr 172.21.15.133:0/1716680624 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcaabfff640 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56201239d4a0 0x5620123b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcab0dd5640 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620123a8680 0x5620123ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcaab7fe640 1 -- 172.21.15.133:0/1716680624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620123a8680 msgr2=0x5620123ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcaab7fe640 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620123a8680 0x5620123ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcaab7fe640 1 -- 172.21.15.133:0/1716680624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56201239d4a0 msgr2=0x5620123b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcaab7fe640 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56201239d4a0 0x5620123b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcaab7fe640 1 -- 172.21.15.133:0/1716680624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620123ae9f0 con 0x56201239ddc0 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcaabfff640 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56201239d4a0 0x5620123b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcab0dd5640 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620123a8680 0x5620123ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcaab7fe640 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239ddc0 0x5620123adef0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ed200 tx=0x7fcaa00bbae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43451062495a2ded server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fca8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fca8b7fe640 1 -- 172.21.15.133:0/1716680624 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00c0290 con 0x56201239ddc0 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fca8b7fe640 1 -- 172.21.15.133:0/1716680624 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fcaa003bd00 con 0x56201239ddc0 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.626+0000 7fcab4c11740 1 -- 172.21.15.133:0/1716680624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9c028070 con 0x56201239ddc0 2024-09-11T23:37:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fca8b7fe640 1 -- 172.21.15.133:0/1716680624 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcaa009f050 con 0x56201239ddc0 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fcab4c11740 10 client.15651 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fcab4c11740 1 -- 172.21.15.133:0/1716680624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5620124228c0 con 0x56201239ddc0 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fca8b7fe640 1 -- 172.21.15.133:0/1716680624 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca9c028070 con 0x56201239ddc0 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fca8b7fe640 1 -- 172.21.15.133:0/1716680624 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcaa0046d30 con 0x56201239ddc0 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fcab4c11740 10 client.15651 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fcab4c11740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fcab4c11740 20 client.15651 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fcab4c11740 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562012423020 0x562012425440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fcab4c11740 4 client.15651 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fcab4c11740 1 -- 172.21.15.133:0/1716680624 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562012399300 con 0x562012423020 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.627+0000 7fcaabfff640 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562012423020 0x562012425440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcaabfff640 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562012423020 0x562012425440 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f85bab09b3095fc8 server_cookie=b2520313034a6098 in_seq=0 out_seq=0 2024-09-11T23:37:30.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fca8b7fe640 10 client.15651 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fca8b7fe640 1 -- 172.21.15.133:0/1716680624 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562012399300 con 0x562012423020 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fca8b7fe640 10 client.15651 handle_command_reply: tid=0 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 1 client.15651 shutdown 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 1 -- 172.21.15.133:0/1716680624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239ddc0 msgr2=0x5620123adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239ddc0 0x5620123adef0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ed200 tx=0x7fcaa00bbae0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 1 -- 172.21.15.133:0/1716680624 shutdown_connections 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562012423020 0x562012425440 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620123a8680 0x5620123ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56201239ddc0 0x5620123adef0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 1 --2- 172.21.15.133:0/1716680624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56201239d4a0 0x5620123b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 1 -- 172.21.15.133:0/1716680624 >> 172.21.15.133:0/1716680624 conn(0x562012399a30 msgr2=0x5620123a3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fca8b7fe640 0 client.15651 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 1 -- 172.21.15.133:0/1716680624 shutdown_connections 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 1 -- 172.21.15.133:0/1716680624 wait complete. 2024-09-11T23:37:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.628+0000 7fcab4c11740 20 client.15651 trim_cache size 0 max 16384 2024-09-11T23:37:30.634 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:37:30.634 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:37:30.634 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:37:30.634 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:37:30.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 -- 172.21.15.133:0/2894962381 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fca940a2ea0 msgr2=0x7fca940a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:30.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fca940a2ea0 0x7fca940a5360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa400aba0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 -- 172.21.15.133:0/2894962381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaac102790 msgr2=0x7fcaac198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:30.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaac102790 0x7fcaac198930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0005ab0 tx=0x7fcaa00042f0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 -- 172.21.15.133:0/2894962381 shutdown_connections 2024-09-11T23:37:30.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fca940a2ea0 0x7fca940a5360 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaac106f10 0x7fcaac19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaac102790 0x7fcaac198930 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 --2- 172.21.15.133:0/2894962381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaac101de0 0x7fcaac1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:30.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 -- 172.21.15.133:0/2894962381 >> 172.21.15.133:0/2894962381 conn(0x7fcaac0fd8c0 msgr2=0x7fcaac0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:30.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 -- 172.21.15.133:0/2894962381 shutdown_connections 2024-09-11T23:37:30.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:30.629+0000 7fcab2838640 1 -- 172.21.15.133:0/2894962381 wait complete. 2024-09-11T23:37:31.422 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:31 smithi133 ceph-mon[31430]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:37:31.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:31 smithi138 ceph-mon[35142]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:37:31.449 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:37:31.449 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:37:31.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:31 smithi178 ceph-mon[35882]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:37:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.238+0000 7fbf0788d640 1 -- 172.21.15.133:0/3538129465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf001036e0 msgr2=0x7fbf00107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.238+0000 7fbf0788d640 1 --2- 172.21.15.133:0/3538129465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf001036e0 0x7fbf00107b90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009930 tx=0x7fbef002a420 comp rx=0 tx=0).stop 2024-09-11T23:37:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.238+0000 7fbf0788d640 1 -- 172.21.15.133:0/3538129465 shutdown_connections 2024-09-11T23:37:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.238+0000 7fbf0788d640 1 --2- 172.21.15.133:0/3538129465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf001082a0 0x7fbf0010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.238+0000 7fbf0788d640 1 --2- 172.21.15.133:0/3538129465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf001036e0 0x7fbf00107b90 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.238+0000 7fbf0788d640 1 --2- 172.21.15.133:0/3538129465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf00102d30 0x7fbf00103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.238+0000 7fbf0788d640 1 -- 172.21.15.133:0/3538129465 >> 172.21.15.133:0/3538129465 conn(0x7fbf000fc7d0 msgr2=0x7fbf000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.238+0000 7fbf0788d640 1 -- 172.21.15.133:0/3538129465 shutdown_connections 2024-09-11T23:37:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.238+0000 7fbf0788d640 1 -- 172.21.15.133:0/3538129465 wait complete. 2024-09-11T23:37:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.238+0000 7fbf0788d640 1 Processor -- start 2024-09-11T23:37:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.238+0000 7fbf0788d640 1 -- start start 2024-09-11T23:37:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf0788d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf00102d30 0x7fbf001985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf0788d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf001036e0 0x7fbf00198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf0788d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf001082a0 0x7fbf0019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf0788d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001120a0 con 0x7fbf001082a0 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf0788d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf00111f20 con 0x7fbf00102d30 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf0788d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf00112220 con 0x7fbf001036e0 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf05e2a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf001082a0 0x7fbf0019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf05e2a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf001082a0 0x7fbf0019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33888/0 (socket says 172.21.15.133:33888) 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf05e2a640 1 -- 172.21.15.133:0/744412858 learned_addr learned my addr 172.21.15.133:0/744412858 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf04e28640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf001036e0 0x7fbf00198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf05629640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf00102d30 0x7fbf001985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf05e2a640 1 -- 172.21.15.133:0/744412858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf001036e0 msgr2=0x7fbf00198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf05e2a640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf001036e0 0x7fbf00198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf05e2a640 1 -- 172.21.15.133:0/744412858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf00102d30 msgr2=0x7fbf001985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf05e2a640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf00102d30 0x7fbf001985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf05e2a640 1 -- 172.21.15.133:0/744412858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0019d570 con 0x7fbf001082a0 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.239+0000 7fbf05e2a640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf001082a0 0x7fbf0019ce90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbef800eaa0 tx=0x7fbef800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.240+0000 7fbef67fc640 1 -- 172.21.15.133:0/744412858 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef800cdb0 con 0x7fbf001082a0 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.240+0000 7fbf0788d640 1 -- 172.21.15.133:0/744412858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0019d860 con 0x7fbf001082a0 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.240+0000 7fbef67fc640 1 -- 172.21.15.133:0/744412858 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef8005020 con 0x7fbf001082a0 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.240+0000 7fbef67fc640 1 -- 172.21.15.133:0/744412858 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef80116b0 con 0x7fbf001082a0 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.240+0000 7fbf0788d640 1 -- 172.21.15.133:0/744412858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf001a5140 con 0x7fbf001082a0 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.241+0000 7fbf0788d640 1 -- 172.21.15.133:0/744412858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0006a850 con 0x7fbf001082a0 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.242+0000 7fbef67fc640 1 -- 172.21.15.133:0/744412858 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fbef8005560 con 0x7fbf001082a0 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.243+0000 7fbef67fc640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbeec0a2f20 0x7fbeec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.243+0000 7fbf05629640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbeec0a2f20 0x7fbeec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.243+0000 7fbef67fc640 1 -- 172.21.15.133:0/744412858 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fbef80cf820 con 0x7fbf001082a0 2024-09-11T23:37:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.243+0000 7fbf05629640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbeec0a2f20 0x7fbeec0a53e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009bd0 tx=0x7fbefc005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.248+0000 7fbef67fc640 1 -- 172.21.15.133:0/744412858 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbef8016030 con 0x7fbf001082a0 2024-09-11T23:37:32.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.473+0000 7fbf0788d640 1 -- 172.21.15.133:0/744412858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf0010a2b0 con 0x7fbf001082a0 2024-09-11T23:37:32.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.474+0000 7fbef67fc640 1 -- 172.21.15.133:0/744412858 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7fbef809c270 con 0x7fbf001082a0 2024-09-11T23:37:32.474 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:37:32.475 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:37:32.475 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:37:32.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 -- 172.21.15.133:0/744412858 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbeec0a2f20 msgr2=0x7fbeec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbeec0a2f20 0x7fbeec0a53e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009bd0 tx=0x7fbefc005f90 comp rx=0 tx=0).stop 2024-09-11T23:37:32.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 -- 172.21.15.133:0/744412858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf001082a0 msgr2=0x7fbf0019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:32.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf001082a0 0x7fbf0019ce90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbef800eaa0 tx=0x7fbef800ef60 comp rx=0 tx=0).stop 2024-09-11T23:37:32.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 -- 172.21.15.133:0/744412858 shutdown_connections 2024-09-11T23:37:32.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbeec0a2f20 0x7fbeec0a53e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf001082a0 0x7fbf0019ce90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf001036e0 0x7fbf00198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 --2- 172.21.15.133:0/744412858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf00102d30 0x7fbf001985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:32.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 -- 172.21.15.133:0/744412858 >> 172.21.15.133:0/744412858 conn(0x7fbf000fc7d0 msgr2=0x7fbf00105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:32.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 -- 172.21.15.133:0/744412858 shutdown_connections 2024-09-11T23:37:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:32.477+0000 7fbf0788d640 1 -- 172.21.15.133:0/744412858 wait complete. 2024-09-11T23:37:32.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:32 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/744412858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:32.879 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:32 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/744412858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:32.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:32 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/744412858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:33.244 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:37:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:33.789 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:33 smithi133 ceph-mon[31430]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T23:37:33.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:33 smithi178 ceph-mon[35882]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T23:37:33.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:33 smithi138 ceph-mon[35142]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T23:37:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.023+0000 7f6b8ad85640 1 -- 172.21.15.133:0/1362429995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b84106a60 msgr2=0x7f6b84106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.023+0000 7f6b8ad85640 1 --2- 172.21.15.133:0/1362429995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b84106a60 0x7f6b84106e40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009930 tx=0x7f6b7802a420 comp rx=0 tx=0).stop 2024-09-11T23:37:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.024+0000 7f6b8ad85640 1 -- 172.21.15.133:0/1362429995 shutdown_connections 2024-09-11T23:37:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.024+0000 7f6b8ad85640 1 --2- 172.21.15.133:0/1362429995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b84101400 0x7f6b8410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.024+0000 7f6b8ad85640 1 --2- 172.21.15.133:0/1362429995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b84100a60 0x7f6b84100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.024+0000 7f6b8ad85640 1 --2- 172.21.15.133:0/1362429995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b84106a60 0x7f6b84106e40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.024+0000 7f6b8ad85640 1 -- 172.21.15.133:0/1362429995 >> 172.21.15.133:0/1362429995 conn(0x7f6b840fc780 msgr2=0x7f6b840feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.024+0000 7f6b8ad85640 1 -- 172.21.15.133:0/1362429995 shutdown_connections 2024-09-11T23:37:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.024+0000 7f6b8ad85640 1 -- 172.21.15.133:0/1362429995 wait complete. 2024-09-11T23:37:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.024+0000 7f6b8ad85640 1 Processor -- start 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.024+0000 7f6b8ad85640 1 -- start start 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b8ad85640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b84100a60 0x7f6b841973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b8ad85640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b84101400 0x7f6b84197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b8ad85640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b84106a60 0x7f6b8419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b8ad85640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84110e60 con 0x7f6b84106a60 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b8ad85640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84110ce0 con 0x7f6b84100a60 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b8ad85640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84110fe0 con 0x7f6b84101400 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b89322640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b84106a60 0x7f6b8419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b89322640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b84106a60 0x7f6b8419bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33914/0 (socket says 172.21.15.133:33914) 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b89322640 1 -- 172.21.15.133:0/3306490476 learned_addr learned my addr 172.21.15.133:0/3306490476 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b83fff640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b84101400 0x7f6b84197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b88b21640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b84100a60 0x7f6b841973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b89322640 1 -- 172.21.15.133:0/3306490476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b84101400 msgr2=0x7f6b84197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:34.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b89322640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b84101400 0x7f6b84197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b89322640 1 -- 172.21.15.133:0/3306490476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b84100a60 msgr2=0x7f6b841973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:34.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b89322640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b84100a60 0x7f6b841973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.025+0000 7f6b89322640 1 -- 172.21.15.133:0/3306490476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8419c390 con 0x7f6b84106a60 2024-09-11T23:37:34.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.026+0000 7f6b89322640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b84106a60 0x7f6b8419bcb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00cad0 tx=0x7f6b7c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:34.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.026+0000 7f6b81ffb640 1 -- 172.21.15.133:0/3306490476 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00ae20 con 0x7f6b84106a60 2024-09-11T23:37:34.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.026+0000 7f6b81ffb640 1 -- 172.21.15.133:0/3306490476 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b7c005020 con 0x7f6b84106a60 2024-09-11T23:37:34.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.026+0000 7f6b8ad85640 1 -- 172.21.15.133:0/3306490476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8419c680 con 0x7f6b84106a60 2024-09-11T23:37:34.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.026+0000 7f6b81ffb640 1 -- 172.21.15.133:0/3306490476 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00f6f0 con 0x7f6b84106a60 2024-09-11T23:37:34.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.026+0000 7f6b8ad85640 1 -- 172.21.15.133:0/3306490476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b841a3f60 con 0x7f6b84106a60 2024-09-11T23:37:34.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.028+0000 7f6b81ffb640 1 -- 172.21.15.133:0/3306490476 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f6b7c0051d0 con 0x7f6b84106a60 2024-09-11T23:37:34.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b8d15e740 1 Processor -- start 2024-09-11T23:37:34.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b8d15e740 1 -- start start 2024-09-11T23:37:34.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b81ffb640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b740a2ea0 0x7f6b740a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad73090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad73a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad82970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55af1acd92f0 con 0x55af1ad74b40 2024-09-11T23:37:34.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55af1ac80cf0 con 0x55af1ad735d0 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55af1acd52b0 con 0x55af1ad80580 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b83fff640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b740a2ea0 0x7f6b740a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b88b21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad73090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b81ffb640 1 -- 172.21.15.133:0/3306490476 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6b7c0cf680 con 0x7f6b84106a60 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b88b21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad73090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33930/0 (socket says 172.21.15.133:33930) 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b88b21640 1 -- 172.21.15.133:0/1221592775 learned_addr learned my addr 172.21.15.133:0/1221592775 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.029+0000 7f6b83fff640 1 --2- 172.21.15.133:0/1221592775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad73a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.030+0000 7f6b89322640 1 --2- 172.21.15.133:0/1221592775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad82970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.030+0000 7f6b88b21640 1 -- 172.21.15.133:0/1221592775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 msgr2=0x55af1ad82970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.030+0000 7f6b88b21640 1 --2- 172.21.15.133:0/1221592775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad82970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.030+0000 7f6b88b21640 1 -- 172.21.15.133:0/1221592775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 msgr2=0x55af1ad73a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.030+0000 7f6b88b21640 1 --2- 172.21.15.133:0/1221592775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad73a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.030+0000 7f6b88b21640 1 -- 172.21.15.133:0/1221592775 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1ac653a0 con 0x55af1ad74b40 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.030+0000 7f6b83fff640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b740a2ea0 0x7f6b740a5360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000b620 tx=0x7f6b7000bb50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.030+0000 7f6b88b21640 1 --2- 172.21.15.133:0/1221592775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad73090 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009930 tx=0x7f6b78032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf8582a74e9a857 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.030+0000 7f6b6effd640 1 -- 172.21.15.133:0/1221592775 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7803f070 con 0x55af1ad74b40 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.030+0000 7f6b6effd640 1 -- 172.21.15.133:0/1221592775 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b780317b0 con 0x55af1ad74b40 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.030+0000 7f6b6effd640 1 -- 172.21.15.133:0/1221592775 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b78035070 con 0x55af1ad74b40 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.031+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1221592775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 msgr2=0x55af1ad73090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.031+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1221592775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad73090 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009930 tx=0x7f6b78032ce0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.031+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1221592775 shutdown_connections 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.031+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1221592775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad82970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.031+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1221592775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad73a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.031+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1221592775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad73090 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.031+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1221592775 >> 172.21.15.133:0/1221592775 conn(0x55af1ad3c310 msgr2=0x55af1ad6fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.031+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1221592775 shutdown_connections 2024-09-11T23:37:34.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.031+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1221592775 wait complete. 2024-09-11T23:37:34.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.031+0000 7f6b8d15e740 1 -- start start 2024-09-11T23:37:34.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b8d15e740 1 Processor -- start 2024-09-11T23:37:34.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad735d0 0x55af1ad7ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad74b40 0x55af1ad7d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad7d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55af1ad83220 con 0x55af1ad735d0 2024-09-11T23:37:34.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55af1ac94060 con 0x55af1ad74b40 2024-09-11T23:37:34.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55af1ad833a0 con 0x55af1ad80580 2024-09-11T23:37:34.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b88b21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad735d0 0x55af1ad7ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b88b21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad735d0 0x55af1ad7ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33940/0 (socket says 172.21.15.133:33940) 2024-09-11T23:37:34.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b88b21640 1 -- 172.21.15.133:0/1742399084 learned_addr learned my addr 172.21.15.133:0/1742399084 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:34.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b8d15e740 10 client.0 _ll_register_callbacks cb 0x55af1a9e78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:34.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b83fff640 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad74b40 0x55af1ad7d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b89322640 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad7d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.032+0000 7f6b88b21640 1 -- 172.21.15.133:0/1742399084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 msgr2=0x55af1ad7d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b88b21640 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad7d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b88b21640 1 -- 172.21.15.133:0/1742399084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad74b40 msgr2=0x55af1ad7d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b88b21640 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad74b40 0x55af1ad7d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b88b21640 1 -- 172.21.15.133:0/1742399084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af1aa4a3c0 con 0x55af1ad735d0 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b83fff640 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad74b40 0x55af1ad7d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b89322640 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad7d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b88b21640 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad735d0 0x55af1ad7ff30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6b7802a460 tx=0x7f6b78031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3936d7711a19d7a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b63fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b63fff640 1 -- 172.21.15.133:0/1742399084 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b78035070 con 0x55af1ad735d0 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b63fff640 1 -- 172.21.15.133:0/1742399084 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6b780048d0 con 0x55af1ad735d0 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1742399084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b780317b0 con 0x55af1ad735d0 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b63fff640 1 -- 172.21.15.133:0/1742399084 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b78055050 con 0x55af1ad735d0 2024-09-11T23:37:34.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b8d15e740 10 client.15669 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1742399084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af1ad8ec50 con 0x55af1ad735d0 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.033+0000 7f6b63fff640 1 -- 172.21.15.133:0/1742399084 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b780317b0 con 0x55af1ad735d0 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.034+0000 7f6b63fff640 1 -- 172.21.15.133:0/1742399084 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b7803bab0 con 0x55af1ad735d0 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.034+0000 7f6b8d15e740 10 client.15669 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.034+0000 7f6b8d15e740 10 client.15669 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.034+0000 7f6b8d15e740 20 client.15669 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.034+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55af1ad8ffa0 0x55af1ad92390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.034+0000 7f6b8d15e740 4 client.15669 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.034+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1742399084 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af1ace6860 con 0x55af1ad8ffa0 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.034+0000 7f6b83fff640 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55af1ad8ffa0 0x55af1ad92390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.034+0000 7f6b83fff640 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55af1ad8ffa0 0x55af1ad92390 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=84283a9ec9a2606b server_cookie=97501fba3d68022a in_seq=0 out_seq=0 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.034+0000 7f6b63fff640 10 client.15669 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b63fff640 1 -- 172.21.15.133:0/1742399084 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af1ace6860 con 0x55af1ad8ffa0 2024-09-11T23:37:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b63fff640 10 client.15669 handle_command_reply: tid=0 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b8d15e740 1 client.15669 shutdown 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1742399084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad735d0 msgr2=0x55af1ad7ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad735d0 0x55af1ad7ff30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6b7802a460 tx=0x7f6b78031510 comp rx=0 tx=0).stop 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1742399084 shutdown_connections 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55af1ad8ffa0 0x55af1ad92390 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad7d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b63fff640 0 client.15669 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad74b40 0x55af1ad7d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1742399084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad735d0 0x55af1ad7ff30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.035+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1742399084 >> 172.21.15.133:0/1742399084 conn(0x55af1ad3c310 msgr2=0x55af1ad75960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.036+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1742399084 shutdown_connections 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.036+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1742399084 wait complete. 2024-09-11T23:37:34.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.036+0000 7f6b8d15e740 20 client.15669 trim_cache size 0 max 16384 2024-09-11T23:37:34.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b8d15e740 1 Processor -- start 2024-09-11T23:37:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b8d15e740 1 -- start start 2024-09-11T23:37:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad8c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad74b40 0x55af1ad97990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad80580 0x55af1ad97ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841ab580 con 0x55af1ad80580 2024-09-11T23:37:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841ad0c0 con 0x55af1ad735d0 2024-09-11T23:37:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841abc50 con 0x55af1ad74b40 2024-09-11T23:37:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b89322640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad80580 0x55af1ad97ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b89322640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad80580 0x55af1ad97ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33944/0 (socket says 172.21.15.133:33944) 2024-09-11T23:37:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b89322640 1 -- 172.21.15.133:0/1555892353 learned_addr learned my addr 172.21.15.133:0/1555892353 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.051+0000 7f6b88b21640 1 --2- 172.21.15.133:0/1555892353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad8c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b83fff640 1 --2- 172.21.15.133:0/1555892353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad74b40 0x55af1ad97990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b89322640 1 -- 172.21.15.133:0/1555892353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad74b40 msgr2=0x55af1ad97990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b89322640 1 --2- 172.21.15.133:0/1555892353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad74b40 0x55af1ad97990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b89322640 1 -- 172.21.15.133:0/1555892353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 msgr2=0x55af1ad8c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b89322640 1 --2- 172.21.15.133:0/1555892353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad8c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b89322640 1 -- 172.21.15.133:0/1555892353 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1ad73d50 con 0x55af1ad80580 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b83fff640 1 --2- 172.21.15.133:0/1555892353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad74b40 0x55af1ad97990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b88b21640 1 --2- 172.21.15.133:0/1555892353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad8c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b89322640 1 --2- 172.21.15.133:0/1555892353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad80580 0x55af1ad97ed0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c09b6d0 tx=0x7f6b7c04a190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5579a59ef4eb9b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b6e7fc640 1 -- 172.21.15.133:0/1555892353 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c01d040 con 0x55af1ad80580 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b6e7fc640 1 -- 172.21.15.133:0/1555892353 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b7c09fd80 con 0x55af1ad80580 2024-09-11T23:37:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.052+0000 7f6b6e7fc640 1 -- 172.21.15.133:0/1555892353 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0860c0 con 0x55af1ad80580 2024-09-11T23:37:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.053+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1555892353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad80580 msgr2=0x55af1ad97ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.053+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1555892353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad80580 0x55af1ad97ed0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c09b6d0 tx=0x7f6b7c04a190 comp rx=0 tx=0).stop 2024-09-11T23:37:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.053+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1555892353 shutdown_connections 2024-09-11T23:37:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.053+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1555892353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad80580 0x55af1ad97ed0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.053+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1555892353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad74b40 0x55af1ad97990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.053+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/1555892353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad8c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.053+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1555892353 >> 172.21.15.133:0/1555892353 conn(0x55af1ad3c310 msgr2=0x55af1ad78520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.053+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1555892353 shutdown_connections 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.053+0000 7f6b8d15e740 1 -- 172.21.15.133:0/1555892353 wait complete. 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.054+0000 7f6b8d15e740 1 -- start start 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.054+0000 7f6b8d15e740 1 Processor -- start 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.054+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad88640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.054+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad88b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.054+0000 7f6b8d15e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.054+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841ac9f0 con 0x55af1ad74b40 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.054+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841ac320 con 0x55af1ad735d0 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.054+0000 7f6b8d15e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841a95a0 con 0x55af1ad80580 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.054+0000 7f6b8d15e740 10 client.0 _ll_register_callbacks cb 0x7f6b70012490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:34.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.054+0000 7f6b83fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad88b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b83fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad88b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33958/0 (socket says 172.21.15.133:33958) 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b83fff640 1 -- 172.21.15.133:0/3186026068 learned_addr learned my addr 172.21.15.133:0/3186026068 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b89322640 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b88b21640 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad88640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b83fff640 1 -- 172.21.15.133:0/3186026068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 msgr2=0x55af1ad835f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b83fff640 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad835f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b83fff640 1 -- 172.21.15.133:0/3186026068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 msgr2=0x55af1ad88640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b83fff640 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad88640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b83fff640 1 -- 172.21.15.133:0/3186026068 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af1ad891c0 con 0x55af1ad74b40 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b83fff640 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad88b80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6b70000c00 tx=0x7f6b7000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e367345d8bbedf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b63fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b63fff640 1 -- 172.21.15.133:0/3186026068 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b70030060 con 0x55af1ad74b40 2024-09-11T23:37:34.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b63fff640 1 -- 172.21.15.133:0/3186026068 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6b7001b040 con 0x55af1ad74b40 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.055+0000 7f6b8d15e740 1 -- 172.21.15.133:0/3186026068 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b7c09fd80 con 0x55af1ad74b40 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.056+0000 7f6b63fff640 1 -- 172.21.15.133:0/3186026068 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b70004e00 con 0x55af1ad74b40 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.056+0000 7f6b8d15e740 10 client.15675 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.056+0000 7f6b8d15e740 1 -- 172.21.15.133:0/3186026068 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af1adfd240 con 0x55af1ad74b40 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.056+0000 7f6b63fff640 1 -- 172.21.15.133:0/3186026068 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b7c09fd80 con 0x55af1ad74b40 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.056+0000 7f6b63fff640 1 -- 172.21.15.133:0/3186026068 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b70033020 con 0x55af1ad74b40 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.056+0000 7f6b8d15e740 10 client.15675 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.056+0000 7f6b8d15e740 10 client.15675 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.056+0000 7f6b8d15e740 20 client.15675 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.056+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55af1adfd9a0 0x55af1adffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.056+0000 7f6b8d15e740 4 client.15675 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.056+0000 7f6b8d15e740 1 -- 172.21.15.133:0/3186026068 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af1ace0d70 con 0x55af1adfd9a0 2024-09-11T23:37:34.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b88b21640 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55af1adfd9a0 0x55af1adffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:34.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b88b21640 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55af1adfd9a0 0x55af1adffdc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=befbdcce16ce5049 server_cookie=7b65c8bb4125bf85 in_seq=0 out_seq=0 2024-09-11T23:37:34.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b63fff640 10 client.15675 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:34.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b63fff640 1 -- 172.21.15.133:0/3186026068 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af1ace0d70 con 0x55af1adfd9a0 2024-09-11T23:37:34.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b63fff640 10 client.15675 handle_command_reply: tid=0 2024-09-11T23:37:34.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b8d15e740 1 client.15675 shutdown 2024-09-11T23:37:34.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b8d15e740 1 -- 172.21.15.133:0/3186026068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 msgr2=0x55af1ad88b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:34.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad88b80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6b70000c00 tx=0x7f6b7000b390 comp rx=0 tx=0).stop 2024-09-11T23:37:34.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b8d15e740 1 -- 172.21.15.133:0/3186026068 shutdown_connections 2024-09-11T23:37:34.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55af1adfd9a0 0x55af1adffdc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af1ad80580 0x55af1ad835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.057+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af1ad74b40 0x55af1ad88b80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.058+0000 7f6b8d15e740 1 --2- 172.21.15.133:0/3186026068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af1ad735d0 0x55af1ad88640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.058+0000 7f6b8d15e740 1 -- 172.21.15.133:0/3186026068 >> 172.21.15.133:0/3186026068 conn(0x55af1ad3c310 msgr2=0x55af1ad6ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:34.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.058+0000 7f6b63fff640 0 client.15675 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:34.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.058+0000 7f6b8d15e740 1 -- 172.21.15.133:0/3186026068 shutdown_connections 2024-09-11T23:37:34.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.058+0000 7f6b8d15e740 1 -- 172.21.15.133:0/3186026068 wait complete. 2024-09-11T23:37:34.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.058+0000 7f6b8d15e740 20 client.15675 trim_cache size 0 max 16384 2024-09-11T23:37:34.062 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:37:34.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.058+0000 7f6b8ad85640 1 -- 172.21.15.133:0/3306490476 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b740a2ea0 msgr2=0x7f6b740a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:34.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.058+0000 7f6b8ad85640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b740a2ea0 0x7f6b740a5360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000b620 tx=0x7f6b7000bb50 comp rx=0 tx=0).stop 2024-09-11T23:37:34.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.058+0000 7f6b8ad85640 1 -- 172.21.15.133:0/3306490476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b84106a60 msgr2=0x7f6b8419bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:34.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.058+0000 7f6b8ad85640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b84106a60 0x7f6b8419bcb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00cad0 tx=0x7f6b7c00cf90 comp rx=0 tx=0).stop 2024-09-11T23:37:34.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.059+0000 7f6b8ad85640 1 -- 172.21.15.133:0/3306490476 shutdown_connections 2024-09-11T23:37:34.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.059+0000 7f6b8ad85640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b740a2ea0 0x7f6b740a5360 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.059+0000 7f6b8ad85640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b84106a60 0x7f6b8419bcb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.059+0000 7f6b8ad85640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b84101400 0x7f6b84197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.059+0000 7f6b8ad85640 1 --2- 172.21.15.133:0/3306490476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b84100a60 0x7f6b841973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:34.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.059+0000 7f6b8ad85640 1 -- 172.21.15.133:0/3306490476 >> 172.21.15.133:0/3306490476 conn(0x7f6b840fc780 msgr2=0x7f6b840fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:34.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.059+0000 7f6b8ad85640 1 -- 172.21.15.133:0/3306490476 shutdown_connections 2024-09-11T23:37:34.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:34.059+0000 7f6b8ad85640 1 -- 172.21.15.133:0/3306490476 wait complete. 2024-09-11T23:37:34.724 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:37:34.958 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-11T23:37:34.959 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:37:34.981 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:35.257 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:35 smithi133 ceph-mon[31430]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T23:37:35.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:35 smithi138 ceph-mon[35142]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T23:37:35.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:35 smithi178 ceph-mon[35882]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T23:37:35.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.497+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1743149903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc4108290 msgr2=0x7f0dc410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:35.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.497+0000 7f0dc9d15640 1 --2- 172.21.15.133:0/1743149903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc4108290 0x7f0dc410ec30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009cd0 tx=0x7f0dbc02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.497+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1743149903 shutdown_connections 2024-09-11T23:37:35.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.497+0000 7f0dc9d15640 1 --2- 172.21.15.133:0/1743149903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc4108290 0x7f0dc410ec30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.497+0000 7f0dc9d15640 1 --2- 172.21.15.133:0/1743149903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc4103740 0x7f0dc4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.497+0000 7f0dc9d15640 1 --2- 172.21.15.133:0/1743149903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dc4102d90 0x7f0dc4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.497+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1743149903 >> 172.21.15.133:0/1743149903 conn(0x7f0dc40fc7d0 msgr2=0x7f0dc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.497+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1743149903 shutdown_connections 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.497+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1743149903 wait complete. 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.497+0000 7f0dc9d15640 1 Processor -- start 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc9d15640 1 -- start start 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc9d15640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc4102d90 0x7f0dc41985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc9d15640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc4103740 0x7f0dc4198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc9d15640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dc4108290 0x7f0dc419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc9d15640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4112090 con 0x7f0dc4102d90 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc9d15640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4111f10 con 0x7f0dc4108290 2024-09-11T23:37:35.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc9d15640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4112210 con 0x7f0dc4103740 2024-09-11T23:37:35.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc4102d90 0x7f0dc41985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc4102d90 0x7f0dc41985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33974/0 (socket says 172.21.15.133:33974) 2024-09-11T23:37:35.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc37fe640 1 -- 172.21.15.133:0/1616825543 learned_addr learned my addr 172.21.15.133:0/1616825543 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:35.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc2ffd640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc4103740 0x7f0dc4198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dc4108290 0x7f0dc419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc37fe640 1 -- 172.21.15.133:0/1616825543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc4103740 msgr2=0x7f0dc4198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:35.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc37fe640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc4103740 0x7f0dc4198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc37fe640 1 -- 172.21.15.133:0/1616825543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dc4108290 msgr2=0x7f0dc419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:35.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc37fe640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dc4108290 0x7f0dc419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc37fe640 1 -- 172.21.15.133:0/1616825543 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc419d590 con 0x7f0dc4102d90 2024-09-11T23:37:35.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.498+0000 7f0dc37fe640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc4102d90 0x7f0dc41985e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0db400d950 tx=0x7f0db400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.499+0000 7f0dc0ff9640 1 -- 172.21.15.133:0/1616825543 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4017070 con 0x7f0dc4102d90 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.499+0000 7f0dc0ff9640 1 -- 172.21.15.133:0/1616825543 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db4009c30 con 0x7f0dc4102d90 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.499+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1616825543 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc419d880 con 0x7f0dc4102d90 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.499+0000 7f0dc0ff9640 1 -- 172.21.15.133:0/1616825543 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db400f6b0 con 0x7f0dc4102d90 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.499+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1616825543 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc41a5160 con 0x7f0dc4102d90 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.501+0000 7f0dcc0ee740 1 Processor -- start 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.501+0000 7f0dcc0ee740 1 -- start start 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ecd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9ecd5d0 0x5645f9ecda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 0x5645f9edeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5645f9e332f0 con 0x5645f9eceb40 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5645f9ddacf0 con 0x5645f9edc780 2024-09-11T23:37:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5645f9e2f2b0 con 0x5645f9ecd5d0 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ecd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ecd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33976/0 (socket says 172.21.15.133:33976) 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dc3fff640 1 -- 172.21.15.133:0/3170583362 learned_addr learned my addr 172.21.15.133:0/3170583362 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dc0ff9640 1 -- 172.21.15.133:0/1616825543 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f0db40109d0 con 0x7f0dc4102d90 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dc3fff640 1 -- 172.21.15.133:0/3170583362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9ecd5d0 msgr2=0x5645f9ecda10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dc2ffd640 1 --2- 172.21.15.133:0/3170583362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 0x5645f9edeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/3170583362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9ecd5d0 0x5645f9ecda10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dc3fff640 1 -- 172.21.15.133:0/3170583362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 msgr2=0x5645f9edeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/3170583362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 0x5645f9edeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dc3fff640 1 -- 172.21.15.133:0/3170583362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645f9dbf3a0 con 0x5645f9eceb40 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.502+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/3170583362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ecd090 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc004ab0 tx=0x7f0dbc009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fcb09cdd6ddc3d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.503+0000 7f0da9ffb640 1 -- 172.21.15.133:0/3170583362 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc048070 con 0x5645f9eceb40 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.503+0000 7f0da9ffb640 1 -- 172.21.15.133:0/3170583362 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dbc036070 con 0x5645f9eceb40 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.503+0000 7f0da9ffb640 1 -- 172.21.15.133:0/3170583362 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc004580 con 0x5645f9eceb40 2024-09-11T23:37:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.503+0000 7f0dc0ff9640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0dac0a2ea0 0x7f0dac0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.503+0000 7f0dc37fe640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0dac0a2ea0 0x7f0dac0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.503+0000 7f0dc0ff9640 1 -- 172.21.15.133:0/1616825543 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0db40cf2a0 con 0x7f0dc4102d90 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.503+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/3170583362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 msgr2=0x5645f9ecd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.503+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/3170583362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ecd090 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc004ab0 tx=0x7f0dbc009a20 comp rx=0 tx=0).stop 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.503+0000 7f0dc37fe640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0dac0a2ea0 0x7f0dac0a5360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0db40cc880 tx=0x7f0db409e040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.504+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/3170583362 shutdown_connections 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.504+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/3170583362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 0x5645f9edeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.504+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/3170583362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9ecd5d0 0x5645f9ecda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.504+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/3170583362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ecd090 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.504+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/3170583362 >> 172.21.15.133:0/3170583362 conn(0x5645f9e96310 msgr2=0x5645f9ec9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.504+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/3170583362 shutdown_connections 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.504+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/3170583362 wait complete. 2024-09-11T23:37:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.504+0000 7f0dcc0ee740 1 -- start start 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.504+0000 7f0dcc0ee740 1 Processor -- start 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9edc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9eceb40 0x5645f9ed94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 0x5645f9ed9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5645f9edf420 con 0x5645f9ecd5d0 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5645f9dee060 con 0x5645f9edc780 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5645f9edf5a0 con 0x5645f9eceb40 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dcc0ee740 10 client.0 _ll_register_callbacks cb 0x5645f9b418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9edc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9edc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33992/0 (socket says 172.21.15.133:33992) 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dc3fff640 1 -- 172.21.15.133:0/1075855851 learned_addr learned my addr 172.21.15.133:0/1075855851 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dc2ffd640 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9eceb40 0x5645f9ed94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 0x5645f9ed9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dc3fff640 1 -- 172.21.15.133:0/1075855851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9eceb40 msgr2=0x5645f9ed94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9eceb40 0x5645f9ed94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dc3fff640 1 -- 172.21.15.133:0/1075855851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 msgr2=0x5645f9ed9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 0x5645f9ed9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dc3fff640 1 -- 172.21.15.133:0/1075855851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645f9ba43c0 con 0x5645f9ecd5d0 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9edc230 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009620 tx=0x7f0dbc0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab89c19793e53970 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.505+0000 7f0da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0da2ffd640 1 -- 172.21.15.133:0/1075855851 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc054020 con 0x5645f9ecd5d0 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0da2ffd640 1 -- 172.21.15.133:0/1075855851 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0dbc004280 con 0x5645f9ecd5d0 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1075855851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dbc036070 con 0x5645f9ecd5d0 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0da2ffd640 1 -- 172.21.15.133:0/1075855851 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0dbc05f050 con 0x5645f9ecd5d0 2024-09-11T23:37:35.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0dcc0ee740 10 client.15687 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1075855851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645f9edf420 con 0x5645f9ecd5d0 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0da2ffd640 1 -- 172.21.15.133:0/1075855851 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dbc036070 con 0x5645f9ecd5d0 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0da2ffd640 1 -- 172.21.15.133:0/1075855851 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0dbc042030 con 0x5645f9ecd5d0 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0dcc0ee740 10 client.15687 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0dcc0ee740 10 client.15687 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0dcc0ee740 20 client.15687 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5645f9eec3b0 0x5645f9eee7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0dcc0ee740 4 client.15687 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.506+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1075855851 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645f9e40860 con 0x5645f9eec3b0 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.507+0000 7f0dc2ffd640 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5645f9eec3b0 0x5645f9eee7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.507+0000 7f0da2ffd640 10 client.15687 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.507+0000 7f0dc2ffd640 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5645f9eec3b0 0x5645f9eee7d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=febf9ffdb24710de server_cookie=8df208616242add4 in_seq=0 out_seq=0 2024-09-11T23:37:35.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0da2ffd640 1 -- 172.21.15.133:0/1075855851 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645f9e40860 con 0x5645f9eec3b0 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0da2ffd640 10 client.15687 handle_command_reply: tid=0 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 1 client.15687 shutdown 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1075855851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 msgr2=0x5645f9edc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9edc230 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009620 tx=0x7f0dbc0040e0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1075855851 shutdown_connections 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5645f9eec3b0 0x5645f9eee7d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 0x5645f9ed9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9eceb40 0x5645f9ed94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/1075855851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9edc230 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1075855851 >> 172.21.15.133:0/1075855851 conn(0x5645f9e96310 msgr2=0x5645f9ecf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0da2ffd640 0 client.15687 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1075855851 shutdown_connections 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1075855851 wait complete. 2024-09-11T23:37:35.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.508+0000 7f0dcc0ee740 20 client.15687 trim_cache size 0 max 16384 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.521+0000 7f0dcc0ee740 1 Processor -- start 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dcc0ee740 1 -- start start 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9ecd5d0 0x5645f9ef40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ef4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9edc780 0x5645f9ec9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc41ab490 con 0x5645f9eceb40 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc41acfd0 con 0x5645f9ecd5d0 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc41abb60 con 0x5645f9edc780 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ef4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ef4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34000/0 (socket says 172.21.15.133:34000) 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc2ffd640 1 -- 172.21.15.133:0/1280301477 learned_addr learned my addr 172.21.15.133:0/1280301477 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/1280301477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9ecd5d0 0x5645f9ef40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/1280301477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9edc780 0x5645f9ec9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc2ffd640 1 -- 172.21.15.133:0/1280301477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9edc780 msgr2=0x5645f9ec9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:35.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc2ffd640 1 --2- 172.21.15.133:0/1280301477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9edc780 0x5645f9ec9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc2ffd640 1 -- 172.21.15.133:0/1280301477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9ecd5d0 msgr2=0x5645f9ef40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:35.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc2ffd640 1 --2- 172.21.15.133:0/1280301477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9ecd5d0 0x5645f9ef40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc2ffd640 1 -- 172.21.15.133:0/1280301477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645f9ed0120 con 0x5645f9eceb40 2024-09-11T23:37:35.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/1280301477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9edc780 0x5645f9ec9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:35.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.522+0000 7f0dc2ffd640 1 --2- 172.21.15.133:0/1280301477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ef4600 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40feda0 tx=0x7f0db800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd48d1ec3aac9ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:35.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.523+0000 7f0da97fa640 1 -- 172.21.15.133:0/1280301477 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8003ca0 con 0x5645f9eceb40 2024-09-11T23:37:35.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.523+0000 7f0da97fa640 1 -- 172.21.15.133:0/1280301477 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db8003e40 con 0x5645f9eceb40 2024-09-11T23:37:35.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.523+0000 7f0da97fa640 1 -- 172.21.15.133:0/1280301477 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8011900 con 0x5645f9eceb40 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.523+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1280301477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 msgr2=0x5645f9ef4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.523+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/1280301477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ef4600 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40feda0 tx=0x7f0db800b390 comp rx=0 tx=0).stop 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.523+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1280301477 shutdown_connections 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.523+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/1280301477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9edc780 0x5645f9ec9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.523+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/1280301477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9eceb40 0x5645f9ef4600 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.523+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/1280301477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9ecd5d0 0x5645f9ef40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.523+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1280301477 >> 172.21.15.133:0/1280301477 conn(0x5645f9e96310 msgr2=0x5645f9ec4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.524+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1280301477 shutdown_connections 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.524+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/1280301477 wait complete. 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.524+0000 7f0dcc0ee740 1 -- start start 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.524+0000 7f0dcc0ee740 1 Processor -- start 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9ee4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9eceb40 0x5645f9ee4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dcc0ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 0x5645f9ed57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc41ac900 con 0x5645f9ecd5d0 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc41ac230 con 0x5645f9edc780 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dcc0ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc41a94f0 con 0x5645f9eceb40 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dcc0ee740 10 client.0 _ll_register_callbacks cb 0x7f0dbc046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9ee4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9ee4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34002/0 (socket says 172.21.15.133:34002) 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dc3fff640 1 -- 172.21.15.133:0/981158259 learned_addr learned my addr 172.21.15.133:0/981158259 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dc2ffd640 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9eceb40 0x5645f9ee4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dc3fff640 1 -- 172.21.15.133:0/981158259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9eceb40 msgr2=0x5645f9ee4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9eceb40 0x5645f9ee4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dc3fff640 1 -- 172.21.15.133:0/981158259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 msgr2=0x5645f9ed57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 0x5645f9ed57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dc3fff640 1 -- 172.21.15.133:0/981158259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645f9ee54b0 con 0x5645f9ecd5d0 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dc3fff640 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9ee4830 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc002230 tx=0x7f0dbc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b1fe7ba6ae0e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0da2ffd640 1 -- 172.21.15.133:0/981158259 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc054030 con 0x5645f9ecd5d0 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0da2ffd640 1 -- 172.21.15.133:0/981158259 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0dbc003b80 con 0x5645f9ecd5d0 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.525+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/981158259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db8003e40 con 0x5645f9ecd5d0 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0da2ffd640 1 -- 172.21.15.133:0/981158259 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0dbc048040 con 0x5645f9ecd5d0 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0dcc0ee740 10 client.15693 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/981158259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645f9f565e0 con 0x5645f9ecd5d0 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0da2ffd640 1 -- 172.21.15.133:0/981158259 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db8003e40 con 0x5645f9ecd5d0 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0da2ffd640 1 -- 172.21.15.133:0/981158259 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0dbc042040 con 0x5645f9ecd5d0 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0dcc0ee740 10 client.15693 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0dcc0ee740 10 client.15693 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0dcc0ee740 20 client.15693 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:35.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5645f9f56ca0 0x5645f9f590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0dcc0ee740 4 client.15693 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/981158259 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645f9e3ad70 con 0x5645f9f56ca0 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.526+0000 7f0dc2ffd640 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5645f9f56ca0 0x5645f9f590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0da2ffd640 10 client.15693 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0dc2ffd640 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5645f9f56ca0 0x5645f9f590c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb56e5dc7ec20ac6 server_cookie=bb412f99fc365dc8 in_seq=0 out_seq=0 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0da2ffd640 1 -- 172.21.15.133:0/981158259 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645f9e3ad70 con 0x5645f9f56ca0 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0da2ffd640 10 client.15693 handle_command_reply: tid=0 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0dcc0ee740 1 client.15693 shutdown 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/981158259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 msgr2=0x5645f9ee4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9ee4830 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc002230 tx=0x7f0dbc009650 comp rx=0 tx=0).stop 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/981158259 shutdown_connections 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5645f9f56ca0 0x5645f9f590c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645f9edc780 0x5645f9ed57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5645f9eceb40 0x5645f9ee4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0dcc0ee740 1 --2- 172.21.15.133:0/981158259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645f9ecd5d0 0x5645f9ee4830 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.527+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/981158259 >> 172.21.15.133:0/981158259 conn(0x5645f9e96310 msgr2=0x5645f9ed7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:35.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0da2ffd640 0 client.15693 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:35.558 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:37:35.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/981158259 shutdown_connections 2024-09-11T23:37:35.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dcc0ee740 1 -- 172.21.15.133:0/981158259 wait complete. 2024-09-11T23:37:35.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dcc0ee740 20 client.15693 trim_cache size 0 max 16384 2024-09-11T23:37:35.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1616825543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0dac0a2ea0 msgr2=0x7f0dac0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:35.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dc9d15640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0dac0a2ea0 0x7f0dac0a5360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0db40cc880 tx=0x7f0db409e040 comp rx=0 tx=0).stop 2024-09-11T23:37:35.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1616825543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc4102d90 msgr2=0x7f0dc41985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:35.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dc9d15640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc4102d90 0x7f0dc41985e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0db400d950 tx=0x7f0db400de10 comp rx=0 tx=0).stop 2024-09-11T23:37:35.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1616825543 shutdown_connections 2024-09-11T23:37:35.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dc9d15640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0dac0a2ea0 0x7f0dac0a5360 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dc9d15640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dc4108290 0x7f0dc419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dc9d15640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dc4103740 0x7f0dc4198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dc9d15640 1 --2- 172.21.15.133:0/1616825543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc4102d90 0x7f0dc41985e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:35.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.528+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1616825543 >> 172.21.15.133:0/1616825543 conn(0x7f0dc40fc7d0 msgr2=0x7f0dc4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:35.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.529+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1616825543 shutdown_connections 2024-09-11T23:37:35.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:35.529+0000 7f0dc9d15640 1 -- 172.21.15.133:0/1616825543 wait complete. 2024-09-11T23:37:37.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:37 smithi138 ceph-mon[35142]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:37:37.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:37 smithi133 ceph-mon[31430]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:37:37.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:37 smithi178 ceph-mon[35882]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:37:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.604+0000 7f0d9fce6640 1 -- 172.21.15.133:0/3522304304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d981009b0 msgr2=0x7f0d98100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.604+0000 7f0d9fce6640 1 --2- 172.21.15.133:0/3522304304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d981009b0 0x7f0d98100e10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009930 tx=0x7f0d8802a420 comp rx=0 tx=0).stop 2024-09-11T23:37:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.604+0000 7f0d9fce6640 1 -- 172.21.15.133:0/3522304304 shutdown_connections 2024-09-11T23:37:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.604+0000 7f0d9fce6640 1 --2- 172.21.15.133:0/3522304304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d98101350 0x7f0d9810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.604+0000 7f0d9fce6640 1 --2- 172.21.15.133:0/3522304304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d981009b0 0x7f0d98100e10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.604+0000 7f0d9fce6640 1 --2- 172.21.15.133:0/3522304304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d981069c0 0x7f0d98106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.604+0000 7f0d9fce6640 1 -- 172.21.15.133:0/3522304304 >> 172.21.15.133:0/3522304304 conn(0x7f0d98077990 msgr2=0x7f0d980fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.604+0000 7f0d9fce6640 1 -- 172.21.15.133:0/3522304304 shutdown_connections 2024-09-11T23:37:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.604+0000 7f0d9fce6640 1 -- 172.21.15.133:0/3522304304 wait complete. 2024-09-11T23:37:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9fce6640 1 Processor -- start 2024-09-11T23:37:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9fce6640 1 -- start start 2024-09-11T23:37:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9fce6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d981009b0 0x7f0d981982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9fce6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d98101350 0x7f0d98198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9fce6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d981069c0 0x7f0d9819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9fce6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98110e50 con 0x7f0d981009b0 2024-09-11T23:37:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9fce6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98110cd0 con 0x7f0d98101350 2024-09-11T23:37:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9fce6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98110fd0 con 0x7f0d981069c0 2024-09-11T23:37:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9da82640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d981009b0 0x7f0d981982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9da82640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d981009b0 0x7f0d981982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34008/0 (socket says 172.21.15.133:34008) 2024-09-11T23:37:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9da82640 1 -- 172.21.15.133:0/2584786500 learned_addr learned my addr 172.21.15.133:0/2584786500 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9e283640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d981069c0 0x7f0d9819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.605+0000 7f0d9d281640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d98101350 0x7f0d98198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.606+0000 7f0d9da82640 1 -- 172.21.15.133:0/2584786500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d981069c0 msgr2=0x7f0d9819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.606+0000 7f0d9da82640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d981069c0 0x7f0d9819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.606+0000 7f0d9da82640 1 -- 172.21.15.133:0/2584786500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d98101350 msgr2=0x7f0d98198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.606+0000 7f0d9da82640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d98101350 0x7f0d98198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.606+0000 7f0d9da82640 1 -- 172.21.15.133:0/2584786500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9819d2e0 con 0x7f0d981009b0 2024-09-11T23:37:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.606+0000 7f0d9da82640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d981009b0 0x7f0d981982f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400d770 tx=0x7f0d9400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.606+0000 7f0d8effd640 1 -- 172.21.15.133:0/2584786500 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94017070 con 0x7f0d981009b0 2024-09-11T23:37:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.606+0000 7f0d9fce6640 1 -- 172.21.15.133:0/2584786500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9819d5d0 con 0x7f0d981009b0 2024-09-11T23:37:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.606+0000 7f0d8effd640 1 -- 172.21.15.133:0/2584786500 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d94005500 con 0x7f0d981009b0 2024-09-11T23:37:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.606+0000 7f0d8effd640 1 -- 172.21.15.133:0/2584786500 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94004e00 con 0x7f0d981009b0 2024-09-11T23:37:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.606+0000 7f0d9fce6640 1 -- 172.21.15.133:0/2584786500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d981a4e60 con 0x7f0d981009b0 2024-09-11T23:37:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.609+0000 7f0da2a7f740 1 Processor -- start 2024-09-11T23:37:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.609+0000 7f0da2a7f740 1 -- start start 2024-09-11T23:37:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.609+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.609+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 0x55f9bb4fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.609+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb5000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.609+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb4572f0 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.609+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb3ff270 con 0x55f9bb4f2af0 2024-09-11T23:37:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.609+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb4532b0 con 0x55f9bb4f3410 2024-09-11T23:37:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.609+0000 7f0d9d281640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb5000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.609+0000 7f0d9d281640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb5000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34010/0 (socket says 172.21.15.133:34010) 2024-09-11T23:37:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d9d281640 1 -- 172.21.15.133:0/2666737454 learned_addr learned my addr 172.21.15.133:0/2666737454 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d8effd640 1 -- 172.21.15.133:0/2584786500 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f0d94004fa0 con 0x7f0d981009b0 2024-09-11T23:37:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d9e283640 1 --2- 172.21.15.133:0/2666737454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d9d281640 1 -- 172.21.15.133:0/2666737454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 msgr2=0x55f9bb4fd770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d9d281640 1 --2- 172.21.15.133:0/2666737454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 0x55f9bb4fd770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d9d281640 1 -- 172.21.15.133:0/2666737454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 msgr2=0x55f9bb4f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d9d281640 1 --2- 172.21.15.133:0/2666737454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d9d281640 1 -- 172.21.15.133:0/2666737454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bb3fe340 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d9e283640 1 --2- 172.21.15.133:0/2666737454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d9d281640 1 --2- 172.21.15.133:0/2666737454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb5000a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009930 tx=0x7f0d88004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d90a8da123a053c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d83fff640 1 -- 172.21.15.133:0/2666737454 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88035040 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d83fff640 1 -- 172.21.15.133:0/2666737454 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d88005520 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.610+0000 7f0d83fff640 1 -- 172.21.15.133:0/2666737454 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88003850 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0d8effd640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0d840a2f50 0x7f0d840a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0d9e283640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0d840a2f50 0x7f0d840a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0d8effd640 1 -- 172.21.15.133:0/2584786500 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0d940cefe0 con 0x7f0d981009b0 2024-09-11T23:37:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2666737454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 msgr2=0x55f9bb5000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/2666737454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb5000a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009930 tx=0x7f0d88004180 comp rx=0 tx=0).stop 2024-09-11T23:37:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0d9e283640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0d840a2f50 0x7f0d840a5410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d90007f90 tx=0x7f0d90007520 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2666737454 shutdown_connections 2024-09-11T23:37:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/2666737454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb5000a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/2666737454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 0x55f9bb4fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/2666737454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2666737454 >> 172.21.15.133:0/2666737454 conn(0x55f9bb4ef060 msgr2=0x55f9bb4ef440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2666737454 shutdown_connections 2024-09-11T23:37:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.611+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2666737454 wait complete. 2024-09-11T23:37:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.612+0000 7f0da2a7f740 1 -- start start 2024-09-11T23:37:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.612+0000 7f0da2a7f740 1 Processor -- start 2024-09-11T23:37:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.612+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4fac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.612+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 0x55f9bb4fb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.612+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4fb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.612+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb500880 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.612+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb3e0290 con 0x55f9bb4f2af0 2024-09-11T23:37:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.612+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb500a00 con 0x55f9bb4f3410 2024-09-11T23:37:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.612+0000 7f0da2a7f740 10 client.0 _ll_register_callbacks cb 0x55f9bb411140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.612+0000 7f0d9e283640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4fb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d9e283640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4fb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34012/0 (socket says 172.21.15.133:34012) 2024-09-11T23:37:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d9e283640 1 -- 172.21.15.133:0/2026084955 learned_addr learned my addr 172.21.15.133:0/2026084955 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d9da82640 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4fac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d9d281640 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 0x55f9bb4fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d9e283640 1 -- 172.21.15.133:0/2026084955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 msgr2=0x55f9bb4fb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d9e283640 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 0x55f9bb4fb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d9e283640 1 -- 172.21.15.133:0/2026084955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 msgr2=0x55f9bb4fac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d9e283640 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4fac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d9e283640 1 -- 172.21.15.133:0/2026084955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bb16a210 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d9e283640 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4fb6d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000fc20 tx=0x7f0d90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a92245416d62b3f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d817fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d817fa640 1 -- 172.21.15.133:0/2026084955 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9002b060 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d817fa640 1 -- 172.21.15.133:0/2026084955 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0d9001cd10 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2026084955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d88005520 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0d817fa640 1 -- 172.21.15.133:0/2026084955 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d90003970 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0da2a7f740 10 client.15705 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.613+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2026084955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9bb236620 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.614+0000 7f0d817fa640 1 -- 172.21.15.133:0/2026084955 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d88005520 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.614+0000 7f0d817fa640 1 -- 172.21.15.133:0/2026084955 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0d9002e030 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.614+0000 7f0da2a7f740 10 client.15705 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.614+0000 7f0da2a7f740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.614+0000 7f0da2a7f740 20 client.15705 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.614+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f9bb50dd10 0x55f9bb510100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.614+0000 7f0da2a7f740 4 client.15705 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.614+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2026084955 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9bb464a20 con 0x55f9bb50dd10 2024-09-11T23:37:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.614+0000 7f0d9da82640 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f9bb50dd10 0x55f9bb510100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.615+0000 7f0d817fa640 10 client.15705 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.615+0000 7f0d9da82640 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f9bb50dd10 0x55f9bb510100 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47c6ec2d8a893b1 server_cookie=5dc676db2a3214bd in_seq=0 out_seq=0 2024-09-11T23:37:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.615+0000 7f0d817fa640 1 -- 172.21.15.133:0/2026084955 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9bb464a20 con 0x55f9bb50dd10 2024-09-11T23:37:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.615+0000 7f0d817fa640 10 client.15705 handle_command_reply: tid=0 2024-09-11T23:37:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.615+0000 7f0da2a7f740 1 client.15705 shutdown 2024-09-11T23:37:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2026084955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 msgr2=0x55f9bb4fb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4fb6d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000fc20 tx=0x7f0d90004a40 comp rx=0 tx=0).stop 2024-09-11T23:37:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2026084955 shutdown_connections 2024-09-11T23:37:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f9bb50dd10 0x55f9bb510100 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4fb6d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0d817fa640 0 client.15705 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 0x55f9bb4fb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/2026084955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4fac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2026084955 >> 172.21.15.133:0/2026084955 conn(0x55f9bb4ef060 msgr2=0x55f9bb4f4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2026084955 shutdown_connections 2024-09-11T23:37:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0da2a7f740 1 -- 172.21.15.133:0/2026084955 wait complete. 2024-09-11T23:37:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.616+0000 7f0da2a7f740 20 client.15705 trim_cache size 0 max 16384 2024-09-11T23:37:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.630+0000 7f0da2a7f740 1 Processor -- start 2024-09-11T23:37:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.630+0000 7f0da2a7f740 1 -- start start 2024-09-11T23:37:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.630+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4fb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.630+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f3410 0x55f9bb515a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.630+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb515f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.630+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d981ad5f0 con 0x55f9bb4f2af0 2024-09-11T23:37:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.630+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d981af130 con 0x55f9bb4f3410 2024-09-11T23:37:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.630+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d981adcc0 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d9da82640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4fb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d9da82640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4fb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34022/0 (socket says 172.21.15.133:34022) 2024-09-11T23:37:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d9da82640 1 -- 172.21.15.133:0/1896667365 learned_addr learned my addr 172.21.15.133:0/1896667365 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d9d281640 1 --2- 172.21.15.133:0/1896667365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f3410 0x55f9bb515a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d9e283640 1 --2- 172.21.15.133:0/1896667365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb515f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d9da82640 1 -- 172.21.15.133:0/1896667365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4fdcb0 msgr2=0x55f9bb515f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d9da82640 1 --2- 172.21.15.133:0/1896667365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb515f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d9da82640 1 -- 172.21.15.133:0/1896667365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f3410 msgr2=0x55f9bb515a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d9da82640 1 --2- 172.21.15.133:0/1896667365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f3410 0x55f9bb515a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d9da82640 1 -- 172.21.15.133:0/1896667365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bb4fd7e0 con 0x55f9bb4f2af0 2024-09-11T23:37:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d9da82640 1 --2- 172.21.15.133:0/1896667365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4fb500 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0d940cc6a0 tx=0x7f0d940048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe4778ea13c327c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d837fe640 1 -- 172.21.15.133:0/1896667365 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d940b4070 con 0x55f9bb4f2af0 2024-09-11T23:37:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d837fe640 1 -- 172.21.15.133:0/1896667365 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9409f6e0 con 0x55f9bb4f2af0 2024-09-11T23:37:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.631+0000 7f0d837fe640 1 -- 172.21.15.133:0/1896667365 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94020040 con 0x55f9bb4f2af0 2024-09-11T23:37:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.632+0000 7f0da2a7f740 1 -- 172.21.15.133:0/1896667365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4f2af0 msgr2=0x55f9bb4fb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.632+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/1896667365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4fb500 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0d940cc6a0 tx=0x7f0d940048a0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.632+0000 7f0da2a7f740 1 -- 172.21.15.133:0/1896667365 shutdown_connections 2024-09-11T23:37:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.632+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/1896667365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb515f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.632+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/1896667365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f3410 0x55f9bb515a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.632+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/1896667365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4fb500 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.632+0000 7f0da2a7f740 1 -- 172.21.15.133:0/1896667365 >> 172.21.15.133:0/1896667365 conn(0x55f9bb4ef060 msgr2=0x55f9bb4fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.632+0000 7f0da2a7f740 1 -- 172.21.15.133:0/1896667365 shutdown_connections 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.632+0000 7f0da2a7f740 1 -- 172.21.15.133:0/1896667365 wait complete. 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0da2a7f740 1 -- start start 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0da2a7f740 1 Processor -- start 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 0x55f9bb4f8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0da2a7f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d981aea60 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d981ae390 con 0x55f9bb4f2af0 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0da2a7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d981ab5d0 con 0x55f9bb4f3410 2024-09-11T23:37:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0da2a7f740 10 client.0 _ll_register_callbacks cb 0x7f0d90007be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0d9e283640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0d9e283640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4f85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34032/0 (socket says 172.21.15.133:34032) 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0d9e283640 1 -- 172.21.15.133:0/4067329212 learned_addr learned my addr 172.21.15.133:0/4067329212 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0d9da82640 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.633+0000 7f0d9d281640 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 0x55f9bb4f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d9e283640 1 -- 172.21.15.133:0/4067329212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 msgr2=0x55f9bb4f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d9e283640 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 0x55f9bb4f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d9e283640 1 -- 172.21.15.133:0/4067329212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 msgr2=0x55f9bb4f7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d9e283640 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4f7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d9e283640 1 -- 172.21.15.133:0/4067329212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bb4fba40 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d9da82640 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d9e283640 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4f85b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000fbf0 tx=0x7f0d9001cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3666809430c394 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d817fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d817fa640 1 -- 172.21.15.133:0/4067329212 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9001c8a0 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d817fa640 1 -- 172.21.15.133:0/4067329212 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0d90026a80 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0da2a7f740 1 -- 172.21.15.133:0/4067329212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d9409f6e0 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d817fa640 1 -- 172.21.15.133:0/4067329212 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d9002b030 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0da2a7f740 10 client.15711 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0da2a7f740 1 -- 172.21.15.133:0/4067329212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9bb578a00 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.634+0000 7f0d817fa640 1 -- 172.21.15.133:0/4067329212 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9409f6e0 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.635+0000 7f0d817fa640 1 -- 172.21.15.133:0/4067329212 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0d90041450 con 0x55f9bb4fdcb0 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.635+0000 7f0da2a7f740 10 client.15711 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.635+0000 7f0da2a7f740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.635+0000 7f0da2a7f740 20 client.15711 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.635+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f9bb50dd10 0x55f9bb57b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.635+0000 7f0da2a7f740 4 client.15711 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:37:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.635+0000 7f0da2a7f740 1 -- 172.21.15.133:0/4067329212 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9bb45ed70 con 0x55f9bb50dd10 2024-09-11T23:37:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.635+0000 7f0d9da82640 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f9bb50dd10 0x55f9bb57b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.635+0000 7f0d817fa640 10 client.15711 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.635+0000 7f0d9da82640 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f9bb50dd10 0x55f9bb57b180 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf96d18a2efd133 server_cookie=24497d469859f266 in_seq=0 out_seq=0 2024-09-11T23:37:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0d817fa640 1 -- 172.21.15.133:0/4067329212 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9bb45ed70 con 0x55f9bb50dd10 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0d817fa640 10 client.15711 handle_command_reply: tid=0 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 1 client.15711 shutdown 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 1 -- 172.21.15.133:0/4067329212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 msgr2=0x55f9bb4f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4f85b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000fbf0 tx=0x7f0d9001cdc0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 1 -- 172.21.15.133:0/4067329212 shutdown_connections 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f9bb50dd10 0x55f9bb57b180 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9bb4fdcb0 0x55f9bb4f85b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f9bb4f3410 0x55f9bb4f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 1 --2- 172.21.15.133:0/4067329212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9bb4f2af0 0x55f9bb4f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0d817fa640 0 client.15711 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 1 -- 172.21.15.133:0/4067329212 >> 172.21.15.133:0/4067329212 conn(0x55f9bb4ef060 msgr2=0x55f9bb4ed9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 1 -- 172.21.15.133:0/4067329212 shutdown_connections 2024-09-11T23:37:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 1 -- 172.21.15.133:0/4067329212 wait complete. 2024-09-11T23:37:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.639+0000 7f0da2a7f740 20 client.15711 trim_cache size 0 max 16384 2024-09-11T23:37:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:37:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:37:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:37:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:37:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 -- 172.21.15.133:0/2584786500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0d840a2f50 msgr2=0x7f0d840a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0d840a2f50 0x7f0d840a5410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d90007f90 tx=0x7f0d90007520 comp rx=0 tx=0).stop 2024-09-11T23:37:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 -- 172.21.15.133:0/2584786500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d981009b0 msgr2=0x7f0d981982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d981009b0 0x7f0d981982f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400d770 tx=0x7f0d9400dc30 comp rx=0 tx=0).stop 2024-09-11T23:37:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 -- 172.21.15.133:0/2584786500 shutdown_connections 2024-09-11T23:37:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0d840a2f50 0x7f0d840a5410 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d981069c0 0x7f0d9819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d98101350 0x7f0d98198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 --2- 172.21.15.133:0/2584786500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d981009b0 0x7f0d981982f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 -- 172.21.15.133:0/2584786500 >> 172.21.15.133:0/2584786500 conn(0x7f0d98077990 msgr2=0x7f0d980ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 -- 172.21.15.133:0/2584786500 shutdown_connections 2024-09-11T23:37:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:38.640+0000 7f0d9fce6640 1 -- 172.21.15.133:0/2584786500 wait complete. 2024-09-11T23:37:39.015 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:39.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:37:40.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:40 smithi138 ceph-mon[35142]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T23:37:40.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:40 smithi133 ceph-mon[31430]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T23:37:40.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:40 smithi178 ceph-mon[35882]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T23:37:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.919+0000 7f0cabe74640 1 -- 172.21.15.133:0/4204736123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca4100a60 msgr2=0x7f0ca4100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.919+0000 7f0cabe74640 1 --2- 172.21.15.133:0/4204736123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca4100a60 0x7f0ca4100ec0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009930 tx=0x7f0c9402a420 comp rx=0 tx=0).stop 2024-09-11T23:37:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.919+0000 7f0cabe74640 1 -- 172.21.15.133:0/4204736123 shutdown_connections 2024-09-11T23:37:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.919+0000 7f0cabe74640 1 --2- 172.21.15.133:0/4204736123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca4101400 0x7f0ca410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.919+0000 7f0cabe74640 1 --2- 172.21.15.133:0/4204736123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca4100a60 0x7f0ca4100ec0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.919+0000 7f0cabe74640 1 --2- 172.21.15.133:0/4204736123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ca4106a60 0x7f0ca4106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.919+0000 7f0cabe74640 1 -- 172.21.15.133:0/4204736123 >> 172.21.15.133:0/4204736123 conn(0x7f0ca40fc780 msgr2=0x7f0ca40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:40.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.919+0000 7f0cabe74640 1 -- 172.21.15.133:0/4204736123 shutdown_connections 2024-09-11T23:37:40.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.919+0000 7f0cabe74640 1 -- 172.21.15.133:0/4204736123 wait complete. 2024-09-11T23:37:40.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.920+0000 7f0cabe74640 1 Processor -- start 2024-09-11T23:37:40.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.920+0000 7f0cabe74640 1 -- start start 2024-09-11T23:37:40.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.920+0000 7f0cabe74640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca4100a60 0x7f0ca419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.920+0000 7f0cabe74640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ca4101400 0x7f0ca419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0cabe74640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca4106a60 0x7f0ca419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0cabe74640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca4110e60 con 0x7f0ca4106a60 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0cabe74640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca4110ce0 con 0x7f0ca4100a60 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0cabe74640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca4110fe0 con 0x7f0ca4101400 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0ca9c10640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca4100a60 0x7f0ca419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0ca9c10640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca4100a60 0x7f0ca419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:39112/0 (socket says 172.21.15.133:39112) 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0ca9c10640 1 -- 172.21.15.133:0/4090222165 learned_addr learned my addr 172.21.15.133:0/4090222165 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0ca940f640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ca4101400 0x7f0ca419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0caa411640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca4106a60 0x7f0ca419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0ca9c10640 1 -- 172.21.15.133:0/4090222165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ca4101400 msgr2=0x7f0ca419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:40.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0ca9c10640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ca4101400 0x7f0ca419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:40.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0ca9c10640 1 -- 172.21.15.133:0/4090222165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca4106a60 msgr2=0x7f0ca419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:40.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0ca9c10640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca4106a60 0x7f0ca419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:40.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.921+0000 7f0ca9c10640 1 -- 172.21.15.133:0/4090222165 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca419f5f0 con 0x7f0ca4100a60 2024-09-11T23:37:40.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.922+0000 7f0ca9c10640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca4100a60 0x7f0ca419a640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000d950 tx=0x7f0ca000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:40.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.922+0000 7f0c9affd640 1 -- 172.21.15.133:0/4090222165 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0017070 con 0x7f0ca4100a60 2024-09-11T23:37:40.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.922+0000 7f0cabe74640 1 -- 172.21.15.133:0/4090222165 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca419f8e0 con 0x7f0ca4100a60 2024-09-11T23:37:40.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.922+0000 7f0c9affd640 1 -- 172.21.15.133:0/4090222165 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca0009c30 con 0x7f0ca4100a60 2024-09-11T23:37:40.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.922+0000 7f0c9affd640 1 -- 172.21.15.133:0/4090222165 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca000f6b0 con 0x7f0ca4100a60 2024-09-11T23:37:40.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.922+0000 7f0cabe74640 1 -- 172.21.15.133:0/4090222165 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca41a71c0 con 0x7f0ca4100a60 2024-09-11T23:37:40.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.923+0000 7f0cabe74640 1 -- 172.21.15.133:0/4090222165 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca4100ec0 con 0x7f0ca4100a60 2024-09-11T23:37:40.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.926+0000 7f0c9affd640 1 -- 172.21.15.133:0/4090222165 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f0ca00109d0 con 0x7f0ca4100a60 2024-09-11T23:37:40.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.926+0000 7f0c9affd640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0c900a2ea0 0x7f0c900a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:40.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.926+0000 7f0c9affd640 1 -- 172.21.15.133:0/4090222165 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0ca00cf540 con 0x7f0ca4100a60 2024-09-11T23:37:40.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.927+0000 7f0ca940f640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0c900a2ea0 0x7f0c900a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:40.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.927+0000 7f0ca940f640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0c900a2ea0 0x7f0c900a5360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009900 tx=0x7f0c940058d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:40.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:40.929+0000 7f0c9affd640 1 -- 172.21.15.133:0/4090222165 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ca0009de0 con 0x7f0ca4100a60 2024-09-11T23:37:40.973 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:41.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.233+0000 7f0cabe74640 1 -- 172.21.15.133:0/4090222165 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ca4103ca0 con 0x7f0ca4100a60 2024-09-11T23:37:41.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.234+0000 7f0c9affd640 1 -- 172.21.15.133:0/4090222165 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f0ca009b940 con 0x7f0ca4100a60 2024-09-11T23:37:41.235 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:37:41.235 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:37:41.236 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:37:41.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 -- 172.21.15.133:0/4090222165 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0c900a2ea0 msgr2=0x7f0c900a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0c900a2ea0 0x7f0c900a5360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009900 tx=0x7f0c940058d0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 -- 172.21.15.133:0/4090222165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca4100a60 msgr2=0x7f0ca419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca4100a60 0x7f0ca419a640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000d950 tx=0x7f0ca000de10 comp rx=0 tx=0).stop 2024-09-11T23:37:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 -- 172.21.15.133:0/4090222165 shutdown_connections 2024-09-11T23:37:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0c900a2ea0 0x7f0c900a5360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca4106a60 0x7f0ca419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ca4101400 0x7f0ca419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 --2- 172.21.15.133:0/4090222165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca4100a60 0x7f0ca419a640 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 -- 172.21.15.133:0/4090222165 >> 172.21.15.133:0/4090222165 conn(0x7f0ca40fc780 msgr2=0x7f0ca406ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 -- 172.21.15.133:0/4090222165 shutdown_connections 2024-09-11T23:37:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.237+0000 7f0cabe74640 1 -- 172.21.15.133:0/4090222165 wait complete. 2024-09-11T23:37:41.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.512+0000 7f8f8d26d640 1 -- 172.21.15.133:0/2672734854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f88100e20 msgr2=0x7f8f88101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:41.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.512+0000 7f8f8d26d640 1 --2- 172.21.15.133:0/2672734854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f88100e20 0x7f8f88101200 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009900 tx=0x7f8f7802a480 comp rx=0 tx=0).stop 2024-09-11T23:37:41.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.512+0000 7f8f8d26d640 1 -- 172.21.15.133:0/2672734854 shutdown_connections 2024-09-11T23:37:41.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.512+0000 7f8f8d26d640 1 --2- 172.21.15.133:0/2672734854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f88102ef0 0x7f8f8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.512+0000 7f8f8d26d640 1 --2- 172.21.15.133:0/2672734854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f88102570 0x7f8f881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.512+0000 7f8f8d26d640 1 --2- 172.21.15.133:0/2672734854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f88100e20 0x7f8f88101200 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.512+0000 7f8f8d26d640 1 -- 172.21.15.133:0/2672734854 >> 172.21.15.133:0/2672734854 conn(0x7f8f880fc7d0 msgr2=0x7f8f880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:41.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.512+0000 7f8f8d26d640 1 -- 172.21.15.133:0/2672734854 shutdown_connections 2024-09-11T23:37:41.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.512+0000 7f8f8d26d640 1 -- 172.21.15.133:0/2672734854 wait complete. 2024-09-11T23:37:41.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.513+0000 7f8f8d26d640 1 Processor -- start 2024-09-11T23:37:41.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.513+0000 7f8f8d26d640 1 -- start start 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f8d26d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f88100e20 0x7f8f881993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f8d26d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f88102570 0x7f8f88199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f8d26d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f88102ef0 0x7f8f881a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f8d26d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88110f80 con 0x7f8f88100e20 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f8d26d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88110e00 con 0x7f8f88102ef0 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f8d26d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88111100 con 0x7f8f88102570 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f86ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f88100e20 0x7f8f881993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f86ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f88100e20 0x7f8f881993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34080/0 (socket says 172.21.15.133:34080) 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f86ffd640 1 -- 172.21.15.133:0/974361682 learned_addr learned my addr 172.21.15.133:0/974361682 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f877fe640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f88102ef0 0x7f8f881a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f867fc640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f88102570 0x7f8f88199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f86ffd640 1 -- 172.21.15.133:0/974361682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f88102570 msgr2=0x7f8f88199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f86ffd640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f88102570 0x7f8f88199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f86ffd640 1 -- 172.21.15.133:0/974361682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f88102ef0 msgr2=0x7f8f881a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f86ffd640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f88102ef0 0x7f8f881a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f86ffd640 1 -- 172.21.15.133:0/974361682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f881971c0 con 0x7f8f88100e20 2024-09-11T23:37:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f867fc640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f88102570 0x7f8f88199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.514+0000 7f8f86ffd640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f88100e20 0x7f8f881993d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8f78006010 tx=0x7f8f78003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.515+0000 7f8f6ffff640 1 -- 172.21.15.133:0/974361682 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7803f070 con 0x7f8f88100e20 2024-09-11T23:37:41.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.515+0000 7f8f8d26d640 1 -- 172.21.15.133:0/974361682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f88197450 con 0x7f8f88100e20 2024-09-11T23:37:41.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.515+0000 7f8f6ffff640 1 -- 172.21.15.133:0/974361682 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f78031c30 con 0x7f8f88100e20 2024-09-11T23:37:41.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.515+0000 7f8f6ffff640 1 -- 172.21.15.133:0/974361682 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78035030 con 0x7f8f88100e20 2024-09-11T23:37:41.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.515+0000 7f8f8d26d640 1 -- 172.21.15.133:0/974361682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f88197980 con 0x7f8f88100e20 2024-09-11T23:37:41.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.518+0000 7f8f8f646740 1 Processor -- start 2024-09-11T23:37:41.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.518+0000 7f8f6ffff640 1 -- 172.21.15.133:0/974361682 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f8f78042070 con 0x7f8f88100e20 2024-09-11T23:37:41.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f8f646740 1 -- start start 2024-09-11T23:37:41.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0db40 0x56243ce0c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce0ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce19760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f8f646740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56243cd722f0 con 0x56243ce17370 2024-09-11T23:37:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f8f646740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56243cd19cf0 con 0x56243ce0db40 2024-09-11T23:37:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f8f646740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56243cd6e2b0 con 0x56243ce0c5d0 2024-09-11T23:37:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f867fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce19760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f867fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce19760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34090/0 (socket says 172.21.15.133:34090) 2024-09-11T23:37:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f867fc640 1 -- 172.21.15.133:0/1958351375 learned_addr learned my addr 172.21.15.133:0/1958351375 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f877fe640 1 --2- 172.21.15.133:0/1958351375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0db40 0x56243ce0c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f86ffd640 1 --2- 172.21.15.133:0/1958351375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce0ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f877fe640 1 -- 172.21.15.133:0/1958351375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 msgr2=0x56243ce0ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:41.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.519+0000 7f8f877fe640 1 --2- 172.21.15.133:0/1958351375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce0ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.520+0000 7f8f877fe640 1 -- 172.21.15.133:0/1958351375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 msgr2=0x56243ce19760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:41.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.520+0000 7f8f877fe640 1 --2- 172.21.15.133:0/1958351375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce19760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.520+0000 7f8f877fe640 1 -- 172.21.15.133:0/1958351375 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56243ccfe3a0 con 0x56243ce0db40 2024-09-11T23:37:41.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.520+0000 7f8f6ffff640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f700a2ea0 0x7f8f700a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.520+0000 7f8f867fc640 1 --2- 172.21.15.133:0/1958351375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce19760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:41.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.520+0000 7f8f86ffd640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f700a2ea0 0x7f8f700a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.520+0000 7f8f6ffff640 1 -- 172.21.15.133:0/974361682 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8f780efa50 con 0x7f8f88100e20 2024-09-11T23:37:41.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.520+0000 7f8f877fe640 1 --2- 172.21.15.133:0/1958351375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0db40 0x56243ce0c090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009f00 tx=0x7f8f80011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e60eb7800832b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:41.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.520+0000 7f8f6d7fa640 1 -- 172.21.15.133:0/1958351375 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80004d20 con 0x56243ce0db40 2024-09-11T23:37:41.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.520+0000 7f8f6d7fa640 1 -- 172.21.15.133:0/1958351375 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f80004050 con 0x56243ce0db40 2024-09-11T23:37:41.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.520+0000 7f8f86ffd640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f700a2ea0 0x7f8f700a5360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8f780ed030 tx=0x7f8f780f34a0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:41.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.521+0000 7f8f8f646740 1 -- 172.21.15.133:0/1958351375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0db40 msgr2=0x56243ce0c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:41.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.521+0000 7f8f8f646740 1 --2- 172.21.15.133:0/1958351375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0db40 0x56243ce0c090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009f00 tx=0x7f8f80011810 comp rx=0 tx=0).stop 2024-09-11T23:37:41.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.521+0000 7f8f8f646740 1 -- 172.21.15.133:0/1958351375 shutdown_connections 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.521+0000 7f8f8f646740 1 --2- 172.21.15.133:0/1958351375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce19760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.521+0000 7f8f8f646740 1 --2- 172.21.15.133:0/1958351375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce0ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.521+0000 7f8f8f646740 1 --2- 172.21.15.133:0/1958351375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0db40 0x56243ce0c090 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.521+0000 7f8f8f646740 1 -- 172.21.15.133:0/1958351375 >> 172.21.15.133:0/1958351375 conn(0x56243cdd5310 msgr2=0x56243ce08c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.521+0000 7f8f8f646740 1 -- 172.21.15.133:0/1958351375 shutdown_connections 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.521+0000 7f8f8f646740 1 -- 172.21.15.133:0/1958351375 wait complete. 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.522+0000 7f8f8f646740 1 -- start start 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.522+0000 7f8f8f646740 1 Processor -- start 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.522+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce16c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.522+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0db40 0x56243ce13eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.522+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce143f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f877fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f877fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce143f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34096/0 (socket says 172.21.15.133:34096) 2024-09-11T23:37:41.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f877fe640 1 -- 172.21.15.133:0/2885963568 learned_addr learned my addr 172.21.15.133:0/2885963568 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f8f646740 1 -- 172.21.15.133:0/2885963568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56243ce1a110 con 0x56243ce17370 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f8f646740 1 -- 172.21.15.133:0/2885963568 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56243cd2d060 con 0x56243ce0db40 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f8f646740 1 -- 172.21.15.133:0/2885963568 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56243ce1a290 con 0x56243ce0c5d0 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f867fc640 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0db40 0x56243ce13eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f877fe640 1 -- 172.21.15.133:0/2885963568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 msgr2=0x56243ce16c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f877fe640 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce16c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f877fe640 1 -- 172.21.15.133:0/2885963568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0db40 msgr2=0x56243ce13eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f877fe640 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0db40 0x56243ce13eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f877fe640 1 -- 172.21.15.133:0/2885963568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56243cae33c0 con 0x56243ce17370 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f8f646740 10 client.0 _ll_register_callbacks cb 0x56243ca808c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f8f646740 1 -- 172.21.15.133:0/2885963568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f80004050 con 0x56243ce17370 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f877fe640 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce143f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8f80003f70 tx=0x7f8f800048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3aafd66ef3f5f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.523+0000 7f8f6aff5640 10 client.15729 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:41.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f6aff5640 1 -- 172.21.15.133:0/2885963568 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80033050 con 0x56243ce17370 2024-09-11T23:37:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f6aff5640 1 -- 172.21.15.133:0/2885963568 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8f800044c0 con 0x56243ce17370 2024-09-11T23:37:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f6aff5640 1 -- 172.21.15.133:0/2885963568 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f80029070 con 0x56243ce17370 2024-09-11T23:37:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f8f646740 10 client.15729 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f8f646740 1 -- 172.21.15.133:0/2885963568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56243ce25910 con 0x56243ce17370 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f6aff5640 1 -- 172.21.15.133:0/2885963568 <== mon.0 v2:172.21.15.133:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f800054e0 con 0x56243ce17370 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f8f646740 10 client.15729 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f8f646740 10 client.15729 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f8f646740 20 client.15729 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56243ce270d0 0x56243ce29570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f8f646740 4 client.15729 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.524+0000 7f8f8f646740 1 -- 172.21.15.133:0/2885963568 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56243cd7f860 con 0x56243ce270d0 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.525+0000 7f8f867fc640 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56243ce270d0 0x56243ce29570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.525+0000 7f8f867fc640 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56243ce270d0 0x56243ce29570 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b187f10be1a794c3 server_cookie=2273eb5d46f34685 in_seq=0 out_seq=0 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.525+0000 7f8f6aff5640 10 client.15729 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.526+0000 7f8f6aff5640 1 -- 172.21.15.133:0/2885963568 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56243cd7f860 con 0x56243ce270d0 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.526+0000 7f8f6aff5640 10 client.15729 handle_command_reply: tid=0 2024-09-11T23:37:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.526+0000 7f8f8f646740 1 client.15729 shutdown 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.526+0000 7f8f8f646740 1 -- 172.21.15.133:0/2885963568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 msgr2=0x56243ce143f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.526+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce143f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8f80003f70 tx=0x7f8f800048d0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.526+0000 7f8f8f646740 1 -- 172.21.15.133:0/2885963568 shutdown_connections 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.526+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56243ce270d0 0x56243ce29570 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.526+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce143f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.526+0000 7f8f6aff5640 0 client.15729 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.527+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0db40 0x56243ce13eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.527+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2885963568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce16c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.527+0000 7f8f8f646740 1 -- 172.21.15.133:0/2885963568 >> 172.21.15.133:0/2885963568 conn(0x56243cdd5310 msgr2=0x56243ce0e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.527+0000 7f8f8f646740 1 -- 172.21.15.133:0/2885963568 shutdown_connections 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.527+0000 7f8f8f646740 1 -- 172.21.15.133:0/2885963568 wait complete. 2024-09-11T23:37:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.527+0000 7f8f8f646740 20 client.15729 trim_cache size 0 max 16384 2024-09-11T23:37:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f8f646740 1 Processor -- start 2024-09-11T23:37:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f8f646740 1 -- start start 2024-09-11T23:37:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0c5d0 0x56243ce07c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0db40 0x56243ce0efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce0f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f8f646740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881af9e0 con 0x56243ce17370 2024-09-11T23:37:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f8f646740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881b1520 con 0x56243ce0c5d0 2024-09-11T23:37:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f8f646740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881b00b0 con 0x56243ce0db40 2024-09-11T23:37:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f877fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce0f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f877fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce0f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34108/0 (socket says 172.21.15.133:34108) 2024-09-11T23:37:41.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f877fe640 1 -- 172.21.15.133:0/2856387642 learned_addr learned my addr 172.21.15.133:0/2856387642 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:41.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f877fe640 1 --2- 172.21.15.133:0/2856387642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0c5d0 0x56243ce07c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f867fc640 1 --2- 172.21.15.133:0/2856387642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0db40 0x56243ce0efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f877fe640 1 -- 172.21.15.133:0/2856387642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0db40 msgr2=0x56243ce0efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:41.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f877fe640 1 --2- 172.21.15.133:0/2856387642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0db40 0x56243ce0efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f877fe640 1 -- 172.21.15.133:0/2856387642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 msgr2=0x56243ce0f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:41.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.551+0000 7f8f877fe640 1 --2- 172.21.15.133:0/2856387642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce0f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.552+0000 7f8f877fe640 1 -- 172.21.15.133:0/2856387642 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56243ce09960 con 0x56243ce0c5d0 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.552+0000 7f8f877fe640 1 --2- 172.21.15.133:0/2856387642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0c5d0 0x56243ce07c10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000f80 tx=0x7f8f80025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bde987ed6626622 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.552+0000 7f8f6cff9640 1 -- 172.21.15.133:0/2856387642 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80033040 con 0x56243ce0c5d0 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.552+0000 7f8f6cff9640 1 -- 172.21.15.133:0/2856387642 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f80004050 con 0x56243ce0c5d0 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.552+0000 7f8f6cff9640 1 -- 172.21.15.133:0/2856387642 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8002e920 con 0x56243ce0c5d0 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.553+0000 7f8f8f646740 1 -- 172.21.15.133:0/2856387642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0c5d0 msgr2=0x56243ce07c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.553+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2856387642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0c5d0 0x56243ce07c10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000f80 tx=0x7f8f80025000 comp rx=0 tx=0).stop 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.553+0000 7f8f8f646740 1 -- 172.21.15.133:0/2856387642 shutdown_connections 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.553+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2856387642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce17370 0x56243ce0f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.553+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2856387642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0db40 0x56243ce0efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.553+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2856387642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce0c5d0 0x56243ce07c10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.553+0000 7f8f8f646740 1 -- 172.21.15.133:0/2856387642 >> 172.21.15.133:0/2856387642 conn(0x56243cdd5310 msgr2=0x56243ce07500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.553+0000 7f8f8f646740 1 -- 172.21.15.133:0/2856387642 shutdown_connections 2024-09-11T23:37:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.553+0000 7f8f8f646740 1 -- 172.21.15.133:0/2856387642 wait complete. 2024-09-11T23:37:41.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.554+0000 7f8f8f646740 1 -- start start 2024-09-11T23:37:41.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.554+0000 7f8f8f646740 1 Processor -- start 2024-09-11T23:37:41.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.554+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce8a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.554+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce0db40 0x56243ce8a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f8f646740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce17370 0x56243ce1aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f877fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce8a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f8f646740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881b0e50 con 0x56243ce0db40 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f877fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce8a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:60078/0 (socket says 172.21.15.133:60078) 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f877fe640 1 -- 172.21.15.133:0/2605416053 learned_addr learned my addr 172.21.15.133:0/2605416053 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f8f646740 1 -- 172.21.15.133:0/2605416053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881b0780 con 0x56243ce17370 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f8f646740 1 -- 172.21.15.133:0/2605416053 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881ad9e0 con 0x56243ce0c5d0 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f867fc640 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce0db40 0x56243ce8a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f8f646740 10 client.0 _ll_register_callbacks cb 0x7f8f7c009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f877fe640 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce17370 0x56243ce1aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f877fe640 1 -- 172.21.15.133:0/2605416053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce17370 msgr2=0x56243ce1aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:41.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f877fe640 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce17370 0x56243ce1aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f877fe640 1 -- 172.21.15.133:0/2605416053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce0db40 msgr2=0x56243ce8a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f877fe640 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce0db40 0x56243ce8a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f877fe640 1 -- 172.21.15.133:0/2605416053 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56243ce1b1d0 con 0x56243ce0c5d0 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f867fc640 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce0db40 0x56243ce8a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f877fe640 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce8a140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f80031040 tx=0x7f8f800096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=453e4bd0e4cee964 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.555+0000 7f8f6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.556+0000 7f8f6aff5640 1 -- 172.21.15.133:0/2605416053 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80033070 con 0x56243ce0c5d0 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.556+0000 7f8f6aff5640 1 -- 172.21.15.133:0/2605416053 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8f80023430 con 0x56243ce0c5d0 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.556+0000 7f8f8f646740 1 -- 172.21.15.133:0/2605416053 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f80004050 con 0x56243ce0c5d0 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.556+0000 7f8f6aff5640 1 -- 172.21.15.133:0/2605416053 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f8003d040 con 0x56243ce0c5d0 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.556+0000 7f8f8f646740 10 client.25219 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.556+0000 7f8f8f646740 1 -- 172.21.15.133:0/2605416053 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56243ce91910 con 0x56243ce0c5d0 2024-09-11T23:37:41.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.556+0000 7f8f6aff5640 1 -- 172.21.15.133:0/2605416053 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f80004880 con 0x56243ce0c5d0 2024-09-11T23:37:41.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.556+0000 7f8f6aff5640 1 -- 172.21.15.133:0/2605416053 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f80019550 con 0x56243ce0c5d0 2024-09-11T23:37:41.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f8f646740 10 client.25219 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:41.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f8f646740 10 client.25219 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:37:41.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f8f646740 20 client.25219 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56243ce270d0 0x56243ce93f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f8f646740 4 client.25219 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:37:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f8f646740 1 -- 172.21.15.133:0/2605416053 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8f80004050 con 0x56243ce270d0 2024-09-11T23:37:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f867fc640 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56243ce270d0 0x56243ce93f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f867fc640 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56243ce270d0 0x56243ce93f80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d69ea6c13934a512 server_cookie=70a00b71f42060b9 in_seq=0 out_seq=0 2024-09-11T23:37:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f6aff5640 10 client.25219 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f6aff5640 1 -- 172.21.15.133:0/2605416053 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8f80004050 con 0x56243ce270d0 2024-09-11T23:37:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f6aff5640 10 client.25219 handle_command_reply: tid=0 2024-09-11T23:37:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.557+0000 7f8f8f646740 1 client.25219 shutdown 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8f646740 1 -- 172.21.15.133:0/2605416053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 msgr2=0x56243ce8a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce8a140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f80031040 tx=0x7f8f800096c0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8f646740 1 -- 172.21.15.133:0/2605416053 shutdown_connections 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56243ce270d0 0x56243ce93f80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f6aff5640 0 client.25219 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56243ce17370 0x56243ce1aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243ce0db40 0x56243ce8a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8f646740 1 --2- 172.21.15.133:0/2605416053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56243ce0c5d0 0x56243ce8a140 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8f646740 1 -- 172.21.15.133:0/2605416053 >> 172.21.15.133:0/2605416053 conn(0x56243cdd5310 msgr2=0x56243ce07d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8f646740 1 -- 172.21.15.133:0/2605416053 shutdown_connections 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8f646740 1 -- 172.21.15.133:0/2605416053 wait complete. 2024-09-11T23:37:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8f646740 20 client.25219 trim_cache size 0 max 16384 2024-09-11T23:37:41.564 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:37:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8d26d640 1 -- 172.21.15.133:0/974361682 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f700a2ea0 msgr2=0x7f8f700a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8d26d640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f700a2ea0 0x7f8f700a5360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8f780ed030 tx=0x7f8f780f34a0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8d26d640 1 -- 172.21.15.133:0/974361682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f88100e20 msgr2=0x7f8f881993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.558+0000 7f8f8d26d640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f88100e20 0x7f8f881993d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8f78006010 tx=0x7f8f78003c10 comp rx=0 tx=0).stop 2024-09-11T23:37:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.559+0000 7f8f8d26d640 1 -- 172.21.15.133:0/974361682 shutdown_connections 2024-09-11T23:37:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.559+0000 7f8f8d26d640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f700a2ea0 0x7f8f700a5360 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.559+0000 7f8f8d26d640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f88102ef0 0x7f8f881a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.559+0000 7f8f8d26d640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f88102570 0x7f8f88199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.559+0000 7f8f8d26d640 1 --2- 172.21.15.133:0/974361682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f88100e20 0x7f8f881993d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.559+0000 7f8f8d26d640 1 -- 172.21.15.133:0/974361682 >> 172.21.15.133:0/974361682 conn(0x7f8f880fc7d0 msgr2=0x7f8f880fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.559+0000 7f8f8d26d640 1 -- 172.21.15.133:0/974361682 shutdown_connections 2024-09-11T23:37:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:41.559+0000 7f8f8d26d640 1 -- 172.21.15.133:0/974361682 wait complete. 2024-09-11T23:37:42.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:42 smithi133 ceph-mon[31430]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:37:42.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:42 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/4090222165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:42.415 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-11T23:37:42.415 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-11T23:37:42.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:42 smithi138 ceph-mon[35142]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:37:42.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:42 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/4090222165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:42.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:42 smithi178 ceph-mon[35882]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:37:42.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:42 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/4090222165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:42.853 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:37:42.895 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.350+0000 7f8b42ce3640 1 -- 172.21.15.133:0/2407322652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c10e3d0 msgr2=0x7f8b3c110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.350+0000 7f8b42ce3640 1 --2- 172.21.15.133:0/2407322652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c10e3d0 0x7f8b3c110820 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009ca0 tx=0x7f8b3402a950 comp rx=0 tx=0).stop 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.350+0000 7f8b42ce3640 1 -- 172.21.15.133:0/2407322652 shutdown_connections 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.350+0000 7f8b42ce3640 1 --2- 172.21.15.133:0/2407322652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c10e3d0 0x7f8b3c110820 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.350+0000 7f8b42ce3640 1 --2- 172.21.15.133:0/2407322652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b3c074f40 0x7f8b3c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.350+0000 7f8b42ce3640 1 --2- 172.21.15.133:0/2407322652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c074620 0x7f8b3c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.350+0000 7f8b42ce3640 1 -- 172.21.15.133:0/2407322652 >> 172.21.15.133:0/2407322652 conn(0x7f8b3c0fd8f0 msgr2=0x7f8b3c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.350+0000 7f8b42ce3640 1 -- 172.21.15.133:0/2407322652 shutdown_connections 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.350+0000 7f8b42ce3640 1 -- 172.21.15.133:0/2407322652 wait complete. 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.351+0000 7f8b42ce3640 1 Processor -- start 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.351+0000 7f8b42ce3640 1 -- start start 2024-09-11T23:37:43.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.351+0000 7f8b42ce3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c074620 0x7f8b3c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.351+0000 7f8b42ce3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c074f40 0x7f8b3c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.351+0000 7f8b42ce3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b3c10e3d0 0x7f8b3c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.351+0000 7f8b3bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c074f40 0x7f8b3c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.351+0000 7f8b3bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c074f40 0x7f8b3c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42556/0 (socket says 172.21.15.133:42556) 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b42ce3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c114160 con 0x7f8b3c074f40 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b42ce3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c113fe0 con 0x7f8b3c10e3d0 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b42ce3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1142e0 con 0x7f8b3c074620 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b3bfff640 1 -- 172.21.15.133:0/1771772548 learned_addr learned my addr 172.21.15.133:0/1771772548 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b40a7f640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c074620 0x7f8b3c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b3bfff640 1 -- 172.21.15.133:0/1771772548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c074620 msgr2=0x7f8b3c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b3bfff640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c074620 0x7f8b3c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b3bfff640 1 -- 172.21.15.133:0/1771772548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b3c10e3d0 msgr2=0x7f8b3c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b3bfff640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b3c10e3d0 0x7f8b3c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b3bfff640 1 -- 172.21.15.133:0/1771772548 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c19f5f0 con 0x7f8b3c074f40 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b3bfff640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c074f40 0x7f8b3c19ab80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800d950 tx=0x7f8b2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b39ffb640 1 -- 172.21.15.133:0/1771772548 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28017070 con 0x7f8b3c074f40 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b42ce3640 1 -- 172.21.15.133:0/1771772548 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3c19f8e0 con 0x7f8b3c074f40 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b39ffb640 1 -- 172.21.15.133:0/1771772548 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b28009c30 con 0x7f8b3c074f40 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.352+0000 7f8b39ffb640 1 -- 172.21.15.133:0/1771772548 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800f6b0 con 0x7f8b3c074f40 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.353+0000 7f8b42ce3640 1 -- 172.21.15.133:0/1771772548 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b3c1a71c0 con 0x7f8b3c074f40 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.355+0000 7f8b450bc740 1 Processor -- start 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b450bc740 1 -- start start 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x5652571feb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571ff080 0x565257209400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565257209940 0x56525720bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b450bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5652571642a0 con 0x5652571ff080 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b450bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56525710bb10 con 0x565257209940 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b450bc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x565257160260 con 0x5652571fe760 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b40a7f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571ff080 0x565257209400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b40a7f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571ff080 0x565257209400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42562/0 (socket says 172.21.15.133:42562) 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b40a7f640 1 -- 172.21.15.133:0/135416362 learned_addr learned my addr 172.21.15.133:0/135416362 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b41280640 1 --2- 172.21.15.133:0/135416362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x5652571feb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b40a7f640 1 -- 172.21.15.133:0/135416362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 msgr2=0x5652571feb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b40a7f640 1 --2- 172.21.15.133:0/135416362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x5652571feb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b40a7f640 1 -- 172.21.15.133:0/135416362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565257209940 msgr2=0x56525720bd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b40a7f640 1 --2- 172.21.15.133:0/135416362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565257209940 0x56525720bd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b40a7f640 1 -- 172.21.15.133:0/135416362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565256ed5b90 con 0x5652571ff080 2024-09-11T23:37:43.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.356+0000 7f8b39ffb640 1 -- 172.21.15.133:0/1771772548 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f8b280109d0 con 0x7f8b3c074f40 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.357+0000 7f8b40a7f640 1 --2- 172.21.15.133:0/135416362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571ff080 0x565257209400 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000bd10 tx=0x7f8b300107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6636d1a299207190 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.357+0000 7f8b26ffd640 1 -- 172.21.15.133:0/135416362 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30004030 con 0x5652571ff080 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.357+0000 7f8b26ffd640 1 -- 172.21.15.133:0/135416362 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b300041d0 con 0x5652571ff080 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.357+0000 7f8b26ffd640 1 -- 172.21.15.133:0/135416362 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30011840 con 0x5652571ff080 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b39ffb640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8b2c0a2ea0 0x7f8b2c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b450bc740 1 -- 172.21.15.133:0/135416362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571ff080 msgr2=0x565257209400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b450bc740 1 --2- 172.21.15.133:0/135416362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571ff080 0x565257209400 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000bd10 tx=0x7f8b300107a0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b39ffb640 1 -- 172.21.15.133:0/1771772548 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8b280cf2c0 con 0x7f8b3c074f40 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b41280640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8b2c0a2ea0 0x7f8b2c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b450bc740 1 -- 172.21.15.133:0/135416362 shutdown_connections 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b450bc740 1 --2- 172.21.15.133:0/135416362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565257209940 0x56525720bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b450bc740 1 --2- 172.21.15.133:0/135416362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571ff080 0x565257209400 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b450bc740 1 --2- 172.21.15.133:0/135416362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x5652571feb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b450bc740 1 -- 172.21.15.133:0/135416362 >> 172.21.15.133:0/135416362 conn(0x5652571facf0 msgr2=0x5652571fb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b450bc740 1 -- 172.21.15.133:0/135416362 shutdown_connections 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.358+0000 7f8b450bc740 1 -- 172.21.15.133:0/135416362 wait complete. 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.359+0000 7f8b41280640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8b2c0a2ea0 0x7f8b2c0a5360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005e30 tx=0x7f8b34005da0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.359+0000 7f8b450bc740 1 -- start start 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.359+0000 7f8b450bc740 1 Processor -- start 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.359+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571fe760 0x565257206860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.359+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x565257206da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.359+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565257209940 0x5652572072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.359+0000 7f8b450bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56525720c5b0 con 0x5652571fe760 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.359+0000 7f8b450bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5652570ec6e0 con 0x5652571ff080 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.359+0000 7f8b450bc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56525720c730 con 0x565257209940 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.359+0000 7f8b450bc740 10 client.0 _ll_register_callbacks cb 0x565257110a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b40a7f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571fe760 0x565257206860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b40a7f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571fe760 0x565257206860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42574/0 (socket says 172.21.15.133:42574) 2024-09-11T23:37:43.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b40a7f640 1 -- 172.21.15.133:0/180139382 learned_addr learned my addr 172.21.15.133:0/180139382 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b40a7f640 1 -- 172.21.15.133:0/180139382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565257209940 msgr2=0x5652572072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b40a7f640 1 --2- 172.21.15.133:0/180139382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565257209940 0x5652572072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b40a7f640 1 -- 172.21.15.133:0/180139382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 msgr2=0x565257206da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b40a7f640 1 --2- 172.21.15.133:0/180139382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x565257206da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b40a7f640 1 -- 172.21.15.133:0/180139382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525721e2e0 con 0x5652571fe760 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b40a7f640 1 --2- 172.21.15.133:0/180139382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571fe760 0x565257206860 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8b30003e30 tx=0x7f8b30005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1599c20a6cbdce58 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b1bfff640 1 -- 172.21.15.133:0/180139382 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30029060 con 0x5652571fe760 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b1bfff640 1 -- 172.21.15.133:0/180139382 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8b300035f0 con 0x5652571fe760 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.360+0000 7f8b450bc740 1 -- 172.21.15.133:0/180139382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b300041d0 con 0x5652571fe760 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.361+0000 7f8b1bfff640 1 -- 172.21.15.133:0/180139382 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8b30005600 con 0x5652571fe760 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.361+0000 7f8b450bc740 10 client.15747 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:43.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.361+0000 7f8b450bc740 1 -- 172.21.15.133:0/180139382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565257220c70 con 0x5652571fe760 2024-09-11T23:37:43.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.361+0000 7f8b1bfff640 1 -- 172.21.15.133:0/180139382 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b300041d0 con 0x5652571fe760 2024-09-11T23:37:43.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.361+0000 7f8b1bfff640 1 -- 172.21.15.133:0/180139382 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8b3001b420 con 0x5652571fe760 2024-09-11T23:37:43.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.361+0000 7f8b450bc740 10 client.15747 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:43.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.361+0000 7f8b450bc740 10 client.15747 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:37:43.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.361+0000 7f8b450bc740 20 client.15747 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:43.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.361+0000 7f8b450bc740 1 --2- 172.21.15.133:0/180139382 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565257222400 0x565257224890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.361+0000 7f8b450bc740 4 client.15747 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:43.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.361+0000 7f8b450bc740 1 -- 172.21.15.133:0/180139382 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652571719d0 con 0x565257222400 2024-09-11T23:37:43.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.362+0000 7f8b3bfff640 1 --2- 172.21.15.133:0/180139382 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565257222400 0x565257224890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.362+0000 7f8b1bfff640 10 client.15747 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.362+0000 7f8b3bfff640 1 --2- 172.21.15.133:0/180139382 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565257222400 0x565257224890 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e62237420834e68 server_cookie=1bfeeba20a446132 in_seq=0 out_seq=0 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b1bfff640 1 -- 172.21.15.133:0/180139382 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652571719d0 con 0x565257222400 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b1bfff640 10 client.15747 handle_command_reply: tid=0 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b450bc740 1 client.15747 shutdown 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b450bc740 1 -- 172.21.15.133:0/180139382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571fe760 msgr2=0x565257206860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b450bc740 1 --2- 172.21.15.133:0/180139382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571fe760 0x565257206860 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8b30003e30 tx=0x7f8b30005c90 comp rx=0 tx=0).stop 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b450bc740 1 -- 172.21.15.133:0/180139382 shutdown_connections 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b450bc740 1 --2- 172.21.15.133:0/180139382 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565257222400 0x565257224890 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b450bc740 1 --2- 172.21.15.133:0/180139382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565257209940 0x5652572072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b450bc740 1 --2- 172.21.15.133:0/180139382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x565257206da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b450bc740 1 --2- 172.21.15.133:0/180139382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652571fe760 0x565257206860 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b450bc740 1 -- 172.21.15.133:0/180139382 >> 172.21.15.133:0/180139382 conn(0x5652571facf0 msgr2=0x5652572009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.363+0000 7f8b1bfff640 0 client.15747 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:43.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.364+0000 7f8b450bc740 1 -- 172.21.15.133:0/180139382 shutdown_connections 2024-09-11T23:37:43.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.364+0000 7f8b450bc740 1 -- 172.21.15.133:0/180139382 wait complete. 2024-09-11T23:37:43.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.364+0000 7f8b450bc740 20 client.15747 trim_cache size 0 max 16384 2024-09-11T23:37:43.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.395+0000 7f8b450bc740 1 Processor -- start 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.395+0000 7f8b450bc740 1 -- start start 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x5652571fb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x56525722a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565257209940 0x56525722a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b450bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1ab590 con 0x565257209940 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b450bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1ad0d0 con 0x5652571ff080 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b450bc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1abc60 con 0x5652571fe760 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b3bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x56525722a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b41280640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565257209940 0x56525722a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b3bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x56525722a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38342/0 (socket says 172.21.15.133:38342) 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b3bfff640 1 -- 172.21.15.133:0/3370696936 learned_addr learned my addr 172.21.15.133:0/3370696936 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b3bfff640 1 -- 172.21.15.133:0/3370696936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 msgr2=0x5652571fb0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b3bfff640 1 --2- 172.21.15.133:0/3370696936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x5652571fb0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b3bfff640 1 -- 172.21.15.133:0/3370696936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565257209940 msgr2=0x56525722a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b3bfff640 1 --2- 172.21.15.133:0/3370696936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565257209940 0x56525722a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.396+0000 7f8b3bfff640 1 -- 172.21.15.133:0/3370696936 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652572021c0 con 0x5652571ff080 2024-09-11T23:37:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.397+0000 7f8b3bfff640 1 --2- 172.21.15.133:0/3370696936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x56525722a1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8b28097fe0 tx=0x7f8b2809b150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9c3490320d3b7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.397+0000 7f8b267fc640 1 -- 172.21.15.133:0/3370696936 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28020040 con 0x5652571ff080 2024-09-11T23:37:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.397+0000 7f8b267fc640 1 -- 172.21.15.133:0/3370696936 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b28015070 con 0x5652571ff080 2024-09-11T23:37:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.397+0000 7f8b267fc640 1 -- 172.21.15.133:0/3370696936 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28097c80 con 0x5652571ff080 2024-09-11T23:37:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.398+0000 7f8b450bc740 1 -- 172.21.15.133:0/3370696936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 msgr2=0x56525722a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.398+0000 7f8b450bc740 1 --2- 172.21.15.133:0/3370696936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x56525722a1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8b28097fe0 tx=0x7f8b2809b150 comp rx=0 tx=0).stop 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.398+0000 7f8b450bc740 1 -- 172.21.15.133:0/3370696936 shutdown_connections 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.398+0000 7f8b450bc740 1 --2- 172.21.15.133:0/3370696936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565257209940 0x56525722a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.398+0000 7f8b450bc740 1 --2- 172.21.15.133:0/3370696936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x56525722a1a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.398+0000 7f8b450bc740 1 --2- 172.21.15.133:0/3370696936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x5652571fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.398+0000 7f8b450bc740 1 -- 172.21.15.133:0/3370696936 >> 172.21.15.133:0/3370696936 conn(0x5652571facf0 msgr2=0x5652571f9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.398+0000 7f8b450bc740 1 -- 172.21.15.133:0/3370696936 shutdown_connections 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.398+0000 7f8b450bc740 1 -- 172.21.15.133:0/3370696936 wait complete. 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.399+0000 7f8b450bc740 1 -- start start 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.399+0000 7f8b450bc740 1 Processor -- start 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.399+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x56525721a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x565257215b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b450bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565257209940 0x565257216070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b450bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1aca00 con 0x565257209940 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b450bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1ac330 con 0x5652571ff080 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b450bc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c1a9570 con 0x5652571fe760 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b450bc740 10 client.0 _ll_register_callbacks cb 0x7f8b30004000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b3bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x565257215b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b41280640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565257209940 0x565257216070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b41280640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565257209940 0x565257216070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42588/0 (socket says 172.21.15.133:42588) 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b3bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x565257215b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38346/0 (socket says 172.21.15.133:38346) 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b41280640 1 -- 172.21.15.133:0/1652739458 learned_addr learned my addr 172.21.15.133:0/1652739458 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b40a7f640 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x56525721a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b40a7f640 1 -- 172.21.15.133:0/1652739458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 msgr2=0x565257215b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b40a7f640 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x565257215b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b40a7f640 1 -- 172.21.15.133:0/1652739458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565257209940 msgr2=0x565257216070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b40a7f640 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565257209940 0x565257216070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.400+0000 7f8b40a7f640 1 -- 172.21.15.133:0/1652739458 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565257216710 con 0x5652571fe760 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.401+0000 7f8b40a7f640 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x56525721a800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8b30000c00 tx=0x7f8b30010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae1df0afa6ed00f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.401+0000 7f8b1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.401+0000 7f8b1bfff640 1 -- 172.21.15.133:0/1652739458 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30029040 con 0x5652571fe760 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.401+0000 7f8b1bfff640 1 -- 172.21.15.133:0/1652739458 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8b30003800 con 0x5652571fe760 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.401+0000 7f8b450bc740 1 -- 172.21.15.133:0/1652739458 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b28015070 con 0x5652571fe760 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.401+0000 7f8b1bfff640 1 -- 172.21.15.133:0/1652739458 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8b30033080 con 0x5652571fe760 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.401+0000 7f8b450bc740 10 client.25231 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.401+0000 7f8b450bc740 1 -- 172.21.15.133:0/1652739458 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56525728cfe0 con 0x5652571fe760 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.401+0000 7f8b1bfff640 1 -- 172.21.15.133:0/1652739458 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b28015070 con 0x5652571fe760 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.402+0000 7f8b1bfff640 1 -- 172.21.15.133:0/1652739458 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8b30041950 con 0x5652571fe760 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.402+0000 7f8b450bc740 10 client.25231 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.402+0000 7f8b450bc740 10 client.25231 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.402+0000 7f8b450bc740 20 client.25231 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.402+0000 7f8b450bc740 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56525728d770 0x56525728fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.402+0000 7f8b450bc740 4 client.25231 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-11T23:37:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.402+0000 7f8b450bc740 1 -- 172.21.15.133:0/1652739458 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x56525716bd20 con 0x56525728d770 2024-09-11T23:37:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.402+0000 7f8b3bfff640 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56525728d770 0x56525728fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.403+0000 7f8b3bfff640 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56525728d770 0x56525728fb60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a8d10e25c7f682 server_cookie=37b7c7b9a654a638 in_seq=0 out_seq=0 2024-09-11T23:37:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.403+0000 7f8b1bfff640 10 client.25231 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:43.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.415+0000 7f8b1bfff640 1 -- 172.21.15.133:0/1652739458 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56525716bd20 con 0x56525728d770 2024-09-11T23:37:43.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.415+0000 7f8b1bfff640 10 client.25231 handle_command_reply: tid=0 2024-09-11T23:37:43.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.415+0000 7f8b450bc740 1 client.25231 shutdown 2024-09-11T23:37:43.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b450bc740 1 -- 172.21.15.133:0/1652739458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 msgr2=0x56525721a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:43.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b450bc740 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x56525721a800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8b30000c00 tx=0x7f8b30010ca0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b450bc740 1 -- 172.21.15.133:0/1652739458 shutdown_connections 2024-09-11T23:37:43.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b450bc740 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56525728d770 0x56525728fb60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b450bc740 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565257209940 0x565257216070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b450bc740 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652571ff080 0x565257215b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b450bc740 1 --2- 172.21.15.133:0/1652739458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652571fe760 0x56525721a800 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b450bc740 1 -- 172.21.15.133:0/1652739458 >> 172.21.15.133:0/1652739458 conn(0x5652571facf0 msgr2=0x5652571f9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:43.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b1bfff640 0 client.25231 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:43.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b450bc740 1 -- 172.21.15.133:0/1652739458 shutdown_connections 2024-09-11T23:37:43.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b450bc740 1 -- 172.21.15.133:0/1652739458 wait complete. 2024-09-11T23:37:43.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.416+0000 7f8b450bc740 20 client.25231 trim_cache size 0 max 16384 2024-09-11T23:37:43.418 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:37:43.418 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 28, 2024-09-11T23:37:43.418 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:37:43.418 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 2, 2024-09-11T23:37:43.418 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:37:43.418 INFO:teuthology.orchestra.run.smithi133.stdout: "942f90c7": { 2024-09-11T23:37:43.418 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 2, 2024-09-11T23:37:43.418 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:37:43.418 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 144.5, 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:37:43.419 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:37:43.420 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:37:43.420 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:37:43.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.417+0000 7f8b42ce3640 1 -- 172.21.15.133:0/1771772548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8b2c0a2ea0 msgr2=0x7f8b2c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:43.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.417+0000 7f8b42ce3640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8b2c0a2ea0 0x7f8b2c0a5360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b34005e30 tx=0x7f8b34005da0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.417+0000 7f8b42ce3640 1 -- 172.21.15.133:0/1771772548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c074f40 msgr2=0x7f8b3c19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:43.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.417+0000 7f8b42ce3640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c074f40 0x7f8b3c19ab80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800d950 tx=0x7f8b2800de10 comp rx=0 tx=0).stop 2024-09-11T23:37:43.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.417+0000 7f8b42ce3640 1 -- 172.21.15.133:0/1771772548 shutdown_connections 2024-09-11T23:37:43.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.418+0000 7f8b42ce3640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8b2c0a2ea0 0x7f8b2c0a5360 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.418+0000 7f8b42ce3640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b3c10e3d0 0x7f8b3c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.418+0000 7f8b42ce3640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c074f40 0x7f8b3c19ab80 secure :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800d950 tx=0x7f8b2800de10 comp rx=0 tx=0).stop 2024-09-11T23:37:43.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.418+0000 7f8b42ce3640 1 --2- 172.21.15.133:0/1771772548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b3c074620 0x7f8b3c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:43.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.418+0000 7f8b42ce3640 1 -- 172.21.15.133:0/1771772548 >> 172.21.15.133:0/1771772548 conn(0x7f8b3c0fd8f0 msgr2=0x7f8b3c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:43.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.418+0000 7f8b42ce3640 1 -- 172.21.15.133:0/1771772548 shutdown_connections 2024-09-11T23:37:43.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:43.418+0000 7f8b42ce3640 1 -- 172.21.15.133:0/1771772548 wait complete. 2024-09-11T23:37:44.072 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:44.073 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '942f90c7', quiesce took 9.1 seconds. Will release after: 15.4 2024-09-11T23:37:44.327 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:44 smithi133 ceph-mon[31430]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T23:37:44.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:44 smithi138 ceph-mon[35142]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T23:37:44.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.536+0000 7fa9214f9640 1 -- 172.21.15.133:0/4205351018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91c102d30 msgr2=0x7fa91c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:44.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.536+0000 7fa9214f9640 1 --2- 172.21.15.133:0/4205351018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91c102d30 0x7fa91c103110 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009900 tx=0x7fa90c02a480 comp rx=0 tx=0).stop 2024-09-11T23:37:44.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.537+0000 7fa9214f9640 1 -- 172.21.15.133:0/4205351018 shutdown_connections 2024-09-11T23:37:44.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.537+0000 7fa9214f9640 1 --2- 172.21.15.133:0/4205351018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa91c10c160 0x7fa91c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.537+0000 7fa9214f9640 1 --2- 172.21.15.133:0/4205351018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa91c103650 0x7fa91c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.537+0000 7fa9214f9640 1 --2- 172.21.15.133:0/4205351018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91c102d30 0x7fa91c103110 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.537+0000 7fa9214f9640 1 -- 172.21.15.133:0/4205351018 >> 172.21.15.133:0/4205351018 conn(0x7fa91c0fc790 msgr2=0x7fa91c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:44.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.537+0000 7fa9214f9640 1 -- 172.21.15.133:0/4205351018 shutdown_connections 2024-09-11T23:37:44.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.537+0000 7fa9214f9640 1 -- 172.21.15.133:0/4205351018 wait complete. 2024-09-11T23:37:44.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.537+0000 7fa9214f9640 1 Processor -- start 2024-09-11T23:37:44.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.537+0000 7fa9214f9640 1 -- start start 2024-09-11T23:37:44.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa9214f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91c102d30 0x7fa91c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa9214f9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa91c103650 0x7fa91c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa9214f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa91c10c160 0x7fa91c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa9214f9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c111f60 con 0x7fa91c102d30 2024-09-11T23:37:44.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa9214f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c111de0 con 0x7fa91c10c160 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa9214f9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1120e0 con 0x7fa91c103650 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa91affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91c102d30 0x7fa91c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa91affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91c102d30 0x7fa91c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42606/0 (socket says 172.21.15.133:42606) 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa91affd640 1 -- 172.21.15.133:0/3933840250 learned_addr learned my addr 172.21.15.133:0/3933840250 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa913fff640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa91c103650 0x7fa91c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa91b7fe640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa91c10c160 0x7fa91c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa913fff640 1 -- 172.21.15.133:0/3933840250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa91c10c160 msgr2=0x7fa91c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa913fff640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa91c10c160 0x7fa91c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa913fff640 1 -- 172.21.15.133:0/3933840250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91c102d30 msgr2=0x7fa91c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa913fff640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91c102d30 0x7fa91c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.538+0000 7fa913fff640 1 -- 172.21.15.133:0/3933840250 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c1a6900 con 0x7fa91c103650 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.539+0000 7fa91affd640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91c102d30 0x7fa91c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.539+0000 7fa913fff640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa91c103650 0x7fa91c100ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa904005fd0 tx=0x7fa90400db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:44.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.539+0000 7fa918ff9640 1 -- 172.21.15.133:0/3933840250 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904004230 con 0x7fa91c103650 2024-09-11T23:37:44.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.539+0000 7fa9214f9640 1 -- 172.21.15.133:0/3933840250 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c1a6bf0 con 0x7fa91c103650 2024-09-11T23:37:44.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.539+0000 7fa918ff9640 1 -- 172.21.15.133:0/3933840250 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9040043d0 con 0x7fa91c103650 2024-09-11T23:37:44.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.539+0000 7fa918ff9640 1 -- 172.21.15.133:0/3933840250 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904010460 con 0x7fa91c103650 2024-09-11T23:37:44.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.539+0000 7fa9214f9640 1 -- 172.21.15.133:0/3933840250 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c1a7180 con 0x7fa91c103650 2024-09-11T23:37:44.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:44 smithi178 ceph-mon[35882]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T23:37:44.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.542+0000 7fa918ff9640 1 -- 172.21.15.133:0/3933840250 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fa904009820 con 0x7fa91c103650 2024-09-11T23:37:44.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.543+0000 7fa9238d2740 1 Processor -- start 2024-09-11T23:37:44.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.543+0000 7fa9238d2740 1 -- start start 2024-09-11T23:37:44.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.543+0000 7fa9238d2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.543+0000 7fa9238d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0ceaba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.543+0000 7fa9238d2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ceb8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.543+0000 7fa9238d2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555d0ce112f0 con 0x555d0ceacb40 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.543+0000 7fa9238d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d0cdb8cf0 con 0x555d0ceab5d0 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.543+0000 7fa9238d2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555d0ce0d2b0 con 0x555d0ceb6370 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.544+0000 7fa91b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.544+0000 7fa91b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42616/0 (socket says 172.21.15.133:42616) 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.544+0000 7fa91b7fe640 1 -- 172.21.15.133:0/2322820067 learned_addr learned my addr 172.21.15.133:0/2322820067 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.544+0000 7fa91affd640 1 --2- 172.21.15.133:0/2322820067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0ceaba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.544+0000 7fa91b7fe640 1 -- 172.21.15.133:0/2322820067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 msgr2=0x555d0ceb8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.544+0000 7fa913fff640 1 --2- 172.21.15.133:0/2322820067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ceb8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa918ff9640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9080a2f20 0x7fa9080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa91b7fe640 1 --2- 172.21.15.133:0/2322820067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ceb8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa91b7fe640 1 -- 172.21.15.133:0/2322820067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 msgr2=0x555d0ceaba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:44.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa91b7fe640 1 --2- 172.21.15.133:0/2322820067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0ceaba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa91b7fe640 1 -- 172.21.15.133:0/2322820067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0cd9d3a0 con 0x555d0ceacb40 2024-09-11T23:37:44.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa91affd640 1 --2- 172.21.15.133:0/2322820067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0ceaba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:44.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa913fff640 1 --2- 172.21.15.133:0/2322820067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ceb8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:44.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa918ff9640 1 -- 172.21.15.133:0/3933840250 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa9040cef90 con 0x7fa91c103650 2024-09-11T23:37:44.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa91b7fe640 1 --2- 172.21.15.133:0/2322820067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceab090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa914009f70 tx=0x7fa914010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c71537696ae6fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:44.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa91b7fe640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9080a2f20 0x7fa9080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa9117fa640 1 -- 172.21.15.133:0/2322820067 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914016070 con 0x555d0ceacb40 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa9117fa640 1 -- 172.21.15.133:0/2322820067 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa914012450 con 0x555d0ceacb40 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.545+0000 7fa9117fa640 1 -- 172.21.15.133:0/2322820067 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914011710 con 0x555d0ceacb40 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.546+0000 7fa9238d2740 1 -- 172.21.15.133:0/2322820067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 msgr2=0x555d0ceab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.546+0000 7fa91b7fe640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9080a2f20 0x7fa9080a53e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa914011010 tx=0x7fa914011dc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.546+0000 7fa9238d2740 1 --2- 172.21.15.133:0/2322820067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceab090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa914009f70 tx=0x7fa914010800 comp rx=0 tx=0).stop 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.546+0000 7fa9238d2740 1 -- 172.21.15.133:0/2322820067 shutdown_connections 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.546+0000 7fa9238d2740 1 --2- 172.21.15.133:0/2322820067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ceb8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.546+0000 7fa9238d2740 1 --2- 172.21.15.133:0/2322820067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0ceaba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.546+0000 7fa9238d2740 1 --2- 172.21.15.133:0/2322820067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceab090 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.546+0000 7fa9238d2740 1 -- 172.21.15.133:0/2322820067 >> 172.21.15.133:0/2322820067 conn(0x555d0ce74310 msgr2=0x555d0cea7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.547+0000 7fa9238d2740 1 -- 172.21.15.133:0/2322820067 shutdown_connections 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.547+0000 7fa9238d2740 1 -- 172.21.15.133:0/2322820067 wait complete. 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.547+0000 7fa9238d2740 1 -- start start 2024-09-11T23:37:44.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.547+0000 7fa9238d2740 1 Processor -- start 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.547+0000 7fa9238d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0ceb5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.547+0000 7fa9238d2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceb30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa913fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceb30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa913fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceb30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42618/0 (socket says 172.21.15.133:42618) 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa913fff640 1 -- 172.21.15.133:0/3469073055 learned_addr learned my addr 172.21.15.133:0/3469073055 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa91affd640 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0ceb5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa9238d2740 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ceb37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa9238d2740 1 -- 172.21.15.133:0/3469073055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555d0ceb9070 con 0x555d0ceacb40 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa9238d2740 1 -- 172.21.15.133:0/3469073055 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d0cdcc060 con 0x555d0ceab5d0 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa9238d2740 1 -- 172.21.15.133:0/3469073055 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555d0ceb91f0 con 0x555d0ceb6370 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa91b7fe640 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ceb37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa913fff640 1 -- 172.21.15.133:0/3469073055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 msgr2=0x555d0ceb37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:44.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa913fff640 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ceb37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa913fff640 1 -- 172.21.15.133:0/3469073055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 msgr2=0x555d0ceb5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:44.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa913fff640 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0ceb5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa913fff640 1 -- 172.21.15.133:0/3469073055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0cb823c0 con 0x555d0ceacb40 2024-09-11T23:37:44.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa91affd640 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0ceb5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa9238d2740 10 client.0 _ll_register_callbacks cb 0x555d0cb1f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa913fff640 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceb30a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa904009610 tx=0x7fa90409af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da72816a3d5658d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa9027fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa9238d2740 1 -- 172.21.15.133:0/3469073055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa914012450 con 0x555d0ceacb40 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa9027fc640 1 -- 172.21.15.133:0/3469073055 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904020050 con 0x555d0ceacb40 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa9027fc640 1 -- 172.21.15.133:0/3469073055 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa9040a3ac0 con 0x555d0ceacb40 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.548+0000 7fa9027fc640 1 -- 172.21.15.133:0/3469073055 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa9040a3d40 con 0x555d0ceacb40 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.549+0000 7fa9238d2740 10 client.15765 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.549+0000 7fa9238d2740 1 -- 172.21.15.133:0/3469073055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({fsmap=28}) -- 0x555d0cecd1a0 con 0x555d0ceacb40 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.549+0000 7fa9027fc640 1 -- 172.21.15.133:0/3469073055 <== mon.0 v2:172.21.15.133:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa9040bb0b0 con 0x555d0ceacb40 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.549+0000 7fa9238d2740 10 client.15765 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.549+0000 7fa9238d2740 10 client.15765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.549+0000 7fa9238d2740 20 client.15765 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.549+0000 7fa9238d2740 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x555d0cece550 0x555d0ced09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.549+0000 7fa9238d2740 4 client.15765 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.549+0000 7fa9238d2740 1 -- 172.21.15.133:0/3469073055 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa914012450 con 0x555d0cece550 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.549+0000 7fa91affd640 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x555d0cece550 0x555d0ced09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.550+0000 7fa91affd640 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x555d0cece550 0x555d0ced09c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=feb400943c57ffd3 server_cookie=1f9b34c00edc2338 in_seq=0 out_seq=0 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.550+0000 7fa9027fc640 10 client.15765 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9027fc640 1 -- 172.21.15.133:0/3469073055 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa914012450 con 0x555d0cece550 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9027fc640 10 client.15765 handle_command_reply: tid=0 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 1 client.15765 shutdown 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 1 -- 172.21.15.133:0/3469073055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 msgr2=0x555d0ceb30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceb30a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa904009610 tx=0x7fa90409af90 comp rx=0 tx=0).stop 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 1 -- 172.21.15.133:0/3469073055 shutdown_connections 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x555d0cece550 0x555d0ced09c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9027fc640 0 client.15765 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ceb37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceacb40 0x555d0ceb30a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 1 --2- 172.21.15.133:0/3469073055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0ceb5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 1 -- 172.21.15.133:0/3469073055 >> 172.21.15.133:0/3469073055 conn(0x555d0ce74310 msgr2=0x555d0cead960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 1 -- 172.21.15.133:0/3469073055 shutdown_connections 2024-09-11T23:37:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 1 -- 172.21.15.133:0/3469073055 wait complete. 2024-09-11T23:37:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.551+0000 7fa9238d2740 20 client.15765 trim_cache size 0 max 16384 2024-09-11T23:37:44.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.564+0000 7fa9238d2740 1 Processor -- start 2024-09-11T23:37:44.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.564+0000 7fa9238d2740 1 -- start start 2024-09-11T23:37:44.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa9238d2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceab5d0 0x555d0cecaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa9238d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceacb40 0x555d0cea6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa9238d2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ced6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa9238d2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c06fe90 con 0x555d0ceab5d0 2024-09-11T23:37:44.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa9238d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c0719d0 con 0x555d0ceacb40 2024-09-11T23:37:44.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa9238d2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c070560 con 0x555d0ceb6370 2024-09-11T23:37:44.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa91affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceab5d0 0x555d0cecaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa91affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceab5d0 0x555d0cecaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42622/0 (socket says 172.21.15.133:42622) 2024-09-11T23:37:44.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa91affd640 1 -- 172.21.15.133:0/1636973577 learned_addr learned my addr 172.21.15.133:0/1636973577 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:44.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa91affd640 1 -- 172.21.15.133:0/1636973577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 msgr2=0x555d0ced6240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:44.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa91affd640 1 --2- 172.21.15.133:0/1636973577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ced6240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa91affd640 1 -- 172.21.15.133:0/1636973577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceacb40 msgr2=0x555d0cea6bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:44.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa91affd640 1 --2- 172.21.15.133:0/1636973577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceacb40 0x555d0cea6bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.565+0000 7fa91affd640 1 -- 172.21.15.133:0/1636973577 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0cea8960 con 0x555d0ceab5d0 2024-09-11T23:37:44.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.566+0000 7fa91affd640 1 --2- 172.21.15.133:0/1636973577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceab5d0 0x555d0cecaba0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa90c006010 tx=0x7fa90c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5884a037f4545995 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:44.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.566+0000 7fa910ff9640 1 -- 172.21.15.133:0/1636973577 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c04b070 con 0x555d0ceab5d0 2024-09-11T23:37:44.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.566+0000 7fa910ff9640 1 -- 172.21.15.133:0/1636973577 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c005530 con 0x555d0ceab5d0 2024-09-11T23:37:44.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.566+0000 7fa910ff9640 1 -- 172.21.15.133:0/1636973577 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c035040 con 0x555d0ceab5d0 2024-09-11T23:37:44.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.567+0000 7fa9238d2740 1 -- 172.21.15.133:0/1636973577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceab5d0 msgr2=0x555d0cecaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:44.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.567+0000 7fa9238d2740 1 --2- 172.21.15.133:0/1636973577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceab5d0 0x555d0cecaba0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa90c006010 tx=0x7fa90c005a20 comp rx=0 tx=0).stop 2024-09-11T23:37:44.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.567+0000 7fa9238d2740 1 -- 172.21.15.133:0/1636973577 shutdown_connections 2024-09-11T23:37:44.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.567+0000 7fa9238d2740 1 --2- 172.21.15.133:0/1636973577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceb6370 0x555d0ced6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.567+0000 7fa9238d2740 1 --2- 172.21.15.133:0/1636973577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceacb40 0x555d0cea6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.567+0000 7fa9238d2740 1 --2- 172.21.15.133:0/1636973577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceab5d0 0x555d0cecaba0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.567+0000 7fa9238d2740 1 -- 172.21.15.133:0/1636973577 >> 172.21.15.133:0/1636973577 conn(0x555d0ce74310 msgr2=0x555d0cea64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:44.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.567+0000 7fa9238d2740 1 -- 172.21.15.133:0/1636973577 shutdown_connections 2024-09-11T23:37:44.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.567+0000 7fa9238d2740 1 -- 172.21.15.133:0/1636973577 wait complete. 2024-09-11T23:37:44.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.567+0000 7fa9238d2740 1 -- start start 2024-09-11T23:37:44.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa9238d2740 1 Processor -- start 2024-09-11T23:37:44.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa9238d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0cec6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa9238d2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceacb40 0x555d0cec7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa9238d2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceb6370 0x555d0cec3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa9238d2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c071300 con 0x555d0ceb6370 2024-09-11T23:37:44.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa9238d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c070c30 con 0x555d0ceab5d0 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa9238d2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c06de90 con 0x555d0ceacb40 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa9238d2740 10 client.0 _ll_register_callbacks cb 0x7fa904003880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa91b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceb6370 0x555d0cec3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa91affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0cec6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa91affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0cec6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38376/0 (socket says 172.21.15.133:38376) 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa91affd640 1 -- 172.21.15.133:0/1208239565 learned_addr learned my addr 172.21.15.133:0/1208239565 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.568+0000 7fa913fff640 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceacb40 0x555d0cec7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.569+0000 7fa91b7fe640 1 -- 172.21.15.133:0/1208239565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceacb40 msgr2=0x555d0cec7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.569+0000 7fa91b7fe640 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceacb40 0x555d0cec7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.569+0000 7fa91b7fe640 1 -- 172.21.15.133:0/1208239565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 msgr2=0x555d0cec6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.569+0000 7fa91b7fe640 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0cec6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.569+0000 7fa91b7fe640 1 -- 172.21.15.133:0/1208239565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0cec7960 con 0x555d0ceb6370 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.569+0000 7fa91b7fe640 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceb6370 0x555d0cec3ce0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x555d0ceb3f50 tx=0x7fa91401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cff696834c6681c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:44.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.569+0000 7fa9027fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.569+0000 7fa9027fc640 1 -- 172.21.15.133:0/1208239565 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91404e070 con 0x555d0ceb6370 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.569+0000 7fa9027fc640 1 -- 172.21.15.133:0/1208239565 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa914004ba0 con 0x555d0ceb6370 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.569+0000 7fa9238d2740 1 -- 172.21.15.133:0/1208239565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90c005530 con 0x555d0ceb6370 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.570+0000 7fa9027fc640 1 -- 172.21.15.133:0/1208239565 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa914061050 con 0x555d0ceb6370 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.570+0000 7fa9238d2740 10 client.15771 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.570+0000 7fa9238d2740 1 -- 172.21.15.133:0/1208239565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d0cf39830 con 0x555d0ceb6370 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.570+0000 7fa9027fc640 1 -- 172.21.15.133:0/1208239565 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c005530 con 0x555d0ceb6370 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.570+0000 7fa9027fc640 1 -- 172.21.15.133:0/1208239565 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa91400f040 con 0x555d0ceb6370 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.570+0000 7fa9238d2740 10 client.15771 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.570+0000 7fa9238d2740 10 client.15771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.570+0000 7fa9238d2740 20 client.15771 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.570+0000 7fa9238d2740 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x555d0cf39c60 0x555d0cf3c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.570+0000 7fa9238d2740 4 client.15771 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:37:44.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.570+0000 7fa9238d2740 1 -- 172.21.15.133:0/1208239565 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d0ce1e860 con 0x555d0cf39c60 2024-09-11T23:37:44.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.571+0000 7fa91affd640 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x555d0cf39c60 0x555d0cf3c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:44.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.571+0000 7fa91affd640 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x555d0cf39c60 0x555d0cf3c080 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=932836281d5ad0f server_cookie=7fda25b9965ee128 in_seq=0 out_seq=0 2024-09-11T23:37:44.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.571+0000 7fa9027fc640 10 client.15771 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:44.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9027fc640 1 -- 172.21.15.133:0/1208239565 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d0ce1e860 con 0x555d0cf39c60 2024-09-11T23:37:44.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9027fc640 10 client.15771 handle_command_reply: tid=0 2024-09-11T23:37:44.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9238d2740 1 client.15771 shutdown 2024-09-11T23:37:44.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9238d2740 1 -- 172.21.15.133:0/1208239565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceb6370 msgr2=0x555d0cec3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:44.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9238d2740 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceb6370 0x555d0cec3ce0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x555d0ceb3f50 tx=0x7fa91401b000 comp rx=0 tx=0).stop 2024-09-11T23:37:44.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9238d2740 1 -- 172.21.15.133:0/1208239565 shutdown_connections 2024-09-11T23:37:44.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9238d2740 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x555d0cf39c60 0x555d0cf3c080 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9238d2740 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d0ceb6370 0x555d0cec3ce0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9238d2740 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555d0ceacb40 0x555d0cec7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9238d2740 1 --2- 172.21.15.133:0/1208239565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d0ceab5d0 0x555d0cec6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9027fc640 0 client.15771 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:44.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9238d2740 1 -- 172.21.15.133:0/1208239565 >> 172.21.15.133:0/1208239565 conn(0x555d0ce74310 msgr2=0x555d0ceb5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:44.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9238d2740 1 -- 172.21.15.133:0/1208239565 shutdown_connections 2024-09-11T23:37:44.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.572+0000 7fa9238d2740 1 -- 172.21.15.133:0/1208239565 wait complete. 2024-09-11T23:37:44.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.573+0000 7fa9238d2740 20 client.15771 trim_cache size 0 max 16384 2024-09-11T23:37:44.575 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:37:44.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.573+0000 7fa9214f9640 1 -- 172.21.15.133:0/3933840250 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9080a2f20 msgr2=0x7fa9080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:44.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.573+0000 7fa9214f9640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9080a2f20 0x7fa9080a53e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa914011010 tx=0x7fa914011dc0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.573+0000 7fa9214f9640 1 -- 172.21.15.133:0/3933840250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa91c103650 msgr2=0x7fa91c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:44.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.573+0000 7fa9214f9640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa91c103650 0x7fa91c100ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa904005fd0 tx=0x7fa90400db60 comp rx=0 tx=0).stop 2024-09-11T23:37:44.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.573+0000 7fa9214f9640 1 -- 172.21.15.133:0/3933840250 shutdown_connections 2024-09-11T23:37:44.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.573+0000 7fa9214f9640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa9080a2f20 0x7fa9080a53e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.573+0000 7fa9214f9640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa91c10c160 0x7fa91c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.573+0000 7fa9214f9640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa91c103650 0x7fa91c100ed0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.573+0000 7fa9214f9640 1 --2- 172.21.15.133:0/3933840250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91c102d30 0x7fa91c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:44.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.573+0000 7fa9214f9640 1 -- 172.21.15.133:0/3933840250 >> 172.21.15.133:0/3933840250 conn(0x7fa91c0fc790 msgr2=0x7fa91c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:44.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.574+0000 7fa9214f9640 1 -- 172.21.15.133:0/3933840250 shutdown_connections 2024-09-11T23:37:44.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:44.574+0000 7fa9214f9640 1 -- 172.21.15.133:0/3933840250 wait complete. 2024-09-11T23:37:45.423 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:37:45.741 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:46.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.295+0000 7f670f4aa640 1 -- 172.21.15.133:0/3850019794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6708102ef0 msgr2=0x7f670810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:46.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.295+0000 7f670f4aa640 1 --2- 172.21.15.133:0/3850019794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6708102ef0 0x7f670810da40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6700009ca0 tx=0x7f670002a950 comp rx=0 tx=0).stop 2024-09-11T23:37:46.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.295+0000 7f670f4aa640 1 -- 172.21.15.133:0/3850019794 shutdown_connections 2024-09-11T23:37:46.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.295+0000 7f670f4aa640 1 --2- 172.21.15.133:0/3850019794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6708102ef0 0x7f670810da40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.295+0000 7f670f4aa640 1 --2- 172.21.15.133:0/3850019794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6708102570 0x7f67081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.295+0000 7f670f4aa640 1 --2- 172.21.15.133:0/3850019794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6708100e20 0x7f6708101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.295+0000 7f670f4aa640 1 -- 172.21.15.133:0/3850019794 >> 172.21.15.133:0/3850019794 conn(0x7f67080fc7d0 msgr2=0x7f67080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.295+0000 7f670f4aa640 1 -- 172.21.15.133:0/3850019794 shutdown_connections 2024-09-11T23:37:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.295+0000 7f670f4aa640 1 -- 172.21.15.133:0/3850019794 wait complete. 2024-09-11T23:37:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.296+0000 7f670f4aa640 1 Processor -- start 2024-09-11T23:37:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.296+0000 7f670f4aa640 1 -- start start 2024-09-11T23:37:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670f4aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6708100e20 0x7f67081973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670f4aa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6708102570 0x7f6708197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670d246640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6708100e20 0x7f67081973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670d246640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6708100e20 0x7f67081973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42654/0 (socket says 172.21.15.133:42654) 2024-09-11T23:37:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670d246640 1 -- 172.21.15.133:0/1074772376 learned_addr learned my addr 172.21.15.133:0/1074772376 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670f4aa640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6708102ef0 0x7f670819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670f4aa640 1 -- 172.21.15.133:0/1074772376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6708110fd0 con 0x7f6708100e20 2024-09-11T23:37:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670f4aa640 1 -- 172.21.15.133:0/1074772376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6708110e50 con 0x7f6708102ef0 2024-09-11T23:37:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670f4aa640 1 -- 172.21.15.133:0/1074772376 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6708111150 con 0x7f6708102570 2024-09-11T23:37:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670ca45640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6708102570 0x7f6708197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670da47640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6708102ef0 0x7f670819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.297+0000 7f670d246640 1 -- 172.21.15.133:0/1074772376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6708102570 msgr2=0x7f6708197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.298+0000 7f670d246640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6708102570 0x7f6708197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.298+0000 7f670d246640 1 -- 172.21.15.133:0/1074772376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6708102ef0 msgr2=0x7f670819bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.298+0000 7f670d246640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6708102ef0 0x7f670819bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.298+0000 7f670d246640 1 -- 172.21.15.133:0/1074772376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670819c380 con 0x7f6708100e20 2024-09-11T23:37:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.298+0000 7f670d246640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6708100e20 0x7f67081973d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f670400d950 tx=0x7f670400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.298+0000 7f66fe7fc640 1 -- 172.21.15.133:0/1074772376 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704017030 con 0x7f6708100e20 2024-09-11T23:37:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.298+0000 7f66fe7fc640 1 -- 172.21.15.133:0/1074772376 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6704004020 con 0x7f6708100e20 2024-09-11T23:37:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.298+0000 7f670f4aa640 1 -- 172.21.15.133:0/1074772376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670819c670 con 0x7f6708100e20 2024-09-11T23:37:46.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.298+0000 7f66fe7fc640 1 -- 172.21.15.133:0/1074772376 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67040047a0 con 0x7f6708100e20 2024-09-11T23:37:46.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.298+0000 7f670f4aa640 1 -- 172.21.15.133:0/1074772376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67081a3f50 con 0x7f6708100e20 2024-09-11T23:37:46.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f66fe7fc640 1 -- 172.21.15.133:0/1074772376 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f67040041d0 con 0x7f6708100e20 2024-09-11T23:37:46.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f6711883740 1 Processor -- start 2024-09-11T23:37:46.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f6711883740 1 -- start start 2024-09-11T23:37:46.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f6711883740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 0x55ee76013d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f6711883740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760142c0 0x55ee76028430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f6711883740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee76028970 0x55ee7602ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f6711883740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ee75f7b2f0 con 0x55ee76028970 2024-09-11T23:37:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f6711883740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ee75f22cf0 con 0x55ee760142c0 2024-09-11T23:37:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f6711883740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ee75f772b0 con 0x55ee760139a0 2024-09-11T23:37:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f670da47640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 0x55ee76013d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f670da47640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 0x55ee76013d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:41790/0 (socket says 172.21.15.133:41790) 2024-09-11T23:37:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.302+0000 7f670da47640 1 -- 172.21.15.133:0/2152745035 learned_addr learned my addr 172.21.15.133:0/2152745035 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.303+0000 7f670ca45640 1 --2- 172.21.15.133:0/2152745035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee76028970 0x55ee7602ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.303+0000 7f66fe7fc640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66f40a2fa0 0x7f66f40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.303+0000 7f670ca45640 1 -- 172.21.15.133:0/2152745035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 msgr2=0x55ee76013d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.303+0000 7f670ca45640 1 --2- 172.21.15.133:0/2152745035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 0x55ee76013d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.303+0000 7f670ca45640 1 -- 172.21.15.133:0/2152745035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760142c0 msgr2=0x55ee76028430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.303+0000 7f670ca45640 1 --2- 172.21.15.133:0/2152745035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760142c0 0x55ee76028430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.303+0000 7f670ca45640 1 -- 172.21.15.133:0/2152745035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee75f073a0 con 0x55ee76028970 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.303+0000 7f66fe7fc640 1 -- 172.21.15.133:0/1074772376 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f67040cf030 con 0x7f6708100e20 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.303+0000 7f670ca45640 1 --2- 172.21.15.133:0/2152745035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee76028970 0x55ee7602ad60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f66f800bb90 tx=0x7f66f80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4fb1fa231e679e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.304+0000 7f670da47640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66f40a2fa0 0x7f66f40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.304+0000 7f66f37fe640 1 -- 172.21.15.133:0/2152745035 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8004030 con 0x55ee76028970 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.304+0000 7f66f37fe640 1 -- 172.21.15.133:0/2152745035 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f80041d0 con 0x55ee76028970 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.304+0000 7f66f37fe640 1 -- 172.21.15.133:0/2152745035 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8011800 con 0x55ee76028970 2024-09-11T23:37:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.305+0000 7f6711883740 1 -- 172.21.15.133:0/2152745035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee76028970 msgr2=0x55ee7602ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.305+0000 7f6711883740 1 --2- 172.21.15.133:0/2152745035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee76028970 0x55ee7602ad60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f66f800bb90 tx=0x7f66f80107a0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.305+0000 7f670da47640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66f40a2fa0 0x7f66f40a5460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6700005fa0 tx=0x7f670002a950 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.305+0000 7f6711883740 1 -- 172.21.15.133:0/2152745035 shutdown_connections 2024-09-11T23:37:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.305+0000 7f6711883740 1 --2- 172.21.15.133:0/2152745035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee76028970 0x55ee7602ad60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.305+0000 7f6711883740 1 --2- 172.21.15.133:0/2152745035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760142c0 0x55ee76028430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.305+0000 7f6711883740 1 --2- 172.21.15.133:0/2152745035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 0x55ee76013d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.305+0000 7f6711883740 1 -- 172.21.15.133:0/2152745035 >> 172.21.15.133:0/2152745035 conn(0x55ee75fde310 msgr2=0x55ee760116a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.305+0000 7f6711883740 1 -- 172.21.15.133:0/2152745035 shutdown_connections 2024-09-11T23:37:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.305+0000 7f6711883740 1 -- 172.21.15.133:0/2152745035 wait complete. 2024-09-11T23:37:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.306+0000 7f6711883740 1 -- start start 2024-09-11T23:37:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.306+0000 7f6711883740 1 Processor -- start 2024-09-11T23:37:46.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.306+0000 7f6711883740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760139a0 0x55ee760223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.306+0000 7f6711883740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760142c0 0x55ee76022910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.306+0000 7f6711883740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee76028970 0x55ee76028100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.306+0000 7f6711883740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ee7601b020 con 0x55ee760139a0 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.306+0000 7f6711883740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ee75f36060 con 0x55ee760142c0 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.306+0000 7f6711883740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ee7601b1a0 con 0x55ee76028970 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.306+0000 7f6711883740 10 client.0 _ll_register_callbacks cb 0x55ee75c898c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f670ca45640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760142c0 0x55ee76022910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f670ca45640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760142c0 0x55ee76022910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38422/0 (socket says 172.21.15.133:38422) 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f670ca45640 1 -- 172.21.15.133:0/566339635 learned_addr learned my addr 172.21.15.133:0/566339635 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f670d246640 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760139a0 0x55ee760223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f670ca45640 1 -- 172.21.15.133:0/566339635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee76028970 msgr2=0x55ee76028100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f670ca45640 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee76028970 0x55ee76028100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f670ca45640 1 -- 172.21.15.133:0/566339635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760139a0 msgr2=0x55ee760223d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f670ca45640 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760139a0 0x55ee760223d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f670ca45640 1 -- 172.21.15.133:0/566339635 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee75cec3c0 con 0x55ee760142c0 2024-09-11T23:37:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f670d246640 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760139a0 0x55ee760223d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:46.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f670ca45640 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760142c0 0x55ee76022910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000c00 tx=0x7f66f80045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e8a7b5db12b3157 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:46.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f66f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:37:46.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.307+0000 7f66f0ff9640 1 -- 172.21.15.133:0/566339635 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f80035f0 con 0x55ee760142c0 2024-09-11T23:37:46.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f66f0ff9640 1 -- 172.21.15.133:0/566339635 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f66f8003790 con 0x55ee760142c0 2024-09-11T23:37:46.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f6711883740 1 -- 172.21.15.133:0/566339635 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f80041d0 con 0x55ee760142c0 2024-09-11T23:37:46.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f66f0ff9640 1 -- 172.21.15.133:0/566339635 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f66f8003990 con 0x55ee760142c0 2024-09-11T23:37:46.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f6711883740 10 client.25208 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:46.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f6711883740 1 -- 172.21.15.133:0/566339635 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee7603fc00 con 0x55ee760142c0 2024-09-11T23:37:46.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f66f0ff9640 1 -- 172.21.15.133:0/566339635 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f8005520 con 0x55ee760142c0 2024-09-11T23:37:46.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f66f0ff9640 1 -- 172.21.15.133:0/566339635 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f66f8011440 con 0x55ee760142c0 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f6711883740 10 client.25208 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f6711883740 10 client.25208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f6711883740 20 client.25208 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f6711883740 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ee76040f50 0x55ee76043340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f6711883740 4 client.25208 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.308+0000 7f6711883740 1 -- 172.21.15.133:0/566339635 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f66f80041d0 con 0x55ee76040f50 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.309+0000 7f670d246640 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ee76040f50 0x55ee76043340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.309+0000 7f66f0ff9640 10 client.25208 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.309+0000 7f670d246640 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ee76040f50 0x55ee76043340 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4fb6416d4f8a5c server_cookie=5c0bb4c498e4f85 in_seq=0 out_seq=0 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f66f0ff9640 1 -- 172.21.15.133:0/566339635 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f66f80041d0 con 0x55ee76040f50 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f66f0ff9640 10 client.25208 handle_command_reply: tid=0 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f6711883740 1 client.25208 shutdown 2024-09-11T23:37:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f6711883740 1 -- 172.21.15.133:0/566339635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760142c0 msgr2=0x55ee76022910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f6711883740 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760142c0 0x55ee76022910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000c00 tx=0x7f66f80045b0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f6711883740 1 -- 172.21.15.133:0/566339635 shutdown_connections 2024-09-11T23:37:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f6711883740 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ee76040f50 0x55ee76043340 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f6711883740 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee76028970 0x55ee76028100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f66f0ff9640 0 client.25208 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f6711883740 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760142c0 0x55ee76022910 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f6711883740 1 --2- 172.21.15.133:0/566339635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760139a0 0x55ee760223d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.310+0000 7f6711883740 1 -- 172.21.15.133:0/566339635 >> 172.21.15.133:0/566339635 conn(0x55ee75fde310 msgr2=0x55ee76010d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.312+0000 7f6711883740 1 -- 172.21.15.133:0/566339635 shutdown_connections 2024-09-11T23:37:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.312+0000 7f6711883740 1 -- 172.21.15.133:0/566339635 wait complete. 2024-09-11T23:37:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.312+0000 7f6711883740 20 client.25208 trim_cache size 0 max 16384 2024-09-11T23:37:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.337+0000 7f6711883740 1 Processor -- start 2024-09-11T23:37:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.337+0000 7f6711883740 1 -- start start 2024-09-11T23:37:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.337+0000 7f6711883740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760139a0 0x55ee76028050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.337+0000 7f6711883740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee7601a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.337+0000 7f6711883740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee76028970 0x55ee7601a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.337+0000 7f6711883740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67081ab500 con 0x55ee760142c0 2024-09-11T23:37:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.337+0000 7f6711883740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67081ad040 con 0x55ee760139a0 2024-09-11T23:37:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.337+0000 7f6711883740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67081abbd0 con 0x55ee76028970 2024-09-11T23:37:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.338+0000 7f670ca45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee7601a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.338+0000 7f670d246640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760139a0 0x55ee76028050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.338+0000 7f670ca45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee7601a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42674/0 (socket says 172.21.15.133:42674) 2024-09-11T23:37:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.338+0000 7f670ca45640 1 -- 172.21.15.133:0/681235371 learned_addr learned my addr 172.21.15.133:0/681235371 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.338+0000 7f670da47640 1 --2- 172.21.15.133:0/681235371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee76028970 0x55ee7601a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.338+0000 7f670ca45640 1 -- 172.21.15.133:0/681235371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee76028970 msgr2=0x55ee7601a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:46.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.338+0000 7f670ca45640 1 --2- 172.21.15.133:0/681235371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee76028970 0x55ee7601a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.338+0000 7f670ca45640 1 -- 172.21.15.133:0/681235371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760139a0 msgr2=0x55ee76028050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:46.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.338+0000 7f670ca45640 1 --2- 172.21.15.133:0/681235371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760139a0 0x55ee76028050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.338+0000 7f670ca45640 1 -- 172.21.15.133:0/681235371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee76011ba0 con 0x55ee760142c0 2024-09-11T23:37:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.339+0000 7f670ca45640 1 --2- 172.21.15.133:0/681235371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee7601a100 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f66f8011a00 tx=0x7f66f80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a41339f34455041 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.339+0000 7f66f2ffd640 1 -- 172.21.15.133:0/681235371 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8033070 con 0x55ee760142c0 2024-09-11T23:37:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.339+0000 7f66f2ffd640 1 -- 172.21.15.133:0/681235371 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f8011c80 con 0x55ee760142c0 2024-09-11T23:37:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.339+0000 7f66f2ffd640 1 -- 172.21.15.133:0/681235371 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f80052a0 con 0x55ee760142c0 2024-09-11T23:37:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.340+0000 7f6711883740 1 -- 172.21.15.133:0/681235371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 msgr2=0x55ee7601a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.340+0000 7f6711883740 1 --2- 172.21.15.133:0/681235371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee7601a100 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f66f8011a00 tx=0x7f66f80107a0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.340+0000 7f6711883740 1 -- 172.21.15.133:0/681235371 shutdown_connections 2024-09-11T23:37:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.340+0000 7f6711883740 1 --2- 172.21.15.133:0/681235371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee76028970 0x55ee7601a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.340+0000 7f6711883740 1 --2- 172.21.15.133:0/681235371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee7601a100 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.340+0000 7f6711883740 1 --2- 172.21.15.133:0/681235371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee760139a0 0x55ee76028050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.340+0000 7f6711883740 1 -- 172.21.15.133:0/681235371 >> 172.21.15.133:0/681235371 conn(0x55ee75fde310 msgr2=0x55ee7601c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.340+0000 7f6711883740 1 -- 172.21.15.133:0/681235371 shutdown_connections 2024-09-11T23:37:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.341+0000 7f6711883740 1 -- 172.21.15.133:0/681235371 wait complete. 2024-09-11T23:37:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.341+0000 7f6711883740 1 -- start start 2024-09-11T23:37:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.341+0000 7f6711883740 1 Processor -- start 2024-09-11T23:37:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.342+0000 7f6711883740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 0x55ee76054790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.342+0000 7f6711883740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee76054cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.342+0000 7f670ca45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee76054cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.342+0000 7f670ca45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee76054cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42684/0 (socket says 172.21.15.133:42684) 2024-09-11T23:37:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.342+0000 7f670ca45640 1 -- 172.21.15.133:0/991012298 learned_addr learned my addr 172.21.15.133:0/991012298 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.342+0000 7f6711883740 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee76028970 0x55ee76051760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.342+0000 7f6711883740 1 -- 172.21.15.133:0/991012298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67081ac970 con 0x55ee760142c0 2024-09-11T23:37:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.342+0000 7f6711883740 1 -- 172.21.15.133:0/991012298 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67081ac2a0 con 0x55ee76028970 2024-09-11T23:37:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.342+0000 7f6711883740 1 -- 172.21.15.133:0/991012298 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67081a9500 con 0x55ee760139a0 2024-09-11T23:37:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.342+0000 7f670d246640 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 0x55ee76054790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.343+0000 7f670d246640 1 -- 172.21.15.133:0/991012298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee76028970 msgr2=0x55ee76051760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.343+0000 7f670d246640 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee76028970 0x55ee76051760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.343+0000 7f670d246640 1 -- 172.21.15.133:0/991012298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 msgr2=0x55ee76054cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.343+0000 7f670d246640 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee76054cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.343+0000 7f670d246640 1 -- 172.21.15.133:0/991012298 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee76051ca0 con 0x55ee760139a0 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.343+0000 7f670ca45640 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee76054cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.343+0000 7f6711883740 10 client.0 _ll_register_callbacks cb 0x7f6700009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.343+0000 7f670d246640 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 0x55ee76054790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67040cc5b0 tx=0x7f6704005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1900e01caceffaaf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.343+0000 7f66f0ff9640 10 client.25258 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.343+0000 7f6711883740 1 -- 172.21.15.133:0/991012298 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f8011c80 con 0x55ee760139a0 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.343+0000 7f66f0ff9640 1 -- 172.21.15.133:0/991012298 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670409b380 con 0x55ee760139a0 2024-09-11T23:37:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.344+0000 7f66f0ff9640 1 -- 172.21.15.133:0/991012298 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f67040979f0 con 0x55ee760139a0 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.344+0000 7f66f0ff9640 1 -- 172.21.15.133:0/991012298 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f670409f730 con 0x55ee760139a0 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.344+0000 7f6711883740 10 client.25258 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.344+0000 7f6711883740 1 -- 172.21.15.133:0/991012298 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55ee760abd80 con 0x55ee760139a0 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.344+0000 7f66f0ff9640 1 -- 172.21.15.133:0/991012298 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f670409a7d0 con 0x55ee760139a0 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.344+0000 7f6711883740 10 client.25258 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.344+0000 7f6711883740 10 client.25258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.344+0000 7f6711883740 20 client.25258 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.344+0000 7f6711883740 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ee760ac440 0x55ee760ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.344+0000 7f6711883740 4 client.25258 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.345+0000 7f6711883740 1 -- 172.21.15.133:0/991012298 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee75f88860 con 0x55ee760ac440 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.345+0000 7f670ca45640 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ee760ac440 0x55ee760ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.345+0000 7f66f0ff9640 10 client.25258 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.345+0000 7f670ca45640 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ee760ac440 0x55ee760ae900 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866a7d9ad910b394 server_cookie=f63e779b2546b286 in_seq=0 out_seq=0 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.345+0000 7f66f0ff9640 1 -- 172.21.15.133:0/991012298 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee75f88860 con 0x55ee760ac440 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.345+0000 7f66f0ff9640 10 client.25258 handle_command_reply: tid=0 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.345+0000 7f6711883740 1 client.25258 shutdown 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f6711883740 1 -- 172.21.15.133:0/991012298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 msgr2=0x55ee76054790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f6711883740 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 0x55ee76054790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67040cc5b0 tx=0x7f6704005db0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f6711883740 1 -- 172.21.15.133:0/991012298 shutdown_connections 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f6711883740 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ee760ac440 0x55ee760ae900 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f6711883740 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee76028970 0x55ee76051760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f6711883740 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee760142c0 0x55ee76054cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f6711883740 1 --2- 172.21.15.133:0/991012298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee760139a0 0x55ee76054790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f6711883740 1 -- 172.21.15.133:0/991012298 >> 172.21.15.133:0/991012298 conn(0x55ee75fde310 msgr2=0x55ee76020f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f66f0ff9640 0 client.25258 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f6711883740 1 -- 172.21.15.133:0/991012298 shutdown_connections 2024-09-11T23:37:46.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f6711883740 1 -- 172.21.15.133:0/991012298 wait complete. 2024-09-11T23:37:46.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.346+0000 7f6711883740 20 client.25258 trim_cache size 0 max 16384 2024-09-11T23:37:46.349 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:37:46.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.347+0000 7f670f4aa640 1 -- 172.21.15.133:0/1074772376 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66f40a2fa0 msgr2=0x7f66f40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:46.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.347+0000 7f670f4aa640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66f40a2fa0 0x7f66f40a5460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6700005fa0 tx=0x7f670002a950 comp rx=0 tx=0).stop 2024-09-11T23:37:46.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.347+0000 7f670f4aa640 1 -- 172.21.15.133:0/1074772376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6708100e20 msgr2=0x7f67081973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:46.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.347+0000 7f670f4aa640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6708100e20 0x7f67081973d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f670400d950 tx=0x7f670400de10 comp rx=0 tx=0).stop 2024-09-11T23:37:46.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.347+0000 7f670f4aa640 1 -- 172.21.15.133:0/1074772376 shutdown_connections 2024-09-11T23:37:46.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.347+0000 7f670f4aa640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66f40a2fa0 0x7f66f40a5460 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.347+0000 7f670f4aa640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6708102ef0 0x7f670819bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.347+0000 7f670f4aa640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6708102570 0x7f6708197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.347+0000 7f670f4aa640 1 --2- 172.21.15.133:0/1074772376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6708100e20 0x7f67081973d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:46.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.347+0000 7f670f4aa640 1 -- 172.21.15.133:0/1074772376 >> 172.21.15.133:0/1074772376 conn(0x7f67080fc7d0 msgr2=0x7f6708106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:46.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.348+0000 7f670f4aa640 1 -- 172.21.15.133:0/1074772376 shutdown_connections 2024-09-11T23:37:46.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:46.348+0000 7f670f4aa640 1 -- 172.21.15.133:0/1074772376 wait complete. 2024-09-11T23:37:46.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:46 smithi138 ceph-mon[35142]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T23:37:46.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:46 smithi133 ceph-mon[31430]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T23:37:46.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:46 smithi178 ceph-mon[35882]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T23:37:48.198 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:37:48.199 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:37:48.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:48 smithi138 ceph-mon[35142]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:37:48.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:48 smithi133 ceph-mon[31430]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:37:48.482 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:48.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:48 smithi178 ceph-mon[35882]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T23:37:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.997+0000 7f078368c640 1 -- 172.21.15.133:0/830497852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c1027c0 msgr2=0x7f077c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.997+0000 7f0773fff640 1 -- 172.21.15.133:0/830497852 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c035020 con 0x7f077c1027c0 2024-09-11T23:37:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.997+0000 7f078368c640 1 --2- 172.21.15.133:0/830497852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c1027c0 0x7f077c106810 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f076c009900 tx=0x7f076c02a410 comp rx=0 tx=0).stop 2024-09-11T23:37:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.998+0000 7f078368c640 1 -- 172.21.15.133:0/830497852 shutdown_connections 2024-09-11T23:37:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.998+0000 7f078368c640 1 --2- 172.21.15.133:0/830497852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f077c106f40 0x7f077c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.998+0000 7f078368c640 1 --2- 172.21.15.133:0/830497852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c1027c0 0x7f077c106810 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.998+0000 7f078368c640 1 --2- 172.21.15.133:0/830497852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f077c101e10 0x7f077c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.998+0000 7f078368c640 1 -- 172.21.15.133:0/830497852 >> 172.21.15.133:0/830497852 conn(0x7f077c0fd8d0 msgr2=0x7f077c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.998+0000 7f078368c640 1 -- 172.21.15.133:0/830497852 shutdown_connections 2024-09-11T23:37:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.998+0000 7f078368c640 1 -- 172.21.15.133:0/830497852 wait complete. 2024-09-11T23:37:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f078368c640 1 Processor -- start 2024-09-11T23:37:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f078368c640 1 -- start start 2024-09-11T23:37:49.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f078368c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f077c101e10 0x7f077c075ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f078368c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f077c106f40 0x7f077c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f078368c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c06d1a0 0x7f077c06d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f078368c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f077c111ff0 con 0x7f077c06d1a0 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f078368c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f077c111e70 con 0x7f077c106f40 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f078368c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f077c112170 con 0x7f077c101e10 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f0781c29640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c06d1a0 0x7f077c06d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f0781c29640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c06d1a0 0x7f077c06d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42696/0 (socket says 172.21.15.133:42696) 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f0781c29640 1 -- 172.21.15.133:0/1569991959 learned_addr learned my addr 172.21.15.133:0/1569991959 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:48.999+0000 7f0781428640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f077c101e10 0x7f077c075ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.000+0000 7f0781c29640 1 -- 172.21.15.133:0/1569991959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f077c101e10 msgr2=0x7f077c075ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.000+0000 7f0781c29640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f077c101e10 0x7f077c075ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.000+0000 7f0781c29640 1 -- 172.21.15.133:0/1569991959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f077c106f40 msgr2=0x7f077c06cc60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.000+0000 7f0781c29640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f077c106f40 0x7f077c06cc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.000+0000 7f0781c29640 1 -- 172.21.15.133:0/1569991959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c0716f0 con 0x7f077c06d1a0 2024-09-11T23:37:49.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.000+0000 7f0781c29640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c06d1a0 0x7f077c06d650 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f077400eaa0 tx=0x7f077400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:49.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.000+0000 7f07727fc640 1 -- 172.21.15.133:0/1569991959 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077400cdb0 con 0x7f077c06d1a0 2024-09-11T23:37:49.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.000+0000 7f07727fc640 1 -- 172.21.15.133:0/1569991959 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0774005020 con 0x7f077c06d1a0 2024-09-11T23:37:49.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.000+0000 7f078368c640 1 -- 172.21.15.133:0/1569991959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077c071980 con 0x7f077c06d1a0 2024-09-11T23:37:49.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.001+0000 7f078368c640 1 -- 172.21.15.133:0/1569991959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077c071e60 con 0x7f077c06d1a0 2024-09-11T23:37:49.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.001+0000 7f07727fc640 1 -- 172.21.15.133:0/1569991959 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07740116b0 con 0x7f077c06d1a0 2024-09-11T23:37:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.004+0000 7f07727fc640 1 -- 172.21.15.133:0/1569991959 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f0774011850 con 0x7f077c06d1a0 2024-09-11T23:37:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.004+0000 7f0785a65740 1 Processor -- start 2024-09-11T23:37:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.005+0000 7f0785a65740 1 -- start start 2024-09-11T23:37:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.005+0000 7f0785a65740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b00b60 0x556b29b00f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.005+0000 7f0785a65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b01480 0x556b29b0b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.005+0000 7f0785a65740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b0bd40 0x556b29b0e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.005+0000 7f0785a65740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556b29a66280 con 0x556b29b0bd40 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.005+0000 7f0785a65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556b29a0db10 con 0x556b29b01480 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.005+0000 7f0785a65740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556b29a62240 con 0x556b29b00b60 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.006+0000 7f07727fc640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07680a2f60 0x7f07680a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.006+0000 7f0781428640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b01480 0x556b29b0b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.006+0000 7f07727fc640 1 -- 172.21.15.133:0/1569991959 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f07740cf7c0 con 0x7f077c06d1a0 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.006+0000 7f0781428640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b01480 0x556b29b0b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38460/0 (socket says 172.21.15.133:38460) 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.006+0000 7f0781428640 1 -- 172.21.15.133:0/3568940804 learned_addr learned my addr 172.21.15.133:0/3568940804 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.006+0000 7f0781c29640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07680a2f60 0x7f07680a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.006+0000 7f0780c27640 1 --2- 172.21.15.133:0/3568940804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b0bd40 0x556b29b0e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.007+0000 7f0781c29640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07680a2f60 0x7f07680a5420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f07740ccda0 tx=0x7f0774013000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.007+0000 7f0781428640 1 -- 172.21.15.133:0/3568940804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b00b60 msgr2=0x556b29b00f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.007+0000 7f0781428640 1 --2- 172.21.15.133:0/3568940804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b00b60 0x556b29b00f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.007+0000 7f0781428640 1 -- 172.21.15.133:0/3568940804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b0bd40 msgr2=0x556b29b0e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.007+0000 7f0781428640 1 --2- 172.21.15.133:0/3568940804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b0bd40 0x556b29b0e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.007+0000 7f0781428640 1 -- 172.21.15.133:0/3568940804 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b29a1a1f0 con 0x556b29b01480 2024-09-11T23:37:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.007+0000 7f0780c27640 1 --2- 172.21.15.133:0/3568940804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b0bd40 0x556b29b0e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.008+0000 7f0781428640 1 --2- 172.21.15.133:0/3568940804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b01480 0x556b29b0b800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0778009d20 tx=0x7f0778011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cc5dff9f2d20835 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.008+0000 7f07677fe640 1 -- 172.21.15.133:0/3568940804 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077801e070 con 0x556b29b01480 2024-09-11T23:37:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.008+0000 7f07677fe640 1 -- 172.21.15.133:0/3568940804 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0778019b10 con 0x556b29b01480 2024-09-11T23:37:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.008+0000 7f07677fe640 1 -- 172.21.15.133:0/3568940804 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077800ebf0 con 0x556b29b01480 2024-09-11T23:37:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.009+0000 7f0785a65740 1 -- 172.21.15.133:0/3568940804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b01480 msgr2=0x556b29b0b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.009+0000 7f0785a65740 1 --2- 172.21.15.133:0/3568940804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b01480 0x556b29b0b800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0778009d20 tx=0x7f0778011810 comp rx=0 tx=0).stop 2024-09-11T23:37:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.009+0000 7f0785a65740 1 -- 172.21.15.133:0/3568940804 shutdown_connections 2024-09-11T23:37:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.009+0000 7f0785a65740 1 --2- 172.21.15.133:0/3568940804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b0bd40 0x556b29b0e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.009+0000 7f0785a65740 1 --2- 172.21.15.133:0/3568940804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b01480 0x556b29b0b800 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.009+0000 7f0785a65740 1 --2- 172.21.15.133:0/3568940804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b00b60 0x556b29b00f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.009+0000 7f0785a65740 1 -- 172.21.15.133:0/3568940804 >> 172.21.15.133:0/3568940804 conn(0x556b29afd0f0 msgr2=0x556b29afd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.009+0000 7f0785a65740 1 -- 172.21.15.133:0/3568940804 shutdown_connections 2024-09-11T23:37:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.009+0000 7f0785a65740 1 -- 172.21.15.133:0/3568940804 wait complete. 2024-09-11T23:37:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.010+0000 7f0785a65740 1 -- start start 2024-09-11T23:37:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.010+0000 7f0785a65740 1 Processor -- start 2024-09-11T23:37:49.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0785a65740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b00b60 0x556b29b08d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0785a65740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 0x556b29b09290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0785a65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b097d0 0x556b29b17de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0785a65740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556b29b0e910 con 0x556b29b00b60 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0785a65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556b29a20f10 con 0x556b29b097d0 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0785a65740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556b29b0ea90 con 0x556b29b0bd40 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0781428640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 0x556b29b09290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0781428640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 0x556b29b09290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:41872/0 (socket says 172.21.15.133:41872) 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0781428640 1 -- 172.21.15.133:0/3228907348 learned_addr learned my addr 172.21.15.133:0/3228907348 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0785a65740 10 client.0 _ll_register_callbacks cb 0x556b299e9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0781428640 1 -- 172.21.15.133:0/3228907348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b097d0 msgr2=0x556b29b17de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0781428640 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b097d0 0x556b29b17de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.011+0000 7f0781428640 1 -- 172.21.15.133:0/3228907348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b00b60 msgr2=0x556b29b08d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.012+0000 7f0780c27640 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b00b60 0x556b29b08d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.012+0000 7f0781428640 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b00b60 0x556b29b08d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.012+0000 7f0781428640 1 -- 172.21.15.133:0/3228907348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b297d76a0 con 0x556b29b0bd40 2024-09-11T23:37:49.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.012+0000 7f0780c27640 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b00b60 0x556b29b08d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:49.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.012+0000 7f0785a65740 1 -- 172.21.15.133:0/3228907348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0778019b10 con 0x556b29b0bd40 2024-09-11T23:37:49.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.012+0000 7f0781428640 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 0x556b29b09290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0778011d10 tx=0x7f0778019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d418afe8712a3cc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:49.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.012+0000 7f0764ff9640 10 client.25267 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:37:49.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.012+0000 7f0764ff9640 1 -- 172.21.15.133:0/3228907348 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077801e070 con 0x556b29b0bd40 2024-09-11T23:37:49.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.012+0000 7f0764ff9640 1 -- 172.21.15.133:0/3228907348 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f077800e8f0 con 0x556b29b0bd40 2024-09-11T23:37:49.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.012+0000 7f0764ff9640 1 -- 172.21.15.133:0/3228907348 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f077800eb30 con 0x556b29b0bd40 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.013+0000 7f0785a65740 10 client.25267 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.013+0000 7f0785a65740 1 -- 172.21.15.133:0/3228907348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=28}) -- 0x556b29b1aac0 con 0x556b29b0bd40 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.013+0000 7f0764ff9640 1 -- 172.21.15.133:0/3228907348 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f077800ecd0 con 0x556b29b0bd40 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.013+0000 7f0785a65740 10 client.25267 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.013+0000 7f0785a65740 10 client.25267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.013+0000 7f0785a65740 20 client.25267 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.013+0000 7f0785a65740 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556b29b1be10 0x556b29b1e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.013+0000 7f0785a65740 4 client.25267 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.013+0000 7f0785a65740 1 -- 172.21.15.133:0/3228907348 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b29a737f0 con 0x556b29b1be10 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.013+0000 7f0780c27640 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556b29b1be10 0x556b29b1e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.014+0000 7f0764ff9640 10 client.25267 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.014+0000 7f0780c27640 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556b29b1be10 0x556b29b1e200 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d7d772d9a52bcf server_cookie=a11ef095e810be2c in_seq=0 out_seq=0 2024-09-11T23:37:49.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0764ff9640 1 -- 172.21.15.133:0/3228907348 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b29a737f0 con 0x556b29b1be10 2024-09-11T23:37:49.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0764ff9640 10 client.25267 handle_command_reply: tid=0 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0785a65740 1 client.25267 shutdown 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0785a65740 1 -- 172.21.15.133:0/3228907348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 msgr2=0x556b29b09290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0785a65740 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 0x556b29b09290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0778011d10 tx=0x7f0778019660 comp rx=0 tx=0).stop 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0785a65740 1 -- 172.21.15.133:0/3228907348 shutdown_connections 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0785a65740 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556b29b1be10 0x556b29b1e200 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0764ff9640 0 client.25267 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0785a65740 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b097d0 0x556b29b17de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0785a65740 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 0x556b29b09290 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0785a65740 1 --2- 172.21.15.133:0/3228907348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b00b60 0x556b29b08d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0785a65740 1 -- 172.21.15.133:0/3228907348 >> 172.21.15.133:0/3228907348 conn(0x556b29afd0f0 msgr2=0x556b29b02e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0785a65740 1 -- 172.21.15.133:0/3228907348 shutdown_connections 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.015+0000 7f0785a65740 1 -- 172.21.15.133:0/3228907348 wait complete. 2024-09-11T23:37:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.016+0000 7f0785a65740 20 client.25267 trim_cache size 0 max 16384 2024-09-11T23:37:49.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.048+0000 7f0785a65740 1 Processor -- start 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.048+0000 7f0785a65740 1 -- start start 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.048+0000 7f0785a65740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b00b60 0x556b29b0ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.048+0000 7f0785a65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b097d0 0x556b29b17bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.048+0000 7f0785a65740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b0bd40 0x556b29b23970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.048+0000 7f0785a65740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f076000e540 con 0x556b29b0bd40 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.048+0000 7f0785a65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0760010080 con 0x556b29b097d0 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.048+0000 7f0785a65740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f076000ec10 con 0x556b29b00b60 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.049+0000 7f0781428640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b097d0 0x556b29b17bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.049+0000 7f0781428640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b097d0 0x556b29b17bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38476/0 (socket says 172.21.15.133:38476) 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.049+0000 7f0781428640 1 -- 172.21.15.133:0/1703757892 learned_addr learned my addr 172.21.15.133:0/1703757892 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.049+0000 7f0781428640 1 -- 172.21.15.133:0/1703757892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b00b60 msgr2=0x556b29b0ac20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.049+0000 7f0780c27640 1 --2- 172.21.15.133:0/1703757892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b00b60 0x556b29b0ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.049+0000 7f0781428640 1 --2- 172.21.15.133:0/1703757892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b00b60 0x556b29b0ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.049+0000 7f0781428640 1 -- 172.21.15.133:0/1703757892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b0bd40 msgr2=0x556b29b23970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.049+0000 7f0781428640 1 --2- 172.21.15.133:0/1703757892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b0bd40 0x556b29b23970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.049+0000 7f0781428640 1 -- 172.21.15.133:0/1703757892 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b29b008e0 con 0x556b29b097d0 2024-09-11T23:37:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.049+0000 7f0781428640 1 --2- 172.21.15.133:0/1703757892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b097d0 0x556b29b17bd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0778009bf0 tx=0x7f077801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=411b72b3dda24bc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.049+0000 7f0780c27640 1 --2- 172.21.15.133:0/1703757892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b00b60 0x556b29b0ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.050+0000 7f0766ffd640 1 -- 172.21.15.133:0/1703757892 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077800e860 con 0x556b29b097d0 2024-09-11T23:37:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.050+0000 7f0766ffd640 1 -- 172.21.15.133:0/1703757892 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0778019b10 con 0x556b29b097d0 2024-09-11T23:37:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.050+0000 7f0766ffd640 1 -- 172.21.15.133:0/1703757892 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778004640 con 0x556b29b097d0 2024-09-11T23:37:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.051+0000 7f0785a65740 1 -- 172.21.15.133:0/1703757892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b097d0 msgr2=0x556b29b17bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.051+0000 7f0785a65740 1 --2- 172.21.15.133:0/1703757892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b097d0 0x556b29b17bd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0778009bf0 tx=0x7f077801e000 comp rx=0 tx=0).stop 2024-09-11T23:37:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.051+0000 7f0785a65740 1 -- 172.21.15.133:0/1703757892 shutdown_connections 2024-09-11T23:37:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.051+0000 7f0785a65740 1 --2- 172.21.15.133:0/1703757892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b0bd40 0x556b29b23970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.051+0000 7f0785a65740 1 --2- 172.21.15.133:0/1703757892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b097d0 0x556b29b17bd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.051+0000 7f0785a65740 1 --2- 172.21.15.133:0/1703757892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b00b60 0x556b29b0ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.051+0000 7f0785a65740 1 -- 172.21.15.133:0/1703757892 >> 172.21.15.133:0/1703757892 conn(0x556b29afd0f0 msgr2=0x556b29afba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.051+0000 7f0785a65740 1 -- 172.21.15.133:0/1703757892 shutdown_connections 2024-09-11T23:37:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.051+0000 7f0785a65740 1 -- 172.21.15.133:0/1703757892 wait complete. 2024-09-11T23:37:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.052+0000 7f0785a65740 1 -- start start 2024-09-11T23:37:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.052+0000 7f0785a65740 1 Processor -- start 2024-09-11T23:37:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.052+0000 7f0785a65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b00b60 0x556b29b13d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.052+0000 7f0785a65740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b097d0 0x556b29b14250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.052+0000 7f0785a65740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 0x556b29b10ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.052+0000 7f0785a65740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f076000f9b0 con 0x556b29b097d0 2024-09-11T23:37:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.052+0000 7f0785a65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f076000f2e0 con 0x556b29b00b60 2024-09-11T23:37:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.052+0000 7f0785a65740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f076000c520 con 0x556b29b0bd40 2024-09-11T23:37:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.052+0000 7f0785a65740 10 client.0 _ll_register_callbacks cb 0x7f077801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.053+0000 7f0780c27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b00b60 0x556b29b13d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.053+0000 7f0780c27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b00b60 0x556b29b13d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38480/0 (socket says 172.21.15.133:38480) 2024-09-11T23:37:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.053+0000 7f0780c27640 1 -- 172.21.15.133:0/271748433 learned_addr learned my addr 172.21.15.133:0/271748433 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.053+0000 7f0780c27640 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 0x556b29b10ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.053+0000 7f0780c27640 1 -- 172.21.15.133:0/271748433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 msgr2=0x556b29b10ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.053+0000 7f0780c27640 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 0x556b29b10ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.053+0000 7f0780c27640 1 -- 172.21.15.133:0/271748433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b097d0 msgr2=0x556b29b14250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.053+0000 7f0781428640 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b097d0 0x556b29b14250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.053+0000 7f0780c27640 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b097d0 0x556b29b14250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.053+0000 7f0780c27640 1 -- 172.21.15.133:0/271748433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b29b11220 con 0x556b29b00b60 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.053+0000 7f0781428640 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b097d0 0x556b29b14250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.054+0000 7f0780c27640 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b00b60 0x556b29b13d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f076c02a450 tx=0x7f076c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa6516e69bf4a6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.054+0000 7f0764ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.054+0000 7f0764ff9640 1 -- 172.21.15.133:0/271748433 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c046040 con 0x556b29b00b60 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.054+0000 7f0785a65740 1 -- 172.21.15.133:0/271748433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0778019b10 con 0x556b29b00b60 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.054+0000 7f0764ff9640 1 -- 172.21.15.133:0/271748433 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f076c005460 con 0x556b29b00b60 2024-09-11T23:37:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.054+0000 7f0764ff9640 1 -- 172.21.15.133:0/271748433 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f076c035040 con 0x556b29b00b60 2024-09-11T23:37:49.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.054+0000 7f0785a65740 10 client.25223 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:49.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.054+0000 7f0785a65740 1 -- 172.21.15.133:0/271748433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b29b87100 con 0x556b29b00b60 2024-09-11T23:37:49.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.055+0000 7f0764ff9640 1 -- 172.21.15.133:0/271748433 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0778019b10 con 0x556b29b00b60 2024-09-11T23:37:49.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.055+0000 7f0764ff9640 1 -- 172.21.15.133:0/271748433 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f076c03a040 con 0x556b29b00b60 2024-09-11T23:37:49.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.055+0000 7f0785a65740 10 client.25223 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:49.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.055+0000 7f0785a65740 10 client.25223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:49.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.055+0000 7f0785a65740 20 client.25223 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:49.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.055+0000 7f0785a65740 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556b29b877e0 0x556b29b89c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:49.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.055+0000 7f0785a65740 4 client.25223 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:37:49.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.055+0000 7f0785a65740 1 -- 172.21.15.133:0/271748433 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556b29b09620 con 0x556b29b877e0 2024-09-11T23:37:49.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.055+0000 7f0781428640 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556b29b877e0 0x556b29b89c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.056+0000 7f0781428640 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556b29b877e0 0x556b29b89c00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2246b10a1507f159 server_cookie=58ec6d18302c7465 in_seq=0 out_seq=0 2024-09-11T23:37:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.056+0000 7f0764ff9640 10 client.25223 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0764ff9640 1 -- 172.21.15.133:0/271748433 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556b29b09620 con 0x556b29b877e0 2024-09-11T23:37:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0764ff9640 10 client.25223 handle_command_reply: tid=0 2024-09-11T23:37:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0785a65740 1 client.25223 shutdown 2024-09-11T23:37:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0785a65740 1 -- 172.21.15.133:0/271748433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b00b60 msgr2=0x556b29b13d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0785a65740 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b00b60 0x556b29b13d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f076c02a450 tx=0x7f076c0058e0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0785a65740 1 -- 172.21.15.133:0/271748433 shutdown_connections 2024-09-11T23:37:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0785a65740 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556b29b877e0 0x556b29b89c00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0764ff9640 0 client.25223 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0785a65740 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b29b0bd40 0x556b29b10ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0785a65740 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b29b097d0 0x556b29b14250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0785a65740 1 --2- 172.21.15.133:0/271748433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b29b00b60 0x556b29b13d10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.057+0000 7f0785a65740 1 -- 172.21.15.133:0/271748433 >> 172.21.15.133:0/271748433 conn(0x556b29afd0f0 msgr2=0x556b29afba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:49.060 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:37:49.060 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:37:49.060 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "9d26949e-bccc-40ae-96d7-5f434e68c055", 2024-09-11T23:37:49.060 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:37:49.061 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:37:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.058+0000 7f0785a65740 1 -- 172.21.15.133:0/271748433 shutdown_connections 2024-09-11T23:37:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.058+0000 7f0785a65740 1 -- 172.21.15.133:0/271748433 wait complete. 2024-09-11T23:37:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.058+0000 7f0785a65740 20 client.25223 trim_cache size 0 max 16384 2024-09-11T23:37:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.058+0000 7f078368c640 1 -- 172.21.15.133:0/1569991959 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07680a2f60 msgr2=0x7f07680a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.058+0000 7f078368c640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07680a2f60 0x7f07680a5420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f07740ccda0 tx=0x7f0774013000 comp rx=0 tx=0).stop 2024-09-11T23:37:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.058+0000 7f078368c640 1 -- 172.21.15.133:0/1569991959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c06d1a0 msgr2=0x7f077c06d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:49.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.058+0000 7f078368c640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c06d1a0 0x7f077c06d650 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f077400eaa0 tx=0x7f077400ef60 comp rx=0 tx=0).stop 2024-09-11T23:37:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.059+0000 7f078368c640 1 -- 172.21.15.133:0/1569991959 shutdown_connections 2024-09-11T23:37:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.059+0000 7f078368c640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07680a2f60 0x7f07680a5420 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.059+0000 7f078368c640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c06d1a0 0x7f077c06d650 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.059+0000 7f078368c640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f077c106f40 0x7f077c06cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.059+0000 7f078368c640 1 --2- 172.21.15.133:0/1569991959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f077c101e10 0x7f077c075ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.059+0000 7f078368c640 1 -- 172.21.15.133:0/1569991959 >> 172.21.15.133:0/1569991959 conn(0x7f077c0fd8d0 msgr2=0x7f077c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.059+0000 7f078368c640 1 -- 172.21.15.133:0/1569991959 shutdown_connections 2024-09-11T23:37:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:49.059+0000 7f078368c640 1 -- 172.21.15.133:0/1569991959 wait complete. 2024-09-11T23:37:49.843 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:37:50.124 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:50.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:50 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:37:50.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:50 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:37:50.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:50 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:37:50.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:50 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:37:50.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:50 smithi133 ceph-mon[31430]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-11T23:37:50.400 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:50 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:37:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:50 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:37:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:50 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:37:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:50 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:37:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:50 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:37:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:50 smithi138 ceph-mon[35142]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-11T23:37:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:50 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:37:50.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:50 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:37:50.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:50 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:37:50.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:50 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:37:50.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:50 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:37:50.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:50 smithi178 ceph-mon[35882]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-11T23:37:50.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:50 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:37:50.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.607+0000 7fa4be57e640 1 -- 172.21.15.133:0/1246217713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4b8100e20 msgr2=0x7fa4b8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:50.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.607+0000 7fa4be57e640 1 --2- 172.21.15.133:0/1246217713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4b8100e20 0x7fa4b8101200 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009900 tx=0x7fa4a802a480 comp rx=0 tx=0).stop 2024-09-11T23:37:50.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.607+0000 7fa4be57e640 1 -- 172.21.15.133:0/1246217713 shutdown_connections 2024-09-11T23:37:50.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.607+0000 7fa4be57e640 1 --2- 172.21.15.133:0/1246217713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8102ef0 0x7fa4b810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.607+0000 7fa4be57e640 1 --2- 172.21.15.133:0/1246217713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b8102570 0x7fa4b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.607+0000 7fa4be57e640 1 --2- 172.21.15.133:0/1246217713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4b8100e20 0x7fa4b8101200 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.607+0000 7fa4be57e640 1 -- 172.21.15.133:0/1246217713 >> 172.21.15.133:0/1246217713 conn(0x7fa4b80fc7d0 msgr2=0x7fa4b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:50.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.607+0000 7fa4be57e640 1 -- 172.21.15.133:0/1246217713 shutdown_connections 2024-09-11T23:37:50.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.607+0000 7fa4be57e640 1 -- 172.21.15.133:0/1246217713 wait complete. 2024-09-11T23:37:50.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.608+0000 7fa4be57e640 1 Processor -- start 2024-09-11T23:37:50.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.608+0000 7fa4be57e640 1 -- start start 2024-09-11T23:37:50.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.608+0000 7fa4be57e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b8100e20 0x7fa4b8113340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.608+0000 7fa4be57e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4b8102570 0x7fa4b81116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.608+0000 7fa4be57e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8102ef0 0x7fa4b8111e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4be57e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8110f80 con 0x7fa4b8102570 2024-09-11T23:37:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4be57e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8110e00 con 0x7fa4b8102ef0 2024-09-11T23:37:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4be57e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8111100 con 0x7fa4b8100e20 2024-09-11T23:37:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4b8102570 0x7fa4b81116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4b8102570 0x7fa4b81116e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42746/0 (socket says 172.21.15.133:42746) 2024-09-11T23:37:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4b77fe640 1 -- 172.21.15.133:0/3514044170 learned_addr learned my addr 172.21.15.133:0/3514044170 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4b7fff640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b8100e20 0x7fa4b8113340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4bcb1b640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8102ef0 0x7fa4b8111e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4b77fe640 1 -- 172.21.15.133:0/3514044170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b8100e20 msgr2=0x7fa4b8113340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b8100e20 0x7fa4b8113340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4b77fe640 1 -- 172.21.15.133:0/3514044170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8102ef0 msgr2=0x7fa4b8111e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8102ef0 0x7fa4b8111e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4b77fe640 1 -- 172.21.15.133:0/3514044170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b81a6a80 con 0x7fa4b8102570 2024-09-11T23:37:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4bcb1b640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8102ef0 0x7fa4b8111e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4b7fff640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b8100e20 0x7fa4b8113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.609+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4b8102570 0x7fa4b81116e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00d950 tx=0x7fa4ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.610+0000 7fa4b57fa640 1 -- 172.21.15.133:0/3514044170 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac017030 con 0x7fa4b8102570 2024-09-11T23:37:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.610+0000 7fa4be57e640 1 -- 172.21.15.133:0/3514044170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b81a6c50 con 0x7fa4b8102570 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.610+0000 7fa4b57fa640 1 -- 172.21.15.133:0/3514044170 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac004020 con 0x7fa4b8102570 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.610+0000 7fa4b57fa640 1 -- 172.21.15.133:0/3514044170 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0047a0 con 0x7fa4b8102570 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.610+0000 7fa4be57e640 1 -- 172.21.15.133:0/3514044170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b81a71b0 con 0x7fa4b8102570 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.613+0000 7fa4c0957740 1 Processor -- start 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.613+0000 7fa4b57fa640 1 -- 172.21.15.133:0/3514044170 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fa4ac0041d0 con 0x7fa4b8102570 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.613+0000 7fa4c0957740 1 -- start start 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92954a0 0x5639f9295880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f9295dc0 0x5639f92a0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92a0680 0x5639f92a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4c0957740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5639f91fb2e0 con 0x5639f92954a0 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4c0957740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5639f91a2ce0 con 0x5639f9295dc0 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4c0957740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5639f91f72a0 con 0x5639f92a0680 2024-09-11T23:37:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4bcb1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92954a0 0x5639f9295880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4bcb1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92954a0 0x5639f9295880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42754/0 (socket says 172.21.15.133:42754) 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4bcb1b640 1 -- 172.21.15.133:0/1459860492 learned_addr learned my addr 172.21.15.133:0/1459860492 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4b7fff640 1 --2- 172.21.15.133:0/1459860492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f9295dc0 0x5639f92a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4bcb1b640 1 -- 172.21.15.133:0/1459860492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92a0680 msgr2=0x5639f92a2a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4bcb1b640 1 --2- 172.21.15.133:0/1459860492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92a0680 0x5639f92a2a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4bcb1b640 1 -- 172.21.15.133:0/1459860492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f9295dc0 msgr2=0x5639f92a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4bcb1b640 1 --2- 172.21.15.133:0/1459860492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f9295dc0 0x5639f92a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.614+0000 7fa4bcb1b640 1 -- 172.21.15.133:0/1459860492 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f9187440 con 0x5639f92954a0 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.615+0000 7fa4b57fa640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4a00a2ea0 0x7fa4a00a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.615+0000 7fa4b57fa640 1 -- 172.21.15.133:0/3514044170 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa4ac0cf160 con 0x7fa4b8102570 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.615+0000 7fa4b7fff640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4a00a2ea0 0x7fa4a00a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.615+0000 7fa4bcb1b640 1 --2- 172.21.15.133:0/1459860492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92954a0 0x5639f9295880 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0007e60 tx=0x7fa4b000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2e003f6f7065aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.616+0000 7fa49e7fc640 1 -- 172.21.15.133:0/1459860492 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0014070 con 0x5639f92954a0 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.616+0000 7fa49e7fc640 1 -- 172.21.15.133:0/1459860492 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b0003e50 con 0x5639f92954a0 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.616+0000 7fa49e7fc640 1 -- 172.21.15.133:0/1459860492 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0004810 con 0x5639f92954a0 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4c0957740 1 -- 172.21.15.133:0/1459860492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92954a0 msgr2=0x5639f9295880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4b7fff640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4a00a2ea0 0x7fa4a00a5360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009230 tx=0x7fa4a80058d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4c0957740 1 --2- 172.21.15.133:0/1459860492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92954a0 0x5639f9295880 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0007e60 tx=0x7fa4b000e720 comp rx=0 tx=0).stop 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4c0957740 1 -- 172.21.15.133:0/1459860492 shutdown_connections 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4c0957740 1 --2- 172.21.15.133:0/1459860492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92a0680 0x5639f92a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4c0957740 1 --2- 172.21.15.133:0/1459860492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f9295dc0 0x5639f92a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4c0957740 1 --2- 172.21.15.133:0/1459860492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92954a0 0x5639f9295880 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4c0957740 1 -- 172.21.15.133:0/1459860492 >> 172.21.15.133:0/1459860492 conn(0x5639f9291a30 msgr2=0x5639f9291e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4c0957740 1 -- 172.21.15.133:0/1459860492 shutdown_connections 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4c0957740 1 -- 172.21.15.133:0/1459860492 wait complete. 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4c0957740 1 -- start start 2024-09-11T23:37:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.617+0000 7fa4c0957740 1 Processor -- start 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.618+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92954a0 0x5639f92a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.618+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f9295dc0 0x5639f929d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.618+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92a0680 0x5639f929db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.618+0000 7fa4c0957740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5639f92a3250 con 0x5639f92a0680 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.618+0000 7fa4c0957740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5639f91b6050 con 0x5639f92954a0 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.618+0000 7fa4c0957740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5639f92a33d0 con 0x5639f9295dc0 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.618+0000 7fa4c0957740 10 client.0 _ll_register_callbacks cb 0x5639f91f6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.618+0000 7fa4bcb1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92a0680 0x5639f929db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.618+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f9295dc0 0x5639f929d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.618+0000 7fa4bcb1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92a0680 0x5639f929db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42756/0 (socket says 172.21.15.133:42756) 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.618+0000 7fa4bcb1b640 1 -- 172.21.15.133:0/4093638744 learned_addr learned my addr 172.21.15.133:0/4093638744 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4b77fe640 1 -- 172.21.15.133:0/4093638744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92954a0 msgr2=0x5639f92a00e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/4093638744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92954a0 0x5639f92a00e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4b77fe640 1 -- 172.21.15.133:0/4093638744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92a0680 msgr2=0x5639f929db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/4093638744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92a0680 0x5639f929db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4b77fe640 1 -- 172.21.15.133:0/4093638744 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f8f6c3c0 con 0x5639f9295dc0 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/4093638744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f9295dc0 0x5639f929d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0cc720 tx=0x7fa4ac09b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc0071a6124bdbaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4977fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4977fe640 1 -- 172.21.15.133:0/4093638744 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0b3060 con 0x5639f9295dc0 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4977fe640 1 -- 172.21.15.133:0/4093638744 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa4ac09ab40 con 0x5639f9295dc0 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4c0957740 1 -- 172.21.15.133:0/4093638744 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4b0003e50 con 0x5639f9295dc0 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4977fe640 1 -- 172.21.15.133:0/4093638744 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4ac09bbf0 con 0x5639f9295dc0 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4c0957740 10 client.25279 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.619+0000 7fa4c0957740 1 -- 172.21.15.133:0/4093638744 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5639f92aed90 con 0x5639f9295dc0 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.620+0000 7fa4977fe640 1 -- 172.21.15.133:0/4093638744 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b0003e50 con 0x5639f9295dc0 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.620+0000 7fa4977fe640 1 -- 172.21.15.133:0/4093638744 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4ac097240 con 0x5639f9295dc0 2024-09-11T23:37:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.620+0000 7fa4c0957740 10 client.25279 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.620+0000 7fa4c0957740 10 client.25279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.620+0000 7fa4c0957740 20 client.25279 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.620+0000 7fa4c0957740 1 --2- 172.21.15.133:0/4093638744 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5639f92b01a0 0x5639f92b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.620+0000 7fa4c0957740 4 client.25279 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.620+0000 7fa4c0957740 1 -- 172.21.15.133:0/4093638744 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639f9208850 con 0x5639f92b01a0 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.620+0000 7fa4b7fff640 1 --2- 172.21.15.133:0/4093638744 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5639f92b01a0 0x5639f92b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.621+0000 7fa4977fe640 10 client.25279 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.621+0000 7fa4b7fff640 1 --2- 172.21.15.133:0/4093638744 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5639f92b01a0 0x5639f92b2590 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f81efa0112e697ef server_cookie=ead8b55c4776439f in_seq=0 out_seq=0 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4977fe640 1 -- 172.21.15.133:0/4093638744 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639f9208850 con 0x5639f92b01a0 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4977fe640 10 client.25279 handle_command_reply: tid=0 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 1 client.25279 shutdown 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 1 -- 172.21.15.133:0/4093638744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f9295dc0 msgr2=0x5639f929d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 1 --2- 172.21.15.133:0/4093638744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f9295dc0 0x5639f929d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0cc720 tx=0x7fa4ac09b3b0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 1 -- 172.21.15.133:0/4093638744 shutdown_connections 2024-09-11T23:37:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 1 --2- 172.21.15.133:0/4093638744 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5639f92b01a0 0x5639f92b2590 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 1 --2- 172.21.15.133:0/4093638744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f92a0680 0x5639f929db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 1 --2- 172.21.15.133:0/4093638744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f9295dc0 0x5639f929d390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4977fe640 0 client.25279 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 1 --2- 172.21.15.133:0/4093638744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92954a0 0x5639f92a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 1 -- 172.21.15.133:0/4093638744 >> 172.21.15.133:0/4093638744 conn(0x5639f9291a30 msgr2=0x5639f9297760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 1 -- 172.21.15.133:0/4093638744 shutdown_connections 2024-09-11T23:37:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 1 -- 172.21.15.133:0/4093638744 wait complete. 2024-09-11T23:37:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.622+0000 7fa4c0957740 20 client.25279 trim_cache size 0 max 16384 2024-09-11T23:37:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.655+0000 7fa4c0957740 1 Processor -- start 2024-09-11T23:37:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.655+0000 7fa4c0957740 1 -- start start 2024-09-11T23:37:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.655+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92954a0 0x5639f9297650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.655+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.655+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.655+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92b7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42764/0 (socket says 172.21.15.133:42764) 2024-09-11T23:37:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.655+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92a0680 0x5639f92b8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.655+0000 7fa4b77fe640 1 -- 172.21.15.133:0/92185631 learned_addr learned my addr 172.21.15.133:0/92185631 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.655+0000 7fa4c0957740 1 -- 172.21.15.133:0/92185631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81af920 con 0x5639f9295dc0 2024-09-11T23:37:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.655+0000 7fa4c0957740 1 -- 172.21.15.133:0/92185631 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81b1460 con 0x5639f92a0680 2024-09-11T23:37:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.655+0000 7fa4c0957740 1 -- 172.21.15.133:0/92185631 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81afff0 con 0x5639f92954a0 2024-09-11T23:37:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.656+0000 7fa4b77fe640 1 -- 172.21.15.133:0/92185631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92954a0 msgr2=0x5639f9297650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.656+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/92185631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92954a0 0x5639f9297650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.656+0000 7fa4b77fe640 1 -- 172.21.15.133:0/92185631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92a0680 msgr2=0x5639f92b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.656+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/92185631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92a0680 0x5639f92b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.656+0000 7fa4b77fe640 1 -- 172.21.15.133:0/92185631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f929c380 con 0x5639f9295dc0 2024-09-11T23:37:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.656+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/92185631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92b7e10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac005fd0 tx=0x7fa4ac0cc7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8094f97ee6be2980 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.656+0000 7fa49dffb640 1 -- 172.21.15.133:0/92185631 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac09f8a0 con 0x5639f9295dc0 2024-09-11T23:37:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.656+0000 7fa49dffb640 1 -- 172.21.15.133:0/92185631 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac0a4160 con 0x5639f9295dc0 2024-09-11T23:37:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.656+0000 7fa49dffb640 1 -- 172.21.15.133:0/92185631 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac09a720 con 0x5639f9295dc0 2024-09-11T23:37:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.657+0000 7fa4c0957740 1 -- 172.21.15.133:0/92185631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 msgr2=0x5639f92b7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.657+0000 7fa4c0957740 1 --2- 172.21.15.133:0/92185631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92b7e10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac005fd0 tx=0x7fa4ac0cc7d0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.657+0000 7fa4c0957740 1 -- 172.21.15.133:0/92185631 shutdown_connections 2024-09-11T23:37:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.657+0000 7fa4c0957740 1 --2- 172.21.15.133:0/92185631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92a0680 0x5639f92b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.657+0000 7fa4c0957740 1 --2- 172.21.15.133:0/92185631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92b7e10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.657+0000 7fa4c0957740 1 --2- 172.21.15.133:0/92185631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92954a0 0x5639f9297650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.657+0000 7fa4c0957740 1 -- 172.21.15.133:0/92185631 >> 172.21.15.133:0/92185631 conn(0x5639f9291a30 msgr2=0x5639f9296f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.659+0000 7fa4c0957740 1 -- 172.21.15.133:0/92185631 shutdown_connections 2024-09-11T23:37:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.659+0000 7fa4c0957740 1 -- 172.21.15.133:0/92185631 wait complete. 2024-09-11T23:37:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.659+0000 7fa4c0957740 1 -- start start 2024-09-11T23:37:50.669 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:37:51.155 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:37:51.155 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:37:51.155 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:37:51.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.659+0000 7fa4c0957740 1 Processor -- start 2024-09-11T23:37:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.659+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92954a0 0x5639f92a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.659+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.659+0000 7fa4c0957740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92a0680 0x5639f92a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4c0957740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81b0d90 con 0x5639f9295dc0 2024-09-11T23:37:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4c0957740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81b06c0 con 0x5639f92954a0 2024-09-11T23:37:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4c0957740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81ad900 con 0x5639f92a0680 2024-09-11T23:37:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4c0957740 10 client.0 _ll_register_callbacks cb 0x7fa4ac0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92a8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42766/0 (socket says 172.21.15.133:42766) 2024-09-11T23:37:51.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4b77fe640 1 -- 172.21.15.133:0/2896134118 learned_addr learned my addr 172.21.15.133:0/2896134118 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:51.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4b7fff640 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92954a0 0x5639f92a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4b77fe640 1 -- 172.21.15.133:0/2896134118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92a0680 msgr2=0x5639f92a55f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:51.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92a0680 0x5639f92a55f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4b77fe640 1 -- 172.21.15.133:0/2896134118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92954a0 msgr2=0x5639f92a8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:51.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92954a0 0x5639f92a8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4b77fe640 1 -- 172.21.15.133:0/2896134118 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f92b2900 con 0x5639f9295dc0 2024-09-11T23:37:51.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4b77fe640 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92a8b60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac097950 tx=0x7fa4ac09a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0f1ae32b21f9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:51.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.660+0000 7fa4977fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:51.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4977fe640 1 -- 172.21.15.133:0/2896134118 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac065070 con 0x5639f9295dc0 2024-09-11T23:37:51.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4977fe640 1 -- 172.21.15.133:0/2896134118 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa4ac0cca30 con 0x5639f9295dc0 2024-09-11T23:37:51.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4c0957740 1 -- 172.21.15.133:0/2896134118 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ac0a4160 con 0x5639f9295dc0 2024-09-11T23:37:51.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4977fe640 1 -- 172.21.15.133:0/2896134118 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4ac0b3070 con 0x5639f9295dc0 2024-09-11T23:37:51.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4c0957740 10 client.15813 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:51.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4c0957740 1 -- 172.21.15.133:0/2896134118 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5639f92b2900 con 0x5639f9295dc0 2024-09-11T23:37:51.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4977fe640 1 -- 172.21.15.133:0/2896134118 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac0973b0 con 0x5639f9295dc0 2024-09-11T23:37:51.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4977fe640 1 -- 172.21.15.133:0/2896134118 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4ac0a0a30 con 0x5639f9295dc0 2024-09-11T23:37:51.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4c0957740 10 client.15813 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:51.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4c0957740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:51.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4c0957740 20 client.15813 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:51.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4c0957740 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5639f931b700 0x5639f931db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4c0957740 4 client.15813 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:37:51.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.661+0000 7fa4c0957740 1 -- 172.21.15.133:0/2896134118 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa4ac0a4160 con 0x5639f931b700 2024-09-11T23:37:51.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.662+0000 7fa4b7fff640 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5639f931b700 0x5639f931db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.662+0000 7fa4977fe640 10 client.15813 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:51.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.662+0000 7fa4b7fff640 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5639f931b700 0x5639f931db20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c797b60456505a server_cookie=15b548a40a934353 in_seq=0 out_seq=0 2024-09-11T23:37:51.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.663+0000 7fa4977fe640 1 -- 172.21.15.133:0/2896134118 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fa4ac0a4160 con 0x5639f931b700 2024-09-11T23:37:51.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.663+0000 7fa4977fe640 10 client.15813 handle_command_reply: tid=0 2024-09-11T23:37:51.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.663+0000 7fa4c0957740 1 client.15813 shutdown 2024-09-11T23:37:51.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4c0957740 1 -- 172.21.15.133:0/2896134118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 msgr2=0x5639f92a8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4c0957740 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92a8b60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac097950 tx=0x7fa4ac09a6d0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4c0957740 1 -- 172.21.15.133:0/2896134118 shutdown_connections 2024-09-11T23:37:51.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4c0957740 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5639f931b700 0x5639f931db20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4c0957740 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5639f92a0680 0x5639f92a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4c0957740 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f9295dc0 0x5639f92a8b60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4977fe640 0 client.15813 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:51.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4c0957740 1 --2- 172.21.15.133:0/2896134118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639f92954a0 0x5639f92a8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4c0957740 1 -- 172.21.15.133:0/2896134118 >> 172.21.15.133:0/2896134118 conn(0x5639f9291a30 msgr2=0x5639f9290b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:51.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4c0957740 1 -- 172.21.15.133:0/2896134118 shutdown_connections 2024-09-11T23:37:51.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4c0957740 1 -- 172.21.15.133:0/2896134118 wait complete. 2024-09-11T23:37:51.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.668+0000 7fa4c0957740 20 client.15813 trim_cache size 0 max 16384 2024-09-11T23:37:51.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.669+0000 7fa4be57e640 1 -- 172.21.15.133:0/3514044170 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4a00a2ea0 msgr2=0x7fa4a00a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.669+0000 7fa4be57e640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4a00a2ea0 0x7fa4a00a5360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009230 tx=0x7fa4a80058d0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.669+0000 7fa4be57e640 1 -- 172.21.15.133:0/3514044170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4b8102570 msgr2=0x7fa4b81116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.669+0000 7fa4be57e640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4b8102570 0x7fa4b81116e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00d950 tx=0x7fa4ac00de10 comp rx=0 tx=0).stop 2024-09-11T23:37:51.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.673+0000 7fa4be57e640 1 -- 172.21.15.133:0/3514044170 shutdown_connections 2024-09-11T23:37:51.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.673+0000 7fa4be57e640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4a00a2ea0 0x7fa4a00a5360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.673+0000 7fa4be57e640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8102ef0 0x7fa4b8111e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.673+0000 7fa4be57e640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4b8102570 0x7fa4b81116e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.673+0000 7fa4be57e640 1 --2- 172.21.15.133:0/3514044170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b8100e20 0x7fa4b8113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.673+0000 7fa4be57e640 1 -- 172.21.15.133:0/3514044170 >> 172.21.15.133:0/3514044170 conn(0x7fa4b80fc7d0 msgr2=0x7fa4b80fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:51.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.673+0000 7fa4be57e640 1 -- 172.21.15.133:0/3514044170 shutdown_connections 2024-09-11T23:37:51.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:50.673+0000 7fa4be57e640 1 -- 172.21.15.133:0/3514044170 wait complete. 2024-09-11T23:37:51.416 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:37:51.416 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:37:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.754+0000 7fe670b6d640 1 -- 172.21.15.133:0/3224448588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe66c101210 msgr2=0x7fe66c10ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.754+0000 7fe670b6d640 1 --2- 172.21.15.133:0/3224448588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe66c101210 0x7fe66c10ded0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe660009900 tx=0x7fe66002a410 comp rx=0 tx=0).stop 2024-09-11T23:37:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.754+0000 7fe670b6d640 1 -- 172.21.15.133:0/3224448588 shutdown_connections 2024-09-11T23:37:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.754+0000 7fe670b6d640 1 --2- 172.21.15.133:0/3224448588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c10e410 0x7fe66c110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.754+0000 7fe670b6d640 1 --2- 172.21.15.133:0/3224448588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe66c101210 0x7fe66c10ded0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.754+0000 7fe670b6d640 1 --2- 172.21.15.133:0/3224448588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe66c100860 0x7fe66c100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.754+0000 7fe670b6d640 1 -- 172.21.15.133:0/3224448588 >> 172.21.15.133:0/3224448588 conn(0x7fe66c077990 msgr2=0x7fe66c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.754+0000 7fe670b6d640 1 -- 172.21.15.133:0/3224448588 shutdown_connections 2024-09-11T23:37:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.754+0000 7fe670b6d640 1 -- 172.21.15.133:0/3224448588 wait complete. 2024-09-11T23:37:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.755+0000 7fe670b6d640 1 Processor -- start 2024-09-11T23:37:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.755+0000 7fe670b6d640 1 -- start start 2024-09-11T23:37:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.755+0000 7fe670b6d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe66c100860 0x7fe66c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.755+0000 7fe670b6d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe66c101210 0x7fe66c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.755+0000 7fe670b6d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c10e410 0x7fe66c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.755+0000 7fe670b6d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1140c0 con 0x7fe66c101210 2024-09-11T23:37:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.755+0000 7fe670b6d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c113f40 con 0x7fe66c10e410 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.755+0000 7fe670b6d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c114240 con 0x7fe66c100860 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.755+0000 7fe66a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe66c100860 0x7fe66c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.755+0000 7fe66a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe66c100860 0x7fe66c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:41950/0 (socket says 172.21.15.133:41950) 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.755+0000 7fe66a59c640 1 -- 172.21.15.133:0/4086618693 learned_addr learned my addr 172.21.15.133:0/4086618693 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe66ad9d640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c10e410 0x7fe66c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe66a59c640 1 -- 172.21.15.133:0/4086618693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c10e410 msgr2=0x7fe66c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe66a59c640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c10e410 0x7fe66c19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe66a59c640 1 -- 172.21.15.133:0/4086618693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe66c101210 msgr2=0x7fe66c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe669d9b640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe66c101210 0x7fe66c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe66a59c640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe66c101210 0x7fe66c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe66a59c640 1 -- 172.21.15.133:0/4086618693 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c19f580 con 0x7fe66c100860 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe669d9b640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe66c101210 0x7fe66c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe66a59c640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe66c100860 0x7fe66c19a590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00e9f0 tx=0x7fe65c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe65b7fe640 1 -- 172.21.15.133:0/4086618693 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c00cd80 con 0x7fe66c100860 2024-09-11T23:37:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe65b7fe640 1 -- 172.21.15.133:0/4086618693 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe65c010910 con 0x7fe66c100860 2024-09-11T23:37:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.756+0000 7fe65b7fe640 1 -- 172.21.15.133:0/4086618693 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c018980 con 0x7fe66c100860 2024-09-11T23:37:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.757+0000 7fe670b6d640 1 -- 172.21.15.133:0/4086618693 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c19f870 con 0x7fe66c100860 2024-09-11T23:37:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.757+0000 7fe670b6d640 1 -- 172.21.15.133:0/4086618693 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c1a7100 con 0x7fe66c100860 2024-09-11T23:37:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.759+0000 7fe6739ea740 1 Processor -- start 2024-09-11T23:37:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.760+0000 7fe65b7fe640 1 -- 172.21.15.133:0/4086618693 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fe65c010ac0 con 0x7fe66c100860 2024-09-11T23:37:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.760+0000 7fe6739ea740 1 -- start start 2024-09-11T23:37:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.760+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x55792735a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.760+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927364f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.760+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x5579273678b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.760+0000 7fe6739ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5579272b62f0 con 0x55792735ac20 2024-09-11T23:37:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.760+0000 7fe6739ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55792725e270 con 0x55792735a300 2024-09-11T23:37:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.760+0000 7fe6739ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5579272b22b0 con 0x5579273654c0 2024-09-11T23:37:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.761+0000 7fe66ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x55792735a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.761+0000 7fe66a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927364f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.761+0000 7fe66ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x55792735a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38562/0 (socket says 172.21.15.133:38562) 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.761+0000 7fe66ad9d640 1 -- 172.21.15.133:0/4018773033 learned_addr learned my addr 172.21.15.133:0/4018773033 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.761+0000 7fe669d9b640 1 --2- 172.21.15.133:0/4018773033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x5579273678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.761+0000 7fe65b7fe640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6540a3020 0x7fe6540a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.761+0000 7fe66ad9d640 1 -- 172.21.15.133:0/4018773033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 msgr2=0x5579273678b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.761+0000 7fe66ad9d640 1 --2- 172.21.15.133:0/4018773033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x5579273678b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.761+0000 7fe66ad9d640 1 -- 172.21.15.133:0/4018773033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 msgr2=0x557927364f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.762+0000 7fe66ad9d640 1 --2- 172.21.15.133:0/4018773033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927364f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.762+0000 7fe66ad9d640 1 -- 172.21.15.133:0/4018773033 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55792725d340 con 0x55792735a300 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.762+0000 7fe66a59c640 1 --2- 172.21.15.133:0/4018773033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927364f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.762+0000 7fe65b7fe640 1 -- 172.21.15.133:0/4086618693 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe65c0cfaa0 con 0x7fe66c100860 2024-09-11T23:37:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.762+0000 7fe66ad9d640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6540a3020 0x7fe6540a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.762+0000 7fe66ad9d640 1 --2- 172.21.15.133:0/4018773033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x55792735a6e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe664009fb0 tx=0x7fe66400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f36dfe53a6f40a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.762+0000 7fe658ff9640 1 -- 172.21.15.133:0/4018773033 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664004b20 con 0x55792735a300 2024-09-11T23:37:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.762+0000 7fe658ff9640 1 -- 172.21.15.133:0/4018773033 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe664004cc0 con 0x55792735a300 2024-09-11T23:37:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.762+0000 7fe658ff9640 1 -- 172.21.15.133:0/4018773033 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664005250 con 0x55792735a300 2024-09-11T23:37:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.762+0000 7fe66ad9d640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6540a3020 0x7fe6540a54e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe664020000 tx=0x7fe66401d000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.763+0000 7fe6739ea740 1 -- 172.21.15.133:0/4018773033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 msgr2=0x55792735a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.763+0000 7fe6739ea740 1 --2- 172.21.15.133:0/4018773033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x55792735a6e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe664009fb0 tx=0x7fe66400f7f0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.766+0000 7fe6739ea740 1 -- 172.21.15.133:0/4018773033 shutdown_connections 2024-09-11T23:37:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.766+0000 7fe6739ea740 1 --2- 172.21.15.133:0/4018773033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x5579273678b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.766+0000 7fe6739ea740 1 --2- 172.21.15.133:0/4018773033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927364f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.766+0000 7fe6739ea740 1 --2- 172.21.15.133:0/4018773033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x55792735a6e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.766+0000 7fe6739ea740 1 -- 172.21.15.133:0/4018773033 >> 172.21.15.133:0/4018773033 conn(0x557927356870 msgr2=0x557927356c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.767+0000 7fe6739ea740 1 -- 172.21.15.133:0/4018773033 shutdown_connections 2024-09-11T23:37:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.767+0000 7fe6739ea740 1 -- 172.21.15.133:0/4018773033 wait complete. 2024-09-11T23:37:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.767+0000 7fe6739ea740 1 -- start start 2024-09-11T23:37:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.767+0000 7fe6739ea740 1 Processor -- start 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x557927362340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927362880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x557927362dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe6739ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557927368090 con 0x55792735ac20 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe6739ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55792723f290 con 0x55792735a300 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe6739ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557927368210 con 0x5579273654c0 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe6739ea740 10 client.0 _ll_register_callbacks cb 0x557927270140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe669d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927362880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe669d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927362880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42798/0 (socket says 172.21.15.133:42798) 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe669d9b640 1 -- 172.21.15.133:0/872425149 learned_addr learned my addr 172.21.15.133:0/872425149 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe66a59c640 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x557927362340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.768+0000 7fe66ad9d640 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x557927362dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe66a59c640 1 -- 172.21.15.133:0/872425149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 msgr2=0x557927362dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe66a59c640 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x557927362dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe66a59c640 1 -- 172.21.15.133:0/872425149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 msgr2=0x557927362880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe66a59c640 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927362880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe66a59c640 1 -- 172.21.15.133:0/872425149 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557926fc9210 con 0x55792735a300 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe66a59c640 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x557927362340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0cd020 tx=0x7fe65c09b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60f7272180791fe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe651ffb640 1 -- 172.21.15.133:0/872425149 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0a4a60 con 0x55792735a300 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe651ffb640 1 -- 172.21.15.133:0/872425149 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe65c010430 con 0x55792735a300 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe6739ea740 1 -- 172.21.15.133:0/872425149 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe664004cc0 con 0x55792735a300 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe651ffb640 1 -- 172.21.15.133:0/872425149 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe65c010630 con 0x55792735a300 2024-09-11T23:37:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe6739ea740 10 client.25241 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe6739ea740 1 -- 172.21.15.133:0/872425149 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557927095620 con 0x55792735a300 2024-09-11T23:37:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.769+0000 7fe651ffb640 1 -- 172.21.15.133:0/872425149 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe65c09f070 con 0x55792735a300 2024-09-11T23:37:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.771+0000 7fe651ffb640 1 -- 172.21.15.133:0/872425149 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe65c0cd3c0 con 0x55792735a300 2024-09-11T23:37:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.771+0000 7fe6739ea740 10 client.25241 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.771+0000 7fe6739ea740 10 client.25241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.771+0000 7fe6739ea740 20 client.25241 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.771+0000 7fe6739ea740 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557927375810 0x557927377c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.771+0000 7fe6739ea740 4 client.25241 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.771+0000 7fe6739ea740 1 -- 172.21.15.133:0/872425149 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579272c3a20 con 0x557927375810 2024-09-11T23:37:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.771+0000 7fe669d9b640 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557927375810 0x557927377c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.772+0000 7fe669d9b640 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557927375810 0x557927377c40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d25b7eaba5d44 server_cookie=2177b6696ef31afa in_seq=0 out_seq=0 2024-09-11T23:37:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.772+0000 7fe651ffb640 10 client.25241 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe651ffb640 1 -- 172.21.15.133:0/872425149 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579272c3a20 con 0x557927375810 2024-09-11T23:37:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe651ffb640 10 client.25241 handle_command_reply: tid=0 2024-09-11T23:37:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 1 client.25241 shutdown 2024-09-11T23:37:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 1 -- 172.21.15.133:0/872425149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 msgr2=0x557927362340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x557927362340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0cd020 tx=0x7fe65c09b810 comp rx=0 tx=0).stop 2024-09-11T23:37:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 1 -- 172.21.15.133:0/872425149 shutdown_connections 2024-09-11T23:37:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557927375810 0x557927377c40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x557927362dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927362880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe651ffb640 0 client.25241 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 1 --2- 172.21.15.133:0/872425149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x557927362340 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 1 -- 172.21.15.133:0/872425149 >> 172.21.15.133:0/872425149 conn(0x557927356870 msgr2=0x55792735c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 1 -- 172.21.15.133:0/872425149 shutdown_connections 2024-09-11T23:37:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 1 -- 172.21.15.133:0/872425149 wait complete. 2024-09-11T23:37:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.773+0000 7fe6739ea740 20 client.25241 trim_cache size 0 max 16384 2024-09-11T23:37:51.783 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe6739ea740 1 Processor -- start 2024-09-11T23:37:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe6739ea740 1 -- start start 2024-09-11T23:37:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x5579273642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927362bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x55792737d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe6739ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1ab340 con 0x55792735ac20 2024-09-11T23:37:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe6739ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1ace80 con 0x55792735a300 2024-09-11T23:37:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe6739ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1aba10 con 0x5579273654c0 2024-09-11T23:37:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe669d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927362bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe669d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927362bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42802/0 (socket says 172.21.15.133:42802) 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe669d9b640 1 -- 172.21.15.133:0/1003121906 learned_addr learned my addr 172.21.15.133:0/1003121906 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe66ad9d640 1 --2- 172.21.15.133:0/1003121906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x55792737d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.790+0000 7fe66a59c640 1 --2- 172.21.15.133:0/1003121906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x5579273642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.791+0000 7fe66a59c640 1 -- 172.21.15.133:0/1003121906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 msgr2=0x55792737d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.791+0000 7fe66a59c640 1 --2- 172.21.15.133:0/1003121906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x55792737d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.791+0000 7fe66a59c640 1 -- 172.21.15.133:0/1003121906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 msgr2=0x557927362bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.791+0000 7fe66a59c640 1 --2- 172.21.15.133:0/1003121906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927362bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.791+0000 7fe66a59c640 1 -- 172.21.15.133:0/1003121906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579273566a0 con 0x55792735a300 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.791+0000 7fe669d9b640 1 --2- 172.21.15.133:0/1003121906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927362bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.791+0000 7fe66ad9d640 1 --2- 172.21.15.133:0/1003121906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x55792737d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.791+0000 7fe66a59c640 1 --2- 172.21.15.133:0/1003121906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x5579273642b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe65c000c00 tx=0x7fe65c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e87e7dfaef0b9c2c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.791+0000 7fe653fff640 1 -- 172.21.15.133:0/1003121906 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c09b450 con 0x55792735a300 2024-09-11T23:37:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.791+0000 7fe653fff640 1 -- 172.21.15.133:0/1003121906 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe65c09f070 con 0x55792735a300 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.791+0000 7fe653fff640 1 -- 172.21.15.133:0/1003121906 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c09acb0 con 0x55792735a300 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.792+0000 7fe6739ea740 1 -- 172.21.15.133:0/1003121906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 msgr2=0x5579273642b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.792+0000 7fe6739ea740 1 --2- 172.21.15.133:0/1003121906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x5579273642b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe65c000c00 tx=0x7fe65c009310 comp rx=0 tx=0).stop 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.792+0000 7fe6739ea740 1 -- 172.21.15.133:0/1003121906 shutdown_connections 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.792+0000 7fe6739ea740 1 --2- 172.21.15.133:0/1003121906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x55792737d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.792+0000 7fe6739ea740 1 --2- 172.21.15.133:0/1003121906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x557927362bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.792+0000 7fe6739ea740 1 --2- 172.21.15.133:0/1003121906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x5579273642b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.792+0000 7fe6739ea740 1 -- 172.21.15.133:0/1003121906 >> 172.21.15.133:0/1003121906 conn(0x557927356870 msgr2=0x557927363ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.792+0000 7fe6739ea740 1 -- 172.21.15.133:0/1003121906 shutdown_connections 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.792+0000 7fe6739ea740 1 -- 172.21.15.133:0/1003121906 wait complete. 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe6739ea740 1 -- start start 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe6739ea740 1 Processor -- start 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x557927365c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x5579273661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe6739ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x557927366750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe6739ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1ac7b0 con 0x55792735ac20 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe6739ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1ac0e0 con 0x55792735a300 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe6739ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1a93f0 con 0x5579273654c0 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe6739ea740 10 client.0 _ll_register_callbacks cb 0x7fe65c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe66a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x557927365c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe669d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x5579273661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe66a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x557927365c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38582/0 (socket says 172.21.15.133:38582) 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.793+0000 7fe66a59c640 1 -- 172.21.15.133:0/1249223931 learned_addr learned my addr 172.21.15.133:0/1249223931 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.794+0000 7fe66ad9d640 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x557927366750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.794+0000 7fe66a59c640 1 -- 172.21.15.133:0/1249223931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 msgr2=0x557927366750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.794+0000 7fe66a59c640 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x557927366750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.794+0000 7fe66a59c640 1 -- 172.21.15.133:0/1249223931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 msgr2=0x5579273661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.794+0000 7fe66a59c640 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x5579273661d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.794+0000 7fe66a59c640 1 -- 172.21.15.133:0/1249223931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55792736c790 con 0x55792735a300 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.794+0000 7fe66a59c640 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x557927365c90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe65c09a0a0 tx=0x7fe65c0a1330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f368a196e706022 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.794+0000 7fe651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.794+0000 7fe651ffb640 1 -- 172.21.15.133:0/1249223931 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0cd540 con 0x55792735a300 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.794+0000 7fe6739ea740 1 -- 172.21.15.133:0/1249223931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe65c09f070 con 0x55792735a300 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.794+0000 7fe651ffb640 1 -- 172.21.15.133:0/1249223931 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe65c018b20 con 0x55792735a300 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.795+0000 7fe651ffb640 1 -- 172.21.15.133:0/1249223931 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe65c082050 con 0x55792735a300 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.795+0000 7fe6739ea740 10 client.25247 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.795+0000 7fe6739ea740 1 -- 172.21.15.133:0/1249223931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55792736daa0 con 0x55792735a300 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.795+0000 7fe651ffb640 1 -- 172.21.15.133:0/1249223931 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe65c09f070 con 0x55792735a300 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.795+0000 7fe651ffb640 1 -- 172.21.15.133:0/1249223931 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe65c09ae90 con 0x55792735a300 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.795+0000 7fe6739ea740 10 client.25247 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.795+0000 7fe6739ea740 10 client.25247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:37:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.795+0000 7fe6739ea740 20 client.25247 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.795+0000 7fe6739ea740 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5579273e0520 0x5579273e2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.795+0000 7fe6739ea740 4 client.25247 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:37:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.795+0000 7fe6739ea740 1 -- 172.21.15.133:0/1249223931 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579272bdd70 con 0x5579273e0520 2024-09-11T23:37:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.796+0000 7fe669d9b640 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5579273e0520 0x5579273e2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.797+0000 7fe651ffb640 10 client.25247 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.797+0000 7fe669d9b640 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5579273e0520 0x5579273e2910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3774d5d2d4bb43 server_cookie=c18a7019c9658fe in_seq=0 out_seq=0 2024-09-11T23:37:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe651ffb640 1 -- 172.21.15.133:0/1249223931 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579272bdd70 con 0x5579273e0520 2024-09-11T23:37:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe651ffb640 10 client.25247 handle_command_reply: tid=0 2024-09-11T23:37:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 1 client.25247 shutdown 2024-09-11T23:37:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 1 -- 172.21.15.133:0/1249223931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 msgr2=0x557927365c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x557927365c90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe65c09a0a0 tx=0x7fe65c0a1330 comp rx=0 tx=0).stop 2024-09-11T23:37:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 1 -- 172.21.15.133:0/1249223931 shutdown_connections 2024-09-11T23:37:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5579273e0520 0x5579273e2910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579273654c0 0x557927366750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55792735ac20 0x5579273661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 1 --2- 172.21.15.133:0/1249223931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55792735a300 0x557927365c90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 1 -- 172.21.15.133:0/1249223931 >> 172.21.15.133:0/1249223931 conn(0x557927356870 msgr2=0x55792734d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe651ffb640 0 client.25247 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:37:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 1 -- 172.21.15.133:0/1249223931 shutdown_connections 2024-09-11T23:37:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 1 -- 172.21.15.133:0/1249223931 wait complete. 2024-09-11T23:37:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.805+0000 7fe6739ea740 20 client.25247 trim_cache size 0 max 16384 2024-09-11T23:37:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:37:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:37:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:37:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:37:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.806+0000 7fe670b6d640 1 -- 172.21.15.133:0/4086618693 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6540a3020 msgr2=0x7fe6540a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.806+0000 7fe670b6d640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6540a3020 0x7fe6540a54e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe664020000 tx=0x7fe66401d000 comp rx=0 tx=0).stop 2024-09-11T23:37:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.807+0000 7fe670b6d640 1 -- 172.21.15.133:0/4086618693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe66c100860 msgr2=0x7fe66c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.807+0000 7fe670b6d640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe66c100860 0x7fe66c19a590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00e9f0 tx=0x7fe65c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.807+0000 7fe670b6d640 1 -- 172.21.15.133:0/4086618693 shutdown_connections 2024-09-11T23:37:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.807+0000 7fe670b6d640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe6540a3020 0x7fe6540a54e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.807+0000 7fe670b6d640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66c10e410 0x7fe66c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.807+0000 7fe670b6d640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe66c101210 0x7fe66c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.807+0000 7fe670b6d640 1 --2- 172.21.15.133:0/4086618693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe66c100860 0x7fe66c19a590 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.807+0000 7fe670b6d640 1 -- 172.21.15.133:0/4086618693 >> 172.21.15.133:0/4086618693 conn(0x7fe66c077990 msgr2=0x7fe66c0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.807+0000 7fe670b6d640 1 -- 172.21.15.133:0/4086618693 shutdown_connections 2024-09-11T23:37:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:37:51.807+0000 7fe670b6d640 1 -- 172.21.15.133:0/4086618693 wait complete. 2024-09-11T23:37:52.170 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:52 smithi133 ceph-mon[31430]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T23:37:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.289+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/3666678311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c78102570 msgr2=0x7f3c781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.289+0000 7f3c7d3e5640 1 --2- 172.21.15.133:0/3666678311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c78102570 0x7f3c781029b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009900 tx=0x7f3c6c02a480 comp rx=0 tx=0).stop 2024-09-11T23:37:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.289+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/3666678311 shutdown_connections 2024-09-11T23:37:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.289+0000 7f3c7d3e5640 1 --2- 172.21.15.133:0/3666678311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c78102ef0 0x7f3c7810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.289+0000 7f3c7d3e5640 1 --2- 172.21.15.133:0/3666678311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c78102570 0x7f3c781029b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.289+0000 7f3c7d3e5640 1 --2- 172.21.15.133:0/3666678311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c78100e20 0x7f3c78101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.289+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/3666678311 >> 172.21.15.133:0/3666678311 conn(0x7f3c780fc7d0 msgr2=0x7f3c780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.289+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/3666678311 shutdown_connections 2024-09-11T23:37:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.289+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/3666678311 wait complete. 2024-09-11T23:37:52.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.290+0000 7f3c7d3e5640 1 Processor -- start 2024-09-11T23:37:52.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.290+0000 7f3c7d3e5640 1 -- start start 2024-09-11T23:37:52.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.291+0000 7f3c7d3e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c78100e20 0x7f3c78197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:52.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.291+0000 7f3c7d3e5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c78102570 0x7f3c78197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:52.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.291+0000 7f3c76ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c78100e20 0x7f3c78197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:52.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.291+0000 7f3c76ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c78100e20 0x7f3c78197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47026/0 (socket says 172.21.15.133:47026) 2024-09-11T23:37:52.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.291+0000 7f3c76ffd640 1 -- 172.21.15.133:0/1028666340 learned_addr learned my addr 172.21.15.133:0/1028666340 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:52.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.291+0000 7f3c7d3e5640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c78102ef0 0x7f3c7819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.291+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78110fd0 con 0x7f3c78100e20 2024-09-11T23:37:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.292+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78110e50 con 0x7f3c78102ef0 2024-09-11T23:37:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.292+0000 7f3c767fc640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c78102570 0x7f3c78197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.292+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78111150 con 0x7f3c78102570 2024-09-11T23:37:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.292+0000 7f3c777fe640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c78102ef0 0x7f3c7819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.292+0000 7f3c76ffd640 1 -- 172.21.15.133:0/1028666340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c78102570 msgr2=0x7f3c78197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.292+0000 7f3c76ffd640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c78102570 0x7f3c78197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.292+0000 7f3c76ffd640 1 -- 172.21.15.133:0/1028666340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c78102ef0 msgr2=0x7f3c7819bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.292+0000 7f3c76ffd640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c78102ef0 0x7f3c7819bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.292+0000 7f3c76ffd640 1 -- 172.21.15.133:0/1028666340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7819c3c0 con 0x7f3c78100e20 2024-09-11T23:37:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.292+0000 7f3c777fe640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c78102ef0 0x7f3c7819bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.292+0000 7f3c76ffd640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c78100e20 0x7f3c78197410 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800e9f0 tx=0x7f3c6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.293+0000 7f3c5ffff640 1 -- 172.21.15.133:0/1028666340 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6800cd80 con 0x7f3c78100e20 2024-09-11T23:37:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.293+0000 7f3c5ffff640 1 -- 172.21.15.133:0/1028666340 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c68005500 con 0x7f3c78100e20 2024-09-11T23:37:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.293+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7819c6b0 con 0x7f3c78100e20 2024-09-11T23:37:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.293+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c78069f70 con 0x7f3c78100e20 2024-09-11T23:37:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.293+0000 7f3c5ffff640 1 -- 172.21.15.133:0/1028666340 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68010690 con 0x7f3c78100e20 2024-09-11T23:37:52.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.295+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c781021b0 con 0x7f3c78100e20 2024-09-11T23:37:52.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.297+0000 7f3c5ffff640 1 -- 172.21.15.133:0/1028666340 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f3c680046c0 con 0x7f3c78100e20 2024-09-11T23:37:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.298+0000 7f3c5ffff640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3c600a2fa0 0x7f3c600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.298+0000 7f3c5ffff640 1 -- 172.21.15.133:0/1028666340 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f3c680cf9c0 con 0x7f3c78100e20 2024-09-11T23:37:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.298+0000 7f3c767fc640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3c600a2fa0 0x7f3c600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:52.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.299+0000 7f3c767fc640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3c600a2fa0 0x7f3c600a5460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c000c00 tx=0x7f3c6c005b50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:52.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.306+0000 7f3c5ffff640 1 -- 172.21.15.133:0/1028666340 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c6809c410 con 0x7f3c78100e20 2024-09-11T23:37:52.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:52 smithi138 ceph-mon[35142]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T23:37:52.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:52 smithi178 ceph-mon[35882]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T23:37:52.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.593+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c78198770 con 0x7f3c78100e20 2024-09-11T23:37:52.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.594+0000 7f3c5ffff640 1 -- 172.21.15.133:0/1028666340 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f3c6809bd10 con 0x7f3c78100e20 2024-09-11T23:37:52.594 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:37:52.594 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:37:52.594 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:37:52.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.596+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3c600a2fa0 msgr2=0x7f3c600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:52.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.596+0000 7f3c7d3e5640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3c600a2fa0 0x7f3c600a5460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c000c00 tx=0x7f3c6c005b50 comp rx=0 tx=0).stop 2024-09-11T23:37:52.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.596+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c78100e20 msgr2=0x7f3c78197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.596+0000 7f3c7d3e5640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c78100e20 0x7f3c78197410 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800e9f0 tx=0x7f3c6800edf0 comp rx=0 tx=0).stop 2024-09-11T23:37:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.597+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 shutdown_connections 2024-09-11T23:37:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.597+0000 7f3c7d3e5640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3c600a2fa0 0x7f3c600a5460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.597+0000 7f3c7d3e5640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c78102ef0 0x7f3c7819bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.597+0000 7f3c7d3e5640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c78102570 0x7f3c78197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.597+0000 7f3c7d3e5640 1 --2- 172.21.15.133:0/1028666340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c78100e20 0x7f3c78197410 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.597+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 >> 172.21.15.133:0/1028666340 conn(0x7f3c780fc7d0 msgr2=0x7f3c780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.597+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 shutdown_connections 2024-09-11T23:37:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:52.597+0000 7f3c7d3e5640 1 -- 172.21.15.133:0/1028666340 wait complete. 2024-09-11T23:37:53.294 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:53 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1028666340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:53.335 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:37:53.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:53 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1028666340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:53.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:53 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1028666340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:37:53.691 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:54.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.231+0000 7fbab0bc7640 1 -- 172.21.15.133:0/3469277443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbaac100100 msgr2=0x7fbaac100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:54.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.231+0000 7fbab0bc7640 1 --2- 172.21.15.133:0/3469277443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbaac100100 0x7fbaac100560 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009900 tx=0x7fbaa002a410 comp rx=0 tx=0).stop 2024-09-11T23:37:54.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.231+0000 7fbab0bc7640 1 -- 172.21.15.133:0/3469277443 shutdown_connections 2024-09-11T23:37:54.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.231+0000 7fbab0bc7640 1 --2- 172.21.15.133:0/3469277443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaac100aa0 0x7fbaac10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.231+0000 7fbab0bc7640 1 --2- 172.21.15.133:0/3469277443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbaac100100 0x7fbaac100560 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.231+0000 7fbab0bc7640 1 --2- 172.21.15.133:0/3469277443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbaac106b80 0x7fbaac106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.231+0000 7fbab0bc7640 1 -- 172.21.15.133:0/3469277443 >> 172.21.15.133:0/3469277443 conn(0x7fbaac077230 msgr2=0x7fbaac0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.232+0000 7fbab0bc7640 1 -- 172.21.15.133:0/3469277443 shutdown_connections 2024-09-11T23:37:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.232+0000 7fbab0bc7640 1 -- 172.21.15.133:0/3469277443 wait complete. 2024-09-11T23:37:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.233+0000 7fbab0bc7640 1 Processor -- start 2024-09-11T23:37:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.233+0000 7fbab0bc7640 1 -- start start 2024-09-11T23:37:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.233+0000 7fbab0bc7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbaac100100 0x7fbaac198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.233+0000 7fbab0bc7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbaac100aa0 0x7fbaac198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.233+0000 7fbab0bc7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaac106b80 0x7fbaac19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.233+0000 7fbab0bc7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac112060 con 0x7fbaac100100 2024-09-11T23:37:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbab0bc7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac111ee0 con 0x7fbaac106b80 2024-09-11T23:37:54.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbab0bc7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1121e0 con 0x7fbaac100aa0 2024-09-11T23:37:54.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbaaa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbaac100100 0x7fbaac198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbaaa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbaac100100 0x7fbaac198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47046/0 (socket says 172.21.15.133:47046) 2024-09-11T23:37:54.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbaaa59c640 1 -- 172.21.15.133:0/122693151 learned_addr learned my addr 172.21.15.133:0/122693151 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:54.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaac106b80 0x7fbaac19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbaa9d9b640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbaac100aa0 0x7fbaac198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbaaa59c640 1 -- 172.21.15.133:0/122693151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbaac100aa0 msgr2=0x7fbaac198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:54.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbaaa59c640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbaac100aa0 0x7fbaac198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbaaa59c640 1 -- 172.21.15.133:0/122693151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaac106b80 msgr2=0x7fbaac19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:54.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbaaa59c640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaac106b80 0x7fbaac19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbaaa59c640 1 -- 172.21.15.133:0/122693151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac19d4f0 con 0x7fbaac100100 2024-09-11T23:37:54.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.234+0000 7fbaaa59c640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbaac100100 0x7fbaac198540 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00ea60 tx=0x7fba9c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:54.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.235+0000 7fba9b7fe640 1 -- 172.21.15.133:0/122693151 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c00cde0 con 0x7fbaac100100 2024-09-11T23:37:54.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.235+0000 7fba9b7fe640 1 -- 172.21.15.133:0/122693151 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c0035f0 con 0x7fbaac100100 2024-09-11T23:37:54.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.235+0000 7fba9b7fe640 1 -- 172.21.15.133:0/122693151 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c010670 con 0x7fbaac100100 2024-09-11T23:37:54.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.235+0000 7fbab0bc7640 1 -- 172.21.15.133:0/122693151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaac19d7e0 con 0x7fbaac100100 2024-09-11T23:37:54.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.236+0000 7fbab0bc7640 1 -- 172.21.15.133:0/122693151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaac19dc40 con 0x7fbaac100100 2024-09-11T23:37:54.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.238+0000 7fba9b7fe640 1 -- 172.21.15.133:0/122693151 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fba9c0037a0 con 0x7fbaac100100 2024-09-11T23:37:54.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.239+0000 7fbab2fa0740 1 Processor -- start 2024-09-11T23:37:54.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.239+0000 7fbab2fa0740 1 -- start start 2024-09-11T23:37:54.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbab2fa0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e585b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbab2fa0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e585b5d0 0x55e8e585ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbab2fa0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e5866370 0x55e8e5868760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbab2fa0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e8e57c12f0 con 0x55e8e585cb40 2024-09-11T23:37:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbab2fa0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e8e5768cf0 con 0x55e8e5866370 2024-09-11T23:37:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbab2fa0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e8e57bd2b0 con 0x55e8e585b5d0 2024-09-11T23:37:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbaaad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e585b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbaaad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e585b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47060/0 (socket says 172.21.15.133:47060) 2024-09-11T23:37:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbaaad9d640 1 -- 172.21.15.133:0/4015982224 learned_addr learned my addr 172.21.15.133:0/4015982224 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbaaa59c640 1 --2- 172.21.15.133:0/4015982224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e585b5d0 0x55e8e585ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbaa9d9b640 1 --2- 172.21.15.133:0/4015982224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e5866370 0x55e8e5868760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbaaad9d640 1 -- 172.21.15.133:0/4015982224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e585b5d0 msgr2=0x55e8e585ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/4015982224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e585b5d0 0x55e8e585ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbaaad9d640 1 -- 172.21.15.133:0/4015982224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e5866370 msgr2=0x55e8e5868760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/4015982224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e5866370 0x55e8e5868760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.240+0000 7fbaaad9d640 1 -- 172.21.15.133:0/4015982224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e574d3a0 con 0x55e8e585cb40 2024-09-11T23:37:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.241+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/4015982224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e585b090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009b60 tx=0x7fbaa400e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bc763be6fb4493 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.241+0000 7fba9b7fe640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba940a2f20 0x7fba940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.241+0000 7fba98ff9640 1 -- 172.21.15.133:0/4015982224 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4014070 con 0x55e8e585cb40 2024-09-11T23:37:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.241+0000 7fba98ff9640 1 -- 172.21.15.133:0/4015982224 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaa4009e50 con 0x55e8e585cb40 2024-09-11T23:37:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.241+0000 7fba98ff9640 1 -- 172.21.15.133:0/4015982224 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40042b0 con 0x55e8e585cb40 2024-09-11T23:37:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.241+0000 7fbaaa59c640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba940a2f20 0x7fba940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.242+0000 7fba9b7fe640 1 -- 172.21.15.133:0/122693151 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fba9c0cf960 con 0x7fbaac100100 2024-09-11T23:37:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.242+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4015982224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 msgr2=0x55e8e585b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.242+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/4015982224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e585b090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009b60 tx=0x7fbaa400e780 comp rx=0 tx=0).stop 2024-09-11T23:37:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.242+0000 7fbaaa59c640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba940a2f20 0x7fba940a53e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0ccf40 tx=0x7fba9c0d5530 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.242+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4015982224 shutdown_connections 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.242+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/4015982224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e5866370 0x55e8e5868760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.242+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/4015982224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e585b5d0 0x55e8e585ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.242+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/4015982224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e585b090 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.242+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4015982224 >> 172.21.15.133:0/4015982224 conn(0x55e8e5824310 msgr2=0x55e8e5857c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.243+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4015982224 shutdown_connections 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.243+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4015982224 wait complete. 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.243+0000 7fbab2fa0740 1 -- start start 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.243+0000 7fbab2fa0740 1 Processor -- start 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.244+0000 7fbab2fa0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e5865da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.244+0000 7fbab2fa0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e585cb40 0x55e8e5863050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.244+0000 7fbab2fa0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e5866370 0x55e8e58637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.244+0000 7fbab2fa0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e8e5869070 con 0x55e8e585b5d0 2024-09-11T23:37:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.244+0000 7fbab2fa0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e8e577c060 con 0x55e8e585cb40 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.244+0000 7fbab2fa0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e8e58691f0 con 0x55e8e5866370 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.244+0000 7fbab2fa0740 10 client.0 _ll_register_callbacks cb 0x55e8e54cf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.244+0000 7fbaaad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e5865da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.244+0000 7fbaaad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e5865da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47068/0 (socket says 172.21.15.133:47068) 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.244+0000 7fbaaad9d640 1 -- 172.21.15.133:0/4211143009 learned_addr learned my addr 172.21.15.133:0/4211143009 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.244+0000 7fbaa9d9b640 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e585cb40 0x55e8e5863050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.245+0000 7fbaaad9d640 1 -- 172.21.15.133:0/4211143009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e5866370 msgr2=0x55e8e58637e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.245+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e5866370 0x55e8e58637e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.245+0000 7fbaaad9d640 1 -- 172.21.15.133:0/4211143009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e585cb40 msgr2=0x55e8e5863050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.245+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e585cb40 0x55e8e5863050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.245+0000 7fbaaad9d640 1 -- 172.21.15.133:0/4211143009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e55323c0 con 0x55e8e585b5d0 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.245+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e5865da0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4017ba0 tx=0x7fbaa4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba81dee55e9cab57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.245+0000 7fba91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.245+0000 7fba91ffb640 1 -- 172.21.15.133:0/4211143009 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4027060 con 0x55e8e585b5d0 2024-09-11T23:37:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.245+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4211143009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaa4009e50 con 0x55e8e585b5d0 2024-09-11T23:37:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.246+0000 7fba91ffb640 1 -- 172.21.15.133:0/4211143009 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fbaa4004a40 con 0x55e8e585b5d0 2024-09-11T23:37:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.246+0000 7fba91ffb640 1 -- 172.21.15.133:0/4211143009 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbaa4014070 con 0x55e8e585b5d0 2024-09-11T23:37:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.246+0000 7fbab2fa0740 10 client.15843 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.246+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4211143009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e8e5874b50 con 0x55e8e585b5d0 2024-09-11T23:37:54.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.246+0000 7fba91ffb640 1 -- 172.21.15.133:0/4211143009 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaa4009e50 con 0x55e8e585b5d0 2024-09-11T23:37:54.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.246+0000 7fba91ffb640 1 -- 172.21.15.133:0/4211143009 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbaa402a020 con 0x55e8e585b5d0 2024-09-11T23:37:54.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.246+0000 7fbab2fa0740 10 client.15843 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.246+0000 7fbab2fa0740 10 client.15843 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.246+0000 7fbab2fa0740 20 client.15843 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.246+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e8e58762e0 0x55e8e58786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.247+0000 7fbab2fa0740 4 client.15843 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.247+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4211143009 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8e57ce860 con 0x55e8e58762e0 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.247+0000 7fbaa9d9b640 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e8e58762e0 0x55e8e58786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.248+0000 7fba91ffb640 10 client.15843 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.248+0000 7fbaa9d9b640 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e8e58762e0 0x55e8e58786d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ec514395e5170fe6 server_cookie=f8bee560969280ae in_seq=0 out_seq=0 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fba91ffb640 1 -- 172.21.15.133:0/4211143009 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8e57ce860 con 0x55e8e58762e0 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fba91ffb640 10 client.15843 handle_command_reply: tid=0 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 1 client.15843 shutdown 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4211143009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 msgr2=0x55e8e5865da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e5865da0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4017ba0 tx=0x7fbaa4003ba0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4211143009 shutdown_connections 2024-09-11T23:37:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e8e58762e0 0x55e8e58786d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e5866370 0x55e8e58637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e585cb40 0x55e8e5863050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/4211143009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e5865da0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4211143009 >> 172.21.15.133:0/4211143009 conn(0x55e8e5824310 msgr2=0x55e8e585d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fba91ffb640 0 client.15843 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4211143009 shutdown_connections 2024-09-11T23:37:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 1 -- 172.21.15.133:0/4211143009 wait complete. 2024-09-11T23:37:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.249+0000 7fbab2fa0740 20 client.15843 trim_cache size 0 max 16384 2024-09-11T23:37:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbab2fa0740 1 Processor -- start 2024-09-11T23:37:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbab2fa0740 1 -- start start 2024-09-11T23:37:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbab2fa0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e585b5d0 0x55e8e5857c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbab2fa0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e5862910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbab2fa0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e5866370 0x55e8e587dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbab2fa0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1ab610 con 0x55e8e585cb40 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbab2fa0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1ad150 con 0x55e8e5866370 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbab2fa0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1abce0 con 0x55e8e585b5d0 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbaa9d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e5862910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbaa9d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e5862910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47072/0 (socket says 172.21.15.133:47072) 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbaa9d9b640 1 -- 172.21.15.133:0/1698073050 learned_addr learned my addr 172.21.15.133:0/1698073050 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/1698073050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e5866370 0x55e8e587dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.275+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/1698073050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e585b5d0 0x55e8e5857c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.276+0000 7fbaa9d9b640 1 -- 172.21.15.133:0/1698073050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e585b5d0 msgr2=0x55e8e5857c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.276+0000 7fbaa9d9b640 1 --2- 172.21.15.133:0/1698073050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e585b5d0 0x55e8e5857c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.276+0000 7fbaa9d9b640 1 -- 172.21.15.133:0/1698073050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e5866370 msgr2=0x55e8e587dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.276+0000 7fbaa9d9b640 1 --2- 172.21.15.133:0/1698073050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e5866370 0x55e8e587dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.276+0000 7fbaa9d9b640 1 -- 172.21.15.133:0/1698073050 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e5862020 con 0x55e8e585cb40 2024-09-11T23:37:54.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.276+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/1698073050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e5866370 0x55e8e587dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:54.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.276+0000 7fbaa9d9b640 1 --2- 172.21.15.133:0/1698073050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e5862910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00096c0 tx=0x7fbaa0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8020aec1c4150f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:54.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.276+0000 7fba93fff640 1 -- 172.21.15.133:0/1698073050 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0035040 con 0x55e8e585cb40 2024-09-11T23:37:54.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.276+0000 7fba93fff640 1 -- 172.21.15.133:0/1698073050 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaa0005520 con 0x55e8e585cb40 2024-09-11T23:37:54.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.276+0000 7fba93fff640 1 -- 172.21.15.133:0/1698073050 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0004b20 con 0x55e8e585cb40 2024-09-11T23:37:54.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.277+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1698073050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 msgr2=0x55e8e5862910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.277+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/1698073050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e5862910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00096c0 tx=0x7fbaa0005a20 comp rx=0 tx=0).stop 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.277+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1698073050 shutdown_connections 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.277+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/1698073050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e5866370 0x55e8e587dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.277+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/1698073050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585cb40 0x55e8e5862910 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.277+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/1698073050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e585b5d0 0x55e8e5857c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.277+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1698073050 >> 172.21.15.133:0/1698073050 conn(0x55e8e5824310 msgr2=0x55e8e5857580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.277+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1698073050 shutdown_connections 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.277+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1698073050 wait complete. 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.278+0000 7fbab2fa0740 1 -- start start 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.278+0000 7fbab2fa0740 1 Processor -- start 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.278+0000 7fbab2fa0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e586e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.278+0000 7fbab2fa0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e585cb40 0x55e8e586eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbaaad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e586e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbaaad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e586e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47086/0 (socket says 172.21.15.133:47086) 2024-09-11T23:37:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbaaad9d640 1 -- 172.21.15.133:0/1025376757 learned_addr learned my addr 172.21.15.133:0/1025376757 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e5866370 0x55e8e585f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1025376757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1aca80 con 0x55e8e585b5d0 2024-09-11T23:37:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1025376757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1ac3b0 con 0x55e8e585cb40 2024-09-11T23:37:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1025376757 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1a95f0 con 0x55e8e5866370 2024-09-11T23:37:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbaa9d9b640 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e585cb40 0x55e8e586eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbaaad9d640 1 -- 172.21.15.133:0/1025376757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e5866370 msgr2=0x55e8e585f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e5866370 0x55e8e585f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbaaad9d640 1 -- 172.21.15.133:0/1025376757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e585cb40 msgr2=0x55e8e586eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e585cb40 0x55e8e586eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbaaad9d640 1 -- 172.21.15.133:0/1025376757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e585fce0 con 0x55e8e585b5d0 2024-09-11T23:37:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.279+0000 7fbab2fa0740 10 client.0 _ll_register_callbacks cb 0x7fba9c0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.280+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1025376757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaa0005520 con 0x55e8e585b5d0 2024-09-11T23:37:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.280+0000 7fbaaad9d640 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e586e660 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4017ba0 tx=0x7fbaa4005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9986308a0bd072b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.280+0000 7fba91ffb640 10 client.15849 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.280+0000 7fba91ffb640 1 -- 172.21.15.133:0/1025376757 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4027040 con 0x55e8e585b5d0 2024-09-11T23:37:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.280+0000 7fba91ffb640 1 -- 172.21.15.133:0/1025376757 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fbaa4017880 con 0x55e8e585b5d0 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.280+0000 7fba91ffb640 1 -- 172.21.15.133:0/1025376757 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbaa4009c50 con 0x55e8e585b5d0 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.280+0000 7fbab2fa0740 10 client.15849 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.280+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1025376757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55e8e58e0720 con 0x55e8e585b5d0 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.281+0000 7fba91ffb640 1 -- 172.21.15.133:0/1025376757 <== mon.0 v2:172.21.15.133:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbaa4037480 con 0x55e8e585b5d0 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.281+0000 7fbab2fa0740 10 client.15849 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.281+0000 7fbab2fa0740 10 client.15849 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.281+0000 7fbab2fa0740 20 client.15849 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.281+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e8e58e0d20 0x55e8e58e3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.281+0000 7fbab2fa0740 4 client.15849 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.281+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1025376757 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8e57c8d70 con 0x55e8e58e0d20 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.281+0000 7fbaa9d9b640 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e8e58e0d20 0x55e8e58e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbaa9d9b640 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e8e58e0d20 0x55e8e58e3180 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1343123863b6833e server_cookie=d3cf184f33511f3c in_seq=0 out_seq=0 2024-09-11T23:37:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fba91ffb640 10 client.15849 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fba91ffb640 1 -- 172.21.15.133:0/1025376757 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8e57c8d70 con 0x55e8e58e0d20 2024-09-11T23:37:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fba91ffb640 10 client.15849 handle_command_reply: tid=0 2024-09-11T23:37:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 1 client.15849 shutdown 2024-09-11T23:37:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1025376757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 msgr2=0x55e8e586e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e586e660 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4017ba0 tx=0x7fbaa4005000 comp rx=0 tx=0).stop 2024-09-11T23:37:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1025376757 shutdown_connections 2024-09-11T23:37:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55e8e58e0d20 0x55e8e58e3180 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8e5866370 0x55e8e585f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.285 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e585cb40 0x55e8e586eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 1 --2- 172.21.15.133:0/1025376757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8e585b5d0 0x55e8e586e660 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1025376757 >> 172.21.15.133:0/1025376757 conn(0x55e8e5824310 msgr2=0x55e8e5856d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fba91ffb640 0 client.15849 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1025376757 shutdown_connections 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 1 -- 172.21.15.133:0/1025376757 wait complete. 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.282+0000 7fbab2fa0740 20 client.15849 trim_cache size 0 max 16384 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.283+0000 7fbab0bc7640 1 -- 172.21.15.133:0/122693151 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba940a2f20 msgr2=0x7fba940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.283+0000 7fbab0bc7640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba940a2f20 0x7fba940a53e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0ccf40 tx=0x7fba9c0d5530 comp rx=0 tx=0).stop 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.283+0000 7fbab0bc7640 1 -- 172.21.15.133:0/122693151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbaac100100 msgr2=0x7fbaac198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.283+0000 7fbab0bc7640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbaac100100 0x7fbaac198540 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00ea60 tx=0x7fba9c00ef20 comp rx=0 tx=0).stop 2024-09-11T23:37:54.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.284+0000 7fbab0bc7640 1 -- 172.21.15.133:0/122693151 shutdown_connections 2024-09-11T23:37:54.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.284+0000 7fbab0bc7640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba940a2f20 0x7fba940a53e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.284+0000 7fbab0bc7640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaac106b80 0x7fbaac19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.284+0000 7fbab0bc7640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbaac100aa0 0x7fbaac198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.284+0000 7fbab0bc7640 1 --2- 172.21.15.133:0/122693151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbaac100100 0x7fbaac198540 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:54.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.284+0000 7fbab0bc7640 1 -- 172.21.15.133:0/122693151 >> 172.21.15.133:0/122693151 conn(0x7fbaac077230 msgr2=0x7fbaac104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:54.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.284+0000 7fbab0bc7640 1 -- 172.21.15.133:0/122693151 shutdown_connections 2024-09-11T23:37:54.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:54.284+0000 7fbab0bc7640 1 -- 172.21.15.133:0/122693151 wait complete. 2024-09-11T23:37:54.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:54 smithi138 ceph-mon[35142]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-11T23:37:54.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:54 smithi133 ceph-mon[31430]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-11T23:37:54.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:54 smithi178 ceph-mon[35882]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-11T23:37:54.969 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:37:55.236 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.771+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3750689210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c00102570 msgr2=0x7f9c001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.771+0000 7f9c06bdf640 1 --2- 172.21.15.133:0/3750689210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c00102570 0x7f9c001029b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009900 tx=0x7f9bec02a480 comp rx=0 tx=0).stop 2024-09-11T23:37:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.772+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3750689210 shutdown_connections 2024-09-11T23:37:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.772+0000 7f9c06bdf640 1 --2- 172.21.15.133:0/3750689210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c00102ef0 0x7f9c0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.772+0000 7f9c06bdf640 1 --2- 172.21.15.133:0/3750689210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c00102570 0x7f9c001029b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.772+0000 7f9c06bdf640 1 --2- 172.21.15.133:0/3750689210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c00100e20 0x7f9c00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.772+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3750689210 >> 172.21.15.133:0/3750689210 conn(0x7f9c000fc7d0 msgr2=0x7f9c000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.772+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3750689210 shutdown_connections 2024-09-11T23:37:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.772+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3750689210 wait complete. 2024-09-11T23:37:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.773+0000 7f9c06bdf640 1 Processor -- start 2024-09-11T23:37:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.773+0000 7f9c06bdf640 1 -- start start 2024-09-11T23:37:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.773+0000 7f9c06bdf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c00100e20 0x7f9c00197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.773+0000 7f9c06bdf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c00102570 0x7f9c001978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c06bdf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c00102ef0 0x7f9c0019bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c06bdf640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9c00110fd0 con 0x7f9c00102570 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c06bdf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c00110e50 con 0x7f9c00100e20 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c06bdf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c00111150 con 0x7f9c00102ef0 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c0497b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c00100e20 0x7f9c00197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9bfffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c00102570 0x7f9c001978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9bfffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c00102570 0x7f9c001978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47102/0 (socket says 172.21.15.133:47102) 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c0497b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c00100e20 0x7f9c00197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42948/0 (socket says 172.21.15.133:42948) 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9bfffff640 1 -- 172.21.15.133:0/3573039342 learned_addr learned my addr 172.21.15.133:0/3573039342 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c0517c640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c00102ef0 0x7f9c0019bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c0497b640 1 -- 172.21.15.133:0/3573039342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c00102ef0 msgr2=0x7f9c0019bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c0497b640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c00102ef0 0x7f9c0019bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c0497b640 1 -- 172.21.15.133:0/3573039342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c00102570 msgr2=0x7f9c001978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c0497b640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c00102570 0x7f9c001978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.774+0000 7f9c0497b640 1 -- 172.21.15.133:0/3573039342 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0019c340 con 0x7f9c00100e20 2024-09-11T23:37:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.775+0000 7f9c0497b640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c00100e20 0x7f9c00197390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009ef0 tx=0x7f9bf400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:55.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.775+0000 7f9bfdffb640 1 -- 172.21.15.133:0/3573039342 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40159e0 con 0x7f9c00100e20 2024-09-11T23:37:55.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.775+0000 7f9bfdffb640 1 -- 172.21.15.133:0/3573039342 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf4015b80 con 0x7f9c00100e20 2024-09-11T23:37:55.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.775+0000 7f9bfdffb640 1 -- 172.21.15.133:0/3573039342 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4005020 con 0x7f9c00100e20 2024-09-11T23:37:55.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.775+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3573039342 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0019c630 con 0x7f9c00100e20 2024-09-11T23:37:55.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.776+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3573039342 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c00069f70 con 0x7f9c00100e20 2024-09-11T23:37:55.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.779+0000 7f9bfdffb640 1 -- 172.21.15.133:0/3573039342 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f9bf40051c0 con 0x7f9c00100e20 2024-09-11T23:37:55.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.779+0000 7f9c08fb8740 1 Processor -- start 2024-09-11T23:37:55.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.780+0000 7f9c08fb8740 1 -- start start 2024-09-11T23:37:55.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.780+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134aa350 0x560d134a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.780+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 0x560d134a9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.780+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134b3b80 0x560d134b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.780+0000 7f9c08fb8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d134062f0 con 0x560d134a8de0 2024-09-11T23:37:55.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.780+0000 7f9c08fb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560d133adcf0 con 0x560d134b3b80 2024-09-11T23:37:55.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.780+0000 7f9c08fb8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560d134022b0 con 0x560d134aa350 2024-09-11T23:37:55.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.780+0000 7f9bfffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134b3b80 0x560d134b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.780+0000 7f9c0517c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134aa350 0x560d134a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.780+0000 7f9bfffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134b3b80 0x560d134b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42950/0 (socket says 172.21.15.133:42950) 2024-09-11T23:37:55.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.780+0000 7f9bfffff640 1 -- 172.21.15.133:0/1191558973 learned_addr learned my addr 172.21.15.133:0/1191558973 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:55.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.781+0000 7f9c0497b640 1 --2- 172.21.15.133:0/1191558973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 0x560d134a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.781+0000 7f9bfffff640 1 -- 172.21.15.133:0/1191558973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134aa350 msgr2=0x560d134a88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:55.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.781+0000 7f9bfffff640 1 --2- 172.21.15.133:0/1191558973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134aa350 0x560d134a88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.781+0000 7f9bfffff640 1 -- 172.21.15.133:0/1191558973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 msgr2=0x560d134a9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.781+0000 7f9bfffff640 1 --2- 172.21.15.133:0/1191558973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 0x560d134a9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.781+0000 7f9bfffff640 1 -- 172.21.15.133:0/1191558973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d133923a0 con 0x560d134b3b80 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.781+0000 7f9c0497b640 1 --2- 172.21.15.133:0/1191558973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 0x560d134a9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.781+0000 7f9bfdffb640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9bf00a2f20 0x7f9bf00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.781+0000 7f9bfffff640 1 --2- 172.21.15.133:0/1191558973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134b3b80 0x560d134b5f70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0040c0 tx=0x7f9bec0040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d8971aeff1ebac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.781+0000 7f9c0517c640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9bf00a2f20 0x7f9bf00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.781+0000 7f9beaffd640 1 -- 172.21.15.133:0/1191558973 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec040070 con 0x560d134b3b80 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.782+0000 7f9beaffd640 1 -- 172.21.15.133:0/1191558973 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bec005500 con 0x560d134b3b80 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.782+0000 7f9beaffd640 1 -- 172.21.15.133:0/1191558973 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec035040 con 0x560d134b3b80 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.782+0000 7f9bfdffb640 1 -- 172.21.15.133:0/3573039342 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f9bf40cf780 con 0x7f9c00100e20 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1191558973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134b3b80 msgr2=0x560d134b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1191558973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134b3b80 0x560d134b5f70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0040c0 tx=0x7f9bec0040f0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c0517c640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9bf00a2f20 0x7f9bf00a53e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8007f20 tx=0x7f9bf8007680 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:55.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1191558973 shutdown_connections 2024-09-11T23:37:55.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1191558973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134b3b80 0x560d134b5f70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1191558973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 0x560d134a9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1191558973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134aa350 0x560d134a88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1191558973 >> 172.21.15.133:0/1191558973 conn(0x560d13469310 msgr2=0x560d134a5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:55.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1191558973 shutdown_connections 2024-09-11T23:37:55.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1191558973 wait complete. 2024-09-11T23:37:55.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c08fb8740 1 -- start start 2024-09-11T23:37:55.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.783+0000 7f9c08fb8740 1 Processor -- start 2024-09-11T23:37:55.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134a8de0 0x560d134b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134aa350 0x560d134b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134b3b80 0x560d134b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c08fb8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d134b6820 con 0x560d134b3b80 2024-09-11T23:37:55.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c0517c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134b3b80 0x560d134b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c0517c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134b3b80 0x560d134b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47116/0 (socket says 172.21.15.133:47116) 2024-09-11T23:37:55.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c0517c640 1 -- 172.21.15.133:0/1900906818 learned_addr learned my addr 172.21.15.133:0/1900906818 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:55.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c08fb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560d133c1060 con 0x560d134aa350 2024-09-11T23:37:55.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1900906818 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560d134b69a0 con 0x560d134a8de0 2024-09-11T23:37:55.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c0497b640 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134a8de0 0x560d134b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9bfffff640 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134aa350 0x560d134b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c08fb8740 10 client.0 _ll_register_callbacks cb 0x560d131148c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:55.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c0517c640 1 -- 172.21.15.133:0/1900906818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134a8de0 msgr2=0x560d134b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:55.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c0517c640 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134a8de0 0x560d134b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c0517c640 1 -- 172.21.15.133:0/1900906818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134aa350 msgr2=0x560d134b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:55.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c0517c640 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134aa350 0x560d134b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.784+0000 7f9c0517c640 1 -- 172.21.15.133:0/1900906818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d131773c0 con 0x560d134b3b80 2024-09-11T23:37:55.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.785+0000 7f9c0517c640 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134b3b80 0x560d134b0d80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800fe00 tx=0x7f9bf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c59673251db4ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.785+0000 7f9bdffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.785+0000 7f9bdffff640 1 -- 172.21.15.133:0/1900906818 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8029060 con 0x560d134b3b80 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.785+0000 7f9bdffff640 1 -- 172.21.15.133:0/1900906818 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f9bf801a460 con 0x560d134b3b80 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.785+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1900906818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bec005500 con 0x560d134b3b80 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.785+0000 7f9bdffff640 1 -- 172.21.15.133:0/1900906818 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bf8007b20 con 0x560d134b3b80 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.785+0000 7f9c08fb8740 10 client.15861 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.785+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1900906818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d1349ec10 con 0x560d134b3b80 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.786+0000 7f9bdffff640 1 -- 172.21.15.133:0/1900906818 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf801ab40 con 0x560d134b3b80 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.786+0000 7f9bdffff640 1 -- 172.21.15.133:0/1900906818 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9bf802c020 con 0x560d134b3b80 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.786+0000 7f9c08fb8740 10 client.15861 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.786+0000 7f9c08fb8740 10 client.15861 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:37:55.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.786+0000 7f9c08fb8740 20 client.15861 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:55.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.786+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x560d134a0020 0x560d134a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.786+0000 7f9c08fb8740 4 client.15861 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:55.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.786+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1900906818 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9bec005500 con 0x560d134a0020 2024-09-11T23:37:55.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.786+0000 7f9c0497b640 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x560d134a0020 0x560d134a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.786+0000 7f9bdffff640 10 client.15861 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:55.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.787+0000 7f9c0497b640 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x560d134a0020 0x560d134a24c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=29e64954f419668 server_cookie=5daf1cb1b04ec826 in_seq=0 out_seq=0 2024-09-11T23:37:55.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.787+0000 7f9bdffff640 1 -- 172.21.15.133:0/1900906818 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9bec005500 con 0x560d134a0020 2024-09-11T23:37:55.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.787+0000 7f9bdffff640 10 client.15861 handle_command_reply: tid=0 2024-09-11T23:37:55.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.787+0000 7f9c08fb8740 1 client.15861 shutdown 2024-09-11T23:37:55.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1900906818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134b3b80 msgr2=0x560d134b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:55.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134b3b80 0x560d134b0d80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800fe00 tx=0x7f9bf8004a40 comp rx=0 tx=0).stop 2024-09-11T23:37:55.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1900906818 shutdown_connections 2024-09-11T23:37:55.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x560d134a0020 0x560d134a24c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9bdffff640 0 client.15861 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:55.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134b3b80 0x560d134b0d80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134aa350 0x560d134b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1900906818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134a8de0 0x560d134b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1900906818 >> 172.21.15.133:0/1900906818 conn(0x560d13469310 msgr2=0x560d134ab170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:55.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1900906818 shutdown_connections 2024-09-11T23:37:55.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1900906818 wait complete. 2024-09-11T23:37:55.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.788+0000 7f9c08fb8740 20 client.15861 trim_cache size 0 max 16384 2024-09-11T23:37:55.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9c08fb8740 1 Processor -- start 2024-09-11T23:37:55.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9c08fb8740 1 -- start start 2024-09-11T23:37:55.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134a0020 0x560d134a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 0x560d1349c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134aa350 0x560d134bbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9c08fb8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001ab3d0 con 0x560d134a8de0 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9c08fb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001acf10 con 0x560d134a0020 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9c08fb8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001abaa0 con 0x560d134aa350 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9bfffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 0x560d1349c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9bfffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 0x560d1349c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47126/0 (socket says 172.21.15.133:47126) 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9bfffff640 1 -- 172.21.15.133:0/737758467 learned_addr learned my addr 172.21.15.133:0/737758467 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.813+0000 7f9bfffff640 1 -- 172.21.15.133:0/737758467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134aa350 msgr2=0x560d134bbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.814+0000 7f9c0517c640 1 --2- 172.21.15.133:0/737758467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134aa350 0x560d134bbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.814+0000 7f9bfffff640 1 --2- 172.21.15.133:0/737758467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134aa350 0x560d134bbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.814+0000 7f9bfffff640 1 -- 172.21.15.133:0/737758467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134a0020 msgr2=0x560d134a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:55.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.814+0000 7f9c0497b640 1 --2- 172.21.15.133:0/737758467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134a0020 0x560d134a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.814+0000 7f9bfffff640 1 --2- 172.21.15.133:0/737758467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134a0020 0x560d134a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.814+0000 7f9bfffff640 1 -- 172.21.15.133:0/737758467 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d134ab930 con 0x560d134a8de0 2024-09-11T23:37:55.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.814+0000 7f9c0497b640 1 --2- 172.21.15.133:0/737758467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134a0020 0x560d134a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:55.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.814+0000 7f9bfffff640 1 --2- 172.21.15.133:0/737758467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 0x560d1349c4f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9bec004e70 tx=0x7f9bec02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0216d66a4b601 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:55.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.814+0000 7f9bea7fc640 1 -- 172.21.15.133:0/737758467 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec040050 con 0x560d134a8de0 2024-09-11T23:37:55.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.814+0000 7f9bea7fc640 1 -- 172.21.15.133:0/737758467 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bec004490 con 0x560d134a8de0 2024-09-11T23:37:55.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.814+0000 7f9bea7fc640 1 -- 172.21.15.133:0/737758467 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec035040 con 0x560d134a8de0 2024-09-11T23:37:55.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.815+0000 7f9c08fb8740 1 -- 172.21.15.133:0/737758467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 msgr2=0x560d1349c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:55.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.815+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/737758467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 0x560d1349c4f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9bec004e70 tx=0x7f9bec02a480 comp rx=0 tx=0).stop 2024-09-11T23:37:55.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.815+0000 7f9c08fb8740 1 -- 172.21.15.133:0/737758467 shutdown_connections 2024-09-11T23:37:55.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.815+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/737758467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134aa350 0x560d134bbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.815+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/737758467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134a8de0 0x560d1349c4f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.815+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/737758467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134a0020 0x560d134a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.815+0000 7f9c08fb8740 1 -- 172.21.15.133:0/737758467 >> 172.21.15.133:0/737758467 conn(0x560d13469310 msgr2=0x560d134a4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.816+0000 7f9c08fb8740 1 -- 172.21.15.133:0/737758467 shutdown_connections 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.816+0000 7f9c08fb8740 1 -- 172.21.15.133:0/737758467 wait complete. 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.816+0000 7f9c08fb8740 1 -- start start 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.816+0000 7f9c08fb8740 1 Processor -- start 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134a0020 0x560d134ba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134a8de0 0x560d134ba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c08fb8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134aa350 0x560d1352fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c0517c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134aa350 0x560d1352fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c0517c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134aa350 0x560d1352fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47140/0 (socket says 172.21.15.133:47140) 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c0497b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134a0020 0x560d134ba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c0517c640 1 -- 172.21.15.133:0/1869194536 learned_addr learned my addr 172.21.15.133:0/1869194536 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1869194536 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001ac840 con 0x560d134aa350 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9bfffff640 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134a8de0 0x560d134ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1869194536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001ac170 con 0x560d134a8de0 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1869194536 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001a93d0 con 0x560d134a0020 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c0497b640 1 -- 172.21.15.133:0/1869194536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134a8de0 msgr2=0x560d134ba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c0497b640 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134a8de0 0x560d134ba810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c08fb8740 10 client.0 _ll_register_callbacks cb 0x7f9bf8011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c0497b640 1 -- 172.21.15.133:0/1869194536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134aa350 msgr2=0x560d1352fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c0497b640 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134aa350 0x560d1352fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.817+0000 7f9c0497b640 1 -- 172.21.15.133:0/1869194536 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d13530360 con 0x560d134a0020 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.818+0000 7f9c0517c640 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134aa350 0x560d1352fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.818+0000 7f9c0497b640 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134a0020 0x560d134ba2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40cccd0 tx=0x7f9bf4009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2379adb2050ef324 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.818+0000 7f9bdffff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:37:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.818+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1869194536 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bec004490 con 0x560d134a0020 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.818+0000 7f9bdffff640 1 -- 172.21.15.133:0/1869194536 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4023040 con 0x560d134a0020 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.818+0000 7f9bdffff640 1 -- 172.21.15.133:0/1869194536 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f9bf409c550 con 0x560d134a0020 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.818+0000 7f9bdffff640 1 -- 172.21.15.133:0/1869194536 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bf40a8b50 con 0x560d134a0020 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.818+0000 7f9c08fb8740 10 client.25327 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.818+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1869194536 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=28}) -- 0x560d135375c0 con 0x560d134a0020 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.819+0000 7f9bdffff640 1 -- 172.21.15.133:0/1869194536 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9bf40d2ce0 con 0x560d134a0020 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.819+0000 7f9c08fb8740 10 client.25327 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.819+0000 7f9c08fb8740 10 client.25327 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.819+0000 7f9c08fb8740 20 client.25327 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.819+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x560d134b3b80 0x560d13539be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.819+0000 7f9c08fb8740 4 client.25327 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.819+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1869194536 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d13413860 con 0x560d134b3b80 2024-09-11T23:37:55.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.819+0000 7f9bfffff640 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x560d134b3b80 0x560d13539be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.820+0000 7f9bfffff640 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x560d134b3b80 0x560d13539be0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3577314cf2af6166 server_cookie=dc2e08f1b3e5de4b in_seq=0 out_seq=0 2024-09-11T23:37:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.820+0000 7f9bdffff640 10 client.25327 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.820+0000 7f9bdffff640 1 -- 172.21.15.133:0/1869194536 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d13413860 con 0x560d134b3b80 2024-09-11T23:37:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.820+0000 7f9bdffff640 10 client.25327 handle_command_reply: tid=0 2024-09-11T23:37:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.820+0000 7f9c08fb8740 1 client.25327 shutdown 2024-09-11T23:37:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.820+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1869194536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134a0020 msgr2=0x560d134ba2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.820+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134a0020 0x560d134ba2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40cccd0 tx=0x7f9bf4009620 comp rx=0 tx=0).stop 2024-09-11T23:37:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.820+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1869194536 shutdown_connections 2024-09-11T23:37:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.820+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x560d134b3b80 0x560d13539be0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.820+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d134aa350 0x560d1352fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.821+0000 7f9bdffff640 0 client.25327 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:37:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.821+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d134a8de0 0x560d134ba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.821+0000 7f9c08fb8740 1 --2- 172.21.15.133:0/1869194536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d134a0020 0x560d134ba2d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.821+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1869194536 >> 172.21.15.133:0/1869194536 conn(0x560d13469310 msgr2=0x560d134c4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.821+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1869194536 shutdown_connections 2024-09-11T23:37:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.821+0000 7f9c08fb8740 1 -- 172.21.15.133:0/1869194536 wait complete. 2024-09-11T23:37:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.821+0000 7f9c08fb8740 20 client.25327 trim_cache size 0 max 16384 2024-09-11T23:37:55.826 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:37:55.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.822+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3573039342 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9bf00a2f20 msgr2=0x7f9bf00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:55.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.822+0000 7f9c06bdf640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9bf00a2f20 0x7f9bf00a53e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8007f20 tx=0x7f9bf8007680 comp rx=0 tx=0).stop 2024-09-11T23:37:55.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.822+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3573039342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c00100e20 msgr2=0x7f9c00197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:55.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.822+0000 7f9c06bdf640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c00100e20 0x7f9c00197390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009ef0 tx=0x7f9bf400ed10 comp rx=0 tx=0).stop 2024-09-11T23:37:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.822+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3573039342 shutdown_connections 2024-09-11T23:37:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.822+0000 7f9c06bdf640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9bf00a2f20 0x7f9bf00a53e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.822+0000 7f9c06bdf640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c00102ef0 0x7f9c0019bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.822+0000 7f9c06bdf640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c00102570 0x7f9c001978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.822+0000 7f9c06bdf640 1 --2- 172.21.15.133:0/3573039342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c00100e20 0x7f9c00197390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.822+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3573039342 >> 172.21.15.133:0/3573039342 conn(0x7f9c000fc7d0 msgr2=0x7f9c00106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.823+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3573039342 shutdown_connections 2024-09-11T23:37:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:55.823+0000 7f9c06bdf640 1 -- 172.21.15.133:0/3573039342 wait complete. 2024-09-11T23:37:56.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:56 smithi138 ceph-mon[35142]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-11T23:37:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:56 smithi133 ceph-mon[31430]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-11T23:37:56.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:56 smithi178 ceph-mon[35882]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-11T23:37:56.726 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:37:57.026 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:57.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.529+0000 7f831568a640 1 -- 172.21.15.133:0/1914531182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8310102780 msgr2=0x7f83101067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.529+0000 7f831568a640 1 --2- 172.21.15.133:0/1914531182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8310102780 0x7f83101067d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8304009900 tx=0x7f830402a410 comp rx=0 tx=0).stop 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.529+0000 7f831568a640 1 -- 172.21.15.133:0/1914531182 shutdown_connections 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.529+0000 7f831568a640 1 --2- 172.21.15.133:0/1914531182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310106f00 0x7f831010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.529+0000 7f831568a640 1 --2- 172.21.15.133:0/1914531182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8310102780 0x7f83101067d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.529+0000 7f831568a640 1 --2- 172.21.15.133:0/1914531182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8310101dd0 0x7f83101021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.529+0000 7f831568a640 1 -- 172.21.15.133:0/1914531182 >> 172.21.15.133:0/1914531182 conn(0x7f83100fd8b0 msgr2=0x7f83100ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.529+0000 7f831568a640 1 -- 172.21.15.133:0/1914531182 shutdown_connections 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.529+0000 7f831568a640 1 -- 172.21.15.133:0/1914531182 wait complete. 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.530+0000 7f831568a640 1 Processor -- start 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.530+0000 7f831568a640 1 -- start start 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.530+0000 7f831568a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8310101dd0 0x7f83101984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.530+0000 7f831568a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8310102780 0x7f8310198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f831568a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310106f00 0x7f831019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f831568a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8310111f60 con 0x7f8310101dd0 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f831568a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8310111de0 con 0x7f8310102780 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f831568a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83101120e0 con 0x7f8310106f00 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8310101dd0 0x7f83101984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8310101dd0 0x7f83101984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47160/0 (socket says 172.21.15.133:47160) 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830effd640 1 -- 172.21.15.133:0/1239079890 learned_addr learned my addr 172.21.15.133:0/1239079890 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830e7fc640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8310102780 0x7f8310198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830f7fe640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310106f00 0x7f831019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830effd640 1 -- 172.21.15.133:0/1239079890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310106f00 msgr2=0x7f831019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830effd640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310106f00 0x7f831019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830effd640 1 -- 172.21.15.133:0/1239079890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8310102780 msgr2=0x7f8310198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830effd640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8310102780 0x7f8310198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830effd640 1 -- 172.21.15.133:0/1239079890 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831019d470 con 0x7f8310101dd0 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830f7fe640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310106f00 0x7f831019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.531+0000 7f830effd640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8310101dd0 0x7f83101984c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f830000d950 tx=0x7f830000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.532+0000 7f82f7fff640 1 -- 172.21.15.133:0/1239079890 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300017070 con 0x7f8310101dd0 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.532+0000 7f831568a640 1 -- 172.21.15.133:0/1239079890 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831019d760 con 0x7f8310101dd0 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.532+0000 7f831568a640 1 -- 172.21.15.133:0/1239079890 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83101a5040 con 0x7f8310101dd0 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.532+0000 7f82f7fff640 1 -- 172.21.15.133:0/1239079890 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8300009c30 con 0x7f8310101dd0 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.532+0000 7f82f7fff640 1 -- 172.21.15.133:0/1239079890 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300020060 con 0x7f8310101dd0 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.535+0000 7f8317a63740 1 Processor -- start 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.535+0000 7f8317a63740 1 -- start start 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.535+0000 7f82f7fff640 1 -- 172.21.15.133:0/1239079890 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f830000f6b0 con 0x7f8310101dd0 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f8317a63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1bb40 0x56330ff1a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f8317a63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1a5d0 0x56330ff1aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f8317a63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff25370 0x56330ff27760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f8317a63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56330fe802f0 con 0x56330ff1a5d0 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f8317a63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56330fe27cf0 con 0x56330ff25370 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f8317a63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56330fe7c2b0 con 0x56330ff1bb40 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f830effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1a5d0 0x56330ff1aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f830effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1a5d0 0x56330ff1aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47174/0 (socket says 172.21.15.133:47174) 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f830effd640 1 -- 172.21.15.133:0/3402131124 learned_addr learned my addr 172.21.15.133:0/3402131124 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f830f7fe640 1 --2- 172.21.15.133:0/3402131124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1bb40 0x56330ff1a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f830e7fc640 1 --2- 172.21.15.133:0/3402131124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff25370 0x56330ff27760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f830effd640 1 -- 172.21.15.133:0/3402131124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1bb40 msgr2=0x56330ff1a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f830effd640 1 --2- 172.21.15.133:0/3402131124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1bb40 0x56330ff1a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f830effd640 1 -- 172.21.15.133:0/3402131124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff25370 msgr2=0x56330ff27760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f830effd640 1 --2- 172.21.15.133:0/3402131124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff25370 0x56330ff27760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.536+0000 7f830effd640 1 -- 172.21.15.133:0/3402131124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330fe0c3a0 con 0x56330ff1a5d0 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.537+0000 7f830e7fc640 1 --2- 172.21.15.133:0/3402131124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff25370 0x56330ff27760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.537+0000 7f830effd640 1 --2- 172.21.15.133:0/3402131124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1a5d0 0x56330ff1aa10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f83000cc730 tx=0x7f83000d7af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba774df7f3653e25 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.537+0000 7f82f7fff640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f80a2f20 0x7f82f80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.537+0000 7f82f57fa640 1 -- 172.21.15.133:0/3402131124 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83000e2070 con 0x56330ff1a5d0 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.537+0000 7f82f57fa640 1 -- 172.21.15.133:0/3402131124 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830009bc90 con 0x56330ff1a5d0 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.537+0000 7f82f57fa640 1 -- 172.21.15.133:0/3402131124 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83000aebc0 con 0x56330ff1a5d0 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.537+0000 7f830f7fe640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f80a2f20 0x7f82f80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.537+0000 7f82f7fff640 1 -- 172.21.15.133:0/1239079890 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f83000cf240 con 0x7f8310101dd0 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.538+0000 7f8317a63740 1 -- 172.21.15.133:0/3402131124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1a5d0 msgr2=0x56330ff1aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.538+0000 7f8317a63740 1 --2- 172.21.15.133:0/3402131124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1a5d0 0x56330ff1aa10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f83000cc730 tx=0x7f83000d7af0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.538+0000 7f830f7fe640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f80a2f20 0x7f82f80a53e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8308005fd0 tx=0x7f8308009500 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.538+0000 7f8317a63740 1 -- 172.21.15.133:0/3402131124 shutdown_connections 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.538+0000 7f8317a63740 1 --2- 172.21.15.133:0/3402131124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff25370 0x56330ff27760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.538+0000 7f8317a63740 1 --2- 172.21.15.133:0/3402131124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1a5d0 0x56330ff1aa10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.538+0000 7f8317a63740 1 --2- 172.21.15.133:0/3402131124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1bb40 0x56330ff1a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.538+0000 7f8317a63740 1 -- 172.21.15.133:0/3402131124 >> 172.21.15.133:0/3402131124 conn(0x56330fee3310 msgr2=0x56330ff16c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.538+0000 7f8317a63740 1 -- 172.21.15.133:0/3402131124 shutdown_connections 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.538+0000 7f8317a63740 1 -- 172.21.15.133:0/3402131124 wait complete. 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f8317a63740 1 -- start start 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f8317a63740 1 Processor -- start 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f8317a63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1bb40 0x56330ff1e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f8317a63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff25370 0x56330ff1ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f8317a63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 0x56330ff22da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f8317a63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56330ff27ee0 con 0x56330ff1bb40 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f8317a63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56330fe3b060 con 0x56330ff25370 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f8317a63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56330ff28060 con 0x56330ff1efe0 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f8317a63740 10 client.0 _ll_register_callbacks cb 0x56330fb8e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f830effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1bb40 0x56330ff1e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 0x56330ff22da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.539+0000 7f830effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1bb40 0x56330ff1e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47186/0 (socket says 172.21.15.133:47186) 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.540+0000 7f830effd640 1 -- 172.21.15.133:0/3315107703 learned_addr learned my addr 172.21.15.133:0/3315107703 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.540+0000 7f830e7fc640 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff25370 0x56330ff1ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.540+0000 7f830effd640 1 -- 172.21.15.133:0/3315107703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 msgr2=0x56330ff22da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.540+0000 7f830effd640 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 0x56330ff22da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.540+0000 7f830effd640 1 -- 172.21.15.133:0/3315107703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff25370 msgr2=0x56330ff1ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.540+0000 7f830effd640 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff25370 0x56330ff1ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.540+0000 7f830effd640 1 -- 172.21.15.133:0/3315107703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330fbf13c0 con 0x56330ff1bb40 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.540+0000 7f830f7fe640 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 0x56330ff22da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.540+0000 7f830effd640 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1bb40 0x56330ff1e510 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8300010ef0 tx=0x7f8300000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad43e679d4c0ee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.540+0000 7f82f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.540+0000 7f82f2ff5640 1 -- 172.21.15.133:0/3315107703 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300097830 con 0x56330ff1bb40 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.541+0000 7f82f2ff5640 1 -- 172.21.15.133:0/3315107703 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8300097c00 con 0x56330ff1bb40 2024-09-11T23:37:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.541+0000 7f8317a63740 1 -- 172.21.15.133:0/3315107703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f830009bc90 con 0x56330ff1bb40 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.541+0000 7f82f2ff5640 1 -- 172.21.15.133:0/3315107703 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f830009b0d0 con 0x56330ff1bb40 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.541+0000 7f8317a63740 10 client.15879 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.541+0000 7f8317a63740 1 -- 172.21.15.133:0/3315107703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56330ff27ee0 con 0x56330ff1bb40 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.541+0000 7f82f2ff5640 1 -- 172.21.15.133:0/3315107703 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830009bc90 con 0x56330ff1bb40 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.541+0000 7f82f2ff5640 1 -- 172.21.15.133:0/3315107703 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83000d2820 con 0x56330ff1bb40 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.541+0000 7f8317a63740 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.541+0000 7f8317a63740 10 client.15879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.541+0000 7f8317a63740 20 client.15879 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.541+0000 7f8317a63740 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56330ff3daf0 0x56330ff3fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.542+0000 7f8317a63740 4 client.15879 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.542+0000 7f8317a63740 1 -- 172.21.15.133:0/3315107703 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56330fe8d860 con 0x56330ff3daf0 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.542+0000 7f830e7fc640 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56330ff3daf0 0x56330ff3fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.542+0000 7f830e7fc640 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56330ff3daf0 0x56330ff3fee0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff020962c1683754 server_cookie=f8c35d73614eef31 in_seq=0 out_seq=0 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.542+0000 7f82f2ff5640 10 client.15879 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.543+0000 7f82f2ff5640 1 -- 172.21.15.133:0/3315107703 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56330fe8d860 con 0x56330ff3daf0 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.543+0000 7f82f2ff5640 10 client.15879 handle_command_reply: tid=0 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.543+0000 7f8317a63740 1 client.15879 shutdown 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.543+0000 7f8317a63740 1 -- 172.21.15.133:0/3315107703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1bb40 msgr2=0x56330ff1e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.543+0000 7f8317a63740 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1bb40 0x56330ff1e510 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8300010ef0 tx=0x7f8300000bc0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.543+0000 7f8317a63740 1 -- 172.21.15.133:0/3315107703 shutdown_connections 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.543+0000 7f8317a63740 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56330ff3daf0 0x56330ff3fee0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.543+0000 7f8317a63740 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 0x56330ff22da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.543+0000 7f8317a63740 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff25370 0x56330ff1ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.543+0000 7f8317a63740 1 --2- 172.21.15.133:0/3315107703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1bb40 0x56330ff1e510 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.543+0000 7f8317a63740 1 -- 172.21.15.133:0/3315107703 >> 172.21.15.133:0/3315107703 conn(0x56330fee3310 msgr2=0x56330ff1c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.544+0000 7f82f2ff5640 0 client.15879 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.544+0000 7f8317a63740 1 -- 172.21.15.133:0/3315107703 shutdown_connections 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.544+0000 7f8317a63740 1 -- 172.21.15.133:0/3315107703 wait complete. 2024-09-11T23:37:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.544+0000 7f8317a63740 20 client.15879 trim_cache size 0 max 16384 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f8317a63740 1 Processor -- start 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f8317a63740 1 -- start start 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f8317a63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff1bb40 0x56330ff26760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f8317a63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 0x56330ff26ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f8317a63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff25370 0x56330ff456c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f8317a63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc00e580 con 0x56330ff25370 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f8317a63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc0100c0 con 0x56330ff1bb40 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f8317a63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc00ec50 con 0x56330ff1efe0 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff25370 0x56330ff456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f830effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff1bb40 0x56330ff26760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f830e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 0x56330ff26ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff25370 0x56330ff456c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47192/0 (socket says 172.21.15.133:47192) 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.557+0000 7f830f7fe640 1 -- 172.21.15.133:0/1080011589 learned_addr learned my addr 172.21.15.133:0/1080011589 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.558+0000 7f830e7fc640 1 -- 172.21.15.133:0/1080011589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff1bb40 msgr2=0x56330ff26760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.558+0000 7f830e7fc640 1 --2- 172.21.15.133:0/1080011589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff1bb40 0x56330ff26760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.558+0000 7f830e7fc640 1 -- 172.21.15.133:0/1080011589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff25370 msgr2=0x56330ff456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.558+0000 7f830e7fc640 1 --2- 172.21.15.133:0/1080011589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff25370 0x56330ff456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.558+0000 7f830e7fc640 1 -- 172.21.15.133:0/1080011589 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330ff16410 con 0x56330ff1efe0 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.558+0000 7f830e7fc640 1 --2- 172.21.15.133:0/1080011589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 0x56330ff26ca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8304009230 tx=0x7f83040058b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e370a9226561e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.558+0000 7f82f4ff9640 1 -- 172.21.15.133:0/1080011589 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304049070 con 0x56330ff1efe0 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.558+0000 7f82f4ff9640 1 -- 172.21.15.133:0/1080011589 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8304005520 con 0x56330ff1efe0 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.558+0000 7f82f4ff9640 1 -- 172.21.15.133:0/1080011589 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304035040 con 0x56330ff1efe0 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.559+0000 7f8317a63740 1 -- 172.21.15.133:0/1080011589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 msgr2=0x56330ff26ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.559+0000 7f8317a63740 1 --2- 172.21.15.133:0/1080011589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 0x56330ff26ca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8304009230 tx=0x7f83040058b0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.559+0000 7f8317a63740 1 -- 172.21.15.133:0/1080011589 shutdown_connections 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.559+0000 7f8317a63740 1 --2- 172.21.15.133:0/1080011589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff25370 0x56330ff456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.559+0000 7f8317a63740 1 --2- 172.21.15.133:0/1080011589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff1efe0 0x56330ff26ca0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.559+0000 7f8317a63740 1 --2- 172.21.15.133:0/1080011589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff1bb40 0x56330ff26760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.559+0000 7f8317a63740 1 -- 172.21.15.133:0/1080011589 >> 172.21.15.133:0/1080011589 conn(0x56330fee3310 msgr2=0x56330ff15d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.559+0000 7f8317a63740 1 -- 172.21.15.133:0/1080011589 shutdown_connections 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.559+0000 7f8317a63740 1 -- 172.21.15.133:0/1080011589 wait complete. 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f8317a63740 1 -- start start 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f8317a63740 1 Processor -- start 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f8317a63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff1bb40 0x56330ff3a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f8317a63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1efe0 0x56330ff1dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f8317a63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff25370 0x56330ff1e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f8317a63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc00f9f0 con 0x56330ff1efe0 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f8317a63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc00f320 con 0x56330ff1bb40 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f8317a63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc00c560 con 0x56330ff25370 2024-09-11T23:37:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f8317a63740 10 client.0 _ll_register_callbacks cb 0x7f83000e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:57.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f830e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1efe0 0x56330ff1dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f830e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1efe0 0x56330ff1dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47196/0 (socket says 172.21.15.133:47196) 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.560+0000 7f830e7fc640 1 -- 172.21.15.133:0/1986657490 learned_addr learned my addr 172.21.15.133:0/1986657490 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.561+0000 7f830effd640 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff1bb40 0x56330ff3a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.561+0000 7f830e7fc640 1 -- 172.21.15.133:0/1986657490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff25370 msgr2=0x56330ff1e1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.561+0000 7f830e7fc640 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff25370 0x56330ff1e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.561+0000 7f830e7fc640 1 -- 172.21.15.133:0/1986657490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff1bb40 msgr2=0x56330ff3a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.561+0000 7f830e7fc640 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff1bb40 0x56330ff3a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.561+0000 7f830e7fc640 1 -- 172.21.15.133:0/1986657490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330ff1e730 con 0x56330ff1efe0 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.561+0000 7f830e7fc640 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1efe0 0x56330ff1dcb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8304005940 tx=0x7f8304009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f42cb5937781e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.561+0000 7f82f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.561+0000 7f82f2ff5640 1 -- 172.21.15.133:0/1986657490 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304052060 con 0x56330ff1efe0 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.561+0000 7f82f2ff5640 1 -- 172.21.15.133:0/1986657490 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f83040035f0 con 0x56330ff1efe0 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.561+0000 7f8317a63740 1 -- 172.21.15.133:0/1986657490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8304005520 con 0x56330ff1efe0 2024-09-11T23:37:57.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.562+0000 7f82f2ff5640 1 -- 172.21.15.133:0/1986657490 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8304049040 con 0x56330ff1efe0 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.562+0000 7f8317a63740 10 client.15885 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.562+0000 7f8317a63740 1 -- 172.21.15.133:0/1986657490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56330ff38ba0 con 0x56330ff1efe0 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.562+0000 7f82f2ff5640 1 -- 172.21.15.133:0/1986657490 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83040046c0 con 0x56330ff1efe0 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.563+0000 7f82f2ff5640 1 -- 172.21.15.133:0/1986657490 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8304055020 con 0x56330ff1efe0 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.563+0000 7f8317a63740 10 client.15885 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.563+0000 7f8317a63740 10 client.15885 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.563+0000 7f8317a63740 20 client.15885 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.563+0000 7f8317a63740 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56330ffa8a10 0x56330ffaae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.563+0000 7f8317a63740 4 client.15885 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.563+0000 7f8317a63740 1 -- 172.21.15.133:0/1986657490 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8304005520 con 0x56330ffa8a10 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.563+0000 7f830effd640 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56330ffa8a10 0x56330ffaae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f830effd640 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56330ffa8a10 0x56330ffaae30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84036b3668dbf290 server_cookie=dde04e89eb0b495 in_seq=0 out_seq=0 2024-09-11T23:37:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f82f2ff5640 10 client.15885 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f82f2ff5640 1 -- 172.21.15.133:0/1986657490 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8304005520 con 0x56330ffa8a10 2024-09-11T23:37:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f82f2ff5640 10 client.15885 handle_command_reply: tid=0 2024-09-11T23:37:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f8317a63740 1 client.15885 shutdown 2024-09-11T23:37:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f8317a63740 1 -- 172.21.15.133:0/1986657490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1efe0 msgr2=0x56330ff1dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f8317a63740 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1efe0 0x56330ff1dcb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8304005940 tx=0x7f8304009260 comp rx=0 tx=0).stop 2024-09-11T23:37:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f8317a63740 1 -- 172.21.15.133:0/1986657490 shutdown_connections 2024-09-11T23:37:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f8317a63740 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x56330ffa8a10 0x56330ffaae30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f8317a63740 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330ff25370 0x56330ff1e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f8317a63740 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56330ff1efe0 0x56330ff1dcb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f8317a63740 1 --2- 172.21.15.133:0/1986657490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56330ff1bb40 0x56330ff3a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f8317a63740 1 -- 172.21.15.133:0/1986657490 >> 172.21.15.133:0/1986657490 conn(0x56330fee3310 msgr2=0x56330ff23ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:57.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.564+0000 7f82f2ff5640 0 client.15885 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:37:57.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f8317a63740 1 -- 172.21.15.133:0/1986657490 shutdown_connections 2024-09-11T23:37:57.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f8317a63740 1 -- 172.21.15.133:0/1986657490 wait complete. 2024-09-11T23:37:57.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f8317a63740 20 client.15885 trim_cache size 0 max 16384 2024-09-11T23:37:57.578 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:37:57.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f831568a640 1 -- 172.21.15.133:0/1239079890 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f80a2f20 msgr2=0x7f82f80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:57.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f831568a640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f80a2f20 0x7f82f80a53e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8308005fd0 tx=0x7f8308009500 comp rx=0 tx=0).stop 2024-09-11T23:37:57.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f831568a640 1 -- 172.21.15.133:0/1239079890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8310101dd0 msgr2=0x7f83101984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:57.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f831568a640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8310101dd0 0x7f83101984c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f830000d950 tx=0x7f830000de10 comp rx=0 tx=0).stop 2024-09-11T23:37:57.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f831568a640 1 -- 172.21.15.133:0/1239079890 shutdown_connections 2024-09-11T23:37:57.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f831568a640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f80a2f20 0x7f82f80a53e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f831568a640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310106f00 0x7f831019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f831568a640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8310102780 0x7f8310198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f831568a640 1 --2- 172.21.15.133:0/1239079890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8310101dd0 0x7f83101984c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:57.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.565+0000 7f831568a640 1 -- 172.21.15.133:0/1239079890 >> 172.21.15.133:0/1239079890 conn(0x7f83100fd8b0 msgr2=0x7f83100fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:57.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.566+0000 7f831568a640 1 -- 172.21.15.133:0/1239079890 shutdown_connections 2024-09-11T23:37:57.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:57.566+0000 7f831568a640 1 -- 172.21.15.133:0/1239079890 wait complete. 2024-09-11T23:37:58.355 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:37:58.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:58 smithi138 ceph-mon[35142]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T23:37:58.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:37:58.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:58 smithi133 ceph-mon[31430]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T23:37:58.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:37:58.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:58 smithi178 ceph-mon[35882]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T23:37:58.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:37:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:37:59.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.198+0000 7eff7939c640 1 -- 172.21.15.133:0/2239820054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff74100a70 msgr2=0x7eff74100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:59.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.198+0000 7eff7939c640 1 --2- 172.21.15.133:0/2239820054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff74100a70 0x7eff74100ed0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009930 tx=0x7eff5c02a420 comp rx=0 tx=0).stop 2024-09-11T23:37:59.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.198+0000 7eff7939c640 1 -- 172.21.15.133:0/2239820054 shutdown_connections 2024-09-11T23:37:59.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.198+0000 7eff7939c640 1 --2- 172.21.15.133:0/2239820054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff74101410 0x7eff7410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.198+0000 7eff7939c640 1 --2- 172.21.15.133:0/2239820054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff74100a70 0x7eff74100ed0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.198+0000 7eff7939c640 1 --2- 172.21.15.133:0/2239820054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74106a70 0x7eff74106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.198+0000 7eff7939c640 1 -- 172.21.15.133:0/2239820054 >> 172.21.15.133:0/2239820054 conn(0x7eff740fc7d0 msgr2=0x7eff740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:59.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.198+0000 7eff7939c640 1 -- 172.21.15.133:0/2239820054 shutdown_connections 2024-09-11T23:37:59.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.198+0000 7eff7939c640 1 -- 172.21.15.133:0/2239820054 wait complete. 2024-09-11T23:37:59.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.199+0000 7eff7939c640 1 Processor -- start 2024-09-11T23:37:59.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.199+0000 7eff7939c640 1 -- start start 2024-09-11T23:37:59.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff7939c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff74100a70 0x7eff74198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff7939c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff74101410 0x7eff74198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff7939c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74106a70 0x7eff7419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff7939c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff74111f40 con 0x7eff74100a70 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff7939c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff74111dc0 con 0x7eff74106a70 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff7939c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7eff741120c0 con 0x7eff74101410 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff72ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff74100a70 0x7eff74198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff72ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff74100a70 0x7eff74198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47212/0 (socket says 172.21.15.133:47212) 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff72ffd640 1 -- 172.21.15.133:0/4258658108 learned_addr learned my addr 172.21.15.133:0/4258658108 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff737fe640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74106a70 0x7eff7419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff6bfff640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff74101410 0x7eff74198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff737fe640 1 -- 172.21.15.133:0/4258658108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff74101410 msgr2=0x7eff74198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff737fe640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff74101410 0x7eff74198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff737fe640 1 -- 172.21.15.133:0/4258658108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff74100a70 msgr2=0x7eff74198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff737fe640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff74100a70 0x7eff74198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff737fe640 1 -- 172.21.15.133:0/4258658108 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7419d3f0 con 0x7eff74106a70 2024-09-11T23:37:59.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff72ffd640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff74100a70 0x7eff74198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:59.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.200+0000 7eff6bfff640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff74101410 0x7eff74198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:59.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.201+0000 7eff737fe640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74106a70 0x7eff7419cd10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00e940 tx=0x7eff6c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:59.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.201+0000 7eff70ff9640 1 -- 172.21.15.133:0/4258658108 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c004d20 con 0x7eff74106a70 2024-09-11T23:37:59.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.201+0000 7eff7939c640 1 -- 172.21.15.133:0/4258658108 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7419d6e0 con 0x7eff74106a70 2024-09-11T23:37:59.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.201+0000 7eff70ff9640 1 -- 172.21.15.133:0/4258658108 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6c005020 con 0x7eff74106a70 2024-09-11T23:37:59.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.201+0000 7eff70ff9640 1 -- 172.21.15.133:0/4258658108 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c0116d0 con 0x7eff74106a70 2024-09-11T23:37:59.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.201+0000 7eff7939c640 1 -- 172.21.15.133:0/4258658108 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff741a4fc0 con 0x7eff74106a70 2024-09-11T23:37:59.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.204+0000 7eff70ff9640 1 -- 172.21.15.133:0/4258658108 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7eff6c011870 con 0x7eff74106a70 2024-09-11T23:37:59.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.204+0000 7eff7b775740 1 Processor -- start 2024-09-11T23:37:59.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.205+0000 7eff7b775740 1 -- start start 2024-09-11T23:37:59.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.205+0000 7eff7b775740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e3c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.205+0000 7eff7b775740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e3ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.205+0000 7eff7b775740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e49760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.205+0000 7eff7b775740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559d10da22f0 con 0x559d10e47370 2024-09-11T23:37:59.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.205+0000 7eff7b775740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d10d49cf0 con 0x559d10e3c5d0 2024-09-11T23:37:59.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.205+0000 7eff7b775740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559d10d9e2b0 con 0x559d10e3db40 2024-09-11T23:37:59.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.205+0000 7eff70ff9640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7eff600a2f20 0x7eff600a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.205+0000 7eff72ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e3ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.205+0000 7eff72ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e3ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:43056/0 (socket says 172.21.15.133:43056) 2024-09-11T23:37:59.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.205+0000 7eff72ffd640 1 -- 172.21.15.133:0/743022831 learned_addr learned my addr 172.21.15.133:0/743022831 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:59.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.206+0000 7eff70ff9640 1 -- 172.21.15.133:0/4258658108 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7eff6c0cf980 con 0x7eff74106a70 2024-09-11T23:37:59.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.206+0000 7eff737fe640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7eff600a2f20 0x7eff600a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.206+0000 7eff72ffd640 1 -- 172.21.15.133:0/743022831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 msgr2=0x559d10e3c090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:37:59.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.206+0000 7eff72ffd640 1 --2- 172.21.15.133:0/743022831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e3c090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.206+0000 7eff72ffd640 1 -- 172.21.15.133:0/743022831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 msgr2=0x559d10e49760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:37:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.206+0000 7eff6bfff640 1 --2- 172.21.15.133:0/743022831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e49760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.207+0000 7eff737fe640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7eff600a2f20 0x7eff600a53e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0ccf60 tx=0x7eff6c06dc90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.207+0000 7eff72ffd640 1 --2- 172.21.15.133:0/743022831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e49760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.207+0000 7eff72ffd640 1 -- 172.21.15.133:0/743022831 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d10d2e3a0 con 0x559d10e3c5d0 2024-09-11T23:37:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.207+0000 7eff6bfff640 1 --2- 172.21.15.133:0/743022831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e49760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:37:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.207+0000 7eff72ffd640 1 --2- 172.21.15.133:0/743022831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e3ca10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff64010580 tx=0x7eff64010a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9588296fe694fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.207+0000 7eff697fa640 1 -- 172.21.15.133:0/743022831 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6400e7d0 con 0x559d10e3c5d0 2024-09-11T23:37:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.207+0000 7eff697fa640 1 -- 172.21.15.133:0/743022831 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6400ee50 con 0x559d10e3c5d0 2024-09-11T23:37:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.207+0000 7eff697fa640 1 -- 172.21.15.133:0/743022831 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64019630 con 0x559d10e3c5d0 2024-09-11T23:37:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.208+0000 7eff7b775740 1 -- 172.21.15.133:0/743022831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 msgr2=0x559d10e3ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.208+0000 7eff7b775740 1 --2- 172.21.15.133:0/743022831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e3ca10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff64010580 tx=0x7eff64010a40 comp rx=0 tx=0).stop 2024-09-11T23:37:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.209+0000 7eff7b775740 1 -- 172.21.15.133:0/743022831 shutdown_connections 2024-09-11T23:37:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.209+0000 7eff7b775740 1 --2- 172.21.15.133:0/743022831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e49760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.209+0000 7eff7b775740 1 --2- 172.21.15.133:0/743022831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e3ca10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.209+0000 7eff7b775740 1 --2- 172.21.15.133:0/743022831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e3c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.209+0000 7eff7b775740 1 -- 172.21.15.133:0/743022831 >> 172.21.15.133:0/743022831 conn(0x559d10e05310 msgr2=0x559d10e38c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.209+0000 7eff7b775740 1 -- 172.21.15.133:0/743022831 shutdown_connections 2024-09-11T23:37:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.209+0000 7eff7b775740 1 -- 172.21.15.133:0/743022831 wait complete. 2024-09-11T23:37:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.209+0000 7eff7b775740 1 -- start start 2024-09-11T23:37:59.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.209+0000 7eff7b775740 1 Processor -- start 2024-09-11T23:37:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff7b775740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3c5d0 0x559d10e46cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff7b775740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e3db40 0x559d10e44000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff7b775740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e47370 0x559d10e44540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff7b775740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559d10e4a070 con 0x559d10e3db40 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff7b775740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d10d5d060 con 0x559d10e47370 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff7b775740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559d10e4a1f0 con 0x559d10e3c5d0 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff7b775740 10 client.0 _ll_register_callbacks cb 0x559d10ab08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff72ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e3db40 0x559d10e44000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff72ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e3db40 0x559d10e44000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47226/0 (socket says 172.21.15.133:47226) 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff72ffd640 1 -- 172.21.15.133:0/3850763611 learned_addr learned my addr 172.21.15.133:0/3850763611 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff6bfff640 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e47370 0x559d10e44540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff6bfff640 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3c5d0 0x559d10e46cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff72ffd640 1 -- 172.21.15.133:0/3850763611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3c5d0 msgr2=0x559d10e46cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff72ffd640 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3c5d0 0x559d10e46cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.210+0000 7eff72ffd640 1 -- 172.21.15.133:0/3850763611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e47370 msgr2=0x559d10e44540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:59.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.211+0000 7eff72ffd640 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e47370 0x559d10e44540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.211+0000 7eff72ffd640 1 -- 172.21.15.133:0/3850763611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d10b133c0 con 0x559d10e3db40 2024-09-11T23:37:59.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.211+0000 7eff6bfff640 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3c5d0 0x559d10e46cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.211+0000 7eff72ffd640 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e3db40 0x559d10e44000 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7eff64000c00 tx=0x7eff6400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4696a0297d1439 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.211+0000 7eff5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.211+0000 7eff5a7fc640 1 -- 172.21.15.133:0/3850763611 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64011440 con 0x559d10e3db40 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.211+0000 7eff7b775740 1 -- 172.21.15.133:0/3850763611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff6400ee50 con 0x559d10e3db40 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.211+0000 7eff5a7fc640 1 -- 172.21.15.133:0/3850763611 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7eff640115e0 con 0x559d10e3db40 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.211+0000 7eff5a7fc640 1 -- 172.21.15.133:0/3850763611 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff64031050 con 0x559d10e3db40 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.211+0000 7eff7b775740 10 client.15894 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.211+0000 7eff7b775740 1 -- 172.21.15.133:0/3850763611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d10e55b40 con 0x559d10e3db40 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.212+0000 7eff5a7fc640 1 -- 172.21.15.133:0/3850763611 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6400ee50 con 0x559d10e3db40 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.212+0000 7eff5a7fc640 1 -- 172.21.15.133:0/3850763611 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7eff6400e5d0 con 0x559d10e3db40 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.212+0000 7eff7b775740 10 client.15894 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.212+0000 7eff7b775740 10 client.15894 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:37:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.212+0000 7eff7b775740 20 client.15894 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:59.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.212+0000 7eff7b775740 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x559d10e56ef0 0x559d10e592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.212+0000 7eff7b775740 4 client.15894 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:37:59.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.212+0000 7eff7b775740 1 -- 172.21.15.133:0/3850763611 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d10daf860 con 0x559d10e56ef0 2024-09-11T23:37:59.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.212+0000 7eff6bfff640 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x559d10e56ef0 0x559d10e592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.213+0000 7eff6bfff640 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x559d10e56ef0 0x559d10e592e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea62f66964b6f98e server_cookie=fb9f3596b47596e3 in_seq=0 out_seq=0 2024-09-11T23:37:59.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.213+0000 7eff5a7fc640 10 client.15894 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.213+0000 7eff5a7fc640 1 -- 172.21.15.133:0/3850763611 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d10daf860 con 0x559d10e56ef0 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.213+0000 7eff5a7fc640 10 client.15894 handle_command_reply: tid=0 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 1 client.15894 shutdown 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 1 -- 172.21.15.133:0/3850763611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e3db40 msgr2=0x559d10e44000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e3db40 0x559d10e44000 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7eff64000c00 tx=0x7eff6400bd80 comp rx=0 tx=0).stop 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 1 -- 172.21.15.133:0/3850763611 shutdown_connections 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x559d10e56ef0 0x559d10e592e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e47370 0x559d10e44540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e3db40 0x559d10e44000 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 1 --2- 172.21.15.133:0/3850763611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3c5d0 0x559d10e46cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 1 -- 172.21.15.133:0/3850763611 >> 172.21.15.133:0/3850763611 conn(0x559d10e05310 msgr2=0x559d10e3e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff5a7fc640 0 client.15894 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 1 -- 172.21.15.133:0/3850763611 shutdown_connections 2024-09-11T23:37:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 1 -- 172.21.15.133:0/3850763611 wait complete. 2024-09-11T23:37:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.214+0000 7eff7b775740 20 client.15894 trim_cache size 0 max 16384 2024-09-11T23:37:59.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.239+0000 7eff7b775740 1 Processor -- start 2024-09-11T23:37:59.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.239+0000 7eff7b775740 1 -- start start 2024-09-11T23:37:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.240+0000 7eff7b775740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.240+0000 7eff7b775740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e37c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.240+0000 7eff7b775740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e3f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.240+0000 7eff7b775740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff5400e5d0 con 0x559d10e47370 2024-09-11T23:37:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.240+0000 7eff7b775740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff54010110 con 0x559d10e3c5d0 2024-09-11T23:37:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.240+0000 7eff7b775740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7eff5400eca0 con 0x559d10e3db40 2024-09-11T23:37:59.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.240+0000 7eff6bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.240+0000 7eff6bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:43080/0 (socket says 172.21.15.133:43080) 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.240+0000 7eff72ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e37c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.240+0000 7eff6bfff640 1 -- 172.21.15.133:0/1401200365 learned_addr learned my addr 172.21.15.133:0/1401200365 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.240+0000 7eff6bfff640 1 --2- 172.21.15.133:0/1401200365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e3f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.241+0000 7eff72ffd640 1 -- 172.21.15.133:0/1401200365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 msgr2=0x559d10e533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.241+0000 7eff72ffd640 1 --2- 172.21.15.133:0/1401200365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.241+0000 7eff72ffd640 1 -- 172.21.15.133:0/1401200365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 msgr2=0x559d10e3f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.241+0000 7eff72ffd640 1 --2- 172.21.15.133:0/1401200365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e3f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.241+0000 7eff72ffd640 1 -- 172.21.15.133:0/1401200365 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d10e42570 con 0x559d10e3db40 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.241+0000 7eff6bfff640 1 --2- 172.21.15.133:0/1401200365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.241+0000 7eff72ffd640 1 --2- 172.21.15.133:0/1401200365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e37c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7eff6400e9a0 tx=0x7eff64001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e179b0e672bf045 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.241+0000 7eff68ff9640 1 -- 172.21.15.133:0/1401200365 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6400e5d0 con 0x559d10e3db40 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.241+0000 7eff68ff9640 1 -- 172.21.15.133:0/1401200365 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff640055f0 con 0x559d10e3db40 2024-09-11T23:37:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.241+0000 7eff68ff9640 1 -- 172.21.15.133:0/1401200365 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64011440 con 0x559d10e3db40 2024-09-11T23:37:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.242+0000 7eff7b775740 1 -- 172.21.15.133:0/1401200365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 msgr2=0x559d10e37c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.242+0000 7eff7b775740 1 --2- 172.21.15.133:0/1401200365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e37c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7eff6400e9a0 tx=0x7eff64001e70 comp rx=0 tx=0).stop 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.243+0000 7eff7b775740 1 -- 172.21.15.133:0/1401200365 shutdown_connections 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.243+0000 7eff7b775740 1 --2- 172.21.15.133:0/1401200365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e3f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.243+0000 7eff7b775740 1 --2- 172.21.15.133:0/1401200365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e37c10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.243+0000 7eff7b775740 1 --2- 172.21.15.133:0/1401200365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.243+0000 7eff7b775740 1 -- 172.21.15.133:0/1401200365 >> 172.21.15.133:0/1401200365 conn(0x559d10e05310 msgr2=0x559d10e37500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.243+0000 7eff7b775740 1 -- 172.21.15.133:0/1401200365 shutdown_connections 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.243+0000 7eff7b775740 1 -- 172.21.15.133:0/1401200365 wait complete. 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.243+0000 7eff7b775740 1 -- start start 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.243+0000 7eff7b775740 1 Processor -- start 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff7b775740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e4f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff7b775740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e4fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff7b775740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e40830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff7b775740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff5400fa40 con 0x559d10e47370 2024-09-11T23:37:59.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff7b775740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff5400f370 con 0x559d10e3c5d0 2024-09-11T23:37:59.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff7b775740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7eff5400c5b0 con 0x559d10e3db40 2024-09-11T23:37:59.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff6bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e4f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff6bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e4f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:43094/0 (socket says 172.21.15.133:43094) 2024-09-11T23:37:59.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff6bfff640 1 -- 172.21.15.133:0/1550464365 learned_addr learned my addr 172.21.15.133:0/1550464365 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:37:59.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff7b775740 10 client.0 _ll_register_callbacks cb 0x7eff640103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:37:59.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff72ffd640 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e4fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff6bfff640 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e40830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff72ffd640 1 -- 172.21.15.133:0/1550464365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 msgr2=0x559d10e4f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff72ffd640 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e4f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff72ffd640 1 -- 172.21.15.133:0/1550464365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 msgr2=0x559d10e40830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:37:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff72ffd640 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e40830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.244+0000 7eff72ffd640 1 -- 172.21.15.133:0/1550464365 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d10e40ef0 con 0x559d10e3db40 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.245+0000 7eff72ffd640 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e4fc30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x559d10e2d380 tx=0x7eff6400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faf0f8e359082fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.245+0000 7eff5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.245+0000 7eff5a7fc640 1 -- 172.21.15.133:0/1550464365 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6400bd60 con 0x559d10e3db40 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.245+0000 7eff5a7fc640 1 -- 172.21.15.133:0/1550464365 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7eff64005320 con 0x559d10e3db40 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.245+0000 7eff7b775740 1 -- 172.21.15.133:0/1550464365 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff640055f0 con 0x559d10e3db40 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.245+0000 7eff5a7fc640 1 -- 172.21.15.133:0/1550464365 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff64011440 con 0x559d10e3db40 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.245+0000 7eff7b775740 10 client.25351 fetch_fsmap learned FSMap version 28 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.245+0000 7eff7b775740 1 -- 172.21.15.133:0/1550464365 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d10ec1280 con 0x559d10e3db40 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.245+0000 7eff5a7fc640 1 -- 172.21.15.133:0/1550464365 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff640055f0 con 0x559d10e3db40 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.246+0000 7eff5a7fc640 1 -- 172.21.15.133:0/1550464365 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7eff640036f0 con 0x559d10e3db40 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.246+0000 7eff7b775740 10 client.25351 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.246+0000 7eff7b775740 10 client.25351 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:37:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.246+0000 7eff7b775740 20 client.25351 populate_metadata read hostname 'smithi133' 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.246+0000 7eff7b775740 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x559d10ec1930 0x559d10ec3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.246+0000 7eff7b775740 4 client.25351 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.246+0000 7eff7b775740 1 -- 172.21.15.133:0/1550464365 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d10da9d70 con 0x559d10ec1930 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.246+0000 7eff6bfff640 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x559d10ec1930 0x559d10ec3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.246+0000 7eff5a7fc640 10 client.25351 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.246+0000 7eff6bfff640 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x559d10ec1930 0x559d10ec3d50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f1cf0d02d9a6a77 server_cookie=c2f99140117b14fc in_seq=0 out_seq=0 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff5a7fc640 1 -- 172.21.15.133:0/1550464365 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d10da9d70 con 0x559d10ec1930 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff5a7fc640 10 client.25351 handle_command_reply: tid=0 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 1 client.25351 shutdown 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 1 -- 172.21.15.133:0/1550464365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 msgr2=0x559d10e4fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e4fc30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x559d10e2d380 tx=0x7eff6400ee40 comp rx=0 tx=0).stop 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 1 -- 172.21.15.133:0/1550464365 shutdown_connections 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x559d10ec1930 0x559d10ec3d50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff5a7fc640 0 client.25351 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:37:59.251 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d10e47370 0x559d10e40830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d10e3db40 0x559d10e4fc30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 1 --2- 172.21.15.133:0/1550464365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d10e3c5d0 0x559d10e4f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 1 -- 172.21.15.133:0/1550464365 >> 172.21.15.133:0/1550464365 conn(0x559d10e05310 msgr2=0x559d10e37d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 1 -- 172.21.15.133:0/1550464365 shutdown_connections 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 1 -- 172.21.15.133:0/1550464365 wait complete. 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.247+0000 7eff7b775740 20 client.25351 trim_cache size 0 max 16384 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 -- 172.21.15.133:0/4258658108 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7eff600a2f20 msgr2=0x7eff600a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7eff600a2f20 0x7eff600a53e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0ccf60 tx=0x7eff6c06dc90 comp rx=0 tx=0).stop 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 -- 172.21.15.133:0/4258658108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74106a70 msgr2=0x7eff7419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74106a70 0x7eff7419cd10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00e940 tx=0x7eff6c00ee00 comp rx=0 tx=0).stop 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 -- 172.21.15.133:0/4258658108 shutdown_connections 2024-09-11T23:37:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7eff600a2f20 0x7eff600a53e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74106a70 0x7eff7419cd10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff74101410 0x7eff74198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 --2- 172.21.15.133:0/4258658108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff74100a70 0x7eff74198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:37:59.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 -- 172.21.15.133:0/4258658108 >> 172.21.15.133:0/4258658108 conn(0x7eff740fc7d0 msgr2=0x7eff740fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:37:59.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 -- 172.21.15.133:0/4258658108 shutdown_connections 2024-09-11T23:37:59.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:37:59.248+0000 7eff7939c640 1 -- 172.21.15.133:0/4258658108 wait complete. 2024-09-11T23:37:59.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:59 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:37:59.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:59 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:37:59.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:37:59 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:37:59.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:59 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:37:59.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:59 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:37:59.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:37:59 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:37:59.474 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:37:59.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:59 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:37:59.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:59 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:37:59.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:37:59 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:38:00.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:38:00.109 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:00.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.534+0000 7f859f16b640 1 -- 172.21.15.133:0/973683240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85981027c0 msgr2=0x7f8598106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:00.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.534+0000 7f859f16b640 1 --2- 172.21.15.133:0/973683240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85981027c0 0x7f8598106810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8584009900 tx=0x7f858402a410 comp rx=0 tx=0).stop 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.534+0000 7f859f16b640 1 -- 172.21.15.133:0/973683240 shutdown_connections 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.534+0000 7f859f16b640 1 --2- 172.21.15.133:0/973683240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8598106f40 0x7f859810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.534+0000 7f859f16b640 1 --2- 172.21.15.133:0/973683240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85981027c0 0x7f8598106810 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.534+0000 7f859f16b640 1 --2- 172.21.15.133:0/973683240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598101e10 0x7f85981021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.535+0000 7f859f16b640 1 -- 172.21.15.133:0/973683240 >> 172.21.15.133:0/973683240 conn(0x7f85980fd8d0 msgr2=0x7f85980ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.535+0000 7f859f16b640 1 -- 172.21.15.133:0/973683240 shutdown_connections 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.535+0000 7f859f16b640 1 -- 172.21.15.133:0/973683240 wait complete. 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.536+0000 7f859f16b640 1 Processor -- start 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.536+0000 7f859f16b640 1 -- start start 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.536+0000 7f859f16b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8598101e10 0x7f8598075c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.536+0000 7f859f16b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85981027c0 0x7f8598070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.536+0000 7f859f16b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598106f40 0x7f85980711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.536+0000 7f859cf07640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8598101e10 0x7f8598075c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.536+0000 7f8597fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85981027c0 0x7f8598070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.536+0000 7f8597fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85981027c0 0x7f8598070ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:43112/0 (socket says 172.21.15.133:43112) 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.536+0000 7f859cf07640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8598101e10 0x7f8598075c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:32818/0 (socket says 172.21.15.133:32818) 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.536+0000 7f8597fff640 1 -- 172.21.15.133:0/1642595688 learned_addr learned my addr 172.21.15.133:0/1642595688 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.537+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8598111ff0 con 0x7f8598106f40 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.537+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8598111e70 con 0x7f85981027c0 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.537+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8598112170 con 0x7f8598101e10 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.537+0000 7f859d708640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598106f40 0x7f85980711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.537+0000 7f8597fff640 1 -- 172.21.15.133:0/1642595688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8598101e10 msgr2=0x7f8598075c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.537+0000 7f8597fff640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8598101e10 0x7f8598075c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.537+0000 7f8597fff640 1 -- 172.21.15.133:0/1642595688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598106f40 msgr2=0x7f85980711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.537+0000 7f8597fff640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598106f40 0x7f85980711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.537+0000 7f8597fff640 1 -- 172.21.15.133:0/1642595688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8598071750 con 0x7f85981027c0 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.537+0000 7f8597fff640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85981027c0 0x7f8598070ca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8584009850 tx=0x7f8584030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.537+0000 7f8595ffb640 1 -- 172.21.15.133:0/1642595688 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858403f070 con 0x7f85981027c0 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.538+0000 7f8595ffb640 1 -- 172.21.15.133:0/1642595688 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8584004460 con 0x7f85981027c0 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.538+0000 7f8595ffb640 1 -- 172.21.15.133:0/1642595688 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584035030 con 0x7f85981027c0 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.538+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85980719e0 con 0x7f85981027c0 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.538+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85981af6a0 con 0x7f85981027c0 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.540+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85981032f0 con 0x7f85981027c0 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.541+0000 7f8595ffb640 1 -- 172.21.15.133:0/1642595688 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f8584042070 con 0x7f85981027c0 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.542+0000 7f8595ffb640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f85880a2ea0 0x7f85880a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.542+0000 7f8595ffb640 1 -- 172.21.15.133:0/1642595688 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f85840eff80 con 0x7f85981027c0 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.542+0000 7f859cf07640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f85880a2ea0 0x7f85880a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.543+0000 7f859cf07640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f85880a2ea0 0x7f85880a5360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f858c0095f0 tx=0x7f858c005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.548+0000 7f8595ffb640 1 -- 172.21.15.133:0/1642595688 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85840bc9d0 con 0x7f85981027c0 2024-09-11T23:38:00.656 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:00.820 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:00 smithi133 ceph-mon[31430]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-09-11T23:38:00.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.913+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85981afa20 con 0x7f85981027c0 2024-09-11T23:38:00.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.914+0000 7f8595ffb640 1 -- 172.21.15.133:0/1642595688 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f85840bc2d0 con 0x7f85981027c0 2024-09-11T23:38:00.914 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:38:00.914 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:38:00.915 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:38:00.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f85880a2ea0 msgr2=0x7f85880a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:00.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f85880a2ea0 0x7f85880a5360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f858c0095f0 tx=0x7f858c005f90 comp rx=0 tx=0).stop 2024-09-11T23:38:00.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85981027c0 msgr2=0x7f8598070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:00.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85981027c0 0x7f8598070ca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8584009850 tx=0x7f8584030650 comp rx=0 tx=0).stop 2024-09-11T23:38:00.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 shutdown_connections 2024-09-11T23:38:00.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f85880a2ea0 0x7f85880a5360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:00.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598106f40 0x7f85980711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:00.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85981027c0 0x7f8598070ca0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:00.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 --2- 172.21.15.133:0/1642595688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8598101e10 0x7f8598075c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:00.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 >> 172.21.15.133:0/1642595688 conn(0x7f85980fd8d0 msgr2=0x7f85980fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:00.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 shutdown_connections 2024-09-11T23:38:00.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:00.918+0000 7f859f16b640 1 -- 172.21.15.133:0/1642595688 wait complete. 2024-09-11T23:38:00.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:00 smithi138 ceph-mon[35142]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-09-11T23:38:01.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:00 smithi178 ceph-mon[35882]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-09-11T23:38:01.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.435+0000 7f86e9ab7640 1 -- 172.21.15.133:0/1265841375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e4103650 msgr2=0x7f86e410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:01.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.435+0000 7f86e9ab7640 1 --2- 172.21.15.133:0/1265841375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e4103650 0x7f86e410bb40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009900 tx=0x7f86d802a480 comp rx=0 tx=0).stop 2024-09-11T23:38:01.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.435+0000 7f86e9ab7640 1 -- 172.21.15.133:0/1265841375 shutdown_connections 2024-09-11T23:38:01.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.435+0000 7f86e9ab7640 1 --2- 172.21.15.133:0/1265841375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e410c160 0x7f86e410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.435+0000 7f86e9ab7640 1 --2- 172.21.15.133:0/1265841375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e4103650 0x7f86e410bb40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.435+0000 7f86e9ab7640 1 --2- 172.21.15.133:0/1265841375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e4102d30 0x7f86e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.435+0000 7f86e9ab7640 1 -- 172.21.15.133:0/1265841375 >> 172.21.15.133:0/1265841375 conn(0x7f86e40fc790 msgr2=0x7f86e40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:01.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.435+0000 7f86e9ab7640 1 -- 172.21.15.133:0/1265841375 shutdown_connections 2024-09-11T23:38:01.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.435+0000 7f86e9ab7640 1 -- 172.21.15.133:0/1265841375 wait complete. 2024-09-11T23:38:01.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.436+0000 7f86e9ab7640 1 Processor -- start 2024-09-11T23:38:01.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.436+0000 7f86e9ab7640 1 -- start start 2024-09-11T23:38:01.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.436+0000 7f86e9ab7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e4102d30 0x7f86e4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.436+0000 7f86e9ab7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e4103650 0x7f86e4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.436+0000 7f86e9ab7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e410c160 0x7f86e4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.436+0000 7f86e9ab7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4111f60 con 0x7f86e4103650 2024-09-11T23:38:01.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.436+0000 7f86e9ab7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4111de0 con 0x7f86e4102d30 2024-09-11T23:38:01.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.436+0000 7f86e9ab7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41120e0 con 0x7f86e410c160 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.437+0000 7f86e2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e4103650 0x7f86e4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.437+0000 7f86e2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e4103650 0x7f86e4100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47292/0 (socket says 172.21.15.133:47292) 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.437+0000 7f86e2ffd640 1 -- 172.21.15.133:0/534881391 learned_addr learned my addr 172.21.15.133:0/534881391 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.437+0000 7f86e3fff640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e410c160 0x7f86e4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.437+0000 7f86e37fe640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e4102d30 0x7f86e4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.437+0000 7f86e2ffd640 1 -- 172.21.15.133:0/534881391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e410c160 msgr2=0x7f86e4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.437+0000 7f86e2ffd640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e410c160 0x7f86e4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.437+0000 7f86e2ffd640 1 -- 172.21.15.133:0/534881391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e4102d30 msgr2=0x7f86e4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.437+0000 7f86e2ffd640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e4102d30 0x7f86e4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.437+0000 7f86e2ffd640 1 -- 172.21.15.133:0/534881391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e41a69f0 con 0x7f86e4103650 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.437+0000 7f86e37fe640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e4102d30 0x7f86e4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.438+0000 7f86e2ffd640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e4103650 0x7f86e4100ed0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f86d8006010 tx=0x7f86d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:01.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.438+0000 7f86e0ff9640 1 -- 172.21.15.133:0/534881391 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d803f030 con 0x7f86e4103650 2024-09-11T23:38:01.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.438+0000 7f86e0ff9640 1 -- 172.21.15.133:0/534881391 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d8003ce0 con 0x7f86e4103650 2024-09-11T23:38:01.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.438+0000 7f86e9ab7640 1 -- 172.21.15.133:0/534881391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e41a6c80 con 0x7f86e4103650 2024-09-11T23:38:01.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.438+0000 7f86e0ff9640 1 -- 172.21.15.133:0/534881391 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8035070 con 0x7f86e4103650 2024-09-11T23:38:01.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.439+0000 7f86e9ab7640 1 -- 172.21.15.133:0/534881391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e41a71b0 con 0x7f86e4103650 2024-09-11T23:38:01.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.440+0000 7f86e0ff9640 1 -- 172.21.15.133:0/534881391 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f86d8042030 con 0x7f86e4103650 2024-09-11T23:38:01.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.441+0000 7f86e0ff9640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f86cc0a2f20 0x7f86cc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.441+0000 7f86e0ff9640 1 -- 172.21.15.133:0/534881391 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f86d80ef690 con 0x7f86e4103650 2024-09-11T23:38:01.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.441+0000 7f86e37fe640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f86cc0a2f20 0x7f86cc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.442+0000 7f86e37fe640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f86cc0a2f20 0x7f86cc0a53e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86d40058c0 tx=0x7f86d4005850 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:01.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.442+0000 7f86ebe90740 1 Processor -- start 2024-09-11T23:38:01.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.442+0000 7f86ebe90740 1 -- start start 2024-09-11T23:38:01.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 0x5577e2041090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2041a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 0x5577e2056f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86ebe90740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5577e1fa72f0 con 0x5577e2054b80 2024-09-11T23:38:01.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86ebe90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5577e1f4ecf0 con 0x5577e20415d0 2024-09-11T23:38:01.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86ebe90740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5577e1fa32b0 con 0x5577e2042b40 2024-09-11T23:38:01.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 0x5577e2041090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 0x5577e2056f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 0x5577e2041090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:32842/0 (socket says 172.21.15.133:32842) 2024-09-11T23:38:01.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e37fe640 1 -- 172.21.15.133:0/3324136960 learned_addr learned my addr 172.21.15.133:0/3324136960 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:01.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e2ffd640 1 --2- 172.21.15.133:0/3324136960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2041a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e37fe640 1 -- 172.21.15.133:0/3324136960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 msgr2=0x5577e2041a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:01.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e37fe640 1 --2- 172.21.15.133:0/3324136960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2041a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e37fe640 1 -- 172.21.15.133:0/3324136960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 msgr2=0x5577e2056f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:01.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e37fe640 1 --2- 172.21.15.133:0/3324136960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 0x5577e2056f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e37fe640 1 -- 172.21.15.133:0/3324136960 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e1f333a0 con 0x5577e2042b40 2024-09-11T23:38:01.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e3fff640 1 --2- 172.21.15.133:0/3324136960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 0x5577e2056f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:01.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.443+0000 7f86e37fe640 1 --2- 172.21.15.133:0/3324136960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 0x5577e2041090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f86d4010010 tx=0x7f86d4012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a5f5ae84b17588a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:01.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.444+0000 7f86c9ffb640 1 -- 172.21.15.133:0/3324136960 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4033070 con 0x5577e2042b40 2024-09-11T23:38:01.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.444+0000 7f86c9ffb640 1 -- 172.21.15.133:0/3324136960 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d402fb40 con 0x5577e2042b40 2024-09-11T23:38:01.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.444+0000 7f86c9ffb640 1 -- 172.21.15.133:0/3324136960 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4010af0 con 0x5577e2042b40 2024-09-11T23:38:01.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.445+0000 7f86ebe90740 1 -- 172.21.15.133:0/3324136960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 msgr2=0x5577e2041090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:01.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.445+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3324136960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 0x5577e2041090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f86d4010010 tx=0x7f86d4012000 comp rx=0 tx=0).stop 2024-09-11T23:38:01.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.445+0000 7f86ebe90740 1 -- 172.21.15.133:0/3324136960 shutdown_connections 2024-09-11T23:38:01.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.445+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3324136960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 0x5577e2056f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.445+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3324136960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2041a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.445+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3324136960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 0x5577e2041090 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.445+0000 7f86ebe90740 1 -- 172.21.15.133:0/3324136960 >> 172.21.15.133:0/3324136960 conn(0x5577e200a310 msgr2=0x5577e203dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:01.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.445+0000 7f86ebe90740 1 -- 172.21.15.133:0/3324136960 shutdown_connections 2024-09-11T23:38:01.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.445+0000 7f86ebe90740 1 -- 172.21.15.133:0/3324136960 wait complete. 2024-09-11T23:38:01.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.446+0000 7f86ebe90740 1 -- start start 2024-09-11T23:38:01.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.446+0000 7f86ebe90740 1 Processor -- start 2024-09-11T23:38:01.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.447+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e20544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.447+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 0x5577e2051780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.447+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 0x5577e2051ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.447+0000 7f86e37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e20544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.447+0000 7f86e37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e20544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:43150/0 (socket says 172.21.15.133:43150) 2024-09-11T23:38:01.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.447+0000 7f86e2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 0x5577e2051780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.447+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 0x5577e2051ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.447+0000 7f86e2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 0x5577e2051780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:32844/0 (socket says 172.21.15.133:32844) 2024-09-11T23:38:01.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.447+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 0x5577e2051ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47324/0 (socket says 172.21.15.133:47324) 2024-09-11T23:38:01.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86e37fe640 1 -- 172.21.15.133:0/2416273566 learned_addr learned my addr 172.21.15.133:0/2416273566 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:01.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86ebe90740 1 -- 172.21.15.133:0/2416273566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5577e2057880 con 0x5577e2054b80 2024-09-11T23:38:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86ebe90740 1 -- 172.21.15.133:0/2416273566 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5577e1f62060 con 0x5577e20415d0 2024-09-11T23:38:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86ebe90740 1 -- 172.21.15.133:0/2416273566 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5577e2057a00 con 0x5577e2042b40 2024-09-11T23:38:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86ebe90740 10 client.0 _ll_register_callbacks cb 0x5577e1cb58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86e3fff640 1 -- 172.21.15.133:0/2416273566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 msgr2=0x5577e2051780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86e3fff640 1 --2- 172.21.15.133:0/2416273566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 0x5577e2051780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86e3fff640 1 -- 172.21.15.133:0/2416273566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 msgr2=0x5577e20544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86e3fff640 1 --2- 172.21.15.133:0/2416273566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e20544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86e3fff640 1 -- 172.21.15.133:0/2416273566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e1d183c0 con 0x5577e2054b80 2024-09-11T23:38:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86e3fff640 1 --2- 172.21.15.133:0/2416273566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 0x5577e2051ce0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005fd0 tx=0x7f86dc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c9ea87a76273f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.448+0000 7f86c2ffd640 1 -- 172.21.15.133:0/2416273566 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc003b80 con 0x5577e2054b80 2024-09-11T23:38:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.449+0000 7f86c2ffd640 1 -- 172.21.15.133:0/2416273566 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f86dc003d20 con 0x5577e2054b80 2024-09-11T23:38:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.449+0000 7f86ebe90740 1 -- 172.21.15.133:0/2416273566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d402fb40 con 0x5577e2054b80 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.449+0000 7f86c2ffd640 1 -- 172.21.15.133:0/2416273566 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86dc010990 con 0x5577e2054b80 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.449+0000 7f86ebe90740 10 client.15918 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.449+0000 7f86ebe90740 1 -- 172.21.15.133:0/2416273566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5577e20632b0 con 0x5577e2054b80 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.449+0000 7f86c2ffd640 1 -- 172.21.15.133:0/2416273566 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d402fb40 con 0x5577e2054b80 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.449+0000 7f86c2ffd640 1 -- 172.21.15.133:0/2416273566 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f86dc025420 con 0x5577e2054b80 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.449+0000 7f86ebe90740 10 client.15918 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.449+0000 7f86ebe90740 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.450+0000 7f86ebe90740 20 client.15918 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.450+0000 7f86ebe90740 1 --2- 172.21.15.133:0/2416273566 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5577e20646c0 0x5577e2066ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.450+0000 7f86ebe90740 4 client.15918 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.450+0000 7f86ebe90740 1 -- 172.21.15.133:0/2416273566 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577e1fb4860 con 0x5577e20646c0 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.450+0000 7f86e37fe640 1 --2- 172.21.15.133:0/2416273566 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5577e20646c0 0x5577e2066ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.450+0000 7f86e37fe640 1 --2- 172.21.15.133:0/2416273566 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5577e20646c0 0x5577e2066ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfbcd7d42f0b44db server_cookie=c9bb087e056c4dd6 in_seq=0 out_seq=0 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.450+0000 7f86c2ffd640 10 client.15918 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.451+0000 7f86c2ffd640 1 -- 172.21.15.133:0/2416273566 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577e1fb4860 con 0x5577e20646c0 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.451+0000 7f86c2ffd640 10 client.15918 handle_command_reply: tid=0 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.451+0000 7f86ebe90740 1 client.15918 shutdown 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.451+0000 7f86ebe90740 1 -- 172.21.15.133:0/2416273566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 msgr2=0x5577e2051ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.451+0000 7f86ebe90740 1 --2- 172.21.15.133:0/2416273566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 0x5577e2051ce0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005fd0 tx=0x7f86dc011b50 comp rx=0 tx=0).stop 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.452+0000 7f86ebe90740 1 -- 172.21.15.133:0/2416273566 shutdown_connections 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.452+0000 7f86ebe90740 1 --2- 172.21.15.133:0/2416273566 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5577e20646c0 0x5577e2066ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.452+0000 7f86c2ffd640 0 client.15918 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.452+0000 7f86ebe90740 1 --2- 172.21.15.133:0/2416273566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2054b80 0x5577e2051ce0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.452+0000 7f86ebe90740 1 --2- 172.21.15.133:0/2416273566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2042b40 0x5577e2051780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.452+0000 7f86ebe90740 1 --2- 172.21.15.133:0/2416273566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e20544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.452+0000 7f86ebe90740 1 -- 172.21.15.133:0/2416273566 >> 172.21.15.133:0/2416273566 conn(0x5577e200a310 msgr2=0x5577e2043960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:01.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.452+0000 7f86ebe90740 1 -- 172.21.15.133:0/2416273566 shutdown_connections 2024-09-11T23:38:01.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.452+0000 7f86ebe90740 1 -- 172.21.15.133:0/2416273566 wait complete. 2024-09-11T23:38:01.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.452+0000 7f86ebe90740 20 client.15918 trim_cache size 0 max 16384 2024-09-11T23:38:01.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86ebe90740 1 Processor -- start 2024-09-11T23:38:01.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86ebe90740 1 -- start start 2024-09-11T23:38:01.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2060b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 0x5577e206c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2054b80 0x5577e206c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86ebe90740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f86e406fde0 con 0x5577e2042b40 2024-09-11T23:38:01.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86ebe90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4071920 con 0x5577e20415d0 2024-09-11T23:38:01.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86ebe90740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f86e40704b0 con 0x5577e2054b80 2024-09-11T23:38:01.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86e2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 0x5577e206c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86e2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 0x5577e206c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47338/0 (socket says 172.21.15.133:47338) 2024-09-11T23:38:01.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86e2ffd640 1 -- 172.21.15.133:0/3897983376 learned_addr learned my addr 172.21.15.133:0/3897983376 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:01.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.465+0000 7f86e37fe640 1 --2- 172.21.15.133:0/3897983376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2060b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.466+0000 7f86e3fff640 1 --2- 172.21.15.133:0/3897983376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2054b80 0x5577e206c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.466+0000 7f86e37fe640 1 -- 172.21.15.133:0/3897983376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2054b80 msgr2=0x5577e206c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:01.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.466+0000 7f86e37fe640 1 --2- 172.21.15.133:0/3897983376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2054b80 0x5577e206c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.466+0000 7f86e37fe640 1 -- 172.21.15.133:0/3897983376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 msgr2=0x5577e206c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:01.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.466+0000 7f86e37fe640 1 --2- 172.21.15.133:0/3897983376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 0x5577e206c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.466+0000 7f86e37fe640 1 -- 172.21.15.133:0/3897983376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e2054840 con 0x5577e20415d0 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.466+0000 7f86e2ffd640 1 --2- 172.21.15.133:0/3897983376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 0x5577e206c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.466+0000 7f86e37fe640 1 --2- 172.21.15.133:0/3897983376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2060b20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5577e20672c0 tx=0x7f86d4010770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=891863621bea387f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.466+0000 7f86c97fa640 1 -- 172.21.15.133:0/3897983376 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4033040 con 0x5577e20415d0 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.466+0000 7f86c97fa640 1 -- 172.21.15.133:0/3897983376 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d4010cc0 con 0x5577e20415d0 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.466+0000 7f86c97fa640 1 -- 172.21.15.133:0/3897983376 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d402f6c0 con 0x5577e20415d0 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.467+0000 7f86ebe90740 1 -- 172.21.15.133:0/3897983376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 msgr2=0x5577e2060b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.467+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3897983376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2060b20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5577e20672c0 tx=0x7f86d4010770 comp rx=0 tx=0).stop 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.467+0000 7f86ebe90740 1 -- 172.21.15.133:0/3897983376 shutdown_connections 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.467+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3897983376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2054b80 0x5577e206c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.467+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3897983376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 0x5577e206c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.467+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3897983376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2060b20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.467+0000 7f86ebe90740 1 -- 172.21.15.133:0/3897983376 >> 172.21.15.133:0/3897983376 conn(0x5577e200a310 msgr2=0x5577e2051740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:01.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.468+0000 7f86ebe90740 1 -- 172.21.15.133:0/3897983376 shutdown_connections 2024-09-11T23:38:01.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.468+0000 7f86ebe90740 1 -- 172.21.15.133:0/3897983376 wait complete. 2024-09-11T23:38:01.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.468+0000 7f86ebe90740 1 -- start start 2024-09-11T23:38:01.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.468+0000 7f86ebe90740 1 Processor -- start 2024-09-11T23:38:01.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2059150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 0x5577e2059690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86ebe90740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2054b80 0x5577e20ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86ebe90740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4071250 con 0x5577e2042b40 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86ebe90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4070b80 con 0x5577e20415d0 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86ebe90740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f86e406dde0 con 0x5577e2054b80 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86ebe90740 10 client.0 _ll_register_callbacks cb 0x7f86dc009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86e37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2059150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86e37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2059150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:43166/0 (socket says 172.21.15.133:43166) 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86e37fe640 1 -- 172.21.15.133:0/3529772049 learned_addr learned my addr 172.21.15.133:0/3529772049 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86e3fff640 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2054b80 0x5577e20ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86e2ffd640 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 0x5577e2059690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86e37fe640 1 -- 172.21.15.133:0/3529772049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2054b80 msgr2=0x5577e20ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86e37fe640 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2054b80 0x5577e20ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86e37fe640 1 -- 172.21.15.133:0/3529772049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 msgr2=0x5577e2059690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86e37fe640 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 0x5577e2059690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86e37fe640 1 -- 172.21.15.133:0/3529772049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e20ce890 con 0x5577e20415d0 2024-09-11T23:38:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.469+0000 7f86e2ffd640 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 0x5577e2059690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.470+0000 7f86e37fe640 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2059150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f86d400bcc0 tx=0x7f86d400b360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c17c37f67ea787a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.470+0000 7f86c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.470+0000 7f86c2ffd640 1 -- 172.21.15.133:0/3529772049 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4033050 con 0x5577e20415d0 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.470+0000 7f86c2ffd640 1 -- 172.21.15.133:0/3529772049 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f86d40109e0 con 0x5577e20415d0 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.470+0000 7f86ebe90740 1 -- 172.21.15.133:0/3529772049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d4010cc0 con 0x5577e20415d0 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.470+0000 7f86c2ffd640 1 -- 172.21.15.133:0/3529772049 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86d4005510 con 0x5577e20415d0 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.470+0000 7f86ebe90740 10 client.25340 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.470+0000 7f86ebe90740 1 -- 172.21.15.133:0/3529772049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5577e20cfa90 con 0x5577e20415d0 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.471+0000 7f86c2ffd640 1 -- 172.21.15.133:0/3529772049 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d4010cc0 con 0x5577e20415d0 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.471+0000 7f86c2ffd640 1 -- 172.21.15.133:0/3529772049 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f86d40384c0 con 0x5577e20415d0 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.471+0000 7f86ebe90740 10 client.25340 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.471+0000 7f86ebe90740 10 client.25340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.471+0000 7f86ebe90740 20 client.25340 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.471+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5577e20646c0 0x5577e20d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.471+0000 7f86ebe90740 4 client.25340 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:38:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.471+0000 7f86ebe90740 1 -- 172.21.15.133:0/3529772049 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577e1faed70 con 0x5577e20646c0 2024-09-11T23:38:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.471+0000 7f86e2ffd640 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5577e20646c0 0x5577e20d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.471+0000 7f86c2ffd640 10 client.25340 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.471+0000 7f86e2ffd640 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5577e20646c0 0x5577e20d2010 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbab70a5fa364fad server_cookie=f754fb367b215117 in_seq=0 out_seq=0 2024-09-11T23:38:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86c2ffd640 1 -- 172.21.15.133:0/3529772049 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577e1faed70 con 0x5577e20646c0 2024-09-11T23:38:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86c2ffd640 10 client.25340 handle_command_reply: tid=0 2024-09-11T23:38:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 1 client.25340 shutdown 2024-09-11T23:38:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 1 -- 172.21.15.133:0/3529772049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 msgr2=0x5577e2059150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2059150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f86d400bcc0 tx=0x7f86d400b360 comp rx=0 tx=0).stop 2024-09-11T23:38:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 1 -- 172.21.15.133:0/3529772049 shutdown_connections 2024-09-11T23:38:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5577e20646c0 0x5577e20d2010 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577e2054b80 0x5577e20ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577e2042b40 0x5577e2059690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 1 --2- 172.21.15.133:0/3529772049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577e20415d0 0x5577e2059150 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86c2ffd640 0 client.25340 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 1 -- 172.21.15.133:0/3529772049 >> 172.21.15.133:0/3529772049 conn(0x5577e200a310 msgr2=0x5577e2052fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 1 -- 172.21.15.133:0/3529772049 shutdown_connections 2024-09-11T23:38:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 1 -- 172.21.15.133:0/3529772049 wait complete. 2024-09-11T23:38:01.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.472+0000 7f86ebe90740 20 client.25340 trim_cache size 0 max 16384 2024-09-11T23:38:01.477 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:38:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.473+0000 7f86e9ab7640 1 -- 172.21.15.133:0/534881391 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f86cc0a2f20 msgr2=0x7f86cc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.473+0000 7f86e9ab7640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f86cc0a2f20 0x7f86cc0a53e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86d40058c0 tx=0x7f86d4005850 comp rx=0 tx=0).stop 2024-09-11T23:38:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.473+0000 7f86e9ab7640 1 -- 172.21.15.133:0/534881391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e4103650 msgr2=0x7f86e4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.473+0000 7f86e9ab7640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e4103650 0x7f86e4100ed0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f86d8006010 tx=0x7f86d80042f0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.473+0000 7f86e9ab7640 1 -- 172.21.15.133:0/534881391 shutdown_connections 2024-09-11T23:38:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.473+0000 7f86e9ab7640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f86cc0a2f20 0x7f86cc0a53e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.473+0000 7f86e9ab7640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e410c160 0x7f86e4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.473+0000 7f86e9ab7640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e4103650 0x7f86e4100ed0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.473+0000 7f86e9ab7640 1 --2- 172.21.15.133:0/534881391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e4102d30 0x7f86e4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.473+0000 7f86e9ab7640 1 -- 172.21.15.133:0/534881391 >> 172.21.15.133:0/534881391 conn(0x7f86e40fc790 msgr2=0x7f86e40fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.474+0000 7f86e9ab7640 1 -- 172.21.15.133:0/534881391 shutdown_connections 2024-09-11T23:38:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:01.474+0000 7f86e9ab7640 1 -- 172.21.15.133:0/534881391 wait complete. 2024-09-11T23:38:01.735 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:01 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1642595688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:01.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:01 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1642595688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:02.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:01 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1642595688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:02.189 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id 942f90c7 --release --await' 2024-09-11T23:38:02.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id 942f90c7 --release --await 2024-09-11T23:38:02.877 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:02 smithi133 ceph-mon[31430]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-11T23:38:02.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:02 smithi138 ceph-mon[35142]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-11T23:38:02.988 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:03.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:02 smithi178 ceph-mon[35882]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-11T23:38:03.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.493+0000 7f169fcb6640 1 -- 172.21.15.133:0/2003149412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16981082a0 msgr2=0x7f169810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.493+0000 7f169fcb6640 1 --2- 172.21.15.133:0/2003149412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16981082a0 0x7f169810ec40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1690009cd0 tx=0x7f169002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.494+0000 7f169fcb6640 1 -- 172.21.15.133:0/2003149412 shutdown_connections 2024-09-11T23:38:03.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.494+0000 7f169fcb6640 1 --2- 172.21.15.133:0/2003149412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16981082a0 0x7f169810ec40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.494+0000 7f169fcb6640 1 --2- 172.21.15.133:0/2003149412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16981036e0 0x7f1698107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.494+0000 7f169fcb6640 1 --2- 172.21.15.133:0/2003149412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1698102d30 0x7f1698103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.494+0000 7f169fcb6640 1 -- 172.21.15.133:0/2003149412 >> 172.21.15.133:0/2003149412 conn(0x7f16980fc7d0 msgr2=0x7f16980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:03.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.494+0000 7f169fcb6640 1 -- 172.21.15.133:0/2003149412 shutdown_connections 2024-09-11T23:38:03.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.495+0000 7f169fcb6640 1 -- 172.21.15.133:0/2003149412 wait complete. 2024-09-11T23:38:03.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.495+0000 7f169fcb6640 1 Processor -- start 2024-09-11T23:38:03.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.495+0000 7f169fcb6640 1 -- start start 2024-09-11T23:38:03.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169fcb6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1698102d30 0x7f1698198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169fcb6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16981036e0 0x7f1698198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169fcb6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16981082a0 0x7f169819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169fcb6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f16981120d0 con 0x7f16981036e0 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169fcb6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1698111f50 con 0x7f16981082a0 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169fcb6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1698112250 con 0x7f1698102d30 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169d251640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16981036e0 0x7f1698198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169d251640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16981036e0 0x7f1698198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33026/0 (socket says 172.21.15.133:33026) 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169d251640 1 -- 172.21.15.133:0/2750017846 learned_addr learned my addr 172.21.15.133:0/2750017846 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169da52640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1698102d30 0x7f1698198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169e253640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16981082a0 0x7f169819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169d251640 1 -- 172.21.15.133:0/2750017846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1698102d30 msgr2=0x7f1698198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169d251640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1698102d30 0x7f1698198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169d251640 1 -- 172.21.15.133:0/2750017846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16981082a0 msgr2=0x7f169819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169d251640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16981082a0 0x7f169819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.496+0000 7f169d251640 1 -- 172.21.15.133:0/2750017846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169819d2b0 con 0x7f16981036e0 2024-09-11T23:38:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.497+0000 7f169e253640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16981082a0 0x7f169819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.497+0000 7f169d251640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16981036e0 0x7f1698198840 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f168800d740 tx=0x7f168800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.497+0000 7f168effd640 1 -- 172.21.15.133:0/2750017846 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688017070 con 0x7f16981036e0 2024-09-11T23:38:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.497+0000 7f168effd640 1 -- 172.21.15.133:0/2750017846 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168800de50 con 0x7f16981036e0 2024-09-11T23:38:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.497+0000 7f168effd640 1 -- 172.21.15.133:0/2750017846 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168800f6b0 con 0x7f16981036e0 2024-09-11T23:38:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.497+0000 7f169fcb6640 1 -- 172.21.15.133:0/2750017846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169819d5a0 con 0x7f16981036e0 2024-09-11T23:38:03.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.498+0000 7f169fcb6640 1 -- 172.21.15.133:0/2750017846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16981a4e80 con 0x7f16981036e0 2024-09-11T23:38:03.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.502+0000 7f168effd640 1 -- 172.21.15.133:0/2750017846 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f168801a070 con 0x7f16981036e0 2024-09-11T23:38:03.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.502+0000 7f16a208f740 1 Processor -- start 2024-09-11T23:38:03.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.502+0000 7f16a208f740 1 -- start start 2024-09-11T23:38:03.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.502+0000 7f16a208f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c52a0 0x55c5d50c5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.503+0000 7f16a208f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c5bc0 0x55c5d50bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.503+0000 7f16a208f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c0260 0x55c5d50c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.503+0000 7f16a208f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c5d501b620 con 0x55c5d50c0260 2024-09-11T23:38:03.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.503+0000 7f16a208f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c5d4fc2e50 con 0x55c5d50c5bc0 2024-09-11T23:38:03.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.503+0000 7f16a208f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c5d50175e0 con 0x55c5d50c52a0 2024-09-11T23:38:03.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.503+0000 7f169e253640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c52a0 0x55c5d50c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.503+0000 7f169e253640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c52a0 0x55c5d50c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:36464/0 (socket says 172.21.15.133:36464) 2024-09-11T23:38:03.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.503+0000 7f169e253640 1 -- 172.21.15.133:0/2925088995 learned_addr learned my addr 172.21.15.133:0/2925088995 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.503+0000 7f168effd640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16840a2fa0 0x7f16840a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.503+0000 7f169da52640 1 --2- 172.21.15.133:0/2925088995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c5bc0 0x55c5d50bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.504+0000 7f168effd640 1 -- 172.21.15.133:0/2750017846 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f16880cea30 con 0x7f16981036e0 2024-09-11T23:38:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.504+0000 7f169e253640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16840a2fa0 0x7f16840a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.504+0000 7f169e253640 1 -- 172.21.15.133:0/2925088995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c5bc0 msgr2=0x55c5d50bfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.504+0000 7f169e253640 1 --2- 172.21.15.133:0/2925088995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c5bc0 0x55c5d50bfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.504+0000 7f169e253640 1 -- 172.21.15.133:0/2925088995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c0260 msgr2=0x55c5d50c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.504+0000 7f169d251640 1 --2- 172.21.15.133:0/2925088995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c0260 0x55c5d50c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.504+0000 7f169e253640 1 --2- 172.21.15.133:0/2925088995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c0260 0x55c5d50c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.504+0000 7f169e253640 1 -- 172.21.15.133:0/2925088995 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5d4fb3d40 con 0x55c5d50c52a0 2024-09-11T23:38:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.504+0000 7f169d251640 1 --2- 172.21.15.133:0/2925088995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c0260 0x55c5d50c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:38:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.504+0000 7f169e253640 1 --2- 172.21.15.133:0/2925088995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c52a0 0x55c5d50c5680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1690035000 tx=0x7f1690031700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e8a6a3c2882aa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.505+0000 7f169e253640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16840a2fa0 0x7f16840a5460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f169004a000 tx=0x7f1690046000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.505+0000 7f1683fff640 1 -- 172.21.15.133:0/2925088995 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169004d070 con 0x55c5d50c52a0 2024-09-11T23:38:03.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.505+0000 7f1683fff640 1 -- 172.21.15.133:0/2925088995 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1690036040 con 0x55c5d50c52a0 2024-09-11T23:38:03.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.505+0000 7f1683fff640 1 -- 172.21.15.133:0/2925088995 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690048410 con 0x55c5d50c52a0 2024-09-11T23:38:03.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.506+0000 7f16a208f740 1 -- 172.21.15.133:0/2925088995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c52a0 msgr2=0x55c5d50c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.506+0000 7f16a208f740 1 --2- 172.21.15.133:0/2925088995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c52a0 0x55c5d50c5680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1690035000 tx=0x7f1690031700 comp rx=0 tx=0).stop 2024-09-11T23:38:03.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.506+0000 7f16a208f740 1 -- 172.21.15.133:0/2925088995 shutdown_connections 2024-09-11T23:38:03.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.506+0000 7f16a208f740 1 --2- 172.21.15.133:0/2925088995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c0260 0x55c5d50c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.506+0000 7f16a208f740 1 --2- 172.21.15.133:0/2925088995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c5bc0 0x55c5d50bfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.506+0000 7f16a208f740 1 --2- 172.21.15.133:0/2925088995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c52a0 0x55c5d50c5680 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.506+0000 7f16a208f740 1 -- 172.21.15.133:0/2925088995 >> 172.21.15.133:0/2925088995 conn(0x55c5d50b20a0 msgr2=0x55c5d50b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:03.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.506+0000 7f16a208f740 1 -- 172.21.15.133:0/2925088995 shutdown_connections 2024-09-11T23:38:03.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.506+0000 7f16a208f740 1 -- 172.21.15.133:0/2925088995 wait complete. 2024-09-11T23:38:03.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.507+0000 7f16a208f740 1 -- start start 2024-09-11T23:38:03.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.507+0000 7f16a208f740 1 Processor -- start 2024-09-11T23:38:03.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.507+0000 7f16a208f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c0260 0x55c5d50c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.507+0000 7f16a208f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f16a208f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c5bc0 0x55c5d50c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f16a208f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c5d50cc5e0 con 0x55c5d50c52a0 2024-09-11T23:38:03.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f16a208f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c5d4fa3f10 con 0x55c5d50c5bc0 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f16a208f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c5d50cc760 con 0x55c5d50c0260 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f16a208f740 10 client.0 _ll_register_callbacks cb 0x55c5d4fc7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f169da52640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c0260 0x55c5d50c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f169d251640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f169da52640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c0260 0x55c5d50c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:36476/0 (socket says 172.21.15.133:36476) 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f169da52640 1 -- 172.21.15.133:0/1790313350 learned_addr learned my addr 172.21.15.133:0/1790313350 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f169e253640 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c5bc0 0x55c5d50c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f169d251640 1 -- 172.21.15.133:0/1790313350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c0260 msgr2=0x55c5d50c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f169d251640 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c0260 0x55c5d50c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f169d251640 1 -- 172.21.15.133:0/1790313350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c5bc0 msgr2=0x55c5d50c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f169d251640 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c5bc0 0x55c5d50c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.508+0000 7f169d251640 1 -- 172.21.15.133:0/1790313350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5d4d2a580 con 0x55c5d50c52a0 2024-09-11T23:38:03.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.509+0000 7f169d251640 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50c9630 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f16880cbfb0 tx=0x7f1688009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fc515416914fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:03.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.509+0000 7f16817fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:03.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.509+0000 7f16817fa640 1 -- 172.21.15.133:0/1790313350 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688021070 con 0x55c5d50c52a0 2024-09-11T23:38:03.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.509+0000 7f16817fa640 1 -- 172.21.15.133:0/1790313350 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f168809a3d0 con 0x55c5d50c52a0 2024-09-11T23:38:03.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.509+0000 7f16a208f740 1 -- 172.21.15.133:0/1790313350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1690036040 con 0x55c5d50c52a0 2024-09-11T23:38:03.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.509+0000 7f16817fa640 1 -- 172.21.15.133:0/1790313350 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16880a39f0 con 0x55c5d50c52a0 2024-09-11T23:38:03.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.509+0000 7f16a208f740 10 client.15930 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:03.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.509+0000 7f16a208f740 1 -- 172.21.15.133:0/1790313350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5d4df4960 con 0x55c5d50c52a0 2024-09-11T23:38:03.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.510+0000 7f16817fa640 1 -- 172.21.15.133:0/1790313350 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1690036040 con 0x55c5d50c52a0 2024-09-11T23:38:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.510+0000 7f16817fa640 1 -- 172.21.15.133:0/1790313350 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1688099cd0 con 0x55c5d50c52a0 2024-09-11T23:38:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.510+0000 7f16a208f740 10 client.15930 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.510+0000 7f16a208f740 10 client.15930 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:38:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.510+0000 7f16a208f740 20 client.15930 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.510+0000 7f16a208f740 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c5d50d9520 0x55c5d50db930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.510+0000 7f16a208f740 4 client.15930 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.510+0000 7f16a208f740 1 -- 172.21.15.133:0/1790313350 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5d5028b90 con 0x55c5d50d9520 2024-09-11T23:38:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.511+0000 7f169da52640 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c5d50d9520 0x55c5d50db930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.511+0000 7f16817fa640 10 client.15930 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.511+0000 7f169da52640 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c5d50d9520 0x55c5d50db930 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9280baf8390fa58 server_cookie=2eddfbbb23fc286d in_seq=0 out_seq=0 2024-09-11T23:38:03.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16817fa640 1 -- 172.21.15.133:0/1790313350 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5d5028b90 con 0x55c5d50d9520 2024-09-11T23:38:03.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16817fa640 10 client.15930 handle_command_reply: tid=0 2024-09-11T23:38:03.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16a208f740 1 client.15930 shutdown 2024-09-11T23:38:03.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16a208f740 1 -- 172.21.15.133:0/1790313350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 msgr2=0x55c5d50c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16a208f740 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50c9630 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f16880cbfb0 tx=0x7f1688009cf0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16a208f740 1 -- 172.21.15.133:0/1790313350 shutdown_connections 2024-09-11T23:38:03.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16a208f740 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c5d50d9520 0x55c5d50db930 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16a208f740 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c5bc0 0x55c5d50c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16817fa640 0 client.15930 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:03.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16a208f740 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50c9630 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16a208f740 1 --2- 172.21.15.133:0/1790313350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c0260 0x55c5d50c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.512+0000 7f16a208f740 1 -- 172.21.15.133:0/1790313350 >> 172.21.15.133:0/1790313350 conn(0x55c5d50b20a0 msgr2=0x55c5d50b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:03.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.513+0000 7f16a208f740 1 -- 172.21.15.133:0/1790313350 shutdown_connections 2024-09-11T23:38:03.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.513+0000 7f16a208f740 1 -- 172.21.15.133:0/1790313350 wait complete. 2024-09-11T23:38:03.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.513+0000 7f16a208f740 20 client.15930 trim_cache size 0 max 16384 2024-09-11T23:38:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.530+0000 7f16a208f740 1 Processor -- start 2024-09-11T23:38:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.530+0000 7f16a208f740 1 -- start start 2024-09-11T23:38:03.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.530+0000 7f16a208f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c0260 0x55c5d50bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.530+0000 7f16a208f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.530+0000 7f16a208f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c5bc0 0x55c5d50e1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.530+0000 7f16a208f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f16981ab610 con 0x55c5d50c52a0 2024-09-11T23:38:03.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.530+0000 7f16a208f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16981ad150 con 0x55c5d50c0260 2024-09-11T23:38:03.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.530+0000 7f16a208f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16981abce0 con 0x55c5d50c5bc0 2024-09-11T23:38:03.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.531+0000 7f169d251640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.531+0000 7f169da52640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c0260 0x55c5d50bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.531+0000 7f169d251640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33068/0 (socket says 172.21.15.133:33068) 2024-09-11T23:38:03.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.531+0000 7f169d251640 1 -- 172.21.15.133:0/2315727795 learned_addr learned my addr 172.21.15.133:0/2315727795 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:03.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.531+0000 7f169e253640 1 --2- 172.21.15.133:0/2315727795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c5bc0 0x55c5d50e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.531+0000 7f169d251640 1 -- 172.21.15.133:0/2315727795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c5bc0 msgr2=0x55c5d50e1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:03.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.531+0000 7f169d251640 1 --2- 172.21.15.133:0/2315727795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c5bc0 0x55c5d50e1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.531+0000 7f169d251640 1 -- 172.21.15.133:0/2315727795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c0260 msgr2=0x55c5d50bfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:03.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.531+0000 7f169d251640 1 --2- 172.21.15.133:0/2315727795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c0260 0x55c5d50bfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.531+0000 7f169d251640 1 -- 172.21.15.133:0/2315727795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5d50ac820 con 0x55c5d50c52a0 2024-09-11T23:38:03.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.531+0000 7f169d251640 1 --2- 172.21.15.133:0/2315727795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50c80b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1688010cf0 tx=0x7f1688009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb034c17dee7a7ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:03.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.532+0000 7f16837fe640 1 -- 172.21.15.133:0/2315727795 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688021050 con 0x55c5d50c52a0 2024-09-11T23:38:03.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.532+0000 7f16837fe640 1 -- 172.21.15.133:0/2315727795 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1688015070 con 0x55c5d50c52a0 2024-09-11T23:38:03.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.532+0000 7f16837fe640 1 -- 172.21.15.133:0/2315727795 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168809a430 con 0x55c5d50c52a0 2024-09-11T23:38:03.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.532+0000 7f16a208f740 1 -- 172.21.15.133:0/2315727795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 msgr2=0x55c5d50c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.532+0000 7f16a208f740 1 --2- 172.21.15.133:0/2315727795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50c80b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1688010cf0 tx=0x7f1688009d60 comp rx=0 tx=0).stop 2024-09-11T23:38:03.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.533+0000 7f16a208f740 1 -- 172.21.15.133:0/2315727795 shutdown_connections 2024-09-11T23:38:03.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.533+0000 7f16a208f740 1 --2- 172.21.15.133:0/2315727795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c5bc0 0x55c5d50e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.533+0000 7f16a208f740 1 --2- 172.21.15.133:0/2315727795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50c80b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.533+0000 7f16a208f740 1 --2- 172.21.15.133:0/2315727795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c0260 0x55c5d50bfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.533+0000 7f16a208f740 1 -- 172.21.15.133:0/2315727795 >> 172.21.15.133:0/2315727795 conn(0x55c5d50b20a0 msgr2=0x55c5d50cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.533+0000 7f16a208f740 1 -- 172.21.15.133:0/2315727795 shutdown_connections 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.533+0000 7f16a208f740 1 -- 172.21.15.133:0/2315727795 wait complete. 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.533+0000 7f16a208f740 1 -- start start 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.533+0000 7f16a208f740 1 Processor -- start 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.534+0000 7f16a208f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c0260 0x55c5d50d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.534+0000 7f16a208f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.534+0000 7f16a208f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c5bc0 0x55c5d50ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.534+0000 7f169da52640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c0260 0x55c5d50d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.534+0000 7f169da52640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c0260 0x55c5d50d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47804/0 (socket says 172.21.15.133:47804) 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.534+0000 7f169da52640 1 -- 172.21.15.133:0/2840882550 learned_addr learned my addr 172.21.15.133:0/2840882550 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:03.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.534+0000 7f16a208f740 1 -- 172.21.15.133:0/2840882550 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f16981aca80 con 0x55c5d50c52a0 2024-09-11T23:38:03.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.534+0000 7f169d251640 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.534+0000 7f169e253640 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c5bc0 0x55c5d50ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.534+0000 7f16a208f740 1 -- 172.21.15.133:0/2840882550 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16981ac3b0 con 0x55c5d50c0260 2024-09-11T23:38:03.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.534+0000 7f16a208f740 1 -- 172.21.15.133:0/2840882550 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16981a9630 con 0x55c5d50c5bc0 2024-09-11T23:38:03.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.535+0000 7f16a208f740 10 client.0 _ll_register_callbacks cb 0x7f168809a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:03.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.535+0000 7f169d251640 1 -- 172.21.15.133:0/2840882550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c5bc0 msgr2=0x55c5d50ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:03.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.535+0000 7f169d251640 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c5bc0 0x55c5d50ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.535+0000 7f169d251640 1 -- 172.21.15.133:0/2840882550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c0260 msgr2=0x55c5d50d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:03.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.535+0000 7f169d251640 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c0260 0x55c5d50d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.535+0000 7f169d251640 1 -- 172.21.15.133:0/2840882550 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5d50cf1f0 con 0x55c5d50c52a0 2024-09-11T23:38:03.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.535+0000 7f169d251640 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50d2110 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55c5d50c5b60 tx=0x7f1688009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a85c0707c258ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:03.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.535+0000 7f16817fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:03.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.536+0000 7f16817fa640 1 -- 172.21.15.133:0/2840882550 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688021050 con 0x55c5d50c52a0 2024-09-11T23:38:03.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.536+0000 7f16a208f740 1 -- 172.21.15.133:0/2840882550 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1688015070 con 0x55c5d50c52a0 2024-09-11T23:38:03.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.536+0000 7f16817fa640 1 -- 172.21.15.133:0/2840882550 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f1688096cd0 con 0x55c5d50c52a0 2024-09-11T23:38:03.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.536+0000 7f16817fa640 1 -- 172.21.15.133:0/2840882550 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1688097020 con 0x55c5d50c52a0 2024-09-11T23:38:03.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.536+0000 7f16a208f740 10 client.15936 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:03.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.536+0000 7f16a208f740 1 -- 172.21.15.133:0/2840882550 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5d50cf1f0 con 0x55c5d50c52a0 2024-09-11T23:38:03.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.537+0000 7f16817fa640 1 -- 172.21.15.133:0/2840882550 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1688015070 con 0x55c5d50c52a0 2024-09-11T23:38:03.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.537+0000 7f16817fa640 1 -- 172.21.15.133:0/2840882550 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1688010730 con 0x55c5d50c52a0 2024-09-11T23:38:03.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.537+0000 7f16a208f740 10 client.15936 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:03.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.537+0000 7f16a208f740 10 client.15936 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:38:03.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.537+0000 7f16a208f740 20 client.15936 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:03.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.537+0000 7f16a208f740 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c5d50d9520 0x55c5d5146f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:03.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.537+0000 7f16a208f740 4 client.15936 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "942f90c7", "release": true, "await": true}] 2024-09-11T23:38:03.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.537+0000 7f16a208f740 1 -- 172.21.15.133:0/2840882550 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "set_id": "942f90c7", "release": true, "await": true}) -- 0x55c5d50230a0 con 0x55c5d50d9520 2024-09-11T23:38:03.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.537+0000 7f169da52640 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c5d50d9520 0x55c5d5146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:03.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.538+0000 7f169da52640 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c5d50d9520 0x55c5d5146f60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa88aa5f3ae76c3 server_cookie=b61a6fcb2adc3b4f in_seq=0 out_seq=0 2024-09-11T23:38:03.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.538+0000 7f16817fa640 10 client.15936 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.539+0000 7f16817fa640 1 -- 172.21.15.133:0/2840882550 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55c5d50230a0 con 0x55c5d50d9520 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.539+0000 7f16817fa640 10 client.15936 handle_command_reply: tid=0 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.539+0000 7f16a208f740 1 client.15936 shutdown 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.539+0000 7f16a208f740 1 -- 172.21.15.133:0/2840882550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 msgr2=0x55c5d50d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.539+0000 7f16a208f740 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50d2110 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55c5d50c5b60 tx=0x7f1688009cf0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.540+0000 7f16a208f740 1 -- 172.21.15.133:0/2840882550 shutdown_connections 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.540+0000 7f16a208f740 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c5d50d9520 0x55c5d5146f60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.540+0000 7f16817fa640 0 client.15936 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.540+0000 7f16a208f740 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5d50c5bc0 0x55c5d50ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.540+0000 7f16a208f740 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5d50c52a0 0x55c5d50d2110 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.540+0000 7f16a208f740 1 --2- 172.21.15.133:0/2840882550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d50c0260 0x55c5d50d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.540+0000 7f16a208f740 1 -- 172.21.15.133:0/2840882550 >> 172.21.15.133:0/2840882550 conn(0x55c5d50b20a0 msgr2=0x55c5d50b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:03.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.540+0000 7f16a208f740 1 -- 172.21.15.133:0/2840882550 shutdown_connections 2024-09-11T23:38:03.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.540+0000 7f16a208f740 1 -- 172.21.15.133:0/2840882550 wait complete. 2024-09-11T23:38:03.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.540+0000 7f16a208f740 20 client.15936 trim_cache size 0 max 16384 2024-09-11T23:38:03.543 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:38:03.543 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 28, 2024-09-11T23:38:03.543 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:38:03.543 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 4, 2024-09-11T23:38:03.543 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:38:03.543 INFO:teuthology.orchestra.run.smithi133.stdout: "942f90c7": { 2024-09-11T23:38:03.543 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 4, 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 144.5, 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:38:03.544 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:38:03.545 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:38:03.545 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:03.545 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:03.545 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:03.545 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:03.545 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:03.545 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:38:03.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.541+0000 7f169fcb6640 1 -- 172.21.15.133:0/2750017846 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16840a2fa0 msgr2=0x7f16840a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.541+0000 7f169fcb6640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16840a2fa0 0x7f16840a5460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f169004a000 tx=0x7f1690046000 comp rx=0 tx=0).stop 2024-09-11T23:38:03.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.541+0000 7f169fcb6640 1 -- 172.21.15.133:0/2750017846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16981036e0 msgr2=0x7f1698198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:03.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.541+0000 7f169fcb6640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16981036e0 0x7f1698198840 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f168800d740 tx=0x7f168800dc00 comp rx=0 tx=0).stop 2024-09-11T23:38:03.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.541+0000 7f169fcb6640 1 -- 172.21.15.133:0/2750017846 shutdown_connections 2024-09-11T23:38:03.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.541+0000 7f169fcb6640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16840a2fa0 0x7f16840a5460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.542+0000 7f169fcb6640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16981082a0 0x7f169819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.542+0000 7f169fcb6640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16981036e0 0x7f1698198840 secure :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f168800d740 tx=0x7f168800dc00 comp rx=0 tx=0).stop 2024-09-11T23:38:03.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.542+0000 7f169fcb6640 1 --2- 172.21.15.133:0/2750017846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1698102d30 0x7f1698198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:03.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.542+0000 7f169fcb6640 1 -- 172.21.15.133:0/2750017846 >> 172.21.15.133:0/2750017846 conn(0x7f16980fc7d0 msgr2=0x7f16980fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:03.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.542+0000 7f169fcb6640 1 -- 172.21.15.133:0/2750017846 shutdown_connections 2024-09-11T23:38:03.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:03.542+0000 7f169fcb6640 1 -- 172.21.15.133:0/2750017846 wait complete. 2024-09-11T23:38:03.904 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:38:03.904 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:38:04.254 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '942f90c7', total seconds elapsed: 29.3 2024-09-11T23:38:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:04.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.782+0000 7f580eba4640 1 -- 172.21.15.133:0/1198460189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5808101420 msgr2=0x7f580810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.782+0000 7f580eba4640 1 --2- 172.21.15.133:0/1198460189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5808101420 0x7f580810d950 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5800009cd0 tx=0x7f580002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.782+0000 7f580eba4640 1 -- 172.21.15.133:0/1198460189 shutdown_connections 2024-09-11T23:38:04.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.782+0000 7f580eba4640 1 --2- 172.21.15.133:0/1198460189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5808101420 0x7f580810d950 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.782+0000 7f580eba4640 1 --2- 172.21.15.133:0/1198460189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5808100a80 0x7f5808100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.782+0000 7f580eba4640 1 --2- 172.21.15.133:0/1198460189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5808106a80 0x7f5808106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.782+0000 7f580eba4640 1 -- 172.21.15.133:0/1198460189 >> 172.21.15.133:0/1198460189 conn(0x7f58080fc7a0 msgr2=0x7f58080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.783+0000 7f580eba4640 1 -- 172.21.15.133:0/1198460189 shutdown_connections 2024-09-11T23:38:04.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.783+0000 7f580eba4640 1 -- 172.21.15.133:0/1198460189 wait complete. 2024-09-11T23:38:04.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.784+0000 7f580eba4640 1 Processor -- start 2024-09-11T23:38:04.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.784+0000 7f580eba4640 1 -- start start 2024-09-11T23:38:04.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.784+0000 7f580eba4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5808100a80 0x7f5808197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.784+0000 7f580eba4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5808101420 0x7f5808197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.784+0000 7f580eba4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5808106a80 0x7f580819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.784+0000 7f580eba4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5808110e80 con 0x7f5808100a80 2024-09-11T23:38:04.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.784+0000 7f580eba4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5808110d00 con 0x7f5808101420 2024-09-11T23:38:04.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.784+0000 7f580eba4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5808111000 con 0x7f5808106a80 2024-09-11T23:38:04.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.784+0000 7f580c940640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5808100a80 0x7f5808197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f5807fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5808101420 0x7f5808197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f580c940640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5808100a80 0x7f5808197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33086/0 (socket says 172.21.15.133:33086) 2024-09-11T23:38:04.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f580c940640 1 -- 172.21.15.133:0/3679999889 learned_addr learned my addr 172.21.15.133:0/3679999889 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:04.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f580c940640 1 -- 172.21.15.133:0/3679999889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5808106a80 msgr2=0x7f580819bbf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:38:04.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f580c940640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5808106a80 0x7f580819bbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f580c940640 1 -- 172.21.15.133:0/3679999889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5808101420 msgr2=0x7f5808197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f580c940640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5808101420 0x7f5808197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f580c940640 1 -- 172.21.15.133:0/3679999889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580819c2f0 con 0x7f5808100a80 2024-09-11T23:38:04.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f580c940640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5808100a80 0x7f5808197320 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00d950 tx=0x7f57fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f5805ffb640 1 -- 172.21.15.133:0/3679999889 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc017030 con 0x7f5808100a80 2024-09-11T23:38:04.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f5805ffb640 1 -- 172.21.15.133:0/3679999889 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57fc004020 con 0x7f5808100a80 2024-09-11T23:38:04.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f5805ffb640 1 -- 172.21.15.133:0/3679999889 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0047a0 con 0x7f5808100a80 2024-09-11T23:38:04.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.785+0000 7f580eba4640 1 -- 172.21.15.133:0/3679999889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580819c5e0 con 0x7f5808100a80 2024-09-11T23:38:04.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.786+0000 7f580eba4640 1 -- 172.21.15.133:0/3679999889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580806a790 con 0x7f5808100a80 2024-09-11T23:38:04.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.788+0000 7f5810f7d740 1 Processor -- start 2024-09-11T23:38:04.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f5810f7d740 1 -- start start 2024-09-11T23:38:04.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f5810f7d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175306b60 0x55a175306f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f5810f7d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175307480 0x55a175311800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f5810f7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175311d40 0x55a175314130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f5810f7d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a17526c280 con 0x55a175307480 2024-09-11T23:38:04.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f5810f7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a175213b10 con 0x55a175311d40 2024-09-11T23:38:04.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f5810f7d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a175268240 con 0x55a175306b60 2024-09-11T23:38:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f5805ffb640 1 -- 172.21.15.133:0/3679999889 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f57fc0041d0 con 0x7f5808100a80 2024-09-11T23:38:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f580d141640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175306b60 0x55a175306f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f580d141640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175306b60 0x55a175306f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:36512/0 (socket says 172.21.15.133:36512) 2024-09-11T23:38:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f580d141640 1 -- 172.21.15.133:0/3473936608 learned_addr learned my addr 172.21.15.133:0/3473936608 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.789+0000 7f5807fff640 1 --2- 172.21.15.133:0/3473936608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175311d40 0x55a175314130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.790+0000 7f580c940640 1 --2- 172.21.15.133:0/3473936608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175307480 0x55a175311800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.790+0000 7f580d141640 1 -- 172.21.15.133:0/3473936608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175311d40 msgr2=0x55a175314130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.790+0000 7f580d141640 1 --2- 172.21.15.133:0/3473936608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175311d40 0x55a175314130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.790+0000 7f580d141640 1 -- 172.21.15.133:0/3473936608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175307480 msgr2=0x55a175311800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.790+0000 7f580d141640 1 --2- 172.21.15.133:0/3473936608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175307480 0x55a175311800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.790+0000 7f580d141640 1 -- 172.21.15.133:0/3473936608 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1752201f0 con 0x55a175306b60 2024-09-11T23:38:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.790+0000 7f580c940640 1 --2- 172.21.15.133:0/3473936608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175307480 0x55a175311800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:38:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.790+0000 7f580d141640 1 --2- 172.21.15.133:0/3473936608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175306b60 0x55a175306f40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5800005cc0 tx=0x7f5800009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6201f59b554244e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.790+0000 7f57f2ffd640 1 -- 172.21.15.133:0/3473936608 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800031630 con 0x55a175306b60 2024-09-11T23:38:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.791+0000 7f57f2ffd640 1 -- 172.21.15.133:0/3473936608 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5800036040 con 0x55a175306b60 2024-09-11T23:38:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.791+0000 7f5805ffb640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57f80a2f20 0x7f57f80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.791+0000 7f57f2ffd640 1 -- 172.21.15.133:0/3473936608 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000042a0 con 0x55a175306b60 2024-09-11T23:38:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.791+0000 7f5805ffb640 1 -- 172.21.15.133:0/3679999889 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f57fc0cefd0 con 0x7f5808100a80 2024-09-11T23:38:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.791+0000 7f580c940640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57f80a2f20 0x7f57f80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.791+0000 7f5810f7d740 1 -- 172.21.15.133:0/3473936608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175306b60 msgr2=0x55a175306f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.791+0000 7f5810f7d740 1 --2- 172.21.15.133:0/3473936608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175306b60 0x55a175306f40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5800005cc0 tx=0x7f5800009a20 comp rx=0 tx=0).stop 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.792+0000 7f5810f7d740 1 -- 172.21.15.133:0/3473936608 shutdown_connections 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.792+0000 7f5810f7d740 1 --2- 172.21.15.133:0/3473936608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175311d40 0x55a175314130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.792+0000 7f5810f7d740 1 --2- 172.21.15.133:0/3473936608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175307480 0x55a175311800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.792+0000 7f5810f7d740 1 --2- 172.21.15.133:0/3473936608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175306b60 0x55a175306f40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.792+0000 7f5810f7d740 1 -- 172.21.15.133:0/3473936608 >> 172.21.15.133:0/3473936608 conn(0x55a1753030f0 msgr2=0x55a1753034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.792+0000 7f5810f7d740 1 -- 172.21.15.133:0/3473936608 shutdown_connections 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.792+0000 7f580c940640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57f80a2f20 0x7f57f80a53e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0cc5b0 tx=0x7f57fc00df40 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.792+0000 7f5810f7d740 1 -- 172.21.15.133:0/3473936608 wait complete. 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.792+0000 7f5810f7d740 1 -- start start 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.792+0000 7f5810f7d740 1 Processor -- start 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.793+0000 7f5810f7d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a17530ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.793+0000 7f5810f7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a17530b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.793+0000 7f5810f7d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 0x55a17530b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.793+0000 7f5810f7d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a1753148a0 con 0x55a175306b60 2024-09-11T23:38:04.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.793+0000 7f5810f7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a175226f10 con 0x55a175307480 2024-09-11T23:38:04.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.793+0000 7f5810f7d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a175314a20 con 0x55a175311d40 2024-09-11T23:38:04.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.793+0000 7f5810f7d740 10 client.0 _ll_register_callbacks cb 0x55a1751efa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:04.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.793+0000 7f580d141640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a17530ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.793+0000 7f580d141640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a17530ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33100/0 (socket says 172.21.15.133:33100) 2024-09-11T23:38:04.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.793+0000 7f580d141640 1 -- 172.21.15.133:0/1993118810 learned_addr learned my addr 172.21.15.133:0/1993118810 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:04.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.793+0000 7f580d141640 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 0x55a17530b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.794+0000 7f5807fff640 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a17530b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.794+0000 7f580d141640 1 -- 172.21.15.133:0/1993118810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 msgr2=0x55a17530b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.794+0000 7f580d141640 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 0x55a17530b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.794+0000 7f580d141640 1 -- 172.21.15.133:0/1993118810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 msgr2=0x55a17530b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.794+0000 7f580d141640 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a17530b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.794+0000 7f580d141640 1 -- 172.21.15.133:0/1993118810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a174fdd6a0 con 0x55a175306b60 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.794+0000 7f580d141640 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a17530ae90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5800006010 tx=0x7f5800003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9888c942168aa36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.794+0000 7f57e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.794+0000 7f57e7fff640 1 -- 172.21.15.133:0/1993118810 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580004e040 con 0x55a175306b60 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.795+0000 7f5810f7d740 1 -- 172.21.15.133:0/1993118810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5800036040 con 0x55a175306b60 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.795+0000 7f57e7fff640 1 -- 172.21.15.133:0/1993118810 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f5800004000 con 0x55a175306b60 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.795+0000 7f57e7fff640 1 -- 172.21.15.133:0/1993118810 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5800004200 con 0x55a175306b60 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.795+0000 7f5810f7d740 10 client.15945 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.795+0000 7f5810f7d740 1 -- 172.21.15.133:0/1993118810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a175328d20 con 0x55a175306b60 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.795+0000 7f57e7fff640 1 -- 172.21.15.133:0/1993118810 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5800036040 con 0x55a175306b60 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.795+0000 7f57e7fff640 1 -- 172.21.15.133:0/1993118810 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5800030070 con 0x55a175306b60 2024-09-11T23:38:04.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.796+0000 7f5810f7d740 10 client.15945 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.796+0000 7f5810f7d740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.796+0000 7f5810f7d740 20 client.15945 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.796+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a17532a070 0x55a17532c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.796+0000 7f5810f7d740 4 client.15945 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.796+0000 7f5810f7d740 1 -- 172.21.15.133:0/1993118810 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1752797f0 con 0x55a17532a070 2024-09-11T23:38:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.796+0000 7f5807fff640 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a17532a070 0x55a17532c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.796+0000 7f57e7fff640 10 client.15945 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.796+0000 7f5807fff640 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a17532a070 0x55a17532c460 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c206373f44eaef server_cookie=5166276ecdba245 in_seq=0 out_seq=0 2024-09-11T23:38:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.797+0000 7f57e7fff640 1 -- 172.21.15.133:0/1993118810 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1752797f0 con 0x55a17532a070 2024-09-11T23:38:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.797+0000 7f57e7fff640 10 client.15945 handle_command_reply: tid=0 2024-09-11T23:38:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.797+0000 7f5810f7d740 1 client.15945 shutdown 2024-09-11T23:38:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.797+0000 7f5810f7d740 1 -- 172.21.15.133:0/1993118810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 msgr2=0x55a17530ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.797+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a17530ae90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5800006010 tx=0x7f5800003e50 comp rx=0 tx=0).stop 2024-09-11T23:38:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.798+0000 7f5810f7d740 1 -- 172.21.15.133:0/1993118810 shutdown_connections 2024-09-11T23:38:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.798+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a17532a070 0x55a17532c460 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.798+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 0x55a17530b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.798+0000 7f57e7fff640 0 client.15945 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.798+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a17530b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.798+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1993118810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a17530ae90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.798+0000 7f5810f7d740 1 -- 172.21.15.133:0/1993118810 >> 172.21.15.133:0/1993118810 conn(0x55a1753030f0 msgr2=0x55a175302270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.798+0000 7f5810f7d740 1 -- 172.21.15.133:0/1993118810 shutdown_connections 2024-09-11T23:38:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.798+0000 7f5810f7d740 1 -- 172.21.15.133:0/1993118810 wait complete. 2024-09-11T23:38:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.798+0000 7f5810f7d740 20 client.15945 trim_cache size 0 max 16384 2024-09-11T23:38:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.832+0000 7f5810f7d740 1 Processor -- start 2024-09-11T23:38:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.832+0000 7f5810f7d740 1 -- start start 2024-09-11T23:38:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.832+0000 7f5810f7d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175308cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.833+0000 7f580d141640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175308cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.833+0000 7f580d141640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175308cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33112/0 (socket says 172.21.15.133:33112) 2024-09-11T23:38:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.833+0000 7f5810f7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a175331a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.833+0000 7f580d141640 1 -- 172.21.15.133:0/2773201693 learned_addr learned my addr 172.21.15.133:0/2773201693 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.833+0000 7f5810f7d740 1 --2- 172.21.15.133:0/2773201693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 0x55a175331fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.833+0000 7f5810f7d740 1 -- 172.21.15.133:0/2773201693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f58081ab5b0 con 0x55a175306b60 2024-09-11T23:38:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.833+0000 7f5810f7d740 1 -- 172.21.15.133:0/2773201693 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58081ad0f0 con 0x55a175307480 2024-09-11T23:38:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.833+0000 7f5807fff640 1 --2- 172.21.15.133:0/2773201693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a175331a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.833+0000 7f5810f7d740 1 -- 172.21.15.133:0/2773201693 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58081abc80 con 0x55a175311d40 2024-09-11T23:38:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.834+0000 7f5807fff640 1 -- 172.21.15.133:0/2773201693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 msgr2=0x55a175331fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.834+0000 7f5807fff640 1 --2- 172.21.15.133:0/2773201693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 0x55a175331fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.834+0000 7f5807fff640 1 -- 172.21.15.133:0/2773201693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 msgr2=0x55a175308cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.834+0000 7f5807fff640 1 --2- 172.21.15.133:0/2773201693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175308cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.834+0000 7f5807fff640 1 -- 172.21.15.133:0/2773201693 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1752fe830 con 0x55a175307480 2024-09-11T23:38:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.834+0000 7f580d141640 1 --2- 172.21.15.133:0/2773201693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175308cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.834+0000 7f5807fff640 1 --2- 172.21.15.133:0/2773201693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a175331a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55a17530a720 tx=0x7f57f400beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=789300fe1401c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.835 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:04 smithi133 ceph-mon[31430]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T23:38:04.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.839+0000 7f57f27fc640 1 -- 172.21.15.133:0/2773201693 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4011d50 con 0x55a175307480 2024-09-11T23:38:04.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.839+0000 7f57f27fc640 1 -- 172.21.15.133:0/2773201693 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f4005500 con 0x55a175307480 2024-09-11T23:38:04.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.839+0000 7f57f27fc640 1 -- 172.21.15.133:0/2773201693 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4003850 con 0x55a175307480 2024-09-11T23:38:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.840+0000 7f5810f7d740 1 -- 172.21.15.133:0/2773201693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 msgr2=0x55a175331a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.840+0000 7f5810f7d740 1 --2- 172.21.15.133:0/2773201693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a175331a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55a17530a720 tx=0x7f57f400beb0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.841+0000 7f5810f7d740 1 -- 172.21.15.133:0/2773201693 shutdown_connections 2024-09-11T23:38:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.841+0000 7f5810f7d740 1 --2- 172.21.15.133:0/2773201693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 0x55a175331fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.841+0000 7f5810f7d740 1 --2- 172.21.15.133:0/2773201693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a175331a80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.841+0000 7f5810f7d740 1 --2- 172.21.15.133:0/2773201693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175308cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.841+0000 7f5810f7d740 1 -- 172.21.15.133:0/2773201693 >> 172.21.15.133:0/2773201693 conn(0x55a1753030f0 msgr2=0x55a1753085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.841+0000 7f5810f7d740 1 -- 172.21.15.133:0/2773201693 shutdown_connections 2024-09-11T23:38:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.841+0000 7f5810f7d740 1 -- 172.21.15.133:0/2773201693 wait complete. 2024-09-11T23:38:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.842+0000 7f5810f7d740 1 -- start start 2024-09-11T23:38:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.842+0000 7f5810f7d740 1 Processor -- start 2024-09-11T23:38:04.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.842+0000 7f5810f7d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175326330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.843+0000 7f580d141640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175326330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.843+0000 7f580d141640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175326330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33118/0 (socket says 172.21.15.133:33118) 2024-09-11T23:38:04.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.843+0000 7f580d141640 1 -- 172.21.15.133:0/1039941853 learned_addr learned my addr 172.21.15.133:0/1039941853 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:04.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.843+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a175326870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.843+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 0x55a175323300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.843+0000 7f5810f7d740 1 -- 172.21.15.133:0/1039941853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f58081aca20 con 0x55a175306b60 2024-09-11T23:38:04.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.843+0000 7f5810f7d740 1 -- 172.21.15.133:0/1039941853 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58081ac350 con 0x55a175307480 2024-09-11T23:38:04.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.843+0000 7f5807fff640 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a175326870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.843+0000 7f5810f7d740 1 -- 172.21.15.133:0/1039941853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58081a9590 con 0x55a175311d40 2024-09-11T23:38:04.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.844+0000 7f580d141640 1 -- 172.21.15.133:0/1039941853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 msgr2=0x55a175323300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:38:04.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.844+0000 7f580d141640 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 0x55a175323300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.844+0000 7f5810f7d740 10 client.0 _ll_register_callbacks cb 0x7f580003f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:04.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.844+0000 7f580d141640 1 -- 172.21.15.133:0/1039941853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 msgr2=0x55a175326870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.844+0000 7f580d141640 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a175326870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.844+0000 7f580d141640 1 -- 172.21.15.133:0/1039941853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1753072b0 con 0x55a175306b60 2024-09-11T23:38:04.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.844+0000 7f5810f7d740 1 -- 172.21.15.133:0/1039941853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f4005500 con 0x55a175306b60 2024-09-11T23:38:04.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.845+0000 7f580d141640 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175326330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5800030c40 tx=0x7f5800005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817698ce1c7be5e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.845+0000 7f57e7fff640 10 client.15951 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:04.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.845+0000 7f57e7fff640 1 -- 172.21.15.133:0/1039941853 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580004a070 con 0x55a175306b60 2024-09-11T23:38:04.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.845+0000 7f57e7fff640 1 -- 172.21.15.133:0/1039941853 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f5800004000 con 0x55a175306b60 2024-09-11T23:38:04.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.845+0000 7f57e7fff640 1 -- 172.21.15.133:0/1039941853 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5800031b50 con 0x55a175306b60 2024-09-11T23:38:04.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.846+0000 7f5810f7d740 10 client.15951 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:04.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.846+0000 7f5810f7d740 1 -- 172.21.15.133:0/1039941853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55a175394e80 con 0x55a175306b60 2024-09-11T23:38:04.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.846+0000 7f57e7fff640 1 -- 172.21.15.133:0/1039941853 <== mon.0 v2:172.21.15.133:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5800004b40 con 0x55a175306b60 2024-09-11T23:38:04.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.846+0000 7f5810f7d740 10 client.15951 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:04.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.846+0000 7f5810f7d740 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:04.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.846+0000 7f5810f7d740 20 client.15951 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:04.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.846+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a17532a070 0x55a1753972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.846+0000 7f5810f7d740 4 client.15951 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:38:04.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.846+0000 7f5810f7d740 1 -- 172.21.15.133:0/1039941853 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a175273d00 con 0x55a17532a070 2024-09-11T23:38:04.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.847+0000 7f5807fff640 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a17532a070 0x55a1753972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.847+0000 7f57e7fff640 10 client.15951 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:04.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.847+0000 7f5807fff640 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a17532a070 0x55a1753972a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9821964e29a205c9 server_cookie=916657ffe69fd0b0 in_seq=0 out_seq=0 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.848+0000 7f57e7fff640 1 -- 172.21.15.133:0/1039941853 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a175273d00 con 0x55a17532a070 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.848+0000 7f57e7fff640 10 client.15951 handle_command_reply: tid=0 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.848+0000 7f5810f7d740 1 client.15951 shutdown 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.848+0000 7f5810f7d740 1 -- 172.21.15.133:0/1039941853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 msgr2=0x55a175326330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.848+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175326330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5800030c40 tx=0x7f5800005ef0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.849+0000 7f5810f7d740 1 -- 172.21.15.133:0/1039941853 shutdown_connections 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.849+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a17532a070 0x55a1753972a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.849+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a175311d40 0x55a175323300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.849+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a175307480 0x55a175326870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.849+0000 7f5810f7d740 1 --2- 172.21.15.133:0/1039941853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a175306b60 0x55a175326330 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.849+0000 7f5810f7d740 1 -- 172.21.15.133:0/1039941853 >> 172.21.15.133:0/1039941853 conn(0x55a1753030f0 msgr2=0x55a17530eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.849+0000 7f57e7fff640 0 client.15951 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:04.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.849+0000 7f5810f7d740 1 -- 172.21.15.133:0/1039941853 shutdown_connections 2024-09-11T23:38:04.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.849+0000 7f5810f7d740 1 -- 172.21.15.133:0/1039941853 wait complete. 2024-09-11T23:38:04.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.849+0000 7f5810f7d740 20 client.15951 trim_cache size 0 max 16384 2024-09-11T23:38:04.850 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:38:04.850 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:38:04.850 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "db0d0c87-c6a5-413b-aa70-09ad237f43d6", 2024-09-11T23:38:04.851 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:38:04.851 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:38:04.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.850+0000 7f580eba4640 1 -- 172.21.15.133:0/3679999889 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57f80a2f20 msgr2=0x7f57f80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.850+0000 7f580eba4640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57f80a2f20 0x7f57f80a53e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0cc5b0 tx=0x7f57fc00df40 comp rx=0 tx=0).stop 2024-09-11T23:38:04.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.850+0000 7f580eba4640 1 -- 172.21.15.133:0/3679999889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5808100a80 msgr2=0x7f5808197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.850+0000 7f580eba4640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5808100a80 0x7f5808197320 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00d950 tx=0x7f57fc00de10 comp rx=0 tx=0).stop 2024-09-11T23:38:04.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.851+0000 7f580eba4640 1 -- 172.21.15.133:0/3679999889 shutdown_connections 2024-09-11T23:38:04.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.851+0000 7f580eba4640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57f80a2f20 0x7f57f80a53e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.851+0000 7f580eba4640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5808106a80 0x7f580819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.851+0000 7f580eba4640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5808101420 0x7f5808197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.851+0000 7f580eba4640 1 --2- 172.21.15.133:0/3679999889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5808100a80 0x7f5808197320 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.851+0000 7f580eba4640 1 -- 172.21.15.133:0/3679999889 >> 172.21.15.133:0/3679999889 conn(0x7f58080fc7a0 msgr2=0x7f58080fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.851+0000 7f580eba4640 1 -- 172.21.15.133:0/3679999889 shutdown_connections 2024-09-11T23:38:04.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:04.851+0000 7f580eba4640 1 -- 172.21.15.133:0/3679999889 wait complete. 2024-09-11T23:38:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.935+0000 7f806d591640 1 -- 172.21.15.133:0/2391740340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8068102ce0 msgr2=0x7f80681030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.935+0000 7f806d591640 1 --2- 172.21.15.133:0/2391740340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8068102ce0 0x7f80681030c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8058009900 tx=0x7f805802a480 comp rx=0 tx=0).stop 2024-09-11T23:38:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.935+0000 7f806d591640 1 -- 172.21.15.133:0/2391740340 shutdown_connections 2024-09-11T23:38:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.935+0000 7f806d591640 1 --2- 172.21.15.133:0/2391740340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806810c0b0 0x7f806810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.935+0000 7f806d591640 1 --2- 172.21.15.133:0/2391740340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8068103600 0x7f806810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.935+0000 7f806d591640 1 --2- 172.21.15.133:0/2391740340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8068102ce0 0x7f80681030c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.935+0000 7f806d591640 1 -- 172.21.15.133:0/2391740340 >> 172.21.15.133:0/2391740340 conn(0x7f8068077990 msgr2=0x7f80680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.935+0000 7f806d591640 1 -- 172.21.15.133:0/2391740340 shutdown_connections 2024-09-11T23:38:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.935+0000 7f806d591640 1 -- 172.21.15.133:0/2391740340 wait complete. 2024-09-11T23:38:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.936+0000 7f806d591640 1 Processor -- start 2024-09-11T23:38:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.936+0000 7f806d591640 1 -- start start 2024-09-11T23:38:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f806d591640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8068102ce0 0x7f8068198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f806d591640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8068103600 0x7f80681988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f806d591640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f806810c0b0 0x7f806819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f806d591640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8068111eb0 con 0x7f8068102ce0 2024-09-11T23:38:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f806d591640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8068111d30 con 0x7f8068103600 2024-09-11T23:38:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f806d591640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8068112030 con 0x7f806810c0b0 2024-09-11T23:38:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f8066ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8068102ce0 0x7f8068198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f8066ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8068102ce0 0x7f8068198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33140/0 (socket says 172.21.15.133:33140) 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f8066ffd640 1 -- 172.21.15.133:0/4178449883 learned_addr learned my addr 172.21.15.133:0/4178449883 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f80677fe640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f806810c0b0 0x7f806819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f80667fc640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8068103600 0x7f80681988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f8066ffd640 1 -- 172.21.15.133:0/4178449883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f806810c0b0 msgr2=0x7f806819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f8066ffd640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f806810c0b0 0x7f806819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f8066ffd640 1 -- 172.21.15.133:0/4178449883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8068103600 msgr2=0x7f80681988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f8066ffd640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8068103600 0x7f80681988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f8066ffd640 1 -- 172.21.15.133:0/4178449883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806819d350 con 0x7f8068102ce0 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.937+0000 7f80667fc640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8068103600 0x7f80681988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.938+0000 7f8066ffd640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8068102ce0 0x7f8068198360 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8058006010 tx=0x7f80580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.938+0000 7f804ffff640 1 -- 172.21.15.133:0/4178449883 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805803f030 con 0x7f8068102ce0 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.938+0000 7f806d591640 1 -- 172.21.15.133:0/4178449883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806819d5e0 con 0x7f8068102ce0 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.938+0000 7f804ffff640 1 -- 172.21.15.133:0/4178449883 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8058003ce0 con 0x7f8068102ce0 2024-09-11T23:38:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.938+0000 7f804ffff640 1 -- 172.21.15.133:0/4178449883 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058035070 con 0x7f8068102ce0 2024-09-11T23:38:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.938+0000 7f806d591640 1 -- 172.21.15.133:0/4178449883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80681a4ed0 con 0x7f8068102ce0 2024-09-11T23:38:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.941+0000 7f807040b740 1 Processor -- start 2024-09-11T23:38:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.941+0000 7f807040b740 1 -- start start 2024-09-11T23:38:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f807040b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f804ffff640 1 -- 172.21.15.133:0/4178449883 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f8058042030 con 0x7f8068102ce0 2024-09-11T23:38:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f807040b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074cd660 0x557c074cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f807040b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074ce000 0x557c074d99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f807040b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557c074282f0 con 0x557c074ce000 2024-09-11T23:38:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f807040b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c073d0270 con 0x557c074cd660 2024-09-11T23:38:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f807040b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557c074242b0 con 0x557c074d1cc0 2024-09-11T23:38:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f80667fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074ce000 0x557c074d99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f80667fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074ce000 0x557c074d99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33146/0 (socket says 172.21.15.133:33146) 2024-09-11T23:38:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f80667fc640 1 -- 172.21.15.133:0/3116540237 learned_addr learned my addr 172.21.15.133:0/3116540237 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f8066ffd640 1 --2- 172.21.15.133:0/3116540237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074cd660 0x557c074cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f80677fe640 1 --2- 172.21.15.133:0/3116540237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f80667fc640 1 -- 172.21.15.133:0/3116540237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 msgr2=0x557c074d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f80667fc640 1 --2- 172.21.15.133:0/3116540237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f80667fc640 1 -- 172.21.15.133:0/3116540237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074cd660 msgr2=0x557c074cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f80667fc640 1 --2- 172.21.15.133:0/3116540237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074cd660 0x557c074cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f80667fc640 1 -- 172.21.15.133:0/3116540237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c073cf340 con 0x557c074ce000 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f80677fe640 1 --2- 172.21.15.133:0/3116540237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.942+0000 7f80667fc640 1 --2- 172.21.15.133:0/3116540237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074ce000 0x557c074d99c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f805c00bd30 tx=0x7f805c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7994e6865ab478 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.943+0000 7f804ffff640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80500a2f50 0x7f80500a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.943+0000 7f804d7fa640 1 -- 172.21.15.133:0/3116540237 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c015070 con 0x557c074ce000 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.943+0000 7f804d7fa640 1 -- 172.21.15.133:0/3116540237 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f805c00be20 con 0x557c074ce000 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.943+0000 7f804d7fa640 1 -- 172.21.15.133:0/3116540237 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c011aa0 con 0x557c074ce000 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.943+0000 7f80677fe640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80500a2f50 0x7f80500a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.943+0000 7f804ffff640 1 -- 172.21.15.133:0/4178449883 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f80580ef4d0 con 0x7f8068102ce0 2024-09-11T23:38:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.944+0000 7f807040b740 1 -- 172.21.15.133:0/3116540237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074ce000 msgr2=0x557c074d99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.944+0000 7f80677fe640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80500a2f50 0x7f80500a5410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8060005fd0 tx=0x7f80600096a0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.944+0000 7f807040b740 1 --2- 172.21.15.133:0/3116540237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074ce000 0x557c074d99c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f805c00bd30 tx=0x7f805c00f730 comp rx=0 tx=0).stop 2024-09-11T23:38:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.944+0000 7f807040b740 1 -- 172.21.15.133:0/3116540237 shutdown_connections 2024-09-11T23:38:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.944+0000 7f807040b740 1 --2- 172.21.15.133:0/3116540237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074ce000 0x557c074d99c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.944+0000 7f807040b740 1 --2- 172.21.15.133:0/3116540237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074cd660 0x557c074cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.944+0000 7f807040b740 1 --2- 172.21.15.133:0/3116540237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.944+0000 7f807040b740 1 -- 172.21.15.133:0/3116540237 >> 172.21.15.133:0/3116540237 conn(0x557c074bfb10 msgr2=0x557c074bfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.944+0000 7f807040b740 1 -- 172.21.15.133:0/3116540237 shutdown_connections 2024-09-11T23:38:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.944+0000 7f807040b740 1 -- 172.21.15.133:0/3116540237 wait complete. 2024-09-11T23:38:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.944+0000 7f807040b740 1 -- start start 2024-09-11T23:38:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f807040b740 1 Processor -- start 2024-09-11T23:38:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f807040b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 0x557c074d6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f807040b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 0x557c074d7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f8066ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 0x557c074d6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f8066ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 0x557c074d6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33162/0 (socket says 172.21.15.133:33162) 2024-09-11T23:38:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f807040b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074d78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f807040b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557c074da0b0 con 0x557c074cd660 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f807040b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c073b1290 con 0x557c074ce000 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f8066ffd640 1 -- 172.21.15.133:0/3437072653 learned_addr learned my addr 172.21.15.133:0/3437072653 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f807040b740 1 -- 172.21.15.133:0/3437072653 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557c074da230 con 0x557c074d1cc0 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f807040b740 10 client.0 _ll_register_callbacks cb 0x557c073e2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f80667fc640 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 0x557c074d7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.945+0000 7f80677fe640 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074d78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f80667fc640 1 -- 172.21.15.133:0/3437072653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 msgr2=0x557c074d78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f80667fc640 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074d78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f80667fc640 1 -- 172.21.15.133:0/3437072653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 msgr2=0x557c074d6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f80667fc640 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 0x557c074d6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f80667fc640 1 -- 172.21.15.133:0/3437072653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c0713b210 con 0x557c074ce000 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f80667fc640 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 0x557c074d7360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f805c00bd30 tx=0x7f805c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=160bd2e8ec588a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f804aff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f804aff5640 1 -- 172.21.15.133:0/3437072653 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c015070 con 0x557c074ce000 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f807040b740 1 -- 172.21.15.133:0/3437072653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f805c00be20 con 0x557c074ce000 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f804aff5640 1 -- 172.21.15.133:0/3437072653 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f805c005570 con 0x557c074ce000 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f804aff5640 1 -- 172.21.15.133:0/3437072653 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f805c011720 con 0x557c074ce000 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f807040b740 10 client.25385 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.946+0000 7f807040b740 1 -- 172.21.15.133:0/3437072653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c07207620 con 0x557c074ce000 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.947+0000 7f804aff5640 1 -- 172.21.15.133:0/3437072653 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f805c00be20 con 0x557c074ce000 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.947+0000 7f804aff5640 1 -- 172.21.15.133:0/3437072653 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f805c01a460 con 0x557c074ce000 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.947+0000 7f807040b740 10 client.25385 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.947+0000 7f807040b740 10 client.25385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.947+0000 7f807040b740 20 client.25385 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.947+0000 7f807040b740 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557c074e7360 0x557c074e9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.947+0000 7f807040b740 4 client.25385 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.947+0000 7f807040b740 1 -- 172.21.15.133:0/3437072653 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c07435a20 con 0x557c074e7360 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.947+0000 7f8066ffd640 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557c074e7360 0x557c074e9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f804aff5640 10 client.25385 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f8066ffd640 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557c074e7360 0x557c074e9750 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87d89a231bb0cf8 server_cookie=eb40db9db124d779 in_seq=0 out_seq=0 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f804aff5640 1 -- 172.21.15.133:0/3437072653 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c07435a20 con 0x557c074e7360 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f804aff5640 10 client.25385 handle_command_reply: tid=0 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f807040b740 1 client.25385 shutdown 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f807040b740 1 -- 172.21.15.133:0/3437072653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 msgr2=0x557c074d7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f807040b740 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 0x557c074d7360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f805c00bd30 tx=0x7f805c005270 comp rx=0 tx=0).stop 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f807040b740 1 -- 172.21.15.133:0/3437072653 shutdown_connections 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f807040b740 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557c074e7360 0x557c074e9750 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f804aff5640 0 client.25385 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f807040b740 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074d78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.948+0000 7f807040b740 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 0x557c074d7360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.949+0000 7f807040b740 1 --2- 172.21.15.133:0/3437072653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 0x557c074d6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.949+0000 7f807040b740 1 -- 172.21.15.133:0/3437072653 >> 172.21.15.133:0/3437072653 conn(0x557c074bfb10 msgr2=0x557c074cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.949+0000 7f807040b740 1 -- 172.21.15.133:0/3437072653 shutdown_connections 2024-09-11T23:38:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.949+0000 7f807040b740 1 -- 172.21.15.133:0/3437072653 wait complete. 2024-09-11T23:38:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.949+0000 7f807040b740 20 client.25385 trim_cache size 0 max 16384 2024-09-11T23:38:04.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:04 smithi138 ceph-mon[35142]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.975+0000 7f807040b740 1 Processor -- start 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.975+0000 7f807040b740 1 -- start start 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.975+0000 7f807040b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 0x557c074d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f807040b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 0x557c074eef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f8066ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 0x557c074d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f8066ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 0x557c074d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33168/0 (socket says 172.21.15.133:33168) 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f807040b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074ef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f8066ffd640 1 -- 172.21.15.133:0/2431416980 learned_addr learned my addr 172.21.15.133:0/2431416980 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f807040b740 1 -- 172.21.15.133:0/2431416980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80681ab4a0 con 0x557c074cd660 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f807040b740 1 -- 172.21.15.133:0/2431416980 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80681acfe0 con 0x557c074ce000 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f807040b740 1 -- 172.21.15.133:0/2431416980 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80681abb70 con 0x557c074d1cc0 2024-09-11T23:38:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f80667fc640 1 --2- 172.21.15.133:0/2431416980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 0x557c074eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f80677fe640 1 --2- 172.21.15.133:0/2431416980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f8066ffd640 1 -- 172.21.15.133:0/2431416980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 msgr2=0x557c074ef460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f8066ffd640 1 --2- 172.21.15.133:0/2431416980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074ef460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f8066ffd640 1 -- 172.21.15.133:0/2431416980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 msgr2=0x557c074eef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f8066ffd640 1 --2- 172.21.15.133:0/2431416980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 0x557c074eef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f8066ffd640 1 -- 172.21.15.133:0/2431416980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c074d5db0 con 0x557c074cd660 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f80677fe640 1 --2- 172.21.15.133:0/2431416980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074ef460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.976+0000 7f80667fc640 1 --2- 172.21.15.133:0/2431416980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 0x557c074eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.977+0000 7f8066ffd640 1 --2- 172.21.15.133:0/2431416980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 0x557c074d0660 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f80580bb5e0 tx=0x7f80580f1960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31bf31f1380ef77 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.977+0000 7f804cff9640 1 -- 172.21.15.133:0/2431416980 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058049070 con 0x557c074cd660 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.977+0000 7f804cff9640 1 -- 172.21.15.133:0/2431416980 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80580bbfe0 con 0x557c074cd660 2024-09-11T23:38:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.977+0000 7f804cff9640 1 -- 172.21.15.133:0/2431416980 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80580c4550 con 0x557c074cd660 2024-09-11T23:38:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.978+0000 7f807040b740 1 -- 172.21.15.133:0/2431416980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 msgr2=0x557c074d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.978+0000 7f807040b740 1 --2- 172.21.15.133:0/2431416980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 0x557c074d0660 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f80580bb5e0 tx=0x7f80580f1960 comp rx=0 tx=0).stop 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.978+0000 7f807040b740 1 -- 172.21.15.133:0/2431416980 shutdown_connections 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.978+0000 7f807040b740 1 --2- 172.21.15.133:0/2431416980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074d1cc0 0x557c074ef460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.978+0000 7f807040b740 1 --2- 172.21.15.133:0/2431416980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074ce000 0x557c074eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.978+0000 7f807040b740 1 --2- 172.21.15.133:0/2431416980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074cd660 0x557c074d0660 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.978+0000 7f807040b740 1 -- 172.21.15.133:0/2431416980 >> 172.21.15.133:0/2431416980 conn(0x557c074bfb10 msgr2=0x557c074cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.978+0000 7f807040b740 1 -- 172.21.15.133:0/2431416980 shutdown_connections 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.978+0000 7f807040b740 1 -- 172.21.15.133:0/2431416980 wait complete. 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.978+0000 7f807040b740 1 -- start start 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.978+0000 7f807040b740 1 Processor -- start 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f807040b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074cd660 0x557c074db420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f807040b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074ce000 0x557c074db960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f807040b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074d1cc0 0x557c074dbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f807040b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80681ac910 con 0x557c074d1cc0 2024-09-11T23:38:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f807040b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80681ac240 con 0x557c074cd660 2024-09-11T23:38:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f807040b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80681a94a0 con 0x557c074ce000 2024-09-11T23:38:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f80677fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074d1cc0 0x557c074dbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f80677fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074d1cc0 0x557c074dbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33176/0 (socket says 172.21.15.133:33176) 2024-09-11T23:38:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f80677fe640 1 -- 172.21.15.133:0/2688039897 learned_addr learned my addr 172.21.15.133:0/2688039897 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f8066ffd640 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074cd660 0x557c074db420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f80667fc640 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074ce000 0x557c074db960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f807040b740 10 client.0 _ll_register_callbacks cb 0x7f80580f2e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f80677fe640 1 -- 172.21.15.133:0/2688039897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074ce000 msgr2=0x557c074db960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f80677fe640 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074ce000 0x557c074db960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f80677fe640 1 -- 172.21.15.133:0/2688039897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074cd660 msgr2=0x557c074db420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f80677fe640 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074cd660 0x557c074db420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f80677fe640 1 -- 172.21.15.133:0/2688039897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c074e9cf0 con 0x557c074d1cc0 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.979+0000 7f8066ffd640 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074cd660 0x557c074db420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.980+0000 7f80677fe640 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074d1cc0 0x557c074dbea0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f80600109b0 tx=0x7f8060004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9959bef34212c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.980+0000 7f804aff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.980+0000 7f804aff5640 1 -- 172.21.15.133:0/2688039897 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806002d060 con 0x557c074d1cc0 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.980+0000 7f804aff5640 1 -- 172.21.15.133:0/2688039897 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f806001d940 con 0x557c074d1cc0 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.980+0000 7f807040b740 1 -- 172.21.15.133:0/2688039897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80580bbfe0 con 0x557c074d1cc0 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.980+0000 7f804aff5640 1 -- 172.21.15.133:0/2688039897 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f806001db40 con 0x557c074d1cc0 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.980+0000 7f807040b740 10 client.15969 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.980+0000 7f807040b740 1 -- 172.21.15.133:0/2688039897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c07552230 con 0x557c074d1cc0 2024-09-11T23:38:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.980+0000 7f804aff5640 1 -- 172.21.15.133:0/2688039897 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80580bbfe0 con 0x557c074d1cc0 2024-09-11T23:38:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.981+0000 7f804aff5640 1 -- 172.21.15.133:0/2688039897 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8060030020 con 0x557c074d1cc0 2024-09-11T23:38:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.981+0000 7f807040b740 10 client.15969 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.981+0000 7f807040b740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.981+0000 7f807040b740 20 client.15969 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.981+0000 7f807040b740 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557c074e7360 0x557c07554960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.981+0000 7f807040b740 4 client.15969 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:38:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.981+0000 7f807040b740 1 -- 172.21.15.133:0/2688039897 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c0742fd70 con 0x557c074e7360 2024-09-11T23:38:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.981+0000 7f8066ffd640 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557c074e7360 0x557c07554960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.982+0000 7f804aff5640 10 client.15969 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.982+0000 7f8066ffd640 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557c074e7360 0x557c07554960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f92b84094ed90f server_cookie=50d2ea4bcaddf779 in_seq=0 out_seq=0 2024-09-11T23:38:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.987+0000 7f804aff5640 1 -- 172.21.15.133:0/2688039897 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c0742fd70 con 0x557c074e7360 2024-09-11T23:38:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.987+0000 7f804aff5640 10 client.15969 handle_command_reply: tid=0 2024-09-11T23:38:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.987+0000 7f807040b740 1 client.15969 shutdown 2024-09-11T23:38:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f807040b740 1 -- 172.21.15.133:0/2688039897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074d1cc0 msgr2=0x557c074dbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f807040b740 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074d1cc0 0x557c074dbea0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f80600109b0 tx=0x7f8060004a40 comp rx=0 tx=0).stop 2024-09-11T23:38:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f807040b740 1 -- 172.21.15.133:0/2688039897 shutdown_connections 2024-09-11T23:38:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f807040b740 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x557c074e7360 0x557c07554960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f807040b740 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c074d1cc0 0x557c074dbea0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f804aff5640 0 client.15969 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f807040b740 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c074ce000 0x557c074db960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f807040b740 1 --2- 172.21.15.133:0/2688039897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c074cd660 0x557c074db420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f807040b740 1 -- 172.21.15.133:0/2688039897 >> 172.21.15.133:0/2688039897 conn(0x557c074bfb10 msgr2=0x557c074d8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f807040b740 1 -- 172.21.15.133:0/2688039897 shutdown_connections 2024-09-11T23:38:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f807040b740 1 -- 172.21.15.133:0/2688039897 wait complete. 2024-09-11T23:38:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.988+0000 7f807040b740 20 client.15969 trim_cache size 0 max 16384 2024-09-11T23:38:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:38:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:38:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:38:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:38:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 -- 172.21.15.133:0/4178449883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80500a2f50 msgr2=0x7f80500a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80500a2f50 0x7f80500a5410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8060005fd0 tx=0x7f80600096a0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 -- 172.21.15.133:0/4178449883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8068102ce0 msgr2=0x7f8068198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8068102ce0 0x7f8068198360 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8058006010 tx=0x7f80580042f0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 -- 172.21.15.133:0/4178449883 shutdown_connections 2024-09-11T23:38:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80500a2f50 0x7f80500a5410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f806810c0b0 0x7f806819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8068103600 0x7f80681988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 --2- 172.21.15.133:0/4178449883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8068102ce0 0x7f8068198360 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 -- 172.21.15.133:0/4178449883 >> 172.21.15.133:0/4178449883 conn(0x7f8068077990 msgr2=0x7f80680fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 -- 172.21.15.133:0/4178449883 shutdown_connections 2024-09-11T23:38:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:04.989+0000 7f806d591640 1 -- 172.21.15.133:0/4178449883 wait complete. 2024-09-11T23:38:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:04 smithi178 ceph-mon[35882]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T23:38:05.629 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:38:05.951 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:06.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.477+0000 7f27f5d20640 1 -- 172.21.15.133:0/3570325303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f0106f20 msgr2=0x7f27f010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:06.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.477+0000 7f27f5d20640 1 --2- 172.21.15.133:0/3570325303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f0106f20 0x7f27f010eaa0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009ca0 tx=0x7f27e802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.477+0000 7f27f5d20640 1 -- 172.21.15.133:0/3570325303 shutdown_connections 2024-09-11T23:38:06.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.477+0000 7f27f5d20640 1 --2- 172.21.15.133:0/3570325303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f0106f20 0x7f27f010eaa0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.477+0000 7f27f5d20640 1 --2- 172.21.15.133:0/3570325303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f01027a0 0x7f27f01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.477+0000 7f27f5d20640 1 --2- 172.21.15.133:0/3570325303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27f0101df0 0x7f27f01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.477+0000 7f27f5d20640 1 -- 172.21.15.133:0/3570325303 >> 172.21.15.133:0/3570325303 conn(0x7f27f00fd8f0 msgr2=0x7f27f00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:06.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.477+0000 7f27f5d20640 1 -- 172.21.15.133:0/3570325303 shutdown_connections 2024-09-11T23:38:06.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.477+0000 7f27f5d20640 1 -- 172.21.15.133:0/3570325303 wait complete. 2024-09-11T23:38:06.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.478+0000 7f27f5d20640 1 Processor -- start 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.479+0000 7f27f5d20640 1 -- start start 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.479+0000 7f27f5d20640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f0101df0 0x7f27f01a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.479+0000 7f27f5d20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f01027a0 0x7f27f01a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.479+0000 7f27f5d20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27f0106f20 0x7f27f010b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.479+0000 7f27f5d20640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0111fd0 con 0x7f27f0101df0 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.479+0000 7f27f5d20640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0111e50 con 0x7f27f01027a0 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.479+0000 7f27f5d20640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0112150 con 0x7f27f0106f20 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.479+0000 7f27eeffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f01027a0 0x7f27f01a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.479+0000 7f27ef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f0101df0 0x7f27f01a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.479+0000 7f27eeffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f01027a0 0x7f27f01a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47898/0 (socket says 172.21.15.133:47898) 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.479+0000 7f27eeffd640 1 -- 172.21.15.133:0/4062981817 learned_addr learned my addr 172.21.15.133:0/4062981817 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.480+0000 7f27effff640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27f0106f20 0x7f27f010b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.480+0000 7f27eeffd640 1 -- 172.21.15.133:0/4062981817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27f0106f20 msgr2=0x7f27f010b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:06.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.480+0000 7f27eeffd640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27f0106f20 0x7f27f010b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.480+0000 7f27eeffd640 1 -- 172.21.15.133:0/4062981817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f0101df0 msgr2=0x7f27f01a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:06.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.480+0000 7f27eeffd640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f0101df0 0x7f27f01a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.480+0000 7f27eeffd640 1 -- 172.21.15.133:0/4062981817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f010bee0 con 0x7f27f01027a0 2024-09-11T23:38:06.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.480+0000 7f27ef7fe640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f0101df0 0x7f27f01a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:06.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.480+0000 7f27eeffd640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f01027a0 0x7f27f01a4940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c920 tx=0x7f27e400cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:06.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.480+0000 7f27ecff9640 1 -- 172.21.15.133:0/4062981817 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4007c70 con 0x7f27f01027a0 2024-09-11T23:38:06.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.481+0000 7f27ecff9640 1 -- 172.21.15.133:0/4062981817 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e4007e10 con 0x7f27f01027a0 2024-09-11T23:38:06.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.481+0000 7f27f5d20640 1 -- 172.21.15.133:0/4062981817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f010c1d0 con 0x7f27f01027a0 2024-09-11T23:38:06.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.481+0000 7f27f5d20640 1 -- 172.21.15.133:0/4062981817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f01ab2d0 con 0x7f27f01027a0 2024-09-11T23:38:06.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.481+0000 7f27ecff9640 1 -- 172.21.15.133:0/4062981817 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e40189a0 con 0x7f27f01027a0 2024-09-11T23:38:06.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.484+0000 7f27f80f9740 1 Processor -- start 2024-09-11T23:38:06.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.484+0000 7f27f80f9740 1 -- start start 2024-09-11T23:38:06.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 0x561711750880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561711750dc0 0x56171175b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 0x56171175da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27f80f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5617116b62e0 con 0x56171175b680 2024-09-11T23:38:06.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27f80f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56171165dce0 con 0x5617117504a0 2024-09-11T23:38:06.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27f80f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5617116b22a0 con 0x561711750dc0 2024-09-11T23:38:06.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27ecff9640 1 -- 172.21.15.133:0/4062981817 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f27e4018b40 con 0x7f27f01027a0 2024-09-11T23:38:06.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27eeffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 0x56171175da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27effff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 0x561711750880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27ef7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561711750dc0 0x56171175b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27ef7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561711750dc0 0x56171175b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:36624/0 (socket says 172.21.15.133:36624) 2024-09-11T23:38:06.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.485+0000 7f27ef7fe640 1 -- 172.21.15.133:0/562344213 learned_addr learned my addr 172.21.15.133:0/562344213 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:06.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.486+0000 7f27ecff9640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27d80a2fa0 0x7f27d80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.486+0000 7f27ef7fe640 1 -- 172.21.15.133:0/562344213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 msgr2=0x561711750880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:06.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.486+0000 7f27ef7fe640 1 --2- 172.21.15.133:0/562344213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 0x561711750880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.486+0000 7f27ef7fe640 1 -- 172.21.15.133:0/562344213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 msgr2=0x56171175da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:06.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.486+0000 7f27ef7fe640 1 --2- 172.21.15.133:0/562344213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 0x56171175da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.486+0000 7f27ef7fe640 1 -- 172.21.15.133:0/562344213 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561711642440 con 0x561711750dc0 2024-09-11T23:38:06.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.486+0000 7f27ecff9640 1 -- 172.21.15.133:0/4062981817 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f27e40cf720 con 0x7f27f01027a0 2024-09-11T23:38:06.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.487+0000 7f27effff640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27d80a2fa0 0x7f27d80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.487+0000 7f27eeffd640 1 --2- 172.21.15.133:0/562344213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 0x56171175da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:06.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.487+0000 7f27ef7fe640 1 --2- 172.21.15.133:0/562344213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561711750dc0 0x56171175b140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27e00104d0 tx=0x7f27e0010990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ada8a4ae7f07a543 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:06.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.487+0000 7f27d5ffb640 1 -- 172.21.15.133:0/562344213 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0011760 con 0x561711750dc0 2024-09-11T23:38:06.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.487+0000 7f27effff640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27d80a2fa0 0x7f27d80a5460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27e8001190 tx=0x7f27e8005fd0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:06.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.487+0000 7f27d5ffb640 1 -- 172.21.15.133:0/562344213 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e0011de0 con 0x561711750dc0 2024-09-11T23:38:06.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.487+0000 7f27d5ffb640 1 -- 172.21.15.133:0/562344213 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e000e430 con 0x561711750dc0 2024-09-11T23:38:06.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.488+0000 7f27f80f9740 1 -- 172.21.15.133:0/562344213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561711750dc0 msgr2=0x56171175b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:06.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.488+0000 7f27f80f9740 1 --2- 172.21.15.133:0/562344213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561711750dc0 0x56171175b140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27e00104d0 tx=0x7f27e0010990 comp rx=0 tx=0).stop 2024-09-11T23:38:06.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.488+0000 7f27f80f9740 1 -- 172.21.15.133:0/562344213 shutdown_connections 2024-09-11T23:38:06.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.488+0000 7f27f80f9740 1 --2- 172.21.15.133:0/562344213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 0x56171175da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.488+0000 7f27f80f9740 1 --2- 172.21.15.133:0/562344213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561711750dc0 0x56171175b140 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.488+0000 7f27f80f9740 1 --2- 172.21.15.133:0/562344213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 0x561711750880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.488+0000 7f27f80f9740 1 -- 172.21.15.133:0/562344213 >> 172.21.15.133:0/562344213 conn(0x56171174ca30 msgr2=0x56171174ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.488+0000 7f27f80f9740 1 -- 172.21.15.133:0/562344213 shutdown_connections 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.488+0000 7f27f80f9740 1 -- 172.21.15.133:0/562344213 wait complete. 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.489+0000 7f27f80f9740 1 -- start start 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.489+0000 7f27f80f9740 1 Processor -- start 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.489+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617117504a0 0x56171175aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.489+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561711750dc0 0x5617117581a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.489+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 0x5617117586e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.489+0000 7f27f80f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56171175e250 con 0x56171175b680 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.489+0000 7f27f80f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561711671050 con 0x561711750dc0 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.489+0000 7f27f80f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56171175e3d0 con 0x5617117504a0 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27f80f9740 10 client.0 _ll_register_callbacks cb 0x5617116b1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27effff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 0x5617117586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27effff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 0x5617117586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33218/0 (socket says 172.21.15.133:33218) 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27effff640 1 -- 172.21.15.133:0/913771912 learned_addr learned my addr 172.21.15.133:0/913771912 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:06.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27eeffd640 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561711750dc0 0x5617117581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27effff640 1 -- 172.21.15.133:0/913771912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617117504a0 msgr2=0x56171175aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:06.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27effff640 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617117504a0 0x56171175aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27effff640 1 -- 172.21.15.133:0/913771912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561711750dc0 msgr2=0x5617117581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:06.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27effff640 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561711750dc0 0x5617117581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27effff640 1 -- 172.21.15.133:0/913771912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617114273c0 con 0x56171175b680 2024-09-11T23:38:06.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27eeffd640 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561711750dc0 0x5617117581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:06.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27effff640 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 0x5617117586e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f27e8035000 tx=0x7f27e802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155d16df4f9d52ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:06.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.490+0000 7f27ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:06.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.491+0000 7f27ceffd640 1 -- 172.21.15.133:0/913771912 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80047c0 con 0x56171175b680 2024-09-11T23:38:06.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.491+0000 7f27f80f9740 1 -- 172.21.15.133:0/913771912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e0011de0 con 0x56171175b680 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.491+0000 7f27ceffd640 1 -- 172.21.15.133:0/913771912 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f27e802f610 con 0x56171175b680 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.491+0000 7f27ceffd640 1 -- 172.21.15.133:0/913771912 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27e802f810 con 0x56171175b680 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.491+0000 7f27f80f9740 10 client.15981 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.491+0000 7f27f80f9740 1 -- 172.21.15.133:0/913771912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561711769b40 con 0x56171175b680 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.491+0000 7f27ceffd640 1 -- 172.21.15.133:0/913771912 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e0011de0 con 0x56171175b680 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.491+0000 7f27ceffd640 1 -- 172.21.15.133:0/913771912 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27e8030a30 con 0x56171175b680 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.492+0000 7f27f80f9740 10 client.15981 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.492+0000 7f27f80f9740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.492+0000 7f27f80f9740 20 client.15981 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.492+0000 7f27f80f9740 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56171176b1c0 0x56171176d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.492+0000 7f27f80f9740 4 client.15981 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.492+0000 7f27f80f9740 1 -- 172.21.15.133:0/913771912 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617116c3850 con 0x56171176b1c0 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.492+0000 7f27ef7fe640 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56171176b1c0 0x56171176d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.492+0000 7f27ef7fe640 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56171176b1c0 0x56171176d680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b16cb66c6ac9040c server_cookie=f3092e80f3773fb8 in_seq=0 out_seq=0 2024-09-11T23:38:06.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.492+0000 7f27ceffd640 10 client.15981 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:06.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.493+0000 7f27ceffd640 1 -- 172.21.15.133:0/913771912 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617116c3850 con 0x56171176b1c0 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.493+0000 7f27ceffd640 10 client.15981 handle_command_reply: tid=0 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.493+0000 7f27f80f9740 1 client.15981 shutdown 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.493+0000 7f27f80f9740 1 -- 172.21.15.133:0/913771912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 msgr2=0x5617117586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.493+0000 7f27f80f9740 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 0x5617117586e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f27e8035000 tx=0x7f27e802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.493+0000 7f27f80f9740 1 -- 172.21.15.133:0/913771912 shutdown_connections 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.493+0000 7f27f80f9740 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56171176b1c0 0x56171176d680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.493+0000 7f27f80f9740 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56171175b680 0x5617117586e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.493+0000 7f27f80f9740 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561711750dc0 0x5617117581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.493+0000 7f27f80f9740 1 --2- 172.21.15.133:0/913771912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617117504a0 0x56171175aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.493+0000 7f27f80f9740 1 -- 172.21.15.133:0/913771912 >> 172.21.15.133:0/913771912 conn(0x56171174ca30 msgr2=0x561711752710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.494+0000 7f27ceffd640 0 client.15981 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.494+0000 7f27f80f9740 1 -- 172.21.15.133:0/913771912 shutdown_connections 2024-09-11T23:38:06.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.494+0000 7f27f80f9740 1 -- 172.21.15.133:0/913771912 wait complete. 2024-09-11T23:38:06.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.494+0000 7f27f80f9740 20 client.15981 trim_cache size 0 max 16384 2024-09-11T23:38:06.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27f80f9740 1 Processor -- start 2024-09-11T23:38:06.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27f80f9740 1 -- start start 2024-09-11T23:38:06.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 0x56171176d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x56171175a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56171175b680 0x561711772f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27f80f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01b3dc0 con 0x561711750dc0 2024-09-11T23:38:06.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27f80f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01b5900 con 0x5617117504a0 2024-09-11T23:38:06.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27f80f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01b4490 con 0x56171175b680 2024-09-11T23:38:06.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27eeffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x56171175a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27eeffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x56171175a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33232/0 (socket says 172.21.15.133:33232) 2024-09-11T23:38:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27eeffd640 1 -- 172.21.15.133:0/3944663076 learned_addr learned my addr 172.21.15.133:0/3944663076 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.511+0000 7f27ef7fe640 1 --2- 172.21.15.133:0/3944663076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 0x56171176d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.512+0000 7f27eeffd640 1 -- 172.21.15.133:0/3944663076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56171175b680 msgr2=0x561711772f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.512+0000 7f27eeffd640 1 --2- 172.21.15.133:0/3944663076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56171175b680 0x561711772f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.512+0000 7f27eeffd640 1 -- 172.21.15.133:0/3944663076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 msgr2=0x56171176d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.512+0000 7f27eeffd640 1 --2- 172.21.15.133:0/3944663076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 0x56171176d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.512+0000 7f27eeffd640 1 -- 172.21.15.133:0/3944663076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561711757150 con 0x561711750dc0 2024-09-11T23:38:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.512+0000 7f27eeffd640 1 --2- 172.21.15.133:0/3944663076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x56171175a110 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f27e409b770 tx=0x7f27e4005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b705f159f23b9af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:06.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.512+0000 7f27d57fa640 1 -- 172.21.15.133:0/3944663076 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e409ba70 con 0x561711750dc0 2024-09-11T23:38:06.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.512+0000 7f27d57fa640 1 -- 172.21.15.133:0/3944663076 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e409fe20 con 0x561711750dc0 2024-09-11T23:38:06.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.512+0000 7f27d57fa640 1 -- 172.21.15.133:0/3944663076 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e40cfb50 con 0x561711750dc0 2024-09-11T23:38:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.513+0000 7f27f80f9740 1 -- 172.21.15.133:0/3944663076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 msgr2=0x56171175a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.513+0000 7f27f80f9740 1 --2- 172.21.15.133:0/3944663076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x56171175a110 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f27e409b770 tx=0x7f27e4005e40 comp rx=0 tx=0).stop 2024-09-11T23:38:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.513+0000 7f27f80f9740 1 -- 172.21.15.133:0/3944663076 shutdown_connections 2024-09-11T23:38:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.513+0000 7f27f80f9740 1 --2- 172.21.15.133:0/3944663076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56171175b680 0x561711772f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.513+0000 7f27f80f9740 1 --2- 172.21.15.133:0/3944663076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x56171175a110 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.513+0000 7f27f80f9740 1 --2- 172.21.15.133:0/3944663076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 0x56171176d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.513+0000 7f27f80f9740 1 -- 172.21.15.133:0/3944663076 >> 172.21.15.133:0/3944663076 conn(0x56171174ca30 msgr2=0x561711759a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.513+0000 7f27f80f9740 1 -- 172.21.15.133:0/3944663076 shutdown_connections 2024-09-11T23:38:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.513+0000 7f27f80f9740 1 -- 172.21.15.133:0/3944663076 wait complete. 2024-09-11T23:38:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27f80f9740 1 -- start start 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27f80f9740 1 Processor -- start 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 0x561711760e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x5617117613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27f80f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56171175b680 0x5617117ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27f80f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01b5230 con 0x561711750dc0 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27f80f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01b4b60 con 0x5617117504a0 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27f80f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01b1da0 con 0x56171175b680 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27eeffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x5617117613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27eeffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x5617117613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33234/0 (socket says 172.21.15.133:33234) 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27eeffd640 1 -- 172.21.15.133:0/3155532992 learned_addr learned my addr 172.21.15.133:0/3155532992 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.514+0000 7f27f80f9740 10 client.0 _ll_register_callbacks cb 0x7f27e80095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.515+0000 7f27eeffd640 1 -- 172.21.15.133:0/3155532992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56171175b680 msgr2=0x5617117ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.515+0000 7f27eeffd640 1 --2- 172.21.15.133:0/3155532992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56171175b680 0x5617117ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.515+0000 7f27eeffd640 1 -- 172.21.15.133:0/3155532992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 msgr2=0x561711760e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.515+0000 7f27eeffd640 1 --2- 172.21.15.133:0/3155532992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 0x561711760e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.515+0000 7f27eeffd640 1 -- 172.21.15.133:0/3155532992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617117cd5c0 con 0x561711750dc0 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.515+0000 7f27eeffd640 1 --2- 172.21.15.133:0/3155532992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x5617117613b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f27e4086010 tx=0x7f27e409ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b7465539fb2fe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.515+0000 7f27ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.515+0000 7f27ceffd640 1 -- 172.21.15.133:0/3155532992 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e40b3070 con 0x561711750dc0 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.515+0000 7f27ceffd640 1 -- 172.21.15.133:0/3155532992 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f27e40cfb50 con 0x561711750dc0 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.515+0000 7f27f80f9740 1 -- 172.21.15.133:0/3155532992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e409fe20 con 0x561711750dc0 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27ceffd640 1 -- 172.21.15.133:0/3155532992 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27e40a44c0 con 0x561711750dc0 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27f80f9740 10 client.15987 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27f80f9740 1 -- 172.21.15.133:0/3155532992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617117d5b40 con 0x561711750dc0 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27ceffd640 1 -- 172.21.15.133:0/3155532992 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e40cfd50 con 0x561711750dc0 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27ceffd640 1 -- 172.21.15.133:0/3155532992 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27e40977c0 con 0x561711750dc0 2024-09-11T23:38:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27f80f9740 10 client.15987 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27f80f9740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27f80f9740 20 client.15987 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27f80f9740 1 --2- 172.21.15.133:0/3155532992 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5617117d61f0 0x5617117d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27f80f9740 4 client.15987 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:38:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27f80f9740 1 -- 172.21.15.133:0/3155532992 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f27e409fe20 con 0x5617117d61f0 2024-09-11T23:38:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.516+0000 7f27ef7fe640 1 --2- 172.21.15.133:0/3155532992 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5617117d61f0 0x5617117d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:06.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.517+0000 7f27ceffd640 10 client.15987 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:06.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.517+0000 7f27ef7fe640 1 --2- 172.21.15.133:0/3155532992 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5617117d61f0 0x5617117d8610 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1035dd34bfc4629a server_cookie=17d3f038d730f592 in_seq=0 out_seq=0 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.517+0000 7f27ceffd640 1 -- 172.21.15.133:0/3155532992 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f27e409fe20 con 0x5617117d61f0 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.517+0000 7f27ceffd640 10 client.15987 handle_command_reply: tid=0 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.517+0000 7f27f80f9740 1 client.15987 shutdown 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.517+0000 7f27f80f9740 1 -- 172.21.15.133:0/3155532992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 msgr2=0x5617117613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.517+0000 7f27f80f9740 1 --2- 172.21.15.133:0/3155532992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x5617117613b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f27e4086010 tx=0x7f27e409ba70 comp rx=0 tx=0).stop 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.517+0000 7f27f80f9740 1 -- 172.21.15.133:0/3155532992 shutdown_connections 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.517+0000 7f27f80f9740 1 --2- 172.21.15.133:0/3155532992 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5617117d61f0 0x5617117d8610 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.517+0000 7f27f80f9740 1 --2- 172.21.15.133:0/3155532992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56171175b680 0x5617117ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.518+0000 7f27f80f9740 1 --2- 172.21.15.133:0/3155532992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561711750dc0 0x5617117613b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.518+0000 7f27f80f9740 1 --2- 172.21.15.133:0/3155532992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617117504a0 0x561711760e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.518+0000 7f27f80f9740 1 -- 172.21.15.133:0/3155532992 >> 172.21.15.133:0/3155532992 conn(0x56171174ca30 msgr2=0x56171175a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.518+0000 7f27ceffd640 0 client.15987 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.518+0000 7f27f80f9740 1 -- 172.21.15.133:0/3155532992 shutdown_connections 2024-09-11T23:38:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.518+0000 7f27f80f9740 1 -- 172.21.15.133:0/3155532992 wait complete. 2024-09-11T23:38:06.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.518+0000 7f27f80f9740 20 client.15987 trim_cache size 0 max 16384 2024-09-11T23:38:06.522 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:38:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:38:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-09-11T23:38:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "db0d0c87-c6a5-413b-aa70-09ad237f43d6": { 2024-09-11T23:38:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-09-11T23:38:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "db0d0c87-c6a5-413b-aa70-09ad237f43d6", 2024-09-11T23:38:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2024-09-11T23:38:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:06.523 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:38:06.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.518+0000 7f27f5d20640 1 -- 172.21.15.133:0/4062981817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27d80a2fa0 msgr2=0x7f27d80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:06.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.518+0000 7f27f5d20640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27d80a2fa0 0x7f27d80a5460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27e8001190 tx=0x7f27e8005fd0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.519+0000 7f27f5d20640 1 -- 172.21.15.133:0/4062981817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f01027a0 msgr2=0x7f27f01a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:06.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.519+0000 7f27f5d20640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f01027a0 0x7f27f01a4940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c920 tx=0x7f27e400cde0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.519+0000 7f27f5d20640 1 -- 172.21.15.133:0/4062981817 shutdown_connections 2024-09-11T23:38:06.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.519+0000 7f27f5d20640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27d80a2fa0 0x7f27d80a5460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.519+0000 7f27f5d20640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27f0106f20 0x7f27f010b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.519+0000 7f27f5d20640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f01027a0 0x7f27f01a4940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.519+0000 7f27f5d20640 1 --2- 172.21.15.133:0/4062981817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f0101df0 0x7f27f01a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:06.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.519+0000 7f27f5d20640 1 -- 172.21.15.133:0/4062981817 >> 172.21.15.133:0/4062981817 conn(0x7f27f00fd8f0 msgr2=0x7f27f010d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:06.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.519+0000 7f27f5d20640 1 -- 172.21.15.133:0/4062981817 shutdown_connections 2024-09-11T23:38:06.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:06.519+0000 7f27f5d20640 1 -- 172.21.15.133:0/4062981817 wait complete. 2024-09-11T23:38:06.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:06 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:38:06.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:06 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:38:06.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:06 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:38:06.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:06 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:38:06.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:06 smithi138 ceph-mon[35142]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-11T23:38:06.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:06 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:38:06.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:06 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:38:06.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:06 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:38:06.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:06 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:38:06.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:06 smithi133 ceph-mon[31430]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-11T23:38:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:06 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:38:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:06 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:38:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:06 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:38:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:06 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:38:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:06 smithi178 ceph-mon[35882]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-11T23:38:07.404 INFO:tasks.cephfs.filesystem:scrub status for tag:db0d0c87-c6a5-413b-aa70-09ad237f43d6 - {'path': '/', 'tag': 'db0d0c87-c6a5-413b-aa70-09ad237f43d6', 'options': 'recursive,force'} 2024-09-11T23:38:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:08 smithi138 ceph-mon[35142]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T23:38:08.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:08 smithi133 ceph-mon[31430]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T23:38:09.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:08 smithi178 ceph-mon[35882]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T23:38:10.672 INFO:tasks.workunit.client.0.smithi133.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-11T23:38:10.672 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-11T23:38:10.672 INFO:tasks.workunit.client.0.smithi133.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T23:38:10.672 INFO:tasks.workunit.client.0.smithi133.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T23:38:10.672 INFO:tasks.workunit.client.0.smithi133.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T23:38:10.672 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-11T23:38:10.672 INFO:tasks.workunit.client.0.smithi133.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T23:38:10.672 INFO:tasks.workunit.client.0.smithi133.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T23:38:10.672 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-11T23:38:10.673 INFO:tasks.workunit.client.0.smithi133.stderr: git switch -c 2024-09-11T23:38:10.673 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-11T23:38:10.673 INFO:tasks.workunit.client.0.smithi133.stderr:Or undo this operation with: 2024-09-11T23:38:10.673 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-11T23:38:10.673 INFO:tasks.workunit.client.0.smithi133.stderr: git switch - 2024-09-11T23:38:10.673 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-11T23:38:10.673 INFO:tasks.workunit.client.0.smithi133.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T23:38:10.673 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-11T23:38:10.673 INFO:tasks.workunit.client.0.smithi133.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-11T23:38:10.678 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T23:38:10.706 INFO:tasks.workunit.client.0.smithi133.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T23:38:10.709 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T23:38:10.709 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T23:38:10.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:10 smithi138 ceph-mon[35142]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-11T23:38:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:10 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:38:10.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:10 smithi133 ceph-mon[31430]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-11T23:38:10.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:10 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:38:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:10 smithi178 ceph-mon[35882]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-11T23:38:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:10 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:38:11.445 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T23:38:11.535 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T23:38:11.576 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T23:38:11.577 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T23:38:11.577 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T23:38:11.617 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T23:38:11.621 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:38:11.621 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T23:38:11.681 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-09-11T23:38:11.682 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-09-11T23:38:11.682 DEBUG:teuthology.orchestra.run.smithi133:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-09-11T23:38:11.743 INFO:tasks.workunit.client.0.smithi133.stdout:getting iogen 2024-09-11T23:38:11.744 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 'getting iogen' 2024-09-11T23:38:11.744 INFO:tasks.workunit.client.0.smithi133.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-11T23:38:11.763 INFO:tasks.workunit.client.0.smithi133.stderr:--2024-09-11 23:38:11-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-11T23:38:11.764 INFO:tasks.workunit.client.0.smithi133.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-11T23:38:11.785 INFO:tasks.workunit.client.0.smithi133.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-11T23:38:11.805 INFO:tasks.workunit.client.0.smithi133.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-11T23:38:11.806 INFO:tasks.workunit.client.0.smithi133.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-09-11T23:38:11.807 INFO:tasks.workunit.client.0.smithi133.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-09-11T23:38:11.810 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-11T23:38:11.810 INFO:tasks.workunit.client.0.smithi133.stderr: 0K ....... 100% 1.30G=0s 2024-09-11T23:38:11.810 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-11T23:38:11.810 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-11 23:38:11 (1.30 GB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-09-11T23:38:11.810 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-11T23:38:11.810 INFO:tasks.workunit.client.0.smithi133.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-09-11T23:38:11.822 INFO:tasks.workunit.client.0.smithi133.stdout:iogen_3.1p0 2024-09-11T23:38:11.824 INFO:tasks.workunit.client.0.smithi133.stdout:iogen_3.1p0/src 2024-09-11T23:38:11.825 INFO:tasks.workunit.client.0.smithi133.stdout:iogen_3.1p0/src/iogen.8 2024-09-11T23:38:11.826 INFO:tasks.workunit.client.0.smithi133.stdout:iogen_3.1p0/src/iogen.c 2024-09-11T23:38:11.827 INFO:tasks.workunit.client.0.smithi133.stdout:iogen_3.1p0/README 2024-09-11T23:38:11.828 INFO:tasks.workunit.client.0.smithi133.stdout:iogen_3.1p0/Makefile.bsd 2024-09-11T23:38:11.829 INFO:tasks.workunit.client.0.smithi133.stdout:iogen_3.1p0/Makefile 2024-09-11T23:38:11.830 INFO:tasks.workunit.client.0.smithi133.stderr:+ cd iogen_3.1p0 2024-09-11T23:38:11.830 INFO:tasks.workunit.client.0.smithi133.stdout:making iogen 2024-09-11T23:38:11.830 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 'making iogen' 2024-09-11T23:38:11.830 INFO:tasks.workunit.client.0.smithi133.stderr:+ make 2024-09-11T23:38:11.838 INFO:tasks.workunit.client.0.smithi133.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-09-11T23:38:12.059 INFO:tasks.workunit.client.0.smithi133.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-09-11T23:38:12.185 INFO:tasks.workunit.client.0.smithi133.stdout:running iogen 2024-09-11T23:38:12.185 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 'running iogen' 2024-09-11T23:38:12.186 INFO:tasks.workunit.client.0.smithi133.stderr:+ ./iogen -n 5 -s 2g 2024-09-11T23:38:12.186 INFO:tasks.workunit.client.0.smithi133.stderr:child spawned 88911 2024-09-11T23:38:12.186 INFO:tasks.workunit.client.0.smithi133.stderr:child spawned 88912 2024-09-11T23:38:12.186 INFO:tasks.workunit.client.0.smithi133.stderr:child spawned 88913 2024-09-11T23:38:12.186 INFO:tasks.workunit.client.0.smithi133.stderr:child spawned 88914 2024-09-11T23:38:12.186 INFO:tasks.workunit.client.0.smithi133.stderr:child spawned 88915 2024-09-11T23:38:12.186 INFO:tasks.workunit.client.0.smithi133.stdout:sleep for 10 min 2024-09-11T23:38:12.186 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 'sleep for 10 min' 2024-09-11T23:38:12.187 INFO:tasks.workunit.client.0.smithi133.stderr:+ sleep 600 2024-09-11T23:38:12.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:12 smithi138 ceph-mon[35142]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T23:38:12.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:12 smithi133 ceph-mon[31430]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T23:38:13.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:12 smithi178 ceph-mon[35882]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T23:38:14.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:14 smithi138 ceph-mon[35142]: pgmap v46: 97 pgs: 97 active+clean; 279 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 57 op/s 2024-09-11T23:38:14.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:14 smithi133 ceph-mon[31430]: pgmap v46: 97 pgs: 97 active+clean; 279 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 57 op/s 2024-09-11T23:38:15.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:14 smithi178 ceph-mon[35882]: pgmap v46: 97 pgs: 97 active+clean; 279 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 23 MiB/s wr, 57 op/s 2024-09-11T23:38:16.961 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:16 smithi133 ceph-mon[31430]: pgmap v47: 97 pgs: 97 active+clean; 315 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-11T23:38:17.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:16 smithi178 ceph-mon[35882]: pgmap v47: 97 pgs: 97 active+clean; 315 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-11T23:38:17.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:16 smithi138 ceph-mon[35142]: pgmap v47: 97 pgs: 97 active+clean; 315 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-09-11T23:38:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.214+0000 7f652ffff640 1 -- 172.21.15.133:0/3925777390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6530101170 msgr2=0x7f653010d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.214+0000 7f652ffff640 1 --2- 172.21.15.133:0/3925777390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6530101170 0x7f653010d6a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6524009900 tx=0x7f652402a420 comp rx=0 tx=0).stop 2024-09-11T23:38:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.215+0000 7f652ffff640 1 -- 172.21.15.133:0/3925777390 shutdown_connections 2024-09-11T23:38:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.215+0000 7f652ffff640 1 --2- 172.21.15.133:0/3925777390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6530101170 0x7f653010d6a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.215+0000 7f652ffff640 1 --2- 172.21.15.133:0/3925777390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65301007d0 0x7f6530100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.215+0000 7f652ffff640 1 --2- 172.21.15.133:0/3925777390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65301067e0 0x7f6530106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.215+0000 7f652ffff640 1 -- 172.21.15.133:0/3925777390 >> 172.21.15.133:0/3925777390 conn(0x7f65300777c0 msgr2=0x7f65300feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.233+0000 7f652ffff640 1 -- 172.21.15.133:0/3925777390 shutdown_connections 2024-09-11T23:38:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.233+0000 7f652ffff640 1 -- 172.21.15.133:0/3925777390 wait complete. 2024-09-11T23:38:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.235+0000 7f652ffff640 1 Processor -- start 2024-09-11T23:38:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.236+0000 7f652ffff640 1 -- start start 2024-09-11T23:38:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.236+0000 7f652ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65301007d0 0x7f6530198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.237+0000 7f652ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101170 0x7f6530198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.237+0000 7f652ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301067e0 0x7f653019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.237+0000 7f652ffff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6530111e00 con 0x7f65301007d0 2024-09-11T23:38:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.237+0000 7f652ffff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6530111c80 con 0x7f65301067e0 2024-09-11T23:38:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.237+0000 7f652ffff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6530111f80 con 0x7f6530101170 2024-09-11T23:38:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.237+0000 7f652e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101170 0x7f6530198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.237+0000 7f652e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101170 0x7f6530198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:36784/0 (socket says 172.21.15.133:36784) 2024-09-11T23:38:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.237+0000 7f652f7fe640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301067e0 0x7f653019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.237+0000 7f652e7fc640 1 -- 172.21.15.133:0/3467123798 learned_addr learned my addr 172.21.15.133:0/3467123798 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.241+0000 7f652f7fe640 1 -- 172.21.15.133:0/3467123798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101170 msgr2=0x7f6530198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.241+0000 7f652f7fe640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101170 0x7f6530198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.241+0000 7f652f7fe640 1 -- 172.21.15.133:0/3467123798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65301007d0 msgr2=0x7f6530198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.237+0000 7f652effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65301007d0 0x7f6530198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.246+0000 7f652f7fe640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65301007d0 0x7f6530198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.246+0000 7f652f7fe640 1 -- 172.21.15.133:0/3467123798 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653019d320 con 0x7f65301067e0 2024-09-11T23:38:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.246+0000 7f652effd640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65301007d0 0x7f6530198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:38:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.248+0000 7f652f7fe640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301067e0 0x7f653019cbf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6524009230 tx=0x7f6524003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.250+0000 7f6517fff640 1 -- 172.21.15.133:0/3467123798 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652403f070 con 0x7f65301067e0 2024-09-11T23:38:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.250+0000 7f6517fff640 1 -- 172.21.15.133:0/3467123798 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6524031e30 con 0x7f65301067e0 2024-09-11T23:38:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.250+0000 7f652ffff640 1 -- 172.21.15.133:0/3467123798 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653019d5b0 con 0x7f65301067e0 2024-09-11T23:38:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.251+0000 7f652ffff640 1 -- 172.21.15.133:0/3467123798 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f653019d9e0 con 0x7f65301067e0 2024-09-11T23:38:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.251+0000 7f6517fff640 1 -- 172.21.15.133:0/3467123798 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524035030 con 0x7f65301067e0 2024-09-11T23:38:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.255+0000 7f6517fff640 1 -- 172.21.15.133:0/3467123798 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f6524042070 con 0x7f65301067e0 2024-09-11T23:38:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.267+0000 7f6538225740 1 Processor -- start 2024-09-11T23:38:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.270+0000 7f6517fff640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65180a3070 0x7f65180a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.270+0000 7f6538225740 1 -- start start 2024-09-11T23:38:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.270+0000 7f6538225740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574627240 0x55b574625b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.270+0000 7f6538225740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574626040 0x55b5746264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.270+0000 7f6538225740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574639520 0x55b57463b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.270+0000 7f6538225740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b57458a2f0 con 0x55b574627240 2024-09-11T23:38:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.270+0000 7f6538225740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b574532270 con 0x55b574626040 2024-09-11T23:38:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.270+0000 7f6538225740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b5745862b0 con 0x55b574639520 2024-09-11T23:38:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.270+0000 7f6517fff640 1 -- 172.21.15.133:0/3467123798 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f65240f0ec0 con 0x7f65301067e0 2024-09-11T23:38:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.272+0000 7f652f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574639520 0x55b57463b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.272+0000 7f652f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574639520 0x55b57463b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:36792/0 (socket says 172.21.15.133:36792) 2024-09-11T23:38:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.272+0000 7f652f7fe640 1 -- 172.21.15.133:0/3881879147 learned_addr learned my addr 172.21.15.133:0/3881879147 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.274+0000 7f652f7fe640 1 -- 172.21.15.133:0/3881879147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574626040 msgr2=0x55b5746264c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:38:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.274+0000 7f652f7fe640 1 --2- 172.21.15.133:0/3881879147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574626040 0x55b5746264c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.274+0000 7f652f7fe640 1 -- 172.21.15.133:0/3881879147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574627240 msgr2=0x55b574625b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:38:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.274+0000 7f652f7fe640 1 --2- 172.21.15.133:0/3881879147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574627240 0x55b574625b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.274+0000 7f652f7fe640 1 -- 172.21.15.133:0/3881879147 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b574531340 con 0x55b574639520 2024-09-11T23:38:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.277+0000 7f652f7fe640 1 --2- 172.21.15.133:0/3881879147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574639520 0x55b57463b910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f652402a940 tx=0x7f65240bd0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75bb87bdfead37ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.277+0000 7f65157fa640 1 -- 172.21.15.133:0/3881879147 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65240d6070 con 0x55b574639520 2024-09-11T23:38:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.278+0000 7f65157fa640 1 -- 172.21.15.133:0/3881879147 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65240dbe50 con 0x55b574639520 2024-09-11T23:38:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.279+0000 7f6538225740 1 -- 172.21.15.133:0/3881879147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574639520 msgr2=0x55b57463b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.279+0000 7f6538225740 1 --2- 172.21.15.133:0/3881879147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574639520 0x55b57463b910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f652402a940 tx=0x7f65240bd0a0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.280+0000 7f6538225740 1 -- 172.21.15.133:0/3881879147 shutdown_connections 2024-09-11T23:38:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.280+0000 7f6538225740 1 --2- 172.21.15.133:0/3881879147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574639520 0x55b57463b910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.280+0000 7f6538225740 1 --2- 172.21.15.133:0/3881879147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574626040 0x55b5746264c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.280+0000 7f6538225740 1 --2- 172.21.15.133:0/3881879147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574627240 0x55b574625b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.280+0000 7f6538225740 1 -- 172.21.15.133:0/3881879147 >> 172.21.15.133:0/3881879147 conn(0x55b574622230 msgr2=0x55b574622610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.292+0000 7f652e7fc640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65180a3070 0x7f65180a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.296+0000 7f652e7fc640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65180a3070 0x7f65180a5530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6520006fd0 tx=0x7f6520008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.297+0000 7f6538225740 1 -- 172.21.15.133:0/3881879147 shutdown_connections 2024-09-11T23:38:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.297+0000 7f6538225740 1 -- 172.21.15.133:0/3881879147 wait complete. 2024-09-11T23:38:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.298+0000 7f6538225740 1 -- start start 2024-09-11T23:38:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.298+0000 7f6538225740 1 Processor -- start 2024-09-11T23:38:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.299+0000 7f6538225740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574626040 0x55b574632e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.299+0000 7f6538225740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574627240 0x55b574633340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.299+0000 7f6538225740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574639520 0x55b574638b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.299+0000 7f6538225740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b57462ce50 con 0x55b574627240 2024-09-11T23:38:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.299+0000 7f6538225740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b574513290 con 0x55b574639520 2024-09-11T23:38:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.299+0000 7f6538225740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b57462cfd0 con 0x55b574626040 2024-09-11T23:38:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.299+0000 7f6538225740 10 client.0 _ll_register_callbacks cb 0x55b574544140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.300+0000 7f652effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574626040 0x55b574632e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.300+0000 7f652effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574626040 0x55b574632e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:36794/0 (socket says 172.21.15.133:36794) 2024-09-11T23:38:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.300+0000 7f652effd640 1 -- 172.21.15.133:0/194564339 learned_addr learned my addr 172.21.15.133:0/194564339 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.300+0000 7f652f7fe640 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574639520 0x55b574638b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.302+0000 7f652effd640 1 -- 172.21.15.133:0/194564339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574639520 msgr2=0x55b574638b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.302+0000 7f652effd640 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574639520 0x55b574638b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.302+0000 7f652effd640 1 -- 172.21.15.133:0/194564339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574627240 msgr2=0x55b574633340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.302+0000 7f652effd640 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574627240 0x55b574633340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.302+0000 7f652effd640 1 -- 172.21.15.133:0/194564339 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57429d210 con 0x55b574626040 2024-09-11T23:38:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.303+0000 7f652effd640 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574626040 0x55b574632e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f652800c8e0 tx=0x7f652800cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d59518450887be84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.303+0000 7f6512ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:38:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.304+0000 7f6512ff5640 1 -- 172.21.15.133:0/194564339 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652800ac30 con 0x55b574626040 2024-09-11T23:38:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.304+0000 7f6538225740 1 -- 172.21.15.133:0/194564339 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65240dbe50 con 0x55b574626040 2024-09-11T23:38:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.304+0000 7f6512ff5640 1 -- 172.21.15.133:0/194564339 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f652800add0 con 0x55b574626040 2024-09-11T23:38:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.305+0000 7f6512ff5640 1 -- 172.21.15.133:0/194564339 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f652800f480 con 0x55b574626040 2024-09-11T23:38:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.309+0000 7f6538225740 10 client.25432 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.309+0000 7f6538225740 1 -- 172.21.15.133:0/194564339 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b574369620 con 0x55b574626040 2024-09-11T23:38:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.311+0000 7f6512ff5640 1 -- 172.21.15.133:0/194564339 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65240dbe50 con 0x55b574626040 2024-09-11T23:38:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.311+0000 7f6512ff5640 1 -- 172.21.15.133:0/194564339 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6528020020 con 0x55b574626040 2024-09-11T23:38:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.311+0000 7f6538225740 10 client.25432 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.311+0000 7f6538225740 10 client.25432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.311+0000 7f6538225740 20 client.25432 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.311+0000 7f6538225740 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b574663040 0x55b5746654e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.311+0000 7f6538225740 4 client.25432 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.311+0000 7f6538225740 1 -- 172.21.15.133:0/194564339 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b574597a20 con 0x55b574663040 2024-09-11T23:38:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.313+0000 7f652e7fc640 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b574663040 0x55b5746654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.320+0000 7f652e7fc640 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b574663040 0x55b5746654e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80628a80895e19e1 server_cookie=159c8f59dc9cd5d6 in_seq=0 out_seq=0 2024-09-11T23:38:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.320+0000 7f6512ff5640 10 client.25432 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.324+0000 7f6512ff5640 1 -- 172.21.15.133:0/194564339 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b574597a20 con 0x55b574663040 2024-09-11T23:38:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.324+0000 7f6512ff5640 10 client.25432 handle_command_reply: tid=0 2024-09-11T23:38:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.324+0000 7f6538225740 1 client.25432 shutdown 2024-09-11T23:38:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6538225740 1 -- 172.21.15.133:0/194564339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574626040 msgr2=0x55b574632e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6538225740 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574626040 0x55b574632e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f652800c8e0 tx=0x7f652800cda0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6538225740 1 -- 172.21.15.133:0/194564339 shutdown_connections 2024-09-11T23:38:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6538225740 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b574663040 0x55b5746654e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6538225740 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574639520 0x55b574638b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6538225740 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574627240 0x55b574633340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6538225740 1 --2- 172.21.15.133:0/194564339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574626040 0x55b574632e00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6538225740 1 -- 172.21.15.133:0/194564339 >> 172.21.15.133:0/194564339 conn(0x55b574622230 msgr2=0x55b574627e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6512ff5640 0 client.25432 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6538225740 1 -- 172.21.15.133:0/194564339 shutdown_connections 2024-09-11T23:38:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6538225740 1 -- 172.21.15.133:0/194564339 wait complete. 2024-09-11T23:38:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.326+0000 7f6538225740 20 client.25432 trim_cache size 0 max 16384 2024-09-11T23:38:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.354+0000 7f6538225740 1 Processor -- start 2024-09-11T23:38:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f6538225740 1 -- start start 2024-09-11T23:38:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f6538225740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574627240 0x55b574621150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f6538225740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574639520 0x55b57466adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f6538225740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b57462c880 0x55b57466b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f6538225740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f651c00e740 con 0x55b57462c880 2024-09-11T23:38:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f6538225740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f651c010280 con 0x55b574627240 2024-09-11T23:38:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f6538225740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f651c00ee10 con 0x55b574639520 2024-09-11T23:38:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f652f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b57462c880 0x55b57466b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f652f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b57462c880 0x55b57466b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37182/0 (socket says 172.21.15.133:37182) 2024-09-11T23:38:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f652f7fe640 1 -- 172.21.15.133:0/648869372 learned_addr learned my addr 172.21.15.133:0/648869372 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f652f7fe640 1 -- 172.21.15.133:0/648869372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574639520 msgr2=0x55b57466adb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f652f7fe640 1 --2- 172.21.15.133:0/648869372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574639520 0x55b57466adb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f652f7fe640 1 -- 172.21.15.133:0/648869372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574627240 msgr2=0x55b574621150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f652f7fe640 1 --2- 172.21.15.133:0/648869372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574627240 0x55b574621150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.355+0000 7f652f7fe640 1 -- 172.21.15.133:0/648869372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5746264c0 con 0x55b57462c880 2024-09-11T23:38:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.356+0000 7f652f7fe640 1 --2- 172.21.15.133:0/648869372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b57462c880 0x55b57466b2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f65240095d0 tx=0x7f6524009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9dd2dfa687a4c58 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.356+0000 7f6514ff9640 1 -- 172.21.15.133:0/648869372 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65240d6040 con 0x55b57462c880 2024-09-11T23:38:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.356+0000 7f6514ff9640 1 -- 172.21.15.133:0/648869372 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65240dbe50 con 0x55b57462c880 2024-09-11T23:38:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.356+0000 7f6514ff9640 1 -- 172.21.15.133:0/648869372 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65240c59f0 con 0x55b57462c880 2024-09-11T23:38:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.357+0000 7f6538225740 1 -- 172.21.15.133:0/648869372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b57462c880 msgr2=0x55b57466b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.357+0000 7f6538225740 1 --2- 172.21.15.133:0/648869372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b57462c880 0x55b57466b2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f65240095d0 tx=0x7f6524009560 comp rx=0 tx=0).stop 2024-09-11T23:38:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.357+0000 7f6538225740 1 -- 172.21.15.133:0/648869372 shutdown_connections 2024-09-11T23:38:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.357+0000 7f6538225740 1 --2- 172.21.15.133:0/648869372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b57462c880 0x55b57466b2f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.357+0000 7f6538225740 1 --2- 172.21.15.133:0/648869372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574639520 0x55b57466adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.357+0000 7f6538225740 1 --2- 172.21.15.133:0/648869372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b574627240 0x55b574621150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.357+0000 7f6538225740 1 -- 172.21.15.133:0/648869372 >> 172.21.15.133:0/648869372 conn(0x55b574622230 msgr2=0x55b574620a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.357+0000 7f6538225740 1 -- 172.21.15.133:0/648869372 shutdown_connections 2024-09-11T23:38:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.357+0000 7f6538225740 1 -- 172.21.15.133:0/648869372 wait complete. 2024-09-11T23:38:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.357+0000 7f6538225740 1 -- start start 2024-09-11T23:38:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.357+0000 7f6538225740 1 Processor -- start 2024-09-11T23:38:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.358+0000 7f6538225740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574627240 0x55b57462a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.358+0000 7f6538225740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b57462c880 0x55b57462ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.358+0000 7f6538225740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574639520 0x55b57462b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.358+0000 7f6538225740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f651c00fbb0 con 0x55b574639520 2024-09-11T23:38:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.358+0000 7f6538225740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f651c00f4e0 con 0x55b57462c880 2024-09-11T23:38:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.358+0000 7f6538225740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f651c00c720 con 0x55b574627240 2024-09-11T23:38:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.358+0000 7f6538225740 10 client.0 _ll_register_callbacks cb 0x7f652800c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.358+0000 7f652f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574639520 0x55b57462b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.358+0000 7f652f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574639520 0x55b57462b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37198/0 (socket says 172.21.15.133:37198) 2024-09-11T23:38:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.358+0000 7f652f7fe640 1 -- 172.21.15.133:0/294084034 learned_addr learned my addr 172.21.15.133:0/294084034 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.358+0000 7f652effd640 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574627240 0x55b57462a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.359+0000 7f652e7fc640 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b57462c880 0x55b57462ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.359+0000 7f652f7fe640 1 -- 172.21.15.133:0/294084034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574627240 msgr2=0x55b57462a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.359+0000 7f652f7fe640 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574627240 0x55b57462a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.359+0000 7f652f7fe640 1 -- 172.21.15.133:0/294084034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b57462c880 msgr2=0x55b57462ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.359+0000 7f652f7fe640 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b57462c880 0x55b57462ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.359+0000 7f652f7fe640 1 -- 172.21.15.133:0/294084034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5746ccc50 con 0x55b574639520 2024-09-11T23:38:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.359+0000 7f652f7fe640 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574639520 0x55b57462b250 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6524009900 tx=0x7f65240c1170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58adb37f05114145 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.359+0000 7f6512ff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.359+0000 7f6512ff5640 1 -- 172.21.15.133:0/294084034 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65240d6040 con 0x55b574639520 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.359+0000 7f6512ff5640 1 -- 172.21.15.133:0/294084034 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f65240dd2b0 con 0x55b574639520 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.359+0000 7f6538225740 1 -- 172.21.15.133:0/294084034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65240dbe50 con 0x55b574639520 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.360+0000 7f6512ff5640 1 -- 172.21.15.133:0/294084034 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65240c59f0 con 0x55b574639520 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.360+0000 7f6538225740 10 client.15993 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.360+0000 7f6538225740 1 -- 172.21.15.133:0/294084034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5746cdf40 con 0x55b574639520 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.360+0000 7f6512ff5640 1 -- 172.21.15.133:0/294084034 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65240dbe50 con 0x55b574639520 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.360+0000 7f6512ff5640 1 -- 172.21.15.133:0/294084034 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f65240b8f60 con 0x55b574639520 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.360+0000 7f6538225740 10 client.15993 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.360+0000 7f6538225740 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.360+0000 7f6538225740 20 client.15993 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.360+0000 7f6538225740 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b5746ce0a0 0x55b5746d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.360+0000 7f6538225740 4 client.15993 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:38:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.361+0000 7f6538225740 1 -- 172.21.15.133:0/294084034 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b574591d70 con 0x55b5746ce0a0 2024-09-11T23:38:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.361+0000 7f652effd640 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b5746ce0a0 0x55b5746d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.366+0000 7f6512ff5640 10 client.15993 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.366+0000 7f652effd640 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b5746ce0a0 0x55b5746d04f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5fb9c805bd3c681 server_cookie=7c0c403aa06a898b in_seq=0 out_seq=0 2024-09-11T23:38:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.436+0000 7f6512ff5640 1 -- 172.21.15.133:0/294084034 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b574591d70 con 0x55b5746ce0a0 2024-09-11T23:38:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.436+0000 7f6512ff5640 10 client.15993 handle_command_reply: tid=0 2024-09-11T23:38:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.436+0000 7f6538225740 1 client.15993 shutdown 2024-09-11T23:38:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.437+0000 7f6538225740 1 -- 172.21.15.133:0/294084034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574639520 msgr2=0x55b57462b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.437+0000 7f6538225740 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574639520 0x55b57462b250 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6524009900 tx=0x7f65240c1170 comp rx=0 tx=0).stop 2024-09-11T23:38:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.437+0000 7f6538225740 1 -- 172.21.15.133:0/294084034 shutdown_connections 2024-09-11T23:38:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.437+0000 7f6538225740 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b5746ce0a0 0x55b5746d04f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.437+0000 7f6538225740 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b574639520 0x55b57462b250 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.437+0000 7f6538225740 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b57462c880 0x55b57462ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.437+0000 7f6538225740 1 --2- 172.21.15.133:0/294084034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b574627240 0x55b57462a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.437+0000 7f6538225740 1 -- 172.21.15.133:0/294084034 >> 172.21.15.133:0/294084034 conn(0x55b574622230 msgr2=0x55b574621260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.437+0000 7f6512ff5640 0 client.15993 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.438+0000 7f6538225740 1 -- 172.21.15.133:0/294084034 shutdown_connections 2024-09-11T23:38:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.439+0000 7f6538225740 1 -- 172.21.15.133:0/294084034 wait complete. 2024-09-11T23:38:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.439+0000 7f6538225740 20 client.15993 trim_cache size 0 max 16384 2024-09-11T23:38:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:38:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:38:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:38:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:38:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.443+0000 7f652ffff640 1 -- 172.21.15.133:0/3467123798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65180a3070 msgr2=0x7f65180a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.443+0000 7f652ffff640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65180a3070 0x7f65180a5530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6520006fd0 tx=0x7f6520008040 comp rx=0 tx=0).stop 2024-09-11T23:38:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.443+0000 7f652ffff640 1 -- 172.21.15.133:0/3467123798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301067e0 msgr2=0x7f653019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.443+0000 7f652ffff640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301067e0 0x7f653019cbf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6524009230 tx=0x7f6524003c00 comp rx=0 tx=0).stop 2024-09-11T23:38:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.443+0000 7f652ffff640 1 -- 172.21.15.133:0/3467123798 shutdown_connections 2024-09-11T23:38:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.444+0000 7f652ffff640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f65180a3070 0x7f65180a5530 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.444+0000 7f652ffff640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301067e0 0x7f653019cbf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.444+0000 7f652ffff640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101170 0x7f6530198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.444+0000 7f652ffff640 1 --2- 172.21.15.133:0/3467123798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65301007d0 0x7f6530198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.444+0000 7f652ffff640 1 -- 172.21.15.133:0/3467123798 >> 172.21.15.133:0/3467123798 conn(0x7f65300777c0 msgr2=0x7f6530103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.444+0000 7f652ffff640 1 -- 172.21.15.133:0/3467123798 shutdown_connections 2024-09-11T23:38:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:18.444+0000 7f652ffff640 1 -- 172.21.15.133:0/3467123798 wait complete. 2024-09-11T23:38:19.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:18 smithi178 ceph-mon[35882]: pgmap v48: 97 pgs: 97 active+clean; 315 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-11T23:38:19.077 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:18 smithi133 ceph-mon[31430]: pgmap v48: 97 pgs: 97 active+clean; 315 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-11T23:38:19.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:18 smithi138 ceph-mon[35142]: pgmap v48: 97 pgs: 97 active+clean; 315 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-11T23:38:21.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:20 smithi138 ceph-mon[35142]: pgmap v49: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 381 MiB/s wr, 301 op/s 2024-09-11T23:38:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:20 smithi138 ceph-mon[35142]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-11T23:38:21.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:20 smithi133 ceph-mon[31430]: pgmap v49: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 381 MiB/s wr, 301 op/s 2024-09-11T23:38:21.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:20 smithi133 ceph-mon[31430]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-11T23:38:21.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:20 smithi178 ceph-mon[35882]: pgmap v49: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 381 MiB/s wr, 301 op/s 2024-09-11T23:38:21.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:20 smithi178 ceph-mon[35882]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-11T23:38:23.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:22 smithi138 ceph-mon[35142]: pgmap v50: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 381 MiB/s wr, 295 op/s 2024-09-11T23:38:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:22 smithi133 ceph-mon[31430]: pgmap v50: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 381 MiB/s wr, 295 op/s 2024-09-11T23:38:23.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:22 smithi178 ceph-mon[35882]: pgmap v50: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 381 MiB/s wr, 295 op/s 2024-09-11T23:38:25.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:24 smithi133 ceph-mon[31430]: pgmap v51: 97 pgs: 97 active+clean; 7.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 669 MiB/s wr, 1.03k op/s 2024-09-11T23:38:25.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:24 smithi178 ceph-mon[35882]: pgmap v51: 97 pgs: 97 active+clean; 7.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 669 MiB/s wr, 1.03k op/s 2024-09-11T23:38:25.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:24 smithi138 ceph-mon[35142]: pgmap v51: 97 pgs: 97 active+clean; 7.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 669 MiB/s wr, 1.03k op/s 2024-09-11T23:38:26.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:26 smithi138 ceph-mon[35142]: pgmap v52: 97 pgs: 97 active+clean; 7.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 701 MiB/s wr, 1.14k op/s 2024-09-11T23:38:26.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:26 smithi133 ceph-mon[31430]: pgmap v52: 97 pgs: 97 active+clean; 7.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 701 MiB/s wr, 1.14k op/s 2024-09-11T23:38:26.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:26 smithi178 ceph-mon[35882]: pgmap v52: 97 pgs: 97 active+clean; 7.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 701 MiB/s wr, 1.14k op/s 2024-09-11T23:38:27.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:27 smithi133 ceph-mon[31430]: pgmap v53: 97 pgs: 97 active+clean; 7.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 698 MiB/s wr, 1.14k op/s 2024-09-11T23:38:27.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:27 smithi178 ceph-mon[35882]: pgmap v53: 97 pgs: 97 active+clean; 7.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 698 MiB/s wr, 1.14k op/s 2024-09-11T23:38:27.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:27 smithi138 ceph-mon[35142]: pgmap v53: 97 pgs: 97 active+clean; 7.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 698 MiB/s wr, 1.14k op/s 2024-09-11T23:38:29.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:29 smithi178 ceph-mon[35882]: pgmap v54: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 GiB/s wr, 1.88k op/s 2024-09-11T23:38:29.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:29 smithi138 ceph-mon[35142]: pgmap v54: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 GiB/s wr, 1.88k op/s 2024-09-11T23:38:29.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:29 smithi133 ceph-mon[31430]: pgmap v54: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 GiB/s wr, 1.88k op/s 2024-09-11T23:38:31.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:31 smithi138 ceph-mon[35142]: pgmap v55: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 680 MiB/s wr, 1.61k op/s 2024-09-11T23:38:31.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:31 smithi133 ceph-mon[31430]: pgmap v55: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 680 MiB/s wr, 1.61k op/s 2024-09-11T23:38:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.717+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1058973516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b41095d0 msgr2=0x7fd4b4109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.717+0000 7fd4b3fff640 1 --2- 172.21.15.133:0/1058973516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b41095d0 0x7fd4b4109a30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009900 tx=0x7fd4a802a440 comp rx=0 tx=0).stop 2024-09-11T23:38:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.719+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1058973516 shutdown_connections 2024-09-11T23:38:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.719+0000 7fd4b3fff640 1 --2- 172.21.15.133:0/1058973516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b411b140 0x7fd4b411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.719+0000 7fd4b3fff640 1 --2- 172.21.15.133:0/1058973516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b41095d0 0x7fd4b4109a30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.719+0000 7fd4b3fff640 1 --2- 172.21.15.133:0/1058973516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41091f0 0x7fd4b4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.719+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1058973516 >> 172.21.15.133:0/1058973516 conn(0x7fd4b406cbf0 msgr2=0x7fd4b406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.746+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1058973516 shutdown_connections 2024-09-11T23:38:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.747+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1058973516 wait complete. 2024-09-11T23:38:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.749+0000 7fd4b3fff640 1 Processor -- start 2024-09-11T23:38:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.750+0000 7fd4b3fff640 1 -- start start 2024-09-11T23:38:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.750+0000 7fd4b3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b41091f0 0x7fd4b41a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.750+0000 7fd4b3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41095d0 0x7fd4b41a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.750+0000 7fd4b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b411b140 0x7fd4b41abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.750+0000 7fd4b3fff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4120c10 con 0x7fd4b411b140 2024-09-11T23:38:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.750+0000 7fd4b3fff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4120a90 con 0x7fd4b41095d0 2024-09-11T23:38:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.750+0000 7fd4b3fff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4120d90 con 0x7fd4b41091f0 2024-09-11T23:38:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.751+0000 7fd4b2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b41091f0 0x7fd4b41a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.751+0000 7fd4b2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b41091f0 0x7fd4b41a71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:47972/0 (socket says 172.21.15.133:47972) 2024-09-11T23:38:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.751+0000 7fd4b2ffd640 1 -- 172.21.15.133:0/1332434190 learned_addr learned my addr 172.21.15.133:0/1332434190 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.751+0000 7fd4b37fe640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b411b140 0x7fd4b41abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.752+0000 7fd4b27fc640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41095d0 0x7fd4b41a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.753+0000 7fd4b27fc640 1 -- 172.21.15.133:0/1332434190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b41091f0 msgr2=0x7fd4b41a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.753+0000 7fd4b27fc640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b41091f0 0x7fd4b41a71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.753+0000 7fd4b27fc640 1 -- 172.21.15.133:0/1332434190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b411b140 msgr2=0x7fd4b41abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.753+0000 7fd4b27fc640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b411b140 0x7fd4b41abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.754+0000 7fd4b27fc640 1 -- 172.21.15.133:0/1332434190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b41ac1a0 con 0x7fd4b41095d0 2024-09-11T23:38:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.755+0000 7fd4b27fc640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41095d0 0x7fd4b41a7720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009850 tx=0x7fd4a8003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.765+0000 7fd49bfff640 1 -- 172.21.15.133:0/1332434190 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a803f070 con 0x7fd4b41095d0 2024-09-11T23:38:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.766+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1332434190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b41ac430 con 0x7fd4b41095d0 2024-09-11T23:38:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.766+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1332434190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b4110920 con 0x7fd4b41095d0 2024-09-11T23:38:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.776+0000 7fd49bfff640 1 -- 172.21.15.133:0/1332434190 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4a803b460 con 0x7fd4b41095d0 2024-09-11T23:38:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.776+0000 7fd49bfff640 1 -- 172.21.15.133:0/1332434190 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8035030 con 0x7fd4b41095d0 2024-09-11T23:38:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.779+0000 7fd4bc5d9740 1 Processor -- start 2024-09-11T23:38:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.780+0000 7fd4bc5d9740 1 -- start start 2024-09-11T23:38:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.780+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283d5cc0 0x55e4283d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.780+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283e0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.780+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 0x55e4283e3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.780+0000 7fd4bc5d9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e42833a2f0 con 0x55e4283d5cc0 2024-09-11T23:38:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.780+0000 7fd4bc5d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e4282e2270 con 0x55e4283d65e0 2024-09-11T23:38:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.780+0000 7fd4bc5d9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e4283362b0 con 0x55e4283e0e80 2024-09-11T23:38:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.780+0000 7fd49bfff640 1 -- 172.21.15.133:0/1332434190 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd4a8042070 con 0x7fd4b41095d0 2024-09-11T23:38:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.781+0000 7fd4b27fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 0x55e4283e3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.781+0000 7fd4b27fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 0x55e4283e3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:47980/0 (socket says 172.21.15.133:47980) 2024-09-11T23:38:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.781+0000 7fd4b27fc640 1 -- 172.21.15.133:0/865204464 learned_addr learned my addr 172.21.15.133:0/865204464 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.782+0000 7fd4b2ffd640 1 --2- 172.21.15.133:0/865204464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283e0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:31 smithi178 ceph-mon[35882]: pgmap v55: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 680 MiB/s wr, 1.61k op/s 2024-09-11T23:38:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.786+0000 7fd4b37fe640 1 --2- 172.21.15.133:0/865204464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283d5cc0 0x55e4283d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.792+0000 7fd49bfff640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd49c0a2f00 0x7fd49c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.793+0000 7fd4b2ffd640 1 -- 172.21.15.133:0/865204464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 msgr2=0x55e4283e3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.793+0000 7fd4b2ffd640 1 --2- 172.21.15.133:0/865204464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 0x55e4283e3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.793+0000 7fd4b2ffd640 1 -- 172.21.15.133:0/865204464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283d5cc0 msgr2=0x55e4283d60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.793+0000 7fd4b2ffd640 1 --2- 172.21.15.133:0/865204464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283d5cc0 0x55e4283d60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.793+0000 7fd4b2ffd640 1 -- 172.21.15.133:0/865204464 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4282e1340 con 0x55e4283d65e0 2024-09-11T23:38:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.794+0000 7fd49bfff640 1 -- 172.21.15.133:0/1332434190 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fd4a80f0c30 con 0x7fd4b41095d0 2024-09-11T23:38:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.797+0000 7fd4b37fe640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd49c0a2f00 0x7fd49c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.797+0000 7fd4b2ffd640 1 --2- 172.21.15.133:0/865204464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283e0940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00e190 tx=0x7fd4ac009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3b9d2e12aa3714b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.798+0000 7fd4997fa640 1 -- 172.21.15.133:0/865204464 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac018020 con 0x55e4283d65e0 2024-09-11T23:38:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.798+0000 7fd4997fa640 1 -- 172.21.15.133:0/865204464 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4ac014b60 con 0x55e4283d65e0 2024-09-11T23:38:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.798+0000 7fd4997fa640 1 -- 172.21.15.133:0/865204464 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac013d20 con 0x55e4283d65e0 2024-09-11T23:38:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.802+0000 7fd4b37fe640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd49c0a2f00 0x7fd49c0a53c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400b8c0 tx=0x7fd4a4004770 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.807+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/865204464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 msgr2=0x55e4283e0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.807+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/865204464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283e0940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00e190 tx=0x7fd4ac009770 comp rx=0 tx=0).stop 2024-09-11T23:38:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.810+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/865204464 shutdown_connections 2024-09-11T23:38:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.810+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/865204464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 0x55e4283e3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.810+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/865204464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283e0940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.810+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/865204464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283d5cc0 0x55e4283d60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.810+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/865204464 >> 172.21.15.133:0/865204464 conn(0x55e4283d2230 msgr2=0x55e4283d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.810+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/865204464 shutdown_connections 2024-09-11T23:38:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.810+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/865204464 wait complete. 2024-09-11T23:38:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.811+0000 7fd4bc5d9740 1 -- start start 2024-09-11T23:38:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.812+0000 7fd4bc5d9740 1 Processor -- start 2024-09-11T23:38:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.813+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283d5cc0 0x55e4283ddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.814+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283de260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.814+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283e0e80 0x55e4283de7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.814+0000 7fd4bc5d9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e4283e3a50 con 0x55e4283e0e80 2024-09-11T23:38:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.814+0000 7fd4bc5d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e4282c3290 con 0x55e4283d65e0 2024-09-11T23:38:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.814+0000 7fd4bc5d9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e4283e3bd0 con 0x55e4283d5cc0 2024-09-11T23:38:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.815+0000 7fd4bc5d9740 10 client.0 _ll_register_callbacks cb 0x55e4282f4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.817+0000 7fd4b27fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283de260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.817+0000 7fd4b27fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283de260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48038/0 (socket says 172.21.15.133:48038) 2024-09-11T23:38:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.817+0000 7fd4b27fc640 1 -- 172.21.15.133:0/1065288840 learned_addr learned my addr 172.21.15.133:0/1065288840 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.818+0000 7fd4b2ffd640 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283d5cc0 0x55e4283ddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.819+0000 7fd4b27fc640 1 -- 172.21.15.133:0/1065288840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283d5cc0 msgr2=0x55e4283ddd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.819+0000 7fd4b27fc640 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283d5cc0 0x55e4283ddd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.819+0000 7fd4b27fc640 1 -- 172.21.15.133:0/1065288840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283e0e80 msgr2=0x55e4283de7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.819+0000 7fd4b27fc640 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283e0e80 0x55e4283de7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.819+0000 7fd4b27fc640 1 -- 172.21.15.133:0/1065288840 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e42804d210 con 0x55e4283d65e0 2024-09-11T23:38:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.819+0000 7fd4b27fc640 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283de260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80b9c20 tx=0x7fd4a8038000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d63e2df1bf95980 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.830+0000 7fd496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:38:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.830+0000 7fd496ff5640 1 -- 172.21.15.133:0/1065288840 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80bcd90 con 0x55e4283d65e0 2024-09-11T23:38:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.830+0000 7fd496ff5640 1 -- 172.21.15.133:0/1065288840 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fd4a80b98c0 con 0x55e4283d65e0 2024-09-11T23:38:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.830+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1065288840 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4ac014b60 con 0x55e4283d65e0 2024-09-11T23:38:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.834+0000 7fd496ff5640 1 -- 172.21.15.133:0/1065288840 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd4a80bcbb0 con 0x55e4283d65e0 2024-09-11T23:38:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.834+0000 7fd4bc5d9740 10 client.25421 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.834+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1065288840 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e428119620 con 0x55e4283d65e0 2024-09-11T23:38:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.836+0000 7fd496ff5640 1 -- 172.21.15.133:0/1065288840 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4ac014b60 con 0x55e4283d65e0 2024-09-11T23:38:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.836+0000 7fd496ff5640 1 -- 172.21.15.133:0/1065288840 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd4a80bd4b0 con 0x55e4283d65e0 2024-09-11T23:38:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.836+0000 7fd4bc5d9740 10 client.25421 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.836+0000 7fd4bc5d9740 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.836+0000 7fd4bc5d9740 20 client.25421 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.836+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e4283f0e10 0x55e4283f3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.836+0000 7fd4bc5d9740 4 client.25421 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.836+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1065288840 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e428347a20 con 0x55e4283f0e10 2024-09-11T23:38:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.837+0000 7fd4b2ffd640 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e4283f0e10 0x55e4283f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.839+0000 7fd496ff5640 10 client.25421 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.839+0000 7fd4b2ffd640 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e4283f0e10 0x55e4283f3290 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98848717e772bced server_cookie=fb3d774f13d5ff54 in_seq=0 out_seq=0 2024-09-11T23:38:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.842+0000 7fd496ff5640 1 -- 172.21.15.133:0/1065288840 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e428347a20 con 0x55e4283f0e10 2024-09-11T23:38:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.842+0000 7fd496ff5640 10 client.25421 handle_command_reply: tid=0 2024-09-11T23:38:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.843+0000 7fd4bc5d9740 1 client.25421 shutdown 2024-09-11T23:38:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.844+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1065288840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 msgr2=0x55e4283de260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.844+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283de260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80b9c20 tx=0x7fd4a8038000 comp rx=0 tx=0).stop 2024-09-11T23:38:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.845+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1065288840 shutdown_connections 2024-09-11T23:38:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.845+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e4283f0e10 0x55e4283f3290 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.845+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283e0e80 0x55e4283de7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.845+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283de260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.845+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1065288840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283d5cc0 0x55e4283ddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.845+0000 7fd496ff5640 0 client.25421 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.845+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1065288840 >> 172.21.15.133:0/1065288840 conn(0x55e4283d2230 msgr2=0x55e4283d7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.845+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1065288840 shutdown_connections 2024-09-11T23:38:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.845+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1065288840 wait complete. 2024-09-11T23:38:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.846+0000 7fd4bc5d9740 20 client.25421 trim_cache size 0 max 16384 2024-09-11T23:38:31.917 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.916+0000 7fd4bc5d9740 1 Processor -- start 2024-09-11T23:38:31.917 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.917+0000 7fd4bc5d9740 1 -- start start 2024-09-11T23:38:31.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.918+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283d5cc0 0x55e4283d8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.918+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.918+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283e0e80 0x55e4283d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.919 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.918+0000 7fd4b2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283d5cc0 0x55e4283d8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.918+0000 7fd4b2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283d5cc0 0x55e4283d8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:47998/0 (socket says 172.21.15.133:47998) 2024-09-11T23:38:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.919+0000 7fd4b2ffd640 1 -- 172.21.15.133:0/3724837533 learned_addr learned my addr 172.21.15.133:0/3724837533 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.919+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/3724837533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a000e680 con 0x55e4283e0e80 2024-09-11T23:38:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.919+0000 7fd4b27fc640 1 --2- 172.21.15.133:0/3724837533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.919+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/3724837533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a00101c0 con 0x55e4283d65e0 2024-09-11T23:38:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.919+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/3724837533 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a000ed50 con 0x55e4283d5cc0 2024-09-11T23:38:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.920+0000 7fd4b37fe640 1 --2- 172.21.15.133:0/3724837533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283e0e80 0x55e4283d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.924+0000 7fd4b27fc640 1 -- 172.21.15.133:0/3724837533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283d5cc0 msgr2=0x55e4283d8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.924+0000 7fd4b27fc640 1 --2- 172.21.15.133:0/3724837533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283d5cc0 0x55e4283d8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.924+0000 7fd4b27fc640 1 -- 172.21.15.133:0/3724837533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283e0e80 msgr2=0x55e4283d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.924+0000 7fd4b27fc640 1 --2- 172.21.15.133:0/3724837533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283e0e80 0x55e4283d1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.924+0000 7fd4b27fc640 1 -- 172.21.15.133:0/3724837533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4283d07b0 con 0x55e4283d65e0 2024-09-11T23:38:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.930+0000 7fd4b27fc640 1 --2- 172.21.15.133:0/3724837533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283de5d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4a802aa30 tx=0x7fd4a80c6020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f921198bf4fba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.931+0000 7fd498ff9640 1 -- 172.21.15.133:0/3724837533 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80ee820 con 0x55e4283d65e0 2024-09-11T23:38:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.932+0000 7fd498ff9640 1 -- 172.21.15.133:0/3724837533 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4a80bd950 con 0x55e4283d65e0 2024-09-11T23:38:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.933+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/3724837533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 msgr2=0x55e4283de5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.933+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/3724837533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283de5d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4a802aa30 tx=0x7fd4a80c6020 comp rx=0 tx=0).stop 2024-09-11T23:38:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.943+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/3724837533 shutdown_connections 2024-09-11T23:38:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.943+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/3724837533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283e0e80 0x55e4283d1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.943+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/3724837533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283de5d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.943+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/3724837533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283d5cc0 0x55e4283d8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.943+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/3724837533 >> 172.21.15.133:0/3724837533 conn(0x55e4283d2230 msgr2=0x55e4283d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.955+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/3724837533 shutdown_connections 2024-09-11T23:38:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.958+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/3724837533 wait complete. 2024-09-11T23:38:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.959+0000 7fd4bc5d9740 1 -- start start 2024-09-11T23:38:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.959+0000 7fd4bc5d9740 1 Processor -- start 2024-09-11T23:38:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.960+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283d5cc0 0x55e4283e9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.960+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283da9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.960+0000 7fd4bc5d9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 0x55e4283daee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.960+0000 7fd4bc5d9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a000faf0 con 0x55e4283d5cc0 2024-09-11T23:38:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.960+0000 7fd4bc5d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a000f420 con 0x55e4283d65e0 2024-09-11T23:38:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.960+0000 7fd4bc5d9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a000c660 con 0x55e4283e0e80 2024-09-11T23:38:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.961+0000 7fd4bc5d9740 10 client.0 _ll_register_callbacks cb 0x7fd4a8003f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.961+0000 7fd4b37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 0x55e4283daee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.961+0000 7fd4b37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 0x55e4283daee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48012/0 (socket says 172.21.15.133:48012) 2024-09-11T23:38:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.961+0000 7fd4b37fe640 1 -- 172.21.15.133:0/1531125313 learned_addr learned my addr 172.21.15.133:0/1531125313 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.961+0000 7fd4b37fe640 1 -- 172.21.15.133:0/1531125313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 msgr2=0x55e4283da9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.961+0000 7fd4b37fe640 1 --2- 172.21.15.133:0/1531125313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283da9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.961+0000 7fd4b37fe640 1 -- 172.21.15.133:0/1531125313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283d5cc0 msgr2=0x55e4283e9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.961+0000 7fd4b37fe640 1 --2- 172.21.15.133:0/1531125313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283d5cc0 0x55e4283e9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.961+0000 7fd4b37fe640 1 -- 172.21.15.133:0/1531125313 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4283db710 con 0x55e4283e0e80 2024-09-11T23:38:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.962+0000 7fd4b37fe640 1 --2- 172.21.15.133:0/1531125313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 0x55e4283daee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40146a0 tx=0x7fd4a4014aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3286f7bdabca4c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.962+0000 7fd496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:38:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.962+0000 7fd496ff5640 1 -- 172.21.15.133:0/1531125313 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4025070 con 0x55e4283e0e80 2024-09-11T23:38:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.962+0000 7fd496ff5640 1 -- 172.21.15.133:0/1531125313 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fd4a4013a10 con 0x55e4283e0e80 2024-09-11T23:38:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.966+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1531125313 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a80bd950 con 0x55e4283e0e80 2024-09-11T23:38:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.968+0000 7fd496ff5640 1 -- 172.21.15.133:0/1531125313 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd4a402f050 con 0x55e4283e0e80 2024-09-11T23:38:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.968+0000 7fd4bc5d9740 10 client.25450 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.968+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1531125313 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e42845bb10 con 0x55e4283e0e80 2024-09-11T23:38:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.969+0000 7fd496ff5640 1 -- 172.21.15.133:0/1531125313 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4a80bd950 con 0x55e4283e0e80 2024-09-11T23:38:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.971+0000 7fd496ff5640 1 -- 172.21.15.133:0/1531125313 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd4a4021400 con 0x55e4283e0e80 2024-09-11T23:38:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.972+0000 7fd4bc5d9740 10 client.25450 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.972+0000 7fd4bc5d9740 10 client.25450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.972+0000 7fd4bc5d9740 20 client.25450 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.972+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1531125313 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e42845c290 0x55e42845e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.972+0000 7fd4bc5d9740 4 client.25450 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:38:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.972+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1531125313 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e428341d70 con 0x55e42845c290 2024-09-11T23:38:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.985+0000 7fd4b2ffd640 1 --2- 172.21.15.133:0/1531125313 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e42845c290 0x55e42845e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.994+0000 7fd496ff5640 10 client.25450 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:31.994+0000 7fd4b2ffd640 1 --2- 172.21.15.133:0/1531125313 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e42845c290 0x55e42845e710 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0a120e9c26ea822 server_cookie=aee0e1ae208006de in_seq=0 out_seq=0 2024-09-11T23:38:33.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:33 smithi178 ceph-mon[35882]: pgmap v56: 97 pgs: 97 active+clean; 10 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 893 MiB/s wr, 2.46k op/s 2024-09-11T23:38:33.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:33 smithi138 ceph-mon[35142]: pgmap v56: 97 pgs: 97 active+clean; 10 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 893 MiB/s wr, 2.46k op/s 2024-09-11T23:38:33.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:33 smithi133 ceph-mon[31430]: pgmap v56: 97 pgs: 97 active+clean; 10 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 893 MiB/s wr, 2.46k op/s 2024-09-11T23:38:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.149+0000 7fd496ff5640 1 -- 172.21.15.133:0/1531125313 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e428341d70 con 0x55e42845c290 2024-09-11T23:38:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.149+0000 7fd496ff5640 10 client.25450 handle_command_reply: tid=0 2024-09-11T23:38:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.149+0000 7fd4bc5d9740 1 client.25450 shutdown 2024-09-11T23:38:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.150+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1531125313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 msgr2=0x55e4283daee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.150+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1531125313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 0x55e4283daee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40146a0 tx=0x7fd4a4014aa0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.150+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1531125313 shutdown_connections 2024-09-11T23:38:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.150+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1531125313 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e42845c290 0x55e42845e710 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.150+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1531125313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e4283e0e80 0x55e4283daee0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.150+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1531125313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4283d65e0 0x55e4283da9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.150+0000 7fd4bc5d9740 1 --2- 172.21.15.133:0/1531125313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4283d5cc0 0x55e4283e9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.150+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1531125313 >> 172.21.15.133:0/1531125313 conn(0x55e4283d2230 msgr2=0x55e4283dc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.150+0000 7fd496ff5640 0 client.25450 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.151+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1531125313 shutdown_connections 2024-09-11T23:38:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.151+0000 7fd4bc5d9740 1 -- 172.21.15.133:0/1531125313 wait complete. 2024-09-11T23:38:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.151+0000 7fd4bc5d9740 20 client.25450 trim_cache size 0 max 16384 2024-09-11T23:38:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:38:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:38:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:38:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:38:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.172+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1332434190 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd49c0a2f00 msgr2=0x7fd49c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.172+0000 7fd4b3fff640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd49c0a2f00 0x7fd49c0a53c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400b8c0 tx=0x7fd4a4004770 comp rx=0 tx=0).stop 2024-09-11T23:38:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.172+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1332434190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41095d0 msgr2=0x7fd4b41a7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.172+0000 7fd4b3fff640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41095d0 0x7fd4b41a7720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009850 tx=0x7fd4a8003cd0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.173+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1332434190 shutdown_connections 2024-09-11T23:38:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.173+0000 7fd4b3fff640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd49c0a2f00 0x7fd49c0a53c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.173+0000 7fd4b3fff640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b411b140 0x7fd4b41abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.173+0000 7fd4b3fff640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b41095d0 0x7fd4b41a7720 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.173+0000 7fd4b3fff640 1 --2- 172.21.15.133:0/1332434190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b41091f0 0x7fd4b41a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.173+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1332434190 >> 172.21.15.133:0/1332434190 conn(0x7fd4b406cbf0 msgr2=0x7fd4b411b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.173+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1332434190 shutdown_connections 2024-09-11T23:38:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:34.173+0000 7fd4b3fff640 1 -- 172.21.15.133:0/1332434190 wait complete. 2024-09-11T23:38:35.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:35 smithi138 ceph-mon[35142]: pgmap v57: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 724 MiB/s wr, 2.25k op/s 2024-09-11T23:38:35.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:35 smithi133 ceph-mon[31430]: pgmap v57: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 724 MiB/s wr, 2.25k op/s 2024-09-11T23:38:35.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:35 smithi178 ceph-mon[35882]: pgmap v57: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 724 MiB/s wr, 2.25k op/s 2024-09-11T23:38:37.406 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:38:37.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:37 smithi178 ceph-mon[35882]: pgmap v58: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 669 MiB/s wr, 2.11k op/s 2024-09-11T23:38:37.580 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:37 smithi133 ceph-mon[31430]: pgmap v58: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 669 MiB/s wr, 2.11k op/s 2024-09-11T23:38:37.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:37 smithi138 ceph-mon[35142]: pgmap v58: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 669 MiB/s wr, 2.11k op/s 2024-09-11T23:38:38.601 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:39.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:39 smithi138 ceph-mon[35142]: pgmap v59: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 996 MiB/s wr, 2.43k op/s 2024-09-11T23:38:39.461 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:39 smithi133 ceph-mon[31430]: pgmap v59: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 996 MiB/s wr, 2.43k op/s 2024-09-11T23:38:39.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:39 smithi178 ceph-mon[35882]: pgmap v59: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 996 MiB/s wr, 2.43k op/s 2024-09-11T23:38:39.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.878+0000 7f7076f76640 1 -- 172.21.15.133:0/2296284864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7070109830 msgr2=0x7f7070109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:39.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.878+0000 7f7076f76640 1 --2- 172.21.15.133:0/2296284864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7070109830 0x7f7070109c90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7060009900 tx=0x7f706002a440 comp rx=0 tx=0).stop 2024-09-11T23:38:39.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.879+0000 7f7076f76640 1 -- 172.21.15.133:0/2296284864 shutdown_connections 2024-09-11T23:38:39.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.879+0000 7f7076f76640 1 --2- 172.21.15.133:0/2296284864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f707011b140 0x7f707011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.879+0000 7f7076f76640 1 --2- 172.21.15.133:0/2296284864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7070109830 0x7f7070109c90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.879+0000 7f7076f76640 1 --2- 172.21.15.133:0/2296284864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7070109450 0x7f7070110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.879+0000 7f7076f76640 1 -- 172.21.15.133:0/2296284864 >> 172.21.15.133:0/2296284864 conn(0x7f707006e140 msgr2=0x7f707006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.879+0000 7f7076f76640 1 -- 172.21.15.133:0/2296284864 shutdown_connections 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.879+0000 7f7076f76640 1 -- 172.21.15.133:0/2296284864 wait complete. 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.880+0000 7f7076f76640 1 Processor -- start 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.881+0000 7f7076f76640 1 -- start start 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.881+0000 7f7076f76640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7070109450 0x7f70701183e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.882+0000 7f7076f76640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f707011b140 0x7f7070118920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.882+0000 7f7076f76640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7070111470 0x7f7070111920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.882+0000 7f7076f76640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7070120df0 con 0x7f7070109450 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.882+0000 7f7076f76640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7070120c70 con 0x7f707011b140 2024-09-11T23:38:39.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.882+0000 7f7076f76640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7070120f70 con 0x7f7070111470 2024-09-11T23:38:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.887+0000 7f7074d12640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7070109450 0x7f70701183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.887+0000 7f7074d12640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7070109450 0x7f70701183e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39908/0 (socket says 172.21.15.133:39908) 2024-09-11T23:38:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.887+0000 7f7074d12640 1 -- 172.21.15.133:0/4205298761 learned_addr learned my addr 172.21.15.133:0/4205298761 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.887+0000 7f7075513640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7070111470 0x7f7070111920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.887+0000 7f706ffff640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f707011b140 0x7f7070118920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.888+0000 7f7075513640 1 -- 172.21.15.133:0/4205298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f707011b140 msgr2=0x7f7070118920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.888+0000 7f7075513640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f707011b140 0x7f7070118920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.888+0000 7f7075513640 1 -- 172.21.15.133:0/4205298761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7070109450 msgr2=0x7f70701183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.888+0000 7f7075513640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7070109450 0x7f70701183e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.888+0000 7f7075513640 1 -- 172.21.15.133:0/4205298761 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70701120f0 con 0x7f7070111470 2024-09-11T23:38:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.889+0000 7f7075513640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7070111470 0x7f7070111920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f706400e970 tx=0x7f706400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:39.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.889+0000 7f706dffb640 1 -- 172.21.15.133:0/4205298761 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706400ccc0 con 0x7f7070111470 2024-09-11T23:38:39.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.889+0000 7f7076f76640 1 -- 172.21.15.133:0/4205298761 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7070115f10 con 0x7f7070111470 2024-09-11T23:38:39.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.890+0000 7f7076f76640 1 -- 172.21.15.133:0/4205298761 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7070116420 con 0x7f7070111470 2024-09-11T23:38:39.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.890+0000 7f706dffb640 1 -- 172.21.15.133:0/4205298761 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7064005020 con 0x7f7070111470 2024-09-11T23:38:39.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.891+0000 7f706dffb640 1 -- 172.21.15.133:0/4205298761 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064004a00 con 0x7f7070111470 2024-09-11T23:38:39.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.907+0000 7f706dffb640 1 -- 172.21.15.133:0/4205298761 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7064004ba0 con 0x7f7070111470 2024-09-11T23:38:39.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.912+0000 7f706dffb640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70580a2e30 0x7f70580a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:39.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.913+0000 7f706dffb640 1 -- 172.21.15.133:0/4205298761 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f70640d0300 con 0x7f7070111470 2024-09-11T23:38:39.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.913+0000 7f7074d12640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70580a2e30 0x7f70580a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.913+0000 7f707934f740 1 Processor -- start 2024-09-11T23:38:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.913+0000 7f707934f740 1 -- start start 2024-09-11T23:38:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.914+0000 7f707934f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 0x5568acc01880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.914+0000 7f707934f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc01dc0 0x5568acc0c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.914+0000 7f707934f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 0x5568acc0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.914+0000 7f707934f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5568acb672e0 con 0x5568acc014a0 2024-09-11T23:38:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.914+0000 7f707934f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5568acb0ece0 con 0x5568acc0c680 2024-09-11T23:38:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.914+0000 7f707934f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5568acb632a0 con 0x5568acc01dc0 2024-09-11T23:38:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.916+0000 7f7075513640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 0x5568acc0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.918+0000 7f7074d12640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 0x5568acc01880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.918+0000 7f7074d12640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 0x5568acc01880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39910/0 (socket says 172.21.15.133:39910) 2024-09-11T23:38:39.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.918+0000 7f7074d12640 1 -- 172.21.15.133:0/3391405873 learned_addr learned my addr 172.21.15.133:0/3391405873 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:39.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.920+0000 7f706ffff640 1 --2- 172.21.15.133:0/3391405873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc01dc0 0x5568acc0c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:39.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.921+0000 7f7075513640 1 -- 172.21.15.133:0/3391405873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc01dc0 msgr2=0x5568acc0c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:39.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.921+0000 7f7075513640 1 --2- 172.21.15.133:0/3391405873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc01dc0 0x5568acc0c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.921+0000 7f7075513640 1 -- 172.21.15.133:0/3391405873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 msgr2=0x5568acc01880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:39.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.921+0000 7f7075513640 1 --2- 172.21.15.133:0/3391405873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 0x5568acc01880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.921+0000 7f7075513640 1 -- 172.21.15.133:0/3391405873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568acaf3440 con 0x5568acc0c680 2024-09-11T23:38:39.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.924+0000 7f7075513640 1 --2- 172.21.15.133:0/3391405873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 0x5568acc0ea70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70640cda60 tx=0x7f706409c250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2878d07a059b5e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.926+0000 7f7056ffd640 1 -- 172.21.15.133:0/3391405873 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706401c040 con 0x5568acc0c680 2024-09-11T23:38:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.927+0000 7f7056ffd640 1 -- 172.21.15.133:0/3391405873 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70640a54a0 con 0x5568acc0c680 2024-09-11T23:38:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.928+0000 7f707934f740 1 -- 172.21.15.133:0/3391405873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 msgr2=0x5568acc0ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.928+0000 7f707934f740 1 --2- 172.21.15.133:0/3391405873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 0x5568acc0ea70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70640cda60 tx=0x7f706409c250 comp rx=0 tx=0).stop 2024-09-11T23:38:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.928+0000 7f707934f740 1 -- 172.21.15.133:0/3391405873 shutdown_connections 2024-09-11T23:38:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.928+0000 7f707934f740 1 --2- 172.21.15.133:0/3391405873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 0x5568acc0ea70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.928+0000 7f707934f740 1 --2- 172.21.15.133:0/3391405873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc01dc0 0x5568acc0c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.928+0000 7f707934f740 1 --2- 172.21.15.133:0/3391405873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 0x5568acc01880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.928+0000 7f707934f740 1 -- 172.21.15.133:0/3391405873 >> 172.21.15.133:0/3391405873 conn(0x5568acbfda30 msgr2=0x5568acbfde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:39.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.930+0000 7f7074d12640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70580a2e30 0x7f70580a52f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f70680098e0 tx=0x7f70680117b0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:39.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.935+0000 7f707934f740 1 -- 172.21.15.133:0/3391405873 shutdown_connections 2024-09-11T23:38:39.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.935+0000 7f707934f740 1 -- 172.21.15.133:0/3391405873 wait complete. 2024-09-11T23:38:39.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.935+0000 7f707934f740 1 -- start start 2024-09-11T23:38:39.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.935+0000 7f707934f740 1 Processor -- start 2024-09-11T23:38:39.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.936+0000 7f707934f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc014a0 0x5568acc0bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:39.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.937+0000 7f707934f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc01dc0 0x5568acc091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:39.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.937+0000 7f707934f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 0x5568acc09710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.937+0000 7f707934f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5568acc0f2a0 con 0x5568acc01dc0 2024-09-11T23:38:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.937+0000 7f7074d12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc014a0 0x5568acc0bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.937+0000 7f7074d12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc014a0 0x5568acc0bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35110/0 (socket says 172.21.15.133:35110) 2024-09-11T23:38:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.937+0000 7f7074d12640 1 -- 172.21.15.133:0/2208070836 learned_addr learned my addr 172.21.15.133:0/2208070836 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.937+0000 7f7075513640 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 0x5568acc09710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.938+0000 7f707934f740 1 -- 172.21.15.133:0/2208070836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5568acb22050 con 0x5568acc0c680 2024-09-11T23:38:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.938+0000 7f707934f740 1 -- 172.21.15.133:0/2208070836 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5568acc0f420 con 0x5568acc014a0 2024-09-11T23:38:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.938+0000 7f707934f740 10 client.0 _ll_register_callbacks cb 0x5568acb62160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.939+0000 7f7075513640 1 -- 172.21.15.133:0/2208070836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc014a0 msgr2=0x5568acc0bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.939+0000 7f7075513640 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc014a0 0x5568acc0bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.939+0000 7f7075513640 1 -- 172.21.15.133:0/2208070836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc01dc0 msgr2=0x5568acc091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.939+0000 7f7075513640 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc01dc0 0x5568acc091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.939+0000 7f7075513640 1 -- 172.21.15.133:0/2208070836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568ac8d83c0 con 0x5568acc0c680 2024-09-11T23:38:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.940+0000 7f7075513640 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 0x5568acc09710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7064009640 tx=0x7f70640a5240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92b3c465e5795982 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:39.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.940+0000 7f704ffff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:38:39.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.940+0000 7f704ffff640 1 -- 172.21.15.133:0/2208070836 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706401c040 con 0x5568acc0c680 2024-09-11T23:38:39.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.940+0000 7f707934f740 1 -- 172.21.15.133:0/2208070836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70640a54a0 con 0x5568acc0c680 2024-09-11T23:38:39.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.940+0000 7f704ffff640 1 -- 172.21.15.133:0/2208070836 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f706409ab30 con 0x5568acc0c680 2024-09-11T23:38:39.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.941+0000 7f704ffff640 1 -- 172.21.15.133:0/2208070836 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f706409ccf0 con 0x5568acc0c680 2024-09-11T23:38:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.943+0000 7f7074d12640 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc014a0 0x5568acc0bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.943+0000 7f707934f740 10 client.25436 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.943+0000 7f707934f740 1 -- 172.21.15.133:0/2208070836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568acc1ac10 con 0x5568acc0c680 2024-09-11T23:38:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.944+0000 7f704ffff640 1 -- 172.21.15.133:0/2208070836 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70640a54a0 con 0x5568acc0c680 2024-09-11T23:38:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.945+0000 7f704ffff640 1 -- 172.21.15.133:0/2208070836 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f706409b6f0 con 0x5568acc0c680 2024-09-11T23:38:39.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.946+0000 7f707934f740 10 client.25436 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:39.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.946+0000 7f707934f740 10 client.25436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:39.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.946+0000 7f707934f740 20 client.25436 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:39.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.946+0000 7f707934f740 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5568acc1c0c0 0x5568acc1e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:39.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.946+0000 7f707934f740 4 client.25436 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:39.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.946+0000 7f707934f740 1 -- 172.21.15.133:0/2208070836 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568acb74850 con 0x5568acc1c0c0 2024-09-11T23:38:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.948+0000 7f7074d12640 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5568acc1c0c0 0x5568acc1e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:39.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.953+0000 7f704ffff640 10 client.25436 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:39.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.953+0000 7f7074d12640 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5568acc1c0c0 0x5568acc1e530 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=656824169a93a244 server_cookie=e7c63f117197ef70 in_seq=0 out_seq=0 2024-09-11T23:38:39.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.962+0000 7f704ffff640 1 -- 172.21.15.133:0/2208070836 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5568acb74850 con 0x5568acc1c0c0 2024-09-11T23:38:39.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.962+0000 7f704ffff640 10 client.25436 handle_command_reply: tid=0 2024-09-11T23:38:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.962+0000 7f707934f740 1 client.25436 shutdown 2024-09-11T23:38:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.964+0000 7f707934f740 1 -- 172.21.15.133:0/2208070836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 msgr2=0x5568acc09710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.964+0000 7f707934f740 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 0x5568acc09710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7064009640 tx=0x7f70640a5240 comp rx=0 tx=0).stop 2024-09-11T23:38:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.964+0000 7f707934f740 1 -- 172.21.15.133:0/2208070836 shutdown_connections 2024-09-11T23:38:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.964+0000 7f707934f740 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5568acc1c0c0 0x5568acc1e530 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.964+0000 7f707934f740 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc0c680 0x5568acc09710 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.964+0000 7f707934f740 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc01dc0 0x5568acc091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.964+0000 7f707934f740 1 --2- 172.21.15.133:0/2208070836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc014a0 0x5568acc0bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.964+0000 7f707934f740 1 -- 172.21.15.133:0/2208070836 >> 172.21.15.133:0/2208070836 conn(0x5568acbfda30 msgr2=0x5568acc03760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.964+0000 7f704ffff640 0 client.25436 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.965+0000 7f707934f740 1 -- 172.21.15.133:0/2208070836 shutdown_connections 2024-09-11T23:38:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.966+0000 7f707934f740 1 -- 172.21.15.133:0/2208070836 wait complete. 2024-09-11T23:38:39.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:39.967+0000 7f707934f740 20 client.25436 trim_cache size 0 max 16384 2024-09-11T23:38:40.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.009+0000 7f707934f740 1 Processor -- start 2024-09-11T23:38:40.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.009+0000 7f707934f740 1 -- start start 2024-09-11T23:38:40.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.010+0000 7f707934f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acbfda30 0x5568acc18550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:40.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.010+0000 7f707934f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc014a0 0x5568acc048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:40.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.010+0000 7f707934f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc01dc0 0x5568acc081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:40.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.010+0000 7f7074d12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acbfda30 0x5568acc18550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:40.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.010+0000 7f7074d12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acbfda30 0x5568acc18550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35116/0 (socket says 172.21.15.133:35116) 2024-09-11T23:38:40.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.011+0000 7f706ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc014a0 0x5568acc048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:40.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.011+0000 7f706ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc014a0 0x5568acc048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:41336/0 (socket says 172.21.15.133:41336) 2024-09-11T23:38:40.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.011+0000 7f7074d12640 1 -- 172.21.15.133:0/2525438889 learned_addr learned my addr 172.21.15.133:0/2525438889 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:40.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.011+0000 7f707934f740 1 -- 172.21.15.133:0/2525438889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f705c00e6f0 con 0x5568acc01dc0 2024-09-11T23:38:40.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.011+0000 7f707934f740 1 -- 172.21.15.133:0/2525438889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f705c010230 con 0x5568acc014a0 2024-09-11T23:38:40.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.011+0000 7f707934f740 1 -- 172.21.15.133:0/2525438889 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f705c00edc0 con 0x5568acbfda30 2024-09-11T23:38:40.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.012+0000 7f7074d12640 1 -- 172.21.15.133:0/2525438889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc014a0 msgr2=0x5568acc048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:40.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.012+0000 7f7074d12640 1 --2- 172.21.15.133:0/2525438889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc014a0 0x5568acc048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.012+0000 7f7074d12640 1 -- 172.21.15.133:0/2525438889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc01dc0 msgr2=0x5568acc081b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:40.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.012+0000 7f7074d12640 1 --2- 172.21.15.133:0/2525438889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc01dc0 0x5568acc081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.012+0000 7f7074d12640 1 -- 172.21.15.133:0/2525438889 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568acc01bf0 con 0x5568acbfda30 2024-09-11T23:38:40.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.012+0000 7f7074d12640 1 --2- 172.21.15.133:0/2525438889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acbfda30 0x5568acc18550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5568acc1ed40 tx=0x7f7068016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbcd1e44bfcadcc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:40.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.013+0000 7f70567fc640 1 -- 172.21.15.133:0/2525438889 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706802e070 con 0x5568acbfda30 2024-09-11T23:38:40.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.013+0000 7f70567fc640 1 -- 172.21.15.133:0/2525438889 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7068020c90 con 0x5568acbfda30 2024-09-11T23:38:40.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.014+0000 7f707934f740 1 -- 172.21.15.133:0/2525438889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acbfda30 msgr2=0x5568acc18550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:40.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.014+0000 7f707934f740 1 --2- 172.21.15.133:0/2525438889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acbfda30 0x5568acc18550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5568acc1ed40 tx=0x7f7068016000 comp rx=0 tx=0).stop 2024-09-11T23:38:40.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.014+0000 7f707934f740 1 -- 172.21.15.133:0/2525438889 shutdown_connections 2024-09-11T23:38:40.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.014+0000 7f707934f740 1 --2- 172.21.15.133:0/2525438889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc01dc0 0x5568acc081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.014+0000 7f707934f740 1 --2- 172.21.15.133:0/2525438889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc014a0 0x5568acc048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.014+0000 7f707934f740 1 --2- 172.21.15.133:0/2525438889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acbfda30 0x5568acc18550 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.014+0000 7f707934f740 1 -- 172.21.15.133:0/2525438889 >> 172.21.15.133:0/2525438889 conn(0x5568acc09870 msgr2=0x5568acc07aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.015+0000 7f707934f740 1 -- 172.21.15.133:0/2525438889 shutdown_connections 2024-09-11T23:38:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.015+0000 7f707934f740 1 -- 172.21.15.133:0/2525438889 wait complete. 2024-09-11T23:38:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.015+0000 7f707934f740 1 -- start start 2024-09-11T23:38:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.015+0000 7f707934f740 1 Processor -- start 2024-09-11T23:38:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.017+0000 7f707934f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 0x5568acc0fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:40.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.017+0000 7f707934f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc01dc0 0x5568acc10160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:40.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.017+0000 7f707934f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc0c680 0x5568acc84d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:40.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.018+0000 7f707934f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f705c00fb60 con 0x5568acc014a0 2024-09-11T23:38:40.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.018+0000 7f707934f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f705c00f490 con 0x5568acc01dc0 2024-09-11T23:38:40.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.018+0000 7f706ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc01dc0 0x5568acc10160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:40.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.018+0000 7f706ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc01dc0 0x5568acc10160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:41340/0 (socket says 172.21.15.133:41340) 2024-09-11T23:38:40.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.018+0000 7f706ffff640 1 -- 172.21.15.133:0/2303459521 learned_addr learned my addr 172.21.15.133:0/2303459521 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:40.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.018+0000 7f7075513640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc0c680 0x5568acc84d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:40.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.018+0000 7f7074d12640 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 0x5568acc0fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:40.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.018+0000 7f707934f740 1 -- 172.21.15.133:0/2303459521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f705c00c6d0 con 0x5568acc0c680 2024-09-11T23:38:40.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.019+0000 7f707934f740 10 client.0 _ll_register_callbacks cb 0x7f706409c370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:40.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.019+0000 7f7075513640 1 -- 172.21.15.133:0/2303459521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc01dc0 msgr2=0x5568acc10160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:40.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.019+0000 7f7075513640 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc01dc0 0x5568acc10160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.019+0000 7f7075513640 1 -- 172.21.15.133:0/2303459521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 msgr2=0x5568acc0fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:40.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.020+0000 7f7075513640 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 0x5568acc0fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.020+0000 7f7075513640 1 -- 172.21.15.133:0/2303459521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568acc853d0 con 0x5568acc0c680 2024-09-11T23:38:40.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.021+0000 7f7074d12640 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 0x5568acc0fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:38:40.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.021+0000 7f7075513640 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc0c680 0x5568acc84d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f706409c7c0 tx=0x7f7064009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f64b77884fff35bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:40.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.021+0000 7f704ffff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:38:40.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.021+0000 7f704ffff640 1 -- 172.21.15.133:0/2303459521 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706401c070 con 0x5568acc0c680 2024-09-11T23:38:40.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.022+0000 7f704ffff640 1 -- 172.21.15.133:0/2303459521 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f706409ab30 con 0x5568acc0c680 2024-09-11T23:38:40.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.022+0000 7f707934f740 1 -- 172.21.15.133:0/2303459521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568acb6ed60 con 0x5568acc0c680 2024-09-11T23:38:40.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.022+0000 7f704ffff640 1 -- 172.21.15.133:0/2303459521 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70640a54a0 con 0x5568acc0c680 2024-09-11T23:38:40.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.022+0000 7f707934f740 10 client.25468 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:40.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.022+0000 7f707934f740 1 -- 172.21.15.133:0/2303459521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568acc853d0 con 0x5568acc0c680 2024-09-11T23:38:40.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.026+0000 7f704ffff640 1 -- 172.21.15.133:0/2303459521 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f706409bc80 con 0x5568acc0c680 2024-09-11T23:38:40.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.026+0000 7f704ffff640 1 -- 172.21.15.133:0/2303459521 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f70640b5a30 con 0x5568acc0c680 2024-09-11T23:38:40.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.026+0000 7f707934f740 10 client.25468 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:40.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.026+0000 7f707934f740 10 client.25468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:40.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.026+0000 7f707934f740 20 client.25468 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:40.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.026+0000 7f707934f740 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5568acc86910 0x5568acc88d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:40.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.026+0000 7f707934f740 4 client.25468 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:38:40.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.026+0000 7f707934f740 1 -- 172.21.15.133:0/2303459521 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5568acb6ed60 con 0x5568acc86910 2024-09-11T23:38:40.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.032+0000 7f7074d12640 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5568acc86910 0x5568acc88d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:40.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.036+0000 7f7074d12640 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5568acc86910 0x5568acc88d00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41bf12d9c8169b6 server_cookie=d201343452135c94 in_seq=0 out_seq=0 2024-09-11T23:38:40.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.037+0000 7f704ffff640 10 client.25468 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:40.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.037+0000 7f704ffff640 1 -- 172.21.15.133:0/2303459521 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5568acb6ed60 con 0x5568acc86910 2024-09-11T23:38:40.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.037+0000 7f704ffff640 10 client.25468 handle_command_reply: tid=0 2024-09-11T23:38:40.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.037+0000 7f707934f740 1 client.25468 shutdown 2024-09-11T23:38:40.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.043+0000 7f707934f740 1 -- 172.21.15.133:0/2303459521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc0c680 msgr2=0x5568acc84d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:40.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.043+0000 7f707934f740 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc0c680 0x5568acc84d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f706409c7c0 tx=0x7f7064009ef0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.043+0000 7f707934f740 1 -- 172.21.15.133:0/2303459521 shutdown_connections 2024-09-11T23:38:40.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.043+0000 7f707934f740 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5568acc86910 0x5568acc88d00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.043+0000 7f707934f740 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568acc0c680 0x5568acc84d90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.043+0000 7f707934f740 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568acc01dc0 0x5568acc10160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.043+0000 7f707934f740 1 --2- 172.21.15.133:0/2303459521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5568acc014a0 0x5568acc0fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.043+0000 7f707934f740 1 -- 172.21.15.133:0/2303459521 >> 172.21.15.133:0/2303459521 conn(0x5568acc09870 msgr2=0x5568acc082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:40.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.043+0000 7f704ffff640 0 client.25468 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:40.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.048+0000 7f707934f740 1 -- 172.21.15.133:0/2303459521 shutdown_connections 2024-09-11T23:38:40.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.048+0000 7f707934f740 1 -- 172.21.15.133:0/2303459521 wait complete. 2024-09-11T23:38:40.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.049+0000 7f707934f740 20 client.25468 trim_cache size 0 max 16384 2024-09-11T23:38:40.055 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:38:40.055 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:38:40.055 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:38:40.055 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:38:40.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 -- 172.21.15.133:0/4205298761 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70580a2e30 msgr2=0x7f70580a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:40.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70580a2e30 0x7f70580a52f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f70680098e0 tx=0x7f70680117b0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 -- 172.21.15.133:0/4205298761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7070111470 msgr2=0x7f7070111920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:40.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7070111470 0x7f7070111920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f706400e970 tx=0x7f706400ee30 comp rx=0 tx=0).stop 2024-09-11T23:38:40.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 -- 172.21.15.133:0/4205298761 shutdown_connections 2024-09-11T23:38:40.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70580a2e30 0x7f70580a52f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7070111470 0x7f7070111920 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f707011b140 0x7f7070118920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 --2- 172.21.15.133:0/4205298761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7070109450 0x7f70701183e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:40.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 -- 172.21.15.133:0/4205298761 >> 172.21.15.133:0/4205298761 conn(0x7f707006e140 msgr2=0x7f707011b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:40.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 -- 172.21.15.133:0/4205298761 shutdown_connections 2024-09-11T23:38:40.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:40.061+0000 7f7076f76640 1 -- 172.21.15.133:0/4205298761 wait complete. 2024-09-11T23:38:41.583 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:41 smithi133 ceph-mon[31430]: pgmap v60: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 659 MiB/s wr, 1.69k op/s 2024-09-11T23:38:41.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:41 smithi138 ceph-mon[35142]: pgmap v60: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 659 MiB/s wr, 1.69k op/s 2024-09-11T23:38:41.708 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:38:41.709 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:38:41.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:41 smithi178 ceph-mon[35882]: pgmap v60: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 659 MiB/s wr, 1.69k op/s 2024-09-11T23:38:42.818 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:43.608 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:43 smithi133 ceph-mon[31430]: pgmap v61: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 855 MiB/s wr, 2.26k op/s 2024-09-11T23:38:43.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:43 smithi138 ceph-mon[35142]: pgmap v61: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 855 MiB/s wr, 2.26k op/s 2024-09-11T23:38:43.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:43 smithi178 ceph-mon[35882]: pgmap v61: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 855 MiB/s wr, 2.26k op/s 2024-09-11T23:38:44.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.137+0000 7f3109342640 1 -- 172.21.15.133:0/1860688590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3104109830 msgr2=0x7f3104109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:44.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.137+0000 7f3109342640 1 --2- 172.21.15.133:0/1860688590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3104109830 0x7f3104109c90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009900 tx=0x7f30f802a440 comp rx=0 tx=0).stop 2024-09-11T23:38:44.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.140+0000 7f3109342640 1 -- 172.21.15.133:0/1860688590 shutdown_connections 2024-09-11T23:38:44.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.140+0000 7f3109342640 1 --2- 172.21.15.133:0/1860688590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310411b140 0x7f310411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:44.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.140+0000 7f3109342640 1 --2- 172.21.15.133:0/1860688590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3104109830 0x7f3104109c90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:44.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.140+0000 7f3109342640 1 --2- 172.21.15.133:0/1860688590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3104109450 0x7f3104110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:44.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.140+0000 7f3109342640 1 -- 172.21.15.133:0/1860688590 >> 172.21.15.133:0/1860688590 conn(0x7f310406e140 msgr2=0x7f310406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:44.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.142+0000 7f3109342640 1 -- 172.21.15.133:0/1860688590 shutdown_connections 2024-09-11T23:38:44.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.142+0000 7f3109342640 1 -- 172.21.15.133:0/1860688590 wait complete. 2024-09-11T23:38:44.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.143+0000 7f3109342640 1 Processor -- start 2024-09-11T23:38:44.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.150+0000 7f3109342640 1 -- start start 2024-09-11T23:38:44.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.150+0000 7f3109342640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3104109450 0x7f31041a7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:44.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.150+0000 7f3109342640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104109830 0x7f31041a7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:44.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.150+0000 7f3109342640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f310411b140 0x7f31041abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:44.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.150+0000 7f3109342640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3104120df0 con 0x7f310411b140 2024-09-11T23:38:44.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.150+0000 7f3109342640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3104120c70 con 0x7f3104109830 2024-09-11T23:38:44.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.150+0000 7f3109342640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3104120f70 con 0x7f3104109450 2024-09-11T23:38:44.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.152+0000 7f31037fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f310411b140 0x7f31041abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:44.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.152+0000 7f31037fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f310411b140 0x7f31041abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47594/0 (socket says 172.21.15.133:47594) 2024-09-11T23:38:44.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.152+0000 7f31037fe640 1 -- 172.21.15.133:0/2137860484 learned_addr learned my addr 172.21.15.133:0/2137860484 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:44.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.153+0000 7f31027fc640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104109830 0x7f31041a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:44.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.153+0000 7f3102ffd640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3104109450 0x7f31041a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:44.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.154+0000 7f31037fe640 1 -- 172.21.15.133:0/2137860484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3104109450 msgr2=0x7f31041a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:44.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.154+0000 7f31037fe640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3104109450 0x7f31041a7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:44.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.154+0000 7f31037fe640 1 -- 172.21.15.133:0/2137860484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104109830 msgr2=0x7f31041a7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:44.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.154+0000 7f31037fe640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104109830 0x7f31041a7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:44.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.154+0000 7f31037fe640 1 -- 172.21.15.133:0/2137860484 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31041ac1e0 con 0x7f310411b140 2024-09-11T23:38:44.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.154+0000 7f31027fc640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104109830 0x7f31041a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:44.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.155+0000 7f31037fe640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f310411b140 0x7f31041abae0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00cab0 tx=0x7f30fc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:44.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.155+0000 7f30ebfff640 1 -- 172.21.15.133:0/2137860484 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc00ae20 con 0x7f310411b140 2024-09-11T23:38:44.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.156+0000 7f3109342640 1 -- 172.21.15.133:0/2137860484 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31041ac4d0 con 0x7f310411b140 2024-09-11T23:38:44.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.156+0000 7f3109342640 1 -- 172.21.15.133:0/2137860484 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3104110920 con 0x7f310411b140 2024-09-11T23:38:44.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.158+0000 7f30ebfff640 1 -- 172.21.15.133:0/2137860484 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30fc00f930 con 0x7f310411b140 2024-09-11T23:38:44.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.158+0000 7f30ebfff640 1 -- 172.21.15.133:0/2137860484 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc018b80 con 0x7f310411b140 2024-09-11T23:38:44.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.165+0000 7f30ebfff640 1 -- 172.21.15.133:0/2137860484 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f30fc00fae0 con 0x7f310411b140 2024-09-11T23:38:44.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.165+0000 7f3109342640 1 -- 172.21.15.133:0/2137860484 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f0005290 con 0x7f310411b140 2024-09-11T23:38:44.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.168+0000 7f30ebfff640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30ec0a2f00 0x7f30ec0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:44.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.169+0000 7f30ebfff640 1 -- 172.21.15.133:0/2137860484 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f30fc0cf7b0 con 0x7f310411b140 2024-09-11T23:38:44.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.170+0000 7f3102ffd640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30ec0a2f00 0x7f30ec0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:44.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.172+0000 7f3102ffd640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30ec0a2f00 0x7f30ec0a53c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30f40058c0 tx=0x7f30f400a4c0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:44.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.179+0000 7f30ebfff640 1 -- 172.21.15.133:0/2137860484 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30fc014030 con 0x7f310411b140 2024-09-11T23:38:44.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.785+0000 7f3109342640 1 -- 172.21.15.133:0/2137860484 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30f0005850 con 0x7f310411b140 2024-09-11T23:38:44.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.786+0000 7f30ebfff640 1 -- 172.21.15.133:0/2137860484 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f30fc09c3e0 con 0x7f310411b140 2024-09-11T23:38:44.786 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:38:44.787 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:38:44.787 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:38:44.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.791+0000 7f3109342640 1 -- 172.21.15.133:0/2137860484 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30ec0a2f00 msgr2=0x7f30ec0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:44.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.791+0000 7f3109342640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30ec0a2f00 0x7f30ec0a53c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30f40058c0 tx=0x7f30f400a4c0 comp rx=0 tx=0).stop 2024-09-11T23:38:44.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.791+0000 7f3109342640 1 -- 172.21.15.133:0/2137860484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f310411b140 msgr2=0x7f31041abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:44.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.791+0000 7f3109342640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f310411b140 0x7f31041abae0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00cab0 tx=0x7f30fc00cf70 comp rx=0 tx=0).stop 2024-09-11T23:38:44.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.792+0000 7f3109342640 1 -- 172.21.15.133:0/2137860484 shutdown_connections 2024-09-11T23:38:44.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.792+0000 7f3109342640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30ec0a2f00 0x7f30ec0a53c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:44.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.792+0000 7f3109342640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f310411b140 0x7f31041abae0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:44.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.792+0000 7f3109342640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104109830 0x7f31041a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:44.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.792+0000 7f3109342640 1 --2- 172.21.15.133:0/2137860484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3104109450 0x7f31041a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:44.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.792+0000 7f3109342640 1 -- 172.21.15.133:0/2137860484 >> 172.21.15.133:0/2137860484 conn(0x7f310406e140 msgr2=0x7f310411be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:44.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.792+0000 7f3109342640 1 -- 172.21.15.133:0/2137860484 shutdown_connections 2024-09-11T23:38:44.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:44.792+0000 7f3109342640 1 -- 172.21.15.133:0/2137860484 wait complete. 2024-09-11T23:38:45.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:44 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2137860484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:45.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:44 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2137860484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:45.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:44 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2137860484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:45.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:38:46.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:45 smithi133 ceph-mon[31430]: pgmap v62: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 753 MiB/s wr, 1.74k op/s 2024-09-11T23:38:46.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:45 smithi138 ceph-mon[35142]: pgmap v62: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 753 MiB/s wr, 1.74k op/s 2024-09-11T23:38:46.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:45 smithi178 ceph-mon[35882]: pgmap v62: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 753 MiB/s wr, 1.74k op/s 2024-09-11T23:38:46.809 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:47.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:47 smithi133 ceph-mon[31430]: pgmap v63: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 633 MiB/s wr, 1.22k op/s 2024-09-11T23:38:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.537+0000 7fe115562640 1 -- 172.21.15.133:0/54571070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1080a5140 msgr2=0x7fe1080b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.537+0000 7fe115562640 1 --2- 172.21.15.133:0/54571070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1080a5140 0x7fe1080b7000 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe110050280 tx=0x7fe11008e3f0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.543+0000 7fe115562640 1 -- 172.21.15.133:0/54571070 shutdown_connections 2024-09-11T23:38:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.543+0000 7fe115562640 1 --2- 172.21.15.133:0/54571070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080b7540 0x7fe1080b9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.543+0000 7fe115562640 1 --2- 172.21.15.133:0/54571070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1080a5140 0x7fe1080b7000 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.543+0000 7fe115562640 1 --2- 172.21.15.133:0/54571070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1080a4820 0x7fe1080a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.543+0000 7fe115562640 1 -- 172.21.15.133:0/54571070 >> 172.21.15.133:0/54571070 conn(0x7fe10801a110 msgr2=0x7fe10801a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.552+0000 7fe115562640 1 -- 172.21.15.133:0/54571070 shutdown_connections 2024-09-11T23:38:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.554+0000 7fe115562640 1 -- 172.21.15.133:0/54571070 wait complete. 2024-09-11T23:38:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.556+0000 7fe115562640 1 Processor -- start 2024-09-11T23:38:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.556+0000 7fe115562640 1 -- start start 2024-09-11T23:38:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.557+0000 7fe115562640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1080a4820 0x7fe1080b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.557+0000 7fe115562640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1080a5140 0x7fe1080ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.557+0000 7fe115562640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080b7540 0x7fe1080aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.557+0000 7fe115562640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1080bcf80 con 0x7fe1080b7540 2024-09-11T23:38:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.557+0000 7fe115562640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1080bce00 con 0x7fe1080a5140 2024-09-11T23:38:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.557+0000 7fe115562640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe1080bd100 con 0x7fe1080a4820 2024-09-11T23:38:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.558+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1080a5140 0x7fe1080ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.559+0000 7fe114d61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080b7540 0x7fe1080aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.559+0000 7fe114d61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080b7540 0x7fe1080aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47618/0 (socket says 172.21.15.133:47618) 2024-09-11T23:38:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.559+0000 7fe114d61640 1 -- 172.21.15.133:0/2521730765 learned_addr learned my addr 172.21.15.133:0/2521730765 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.560+0000 7fe114d61640 1 -- 172.21.15.133:0/2521730765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1080a4820 msgr2=0x7fe1080b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.560+0000 7fe114d61640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1080a4820 0x7fe1080b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.560+0000 7fe114d61640 1 -- 172.21.15.133:0/2521730765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1080a5140 msgr2=0x7fe1080ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.560+0000 7fe114d61640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1080a5140 0x7fe1080ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.560+0000 7fe114d61640 1 -- 172.21.15.133:0/2521730765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1080af240 con 0x7fe1080b7540 2024-09-11T23:38:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.560+0000 7fe114d61640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080b7540 0x7fe1080aec00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe104009fd0 tx=0x7fe10400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.562+0000 7fe10d7fa640 1 -- 172.21.15.133:0/2521730765 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104011830 con 0x7fe1080b7540 2024-09-11T23:38:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.562+0000 7fe10d7fa640 1 -- 172.21.15.133:0/2521730765 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe104016070 con 0x7fe1080b7540 2024-09-11T23:38:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.562+0000 7fe115562640 1 -- 172.21.15.133:0/2521730765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1080b3160 con 0x7fe1080b7540 2024-09-11T23:38:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.564+0000 7fe115562640 1 -- 172.21.15.133:0/2521730765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1080b3680 con 0x7fe1080b7540 2024-09-11T23:38:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.564+0000 7fe10d7fa640 1 -- 172.21.15.133:0/2521730765 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10400f2e0 con 0x7fe1080b7540 2024-09-11T23:38:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.569+0000 7fe10d7fa640 1 -- 172.21.15.133:0/2521730765 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe104010380 con 0x7fe1080b7540 2024-09-11T23:38:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.573+0000 7fe11963a740 1 Processor -- start 2024-09-11T23:38:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.573+0000 7fe11963a740 1 -- start start 2024-09-11T23:38:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.574+0000 7fe11963a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d74cc0 0x556367d750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.574+0000 7fe11963a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d755e0 0x556367d7f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.574+0000 7fe11963a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d7fe80 0x556367d82270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.575+0000 7fe11963a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556367cd92f0 con 0x556367d7fe80 2024-09-11T23:38:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.575+0000 7fe11963a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556367c81270 con 0x556367d755e0 2024-09-11T23:38:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.575+0000 7fe11963a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556367cd52b0 con 0x556367d74cc0 2024-09-11T23:38:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.575+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d7fe80 0x556367d82270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.575+0000 7fe10d7fa640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe1000a2d60 0x7fe1000a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.576+0000 7fe10d7fa640 1 -- 172.21.15.133:0/2521730765 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe1040ce870 con 0x7fe1080b7540 2024-09-11T23:38:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.578+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d7fe80 0x556367d82270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47632/0 (socket says 172.21.15.133:47632) 2024-09-11T23:38:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.578+0000 7fe10f7fe640 1 -- 172.21.15.133:0/4004109436 learned_addr learned my addr 172.21.15.133:0/4004109436 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.578+0000 7fe10f7fe640 1 --2- 172.21.15.133:0/4004109436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d74cc0 0x556367d750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.578+0000 7fe10ffff640 1 --2- 172.21.15.133:0/4004109436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d755e0 0x556367d7f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.580+0000 7fe114d61640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe1000a2d60 0x7fe1000a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.582+0000 7fe10ffff640 1 -- 172.21.15.133:0/4004109436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d74cc0 msgr2=0x556367d750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.582+0000 7fe10ffff640 1 --2- 172.21.15.133:0/4004109436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d74cc0 0x556367d750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.582+0000 7fe10ffff640 1 -- 172.21.15.133:0/4004109436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d7fe80 msgr2=0x556367d82270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.582+0000 7fe10ffff640 1 --2- 172.21.15.133:0/4004109436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d7fe80 0x556367d82270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.582+0000 7fe10ffff640 1 -- 172.21.15.133:0/4004109436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556367c80340 con 0x556367d755e0 2024-09-11T23:38:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.584+0000 7fe10ffff640 1 --2- 172.21.15.133:0/4004109436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d755e0 0x556367d7f940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009af0 tx=0x7fe0fc00e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262ed9c182f9a0e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.584+0000 7fe114d61640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe1000a2d60 0x7fe1000a5220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe104097310 tx=0x7fe1040047d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.589+0000 7fe0fa7fc640 1 -- 172.21.15.133:0/4004109436 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc00ed00 con 0x556367d755e0 2024-09-11T23:38:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.589+0000 7fe0fa7fc640 1 -- 172.21.15.133:0/4004109436 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0fc005020 con 0x556367d755e0 2024-09-11T23:38:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.589+0000 7fe0fa7fc640 1 -- 172.21.15.133:0/4004109436 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc004a80 con 0x556367d755e0 2024-09-11T23:38:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.591+0000 7fe11963a740 1 -- 172.21.15.133:0/4004109436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d755e0 msgr2=0x556367d7f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.591+0000 7fe11963a740 1 --2- 172.21.15.133:0/4004109436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d755e0 0x556367d7f940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009af0 tx=0x7fe0fc00e350 comp rx=0 tx=0).stop 2024-09-11T23:38:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.592+0000 7fe11963a740 1 -- 172.21.15.133:0/4004109436 shutdown_connections 2024-09-11T23:38:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.592+0000 7fe11963a740 1 --2- 172.21.15.133:0/4004109436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d7fe80 0x556367d82270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.592+0000 7fe11963a740 1 --2- 172.21.15.133:0/4004109436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d755e0 0x556367d7f940 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.592+0000 7fe11963a740 1 --2- 172.21.15.133:0/4004109436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d74cc0 0x556367d750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.592+0000 7fe11963a740 1 -- 172.21.15.133:0/4004109436 >> 172.21.15.133:0/4004109436 conn(0x556367d71230 msgr2=0x556367d71610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.592+0000 7fe11963a740 1 -- 172.21.15.133:0/4004109436 shutdown_connections 2024-09-11T23:38:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.597+0000 7fe11963a740 1 -- 172.21.15.133:0/4004109436 wait complete. 2024-09-11T23:38:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.598+0000 7fe11963a740 1 -- start start 2024-09-11T23:38:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.598+0000 7fe11963a740 1 Processor -- start 2024-09-11T23:38:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.600+0000 7fe11963a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 0x556367d7cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.600+0000 7fe11963a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d7d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.600+0000 7fe11963a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d7fe80 0x556367d7d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.600+0000 7fe11963a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556367d82a50 con 0x556367d755e0 2024-09-11T23:38:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.600+0000 7fe11963a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556367c62290 con 0x556367d74cc0 2024-09-11T23:38:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.600+0000 7fe11963a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556367d82bd0 con 0x556367d7fe80 2024-09-11T23:38:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.600+0000 7fe11963a740 10 client.0 _ll_register_callbacks cb 0x556367c93140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.601+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d7d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.601+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d7d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47648/0 (socket says 172.21.15.133:47648) 2024-09-11T23:38:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.601+0000 7fe10ffff640 1 -- 172.21.15.133:0/2633108757 learned_addr learned my addr 172.21.15.133:0/2633108757 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.601+0000 7fe10f7fe640 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 0x556367d7cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.601+0000 7fe10f7fe640 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d7fe80 0x556367d7d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.602+0000 7fe10f7fe640 1 -- 172.21.15.133:0/2633108757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d7fe80 msgr2=0x556367d7d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.602+0000 7fe10f7fe640 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d7fe80 0x556367d7d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.602+0000 7fe10f7fe640 1 -- 172.21.15.133:0/2633108757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 msgr2=0x556367d7d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.602+0000 7fe10f7fe640 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d7d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.602+0000 7fe10f7fe640 1 -- 172.21.15.133:0/2633108757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563679ec210 con 0x556367d74cc0 2024-09-11T23:38:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.603+0000 7fe10f7fe640 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 0x556367d7cd80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe11008e9e0 tx=0x7fe11008e3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ad077bccdc1d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.642+0000 7fe0ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:38:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.642+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2633108757 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110096c70 con 0x556367d74cc0 2024-09-11T23:38:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.642+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2633108757 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe110096e10 con 0x556367d74cc0 2024-09-11T23:38:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.642+0000 7fe11963a740 1 -- 172.21.15.133:0/2633108757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0fc005020 con 0x556367d74cc0 2024-09-11T23:38:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.643+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2633108757 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe11006a0c0 con 0x556367d74cc0 2024-09-11T23:38:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.643+0000 7fe11963a740 10 client.25457 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.643+0000 7fe11963a740 1 -- 172.21.15.133:0/2633108757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556367ab8620 con 0x556367d74cc0 2024-09-11T23:38:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.643+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2633108757 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0fc005020 con 0x556367d74cc0 2024-09-11T23:38:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.645+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2633108757 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe110097710 con 0x556367d74cc0 2024-09-11T23:38:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.645+0000 7fe11963a740 10 client.25457 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.645+0000 7fe11963a740 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.645+0000 7fe11963a740 20 client.25457 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.645+0000 7fe11963a740 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556367d90220 0x556367d92630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.645+0000 7fe11963a740 4 client.25457 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.645+0000 7fe11963a740 1 -- 172.21.15.133:0/2633108757 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556367ce6a20 con 0x556367d90220 2024-09-11T23:38:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.646+0000 7fe10ffff640 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556367d90220 0x556367d92630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.648+0000 7fe10ffff640 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556367d90220 0x556367d92630 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f50c08ffffc35a server_cookie=55e4a73555aa14ab in_seq=0 out_seq=0 2024-09-11T23:38:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.648+0000 7fe0ef7fe640 10 client.25457 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.649+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2633108757 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556367ce6a20 con 0x556367d90220 2024-09-11T23:38:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.649+0000 7fe0ef7fe640 10 client.25457 handle_command_reply: tid=0 2024-09-11T23:38:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.650+0000 7fe11963a740 1 client.25457 shutdown 2024-09-11T23:38:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.650+0000 7fe11963a740 1 -- 172.21.15.133:0/2633108757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 msgr2=0x556367d7cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.650+0000 7fe11963a740 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 0x556367d7cd80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe11008e9e0 tx=0x7fe11008e3f0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.664+0000 7fe11963a740 1 -- 172.21.15.133:0/2633108757 shutdown_connections 2024-09-11T23:38:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.664+0000 7fe11963a740 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556367d90220 0x556367d92630 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.664+0000 7fe11963a740 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d7fe80 0x556367d7d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.664+0000 7fe11963a740 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d7d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.664+0000 7fe11963a740 1 --2- 172.21.15.133:0/2633108757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 0x556367d7cd80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.664+0000 7fe11963a740 1 -- 172.21.15.133:0/2633108757 >> 172.21.15.133:0/2633108757 conn(0x556367d71230 msgr2=0x556367d76f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.664+0000 7fe0ef7fe640 0 client.25457 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.664+0000 7fe11963a740 1 -- 172.21.15.133:0/2633108757 shutdown_connections 2024-09-11T23:38:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.664+0000 7fe11963a740 1 -- 172.21.15.133:0/2633108757 wait complete. 2024-09-11T23:38:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.665+0000 7fe11963a740 20 client.25457 trim_cache size 0 max 16384 2024-09-11T23:38:47.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:47 smithi138 ceph-mon[35142]: pgmap v63: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 633 MiB/s wr, 1.22k op/s 2024-09-11T23:38:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.745+0000 7fe11963a740 1 Processor -- start 2024-09-11T23:38:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.746+0000 7fe11963a740 1 -- start start 2024-09-11T23:38:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.746+0000 7fe11963a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 0x556367d77730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.747+0000 7fe11963a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d7d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.747+0000 7fe11963a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d7fe80 0x556367d70150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.747+0000 7fe11963a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe108160c00 con 0x556367d755e0 2024-09-11T23:38:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.747+0000 7fe11963a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe108162740 con 0x556367d74cc0 2024-09-11T23:38:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.747+0000 7fe11963a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081612d0 con 0x556367d7fe80 2024-09-11T23:38:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.747+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 0x556367d77730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.747+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 0x556367d77730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58118/0 (socket says 172.21.15.133:58118) 2024-09-11T23:38:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.747+0000 7fe10f7fe640 1 -- 172.21.15.133:0/3314233272 learned_addr learned my addr 172.21.15.133:0/3314233272 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.748+0000 7fe10f7fe640 1 -- 172.21.15.133:0/3314233272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d7fe80 msgr2=0x556367d70150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.748+0000 7fe10f7fe640 1 --2- 172.21.15.133:0/3314233272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d7fe80 0x556367d70150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.748+0000 7fe10f7fe640 1 -- 172.21.15.133:0/3314233272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 msgr2=0x556367d7d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.748+0000 7fe10f7fe640 1 --2- 172.21.15.133:0/3314233272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d7d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.748+0000 7fe10f7fe640 1 -- 172.21.15.133:0/3314233272 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556367d7bd10 con 0x556367d74cc0 2024-09-11T23:38:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.749+0000 7fe10f7fe640 1 --2- 172.21.15.133:0/3314233272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 0x556367d77730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe110067f00 tx=0x7fe110068dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8a063a17161a50 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.750+0000 7fe0f9ffb640 1 -- 172.21.15.133:0/3314233272 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1100b6070 con 0x556367d74cc0 2024-09-11T23:38:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.750+0000 7fe0f9ffb640 1 -- 172.21.15.133:0/3314233272 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe110092070 con 0x556367d74cc0 2024-09-11T23:38:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.752+0000 7fe11963a740 1 -- 172.21.15.133:0/3314233272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 msgr2=0x556367d77730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.752+0000 7fe11963a740 1 --2- 172.21.15.133:0/3314233272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 0x556367d77730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe110067f00 tx=0x7fe110068dd0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.765+0000 7fe11963a740 1 -- 172.21.15.133:0/3314233272 shutdown_connections 2024-09-11T23:38:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.765+0000 7fe11963a740 1 --2- 172.21.15.133:0/3314233272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d7fe80 0x556367d70150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.765+0000 7fe11963a740 1 --2- 172.21.15.133:0/3314233272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d7d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.765+0000 7fe11963a740 1 --2- 172.21.15.133:0/3314233272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d74cc0 0x556367d77730 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.765+0000 7fe11963a740 1 -- 172.21.15.133:0/3314233272 >> 172.21.15.133:0/3314233272 conn(0x556367d71230 msgr2=0x556367d6fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.765+0000 7fe11963a740 1 -- 172.21.15.133:0/3314233272 shutdown_connections 2024-09-11T23:38:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.765+0000 7fe11963a740 1 -- 172.21.15.133:0/3314233272 wait complete. 2024-09-11T23:38:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.766+0000 7fe11963a740 1 -- start start 2024-09-11T23:38:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.767+0000 7fe11963a740 1 Processor -- start 2024-09-11T23:38:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.768+0000 7fe11963a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d74cc0 0x556367d82e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.768+0000 7fe11963a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d83360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.768+0000 7fe11963a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d7fe80 0x556367d838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.768+0000 7fe11963a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe108162070 con 0x556367d755e0 2024-09-11T23:38:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.768+0000 7fe11963a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081619a0 con 0x556367d7fe80 2024-09-11T23:38:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.768+0000 7fe11963a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe10815ece0 con 0x556367d74cc0 2024-09-11T23:38:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.768+0000 7fe11963a740 10 client.0 _ll_register_callbacks cb 0x7fe110068f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.769+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d83360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.769+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d83360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47670/0 (socket says 172.21.15.133:47670) 2024-09-11T23:38:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.769+0000 7fe10ffff640 1 -- 172.21.15.133:0/2259675139 learned_addr learned my addr 172.21.15.133:0/2259675139 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.771+0000 7fe10ffff640 1 -- 172.21.15.133:0/2259675139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d74cc0 msgr2=0x556367d82e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.771+0000 7fe10ffff640 1 --2- 172.21.15.133:0/2259675139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d74cc0 0x556367d82e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.771+0000 7fe10ffff640 1 -- 172.21.15.133:0/2259675139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d7fe80 msgr2=0x556367d838a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:38:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.771+0000 7fe10ffff640 1 --2- 172.21.15.133:0/2259675139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d7fe80 0x556367d838a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.771+0000 7fe10ffff640 1 -- 172.21.15.133:0/2259675139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556367df9150 con 0x556367d755e0 2024-09-11T23:38:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.772+0000 7fe10ffff640 1 --2- 172.21.15.133:0/2259675139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d83360 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005c50 tx=0x7fe0fc0097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f305911b18e08e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.773+0000 7fe0ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.774+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2259675139 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc030060 con 0x556367d755e0 2024-09-11T23:38:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.774+0000 7fe11963a740 1 -- 172.21.15.133:0/2259675139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe110092070 con 0x556367d755e0 2024-09-11T23:38:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.775+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2259675139 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe0fc024650 con 0x556367d755e0 2024-09-11T23:38:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.776+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2259675139 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0fc024870 con 0x556367d755e0 2024-09-11T23:38:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.776+0000 7fe11963a740 10 client.16035 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.776+0000 7fe11963a740 1 -- 172.21.15.133:0/2259675139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556367dfa460 con 0x556367d755e0 2024-09-11T23:38:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.777+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2259675139 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe110092070 con 0x556367d755e0 2024-09-11T23:38:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.782+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2259675139 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe0fc010be0 con 0x556367d755e0 2024-09-11T23:38:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.782+0000 7fe11963a740 10 client.16035 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.782+0000 7fe11963a740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:38:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.782+0000 7fe11963a740 20 client.16035 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.782+0000 7fe11963a740 1 --2- 172.21.15.133:0/2259675139 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556367dfac60 0x556367dfd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.782+0000 7fe11963a740 4 client.16035 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:38:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.782+0000 7fe11963a740 1 -- 172.21.15.133:0/2259675139 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556367ce0d70 con 0x556367dfac60 2024-09-11T23:38:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.793+0000 7fe10f7fe640 1 --2- 172.21.15.133:0/2259675139 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556367dfac60 0x556367dfd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:47.794 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:47 smithi178 ceph-mon[35882]: pgmap v63: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 633 MiB/s wr, 1.22k op/s 2024-09-11T23:38:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.795+0000 7fe10f7fe640 1 --2- 172.21.15.133:0/2259675139 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556367dfac60 0x556367dfd080 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2af52ea9e4bbd1 server_cookie=3f07ec2f4627582f in_seq=0 out_seq=0 2024-09-11T23:38:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:47.796+0000 7fe0ef7fe640 10 client.16035 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:48.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.126+0000 7f271e148640 1 -- 172.21.15.133:0/3311090047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2718109e00 msgr2=0x7f271810e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:48.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.128+0000 7f271e148640 1 --2- 172.21.15.133:0/3311090047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2718109e00 0x7f271810e350 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2708009900 tx=0x7f270802a420 comp rx=0 tx=0).stop 2024-09-11T23:38:48.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.130+0000 7f271e148640 1 -- 172.21.15.133:0/3311090047 shutdown_connections 2024-09-11T23:38:48.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.130+0000 7f271e148640 1 --2- 172.21.15.133:0/3311090047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f271810ea80 0x7f27181154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.130+0000 7f271e148640 1 --2- 172.21.15.133:0/3311090047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2718109e00 0x7f271810e350 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.130+0000 7f271e148640 1 --2- 172.21.15.133:0/3311090047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2718109450 0x7f2718109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.130+0000 7f271e148640 1 -- 172.21.15.133:0/3311090047 >> 172.21.15.133:0/3311090047 conn(0x7f271806e2d0 msgr2=0x7f271806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.130+0000 7f271e148640 1 -- 172.21.15.133:0/3311090047 shutdown_connections 2024-09-11T23:38:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.130+0000 7f271e148640 1 -- 172.21.15.133:0/3311090047 wait complete. 2024-09-11T23:38:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.132+0000 7f271e148640 1 Processor -- start 2024-09-11T23:38:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.133+0000 7f271e148640 1 -- start start 2024-09-11T23:38:48.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.138+0000 7f271e148640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2718109450 0x7f271819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.138+0000 7f271e148640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718109e00 0x7f271819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.139+0000 7f271e148640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f271810ea80 0x7f27181a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.139+0000 7f271e148640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2718118ae0 con 0x7f2718109e00 2024-09-11T23:38:48.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.139+0000 7f271e148640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2718118960 con 0x7f2718109450 2024-09-11T23:38:48.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.139+0000 7f271e148640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2718118c60 con 0x7f271810ea80 2024-09-11T23:38:48.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.140+0000 7f2716ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718109e00 0x7f271819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.140+0000 7f2716ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718109e00 0x7f271819f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47680/0 (socket says 172.21.15.133:47680) 2024-09-11T23:38:48.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.140+0000 7f2716ffd640 1 -- 172.21.15.133:0/3078952292 learned_addr learned my addr 172.21.15.133:0/3078952292 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:48.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.141+0000 7f27177fe640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2718109450 0x7f271819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.141+0000 7f2717fff640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f271810ea80 0x7f27181a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.144+0000 7f27177fe640 1 -- 172.21.15.133:0/3078952292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f271810ea80 msgr2=0x7f27181a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.144+0000 7f27177fe640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f271810ea80 0x7f27181a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.144+0000 7f27177fe640 1 -- 172.21.15.133:0/3078952292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718109e00 msgr2=0x7f271819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.144+0000 7f27177fe640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718109e00 0x7f271819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.144+0000 7f27177fe640 1 -- 172.21.15.133:0/3078952292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27181a4010 con 0x7f2718109450 2024-09-11T23:38:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.147+0000 7f27177fe640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2718109450 0x7f271819f060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2700009fd0 tx=0x7f270000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:48.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.150+0000 7f2714ff9640 1 -- 172.21.15.133:0/3078952292 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700012b80 con 0x7f2718109450 2024-09-11T23:38:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.150+0000 7f2714ff9640 1 -- 172.21.15.133:0/3078952292 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2700012d20 con 0x7f2718109450 2024-09-11T23:38:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.150+0000 7f2714ff9640 1 -- 172.21.15.133:0/3078952292 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27000196e0 con 0x7f2718109450 2024-09-11T23:38:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.151+0000 7f271e148640 1 -- 172.21.15.133:0/3078952292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2718069b10 con 0x7f2718109450 2024-09-11T23:38:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.151+0000 7f271e148640 1 -- 172.21.15.133:0/3078952292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27181a4470 con 0x7f2718109450 2024-09-11T23:38:48.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.161+0000 7f2714ff9640 1 -- 172.21.15.133:0/3078952292 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f270001b400 con 0x7f2718109450 2024-09-11T23:38:48.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.164+0000 7f2714ff9640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27040a2df0 0x7f27040a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.170+0000 7f2720521740 1 Processor -- start 2024-09-11T23:38:48.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.171+0000 7f2720521740 1 -- start start 2024-09-11T23:38:48.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.172+0000 7f2714ff9640 1 -- 172.21.15.133:0/3078952292 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f27000d28d0 con 0x7f2718109450 2024-09-11T23:38:48.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.174+0000 7f2720521740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 0x5629aecd6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.175+0000 7f2720521740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aecd6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.175+0000 7f2720521740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aece3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.175+0000 7f2720521740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5629aec3c2f0 con 0x5629aece1530 2024-09-11T23:38:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.175+0000 7f27177fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 0x5629aecd6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.175+0000 7f27177fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 0x5629aecd6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48998/0 (socket says 172.21.15.133:48998) 2024-09-11T23:38:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.175+0000 7f27177fe640 1 -- 172.21.15.133:0/204931448 learned_addr learned my addr 172.21.15.133:0/204931448 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.176+0000 7f2716ffd640 1 --2- 172.21.15.133:0/204931448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aecd6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.176+0000 7f2720521740 1 -- 172.21.15.133:0/204931448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5629aebe3cf0 con 0x5629aecd6790 2024-09-11T23:38:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.176+0000 7f2720521740 1 -- 172.21.15.133:0/204931448 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5629aec382b0 con 0x5629aecd7b90 2024-09-11T23:38:48.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.178+0000 7f2717fff640 1 --2- 172.21.15.133:0/204931448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aece3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.178+0000 7f2716ffd640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27040a2df0 0x7f27040a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.178+0000 7f2716ffd640 1 -- 172.21.15.133:0/204931448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 msgr2=0x5629aecd6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.178+0000 7f2716ffd640 1 --2- 172.21.15.133:0/204931448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 0x5629aecd6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.179+0000 7f2716ffd640 1 -- 172.21.15.133:0/204931448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 msgr2=0x5629aece3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.179+0000 7f2716ffd640 1 --2- 172.21.15.133:0/204931448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aece3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.179+0000 7f2716ffd640 1 -- 172.21.15.133:0/204931448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629aebc83a0 con 0x5629aecd6790 2024-09-11T23:38:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.179+0000 7f27177fe640 1 --2- 172.21.15.133:0/204931448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 0x5629aecd6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.180+0000 7f2716ffd640 1 --2- 172.21.15.133:0/204931448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aecd6bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2708031550 tx=0x7f27080036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef45415e9a7c0862 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.181+0000 7f271e148640 1 -- 172.21.15.133:0/204931448 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270804e070 con 0x5629aecd6790 2024-09-11T23:38:48.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.182+0000 7f271e148640 1 -- 172.21.15.133:0/204931448 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2708003c00 con 0x5629aecd6790 2024-09-11T23:38:48.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.182+0000 7f271e148640 1 -- 172.21.15.133:0/204931448 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708035030 con 0x5629aecd6790 2024-09-11T23:38:48.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.184+0000 7f2716ffd640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27040a2df0 0x7f27040a52b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2708047000 tx=0x7f270803caf0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:48.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.184+0000 7f2720521740 1 -- 172.21.15.133:0/204931448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 msgr2=0x5629aecd6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:48.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.184+0000 7f2720521740 1 --2- 172.21.15.133:0/204931448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aecd6bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2708031550 tx=0x7f27080036f0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.185+0000 7f2720521740 1 -- 172.21.15.133:0/204931448 shutdown_connections 2024-09-11T23:38:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.186+0000 7f2720521740 1 --2- 172.21.15.133:0/204931448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aece3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.186+0000 7f2720521740 1 --2- 172.21.15.133:0/204931448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aecd6bf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.186+0000 7f2720521740 1 --2- 172.21.15.133:0/204931448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 0x5629aecd6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.186+0000 7f2720521740 1 -- 172.21.15.133:0/204931448 >> 172.21.15.133:0/204931448 conn(0x5629aec9f310 msgr2=0x5629aecd2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.186+0000 7f2720521740 1 -- 172.21.15.133:0/204931448 shutdown_connections 2024-09-11T23:38:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.186+0000 7f2720521740 1 -- 172.21.15.133:0/204931448 wait complete. 2024-09-11T23:38:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.187+0000 7f2720521740 1 -- start start 2024-09-11T23:38:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.188+0000 7f2720521740 1 Processor -- start 2024-09-11T23:38:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.189+0000 7f2720521740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd6790 0x5629aece0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.189+0000 7f2720521740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd7b90 0x5629aecde180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.189+0000 7f2720521740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aecde720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.190+0000 7f2720521740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5629aece4040 con 0x5629aece1530 2024-09-11T23:38:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.190+0000 7f2720521740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5629aebf7060 con 0x5629aecd7b90 2024-09-11T23:38:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.190+0000 7f2720521740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5629aece41c0 con 0x5629aecd6790 2024-09-11T23:38:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.190+0000 7f2720521740 10 client.0 _ll_register_callbacks cb 0x5629ae94a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.190+0000 7f2716ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd7b90 0x5629aecde180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.190+0000 7f2716ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd7b90 0x5629aecde180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58154/0 (socket says 172.21.15.133:58154) 2024-09-11T23:38:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.190+0000 7f2716ffd640 1 -- 172.21.15.133:0/4232830952 learned_addr learned my addr 172.21.15.133:0/4232830952 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:48.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.200+0000 7f27177fe640 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd6790 0x5629aece0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.203+0000 7f2716ffd640 1 -- 172.21.15.133:0/4232830952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd6790 msgr2=0x5629aece0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:48.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.203+0000 7f2716ffd640 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd6790 0x5629aece0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.203+0000 7f2716ffd640 1 -- 172.21.15.133:0/4232830952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 msgr2=0x5629aecde720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:48.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.203+0000 7f2716ffd640 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aecde720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.203+0000 7f2716ffd640 1 -- 172.21.15.133:0/4232830952 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ae9ad3c0 con 0x5629aecd7b90 2024-09-11T23:38:48.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.204+0000 7f2716ffd640 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd7b90 0x5629aecde180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2708004650 tx=0x7f2708005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c8f36a1ea5d0c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:48.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.204+0000 7f26f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:38:48.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.204+0000 7f26f6ffd640 1 -- 172.21.15.133:0/4232830952 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270805a030 con 0x5629aecd7b90 2024-09-11T23:38:48.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.205+0000 7f26f6ffd640 1 -- 172.21.15.133:0/4232830952 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f2708003a00 con 0x5629aecd7b90 2024-09-11T23:38:48.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.205+0000 7f2720521740 1 -- 172.21.15.133:0/4232830952 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2708003c00 con 0x5629aecd7b90 2024-09-11T23:38:48.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.205+0000 7f26f6ffd640 1 -- 172.21.15.133:0/4232830952 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2708066050 con 0x5629aecd7b90 2024-09-11T23:38:48.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.205+0000 7f2720521740 10 client.25472 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:48.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.205+0000 7f2720521740 1 -- 172.21.15.133:0/4232830952 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5629aecefb30 con 0x5629aecd7b90 2024-09-11T23:38:48.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.206+0000 7f26f6ffd640 1 -- 172.21.15.133:0/4232830952 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f270803c6b0 con 0x5629aecd7b90 2024-09-11T23:38:48.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.206+0000 7f26f6ffd640 1 -- 172.21.15.133:0/4232830952 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27080047a0 con 0x5629aecd7b90 2024-09-11T23:38:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.206+0000 7f2720521740 10 client.25472 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.206+0000 7f2720521740 10 client.25472 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:38:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.207+0000 7f2720521740 20 client.25472 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.207+0000 7f2720521740 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5629aecf0e80 0x5629aecf3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.207+0000 7f2720521740 4 client.25472 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.207+0000 7f2720521740 1 -- 172.21.15.133:0/4232830952 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2708003c00 con 0x5629aecf0e80 2024-09-11T23:38:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.207+0000 7f27177fe640 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5629aecf0e80 0x5629aecf3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.209+0000 7f26f6ffd640 10 client.25472 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:38:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.209+0000 7f27177fe640 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5629aecf0e80 0x5629aecf3270 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b0d43a75b745c909 server_cookie=3076932d2f974704 in_seq=0 out_seq=0 2024-09-11T23:38:48.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.212+0000 7f26f6ffd640 1 -- 172.21.15.133:0/4232830952 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2708003c00 con 0x5629aecf0e80 2024-09-11T23:38:48.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.212+0000 7f26f6ffd640 10 client.25472 handle_command_reply: tid=0 2024-09-11T23:38:48.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.213+0000 7f2720521740 1 client.25472 shutdown 2024-09-11T23:38:48.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.213+0000 7f2720521740 1 -- 172.21.15.133:0/4232830952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd7b90 msgr2=0x5629aecde180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:48.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.213+0000 7f2720521740 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd7b90 0x5629aecde180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2708004650 tx=0x7f2708005e60 comp rx=0 tx=0).stop 2024-09-11T23:38:48.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.213+0000 7f2720521740 1 -- 172.21.15.133:0/4232830952 shutdown_connections 2024-09-11T23:38:48.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.213+0000 7f2720521740 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5629aecf0e80 0x5629aecf3270 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.213+0000 7f2720521740 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aecde720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.214+0000 7f2720521740 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd7b90 0x5629aecde180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.214+0000 7f2720521740 1 --2- 172.21.15.133:0/4232830952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd6790 0x5629aece0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.214+0000 7f2720521740 1 -- 172.21.15.133:0/4232830952 >> 172.21.15.133:0/4232830952 conn(0x5629aec9f310 msgr2=0x5629aecd8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:48.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.214+0000 7f26f6ffd640 0 client.25472 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:38:48.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.214+0000 7f2720521740 1 -- 172.21.15.133:0/4232830952 shutdown_connections 2024-09-11T23:38:48.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.214+0000 7f2720521740 1 -- 172.21.15.133:0/4232830952 wait complete. 2024-09-11T23:38:48.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.214+0000 7f2720521740 20 client.25472 trim_cache size 0 max 16384 2024-09-11T23:38:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.258+0000 7f2720521740 1 Processor -- start 2024-09-11T23:38:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.258+0000 7f2720521740 1 -- start start 2024-09-11T23:38:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.264+0000 7f2720521740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aeced380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.265+0000 7f2720521740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 0x5629aecd9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.265+0000 7f2720521740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aecd98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.265+0000 7f2720521740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f27181b3d40 con 0x5629aece1530 2024-09-11T23:38:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.265+0000 7f2720521740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27181b5880 con 0x5629aecd6790 2024-09-11T23:38:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.265+0000 7f2720521740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f27181b4410 con 0x5629aecd7b90 2024-09-11T23:38:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.266+0000 7f2717fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aecd98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.266+0000 7f2717fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aecd98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47716/0 (socket says 172.21.15.133:47716) 2024-09-11T23:38:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.266+0000 7f2717fff640 1 -- 172.21.15.133:0/559737182 learned_addr learned my addr 172.21.15.133:0/559737182 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:48.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.267+0000 7f2716ffd640 1 --2- 172.21.15.133:0/559737182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 0x5629aecd9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.267+0000 7f27177fe640 1 --2- 172.21.15.133:0/559737182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aeced380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.268+0000 7f2716ffd640 1 -- 172.21.15.133:0/559737182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 msgr2=0x5629aeced380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:48.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.268+0000 7f2716ffd640 1 --2- 172.21.15.133:0/559737182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aeced380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.268+0000 7f2716ffd640 1 -- 172.21.15.133:0/559737182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 msgr2=0x5629aecd98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:48.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.268+0000 7f2716ffd640 1 --2- 172.21.15.133:0/559737182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aecd98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.268+0000 7f2716ffd640 1 -- 172.21.15.133:0/559737182 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629aecdc6d0 con 0x5629aecd7b90 2024-09-11T23:38:48.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.275+0000 7f2716ffd640 1 --2- 172.21.15.133:0/559737182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 0x5629aecd9390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f27080096c0 tx=0x7f270802a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=197b459cf241d22c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:48.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.275+0000 7f270d7fa640 1 -- 172.21.15.133:0/559737182 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270805a070 con 0x5629aecd7b90 2024-09-11T23:38:48.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.279+0000 7f270d7fa640 1 -- 172.21.15.133:0/559737182 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f270805b040 con 0x5629aecd7b90 2024-09-11T23:38:48.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.280+0000 7f2720521740 1 -- 172.21.15.133:0/559737182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 msgr2=0x5629aecd9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:48.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.280+0000 7f2720521740 1 --2- 172.21.15.133:0/559737182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 0x5629aecd9390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f27080096c0 tx=0x7f270802a420 comp rx=0 tx=0).stop 2024-09-11T23:38:48.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.282+0000 7f2720521740 1 -- 172.21.15.133:0/559737182 shutdown_connections 2024-09-11T23:38:48.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.282+0000 7f2720521740 1 --2- 172.21.15.133:0/559737182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aece1530 0x5629aecd98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.282+0000 7f2720521740 1 --2- 172.21.15.133:0/559737182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aecd7b90 0x5629aecd9390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.282+0000 7f2720521740 1 --2- 172.21.15.133:0/559737182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aeced380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.282+0000 7f2720521740 1 -- 172.21.15.133:0/559737182 >> 172.21.15.133:0/559737182 conn(0x5629aec9f310 msgr2=0x5629aecd82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:48.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.285+0000 7f2720521740 1 -- 172.21.15.133:0/559737182 shutdown_connections 2024-09-11T23:38:48.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.285+0000 7f2720521740 1 -- 172.21.15.133:0/559737182 wait complete. 2024-09-11T23:38:48.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.286+0000 7f2720521740 1 -- start start 2024-09-11T23:38:48.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.287+0000 7f2720521740 1 Processor -- start 2024-09-11T23:38:48.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.287+0000 7f2720521740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aece9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.288+0000 7f2720521740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aecd7b90 0x5629aece9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.288+0000 7f27177fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aece9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.288+0000 7f27177fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aece9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58178/0 (socket says 172.21.15.133:58178) 2024-09-11T23:38:48.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.288+0000 7f27177fe640 1 -- 172.21.15.133:0/512044521 learned_addr learned my addr 172.21.15.133:0/512044521 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:48.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.288+0000 7f2720521740 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aece1530 0x5629aecda9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.288+0000 7f2720521740 1 -- 172.21.15.133:0/512044521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f27181b51b0 con 0x5629aecd7b90 2024-09-11T23:38:48.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.288+0000 7f2720521740 1 -- 172.21.15.133:0/512044521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27181b4ae0 con 0x5629aecd6790 2024-09-11T23:38:48.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.288+0000 7f2720521740 1 -- 172.21.15.133:0/512044521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f27181b1d40 con 0x5629aece1530 2024-09-11T23:38:48.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.288+0000 7f2717fff640 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aece1530 0x5629aecda9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.289+0000 7f2720521740 10 client.0 _ll_register_callbacks cb 0x7f270804c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:48.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.290+0000 7f2717fff640 1 -- 172.21.15.133:0/512044521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 msgr2=0x5629aece9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:48.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.290+0000 7f2717fff640 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aece9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.290+0000 7f2717fff640 1 -- 172.21.15.133:0/512044521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aecd7b90 msgr2=0x5629aece9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:48.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.290+0000 7f2717fff640 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aecd7b90 0x5629aece9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.290+0000 7f2717fff640 1 -- 172.21.15.133:0/512044521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629aecdb1f0 con 0x5629aece1530 2024-09-11T23:38:48.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.294+0000 7f2717fff640 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aece1530 0x5629aecda9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f271000afd0 tx=0x7f2710007e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6be5f1255adf9c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:48.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.301+0000 7f26f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:38:48.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.301+0000 7f26f6ffd640 1 -- 172.21.15.133:0/512044521 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710013ca0 con 0x5629aece1530 2024-09-11T23:38:48.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.301+0000 7f2720521740 1 -- 172.21.15.133:0/512044521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f270805b040 con 0x5629aece1530 2024-09-11T23:38:48.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.301+0000 7f26f6ffd640 1 -- 172.21.15.133:0/512044521 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f2710013e40 con 0x5629aece1530 2024-09-11T23:38:48.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.306+0000 7f26f6ffd640 1 -- 172.21.15.133:0/512044521 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2710016420 con 0x5629aece1530 2024-09-11T23:38:48.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.306+0000 7f2720521740 10 client.25495 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:48.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.306+0000 7f2720521740 1 -- 172.21.15.133:0/512044521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5629aed5b3b0 con 0x5629aece1530 2024-09-11T23:38:48.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.307+0000 7f26f6ffd640 1 -- 172.21.15.133:0/512044521 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f270805b040 con 0x5629aece1530 2024-09-11T23:38:48.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.308+0000 7f26f6ffd640 1 -- 172.21.15.133:0/512044521 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2710029980 con 0x5629aece1530 2024-09-11T23:38:48.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.308+0000 7f2720521740 10 client.25495 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:48.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.308+0000 7f2720521740 10 client.25495 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:38:48.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.308+0000 7f2720521740 20 client.25495 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:48.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.308+0000 7f2720521740 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5629aed5bb10 0x5629aed5df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:48.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.308+0000 7f2720521740 4 client.25495 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:38:48.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.308+0000 7f2720521740 1 -- 172.21.15.133:0/512044521 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629aec49860 con 0x5629aed5bb10 2024-09-11T23:38:48.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.312+0000 7f27177fe640 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5629aed5bb10 0x5629aed5df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:48.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.328+0000 7f27177fe640 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5629aed5bb10 0x5629aed5df70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8baa7c809d0226b server_cookie=d9e62351c09c9bb4 in_seq=0 out_seq=0 2024-09-11T23:38:48.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.328+0000 7f26f6ffd640 10 client.25495 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:38:48.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.331+0000 7f26f6ffd640 1 -- 172.21.15.133:0/512044521 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629aec49860 con 0x5629aed5bb10 2024-09-11T23:38:48.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.331+0000 7f26f6ffd640 10 client.25495 handle_command_reply: tid=0 2024-09-11T23:38:48.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.331+0000 7f2720521740 1 client.25495 shutdown 2024-09-11T23:38:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.336+0000 7f2720521740 1 -- 172.21.15.133:0/512044521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aece1530 msgr2=0x5629aecda9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.336+0000 7f2720521740 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aece1530 0x5629aecda9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f271000afd0 tx=0x7f2710007e60 comp rx=0 tx=0).stop 2024-09-11T23:38:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.336+0000 7f2720521740 1 -- 172.21.15.133:0/512044521 shutdown_connections 2024-09-11T23:38:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.336+0000 7f2720521740 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5629aed5bb10 0x5629aed5df70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.337+0000 7f2720521740 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629aece1530 0x5629aecda9f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.337+0000 7f2720521740 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629aecd7b90 0x5629aece9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.337+0000 7f2720521740 1 --2- 172.21.15.133:0/512044521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629aecd6790 0x5629aece9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.337+0000 7f2720521740 1 -- 172.21.15.133:0/512044521 >> 172.21.15.133:0/512044521 conn(0x5629aec9f310 msgr2=0x5629aecdc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.337+0000 7f26f6ffd640 0 client.25495 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:38:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.337+0000 7f2720521740 1 -- 172.21.15.133:0/512044521 shutdown_connections 2024-09-11T23:38:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.337+0000 7f2720521740 1 -- 172.21.15.133:0/512044521 wait complete. 2024-09-11T23:38:48.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.337+0000 7f2720521740 20 client.25495 trim_cache size 0 max 16384 2024-09-11T23:38:48.342 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:38:48.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.344+0000 7f270e7fc640 1 -- 172.21.15.133:0/3078952292 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27040a2df0 msgr2=0x7f27040a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:48.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.344+0000 7f270e7fc640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27040a2df0 0x7f27040a52b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2708047000 tx=0x7f270803caf0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.344+0000 7f270e7fc640 1 -- 172.21.15.133:0/3078952292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2718109450 msgr2=0x7f271819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.344+0000 7f270e7fc640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2718109450 0x7f271819f060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2700009fd0 tx=0x7f270000b870 comp rx=0 tx=0).stop 2024-09-11T23:38:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.345+0000 7f270e7fc640 1 -- 172.21.15.133:0/3078952292 shutdown_connections 2024-09-11T23:38:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.345+0000 7f270e7fc640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f27040a2df0 0x7f27040a52b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.345+0000 7f270e7fc640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f271810ea80 0x7f27181a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.345+0000 7f270e7fc640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718109e00 0x7f271819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.345+0000 7f270e7fc640 1 --2- 172.21.15.133:0/3078952292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2718109450 0x7f271819f060 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.345+0000 7f270e7fc640 1 -- 172.21.15.133:0/3078952292 >> 172.21.15.133:0/3078952292 conn(0x7f271806e2d0 msgr2=0x7f271810c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:48.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.347+0000 7f270e7fc640 1 -- 172.21.15.133:0/3078952292 shutdown_connections 2024-09-11T23:38:48.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:48.347+0000 7f270e7fc640 1 -- 172.21.15.133:0/3078952292 wait complete. 2024-09-11T23:38:49.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:49 smithi138 ceph-mon[35142]: pgmap v64: 97 pgs: 97 active+clean; 10 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 948 MiB/s wr, 1.52k op/s 2024-09-11T23:38:49.712 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:49 smithi133 ceph-mon[31430]: pgmap v64: 97 pgs: 97 active+clean; 10 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 948 MiB/s wr, 1.52k op/s 2024-09-11T23:38:49.755 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-11T23:38:49.756 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:38:49.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:49 smithi178 ceph-mon[35882]: pgmap v64: 97 pgs: 97 active+clean; 10 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 948 MiB/s wr, 1.52k op/s 2024-09-11T23:38:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.845+0000 7fe0ef7fe640 1 -- 172.21.15.133:0/2259675139 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556367ce0d70 con 0x556367dfac60 2024-09-11T23:38:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.845+0000 7fe0ef7fe640 10 client.16035 handle_command_reply: tid=0 2024-09-11T23:38:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.845+0000 7fe11963a740 1 client.16035 shutdown 2024-09-11T23:38:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.847+0000 7fe11963a740 1 -- 172.21.15.133:0/2259675139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 msgr2=0x556367d83360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.847+0000 7fe11963a740 1 --2- 172.21.15.133:0/2259675139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d83360 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005c50 tx=0x7fe0fc0097c0 comp rx=0 tx=0).stop 2024-09-11T23:38:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.848+0000 7fe11963a740 1 -- 172.21.15.133:0/2259675139 shutdown_connections 2024-09-11T23:38:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.848+0000 7fe11963a740 1 --2- 172.21.15.133:0/2259675139 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556367dfac60 0x556367dfd080 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.848+0000 7fe11963a740 1 --2- 172.21.15.133:0/2259675139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556367d7fe80 0x556367d838a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.848+0000 7fe11963a740 1 --2- 172.21.15.133:0/2259675139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556367d755e0 0x556367d83360 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.848+0000 7fe11963a740 1 --2- 172.21.15.133:0/2259675139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556367d74cc0 0x556367d82e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.848+0000 7fe11963a740 1 -- 172.21.15.133:0/2259675139 >> 172.21.15.133:0/2259675139 conn(0x556367d71230 msgr2=0x556367d70260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.848+0000 7fe0ef7fe640 0 client.16035 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.848+0000 7fe11963a740 1 -- 172.21.15.133:0/2259675139 shutdown_connections 2024-09-11T23:38:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.849+0000 7fe11963a740 1 -- 172.21.15.133:0/2259675139 wait complete. 2024-09-11T23:38:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.849+0000 7fe11963a740 20 client.16035 trim_cache size 0 max 16384 2024-09-11T23:38:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:38:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:38:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:38:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:38:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.867+0000 7fe115562640 1 -- 172.21.15.133:0/2521730765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe1000a2d60 msgr2=0x7fe1000a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.867+0000 7fe115562640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe1000a2d60 0x7fe1000a5220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe104097310 tx=0x7fe1040047d0 comp rx=0 tx=0).stop 2024-09-11T23:38:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.868+0000 7fe115562640 1 -- 172.21.15.133:0/2521730765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080b7540 msgr2=0x7fe1080aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.868+0000 7fe115562640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080b7540 0x7fe1080aec00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe104009fd0 tx=0x7fe10400b8e0 comp rx=0 tx=0).stop 2024-09-11T23:38:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.868+0000 7fe115562640 1 -- 172.21.15.133:0/2521730765 shutdown_connections 2024-09-11T23:38:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.868+0000 7fe115562640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe1000a2d60 0x7fe1000a5220 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.868+0000 7fe115562640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080b7540 0x7fe1080aec00 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.868+0000 7fe115562640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1080a5140 0x7fe1080ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.868+0000 7fe115562640 1 --2- 172.21.15.133:0/2521730765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1080a4820 0x7fe1080b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.868+0000 7fe115562640 1 -- 172.21.15.133:0/2521730765 >> 172.21.15.133:0/2521730765 conn(0x7fe10801a110 msgr2=0x7fe1080a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.869+0000 7fe115562640 1 -- 172.21.15.133:0/2521730765 shutdown_connections 2024-09-11T23:38:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:38:49.869+0000 7fe115562640 1 -- 172.21.15.133:0/2521730765 wait complete. 2024-09-11T23:38:50.000 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:38:51.227 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:51.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:51 smithi178 ceph-mon[35882]: pgmap v65: 97 pgs: 97 active+clean; 10 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 622 MiB/s wr, 1.21k op/s 2024-09-11T23:38:51.618 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:51 smithi133 ceph-mon[31430]: pgmap v65: 97 pgs: 97 active+clean; 10 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 622 MiB/s wr, 1.21k op/s 2024-09-11T23:38:51.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:51 smithi138 ceph-mon[35142]: pgmap v65: 97 pgs: 97 active+clean; 10 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 622 MiB/s wr, 1.21k op/s 2024-09-11T23:38:52.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.079+0000 7efc7f59e640 1 -- 172.21.15.133:0/571984999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc801008d0 msgr2=0x7efc80100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.080+0000 7efc7f59e640 1 --2- 172.21.15.133:0/571984999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc801008d0 0x7efc80100d30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efc74009900 tx=0x7efc7402a440 comp rx=0 tx=0).stop 2024-09-11T23:38:52.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.082+0000 7efc7f59e640 1 -- 172.21.15.133:0/571984999 shutdown_connections 2024-09-11T23:38:52.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.082+0000 7efc7f59e640 1 --2- 172.21.15.133:0/571984999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc80101270 0x7efc8010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.082+0000 7efc7f59e640 1 --2- 172.21.15.133:0/571984999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc801008d0 0x7efc80100d30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.083+0000 7efc7f59e640 1 --2- 172.21.15.133:0/571984999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc801068d0 0x7efc80106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.083+0000 7efc7f59e640 1 -- 172.21.15.133:0/571984999 >> 172.21.15.133:0/571984999 conn(0x7efc800fc610 msgr2=0x7efc800fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:52.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.086+0000 7efc7f59e640 1 -- 172.21.15.133:0/571984999 shutdown_connections 2024-09-11T23:38:52.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.088+0000 7efc7f59e640 1 -- 172.21.15.133:0/571984999 wait complete. 2024-09-11T23:38:52.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.090+0000 7efc7f59e640 1 Processor -- start 2024-09-11T23:38:52.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.090+0000 7efc7f59e640 1 -- start start 2024-09-11T23:38:52.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.091+0000 7efc7f59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc801008d0 0x7efc801972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.091+0000 7efc7f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc80101270 0x7efc80197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.092+0000 7efc7f59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc801068d0 0x7efc8019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.092+0000 7efc7f59e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efc80110de0 con 0x7efc80101270 2024-09-11T23:38:52.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.092+0000 7efc7f59e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc80110c60 con 0x7efc801008d0 2024-09-11T23:38:52.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.092+0000 7efc7f59e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc80110f60 con 0x7efc801068d0 2024-09-11T23:38:52.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.096+0000 7efc7ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc801068d0 0x7efc8019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.096+0000 7efc7ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc801068d0 0x7efc8019bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35258/0 (socket says 172.21.15.133:35258) 2024-09-11T23:38:52.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.096+0000 7efc7ed9d640 1 -- 172.21.15.133:0/270670840 learned_addr learned my addr 172.21.15.133:0/270670840 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:52.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.096+0000 7efc7e59c640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc801008d0 0x7efc801972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.096+0000 7efc7dd9b640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc80101270 0x7efc80197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.099+0000 7efc7ed9d640 1 -- 172.21.15.133:0/270670840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc801008d0 msgr2=0x7efc801972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.099+0000 7efc7ed9d640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc801008d0 0x7efc801972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.099+0000 7efc7ed9d640 1 -- 172.21.15.133:0/270670840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc80101270 msgr2=0x7efc80197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.099+0000 7efc7ed9d640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc80101270 0x7efc80197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.099+0000 7efc7ed9d640 1 -- 172.21.15.133:0/270670840 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8019c270 con 0x7efc801068d0 2024-09-11T23:38:52.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.103+0000 7efc7e59c640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc801008d0 0x7efc801972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:38:52.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.104+0000 7efc7ed9d640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc801068d0 0x7efc8019bb90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efc7800b990 tx=0x7efc7800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:52.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.108+0000 7efc6f7fe640 1 -- 172.21.15.133:0/270670840 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78019070 con 0x7efc801068d0 2024-09-11T23:38:52.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.108+0000 7efc7f59e640 1 -- 172.21.15.133:0/270670840 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8019c560 con 0x7efc801068d0 2024-09-11T23:38:52.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.109+0000 7efc7f59e640 1 -- 172.21.15.133:0/270670840 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc80069f70 con 0x7efc801068d0 2024-09-11T23:38:52.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.111+0000 7efc6f7fe640 1 -- 172.21.15.133:0/270670840 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc7800cba0 con 0x7efc801068d0 2024-09-11T23:38:52.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.111+0000 7efc6d7fa640 1 -- 172.21.15.133:0/270670840 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc60005290 con 0x7efc801068d0 2024-09-11T23:38:52.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.111+0000 7efc6f7fe640 1 -- 172.21.15.133:0/270670840 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78007650 con 0x7efc801068d0 2024-09-11T23:38:52.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.120+0000 7efc6f7fe640 1 -- 172.21.15.133:0/270670840 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7efc7801c070 con 0x7efc801068d0 2024-09-11T23:38:52.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.123+0000 7efc6f7fe640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efc680a2f00 0x7efc680a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:52.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.124+0000 7efc6f7fe640 1 -- 172.21.15.133:0/270670840 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7efc780cf110 con 0x7efc801068d0 2024-09-11T23:38:52.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.125+0000 7efc6f7fe640 1 -- 172.21.15.133:0/270670840 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc780145d0 con 0x7efc801068d0 2024-09-11T23:38:52.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.130+0000 7efc7e59c640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efc680a2f00 0x7efc680a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:52.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.131+0000 7efc7e59c640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efc680a2f00 0x7efc680a53c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc70006fd0 tx=0x7efc70008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:52.224 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:52.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.632+0000 7efc6d7fa640 1 -- 172.21.15.133:0/270670840 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc60005580 con 0x7efc801068d0 2024-09-11T23:38:52.653 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:38:52.653 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:38:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.642+0000 7efc6f7fe640 1 -- 172.21.15.133:0/270670840 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7efc7809bed0 con 0x7efc801068d0 2024-09-11T23:38:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:38:52.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.658+0000 7efc6d7fa640 1 -- 172.21.15.133:0/270670840 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efc680a2f00 msgr2=0x7efc680a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.658+0000 7efc6d7fa640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efc680a2f00 0x7efc680a53c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc70006fd0 tx=0x7efc70008040 comp rx=0 tx=0).stop 2024-09-11T23:38:52.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.658+0000 7efc6d7fa640 1 -- 172.21.15.133:0/270670840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc801068d0 msgr2=0x7efc8019bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:52.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.658+0000 7efc6d7fa640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc801068d0 0x7efc8019bb90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efc7800b990 tx=0x7efc7800be50 comp rx=0 tx=0).stop 2024-09-11T23:38:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.662+0000 7efc6d7fa640 1 -- 172.21.15.133:0/270670840 shutdown_connections 2024-09-11T23:38:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.662+0000 7efc6d7fa640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efc680a2f00 0x7efc680a53c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.663+0000 7efc6d7fa640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc801068d0 0x7efc8019bb90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.663+0000 7efc6d7fa640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc80101270 0x7efc80197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.663+0000 7efc6d7fa640 1 --2- 172.21.15.133:0/270670840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc801008d0 0x7efc801972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.663+0000 7efc6d7fa640 1 -- 172.21.15.133:0/270670840 >> 172.21.15.133:0/270670840 conn(0x7efc800fc610 msgr2=0x7efc800fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:52.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.669+0000 7efc6d7fa640 1 -- 172.21.15.133:0/270670840 shutdown_connections 2024-09-11T23:38:52.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:52.669+0000 7efc6d7fa640 1 -- 172.21.15.133:0/270670840 wait complete. 2024-09-11T23:38:53.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:52 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/270670840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:53.077 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:52 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/270670840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:53.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:52 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/270670840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:38:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.373+0000 7f9dd5ce6640 1 -- 172.21.15.133:0/4191732134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dd01008c0 msgr2=0x7f9dd0100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.373+0000 7f9dd5ce6640 1 --2- 172.21.15.133:0/4191732134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dd01008c0 0x7f9dd0100ca0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9dc001a910 tx=0x7f9dc003b470 comp rx=0 tx=0).stop 2024-09-11T23:38:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.373+0000 7f9dd5ce6640 1 -- 172.21.15.133:0/4191732134 shutdown_connections 2024-09-11T23:38:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.373+0000 7f9dd5ce6640 1 --2- 172.21.15.133:0/4191732134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd0105ec0 0x7f9dd0106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.373+0000 7f9dd5ce6640 1 --2- 172.21.15.133:0/4191732134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dd01011e0 0x7f9dd0105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.373+0000 7f9dd5ce6640 1 --2- 172.21.15.133:0/4191732134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dd01008c0 0x7f9dd0100ca0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.373+0000 7f9dd5ce6640 1 -- 172.21.15.133:0/4191732134 >> 172.21.15.133:0/4191732134 conn(0x7f9dd00fc7d0 msgr2=0x7f9dd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.373+0000 7f9dd5ce6640 1 -- 172.21.15.133:0/4191732134 shutdown_connections 2024-09-11T23:38:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.373+0000 7f9dd5ce6640 1 -- 172.21.15.133:0/4191732134 wait complete. 2024-09-11T23:38:53.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.380+0000 7f9dd5ce6640 1 Processor -- start 2024-09-11T23:38:53.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.380+0000 7f9dd5ce6640 1 -- start start 2024-09-11T23:38:53.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.381+0000 7f9dd5ce6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dd01008c0 0x7f9dd0114d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.385+0000 7f9dd5ce6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dd01011e0 0x7f9dd0110e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.386+0000 7f9dd5ce6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd0105ec0 0x7f9dd0111360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.386+0000 7f9dcf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dd01008c0 0x7f9dd0114d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.386+0000 7f9dd5ce6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0109980 con 0x7f9dd01008c0 2024-09-11T23:38:53.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.386+0000 7f9dd5ce6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0109800 con 0x7f9dd01011e0 2024-09-11T23:38:53.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.386+0000 7f9dd5ce6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0109b00 con 0x7f9dd0105ec0 2024-09-11T23:38:53.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.388+0000 7f9dcf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dd01008c0 0x7f9dd0114d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40222/0 (socket says 172.21.15.133:40222) 2024-09-11T23:38:53.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.388+0000 7f9dcf7fe640 1 -- 172.21.15.133:0/2861015943 learned_addr learned my addr 172.21.15.133:0/2861015943 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:53.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.388+0000 7f9dcffff640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd0105ec0 0x7f9dd0111360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.388+0000 7f9dceffd640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dd01011e0 0x7f9dd0110e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.390+0000 7f9dcf7fe640 1 -- 172.21.15.133:0/2861015943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd0105ec0 msgr2=0x7f9dd0111360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.390+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd0105ec0 0x7f9dd0111360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.390+0000 7f9dcf7fe640 1 -- 172.21.15.133:0/2861015943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dd01011e0 msgr2=0x7f9dd0110e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.390+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dd01011e0 0x7f9dd0110e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.390+0000 7f9dcf7fe640 1 -- 172.21.15.133:0/2861015943 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd0111950 con 0x7f9dd01008c0 2024-09-11T23:38:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.392+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dd01008c0 0x7f9dd0114d80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0004d80 tx=0x7f9dc0004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:53.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.393+0000 7f9dccff9640 1 -- 172.21.15.133:0/2861015943 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc004f020 con 0x7f9dd01008c0 2024-09-11T23:38:53.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.393+0000 7f9dd5ce6640 1 -- 172.21.15.133:0/2861015943 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd0111be0 con 0x7f9dd01008c0 2024-09-11T23:38:53.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.394+0000 7f9dd5ce6640 1 -- 172.21.15.133:0/2861015943 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd01a2c80 con 0x7f9dd01008c0 2024-09-11T23:38:53.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.404+0000 7f9dd80bf740 1 Processor -- start 2024-09-11T23:38:53.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.404+0000 7f9dd80bf740 1 -- start start 2024-09-11T23:38:53.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.405+0000 7f9dccff9640 1 -- 172.21.15.133:0/2861015943 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc0004260 con 0x7f9dd01008c0 2024-09-11T23:38:53.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.405+0000 7f9dccff9640 1 -- 172.21.15.133:0/2861015943 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0046070 con 0x7f9dd01008c0 2024-09-11T23:38:53.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.405+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830fe00 0x55a9d830e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.405+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830e890 0x55a9d830ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.406+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d8319630 0x55a9d831ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.406+0000 7f9dd80bf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a9d82742f0 con 0x55a9d830fe00 2024-09-11T23:38:53.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.406+0000 7f9dd80bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a9d821bcf0 con 0x55a9d8319630 2024-09-11T23:38:53.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.406+0000 7f9dd80bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a9d82702b0 con 0x55a9d830e890 2024-09-11T23:38:53.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.406+0000 7f9dcf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830e890 0x55a9d830ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.406+0000 7f9dcf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830e890 0x55a9d830ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35286/0 (socket says 172.21.15.133:35286) 2024-09-11T23:38:53.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.406+0000 7f9dcf7fe640 1 -- 172.21.15.133:0/619012141 learned_addr learned my addr 172.21.15.133:0/619012141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:53.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.407+0000 7f9dceffd640 1 --2- 172.21.15.133:0/619012141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d8319630 0x55a9d831ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.408+0000 7f9dcf7fe640 1 -- 172.21.15.133:0/619012141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d8319630 msgr2=0x55a9d831ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:53.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.408+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/619012141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d8319630 0x55a9d831ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.408+0000 7f9dcf7fe640 1 -- 172.21.15.133:0/619012141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830fe00 msgr2=0x55a9d830e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:53.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.408+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/619012141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830fe00 0x55a9d830e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.408+0000 7f9dcf7fe640 1 -- 172.21.15.133:0/619012141 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d82003a0 con 0x55a9d830e890 2024-09-11T23:38:53.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.415+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/619012141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830e890 0x55a9d830ecd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0005b70 tx=0x7f9dc0005ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e63a62d06d750010 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:53.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.415+0000 7f9dccff9640 1 -- 172.21.15.133:0/2861015943 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9dc0052020 con 0x7f9dd01008c0 2024-09-11T23:38:53.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.417+0000 7f9dccff9640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9db80a2e40 0x7f9db80a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.417+0000 7f9dccff9640 1 -- 172.21.15.133:0/2861015943 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f9dc0066070 con 0x7f9dd01008c0 2024-09-11T23:38:53.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.418+0000 7f9db57fa640 1 -- 172.21.15.133:0/619012141 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00cb220 con 0x55a9d830e890 2024-09-11T23:38:53.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.418+0000 7f9db57fa640 1 -- 172.21.15.133:0/619012141 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc00cf680 con 0x55a9d830e890 2024-09-11T23:38:53.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.421+0000 7f9dcffff640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9db80a2e40 0x7f9db80a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.423+0000 7f9dd80bf740 1 -- 172.21.15.133:0/619012141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830e890 msgr2=0x55a9d830ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.423+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/619012141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830e890 0x55a9d830ecd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0005b70 tx=0x7f9dc0005ba0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.425+0000 7f9dd80bf740 1 -- 172.21.15.133:0/619012141 shutdown_connections 2024-09-11T23:38:53.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.425+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/619012141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d8319630 0x55a9d831ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.425+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/619012141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830e890 0x55a9d830ecd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.425+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/619012141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830fe00 0x55a9d830e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.425+0000 7f9dd80bf740 1 -- 172.21.15.133:0/619012141 >> 172.21.15.133:0/619012141 conn(0x55a9d82d7310 msgr2=0x55a9d830af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:53.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.425+0000 7f9dd80bf740 1 -- 172.21.15.133:0/619012141 shutdown_connections 2024-09-11T23:38:53.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.425+0000 7f9dd80bf740 1 -- 172.21.15.133:0/619012141 wait complete. 2024-09-11T23:38:53.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.426+0000 7f9dd80bf740 1 -- start start 2024-09-11T23:38:53.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.426+0000 7f9dd80bf740 1 Processor -- start 2024-09-11T23:38:53.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.428+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830e890 0x55a9d8318d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.428+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830fe00 0x55a9d8316000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.428+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d8319630 0x55a9d8316540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.428+0000 7f9dd80bf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a9d82702b0 con 0x55a9d830e890 2024-09-11T23:38:53.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.428+0000 7f9dd80bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a9d822f060 con 0x55a9d8319630 2024-09-11T23:38:53.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.428+0000 7f9dd80bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a9d831c0f0 con 0x55a9d830fe00 2024-09-11T23:38:53.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.428+0000 7f9dd80bf740 10 client.0 _ll_register_callbacks cb 0x55a9d7f828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:53.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.431+0000 7f9dcffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d8319630 0x55a9d8316540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.431+0000 7f9dceffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830fe00 0x55a9d8316000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.431+0000 7f9dceffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830fe00 0x55a9d8316000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35298/0 (socket says 172.21.15.133:35298) 2024-09-11T23:38:53.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.431+0000 7f9dceffd640 1 -- 172.21.15.133:0/998398934 learned_addr learned my addr 172.21.15.133:0/998398934 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:53.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.431+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830e890 0x55a9d8318d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.432+0000 7f9dcffff640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9db80a2e40 0x7f9db80a5300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009ee0 tx=0x7f9dc80094d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:53.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.433+0000 7f9dcffff640 1 -- 172.21.15.133:0/998398934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830fe00 msgr2=0x55a9d8316000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:53.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.433+0000 7f9dcffff640 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830fe00 0x55a9d8316000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.433+0000 7f9dcffff640 1 -- 172.21.15.133:0/998398934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830e890 msgr2=0x55a9d8318d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:53.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.433+0000 7f9dcffff640 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830e890 0x55a9d8318d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.433+0000 7f9dcffff640 1 -- 172.21.15.133:0/998398934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d7fe53c0 con 0x55a9d8319630 2024-09-11T23:38:53.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.437+0000 7f9dcffff640 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d8319630 0x55a9d8316540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8013000 tx=0x7f9dc8004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91790d127eb09f9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:53.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.437+0000 7f9db37f6640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:38:53.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.437+0000 7f9db37f6640 1 -- 172.21.15.133:0/998398934 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8016a80 con 0x55a9d8319630 2024-09-11T23:38:53.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.438+0000 7f9db37f6640 1 -- 172.21.15.133:0/998398934 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f9dc8012040 con 0x55a9d8319630 2024-09-11T23:38:53.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.438+0000 7f9dd80bf740 1 -- 172.21.15.133:0/998398934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dc00cf680 con 0x55a9d8319630 2024-09-11T23:38:53.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.439+0000 7f9db37f6640 1 -- 172.21.15.133:0/998398934 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9dc8016c20 con 0x55a9d8319630 2024-09-11T23:38:53.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.440+0000 7f9dd80bf740 10 client.25487 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:53.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.440+0000 7f9dd80bf740 1 -- 172.21.15.133:0/998398934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a9d83387c0 con 0x55a9d8319630 2024-09-11T23:38:53.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.442+0000 7f9db37f6640 1 -- 172.21.15.133:0/998398934 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc00cf680 con 0x55a9d8319630 2024-09-11T23:38:53.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.442+0000 7f9db37f6640 1 -- 172.21.15.133:0/998398934 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9dc8020070 con 0x55a9d8319630 2024-09-11T23:38:53.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.442+0000 7f9dd80bf740 10 client.25487 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:53.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.442+0000 7f9dd80bf740 10 client.25487 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:38:53.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.442+0000 7f9dd80bf740 20 client.25487 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:53.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.442+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a9d833a120 0x55a9d833c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.442+0000 7f9dd80bf740 4 client.25487 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:53.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.442+0000 7f9dd80bf740 1 -- 172.21.15.133:0/998398934 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9d8281860 con 0x55a9d833a120 2024-09-11T23:38:53.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.444+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a9d833a120 0x55a9d833c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.461+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a9d833a120 0x55a9d833c540 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=76d5b7e236e1d11d server_cookie=3ffb073bf3449045 in_seq=0 out_seq=0 2024-09-11T23:38:53.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.465+0000 7f9db37f6640 10 client.25487 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:38:53.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.465+0000 7f9db37f6640 1 -- 172.21.15.133:0/998398934 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9d8281860 con 0x55a9d833a120 2024-09-11T23:38:53.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.465+0000 7f9db37f6640 10 client.25487 handle_command_reply: tid=0 2024-09-11T23:38:53.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.465+0000 7f9dd80bf740 1 client.25487 shutdown 2024-09-11T23:38:53.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.466+0000 7f9dd80bf740 1 -- 172.21.15.133:0/998398934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d8319630 msgr2=0x55a9d8316540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.466+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d8319630 0x55a9d8316540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8013000 tx=0x7f9dc8004a60 comp rx=0 tx=0).stop 2024-09-11T23:38:53.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.467+0000 7f9dd80bf740 1 -- 172.21.15.133:0/998398934 shutdown_connections 2024-09-11T23:38:53.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.467+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a9d833a120 0x55a9d833c540 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.467+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d8319630 0x55a9d8316540 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.467+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d830fe00 0x55a9d8316000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.467+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/998398934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830e890 0x55a9d8318d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.467+0000 7f9dd80bf740 1 -- 172.21.15.133:0/998398934 >> 172.21.15.133:0/998398934 conn(0x55a9d82d7310 msgr2=0x55a9d830a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.471+0000 7f9db37f6640 0 client.25487 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:38:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.471+0000 7f9dd80bf740 1 -- 172.21.15.133:0/998398934 shutdown_connections 2024-09-11T23:38:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.471+0000 7f9dd80bf740 1 -- 172.21.15.133:0/998398934 wait complete. 2024-09-11T23:38:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.471+0000 7f9dd80bf740 20 client.25487 trim_cache size 0 max 16384 2024-09-11T23:38:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.517+0000 7f9dd80bf740 1 Processor -- start 2024-09-11T23:38:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.518+0000 7f9dd80bf740 1 -- start start 2024-09-11T23:38:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.519+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 0x55a9d8316530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.519+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830fe00 0x55a9d8341e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.519+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d8319630 0x55a9d8342390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.520+0000 7f9dceffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830fe00 0x55a9d8341e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.520+0000 7f9dceffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830fe00 0x55a9d8341e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40246/0 (socket says 172.21.15.133:40246) 2024-09-11T23:38:53.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.520+0000 7f9dceffd640 1 -- 172.21.15.133:0/1343837724 learned_addr learned my addr 172.21.15.133:0/1343837724 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:53.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.520+0000 7f9dd80bf740 1 -- 172.21.15.133:0/1343837724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc00e740 con 0x55a9d830fe00 2024-09-11T23:38:53.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.520+0000 7f9dd80bf740 1 -- 172.21.15.133:0/1343837724 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc010280 con 0x55a9d830e890 2024-09-11T23:38:53.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.521+0000 7f9dd80bf740 1 -- 172.21.15.133:0/1343837724 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc00ee10 con 0x55a9d8319630 2024-09-11T23:38:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.522+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/1343837724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 0x55a9d8316530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.522+0000 7f9dceffd640 1 -- 172.21.15.133:0/1343837724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d8319630 msgr2=0x55a9d8342390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:38:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.522+0000 7f9dcffff640 1 --2- 172.21.15.133:0/1343837724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d8319630 0x55a9d8342390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.522+0000 7f9dceffd640 1 --2- 172.21.15.133:0/1343837724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d8319630 0x55a9d8342390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.522+0000 7f9dceffd640 1 -- 172.21.15.133:0/1343837724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 msgr2=0x55a9d8316530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.522+0000 7f9dceffd640 1 --2- 172.21.15.133:0/1343837724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 0x55a9d8316530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.522+0000 7f9dceffd640 1 -- 172.21.15.133:0/1343837724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d82e4800 con 0x55a9d830fe00 2024-09-11T23:38:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.522+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/1343837724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 0x55a9d8316530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:38:53.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.532+0000 7f9dceffd640 1 --2- 172.21.15.133:0/1343837724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830fe00 0x55a9d8341e50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40099e0 tx=0x7f9dc40090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b05d25b8a7d689 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:53.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.533+0000 7f9db4ff9640 1 -- 172.21.15.133:0/1343837724 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc401b070 con 0x55a9d830fe00 2024-09-11T23:38:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.533+0000 7f9db4ff9640 1 -- 172.21.15.133:0/1343837724 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc4016460 con 0x55a9d830fe00 2024-09-11T23:38:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.535+0000 7f9dd80bf740 1 -- 172.21.15.133:0/1343837724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830fe00 msgr2=0x55a9d8341e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.535+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/1343837724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830fe00 0x55a9d8341e50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40099e0 tx=0x7f9dc40090e0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.535+0000 7f9dd80bf740 1 -- 172.21.15.133:0/1343837724 shutdown_connections 2024-09-11T23:38:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.535+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/1343837724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d8319630 0x55a9d8342390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.535+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/1343837724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d830fe00 0x55a9d8341e50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.535+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/1343837724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 0x55a9d8316530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.535+0000 7f9dd80bf740 1 -- 172.21.15.133:0/1343837724 >> 172.21.15.133:0/1343837724 conn(0x55a9d82d7310 msgr2=0x55a9d8315e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.536+0000 7f9dd80bf740 1 -- 172.21.15.133:0/1343837724 shutdown_connections 2024-09-11T23:38:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.536+0000 7f9dd80bf740 1 -- 172.21.15.133:0/1343837724 wait complete. 2024-09-11T23:38:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.536+0000 7f9dd80bf740 1 -- start start 2024-09-11T23:38:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.537+0000 7f9dd80bf740 1 Processor -- start 2024-09-11T23:38:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.538+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 0x55a9d83213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.540+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d8319630 0x55a9d8321920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.540+0000 7f9dd80bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d8321e60 0x55a9d8312270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.540+0000 7f9dd80bf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc00fbb0 con 0x55a9d8319630 2024-09-11T23:38:53.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.540+0000 7f9dd80bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc00f4e0 con 0x55a9d830e890 2024-09-11T23:38:53.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.540+0000 7f9dd80bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc00c720 con 0x55a9d8321e60 2024-09-11T23:38:53.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.541+0000 7f9dd80bf740 10 client.0 _ll_register_callbacks cb 0x7f9dc80096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:53.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.541+0000 7f9dcf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 0x55a9d83213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.543+0000 7f9dcf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 0x55a9d83213e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:49646/0 (socket says 172.21.15.133:49646) 2024-09-11T23:38:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.543+0000 7f9dcf7fe640 1 -- 172.21.15.133:0/2478290993 learned_addr learned my addr 172.21.15.133:0/2478290993 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.543+0000 7f9dcffff640 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d8321e60 0x55a9d8312270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.543+0000 7f9dceffd640 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d8319630 0x55a9d8321920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.544+0000 7f9dceffd640 1 -- 172.21.15.133:0/2478290993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d8321e60 msgr2=0x55a9d8312270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.544+0000 7f9dceffd640 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d8321e60 0x55a9d8312270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.544+0000 7f9dceffd640 1 -- 172.21.15.133:0/2478290993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 msgr2=0x55a9d83213e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.544+0000 7f9dceffd640 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 0x55a9d83213e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.544+0000 7f9dceffd640 1 -- 172.21.15.133:0/2478290993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d83128e0 con 0x55a9d8319630 2024-09-11T23:38:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.546+0000 7f9dceffd640 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d8319630 0x55a9d8321920 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55a9d8339ca0 tx=0x7f9dc4013c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceceaea98b229e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.546+0000 7f9db2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.547+0000 7f9db2ff5640 1 -- 172.21.15.133:0/2478290993 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc401b040 con 0x55a9d8319630 2024-09-11T23:38:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.548+0000 7f9dd80bf740 1 -- 172.21.15.133:0/2478290993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9d827bd70 con 0x55a9d8319630 2024-09-11T23:38:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.549+0000 7f9db2ff5640 1 -- 172.21.15.133:0/2478290993 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f9dc4015ab0 con 0x55a9d8319630 2024-09-11T23:38:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.550+0000 7f9db2ff5640 1 -- 172.21.15.133:0/2478290993 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9dc4015d10 con 0x55a9d8319630 2024-09-11T23:38:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.550+0000 7f9dd80bf740 10 client.16062 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.550+0000 7f9dd80bf740 1 -- 172.21.15.133:0/2478290993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a9d83a4780 con 0x55a9d8319630 2024-09-11T23:38:53.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.551+0000 7f9db2ff5640 1 -- 172.21.15.133:0/2478290993 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc4015eb0 con 0x55a9d8319630 2024-09-11T23:38:53.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.554+0000 7f9db2ff5640 1 -- 172.21.15.133:0/2478290993 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9dc4016460 con 0x55a9d8319630 2024-09-11T23:38:53.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.554+0000 7f9dd80bf740 10 client.16062 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:53.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.554+0000 7f9dd80bf740 10 client.16062 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:38:53.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.554+0000 7f9dd80bf740 20 client.16062 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:53.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.554+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a9d83a4e50 0x55a9d83a7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:53.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.554+0000 7f9dd80bf740 4 client.16062 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:38:53.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.554+0000 7f9dd80bf740 1 -- 172.21.15.133:0/2478290993 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9d827bd70 con 0x55a9d83a4e50 2024-09-11T23:38:53.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.557+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a9d83a4e50 0x55a9d83a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:53.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.571+0000 7f9dcf7fe640 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a9d83a4e50 0x55a9d83a7240 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=81348d0abd5f749c server_cookie=2199336c1b99c6fb in_seq=0 out_seq=0 2024-09-11T23:38:53.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.571+0000 7f9db2ff5640 10 client.16062 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:38:53.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.571+0000 7f9db2ff5640 1 -- 172.21.15.133:0/2478290993 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9d827bd70 con 0x55a9d83a4e50 2024-09-11T23:38:53.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.571+0000 7f9db2ff5640 10 client.16062 handle_command_reply: tid=0 2024-09-11T23:38:53.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.571+0000 7f9dd80bf740 1 client.16062 shutdown 2024-09-11T23:38:53.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.571+0000 7f9dd80bf740 1 -- 172.21.15.133:0/2478290993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d8319630 msgr2=0x55a9d8321920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.571+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d8319630 0x55a9d8321920 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55a9d8339ca0 tx=0x7f9dc4013c70 comp rx=0 tx=0).stop 2024-09-11T23:38:53.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.572+0000 7f9dd80bf740 1 -- 172.21.15.133:0/2478290993 shutdown_connections 2024-09-11T23:38:53.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.572+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a9d83a4e50 0x55a9d83a7240 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.572+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9d8321e60 0x55a9d8312270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.572+0000 7f9db2ff5640 0 client.16062 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:38:53.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.572+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9d8319630 0x55a9d8321920 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.572+0000 7f9dd80bf740 1 --2- 172.21.15.133:0/2478290993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9d830e890 0x55a9d83213e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.572+0000 7f9dd80bf740 1 -- 172.21.15.133:0/2478290993 >> 172.21.15.133:0/2478290993 conn(0x55a9d82d7310 msgr2=0x55a9d8314d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:53.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.572+0000 7f9dd80bf740 1 -- 172.21.15.133:0/2478290993 shutdown_connections 2024-09-11T23:38:53.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.572+0000 7f9dd80bf740 1 -- 172.21.15.133:0/2478290993 wait complete. 2024-09-11T23:38:53.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.573+0000 7f9dd80bf740 20 client.16062 trim_cache size 0 max 16384 2024-09-11T23:38:53.573 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:38:53.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.575+0000 7f9db67fc640 1 -- 172.21.15.133:0/2861015943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9db80a2e40 msgr2=0x7f9db80a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.575+0000 7f9db67fc640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9db80a2e40 0x7f9db80a5300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009ee0 tx=0x7f9dc80094d0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.575+0000 7f9db67fc640 1 -- 172.21.15.133:0/2861015943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dd01008c0 msgr2=0x7f9dd0114d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:53.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.575+0000 7f9db67fc640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dd01008c0 0x7f9dd0114d80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0004d80 tx=0x7f9dc0004db0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.575+0000 7f9db67fc640 1 -- 172.21.15.133:0/2861015943 shutdown_connections 2024-09-11T23:38:53.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.575+0000 7f9db67fc640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9db80a2e40 0x7f9db80a5300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.575+0000 7f9db67fc640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd0105ec0 0x7f9dd0111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.575+0000 7f9db67fc640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dd01011e0 0x7f9dd0110e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.575+0000 7f9db67fc640 1 --2- 172.21.15.133:0/2861015943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dd01008c0 0x7f9dd0114d80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:53.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.575+0000 7f9db67fc640 1 -- 172.21.15.133:0/2861015943 >> 172.21.15.133:0/2861015943 conn(0x7f9dd00fc7d0 msgr2=0x7f9dd0103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:53.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.576+0000 7f9db67fc640 1 -- 172.21.15.133:0/2861015943 shutdown_connections 2024-09-11T23:38:53.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:53.576+0000 7f9db67fc640 1 -- 172.21.15.133:0/2861015943 wait complete. 2024-09-11T23:38:53.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:53 smithi133 ceph-mon[31430]: pgmap v66: 97 pgs: 97 active+clean; 10 GiB data, 61 GiB used, 1011 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 807 MiB/s wr, 1.76k op/s 2024-09-11T23:38:54.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:53 smithi138 ceph-mon[35142]: pgmap v66: 97 pgs: 97 active+clean; 10 GiB data, 61 GiB used, 1011 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 807 MiB/s wr, 1.76k op/s 2024-09-11T23:38:54.281 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-11T23:38:54.282 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-11T23:38:54.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:53 smithi178 ceph-mon[35882]: pgmap v66: 97 pgs: 97 active+clean; 10 GiB data, 61 GiB used, 1011 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 807 MiB/s wr, 1.76k op/s 2024-09-11T23:38:55.090 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:38:55.395 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:55.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:55 smithi133 ceph-mon[31430]: pgmap v67: 97 pgs: 97 active+clean; 10 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 733 MiB/s wr, 1.54k op/s 2024-09-11T23:38:55.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:55 smithi178 ceph-mon[35882]: pgmap v67: 97 pgs: 97 active+clean; 10 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 733 MiB/s wr, 1.54k op/s 2024-09-11T23:38:55.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:55 smithi138 ceph-mon[35142]: pgmap v67: 97 pgs: 97 active+clean; 10 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 733 MiB/s wr, 1.54k op/s 2024-09-11T23:38:56.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.189+0000 7efc11812640 1 -- 172.21.15.133:0/1198263082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc0c11b140 msgr2=0x7efc0c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.189+0000 7efc11812640 1 --2- 172.21.15.133:0/1198263082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc0c11b140 0x7efc0c11d530 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009860 tx=0x7efbf802a360 comp rx=0 tx=0).stop 2024-09-11T23:38:56.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.190+0000 7efc11812640 1 -- 172.21.15.133:0/1198263082 shutdown_connections 2024-09-11T23:38:56.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.190+0000 7efc11812640 1 --2- 172.21.15.133:0/1198263082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc0c11b140 0x7efc0c11d530 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.190+0000 7efc11812640 1 --2- 172.21.15.133:0/1198263082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c06ea30 0x7efc0c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.190+0000 7efc11812640 1 --2- 172.21.15.133:0/1198263082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc0c110190 0x7efc0c06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.190+0000 7efc11812640 1 -- 172.21.15.133:0/1198263082 >> 172.21.15.133:0/1198263082 conn(0x7efc0c073180 msgr2=0x7efc0c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.191+0000 7efc11812640 1 -- 172.21.15.133:0/1198263082 shutdown_connections 2024-09-11T23:38:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.192+0000 7efc11812640 1 -- 172.21.15.133:0/1198263082 wait complete. 2024-09-11T23:38:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.193+0000 7efc11812640 1 Processor -- start 2024-09-11T23:38:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.194+0000 7efc11812640 1 -- start start 2024-09-11T23:38:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.194+0000 7efc11812640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc0c06ea30 0x7efc0c1a7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.195+0000 7efc11812640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c110190 0x7efc0c1a7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.195+0000 7efc11812640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc0c11b140 0x7efc0c1ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.195+0000 7efc11812640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c120be0 con 0x7efc0c11b140 2024-09-11T23:38:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.195+0000 7efc11812640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c120a60 con 0x7efc0c110190 2024-09-11T23:38:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.195+0000 7efc11812640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c120d60 con 0x7efc0c06ea30 2024-09-11T23:38:56.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.196+0000 7efc11011640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc0c11b140 0x7efc0c1ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.196+0000 7efc11011640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc0c11b140 0x7efc0c1ab910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40282/0 (socket says 172.21.15.133:40282) 2024-09-11T23:38:56.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.196+0000 7efc11011640 1 -- 172.21.15.133:0/1273394020 learned_addr learned my addr 172.21.15.133:0/1273394020 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:56.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.197+0000 7efc11011640 1 -- 172.21.15.133:0/1273394020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc0c06ea30 msgr2=0x7efc0c1a7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.197+0000 7efc11011640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc0c06ea30 0x7efc0c1a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.197+0000 7efc11011640 1 -- 172.21.15.133:0/1273394020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c110190 msgr2=0x7efc0c1a7580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:38:56.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.197+0000 7efc11011640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c110190 0x7efc0c1a7580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.197+0000 7efc11011640 1 -- 172.21.15.133:0/1273394020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c1ac090 con 0x7efc0c11b140 2024-09-11T23:38:56.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.198+0000 7efc11011640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc0c11b140 0x7efc0c1ab910 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009780 tx=0x7efbf8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.198+0000 7efc09ffb640 1 -- 172.21.15.133:0/1273394020 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8037040 con 0x7efc0c11b140 2024-09-11T23:38:56.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.198+0000 7efc11812640 1 -- 172.21.15.133:0/1273394020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0c1ac320 con 0x7efc0c11b140 2024-09-11T23:38:56.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.199+0000 7efc11812640 1 -- 172.21.15.133:0/1273394020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0c110920 con 0x7efc0c11b140 2024-09-11T23:38:56.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.214+0000 7efc09ffb640 1 -- 172.21.15.133:0/1273394020 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf8040070 con 0x7efc0c11b140 2024-09-11T23:38:56.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.230+0000 7efc14e4d740 1 Processor -- start 2024-09-11T23:38:56.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.231+0000 7efc09ffb640 1 -- 172.21.15.133:0/1273394020 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8032c30 con 0x7efc0c11b140 2024-09-11T23:38:56.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.231+0000 7efc09ffb640 1 -- 172.21.15.133:0/1273394020 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7efbf8045460 con 0x7efc0c11b140 2024-09-11T23:38:56.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.231+0000 7efc14e4d740 1 -- start start 2024-09-11T23:38:56.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.233+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2babce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.233+0000 7efc09ffb640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efbf00a2f30 0x7efbf00a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.233+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bb65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.233+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bb8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.233+0000 7efc14e4d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2b112a0 con 0x55a1b2bab900 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.233+0000 7efc14e4d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2ab8b10 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.233+0000 7efc14e4d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2b0d260 con 0x55a1b2bac220 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.234+0000 7efc09ffb640 1 -- 172.21.15.133:0/1273394020 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7efbf80f0290 con 0x7efc0c11b140 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.234+0000 7efc11011640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bb8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.235+0000 7efc10810640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efbf00a2f30 0x7efbf00a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.235+0000 7efc11011640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bb8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:49658/0 (socket says 172.21.15.133:49658) 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.235+0000 7efc11011640 1 -- 172.21.15.133:0/1905875914 learned_addr learned my addr 172.21.15.133:0/1905875914 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.236+0000 7efc11011640 1 -- 172.21.15.133:0/1905875914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 msgr2=0x55a1b2bb65a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.237+0000 7efc11011640 1 --2- 172.21.15.133:0/1905875914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bb65a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.237+0000 7efc11011640 1 -- 172.21.15.133:0/1905875914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 msgr2=0x55a1b2babce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.237+0000 7efc11011640 1 --2- 172.21.15.133:0/1905875914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2babce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.237+0000 7efc11011640 1 -- 172.21.15.133:0/1905875914 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b2882b90 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.239+0000 7efc11011640 1 --2- 172.21.15.133:0/1905875914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bb8ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efbf80c5090 tx=0x7efbf80bc720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a73ab361623f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.245+0000 7efc10810640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efbf00a2f30 0x7efbf00a53f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efbfc005fd0 tx=0x7efbfc009520 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.246+0000 7efc02ffd640 1 -- 172.21.15.133:0/1905875914 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf80d5070 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.246+0000 7efc02ffd640 1 -- 172.21.15.133:0/1905875914 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf80bd050 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.246+0000 7efc02ffd640 1 -- 172.21.15.133:0/1905875914 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8009430 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.248+0000 7efc14e4d740 1 -- 172.21.15.133:0/1905875914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 msgr2=0x55a1b2bb8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.248+0000 7efc14e4d740 1 --2- 172.21.15.133:0/1905875914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bb8ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efbf80c5090 tx=0x7efbf80bc720 comp rx=0 tx=0).stop 2024-09-11T23:38:56.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.249+0000 7efc14e4d740 1 -- 172.21.15.133:0/1905875914 shutdown_connections 2024-09-11T23:38:56.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.249+0000 7efc14e4d740 1 --2- 172.21.15.133:0/1905875914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bb8ed0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.249+0000 7efc14e4d740 1 --2- 172.21.15.133:0/1905875914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bb65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.249+0000 7efc14e4d740 1 --2- 172.21.15.133:0/1905875914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2babce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.249+0000 7efc14e4d740 1 -- 172.21.15.133:0/1905875914 >> 172.21.15.133:0/1905875914 conn(0x55a1b2ba7e90 msgr2=0x55a1b2ba8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.255+0000 7efc14e4d740 1 -- 172.21.15.133:0/1905875914 shutdown_connections 2024-09-11T23:38:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.256+0000 7efc14e4d740 1 -- 172.21.15.133:0/1905875914 wait complete. 2024-09-11T23:38:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.257+0000 7efc14e4d740 1 -- start start 2024-09-11T23:38:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.258+0000 7efc14e4d740 1 Processor -- start 2024-09-11T23:38:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.261+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bab900 0x55a1b2bb3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.261+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bb4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.261+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bb45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.261+0000 7efc14e4d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2bb9470 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.261+0000 7efc14e4d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2a996e0 con 0x55a1b2bab900 2024-09-11T23:38:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.261+0000 7efc14e4d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2bb95f0 con 0x55a1b2bac220 2024-09-11T23:38:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.261+0000 7efc14e4d740 10 client.0 _ll_register_callbacks cb 0x55a1b2abda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.263+0000 7efc0bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bb4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.265+0000 7efc10810640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bab900 0x55a1b2bb3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.265+0000 7efc0bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bb4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35334/0 (socket says 172.21.15.133:35334) 2024-09-11T23:38:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.265+0000 7efc0bfff640 1 -- 172.21.15.133:0/3233402174 learned_addr learned my addr 172.21.15.133:0/3233402174 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.267+0000 7efc11011640 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bb45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.269+0000 7efc11011640 1 -- 172.21.15.133:0/3233402174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 msgr2=0x55a1b2bb4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.269+0000 7efc11011640 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bb4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.269+0000 7efc11011640 1 -- 172.21.15.133:0/3233402174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bab900 msgr2=0x55a1b2bb3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.269+0000 7efc11011640 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bab900 0x55a1b2bb3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.269+0000 7efc11011640 1 -- 172.21.15.133:0/3233402174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b2bc2aa0 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.279+0000 7efc11011640 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bb45c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009830 tx=0x7efbf802a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b757cc3aee620f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.283+0000 7efbeffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.283+0000 7efbeffff640 1 -- 172.21.15.133:0/3233402174 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf80d9020 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.283+0000 7efc14e4d740 1 -- 172.21.15.133:0/3233402174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf80bd050 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.283+0000 7efbeffff640 1 -- 172.21.15.133:0/3233402174 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7efbf80b8f30 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.284+0000 7efbeffff640 1 -- 172.21.15.133:0/3233402174 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efbf80ce050 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.284+0000 7efc14e4d740 10 client.16071 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.284+0000 7efc14e4d740 1 -- 172.21.15.133:0/3233402174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1b2bb9470 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.285+0000 7efbeffff640 1 -- 172.21.15.133:0/3233402174 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf80bd050 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.285+0000 7efbeffff640 1 -- 172.21.15.133:0/3233402174 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efbf803a070 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.285+0000 7efc14e4d740 10 client.16071 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:56.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.285+0000 7efc14e4d740 10 client.16071 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:38:56.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.285+0000 7efc14e4d740 20 client.16071 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:56.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.285+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a1b2bc65e0 0x55a1b2bc89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.285+0000 7efc14e4d740 4 client.16071 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:56.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.285+0000 7efc14e4d740 1 -- 172.21.15.133:0/3233402174 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1b2b1e9d0 con 0x55a1b2bc65e0 2024-09-11T23:38:56.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.287+0000 7efc10810640 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a1b2bc65e0 0x55a1b2bc89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.305+0000 7efbeffff640 10 client.16071 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:56.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.305+0000 7efc10810640 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a1b2bc65e0 0x55a1b2bc89d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2e1e5ae8dd818be server_cookie=b6b3aeacc255cd9b in_seq=0 out_seq=0 2024-09-11T23:38:56.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.311+0000 7efbeffff640 1 -- 172.21.15.133:0/3233402174 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1b2b1e9d0 con 0x55a1b2bc65e0 2024-09-11T23:38:56.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.311+0000 7efbeffff640 10 client.16071 handle_command_reply: tid=0 2024-09-11T23:38:56.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.311+0000 7efc14e4d740 1 client.16071 shutdown 2024-09-11T23:38:56.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.313+0000 7efc14e4d740 1 -- 172.21.15.133:0/3233402174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bb6ae0 msgr2=0x55a1b2bb45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.313+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bb45c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009830 tx=0x7efbf802a360 comp rx=0 tx=0).stop 2024-09-11T23:38:56.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.313+0000 7efc14e4d740 1 -- 172.21.15.133:0/3233402174 shutdown_connections 2024-09-11T23:38:56.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.313+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a1b2bc65e0 0x55a1b2bc89d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.313+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bb45c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.313+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bb4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.313+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3233402174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bab900 0x55a1b2bb3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.313+0000 7efc14e4d740 1 -- 172.21.15.133:0/3233402174 >> 172.21.15.133:0/3233402174 conn(0x55a1b2ba7e90 msgr2=0x55a1b2badbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.313+0000 7efbeffff640 0 client.16071 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:56.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.314+0000 7efc14e4d740 1 -- 172.21.15.133:0/3233402174 shutdown_connections 2024-09-11T23:38:56.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.314+0000 7efc14e4d740 1 -- 172.21.15.133:0/3233402174 wait complete. 2024-09-11T23:38:56.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.315+0000 7efc14e4d740 20 client.16071 trim_cache size 0 max 16384 2024-09-11T23:38:56.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.350+0000 7efc14e4d740 1 Processor -- start 2024-09-11T23:38:56.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.350+0000 7efc14e4d740 1 -- start start 2024-09-11T23:38:56.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.351+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2bb4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.351+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2badab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.351+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.351+0000 7efc14e4d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efbf400e640 con 0x55a1b2bab900 2024-09-11T23:38:56.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.351+0000 7efc14e4d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efbf4010180 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.351+0000 7efc14e4d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efbf400ed10 con 0x55a1b2bac220 2024-09-11T23:38:56.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.352+0000 7efc0bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2badab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.352+0000 7efc0bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2badab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35346/0 (socket says 172.21.15.133:35346) 2024-09-11T23:38:56.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.352+0000 7efc0bfff640 1 -- 172.21.15.133:0/391171955 learned_addr learned my addr 172.21.15.133:0/391171955 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:56.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.352+0000 7efc10810640 1 --2- 172.21.15.133:0/391171955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2bb4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.355+0000 7efc11011640 1 --2- 172.21.15.133:0/391171955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.356+0000 7efc10810640 1 -- 172.21.15.133:0/391171955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 msgr2=0x55a1b2badab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.356+0000 7efc10810640 1 --2- 172.21.15.133:0/391171955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2badab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.356+0000 7efc10810640 1 -- 172.21.15.133:0/391171955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 msgr2=0x55a1b2bce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.356+0000 7efc10810640 1 --2- 172.21.15.133:0/391171955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.356+0000 7efc10810640 1 -- 172.21.15.133:0/391171955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b2bc26d0 con 0x55a1b2bab900 2024-09-11T23:38:56.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.357+0000 7efc10810640 1 --2- 172.21.15.133:0/391171955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2bb4410 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55a1b2bc91f0 tx=0x7efbfc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d078047e2d012cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.358+0000 7efc027fc640 1 -- 172.21.15.133:0/391171955 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc005220 con 0x55a1b2bab900 2024-09-11T23:38:56.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.358+0000 7efc027fc640 1 -- 172.21.15.133:0/391171955 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbfc004c80 con 0x55a1b2bab900 2024-09-11T23:38:56.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.358+0000 7efc027fc640 1 -- 172.21.15.133:0/391171955 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc003b00 con 0x55a1b2bab900 2024-09-11T23:38:56.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.360+0000 7efc14e4d740 1 -- 172.21.15.133:0/391171955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 msgr2=0x55a1b2bb4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.360+0000 7efc14e4d740 1 --2- 172.21.15.133:0/391171955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2bb4410 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55a1b2bc91f0 tx=0x7efbfc015000 comp rx=0 tx=0).stop 2024-09-11T23:38:56.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.363+0000 7efc14e4d740 1 -- 172.21.15.133:0/391171955 shutdown_connections 2024-09-11T23:38:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.363+0000 7efc14e4d740 1 --2- 172.21.15.133:0/391171955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.363+0000 7efc14e4d740 1 --2- 172.21.15.133:0/391171955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2badab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.363+0000 7efc14e4d740 1 --2- 172.21.15.133:0/391171955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2bb4410 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.363+0000 7efc14e4d740 1 -- 172.21.15.133:0/391171955 >> 172.21.15.133:0/391171955 conn(0x55a1b2ba7e90 msgr2=0x55a1b2bad3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.363+0000 7efc14e4d740 1 -- 172.21.15.133:0/391171955 shutdown_connections 2024-09-11T23:38:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.363+0000 7efc14e4d740 1 -- 172.21.15.133:0/391171955 wait complete. 2024-09-11T23:38:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.364+0000 7efc14e4d740 1 -- start start 2024-09-11T23:38:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.364+0000 7efc14e4d740 1 Processor -- start 2024-09-11T23:38:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.365+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2bba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.366+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bbae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.366+0000 7efc14e4d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bbb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.366+0000 7efc10810640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2bba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.366+0000 7efc10810640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2bba8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40296/0 (socket says 172.21.15.133:40296) 2024-09-11T23:38:56.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.366+0000 7efc10810640 1 -- 172.21.15.133:0/3345358857 learned_addr learned my addr 172.21.15.133:0/3345358857 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:56.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.366+0000 7efc0bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bbae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.367+0000 7efc14e4d740 1 -- 172.21.15.133:0/3345358857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efbf400fab0 con 0x55a1b2bab900 2024-09-11T23:38:56.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.367+0000 7efc11011640 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bbb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.367+0000 7efc14e4d740 1 -- 172.21.15.133:0/3345358857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efbf400f3e0 con 0x55a1b2bb6ae0 2024-09-11T23:38:56.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.367+0000 7efc14e4d740 1 -- 172.21.15.133:0/3345358857 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efbf400c620 con 0x55a1b2bac220 2024-09-11T23:38:56.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.367+0000 7efc14e4d740 10 client.0 _ll_register_callbacks cb 0x7efbf80edb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:56.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.367+0000 7efc10810640 1 -- 172.21.15.133:0/3345358857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 msgr2=0x55a1b2bbae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.367+0000 7efc10810640 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bbae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.367+0000 7efc10810640 1 -- 172.21.15.133:0/3345358857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 msgr2=0x55a1b2bbb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.368+0000 7efc10810640 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bbb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.368+0000 7efc10810640 1 -- 172.21.15.133:0/3345358857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b2c30520 con 0x55a1b2bab900 2024-09-11T23:38:56.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.369+0000 7efc14e4d740 1 -- 172.21.15.133:0/3345358857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbfc004c80 con 0x55a1b2bab900 2024-09-11T23:38:56.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.369+0000 7efbeffff640 10 client.16077 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:56.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.369+0000 7efc10810640 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2bba8c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00e720 tx=0x7efbfc0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e527f461afa4f0c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.370+0000 7efbeffff640 1 -- 172.21.15.133:0/3345358857 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc038060 con 0x55a1b2bab900 2024-09-11T23:38:56.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.370+0000 7efbeffff640 1 -- 172.21.15.133:0/3345358857 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7efbfc003700 con 0x55a1b2bab900 2024-09-11T23:38:56.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.370+0000 7efbeffff640 1 -- 172.21.15.133:0/3345358857 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efbfc009790 con 0x55a1b2bab900 2024-09-11T23:38:56.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.372+0000 7efc14e4d740 10 client.16077 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:56.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.372+0000 7efc14e4d740 1 -- 172.21.15.133:0/3345358857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55a1b2c316b0 con 0x55a1b2bab900 2024-09-11T23:38:56.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.380+0000 7efbeffff640 1 -- 172.21.15.133:0/3345358857 <== mon.0 v2:172.21.15.133:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efbfc03b050 con 0x55a1b2bab900 2024-09-11T23:38:56.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.380+0000 7efc14e4d740 10 client.16077 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:56.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.380+0000 7efc14e4d740 10 client.16077 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:38:56.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.380+0000 7efc14e4d740 20 client.16077 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:56.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.380+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a1b2c31c60 0x55a1b2c34080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.380+0000 7efc14e4d740 4 client.16077 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-11T23:38:56.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.380+0000 7efc14e4d740 1 -- 172.21.15.133:0/3345358857 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x55a1b2b18d20 con 0x55a1b2c31c60 2024-09-11T23:38:56.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.383+0000 7efc0bfff640 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a1b2c31c60 0x55a1b2c34080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.389+0000 7efbeffff640 10 client.16077 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:56.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.389+0000 7efc0bfff640 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a1b2c31c60 0x55a1b2c34080 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd23f54332e99147 server_cookie=a241eea4a42fdcad in_seq=0 out_seq=0 2024-09-11T23:38:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efbeffff640 1 -- 172.21.15.133:0/3345358857 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a1b2b18d20 con 0x55a1b2c31c60 2024-09-11T23:38:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efbeffff640 10 client.16077 handle_command_reply: tid=0 2024-09-11T23:38:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 1 client.16077 shutdown 2024-09-11T23:38:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 1 -- 172.21.15.133:0/3345358857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 msgr2=0x55a1b2bba8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2bba8c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00e720 tx=0x7efbfc0049d0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 1 -- 172.21.15.133:0/3345358857 shutdown_connections 2024-09-11T23:38:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a1b2c31c60 0x55a1b2c34080 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b2bb6ae0 0x55a1b2bbb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1b2bac220 0x55a1b2bbae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efbeffff640 0 client.16077 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:38:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 1 --2- 172.21.15.133:0/3345358857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1b2bab900 0x55a1b2bba8c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 1 -- 172.21.15.133:0/3345358857 >> 172.21.15.133:0/3345358857 conn(0x55a1b2ba7e90 msgr2=0x55a1b2bc1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 1 -- 172.21.15.133:0/3345358857 shutdown_connections 2024-09-11T23:38:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 1 -- 172.21.15.133:0/3345358857 wait complete. 2024-09-11T23:38:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.899+0000 7efc14e4d740 20 client.16077 trim_cache size 0 max 16384 2024-09-11T23:38:56.900 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:38:56.900 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 28, 2024-09-11T23:38:56.900 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:38:56.900 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 6, 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "1fb2b5b2": { 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 6, 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 133.4, 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:38:56.901 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:38:56.902 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:38:56.902 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:56.902 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:56.902 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:56.902 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:56.902 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:38:56.902 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 -- 172.21.15.133:0/1273394020 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efbf00a2f30 msgr2=0x7efbf00a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efbf00a2f30 0x7efbf00a53f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efbfc005fd0 tx=0x7efbfc009520 comp rx=0 tx=0).stop 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 -- 172.21.15.133:0/1273394020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc0c11b140 msgr2=0x7efc0c1ab910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc0c11b140 0x7efc0c1ab910 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009780 tx=0x7efbf8004530 comp rx=0 tx=0).stop 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 -- 172.21.15.133:0/1273394020 shutdown_connections 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efbf00a2f30 0x7efbf00a53f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc0c11b140 0x7efc0c1ab910 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c110190 0x7efc0c1a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 --2- 172.21.15.133:0/1273394020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc0c06ea30 0x7efc0c1a7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 -- 172.21.15.133:0/1273394020 >> 172.21.15.133:0/1273394020 conn(0x7efc0c073180 msgr2=0x7efc0c119450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 -- 172.21.15.133:0/1273394020 shutdown_connections 2024-09-11T23:38:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.900+0000 7efc11812640 1 -- 172.21.15.133:0/1273394020 wait complete. 2024-09-11T23:38:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.950+0000 7fc3c95a1640 1 -- 172.21.15.133:0/558642539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3c4102ef0 msgr2=0x7fc3c410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.950+0000 7fc3c95a1640 1 --2- 172.21.15.133:0/558642539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3c4102ef0 0x7fc3c410da40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009ca0 tx=0x7fc3bc02a950 comp rx=0 tx=0).stop 2024-09-11T23:38:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.950+0000 7fc3c95a1640 1 -- 172.21.15.133:0/558642539 shutdown_connections 2024-09-11T23:38:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.950+0000 7fc3c95a1640 1 --2- 172.21.15.133:0/558642539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3c4102ef0 0x7fc3c410da40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.950+0000 7fc3c95a1640 1 --2- 172.21.15.133:0/558642539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4102570 0x7fc3c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.950+0000 7fc3c95a1640 1 --2- 172.21.15.133:0/558642539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3c4100e20 0x7fc3c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.950+0000 7fc3c95a1640 1 -- 172.21.15.133:0/558642539 >> 172.21.15.133:0/558642539 conn(0x7fc3c40fc7d0 msgr2=0x7fc3c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.950+0000 7fc3c95a1640 1 -- 172.21.15.133:0/558642539 shutdown_connections 2024-09-11T23:38:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.950+0000 7fc3c95a1640 1 -- 172.21.15.133:0/558642539 wait complete. 2024-09-11T23:38:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.951+0000 7fc3c95a1640 1 Processor -- start 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.951+0000 7fc3c95a1640 1 -- start start 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.951+0000 7fc3c95a1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4100e20 0x7fc3c41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.951+0000 7fc3c95a1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3c4102570 0x7fc3c4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.951+0000 7fc3c95a1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3c4102ef0 0x7fc3c419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.951+0000 7fc3c95a1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4110f80 con 0x7fc3c4102ef0 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.951+0000 7fc3c95a1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4110e00 con 0x7fc3c4100e20 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.951+0000 7fc3c95a1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4111100 con 0x7fc3c4102570 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.951+0000 7fc3c37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3c4102ef0 0x7fc3c419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.951+0000 7fc3c2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4100e20 0x7fc3c41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c27fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3c4102570 0x7fc3c4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4100e20 0x7fc3c41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:49712/0 (socket says 172.21.15.133:49712) 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c27fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3c4102570 0x7fc3c4198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35372/0 (socket says 172.21.15.133:35372) 2024-09-11T23:38:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/2350360527 learned_addr learned my addr 172.21.15.133:0/2350360527 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c37fe640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3c4102ef0 0x7fc3c419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40320/0 (socket says 172.21.15.133:40320) 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/2350360527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3c4102570 msgr2=0x7fc3c4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3c4102570 0x7fc3c4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/2350360527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3c4102ef0 msgr2=0x7fc3c419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3c4102ef0 0x7fc3c419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/2350360527 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c419d480 con 0x7fc3c4100e20 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c27fc640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3c4102570 0x7fc3c4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4100e20 0x7fc3c41984d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400d770 tx=0x7fc3b400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3abfff640 1 -- 172.21.15.133:0/2350360527 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4017070 con 0x7fc3c4100e20 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c95a1640 1 -- 172.21.15.133:0/2350360527 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c419d770 con 0x7fc3c4100e20 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3abfff640 1 -- 172.21.15.133:0/2350360527 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b4005500 con 0x7fc3c4100e20 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3abfff640 1 -- 172.21.15.133:0/2350360527 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4004e00 con 0x7fc3c4100e20 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.952+0000 7fc3c95a1640 1 -- 172.21.15.133:0/2350360527 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c41a5050 con 0x7fc3c4100e20 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.954+0000 7fc3abfff640 1 -- 172.21.15.133:0/2350360527 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc3b4004fa0 con 0x7fc3c4100e20 2024-09-11T23:38:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3cb97a740 1 Processor -- start 2024-09-11T23:38:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3cb97a740 1 -- start start 2024-09-11T23:38:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06bb40 0x55966c06a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c06aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c075370 0x55966c077760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55966bfd02f0 con 0x55966c075370 2024-09-11T23:38:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55966bf77cf0 con 0x55966c06bb40 2024-09-11T23:38:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55966bfcc2b0 con 0x55966c06a5d0 2024-09-11T23:38:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3c37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06bb40 0x55966c06a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3c37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06bb40 0x55966c06a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:49720/0 (socket says 172.21.15.133:49720) 2024-09-11T23:38:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3c37fe640 1 -- 172.21.15.133:0/1110440498 learned_addr learned my addr 172.21.15.133:0/1110440498 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3c27fc640 1 --2- 172.21.15.133:0/1110440498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c075370 0x55966c077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3abfff640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc3ac0a2cf0 0x7fc3ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.955+0000 7fc3abfff640 1 -- 172.21.15.133:0/2350360527 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc3b40cee10 con 0x7fc3c4100e20 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.956+0000 7fc3c27fc640 1 -- 172.21.15.133:0/1110440498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 msgr2=0x55966c06aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.956+0000 7fc3c27fc640 1 --2- 172.21.15.133:0/1110440498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c06aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.956+0000 7fc3c27fc640 1 -- 172.21.15.133:0/1110440498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06bb40 msgr2=0x55966c06a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.956+0000 7fc3c27fc640 1 --2- 172.21.15.133:0/1110440498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06bb40 0x55966c06a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.956+0000 7fc3c27fc640 1 -- 172.21.15.133:0/1110440498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55966bf5c3a0 con 0x55966c075370 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.956+0000 7fc3c27fc640 1 --2- 172.21.15.133:0/1110440498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c075370 0x55966c077760 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800bfb0 tx=0x7fc3b8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c72e6f3c567666e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.956+0000 7fc3c37fe640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc3ac0a2cf0 0x7fc3ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.956+0000 7fc3a97fa640 1 -- 172.21.15.133:0/1110440498 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8004220 con 0x55966c075370 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.956+0000 7fc3a97fa640 1 -- 172.21.15.133:0/1110440498 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b80043c0 con 0x55966c075370 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.956+0000 7fc3a97fa640 1 -- 172.21.15.133:0/1110440498 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8011ac0 con 0x55966c075370 2024-09-11T23:38:56.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1110440498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c075370 msgr2=0x55966c077760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1110440498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c075370 0x55966c077760 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800bfb0 tx=0x7fc3b8010860 comp rx=0 tx=0).stop 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3c37fe640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc3ac0a2cf0 0x7fc3ac0a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009ca0 tx=0x7fc3bc0095c0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1110440498 shutdown_connections 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1110440498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c075370 0x55966c077760 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1110440498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c06aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1110440498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06bb40 0x55966c06a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1110440498 >> 172.21.15.133:0/1110440498 conn(0x55966c033310 msgr2=0x55966c066c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1110440498 shutdown_connections 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1110440498 wait complete. 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3cb97a740 1 -- start start 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.957+0000 7fc3cb97a740 1 Processor -- start 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c074e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 0x55966c0720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 0x55966c072860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55966c078070 con 0x55966c06bb40 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55966bf8b060 con 0x55966c075370 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55966c0781f0 con 0x55966c06a5d0 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3cb97a740 10 client.0 _ll_register_callbacks cb 0x55966bcde8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 0x55966c0720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c074e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c074e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35392/0 (socket says 172.21.15.133:35392) 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 0x55966c0720d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40326/0 (socket says 172.21.15.133:40326) 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/3631885849 learned_addr learned my addr 172.21.15.133:0/3631885849 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c37fe640 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 0x55966c072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/3631885849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 msgr2=0x55966c072860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 0x55966c072860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/3631885849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 msgr2=0x55966c0720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 0x55966c0720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/3631885849 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55966bd413c0 con 0x55966c06a5d0 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c27fc640 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 0x55966c0720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c074e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40cc5d0 tx=0x7fc3b409b2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9878ff9b7dc8dd64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/3631885849 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4020050 con 0x55966c06a5d0 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3cb97a740 1 -- 172.21.15.133:0/3631885849 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3b80043c0 con 0x55966c06a5d0 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.958+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/3631885849 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc3b4097a60 con 0x55966c06a5d0 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/3631885849 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc3b409f6f0 con 0x55966c06a5d0 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3cb97a740 10 client.25531 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3cb97a740 1 -- 172.21.15.133:0/3631885849 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55966c083c40 con 0x55966c06a5d0 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/3631885849 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b80043c0 con 0x55966c06a5d0 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/3631885849 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc3b409a840 con 0x55966c06a5d0 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3cb97a740 10 client.25531 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3cb97a740 10 client.25531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:38:56.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3cb97a740 20 client.25531 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55966c085050 0x55966c087440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3cb97a740 4 client.25531 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3cb97a740 1 -- 172.21.15.133:0/3631885849 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55966bfdd860 con 0x55966c085050 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.959+0000 7fc3c27fc640 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55966c085050 0x55966c087440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3c27fc640 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55966c085050 0x55966c087440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21c35890b8cd86b server_cookie=697cc8145875c63a in_seq=0 out_seq=0 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3a6ff5640 10 client.25531 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/3631885849 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55966bfdd860 con 0x55966c085050 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3a6ff5640 10 client.25531 handle_command_reply: tid=0 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3cb97a740 1 client.25531 shutdown 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3cb97a740 1 -- 172.21.15.133:0/3631885849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 msgr2=0x55966c074e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c074e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40cc5d0 tx=0x7fc3b409b2e0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3cb97a740 1 -- 172.21.15.133:0/3631885849 shutdown_connections 2024-09-11T23:38:56.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55966c085050 0x55966c087440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3a6ff5640 0 client.25531 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:38:56.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 0x55966c072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 0x55966c0720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/3631885849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c074e20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.960+0000 7fc3cb97a740 1 -- 172.21.15.133:0/3631885849 >> 172.21.15.133:0/3631885849 conn(0x55966c033310 msgr2=0x55966c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.961+0000 7fc3cb97a740 1 -- 172.21.15.133:0/3631885849 shutdown_connections 2024-09-11T23:38:56.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.961+0000 7fc3cb97a740 1 -- 172.21.15.133:0/3631885849 wait complete. 2024-09-11T23:38:56.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.961+0000 7fc3cb97a740 20 client.25531 trim_cache size 0 max 16384 2024-09-11T23:38:56.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3cb97a740 1 Processor -- start 2024-09-11T23:38:56.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3cb97a740 1 -- start start 2024-09-11T23:38:56.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06a5d0 0x55966c066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06bb40 0x55966c071990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c075370 0x55966c08cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41af990 con 0x55966c075370 2024-09-11T23:38:56.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41b14d0 con 0x55966c06a5d0 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41b0060 con 0x55966c06bb40 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3c2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06a5d0 0x55966c066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3c37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c075370 0x55966c08cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3c2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06a5d0 0x55966c066e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:49732/0 (socket says 172.21.15.133:49732) 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/1809063491 learned_addr learned my addr 172.21.15.133:0/1809063491 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.974+0000 7fc3c27fc640 1 --2- 172.21.15.133:0/1809063491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06bb40 0x55966c071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.975+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/1809063491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06bb40 msgr2=0x55966c071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.975+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/1809063491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06bb40 0x55966c071990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.975+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/1809063491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c075370 msgr2=0x55966c08cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.975+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/1809063491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c075370 0x55966c08cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.975+0000 7fc3c2ffd640 1 -- 172.21.15.133:0/1809063491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55966c066510 con 0x55966c06a5d0 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.975+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/1809063491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06a5d0 0x55966c066e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc3b409b380 tx=0x7fc3b4099900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb51bc63d90b646c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.975+0000 7fc3a8ff9640 1 -- 172.21.15.133:0/1809063491 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4020070 con 0x55966c06a5d0 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.975+0000 7fc3a8ff9640 1 -- 172.21.15.133:0/1809063491 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b40a3fb0 con 0x55966c06a5d0 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.975+0000 7fc3a8ff9640 1 -- 172.21.15.133:0/1809063491 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b409a840 con 0x55966c06a5d0 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.976+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1809063491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06a5d0 msgr2=0x55966c066e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.976+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1809063491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06a5d0 0x55966c066e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc3b409b380 tx=0x7fc3b4099900 comp rx=0 tx=0).stop 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.976+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1809063491 shutdown_connections 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.976+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1809063491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c075370 0x55966c08cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.976+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1809063491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06bb40 0x55966c071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.976+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1809063491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c06a5d0 0x55966c066e10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.976+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1809063491 >> 172.21.15.133:0/1809063491 conn(0x55966c033310 msgr2=0x55966c066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.976+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1809063491 shutdown_connections 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.976+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1809063491 wait complete. 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.976+0000 7fc3cb97a740 1 -- start start 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.976+0000 7fc3cb97a740 1 Processor -- start 2024-09-11T23:38:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c07dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 0x55966c06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3cb97a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 0x55966c06f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41b0e00 con 0x55966c06bb40 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41b0730 con 0x55966c075370 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3cb97a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41ad970 con 0x55966c06a5d0 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3cb97a740 10 client.0 _ll_register_callbacks cb 0x7fc3b40056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 0x55966c06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 0x55966c06f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 0x55966c06f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:49748/0 (socket says 172.21.15.133:49748) 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c37fe640 1 -- 172.21.15.133:0/1453133580 learned_addr learned my addr 172.21.15.133:0/1453133580 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c07dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c37fe640 1 -- 172.21.15.133:0/1453133580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 msgr2=0x55966c07dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c37fe640 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c07dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c37fe640 1 -- 172.21.15.133:0/1453133580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 msgr2=0x55966c06ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c37fe640 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 0x55966c06ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c37fe640 1 -- 172.21.15.133:0/1453133580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55966c0eec40 con 0x55966c075370 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c27fc640 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 0x55966c06ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3c37fe640 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 0x55966c06f6c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc035000 tx=0x7fc3bc041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b973b415d09c1bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.977+0000 7fc3a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/1453133580 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc034040 con 0x55966c075370 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1453133580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3b40a3fb0 con 0x55966c075370 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/1453133580 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc3bc030b50 con 0x55966c075370 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/1453133580 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc3bc030d50 con 0x55966c075370 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3cb97a740 10 client.25520 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1453133580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55966c0efe30 con 0x55966c075370 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/1453133580 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b40a3fb0 con 0x55966c075370 2024-09-11T23:38:56.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/1453133580 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc3bc059030 con 0x55966c075370 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3cb97a740 10 client.25520 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3cb97a740 10 client.25520 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3cb97a740 20 client.25520 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55966c085050 0x55966c0f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3cb97a740 4 client.25520 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1453133580 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55966bfd7d70 con 0x55966c085050 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.978+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55966c085050 0x55966c0f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3a6ff5640 10 client.25520 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3c2ffd640 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55966c085050 0x55966c0f25f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a4c84ae42040292 server_cookie=8ff2dfc2e4b89b9 in_seq=0 out_seq=0 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3a6ff5640 1 -- 172.21.15.133:0/1453133580 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55966bfd7d70 con 0x55966c085050 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3a6ff5640 10 client.25520 handle_command_reply: tid=0 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 1 client.25520 shutdown 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1453133580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 msgr2=0x55966c06f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 0x55966c06f6c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc035000 tx=0x7fc3bc041000 comp rx=0 tx=0).stop 2024-09-11T23:38:56.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1453133580 shutdown_connections 2024-09-11T23:38:56.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55966c085050 0x55966c0f25f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55966c075370 0x55966c06f6c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55966c06bb40 0x55966c06ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 1 --2- 172.21.15.133:0/1453133580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55966c06a5d0 0x55966c07dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3a6ff5640 0 client.25520 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:38:56.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1453133580 >> 172.21.15.133:0/1453133580 conn(0x55966c033310 msgr2=0x55966c0654e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1453133580 shutdown_connections 2024-09-11T23:38:56.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 1 -- 172.21.15.133:0/1453133580 wait complete. 2024-09-11T23:38:56.985 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:38:56.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.979+0000 7fc3cb97a740 20 client.25520 trim_cache size 0 max 16384 2024-09-11T23:38:56.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 -- 172.21.15.133:0/2350360527 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc3ac0a2cf0 msgr2=0x7fc3ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc3ac0a2cf0 0x7fc3ac0a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009ca0 tx=0x7fc3bc0095c0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 -- 172.21.15.133:0/2350360527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4100e20 msgr2=0x7fc3c41984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:56.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4100e20 0x7fc3c41984d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400d770 tx=0x7fc3b400dc30 comp rx=0 tx=0).stop 2024-09-11T23:38:56.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 -- 172.21.15.133:0/2350360527 shutdown_connections 2024-09-11T23:38:56.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc3ac0a2cf0 0x7fc3ac0a51b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3c4102ef0 0x7fc3c419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3c4102570 0x7fc3c4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 --2- 172.21.15.133:0/2350360527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4100e20 0x7fc3c41984d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:56.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 -- 172.21.15.133:0/2350360527 >> 172.21.15.133:0/2350360527 conn(0x7fc3c40fc7d0 msgr2=0x7fc3c406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:56.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 -- 172.21.15.133:0/2350360527 shutdown_connections 2024-09-11T23:38:56.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:56.980+0000 7fc3c95a1640 1 -- 172.21.15.133:0/2350360527 wait complete. 2024-09-11T23:38:57.887 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:57 smithi133 ceph-mon[31430]: pgmap v68: 97 pgs: 97 active+clean; 10 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 623 MiB/s wr, 1.22k op/s 2024-09-11T23:38:57.888 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1fb2b5b2', quiesce took 8.1 seconds. Will release after: 5.300000000000001 2024-09-11T23:38:58.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:57 smithi178 ceph-mon[35882]: pgmap v68: 97 pgs: 97 active+clean; 10 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 623 MiB/s wr, 1.22k op/s 2024-09-11T23:38:58.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:57 smithi138 ceph-mon[35142]: pgmap v68: 97 pgs: 97 active+clean; 10 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 623 MiB/s wr, 1.22k op/s 2024-09-11T23:38:58.545 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:38:58.803 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:38:58.925 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:38:58 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:38:59.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:38:58 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:38:59.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:38:58 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:38:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.321+0000 7fd5d12a3640 1 -- 172.21.15.133:0/993418985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5cc102ef0 msgr2=0x7fd5cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.321+0000 7fd5d12a3640 1 --2- 172.21.15.133:0/993418985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5cc102ef0 0x7fd5cc10da40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009ca0 tx=0x7fd5c402a950 comp rx=0 tx=0).stop 2024-09-11T23:38:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.321+0000 7fd5d12a3640 1 -- 172.21.15.133:0/993418985 shutdown_connections 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.321+0000 7fd5d12a3640 1 --2- 172.21.15.133:0/993418985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5cc102ef0 0x7fd5cc10da40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.321+0000 7fd5d12a3640 1 --2- 172.21.15.133:0/993418985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc102570 0x7fd5cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.321+0000 7fd5d12a3640 1 --2- 172.21.15.133:0/993418985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5cc100e20 0x7fd5cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.321+0000 7fd5d12a3640 1 -- 172.21.15.133:0/993418985 >> 172.21.15.133:0/993418985 conn(0x7fd5cc0fc7d0 msgr2=0x7fd5cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.321+0000 7fd5d12a3640 1 -- 172.21.15.133:0/993418985 shutdown_connections 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.321+0000 7fd5d12a3640 1 -- 172.21.15.133:0/993418985 wait complete. 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5d12a3640 1 Processor -- start 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5d12a3640 1 -- start start 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5d12a3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5cc100e20 0x7fd5cc197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5d12a3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc102570 0x7fd5cc1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5d12a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5cc102ef0 0x7fd5cc19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5d12a3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc110f80 con 0x7fd5cc102ef0 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5d12a3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc110e00 con 0x7fd5cc100e20 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5d12a3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc111100 con 0x7fd5cc102570 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5caffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5cc100e20 0x7fd5cc197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5ca7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc102570 0x7fd5cc1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5caffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5cc100e20 0x7fd5cc197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:49780/0 (socket says 172.21.15.133:49780) 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5ca7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc102570 0x7fd5cc1978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35430/0 (socket says 172.21.15.133:35430) 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5caffd640 1 -- 172.21.15.133:0/3606773230 learned_addr learned my addr 172.21.15.133:0/3606773230 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.322+0000 7fd5cb7fe640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5cc102ef0 0x7fd5cc19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.323+0000 7fd5caffd640 1 -- 172.21.15.133:0/3606773230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc102570 msgr2=0x7fd5cc1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.323+0000 7fd5caffd640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc102570 0x7fd5cc1978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.323+0000 7fd5caffd640 1 -- 172.21.15.133:0/3606773230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5cc102ef0 msgr2=0x7fd5cc19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.323+0000 7fd5caffd640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5cc102ef0 0x7fd5cc19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.323+0000 7fd5caffd640 1 -- 172.21.15.133:0/3606773230 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc19c330 con 0x7fd5cc100e20 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.323+0000 7fd5caffd640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5cc100e20 0x7fd5cc197380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00e9f0 tx=0x7fd5bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.323+0000 7fd5b3fff640 1 -- 172.21.15.133:0/3606773230 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc00cd80 con 0x7fd5cc100e20 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.323+0000 7fd5d12a3640 1 -- 172.21.15.133:0/3606773230 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5cc19c620 con 0x7fd5cc100e20 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.323+0000 7fd5d12a3640 1 -- 172.21.15.133:0/3606773230 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5cc1a3f00 con 0x7fd5cc100e20 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.323+0000 7fd5b3fff640 1 -- 172.21.15.133:0/3606773230 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc010910 con 0x7fd5cc100e20 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.323+0000 7fd5b3fff640 1 -- 172.21.15.133:0/3606773230 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc018980 con 0x7fd5cc100e20 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.325+0000 7fd5b3fff640 1 -- 172.21.15.133:0/3606773230 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd5bc010ac0 con 0x7fd5cc100e20 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5d367c740 1 Processor -- start 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5d367c740 1 -- start start 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61bb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 0x562ee61bba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 0x562ee61ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5d367c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562ee61212f0 con 0x562ee61c8580 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5d367c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562ee60c8cf0 con 0x562ee61bb5d0 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5b3fff640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd5b40a2cf0 0x7fd5b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5d367c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ee611d2b0 con 0x562ee61bcb40 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5cb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61bb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5caffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 0x562ee61bba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5cb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61bb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35438/0 (socket says 172.21.15.133:35438) 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5cb7fe640 1 -- 172.21.15.133:0/1042250610 learned_addr learned my addr 172.21.15.133:0/1042250610 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5ca7fc640 1 --2- 172.21.15.133:0/1042250610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 0x562ee61ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.326+0000 7fd5cb7fe640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd5b40a2cf0 0x7fd5b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5b3fff640 1 -- 172.21.15.133:0/3606773230 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fd5bc0cf7b0 con 0x7fd5cc100e20 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5caffd640 1 -- 172.21.15.133:0/1042250610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 msgr2=0x562ee61bb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5caffd640 1 --2- 172.21.15.133:0/1042250610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61bb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5caffd640 1 -- 172.21.15.133:0/1042250610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 msgr2=0x562ee61ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5caffd640 1 --2- 172.21.15.133:0/1042250610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 0x562ee61ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5caffd640 1 -- 172.21.15.133:0/1042250610 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee60ad3a0 con 0x562ee61bb5d0 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5ca7fc640 1 --2- 172.21.15.133:0/1042250610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 0x562ee61ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5cb7fe640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd5b40a2cf0 0x7fd5b40a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4005fe0 tx=0x7fd5c4005c50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5caffd640 1 --2- 172.21.15.133:0/1042250610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 0x562ee61bba10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0ccf70 tx=0x7fd5bc06ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e632672dd561d9e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5b17fa640 1 -- 172.21.15.133:0/1042250610 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0a00f0 con 0x562ee61bb5d0 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5b17fa640 1 -- 172.21.15.133:0/1042250610 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc0a49b0 con 0x562ee61bb5d0 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.327+0000 7fd5b17fa640 1 -- 172.21.15.133:0/1042250610 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc09b4a0 con 0x562ee61bb5d0 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.328+0000 7fd5d367c740 1 -- 172.21.15.133:0/1042250610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 msgr2=0x562ee61bba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.328+0000 7fd5d367c740 1 --2- 172.21.15.133:0/1042250610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 0x562ee61bba10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0ccf70 tx=0x7fd5bc06ef20 comp rx=0 tx=0).stop 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.328+0000 7fd5d367c740 1 -- 172.21.15.133:0/1042250610 shutdown_connections 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.328+0000 7fd5d367c740 1 --2- 172.21.15.133:0/1042250610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 0x562ee61ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.328+0000 7fd5d367c740 1 --2- 172.21.15.133:0/1042250610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 0x562ee61bba10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.328+0000 7fd5d367c740 1 --2- 172.21.15.133:0/1042250610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61bb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.328+0000 7fd5d367c740 1 -- 172.21.15.133:0/1042250610 >> 172.21.15.133:0/1042250610 conn(0x562ee6184310 msgr2=0x562ee61b7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:59.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.328+0000 7fd5d367c740 1 -- 172.21.15.133:0/1042250610 shutdown_connections 2024-09-11T23:38:59.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.328+0000 7fd5d367c740 1 -- 172.21.15.133:0/1042250610 wait complete. 2024-09-11T23:38:59.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.328+0000 7fd5d367c740 1 -- start start 2024-09-11T23:38:59.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.328+0000 7fd5d367c740 1 Processor -- start 2024-09-11T23:38:59.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 0x562ee61c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61c5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 0x562ee61c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5d367c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562ee61cb1c0 con 0x562ee61c8580 2024-09-11T23:38:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5d367c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562ee60dc060 con 0x562ee61bb5d0 2024-09-11T23:38:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5d367c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ee61cb340 con 0x562ee61bcb40 2024-09-11T23:38:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5d367c740 10 client.0 _ll_register_callbacks cb 0x562ee5e2f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5caffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 0x562ee61c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5cb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 0x562ee61c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5caffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 0x562ee61c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:49794/0 (socket says 172.21.15.133:49794) 2024-09-11T23:38:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5ca7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5caffd640 1 -- 172.21.15.133:0/3051502754 learned_addr learned my addr 172.21.15.133:0/3051502754 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5caffd640 1 -- 172.21.15.133:0/3051502754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 msgr2=0x562ee61c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5caffd640 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5caffd640 1 -- 172.21.15.133:0/3051502754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 msgr2=0x562ee61c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5caffd640 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 0x562ee61c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5caffd640 1 -- 172.21.15.133:0/3051502754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee5e923c0 con 0x562ee61bb5d0 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5cb7fe640 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 0x562ee61c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5caffd640 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 0x562ee61c7f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0cd3d0 tx=0x7fd5bc00c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa87f50efdb75674 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.329+0000 7fd5aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3051502754 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0a00f0 con 0x562ee61bb5d0 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3051502754 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fd5bc09b450 con 0x562ee61bb5d0 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5d367c740 1 -- 172.21.15.133:0/3051502754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5bc0a49b0 con 0x562ee61bb5d0 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3051502754 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd5bc06b050 con 0x562ee61bb5d0 2024-09-11T23:38:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5d367c740 10 client.25532 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5d367c740 1 -- 172.21.15.133:0/3051502754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ee61d6c60 con 0x562ee61bb5d0 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3051502754 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc0a49b0 con 0x562ee61bb5d0 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3051502754 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd5bc09acb0 con 0x562ee61bb5d0 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5d367c740 10 client.25532 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5d367c740 10 client.25532 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5d367c740 20 client.25532 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x562ee61d7fb0 0x562ee61da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5d367c740 4 client.25532 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5d367c740 1 -- 172.21.15.133:0/3051502754 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ee612e860 con 0x562ee61d7fb0 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.330+0000 7fd5ca7fc640 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x562ee61d7fb0 0x562ee61da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.331+0000 7fd5aeff5640 10 client.25532 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.331+0000 7fd5ca7fc640 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x562ee61d7fb0 0x562ee61da3a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc046fbe3e79e8ae server_cookie=314a77cfdad6c7fc in_seq=0 out_seq=0 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3051502754 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ee612e860 con 0x562ee61d7fb0 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5aeff5640 10 client.25532 handle_command_reply: tid=0 2024-09-11T23:38:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 1 client.25532 shutdown 2024-09-11T23:38:59.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 1 -- 172.21.15.133:0/3051502754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 msgr2=0x562ee61c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:59.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 0x562ee61c7f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0cd3d0 tx=0x7fd5bc00c7d0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 1 -- 172.21.15.133:0/3051502754 shutdown_connections 2024-09-11T23:38:59.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x562ee61d7fb0 0x562ee61da3a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61c8580 0x562ee61c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5aeff5640 0 client.25532 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:38:59.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3051502754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bb5d0 0x562ee61c7f80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 1 -- 172.21.15.133:0/3051502754 >> 172.21.15.133:0/3051502754 conn(0x562ee6184310 msgr2=0x562ee61bd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:59.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 1 -- 172.21.15.133:0/3051502754 shutdown_connections 2024-09-11T23:38:59.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 1 -- 172.21.15.133:0/3051502754 wait complete. 2024-09-11T23:38:59.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.332+0000 7fd5d367c740 20 client.25532 trim_cache size 0 max 16384 2024-09-11T23:38:59.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.345+0000 7fd5d367c740 1 Processor -- start 2024-09-11T23:38:59.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.345+0000 7fd5d367c740 1 -- start start 2024-09-11T23:38:59.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 0x562ee61d4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bcb40 0x562ee61c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61c8580 0x562ee61df940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:38:59.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5d367c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc1ab3a0 con 0x562ee61bb5d0 2024-09-11T23:38:59.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5d367c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc1acee0 con 0x562ee61bcb40 2024-09-11T23:38:59.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5d367c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc1aba70 con 0x562ee61c8580 2024-09-11T23:38:59.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5caffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 0x562ee61d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5caffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 0x562ee61d4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40382/0 (socket says 172.21.15.133:40382) 2024-09-11T23:38:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5caffd640 1 -- 172.21.15.133:0/2847485933 learned_addr learned my addr 172.21.15.133:0/2847485933 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5ca7fc640 1 --2- 172.21.15.133:0/2847485933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bcb40 0x562ee61c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5caffd640 1 -- 172.21.15.133:0/2847485933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61c8580 msgr2=0x562ee61df940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5caffd640 1 --2- 172.21.15.133:0/2847485933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61c8580 0x562ee61df940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5caffd640 1 -- 172.21.15.133:0/2847485933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bcb40 msgr2=0x562ee61c40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5caffd640 1 --2- 172.21.15.133:0/2847485933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bcb40 0x562ee61c40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5caffd640 1 -- 172.21.15.133:0/2847485933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee61b8960 con 0x562ee61bb5d0 2024-09-11T23:38:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5caffd640 1 --2- 172.21.15.133:0/2847485933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 0x562ee61d4460 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc09baf0 tx=0x7fd5bc0a4830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34411535e8ab7c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5b0ff9640 1 -- 172.21.15.133:0/2847485933 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc065030 con 0x562ee61bb5d0 2024-09-11T23:38:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5b0ff9640 1 -- 172.21.15.133:0/2847485933 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc0a49b0 con 0x562ee61bb5d0 2024-09-11T23:38:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.346+0000 7fd5b0ff9640 1 -- 172.21.15.133:0/2847485933 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc09acb0 con 0x562ee61bb5d0 2024-09-11T23:38:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.347+0000 7fd5d367c740 1 -- 172.21.15.133:0/2847485933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 msgr2=0x562ee61d4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.347+0000 7fd5d367c740 1 --2- 172.21.15.133:0/2847485933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 0x562ee61d4460 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc09baf0 tx=0x7fd5bc0a4830 comp rx=0 tx=0).stop 2024-09-11T23:38:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.347+0000 7fd5d367c740 1 -- 172.21.15.133:0/2847485933 shutdown_connections 2024-09-11T23:38:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.347+0000 7fd5d367c740 1 --2- 172.21.15.133:0/2847485933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61c8580 0x562ee61df940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.347+0000 7fd5d367c740 1 --2- 172.21.15.133:0/2847485933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61bcb40 0x562ee61c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.347+0000 7fd5d367c740 1 --2- 172.21.15.133:0/2847485933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 0x562ee61d4460 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.347+0000 7fd5d367c740 1 -- 172.21.15.133:0/2847485933 >> 172.21.15.133:0/2847485933 conn(0x562ee6184310 msgr2=0x562ee61c39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.347+0000 7fd5d367c740 1 -- 172.21.15.133:0/2847485933 shutdown_connections 2024-09-11T23:38:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.347+0000 7fd5d367c740 1 -- 172.21.15.133:0/2847485933 wait complete. 2024-09-11T23:38:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5d367c740 1 -- start start 2024-09-11T23:38:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5d367c740 1 Processor -- start 2024-09-11T23:38:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 0x562ee61bec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5d367c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61c8580 0x562ee61cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5d367c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc1ac810 con 0x562ee61bb5d0 2024-09-11T23:38:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5d367c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc1ac140 con 0x562ee61c8580 2024-09-11T23:38:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5d367c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc1a93a0 con 0x562ee61bcb40 2024-09-11T23:38:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5d367c740 10 client.0 _ll_register_callbacks cb 0x7fd5c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:38:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5caffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 0x562ee61bec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5ca7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5ca7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61bf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35452/0 (socket says 172.21.15.133:35452) 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5ca7fc640 1 -- 172.21.15.133:0/3352334816 learned_addr learned my addr 172.21.15.133:0/3352334816 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.348+0000 7fd5cb7fe640 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61c8580 0x562ee61cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5caffd640 1 -- 172.21.15.133:0/3352334816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 msgr2=0x562ee61bf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5caffd640 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61bf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5caffd640 1 -- 172.21.15.133:0/3352334816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61c8580 msgr2=0x562ee61cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5caffd640 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61c8580 0x562ee61cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5caffd640 1 -- 172.21.15.133:0/3352334816 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee61cfc10 con 0x562ee61bb5d0 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5ca7fc640 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61bf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5cb7fe640 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61c8580 0x562ee61cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5caffd640 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 0x562ee61bec60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc09baf0 tx=0x7fd5bc097d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da471f717dbaca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:38:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3352334816 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc065070 con 0x562ee61bb5d0 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3352334816 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fd5bc018d30 con 0x562ee61bb5d0 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5d367c740 1 -- 172.21.15.133:0/3352334816 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5bc0a49b0 con 0x562ee61bb5d0 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3352334816 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd5bc0a00f0 con 0x562ee61bb5d0 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5d367c740 10 client.16113 fetch_fsmap learned FSMap version 28 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5d367c740 1 -- 172.21.15.133:0/3352334816 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ee61cfc10 con 0x562ee61bb5d0 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3352334816 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc097b40 con 0x562ee61bb5d0 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.349+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3352334816 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd5bc086390 con 0x562ee61bb5d0 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5d367c740 10 client.16113 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5d367c740 10 client.16113 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5d367c740 20 client.16113 populate_metadata read hostname 'smithi133' 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x562ee6244d10 0x562ee6247100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5d367c740 4 client.16113 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:38:59.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5d367c740 1 -- 172.21.15.133:0/3352334816 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd5bc0a49b0 con 0x562ee6244d10 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5ca7fc640 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x562ee6244d10 0x562ee6247100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5aeff5640 10 client.16113 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5ca7fc640 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x562ee6244d10 0x562ee6247100 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=985da3dcf5eddcb7 server_cookie=3c45910b855590f2 in_seq=0 out_seq=0 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5aeff5640 1 -- 172.21.15.133:0/3352334816 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd5bc0a49b0 con 0x562ee6244d10 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5aeff5640 10 client.16113 handle_command_reply: tid=0 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5d367c740 1 client.16113 shutdown 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5d367c740 1 -- 172.21.15.133:0/3352334816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 msgr2=0x562ee61bec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.350+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 0x562ee61bec60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc09baf0 tx=0x7fd5bc097d90 comp rx=0 tx=0).stop 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.351+0000 7fd5d367c740 1 -- 172.21.15.133:0/3352334816 shutdown_connections 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.351+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x562ee6244d10 0x562ee6247100 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.351+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ee61c8580 0x562ee61cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.351+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ee61bcb40 0x562ee61bf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.351+0000 7fd5d367c740 1 --2- 172.21.15.133:0/3352334816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ee61bb5d0 0x562ee61bec60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.351+0000 7fd5d367c740 1 -- 172.21.15.133:0/3352334816 >> 172.21.15.133:0/3352334816 conn(0x562ee6184310 msgr2=0x562ee61b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.351+0000 7fd5aeff5640 0 client.16113 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.351+0000 7fd5d367c740 1 -- 172.21.15.133:0/3352334816 shutdown_connections 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.351+0000 7fd5d367c740 1 -- 172.21.15.133:0/3352334816 wait complete. 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.351+0000 7fd5d367c740 20 client.16113 trim_cache size 0 max 16384 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 -- 172.21.15.133:0/3606773230 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd5b40a2cf0 msgr2=0x7fd5b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd5b40a2cf0 0x7fd5b40a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4005fe0 tx=0x7fd5c4005c50 comp rx=0 tx=0).stop 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 -- 172.21.15.133:0/3606773230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5cc100e20 msgr2=0x7fd5cc197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5cc100e20 0x7fd5cc197380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00e9f0 tx=0x7fd5bc00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 -- 172.21.15.133:0/3606773230 shutdown_connections 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd5b40a2cf0 0x7fd5b40a51b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5cc102ef0 0x7fd5cc19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc102570 0x7fd5cc1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 --2- 172.21.15.133:0/3606773230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5cc100e20 0x7fd5cc197380 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:38:59.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 -- 172.21.15.133:0/3606773230 >> 172.21.15.133:0/3606773230 conn(0x7fd5cc0fc7d0 msgr2=0x7fd5cc106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:38:59.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 -- 172.21.15.133:0/3606773230 shutdown_connections 2024-09-11T23:38:59.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:38:59.352+0000 7fd5d12a3640 1 -- 172.21.15.133:0/3606773230 wait complete. 2024-09-11T23:39:00.127 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:39:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:00 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:00 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:00 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:00 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:00 smithi178 ceph-mon[35882]: pgmap v69: 97 pgs: 97 active+clean; 10 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 895 MiB/s wr, 1.99k op/s 2024-09-11T23:39:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:00 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:00 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:00 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:39:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:00 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:39:00.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:00 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.375 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:00.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:00 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:00 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:00 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:00 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:00 smithi138 ceph-mon[35142]: pgmap v69: 97 pgs: 97 active+clean; 10 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 895 MiB/s wr, 1.99k op/s 2024-09-11T23:39:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:00 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:00 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:00 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:39:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:00 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:39:00.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:00 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:00 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:00 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:00 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:00 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:00 smithi133 ceph-mon[31430]: pgmap v69: 97 pgs: 97 active+clean; 10 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 895 MiB/s wr, 1.99k op/s 2024-09-11T23:39:00.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:00 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:00 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:00 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:39:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:00 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:39:00.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:00 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:39:00.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.821+0000 7f0556139640 1 -- 172.21.15.133:0/3655881675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05501027c0 msgr2=0x7f0550106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:00.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.821+0000 7f0556139640 1 --2- 172.21.15.133:0/3655881675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05501027c0 0x7f0550106810 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0538009900 tx=0x7f053802a410 comp rx=0 tx=0).stop 2024-09-11T23:39:00.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.821+0000 7f0556139640 1 -- 172.21.15.133:0/3655881675 shutdown_connections 2024-09-11T23:39:00.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.821+0000 7f0556139640 1 --2- 172.21.15.133:0/3655881675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0550106f40 0x7f055010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.821+0000 7f0556139640 1 --2- 172.21.15.133:0/3655881675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05501027c0 0x7f0550106810 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.821+0000 7f0556139640 1 --2- 172.21.15.133:0/3655881675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0550101e10 0x7f05501021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.821+0000 7f0556139640 1 -- 172.21.15.133:0/3655881675 >> 172.21.15.133:0/3655881675 conn(0x7f05500fd8d0 msgr2=0x7f05500ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.821+0000 7f0556139640 1 -- 172.21.15.133:0/3655881675 shutdown_connections 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.821+0000 7f0556139640 1 -- 172.21.15.133:0/3655881675 wait complete. 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0556139640 1 Processor -- start 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0556139640 1 -- start start 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0556139640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0550101e10 0x7f05501983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0556139640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05501027c0 0x7f0550198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0556139640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0550106f40 0x7f055019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0556139640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0550111f60 con 0x7f0550101e10 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0556139640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0550111de0 con 0x7f05501027c0 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0556139640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f05501120e0 con 0x7f0550106f40 2024-09-11T23:39:00.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f054f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0550101e10 0x7f05501983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0547fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05501027c0 0x7f0550198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0547fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05501027c0 0x7f0550198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:49842/0 (socket says 172.21.15.133:49842) 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0547fff640 1 -- 172.21.15.133:0/1360719104 learned_addr learned my addr 172.21.15.133:0/1360719104 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f054f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0550101e10 0x7f05501983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40406/0 (socket says 172.21.15.133:40406) 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f054ffff640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0550106f40 0x7f055019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0547fff640 1 -- 172.21.15.133:0/1360719104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0550106f40 msgr2=0x7f055019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0547fff640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0550106f40 0x7f055019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0547fff640 1 -- 172.21.15.133:0/1360719104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0550101e10 msgr2=0x7f05501983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0547fff640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0550101e10 0x7f05501983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.822+0000 7f0547fff640 1 -- 172.21.15.133:0/1360719104 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055019d380 con 0x7f05501027c0 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.823+0000 7f054f7fe640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0550101e10 0x7f05501983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.823+0000 7f0547fff640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05501027c0 0x7f0550198910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0538006010 tx=0x7f05380042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.823+0000 7f054d7fa640 1 -- 172.21.15.133:0/1360719104 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053803f030 con 0x7f05501027c0 2024-09-11T23:39:00.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.823+0000 7f0556139640 1 -- 172.21.15.133:0/1360719104 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055019d610 con 0x7f05501027c0 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.823+0000 7f054d7fa640 1 -- 172.21.15.133:0/1360719104 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0538003ce0 con 0x7f05501027c0 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.823+0000 7f054d7fa640 1 -- 172.21.15.133:0/1360719104 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538035070 con 0x7f05501027c0 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.823+0000 7f0556139640 1 -- 172.21.15.133:0/1360719104 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05501a4f50 con 0x7f05501027c0 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.825+0000 7f054d7fa640 1 -- 172.21.15.133:0/1360719104 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0538042030 con 0x7f05501027c0 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.825+0000 7f0558512740 1 Processor -- start 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.825+0000 7f0558512740 1 -- start start 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f0558512740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f0558512740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f0558512740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b1502760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f0558512740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5611b145b2f0 con 0x5611b14f6b40 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f0558512740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5611b1402cf0 con 0x5611b14f55d0 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f0558512740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5611b14572b0 con 0x5611b1500370 2024-09-11T23:39:00.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40414/0 (socket says 172.21.15.133:40414) 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054ffff640 1 -- 172.21.15.133:0/167440460 learned_addr learned my addr 172.21.15.133:0/167440460 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054f7fe640 1 --2- 172.21.15.133:0/167440460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054d7fa640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f053c0a2cf0 0x7f053c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f0547fff640 1 --2- 172.21.15.133:0/167440460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b1502760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054ffff640 1 -- 172.21.15.133:0/167440460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 msgr2=0x5611b1502760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054ffff640 1 --2- 172.21.15.133:0/167440460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b1502760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054ffff640 1 -- 172.21.15.133:0/167440460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 msgr2=0x5611b14f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054ffff640 1 --2- 172.21.15.133:0/167440460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054ffff640 1 -- 172.21.15.133:0/167440460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611b13e73a0 con 0x5611b14f6b40 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f0547fff640 1 --2- 172.21.15.133:0/167440460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b1502760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054d7fa640 1 -- 172.21.15.133:0/1360719104 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f05380efa60 con 0x7f05501027c0 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054ffff640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f053c0a2cf0 0x7f053c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.826+0000 7f054ffff640 1 --2- 172.21.15.133:0/167440460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f5090 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0548011590 tx=0x7f0548011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cfc7afb54236bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0545ffb640 1 -- 172.21.15.133:0/167440460 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548017050 con 0x5611b14f6b40 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0545ffb640 1 -- 172.21.15.133:0/167440460 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f054801bb90 con 0x5611b14f6b40 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0545ffb640 1 -- 172.21.15.133:0/167440460 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054800e3b0 con 0x5611b14f6b40 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0558512740 1 -- 172.21.15.133:0/167440460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 msgr2=0x5611b14f5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f054ffff640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f053c0a2cf0 0x7f053c0a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0548022000 tx=0x7f0548023000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0558512740 1 --2- 172.21.15.133:0/167440460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f5090 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0548011590 tx=0x7f0548011a50 comp rx=0 tx=0).stop 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0558512740 1 -- 172.21.15.133:0/167440460 shutdown_connections 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0558512740 1 --2- 172.21.15.133:0/167440460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b1502760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0558512740 1 --2- 172.21.15.133:0/167440460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0558512740 1 --2- 172.21.15.133:0/167440460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f5090 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0558512740 1 -- 172.21.15.133:0/167440460 >> 172.21.15.133:0/167440460 conn(0x5611b14be310 msgr2=0x5611b14f1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0558512740 1 -- 172.21.15.133:0/167440460 shutdown_connections 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.827+0000 7f0558512740 1 -- 172.21.15.133:0/167440460 wait complete. 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f0558512740 1 -- start start 2024-09-11T23:39:00.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f0558512740 1 Processor -- start 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f0558512740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f0558512740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14fd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f0558512740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14fd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f0558512740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5611b1503070 con 0x5611b14f6b40 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f0558512740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5611b1416060 con 0x5611b14f55d0 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f0558512740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5611b15031f0 con 0x5611b1500370 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f0558512740 10 client.0 _ll_register_callbacks cb 0x5611b11698c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f0547fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f054f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f054f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14ffd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:49860/0 (socket says 172.21.15.133:49860) 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f0547fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14fd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40420/0 (socket says 172.21.15.133:40420) 2024-09-11T23:39:00.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f054f7fe640 1 -- 172.21.15.133:0/414792137 learned_addr learned my addr 172.21.15.133:0/414792137 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.828+0000 7f054ffff640 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f0547fff640 1 -- 172.21.15.133:0/414792137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 msgr2=0x5611b14fd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f0547fff640 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14fd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f0547fff640 1 -- 172.21.15.133:0/414792137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 msgr2=0x5611b14ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f0547fff640 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f0547fff640 1 -- 172.21.15.133:0/414792137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611b11cc3c0 con 0x5611b14f6b40 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f054f7fe640 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f0547fff640 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14fd030 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f05380ed220 tx=0x7f053808af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da52e9a4898a277c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f052effd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f052effd640 1 -- 172.21.15.133:0/414792137 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538049070 con 0x5611b14f6b40 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f052effd640 1 -- 172.21.15.133:0/414792137 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f05380bc250 con 0x5611b14f6b40 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f0558512740 1 -- 172.21.15.133:0/414792137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f054801bb90 con 0x5611b14f6b40 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f052effd640 1 -- 172.21.15.133:0/414792137 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05380bbb30 con 0x5611b14f6b40 2024-09-11T23:39:00.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f0558512740 10 client.16125 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f0558512740 1 -- 172.21.15.133:0/414792137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611b150eab0 con 0x5611b14f6b40 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.829+0000 7f052effd640 1 -- 172.21.15.133:0/414792137 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f054801bb90 con 0x5611b14f6b40 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.830+0000 7f052effd640 1 -- 172.21.15.133:0/414792137 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f05380c47d0 con 0x5611b14f6b40 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.830+0000 7f0558512740 10 client.16125 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.830+0000 7f0558512740 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.830+0000 7f0558512740 20 client.16125 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.830+0000 7f0558512740 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611b150fe00 0x5611b15121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.830+0000 7f0558512740 4 client.16125 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.830+0000 7f0558512740 1 -- 172.21.15.133:0/414792137 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611b1468860 con 0x5611b150fe00 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.830+0000 7f054f7fe640 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611b150fe00 0x5611b15121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f052effd640 10 client.16125 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f054f7fe640 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611b150fe00 0x5611b15121f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c96dd9e77045d286 server_cookie=5b4a4b2fe7f4b01d in_seq=0 out_seq=0 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f052effd640 1 -- 172.21.15.133:0/414792137 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611b1468860 con 0x5611b150fe00 2024-09-11T23:39:00.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f052effd640 10 client.16125 handle_command_reply: tid=0 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f0558512740 1 client.16125 shutdown 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f0558512740 1 -- 172.21.15.133:0/414792137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 msgr2=0x5611b14fd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f0558512740 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14fd030 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f05380ed220 tx=0x7f053808af20 comp rx=0 tx=0).stop 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f0558512740 1 -- 172.21.15.133:0/414792137 shutdown_connections 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f0558512740 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611b150fe00 0x5611b15121f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f0558512740 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14fd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f0558512740 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14fd030 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f0558512740 1 --2- 172.21.15.133:0/414792137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f0558512740 1 -- 172.21.15.133:0/414792137 >> 172.21.15.133:0/414792137 conn(0x5611b14be310 msgr2=0x5611b14f7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.831+0000 7f052effd640 0 client.16125 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.832+0000 7f0558512740 1 -- 172.21.15.133:0/414792137 shutdown_connections 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.832+0000 7f0558512740 1 -- 172.21.15.133:0/414792137 wait complete. 2024-09-11T23:39:00.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.832+0000 7f0558512740 20 client.16125 trim_cache size 0 max 16384 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f0558512740 1 Processor -- start 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f0558512740 1 -- start start 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f0558512740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f0558512740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f0558512740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14f8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f0558512740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f05501ad560 con 0x5611b14f6b40 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f0558512740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05501af0a0 con 0x5611b14f55d0 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f0558512740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f05501adc30 con 0x5611b1500370 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f0547fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f0547fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40432/0 (socket says 172.21.15.133:40432) 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f0547fff640 1 -- 172.21.15.133:0/3000062109 learned_addr learned my addr 172.21.15.133:0/3000062109 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:00.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.845+0000 7f054f7fe640 1 --2- 172.21.15.133:0/3000062109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.846+0000 7f054ffff640 1 --2- 172.21.15.133:0/3000062109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.846+0000 7f0547fff640 1 -- 172.21.15.133:0/3000062109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 msgr2=0x5611b14f8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.846+0000 7f0547fff640 1 --2- 172.21.15.133:0/3000062109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14f8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.846+0000 7f0547fff640 1 -- 172.21.15.133:0/3000062109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 msgr2=0x5611b14fefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.846+0000 7f0547fff640 1 --2- 172.21.15.133:0/3000062109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14fefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.846+0000 7f0547fff640 1 -- 172.21.15.133:0/3000062109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611b14f52d0 con 0x5611b14f6b40 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.846+0000 7f0547fff640 1 --2- 172.21.15.133:0/3000062109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f8120 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f05380b89c0 tx=0x7f053802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f87b793902d355f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.846+0000 7f05457fa640 1 -- 172.21.15.133:0/3000062109 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538049070 con 0x5611b14f6b40 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.846+0000 7f05457fa640 1 -- 172.21.15.133:0/3000062109 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05380bc6f0 con 0x5611b14f6b40 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.846+0000 7f05457fa640 1 -- 172.21.15.133:0/3000062109 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05380bb950 con 0x5611b14f6b40 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.847+0000 7f0558512740 1 -- 172.21.15.133:0/3000062109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 msgr2=0x5611b14f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.847+0000 7f0558512740 1 --2- 172.21.15.133:0/3000062109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f8120 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f05380b89c0 tx=0x7f053802a410 comp rx=0 tx=0).stop 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.847+0000 7f0558512740 1 -- 172.21.15.133:0/3000062109 shutdown_connections 2024-09-11T23:39:00.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.847+0000 7f0558512740 1 --2- 172.21.15.133:0/3000062109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.847+0000 7f0558512740 1 --2- 172.21.15.133:0/3000062109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b14f8120 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.847+0000 7f0558512740 1 --2- 172.21.15.133:0/3000062109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b14fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.847+0000 7f0558512740 1 -- 172.21.15.133:0/3000062109 >> 172.21.15.133:0/3000062109 conn(0x5611b14be310 msgr2=0x5611b14fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.847+0000 7f0558512740 1 -- 172.21.15.133:0/3000062109 shutdown_connections 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.847+0000 7f0558512740 1 -- 172.21.15.133:0/3000062109 wait complete. 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.847+0000 7f0558512740 1 -- start start 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.847+0000 7f0558512740 1 Processor -- start 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0558512740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b1508760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0558512740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b1508ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0558512740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14f9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0558512740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f05501ae9d0 con 0x5611b14f6b40 2024-09-11T23:39:00.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0558512740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05501ae300 con 0x5611b14f55d0 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0558512740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f05501ab540 con 0x5611b1500370 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0558512740 10 client.0 _ll_register_callbacks cb 0x7f05380314b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f054f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b1508760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0547fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b1508ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f054f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b1508760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:49870/0 (socket says 172.21.15.133:49870) 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f054f7fe640 1 -- 172.21.15.133:0/4219473047 learned_addr learned my addr 172.21.15.133:0/4219473047 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f054ffff640 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14f9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0547fff640 1 -- 172.21.15.133:0/4219473047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 msgr2=0x5611b14f9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0547fff640 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14f9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0547fff640 1 -- 172.21.15.133:0/4219473047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 msgr2=0x5611b1508760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0547fff640 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b1508760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0547fff640 1 -- 172.21.15.133:0/4219473047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611b14fa280 con 0x5611b14f6b40 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f054f7fe640 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b1508760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:00.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f054ffff640 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14f9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0547fff640 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b1508ca0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f05380b89c0 tx=0x7f05380c4a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c248c6f766e574f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f052effd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f052effd640 1 -- 172.21.15.133:0/4219473047 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538049070 con 0x5611b14f6b40 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f052effd640 1 -- 172.21.15.133:0/4219473047 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f053803bd90 con 0x5611b14f6b40 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.848+0000 7f0558512740 1 -- 172.21.15.133:0/4219473047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05380bc6f0 con 0x5611b14f6b40 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f052effd640 1 -- 172.21.15.133:0/4219473047 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05380bb950 con 0x5611b14f6b40 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f0558512740 10 client.16131 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f0558512740 1 -- 172.21.15.133:0/4219473047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611b14fa280 con 0x5611b14f6b40 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f052effd640 1 -- 172.21.15.133:0/4219473047 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05380bc6f0 con 0x5611b14f6b40 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f052effd640 1 -- 172.21.15.133:0/4219473047 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f05380bb4d0 con 0x5611b14f6b40 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f0558512740 10 client.16131 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f0558512740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f0558512740 20 client.16131 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f0558512740 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611b157a970 0x5611b157cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f0558512740 4 client.16131 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f0558512740 1 -- 172.21.15.133:0/4219473047 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611b1462d70 con 0x5611b157a970 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.849+0000 7f054f7fe640 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611b157a970 0x5611b157cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f054f7fe640 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611b157a970 0x5611b157cd60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba1473b74123259b server_cookie=52380d4372febaf2 in_seq=0 out_seq=0 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f052effd640 10 client.16131 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f052effd640 1 -- 172.21.15.133:0/4219473047 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611b1462d70 con 0x5611b157a970 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f052effd640 10 client.16131 handle_command_reply: tid=0 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 1 client.16131 shutdown 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 1 -- 172.21.15.133:0/4219473047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 msgr2=0x5611b1508ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b1508ca0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f05380b89c0 tx=0x7f05380c4a80 comp rx=0 tx=0).stop 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 1 -- 172.21.15.133:0/4219473047 shutdown_connections 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611b157a970 0x5611b157cd60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611b1500370 0x5611b14f9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611b14f6b40 0x5611b1508ca0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f052effd640 0 client.16131 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:00.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 1 --2- 172.21.15.133:0/4219473047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611b14f55d0 0x5611b1508760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 1 -- 172.21.15.133:0/4219473047 >> 172.21.15.133:0/4219473047 conn(0x5611b14be310 msgr2=0x5611b14ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:00.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 1 -- 172.21.15.133:0/4219473047 shutdown_connections 2024-09-11T23:39:00.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 1 -- 172.21.15.133:0/4219473047 wait complete. 2024-09-11T23:39:00.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.850+0000 7f0558512740 20 client.16131 trim_cache size 0 max 16384 2024-09-11T23:39:00.866 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:39:00.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 -- 172.21.15.133:0/1360719104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f053c0a2cf0 msgr2=0x7f053c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:00.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f053c0a2cf0 0x7f053c0a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0548022000 tx=0x7f0548023000 comp rx=0 tx=0).stop 2024-09-11T23:39:00.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 -- 172.21.15.133:0/1360719104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05501027c0 msgr2=0x7f0550198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:00.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05501027c0 0x7f0550198910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0538006010 tx=0x7f05380042f0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 -- 172.21.15.133:0/1360719104 shutdown_connections 2024-09-11T23:39:00.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f053c0a2cf0 0x7f053c0a51b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0550106f40 0x7f055019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05501027c0 0x7f0550198910 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 --2- 172.21.15.133:0/1360719104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0550101e10 0x7f05501983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 -- 172.21.15.133:0/1360719104 >> 172.21.15.133:0/1360719104 conn(0x7f05500fd8d0 msgr2=0x7f05500fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 -- 172.21.15.133:0/1360719104 shutdown_connections 2024-09-11T23:39:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:00.851+0000 7f0556139640 1 -- 172.21.15.133:0/1360719104 wait complete. 2024-09-11T23:39:02.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:02 smithi178 ceph-mon[35882]: pgmap v70: 97 pgs: 97 active+clean; 10 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 580 MiB/s wr, 1.68k op/s 2024-09-11T23:39:02.587 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:39:02.588 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:39:02.626 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:02 smithi133 ceph-mon[31430]: pgmap v70: 97 pgs: 97 active+clean; 10 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 580 MiB/s wr, 1.68k op/s 2024-09-11T23:39:02.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:02 smithi138 ceph-mon[35142]: pgmap v70: 97 pgs: 97 active+clean; 10 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 580 MiB/s wr, 1.68k op/s 2024-09-11T23:39:02.922 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.032+0000 7f43c25cf640 1 -- 172.21.15.133:0/853945723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43bc101ad0 msgr2=0x7f43bc101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.032+0000 7f43c25cf640 1 --2- 172.21.15.133:0/853945723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43bc101ad0 0x7f43bc101f30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009930 tx=0x7f43b002a420 comp rx=0 tx=0).stop 2024-09-11T23:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.032+0000 7f43c25cf640 1 -- 172.21.15.133:0/853945723 shutdown_connections 2024-09-11T23:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.032+0000 7f43c25cf640 1 --2- 172.21.15.133:0/853945723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43bc102470 0x7f43bc10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.032+0000 7f43c25cf640 1 --2- 172.21.15.133:0/853945723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43bc101ad0 0x7f43bc101f30 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.032+0000 7f43c25cf640 1 --2- 172.21.15.133:0/853945723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43bc107b70 0x7f43bc107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.032+0000 7f43c25cf640 1 -- 172.21.15.133:0/853945723 >> 172.21.15.133:0/853945723 conn(0x7f43bc0fd7d0 msgr2=0x7f43bc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.032+0000 7f43c25cf640 1 -- 172.21.15.133:0/853945723 shutdown_connections 2024-09-11T23:39:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.032+0000 7f43c25cf640 1 -- 172.21.15.133:0/853945723 wait complete. 2024-09-11T23:39:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.033+0000 7f43c25cf640 1 Processor -- start 2024-09-11T23:39:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.033+0000 7f43c25cf640 1 -- start start 2024-09-11T23:39:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.033+0000 7f43c25cf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43bc101ad0 0x7f43bc198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.033+0000 7f43c25cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43bc102470 0x7f43bc1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.033+0000 7f43c25cf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43bc107b70 0x7f43bc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.033+0000 7f43c25cf640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc111ee0 con 0x7f43bc102470 2024-09-11T23:39:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.033+0000 7f43c25cf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc111d60 con 0x7f43bc107b70 2024-09-11T23:39:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.033+0000 7f43c25cf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc112060 con 0x7f43bc101ad0 2024-09-11T23:39:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.033+0000 7f43bb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43bc102470 0x7f43bc1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.033+0000 7f43bb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43bc102470 0x7f43bc1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35106/0 (socket says 172.21.15.133:35106) 2024-09-11T23:39:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.033+0000 7f43bb7fe640 1 -- 172.21.15.133:0/1402897291 learned_addr learned my addr 172.21.15.133:0/1402897291 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43bbfff640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43bc101ad0 0x7f43bc198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43c0b6c640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43bc107b70 0x7f43bc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43bb7fe640 1 -- 172.21.15.133:0/1402897291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43bc101ad0 msgr2=0x7f43bc198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43bc101ad0 0x7f43bc198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43bb7fe640 1 -- 172.21.15.133:0/1402897291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43bc107b70 msgr2=0x7f43bc19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43bc107b70 0x7f43bc19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43bb7fe640 1 -- 172.21.15.133:0/1402897291 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc19d380 con 0x7f43bc102470 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43bc102470 0x7f43bc1988d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009230 tx=0x7f43b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43b97fa640 1 -- 172.21.15.133:0/1402897291 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b003f030 con 0x7f43bc102470 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43b97fa640 1 -- 172.21.15.133:0/1402897291 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b0003ad0 con 0x7f43bc102470 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43b97fa640 1 -- 172.21.15.133:0/1402897291 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0035070 con 0x7f43bc102470 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43c25cf640 1 -- 172.21.15.133:0/1402897291 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43bc19d610 con 0x7f43bc102470 2024-09-11T23:39:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.034+0000 7f43c25cf640 1 -- 172.21.15.133:0/1402897291 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43bc1a4f00 con 0x7f43bc102470 2024-09-11T23:39:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.036+0000 7f43b97fa640 1 -- 172.21.15.133:0/1402897291 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f43b0042030 con 0x7f43bc102470 2024-09-11T23:39:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c5443740 1 Processor -- start 2024-09-11T23:39:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c5443740 1 -- start start 2024-09-11T23:39:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c5443740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x558506073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43b97fa640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f43a40a2da0 0x7f43a40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c5443740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x55850607e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c5443740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850607ecb0 0x5585060810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c5443740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558505fd82f0 con 0x558506073af0 2024-09-11T23:39:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c5443740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558505f80270 con 0x55850607ecb0 2024-09-11T23:39:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c5443740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558505fd42b0 con 0x558506074410 2024-09-11T23:39:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c0b6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x558506073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43bbfff640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f43a40a2da0 0x7f43a40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c0b6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x558506073ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35110/0 (socket says 172.21.15.133:35110) 2024-09-11T23:39:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c0b6c640 1 -- 172.21.15.133:0/1165010467 learned_addr learned my addr 172.21.15.133:0/1165010467 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/1165010467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x55850607e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43c0b6c640 1 --2- 172.21.15.133:0/1165010467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850607ecb0 0x5585060810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43b97fa640 1 -- 172.21.15.133:0/1402897291 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f43b00ef970 con 0x7f43bc102470 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43bb7fe640 1 -- 172.21.15.133:0/1165010467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850607ecb0 msgr2=0x5585060810a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/1165010467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850607ecb0 0x5585060810a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43bb7fe640 1 -- 172.21.15.133:0/1165010467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 msgr2=0x558506073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/1165010467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x558506073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.037+0000 7f43bb7fe640 1 -- 172.21.15.133:0/1165010467 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558505f7f340 con 0x558506074410 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.038+0000 7f43c0b6c640 1 --2- 172.21.15.133:0/1165010467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x558506073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.038+0000 7f43bbfff640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f43a40a2da0 0x7f43a40a5260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009b50 tx=0x7f43ac005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.038+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/1165010467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x55850607e770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f43b00ed130 tx=0x7f43b002a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d0eb0f62b0ad36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.038+0000 7f43a27fc640 1 -- 172.21.15.133:0/1165010467 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0049070 con 0x558506074410 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.038+0000 7f43a27fc640 1 -- 172.21.15.133:0/1165010467 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b00bbfb0 con 0x558506074410 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.038+0000 7f43a27fc640 1 -- 172.21.15.133:0/1165010467 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b00c1c20 con 0x558506074410 2024-09-11T23:39:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 -- 172.21.15.133:0/1165010467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 msgr2=0x55850607e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 --2- 172.21.15.133:0/1165010467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x55850607e770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f43b00ed130 tx=0x7f43b002a420 comp rx=0 tx=0).stop 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 -- 172.21.15.133:0/1165010467 shutdown_connections 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 --2- 172.21.15.133:0/1165010467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850607ecb0 0x5585060810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 --2- 172.21.15.133:0/1165010467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x55850607e770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 --2- 172.21.15.133:0/1165010467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x558506073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 -- 172.21.15.133:0/1165010467 >> 172.21.15.133:0/1165010467 conn(0x558506070060 msgr2=0x558506070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 -- 172.21.15.133:0/1165010467 shutdown_connections 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 -- 172.21.15.133:0/1165010467 wait complete. 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 -- start start 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 Processor -- start 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506074410 0x55850607c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55850607ecb0 0x55850607c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558506081640 con 0x558506073af0 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.039+0000 7f43c5443740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558505f61290 con 0x558506074410 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43c5443740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5585060817c0 con 0x55850607ecb0 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43c5443740 10 client.0 _ll_register_callbacks cb 0x558505f92140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43bbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43bbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35118/0 (socket says 172.21.15.133:35118) 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43bbfff640 1 -- 172.21.15.133:0/1502479670 learned_addr learned my addr 172.21.15.133:0/1502479670 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43c0b6c640 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55850607ecb0 0x55850607c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506074410 0x55850607c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43bbfff640 1 -- 172.21.15.133:0/1502479670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55850607ecb0 msgr2=0x55850607c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43bbfff640 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55850607ecb0 0x55850607c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43bbfff640 1 -- 172.21.15.133:0/1502479670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506074410 msgr2=0x55850607c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43bbfff640 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506074410 0x55850607c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43bbfff640 1 -- 172.21.15.133:0/1502479670 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558505ceb210 con 0x558506073af0 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506074410 0x55850607c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43bbfff640 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607bbd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0039a0 tx=0x7f43ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf0b4a68a07d827 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f439b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f439b7fe640 1 -- 172.21.15.133:0/1502479670 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac003e60 con 0x558506073af0 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43c5443740 1 -- 172.21.15.133:0/1502479670 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b00bbfb0 con 0x558506073af0 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f439b7fe640 1 -- 172.21.15.133:0/1502479670 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f43ac00ecc0 con 0x558506073af0 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f439b7fe640 1 -- 172.21.15.133:0/1502479670 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43ac01e420 con 0x558506073af0 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43c5443740 10 client.16143 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.040+0000 7f43c5443740 1 -- 172.21.15.133:0/1502479670 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558505db7620 con 0x558506073af0 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.041+0000 7f439b7fe640 1 -- 172.21.15.133:0/1502479670 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b00bbfb0 con 0x558506073af0 2024-09-11T23:39:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.041+0000 7f439b7fe640 1 -- 172.21.15.133:0/1502479670 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f43ac02e020 con 0x558506073af0 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.041+0000 7f43c5443740 10 client.16143 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.041+0000 7f43c5443740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.041+0000 7f43c5443740 20 client.16143 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.041+0000 7f43c5443740 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55850608e990 0x558506090d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.041+0000 7f43c5443740 4 client.16143 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.041+0000 7f43c5443740 1 -- 172.21.15.133:0/1502479670 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558505fe5a20 con 0x55850608e990 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.041+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55850608e990 0x558506090d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.041+0000 7f439b7fe640 10 client.16143 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.041+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55850608e990 0x558506090d80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e46ef092cd46cf6 server_cookie=653c7704b46364e3 in_seq=0 out_seq=0 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f439b7fe640 1 -- 172.21.15.133:0/1502479670 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558505fe5a20 con 0x55850608e990 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f439b7fe640 10 client.16143 handle_command_reply: tid=0 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 1 client.16143 shutdown 2024-09-11T23:39:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 1 -- 172.21.15.133:0/1502479670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 msgr2=0x55850607bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607bbd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0039a0 tx=0x7f43ac004730 comp rx=0 tx=0).stop 2024-09-11T23:39:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 1 -- 172.21.15.133:0/1502479670 shutdown_connections 2024-09-11T23:39:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55850608e990 0x558506090d80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55850607ecb0 0x55850607c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506074410 0x55850607c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f439b7fe640 0 client.16143 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 1 --2- 172.21.15.133:0/1502479670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607bbd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 1 -- 172.21.15.133:0/1502479670 >> 172.21.15.133:0/1502479670 conn(0x558506070060 msgr2=0x558506075dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 1 -- 172.21.15.133:0/1502479670 shutdown_connections 2024-09-11T23:39:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 1 -- 172.21.15.133:0/1502479670 wait complete. 2024-09-11T23:39:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.042+0000 7f43c5443740 20 client.16143 trim_cache size 0 max 16384 2024-09-11T23:39:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.056+0000 7f43c5443740 1 Processor -- start 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.056+0000 7f43c5443740 1 -- start start 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43c5443740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43c5443740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x55850607db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43c5443740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850607ecb0 0x558506096650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43c5443740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc1ad700 con 0x558506073af0 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43c5443740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc1af240 con 0x55850607ecb0 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43c5443740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc1addd0 con 0x558506074410 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43bbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43bbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35130/0 (socket says 172.21.15.133:35130) 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43bbfff640 1 -- 172.21.15.133:0/696809070 learned_addr learned my addr 172.21.15.133:0/696809070 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43c0b6c640 1 --2- 172.21.15.133:0/696809070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850607ecb0 0x558506096650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43bbfff640 1 -- 172.21.15.133:0/696809070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 msgr2=0x55850607db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/696809070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x55850607db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43bbfff640 1 --2- 172.21.15.133:0/696809070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x55850607db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43bbfff640 1 -- 172.21.15.133:0/696809070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850607ecb0 msgr2=0x558506096650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43bbfff640 1 --2- 172.21.15.133:0/696809070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850607ecb0 0x558506096650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43bbfff640 1 -- 172.21.15.133:0/696809070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585060910f0 con 0x558506073af0 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43c0b6c640 1 --2- 172.21.15.133:0/696809070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850607ecb0 0x558506096650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/696809070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x55850607db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43bbfff640 1 --2- 172.21.15.133:0/696809070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607c480 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f43ac003690 tx=0x7f43ac00e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3ef7dac309c977 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43a1ffb640 1 -- 172.21.15.133:0/696809070 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac036070 con 0x558506073af0 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43a1ffb640 1 -- 172.21.15.133:0/696809070 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43ac011950 con 0x558506073af0 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.057+0000 7f43a1ffb640 1 -- 172.21.15.133:0/696809070 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac01e800 con 0x558506073af0 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.058+0000 7f43c5443740 1 -- 172.21.15.133:0/696809070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 msgr2=0x55850607c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.058+0000 7f43c5443740 1 --2- 172.21.15.133:0/696809070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607c480 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f43ac003690 tx=0x7f43ac00e780 comp rx=0 tx=0).stop 2024-09-11T23:39:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.058+0000 7f43c5443740 1 -- 172.21.15.133:0/696809070 shutdown_connections 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.058+0000 7f43c5443740 1 --2- 172.21.15.133:0/696809070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850607ecb0 0x558506096650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.058+0000 7f43c5443740 1 --2- 172.21.15.133:0/696809070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x55850607db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.058+0000 7f43c5443740 1 --2- 172.21.15.133:0/696809070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558506073af0 0x55850607c480 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.058+0000 7f43c5443740 1 -- 172.21.15.133:0/696809070 >> 172.21.15.133:0/696809070 conn(0x558506070060 msgr2=0x55850607d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.058+0000 7f43c5443740 1 -- 172.21.15.133:0/696809070 shutdown_connections 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.058+0000 7f43c5443740 1 -- 172.21.15.133:0/696809070 wait complete. 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.058+0000 7f43c5443740 1 -- start start 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c5443740 1 Processor -- start 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c5443740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506073af0 0x558506086c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c5443740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x558506087140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c5443740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55850607ecb0 0x5585060783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c5443740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc1aeb70 con 0x55850607ecb0 2024-09-11T23:39:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c5443740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc1ae4a0 con 0x558506073af0 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c5443740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc1ab6e0 con 0x558506074410 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c5443740 10 client.0 _ll_register_callbacks cb 0x7f43ac004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c0b6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55850607ecb0 0x5585060783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c0b6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55850607ecb0 0x5585060783c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35138/0 (socket says 172.21.15.133:35138) 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c0b6c640 1 -- 172.21.15.133:0/2196665911 learned_addr learned my addr 172.21.15.133:0/2196665911 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43bbfff640 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506073af0 0x558506086c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x558506087140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43bbfff640 1 -- 172.21.15.133:0/2196665911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 msgr2=0x558506087140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43bbfff640 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x558506087140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43bbfff640 1 -- 172.21.15.133:0/2196665911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55850607ecb0 msgr2=0x5585060783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43bbfff640 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55850607ecb0 0x5585060783c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43bbfff640 1 -- 172.21.15.133:0/2196665911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558506078a50 con 0x558506073af0 2024-09-11T23:39:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43c0b6c640 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55850607ecb0 0x5585060783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x558506087140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f43bbfff640 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506073af0 0x558506086c00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f43ac030840 tx=0x7f43ac030d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee2044bad8835573 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.059+0000 7f439b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f439b7fe640 1 -- 172.21.15.133:0/2196665911 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac047060 con 0x558506073af0 2024-09-11T23:39:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f43c5443740 1 -- 172.21.15.133:0/2196665911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43ac011950 con 0x558506073af0 2024-09-11T23:39:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f439b7fe640 1 -- 172.21.15.133:0/2196665911 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f43ac00ecc0 con 0x558506073af0 2024-09-11T23:39:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f439b7fe640 1 -- 172.21.15.133:0/2196665911 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43ac036070 con 0x558506073af0 2024-09-11T23:39:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f43c5443740 10 client.25574 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f43c5443740 1 -- 172.21.15.133:0/2196665911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5585060f8f20 con 0x558506073af0 2024-09-11T23:39:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f439b7fe640 1 -- 172.21.15.133:0/2196665911 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43ac011950 con 0x558506073af0 2024-09-11T23:39:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f439b7fe640 1 -- 172.21.15.133:0/2196665911 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f43ac04a020 con 0x558506073af0 2024-09-11T23:39:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f43c5443740 10 client.25574 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f43c5443740 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f43c5443740 20 client.25574 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f43c5443740 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5585060f94d0 0x5585060fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f43c5443740 4 client.25574 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:39:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f43c5443740 1 -- 172.21.15.133:0/2196665911 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558505fdfd70 con 0x5585060f94d0 2024-09-11T23:39:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.060+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5585060f94d0 0x5585060fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.061+0000 7f43bb7fe640 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5585060f94d0 0x5585060fb8c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea035542599b6ee6 server_cookie=bc10e40d8197a280 in_seq=0 out_seq=0 2024-09-11T23:39:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.061+0000 7f439b7fe640 10 client.25574 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f439b7fe640 1 -- 172.21.15.133:0/2196665911 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558505fdfd70 con 0x5585060f94d0 2024-09-11T23:39:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f439b7fe640 10 client.25574 handle_command_reply: tid=0 2024-09-11T23:39:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 1 client.25574 shutdown 2024-09-11T23:39:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 1 -- 172.21.15.133:0/2196665911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506073af0 msgr2=0x558506086c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506073af0 0x558506086c00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f43ac030840 tx=0x7f43ac030d70 comp rx=0 tx=0).stop 2024-09-11T23:39:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 1 -- 172.21.15.133:0/2196665911 shutdown_connections 2024-09-11T23:39:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5585060f94d0 0x5585060fb8c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55850607ecb0 0x5585060783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f439b7fe640 0 client.25574 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558506074410 0x558506087140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 1 --2- 172.21.15.133:0/2196665911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558506073af0 0x558506086c00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 1 -- 172.21.15.133:0/2196665911 >> 172.21.15.133:0/2196665911 conn(0x558506070060 msgr2=0x558506075d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 1 -- 172.21.15.133:0/2196665911 shutdown_connections 2024-09-11T23:39:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 1 -- 172.21.15.133:0/2196665911 wait complete. 2024-09-11T23:39:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.078+0000 7f43c5443740 20 client.25574 trim_cache size 0 max 16384 2024-09-11T23:39:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:39:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:39:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:39:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:39:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 -- 172.21.15.133:0/1402897291 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f43a40a2da0 msgr2=0x7f43a40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f43a40a2da0 0x7f43a40a5260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009b50 tx=0x7f43ac005f90 comp rx=0 tx=0).stop 2024-09-11T23:39:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 -- 172.21.15.133:0/1402897291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43bc102470 msgr2=0x7f43bc1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43bc102470 0x7f43bc1988d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009230 tx=0x7f43b0004270 comp rx=0 tx=0).stop 2024-09-11T23:39:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 -- 172.21.15.133:0/1402897291 shutdown_connections 2024-09-11T23:39:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f43a40a2da0 0x7f43a40a5260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43bc107b70 0x7f43bc19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43bc102470 0x7f43bc1988d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 --2- 172.21.15.133:0/1402897291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43bc101ad0 0x7f43bc198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 -- 172.21.15.133:0/1402897291 >> 172.21.15.133:0/1402897291 conn(0x7f43bc0fd7d0 msgr2=0x7f43bc0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 -- 172.21.15.133:0/1402897291 shutdown_connections 2024-09-11T23:39:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:03.079+0000 7f43c25cf640 1 -- 172.21.15.133:0/1402897291 wait complete. 2024-09-11T23:39:03.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:39:03.347 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:03 smithi133 ceph-mon[31430]: pgmap v71: 97 pgs: 97 active+clean; 10 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 601 MiB/s wr, 1.71k op/s 2024-09-11T23:39:03.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.438+0000 7fc634f4a640 1 -- 172.21.15.133:0/1052482172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc630101440 msgr2=0x7fc63010df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.438+0000 7fc634f4a640 1 --2- 172.21.15.133:0/1052482172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc630101440 0x7fc63010df30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc624009900 tx=0x7fc62402a410 comp rx=0 tx=0).stop 2024-09-11T23:39:03.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.438+0000 7fc634f4a640 1 -- 172.21.15.133:0/1052482172 shutdown_connections 2024-09-11T23:39:03.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.438+0000 7fc634f4a640 1 --2- 172.21.15.133:0/1052482172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc63010e500 0x7fc630110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.438+0000 7fc634f4a640 1 --2- 172.21.15.133:0/1052482172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc630101440 0x7fc63010df30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.438+0000 7fc634f4a640 1 --2- 172.21.15.133:0/1052482172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc630100b20 0x7fc630100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.438+0000 7fc634f4a640 1 -- 172.21.15.133:0/1052482172 >> 172.21.15.133:0/1052482172 conn(0x7fc6300fc7d0 msgr2=0x7fc6300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.439+0000 7fc634f4a640 1 -- 172.21.15.133:0/1052482172 shutdown_connections 2024-09-11T23:39:03.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.439+0000 7fc634f4a640 1 -- 172.21.15.133:0/1052482172 wait complete. 2024-09-11T23:39:03.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.439+0000 7fc634f4a640 1 Processor -- start 2024-09-11T23:39:03.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.439+0000 7fc634f4a640 1 -- start start 2024-09-11T23:39:03.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc634f4a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc630100b20 0x7fc6301a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc634f4a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc630101440 0x7fc6301a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc634f4a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc63010e500 0x7fc63019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc634f4a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc630114270 con 0x7fc630100b20 2024-09-11T23:39:03.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc634f4a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6301140f0 con 0x7fc630101440 2024-09-11T23:39:03.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc634f4a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6301143f0 con 0x7fc63010e500 2024-09-11T23:39:03.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc62e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc630100b20 0x7fc6301a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc62e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc630100b20 0x7fc6301a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35156/0 (socket says 172.21.15.133:35156) 2024-09-11T23:39:03.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc62e59c640 1 -- 172.21.15.133:0/2410625894 learned_addr learned my addr 172.21.15.133:0/2410625894 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:03.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc62dd9b640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc630101440 0x7fc6301a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc62e59c640 1 -- 172.21.15.133:0/2410625894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc63010e500 msgr2=0x7fc63019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc62e59c640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc63010e500 0x7fc63019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc62e59c640 1 -- 172.21.15.133:0/2410625894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc630101440 msgr2=0x7fc6301a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc62e59c640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc630101440 0x7fc6301a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc62e59c640 1 -- 172.21.15.133:0/2410625894 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63019ad10 con 0x7fc630100b20 2024-09-11T23:39:03.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc62e59c640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc630100b20 0x7fc6301a0420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc62000e9f0 tx=0x7fc62000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc61f7fe640 1 -- 172.21.15.133:0/2410625894 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62000cd80 con 0x7fc630100b20 2024-09-11T23:39:03.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc634f4a640 1 -- 172.21.15.133:0/2410625894 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63019b000 con 0x7fc630100b20 2024-09-11T23:39:03.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc61f7fe640 1 -- 172.21.15.133:0/2410625894 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc620005500 con 0x7fc630100b20 2024-09-11T23:39:03.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc61f7fe640 1 -- 172.21.15.133:0/2410625894 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620010690 con 0x7fc630100b20 2024-09-11T23:39:03.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.440+0000 7fc634f4a640 1 -- 172.21.15.133:0/2410625894 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6301a72f0 con 0x7fc630100b20 2024-09-11T23:39:03.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.443+0000 7fc61f7fe640 1 -- 172.21.15.133:0/2410625894 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc6200046c0 con 0x7fc630100b20 2024-09-11T23:39:03.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.443+0000 7fc637323740 1 Processor -- start 2024-09-11T23:39:03.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.443+0000 7fc637323740 1 -- start start 2024-09-11T23:39:03.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.443+0000 7fc61f7fe640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6180a2cf0 0x7fc6180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.443+0000 7fc637323740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f11f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.443+0000 7fc637323740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f1c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.443+0000 7fc637323740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f1f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.443+0000 7fc637323740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d835e77280 con 0x55d835f1cd40 2024-09-11T23:39:03.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc637323740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d835e1eb10 con 0x55d835f11b60 2024-09-11T23:39:03.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc637323740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d835e73240 con 0x55d835f12480 2024-09-11T23:39:03.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62ed9d640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6180a2cf0 0x7fc6180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f11f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f1f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc61f7fe640 1 -- 172.21.15.133:0/2410625894 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc6200cf870 con 0x7fc630100b20 2024-09-11T23:39:03.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f11f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59340/0 (socket says 172.21.15.133:59340) 2024-09-11T23:39:03.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62e59c640 1 -- 172.21.15.133:0/2211048410 learned_addr learned my addr 172.21.15.133:0/2211048410 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:03.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62dd9b640 1 --2- 172.21.15.133:0/2211048410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f1c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62e59c640 1 -- 172.21.15.133:0/2211048410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 msgr2=0x55d835f1c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62e59c640 1 --2- 172.21.15.133:0/2211048410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f1c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62e59c640 1 -- 172.21.15.133:0/2211048410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 msgr2=0x55d835f1f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62e59c640 1 --2- 172.21.15.133:0/2211048410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f1f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62e59c640 1 -- 172.21.15.133:0/2211048410 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d835e2b1f0 con 0x55d835f11b60 2024-09-11T23:39:03.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62ed9d640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6180a2cf0 0x7fc6180a51b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc630069760 tx=0x7fc628009bd0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc62e59c640 1 --2- 172.21.15.133:0/2211048410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f11f40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c6a0 tx=0x7fc62009bbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a77daf4abb0a8407 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc61cff9640 1 -- 172.21.15.133:0/2211048410 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62009c500 con 0x55d835f11b60 2024-09-11T23:39:03.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc61cff9640 1 -- 172.21.15.133:0/2211048410 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62009f070 con 0x55d835f11b60 2024-09-11T23:39:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.444+0000 7fc61cff9640 1 -- 172.21.15.133:0/2211048410 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6200cd2b0 con 0x55d835f11b60 2024-09-11T23:39:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.445+0000 7fc637323740 1 -- 172.21.15.133:0/2211048410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 msgr2=0x55d835f11f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.445+0000 7fc637323740 1 --2- 172.21.15.133:0/2211048410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f11f40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c6a0 tx=0x7fc62009bbd0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.445+0000 7fc637323740 1 -- 172.21.15.133:0/2211048410 shutdown_connections 2024-09-11T23:39:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.445+0000 7fc637323740 1 --2- 172.21.15.133:0/2211048410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f1f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.445+0000 7fc637323740 1 --2- 172.21.15.133:0/2211048410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f1c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.445+0000 7fc637323740 1 --2- 172.21.15.133:0/2211048410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f11f40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.445+0000 7fc637323740 1 -- 172.21.15.133:0/2211048410 >> 172.21.15.133:0/2211048410 conn(0x55d835f0e0f0 msgr2=0x55d835f0e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.445+0000 7fc637323740 1 -- 172.21.15.133:0/2211048410 shutdown_connections 2024-09-11T23:39:03.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.445+0000 7fc637323740 1 -- 172.21.15.133:0/2211048410 wait complete. 2024-09-11T23:39:03.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc637323740 1 -- start start 2024-09-11T23:39:03.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc637323740 1 Processor -- start 2024-09-11T23:39:03.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc637323740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f19cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc637323740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f1a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc637323740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f1a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc637323740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d835f1f910 con 0x55d835f1cd40 2024-09-11T23:39:03.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc637323740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d835e31f10 con 0x55d835f11b60 2024-09-11T23:39:03.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc637323740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d835f1fa90 con 0x55d835f12480 2024-09-11T23:39:03.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc62ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f1a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc62ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f1a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35174/0 (socket says 172.21.15.133:35174) 2024-09-11T23:39:03.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc637323740 10 client.0 _ll_register_callbacks cb 0x55d835dfaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:03.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc62ed9d640 1 -- 172.21.15.133:0/4209175245 learned_addr learned my addr 172.21.15.133:0/4209175245 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:03.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc62e59c640 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f19cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc62dd9b640 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f1a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc62ed9d640 1 -- 172.21.15.133:0/4209175245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 msgr2=0x55d835f1a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc62ed9d640 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f1a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc62ed9d640 1 -- 172.21.15.133:0/4209175245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 msgr2=0x55d835f19cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc62ed9d640 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f19cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc62ed9d640 1 -- 172.21.15.133:0/4209175245 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d835be86a0 con 0x55d835f1cd40 2024-09-11T23:39:03.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.446+0000 7fc62e59c640 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f19cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:03.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc62ed9d640 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f1a730 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc62800fbc0 tx=0x7fc628009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d093de0a7560f4f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:03.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc615ffb640 1 -- 172.21.15.133:0/4209175245 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628024070 con 0x55d835f1cd40 2024-09-11T23:39:03.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc637323740 1 -- 172.21.15.133:0/4209175245 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc62009f070 con 0x55d835f1cd40 2024-09-11T23:39:03.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc615ffb640 1 -- 172.21.15.133:0/4209175245 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc628004b20 con 0x55d835f1cd40 2024-09-11T23:39:03.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc615ffb640 1 -- 172.21.15.133:0/4209175245 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc628004d20 con 0x55d835f1cd40 2024-09-11T23:39:03.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc637323740 10 client.16161 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:03.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc637323740 1 -- 172.21.15.133:0/4209175245 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d835f2b5f0 con 0x55d835f1cd40 2024-09-11T23:39:03.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc615ffb640 1 -- 172.21.15.133:0/4209175245 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62009f070 con 0x55d835f1cd40 2024-09-11T23:39:03.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc615ffb640 1 -- 172.21.15.133:0/4209175245 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc62801b040 con 0x55d835f1cd40 2024-09-11T23:39:03.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc637323740 10 client.16161 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:03.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc637323740 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:03.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc637323740 20 client.16161 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:03.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc637323740 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d835f2ca00 0x55d835f2ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc637323740 4 client.16161 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:03.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.447+0000 7fc637323740 1 -- 172.21.15.133:0/4209175245 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d835e847f0 con 0x55d835f2ca00 2024-09-11T23:39:03.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.448+0000 7fc62e59c640 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d835f2ca00 0x55d835f2ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.448+0000 7fc615ffb640 10 client.16161 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:03.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.448+0000 7fc62e59c640 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d835f2ca00 0x55d835f2ee30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7fd44d1c9712b37 server_cookie=c527d89d61604f1a in_seq=0 out_seq=0 2024-09-11T23:39:03.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.448+0000 7fc615ffb640 1 -- 172.21.15.133:0/4209175245 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d835e847f0 con 0x55d835f2ca00 2024-09-11T23:39:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.448+0000 7fc615ffb640 10 client.16161 handle_command_reply: tid=0 2024-09-11T23:39:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.448+0000 7fc637323740 1 client.16161 shutdown 2024-09-11T23:39:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc637323740 1 -- 172.21.15.133:0/4209175245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 msgr2=0x55d835f1a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc637323740 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f1a730 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc62800fbc0 tx=0x7fc628009200 comp rx=0 tx=0).stop 2024-09-11T23:39:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc637323740 1 -- 172.21.15.133:0/4209175245 shutdown_connections 2024-09-11T23:39:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc637323740 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d835f2ca00 0x55d835f2ee30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc637323740 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f1a730 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc637323740 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f1a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc615ffb640 0 client.16161 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc637323740 1 --2- 172.21.15.133:0/4209175245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f19cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc637323740 1 -- 172.21.15.133:0/4209175245 >> 172.21.15.133:0/4209175245 conn(0x55d835f0e0f0 msgr2=0x55d835f13e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc637323740 1 -- 172.21.15.133:0/4209175245 shutdown_connections 2024-09-11T23:39:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc637323740 1 -- 172.21.15.133:0/4209175245 wait complete. 2024-09-11T23:39:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.449+0000 7fc637323740 20 client.16161 trim_cache size 0 max 16384 2024-09-11T23:39:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.466+0000 7fc637323740 1 Processor -- start 2024-09-11T23:39:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.466+0000 7fc637323740 1 -- start start 2024-09-11T23:39:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc637323740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc637323740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f34750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc637323740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f34c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc637323740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc6301af970 con 0x55d835f1cd40 2024-09-11T23:39:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc637323740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6301b14b0 con 0x55d835f11b60 2024-09-11T23:39:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc637323740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6301b0040 con 0x55d835f12480 2024-09-11T23:39:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f34c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f34c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35180/0 (socket says 172.21.15.133:35180) 2024-09-11T23:39:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f145c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59364/0 (socket says 172.21.15.133:59364) 2024-09-11T23:39:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62ed9d640 1 -- 172.21.15.133:0/2423951434 learned_addr learned my addr 172.21.15.133:0/2423951434 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62dd9b640 1 --2- 172.21.15.133:0/2423951434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f34750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62ed9d640 1 -- 172.21.15.133:0/2423951434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 msgr2=0x55d835f34750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62ed9d640 1 --2- 172.21.15.133:0/2423951434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f34750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62ed9d640 1 -- 172.21.15.133:0/2423951434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 msgr2=0x55d835f145c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62ed9d640 1 --2- 172.21.15.133:0/2423951434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f145c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62ed9d640 1 -- 172.21.15.133:0/2423951434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d835f2f4b0 con 0x55d835f1cd40 2024-09-11T23:39:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62e59c640 1 --2- 172.21.15.133:0/2423951434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc62ed9d640 1 --2- 172.21.15.133:0/2423951434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f34c90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc6280097d0 tx=0x7fc6280095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a73318b37e199e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.467+0000 7fc617fff640 1 -- 172.21.15.133:0/2423951434 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628035070 con 0x55d835f1cd40 2024-09-11T23:39:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.468+0000 7fc617fff640 1 -- 172.21.15.133:0/2423951434 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc628003e50 con 0x55d835f1cd40 2024-09-11T23:39:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.468+0000 7fc617fff640 1 -- 172.21.15.133:0/2423951434 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628024040 con 0x55d835f1cd40 2024-09-11T23:39:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.468+0000 7fc637323740 1 -- 172.21.15.133:0/2423951434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 msgr2=0x55d835f34c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.468+0000 7fc637323740 1 --2- 172.21.15.133:0/2423951434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f34c90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc6280097d0 tx=0x7fc6280095d0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.468+0000 7fc637323740 1 -- 172.21.15.133:0/2423951434 shutdown_connections 2024-09-11T23:39:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.468+0000 7fc637323740 1 --2- 172.21.15.133:0/2423951434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f34c90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.468+0000 7fc637323740 1 --2- 172.21.15.133:0/2423951434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f34750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.468+0000 7fc637323740 1 --2- 172.21.15.133:0/2423951434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.468+0000 7fc637323740 1 -- 172.21.15.133:0/2423951434 >> 172.21.15.133:0/2423951434 conn(0x55d835f0e0f0 msgr2=0x55d835f1bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.468+0000 7fc637323740 1 -- 172.21.15.133:0/2423951434 shutdown_connections 2024-09-11T23:39:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.468+0000 7fc637323740 1 -- 172.21.15.133:0/2423951434 wait complete. 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc637323740 1 -- start start 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc637323740 1 Processor -- start 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc637323740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f24e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc637323740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc637323740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f21c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc637323740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc6301b0de0 con 0x55d835f1cd40 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc637323740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6301b0710 con 0x55d835f11b60 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc637323740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6301ad950 con 0x55d835f12480 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc637323740 10 client.0 _ll_register_callbacks cb 0x7fc628009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc62e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f24e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc62ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f21c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc62ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f21c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35196/0 (socket says 172.21.15.133:35196) 2024-09-11T23:39:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc62e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f24e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59378/0 (socket says 172.21.15.133:59378) 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.469+0000 7fc62ed9d640 1 -- 172.21.15.133:0/2160082630 learned_addr learned my addr 172.21.15.133:0/2160082630 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc62dd9b640 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc62e59c640 1 -- 172.21.15.133:0/2160082630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 msgr2=0x55d835f253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc62e59c640 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f253d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc62e59c640 1 -- 172.21.15.133:0/2160082630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 msgr2=0x55d835f21c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc62e59c640 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f21c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc62e59c640 1 -- 172.21.15.133:0/2160082630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d835f22370 con 0x55d835f11b60 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc62ed9d640 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f21c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc62e59c640 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f24e90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c6a0 tx=0x7fc620004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8b45bcb2f1e75fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc615ffb640 1 -- 172.21.15.133:0/2160082630 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62009c500 con 0x55d835f11b60 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc615ffb640 1 -- 172.21.15.133:0/2160082630 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc6200a45e0 con 0x55d835f11b60 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc637323740 1 -- 172.21.15.133:0/2160082630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc628003e50 con 0x55d835f11b60 2024-09-11T23:39:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc615ffb640 1 -- 172.21.15.133:0/2160082630 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6200a47e0 con 0x55d835f11b60 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc637323740 10 client.25592 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.470+0000 7fc637323740 1 -- 172.21.15.133:0/2160082630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d835f98130 con 0x55d835f11b60 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.471+0000 7fc615ffb640 1 -- 172.21.15.133:0/2160082630 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc628003e50 con 0x55d835f11b60 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.471+0000 7fc615ffb640 1 -- 172.21.15.133:0/2160082630 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc62009acc0 con 0x55d835f11b60 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.471+0000 7fc637323740 10 client.25592 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.471+0000 7fc637323740 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.471+0000 7fc637323740 20 client.25592 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.471+0000 7fc637323740 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d835f98470 0x55d835f9a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.471+0000 7fc637323740 4 client.25592 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.471+0000 7fc637323740 1 -- 172.21.15.133:0/2160082630 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d835e7ed00 con 0x55d835f98470 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.471+0000 7fc62dd9b640 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d835f98470 0x55d835f9a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.471+0000 7fc615ffb640 10 client.25592 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.471+0000 7fc62dd9b640 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d835f98470 0x55d835f9a860 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f237c34c8e32a2 server_cookie=d51a44f98ae707b3 in_seq=0 out_seq=0 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc615ffb640 1 -- 172.21.15.133:0/2160082630 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d835e7ed00 con 0x55d835f98470 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc615ffb640 10 client.25592 handle_command_reply: tid=0 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc637323740 1 client.25592 shutdown 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc637323740 1 -- 172.21.15.133:0/2160082630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 msgr2=0x55d835f24e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc637323740 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f24e90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c6a0 tx=0x7fc620004a40 comp rx=0 tx=0).stop 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc637323740 1 -- 172.21.15.133:0/2160082630 shutdown_connections 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc637323740 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d835f98470 0x55d835f9a860 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc637323740 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d835f1cd40 0x55d835f21c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc615ffb640 0 client.25592 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc637323740 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d835f12480 0x55d835f253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc637323740 1 --2- 172.21.15.133:0/2160082630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d835f11b60 0x55d835f24e90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc637323740 1 -- 172.21.15.133:0/2160082630 >> 172.21.15.133:0/2160082630 conn(0x55d835f0e0f0 msgr2=0x55d835f19c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc637323740 1 -- 172.21.15.133:0/2160082630 shutdown_connections 2024-09-11T23:39:03.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.473+0000 7fc637323740 1 -- 172.21.15.133:0/2160082630 wait complete. 2024-09-11T23:39:03.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc637323740 20 client.25592 trim_cache size 0 max 16384 2024-09-11T23:39:03.498 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:39:03.498 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:39:03.498 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "c7ea0928-ddf8-4743-a54b-9144daf1880d", 2024-09-11T23:39:03.498 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:39:03.498 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:39:03.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 -- 172.21.15.133:0/2410625894 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6180a2cf0 msgr2=0x7fc6180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6180a2cf0 0x7fc6180a51b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc630069760 tx=0x7fc628009bd0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 -- 172.21.15.133:0/2410625894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc630100b20 msgr2=0x7fc6301a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc630100b20 0x7fc6301a0420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc62000e9f0 tx=0x7fc62000edf0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 -- 172.21.15.133:0/2410625894 shutdown_connections 2024-09-11T23:39:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6180a2cf0 0x7fc6180a51b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc63010e500 0x7fc63019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc630101440 0x7fc6301a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 --2- 172.21.15.133:0/2410625894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc630100b20 0x7fc6301a0420 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 -- 172.21.15.133:0/2410625894 >> 172.21.15.133:0/2410625894 conn(0x7fc6300fc7d0 msgr2=0x7fc63010f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 -- 172.21.15.133:0/2410625894 shutdown_connections 2024-09-11T23:39:03.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:03.474+0000 7fc634f4a640 1 -- 172.21.15.133:0/2410625894 wait complete. 2024-09-11T23:39:03.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:03 smithi178 ceph-mon[35882]: pgmap v71: 97 pgs: 97 active+clean; 10 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 601 MiB/s wr, 1.71k op/s 2024-09-11T23:39:03.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:03 smithi138 ceph-mon[35142]: pgmap v71: 97 pgs: 97 active+clean; 10 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 601 MiB/s wr, 1.71k op/s 2024-09-11T23:39:04.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:39:04.285 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:04.413 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:04 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:39:04.413 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:04 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:04.413 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:04 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:39:04.413 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:04 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:04.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:04 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:39:04.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:04 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:04.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:04 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:39:04.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:04 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:04.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:04 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:39:04.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:04 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:04.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:04 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:39:04.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:04 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.781+0000 7fe32654c640 1 -- 172.21.15.133:0/339305888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3201082a0 msgr2=0x7fe32010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.781+0000 7fe32654c640 1 --2- 172.21.15.133:0/339305888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3201082a0 0x7fe32010ec40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe318009cd0 tx=0x7fe31802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:39:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.781+0000 7fe32654c640 1 -- 172.21.15.133:0/339305888 shutdown_connections 2024-09-11T23:39:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.781+0000 7fe32654c640 1 --2- 172.21.15.133:0/339305888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3201082a0 0x7fe32010ec40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.781+0000 7fe32654c640 1 --2- 172.21.15.133:0/339305888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3201036e0 0x7fe320107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.781+0000 7fe32654c640 1 --2- 172.21.15.133:0/339305888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe320102d30 0x7fe320103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.781+0000 7fe32654c640 1 -- 172.21.15.133:0/339305888 >> 172.21.15.133:0/339305888 conn(0x7fe3200fc7d0 msgr2=0x7fe3200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.781+0000 7fe32654c640 1 -- 172.21.15.133:0/339305888 shutdown_connections 2024-09-11T23:39:04.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.781+0000 7fe32654c640 1 -- 172.21.15.133:0/339305888 wait complete. 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.782+0000 7fe32654c640 1 Processor -- start 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.782+0000 7fe32654c640 1 -- start start 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.782+0000 7fe32654c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe320102d30 0x7fe320100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.782+0000 7fe32654c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3201036e0 0x7fe320100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.782+0000 7fe32654c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3201082a0 0x7fe320101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.782+0000 7fe32654c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201120a0 con 0x7fe3201036e0 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.782+0000 7fe32654c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe320111f20 con 0x7fe3201082a0 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.782+0000 7fe32654c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe320112220 con 0x7fe320102d30 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.782+0000 7fe31f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3201036e0 0x7fe320100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3201036e0 0x7fe320100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35212/0 (socket says 172.21.15.133:35212) 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31f7fe640 1 -- 172.21.15.133:0/673668233 learned_addr learned my addr 172.21.15.133:0/673668233 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe324ae9640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3201082a0 0x7fe320101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31ffff640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe320102d30 0x7fe320100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31f7fe640 1 -- 172.21.15.133:0/673668233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe320102d30 msgr2=0x7fe320100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31f7fe640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe320102d30 0x7fe320100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31f7fe640 1 -- 172.21.15.133:0/673668233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3201082a0 msgr2=0x7fe320101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31f7fe640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3201082a0 0x7fe320101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31f7fe640 1 -- 172.21.15.133:0/673668233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3201a6a20 con 0x7fe3201036e0 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe324ae9640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3201082a0 0x7fe320101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31f7fe640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3201036e0 0x7fe320100ed0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe31400d750 tx=0x7fe31400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31d7fa640 1 -- 172.21.15.133:0/673668233 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314017070 con 0x7fe3201036e0 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe32654c640 1 -- 172.21.15.133:0/673668233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3201a6d10 con 0x7fe3201036e0 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31d7fa640 1 -- 172.21.15.133:0/673668233 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe314005500 con 0x7fe3201036e0 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe31d7fa640 1 -- 172.21.15.133:0/673668233 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314004e00 con 0x7fe3201036e0 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.783+0000 7fe32654c640 1 -- 172.21.15.133:0/673668233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3201a71f0 con 0x7fe3201036e0 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.784+0000 7fe32654c640 1 -- 172.21.15.133:0/673668233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe320103110 con 0x7fe3201036e0 2024-09-11T23:39:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.786+0000 7fe31d7fa640 1 -- 172.21.15.133:0/673668233 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe314004fa0 con 0x7fe3201036e0 2024-09-11T23:39:04.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.786+0000 7fe31d7fa640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe3080a2cf0 0x7fe3080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:04.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.786+0000 7fe31ffff640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe3080a2cf0 0x7fe3080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:04.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.786+0000 7fe31d7fa640 1 -- 172.21.15.133:0/673668233 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe3140cf190 con 0x7fe3201036e0 2024-09-11T23:39:04.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.787+0000 7fe31ffff640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe3080a2cf0 0x7fe3080a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe310005fd0 tx=0x7fe310005850 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:04.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:04.790+0000 7fe31d7fa640 1 -- 172.21.15.133:0/673668233 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe31409be20 con 0x7fe3201036e0 2024-09-11T23:39:04.847 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:05.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.008+0000 7fe32654c640 1 -- 172.21.15.133:0/673668233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe32010a2b0 con 0x7fe3201036e0 2024-09-11T23:39:05.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.009+0000 7fe31d7fa640 1 -- 172.21.15.133:0/673668233 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7fe31409b720 con 0x7fe3201036e0 2024-09-11T23:39:05.009 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:39:05.009 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:39:05.009 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:39:05.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 -- 172.21.15.133:0/673668233 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe3080a2cf0 msgr2=0x7fe3080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:05.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe3080a2cf0 0x7fe3080a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe310005fd0 tx=0x7fe310005850 comp rx=0 tx=0).stop 2024-09-11T23:39:05.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 -- 172.21.15.133:0/673668233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3201036e0 msgr2=0x7fe320100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:05.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3201036e0 0x7fe320100ed0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe31400d750 tx=0x7fe31400dc10 comp rx=0 tx=0).stop 2024-09-11T23:39:05.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 -- 172.21.15.133:0/673668233 shutdown_connections 2024-09-11T23:39:05.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe3080a2cf0 0x7fe3080a51b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3201082a0 0x7fe320101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3201036e0 0x7fe320100ed0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 --2- 172.21.15.133:0/673668233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe320102d30 0x7fe320100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:05.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 -- 172.21.15.133:0/673668233 >> 172.21.15.133:0/673668233 conn(0x7fe3200fc7d0 msgr2=0x7fe320105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:05.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 -- 172.21.15.133:0/673668233 shutdown_connections 2024-09-11T23:39:05.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:05.011+0000 7fe32654c640 1 -- 172.21.15.133:0/673668233 wait complete. 2024-09-11T23:39:05.533 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:05 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:39:05.533 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:05 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/673668233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:05.533 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:05 smithi133 ceph-mon[31430]: pgmap v72: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 831 KiB/s rd, 426 MiB/s wr, 1.18k op/s 2024-09-11T23:39:05.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:05 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:39:05.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:05 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/673668233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:05.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:05 smithi178 ceph-mon[35882]: pgmap v72: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 831 KiB/s rd, 426 MiB/s wr, 1.18k op/s 2024-09-11T23:39:05.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:05 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:39:05.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:05 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/673668233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:05.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:05 smithi138 ceph-mon[35142]: pgmap v72: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 831 KiB/s rd, 426 MiB/s wr, 1.18k op/s 2024-09-11T23:39:05.954 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id 1fb2b5b2 --release --await' 2024-09-11T23:39:05.954 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id 1fb2b5b2 --release --await 2024-09-11T23:39:06.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.161+0000 7f9454ab8640 1 -- 172.21.15.133:0/3823618763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9450102570 msgr2=0x7f94501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:06.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.161+0000 7f9454ab8640 1 --2- 172.21.15.133:0/3823618763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9450102570 0x7f94501029b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9444009900 tx=0x7f944402a480 comp rx=0 tx=0).stop 2024-09-11T23:39:06.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.161+0000 7f9454ab8640 1 -- 172.21.15.133:0/3823618763 shutdown_connections 2024-09-11T23:39:06.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.161+0000 7f9454ab8640 1 --2- 172.21.15.133:0/3823618763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9450102ef0 0x7f945010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.161+0000 7f9454ab8640 1 --2- 172.21.15.133:0/3823618763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9450102570 0x7f94501029b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.161+0000 7f9454ab8640 1 --2- 172.21.15.133:0/3823618763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9450100e20 0x7f9450101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.161+0000 7f9454ab8640 1 -- 172.21.15.133:0/3823618763 >> 172.21.15.133:0/3823618763 conn(0x7f94500fc7d0 msgr2=0x7f94500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.161+0000 7f9454ab8640 1 -- 172.21.15.133:0/3823618763 shutdown_connections 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.161+0000 7f9454ab8640 1 -- 172.21.15.133:0/3823618763 wait complete. 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f9454ab8640 1 Processor -- start 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f9454ab8640 1 -- start start 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f9454ab8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9450100e20 0x7f94500fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f9454ab8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9450102570 0x7f94500fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f9454ab8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9450102ef0 0x7f94500fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f9454ab8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9450110f80 con 0x7f9450102570 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f9454ab8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9450110e00 con 0x7f9450102ef0 2024-09-11T23:39:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f9454ab8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9450111100 con 0x7f9450100e20 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f944dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9450102570 0x7f94500fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f944dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9450102570 0x7f94500fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35222/0 (socket says 172.21.15.133:35222) 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f944dd9b640 1 -- 172.21.15.133:0/2840577166 learned_addr learned my addr 172.21.15.133:0/2840577166 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f944e59c640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9450100e20 0x7f94500fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.162+0000 7f944ed9d640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9450102ef0 0x7f94500fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.163+0000 7f944dd9b640 1 -- 172.21.15.133:0/2840577166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9450100e20 msgr2=0x7f94500fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.163+0000 7f944dd9b640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9450100e20 0x7f94500fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.163+0000 7f944dd9b640 1 -- 172.21.15.133:0/2840577166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9450102ef0 msgr2=0x7f94500fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.163+0000 7f944dd9b640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9450102ef0 0x7f94500fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.163+0000 7f944dd9b640 1 -- 172.21.15.133:0/2840577166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94501a6930 con 0x7f9450102570 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.163+0000 7f944dd9b640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9450102570 0x7f94500fd1c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9444006010 tx=0x7f9444003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.163+0000 7f943f7fe640 1 -- 172.21.15.133:0/2840577166 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944403f070 con 0x7f9450102570 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.163+0000 7f943f7fe640 1 -- 172.21.15.133:0/2840577166 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9444031e30 con 0x7f9450102570 2024-09-11T23:39:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.163+0000 7f9454ab8640 1 -- 172.21.15.133:0/2840577166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94501a6bc0 con 0x7f9450102570 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.163+0000 7f943f7fe640 1 -- 172.21.15.133:0/2840577166 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444035030 con 0x7f9450102570 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.163+0000 7f9454ab8640 1 -- 172.21.15.133:0/2840577166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94501a70a0 con 0x7f9450102570 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.165+0000 7f943f7fe640 1 -- 172.21.15.133:0/2840577166 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9444042070 con 0x7f9450102570 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f9456e91740 1 Processor -- start 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f9456e91740 1 -- start start 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f9456e91740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 0x56003e58c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f9456e91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 0x56003e597140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f9456e91740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e597680 0x56003e599a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f9456e91740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56003e4f22e0 con 0x56003e58c4a0 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f9456e91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56003e499ce0 con 0x56003e58cdc0 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f9456e91740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56003e4ee2a0 con 0x56003e597680 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f943f7fe640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94380a2cf0 0x7f94380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f944e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 0x56003e597140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f944ed9d640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94380a2cf0 0x7f94380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f944e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 0x56003e597140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59438/0 (socket says 172.21.15.133:59438) 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f944e59c640 1 -- 172.21.15.133:0/2467677948 learned_addr learned my addr 172.21.15.133:0/2467677948 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f944ed9d640 1 --2- 172.21.15.133:0/2467677948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 0x56003e58c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f943f7fe640 1 -- 172.21.15.133:0/2840577166 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f94440efbf0 con 0x7f9450102570 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.166+0000 7f944dd9b640 1 --2- 172.21.15.133:0/2467677948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e597680 0x56003e599a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.167+0000 7f944e59c640 1 -- 172.21.15.133:0/2467677948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e597680 msgr2=0x56003e599a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.167+0000 7f944e59c640 1 --2- 172.21.15.133:0/2467677948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e597680 0x56003e599a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.167+0000 7f944e59c640 1 -- 172.21.15.133:0/2467677948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 msgr2=0x56003e58c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.167+0000 7f944e59c640 1 --2- 172.21.15.133:0/2467677948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 0x56003e58c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.167+0000 7f944e59c640 1 -- 172.21.15.133:0/2467677948 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003e47e440 con 0x56003e58cdc0 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.167+0000 7f944ed9d640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94380a2cf0 0x7f94380a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f944800afd0 tx=0x7f944800ad60 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.167+0000 7f944e59c640 1 --2- 172.21.15.133:0/2467677948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 0x56003e597140 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f944000ad00 tx=0x7f944000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44e85feed44413b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.167+0000 7f943cff9640 1 -- 172.21.15.133:0/2467677948 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944001d070 con 0x56003e58cdc0 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.167+0000 7f943cff9640 1 -- 172.21.15.133:0/2467677948 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9440018b20 con 0x56003e58cdc0 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.167+0000 7f943cff9640 1 -- 172.21.15.133:0/2467677948 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440010a90 con 0x56003e58cdc0 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.168+0000 7f9456e91740 1 -- 172.21.15.133:0/2467677948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 msgr2=0x56003e597140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.168+0000 7f9456e91740 1 --2- 172.21.15.133:0/2467677948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 0x56003e597140 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f944000ad00 tx=0x7f944000f790 comp rx=0 tx=0).stop 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.168+0000 7f9456e91740 1 -- 172.21.15.133:0/2467677948 shutdown_connections 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.168+0000 7f9456e91740 1 --2- 172.21.15.133:0/2467677948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e597680 0x56003e599a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.168+0000 7f9456e91740 1 --2- 172.21.15.133:0/2467677948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 0x56003e597140 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.168+0000 7f9456e91740 1 --2- 172.21.15.133:0/2467677948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 0x56003e58c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.168+0000 7f9456e91740 1 -- 172.21.15.133:0/2467677948 >> 172.21.15.133:0/2467677948 conn(0x56003e588a30 msgr2=0x56003e588e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.168+0000 7f9456e91740 1 -- 172.21.15.133:0/2467677948 shutdown_connections 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.168+0000 7f9456e91740 1 -- 172.21.15.133:0/2467677948 wait complete. 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.168+0000 7f9456e91740 1 -- start start 2024-09-11T23:39:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.168+0000 7f9456e91740 1 Processor -- start 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f9456e91740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e58c4a0 0x56003e596fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f9456e91740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 0x56003e594260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f9456e91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e597680 0x56003e5947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f9456e91740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56003e59a250 con 0x56003e58cdc0 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f9456e91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56003e4ad050 con 0x56003e597680 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f9456e91740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56003e59a3d0 con 0x56003e58c4a0 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f9456e91740 10 client.0 _ll_register_callbacks cb 0x56003e4ed160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f944dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 0x56003e594260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f944dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 0x56003e594260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35240/0 (socket says 172.21.15.133:35240) 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f944dd9b640 1 -- 172.21.15.133:0/1466476208 learned_addr learned my addr 172.21.15.133:0/1466476208 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f944ed9d640 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e597680 0x56003e5947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f944dd9b640 1 -- 172.21.15.133:0/1466476208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e58c4a0 msgr2=0x56003e596fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f944dd9b640 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e58c4a0 0x56003e596fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f944dd9b640 1 -- 172.21.15.133:0/1466476208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e597680 msgr2=0x56003e5947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f944dd9b640 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e597680 0x56003e5947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f944dd9b640 1 -- 172.21.15.133:0/1466476208 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56003e2633c0 con 0x56003e58cdc0 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f944ed9d640 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e597680 0x56003e5947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f944dd9b640 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 0x56003e594260 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f94440b8b50 tx=0x7f94440bc010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c04dec43767611b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.169+0000 7f9435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9435ffb640 1 -- 172.21.15.133:0/1466476208 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444049070 con 0x56003e58cdc0 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9456e91740 1 -- 172.21.15.133:0/1466476208 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9440018b20 con 0x56003e58cdc0 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9435ffb640 1 -- 172.21.15.133:0/1466476208 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f94440dc0b0 con 0x56003e58cdc0 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9435ffb640 1 -- 172.21.15.133:0/1466476208 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f94440bc880 con 0x56003e58cdc0 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9456e91740 10 client.16185 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9456e91740 1 -- 172.21.15.133:0/1466476208 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56003e5a5b70 con 0x56003e58cdc0 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9435ffb640 1 -- 172.21.15.133:0/1466476208 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9440018b20 con 0x56003e58cdc0 2024-09-11T23:39:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9435ffb640 1 -- 172.21.15.133:0/1466476208 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f94440c0120 con 0x56003e58cdc0 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9456e91740 10 client.16185 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9456e91740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9456e91740 20 client.16185 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9456e91740 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56003e5a6f60 0x56003e5a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9456e91740 4 client.16185 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f9456e91740 1 -- 172.21.15.133:0/1466476208 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56003e4ff850 con 0x56003e5a6f60 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.170+0000 7f944e59c640 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56003e5a6f60 0x56003e5a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.171+0000 7f9435ffb640 10 client.16185 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.171+0000 7f944e59c640 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56003e5a6f60 0x56003e5a93f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26b92173eda43c8 server_cookie=2c9278cc562841a2 in_seq=0 out_seq=0 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9435ffb640 1 -- 172.21.15.133:0/1466476208 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56003e4ff850 con 0x56003e5a6f60 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9435ffb640 10 client.16185 handle_command_reply: tid=0 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 1 client.16185 shutdown 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 1 -- 172.21.15.133:0/1466476208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 msgr2=0x56003e594260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 0x56003e594260 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f94440b8b50 tx=0x7f94440bc010 comp rx=0 tx=0).stop 2024-09-11T23:39:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 1 -- 172.21.15.133:0/1466476208 shutdown_connections 2024-09-11T23:39:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56003e5a6f60 0x56003e5a93f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e597680 0x56003e5947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9435ffb640 0 client.16185 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 0x56003e594260 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 1 --2- 172.21.15.133:0/1466476208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e58c4a0 0x56003e596fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 1 -- 172.21.15.133:0/1466476208 >> 172.21.15.133:0/1466476208 conn(0x56003e588a30 msgr2=0x56003e58e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 1 -- 172.21.15.133:0/1466476208 shutdown_connections 2024-09-11T23:39:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 1 -- 172.21.15.133:0/1466476208 wait complete. 2024-09-11T23:39:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.172+0000 7f9456e91740 20 client.16185 trim_cache size 0 max 16384 2024-09-11T23:39:06.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.190+0000 7f9456e91740 1 Processor -- start 2024-09-11T23:39:06.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.190+0000 7f9456e91740 1 -- start start 2024-09-11T23:39:06.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.190+0000 7f9456e91740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e588a30 0x56003e587a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.190+0000 7f9456e91740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 0x56003e5aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.190+0000 7f9456e91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 0x56003e5af210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.190+0000 7f9456e91740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94501af840 con 0x56003e58c4a0 2024-09-11T23:39:06.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.190+0000 7f9456e91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94501b1380 con 0x56003e58cdc0 2024-09-11T23:39:06.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.190+0000 7f9456e91740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94501aff10 con 0x56003e588a30 2024-09-11T23:39:06.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 0x56003e5aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 0x56003e5aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35242/0 (socket says 172.21.15.133:35242) 2024-09-11T23:39:06.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944dd9b640 1 -- 172.21.15.133:0/2321724767 learned_addr learned my addr 172.21.15.133:0/2321724767 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:06.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944e59c640 1 --2- 172.21.15.133:0/2321724767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e588a30 0x56003e587a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944ed9d640 1 --2- 172.21.15.133:0/2321724767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 0x56003e5af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944dd9b640 1 -- 172.21.15.133:0/2321724767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e588a30 msgr2=0x56003e587a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944dd9b640 1 --2- 172.21.15.133:0/2321724767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e588a30 0x56003e587a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944dd9b640 1 -- 172.21.15.133:0/2321724767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 msgr2=0x56003e5af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944dd9b640 1 --2- 172.21.15.133:0/2321724767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 0x56003e5af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944dd9b640 1 -- 172.21.15.133:0/2321724767 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003e58cbf0 con 0x56003e58c4a0 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944e59c640 1 --2- 172.21.15.133:0/2321724767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e588a30 0x56003e587a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f944dd9b640 1 --2- 172.21.15.133:0/2321724767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 0x56003e5aecd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f94440bbe80 tx=0x7f9444005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c33815c423df93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f9437fff640 1 -- 172.21.15.133:0/2321724767 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444049070 con 0x56003e58c4a0 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f9437fff640 1 -- 172.21.15.133:0/2321724767 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944403be50 con 0x56003e58c4a0 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.191+0000 7f9437fff640 1 -- 172.21.15.133:0/2321724767 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94440b8980 con 0x56003e58c4a0 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.192+0000 7f9456e91740 1 -- 172.21.15.133:0/2321724767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 msgr2=0x56003e5aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.192+0000 7f9456e91740 1 --2- 172.21.15.133:0/2321724767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 0x56003e5aecd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f94440bbe80 tx=0x7f9444005c00 comp rx=0 tx=0).stop 2024-09-11T23:39:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.192+0000 7f9456e91740 1 -- 172.21.15.133:0/2321724767 shutdown_connections 2024-09-11T23:39:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.192+0000 7f9456e91740 1 --2- 172.21.15.133:0/2321724767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e58cdc0 0x56003e5af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.192+0000 7f9456e91740 1 --2- 172.21.15.133:0/2321724767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58c4a0 0x56003e5aecd0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.192+0000 7f9456e91740 1 --2- 172.21.15.133:0/2321724767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e588a30 0x56003e587a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.192+0000 7f9456e91740 1 -- 172.21.15.133:0/2321724767 >> 172.21.15.133:0/2321724767 conn(0x56003e594900 msgr2=0x56003e587350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.192+0000 7f9456e91740 1 -- 172.21.15.133:0/2321724767 shutdown_connections 2024-09-11T23:39:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.192+0000 7f9456e91740 1 -- 172.21.15.133:0/2321724767 wait complete. 2024-09-11T23:39:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.193+0000 7f9456e91740 1 -- start start 2024-09-11T23:39:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.193+0000 7f9456e91740 1 Processor -- start 2024-09-11T23:39:06.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.193+0000 7f9456e91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e588a30 0x56003e59d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.193+0000 7f9456e91740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e58c4a0 0x56003e59dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.193+0000 7f9456e91740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 0x56003e59dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.193+0000 7f9456e91740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94501b0cb0 con 0x56003e58cdc0 2024-09-11T23:39:06.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.193+0000 7f9456e91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94501b05e0 con 0x56003e588a30 2024-09-11T23:39:06.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.193+0000 7f9456e91740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94501ad860 con 0x56003e58c4a0 2024-09-11T23:39:06.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f9456e91740 10 client.0 _ll_register_callbacks cb 0x7f9444005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:06.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f944ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 0x56003e59dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f944e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e588a30 0x56003e59d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f944ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 0x56003e59dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35246/0 (socket says 172.21.15.133:35246) 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f944ed9d640 1 -- 172.21.15.133:0/179511413 learned_addr learned my addr 172.21.15.133:0/179511413 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f944dd9b640 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e58c4a0 0x56003e59dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f944e59c640 1 -- 172.21.15.133:0/179511413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e58c4a0 msgr2=0x56003e59dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f944e59c640 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e58c4a0 0x56003e59dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f944e59c640 1 -- 172.21.15.133:0/179511413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 msgr2=0x56003e59dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f944e59c640 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 0x56003e59dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f944e59c640 1 -- 172.21.15.133:0/179511413 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56003e610eb0 con 0x56003e588a30 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f944e59c640 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e588a30 0x56003e59d570 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9440005910 tx=0x7f9440005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0f4d39baf47402 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f9435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.194+0000 7f9435ffb640 1 -- 172.21.15.133:0/179511413 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944001d040 con 0x56003e588a30 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9435ffb640 1 -- 172.21.15.133:0/179511413 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f9440005040 con 0x56003e588a30 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9456e91740 1 -- 172.21.15.133:0/179511413 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944403be50 con 0x56003e588a30 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9435ffb640 1 -- 172.21.15.133:0/179511413 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9440005240 con 0x56003e588a30 2024-09-11T23:39:06.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9456e91740 10 client.25613 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9456e91740 1 -- 172.21.15.133:0/179511413 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56003e612160 con 0x56003e588a30 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9435ffb640 1 -- 172.21.15.133:0/179511413 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944403be50 con 0x56003e588a30 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9435ffb640 1 -- 172.21.15.133:0/179511413 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9440035020 con 0x56003e588a30 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9456e91740 10 client.25613 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9456e91740 10 client.25613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9456e91740 20 client.25613 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9456e91740 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56003e597680 0x56003e614510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9456e91740 4 client.25613 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.195+0000 7f9456e91740 1 -- 172.21.15.133:0/179511413 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56003e4f9d60 con 0x56003e597680 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f944dd9b640 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56003e597680 0x56003e614510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9435ffb640 10 client.25613 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f944dd9b640 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56003e597680 0x56003e614510 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1d5fb7747bf15a server_cookie=89b4377f8680a226 in_seq=0 out_seq=0 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9435ffb640 1 -- 172.21.15.133:0/179511413 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56003e4f9d60 con 0x56003e597680 2024-09-11T23:39:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9435ffb640 10 client.25613 handle_command_reply: tid=0 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9456e91740 1 client.25613 shutdown 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9456e91740 1 -- 172.21.15.133:0/179511413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e588a30 msgr2=0x56003e59d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9456e91740 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e588a30 0x56003e59d570 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9440005910 tx=0x7f9440005ed0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9456e91740 1 -- 172.21.15.133:0/179511413 shutdown_connections 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9456e91740 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56003e597680 0x56003e614510 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9456e91740 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56003e58cdc0 0x56003e59dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9456e91740 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56003e58c4a0 0x56003e59dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9435ffb640 0 client.25613 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9456e91740 1 --2- 172.21.15.133:0/179511413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56003e588a30 0x56003e59d570 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9456e91740 1 -- 172.21.15.133:0/179511413 >> 172.21.15.133:0/179511413 conn(0x56003e594900 msgr2=0x56003e587b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9456e91740 1 -- 172.21.15.133:0/179511413 shutdown_connections 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.196+0000 7f9456e91740 1 -- 172.21.15.133:0/179511413 wait complete. 2024-09-11T23:39:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9456e91740 20 client.25613 trim_cache size 0 max 16384 2024-09-11T23:39:06.201 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:39:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:39:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:39:06.201 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:39:06.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 -- 172.21.15.133:0/2840577166 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94380a2cf0 msgr2=0x7f94380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94380a2cf0 0x7f94380a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f944800afd0 tx=0x7f944800ad60 comp rx=0 tx=0).stop 2024-09-11T23:39:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 -- 172.21.15.133:0/2840577166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9450102570 msgr2=0x7f94500fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9450102570 0x7f94500fd1c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9444006010 tx=0x7f9444003c80 comp rx=0 tx=0).stop 2024-09-11T23:39:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 -- 172.21.15.133:0/2840577166 shutdown_connections 2024-09-11T23:39:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94380a2cf0 0x7f94380a51b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9450102ef0 0x7f94500fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9450102570 0x7f94500fd1c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 --2- 172.21.15.133:0/2840577166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9450100e20 0x7f94500fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 -- 172.21.15.133:0/2840577166 >> 172.21.15.133:0/2840577166 conn(0x7f94500fc7d0 msgr2=0x7f945006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 -- 172.21.15.133:0/2840577166 shutdown_connections 2024-09-11T23:39:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:06.197+0000 7f9454ab8640 1 -- 172.21.15.133:0/2840577166 wait complete. 2024-09-11T23:39:07.000 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:39:07.001 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:39:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:07.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.474+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3630545568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1c4106f40 msgr2=0x7fd1c410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:07.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.474+0000 7fd1cbd53640 1 --2- 172.21.15.133:0/3630545568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1c4106f40 0x7fd1c410eac0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009ca0 tx=0x7fd1bc02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.474+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3630545568 shutdown_connections 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.474+0000 7fd1cbd53640 1 --2- 172.21.15.133:0/3630545568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1c4106f40 0x7fd1c410eac0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.474+0000 7fd1cbd53640 1 --2- 172.21.15.133:0/3630545568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1c41027c0 0x7fd1c4106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.474+0000 7fd1cbd53640 1 --2- 172.21.15.133:0/3630545568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1c4101e10 0x7fd1c41021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.474+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3630545568 >> 172.21.15.133:0/3630545568 conn(0x7fd1c40fd8d0 msgr2=0x7fd1c40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.474+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3630545568 shutdown_connections 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.474+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3630545568 wait complete. 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1cbd53640 1 Processor -- start 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1cbd53640 1 -- start start 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1cbd53640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1c4101e10 0x7fd1c41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1cbd53640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1c41027c0 0x7fd1c4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1cbd53640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1c4106f40 0x7fd1c419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1cbd53640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4111fa0 con 0x7fd1c4101e10 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1cbd53640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4111e20 con 0x7fd1c41027c0 2024-09-11T23:39:07.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1cbd53640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4112120 con 0x7fd1c4106f40 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1c9aef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1c4101e10 0x7fd1c41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1c9aef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1c4101e10 0x7fd1c41983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35274/0 (socket says 172.21.15.133:35274) 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1c9aef640 1 -- 172.21.15.133:0/3355882922 learned_addr learned my addr 172.21.15.133:0/3355882922 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1c92ee640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1c41027c0 0x7fd1c4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1c4106f40 0x7fd1c419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1c9aef640 1 -- 172.21.15.133:0/3355882922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1c4106f40 msgr2=0x7fd1c419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1c9aef640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1c4106f40 0x7fd1c419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1c9aef640 1 -- 172.21.15.133:0/3355882922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1c41027c0 msgr2=0x7fd1c4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1c9aef640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1c41027c0 0x7fd1c4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.475+0000 7fd1c9aef640 1 -- 172.21.15.133:0/3355882922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c419d370 con 0x7fd1c4101e10 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.476+0000 7fd1c9aef640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1c4101e10 0x7fd1c41983c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000d950 tx=0x7fd1c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.476+0000 7fd1baffd640 1 -- 172.21.15.133:0/3355882922 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0017070 con 0x7fd1c4101e10 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.476+0000 7fd1baffd640 1 -- 172.21.15.133:0/3355882922 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c0009c30 con 0x7fd1c4101e10 2024-09-11T23:39:07.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.476+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3355882922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c419d660 con 0x7fd1c4101e10 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.476+0000 7fd1baffd640 1 -- 172.21.15.133:0/3355882922 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c000f6b0 con 0x7fd1c4101e10 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.476+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3355882922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c41a4f40 con 0x7fd1c4101e10 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.478+0000 7fd1baffd640 1 -- 172.21.15.133:0/3355882922 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd1c00109d0 con 0x7fd1c4101e10 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.478+0000 7fd1ce12c740 1 Processor -- start 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1ce12c740 1 -- start start 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c908440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1baffd640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1b00a2cf0 0x7fd1b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c912d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c915630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55871c86d620 con 0x55871c908980 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55871c814e50 con 0x55871c908060 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55871c8695e0 con 0x55871c913240 2024-09-11T23:39:07.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1c92ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c912d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1baffd640 1 -- 172.21.15.133:0/3355882922 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fd1c00cf0d0 con 0x7fd1c4101e10 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1c92ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c912d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35282/0 (socket says 172.21.15.133:35282) 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1c92ee640 1 -- 172.21.15.133:0/2864387691 learned_addr learned my addr 172.21.15.133:0/2864387691 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1ca2f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c908440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1c9aef640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1b00a2cf0 0x7fd1b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/2864387691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c915630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1c92ee640 1 -- 172.21.15.133:0/2864387691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 msgr2=0x55871c915630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1c92ee640 1 --2- 172.21.15.133:0/2864387691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c915630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1c92ee640 1 -- 172.21.15.133:0/2864387691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 msgr2=0x55871c908440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1c92ee640 1 --2- 172.21.15.133:0/2864387691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c908440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1c92ee640 1 -- 172.21.15.133:0/2864387691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871c805d40 con 0x55871c908980 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.479+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/2864387691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c915630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.480+0000 7fd1c9aef640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1b00a2cf0 0x7fd1b00a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00cc890 tx=0x7fd1c0010d70 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:07.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.480+0000 7fd1c92ee640 1 --2- 172.21.15.133:0/2864387691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c912d00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009c10 tx=0x7fd1b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b700ea725e50540 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:07.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.480+0000 7fd1affff640 1 -- 172.21.15.133:0/2864387691 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4004230 con 0x55871c908980 2024-09-11T23:39:07.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.480+0000 7fd1affff640 1 -- 172.21.15.133:0/2864387691 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b40043d0 con 0x55871c908980 2024-09-11T23:39:07.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.480+0000 7fd1affff640 1 -- 172.21.15.133:0/2864387691 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4004e10 con 0x55871c908980 2024-09-11T23:39:07.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.480+0000 7fd1ce12c740 1 -- 172.21.15.133:0/2864387691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 msgr2=0x55871c912d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:07.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.480+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/2864387691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c912d00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009c10 tx=0x7fd1b4011810 comp rx=0 tx=0).stop 2024-09-11T23:39:07.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 -- 172.21.15.133:0/2864387691 shutdown_connections 2024-09-11T23:39:07.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/2864387691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c915630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/2864387691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c912d00 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/2864387691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c908440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 -- 172.21.15.133:0/2864387691 >> 172.21.15.133:0/2864387691 conn(0x55871c9045f0 msgr2=0x55871c9049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 -- 172.21.15.133:0/2864387691 shutdown_connections 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 -- 172.21.15.133:0/2864387691 wait complete. 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 -- start start 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 Processor -- start 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c910080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c9105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c910b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55871c915e10 con 0x55871c908980 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55871c7f5f10 con 0x55871c908060 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.481+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55871c915f90 con 0x55871c913240 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1ce12c740 10 client.0 _ll_register_callbacks cb 0x55871c819db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1c92ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c9105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1c92ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c9105c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35294/0 (socket says 172.21.15.133:35294) 2024-09-11T23:39:07.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1c92ee640 1 -- 172.21.15.133:0/1012450982 learned_addr learned my addr 172.21.15.133:0/1012450982 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c910080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1c92ee640 1 -- 172.21.15.133:0/1012450982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 msgr2=0x55871c910b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1c92ee640 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c910b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1c92ee640 1 -- 172.21.15.133:0/1012450982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 msgr2=0x55871c910080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1c92ee640 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c910080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1c92ee640 1 -- 172.21.15.133:0/1012450982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871c57c580 con 0x55871c908980 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c910080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1c92ee640 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c9105c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40051d0 tx=0x7fd1b4005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81eb6e210799f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/1012450982 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4024620 con 0x55871c908980 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/1012450982 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fd1b40247c0 con 0x55871c908980 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1ce12c740 1 -- 172.21.15.133:0/1012450982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1b40043d0 con 0x55871c908980 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/1012450982 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1b4032050 con 0x55871c908980 2024-09-11T23:39:07.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1ce12c740 10 client.16203 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.482+0000 7fd1ce12c740 1 -- 172.21.15.133:0/1012450982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55871c646960 con 0x55871c908980 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.483+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/1012450982 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b40043d0 con 0x55871c908980 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.483+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/1012450982 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd1b4018070 con 0x55871c908980 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.483+0000 7fd1ce12c740 10 client.16203 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.483+0000 7fd1ce12c740 10 client.16203 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.483+0000 7fd1ce12c740 20 client.16203 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.483+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55871c922ca0 0x55871c925090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.483+0000 7fd1ce12c740 4 client.16203 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.483+0000 7fd1ce12c740 1 -- 172.21.15.133:0/1012450982 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55871c87ab90 con 0x55871c922ca0 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.483+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55871c922ca0 0x55871c925090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ad7fa640 10 client.16203 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55871c922ca0 0x55871c925090 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3973870ab9806869 server_cookie=85c0d544d6e2080a in_seq=0 out_seq=0 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/1012450982 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55871c87ab90 con 0x55871c922ca0 2024-09-11T23:39:07.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ad7fa640 10 client.16203 handle_command_reply: tid=0 2024-09-11T23:39:07.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ce12c740 1 client.16203 shutdown 2024-09-11T23:39:07.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ce12c740 1 -- 172.21.15.133:0/1012450982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 msgr2=0x55871c9105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:07.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c9105c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40051d0 tx=0x7fd1b4005200 comp rx=0 tx=0).stop 2024-09-11T23:39:07.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ce12c740 1 -- 172.21.15.133:0/1012450982 shutdown_connections 2024-09-11T23:39:07.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55871c922ca0 0x55871c925090 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c910b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c9105c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ad7fa640 0 client.16203 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:07.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/1012450982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c910080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.484+0000 7fd1ce12c740 1 -- 172.21.15.133:0/1012450982 >> 172.21.15.133:0/1012450982 conn(0x55871c9045f0 msgr2=0x55871c903750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:07.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.485+0000 7fd1ce12c740 1 -- 172.21.15.133:0/1012450982 shutdown_connections 2024-09-11T23:39:07.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.485+0000 7fd1ce12c740 1 -- 172.21.15.133:0/1012450982 wait complete. 2024-09-11T23:39:07.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.485+0000 7fd1ce12c740 20 client.16203 trim_cache size 0 max 16384 2024-09-11T23:39:07.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ce12c740 1 Processor -- start 2024-09-11T23:39:07.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ce12c740 1 -- start start 2024-09-11T23:39:07.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908060 0x55871c903640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c908980 0x55871c92a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c913240 0x55871c92ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c41ad720 con 0x55871c908060 2024-09-11T23:39:07.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c41af260 con 0x55871c913240 2024-09-11T23:39:07.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c41addf0 con 0x55871c908980 2024-09-11T23:39:07.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ca2f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908060 0x55871c903640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ca2f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908060 0x55871c903640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35310/0 (socket says 172.21.15.133:35310) 2024-09-11T23:39:07.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ca2f0640 1 -- 172.21.15.133:0/208469464 learned_addr learned my addr 172.21.15.133:0/208469464 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:07.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1c92ee640 1 --2- 172.21.15.133:0/208469464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c908980 0x55871c92a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.502+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/208469464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c913240 0x55871c92ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.503+0000 7fd1ca2f0640 1 -- 172.21.15.133:0/208469464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c908980 msgr2=0x55871c92a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:07.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.503+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/208469464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c908980 0x55871c92a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.503+0000 7fd1ca2f0640 1 -- 172.21.15.133:0/208469464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c913240 msgr2=0x55871c92ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:07.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.503+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/208469464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c913240 0x55871c92ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.503+0000 7fd1ca2f0640 1 -- 172.21.15.133:0/208469464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871c9256c0 con 0x55871c908060 2024-09-11T23:39:07.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.503+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/208469464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908060 0x55871c903640 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005e30 tx=0x7fd1bc03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7251664cb4629b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:07.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.503+0000 7fd1af7fe640 1 -- 172.21.15.133:0/208469464 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc046040 con 0x55871c908060 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.503+0000 7fd1af7fe640 1 -- 172.21.15.133:0/208469464 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1bc051070 con 0x55871c908060 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.503+0000 7fd1af7fe640 1 -- 172.21.15.133:0/208469464 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0305c0 con 0x55871c908060 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.504+0000 7fd1ce12c740 1 -- 172.21.15.133:0/208469464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908060 msgr2=0x55871c903640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.504+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/208469464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908060 0x55871c903640 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005e30 tx=0x7fd1bc03f040 comp rx=0 tx=0).stop 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.504+0000 7fd1ce12c740 1 -- 172.21.15.133:0/208469464 shutdown_connections 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.504+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/208469464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c913240 0x55871c92ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.504+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/208469464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c908980 0x55871c92a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.504+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/208469464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908060 0x55871c903640 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.504+0000 7fd1ce12c740 1 -- 172.21.15.133:0/208469464 >> 172.21.15.133:0/208469464 conn(0x55871c9045f0 msgr2=0x55871c902f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.504+0000 7fd1ce12c740 1 -- 172.21.15.133:0/208469464 shutdown_connections 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.504+0000 7fd1ce12c740 1 -- 172.21.15.133:0/208469464 wait complete. 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ce12c740 1 -- start start 2024-09-11T23:39:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ce12c740 1 Processor -- start 2024-09-11T23:39:07.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c91b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c91b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ce12c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c9161a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c41aeb90 con 0x55871c908980 2024-09-11T23:39:07.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c41ae4c0 con 0x55871c908060 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ce12c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c41ab700 con 0x55871c913240 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ce12c740 10 client.0 _ll_register_callbacks cb 0x7fd1b4004e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ca2f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c91b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1c92ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c91b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ca2f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c91b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59512/0 (socket says 172.21.15.133:59512) 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.505+0000 7fd1ca2f0640 1 -- 172.21.15.133:0/433549766 learned_addr learned my addr 172.21.15.133:0/433549766 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c9161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ca2f0640 1 -- 172.21.15.133:0/433549766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 msgr2=0x55871c9161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c9161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ca2f0640 1 -- 172.21.15.133:0/433549766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 msgr2=0x55871c91b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c91b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ca2f0640 1 -- 172.21.15.133:0/433549766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871c904420 con 0x55871c908060 2024-09-11T23:39:07.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ca2f0640 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c91b3d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009620 tx=0x7fd1bc02fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e9d1738fc50011 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/433549766 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc004d40 con 0x55871c908060 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ce12c740 1 -- 172.21.15.133:0/433549766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1bc051070 con 0x55871c908060 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/433549766 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fd1bc02f610 con 0x55871c908060 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/433549766 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1bc02f810 con 0x55871c908060 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ce12c740 10 client.25631 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.506+0000 7fd1ce12c740 1 -- 172.21.15.133:0/433549766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55871c98e0c0 con 0x55871c908060 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.507+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/433549766 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1bc051070 con 0x55871c908060 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.507+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/433549766 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd1bc0307f0 con 0x55871c908060 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.507+0000 7fd1ce12c740 10 client.25631 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.507+0000 7fd1ce12c740 10 client.25631 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.507+0000 7fd1ce12c740 20 client.25631 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.507+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55871c98e690 0x55871c990a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:07.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.507+0000 7fd1ce12c740 4 client.25631 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1fb2b5b2", "release": true, "await": true}] 2024-09-11T23:39:07.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.507+0000 7fd1ce12c740 1 -- 172.21.15.133:0/433549766 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1fb2b5b2", "release": true, "await": true}) -- 0x55871c8ff950 con 0x55871c98e690 2024-09-11T23:39:07.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.507+0000 7fd1c92ee640 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55871c98e690 0x55871c990a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:07.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.508+0000 7fd1ad7fa640 10 client.25631 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:07.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.508+0000 7fd1c92ee640 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55871c98e690 0x55871c990a80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1575a86de9335176 server_cookie=7e6d2b6e68c9e35e in_seq=0 out_seq=0 2024-09-11T23:39:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ad7fa640 1 -- 172.21.15.133:0/433549766 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55871c8ff950 con 0x55871c98e690 2024-09-11T23:39:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ad7fa640 10 client.25631 handle_command_reply: tid=0 2024-09-11T23:39:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ce12c740 1 client.25631 shutdown 2024-09-11T23:39:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ce12c740 1 -- 172.21.15.133:0/433549766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 msgr2=0x55871c91b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c91b3d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009620 tx=0x7fd1bc02fc00 comp rx=0 tx=0).stop 2024-09-11T23:39:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ce12c740 1 -- 172.21.15.133:0/433549766 shutdown_connections 2024-09-11T23:39:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55871c98e690 0x55871c990a80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55871c913240 0x55871c9161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55871c908980 0x55871c91b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ce12c740 1 --2- 172.21.15.133:0/433549766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55871c908060 0x55871c91b3d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ad7fa640 0 client.25631 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ce12c740 1 -- 172.21.15.133:0/433549766 >> 172.21.15.133:0/433549766 conn(0x55871c9045f0 msgr2=0x55871c910810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.511+0000 7fd1ce12c740 1 -- 172.21.15.133:0/433549766 shutdown_connections 2024-09-11T23:39:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1ce12c740 1 -- 172.21.15.133:0/433549766 wait complete. 2024-09-11T23:39:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1ce12c740 20 client.25631 trim_cache size 0 max 16384 2024-09-11T23:39:07.512 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:39:07.512 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 28, 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 8, 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "1fb2b5b2": { 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 8, 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 133.4, 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:39:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:39:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:39:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:39:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:39:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:39:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:07.514 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:39:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3355882922 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1b00a2cf0 msgr2=0x7fd1b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1cbd53640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1b00a2cf0 0x7fd1b00a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00cc890 tx=0x7fd1c0010d70 comp rx=0 tx=0).stop 2024-09-11T23:39:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3355882922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1c4101e10 msgr2=0x7fd1c41983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1cbd53640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1c4101e10 0x7fd1c41983c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000d950 tx=0x7fd1c000de10 comp rx=0 tx=0).stop 2024-09-11T23:39:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3355882922 shutdown_connections 2024-09-11T23:39:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1cbd53640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1b00a2cf0 0x7fd1b00a51b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1cbd53640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1c4106f40 0x7fd1c419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1cbd53640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1c41027c0 0x7fd1c4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1cbd53640 1 --2- 172.21.15.133:0/3355882922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1c4101e10 0x7fd1c41983c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.512+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3355882922 >> 172.21.15.133:0/3355882922 conn(0x7fd1c40fd8d0 msgr2=0x7fd1c40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.513+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3355882922 shutdown_connections 2024-09-11T23:39:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:07.513+0000 7fd1cbd53640 1 -- 172.21.15.133:0/3355882922 wait complete. 2024-09-11T23:39:07.531 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:08.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:08 smithi133 ceph-mon[31430]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 831 KiB/s rd, 305 MiB/s wr, 824 op/s 2024-09-11T23:39:08.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:08 smithi178 ceph-mon[35882]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 831 KiB/s rd, 305 MiB/s wr, 824 op/s 2024-09-11T23:39:08.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:08 smithi138 ceph-mon[35142]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 831 KiB/s rd, 305 MiB/s wr, 824 op/s 2024-09-11T23:39:08.963 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1fb2b5b2', total seconds elapsed: 19.2 2024-09-11T23:39:09.533 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:09 smithi133 ceph-mon[31430]: pgmap v74: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 336 MiB/s wr, 1.04k op/s 2024-09-11T23:39:09.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.534+0000 7f504f7f1640 1 -- 172.21.15.133:0/2279644470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048102ef0 msgr2=0x7f504810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:09.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.534+0000 7f504f7f1640 1 --2- 172.21.15.133:0/2279644470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048102ef0 0x7f504810da40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5040009ca0 tx=0x7f504002a950 comp rx=0 tx=0).stop 2024-09-11T23:39:09.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.534+0000 7f504f7f1640 1 -- 172.21.15.133:0/2279644470 shutdown_connections 2024-09-11T23:39:09.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.534+0000 7f504f7f1640 1 --2- 172.21.15.133:0/2279644470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048102ef0 0x7f504810da40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:09.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.534+0000 7f504f7f1640 1 --2- 172.21.15.133:0/2279644470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5048102570 0x7f50481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.534+0000 7f504f7f1640 1 --2- 172.21.15.133:0/2279644470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5048100e20 0x7f5048101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.534+0000 7f504f7f1640 1 -- 172.21.15.133:0/2279644470 >> 172.21.15.133:0/2279644470 conn(0x7f50480fc7d0 msgr2=0x7f50480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.534+0000 7f504f7f1640 1 -- 172.21.15.133:0/2279644470 shutdown_connections 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.534+0000 7f504f7f1640 1 -- 172.21.15.133:0/2279644470 wait complete. 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504f7f1640 1 Processor -- start 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504f7f1640 1 -- start start 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504f7f1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048100e20 0x7f50480fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504f7f1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5048102570 0x7f50480fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504f7f1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5048102ef0 0x7f50480fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504f7f1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5048110f80 con 0x7f5048100e20 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504f7f1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5048110e00 con 0x7f5048102570 2024-09-11T23:39:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504f7f1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5048111100 con 0x7f5048102ef0 2024-09-11T23:39:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504cd8c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5048102570 0x7f50480fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504cd8c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5048102570 0x7f50480fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59532/0 (socket says 172.21.15.133:59532) 2024-09-11T23:39:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504cd8c640 1 -- 172.21.15.133:0/1526525491 learned_addr learned my addr 172.21.15.133:0/1526525491 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504d58d640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048100e20 0x7f50480fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.535+0000 7f504dd8e640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5048102ef0 0x7f50480fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.536+0000 7f504cd8c640 1 -- 172.21.15.133:0/1526525491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5048102ef0 msgr2=0x7f50480fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.536+0000 7f504cd8c640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5048102ef0 0x7f50480fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:09.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.536+0000 7f504cd8c640 1 -- 172.21.15.133:0/1526525491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048100e20 msgr2=0x7f50480fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:09.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.536+0000 7f504cd8c640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048100e20 0x7f50480fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:09.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.536+0000 7f504cd8c640 1 -- 172.21.15.133:0/1526525491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50481a6910 con 0x7f5048102570 2024-09-11T23:39:09.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.536+0000 7f504cd8c640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5048102570 0x7f50480fd1c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f503800e9f0 tx=0x7f503800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:09.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.536+0000 7f503e7fc640 1 -- 172.21.15.133:0/1526525491 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503800cd80 con 0x7f5048102570 2024-09-11T23:39:09.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.536+0000 7f504f7f1640 1 -- 172.21.15.133:0/1526525491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50481a6c00 con 0x7f5048102570 2024-09-11T23:39:09.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.536+0000 7f503e7fc640 1 -- 172.21.15.133:0/1526525491 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5038010910 con 0x7f5048102570 2024-09-11T23:39:09.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.536+0000 7f503e7fc640 1 -- 172.21.15.133:0/1526525491 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038018990 con 0x7f5048102570 2024-09-11T23:39:09.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.536+0000 7f504f7f1640 1 -- 172.21.15.133:0/1526525491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50481a7140 con 0x7f5048102570 2024-09-11T23:39:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.537+0000 7f504f7f1640 1 -- 172.21.15.133:0/1526525491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5048101200 con 0x7f5048102570 2024-09-11T23:39:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.538+0000 7f503e7fc640 1 -- 172.21.15.133:0/1526525491 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5038010ac0 con 0x7f5048102570 2024-09-11T23:39:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.539+0000 7f503e7fc640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f50340a2cf0 0x7f50340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.539+0000 7f504d58d640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f50340a2cf0 0x7f50340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.539+0000 7f503e7fc640 1 -- 172.21.15.133:0/1526525491 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f50380cfb70 con 0x7f5048102570 2024-09-11T23:39:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.539+0000 7f504d58d640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f50340a2cf0 0x7f50340a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f50440058c0 tx=0x7f504400a2d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:09.542+0000 7f503e7fc640 1 -- 172.21.15.133:0/1526525491 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f503809c1b0 con 0x7f5048102570 2024-09-11T23:39:09.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:09 smithi138 ceph-mon[35142]: pgmap v74: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 336 MiB/s wr, 1.04k op/s 2024-09-11T23:39:09.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:09 smithi178 ceph-mon[35882]: pgmap v74: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 336 MiB/s wr, 1.04k op/s 2024-09-11T23:39:10.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.020+0000 7f504f7f1640 1 -- 172.21.15.133:0/1526525491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50480fe1b0 con 0x7f5048102570 2024-09-11T23:39:10.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.020+0000 7f503e7fc640 1 -- 172.21.15.133:0/1526525491 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f50480fe1b0 con 0x7f5048102570 2024-09-11T23:39:10.021 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:39:10.021 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:39:10.021 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:39:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.024+0000 7f504f7f1640 1 -- 172.21.15.133:0/1526525491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f50340a2cf0 msgr2=0x7f50340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.024+0000 7f504f7f1640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f50340a2cf0 0x7f50340a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f50440058c0 tx=0x7f504400a2d0 comp rx=0 tx=0).stop 2024-09-11T23:39:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.024+0000 7f504f7f1640 1 -- 172.21.15.133:0/1526525491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5048102570 msgr2=0x7f50480fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.024+0000 7f504f7f1640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5048102570 0x7f50480fd1c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f503800e9f0 tx=0x7f503800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:39:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.024+0000 7f504f7f1640 1 -- 172.21.15.133:0/1526525491 shutdown_connections 2024-09-11T23:39:10.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.024+0000 7f504f7f1640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f50340a2cf0 0x7f50340a51b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.024+0000 7f504f7f1640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5048102ef0 0x7f50480fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.024+0000 7f504f7f1640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5048102570 0x7f50480fd1c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.024+0000 7f504f7f1640 1 --2- 172.21.15.133:0/1526525491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048100e20 0x7f50480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.024+0000 7f504f7f1640 1 -- 172.21.15.133:0/1526525491 >> 172.21.15.133:0/1526525491 conn(0x7f50480fc7d0 msgr2=0x7f504806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.024+0000 7f504f7f1640 1 -- 172.21.15.133:0/1526525491 shutdown_connections 2024-09-11T23:39:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:10.025+0000 7f504f7f1640 1 -- 172.21.15.133:0/1526525491 wait complete. 2024-09-11T23:39:10.932 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:10 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1526525491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:10.939 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:39:10.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:10 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1526525491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:10 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1526525491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:11 smithi138 ceph-mon[35142]: pgmap v75: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 74 MiB/s wr, 361 op/s 2024-09-11T23:39:11.974 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:11 smithi133 ceph-mon[31430]: pgmap v75: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 74 MiB/s wr, 361 op/s 2024-09-11T23:39:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:11 smithi178 ceph-mon[35882]: pgmap v75: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 74 MiB/s wr, 361 op/s 2024-09-11T23:39:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.618+0000 7f6e05ad9640 1 -- 172.21.15.133:0/4237768232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e00112b50 msgr2=0x7f6e00114f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:12.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.618+0000 7f6e05ad9640 1 --2- 172.21.15.133:0/4237768232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e00112b50 0x7f6e00114f40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009be0 tx=0x7f6df802a710 comp rx=0 tx=0).stop 2024-09-11T23:39:12.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.619+0000 7f6e05ad9640 1 -- 172.21.15.133:0/4237768232 shutdown_connections 2024-09-11T23:39:12.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.619+0000 7f6e05ad9640 1 --2- 172.21.15.133:0/4237768232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e00112b50 0x7f6e00114f40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.619+0000 7f6e05ad9640 1 --2- 172.21.15.133:0/4237768232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00107160 0x7f6e00112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.619+0000 7f6e05ad9640 1 --2- 172.21.15.133:0/4237768232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e00106840 0x7f6e00106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.619+0000 7f6e05ad9640 1 -- 172.21.15.133:0/4237768232 >> 172.21.15.133:0/4237768232 conn(0x7f6e00075f70 msgr2=0x7f6e00076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.619+0000 7f6e05ad9640 1 -- 172.21.15.133:0/4237768232 shutdown_connections 2024-09-11T23:39:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.621+0000 7f6e05ad9640 1 -- 172.21.15.133:0/4237768232 wait complete. 2024-09-11T23:39:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.622+0000 7f6e05ad9640 1 Processor -- start 2024-09-11T23:39:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.624+0000 7f6e05ad9640 1 -- start start 2024-09-11T23:39:12.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.624+0000 7f6e05ad9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00106840 0x7f6e0010a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.624+0000 7f6e05ad9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e00107160 0x7f6e0010ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.624+0000 7f6e05ad9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e00112b50 0x7f6e001036d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.624+0000 7f6e05ad9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0006c660 con 0x7f6e00107160 2024-09-11T23:39:12.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.624+0000 7f6e05ad9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0006c4e0 con 0x7f6e00106840 2024-09-11T23:39:12.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.624+0000 7f6e05ad9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0006c7e0 con 0x7f6e00112b50 2024-09-11T23:39:12.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.625+0000 7f6dfffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e00112b50 0x7f6e001036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.625+0000 7f6dff7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00106840 0x7f6e0010a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.625+0000 7f6dfffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e00112b50 0x7f6e001036d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:52972/0 (socket says 172.21.15.133:52972) 2024-09-11T23:39:12.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.625+0000 7f6dfffff640 1 -- 172.21.15.133:0/560434075 learned_addr learned my addr 172.21.15.133:0/560434075 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:12.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.625+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e00107160 0x7f6e0010ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.626+0000 7f6dfeffd640 1 -- 172.21.15.133:0/560434075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e00112b50 msgr2=0x7f6e001036d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.626+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e00112b50 0x7f6e001036d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.626+0000 7f6dfeffd640 1 -- 172.21.15.133:0/560434075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00106840 msgr2=0x7f6e0010a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.626+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00106840 0x7f6e0010a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.626+0000 7f6dfeffd640 1 -- 172.21.15.133:0/560434075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e00103c90 con 0x7f6e00107160 2024-09-11T23:39:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.626+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e00107160 0x7f6e0010ae40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6df000b3e0 tx=0x7f6df000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.627+0000 7f6dfcff9640 1 -- 172.21.15.133:0/560434075 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0015020 con 0x7f6e00107160 2024-09-11T23:39:12.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.627+0000 7f6e05ad9640 1 -- 172.21.15.133:0/560434075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e00103f80 con 0x7f6e00107160 2024-09-11T23:39:12.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.628+0000 7f6e05ad9640 1 -- 172.21.15.133:0/560434075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e00104510 con 0x7f6e00107160 2024-09-11T23:39:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.628+0000 7f6dfcff9640 1 -- 172.21.15.133:0/560434075 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df0011d10 con 0x7f6e00107160 2024-09-11T23:39:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.629+0000 7f6dfcff9640 1 -- 172.21.15.133:0/560434075 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0015020 con 0x7f6e00107160 2024-09-11T23:39:12.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.633+0000 7f6e07eb2740 1 Processor -- start 2024-09-11T23:39:12.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.634+0000 7f6e07eb2740 1 -- start start 2024-09-11T23:39:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.634+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d29128090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.634+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 0x557d29128a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.634+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 0x557d29135760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.634+0000 7f6e07eb2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557d2908e2f0 con 0x557d291285d0 2024-09-11T23:39:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.634+0000 7f6e07eb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557d29035cf0 con 0x557d29133370 2024-09-11T23:39:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.634+0000 7f6e07eb2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557d2908a2b0 con 0x557d29129b40 2024-09-11T23:39:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.635+0000 7f6dfcff9640 1 -- 172.21.15.133:0/560434075 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6df0018020 con 0x7f6e00107160 2024-09-11T23:39:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.635+0000 7f6dff7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 0x557d29128a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.635+0000 7f6dff7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 0x557d29128a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39088/0 (socket says 172.21.15.133:39088) 2024-09-11T23:39:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.635+0000 7f6dff7fe640 1 -- 172.21.15.133:0/1320883183 learned_addr learned my addr 172.21.15.133:0/1320883183 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:12.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.636+0000 7f6dff7fe640 1 -- 172.21.15.133:0/1320883183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 msgr2=0x557d29128090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:12.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.636+0000 7f6dff7fe640 1 --2- 172.21.15.133:0/1320883183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d29128090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.636+0000 7f6dff7fe640 1 -- 172.21.15.133:0/1320883183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 msgr2=0x557d29135760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.636+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/1320883183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 0x557d29135760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.636+0000 7f6dff7fe640 1 --2- 172.21.15.133:0/1320883183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 0x557d29135760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.636+0000 7f6dff7fe640 1 -- 172.21.15.133:0/1320883183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d2901a3a0 con 0x557d291285d0 2024-09-11T23:39:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.636+0000 7f6dfcff9640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6de80a31b0 0x7f6de80a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.636+0000 7f6dff7fe640 1 --2- 172.21.15.133:0/1320883183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 0x557d29128a10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009c50 tx=0x7f6df4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06231709bb18db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.637+0000 7f6dfcff9640 1 -- 172.21.15.133:0/560434075 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6df0011630 con 0x7f6e00107160 2024-09-11T23:39:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.637+0000 7f6de5ffb640 1 -- 172.21.15.133:0/1320883183 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df401e070 con 0x557d291285d0 2024-09-11T23:39:12.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.637+0000 7f6de5ffb640 1 -- 172.21.15.133:0/1320883183 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df4019b10 con 0x557d291285d0 2024-09-11T23:39:12.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.637+0000 7f6de5ffb640 1 -- 172.21.15.133:0/1320883183 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df400eae0 con 0x557d291285d0 2024-09-11T23:39:12.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.638+0000 7f6dfffff640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6de80a31b0 0x7f6de80a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.638+0000 7f6e07eb2740 1 -- 172.21.15.133:0/1320883183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 msgr2=0x557d29128a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:12.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.638+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/1320883183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 0x557d29128a10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009c50 tx=0x7f6df4011810 comp rx=0 tx=0).stop 2024-09-11T23:39:12.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.639+0000 7f6e07eb2740 1 -- 172.21.15.133:0/1320883183 shutdown_connections 2024-09-11T23:39:12.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.639+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/1320883183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 0x557d29135760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.639+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/1320883183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 0x557d29128a10 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.639+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/1320883183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d29128090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.639+0000 7f6e07eb2740 1 -- 172.21.15.133:0/1320883183 >> 172.21.15.133:0/1320883183 conn(0x557d290f1310 msgr2=0x557d29124c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.639+0000 7f6e07eb2740 1 -- 172.21.15.133:0/1320883183 shutdown_connections 2024-09-11T23:39:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.639+0000 7f6dfffff640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6de80a31b0 0x7f6de80a5670 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6df80099a0 tx=0x7f6df8005ea0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.639+0000 7f6e07eb2740 1 -- 172.21.15.133:0/1320883183 wait complete. 2024-09-11T23:39:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.639+0000 7f6e07eb2740 1 -- start start 2024-09-11T23:39:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.640+0000 7f6e07eb2740 1 Processor -- start 2024-09-11T23:39:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.640+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 0x557d29132ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.640+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d2912ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.640+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 0x557d291304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.640+0000 7f6e07eb2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557d291360c0 con 0x557d291285d0 2024-09-11T23:39:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.640+0000 7f6e07eb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557d29049060 con 0x557d29133370 2024-09-11T23:39:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.640+0000 7f6e07eb2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557d29136240 con 0x557d29129b40 2024-09-11T23:39:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.640+0000 7f6e07eb2740 10 client.0 _ll_register_callbacks cb 0x557d28d9c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.641+0000 7f6dfffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 0x557d291304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.641+0000 7f6dfffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 0x557d291304f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47306/0 (socket says 172.21.15.133:47306) 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.641+0000 7f6dfffff640 1 -- 172.21.15.133:0/3208859256 learned_addr learned my addr 172.21.15.133:0/3208859256 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.641+0000 7f6dff7fe640 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 0x557d29132ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.641+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d2912ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.643+0000 7f6dff7fe640 1 -- 172.21.15.133:0/3208859256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 msgr2=0x557d2912ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.643+0000 7f6dff7fe640 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d2912ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.643+0000 7f6dff7fe640 1 -- 172.21.15.133:0/3208859256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 msgr2=0x557d291304f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.643+0000 7f6dff7fe640 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 0x557d291304f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.643+0000 7f6dff7fe640 1 -- 172.21.15.133:0/3208859256 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d28dff3c0 con 0x557d291285d0 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.643+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d2912ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.643+0000 7f6dfffff640 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 0x557d291304f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.644+0000 7f6dff7fe640 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 0x557d29132ce0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009c20 tx=0x7f6df40094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ace98c35812ea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.644+0000 7f6ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.645+0000 7f6ddeffd640 1 -- 172.21.15.133:0/3208859256 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4029020 con 0x557d291285d0 2024-09-11T23:39:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.646+0000 7f6e07eb2740 1 -- 172.21.15.133:0/3208859256 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df4019b10 con 0x557d291285d0 2024-09-11T23:39:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.646+0000 7f6ddeffd640 1 -- 172.21.15.133:0/3208859256 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6df4005040 con 0x557d291285d0 2024-09-11T23:39:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.646+0000 7f6ddeffd640 1 -- 172.21.15.133:0/3208859256 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6df401e040 con 0x557d291285d0 2024-09-11T23:39:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.646+0000 7f6e07eb2740 10 client.16227 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.646+0000 7f6e07eb2740 1 -- 172.21.15.133:0/3208859256 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d29141b60 con 0x557d291285d0 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.647+0000 7f6ddeffd640 1 -- 172.21.15.133:0/3208859256 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df4019b10 con 0x557d291285d0 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.648+0000 7f6ddeffd640 1 -- 172.21.15.133:0/3208859256 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6df400eca0 con 0x557d291285d0 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.648+0000 7f6e07eb2740 10 client.16227 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.648+0000 7f6e07eb2740 10 client.16227 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.648+0000 7f6e07eb2740 20 client.16227 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.648+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x557d29143370 0x557d291457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.648+0000 7f6e07eb2740 4 client.16227 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.648+0000 7f6e07eb2740 1 -- 172.21.15.133:0/3208859256 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d2909b860 con 0x557d29143370 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.648+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x557d29143370 0x557d291457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.649+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x557d29143370 0x557d291457f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f3a7a960ba27651b server_cookie=9ba8b081d117f6ca in_seq=0 out_seq=0 2024-09-11T23:39:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.650+0000 7f6ddeffd640 10 client.16227 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:39:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.651+0000 7f6ddeffd640 1 -- 172.21.15.133:0/3208859256 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d2909b860 con 0x557d29143370 2024-09-11T23:39:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.651+0000 7f6ddeffd640 10 client.16227 handle_command_reply: tid=0 2024-09-11T23:39:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.651+0000 7f6e07eb2740 1 client.16227 shutdown 2024-09-11T23:39:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.651+0000 7f6e07eb2740 1 -- 172.21.15.133:0/3208859256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 msgr2=0x557d29132ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.651+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 0x557d29132ce0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009c20 tx=0x7f6df40094a0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.652+0000 7f6e07eb2740 1 -- 172.21.15.133:0/3208859256 shutdown_connections 2024-09-11T23:39:12.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.652+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x557d29143370 0x557d291457f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.652+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d29133370 0x557d291304f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.652+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d2912ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.652+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/3208859256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d291285d0 0x557d29132ce0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.652+0000 7f6e07eb2740 1 -- 172.21.15.133:0/3208859256 >> 172.21.15.133:0/3208859256 conn(0x557d290f1310 msgr2=0x557d2912a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:12.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.652+0000 7f6ddeffd640 0 client.16227 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:39:12.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.652+0000 7f6e07eb2740 1 -- 172.21.15.133:0/3208859256 shutdown_connections 2024-09-11T23:39:12.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.653+0000 7f6e07eb2740 1 -- 172.21.15.133:0/3208859256 wait complete. 2024-09-11T23:39:12.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.653+0000 7f6e07eb2740 20 client.16227 trim_cache size 0 max 16384 2024-09-11T23:39:12.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.699+0000 7f6e07eb2740 1 Processor -- start 2024-09-11T23:39:12.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.703+0000 7f6e07eb2740 1 -- start start 2024-09-11T23:39:12.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.703+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 0x557d2913f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.703+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29129b40 0x557d29135410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.703+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29133370 0x557d2912e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.703+0000 7f6e07eb2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec00e5d0 con 0x557d29129b40 2024-09-11T23:39:12.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.703+0000 7f6e07eb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec010110 con 0x557d291285d0 2024-09-11T23:39:12.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.703+0000 7f6e07eb2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec00eca0 con 0x557d29133370 2024-09-11T23:39:12.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.706+0000 7f6dfeffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29129b40 0x557d29135410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.707+0000 7f6dfeffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29129b40 0x557d29135410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39108/0 (socket says 172.21.15.133:39108) 2024-09-11T23:39:12.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.707+0000 7f6dfeffd640 1 -- 172.21.15.133:0/4120397181 learned_addr learned my addr 172.21.15.133:0/4120397181 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:12.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.708+0000 7f6dfffff640 1 --2- 172.21.15.133:0/4120397181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29133370 0x557d2912e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.708+0000 7f6dff7fe640 1 --2- 172.21.15.133:0/4120397181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 0x557d2913f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.709+0000 7f6dfeffd640 1 -- 172.21.15.133:0/4120397181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29133370 msgr2=0x557d2912e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:12.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.709+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/4120397181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29133370 0x557d2912e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.709+0000 7f6dfeffd640 1 -- 172.21.15.133:0/4120397181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 msgr2=0x557d2913f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:12.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.709+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/4120397181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 0x557d2913f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.709+0000 7f6dfeffd640 1 -- 172.21.15.133:0/4120397181 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d29128e10 con 0x557d29129b40 2024-09-11T23:39:12.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.712+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/4120397181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29129b40 0x557d29135410 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6df000bce0 tx=0x7f6df0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4267d0fd51f52ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:12.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.718+0000 7f6de57fa640 1 -- 172.21.15.133:0/4120397181 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0015040 con 0x557d29129b40 2024-09-11T23:39:12.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.718+0000 7f6de57fa640 1 -- 172.21.15.133:0/4120397181 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df0011d10 con 0x557d29129b40 2024-09-11T23:39:12.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.718+0000 7f6de57fa640 1 -- 172.21.15.133:0/4120397181 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df00cb220 con 0x557d29129b40 2024-09-11T23:39:12.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.723+0000 7f6e07eb2740 1 -- 172.21.15.133:0/4120397181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29129b40 msgr2=0x557d29135410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:12.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.723+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/4120397181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29129b40 0x557d29135410 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6df000bce0 tx=0x7f6df0000bc0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.723+0000 7f6e07eb2740 1 -- 172.21.15.133:0/4120397181 shutdown_connections 2024-09-11T23:39:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.723+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/4120397181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29133370 0x557d2912e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.723+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/4120397181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29129b40 0x557d29135410 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.723+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/4120397181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 0x557d2913f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.724+0000 7f6e07eb2740 1 -- 172.21.15.133:0/4120397181 >> 172.21.15.133:0/4120397181 conn(0x557d290f1310 msgr2=0x557d2911fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.724+0000 7f6e07eb2740 1 -- 172.21.15.133:0/4120397181 shutdown_connections 2024-09-11T23:39:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.724+0000 7f6e07eb2740 1 -- 172.21.15.133:0/4120397181 wait complete. 2024-09-11T23:39:12.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.725+0000 7f6e07eb2740 1 -- start start 2024-09-11T23:39:12.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.726+0000 7f6e07eb2740 1 Processor -- start 2024-09-11T23:39:12.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.727+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 0x557d2913a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.727+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d2913ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.728+0000 7f6e07eb2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29133370 0x557d2913b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.730+0000 7f6dff7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 0x557d2913a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.730+0000 7f6dff7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 0x557d2913a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47326/0 (socket says 172.21.15.133:47326) 2024-09-11T23:39:12.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.730+0000 7f6dfeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d2913ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.730+0000 7f6dfeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d2913ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53016/0 (socket says 172.21.15.133:53016) 2024-09-11T23:39:12.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.730+0000 7f6dff7fe640 1 -- 172.21.15.133:0/497731709 learned_addr learned my addr 172.21.15.133:0/497731709 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:12.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.731+0000 7f6e07eb2740 1 -- 172.21.15.133:0/497731709 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec00fa40 con 0x557d29133370 2024-09-11T23:39:12.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.731+0000 7f6e07eb2740 1 -- 172.21.15.133:0/497731709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec00f370 con 0x557d291285d0 2024-09-11T23:39:12.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.731+0000 7f6e07eb2740 1 -- 172.21.15.133:0/497731709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec00c5b0 con 0x557d29129b40 2024-09-11T23:39:12.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.731+0000 7f6e07eb2740 10 client.0 _ll_register_callbacks cb 0x7f6df401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:12.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.734+0000 7f6dfffff640 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29133370 0x557d2913b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.736+0000 7f6dff7fe640 1 -- 172.21.15.133:0/497731709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 msgr2=0x557d2913ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:12.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.736+0000 7f6dff7fe640 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d2913ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.736+0000 7f6dff7fe640 1 -- 172.21.15.133:0/497731709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29133370 msgr2=0x557d2913b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:12.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.736+0000 7f6dff7fe640 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29133370 0x557d2913b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.736+0000 7f6dff7fe640 1 -- 172.21.15.133:0/497731709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d2913bb50 con 0x557d291285d0 2024-09-11T23:39:12.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.737+0000 7f6dff7fe640 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 0x557d2913a740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6df4001eb0 tx=0x7f6df4019740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54ee05e4d9d340e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:12.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.737+0000 7f6ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:12.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.738+0000 7f6ddeffd640 1 -- 172.21.15.133:0/497731709 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df403c020 con 0x557d291285d0 2024-09-11T23:39:12.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.738+0000 7f6ddeffd640 1 -- 172.21.15.133:0/497731709 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6df4005140 con 0x557d291285d0 2024-09-11T23:39:12.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.738+0000 7f6e07eb2740 1 -- 172.21.15.133:0/497731709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df0011d10 con 0x557d291285d0 2024-09-11T23:39:12.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.739+0000 7f6ddeffd640 1 -- 172.21.15.133:0/497731709 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6df4029040 con 0x557d291285d0 2024-09-11T23:39:12.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.739+0000 7f6e07eb2740 10 client.25649 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:12.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.739+0000 7f6e07eb2740 1 -- 172.21.15.133:0/497731709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d2912cc40 con 0x557d291285d0 2024-09-11T23:39:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.740+0000 7f6ddeffd640 1 -- 172.21.15.133:0/497731709 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df4025420 con 0x557d291285d0 2024-09-11T23:39:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.740+0000 7f6ddeffd640 1 -- 172.21.15.133:0/497731709 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6df4024470 con 0x557d291285d0 2024-09-11T23:39:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.740+0000 7f6e07eb2740 10 client.25649 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.740+0000 7f6e07eb2740 10 client.25649 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:39:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.740+0000 7f6e07eb2740 20 client.25649 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.740+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x557d2912d340 0x557d2912d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.741+0000 7f6e07eb2740 4 client.25649 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:39:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.741+0000 7f6e07eb2740 1 -- 172.21.15.133:0/497731709 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6df0011d10 con 0x557d2912d340 2024-09-11T23:39:12.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.741+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x557d2912d340 0x557d2912d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:12.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.743+0000 7f6dfeffd640 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x557d2912d340 0x557d2912d750 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d489e752013f3e34 server_cookie=3454a5cdf1511086 in_seq=0 out_seq=0 2024-09-11T23:39:12.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.743+0000 7f6ddeffd640 10 client.25649 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:39:12.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.744+0000 7f6ddeffd640 1 -- 172.21.15.133:0/497731709 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6df0011d10 con 0x557d2912d340 2024-09-11T23:39:12.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.744+0000 7f6ddeffd640 10 client.25649 handle_command_reply: tid=0 2024-09-11T23:39:12.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.744+0000 7f6e07eb2740 1 client.25649 shutdown 2024-09-11T23:39:12.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.755+0000 7f6e07eb2740 1 -- 172.21.15.133:0/497731709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 msgr2=0x557d2913a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:12.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.755+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 0x557d2913a740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6df4001eb0 tx=0x7f6df4019740 comp rx=0 tx=0).stop 2024-09-11T23:39:12.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.755+0000 7f6e07eb2740 1 -- 172.21.15.133:0/497731709 shutdown_connections 2024-09-11T23:39:12.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.755+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x557d2912d340 0x557d2912d750 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.755+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d29133370 0x557d2913b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.755+0000 7f6ddeffd640 0 client.25649 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:39:12.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.755+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557d29129b40 0x557d2913ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.755+0000 7f6e07eb2740 1 --2- 172.21.15.133:0/497731709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557d291285d0 0x557d2913a740 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.756+0000 7f6e07eb2740 1 -- 172.21.15.133:0/497731709 >> 172.21.15.133:0/497731709 conn(0x557d290f1310 msgr2=0x557d2912e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:12.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.756+0000 7f6e07eb2740 1 -- 172.21.15.133:0/497731709 shutdown_connections 2024-09-11T23:39:12.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.756+0000 7f6e07eb2740 1 -- 172.21.15.133:0/497731709 wait complete. 2024-09-11T23:39:12.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.756+0000 7f6e07eb2740 20 client.25649 trim_cache size 0 max 16384 2024-09-11T23:39:12.766 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:39:12.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.764+0000 7f6e05ad9640 1 -- 172.21.15.133:0/560434075 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6de80a31b0 msgr2=0x7f6de80a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:12.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.764+0000 7f6e05ad9640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6de80a31b0 0x7f6de80a5670 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6df80099a0 tx=0x7f6df8005ea0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.764+0000 7f6e05ad9640 1 -- 172.21.15.133:0/560434075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e00107160 msgr2=0x7f6e0010ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:12.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.764+0000 7f6e05ad9640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e00107160 0x7f6e0010ae40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6df000b3e0 tx=0x7f6df000b7e0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.764+0000 7f6e05ad9640 1 -- 172.21.15.133:0/560434075 shutdown_connections 2024-09-11T23:39:12.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.764+0000 7f6e05ad9640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6de80a31b0 0x7f6de80a5670 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.764+0000 7f6e05ad9640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e00112b50 0x7f6e001036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.764+0000 7f6e05ad9640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e00107160 0x7f6e0010ae40 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.764+0000 7f6e05ad9640 1 --2- 172.21.15.133:0/560434075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e00106840 0x7f6e0010a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:12.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.765+0000 7f6e05ad9640 1 -- 172.21.15.133:0/560434075 >> 172.21.15.133:0/560434075 conn(0x7f6e00075f70 msgr2=0x7f6e001142b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:12.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.765+0000 7f6e05ad9640 1 -- 172.21.15.133:0/560434075 shutdown_connections 2024-09-11T23:39:12.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:12.765+0000 7f6e05ad9640 1 -- 172.21.15.133:0/560434075 wait complete. 2024-09-11T23:39:14.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:39:14.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:14 smithi133 ceph-mon[31430]: pgmap v76: 97 pgs: 97 active+clean; 11 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 253 MiB/s wr, 1.67k op/s 2024-09-11T23:39:14.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:14 smithi178 ceph-mon[35882]: pgmap v76: 97 pgs: 97 active+clean; 11 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 253 MiB/s wr, 1.67k op/s 2024-09-11T23:39:14.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:14 smithi138 ceph-mon[35142]: pgmap v76: 97 pgs: 97 active+clean; 11 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 253 MiB/s wr, 1.67k op/s 2024-09-11T23:39:15.217 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:15.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:15 smithi178 ceph-mon[35882]: pgmap v77: 97 pgs: 97 active+clean; 12 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 340 MiB/s wr, 2.39k op/s 2024-09-11T23:39:15.658 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:15 smithi133 ceph-mon[31430]: pgmap v77: 97 pgs: 97 active+clean; 12 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 340 MiB/s wr, 2.39k op/s 2024-09-11T23:39:15.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:15 smithi138 ceph-mon[35142]: pgmap v77: 97 pgs: 97 active+clean; 12 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 340 MiB/s wr, 2.39k op/s 2024-09-11T23:39:16.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.133+0000 7ff5aa7fc640 1 -- 172.21.15.133:0/2766537823 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a00319b0 con 0x7ff5ac109120 2024-09-11T23:39:16.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.134+0000 7ff5b1d90640 1 -- 172.21.15.133:0/2766537823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac109120 msgr2=0x7ff5ac110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.134+0000 7ff5b1d90640 1 --2- 172.21.15.133:0/2766537823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac109120 0x7ff5ac110ca0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009900 tx=0x7ff5a002a410 comp rx=0 tx=0).stop 2024-09-11T23:39:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.134+0000 7ff5b1d90640 1 -- 172.21.15.133:0/2766537823 shutdown_connections 2024-09-11T23:39:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.134+0000 7ff5b1d90640 1 --2- 172.21.15.133:0/2766537823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac109120 0x7ff5ac110ca0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.134+0000 7ff5b1d90640 1 --2- 172.21.15.133:0/2766537823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5ac1049a0 0x7ff5ac1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.134+0000 7ff5b1d90640 1 --2- 172.21.15.133:0/2766537823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5ac103ff0 0x7ff5ac1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.134+0000 7ff5b1d90640 1 -- 172.21.15.133:0/2766537823 >> 172.21.15.133:0/2766537823 conn(0x7ff5ac0ffab0 msgr2=0x7ff5ac101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.134+0000 7ff5b1d90640 1 -- 172.21.15.133:0/2766537823 shutdown_connections 2024-09-11T23:39:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.134+0000 7ff5b1d90640 1 -- 172.21.15.133:0/2766537823 wait complete. 2024-09-11T23:39:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.135+0000 7ff5b1d90640 1 Processor -- start 2024-09-11T23:39:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.135+0000 7ff5b1d90640 1 -- start start 2024-09-11T23:39:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.136+0000 7ff5b1d90640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5ac103ff0 0x7ff5ac19c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.136+0000 7ff5b1d90640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac1049a0 0x7ff5ac19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.136+0000 7ff5b1d90640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5ac109120 0x7ff5ac10f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.136+0000 7ff5b1d90640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac077ae0 con 0x7ff5ac1049a0 2024-09-11T23:39:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.136+0000 7ff5b1d90640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac077960 con 0x7ff5ac109120 2024-09-11T23:39:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.136+0000 7ff5b1d90640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac077c60 con 0x7ff5ac103ff0 2024-09-11T23:39:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.136+0000 7ff5abfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5ac109120 0x7ff5ac10f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.136+0000 7ff5abfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5ac109120 0x7ff5ac10f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47350/0 (socket says 172.21.15.133:47350) 2024-09-11T23:39:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.136+0000 7ff5abfff640 1 -- 172.21.15.133:0/1671989909 learned_addr learned my addr 172.21.15.133:0/1671989909 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.136+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5ac103ff0 0x7ff5ac19c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.137+0000 7ff5abfff640 1 -- 172.21.15.133:0/1671989909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5ac103ff0 msgr2=0x7ff5ac19c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.137+0000 7ff5abfff640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5ac103ff0 0x7ff5ac19c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.137+0000 7ff5abfff640 1 -- 172.21.15.133:0/1671989909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac1049a0 msgr2=0x7ff5ac19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.137+0000 7ff5abfff640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac1049a0 0x7ff5ac19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.137+0000 7ff5abfff640 1 -- 172.21.15.133:0/1671989909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac1101a0 con 0x7ff5ac109120 2024-09-11T23:39:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.137+0000 7ff5abfff640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5ac109120 0x7ff5ac10f8e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009230 tx=0x7ff5a00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.138+0000 7ff5a8ff9640 1 -- 172.21.15.133:0/1671989909 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a003f030 con 0x7ff5ac109120 2024-09-11T23:39:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.138+0000 7ff5a8ff9640 1 -- 172.21.15.133:0/1671989909 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a0003ce0 con 0x7ff5ac109120 2024-09-11T23:39:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.138+0000 7ff5b1d90640 1 -- 172.21.15.133:0/1671989909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ac110430 con 0x7ff5ac109120 2024-09-11T23:39:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.138+0000 7ff5b1d90640 1 -- 172.21.15.133:0/1671989909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ac1a3110 con 0x7ff5ac109120 2024-09-11T23:39:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.138+0000 7ff5a8ff9640 1 -- 172.21.15.133:0/1671989909 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0035070 con 0x7ff5ac109120 2024-09-11T23:39:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.142+0000 7ff5b4169740 1 Processor -- start 2024-09-11T23:39:16.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.143+0000 7ff5a8ff9640 1 -- 172.21.15.133:0/1671989909 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff5a0042030 con 0x7ff5ac109120 2024-09-11T23:39:16.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.144+0000 7ff5b4169740 1 -- start start 2024-09-11T23:39:16.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.144+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c671896b40 0x55c671895090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.144+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 0x55c671895a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.144+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718a0370 0x55c6718a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.144+0000 7ff5b4169740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c6717fb2f0 con 0x55c671896b40 2024-09-11T23:39:16.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.144+0000 7ff5b4169740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c6717a2cf0 con 0x55c6718955d0 2024-09-11T23:39:16.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.144+0000 7ff5b4169740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6717f72b0 con 0x55c6718a0370 2024-09-11T23:39:16.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.145+0000 7ff5ab7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 0x55c671895a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.145+0000 7ff5ab7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 0x55c671895a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47354/0 (socket says 172.21.15.133:47354) 2024-09-11T23:39:16.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.145+0000 7ff5ab7fe640 1 -- 172.21.15.133:0/952855960 learned_addr learned my addr 172.21.15.133:0/952855960 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:16.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.145+0000 7ff5a8ff9640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5940b3ed0 0x7ff5940b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.146+0000 7ff5ab7fe640 1 -- 172.21.15.133:0/952855960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718a0370 msgr2=0x55c6718a2760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:39:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.146+0000 7ff5a8ff9640 1 -- 172.21.15.133:0/1671989909 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff5a00efc10 con 0x7ff5ac109120 2024-09-11T23:39:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.146+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/952855960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718a0370 0x55c6718a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.146+0000 7ff5ab7fe640 1 -- 172.21.15.133:0/952855960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c671896b40 msgr2=0x55c671895090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.146+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/952855960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c671896b40 0x55c671895090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.146+0000 7ff5abfff640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5940b3ed0 0x7ff5940b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.146+0000 7ff5ab7fe640 1 -- 172.21.15.133:0/952855960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6717873a0 con 0x55c6718955d0 2024-09-11T23:39:16.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.146+0000 7ff5abfff640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5940b3ed0 0x7ff5940b6390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0005ab0 tx=0x7ff5a00314b0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.149+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/952855960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 0x55c671895a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400a5b0 tx=0x7ff5a40096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4b19e829f78665a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.149+0000 7ff591ffb640 1 -- 172.21.15.133:0/952855960 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4018020 con 0x55c6718955d0 2024-09-11T23:39:16.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.149+0000 7ff591ffb640 1 -- 172.21.15.133:0/952855960 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a4014cf0 con 0x55c6718955d0 2024-09-11T23:39:16.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.150+0000 7ff591ffb640 1 -- 172.21.15.133:0/952855960 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4013e10 con 0x55c6718955d0 2024-09-11T23:39:16.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.151+0000 7ff5b4169740 1 -- 172.21.15.133:0/952855960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 msgr2=0x55c671895a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.151+0000 7ff5b4169740 1 --2- 172.21.15.133:0/952855960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 0x55c671895a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400a5b0 tx=0x7ff5a40096d0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.151+0000 7ff5b4169740 1 -- 172.21.15.133:0/952855960 shutdown_connections 2024-09-11T23:39:16.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.151+0000 7ff5b4169740 1 --2- 172.21.15.133:0/952855960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718a0370 0x55c6718a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.151+0000 7ff5b4169740 1 --2- 172.21.15.133:0/952855960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 0x55c671895a10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.151+0000 7ff5b4169740 1 --2- 172.21.15.133:0/952855960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c671896b40 0x55c671895090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.151+0000 7ff5b4169740 1 -- 172.21.15.133:0/952855960 >> 172.21.15.133:0/952855960 conn(0x55c67185e310 msgr2=0x55c671891c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:16.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.152+0000 7ff5b4169740 1 -- 172.21.15.133:0/952855960 shutdown_connections 2024-09-11T23:39:16.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.152+0000 7ff5b4169740 1 -- 172.21.15.133:0/952855960 wait complete. 2024-09-11T23:39:16.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.152+0000 7ff5b4169740 1 -- start start 2024-09-11T23:39:16.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.152+0000 7ff5b4169740 1 Processor -- start 2024-09-11T23:39:16.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.153+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718955d0 0x55c67189fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.154+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c671896b40 0x55c67189cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.154+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718a0370 0x55c67189d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.154+0000 7ff5b4169740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c6718a3110 con 0x55c671896b40 2024-09-11T23:39:16.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.154+0000 7ff5b4169740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c6717b6060 con 0x55c6718a0370 2024-09-11T23:39:16.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.154+0000 7ff5aaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718955d0 0x55c67189fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.154+0000 7ff5aaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718955d0 0x55c67189fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53058/0 (socket says 172.21.15.133:53058) 2024-09-11T23:39:16.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.154+0000 7ff5aaffd640 1 -- 172.21.15.133:0/2469707237 learned_addr learned my addr 172.21.15.133:0/2469707237 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:16.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.154+0000 7ff5b4169740 1 -- 172.21.15.133:0/2469707237 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6718a3290 con 0x55c6718955d0 2024-09-11T23:39:16.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.154+0000 7ff5aaffd640 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718a0370 0x55c67189d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.154+0000 7ff5b4169740 10 client.0 _ll_register_callbacks cb 0x55c6715098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:16.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.155+0000 7ff5aaffd640 1 -- 172.21.15.133:0/2469707237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718955d0 msgr2=0x55c67189fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.155+0000 7ff5aaffd640 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718955d0 0x55c67189fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.155+0000 7ff5aaffd640 1 -- 172.21.15.133:0/2469707237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c671896b40 msgr2=0x55c67189cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.155+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c671896b40 0x55c67189cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.155+0000 7ff5aaffd640 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c671896b40 0x55c67189cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.155+0000 7ff5aaffd640 1 -- 172.21.15.133:0/2469707237 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67156c3c0 con 0x55c6718a0370 2024-09-11T23:39:16.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.155+0000 7ff5aaffd640 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718a0370 0x55c67189d440 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff59c014160 tx=0x7ff59c006960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=259213469c174fb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.155+0000 7ff58affd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:16.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.155+0000 7ff58affd640 1 -- 172.21.15.133:0/2469707237 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0149e0 con 0x55c6718a0370 2024-09-11T23:39:16.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.155+0000 7ff58affd640 1 -- 172.21.15.133:0/2469707237 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff59c014b80 con 0x55c6718a0370 2024-09-11T23:39:16.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.156+0000 7ff5b4169740 1 -- 172.21.15.133:0/2469707237 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a4014cf0 con 0x55c6718a0370 2024-09-11T23:39:16.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.156+0000 7ff58affd640 1 -- 172.21.15.133:0/2469707237 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff59c014d80 con 0x55c6718a0370 2024-09-11T23:39:16.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.157+0000 7ff5b4169740 10 client.25658 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:16.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.157+0000 7ff5b4169740 1 -- 172.21.15.133:0/2469707237 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c6718aeb80 con 0x55c6718a0370 2024-09-11T23:39:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.157+0000 7ff58affd640 1 -- 172.21.15.133:0/2469707237 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff59c004890 con 0x55c6718a0370 2024-09-11T23:39:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.157+0000 7ff58affd640 1 -- 172.21.15.133:0/2469707237 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff59c0061e0 con 0x55c6718a0370 2024-09-11T23:39:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.159+0000 7ff5b4169740 10 client.25658 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.159+0000 7ff5b4169740 10 client.25658 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:39:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.159+0000 7ff5b4169740 20 client.25658 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.159+0000 7ff5b4169740 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c6718aff90 0x55c6718b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.159+0000 7ff5b4169740 4 client.25658 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.159+0000 7ff5b4169740 1 -- 172.21.15.133:0/2469707237 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff5a4014cf0 con 0x55c6718aff90 2024-09-11T23:39:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.162+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c6718aff90 0x55c6718b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.163+0000 7ff58affd640 10 client.25658 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:39:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.163+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c6718aff90 0x55c6718b2380 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=74eb20131d62beaa server_cookie=e5b7f056072b38a1 in_seq=0 out_seq=0 2024-09-11T23:39:16.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.165+0000 7ff58affd640 1 -- 172.21.15.133:0/2469707237 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff5a4014cf0 con 0x55c6718aff90 2024-09-11T23:39:16.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.165+0000 7ff58affd640 10 client.25658 handle_command_reply: tid=0 2024-09-11T23:39:16.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.165+0000 7ff5b4169740 1 client.25658 shutdown 2024-09-11T23:39:16.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.166+0000 7ff5b4169740 1 -- 172.21.15.133:0/2469707237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718a0370 msgr2=0x55c67189d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.166+0000 7ff5b4169740 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718a0370 0x55c67189d440 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff59c014160 tx=0x7ff59c006960 comp rx=0 tx=0).stop 2024-09-11T23:39:16.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.170+0000 7ff5b4169740 1 -- 172.21.15.133:0/2469707237 shutdown_connections 2024-09-11T23:39:16.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.170+0000 7ff5b4169740 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c6718aff90 0x55c6718b2380 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.170+0000 7ff5b4169740 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718a0370 0x55c67189d440 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff59c014160 tx=0x7ff59c006960 comp rx=0 tx=0).stop 2024-09-11T23:39:16.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.171+0000 7ff5b4169740 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c671896b40 0x55c67189cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.171+0000 7ff5b4169740 1 --2- 172.21.15.133:0/2469707237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718955d0 0x55c67189fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.171+0000 7ff5b4169740 1 -- 172.21.15.133:0/2469707237 >> 172.21.15.133:0/2469707237 conn(0x55c67185e310 msgr2=0x55c671897930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:16.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.173+0000 7ff58affd640 0 client.25658 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:39:16.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.176+0000 7ff5b4169740 1 -- 172.21.15.133:0/2469707237 shutdown_connections 2024-09-11T23:39:16.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.176+0000 7ff5b4169740 1 -- 172.21.15.133:0/2469707237 wait complete. 2024-09-11T23:39:16.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.176+0000 7ff5b4169740 20 client.25658 trim_cache size 0 max 16384 2024-09-11T23:39:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.213+0000 7ff5b4169740 1 Processor -- start 2024-09-11T23:39:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.214+0000 7ff5b4169740 1 -- start start 2024-09-11T23:39:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.214+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718955d0 0x55c6718ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.214+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c671896b40 0x55c6718b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.215+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6718b8140 0x55c67189a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.215+0000 7ff5b4169740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1ab740 con 0x55c6718b8140 2024-09-11T23:39:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.215+0000 7ff5b4169740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1ad280 con 0x55c671896b40 2024-09-11T23:39:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.215+0000 7ff5b4169740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1abe10 con 0x55c6718955d0 2024-09-11T23:39:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.215+0000 7ff5ab7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c671896b40 0x55c6718b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.215+0000 7ff5ab7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c671896b40 0x55c6718b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47372/0 (socket says 172.21.15.133:47372) 2024-09-11T23:39:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.215+0000 7ff5ab7fe640 1 -- 172.21.15.133:0/2687689683 learned_addr learned my addr 172.21.15.133:0/2687689683 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:16.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.215+0000 7ff5aaffd640 1 --2- 172.21.15.133:0/2687689683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718955d0 0x55c6718ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.216+0000 7ff5aaffd640 1 --2- 172.21.15.133:0/2687689683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6718b8140 0x55c67189a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.216+0000 7ff5ab7fe640 1 -- 172.21.15.133:0/2687689683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718955d0 msgr2=0x55c6718ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.216+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/2687689683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718955d0 0x55c6718ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.216+0000 7ff5ab7fe640 1 -- 172.21.15.133:0/2687689683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6718b8140 msgr2=0x55c67189a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.216+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/2687689683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6718b8140 0x55c67189a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.216+0000 7ff5ab7fe640 1 -- 172.21.15.133:0/2687689683 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67189ffc0 con 0x55c671896b40 2024-09-11T23:39:16.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.217+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/2687689683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c671896b40 0x55c6718b7c00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400a4d0 tx=0x7ff5a400d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51e11ad5dbce9e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.218+0000 7ff5917fa640 1 -- 172.21.15.133:0/2687689683 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4018040 con 0x55c671896b40 2024-09-11T23:39:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.218+0000 7ff5917fa640 1 -- 172.21.15.133:0/2687689683 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a4014cf0 con 0x55c671896b40 2024-09-11T23:39:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.220+0000 7ff5b4169740 1 -- 172.21.15.133:0/2687689683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c671896b40 msgr2=0x55c6718b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.220+0000 7ff5b4169740 1 --2- 172.21.15.133:0/2687689683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c671896b40 0x55c6718b7c00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400a4d0 tx=0x7ff5a400d040 comp rx=0 tx=0).stop 2024-09-11T23:39:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.220+0000 7ff5b4169740 1 -- 172.21.15.133:0/2687689683 shutdown_connections 2024-09-11T23:39:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.220+0000 7ff5b4169740 1 --2- 172.21.15.133:0/2687689683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6718b8140 0x55c67189a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.220+0000 7ff5b4169740 1 --2- 172.21.15.133:0/2687689683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c671896b40 0x55c6718b7c00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.220+0000 7ff5b4169740 1 --2- 172.21.15.133:0/2687689683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718955d0 0x55c6718ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.220+0000 7ff5b4169740 1 -- 172.21.15.133:0/2687689683 >> 172.21.15.133:0/2687689683 conn(0x55c67185e310 msgr2=0x55c67189ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:16.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.222+0000 7ff5b4169740 1 -- 172.21.15.133:0/2687689683 shutdown_connections 2024-09-11T23:39:16.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.222+0000 7ff5b4169740 1 -- 172.21.15.133:0/2687689683 wait complete. 2024-09-11T23:39:16.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.223+0000 7ff5b4169740 1 -- start start 2024-09-11T23:39:16.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.223+0000 7ff5b4169740 1 Processor -- start 2024-09-11T23:39:16.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.224+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 0x55c6718a8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.224+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718b8140 0x55c6718a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.224+0000 7ff5b4169740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6718a9080 0x55c6718a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.224+0000 7ff5b4169740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1acbb0 con 0x55c6718a9080 2024-09-11T23:39:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.224+0000 7ff5b4169740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1ac4e0 con 0x55c6718955d0 2024-09-11T23:39:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.224+0000 7ff5b4169740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1a9740 con 0x55c6718b8140 2024-09-11T23:39:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.224+0000 7ff5b4169740 10 client.0 _ll_register_callbacks cb 0x7ff59c00bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:16.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.226+0000 7ff5aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 0x55c6718a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.226+0000 7ff5aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 0x55c6718a8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47374/0 (socket says 172.21.15.133:47374) 2024-09-11T23:39:16.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.226+0000 7ff5aaffd640 1 -- 172.21.15.133:0/1179831571 learned_addr learned my addr 172.21.15.133:0/1179831571 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:16.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.226+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718b8140 0x55c6718a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.227+0000 7ff5ab7fe640 1 -- 172.21.15.133:0/1179831571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 msgr2=0x55c6718a8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.227+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 0x55c6718a8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.227+0000 7ff5ab7fe640 1 -- 172.21.15.133:0/1179831571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6718a9080 msgr2=0x55c6718a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.227+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6718a9080 0x55c6718a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.227+0000 7ff5ab7fe640 1 -- 172.21.15.133:0/1179831571 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6718a5e00 con 0x55c6718b8140 2024-09-11T23:39:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.228+0000 7ff5ab7fe640 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718b8140 0x55c6718a8b40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55c67189ff90 tx=0x7ff5a4013f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fbb688e52aeeff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.231+0000 7ff58affd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:39:16.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.232+0000 7ff58affd640 1 -- 172.21.15.133:0/1179831571 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4018050 con 0x55c6718b8140 2024-09-11T23:39:16.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.232+0000 7ff5b4169740 1 -- 172.21.15.133:0/1179831571 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c671808860 con 0x55c6718b8140 2024-09-11T23:39:16.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.232+0000 7ff58affd640 1 -- 172.21.15.133:0/1179831571 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff5a4013870 con 0x55c6718b8140 2024-09-11T23:39:16.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.235+0000 7ff58affd640 1 -- 172.21.15.133:0/1179831571 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5a4013360 con 0x55c6718b8140 2024-09-11T23:39:16.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.235+0000 7ff5b4169740 10 client.25651 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:16.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.235+0000 7ff5b4169740 1 -- 172.21.15.133:0/1179831571 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c67191b730 con 0x55c6718b8140 2024-09-11T23:39:16.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.236+0000 7ff58affd640 1 -- 172.21.15.133:0/1179831571 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c671808860 con 0x55c6718b8140 2024-09-11T23:39:16.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.238+0000 7ff58affd640 1 -- 172.21.15.133:0/1179831571 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff5a4038050 con 0x55c6718b8140 2024-09-11T23:39:16.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.239+0000 7ff5b4169740 10 client.25651 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:16.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.239+0000 7ff5b4169740 10 client.25651 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:39:16.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.239+0000 7ff5b4169740 20 client.25651 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:16.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.239+0000 7ff5b4169740 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c67191bc40 0x55c67191e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.239+0000 7ff5b4169740 4 client.25651 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:39:16.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.239+0000 7ff5b4169740 1 -- 172.21.15.133:0/1179831571 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c671802d70 con 0x55c67191bc40 2024-09-11T23:39:16.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.241+0000 7ff5aaffd640 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c67191bc40 0x55c67191e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.245+0000 7ff5aaffd640 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c67191bc40 0x55c67191e030 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=12293259d4acf04c server_cookie=dd644efcd36dbfb7 in_seq=0 out_seq=0 2024-09-11T23:39:16.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.245+0000 7ff58affd640 10 client.25651 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:39:16.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.246+0000 7ff58affd640 1 -- 172.21.15.133:0/1179831571 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c671802d70 con 0x55c67191bc40 2024-09-11T23:39:16.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.246+0000 7ff58affd640 10 client.25651 handle_command_reply: tid=0 2024-09-11T23:39:16.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.246+0000 7ff5b4169740 1 client.25651 shutdown 2024-09-11T23:39:16.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.250+0000 7ff5b4169740 1 -- 172.21.15.133:0/1179831571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718b8140 msgr2=0x55c6718a8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.250+0000 7ff5b4169740 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718b8140 0x55c6718a8b40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55c67189ff90 tx=0x7ff5a4013f40 comp rx=0 tx=0).stop 2024-09-11T23:39:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.251+0000 7ff5b4169740 1 -- 172.21.15.133:0/1179831571 shutdown_connections 2024-09-11T23:39:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.251+0000 7ff5b4169740 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c67191bc40 0x55c67191e030 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.251+0000 7ff5b4169740 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6718a9080 0x55c6718a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.251+0000 7ff5b4169740 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6718b8140 0x55c6718a8b40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.251+0000 7ff5b4169740 1 --2- 172.21.15.133:0/1179831571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6718955d0 0x55c6718a8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.251+0000 7ff5b4169740 1 -- 172.21.15.133:0/1179831571 >> 172.21.15.133:0/1179831571 conn(0x55c67185e310 msgr2=0x55c671890500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.252+0000 7ff58affd640 0 client.25651 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:39:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.252+0000 7ff5b4169740 1 -- 172.21.15.133:0/1179831571 shutdown_connections 2024-09-11T23:39:16.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.252+0000 7ff5b4169740 1 -- 172.21.15.133:0/1179831571 wait complete. 2024-09-11T23:39:16.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.252+0000 7ff5b4169740 20 client.25651 trim_cache size 0 max 16384 2024-09-11T23:39:16.254 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:39:16.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.257+0000 7ff5b1d90640 1 -- 172.21.15.133:0/1671989909 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5940b3ed0 msgr2=0x7ff5940b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.257+0000 7ff5b1d90640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5940b3ed0 0x7ff5940b6390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0005ab0 tx=0x7ff5a00314b0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.257+0000 7ff5b1d90640 1 -- 172.21.15.133:0/1671989909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5ac109120 msgr2=0x7ff5ac10f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.257+0000 7ff5b1d90640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5ac109120 0x7ff5ac10f8e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009230 tx=0x7ff5a00042f0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.257+0000 7ff5b1d90640 1 -- 172.21.15.133:0/1671989909 shutdown_connections 2024-09-11T23:39:16.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.257+0000 7ff5b1d90640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5940b3ed0 0x7ff5940b6390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.258+0000 7ff5b1d90640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5ac109120 0x7ff5ac10f8e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.258+0000 7ff5b1d90640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac1049a0 0x7ff5ac19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.258+0000 7ff5b1d90640 1 --2- 172.21.15.133:0/1671989909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5ac103ff0 0x7ff5ac19c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.258+0000 7ff5b1d90640 1 -- 172.21.15.133:0/1671989909 >> 172.21.15.133:0/1671989909 conn(0x7ff5ac0ffab0 msgr2=0x7ff5ac07a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:16.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.259+0000 7ff5b1d90640 1 -- 172.21.15.133:0/1671989909 shutdown_connections 2024-09-11T23:39:16.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:16.259+0000 7ff5b1d90640 1 -- 172.21.15.133:0/1671989909 wait complete. 2024-09-11T23:39:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.361+0000 7f4317ff6640 1 -- 172.21.15.133:0/2719096285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43101008d0 msgr2=0x7f4310100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.361+0000 7f4317ff6640 1 --2- 172.21.15.133:0/2719096285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43101008d0 0x7f4310100cb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f430c009890 tx=0x7f430c02a440 comp rx=0 tx=0).stop 2024-09-11T23:39:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.367+0000 7f4317ff6640 1 -- 172.21.15.133:0/2719096285 shutdown_connections 2024-09-11T23:39:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.367+0000 7f4317ff6640 1 --2- 172.21.15.133:0/2719096285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4310105d00 0x7f4310106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.367+0000 7f4317ff6640 1 --2- 172.21.15.133:0/2719096285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43101011f0 0x7f43101056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.367+0000 7f4317ff6640 1 --2- 172.21.15.133:0/2719096285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43101008d0 0x7f4310100cb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.367+0000 7f4317ff6640 1 -- 172.21.15.133:0/2719096285 >> 172.21.15.133:0/2719096285 conn(0x7f43100777c0 msgr2=0x7f43100feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.372+0000 7f4317ff6640 1 -- 172.21.15.133:0/2719096285 shutdown_connections 2024-09-11T23:39:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.373+0000 7f4317ff6640 1 -- 172.21.15.133:0/2719096285 wait complete. 2024-09-11T23:39:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.374+0000 7f4317ff6640 1 Processor -- start 2024-09-11T23:39:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.375+0000 7f4317ff6640 1 -- start start 2024-09-11T23:39:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.376+0000 7f4317ff6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43101008d0 0x7f43101a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.376+0000 7f4317ff6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43101011f0 0x7f43101a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.376+0000 7f4317ff6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4310105d00 0x7f431019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.376+0000 7f4317ff6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f43101183b0 con 0x7f4310105d00 2024-09-11T23:39:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.376+0000 7f4317ff6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4310118230 con 0x7f43101008d0 2024-09-11T23:39:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.376+0000 7f4317ff6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4310118530 con 0x7f43101011f0 2024-09-11T23:39:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.378+0000 7f4316593640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4310105d00 0x7f431019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.378+0000 7f4316593640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4310105d00 0x7f431019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39206/0 (socket says 172.21.15.133:39206) 2024-09-11T23:39:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.378+0000 7f4316593640 1 -- 172.21.15.133:0/4285372884 learned_addr learned my addr 172.21.15.133:0/4285372884 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.379+0000 7f4315591640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43101011f0 0x7f43101a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.380+0000 7f4315d92640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43101008d0 0x7f43101a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.385+0000 7f4316593640 1 -- 172.21.15.133:0/4285372884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43101011f0 msgr2=0x7f43101a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.385+0000 7f4316593640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43101011f0 0x7f43101a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.385+0000 7f4316593640 1 -- 172.21.15.133:0/4285372884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43101008d0 msgr2=0x7f43101a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.385+0000 7f4316593640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43101008d0 0x7f43101a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.385+0000 7f4316593640 1 -- 172.21.15.133:0/4285372884 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431019efb0 con 0x7f4310105d00 2024-09-11T23:39:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.389+0000 7f4316593640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4310105d00 0x7f431019e7e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4308009fd0 tx=0x7f430800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.393+0000 7f4306ffd640 1 -- 172.21.15.133:0/4285372884 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308004b20 con 0x7f4310105d00 2024-09-11T23:39:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.393+0000 7f4306ffd640 1 -- 172.21.15.133:0/4285372884 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4308003e50 con 0x7f4310105d00 2024-09-11T23:39:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.393+0000 7f4306ffd640 1 -- 172.21.15.133:0/4285372884 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430800b570 con 0x7f4310105d00 2024-09-11T23:39:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.395+0000 7f4317ff6640 1 -- 172.21.15.133:0/4285372884 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431019f2a0 con 0x7f4310105d00 2024-09-11T23:39:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.395+0000 7f4317ff6640 1 -- 172.21.15.133:0/4285372884 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43101ab4c0 con 0x7f4310105d00 2024-09-11T23:39:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.406+0000 7f4306ffd640 1 -- 172.21.15.133:0/4285372884 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f430801d020 con 0x7f4310105d00 2024-09-11T23:39:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.407+0000 7f4306ffd640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f42fc0a2e80 0x7f42fc0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.416+0000 7f4306ffd640 1 -- 172.21.15.133:0/4285372884 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f43080998c0 con 0x7f4310105d00 2024-09-11T23:39:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.418+0000 7f431ae6c740 1 Processor -- start 2024-09-11T23:39:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.418+0000 7f431ae6c740 1 -- start start 2024-09-11T23:39:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.419+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 0x5578041cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.419+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.420+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041d9e80 0x5578041dc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.420+0000 7f4315d92640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 0x5578041cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.420+0000 7f4315d92640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 0x5578041cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53098/0 (socket says 172.21.15.133:53098) 2024-09-11T23:39:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.429+0000 7f4315591640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.429+0000 7f4316593640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041d9e80 0x5578041dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.429+0000 7f4315591640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47400/0 (socket says 172.21.15.133:47400) 2024-09-11T23:39:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.429+0000 7f4315d92640 1 -- 172.21.15.133:0/680115824 learned_addr learned my addr 172.21.15.133:0/680115824 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.430+0000 7f431ae6c740 1 -- 172.21.15.133:0/680115824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5578041332f0 con 0x5578041d9e80 2024-09-11T23:39:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.430+0000 7f431ae6c740 1 -- 172.21.15.133:0/680115824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5578040db270 con 0x5578041cf5e0 2024-09-11T23:39:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.430+0000 7f431ae6c740 1 -- 172.21.15.133:0/680115824 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55780412f2b0 con 0x5578041cecc0 2024-09-11T23:39:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.430+0000 7f4315d92640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f42fc0a2e80 0x7f42fc0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.431+0000 7f4315d92640 1 -- 172.21.15.133:0/680115824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 msgr2=0x5578041d9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.431+0000 7f4315d92640 1 --2- 172.21.15.133:0/680115824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.431+0000 7f4315d92640 1 -- 172.21.15.133:0/680115824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041d9e80 msgr2=0x5578041dc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.431+0000 7f4315d92640 1 --2- 172.21.15.133:0/680115824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041d9e80 0x5578041dc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.431+0000 7f4315d92640 1 -- 172.21.15.133:0/680115824 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578040da340 con 0x5578041cecc0 2024-09-11T23:39:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.432+0000 7f4315d92640 1 --2- 172.21.15.133:0/680115824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 0x5578041cf0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f430c0095f0 tx=0x7f430c030760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=955a475ade9e960c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.433+0000 7f42fbfff640 1 -- 172.21.15.133:0/680115824 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c050070 con 0x5578041cecc0 2024-09-11T23:39:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.433+0000 7f42fbfff640 1 -- 172.21.15.133:0/680115824 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430c005040 con 0x5578041cecc0 2024-09-11T23:39:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.434+0000 7f42fbfff640 1 -- 172.21.15.133:0/680115824 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c004990 con 0x5578041cecc0 2024-09-11T23:39:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.434+0000 7f431ae6c740 1 -- 172.21.15.133:0/680115824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 msgr2=0x5578041cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.434+0000 7f431ae6c740 1 --2- 172.21.15.133:0/680115824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 0x5578041cf0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f430c0095f0 tx=0x7f430c030760 comp rx=0 tx=0).stop 2024-09-11T23:39:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.435+0000 7f431ae6c740 1 -- 172.21.15.133:0/680115824 shutdown_connections 2024-09-11T23:39:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.435+0000 7f431ae6c740 1 --2- 172.21.15.133:0/680115824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041d9e80 0x5578041dc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.435+0000 7f431ae6c740 1 --2- 172.21.15.133:0/680115824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.435+0000 7f431ae6c740 1 --2- 172.21.15.133:0/680115824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 0x5578041cf0a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.435+0000 7f431ae6c740 1 -- 172.21.15.133:0/680115824 >> 172.21.15.133:0/680115824 conn(0x5578041cb230 msgr2=0x5578041cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.438+0000 7f431ae6c740 1 -- 172.21.15.133:0/680115824 shutdown_connections 2024-09-11T23:39:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.439+0000 7f4315d92640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f42fc0a2e80 0x7f42fc0a5340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f430c04d000 tx=0x7f430c04e000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.439+0000 7f431ae6c740 1 -- 172.21.15.133:0/680115824 wait complete. 2024-09-11T23:39:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.440+0000 7f431ae6c740 1 -- start start 2024-09-11T23:39:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.440+0000 7f431ae6c740 1 Processor -- start 2024-09-11T23:39:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.441+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 0x5578041d6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.441+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.441+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041d9e80 0x5578041d7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.441+0000 7f431ae6c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5578041dcaa0 con 0x5578041d9e80 2024-09-11T23:39:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.441+0000 7f431ae6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5578040bc290 con 0x5578041cf5e0 2024-09-11T23:39:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.441+0000 7f431ae6c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5578041dcc20 con 0x5578041cecc0 2024-09-11T23:39:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.442+0000 7f431ae6c740 10 client.0 _ll_register_callbacks cb 0x5578040ed140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.443+0000 7f4316593640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041d9e80 0x5578041d7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.443+0000 7f4315591640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.443+0000 7f4315591640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47406/0 (socket says 172.21.15.133:47406) 2024-09-11T23:39:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.443+0000 7f4315591640 1 -- 172.21.15.133:0/1123161043 learned_addr learned my addr 172.21.15.133:0/1123161043 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.443+0000 7f4315d92640 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 0x5578041d6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.444+0000 7f4315591640 1 -- 172.21.15.133:0/1123161043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 msgr2=0x5578041d6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.444+0000 7f4315591640 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 0x5578041d6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.444+0000 7f4315591640 1 -- 172.21.15.133:0/1123161043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041d9e80 msgr2=0x5578041d7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.444+0000 7f4315591640 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041d9e80 0x5578041d7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.444+0000 7f4315591640 1 -- 172.21.15.133:0/1123161043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557803e46210 con 0x5578041cf5e0 2024-09-11T23:39:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.445+0000 7f4315591640 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d7420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4300011570 tx=0x7f4300011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47260b8d88ea0efb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.445+0000 7f42f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.445+0000 7f42f97fa640 1 -- 172.21.15.133:0/1123161043 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300013670 con 0x5578041cf5e0 2024-09-11T23:39:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.446+0000 7f431ae6c740 1 -- 172.21.15.133:0/1123161043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430c005040 con 0x5578041cf5e0 2024-09-11T23:39:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.446+0000 7f42f97fa640 1 -- 172.21.15.133:0/1123161043 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f4300013810 con 0x5578041cf5e0 2024-09-11T23:39:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.446+0000 7f42f97fa640 1 -- 172.21.15.133:0/1123161043 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4300013a10 con 0x5578041cf5e0 2024-09-11T23:39:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.447+0000 7f431ae6c740 10 client.25676 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.447+0000 7f431ae6c740 1 -- 172.21.15.133:0/1123161043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557803f12620 con 0x5578041cf5e0 2024-09-11T23:39:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.448+0000 7f42f97fa640 1 -- 172.21.15.133:0/1123161043 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430c005040 con 0x5578041cf5e0 2024-09-11T23:39:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.448+0000 7f42f97fa640 1 -- 172.21.15.133:0/1123161043 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4300025020 con 0x5578041cf5e0 2024-09-11T23:39:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.452+0000 7f431ae6c740 10 client.25676 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.452+0000 7f431ae6c740 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.452+0000 7f431ae6c740 20 client.25676 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.452+0000 7f431ae6c740 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5578041fb050 0x5578041fd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.452+0000 7f431ae6c740 4 client.25676 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.452+0000 7f431ae6c740 1 -- 172.21.15.133:0/1123161043 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557804140a20 con 0x5578041fb050 2024-09-11T23:39:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.453+0000 7f4315d92640 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5578041fb050 0x5578041fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.454+0000 7f42f97fa640 10 client.25676 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.454+0000 7f4315d92640 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5578041fb050 0x5578041fd440 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e84075581fd026 server_cookie=d1080a2509874499 in_seq=0 out_seq=0 2024-09-11T23:39:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.456+0000 7f42f97fa640 1 -- 172.21.15.133:0/1123161043 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557804140a20 con 0x5578041fb050 2024-09-11T23:39:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.456+0000 7f42f97fa640 10 client.25676 handle_command_reply: tid=0 2024-09-11T23:39:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.464+0000 7f431ae6c740 1 client.25676 shutdown 2024-09-11T23:39:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.465+0000 7f431ae6c740 1 -- 172.21.15.133:0/1123161043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 msgr2=0x5578041d7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.465+0000 7f431ae6c740 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d7420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4300011570 tx=0x7f4300011a30 comp rx=0 tx=0).stop 2024-09-11T23:39:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.465+0000 7f431ae6c740 1 -- 172.21.15.133:0/1123161043 shutdown_connections 2024-09-11T23:39:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.465+0000 7f431ae6c740 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5578041fb050 0x5578041fd440 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.465+0000 7f431ae6c740 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041d9e80 0x5578041d7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.465+0000 7f431ae6c740 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d7420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.465+0000 7f431ae6c740 1 --2- 172.21.15.133:0/1123161043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cecc0 0x5578041d6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.465+0000 7f431ae6c740 1 -- 172.21.15.133:0/1123161043 >> 172.21.15.133:0/1123161043 conn(0x5578041cb230 msgr2=0x5578041d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.466+0000 7f42f97fa640 0 client.25676 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.466+0000 7f431ae6c740 1 -- 172.21.15.133:0/1123161043 shutdown_connections 2024-09-11T23:39:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.466+0000 7f431ae6c740 1 -- 172.21.15.133:0/1123161043 wait complete. 2024-09-11T23:39:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.466+0000 7f431ae6c740 20 client.25676 trim_cache size 0 max 16384 2024-09-11T23:39:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.504+0000 7f431ae6c740 1 Processor -- start 2024-09-11T23:39:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.504+0000 7f431ae6c740 1 -- start start 2024-09-11T23:39:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.505+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041cecc0 0x5578041d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.505+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.505+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041d9e80 0x557804202d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.505+0000 7f431ae6c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4310074080 con 0x5578041cecc0 2024-09-11T23:39:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.505+0000 7f431ae6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43101acef0 con 0x5578041cf5e0 2024-09-11T23:39:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.505+0000 7f431ae6c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43101aba80 con 0x5578041d9e80 2024-09-11T23:39:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.505+0000 7f4316593640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041d9e80 0x557804202d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.505+0000 7f4316593640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041d9e80 0x557804202d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53116/0 (socket says 172.21.15.133:53116) 2024-09-11T23:39:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.506+0000 7f4316593640 1 -- 172.21.15.133:0/990561254 learned_addr learned my addr 172.21.15.133:0/990561254 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.510+0000 7f4316593640 1 -- 172.21.15.133:0/990561254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 msgr2=0x5578041d7790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:39:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.510+0000 7f4316593640 1 --2- 172.21.15.133:0/990561254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d7790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.510+0000 7f4316593640 1 -- 172.21.15.133:0/990561254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041cecc0 msgr2=0x5578041d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.510+0000 7f4316593640 1 --2- 172.21.15.133:0/990561254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041cecc0 0x5578041d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.510+0000 7f4316593640 1 -- 172.21.15.133:0/990561254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578041ceaf0 con 0x5578041d9e80 2024-09-11T23:39:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.510+0000 7f4316593640 1 --2- 172.21.15.133:0/990561254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041d9e80 0x557804202d30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f430809b7b0 tx=0x7f4308021040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dec3b8c70c24f529 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.511+0000 7f42fb7fe640 1 -- 172.21.15.133:0/990561254 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430801a040 con 0x5578041d9e80 2024-09-11T23:39:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.511+0000 7f42fb7fe640 1 -- 172.21.15.133:0/990561254 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430809fbd0 con 0x5578041d9e80 2024-09-11T23:39:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.512+0000 7f431ae6c740 1 -- 172.21.15.133:0/990561254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041d9e80 msgr2=0x557804202d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.512+0000 7f431ae6c740 1 --2- 172.21.15.133:0/990561254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041d9e80 0x557804202d30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f430809b7b0 tx=0x7f4308021040 comp rx=0 tx=0).stop 2024-09-11T23:39:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.516+0000 7f431ae6c740 1 -- 172.21.15.133:0/990561254 shutdown_connections 2024-09-11T23:39:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.516+0000 7f431ae6c740 1 --2- 172.21.15.133:0/990561254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041d9e80 0x557804202d30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.516+0000 7f431ae6c740 1 --2- 172.21.15.133:0/990561254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041cf5e0 0x5578041d7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.516+0000 7f431ae6c740 1 --2- 172.21.15.133:0/990561254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041cecc0 0x5578041d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.516+0000 7f431ae6c740 1 -- 172.21.15.133:0/990561254 >> 172.21.15.133:0/990561254 conn(0x5578041cb230 msgr2=0x5578041d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.522+0000 7f431ae6c740 1 -- 172.21.15.133:0/990561254 shutdown_connections 2024-09-11T23:39:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.524+0000 7f431ae6c740 1 -- 172.21.15.133:0/990561254 wait complete. 2024-09-11T23:39:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.525+0000 7f431ae6c740 1 -- start start 2024-09-11T23:39:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.525+0000 7f431ae6c740 1 Processor -- start 2024-09-11T23:39:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.526+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041cecc0 0x5578041ddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.526+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cf5e0 0x5578041de1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.526+0000 7f431ae6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041d9e80 0x5578041de730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.526+0000 7f431ae6c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f43101ac820 con 0x5578041cecc0 2024-09-11T23:39:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.526+0000 7f431ae6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43101ac150 con 0x5578041d9e80 2024-09-11T23:39:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.527+0000 7f431ae6c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43101056b0 con 0x5578041cf5e0 2024-09-11T23:39:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.527+0000 7f431ae6c740 10 client.0 _ll_register_callbacks cb 0x7f43000094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.527+0000 7f4316593640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041d9e80 0x5578041de730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.527+0000 7f4316593640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041d9e80 0x5578041de730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47418/0 (socket says 172.21.15.133:47418) 2024-09-11T23:39:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.527+0000 7f4316593640 1 -- 172.21.15.133:0/3612364391 learned_addr learned my addr 172.21.15.133:0/3612364391 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.529+0000 7f4316593640 1 -- 172.21.15.133:0/3612364391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cf5e0 msgr2=0x5578041de1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:39:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.529+0000 7f4316593640 1 --2- 172.21.15.133:0/3612364391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cf5e0 0x5578041de1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.529+0000 7f4316593640 1 -- 172.21.15.133:0/3612364391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041cecc0 msgr2=0x5578041ddcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.529+0000 7f4316593640 1 --2- 172.21.15.133:0/3612364391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041cecc0 0x5578041ddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.529+0000 7f4316593640 1 -- 172.21.15.133:0/3612364391 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557804275d90 con 0x5578041d9e80 2024-09-11T23:39:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.534+0000 7f4316593640 1 --2- 172.21.15.133:0/3612364391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041d9e80 0x5578041de730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4308009a20 tx=0x7f43080a4540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7d843188edc5c79 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.534+0000 7f42f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.534+0000 7f42f97fa640 1 -- 172.21.15.133:0/3612364391 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43080b8070 con 0x5578041d9e80 2024-09-11T23:39:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.534+0000 7f42f97fa640 1 -- 172.21.15.133:0/3612364391 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f430809cb50 con 0x5578041d9e80 2024-09-11T23:39:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.535+0000 7f431ae6c740 1 -- 172.21.15.133:0/3612364391 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430809fbd0 con 0x5578041d9e80 2024-09-11T23:39:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.535+0000 7f42f97fa640 1 -- 172.21.15.133:0/3612364391 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f430801a030 con 0x5578041d9e80 2024-09-11T23:39:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.539+0000 7f431ae6c740 10 client.25679 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.539+0000 7f431ae6c740 1 -- 172.21.15.133:0/3612364391 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557804277060 con 0x5578041d9e80 2024-09-11T23:39:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.540+0000 7f42f97fa640 1 -- 172.21.15.133:0/3612364391 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430809fbd0 con 0x5578041d9e80 2024-09-11T23:39:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.540+0000 7f42f97fa640 1 -- 172.21.15.133:0/3612364391 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f430800b040 con 0x5578041d9e80 2024-09-11T23:39:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.542+0000 7f431ae6c740 10 client.25679 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.542+0000 7f431ae6c740 10 client.25679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.542+0000 7f431ae6c740 20 client.25679 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.542+0000 7f431ae6c740 1 --2- 172.21.15.133:0/3612364391 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5578042776d0 0x557804279af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.542+0000 7f431ae6c740 4 client.25679 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:39:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.542+0000 7f431ae6c740 1 -- 172.21.15.133:0/3612364391 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55780413ad70 con 0x5578042776d0 2024-09-11T23:39:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.544+0000 7f4315d92640 1 --2- 172.21.15.133:0/3612364391 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5578042776d0 0x557804279af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.558+0000 7f4315d92640 1 --2- 172.21.15.133:0/3612364391 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5578042776d0 0x557804279af0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e093ba370301b165 server_cookie=789cd0fb1c469d84 in_seq=0 out_seq=0 2024-09-11T23:39:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:16.559+0000 7f42f97fa640 10 client.25679 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.243+0000 7f42f97fa640 1 -- 172.21.15.133:0/3612364391 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55780413ad70 con 0x5578042776d0 2024-09-11T23:39:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.243+0000 7f42f97fa640 10 client.25679 handle_command_reply: tid=0 2024-09-11T23:39:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.243+0000 7f431ae6c740 1 client.25679 shutdown 2024-09-11T23:39:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.247+0000 7f431ae6c740 1 -- 172.21.15.133:0/3612364391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041d9e80 msgr2=0x5578041de730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.247+0000 7f431ae6c740 1 --2- 172.21.15.133:0/3612364391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041d9e80 0x5578041de730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4308009a20 tx=0x7f43080a4540 comp rx=0 tx=0).stop 2024-09-11T23:39:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.248+0000 7f431ae6c740 1 -- 172.21.15.133:0/3612364391 shutdown_connections 2024-09-11T23:39:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.248+0000 7f431ae6c740 1 --2- 172.21.15.133:0/3612364391 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5578042776d0 0x557804279af0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.248+0000 7f431ae6c740 1 --2- 172.21.15.133:0/3612364391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578041d9e80 0x5578041de730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.248+0000 7f431ae6c740 1 --2- 172.21.15.133:0/3612364391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578041cf5e0 0x5578041de1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.248+0000 7f431ae6c740 1 --2- 172.21.15.133:0/3612364391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578041cecc0 0x5578041ddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.248+0000 7f431ae6c740 1 -- 172.21.15.133:0/3612364391 >> 172.21.15.133:0/3612364391 conn(0x5578041cb230 msgr2=0x5578041d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.248+0000 7f42f97fa640 0 client.25679 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.250+0000 7f431ae6c740 1 -- 172.21.15.133:0/3612364391 shutdown_connections 2024-09-11T23:39:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.251+0000 7f431ae6c740 1 -- 172.21.15.133:0/3612364391 wait complete. 2024-09-11T23:39:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.251+0000 7f431ae6c740 20 client.25679 trim_cache size 0 max 16384 2024-09-11T23:39:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:39:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:39:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:39:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:39:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.257+0000 7f4317ff6640 1 -- 172.21.15.133:0/4285372884 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f42fc0a2e80 msgr2=0x7f42fc0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.257+0000 7f4317ff6640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f42fc0a2e80 0x7f42fc0a5340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f430c04d000 tx=0x7f430c04e000 comp rx=0 tx=0).stop 2024-09-11T23:39:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.257+0000 7f4317ff6640 1 -- 172.21.15.133:0/4285372884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4310105d00 msgr2=0x7f431019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.257+0000 7f4317ff6640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4310105d00 0x7f431019e7e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4308009fd0 tx=0x7f430800c740 comp rx=0 tx=0).stop 2024-09-11T23:39:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.271+0000 7f4317ff6640 1 -- 172.21.15.133:0/4285372884 shutdown_connections 2024-09-11T23:39:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.271+0000 7f4317ff6640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f42fc0a2e80 0x7f42fc0a5340 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.271+0000 7f4317ff6640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4310105d00 0x7f431019e7e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.271+0000 7f4317ff6640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43101011f0 0x7f43101a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.271+0000 7f4317ff6640 1 --2- 172.21.15.133:0/4285372884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43101008d0 0x7f43101a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.271+0000 7f4317ff6640 1 -- 172.21.15.133:0/4285372884 >> 172.21.15.133:0/4285372884 conn(0x7f43100777c0 msgr2=0x7f431010f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.271+0000 7f4317ff6640 1 -- 172.21.15.133:0/4285372884 shutdown_connections 2024-09-11T23:39:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:17.272+0000 7f4317ff6640 1 -- 172.21.15.133:0/4285372884 wait complete. 2024-09-11T23:39:17.455 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:39:18.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:18 smithi178 ceph-mon[35882]: pgmap v78: 97 pgs: 97 active+clean; 12 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 329 MiB/s wr, 2.37k op/s 2024-09-11T23:39:18.548 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:18.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:18 smithi138 ceph-mon[35142]: pgmap v78: 97 pgs: 97 active+clean; 12 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 329 MiB/s wr, 2.37k op/s 2024-09-11T23:39:18.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:18 smithi133 ceph-mon[31430]: pgmap v78: 97 pgs: 97 active+clean; 12 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 329 MiB/s wr, 2.37k op/s 2024-09-11T23:39:19.456 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:19 smithi133 ceph-mon[31430]: pgmap v79: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 585 MiB/s wr, 4.45k op/s 2024-09-11T23:39:19.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.456+0000 7fafc17af640 1 -- 172.21.15.133:0/1877649775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafbc109450 msgr2=0x7fafbc110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:19.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.456+0000 7fafc17af640 1 --2- 172.21.15.133:0/1877649775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafbc109450 0x7fafbc110250 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fafac009ca0 tx=0x7fafac02a8d0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.457+0000 7fafc17af640 1 -- 172.21.15.133:0/1877649775 shutdown_connections 2024-09-11T23:39:19.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.457+0000 7fafc17af640 1 --2- 172.21.15.133:0/1877649775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc11b140 0x7fafbc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.457+0000 7fafc17af640 1 --2- 172.21.15.133:0/1877649775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafbc109830 0x7fafbc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.457+0000 7fafc17af640 1 --2- 172.21.15.133:0/1877649775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafbc109450 0x7fafbc110250 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.457+0000 7fafc17af640 1 -- 172.21.15.133:0/1877649775 >> 172.21.15.133:0/1877649775 conn(0x7fafbc06e140 msgr2=0x7fafbc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:19.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.460+0000 7fafc17af640 1 -- 172.21.15.133:0/1877649775 shutdown_connections 2024-09-11T23:39:19.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.463+0000 7fafc17af640 1 -- 172.21.15.133:0/1877649775 wait complete. 2024-09-11T23:39:19.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.465+0000 7fafc17af640 1 Processor -- start 2024-09-11T23:39:19.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.466+0000 7fafc17af640 1 -- start start 2024-09-11T23:39:19.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.466+0000 7fafc17af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc109450 0x7fafbc1a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.467+0000 7fafc17af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafbc109830 0x7fafbc1a7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.467+0000 7fafc17af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafbc11b140 0x7fafbc1abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.467+0000 7fafc17af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc120eb0 con 0x7fafbc109830 2024-09-11T23:39:19.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.467+0000 7fafc17af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc120d30 con 0x7fafbc109450 2024-09-11T23:39:19.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.467+0000 7fafc17af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc121030 con 0x7fafbc11b140 2024-09-11T23:39:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.468+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafbc11b140 0x7fafbc1abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.468+0000 7fafbaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc109450 0x7fafbc1a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.468+0000 7fafbaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc109450 0x7fafbc1a7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47442/0 (socket says 172.21.15.133:47442) 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.468+0000 7fafbaffd640 1 -- 172.21.15.133:0/3291234410 learned_addr learned my addr 172.21.15.133:0/3291234410 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.469+0000 7fafbaffd640 1 -- 172.21.15.133:0/3291234410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafbc11b140 msgr2=0x7fafbc1abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.469+0000 7fafbaffd640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafbc11b140 0x7fafbc1abd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.469+0000 7fafbaffd640 1 -- 172.21.15.133:0/3291234410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafbc109830 msgr2=0x7fafbc1a7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.469+0000 7fafbaffd640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafbc109830 0x7fafbc1a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.469+0000 7fafbaffd640 1 -- 172.21.15.133:0/3291234410 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc1ac400 con 0x7fafbc109450 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.471+0000 7fafbaffd640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc109450 0x7fafbc1a7450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fafac02aec0 tx=0x7fafac031460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.476+0000 7fafb3fff640 1 -- 172.21.15.133:0/3291234410 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac031890 con 0x7fafbc109450 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.477+0000 7fafb3fff640 1 -- 172.21.15.133:0/3291234410 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafac004520 con 0x7fafbc109450 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.477+0000 7fafc17af640 1 -- 172.21.15.133:0/3291234410 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc1ac690 con 0x7fafbc109450 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.477+0000 7fafc17af640 1 -- 172.21.15.133:0/3291234410 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc1b3fd0 con 0x7fafbc109450 2024-09-11T23:39:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.477+0000 7fafb3fff640 1 -- 172.21.15.133:0/3291234410 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac02fe10 con 0x7fafbc109450 2024-09-11T23:39:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.485+0000 7fafb3fff640 1 -- 172.21.15.133:0/3291234410 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fafac0046d0 con 0x7fafbc109450 2024-09-11T23:39:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.493+0000 7fafb3fff640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafa80a2f00 0x7fafa80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.494+0000 7fafb3fff640 1 -- 172.21.15.133:0/3291234410 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fafac0f07e0 con 0x7fafbc109450 2024-09-11T23:39:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.495+0000 7fafc3b88740 1 Processor -- start 2024-09-11T23:39:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.496+0000 7fafc3b88740 1 -- start start 2024-09-11T23:39:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.496+0000 7fafba7fc640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafa80a2f00 0x7fafa80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.498+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 0x564e24fdc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.498+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fdc5d0 0x564e24fdca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.498+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 0x564e24fe9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.498+0000 7fafc3b88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564e24f422f0 con 0x564e24fdc5d0 2024-09-11T23:39:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.498+0000 7fafc3b88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564e24ee9cf0 con 0x564e24fddb40 2024-09-11T23:39:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.498+0000 7fafc3b88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564e24f3e2b0 con 0x564e24fe7370 2024-09-11T23:39:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.499+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 0x564e24fe9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.499+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 0x564e24fe9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53140/0 (socket says 172.21.15.133:53140) 2024-09-11T23:39:19.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.499+0000 7fafbb7fe640 1 -- 172.21.15.133:0/983096601 learned_addr learned my addr 172.21.15.133:0/983096601 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:19.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.499+0000 7fafbaffd640 1 --2- 172.21.15.133:0/983096601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 0x564e24fdc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.503+0000 7fafbb7fe640 1 -- 172.21.15.133:0/983096601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 msgr2=0x564e24fdc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:19.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.503+0000 7fafbb7fe640 1 --2- 172.21.15.133:0/983096601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 0x564e24fdc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.503+0000 7fafbb7fe640 1 -- 172.21.15.133:0/983096601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fdc5d0 msgr2=0x564e24fdca10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:39:19.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.503+0000 7fafba7fc640 1 --2- 172.21.15.133:0/983096601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fdc5d0 0x564e24fdca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.504+0000 7fafbb7fe640 1 --2- 172.21.15.133:0/983096601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fdc5d0 0x564e24fdca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.504+0000 7fafbb7fe640 1 -- 172.21.15.133:0/983096601 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e24ece3a0 con 0x564e24fe7370 2024-09-11T23:39:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.505+0000 7fafbb7fe640 1 --2- 172.21.15.133:0/983096601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 0x564e24fe9760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fafa4001690 tx=0x7fafa4001b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81a566e11dd134 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.505+0000 7fafb17fa640 1 -- 172.21.15.133:0/983096601 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4016030 con 0x564e24fe7370 2024-09-11T23:39:19.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.506+0000 7fafb17fa640 1 -- 172.21.15.133:0/983096601 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa4006e40 con 0x564e24fe7370 2024-09-11T23:39:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.506+0000 7fafb17fa640 1 -- 172.21.15.133:0/983096601 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4007520 con 0x564e24fe7370 2024-09-11T23:39:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.507+0000 7fafc3b88740 1 -- 172.21.15.133:0/983096601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 msgr2=0x564e24fe9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.507+0000 7fafc3b88740 1 --2- 172.21.15.133:0/983096601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 0x564e24fe9760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fafa4001690 tx=0x7fafa4001b50 comp rx=0 tx=0).stop 2024-09-11T23:39:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.510+0000 7fafba7fc640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafa80a2f00 0x7fafa80a53c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fafbc11d8c0 tx=0x7fafb4009d20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:19.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.513+0000 7fafc3b88740 1 -- 172.21.15.133:0/983096601 shutdown_connections 2024-09-11T23:39:19.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.513+0000 7fafc3b88740 1 --2- 172.21.15.133:0/983096601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 0x564e24fe9760 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.513+0000 7fafc3b88740 1 --2- 172.21.15.133:0/983096601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fdc5d0 0x564e24fdca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.513+0000 7fafc3b88740 1 --2- 172.21.15.133:0/983096601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 0x564e24fdc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.513+0000 7fafc3b88740 1 -- 172.21.15.133:0/983096601 >> 172.21.15.133:0/983096601 conn(0x564e24fa5310 msgr2=0x564e24fd8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.513+0000 7fafc3b88740 1 -- 172.21.15.133:0/983096601 shutdown_connections 2024-09-11T23:39:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.513+0000 7fafc3b88740 1 -- 172.21.15.133:0/983096601 wait complete. 2024-09-11T23:39:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.515+0000 7fafc3b88740 1 -- start start 2024-09-11T23:39:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.515+0000 7fafc3b88740 1 Processor -- start 2024-09-11T23:39:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.516+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fdc5d0 0x564e24fe6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.516+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 0x564e24fe3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.517+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fe7370 0x564e24fe4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.517+0000 7fafc3b88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564e24fea070 con 0x564e24fddb40 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.517+0000 7fafc3b88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564e24efd060 con 0x564e24fe7370 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.517+0000 7fafc3b88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564e24fea1f0 con 0x564e24fdc5d0 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.517+0000 7fafbaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fdc5d0 0x564e24fe6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.517+0000 7fafc3b88740 10 client.0 _ll_register_callbacks cb 0x564e24c508c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.517+0000 7fafbaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fdc5d0 0x564e24fe6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53142/0 (socket says 172.21.15.133:53142) 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.517+0000 7fafbaffd640 1 -- 172.21.15.133:0/2939846103 learned_addr learned my addr 172.21.15.133:0/2939846103 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.518+0000 7fafba7fc640 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 0x564e24fe3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.518+0000 7fafbb7fe640 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fe7370 0x564e24fe4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.518+0000 7fafbb7fe640 1 -- 172.21.15.133:0/2939846103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fdc5d0 msgr2=0x564e24fe6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.518+0000 7fafbb7fe640 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fdc5d0 0x564e24fe6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.518+0000 7fafbb7fe640 1 -- 172.21.15.133:0/2939846103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 msgr2=0x564e24fe3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:19.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.518+0000 7fafbb7fe640 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 0x564e24fe3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.518+0000 7fafbb7fe640 1 -- 172.21.15.133:0/2939846103 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e24cb33c0 con 0x564e24fe7370 2024-09-11T23:39:19.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.519+0000 7fafba7fc640 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 0x564e24fe3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:19.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.519+0000 7fafbb7fe640 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fe7370 0x564e24fe4520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fafa4007100 tx=0x7fafa400db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0dd480d1bd8e17d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:19.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.522+0000 7fafa27fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:19.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.522+0000 7fafa27fc640 1 -- 172.21.15.133:0/2939846103 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4007460 con 0x564e24fe7370 2024-09-11T23:39:19.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.522+0000 7fafa27fc640 1 -- 172.21.15.133:0/2939846103 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fafa4007600 con 0x564e24fe7370 2024-09-11T23:39:19.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.522+0000 7fafc3b88740 1 -- 172.21.15.133:0/2939846103 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e24f4f860 con 0x564e24fe7370 2024-09-11T23:39:19.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.525+0000 7fafa27fc640 1 -- 172.21.15.133:0/2939846103 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fafa4025470 con 0x564e24fe7370 2024-09-11T23:39:19.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.526+0000 7fafc3b88740 10 client.25688 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:19.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.526+0000 7fafc3b88740 1 -- 172.21.15.133:0/2939846103 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e24ff5b30 con 0x564e24fe7370 2024-09-11T23:39:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.528+0000 7fafa27fc640 1 -- 172.21.15.133:0/2939846103 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa4020070 con 0x564e24fe7370 2024-09-11T23:39:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.530+0000 7fafa27fc640 1 -- 172.21.15.133:0/2939846103 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fafa401b4b0 con 0x564e24fe7370 2024-09-11T23:39:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.530+0000 7fafc3b88740 10 client.25688 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.530+0000 7fafc3b88740 10 client.25688 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:39:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.530+0000 7fafc3b88740 20 client.25688 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.530+0000 7fafc3b88740 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x564e24ff6ed0 0x564e24ff92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.530+0000 7fafc3b88740 4 client.25688 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:19.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.530+0000 7fafc3b88740 1 -- 172.21.15.133:0/2939846103 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e24f4f860 con 0x564e24ff6ed0 2024-09-11T23:39:19.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.533+0000 7fafbaffd640 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x564e24ff6ed0 0x564e24ff92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.547+0000 7fafa27fc640 10 client.25688 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:39:19.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.547+0000 7fafbaffd640 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x564e24ff6ed0 0x564e24ff92c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6d9446e48d5c050 server_cookie=297f6c7eb35ac5ee in_seq=0 out_seq=0 2024-09-11T23:39:19.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.550+0000 7fafa27fc640 1 -- 172.21.15.133:0/2939846103 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e24f4f860 con 0x564e24ff6ed0 2024-09-11T23:39:19.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.550+0000 7fafa27fc640 10 client.25688 handle_command_reply: tid=0 2024-09-11T23:39:19.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.550+0000 7fafc3b88740 1 client.25688 shutdown 2024-09-11T23:39:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.557+0000 7fafc3b88740 1 -- 172.21.15.133:0/2939846103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fe7370 msgr2=0x564e24fe4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.557+0000 7fafc3b88740 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fe7370 0x564e24fe4520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fafa4007100 tx=0x7fafa400db70 comp rx=0 tx=0).stop 2024-09-11T23:39:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.557+0000 7fafc3b88740 1 -- 172.21.15.133:0/2939846103 shutdown_connections 2024-09-11T23:39:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.558+0000 7fafc3b88740 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x564e24ff6ed0 0x564e24ff92c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.558+0000 7fafc3b88740 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fe7370 0x564e24fe4520 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.558+0000 7fafa27fc640 0 client.25688 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:39:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.558+0000 7fafc3b88740 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 0x564e24fe3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.558+0000 7fafc3b88740 1 --2- 172.21.15.133:0/2939846103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fdc5d0 0x564e24fe6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.558+0000 7fafc3b88740 1 -- 172.21.15.133:0/2939846103 >> 172.21.15.133:0/2939846103 conn(0x564e24fa5310 msgr2=0x564e24fde930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:19.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.558+0000 7fafc3b88740 1 -- 172.21.15.133:0/2939846103 shutdown_connections 2024-09-11T23:39:19.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.559+0000 7fafc3b88740 1 -- 172.21.15.133:0/2939846103 wait complete. 2024-09-11T23:39:19.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.559+0000 7fafc3b88740 20 client.25688 trim_cache size 0 max 16384 2024-09-11T23:39:19.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.614+0000 7fafc3b88740 1 Processor -- start 2024-09-11T23:39:19.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.614+0000 7fafc3b88740 1 -- start start 2024-09-11T23:39:19.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.628+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fdc5d0 0x564e24fe3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.628+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 0x564e24fe5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.629+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fe7370 0x564e24fdf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.629+0000 7fafc3b88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c00e500 con 0x564e24fe7370 2024-09-11T23:39:19.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.629+0000 7fafc3b88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c010040 con 0x564e24fddb40 2024-09-11T23:39:19.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.629+0000 7fafc3b88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c00ebd0 con 0x564e24fdc5d0 2024-09-11T23:39:19.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.629+0000 7fafba7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 0x564e24fe5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.629+0000 7fafba7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 0x564e24fe5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47466/0 (socket says 172.21.15.133:47466) 2024-09-11T23:39:19.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.629+0000 7fafba7fc640 1 -- 172.21.15.133:0/1656660559 learned_addr learned my addr 172.21.15.133:0/1656660559 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:19.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.632+0000 7fafbb7fe640 1 --2- 172.21.15.133:0/1656660559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fe7370 0x564e24fdf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.632+0000 7fafbaffd640 1 --2- 172.21.15.133:0/1656660559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fdc5d0 0x564e24fe3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.635+0000 7fafba7fc640 1 -- 172.21.15.133:0/1656660559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fdc5d0 msgr2=0x564e24fe3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:19.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.635+0000 7fafba7fc640 1 --2- 172.21.15.133:0/1656660559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fdc5d0 0x564e24fe3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.635+0000 7fafba7fc640 1 -- 172.21.15.133:0/1656660559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fe7370 msgr2=0x564e24fdf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:19.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.635+0000 7fafba7fc640 1 --2- 172.21.15.133:0/1656660559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fe7370 0x564e24fdf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.635+0000 7fafba7fc640 1 -- 172.21.15.133:0/1656660559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e24fe2f70 con 0x564e24fddb40 2024-09-11T23:39:19.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.638+0000 7fafba7fc640 1 --2- 172.21.15.133:0/1656660559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 0x564e24fe5f10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fafb4006fd0 tx=0x7fafb4016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd971377bca6f60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:19.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.639+0000 7fafb0ff9640 1 -- 172.21.15.133:0/1656660559 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004220 con 0x564e24fddb40 2024-09-11T23:39:19.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.640+0000 7fafb0ff9640 1 -- 172.21.15.133:0/1656660559 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb4010040 con 0x564e24fddb40 2024-09-11T23:39:19.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.640+0000 7fafb0ff9640 1 -- 172.21.15.133:0/1656660559 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4003cf0 con 0x564e24fddb40 2024-09-11T23:39:19.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.644+0000 7fafc3b88740 1 -- 172.21.15.133:0/1656660559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 msgr2=0x564e24fe5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:19.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.644+0000 7fafc3b88740 1 --2- 172.21.15.133:0/1656660559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 0x564e24fe5f10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fafb4006fd0 tx=0x7fafb4016000 comp rx=0 tx=0).stop 2024-09-11T23:39:19.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.647+0000 7fafc3b88740 1 -- 172.21.15.133:0/1656660559 shutdown_connections 2024-09-11T23:39:19.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.647+0000 7fafc3b88740 1 --2- 172.21.15.133:0/1656660559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fe7370 0x564e24fdf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.647+0000 7fafc3b88740 1 --2- 172.21.15.133:0/1656660559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fddb40 0x564e24fe5f10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.647+0000 7fafc3b88740 1 --2- 172.21.15.133:0/1656660559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fdc5d0 0x564e24fe3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.647+0000 7fafc3b88740 1 -- 172.21.15.133:0/1656660559 >> 172.21.15.133:0/1656660559 conn(0x564e24fa5310 msgr2=0x564e24fe5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:19.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.648+0000 7fafc3b88740 1 -- 172.21.15.133:0/1656660559 shutdown_connections 2024-09-11T23:39:19.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.649+0000 7fafc3b88740 1 -- 172.21.15.133:0/1656660559 wait complete. 2024-09-11T23:39:19.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.650+0000 7fafc3b88740 1 -- start start 2024-09-11T23:39:19.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.651+0000 7fafc3b88740 1 Processor -- start 2024-09-11T23:39:19.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.652+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fdc5d0 0x564e24fefc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.652+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 0x564e24fe0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.652+0000 7fafc3b88740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 0x564e24fe11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.652+0000 7fafc3b88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c00f970 con 0x564e24fddb40 2024-09-11T23:39:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.652+0000 7fafc3b88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c00f2a0 con 0x564e24fdc5d0 2024-09-11T23:39:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.653+0000 7fafc3b88740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c00c590 con 0x564e24fe7370 2024-09-11T23:39:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.653+0000 7fafc3b88740 10 client.0 _ll_register_callbacks cb 0x7fafa40014e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.655+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 0x564e24fe11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.655+0000 7fafbaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fdc5d0 0x564e24fefc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.659+0000 7fafba7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 0x564e24fe0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.659+0000 7fafba7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 0x564e24fe0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39298/0 (socket says 172.21.15.133:39298) 2024-09-11T23:39:19.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.659+0000 7fafba7fc640 1 -- 172.21.15.133:0/1047307080 learned_addr learned my addr 172.21.15.133:0/1047307080 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:19.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.662+0000 7fafbaffd640 1 -- 172.21.15.133:0/1047307080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 msgr2=0x564e24fe11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:19.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.662+0000 7fafbaffd640 1 --2- 172.21.15.133:0/1047307080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 0x564e24fe11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.662+0000 7fafbaffd640 1 -- 172.21.15.133:0/1047307080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 msgr2=0x564e24fe0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:19.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.662+0000 7fafbaffd640 1 --2- 172.21.15.133:0/1047307080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 0x564e24fe0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.662+0000 7fafbaffd640 1 -- 172.21.15.133:0/1047307080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e24ff9630 con 0x564e24fdc5d0 2024-09-11T23:39:19.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.666+0000 7fafbaffd640 1 --2- 172.21.15.133:0/1047307080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fdc5d0 0x564e24fefc50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fafac02f550 tx=0x7fafac0f0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89c7dd9a1a2e3563 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:19.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.669+0000 7fafa27fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:19.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.670+0000 7fafa27fc640 1 -- 172.21.15.133:0/1047307080 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac0c5a90 con 0x564e24fdc5d0 2024-09-11T23:39:19.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.671+0000 7fafa27fc640 1 -- 172.21.15.133:0/1047307080 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fafac0b9490 con 0x564e24fdc5d0 2024-09-11T23:39:19.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.671+0000 7fafc3b88740 1 -- 172.21.15.133:0/1047307080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb4010040 con 0x564e24fdc5d0 2024-09-11T23:39:19.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.672+0000 7fafa27fc640 1 -- 172.21.15.133:0/1047307080 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fafac0d1020 con 0x564e24fdc5d0 2024-09-11T23:39:19.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.673+0000 7fafc3b88740 10 client.25694 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:19.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.673+0000 7fafc3b88740 1 -- 172.21.15.133:0/1047307080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e25061630 con 0x564e24fdc5d0 2024-09-11T23:39:19.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.675+0000 7fafa27fc640 1 -- 172.21.15.133:0/1047307080 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564e24f4f860 con 0x564e24fdc5d0 2024-09-11T23:39:19.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.675+0000 7fafa27fc640 1 -- 172.21.15.133:0/1047307080 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fafac0c5ce0 con 0x564e24fdc5d0 2024-09-11T23:39:19.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.675+0000 7fafc3b88740 10 client.25694 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:19.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.675+0000 7fafc3b88740 10 client.25694 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:39:19.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.675+0000 7fafc3b88740 20 client.25694 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:19.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.675+0000 7fafc3b88740 1 --2- 172.21.15.133:0/1047307080 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x564e24ff6ed0 0x564e25063d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:19.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.675+0000 7fafc3b88740 4 client.25694 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:39:19.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.675+0000 7fafc3b88740 1 -- 172.21.15.133:0/1047307080 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fafb4010040 con 0x564e24ff6ed0 2024-09-11T23:39:19.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.677+0000 7fafba7fc640 1 --2- 172.21.15.133:0/1047307080 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x564e24ff6ed0 0x564e25063d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:19.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.688+0000 7fafba7fc640 1 --2- 172.21.15.133:0/1047307080 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x564e24ff6ed0 0x564e25063d30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35201d9a4a6f57f4 server_cookie=88683fa5c6b66dcd in_seq=0 out_seq=0 2024-09-11T23:39:19.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.688+0000 7fafa27fc640 10 client.25694 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:39:19.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.689+0000 7fafa27fc640 1 -- 172.21.15.133:0/1047307080 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fafb4010040 con 0x564e24ff6ed0 2024-09-11T23:39:19.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.689+0000 7fafa27fc640 10 client.25694 handle_command_reply: tid=0 2024-09-11T23:39:19.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.689+0000 7fafc3b88740 1 client.25694 shutdown 2024-09-11T23:39:19.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.690+0000 7fafc3b88740 1 -- 172.21.15.133:0/1047307080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fdc5d0 msgr2=0x564e24fefc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:19.692 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:39:19.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.690+0000 7fafc3b88740 1 --2- 172.21.15.133:0/1047307080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fdc5d0 0x564e24fefc50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fafac02f550 tx=0x7fafac0f0040 comp rx=0 tx=0).stop 2024-09-11T23:39:19.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.691+0000 7fafc3b88740 1 -- 172.21.15.133:0/1047307080 shutdown_connections 2024-09-11T23:39:19.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.691+0000 7fafc3b88740 1 --2- 172.21.15.133:0/1047307080 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x564e24ff6ed0 0x564e25063d30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.691+0000 7fafc3b88740 1 --2- 172.21.15.133:0/1047307080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e24fe7370 0x564e24fe11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.691+0000 7fafc3b88740 1 --2- 172.21.15.133:0/1047307080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e24fddb40 0x564e24fe0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.691+0000 7fafa27fc640 0 client.25694 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:39:19.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.691+0000 7fafc3b88740 1 --2- 172.21.15.133:0/1047307080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e24fdc5d0 0x564e24fefc50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.691+0000 7fafc3b88740 1 -- 172.21.15.133:0/1047307080 >> 172.21.15.133:0/1047307080 conn(0x564e24fa5310 msgr2=0x564e24fd74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:19.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.691+0000 7fafc3b88740 1 -- 172.21.15.133:0/1047307080 shutdown_connections 2024-09-11T23:39:19.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.691+0000 7fafc3b88740 1 -- 172.21.15.133:0/1047307080 wait complete. 2024-09-11T23:39:19.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.691+0000 7fafc3b88740 20 client.25694 trim_cache size 0 max 16384 2024-09-11T23:39:19.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.694+0000 7fafc17af640 1 -- 172.21.15.133:0/3291234410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafa80a2f00 msgr2=0x7fafa80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:19.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.694+0000 7fafc17af640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafa80a2f00 0x7fafa80a53c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fafbc11d8c0 tx=0x7fafb4009d20 comp rx=0 tx=0).stop 2024-09-11T23:39:19.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.694+0000 7fafc17af640 1 -- 172.21.15.133:0/3291234410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc109450 msgr2=0x7fafbc1a7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:19.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.694+0000 7fafc17af640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc109450 0x7fafbc1a7450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fafac02aec0 tx=0x7fafac031460 comp rx=0 tx=0).stop 2024-09-11T23:39:19.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.694+0000 7fafc17af640 1 -- 172.21.15.133:0/3291234410 shutdown_connections 2024-09-11T23:39:19.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.694+0000 7fafc17af640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafa80a2f00 0x7fafa80a53c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.694+0000 7fafc17af640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafbc11b140 0x7fafbc1abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.694+0000 7fafc17af640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafbc109830 0x7fafbc1a7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.694+0000 7fafc17af640 1 --2- 172.21.15.133:0/3291234410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc109450 0x7fafbc1a7450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:19.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.694+0000 7fafc17af640 1 -- 172.21.15.133:0/3291234410 >> 172.21.15.133:0/3291234410 conn(0x7fafbc06e140 msgr2=0x7fafbc11bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:19.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.695+0000 7fafc17af640 1 -- 172.21.15.133:0/3291234410 shutdown_connections 2024-09-11T23:39:19.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:19.695+0000 7fafc17af640 1 -- 172.21.15.133:0/3291234410 wait complete. 2024-09-11T23:39:19.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:19 smithi138 ceph-mon[35142]: pgmap v79: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 585 MiB/s wr, 4.45k op/s 2024-09-11T23:39:19.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:19 smithi178 ceph-mon[35882]: pgmap v79: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 585 MiB/s wr, 4.45k op/s 2024-09-11T23:39:21.098 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:39:21.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:21 smithi138 ceph-mon[35142]: pgmap v80: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1002 KiB/s rd, 593 MiB/s wr, 4.49k op/s 2024-09-11T23:39:21.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:21 smithi133 ceph-mon[31430]: pgmap v80: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1002 KiB/s rd, 593 MiB/s wr, 4.49k op/s 2024-09-11T23:39:21.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:21 smithi178 ceph-mon[35882]: pgmap v80: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1002 KiB/s rd, 593 MiB/s wr, 4.49k op/s 2024-09-11T23:39:21.962 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:22.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.852+0000 7f0456f40640 1 -- 172.21.15.133:0/2426035682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f045010efb0 msgr2=0x7f0450109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:22.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.853+0000 7f0456f40640 1 --2- 172.21.15.133:0/2426035682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f045010efb0 0x7f0450109450 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0448013490 tx=0x7f0448032b80 comp rx=0 tx=0).stop 2024-09-11T23:39:22.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.857+0000 7f0456f40640 1 -- 172.21.15.133:0/2426035682 shutdown_connections 2024-09-11T23:39:22.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.857+0000 7f0456f40640 1 --2- 172.21.15.133:0/2426035682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109990 0x7f0450109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.857+0000 7f0456f40640 1 --2- 172.21.15.133:0/2426035682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f045010efb0 0x7f0450109450 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.857+0000 7f0456f40640 1 --2- 172.21.15.133:0/2426035682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f045010e690 0x7f045010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.857+0000 7f0456f40640 1 -- 172.21.15.133:0/2426035682 >> 172.21.15.133:0/2426035682 conn(0x7f045006e140 msgr2=0x7f045006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:22.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.857+0000 7f0456f40640 1 -- 172.21.15.133:0/2426035682 shutdown_connections 2024-09-11T23:39:22.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.857+0000 7f0456f40640 1 -- 172.21.15.133:0/2426035682 wait complete. 2024-09-11T23:39:22.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.860+0000 7f0456f40640 1 Processor -- start 2024-09-11T23:39:22.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.860+0000 7f0456f40640 1 -- start start 2024-09-11T23:39:22.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.862+0000 7f0456f40640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109990 0x7f04501a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:22.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.862+0000 7f0456f40640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f045010e690 0x7f04501a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:22.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.864+0000 7f0454cdc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109990 0x7f04501a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:22.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.864+0000 7f0454cdc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109990 0x7f04501a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:36948/0 (socket says 172.21.15.133:36948) 2024-09-11T23:39:22.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.865+0000 7f0454cdc640 1 -- 172.21.15.133:0/4053249132 learned_addr learned my addr 172.21.15.133:0/4053249132 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:22.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.866+0000 7f0456f40640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f045010efb0 0x7f045019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:22.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.866+0000 7f044ffff640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f045010e690 0x7f04501a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:22.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.866+0000 7f0456f40640 1 -- 172.21.15.133:0/4053249132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0450118560 con 0x7f045010efb0 2024-09-11T23:39:22.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.867+0000 7f0456f40640 1 -- 172.21.15.133:0/4053249132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04501183e0 con 0x7f0450109990 2024-09-11T23:39:22.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.867+0000 7f0456f40640 1 -- 172.21.15.133:0/4053249132 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f04501186e0 con 0x7f045010e690 2024-09-11T23:39:22.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.868+0000 7f04554dd640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f045010efb0 0x7f045019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:22.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.870+0000 7f0454cdc640 1 -- 172.21.15.133:0/4053249132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f045010e690 msgr2=0x7f04501a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:22.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.870+0000 7f0454cdc640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f045010e690 0x7f04501a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.870+0000 7f0454cdc640 1 -- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f045010efb0 msgr2=0x7f045019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:22.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.870+0000 7f0454cdc640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f045010efb0 0x7f045019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.870+0000 7f0454cdc640 1 -- 172.21.15.133:0/4053249132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045019efb0 con 0x7f0450109990 2024-09-11T23:39:22.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.870+0000 7f04554dd640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f045010efb0 0x7f045019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:22.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.879+0000 7f0454cdc640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109990 0x7f04501a4690 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f044000e9f0 tx=0x7f044000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:22.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.882+0000 7f044dffb640 1 -- 172.21.15.133:0/4053249132 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044000cd80 con 0x7f0450109990 2024-09-11T23:39:22.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.883+0000 7f0456f40640 1 -- 172.21.15.133:0/4053249132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045019f2a0 con 0x7f0450109990 2024-09-11T23:39:22.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.884+0000 7f0456f40640 1 -- 172.21.15.133:0/4053249132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04501ab4f0 con 0x7f0450109990 2024-09-11T23:39:22.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.884+0000 7f044dffb640 1 -- 172.21.15.133:0/4053249132 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0440010910 con 0x7f0450109990 2024-09-11T23:39:22.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.884+0000 7f044dffb640 1 -- 172.21.15.133:0/4053249132 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440018990 con 0x7f0450109990 2024-09-11T23:39:22.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.898+0000 7f0459319740 1 Processor -- start 2024-09-11T23:39:22.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.899+0000 7f0459319740 1 -- start start 2024-09-11T23:39:22.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.900+0000 7f0459319740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da7b40 0x5573a6da6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:22.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.900+0000 7f0459319740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6da65d0 0x5573a6da6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:22.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.900+0000 7f0459319740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6db1370 0x5573a6db3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:22.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.900+0000 7f0459319740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5573a6d0c2f0 con 0x5573a6db1370 2024-09-11T23:39:22.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.900+0000 7f0459319740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5573a6cb3cf0 con 0x5573a6da7b40 2024-09-11T23:39:22.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.900+0000 7f0459319740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5573a6d082b0 con 0x5573a6da65d0 2024-09-11T23:39:22.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.901+0000 7f044dffb640 1 -- 172.21.15.133:0/4053249132 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0440018b30 con 0x7f0450109990 2024-09-11T23:39:22.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.902+0000 7f0454cdc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6da65d0 0x5573a6da6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:22.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.902+0000 7f04554dd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da7b40 0x5573a6da6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:22.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.902+0000 7f04554dd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da7b40 0x5573a6da6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:36956/0 (socket says 172.21.15.133:36956) 2024-09-11T23:39:22.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.902+0000 7f04554dd640 1 -- 172.21.15.133:0/639356327 learned_addr learned my addr 172.21.15.133:0/639356327 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:22.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.903+0000 7f044dffb640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04380a2ec0 0x7f04380a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:22.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.904+0000 7f04554dd640 1 -- 172.21.15.133:0/639356327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6da65d0 msgr2=0x5573a6da6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:22.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.904+0000 7f04554dd640 1 --2- 172.21.15.133:0/639356327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6da65d0 0x5573a6da6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.904+0000 7f04554dd640 1 -- 172.21.15.133:0/639356327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6db1370 msgr2=0x5573a6db3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:22.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.904+0000 7f04554dd640 1 --2- 172.21.15.133:0/639356327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6db1370 0x5573a6db3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.904+0000 7f04554dd640 1 -- 172.21.15.133:0/639356327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a6c983a0 con 0x5573a6da7b40 2024-09-11T23:39:22.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.907+0000 7f04554dd640 1 --2- 172.21.15.133:0/639356327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da7b40 0x5573a6da6090 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f044400be70 tx=0x7f044400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cdf0e073ca8a668 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:22.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.907+0000 7f04554dd640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04380a2ec0 0x7f04380a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:22.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.908+0000 7f044dffb640 1 -- 172.21.15.133:0/4053249132 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f04400d0700 con 0x7f0450109990 2024-09-11T23:39:22.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.909+0000 7f0436ffd640 1 -- 172.21.15.133:0/639356327 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444021070 con 0x5573a6da7b40 2024-09-11T23:39:22.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.909+0000 7f0436ffd640 1 -- 172.21.15.133:0/639356327 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044401ccd0 con 0x5573a6da7b40 2024-09-11T23:39:22.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.911+0000 7f0459319740 1 -- 172.21.15.133:0/639356327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da7b40 msgr2=0x5573a6da6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:22.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.911+0000 7f0459319740 1 --2- 172.21.15.133:0/639356327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da7b40 0x5573a6da6090 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f044400be70 tx=0x7f044400f730 comp rx=0 tx=0).stop 2024-09-11T23:39:22.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.911+0000 7f0459319740 1 -- 172.21.15.133:0/639356327 shutdown_connections 2024-09-11T23:39:22.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.911+0000 7f0459319740 1 --2- 172.21.15.133:0/639356327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6db1370 0x5573a6db3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.911+0000 7f0459319740 1 --2- 172.21.15.133:0/639356327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6da65d0 0x5573a6da6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.911+0000 7f0459319740 1 --2- 172.21.15.133:0/639356327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da7b40 0x5573a6da6090 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.911+0000 7f0459319740 1 -- 172.21.15.133:0/639356327 >> 172.21.15.133:0/639356327 conn(0x5573a6d6f310 msgr2=0x5573a6da2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:22.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.916+0000 7f04554dd640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04380a2ec0 0x7f04380a5380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f044401e000 tx=0x7f044401b000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:22.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.916+0000 7f0459319740 1 -- 172.21.15.133:0/639356327 shutdown_connections 2024-09-11T23:39:22.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.917+0000 7f0459319740 1 -- 172.21.15.133:0/639356327 wait complete. 2024-09-11T23:39:22.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.918+0000 7f0459319740 1 -- start start 2024-09-11T23:39:22.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.918+0000 7f0459319740 1 Processor -- start 2024-09-11T23:39:22.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.919+0000 7f0459319740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6da65d0 0x5573a6db0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:22.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.919+0000 7f0459319740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6db1370 0x5573a6dadd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:22.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.919+0000 7f0459319740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6dae2c0 0x5573a6dae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:22.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.919+0000 7f0459319740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5573a6dc50d0 con 0x5573a6db1370 2024-09-11T23:39:22.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.920+0000 7f0459319740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5573a6cc7060 con 0x5573a6dae2c0 2024-09-11T23:39:22.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.920+0000 7f0459319740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5573a6dc5250 con 0x5573a6da65d0 2024-09-11T23:39:22.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.920+0000 7f0459319740 10 client.0 _ll_register_callbacks cb 0x5573a6a1a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:22.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.921+0000 7f044ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6db1370 0x5573a6dadd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:22.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.921+0000 7f044ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6db1370 0x5573a6dadd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54644/0 (socket says 172.21.15.133:54644) 2024-09-11T23:39:22.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.921+0000 7f044ffff640 1 -- 172.21.15.133:0/1465102288 learned_addr learned my addr 172.21.15.133:0/1465102288 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:22.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.922+0000 7f044ffff640 1 -- 172.21.15.133:0/1465102288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6da65d0 msgr2=0x5573a6db0ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:39:22.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.922+0000 7f044ffff640 1 --2- 172.21.15.133:0/1465102288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6da65d0 0x5573a6db0ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.922+0000 7f044ffff640 1 -- 172.21.15.133:0/1465102288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6dae2c0 msgr2=0x5573a6dae740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:39:22.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.922+0000 7f044ffff640 1 --2- 172.21.15.133:0/1465102288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6dae2c0 0x5573a6dae740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.923+0000 7f044ffff640 1 -- 172.21.15.133:0/1465102288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a6a7d3c0 con 0x5573a6db1370 2024-09-11T23:39:22.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.926+0000 7f044ffff640 1 --2- 172.21.15.133:0/1465102288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6db1370 0x5573a6dadd80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0448000f80 tx=0x7f0448034a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67dc393436efd7cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:22.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.926+0000 7f042ffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:22.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.935+0000 7f042ffff640 1 -- 172.21.15.133:0/1465102288 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044800fde0 con 0x5573a6db1370 2024-09-11T23:39:22.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.936+0000 7f0459319740 1 -- 172.21.15.133:0/1465102288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5573a6d19860 con 0x5573a6db1370 2024-09-11T23:39:22.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.936+0000 7f042ffff640 1 -- 172.21.15.133:0/1465102288 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f044800fbe0 con 0x5573a6db1370 2024-09-11T23:39:22.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.945+0000 7f042ffff640 1 -- 172.21.15.133:0/1465102288 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f044804e050 con 0x5573a6db1370 2024-09-11T23:39:22.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.945+0000 7f0459319740 10 client.16269 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:22.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.945+0000 7f0459319740 1 -- 172.21.15.133:0/1465102288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5573a6dd0df0 con 0x5573a6db1370 2024-09-11T23:39:22.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.955+0000 7f042ffff640 1 -- 172.21.15.133:0/1465102288 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5573a6d19860 con 0x5573a6db1370 2024-09-11T23:39:22.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.955+0000 7f042ffff640 1 -- 172.21.15.133:0/1465102288 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f044800f030 con 0x5573a6db1370 2024-09-11T23:39:22.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.955+0000 7f0459319740 10 client.16269 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:22.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.955+0000 7f0459319740 10 client.16269 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:39:22.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.955+0000 7f0459319740 20 client.16269 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:22.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.955+0000 7f0459319740 1 --2- 172.21.15.133:0/1465102288 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5573a6dd22c0 0x5573a6dd46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:22.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.955+0000 7f0459319740 4 client.16269 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:22.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.955+0000 7f0459319740 1 -- 172.21.15.133:0/1465102288 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573a6d13d70 con 0x5573a6dd22c0 2024-09-11T23:39:22.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.957+0000 7f0454cdc640 1 --2- 172.21.15.133:0/1465102288 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5573a6dd22c0 0x5573a6dd46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:22.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.961+0000 7f0454cdc640 1 --2- 172.21.15.133:0/1465102288 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5573a6dd22c0 0x5573a6dd46e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=337d8c1ca0fdcdda server_cookie=61e6833841c942da in_seq=0 out_seq=0 2024-09-11T23:39:22.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.962+0000 7f042ffff640 10 client.16269 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:39:22.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.967+0000 7f042ffff640 1 -- 172.21.15.133:0/1465102288 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573a6d13d70 con 0x5573a6dd22c0 2024-09-11T23:39:22.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.967+0000 7f042ffff640 10 client.16269 handle_command_reply: tid=0 2024-09-11T23:39:22.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.967+0000 7f0459319740 1 client.16269 shutdown 2024-09-11T23:39:22.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.969+0000 7f0459319740 1 -- 172.21.15.133:0/1465102288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6db1370 msgr2=0x5573a6dadd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:22.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.969+0000 7f0459319740 1 --2- 172.21.15.133:0/1465102288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6db1370 0x5573a6dadd80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0448000f80 tx=0x7f0448034a60 comp rx=0 tx=0).stop 2024-09-11T23:39:22.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.970+0000 7f0459319740 1 -- 172.21.15.133:0/1465102288 shutdown_connections 2024-09-11T23:39:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.970+0000 7f0459319740 1 --2- 172.21.15.133:0/1465102288 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5573a6dd22c0 0x5573a6dd46e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.970+0000 7f0459319740 1 --2- 172.21.15.133:0/1465102288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6dae2c0 0x5573a6dae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.970+0000 7f0459319740 1 --2- 172.21.15.133:0/1465102288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6db1370 0x5573a6dadd80 secure :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0448000f80 tx=0x7f0448034a60 comp rx=0 tx=0).stop 2024-09-11T23:39:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.970+0000 7f0459319740 1 --2- 172.21.15.133:0/1465102288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6da65d0 0x5573a6db0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.970+0000 7f0459319740 1 -- 172.21.15.133:0/1465102288 >> 172.21.15.133:0/1465102288 conn(0x5573a6d6f310 msgr2=0x5573a6da8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.970+0000 7f042ffff640 0 client.16269 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:39:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.971+0000 7f0459319740 1 -- 172.21.15.133:0/1465102288 shutdown_connections 2024-09-11T23:39:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.971+0000 7f0459319740 1 -- 172.21.15.133:0/1465102288 wait complete. 2024-09-11T23:39:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:22.971+0000 7f0459319740 20 client.16269 trim_cache size 0 max 16384 2024-09-11T23:39:23.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.002+0000 7f0459319740 1 Processor -- start 2024-09-11T23:39:23.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f0459319740 1 -- start start 2024-09-11T23:39:23.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f0459319740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da65d0 0x5573a6dad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:23.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f0459319740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 0x5573a6dadb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:23.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f0459319740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6dd9fb0 0x5573a6dda410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f0459319740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f043c00e6f0 con 0x5573a6dd9fb0 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f0459319740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f043c010230 con 0x5573a6da65d0 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f0459319740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f043c00edc0 con 0x5573a6dae2c0 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f044ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 0x5573a6dadb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f044ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 0x5573a6dadb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:56922/0 (socket says 172.21.15.133:56922) 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f044ffff640 1 -- 172.21.15.133:0/83409418 learned_addr learned my addr 172.21.15.133:0/83409418 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f0454cdc640 1 --2- 172.21.15.133:0/83409418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da65d0 0x5573a6dad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f044ffff640 1 -- 172.21.15.133:0/83409418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da65d0 msgr2=0x5573a6dad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f044ffff640 1 --2- 172.21.15.133:0/83409418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da65d0 0x5573a6dad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f044ffff640 1 -- 172.21.15.133:0/83409418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6dd9fb0 msgr2=0x5573a6dda410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f044ffff640 1 --2- 172.21.15.133:0/83409418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6dd9fb0 0x5573a6dda410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.003+0000 7f044ffff640 1 -- 172.21.15.133:0/83409418 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a6da31a0 con 0x5573a6dae2c0 2024-09-11T23:39:23.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.004+0000 7f044ffff640 1 --2- 172.21.15.133:0/83409418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 0x5573a6dadb80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f04480330a0 tx=0x7f04480021d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9fe23c2cfc66eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:23.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.004+0000 7f04367fc640 1 -- 172.21.15.133:0/83409418 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044800f230 con 0x5573a6dae2c0 2024-09-11T23:39:23.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.004+0000 7f04367fc640 1 -- 172.21.15.133:0/83409418 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0448003e50 con 0x5573a6dae2c0 2024-09-11T23:39:23.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.004+0000 7f04367fc640 1 -- 172.21.15.133:0/83409418 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04480113c0 con 0x5573a6dae2c0 2024-09-11T23:39:23.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.006+0000 7f0459319740 1 -- 172.21.15.133:0/83409418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 msgr2=0x5573a6dadb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:23.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.006+0000 7f0459319740 1 --2- 172.21.15.133:0/83409418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 0x5573a6dadb80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f04480330a0 tx=0x7f04480021d0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.006+0000 7f0459319740 1 -- 172.21.15.133:0/83409418 shutdown_connections 2024-09-11T23:39:23.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.006+0000 7f0459319740 1 --2- 172.21.15.133:0/83409418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6dd9fb0 0x5573a6dda410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.006+0000 7f0459319740 1 --2- 172.21.15.133:0/83409418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 0x5573a6dadb80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.006+0000 7f0459319740 1 --2- 172.21.15.133:0/83409418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da65d0 0x5573a6dad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.006+0000 7f0459319740 1 -- 172.21.15.133:0/83409418 >> 172.21.15.133:0/83409418 conn(0x5573a6d6f310 msgr2=0x5573a6dac650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:23.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.007+0000 7f0459319740 1 -- 172.21.15.133:0/83409418 shutdown_connections 2024-09-11T23:39:23.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.007+0000 7f0459319740 1 -- 172.21.15.133:0/83409418 wait complete. 2024-09-11T23:39:23.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.008+0000 7f0459319740 1 -- start start 2024-09-11T23:39:23.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.008+0000 7f0459319740 1 Processor -- start 2024-09-11T23:39:23.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.009+0000 7f0459319740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da65d0 0x5573a6dca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:23.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.009+0000 7f0459319740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 0x5573a6dcaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:23.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.010+0000 7f0459319740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6dd9fb0 0x5573a6dc7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:23.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.010+0000 7f0459319740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f043c00fb60 con 0x5573a6dd9fb0 2024-09-11T23:39:23.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.010+0000 7f04554dd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6dd9fb0 0x5573a6dc7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:23.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.010+0000 7f04554dd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6dd9fb0 0x5573a6dc7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54662/0 (socket says 172.21.15.133:54662) 2024-09-11T23:39:23.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.010+0000 7f04554dd640 1 -- 172.21.15.133:0/1253090411 learned_addr learned my addr 172.21.15.133:0/1253090411 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:23.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.010+0000 7f044ffff640 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 0x5573a6dcaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:23.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.010+0000 7f0459319740 1 -- 172.21.15.133:0/1253090411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f043c00f490 con 0x5573a6da65d0 2024-09-11T23:39:23.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.011+0000 7f0459319740 1 -- 172.21.15.133:0/1253090411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f043c00c6d0 con 0x5573a6dae2c0 2024-09-11T23:39:23.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.011+0000 7f0454cdc640 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da65d0 0x5573a6dca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:23.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.011+0000 7f0459319740 10 client.0 _ll_register_callbacks cb 0x7f0448013390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.015+0000 7f0454cdc640 1 -- 172.21.15.133:0/1253090411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 msgr2=0x5573a6dcaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.016+0000 7f0454cdc640 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 0x5573a6dcaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.016+0000 7f0454cdc640 1 -- 172.21.15.133:0/1253090411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6dd9fb0 msgr2=0x5573a6dc7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.016+0000 7f0454cdc640 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6dd9fb0 0x5573a6dc7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.016+0000 7f0454cdc640 1 -- 172.21.15.133:0/1253090411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a6da3960 con 0x5573a6da65d0 2024-09-11T23:39:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.016+0000 7f044ffff640 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 0x5573a6dcaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.016+0000 7f04554dd640 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6dd9fb0 0x5573a6dc7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:23.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.023+0000 7f0454cdc640 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da65d0 0x5573a6dca560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f04400cde60 tx=0x7f0440015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928fc3b2eeb58c9f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:23.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.023+0000 7f042ffff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:23.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.023+0000 7f042ffff640 1 -- 172.21.15.133:0/1253090411 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04400a58a0 con 0x5573a6da65d0 2024-09-11T23:39:23.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.024+0000 7f042ffff640 1 -- 172.21.15.133:0/1253090411 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f04400ce0e0 con 0x5573a6da65d0 2024-09-11T23:39:23.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.025+0000 7f0459319740 1 -- 172.21.15.133:0/1253090411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0448003e50 con 0x5573a6da65d0 2024-09-11T23:39:23.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.027+0000 7f042ffff640 1 -- 172.21.15.133:0/1253090411 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f04400ce2e0 con 0x5573a6da65d0 2024-09-11T23:39:23.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.030+0000 7f0459319740 10 client.25709 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.030+0000 7f0459319740 1 -- 172.21.15.133:0/1253090411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5573a6da3dd0 con 0x5573a6da65d0 2024-09-11T23:39:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.031+0000 7f042ffff640 1 -- 172.21.15.133:0/1253090411 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0448003e50 con 0x5573a6da65d0 2024-09-11T23:39:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.032+0000 7f042ffff640 1 -- 172.21.15.133:0/1253090411 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f044009c080 con 0x5573a6da65d0 2024-09-11T23:39:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.032+0000 7f0459319740 10 client.25709 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.032+0000 7f0459319740 10 client.25709 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:39:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.032+0000 7f0459319740 20 client.25709 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.032+0000 7f0459319740 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5573a6e3d840 0x5573a6e3fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.032+0000 7f0459319740 4 client.25709 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:39:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.032+0000 7f0459319740 1 -- 172.21.15.133:0/1253090411 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573a6c90170 con 0x5573a6e3d840 2024-09-11T23:39:23.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.034+0000 7f044ffff640 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5573a6e3d840 0x5573a6e3fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:23.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.038+0000 7f042ffff640 10 client.25709 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:39:23.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.038+0000 7f044ffff640 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5573a6e3d840 0x5573a6e3fc60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6986dd1b04226f4b server_cookie=255b68c51309c190 in_seq=0 out_seq=0 2024-09-11T23:39:23.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.039+0000 7f042ffff640 1 -- 172.21.15.133:0/1253090411 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573a6c90170 con 0x5573a6e3d840 2024-09-11T23:39:23.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.039+0000 7f042ffff640 10 client.25709 handle_command_reply: tid=0 2024-09-11T23:39:23.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.039+0000 7f0459319740 1 client.25709 shutdown 2024-09-11T23:39:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.041+0000 7f0459319740 1 -- 172.21.15.133:0/1253090411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da65d0 msgr2=0x5573a6dca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.041+0000 7f0459319740 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da65d0 0x5573a6dca560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f04400cde60 tx=0x7f0440015000 comp rx=0 tx=0).stop 2024-09-11T23:39:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.041+0000 7f0459319740 1 -- 172.21.15.133:0/1253090411 shutdown_connections 2024-09-11T23:39:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.041+0000 7f0459319740 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5573a6e3d840 0x5573a6e3fc60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.041+0000 7f0459319740 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573a6dd9fb0 0x5573a6dc7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.041+0000 7f0459319740 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5573a6dae2c0 0x5573a6dcaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.041+0000 7f0459319740 1 --2- 172.21.15.133:0/1253090411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573a6da65d0 0x5573a6dca560 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.041+0000 7f042ffff640 0 client.25709 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:39:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.041+0000 7f0459319740 1 -- 172.21.15.133:0/1253090411 >> 172.21.15.133:0/1253090411 conn(0x5573a6d6f310 msgr2=0x5573a6dda410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.041+0000 7f0459319740 1 -- 172.21.15.133:0/1253090411 shutdown_connections 2024-09-11T23:39:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.041+0000 7f0459319740 1 -- 172.21.15.133:0/1253090411 wait complete. 2024-09-11T23:39:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.042+0000 7f0459319740 20 client.25709 trim_cache size 0 max 16384 2024-09-11T23:39:23.042 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:39:23.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.045+0000 7f0456f40640 1 -- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04380a2ec0 msgr2=0x7f04380a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:23.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.046+0000 7f0456f40640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04380a2ec0 0x7f04380a5380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f044401e000 tx=0x7f044401b000 comp rx=0 tx=0).stop 2024-09-11T23:39:23.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.046+0000 7f0456f40640 1 -- 172.21.15.133:0/4053249132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109990 msgr2=0x7f04501a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:23.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.046+0000 7f0456f40640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109990 0x7f04501a4690 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f044000e9f0 tx=0x7f044000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.049+0000 7f0456f40640 1 -- 172.21.15.133:0/4053249132 shutdown_connections 2024-09-11T23:39:23.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.049+0000 7f0456f40640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04380a2ec0 0x7f04380a5380 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.049+0000 7f0456f40640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f045010efb0 0x7f045019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.049+0000 7f0456f40640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f045010e690 0x7f04501a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.049+0000 7f0456f40640 1 --2- 172.21.15.133:0/4053249132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109990 0x7f04501a4690 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:23.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.049+0000 7f0456f40640 1 -- 172.21.15.133:0/4053249132 >> 172.21.15.133:0/4053249132 conn(0x7f045006e140 msgr2=0x7f045010c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:23.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.050+0000 7f0456f40640 1 -- 172.21.15.133:0/4053249132 shutdown_connections 2024-09-11T23:39:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:23.051+0000 7f0456f40640 1 -- 172.21.15.133:0/4053249132 wait complete. 2024-09-11T23:39:23.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:23 smithi138 ceph-mon[35142]: pgmap v81: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 827 MiB/s wr, 5.29k op/s 2024-09-11T23:39:23.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:23 smithi133 ceph-mon[31430]: pgmap v81: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 827 MiB/s wr, 5.29k op/s 2024-09-11T23:39:23.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:23 smithi178 ceph-mon[35882]: pgmap v81: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 827 MiB/s wr, 5.29k op/s 2024-09-11T23:39:23.926 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:39:24.486 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:25.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.562+0000 7f5d55e72640 1 -- 172.21.15.133:0/1276723415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d50114ab0 msgr2=0x7f5d50116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:25.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.562+0000 7f5d55e72640 1 --2- 172.21.15.133:0/1276723415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d50114ab0 0x7f5d50116ea0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009c10 tx=0x7f5d4002a720 comp rx=0 tx=0).stop 2024-09-11T23:39:25.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.563+0000 7f5d55e72640 1 -- 172.21.15.133:0/1276723415 shutdown_connections 2024-09-11T23:39:25.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.563+0000 7f5d55e72640 1 --2- 172.21.15.133:0/1276723415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d50114ab0 0x7f5d50116ea0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.563+0000 7f5d55e72640 1 --2- 172.21.15.133:0/1276723415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d5006ee10 0x7f5d50114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.563+0000 7f5d55e72640 1 --2- 172.21.15.133:0/1276723415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5006e4f0 0x7f5d5006e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.563+0000 7f5d55e72640 1 -- 172.21.15.133:0/1276723415 >> 172.21.15.133:0/1276723415 conn(0x7f5d50073180 msgr2=0x7f5d50073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:25.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.564+0000 7f5d55e72640 1 -- 172.21.15.133:0/1276723415 shutdown_connections 2024-09-11T23:39:25.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.565+0000 7f5d55e72640 1 -- 172.21.15.133:0/1276723415 wait complete. 2024-09-11T23:39:25.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.567+0000 7f5d55e72640 1 Processor -- start 2024-09-11T23:39:25.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.567+0000 7f5d55e72640 1 -- start start 2024-09-11T23:39:25.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.569+0000 7f5d55e72640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d5006e4f0 0x7f5d501118a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.569+0000 7f5d55e72640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5006ee10 0x7f5d50111de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.570+0000 7f5d55e72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d50114ab0 0x7f5d50112320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.570+0000 7f5d55e72640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010c060 con 0x7f5d5006ee10 2024-09-11T23:39:25.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.571+0000 7f5d55671640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d50114ab0 0x7f5d50112320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.571+0000 7f5d55671640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d50114ab0 0x7f5d50112320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:56956/0 (socket says 172.21.15.133:56956) 2024-09-11T23:39:25.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.571+0000 7f5d55671640 1 -- 172.21.15.133:0/23489801 learned_addr learned my addr 172.21.15.133:0/23489801 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:25.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.572+0000 7f5d55e72640 1 -- 172.21.15.133:0/23489801 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010bee0 con 0x7f5d5006e4f0 2024-09-11T23:39:25.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.572+0000 7f5d55e72640 1 -- 172.21.15.133:0/23489801 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010c1e0 con 0x7f5d50114ab0 2024-09-11T23:39:25.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.573+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5006ee10 0x7f5d50111de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.574+0000 7f5d4ffff640 1 -- 172.21.15.133:0/23489801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d50114ab0 msgr2=0x7f5d50112320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:25.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.574+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d50114ab0 0x7f5d50112320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.574+0000 7f5d4ffff640 1 -- 172.21.15.133:0/23489801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d5006e4f0 msgr2=0x7f5d501118a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:25.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.574+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d5006e4f0 0x7f5d501118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.574+0000 7f5d4ffff640 1 -- 172.21.15.133:0/23489801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d501b33f0 con 0x7f5d5006ee10 2024-09-11T23:39:25.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.575+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5006ee10 0x7f5d50111de0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800d6d0 tx=0x7f5d4800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:25.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.575+0000 7f5d4dffb640 1 -- 172.21.15.133:0/23489801 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48017070 con 0x7f5d5006ee10 2024-09-11T23:39:25.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.575+0000 7f5d4dffb640 1 -- 172.21.15.133:0/23489801 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d48010460 con 0x7f5d5006ee10 2024-09-11T23:39:25.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.575+0000 7f5d4dffb640 1 -- 172.21.15.133:0/23489801 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4800f620 con 0x7f5d5006ee10 2024-09-11T23:39:25.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.576+0000 7f5d55e72640 1 -- 172.21.15.133:0/23489801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d501b36e0 con 0x7f5d5006ee10 2024-09-11T23:39:25.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.576+0000 7f5d55e72640 1 -- 172.21.15.133:0/23489801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d501b3c20 con 0x7f5d5006ee10 2024-09-11T23:39:25.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.584+0000 7f5d4dffb640 1 -- 172.21.15.133:0/23489801 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5d4801a070 con 0x7f5d5006ee10 2024-09-11T23:39:25.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.586+0000 7f5d4dffb640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5d380a2f00 0x7f5d380a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.586+0000 7f5d4dffb640 1 -- 172.21.15.133:0/23489801 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f5d480cec60 con 0x7f5d5006ee10 2024-09-11T23:39:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.591+0000 7f5d594ad740 1 Processor -- start 2024-09-11T23:39:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.591+0000 7f5d594ad740 1 -- start start 2024-09-11T23:39:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.592+0000 7f5d54e70640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5d380a2f00 0x7f5d380a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.592+0000 7f5d594ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5ab2d20 0x560ab5ab1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.592+0000 7f5d594ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ab1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.592+0000 7f5d594ad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 0x560ab5abe940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.592+0000 7f5d594ad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560ab5a172f0 con 0x560ab5ab17b0 2024-09-11T23:39:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.592+0000 7f5d594ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560ab59becf0 con 0x560ab5ab2d20 2024-09-11T23:39:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.592+0000 7f5d594ad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560ab5a132b0 con 0x560ab5abc550 2024-09-11T23:39:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.594+0000 7f5d4ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ab1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.594+0000 7f5d4ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ab1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54696/0 (socket says 172.21.15.133:54696) 2024-09-11T23:39:25.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.594+0000 7f5d4ffff640 1 -- 172.21.15.133:0/1351222623 learned_addr learned my addr 172.21.15.133:0/1351222623 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.594+0000 7f5d55671640 1 --2- 172.21.15.133:0/1351222623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 0x560ab5abe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.596+0000 7f5d4ffff640 1 -- 172.21.15.133:0/1351222623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 msgr2=0x560ab5abe940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.596+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/1351222623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 0x560ab5abe940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.596+0000 7f5d4ffff640 1 -- 172.21.15.133:0/1351222623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5ab2d20 msgr2=0x560ab5ab1270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:39:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.596+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/1351222623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5ab2d20 0x560ab5ab1270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.596+0000 7f5d4ffff640 1 -- 172.21.15.133:0/1351222623 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab59a33a0 con 0x560ab5ab17b0 2024-09-11T23:39:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.597+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/1351222623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ab1bf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5d48097b60 tx=0x7f5d4809b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb34781916a2e327 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:25.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.605+0000 7f5d54e70640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5d380a2f00 0x7f5d380a53c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5d440097b0 tx=0x7f5d44006c90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:25.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.677+0000 7f5d36ffd640 1 -- 172.21.15.133:0/1351222623 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48021070 con 0x560ab5ab17b0 2024-09-11T23:39:25.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.677+0000 7f5d36ffd640 1 -- 172.21.15.133:0/1351222623 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d480a3e00 con 0x560ab5ab17b0 2024-09-11T23:39:25.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.679+0000 7f5d594ad740 1 -- 172.21.15.133:0/1351222623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 msgr2=0x560ab5ab1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:25.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.679+0000 7f5d594ad740 1 --2- 172.21.15.133:0/1351222623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ab1bf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5d48097b60 tx=0x7f5d4809b950 comp rx=0 tx=0).stop 2024-09-11T23:39:25.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.679+0000 7f5d594ad740 1 -- 172.21.15.133:0/1351222623 shutdown_connections 2024-09-11T23:39:25.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.679+0000 7f5d594ad740 1 --2- 172.21.15.133:0/1351222623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 0x560ab5abe940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.679+0000 7f5d594ad740 1 --2- 172.21.15.133:0/1351222623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ab1bf0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.679+0000 7f5d594ad740 1 --2- 172.21.15.133:0/1351222623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5ab2d20 0x560ab5ab1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.680+0000 7f5d594ad740 1 -- 172.21.15.133:0/1351222623 >> 172.21.15.133:0/1351222623 conn(0x560ab5a7a310 msgr2=0x560ab5aade40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:25.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.680+0000 7f5d594ad740 1 -- 172.21.15.133:0/1351222623 shutdown_connections 2024-09-11T23:39:25.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.680+0000 7f5d594ad740 1 -- 172.21.15.133:0/1351222623 wait complete. 2024-09-11T23:39:25.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.680+0000 7f5d594ad740 1 -- start start 2024-09-11T23:39:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.681+0000 7f5d594ad740 1 Processor -- start 2024-09-11T23:39:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.682+0000 7f5d594ad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5ab17b0 0x560ab5abbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.682+0000 7f5d594ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 0x560ab5ab9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.683+0000 7f5d594ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5abc550 0x560ab5ab99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.683+0000 7f5d594ad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560ab5abf240 con 0x560ab5ab2d20 2024-09-11T23:39:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.683+0000 7f5d594ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560ab59d2060 con 0x560ab5abc550 2024-09-11T23:39:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.683+0000 7f5d594ad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560ab5abf3c0 con 0x560ab5ab17b0 2024-09-11T23:39:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.683+0000 7f5d594ad740 10 client.0 _ll_register_callbacks cb 0x560ab57258c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.685+0000 7f5d55671640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5abc550 0x560ab5ab99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.685+0000 7f5d54e70640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5ab17b0 0x560ab5abbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.685+0000 7f5d55671640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5abc550 0x560ab5ab99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37010/0 (socket says 172.21.15.133:37010) 2024-09-11T23:39:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.685+0000 7f5d55671640 1 -- 172.21.15.133:0/1355290482 learned_addr learned my addr 172.21.15.133:0/1355290482 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.686+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 0x560ab5ab9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.687+0000 7f5d4ffff640 1 -- 172.21.15.133:0/1355290482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5ab17b0 msgr2=0x560ab5abbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:25.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.687+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5ab17b0 0x560ab5abbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.687+0000 7f5d4ffff640 1 -- 172.21.15.133:0/1355290482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5abc550 msgr2=0x560ab5ab99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:25.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.687+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5abc550 0x560ab5ab99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.688+0000 7f5d4ffff640 1 -- 172.21.15.133:0/1355290482 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab57883c0 con 0x560ab5ab2d20 2024-09-11T23:39:25.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.691+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 0x560ab5ab9230 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5d4809b120 tx=0x7f5d480996a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7479bf7bf82f8d90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:25.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.691+0000 7f5d2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.692+0000 7f5d2ffff640 1 -- 172.21.15.133:0/1355290482 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48021050 con 0x560ab5ab2d20 2024-09-11T23:39:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.692+0000 7f5d2ffff640 1 -- 172.21.15.133:0/1355290482 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f5d480ccb10 con 0x560ab5ab2d20 2024-09-11T23:39:25.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.695+0000 7f5d594ad740 1 -- 172.21.15.133:0/1355290482 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d480a3e00 con 0x560ab5ab2d20 2024-09-11T23:39:25.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.701+0000 7f5d2ffff640 1 -- 172.21.15.133:0/1355290482 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d480ccd10 con 0x560ab5ab2d20 2024-09-11T23:39:25.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.701+0000 7f5d594ad740 10 client.16281 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:25.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.701+0000 7f5d594ad740 1 -- 172.21.15.133:0/1355290482 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560ab5acad70 con 0x560ab5ab2d20 2024-09-11T23:39:25.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.707+0000 7f5d2ffff640 1 -- 172.21.15.133:0/1355290482 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d480a3e00 con 0x560ab5ab2d20 2024-09-11T23:39:25.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.707+0000 7f5d2ffff640 1 -- 172.21.15.133:0/1355290482 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5d480a06d0 con 0x560ab5ab2d20 2024-09-11T23:39:25.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.708+0000 7f5d594ad740 10 client.16281 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:25.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.708+0000 7f5d594ad740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:25.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.708+0000 7f5d594ad740 20 client.16281 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:25.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.708+0000 7f5d594ad740 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560ab5acc0c0 0x560ab5ace4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.708+0000 7f5d594ad740 4 client.16281 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:25.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.708+0000 7f5d594ad740 1 -- 172.21.15.133:0/1355290482 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ab5a24860 con 0x560ab5acc0c0 2024-09-11T23:39:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.714+0000 7f5d54e70640 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560ab5acc0c0 0x560ab5ace4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.726+0000 7f5d2ffff640 10 client.16281 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.726+0000 7f5d54e70640 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560ab5acc0c0 0x560ab5ace4b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570c35d6e915f509 server_cookie=e9b0a8aaf3640446 in_seq=0 out_seq=0 2024-09-11T23:39:25.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.731+0000 7f5d2ffff640 1 -- 172.21.15.133:0/1355290482 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ab5a24860 con 0x560ab5acc0c0 2024-09-11T23:39:25.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.732+0000 7f5d2ffff640 10 client.16281 handle_command_reply: tid=0 2024-09-11T23:39:25.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.732+0000 7f5d594ad740 1 client.16281 shutdown 2024-09-11T23:39:25.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.732+0000 7f5d594ad740 1 -- 172.21.15.133:0/1355290482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 msgr2=0x560ab5ab9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:25.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.732+0000 7f5d594ad740 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 0x560ab5ab9230 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5d4809b120 tx=0x7f5d480996a0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.735+0000 7f5d594ad740 1 -- 172.21.15.133:0/1355290482 shutdown_connections 2024-09-11T23:39:25.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.735+0000 7f5d594ad740 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560ab5acc0c0 0x560ab5ace4b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.735+0000 7f5d594ad740 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5abc550 0x560ab5ab99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.735+0000 7f5d2ffff640 0 client.16281 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:25.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.735+0000 7f5d594ad740 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 0x560ab5ab9230 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.735+0000 7f5d594ad740 1 --2- 172.21.15.133:0/1355290482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5ab17b0 0x560ab5abbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.735+0000 7f5d594ad740 1 -- 172.21.15.133:0/1355290482 >> 172.21.15.133:0/1355290482 conn(0x560ab5a7a310 msgr2=0x560ab5ab3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:25.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.743+0000 7f5d594ad740 1 -- 172.21.15.133:0/1355290482 shutdown_connections 2024-09-11T23:39:25.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.743+0000 7f5d594ad740 1 -- 172.21.15.133:0/1355290482 wait complete. 2024-09-11T23:39:25.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.744+0000 7f5d594ad740 20 client.16281 trim_cache size 0 max 16384 2024-09-11T23:39:25.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.774+0000 7f5d594ad740 1 Processor -- start 2024-09-11T23:39:25.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.774+0000 7f5d594ad740 1 -- start start 2024-09-11T23:39:25.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.775+0000 7f5d594ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5ab17b0 0x560ab5ab8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.776+0000 7f5d594ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 0x560ab5ab8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.776+0000 7f5d54e70640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5ab17b0 0x560ab5ab8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.776+0000 7f5d54e70640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5ab17b0 0x560ab5ab8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37012/0 (socket says 172.21.15.133:37012) 2024-09-11T23:39:25.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.778+0000 7f5d4ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 0x560ab5ab8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.778+0000 7f5d4ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 0x560ab5ab8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54712/0 (socket says 172.21.15.133:54712) 2024-09-11T23:39:25.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.779+0000 7f5d54e70640 1 -- 172.21.15.133:0/2135970337 learned_addr learned my addr 172.21.15.133:0/2135970337 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:25.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.779+0000 7f5d594ad740 1 --2- 172.21.15.133:0/2135970337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 0x560ab5ad14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.780+0000 7f5d594ad740 1 -- 172.21.15.133:0/2135970337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501bc530 con 0x560ab5ab2d20 2024-09-11T23:39:25.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.780+0000 7f5d594ad740 1 -- 172.21.15.133:0/2135970337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501be070 con 0x560ab5ab17b0 2024-09-11T23:39:25.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.780+0000 7f5d55671640 1 --2- 172.21.15.133:0/2135970337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 0x560ab5ad14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.781+0000 7f5d594ad740 1 -- 172.21.15.133:0/2135970337 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501bcc00 con 0x560ab5abc550 2024-09-11T23:39:25.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.782+0000 7f5d4ffff640 1 -- 172.21.15.133:0/2135970337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 msgr2=0x560ab5ad14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:25.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.782+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/2135970337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 0x560ab5ad14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.782+0000 7f5d4ffff640 1 -- 172.21.15.133:0/2135970337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5ab17b0 msgr2=0x560ab5ab8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:25.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.782+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/2135970337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5ab17b0 0x560ab5ab8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.782+0000 7f5d4ffff640 1 -- 172.21.15.133:0/2135970337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab5ab4300 con 0x560ab5ab2d20 2024-09-11T23:39:25.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.785+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/2135970337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 0x560ab5ab8210 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5d480975f0 tx=0x7f5d4809b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a7327b2f2d38ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:25.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.786+0000 7f5d367fc640 1 -- 172.21.15.133:0/2135970337 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48021070 con 0x560ab5ab2d20 2024-09-11T23:39:25.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.787+0000 7f5d367fc640 1 -- 172.21.15.133:0/2135970337 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d480a3e00 con 0x560ab5ab2d20 2024-09-11T23:39:25.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.788+0000 7f5d594ad740 1 -- 172.21.15.133:0/2135970337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 msgr2=0x560ab5ab8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:25.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.788+0000 7f5d594ad740 1 --2- 172.21.15.133:0/2135970337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 0x560ab5ab8210 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5d480975f0 tx=0x7f5d4809b950 comp rx=0 tx=0).stop 2024-09-11T23:39:25.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.790+0000 7f5d594ad740 1 -- 172.21.15.133:0/2135970337 shutdown_connections 2024-09-11T23:39:25.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.790+0000 7f5d594ad740 1 --2- 172.21.15.133:0/2135970337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 0x560ab5ad14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.790+0000 7f5d594ad740 1 --2- 172.21.15.133:0/2135970337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab2d20 0x560ab5ab8210 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.790+0000 7f5d594ad740 1 --2- 172.21.15.133:0/2135970337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5ab17b0 0x560ab5ab8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.790+0000 7f5d594ad740 1 -- 172.21.15.133:0/2135970337 >> 172.21.15.133:0/2135970337 conn(0x560ab5a7a310 msgr2=0x560ab5ab7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:25.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.791+0000 7f5d594ad740 1 -- 172.21.15.133:0/2135970337 shutdown_connections 2024-09-11T23:39:25.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.791+0000 7f5d594ad740 1 -- 172.21.15.133:0/2135970337 wait complete. 2024-09-11T23:39:25.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.792+0000 7f5d594ad740 1 -- start start 2024-09-11T23:39:25.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.792+0000 7f5d594ad740 1 Processor -- start 2024-09-11T23:39:25.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.794+0000 7f5d594ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ace890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.794+0000 7f5d594ad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 0x560ab5acedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.794+0000 7f5d594ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5acf4c0 0x560ab5b34a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.794+0000 7f5d594ad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501bd9a0 con 0x560ab5ab17b0 2024-09-11T23:39:25.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.794+0000 7f5d594ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501bd2d0 con 0x560ab5acf4c0 2024-09-11T23:39:25.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.794+0000 7f5d594ad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501ba530 con 0x560ab5abc550 2024-09-11T23:39:25.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.794+0000 7f5d594ad740 10 client.0 _ll_register_callbacks cb 0x7f5d48013030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:25.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.795+0000 7f5d55671640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5acf4c0 0x560ab5b34a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.795+0000 7f5d54e70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ace890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.795+0000 7f5d54e70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ace890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54726/0 (socket says 172.21.15.133:54726) 2024-09-11T23:39:25.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.795+0000 7f5d54e70640 1 -- 172.21.15.133:0/331328326 learned_addr learned my addr 172.21.15.133:0/331328326 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:25.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.796+0000 7f5d54e70640 1 -- 172.21.15.133:0/331328326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 msgr2=0x560ab5acedd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:39:25.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.796+0000 7f5d54e70640 1 --2- 172.21.15.133:0/331328326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 0x560ab5acedd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.796+0000 7f5d54e70640 1 -- 172.21.15.133:0/331328326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5acf4c0 msgr2=0x560ab5b34a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:25.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.796+0000 7f5d54e70640 1 --2- 172.21.15.133:0/331328326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5acf4c0 0x560ab5b34a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.796+0000 7f5d54e70640 1 -- 172.21.15.133:0/331328326 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab5b35240 con 0x560ab5ab17b0 2024-09-11T23:39:25.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.797+0000 7f5d54e70640 1 --2- 172.21.15.133:0/331328326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ace890 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5d44007010 tx=0x7f5d440047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443184490d02e4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:25.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.797+0000 7f5d2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:25.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.798+0000 7f5d2ffff640 1 -- 172.21.15.133:0/331328326 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4402d060 con 0x560ab5ab17b0 2024-09-11T23:39:25.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.798+0000 7f5d2ffff640 1 -- 172.21.15.133:0/331328326 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f5d440139d0 con 0x560ab5ab17b0 2024-09-11T23:39:25.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.798+0000 7f5d594ad740 1 -- 172.21.15.133:0/331328326 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ab5a1ed70 con 0x560ab5ab17b0 2024-09-11T23:39:25.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.799+0000 7f5d2ffff640 1 -- 172.21.15.133:0/331328326 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d44013c10 con 0x560ab5ab17b0 2024-09-11T23:39:25.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.799+0000 7f5d594ad740 10 client.16287 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:25.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.799+0000 7f5d594ad740 1 -- 172.21.15.133:0/331328326 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560ab5b36630 con 0x560ab5ab17b0 2024-09-11T23:39:25.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.800+0000 7f5d2ffff640 1 -- 172.21.15.133:0/331328326 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560ab5a1ed70 con 0x560ab5ab17b0 2024-09-11T23:39:25.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.800+0000 7f5d2ffff640 1 -- 172.21.15.133:0/331328326 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5d44030020 con 0x560ab5ab17b0 2024-09-11T23:39:25.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.801+0000 7f5d594ad740 10 client.16287 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:25.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.801+0000 7f5d594ad740 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:25.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.801+0000 7f5d594ad740 20 client.16287 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:25.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.801+0000 7f5d594ad740 1 --2- 172.21.15.133:0/331328326 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560ab5b36de0 0x560ab5b39220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:25.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.801+0000 7f5d594ad740 4 client.16287 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:39:25.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.801+0000 7f5d594ad740 1 -- 172.21.15.133:0/331328326 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ab599b170 con 0x560ab5b36de0 2024-09-11T23:39:25.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.804+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/331328326 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560ab5b36de0 0x560ab5b39220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:25.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.810+0000 7f5d4ffff640 1 --2- 172.21.15.133:0/331328326 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560ab5b36de0 0x560ab5b39220 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e937ee61092a4c66 server_cookie=fed4e83d1c6671cf in_seq=0 out_seq=0 2024-09-11T23:39:25.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.810+0000 7f5d2ffff640 10 client.16287 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:25.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.811+0000 7f5d2ffff640 1 -- 172.21.15.133:0/331328326 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ab599b170 con 0x560ab5b36de0 2024-09-11T23:39:25.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.811+0000 7f5d2ffff640 10 client.16287 handle_command_reply: tid=0 2024-09-11T23:39:25.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.811+0000 7f5d594ad740 1 client.16287 shutdown 2024-09-11T23:39:25.818 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:25 smithi133 ceph-mon[31430]: pgmap v82: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 772 MiB/s wr, 4.48k op/s 2024-09-11T23:39:25.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.816+0000 7f5d594ad740 1 -- 172.21.15.133:0/331328326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 msgr2=0x560ab5ace890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:25.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.816+0000 7f5d594ad740 1 --2- 172.21.15.133:0/331328326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ace890 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5d44007010 tx=0x7f5d440047d0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.816+0000 7f5d594ad740 1 -- 172.21.15.133:0/331328326 shutdown_connections 2024-09-11T23:39:25.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.816+0000 7f5d594ad740 1 --2- 172.21.15.133:0/331328326 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560ab5b36de0 0x560ab5b39220 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.816+0000 7f5d594ad740 1 --2- 172.21.15.133:0/331328326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab5acf4c0 0x560ab5b34a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.816+0000 7f5d594ad740 1 --2- 172.21.15.133:0/331328326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ab5abc550 0x560ab5acedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.816+0000 7f5d594ad740 1 --2- 172.21.15.133:0/331328326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560ab5ab17b0 0x560ab5ace890 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.816+0000 7f5d594ad740 1 -- 172.21.15.133:0/331328326 >> 172.21.15.133:0/331328326 conn(0x560ab5a7a310 msgr2=0x560ab5ab3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:25.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.817+0000 7f5d2ffff640 0 client.16287 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:25.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.824+0000 7f5d594ad740 1 -- 172.21.15.133:0/331328326 shutdown_connections 2024-09-11T23:39:25.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.825+0000 7f5d594ad740 1 -- 172.21.15.133:0/331328326 wait complete. 2024-09-11T23:39:25.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.825+0000 7f5d594ad740 20 client.16287 trim_cache size 0 max 16384 2024-09-11T23:39:25.828 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:39:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.829+0000 7f5d55e72640 1 -- 172.21.15.133:0/23489801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5d380a2f00 msgr2=0x7f5d380a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.829+0000 7f5d55e72640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5d380a2f00 0x7f5d380a53c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5d440097b0 tx=0x7f5d44006c90 comp rx=0 tx=0).stop 2024-09-11T23:39:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.829+0000 7f5d55e72640 1 -- 172.21.15.133:0/23489801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5006ee10 msgr2=0x7f5d50111de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.829+0000 7f5d55e72640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5006ee10 0x7f5d50111de0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800d6d0 tx=0x7f5d4800db90 comp rx=0 tx=0).stop 2024-09-11T23:39:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.830+0000 7f5d55e72640 1 -- 172.21.15.133:0/23489801 shutdown_connections 2024-09-11T23:39:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.830+0000 7f5d55e72640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5d380a2f00 0x7f5d380a53c0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5d440097b0 tx=0x7f5d44006c90 comp rx=0 tx=0).stop 2024-09-11T23:39:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.830+0000 7f5d55e72640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d50114ab0 0x7f5d50112320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.830+0000 7f5d55e72640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5006ee10 0x7f5d50111de0 secure :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800d6d0 tx=0x7f5d4800db90 comp rx=0 tx=0).stop 2024-09-11T23:39:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.830+0000 7f5d55e72640 1 --2- 172.21.15.133:0/23489801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d5006e4f0 0x7f5d501118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.830+0000 7f5d55e72640 1 -- 172.21.15.133:0/23489801 >> 172.21.15.133:0/23489801 conn(0x7f5d50073180 msgr2=0x7f5d5006b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:25.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.830+0000 7f5d55e72640 1 -- 172.21.15.133:0/23489801 shutdown_connections 2024-09-11T23:39:25.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:25.830+0000 7f5d55e72640 1 -- 172.21.15.133:0/23489801 wait complete. 2024-09-11T23:39:25.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:25 smithi138 ceph-mon[35142]: pgmap v82: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 772 MiB/s wr, 4.48k op/s 2024-09-11T23:39:26.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:25 smithi178 ceph-mon[35882]: pgmap v82: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 772 MiB/s wr, 4.48k op/s 2024-09-11T23:39:27.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:27 smithi138 ceph-mon[35142]: pgmap v83: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 665 MiB/s wr, 3.73k op/s 2024-09-11T23:39:27.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:27 smithi133 ceph-mon[31430]: pgmap v83: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 665 MiB/s wr, 3.73k op/s 2024-09-11T23:39:27.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:27 smithi178 ceph-mon[35882]: pgmap v83: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 665 MiB/s wr, 3.73k op/s 2024-09-11T23:39:28.064 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:39:28.064 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:39:29.084 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:29.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:29 smithi138 ceph-mon[35142]: pgmap v84: 97 pgs: 97 active+clean; 17 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 914 MiB/s wr, 4.87k op/s 2024-09-11T23:39:29.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:29 smithi133 ceph-mon[31430]: pgmap v84: 97 pgs: 97 active+clean; 17 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 914 MiB/s wr, 4.87k op/s 2024-09-11T23:39:29.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:29 smithi178 ceph-mon[35882]: pgmap v84: 97 pgs: 97 active+clean; 17 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 914 MiB/s wr, 4.87k op/s 2024-09-11T23:39:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.420+0000 7f09d5346640 1 -- 172.21.15.133:0/2320079503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d01008c0 msgr2=0x7f09d0100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.420+0000 7f09d5346640 1 --2- 172.21.15.133:0/2320079503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d01008c0 0x7f09d0100ca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009930 tx=0x7f09c002a420 comp rx=0 tx=0).stop 2024-09-11T23:39:30.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.430+0000 7f09d5346640 1 -- 172.21.15.133:0/2320079503 shutdown_connections 2024-09-11T23:39:30.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.430+0000 7f09d5346640 1 --2- 172.21.15.133:0/2320079503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0105ec0 0x7f09d0106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.430+0000 7f09d5346640 1 --2- 172.21.15.133:0/2320079503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09d01011e0 0x7f09d0105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.430+0000 7f09d5346640 1 --2- 172.21.15.133:0/2320079503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d01008c0 0x7f09d0100ca0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.430+0000 7f09d5346640 1 -- 172.21.15.133:0/2320079503 >> 172.21.15.133:0/2320079503 conn(0x7f09d00fc7d0 msgr2=0x7f09d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:30.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.430+0000 7f09d5346640 1 -- 172.21.15.133:0/2320079503 shutdown_connections 2024-09-11T23:39:30.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.430+0000 7f09d5346640 1 -- 172.21.15.133:0/2320079503 wait complete. 2024-09-11T23:39:30.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.431+0000 7f09d5346640 1 Processor -- start 2024-09-11T23:39:30.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.432+0000 7f09d5346640 1 -- start start 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.432+0000 7f09d5346640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d01008c0 0x7f09d010fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.433+0000 7f09ceffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d01008c0 0x7f09d010fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.433+0000 7f09ceffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d01008c0 0x7f09d010fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57014/0 (socket says 172.21.15.133:57014) 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.433+0000 7f09d5346640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09d01011e0 0x7f09d01101d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.433+0000 7f09d5346640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0105ec0 0x7f09d0108f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.434+0000 7f09d5346640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0118680 con 0x7f09d0105ec0 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.434+0000 7f09d5346640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0118500 con 0x7f09d01011e0 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.434+0000 7f09d5346640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0118800 con 0x7f09d01008c0 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.434+0000 7f09ceffd640 1 -- 172.21.15.133:0/248231064 learned_addr learned my addr 172.21.15.133:0/248231064 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.434+0000 7f09ce7fc640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09d01011e0 0x7f09d01101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.434+0000 7f09cf7fe640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0105ec0 0x7f09d0108f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.435+0000 7f09cf7fe640 1 -- 172.21.15.133:0/248231064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d01008c0 msgr2=0x7f09d010fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.435+0000 7f09cf7fe640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d01008c0 0x7f09d010fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.435+0000 7f09cf7fe640 1 -- 172.21.15.133:0/248231064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09d01011e0 msgr2=0x7f09d01101d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.435+0000 7f09cf7fe640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09d01011e0 0x7f09d01101d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.435+0000 7f09cf7fe640 1 -- 172.21.15.133:0/248231064 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d01097c0 con 0x7f09d0105ec0 2024-09-11T23:39:30.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.436+0000 7f09cf7fe640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0105ec0 0x7f09d0108f00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009fd0 tx=0x7f09c800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.438+0000 7f09b7fff640 1 -- 172.21.15.133:0/248231064 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c800ce20 con 0x7f09d0105ec0 2024-09-11T23:39:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.438+0000 7f09d5346640 1 -- 172.21.15.133:0/248231064 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d0109a50 con 0x7f09d0105ec0 2024-09-11T23:39:30.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.438+0000 7f09d5346640 1 -- 172.21.15.133:0/248231064 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d010da40 con 0x7f09d0105ec0 2024-09-11T23:39:30.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.439+0000 7f09b7fff640 1 -- 172.21.15.133:0/248231064 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c8005020 con 0x7f09d0105ec0 2024-09-11T23:39:30.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.453+0000 7f09b7fff640 1 -- 172.21.15.133:0/248231064 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8010720 con 0x7f09d0105ec0 2024-09-11T23:39:30.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.453+0000 7f09b7fff640 1 -- 172.21.15.133:0/248231064 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f09c8010980 con 0x7f09d0105ec0 2024-09-11T23:39:30.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.455+0000 7f09d771f740 1 Processor -- start 2024-09-11T23:39:30.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.456+0000 7f09d771f740 1 -- start start 2024-09-11T23:39:30.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.456+0000 7f09d771f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca5b60 0x556f2cca5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.457+0000 7f09d771f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 0x556f2ccb0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.457+0000 7f09d771f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccb3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.457+0000 7f09d771f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556f2cc0b280 con 0x556f2ccb0d40 2024-09-11T23:39:30.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.457+0000 7f09d771f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f2cbb2b10 con 0x556f2cca5b60 2024-09-11T23:39:30.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.457+0000 7f09d771f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556f2cc07240 con 0x556f2cca6480 2024-09-11T23:39:30.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.457+0000 7f09b7fff640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09b80a2f30 0x7f09b80a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.458+0000 7f09cf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccb3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.458+0000 7f09cf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccb3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54752/0 (socket says 172.21.15.133:54752) 2024-09-11T23:39:30.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.458+0000 7f09cf7fe640 1 -- 172.21.15.133:0/3532666036 learned_addr learned my addr 172.21.15.133:0/3532666036 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:30.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.458+0000 7f09b7fff640 1 -- 172.21.15.133:0/248231064 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f09c80d0bd0 con 0x7f09d0105ec0 2024-09-11T23:39:30.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.458+0000 7f09ceffd640 1 --2- 172.21.15.133:0/3532666036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca5b60 0x556f2cca5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.459+0000 7f09ce7fc640 1 --2- 172.21.15.133:0/3532666036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 0x556f2ccb0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.459+0000 7f09ceffd640 1 -- 172.21.15.133:0/3532666036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 msgr2=0x556f2ccb0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.459+0000 7f09ceffd640 1 --2- 172.21.15.133:0/3532666036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 0x556f2ccb0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.459+0000 7f09ceffd640 1 -- 172.21.15.133:0/3532666036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 msgr2=0x556f2ccb3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.459+0000 7f09ceffd640 1 --2- 172.21.15.133:0/3532666036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccb3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.459+0000 7f09ceffd640 1 -- 172.21.15.133:0/3532666036 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2cbbf1f0 con 0x556f2cca5b60 2024-09-11T23:39:30.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.459+0000 7f09cf7fe640 1 --2- 172.21.15.133:0/3532666036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccb3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:30.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.460+0000 7f09ceffd640 1 --2- 172.21.15.133:0/3532666036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca5b60 0x556f2cca5f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09c0005ab0 tx=0x7f09c0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5499a641947ef0cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.460+0000 7f09b57fa640 1 -- 172.21.15.133:0/3532666036 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0035030 con 0x556f2cca5b60 2024-09-11T23:39:30.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.460+0000 7f09b57fa640 1 -- 172.21.15.133:0/3532666036 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c0005520 con 0x556f2cca5b60 2024-09-11T23:39:30.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.461+0000 7f09d771f740 1 -- 172.21.15.133:0/3532666036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca5b60 msgr2=0x556f2cca5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.462+0000 7f09d771f740 1 --2- 172.21.15.133:0/3532666036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca5b60 0x556f2cca5f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09c0005ab0 tx=0x7f09c0004040 comp rx=0 tx=0).stop 2024-09-11T23:39:30.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.462+0000 7f09d771f740 1 -- 172.21.15.133:0/3532666036 shutdown_connections 2024-09-11T23:39:30.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.462+0000 7f09d771f740 1 --2- 172.21.15.133:0/3532666036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccb3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.462+0000 7f09d771f740 1 --2- 172.21.15.133:0/3532666036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 0x556f2ccb0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.462+0000 7f09d771f740 1 --2- 172.21.15.133:0/3532666036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca5b60 0x556f2cca5f40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.462+0000 7f09d771f740 1 -- 172.21.15.133:0/3532666036 >> 172.21.15.133:0/3532666036 conn(0x556f2cca20f0 msgr2=0x556f2cca24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:30.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.462+0000 7f09d771f740 1 -- 172.21.15.133:0/3532666036 shutdown_connections 2024-09-11T23:39:30.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.463+0000 7f09d771f740 1 -- 172.21.15.133:0/3532666036 wait complete. 2024-09-11T23:39:30.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.463+0000 7f09d771f740 1 -- start start 2024-09-11T23:39:30.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.464+0000 7f09d771f740 1 Processor -- start 2024-09-11T23:39:30.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.465+0000 7f09d771f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 0x556f2ccb06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.465+0000 7f09d771f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccad990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.465+0000 7f09d771f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2ccaded0 0x556f2ccae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.466+0000 7f09ce7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.466+0000 7f09ce7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54756/0 (socket says 172.21.15.133:54756) 2024-09-11T23:39:30.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.466+0000 7f09cf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2ccaded0 0x556f2ccae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.466+0000 7f09ce7fc640 1 -- 172.21.15.133:0/470181682 learned_addr learned my addr 172.21.15.133:0/470181682 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:30.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.466+0000 7f09d771f740 1 -- 172.21.15.133:0/470181682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556f2ccb39b0 con 0x556f2ccb0d40 2024-09-11T23:39:30.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.466+0000 7f09d771f740 1 -- 172.21.15.133:0/470181682 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f2cbc5f10 con 0x556f2ccaded0 2024-09-11T23:39:30.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.466+0000 7f09d771f740 1 -- 172.21.15.133:0/470181682 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556f2ccb3b30 con 0x556f2cca6480 2024-09-11T23:39:30.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.466+0000 7f09d771f740 10 client.0 _ll_register_callbacks cb 0x556f2cb8ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:30.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.467+0000 7f09ce7fc640 1 -- 172.21.15.133:0/470181682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 msgr2=0x556f2ccb06e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:39:30.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.467+0000 7f09ce7fc640 1 --2- 172.21.15.133:0/470181682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 0x556f2ccb06e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.467+0000 7f09ce7fc640 1 -- 172.21.15.133:0/470181682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2ccaded0 msgr2=0x556f2ccae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.467+0000 7f09ce7fc640 1 --2- 172.21.15.133:0/470181682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2ccaded0 0x556f2ccae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.467+0000 7f09ce7fc640 1 -- 172.21.15.133:0/470181682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2c97c6a0 con 0x556f2ccb0d40 2024-09-11T23:39:30.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.468+0000 7f09ce7fc640 1 --2- 172.21.15.133:0/470181682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccad990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f09c4010440 tx=0x7f09c4010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13cbb044f441fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.468+0000 7f09b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:30.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.469+0000 7f09b2ff5640 1 -- 172.21.15.133:0/470181682 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4012920 con 0x556f2ccb0d40 2024-09-11T23:39:30.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.469+0000 7f09b2ff5640 1 -- 172.21.15.133:0/470181682 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f09c4012ac0 con 0x556f2ccb0d40 2024-09-11T23:39:30.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.469+0000 7f09d771f740 1 -- 172.21.15.133:0/470181682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556f2cc187f0 con 0x556f2ccb0d40 2024-09-11T23:39:30.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.471+0000 7f09ceffd640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09b80a2f30 0x7f09b80a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.474+0000 7f09ceffd640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09b80a2f30 0x7f09b80a53f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09c002a460 tx=0x7f09c0005a20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.474+0000 7f09b2ff5640 1 -- 172.21.15.133:0/470181682 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f09c4012cc0 con 0x556f2ccb0d40 2024-09-11T23:39:30.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.476+0000 7f09d771f740 10 client.16299 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:30.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.476+0000 7f09d771f740 1 -- 172.21.15.133:0/470181682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f2ccbf920 con 0x556f2ccb0d40 2024-09-11T23:39:30.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.478+0000 7f09b2ff5640 1 -- 172.21.15.133:0/470181682 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556f2cc187f0 con 0x556f2ccb0d40 2024-09-11T23:39:30.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.478+0000 7f09b2ff5640 1 -- 172.21.15.133:0/470181682 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f09c400f040 con 0x556f2ccb0d40 2024-09-11T23:39:30.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.478+0000 7f09d771f740 10 client.16299 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:30.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.478+0000 7f09d771f740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:30.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.478+0000 7f09d771f740 20 client.16299 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:30.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.478+0000 7f09d771f740 1 --2- 172.21.15.133:0/470181682 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556f2ccc0eb0 0x556f2ccc32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.478+0000 7f09d771f740 4 client.16299 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:30.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.478+0000 7f09d771f740 1 -- 172.21.15.133:0/470181682 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f2cc12d00 con 0x556f2ccc0eb0 2024-09-11T23:39:30.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.479+0000 7f09ceffd640 1 --2- 172.21.15.133:0/470181682 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556f2ccc0eb0 0x556f2ccc32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.480+0000 7f09b2ff5640 10 client.16299 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:30.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.481+0000 7f09ceffd640 1 --2- 172.21.15.133:0/470181682 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556f2ccc0eb0 0x556f2ccc32d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79deab32abfa5aff server_cookie=b6f83f7b6c415512 in_seq=0 out_seq=0 2024-09-11T23:39:30.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.483+0000 7f09b2ff5640 1 -- 172.21.15.133:0/470181682 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f2cc12d00 con 0x556f2ccc0eb0 2024-09-11T23:39:30.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.483+0000 7f09b2ff5640 10 client.16299 handle_command_reply: tid=0 2024-09-11T23:39:30.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.483+0000 7f09d771f740 1 client.16299 shutdown 2024-09-11T23:39:30.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.484+0000 7f09d771f740 1 -- 172.21.15.133:0/470181682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 msgr2=0x556f2ccad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.484+0000 7f09d771f740 1 --2- 172.21.15.133:0/470181682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccad990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f09c4010440 tx=0x7f09c4010900 comp rx=0 tx=0).stop 2024-09-11T23:39:30.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.485+0000 7f09d771f740 1 -- 172.21.15.133:0/470181682 shutdown_connections 2024-09-11T23:39:30.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.485+0000 7f09d771f740 1 --2- 172.21.15.133:0/470181682 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556f2ccc0eb0 0x556f2ccc32d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.485+0000 7f09d771f740 1 --2- 172.21.15.133:0/470181682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2ccaded0 0x556f2ccae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.485+0000 7f09d771f740 1 --2- 172.21.15.133:0/470181682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccb0d40 0x556f2ccad990 secure :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f09c4010440 tx=0x7f09c4010900 comp rx=0 tx=0).stop 2024-09-11T23:39:30.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.485+0000 7f09d771f740 1 --2- 172.21.15.133:0/470181682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 0x556f2ccb06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.485+0000 7f09d771f740 1 -- 172.21.15.133:0/470181682 >> 172.21.15.133:0/470181682 conn(0x556f2cca20f0 msgr2=0x556f2cca7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:30.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.485+0000 7f09b2ff5640 0 client.16299 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:30.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.486+0000 7f09d771f740 1 -- 172.21.15.133:0/470181682 shutdown_connections 2024-09-11T23:39:30.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.486+0000 7f09d771f740 1 -- 172.21.15.133:0/470181682 wait complete. 2024-09-11T23:39:30.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.486+0000 7f09d771f740 20 client.16299 trim_cache size 0 max 16384 2024-09-11T23:39:30.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.536+0000 7f09d771f740 1 Processor -- start 2024-09-11T23:39:30.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.536+0000 7f09d771f740 1 -- start start 2024-09-11T23:39:30.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.536+0000 7f09d771f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 0x556f2ccac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.537+0000 7f09d771f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccaded0 0x556f2ccac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.537+0000 7f09d771f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca75b0 0x556f2cca7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.537+0000 7f09d771f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc00e760 con 0x556f2ccaded0 2024-09-11T23:39:30.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.537+0000 7f09d771f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc0102a0 con 0x556f2cca75b0 2024-09-11T23:39:30.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.537+0000 7f09d771f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc00ee30 con 0x556f2cca6480 2024-09-11T23:39:30.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.540+0000 7f09cf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca75b0 0x556f2cca7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.540+0000 7f09cf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca75b0 0x556f2cca7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37046/0 (socket says 172.21.15.133:37046) 2024-09-11T23:39:30.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.540+0000 7f09cf7fe640 1 -- 172.21.15.133:0/1062883003 learned_addr learned my addr 172.21.15.133:0/1062883003 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:30.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.540+0000 7f09ceffd640 1 --2- 172.21.15.133:0/1062883003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 0x556f2ccac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.540+0000 7f09cf7fe640 1 -- 172.21.15.133:0/1062883003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 msgr2=0x556f2ccac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.540+0000 7f09cf7fe640 1 --2- 172.21.15.133:0/1062883003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 0x556f2ccac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.540+0000 7f09cf7fe640 1 -- 172.21.15.133:0/1062883003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccaded0 msgr2=0x556f2ccac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.540+0000 7f09cf7fe640 1 --2- 172.21.15.133:0/1062883003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccaded0 0x556f2ccac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.540+0000 7f09cf7fe640 1 -- 172.21.15.133:0/1062883003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2ccb0300 con 0x556f2cca75b0 2024-09-11T23:39:30.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.541+0000 7f09cf7fe640 1 --2- 172.21.15.133:0/1062883003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca75b0 0x556f2cca7a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f09c8099b50 tx=0x7f09c8013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4f759a58266cda server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.541+0000 7f09b4ff9640 1 -- 172.21.15.133:0/1062883003 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c80b6070 con 0x556f2cca75b0 2024-09-11T23:39:30.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.541+0000 7f09b4ff9640 1 -- 172.21.15.133:0/1062883003 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c809d180 con 0x556f2cca75b0 2024-09-11T23:39:30.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.541+0000 7f09b4ff9640 1 -- 172.21.15.133:0/1062883003 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c80bd0b0 con 0x556f2cca75b0 2024-09-11T23:39:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.546+0000 7f41f48a8640 1 -- 172.21.15.133:0/516622301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f0074620 msgr2=0x7f41f007b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.546+0000 7f41f48a8640 1 --2- 172.21.15.133:0/516622301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f0074620 0x7f41f007b000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009ca0 tx=0x7f41e802ba10 comp rx=0 tx=0).stop 2024-09-11T23:39:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.546+0000 7f41f48a8640 1 -- 172.21.15.133:0/516622301 shutdown_connections 2024-09-11T23:39:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.546+0000 7f41f48a8640 1 --2- 172.21.15.133:0/516622301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f0074620 0x7f41f007b000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.546+0000 7f41f48a8640 1 --2- 172.21.15.133:0/516622301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41f0109e10 0x7f41f0073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.546+0000 7f41f48a8640 1 --2- 172.21.15.133:0/516622301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41f0109460 0x7f41f0109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.546+0000 7f41f48a8640 1 -- 172.21.15.133:0/516622301 >> 172.21.15.133:0/516622301 conn(0x7f41f006cbf0 msgr2=0x7f41f006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.546+0000 7f41f48a8640 1 -- 172.21.15.133:0/516622301 shutdown_connections 2024-09-11T23:39:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.546+0000 7f41f48a8640 1 -- 172.21.15.133:0/516622301 wait complete. 2024-09-11T23:39:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.547+0000 7f41f48a8640 1 Processor -- start 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.543+0000 7f09d771f740 1 -- 172.21.15.133:0/1062883003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca75b0 msgr2=0x556f2cca7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.543+0000 7f09d771f740 1 --2- 172.21.15.133:0/1062883003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca75b0 0x556f2cca7a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f09c8099b50 tx=0x7f09c8013000 comp rx=0 tx=0).stop 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.543+0000 7f09d771f740 1 -- 172.21.15.133:0/1062883003 shutdown_connections 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.543+0000 7f09d771f740 1 --2- 172.21.15.133:0/1062883003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2cca75b0 0x556f2cca7a10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.543+0000 7f09d771f740 1 --2- 172.21.15.133:0/1062883003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2ccaded0 0x556f2ccac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.543+0000 7f09d771f740 1 --2- 172.21.15.133:0/1062883003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca6480 0x556f2ccac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.543+0000 7f09d771f740 1 -- 172.21.15.133:0/1062883003 >> 172.21.15.133:0/1062883003 conn(0x556f2cca20f0 msgr2=0x556f2cca8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.543+0000 7f09d771f740 1 -- 172.21.15.133:0/1062883003 shutdown_connections 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.543+0000 7f09d771f740 1 -- 172.21.15.133:0/1062883003 wait complete. 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.544+0000 7f09d771f740 1 -- start start 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.544+0000 7f09d771f740 1 Processor -- start 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.545+0000 7f09d771f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2cca6480 0x556f2ccb8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.545+0000 7f09d771f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca75b0 0x556f2ccb93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.545+0000 7f09d771f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2ccaded0 0x556f2cca9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.545+0000 7f09d771f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc00fbd0 con 0x556f2cca6480 2024-09-11T23:39:30.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.545+0000 7f09d771f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc00f500 con 0x556f2ccaded0 2024-09-11T23:39:30.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.545+0000 7f09d771f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc00c740 con 0x556f2cca75b0 2024-09-11T23:39:30.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.545+0000 7f09d771f740 10 client.0 _ll_register_callbacks cb 0x7f09c4005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.548+0000 7f41f48a8640 1 -- start start 2024-09-11T23:39:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.549+0000 7f41f48a8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41f0109460 0x7f41f0084890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.549+0000 7f41f48a8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41f0084dd0 0x7f41f007f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.549+0000 7f41f48a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f007fe50 0x7f41f0080300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.549+0000 7f41f48a8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f41f007e5d0 con 0x7f41f0084dd0 2024-09-11T23:39:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.549+0000 7f41f48a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41f007e450 con 0x7f41f007fe50 2024-09-11T23:39:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.549+0000 7f41f48a8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41f007e750 con 0x7f41f0109460 2024-09-11T23:39:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f41ef7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41f0109460 0x7f41f0084890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f41ef7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41f0109460 0x7f41f0084890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57062/0 (socket says 172.21.15.133:57062) 2024-09-11T23:39:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f41ef7fe640 1 -- 172.21.15.133:0/3627172464 learned_addr learned my addr 172.21.15.133:0/3627172464 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f41effff640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f007fe50 0x7f41f0080300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f41effff640 1 -- 172.21.15.133:0/3627172464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41f0109460 msgr2=0x7f41f0084890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f41effff640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41f0109460 0x7f41f0084890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f41effff640 1 -- 172.21.15.133:0/3627172464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41f0084dd0 msgr2=0x7f41f007f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f41effff640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41f0084dd0 0x7f41f007f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.551+0000 7f41effff640 1 -- 172.21.15.133:0/3627172464 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f01b5090 con 0x7f41f007fe50 2024-09-11T23:39:30.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f09ce7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca75b0 0x556f2ccb93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f09cf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2ccaded0 0x556f2cca9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f09ce7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca75b0 0x556f2ccb93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57050/0 (socket says 172.21.15.133:57050) 2024-09-11T23:39:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.550+0000 7f09ce7fc640 1 -- 172.21.15.133:0/3587636657 learned_addr learned my addr 172.21.15.133:0/3587636657 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.551+0000 7f09ceffd640 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2cca6480 0x556f2ccb8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.552+0000 7f41effff640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f007fe50 0x7f41f0080300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f41e802c000 tx=0x7f41e80308a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.552+0000 7f09ce7fc640 1 -- 172.21.15.133:0/3587636657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2ccaded0 msgr2=0x556f2cca9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.552+0000 7f09ce7fc640 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2ccaded0 0x556f2cca9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.552+0000 7f09ce7fc640 1 -- 172.21.15.133:0/3587636657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2cca6480 msgr2=0x556f2ccb8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.552+0000 7f09ce7fc640 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2cca6480 0x556f2ccb8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.552+0000 7f09ce7fc640 1 -- 172.21.15.133:0/3587636657 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2ccb99e0 con 0x556f2cca75b0 2024-09-11T23:39:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.553+0000 7f41ecff9640 1 -- 172.21.15.133:0/3627172464 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8034040 con 0x7f41f007fe50 2024-09-11T23:39:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.553+0000 7f41f48a8640 1 -- 172.21.15.133:0/3627172464 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f01b5320 con 0x7f41f007fe50 2024-09-11T23:39:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.553+0000 7f41f48a8640 1 -- 172.21.15.133:0/3627172464 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f01b5810 con 0x7f41f007fe50 2024-09-11T23:39:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.553+0000 7f41ecff9640 1 -- 172.21.15.133:0/3627172464 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e8003970 con 0x7f41f007fe50 2024-09-11T23:39:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.553+0000 7f09ce7fc640 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca75b0 0x556f2ccb93a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09c4011770 tx=0x7f09c4011dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b92254241698de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.554+0000 7f09b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:39:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.554+0000 7f09b2ff5640 1 -- 172.21.15.133:0/3587636657 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4012e60 con 0x556f2cca75b0 2024-09-11T23:39:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.554+0000 7f09b2ff5640 1 -- 172.21.15.133:0/3587636657 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f09c4012930 con 0x556f2cca75b0 2024-09-11T23:39:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.555+0000 7f09d771f740 1 -- 172.21.15.133:0/3587636657 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09c809d180 con 0x556f2cca75b0 2024-09-11T23:39:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.555+0000 7f09b2ff5640 1 -- 172.21.15.133:0/3587636657 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f09c4010f30 con 0x556f2cca75b0 2024-09-11T23:39:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.555+0000 7f09d771f740 10 client.25720 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.555+0000 7f09d771f740 1 -- 172.21.15.133:0/3587636657 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f2cd4f120 con 0x556f2cca75b0 2024-09-11T23:39:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.563+0000 7f41ecff9640 1 -- 172.21.15.133:0/3627172464 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8004440 con 0x7f41f007fe50 2024-09-11T23:39:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.563+0000 7f41ecff9640 1 -- 172.21.15.133:0/3627172464 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f41e8037070 con 0x7f41f007fe50 2024-09-11T23:39:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.560+0000 7f09b2ff5640 1 -- 172.21.15.133:0/3587636657 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c809d180 con 0x556f2cca75b0 2024-09-11T23:39:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.560+0000 7f09b2ff5640 1 -- 172.21.15.133:0/3587636657 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f09c4004820 con 0x556f2cca75b0 2024-09-11T23:39:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.562+0000 7f09d771f740 10 client.25720 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.562+0000 7f09d771f740 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.562+0000 7f09d771f740 20 client.25720 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.562+0000 7f09d771f740 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556f2cd4f7e0 0x556f2cd51c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.562+0000 7f09d771f740 4 client.25720 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:39:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.562+0000 7f09d771f740 1 -- 172.21.15.133:0/3587636657 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556f2cb8f110 con 0x556f2cd4f7e0 2024-09-11T23:39:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.566+0000 7f09ceffd640 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556f2cd4f7e0 0x556f2cd51c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.571+0000 7f41f88a1740 1 Processor -- start 2024-09-11T23:39:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.572+0000 7f41ecff9640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f41d80a2d60 0x7f41d80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.572+0000 7f41ecff9640 1 -- 172.21.15.133:0/3627172464 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f41e8030af0 con 0x7f41f007fe50 2024-09-11T23:39:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.573+0000 7f41f88a1740 1 -- start start 2024-09-11T23:39:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.573+0000 7f41f88a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc112cc0 0x55e7bc1130a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.574+0000 7f09ceffd640 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556f2cd4f7e0 0x556f2cd51c00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=615581a4e4105322 server_cookie=a17cbaad8197717d in_seq=0 out_seq=0 2024-09-11T23:39:30.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.574+0000 7f09b2ff5640 10 client.25720 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.575+0000 7f41f88a1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc11d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.575+0000 7f41ef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc112cc0 0x55e7bc1130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.575+0000 7f41ef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc112cc0 0x55e7bc1130a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54820/0 (socket says 172.21.15.133:54820) 2024-09-11T23:39:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.576+0000 7f41ef7fe640 1 -- 172.21.15.133:0/3600450563 learned_addr learned my addr 172.21.15.133:0/3600450563 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.576+0000 7f41f88a1740 1 --2- 172.21.15.133:0/3600450563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc11de80 0x55e7bc120270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.576+0000 7f41f88a1740 1 -- 172.21.15.133:0/3600450563 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e7bc0772f0 con 0x55e7bc112cc0 2024-09-11T23:39:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.576+0000 7f41f88a1740 1 -- 172.21.15.133:0/3600450563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e7bc01f270 con 0x55e7bc11de80 2024-09-11T23:39:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.576+0000 7f41f88a1740 1 -- 172.21.15.133:0/3600450563 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e7bc0732b0 con 0x55e7bc1135e0 2024-09-11T23:39:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.576+0000 7f41eeffd640 1 --2- 172.21.15.133:0/3600450563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc11d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.576+0000 7f41eeffd640 1 -- 172.21.15.133:0/3600450563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc11de80 msgr2=0x55e7bc120270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.576+0000 7f41eeffd640 1 --2- 172.21.15.133:0/3600450563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc11de80 0x55e7bc120270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.576+0000 7f41eeffd640 1 -- 172.21.15.133:0/3600450563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc112cc0 msgr2=0x55e7bc1130a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.576+0000 7f41eeffd640 1 --2- 172.21.15.133:0/3600450563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc112cc0 0x55e7bc1130a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.577+0000 7f41eeffd640 1 -- 172.21.15.133:0/3600450563 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7bc01e340 con 0x55e7bc1135e0 2024-09-11T23:39:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.577+0000 7f41eeffd640 1 --2- 172.21.15.133:0/3600450563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc11d940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f41e000bd10 tx=0x7f41e000d7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dce91a8923f97b56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.578+0000 7f41eeffd640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f41d80a2d60 0x7f41d80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.580+0000 7f41eeffd640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f41d80a2d60 0x7f41d80a5220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f41e0015fd0 tx=0x7f41e0015f40 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.580+0000 7f41d5ffb640 1 -- 172.21.15.133:0/3600450563 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0004c60 con 0x55e7bc1135e0 2024-09-11T23:39:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.580+0000 7f41d5ffb640 1 -- 172.21.15.133:0/3600450563 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e0004e00 con 0x55e7bc1135e0 2024-09-11T23:39:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.580+0000 7f41d5ffb640 1 -- 172.21.15.133:0/3600450563 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0015b90 con 0x55e7bc1135e0 2024-09-11T23:39:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.582+0000 7f41f88a1740 1 -- 172.21.15.133:0/3600450563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 msgr2=0x55e7bc11d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.582+0000 7f41f88a1740 1 --2- 172.21.15.133:0/3600450563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc11d940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f41e000bd10 tx=0x7f41e000d7e0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.583+0000 7f41f88a1740 1 -- 172.21.15.133:0/3600450563 shutdown_connections 2024-09-11T23:39:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.583+0000 7f41f88a1740 1 --2- 172.21.15.133:0/3600450563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc11de80 0x55e7bc120270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.583+0000 7f41f88a1740 1 --2- 172.21.15.133:0/3600450563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc11d940 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.583+0000 7f41f88a1740 1 --2- 172.21.15.133:0/3600450563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc112cc0 0x55e7bc1130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.583+0000 7f41f88a1740 1 -- 172.21.15.133:0/3600450563 >> 172.21.15.133:0/3600450563 conn(0x55e7bc10f230 msgr2=0x55e7bc10f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.583+0000 7f41f88a1740 1 -- 172.21.15.133:0/3600450563 shutdown_connections 2024-09-11T23:39:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.583+0000 7f41f88a1740 1 -- 172.21.15.133:0/3600450563 wait complete. 2024-09-11T23:39:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.584+0000 7f41f88a1740 1 -- start start 2024-09-11T23:39:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.585+0000 7f41f88a1740 1 Processor -- start 2024-09-11T23:39:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.586+0000 7f41f88a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc112cc0 0x55e7bc11ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.587+0000 7f41f88a1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc11b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.587+0000 7f41f88a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc11de80 0x55e7bc11b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.587+0000 7f41f88a1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e7bc120a50 con 0x55e7bc112cc0 2024-09-11T23:39:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.587+0000 7f41effff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc11de80 0x55e7bc11b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.588+0000 7f41effff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc11de80 0x55e7bc11b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37102/0 (socket says 172.21.15.133:37102) 2024-09-11T23:39:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.588+0000 7f41effff640 1 -- 172.21.15.133:0/1413175675 learned_addr learned my addr 172.21.15.133:0/1413175675 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.588+0000 7f41f88a1740 1 -- 172.21.15.133:0/1413175675 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e7bc000290 con 0x55e7bc11de80 2024-09-11T23:39:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.588+0000 7f41eeffd640 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc11b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.588+0000 7f41f88a1740 1 -- 172.21.15.133:0/1413175675 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e7bc120bd0 con 0x55e7bc1135e0 2024-09-11T23:39:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.591+0000 7f41ef7fe640 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc112cc0 0x55e7bc11ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.592+0000 7f41f88a1740 10 client.0 _ll_register_callbacks cb 0x55e7bc031140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.601+0000 7f41eeffd640 1 -- 172.21.15.133:0/1413175675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc11de80 msgr2=0x55e7bc11b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.601+0000 7f41eeffd640 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc11de80 0x55e7bc11b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.601+0000 7f41eeffd640 1 -- 172.21.15.133:0/1413175675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc112cc0 msgr2=0x55e7bc11ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.601+0000 7f41eeffd640 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc112cc0 0x55e7bc11ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.601+0000 7f41eeffd640 1 -- 172.21.15.133:0/1413175675 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7bbd8a210 con 0x55e7bc1135e0 2024-09-11T23:39:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f41eeffd640 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc11b2e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f41e0016fd0 tx=0x7f41e0015df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57a7c4012fcccd72 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09b2ff5640 1 -- 172.21.15.133:0/3587636657 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556f2cb8f110 con 0x556f2cd4f7e0 2024-09-11T23:39:30.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09b2ff5640 10 client.25720 handle_command_reply: tid=0 2024-09-11T23:39:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f41ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:39:30.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09d771f740 1 client.25720 shutdown 2024-09-11T23:39:30.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09d771f740 1 -- 172.21.15.133:0/3587636657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca75b0 msgr2=0x556f2ccb93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09d771f740 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca75b0 0x556f2ccb93a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09c4011770 tx=0x7f09c4011dd0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09d771f740 1 -- 172.21.15.133:0/3587636657 shutdown_connections 2024-09-11T23:39:30.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09d771f740 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556f2cd4f7e0 0x556f2cd51c00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09d771f740 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f2ccaded0 0x556f2cca9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09d771f740 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556f2cca75b0 0x556f2ccb93a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09d771f740 1 --2- 172.21.15.133:0/3587636657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f2cca6480 0x556f2ccb8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09d771f740 1 -- 172.21.15.133:0/3587636657 >> 172.21.15.133:0/3587636657 conn(0x556f2cca20f0 msgr2=0x556f2cca8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:30.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.605+0000 7f09b2ff5640 0 client.25720 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.606+0000 7f41ceffd640 1 -- 172.21.15.133:0/1413175675 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e001b040 con 0x55e7bc1135e0 2024-09-11T23:39:30.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.606+0000 7f09d771f740 1 -- 172.21.15.133:0/3587636657 shutdown_connections 2024-09-11T23:39:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.606+0000 7f41f88a1740 1 -- 172.21.15.133:0/1413175675 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41e0004e00 con 0x55e7bc1135e0 2024-09-11T23:39:30.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.606+0000 7f09d771f740 1 -- 172.21.15.133:0/3587636657 wait complete. 2024-09-11T23:39:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.606+0000 7f41ceffd640 1 -- 172.21.15.133:0/1413175675 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f41e00164a0 con 0x55e7bc1135e0 2024-09-11T23:39:30.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.606+0000 7f09d771f740 20 client.25720 trim_cache size 0 max 16384 2024-09-11T23:39:30.610 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:39:30.610 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:39:30.610 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "4a10f6cf-eed6-4f16-a7d6-8c0a31be47ed", 2024-09-11T23:39:30.610 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:39:30.610 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:39:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.607+0000 7f41ceffd640 1 -- 172.21.15.133:0/1413175675 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41e0016800 con 0x55e7bc1135e0 2024-09-11T23:39:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.607+0000 7f41f88a1740 10 client.25726 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.607+0000 7f41f88a1740 1 -- 172.21.15.133:0/1413175675 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e7bbe56620 con 0x55e7bc1135e0 2024-09-11T23:39:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 -- 172.21.15.133:0/248231064 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09b80a2f30 msgr2=0x7f09b80a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09b80a2f30 0x7f09b80a53f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09c002a460 tx=0x7f09c0005a20 comp rx=0 tx=0).stop 2024-09-11T23:39:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 -- 172.21.15.133:0/248231064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0105ec0 msgr2=0x7f09d0108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0105ec0 0x7f09d0108f00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009fd0 tx=0x7f09c800ef10 comp rx=0 tx=0).stop 2024-09-11T23:39:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 -- 172.21.15.133:0/248231064 shutdown_connections 2024-09-11T23:39:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09b80a2f30 0x7f09b80a53f0 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09c002a460 tx=0x7f09c0005a20 comp rx=0 tx=0).stop 2024-09-11T23:39:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0105ec0 0x7f09d0108f00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09d01011e0 0x7f09d01101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 --2- 172.21.15.133:0/248231064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d01008c0 0x7f09d010fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 -- 172.21.15.133:0/248231064 >> 172.21.15.133:0/248231064 conn(0x7f09d00fc7d0 msgr2=0x7f09d0103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f41ceffd640 1 -- 172.21.15.133:0/1413175675 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e0004e00 con 0x55e7bc1135e0 2024-09-11T23:39:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 -- 172.21.15.133:0/248231064 shutdown_connections 2024-09-11T23:39:30.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:30.608+0000 7f09d5346640 1 -- 172.21.15.133:0/248231064 wait complete. 2024-09-11T23:39:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.610+0000 7f41ceffd640 1 -- 172.21.15.133:0/1413175675 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f41e0014070 con 0x55e7bc1135e0 2024-09-11T23:39:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.610+0000 7f41f88a1740 10 client.25726 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.610+0000 7f41f88a1740 10 client.25726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.610+0000 7f41f88a1740 20 client.25726 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.610+0000 7f41f88a1740 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7bc12e0b0 0x55e7bc130530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.610+0000 7f41f88a1740 4 client.25726 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.610+0000 7f41f88a1740 1 -- 172.21.15.133:0/1413175675 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7bc084a20 con 0x55e7bc12e0b0 2024-09-11T23:39:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.611+0000 7f41ef7fe640 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7bc12e0b0 0x55e7bc130530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.614+0000 7f41ef7fe640 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7bc12e0b0 0x55e7bc130530 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa12eed3f9b357ee server_cookie=2f1148de73499f65 in_seq=0 out_seq=0 2024-09-11T23:39:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.614+0000 7f41ceffd640 10 client.25726 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.623+0000 7f41ceffd640 1 -- 172.21.15.133:0/1413175675 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7bc084a20 con 0x55e7bc12e0b0 2024-09-11T23:39:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.623+0000 7f41ceffd640 10 client.25726 handle_command_reply: tid=0 2024-09-11T23:39:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.623+0000 7f41f88a1740 1 client.25726 shutdown 2024-09-11T23:39:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.628+0000 7f41f88a1740 1 -- 172.21.15.133:0/1413175675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 msgr2=0x55e7bc11b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.628+0000 7f41f88a1740 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc11b2e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f41e0016fd0 tx=0x7f41e0015df0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.628+0000 7f41f88a1740 1 -- 172.21.15.133:0/1413175675 shutdown_connections 2024-09-11T23:39:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.628+0000 7f41f88a1740 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7bc12e0b0 0x55e7bc130530 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.628+0000 7f41f88a1740 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc11de80 0x55e7bc11b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.628+0000 7f41f88a1740 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc11b2e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.628+0000 7f41f88a1740 1 --2- 172.21.15.133:0/1413175675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc112cc0 0x55e7bc11ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.628+0000 7f41f88a1740 1 -- 172.21.15.133:0/1413175675 >> 172.21.15.133:0/1413175675 conn(0x55e7bc10f230 msgr2=0x55e7bc114f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.628+0000 7f41ceffd640 0 client.25726 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.628+0000 7f41f88a1740 1 -- 172.21.15.133:0/1413175675 shutdown_connections 2024-09-11T23:39:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.637+0000 7f41f88a1740 1 -- 172.21.15.133:0/1413175675 wait complete. 2024-09-11T23:39:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.637+0000 7f41f88a1740 20 client.25726 trim_cache size 0 max 16384 2024-09-11T23:39:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.738+0000 7f41f88a1740 1 Processor -- start 2024-09-11T23:39:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.738+0000 7f41f88a1740 1 -- start start 2024-09-11T23:39:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.738+0000 7f41f88a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc112cc0 0x55e7bc115780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.739+0000 7f41f88a1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc119650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.739+0000 7f41f88a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc11de80 0x55e7bc11b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.739+0000 7f41f88a1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc00e570 con 0x55e7bc11de80 2024-09-11T23:39:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.739+0000 7f41f88a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc0100b0 con 0x55e7bc112cc0 2024-09-11T23:39:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.739+0000 7f41f88a1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc00ec40 con 0x55e7bc1135e0 2024-09-11T23:39:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.743+0000 7f41eeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc119650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.743+0000 7f41eeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc119650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57070/0 (socket says 172.21.15.133:57070) 2024-09-11T23:39:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.743+0000 7f41eeffd640 1 -- 172.21.15.133:0/2012882185 learned_addr learned my addr 172.21.15.133:0/2012882185 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.745+0000 7f41ef7fe640 1 --2- 172.21.15.133:0/2012882185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc112cc0 0x55e7bc115780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.753+0000 7f41eeffd640 1 -- 172.21.15.133:0/2012882185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc112cc0 msgr2=0x55e7bc115780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.753+0000 7f41eeffd640 1 --2- 172.21.15.133:0/2012882185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc112cc0 0x55e7bc115780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.753+0000 7f41eeffd640 1 -- 172.21.15.133:0/2012882185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc11de80 msgr2=0x55e7bc11b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.753+0000 7f41eeffd640 1 --2- 172.21.15.133:0/2012882185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc11de80 0x55e7bc11b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.753+0000 7f41eeffd640 1 -- 172.21.15.133:0/2012882185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7bc113410 con 0x55e7bc1135e0 2024-09-11T23:39:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.760+0000 7f41eeffd640 1 --2- 172.21.15.133:0/2012882185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc119650 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b380 tx=0x7f41e000b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65428249cde6f5db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.764+0000 7f41d57fa640 1 -- 172.21.15.133:0/2012882185 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0011020 con 0x55e7bc1135e0 2024-09-11T23:39:30.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.765+0000 7f41d57fa640 1 -- 172.21.15.133:0/2012882185 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e0006070 con 0x55e7bc1135e0 2024-09-11T23:39:30.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.766+0000 7f41f88a1740 1 -- 172.21.15.133:0/2012882185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 msgr2=0x55e7bc119650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:30.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.767+0000 7f41f88a1740 1 --2- 172.21.15.133:0/2012882185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc119650 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b380 tx=0x7f41e000b650 comp rx=0 tx=0).stop 2024-09-11T23:39:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.771+0000 7f41f88a1740 1 -- 172.21.15.133:0/2012882185 shutdown_connections 2024-09-11T23:39:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.771+0000 7f41f88a1740 1 --2- 172.21.15.133:0/2012882185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc11de80 0x55e7bc11b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.771+0000 7f41f88a1740 1 --2- 172.21.15.133:0/2012882185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc119650 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.771+0000 7f41f88a1740 1 --2- 172.21.15.133:0/2012882185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc112cc0 0x55e7bc115780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.771+0000 7f41f88a1740 1 -- 172.21.15.133:0/2012882185 >> 172.21.15.133:0/2012882185 conn(0x55e7bc10f230 msgr2=0x55e7bc11ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.784+0000 7f41f88a1740 1 -- 172.21.15.133:0/2012882185 shutdown_connections 2024-09-11T23:39:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.784+0000 7f41f88a1740 1 -- 172.21.15.133:0/2012882185 wait complete. 2024-09-11T23:39:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.785+0000 7f41f88a1740 1 -- start start 2024-09-11T23:39:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.785+0000 7f41f88a1740 1 Processor -- start 2024-09-11T23:39:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.786+0000 7f41f88a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc112cc0 0x55e7bc1263c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.786+0000 7f41f88a1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc1216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.787+0000 7f41f88a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc11de80 0x55e7bc121c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.787+0000 7f41f88a1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc00f9e0 con 0x55e7bc11de80 2024-09-11T23:39:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.787+0000 7f41eeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc1216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.787+0000 7f41eeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc1216e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57072/0 (socket says 172.21.15.133:57072) 2024-09-11T23:39:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.787+0000 7f41eeffd640 1 -- 172.21.15.133:0/3014928636 learned_addr learned my addr 172.21.15.133:0/3014928636 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.787+0000 7f41f88a1740 1 -- 172.21.15.133:0/3014928636 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc00f310 con 0x55e7bc112cc0 2024-09-11T23:39:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.788+0000 7f41ef7fe640 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc112cc0 0x55e7bc1263c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.788+0000 7f41f88a1740 1 -- 172.21.15.133:0/3014928636 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc00c600 con 0x55e7bc1135e0 2024-09-11T23:39:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.788+0000 7f41f88a1740 10 client.0 _ll_register_callbacks cb 0x7f41e000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.789+0000 7f41eeffd640 1 -- 172.21.15.133:0/3014928636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc112cc0 msgr2=0x55e7bc1263c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.789+0000 7f41eeffd640 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc112cc0 0x55e7bc1263c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.789+0000 7f41eeffd640 1 -- 172.21.15.133:0/3014928636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc11de80 msgr2=0x55e7bc121c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.789+0000 7f41eeffd640 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc11de80 0x55e7bc121c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.789+0000 7f41eeffd640 1 -- 172.21.15.133:0/3014928636 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7bc1221e0 con 0x55e7bc1135e0 2024-09-11T23:39:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.794+0000 7f41eeffd640 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc1216e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55e7bc0733b0 tx=0x7f41e000b4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72de7c0f0dcd4768 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.794+0000 7f41ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:39:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.794+0000 7f41ceffd640 1 -- 172.21.15.133:0/3014928636 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0011070 con 0x55e7bc1135e0 2024-09-11T23:39:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.795+0000 7f41f88a1740 1 -- 172.21.15.133:0/3014928636 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41e0006070 con 0x55e7bc1135e0 2024-09-11T23:39:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.795+0000 7f41ceffd640 1 -- 172.21.15.133:0/3014928636 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f41e002f040 con 0x55e7bc1135e0 2024-09-11T23:39:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.795+0000 7f41ceffd640 1 -- 172.21.15.133:0/3014928636 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41e0006460 con 0x55e7bc1135e0 2024-09-11T23:39:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.795+0000 7f41f88a1740 10 client.25732 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.795+0000 7f41f88a1740 1 -- 172.21.15.133:0/3014928636 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e7bc198b50 con 0x55e7bc1135e0 2024-09-11T23:39:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.797+0000 7f41ceffd640 1 -- 172.21.15.133:0/3014928636 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e0006070 con 0x55e7bc1135e0 2024-09-11T23:39:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.797+0000 7f41ceffd640 1 -- 172.21.15.133:0/3014928636 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f41e003b070 con 0x55e7bc1135e0 2024-09-11T23:39:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.798+0000 7f41f88a1740 10 client.25732 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.798+0000 7f41f88a1740 10 client.25732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.798+0000 7f41f88a1740 20 client.25732 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.798+0000 7f41f88a1740 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7bc199200 0x55e7bc19b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.798+0000 7f41f88a1740 4 client.25732 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:39:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.798+0000 7f41f88a1740 1 -- 172.21.15.133:0/3014928636 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7bc07ed70 con 0x55e7bc199200 2024-09-11T23:39:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.800+0000 7f41ef7fe640 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7bc199200 0x55e7bc19b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.803+0000 7f41ef7fe640 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7bc199200 0x55e7bc19b620 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=180a597d3ec97117 server_cookie=8df1f05bd5f0ebbd in_seq=0 out_seq=0 2024-09-11T23:39:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:30.803+0000 7f41ceffd640 10 client.25732 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:30 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:39:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:30 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:30 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:39:31.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:30 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:39:31.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:30 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:31.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:30 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:39:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:30 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:39:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:30 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:30 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:39:32.055 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:39:32.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:31 smithi133 ceph-mon[31430]: pgmap v85: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 808 KiB/s rd, 698 MiB/s wr, 2.99k op/s 2024-09-11T23:39:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:31 smithi178 ceph-mon[35882]: pgmap v85: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 808 KiB/s rd, 698 MiB/s wr, 2.99k op/s 2024-09-11T23:39:32.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:31 smithi138 ceph-mon[35142]: pgmap v85: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 808 KiB/s rd, 698 MiB/s wr, 2.99k op/s 2024-09-11T23:39:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.887+0000 7f41ceffd640 1 -- 172.21.15.133:0/3014928636 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7bc07ed70 con 0x55e7bc199200 2024-09-11T23:39:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.887+0000 7f41ceffd640 10 client.25732 handle_command_reply: tid=0 2024-09-11T23:39:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.887+0000 7f41f88a1740 1 client.25732 shutdown 2024-09-11T23:39:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41f88a1740 1 -- 172.21.15.133:0/3014928636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 msgr2=0x55e7bc1216e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41f88a1740 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc1216e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55e7bc0733b0 tx=0x7f41e000b4e0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41f88a1740 1 -- 172.21.15.133:0/3014928636 shutdown_connections 2024-09-11T23:39:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41f88a1740 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7bc199200 0x55e7bc19b620 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41f88a1740 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7bc11de80 0x55e7bc121c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41f88a1740 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7bc1135e0 0x55e7bc1216e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41f88a1740 1 --2- 172.21.15.133:0/3014928636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7bc112cc0 0x55e7bc1263c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41f88a1740 1 -- 172.21.15.133:0/3014928636 >> 172.21.15.133:0/3014928636 conn(0x55e7bc10f230 msgr2=0x55e7bc11c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41ceffd640 0 client.25732 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41f88a1740 1 -- 172.21.15.133:0/3014928636 shutdown_connections 2024-09-11T23:39:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41f88a1740 1 -- 172.21.15.133:0/3014928636 wait complete. 2024-09-11T23:39:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.888+0000 7f41f88a1740 20 client.25732 trim_cache size 0 max 16384 2024-09-11T23:39:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:39:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:39:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:39:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:39:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.889+0000 7f41f48a8640 1 -- 172.21.15.133:0/3627172464 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f41d80a2d60 msgr2=0x7f41d80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.889+0000 7f41f48a8640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f41d80a2d60 0x7f41d80a5220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f41e0015fd0 tx=0x7f41e0015f40 comp rx=0 tx=0).stop 2024-09-11T23:39:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.889+0000 7f41f48a8640 1 -- 172.21.15.133:0/3627172464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f007fe50 msgr2=0x7f41f0080300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.889+0000 7f41f48a8640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f007fe50 0x7f41f0080300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f41e802c000 tx=0x7f41e80308a0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.890+0000 7f41f48a8640 1 -- 172.21.15.133:0/3627172464 shutdown_connections 2024-09-11T23:39:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.890+0000 7f41f48a8640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f41d80a2d60 0x7f41d80a5220 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.890+0000 7f41f48a8640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f007fe50 0x7f41f0080300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.890+0000 7f41f48a8640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41f0084dd0 0x7f41f007f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.890+0000 7f41f48a8640 1 --2- 172.21.15.133:0/3627172464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41f0109460 0x7f41f0084890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.890+0000 7f41f48a8640 1 -- 172.21.15.133:0/3627172464 >> 172.21.15.133:0/3627172464 conn(0x7f41f006cbf0 msgr2=0x7f41f0071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.890+0000 7f41f48a8640 1 -- 172.21.15.133:0/3627172464 shutdown_connections 2024-09-11T23:39:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:32.890+0000 7f41f48a8640 1 -- 172.21.15.133:0/3627172464 wait complete. 2024-09-11T23:39:33.012 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:33 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:33.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:33 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:33.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:33 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:39:33.850 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-11T23:39:33.850 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:39:33.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.957+0000 7fdbb7fff640 1 -- 172.21.15.133:0/1981601800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb8105d90 msgr2=0x7fdbb8106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:33.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.957+0000 7fdbb7fff640 1 --2- 172.21.15.133:0/1981601800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb8105d90 0x7fdbb8106220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009900 tx=0x7fdbac02a480 comp rx=0 tx=0).stop 2024-09-11T23:39:33.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.958+0000 7fdbb7fff640 1 -- 172.21.15.133:0/1981601800 shutdown_connections 2024-09-11T23:39:33.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.958+0000 7fdbb7fff640 1 --2- 172.21.15.133:0/1981601800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb8105d90 0x7fdbb8106220 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:33.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.958+0000 7fdbb7fff640 1 --2- 172.21.15.133:0/1981601800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb81012a0 0x7fdbb8105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:33.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.958+0000 7fdbb7fff640 1 --2- 172.21.15.133:0/1981601800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb8100980 0x7fdbb8100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.958+0000 7fdbb7fff640 1 -- 172.21.15.133:0/1981601800 >> 172.21.15.133:0/1981601800 conn(0x7fdbb80fc610 msgr2=0x7fdbb80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.958+0000 7fdbb7fff640 1 -- 172.21.15.133:0/1981601800 shutdown_connections 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.958+0000 7fdbb7fff640 1 -- 172.21.15.133:0/1981601800 wait complete. 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.959+0000 7fdbb7fff640 1 Processor -- start 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.960+0000 7fdbb7fff640 1 -- start start 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.961+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb8100980 0x7fdbb80736d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.961+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb81012a0 0x7fdbb8073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.961+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb806c760 0x7fdbb806cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.961+0000 7fdbb7fff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81183a0 con 0x7fdbb8100980 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.961+0000 7fdbb7fff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8118220 con 0x7fdbb81012a0 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.961+0000 7fdbb7fff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8118520 con 0x7fdbb806c760 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.962+0000 7fdbb67fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb81012a0 0x7fdbb8073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.962+0000 7fdbb67fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb81012a0 0x7fdbb8073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:57534/0 (socket says 172.21.15.133:57534) 2024-09-11T23:39:33.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.962+0000 7fdbb67fc640 1 -- 172.21.15.133:0/4284669199 learned_addr learned my addr 172.21.15.133:0/4284669199 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.964+0000 7fdbb77fe640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb806c760 0x7fdbb806cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.964+0000 7fdbb67fc640 1 -- 172.21.15.133:0/4284669199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb806c760 msgr2=0x7fdbb806cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.964+0000 7fdbb67fc640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb806c760 0x7fdbb806cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.964+0000 7fdbb67fc640 1 -- 172.21.15.133:0/4284669199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb8100980 msgr2=0x7fdbb80736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:33.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.964+0000 7fdbb67fc640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb8100980 0x7fdbb80736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:33.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.964+0000 7fdbb67fc640 1 -- 172.21.15.133:0/4284669199 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb806d3e0 con 0x7fdbb81012a0 2024-09-11T23:39:33.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.969+0000 7fdbb67fc640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb81012a0 0x7fdbb8073c10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000ea00 tx=0x7fdbb000eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.972+0000 7fdbbca16640 1 -- 172.21.15.133:0/4284669199 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb000cd80 con 0x7fdbb81012a0 2024-09-11T23:39:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.972+0000 7fdbbca16640 1 -- 172.21.15.133:0/4284669199 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb0011950 con 0x7fdbb81012a0 2024-09-11T23:39:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.972+0000 7fdbb7fff640 1 -- 172.21.15.133:0/4284669199 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb8071200 con 0x7fdbb81012a0 2024-09-11T23:39:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.973+0000 7fdbb7fff640 1 -- 172.21.15.133:0/4284669199 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb8071740 con 0x7fdbb81012a0 2024-09-11T23:39:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.973+0000 7fdbbca16640 1 -- 172.21.15.133:0/4284669199 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb001aa10 con 0x7fdbb81012a0 2024-09-11T23:39:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.981+0000 7fdbbfdf1740 1 Processor -- start 2024-09-11T23:39:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.981+0000 7fdbbfdf1740 1 -- start start 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.982+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efcaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.982+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efcaf90 0x56296efd5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.982+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd5850 0x56296efd7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.982+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56296ef302e0 con 0x56296efcaf90 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.982+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56296eed7ce0 con 0x56296efca670 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.982+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56296ef2c2a0 con 0x56296efd5850 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.982+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efcaf90 0x56296efd5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.982+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efcaf90 0x56296efd5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51054/0 (socket says 172.21.15.133:51054) 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.982+0000 7fdbb6ffd640 1 -- 172.21.15.133:0/1089975722 learned_addr learned my addr 172.21.15.133:0/1089975722 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.983+0000 7fdbb6ffd640 1 -- 172.21.15.133:0/1089975722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd5850 msgr2=0x56296efd7c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.983+0000 7fdbb6ffd640 1 --2- 172.21.15.133:0/1089975722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd5850 0x56296efd7c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.983+0000 7fdbb6ffd640 1 -- 172.21.15.133:0/1089975722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 msgr2=0x56296efcaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.983+0000 7fdbb6ffd640 1 --2- 172.21.15.133:0/1089975722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efcaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.983+0000 7fdbb6ffd640 1 -- 172.21.15.133:0/1089975722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56296eebc440 con 0x56296efcaf90 2024-09-11T23:39:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.985+0000 7fdbb6ffd640 1 --2- 172.21.15.133:0/1089975722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efcaf90 0x56296efd5310 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009c40 tx=0x7fdba800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46104c7e97dc2688 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.985+0000 7fdbbca16640 1 -- 172.21.15.133:0/4284669199 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fdbb001ac90 con 0x7fdbb81012a0 2024-09-11T23:39:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.986+0000 7fdbbca16640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdba00a2f00 0x7fdba00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.987+0000 7fdbbca16640 1 -- 172.21.15.133:0/4284669199 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fdbb00cf7a0 con 0x7fdbb81012a0 2024-09-11T23:39:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.987+0000 7fdbb77fe640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdba00a2f00 0x7fdba00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.987+0000 7fdb9dffb640 1 -- 172.21.15.133:0/1089975722 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8005020 con 0x56296efcaf90 2024-09-11T23:39:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.988+0000 7fdb9dffb640 1 -- 172.21.15.133:0/1089975722 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba80047a0 con 0x56296efcaf90 2024-09-11T23:39:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.989+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/1089975722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efcaf90 msgr2=0x56296efd5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.989+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/1089975722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efcaf90 0x56296efd5310 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009c40 tx=0x7fdba800e720 comp rx=0 tx=0).stop 2024-09-11T23:39:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.989+0000 7fdbb77fe640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdba00a2f00 0x7fdba00a53c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009230 tx=0x7fdbac005d50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.990+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/1089975722 shutdown_connections 2024-09-11T23:39:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.990+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/1089975722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd5850 0x56296efd7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.990+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/1089975722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efcaf90 0x56296efd5310 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.990+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/1089975722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efcaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.990+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/1089975722 >> 172.21.15.133:0/1089975722 conn(0x56296efc6c00 msgr2=0x56296efc6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.992+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/1089975722 shutdown_connections 2024-09-11T23:39:33.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.995+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/1089975722 wait complete. 2024-09-11T23:39:34.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.996+0000 7fdbbfdf1740 1 -- start start 2024-09-11T23:39:34.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:33.996+0000 7fdbbfdf1740 1 Processor -- start 2024-09-11T23:39:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.000+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efd4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.000+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd5850 0x56296efd2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.000+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd2740 0x56296efd2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.000+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56296efd8470 con 0x56296efd2740 2024-09-11T23:39:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.000+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56296eeeb050 con 0x56296efca670 2024-09-11T23:39:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.000+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56296efd85f0 con 0x56296efd5850 2024-09-11T23:39:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.000+0000 7fdbbfdf1740 10 client.0 _ll_register_callbacks cb 0x56296ef2b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.002+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efd4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.002+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efd4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:57552/0 (socket says 172.21.15.133:57552) 2024-09-11T23:39:34.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.002+0000 7fdbb6ffd640 1 -- 172.21.15.133:0/156641835 learned_addr learned my addr 172.21.15.133:0/156641835 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.003+0000 7fdbb6ffd640 1 -- 172.21.15.133:0/156641835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd5850 msgr2=0x56296efd2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.003+0000 7fdbb6ffd640 1 --2- 172.21.15.133:0/156641835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd5850 0x56296efd2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.003+0000 7fdbb6ffd640 1 -- 172.21.15.133:0/156641835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd2740 msgr2=0x56296efd2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.003+0000 7fdbb6ffd640 1 --2- 172.21.15.133:0/156641835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd2740 0x56296efd2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.003+0000 7fdbb6ffd640 1 -- 172.21.15.133:0/156641835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56296eca13c0 con 0x56296efca670 2024-09-11T23:39:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.004+0000 7fdbb6ffd640 1 --2- 172.21.15.133:0/156641835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efd4f50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009560 tx=0x7fdba80099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63b829ec10e3924 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.005+0000 7fdb96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.005+0000 7fdb96ffd640 1 -- 172.21.15.133:0/156641835 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8028060 con 0x56296efca670 2024-09-11T23:39:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.005+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/156641835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56296ef3d850 con 0x56296efca670 2024-09-11T23:39:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.005+0000 7fdb96ffd640 1 -- 172.21.15.133:0/156641835 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fdba8011cd0 con 0x56296efca670 2024-09-11T23:39:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.006+0000 7fdb96ffd640 1 -- 172.21.15.133:0/156641835 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdba801d040 con 0x56296efca670 2024-09-11T23:39:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.006+0000 7fdbbfdf1740 10 client.25760 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.006+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/156641835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56296eff5140 con 0x56296efca670 2024-09-11T23:39:34.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.009+0000 7fdb96ffd640 1 -- 172.21.15.133:0/156641835 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56296ef3d850 con 0x56296efca670 2024-09-11T23:39:34.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.010+0000 7fdb96ffd640 1 -- 172.21.15.133:0/156641835 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdba802b020 con 0x56296efca670 2024-09-11T23:39:34.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.012+0000 7fdbbfdf1740 10 client.25760 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:34.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.012+0000 7fdbbfdf1740 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:34.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.012+0000 7fdbbfdf1740 20 client.25760 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:34.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.012+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/156641835 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56296eff6550 0x56296eff8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:34.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.012+0000 7fdbbfdf1740 4 client.25760 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:34.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.012+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/156641835 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56296ef37d60 con 0x56296eff6550 2024-09-11T23:39:34.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.013+0000 7fdbb67fc640 1 --2- 172.21.15.133:0/156641835 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56296eff6550 0x56296eff8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.032+0000 7fdbb67fc640 1 --2- 172.21.15.133:0/156641835 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56296eff6550 0x56296eff8940 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f62bb585893c71 server_cookie=ef1a8ec834c16535 in_seq=0 out_seq=0 2024-09-11T23:39:34.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.032+0000 7fdb96ffd640 10 client.25760 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:34.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.039+0000 7fdb96ffd640 1 -- 172.21.15.133:0/156641835 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56296ef37d60 con 0x56296eff6550 2024-09-11T23:39:34.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.039+0000 7fdb96ffd640 10 client.25760 handle_command_reply: tid=0 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.039+0000 7fdbbfdf1740 1 client.25760 shutdown 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.040+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/156641835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 msgr2=0x56296efd4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.040+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/156641835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efd4f50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009560 tx=0x7fdba80099c0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.040+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/156641835 shutdown_connections 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.040+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/156641835 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56296eff6550 0x56296eff8940 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.040+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/156641835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd2740 0x56296efd2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.040+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/156641835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd5850 0x56296efd2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.040+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/156641835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efd4f50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.040+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/156641835 >> 172.21.15.133:0/156641835 conn(0x56296efc6c00 msgr2=0x56296efc5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.041+0000 7fdb96ffd640 0 client.25760 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.041+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/156641835 shutdown_connections 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.041+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/156641835 wait complete. 2024-09-11T23:39:34.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.042+0000 7fdbbfdf1740 20 client.25760 trim_cache size 0 max 16384 2024-09-11T23:39:34.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.093+0000 7fdbbfdf1740 1 Processor -- start 2024-09-11T23:39:34.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.094+0000 7fdbbfdf1740 1 -- start start 2024-09-11T23:39:34.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.094+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efcd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:34.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.095+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd2740 0x56296efc1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:34.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.095+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd5850 0x56296efc5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:34.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.095+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdba400e5b0 con 0x56296efd5850 2024-09-11T23:39:34.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.095+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdba40100f0 con 0x56296efca670 2024-09-11T23:39:34.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.095+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdba400ec80 con 0x56296efd2740 2024-09-11T23:39:34.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.097+0000 7fdbb67fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd2740 0x56296efc1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:34.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.097+0000 7fdbb67fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd2740 0x56296efc1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53078/0 (socket says 172.21.15.133:53078) 2024-09-11T23:39:34.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.097+0000 7fdbb67fc640 1 -- 172.21.15.133:0/100698050 learned_addr learned my addr 172.21.15.133:0/100698050 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:34.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.097+0000 7fdbb77fe640 1 --2- 172.21.15.133:0/100698050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd5850 0x56296efc5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:34.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.097+0000 7fdbb6ffd640 1 --2- 172.21.15.133:0/100698050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efcd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:34.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.099+0000 7fdbb67fc640 1 -- 172.21.15.133:0/100698050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 msgr2=0x56296efcd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:34.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.099+0000 7fdbb67fc640 1 --2- 172.21.15.133:0/100698050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efcd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.099+0000 7fdbb67fc640 1 -- 172.21.15.133:0/100698050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd5850 msgr2=0x56296efc5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:34.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.099+0000 7fdbb67fc640 1 --2- 172.21.15.133:0/100698050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd5850 0x56296efc5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.100+0000 7fdbb67fc640 1 -- 172.21.15.133:0/100698050 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56296efd11b0 con 0x56296efd2740 2024-09-11T23:39:34.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.101+0000 7fdbb67fc640 1 --2- 172.21.15.133:0/100698050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd2740 0x56296efc1b90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009510 tx=0x7fdbb00094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eadf81c44405ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.101+0000 7fdb9d7fa640 1 -- 172.21.15.133:0/100698050 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb00a0080 con 0x56296efd2740 2024-09-11T23:39:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.102+0000 7fdb9d7fa640 1 -- 172.21.15.133:0/100698050 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb00a4940 con 0x56296efd2740 2024-09-11T23:39:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.102+0000 7fdb9d7fa640 1 -- 172.21.15.133:0/100698050 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb00a4710 con 0x56296efd2740 2024-09-11T23:39:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.103+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/100698050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd2740 msgr2=0x56296efc1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.103+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/100698050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd2740 0x56296efc1b90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009510 tx=0x7fdbb00094a0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.104+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/100698050 shutdown_connections 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.104+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/100698050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd5850 0x56296efc5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.104+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/100698050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efd2740 0x56296efc1b90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.104+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/100698050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efca670 0x56296efcd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.104+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/100698050 >> 172.21.15.133:0/100698050 conn(0x56296efc6c00 msgr2=0x56296efcd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.104+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/100698050 shutdown_connections 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.104+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/100698050 wait complete. 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.105+0000 7fdbbfdf1740 1 -- start start 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.106+0000 7fdbbfdf1740 1 Processor -- start 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.106+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efca670 0x56296efddb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.107+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd2740 0x56296efde050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.107+0000 7fdbbfdf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efd5850 0x56296efd88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:34.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.107+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdba400fa20 con 0x56296efd2740 2024-09-11T23:39:34.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.107+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdba400f350 con 0x56296efd5850 2024-09-11T23:39:34.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.107+0000 7fdbbfdf1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdba400c640 con 0x56296efca670 2024-09-11T23:39:34.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.107+0000 7fdbbfdf1740 10 client.0 _ll_register_callbacks cb 0x56296efd57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:34.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.111+0000 7fdbb77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efd5850 0x56296efd88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:34.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.111+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efca670 0x56296efddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:34.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.111+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efca670 0x56296efddb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53094/0 (socket says 172.21.15.133:53094) 2024-09-11T23:39:34.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.111+0000 7fdbb6ffd640 1 -- 172.21.15.133:0/3119329885 learned_addr learned my addr 172.21.15.133:0/3119329885 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:34.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.114+0000 7fdbb77fe640 1 -- 172.21.15.133:0/3119329885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efca670 msgr2=0x56296efddb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:34.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.114+0000 7fdbb77fe640 1 --2- 172.21.15.133:0/3119329885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efca670 0x56296efddb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.114+0000 7fdbb77fe640 1 -- 172.21.15.133:0/3119329885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd2740 msgr2=0x56296efde050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:34.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.114+0000 7fdbb77fe640 1 --2- 172.21.15.133:0/3119329885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd2740 0x56296efde050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.114+0000 7fdbb77fe640 1 -- 172.21.15.133:0/3119329885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56296efd90c0 con 0x56296efd5850 2024-09-11T23:39:34.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.119+0000 7fdbb77fe640 1 --2- 172.21.15.133:0/3119329885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efd5850 0x56296efd88e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005dc0 tx=0x7fdbac031820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=978719656f0eaed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:34.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.119+0000 7fdb96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:34.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.374+0000 7fdb96ffd640 1 -- 172.21.15.133:0/3119329885 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac04c070 con 0x56296efd5850 2024-09-11T23:39:34.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.374+0000 7fdb96ffd640 1 -- 172.21.15.133:0/3119329885 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fdbac005510 con 0x56296efd5850 2024-09-11T23:39:34.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.374+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/3119329885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb00a4940 con 0x56296efd5850 2024-09-11T23:39:34.375 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:34 smithi133 ceph-mon[31430]: pgmap v86: 97 pgs: 97 active+clean; 18 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 857 MiB/s wr, 3.34k op/s 2024-09-11T23:39:34.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.406+0000 7fdb96ffd640 1 -- 172.21.15.133:0/3119329885 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdbac058050 con 0x56296efd5850 2024-09-11T23:39:34.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.406+0000 7fdbbfdf1740 10 client.25766 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:34.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.406+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/3119329885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56296f061a10 con 0x56296efd5850 2024-09-11T23:39:34.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.407+0000 7fdb96ffd640 1 -- 172.21.15.133:0/3119329885 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb00a4940 con 0x56296efd5850 2024-09-11T23:39:34.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.407+0000 7fdb96ffd640 1 -- 172.21.15.133:0/3119329885 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdbac003a30 con 0x56296efd5850 2024-09-11T23:39:34.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.407+0000 7fdbbfdf1740 10 client.25766 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:34.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.407+0000 7fdbbfdf1740 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:34.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.407+0000 7fdbbfdf1740 20 client.25766 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:34.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.407+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/3119329885 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56296f062110 0x56296f064530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:34.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.408+0000 7fdbbfdf1740 4 client.25766 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:39:34.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.408+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/3119329885 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56296eeb4210 con 0x56296f062110 2024-09-11T23:39:34.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.408+0000 7fdbb6ffd640 1 --2- 172.21.15.133:0/3119329885 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56296f062110 0x56296f064530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:34.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.408+0000 7fdbb6ffd640 1 --2- 172.21.15.133:0/3119329885 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56296f062110 0x56296f064530 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436a4f3ae3472228 server_cookie=d249e5a819e4c2c1 in_seq=0 out_seq=0 2024-09-11T23:39:34.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.408+0000 7fdb96ffd640 10 client.25766 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:34.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.411+0000 7fdb96ffd640 1 -- 172.21.15.133:0/3119329885 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56296eeb4210 con 0x56296f062110 2024-09-11T23:39:34.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.411+0000 7fdb96ffd640 10 client.25766 handle_command_reply: tid=0 2024-09-11T23:39:34.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.411+0000 7fdbbfdf1740 1 client.25766 shutdown 2024-09-11T23:39:34.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.412+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/3119329885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efd5850 msgr2=0x56296efd88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:34.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.412+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/3119329885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efd5850 0x56296efd88e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005dc0 tx=0x7fdbac031820 comp rx=0 tx=0).stop 2024-09-11T23:39:34.414 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:39:34.414 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:39:34.414 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-09-11T23:39:34.414 INFO:teuthology.orchestra.run.smithi133.stdout: "4a10f6cf-eed6-4f16-a7d6-8c0a31be47ed": { 2024-09-11T23:39:34.414 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-09-11T23:39:34.414 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "4a10f6cf-eed6-4f16-a7d6-8c0a31be47ed", 2024-09-11T23:39:34.414 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2024-09-11T23:39:34.414 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:34.414 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:34.414 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:39:34.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.412+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/3119329885 shutdown_connections 2024-09-11T23:39:34.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.412+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/3119329885 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56296f062110 0x56296f064530 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.412+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/3119329885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56296efd5850 0x56296efd88e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.412+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/3119329885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56296efd2740 0x56296efde050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.412+0000 7fdb96ffd640 0 client.25766 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:34.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.412+0000 7fdbbfdf1740 1 --2- 172.21.15.133:0/3119329885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56296efca670 0x56296efddb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.412+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/3119329885 >> 172.21.15.133:0/3119329885 conn(0x56296efc6c00 msgr2=0x56296efc5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:34.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.412+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/3119329885 shutdown_connections 2024-09-11T23:39:34.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.413+0000 7fdbbfdf1740 1 -- 172.21.15.133:0/3119329885 wait complete. 2024-09-11T23:39:34.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.413+0000 7fdbbfdf1740 20 client.25766 trim_cache size 0 max 16384 2024-09-11T23:39:34.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.414+0000 7fdbb7fff640 1 -- 172.21.15.133:0/4284669199 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdba00a2f00 msgr2=0x7fdba00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:34.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.414+0000 7fdbb7fff640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdba00a2f00 0x7fdba00a53c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009230 tx=0x7fdbac005d50 comp rx=0 tx=0).stop 2024-09-11T23:39:34.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.415+0000 7fdbb7fff640 1 -- 172.21.15.133:0/4284669199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb81012a0 msgr2=0x7fdbb8073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:34.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.415+0000 7fdbb7fff640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb81012a0 0x7fdbb8073c10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000ea00 tx=0x7fdbb000eec0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.415+0000 7fdbb7fff640 1 -- 172.21.15.133:0/4284669199 shutdown_connections 2024-09-11T23:39:34.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.415+0000 7fdbb7fff640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdba00a2f00 0x7fdba00a53c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.415+0000 7fdbb7fff640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb806c760 0x7fdbb806cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.415+0000 7fdbb7fff640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb81012a0 0x7fdbb8073c10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.415+0000 7fdbb7fff640 1 --2- 172.21.15.133:0/4284669199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb8100980 0x7fdbb80736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:34.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.415+0000 7fdbb7fff640 1 -- 172.21.15.133:0/4284669199 >> 172.21.15.133:0/4284669199 conn(0x7fdbb80fc610 msgr2=0x7fdbb8105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:34.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.415+0000 7fdbb7fff640 1 -- 172.21.15.133:0/4284669199 shutdown_connections 2024-09-11T23:39:34.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:34.419+0000 7fdbb7fff640 1 -- 172.21.15.133:0/4284669199 wait complete. 2024-09-11T23:39:34.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:34 smithi138 ceph-mon[35142]: pgmap v86: 97 pgs: 97 active+clean; 18 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 857 MiB/s wr, 3.34k op/s 2024-09-11T23:39:34.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:34 smithi178 ceph-mon[35882]: pgmap v86: 97 pgs: 97 active+clean; 18 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 857 MiB/s wr, 3.34k op/s 2024-09-11T23:39:35.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:35 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:39:35.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:35 smithi138 ceph-mon[35142]: pgmap v87: 97 pgs: 97 active+clean; 19 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 721 MiB/s wr, 2.66k op/s 2024-09-11T23:39:35.718 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:35 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:39:35.719 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:35 smithi133 ceph-mon[31430]: pgmap v87: 97 pgs: 97 active+clean; 19 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 721 MiB/s wr, 2.66k op/s 2024-09-11T23:39:35.732 INFO:tasks.cephfs.filesystem:scrub status for tag:4a10f6cf-eed6-4f16-a7d6-8c0a31be47ed - {'path': '/', 'tag': '4a10f6cf-eed6-4f16-a7d6-8c0a31be47ed', 'options': 'recursive,force'} 2024-09-11T23:39:35.744 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:35.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:35 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:39:35.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:35 smithi178 ceph-mon[35882]: pgmap v87: 97 pgs: 97 active+clean; 19 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 721 MiB/s wr, 2.66k op/s 2024-09-11T23:39:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.582+0000 7f82dc955640 1 -- 172.21.15.133:0/2086863563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d806ea30 msgr2=0x7f82d806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.582+0000 7f82dc955640 1 --2- 172.21.15.133:0/2086863563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d806ea30 0x7f82d806ee70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009900 tx=0x7f82cc02a410 comp rx=0 tx=0).stop 2024-09-11T23:39:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.583+0000 7f82dc955640 1 -- 172.21.15.133:0/2086863563 shutdown_connections 2024-09-11T23:39:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.583+0000 7f82dc955640 1 --2- 172.21.15.133:0/2086863563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82d811b140 0x7f82d811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.583+0000 7f82dc955640 1 --2- 172.21.15.133:0/2086863563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d806ea30 0x7f82d806ee70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.583+0000 7f82dc955640 1 --2- 172.21.15.133:0/2086863563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d8110190 0x7f82d806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.583+0000 7f82dc955640 1 -- 172.21.15.133:0/2086863563 >> 172.21.15.133:0/2086863563 conn(0x7f82d8073180 msgr2=0x7f82d8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.589+0000 7f82dc955640 1 -- 172.21.15.133:0/2086863563 shutdown_connections 2024-09-11T23:39:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.589+0000 7f82dc955640 1 -- 172.21.15.133:0/2086863563 wait complete. 2024-09-11T23:39:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.590+0000 7f82dc955640 1 Processor -- start 2024-09-11T23:39:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.591+0000 7f82dc955640 1 -- start start 2024-09-11T23:39:36.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.593+0000 7f82dc955640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d806ea30 0x7f82d8119760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:36.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.593+0000 7f82dc955640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d8110190 0x7f82d81127f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:36.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.593+0000 7f82dc955640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82d811b140 0x7f82d8112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:36.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.593+0000 7f82dc955640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8120be0 con 0x7f82d8110190 2024-09-11T23:39:36.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.594+0000 7f82dc955640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8120a60 con 0x7f82d806ea30 2024-09-11T23:39:36.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.594+0000 7f82d759e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82d811b140 0x7f82d8112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:36.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.594+0000 7f82d759e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82d811b140 0x7f82d8112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53102/0 (socket says 172.21.15.133:53102) 2024-09-11T23:39:36.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.594+0000 7f82d759e640 1 -- 172.21.15.133:0/2977441791 learned_addr learned my addr 172.21.15.133:0/2977441791 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:36.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.594+0000 7f82dc955640 1 -- 172.21.15.133:0/2977441791 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8120d60 con 0x7f82d811b140 2024-09-11T23:39:36.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.596+0000 7f82d6d9d640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d806ea30 0x7f82d8119760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:36.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.599+0000 7f82d659c640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d8110190 0x7f82d81127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:36.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.604+0000 7f82d6d9d640 1 -- 172.21.15.133:0/2977441791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82d811b140 msgr2=0x7f82d8112d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:36.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.604+0000 7f82d6d9d640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82d811b140 0x7f82d8112d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:36.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.604+0000 7f82d6d9d640 1 -- 172.21.15.133:0/2977441791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d8110190 msgr2=0x7f82d81127f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:36.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.604+0000 7f82d6d9d640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d8110190 0x7f82d81127f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:36.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.604+0000 7f82d6d9d640 1 -- 172.21.15.133:0/2977441791 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d81132f0 con 0x7f82d806ea30 2024-09-11T23:39:36.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.611+0000 7f82d6d9d640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d806ea30 0x7f82d8119760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f82c800e9f0 tx=0x7f82c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:36.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.612+0000 7f82bffff640 1 -- 172.21.15.133:0/2977441791 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c800ccc0 con 0x7f82d806ea30 2024-09-11T23:39:36.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.612+0000 7f82bffff640 1 -- 172.21.15.133:0/2977441791 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c8010910 con 0x7f82d806ea30 2024-09-11T23:39:36.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.612+0000 7f82bffff640 1 -- 172.21.15.133:0/2977441791 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c80189f0 con 0x7f82d806ea30 2024-09-11T23:39:36.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.612+0000 7f82dc955640 1 -- 172.21.15.133:0/2977441791 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d81135e0 con 0x7f82d806ea30 2024-09-11T23:39:36.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.615+0000 7f82dc955640 1 -- 172.21.15.133:0/2977441791 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d81174c0 con 0x7f82d806ea30 2024-09-11T23:39:36.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.621+0000 7f82bffff640 1 -- 172.21.15.133:0/2977441791 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f82c8018b90 con 0x7f82d806ea30 2024-09-11T23:39:36.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.624+0000 7f82bffff640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82c00a2e80 0x7f82c00a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:36.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.625+0000 7f82bffff640 1 -- 172.21.15.133:0/2977441791 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f82c80d4170 con 0x7f82d806ea30 2024-09-11T23:39:36.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.632+0000 7f82dc955640 1 -- 172.21.15.133:0/2977441791 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d81096f0 con 0x7f82d806ea30 2024-09-11T23:39:36.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.633+0000 7f82d659c640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82c00a2e80 0x7f82c00a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.635+0000 7f82d659c640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82c00a2e80 0x7f82c00a5340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000c00 tx=0x7f82cc0040c0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:36.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:36.643+0000 7f82bffff640 1 -- 172.21.15.133:0/2977441791 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82c809d280 con 0x7f82d806ea30 2024-09-11T23:39:37.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.229+0000 7f82dc955640 1 -- 172.21.15.133:0/2977441791 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82d81177b0 con 0x7f82d806ea30 2024-09-11T23:39:37.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.234+0000 7f82bffff640 1 -- 172.21.15.133:0/2977441791 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f82c809cb80 con 0x7f82d806ea30 2024-09-11T23:39:37.237 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:39:37.237 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:39:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:39:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.246+0000 7f82dc955640 1 -- 172.21.15.133:0/2977441791 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82c00a2e80 msgr2=0x7f82c00a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.246+0000 7f82dc955640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82c00a2e80 0x7f82c00a5340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000c00 tx=0x7f82cc0040c0 comp rx=0 tx=0).stop 2024-09-11T23:39:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.247+0000 7f82dc955640 1 -- 172.21.15.133:0/2977441791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d806ea30 msgr2=0x7f82d8119760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.247+0000 7f82dc955640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d806ea30 0x7f82d8119760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f82c800e9f0 tx=0x7f82c800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:39:37.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.252+0000 7f82dc955640 1 -- 172.21.15.133:0/2977441791 shutdown_connections 2024-09-11T23:39:37.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.252+0000 7f82dc955640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82c00a2e80 0x7f82c00a5340 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:37.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.252+0000 7f82dc955640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82d811b140 0x7f82d8112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:37.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.252+0000 7f82dc955640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d8110190 0x7f82d81127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:37.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.252+0000 7f82dc955640 1 --2- 172.21.15.133:0/2977441791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d806ea30 0x7f82d8119760 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:37.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.252+0000 7f82dc955640 1 -- 172.21.15.133:0/2977441791 >> 172.21.15.133:0/2977441791 conn(0x7f82d8073180 msgr2=0x7f82d806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:37.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.253+0000 7f82dc955640 1 -- 172.21.15.133:0/2977441791 shutdown_connections 2024-09-11T23:39:37.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:37.253+0000 7f82dc955640 1 -- 172.21.15.133:0/2977441791 wait complete. 2024-09-11T23:39:37.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:37 smithi138 ceph-mon[35142]: pgmap v88: 97 pgs: 97 active+clean; 19 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 596 MiB/s wr, 2.17k op/s 2024-09-11T23:39:37.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:37 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2977441791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:37.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:37 smithi133 ceph-mon[31430]: pgmap v88: 97 pgs: 97 active+clean; 19 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 596 MiB/s wr, 2.17k op/s 2024-09-11T23:39:37.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:37 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2977441791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:37.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:37 smithi178 ceph-mon[35882]: pgmap v88: 97 pgs: 97 active+clean; 19 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 596 MiB/s wr, 2.17k op/s 2024-09-11T23:39:37.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:37 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2977441791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:39:38.291 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-11T23:39:38.291 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-11T23:39:39.354 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:40.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.198+0000 7f358b1fc640 1 -- 172.21.15.133:0/2173134430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3584112670 msgr2=0x7f3584114aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:40.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.198+0000 7f358b1fc640 1 --2- 172.21.15.133:0/2173134430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3584112670 0x7f3584114aa0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f357c009ca0 tx=0x7f357c02a8f0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.199+0000 7f358b1fc640 1 -- 172.21.15.133:0/2173134430 shutdown_connections 2024-09-11T23:39:40.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.199+0000 7f358b1fc640 1 --2- 172.21.15.133:0/2173134430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3584112670 0x7f3584114aa0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.199+0000 7f358b1fc640 1 --2- 172.21.15.133:0/2173134430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3584109bb0 0x7f35841120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.199+0000 7f358b1fc640 1 --2- 172.21.15.133:0/2173134430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3584109290 0x7f3584109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.199+0000 7f358b1fc640 1 -- 172.21.15.133:0/2173134430 >> 172.21.15.133:0/2173134430 conn(0x7f3584077960 msgr2=0x7f3584079d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:40.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.200+0000 7f358b1fc640 1 -- 172.21.15.133:0/2173134430 shutdown_connections 2024-09-11T23:39:40.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.201+0000 7f358b1fc640 1 -- 172.21.15.133:0/2173134430 wait complete. 2024-09-11T23:39:40.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.202+0000 7f358b1fc640 1 Processor -- start 2024-09-11T23:39:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.202+0000 7f358b1fc640 1 -- start start 2024-09-11T23:39:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.202+0000 7f358b1fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3584109290 0x7f3584073940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.202+0000 7f358b1fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3584109bb0 0x7f3584073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.202+0000 7f358b1fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3584112670 0x7f358406cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.202+0000 7f358b1fc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3584118460 con 0x7f3584109bb0 2024-09-11T23:39:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.202+0000 7f358b1fc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35841182e0 con 0x7f3584112670 2024-09-11T23:39:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.202+0000 7f358b1fc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35841185e0 con 0x7f3584109290 2024-09-11T23:39:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.203+0000 7f358a9fb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3584112670 0x7f358406cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.203+0000 7f358a9fb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3584112670 0x7f358406cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:57602/0 (socket says 172.21.15.133:57602) 2024-09-11T23:39:40.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.203+0000 7f358a9fb640 1 -- 172.21.15.133:0/4002979589 learned_addr learned my addr 172.21.15.133:0/4002979589 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:40.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.203+0000 7f35899f9640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3584109bb0 0x7f3584073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.203+0000 7f35899f9640 1 -- 172.21.15.133:0/4002979589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3584109290 msgr2=0x7f3584073940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:39:40.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.203+0000 7f35899f9640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3584109290 0x7f3584073940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.203+0000 7f35899f9640 1 -- 172.21.15.133:0/4002979589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3584112670 msgr2=0x7f358406cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:40.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.203+0000 7f35899f9640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3584112670 0x7f358406cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.203+0000 7f35899f9640 1 -- 172.21.15.133:0/4002979589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358406d470 con 0x7f3584109bb0 2024-09-11T23:39:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.204+0000 7f35899f9640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3584109bb0 0x7f3584073e80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f357400d6d0 tx=0x7f357400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:40.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.209+0000 7f357b7fe640 1 -- 172.21.15.133:0/4002979589 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574017070 con 0x7f3584109bb0 2024-09-11T23:39:40.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.209+0000 7f357b7fe640 1 -- 172.21.15.133:0/4002979589 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3574009820 con 0x7f3584109bb0 2024-09-11T23:39:40.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.209+0000 7f357b7fe640 1 -- 172.21.15.133:0/4002979589 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574004ca0 con 0x7f3584109bb0 2024-09-11T23:39:40.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.211+0000 7f358b1fc640 1 -- 172.21.15.133:0/4002979589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358406d700 con 0x7f3584109bb0 2024-09-11T23:39:40.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.211+0000 7f358b1fc640 1 -- 172.21.15.133:0/4002979589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35840716a0 con 0x7f3584109bb0 2024-09-11T23:39:40.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.217+0000 7f358e837740 1 Processor -- start 2024-09-11T23:39:40.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.217+0000 7f358e837740 1 -- start start 2024-09-11T23:39:40.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.217+0000 7f358e837740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9650900 0x562db9650ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.217+0000 7f358e837740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 0x562db965b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.217+0000 7f358e837740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db965ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.218+0000 7f358e837740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562db95b62a0 con 0x562db9651220 2024-09-11T23:39:40.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.218+0000 7f358e837740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562db955db10 con 0x562db9650900 2024-09-11T23:39:40.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.218+0000 7f358e837740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562db95b2260 con 0x562db965bae0 2024-09-11T23:39:40.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.218+0000 7f357b7fe640 1 -- 172.21.15.133:0/4002979589 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f35740099d0 con 0x7f3584109bb0 2024-09-11T23:39:40.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.218+0000 7f358a9fb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9650900 0x562db9650ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.220+0000 7f357b7fe640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35700a2f00 0x7f35700a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.220+0000 7f35899f9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db965ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.220+0000 7f358a1fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 0x562db965b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.221+0000 7f358a9fb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9650900 0x562db9650ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:57610/0 (socket says 172.21.15.133:57610) 2024-09-11T23:39:40.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.221+0000 7f358a9fb640 1 -- 172.21.15.133:0/3649926679 learned_addr learned my addr 172.21.15.133:0/3649926679 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:40.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.221+0000 7f357b7fe640 1 -- 172.21.15.133:0/4002979589 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f35740cf0a0 con 0x7f3584109bb0 2024-09-11T23:39:40.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.221+0000 7f358a9fb640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35700a2f00 0x7f35700a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.221+0000 7f358a1fa640 1 -- 172.21.15.133:0/3649926679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 msgr2=0x562db965ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:40.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.221+0000 7f358a1fa640 1 --2- 172.21.15.133:0/3649926679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db965ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.222+0000 7f358a1fa640 1 -- 172.21.15.133:0/3649926679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9650900 msgr2=0x562db9650ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:40.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.222+0000 7f358a1fa640 1 --2- 172.21.15.133:0/3649926679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9650900 0x562db9650ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.222+0000 7f358a1fa640 1 -- 172.21.15.133:0/3649926679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db9327b90 con 0x562db9651220 2024-09-11T23:39:40.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.222+0000 7f358a1fa640 1 --2- 172.21.15.133:0/3649926679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 0x562db965b5a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3580009690 tx=0x7f3580009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c568f16cc3e3a9d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:40.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.222+0000 7f35899f9640 1 --2- 172.21.15.133:0/3649926679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db965ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:39:40.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.224+0000 7f3578dea640 1 -- 172.21.15.133:0/3649926679 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580015020 con 0x562db9651220 2024-09-11T23:39:40.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.224+0000 7f3578dea640 1 -- 172.21.15.133:0/3649926679 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f358000bda0 con 0x562db9651220 2024-09-11T23:39:40.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.224+0000 7f3578dea640 1 -- 172.21.15.133:0/3649926679 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580010380 con 0x562db9651220 2024-09-11T23:39:40.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.225+0000 7f358a9fb640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35700a2f00 0x7f35700a53c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f357c02f570 tx=0x7f357c02a8f0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:40.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.230+0000 7f358e837740 1 -- 172.21.15.133:0/3649926679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 msgr2=0x562db965b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:40.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.230+0000 7f358e837740 1 --2- 172.21.15.133:0/3649926679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 0x562db965b5a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3580009690 tx=0x7f3580009b50 comp rx=0 tx=0).stop 2024-09-11T23:39:40.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.230+0000 7f358e837740 1 -- 172.21.15.133:0/3649926679 shutdown_connections 2024-09-11T23:39:40.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.230+0000 7f358e837740 1 --2- 172.21.15.133:0/3649926679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db965ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.230+0000 7f358e837740 1 --2- 172.21.15.133:0/3649926679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 0x562db965b5a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.230+0000 7f358e837740 1 --2- 172.21.15.133:0/3649926679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9650900 0x562db9650ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.230+0000 7f358e837740 1 -- 172.21.15.133:0/3649926679 >> 172.21.15.133:0/3649926679 conn(0x562db964ce90 msgr2=0x562db964d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:40.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.233+0000 7f358e837740 1 -- 172.21.15.133:0/3649926679 shutdown_connections 2024-09-11T23:39:40.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.235+0000 7f358e837740 1 -- 172.21.15.133:0/3649926679 wait complete. 2024-09-11T23:39:40.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.235+0000 7f358e837740 1 -- start start 2024-09-11T23:39:40.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.235+0000 7f358e837740 1 Processor -- start 2024-09-11T23:39:40.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.236+0000 7f358e837740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db9650900 0x562db9658a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.236+0000 7f358e837740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 0x562db9658f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.236+0000 7f358e837740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db965bae0 0x562db96594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.236+0000 7f358e837740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562db966f6c0 con 0x562db9651220 2024-09-11T23:39:40.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.236+0000 7f358e837740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562db953e6e0 con 0x562db965bae0 2024-09-11T23:39:40.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.236+0000 7f358e837740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562db966f840 con 0x562db9650900 2024-09-11T23:39:40.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.236+0000 7f358e837740 10 client.0 _ll_register_callbacks cb 0x562db9562a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.236+0000 7f35899f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 0x562db9658f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.236+0000 7f35899f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 0x562db9658f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51130/0 (socket says 172.21.15.133:51130) 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.236+0000 7f35899f9640 1 -- 172.21.15.133:0/2999464310 learned_addr learned my addr 172.21.15.133:0/2999464310 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.237+0000 7f358a1fa640 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db9650900 0x562db9658a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.237+0000 7f358a9fb640 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db965bae0 0x562db96594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.237+0000 7f35899f9640 1 -- 172.21.15.133:0/2999464310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db9650900 msgr2=0x562db9658a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.237+0000 7f35899f9640 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db9650900 0x562db9658a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.237+0000 7f35899f9640 1 -- 172.21.15.133:0/2999464310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db965bae0 msgr2=0x562db96594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.237+0000 7f35899f9640 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db965bae0 0x562db96594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.237+0000 7f35899f9640 1 -- 172.21.15.133:0/2999464310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db9678c30 con 0x562db9651220 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.238+0000 7f35899f9640 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 0x562db9658f90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3574009610 tx=0x7f357409be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa0241856c6d0ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:40.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.238+0000 7f356dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:40.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.239+0000 7f356dffb640 1 -- 172.21.15.133:0/2999464310 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35740a42c0 con 0x562db9651220 2024-09-11T23:39:40.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.240+0000 7f356dffb640 1 -- 172.21.15.133:0/2999464310 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f3574097cc0 con 0x562db9651220 2024-09-11T23:39:40.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.240+0000 7f358e837740 1 -- 172.21.15.133:0/2999464310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358000bda0 con 0x562db9651220 2024-09-11T23:39:40.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.241+0000 7f356dffb640 1 -- 172.21.15.133:0/2999464310 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3574019bc0 con 0x562db9651220 2024-09-11T23:39:40.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.241+0000 7f358e837740 10 client.16335 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:40.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.241+0000 7f358e837740 1 -- 172.21.15.133:0/2999464310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562db965fcd0 con 0x562db9651220 2024-09-11T23:39:40.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.409+0000 7f356dffb640 1 -- 172.21.15.133:0/2999464310 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f358000bda0 con 0x562db9651220 2024-09-11T23:39:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.409+0000 7f356dffb640 1 -- 172.21.15.133:0/2999464310 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f357409aaa0 con 0x562db9651220 2024-09-11T23:39:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.411+0000 7f358e837740 10 client.16335 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.411+0000 7f358e837740 10 client.16335 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:39:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.411+0000 7f358e837740 20 client.16335 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.411+0000 7f358e837740 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562db9661190 0x562db96635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.411+0000 7f358e837740 4 client.16335 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.411+0000 7f358e837740 1 -- 172.21.15.133:0/2999464310 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562db95c39d0 con 0x562db9661190 2024-09-11T23:39:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.412+0000 7f358a1fa640 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562db9661190 0x562db96635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.413+0000 7f358a1fa640 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562db9661190 0x562db96635c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1579ced30a0c19ac server_cookie=94841981873f07d1 in_seq=0 out_seq=0 2024-09-11T23:39:40.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.413+0000 7f356dffb640 10 client.16335 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:40.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.414+0000 7f356dffb640 1 -- 172.21.15.133:0/2999464310 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562db95c39d0 con 0x562db9661190 2024-09-11T23:39:40.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.414+0000 7f356dffb640 10 client.16335 handle_command_reply: tid=0 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.414+0000 7f358e837740 1 client.16335 shutdown 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.414+0000 7f358e837740 1 -- 172.21.15.133:0/2999464310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 msgr2=0x562db9658f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.414+0000 7f358e837740 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 0x562db9658f90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3574009610 tx=0x7f357409be90 comp rx=0 tx=0).stop 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.414+0000 7f358e837740 1 -- 172.21.15.133:0/2999464310 shutdown_connections 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.414+0000 7f358e837740 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562db9661190 0x562db96635c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.415+0000 7f358e837740 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db965bae0 0x562db96594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.415+0000 7f358e837740 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9651220 0x562db9658f90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.415+0000 7f358e837740 1 --2- 172.21.15.133:0/2999464310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db9650900 0x562db9658a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.415+0000 7f358e837740 1 -- 172.21.15.133:0/2999464310 >> 172.21.15.133:0/2999464310 conn(0x562db964ce90 msgr2=0x562db9652bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.415+0000 7f356dffb640 0 client.16335 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.415+0000 7f358e837740 1 -- 172.21.15.133:0/2999464310 shutdown_connections 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.415+0000 7f358e837740 1 -- 172.21.15.133:0/2999464310 wait complete. 2024-09-11T23:39:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.415+0000 7f358e837740 20 client.16335 trim_cache size 0 max 16384 2024-09-11T23:39:40.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:40 smithi133 ceph-mon[31430]: pgmap v89: 97 pgs: 97 active+clean; 19 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 868 MiB/s wr, 2.96k op/s 2024-09-11T23:39:40.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.464+0000 7f358e837740 1 Processor -- start 2024-09-11T23:39:40.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.464+0000 7f358e837740 1 -- start start 2024-09-11T23:39:40.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.465+0000 7f358e837740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9650900 0x562db966eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.465+0000 7f358e837740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9651220 0x562db96533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.465+0000 7f358a1fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9650900 0x562db966eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.465+0000 7f358a1fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9650900 0x562db966eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51136/0 (socket says 172.21.15.133:51136) 2024-09-11T23:39:40.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.466+0000 7f358a1fa640 1 -- 172.21.15.133:0/3895534926 learned_addr learned my addr 172.21.15.133:0/3895534926 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:40.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.466+0000 7f358e837740 1 --2- 172.21.15.133:0/3895534926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db965aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.467+0000 7f358e837740 1 -- 172.21.15.133:0/3895534926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3584084560 con 0x562db9650900 2024-09-11T23:39:40.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.467+0000 7f358e837740 1 -- 172.21.15.133:0/3895534926 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35841b57b0 con 0x562db9651220 2024-09-11T23:39:40.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.467+0000 7f35899f9640 1 --2- 172.21.15.133:0/3895534926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9651220 0x562db96533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.468+0000 7f358e837740 1 -- 172.21.15.133:0/3895534926 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3584084c30 con 0x562db965bae0 2024-09-11T23:39:40.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.469+0000 7f358a9fb640 1 --2- 172.21.15.133:0/3895534926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db965aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.470+0000 7f358a1fa640 1 -- 172.21.15.133:0/3895534926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 msgr2=0x562db965aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:40.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.470+0000 7f358a1fa640 1 --2- 172.21.15.133:0/3895534926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db965aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.470+0000 7f358a1fa640 1 -- 172.21.15.133:0/3895534926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9651220 msgr2=0x562db96533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:40.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.470+0000 7f358a1fa640 1 --2- 172.21.15.133:0/3895534926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9651220 0x562db96533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.470+0000 7f358a1fa640 1 -- 172.21.15.133:0/3895534926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db9656fa0 con 0x562db9650900 2024-09-11T23:39:40.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.471+0000 7f358a1fa640 1 --2- 172.21.15.133:0/3895534926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9650900 0x562db966eec0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f35800095e0 tx=0x7f3580009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27be4db7e002c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:40.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.494+0000 7f356ffff640 1 -- 172.21.15.133:0/3895534926 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580015050 con 0x562db9650900 2024-09-11T23:39:40.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.494+0000 7f356ffff640 1 -- 172.21.15.133:0/3895534926 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3580004520 con 0x562db9650900 2024-09-11T23:39:40.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.496+0000 7f358e837740 1 -- 172.21.15.133:0/3895534926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9650900 msgr2=0x562db966eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:40.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.496+0000 7f358e837740 1 --2- 172.21.15.133:0/3895534926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9650900 0x562db966eec0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f35800095e0 tx=0x7f3580009b50 comp rx=0 tx=0).stop 2024-09-11T23:39:40.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.496+0000 7f358e837740 1 -- 172.21.15.133:0/3895534926 shutdown_connections 2024-09-11T23:39:40.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.496+0000 7f358e837740 1 --2- 172.21.15.133:0/3895534926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db965aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.496+0000 7f358e837740 1 --2- 172.21.15.133:0/3895534926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9651220 0x562db96533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.496+0000 7f358e837740 1 --2- 172.21.15.133:0/3895534926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9650900 0x562db966eec0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.496+0000 7f358e837740 1 -- 172.21.15.133:0/3895534926 >> 172.21.15.133:0/3895534926 conn(0x562db964ce90 msgr2=0x562db9657bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:40.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.498+0000 7f358e837740 1 -- 172.21.15.133:0/3895534926 shutdown_connections 2024-09-11T23:39:40.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.499+0000 7f358e837740 1 -- 172.21.15.133:0/3895534926 wait complete. 2024-09-11T23:39:40.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.499+0000 7f358e837740 1 -- start start 2024-09-11T23:39:40.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.500+0000 7f358e837740 1 Processor -- start 2024-09-11T23:39:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.500+0000 7f358e837740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9650900 0x562db9671360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.501+0000 7f358e837740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9651220 0x562db96718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.501+0000 7f358e837740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db96748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.501+0000 7f358e837740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f35841b50e0 con 0x562db9650900 2024-09-11T23:39:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.501+0000 7f358e837740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35841b4a10 con 0x562db9651220 2024-09-11T23:39:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.501+0000 7f358e837740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3584082540 con 0x562db965bae0 2024-09-11T23:39:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.501+0000 7f358e837740 10 client.0 _ll_register_callbacks cb 0x7f357409bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.505+0000 7f358a9fb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db96748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.505+0000 7f358a9fb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db96748e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53158/0 (socket says 172.21.15.133:53158) 2024-09-11T23:39:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.505+0000 7f358a9fb640 1 -- 172.21.15.133:0/3324637740 learned_addr learned my addr 172.21.15.133:0/3324637740 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.505+0000 7f35899f9640 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9651220 0x562db96718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.508+0000 7f35899f9640 1 -- 172.21.15.133:0/3324637740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 msgr2=0x562db96748e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:40.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.508+0000 7f35899f9640 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db96748e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.508+0000 7f35899f9640 1 -- 172.21.15.133:0/3324637740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9650900 msgr2=0x562db9671360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:40.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.508+0000 7f35899f9640 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9650900 0x562db9671360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.508+0000 7f35899f9640 1 -- 172.21.15.133:0/3324637740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db9674f80 con 0x562db9651220 2024-09-11T23:39:40.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.511+0000 7f35899f9640 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9651220 0x562db96718a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x562db965a000 tx=0x7f3574019f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57aa3683f11aaead server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:40.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.511+0000 7f356dffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:39:40.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.552+0000 7f356dffb640 1 -- 172.21.15.133:0/3324637740 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35740978f0 con 0x562db9651220 2024-09-11T23:39:40.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.553+0000 7f358e837740 1 -- 172.21.15.133:0/3324637740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3580004520 con 0x562db9651220 2024-09-11T23:39:40.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.553+0000 7f356dffb640 1 -- 172.21.15.133:0/3324637740 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f35740afb70 con 0x562db9651220 2024-09-11T23:39:40.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.553+0000 7f356dffb640 1 -- 172.21.15.133:0/3324637740 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35740a42c0 con 0x562db9651220 2024-09-11T23:39:40.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.554+0000 7f358e837740 10 client.25790 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:40.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.554+0000 7f358e837740 1 -- 172.21.15.133:0/3324637740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562db96f8a50 con 0x562db9651220 2024-09-11T23:39:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.554+0000 7f356dffb640 1 -- 172.21.15.133:0/3324637740 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3580004520 con 0x562db9651220 2024-09-11T23:39:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.554+0000 7f356dffb640 1 -- 172.21.15.133:0/3324637740 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f35740bb040 con 0x562db9651220 2024-09-11T23:39:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.555+0000 7f358e837740 10 client.25790 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.555+0000 7f358e837740 10 client.25790 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:39:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.555+0000 7f358e837740 20 client.25790 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.555+0000 7f358e837740 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562db96f92c0 0x562db96fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.555+0000 7f358e837740 4 client.25790 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-11T23:39:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.555+0000 7f358e837740 1 -- 172.21.15.133:0/3324637740 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x562db95bdd20 con 0x562db96f92c0 2024-09-11T23:39:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.555+0000 7f358a1fa640 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562db96f92c0 0x562db96fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.558+0000 7f356dffb640 10 client.25790 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:40.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.559+0000 7f358a1fa640 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562db96f92c0 0x562db96fb6e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d09cd49e8e294d4 server_cookie=1ee7e595998480b7 in_seq=0 out_seq=0 2024-09-11T23:39:40.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:40 smithi178 ceph-mon[35882]: pgmap v89: 97 pgs: 97 active+clean; 19 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 868 MiB/s wr, 2.96k op/s 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f356dffb640 1 -- 172.21.15.133:0/3324637740 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562db95bdd20 con 0x562db96f92c0 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f356dffb640 10 client.25790 handle_command_reply: tid=0 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f358e837740 1 client.25790 shutdown 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f358e837740 1 -- 172.21.15.133:0/3324637740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9651220 msgr2=0x562db96718a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f358e837740 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9651220 0x562db96718a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x562db965a000 tx=0x7f3574019f70 comp rx=0 tx=0).stop 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f358e837740 1 -- 172.21.15.133:0/3324637740 shutdown_connections 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f358e837740 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562db96f92c0 0x562db96fb6e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f358e837740 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562db965bae0 0x562db96748e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f358e837740 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db9651220 0x562db96718a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f356dffb640 0 client.25790 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f358e837740 1 --2- 172.21.15.133:0/3324637740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562db9650900 0x562db9671360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.810+0000 7f358e837740 1 -- 172.21.15.133:0/3324637740 >> 172.21.15.133:0/3324637740 conn(0x562db964ce90 msgr2=0x562db964c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:40.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358e837740 1 -- 172.21.15.133:0/3324637740 shutdown_connections 2024-09-11T23:39:40.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358e837740 1 -- 172.21.15.133:0/3324637740 wait complete. 2024-09-11T23:39:40.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358e837740 20 client.25790 trim_cache size 0 max 16384 2024-09-11T23:39:40.812 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:39:40.812 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 28, 2024-09-11T23:39:40.812 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:39:40.812 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 10, 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: "a1b0efb3": { 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 10, 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 144.7, 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:39:40.813 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:39:40.814 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:39:40.814 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:39:40.814 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:39:40.814 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:39:40.814 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:40.814 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:40.814 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:40.814 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:40.814 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:39:40.814 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:39:40.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358b1fc640 1 -- 172.21.15.133:0/4002979589 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35700a2f00 msgr2=0x7f35700a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:40.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358b1fc640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35700a2f00 0x7f35700a53c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f357c02f570 tx=0x7f357c02a8f0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358b1fc640 1 -- 172.21.15.133:0/4002979589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3584109bb0 msgr2=0x7f3584073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:40.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358b1fc640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3584109bb0 0x7f3584073e80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f357400d6d0 tx=0x7f357400db90 comp rx=0 tx=0).stop 2024-09-11T23:39:40.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358b1fc640 1 -- 172.21.15.133:0/4002979589 shutdown_connections 2024-09-11T23:39:40.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358b1fc640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35700a2f00 0x7f35700a53c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358b1fc640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3584112670 0x7f358406cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358b1fc640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3584109bb0 0x7f3584073e80 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358b1fc640 1 --2- 172.21.15.133:0/4002979589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3584109290 0x7f3584073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:40.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358b1fc640 1 -- 172.21.15.133:0/4002979589 >> 172.21.15.133:0/4002979589 conn(0x7f3584077960 msgr2=0x7f3584078030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:40.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.811+0000 7f358b1fc640 1 -- 172.21.15.133:0/4002979589 shutdown_connections 2024-09-11T23:39:40.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:40.812+0000 7f358b1fc640 1 -- 172.21.15.133:0/4002979589 wait complete. 2024-09-11T23:39:40.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:40 smithi138 ceph-mon[35142]: pgmap v89: 97 pgs: 97 active+clean; 19 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 868 MiB/s wr, 2.96k op/s 2024-09-11T23:39:41.626 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a1b0efb3', quiesce took 7.8 seconds. Will release after: 16.9 2024-09-11T23:39:41.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:41 smithi133 ceph-mon[31430]: pgmap v90: 97 pgs: 97 active+clean; 19 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 659 MiB/s wr, 1.92k op/s 2024-09-11T23:39:41.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:41 smithi178 ceph-mon[35882]: pgmap v90: 97 pgs: 97 active+clean; 19 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 659 MiB/s wr, 1.92k op/s 2024-09-11T23:39:41.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:41 smithi138 ceph-mon[35142]: pgmap v90: 97 pgs: 97 active+clean; 19 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 659 MiB/s wr, 1.92k op/s 2024-09-11T23:39:44.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:44 smithi178 ceph-mon[35882]: pgmap v91: 97 pgs: 97 active+clean; 19 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 723 MiB/s wr, 1.83k op/s 2024-09-11T23:39:44.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:44 smithi138 ceph-mon[35142]: pgmap v91: 97 pgs: 97 active+clean; 19 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 723 MiB/s wr, 1.83k op/s 2024-09-11T23:39:44.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:44 smithi133 ceph-mon[31430]: pgmap v91: 97 pgs: 97 active+clean; 19 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 723 MiB/s wr, 1.83k op/s 2024-09-11T23:39:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.110+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3055677408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dd01138b0 msgr2=0x7f8dd0115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.110+0000 7f8dd81c4640 1 --2- 172.21.15.133:0/3055677408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dd01138b0 0x7f8dd0115ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009900 tx=0x7f8dc802a3f0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.112+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3055677408 shutdown_connections 2024-09-11T23:39:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.112+0000 7f8dd81c4640 1 --2- 172.21.15.133:0/3055677408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dd01138b0 0x7f8dd0115ca0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.112+0000 7f8dd81c4640 1 --2- 172.21.15.133:0/3055677408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dd010acb0 0x7f8dd0113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.112+0000 7f8dd81c4640 1 --2- 172.21.15.133:0/3055677408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd010a300 0x7f8dd010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.112+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3055677408 >> 172.21.15.133:0/3055677408 conn(0x7f8dd006ce00 msgr2=0x7f8dd006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.113+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3055677408 shutdown_connections 2024-09-11T23:39:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.113+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3055677408 wait complete. 2024-09-11T23:39:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.114+0000 7f8dd81c4640 1 Processor -- start 2024-09-11T23:39:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.114+0000 7f8dd81c4640 1 -- start start 2024-09-11T23:39:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.115+0000 7f8dd81c4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dd010a300 0x7f8dd01a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.115+0000 7f8dd81c4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dd010acb0 0x7f8dd01a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.115+0000 7f8dd81c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd01138b0 0x7f8dd01a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.115+0000 7f8dd81c4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd011a720 con 0x7f8dd01138b0 2024-09-11T23:39:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.115+0000 7f8dd81c4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd011a5a0 con 0x7f8dd010acb0 2024-09-11T23:39:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.115+0000 7f8dd81c4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd011a8a0 con 0x7f8dd010a300 2024-09-11T23:39:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.116+0000 7f8dd6761640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd01138b0 0x7f8dd01a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.116+0000 7f8dd6761640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd01138b0 0x7f8dd01a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59960/0 (socket says 172.21.15.133:59960) 2024-09-11T23:39:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.116+0000 7f8dd6761640 1 -- 172.21.15.133:0/3902943077 learned_addr learned my addr 172.21.15.133:0/3902943077 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:46.118 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.117+0000 7f8dd6761640 1 -- 172.21.15.133:0/3902943077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dd010a300 msgr2=0x7f8dd01a67e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:39:46.118 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.117+0000 7f8dd6761640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dd010a300 0x7f8dd01a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.118 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.117+0000 7f8dd6761640 1 -- 172.21.15.133:0/3902943077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dd010acb0 msgr2=0x7f8dd01a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:46.118 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.117+0000 7f8dd6761640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dd010acb0 0x7f8dd01a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.118 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.117+0000 7f8dd6761640 1 -- 172.21.15.133:0/3902943077 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd01a1110 con 0x7f8dd01138b0 2024-09-11T23:39:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.117+0000 7f8dd6761640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd01138b0 0x7f8dd01a0990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8dc802a960 tx=0x7f8dc8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.118+0000 7f8dc6ffd640 1 -- 172.21.15.133:0/3902943077 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc803f070 con 0x7f8dd01138b0 2024-09-11T23:39:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.118+0000 7f8dc6ffd640 1 -- 172.21.15.133:0/3902943077 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dc8031c90 con 0x7f8dd01138b0 2024-09-11T23:39:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.118+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3902943077 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd01a13a0 con 0x7f8dd01138b0 2024-09-11T23:39:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.119+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3902943077 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd01ad6a0 con 0x7f8dd01138b0 2024-09-11T23:39:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.119+0000 7f8dc6ffd640 1 -- 172.21.15.133:0/3902943077 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8035030 con 0x7f8dd01138b0 2024-09-11T23:39:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.129+0000 7f8ddb03a740 1 Processor -- start 2024-09-11T23:39:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.130+0000 7f8ddb03a740 1 -- start start 2024-09-11T23:39:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.130+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1cecc0 0x56537a1cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.130+0000 7f8dc6ffd640 1 -- 172.21.15.133:0/3902943077 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8dc8042070 con 0x7f8dd01138b0 2024-09-11T23:39:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.132+0000 7f8dc6ffd640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8dbc0a2fd0 0x7f8dbc0a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.132+0000 7f8dc6ffd640 1 -- 172.21.15.133:0/3902943077 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8dc80efbf0 con 0x7f8dd01138b0 2024-09-11T23:39:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.133+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1ca660 0x56537a1caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.133+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 0x56537a1d69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.133+0000 7f8ddb03a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56537a1252f0 con 0x56537a1cecc0 2024-09-11T23:39:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.133+0000 7f8dd575f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1cecc0 0x56537a1cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.133+0000 7f8ddb03a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56537a0cd270 con 0x56537a1cb000 2024-09-11T23:39:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.133+0000 7f8ddb03a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56537a1212b0 con 0x56537a1ca660 2024-09-11T23:39:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.133+0000 7f8dd5f60640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8dbc0a2fd0 0x7f8dbc0a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.133+0000 7f8dd6761640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 0x56537a1d69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.133+0000 7f8dd6761640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 0x56537a1d69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:60506/0 (socket says 172.21.15.133:60506) 2024-09-11T23:39:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.133+0000 7f8dd6761640 1 -- 172.21.15.133:0/725069845 learned_addr learned my addr 172.21.15.133:0/725069845 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.134+0000 7f8dd6761640 1 -- 172.21.15.133:0/725069845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1ca660 msgr2=0x56537a1caac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:39:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.134+0000 7f8dd6761640 1 --2- 172.21.15.133:0/725069845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1ca660 0x56537a1caac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.134+0000 7f8dd6761640 1 -- 172.21.15.133:0/725069845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1cecc0 msgr2=0x56537a1cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.134+0000 7f8dd6761640 1 --2- 172.21.15.133:0/725069845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1cecc0 0x56537a1cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.134+0000 7f8dd6761640 1 -- 172.21.15.133:0/725069845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537a0cc340 con 0x56537a1cb000 2024-09-11T23:39:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.135+0000 7f8dd5f60640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8dbc0a2fd0 0x7f8dbc0a5490 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8dd01a1ad0 tx=0x7f8dcc009250 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.135+0000 7f8dd6761640 1 --2- 172.21.15.133:0/725069845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 0x56537a1d69c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80c4860 tx=0x7f8dc8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1469943e6d612c28 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.136+0000 7f8dbbfff640 1 -- 172.21.15.133:0/725069845 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80c04d0 con 0x56537a1cb000 2024-09-11T23:39:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.136+0000 7f8dbbfff640 1 -- 172.21.15.133:0/725069845 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dc80c4d90 con 0x56537a1cb000 2024-09-11T23:39:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.138+0000 7f8ddb03a740 1 -- 172.21.15.133:0/725069845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 msgr2=0x56537a1d69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.138+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/725069845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 0x56537a1d69c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80c4860 tx=0x7f8dc8005d50 comp rx=0 tx=0).stop 2024-09-11T23:39:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.138+0000 7f8ddb03a740 1 -- 172.21.15.133:0/725069845 shutdown_connections 2024-09-11T23:39:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.138+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/725069845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 0x56537a1d69c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.138+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/725069845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1ca660 0x56537a1caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.138+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/725069845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1cecc0 0x56537a1cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.138+0000 7f8ddb03a740 1 -- 172.21.15.133:0/725069845 >> 172.21.15.133:0/725069845 conn(0x56537a1bcb10 msgr2=0x56537a1bcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.139+0000 7f8ddb03a740 1 -- 172.21.15.133:0/725069845 shutdown_connections 2024-09-11T23:39:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.139+0000 7f8ddb03a740 1 -- 172.21.15.133:0/725069845 wait complete. 2024-09-11T23:39:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.139+0000 7f8ddb03a740 1 -- start start 2024-09-11T23:39:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.140+0000 7f8ddb03a740 1 Processor -- start 2024-09-11T23:39:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.140+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.141+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 0x56537a1d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.141+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cecc0 0x56537a1d47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.141+0000 7f8ddb03a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56537a1d6f00 con 0x56537a1ca660 2024-09-11T23:39:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.141+0000 7f8ddb03a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56537a0ae290 con 0x56537a1cecc0 2024-09-11T23:39:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.141+0000 7f8ddb03a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56537a1d7080 con 0x56537a1cb000 2024-09-11T23:39:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.141+0000 7f8ddb03a740 10 client.0 _ll_register_callbacks cb 0x56537a0df140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.141+0000 7f8dd575f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 0x56537a1d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.142+0000 7f8dd575f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 0x56537a1d42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:36730/0 (socket says 172.21.15.133:36730) 2024-09-11T23:39:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.142+0000 7f8dd575f640 1 -- 172.21.15.133:0/2864565293 learned_addr learned my addr 172.21.15.133:0/2864565293 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.142+0000 7f8dd575f640 1 -- 172.21.15.133:0/2864565293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cecc0 msgr2=0x56537a1d47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.142+0000 7f8dd575f640 1 --2- 172.21.15.133:0/2864565293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cecc0 0x56537a1d47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.142+0000 7f8dd575f640 1 -- 172.21.15.133:0/2864565293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 msgr2=0x56537a1d3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.142+0000 7f8dd575f640 1 --2- 172.21.15.133:0/2864565293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.142+0000 7f8dd575f640 1 -- 172.21.15.133:0/2864565293 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565379e38210 con 0x56537a1cb000 2024-09-11T23:39:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.144+0000 7f8dd575f640 1 --2- 172.21.15.133:0/2864565293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 0x56537a1d42a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000d620 tx=0x7f8dc000dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c39829eb6772b80c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.144+0000 7f8db97fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:39:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.145+0000 7f8db97fa640 1 -- 172.21.15.133:0/2864565293 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc000f8f0 con 0x56537a1cb000 2024-09-11T23:39:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.145+0000 7f8db97fa640 1 -- 172.21.15.133:0/2864565293 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8dc000fa90 con 0x56537a1cb000 2024-09-11T23:39:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.145+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2864565293 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dc80c4d90 con 0x56537a1cb000 2024-09-11T23:39:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.146+0000 7f8db97fa640 1 -- 172.21.15.133:0/2864565293 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8dc000fc90 con 0x56537a1cb000 2024-09-11T23:39:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.146+0000 7f8ddb03a740 10 client.25765 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.146+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2864565293 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565379f04620 con 0x56537a1cb000 2024-09-11T23:39:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.147+0000 7f8db97fa640 1 -- 172.21.15.133:0/2864565293 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dc80c4d90 con 0x56537a1cb000 2024-09-11T23:39:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.147+0000 7f8db97fa640 1 -- 172.21.15.133:0/2864565293 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8dc001f030 con 0x56537a1cb000 2024-09-11T23:39:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.147+0000 7f8ddb03a740 10 client.25765 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.147+0000 7f8ddb03a740 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.147+0000 7f8ddb03a740 20 client.25765 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.147+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2864565293 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56537a1c4d60 0x56537a1c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.147+0000 7f8ddb03a740 4 client.25765 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.147+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2864565293 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56537a132a20 con 0x56537a1c4d60 2024-09-11T23:39:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.148+0000 7f8dd5f60640 1 --2- 172.21.15.133:0/2864565293 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56537a1c4d60 0x56537a1c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.154+0000 7f8dd5f60640 1 --2- 172.21.15.133:0/2864565293 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56537a1c4d60 0x56537a1c71c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1cd4940d210178 server_cookie=f326def2ad565525 in_seq=0 out_seq=0 2024-09-11T23:39:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.155+0000 7f8db97fa640 10 client.25765 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.158+0000 7f8db97fa640 1 -- 172.21.15.133:0/2864565293 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56537a132a20 con 0x56537a1c4d60 2024-09-11T23:39:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.158+0000 7f8db97fa640 10 client.25765 handle_command_reply: tid=0 2024-09-11T23:39:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.158+0000 7f8ddb03a740 1 client.25765 shutdown 2024-09-11T23:39:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.158+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2864565293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 msgr2=0x56537a1d42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.158+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2864565293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 0x56537a1d42a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000d620 tx=0x7f8dc000dae0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.159+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2864565293 shutdown_connections 2024-09-11T23:39:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.159+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2864565293 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56537a1c4d60 0x56537a1c71c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.159+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2864565293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cecc0 0x56537a1d47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.159+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2864565293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 0x56537a1d42a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.159+0000 7f8db97fa640 0 client.25765 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.159+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2864565293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.159+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2864565293 >> 172.21.15.133:0/2864565293 conn(0x56537a1bcb10 msgr2=0x56537a1c9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.159+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2864565293 shutdown_connections 2024-09-11T23:39:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.159+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2864565293 wait complete. 2024-09-11T23:39:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.159+0000 7f8ddb03a740 20 client.25765 trim_cache size 0 max 16384 2024-09-11T23:39:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.190+0000 7f8ddb03a740 1 Processor -- start 2024-09-11T23:39:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.190+0000 7f8ddb03a740 1 -- start start 2024-09-11T23:39:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.190+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.190+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 0x56537a1bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.190+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cecc0 0x56537a1f6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.190+0000 7f8ddb03a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b3db0 con 0x56537a1ca660 2024-09-11T23:39:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.190+0000 7f8ddb03a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b58f0 con 0x56537a1cecc0 2024-09-11T23:39:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.190+0000 7f8ddb03a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b4480 con 0x56537a1cb000 2024-09-11T23:39:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.191+0000 7f8dd5f60640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.191+0000 7f8dd5f60640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59990/0 (socket says 172.21.15.133:59990) 2024-09-11T23:39:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.191+0000 7f8dd5f60640 1 -- 172.21.15.133:0/1834630112 learned_addr learned my addr 172.21.15.133:0/1834630112 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.191+0000 7f8dd575f640 1 --2- 172.21.15.133:0/1834630112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 0x56537a1bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.191+0000 7f8dd5f60640 1 -- 172.21.15.133:0/1834630112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 msgr2=0x56537a1bc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.191+0000 7f8dd5f60640 1 --2- 172.21.15.133:0/1834630112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 0x56537a1bc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.191+0000 7f8dd5f60640 1 -- 172.21.15.133:0/1834630112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cecc0 msgr2=0x56537a1f6370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:39:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.191+0000 7f8dd5f60640 1 --2- 172.21.15.133:0/1834630112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cecc0 0x56537a1f6370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.191+0000 7f8dd5f60640 1 -- 172.21.15.133:0/1834630112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537a1c0980 con 0x56537a1ca660 2024-09-11T23:39:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.191+0000 7f8dd575f640 1 --2- 172.21.15.133:0/1834630112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 0x56537a1bc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:39:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.191+0000 7f8dd5f60640 1 --2- 172.21.15.133:0/1834630112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d4610 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009790 tx=0x7f8dcc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afed82426452705f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.192+0000 7f8dbb7fe640 1 -- 172.21.15.133:0/1834630112 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc026070 con 0x56537a1ca660 2024-09-11T23:39:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.192+0000 7f8dbb7fe640 1 -- 172.21.15.133:0/1834630112 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dcc009c20 con 0x56537a1ca660 2024-09-11T23:39:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.192+0000 7f8dbb7fe640 1 -- 172.21.15.133:0/1834630112 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc019680 con 0x56537a1ca660 2024-09-11T23:39:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.193+0000 7f8ddb03a740 1 -- 172.21.15.133:0/1834630112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 msgr2=0x56537a1d4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.193+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/1834630112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d4610 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009790 tx=0x7f8dcc00f000 comp rx=0 tx=0).stop 2024-09-11T23:39:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.193+0000 7f8ddb03a740 1 -- 172.21.15.133:0/1834630112 shutdown_connections 2024-09-11T23:39:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.193+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/1834630112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cecc0 0x56537a1f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.193+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/1834630112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cb000 0x56537a1bc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.193+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/1834630112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d4610 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.193+0000 7f8ddb03a740 1 -- 172.21.15.133:0/1834630112 >> 172.21.15.133:0/1834630112 conn(0x56537a1bcb10 msgr2=0x56537a1bb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.193+0000 7f8ddb03a740 1 -- 172.21.15.133:0/1834630112 shutdown_connections 2024-09-11T23:39:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.193+0000 7f8ddb03a740 1 -- 172.21.15.133:0/1834630112 wait complete. 2024-09-11T23:39:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.194+0000 7f8ddb03a740 1 -- start start 2024-09-11T23:39:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.195+0000 7f8ddb03a740 1 Processor -- start 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.196+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.196+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 0x56537a1d8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.196+0000 7f8ddb03a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cecc0 0x56537a256330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8ddb03a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b5220 con 0x56537a1ca660 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8dd5f60640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8dd5f60640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60000/0 (socket says 172.21.15.133:60000) 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8dd5f60640 1 -- 172.21.15.133:0/2857048066 learned_addr learned my addr 172.21.15.133:0/2857048066 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2857048066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b4b50 con 0x56537a1cb000 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2857048066 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01b1db0 con 0x56537a1cecc0 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8dd6761640 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cecc0 0x56537a256330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8dd575f640 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 0x56537a1d8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8ddb03a740 10 client.0 _ll_register_callbacks cb 0x7f8dc000bcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8dd5f60640 1 -- 172.21.15.133:0/2857048066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cecc0 msgr2=0x56537a256330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8dd5f60640 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cecc0 0x56537a256330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8dd5f60640 1 -- 172.21.15.133:0/2857048066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 msgr2=0x56537a1d8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8dd5f60640 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 0x56537a1d8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.197+0000 7f8dd5f60640 1 -- 172.21.15.133:0/2857048066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537a1d91b0 con 0x56537a1ca660 2024-09-11T23:39:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.198+0000 7f8dd5f60640 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d83c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc024040 tx=0x7f8dcc021e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9caa30645ff7d6cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.198+0000 7f8db97fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.198+0000 7f8db97fa640 1 -- 172.21.15.133:0/2857048066 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc02f060 con 0x56537a1ca660 2024-09-11T23:39:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.198+0000 7f8db97fa640 1 -- 172.21.15.133:0/2857048066 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8dcc0214d0 con 0x56537a1ca660 2024-09-11T23:39:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.198+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2857048066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dcc009c20 con 0x56537a1ca660 2024-09-11T23:39:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.199+0000 7f8db97fa640 1 -- 172.21.15.133:0/2857048066 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8dcc026040 con 0x56537a1ca660 2024-09-11T23:39:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.199+0000 7f8ddb03a740 10 client.16347 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.199+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2857048066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56537a257890 con 0x56537a1ca660 2024-09-11T23:39:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.199+0000 7f8db97fa640 1 -- 172.21.15.133:0/2857048066 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dcc021710 con 0x56537a1ca660 2024-09-11T23:39:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.200+0000 7f8db97fa640 1 -- 172.21.15.133:0/2857048066 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8dcc032020 con 0x56537a1ca660 2024-09-11T23:39:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.200+0000 7f8ddb03a740 10 client.16347 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.200+0000 7f8ddb03a740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.200+0000 7f8ddb03a740 20 client.16347 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.200+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56537a257ef0 0x56537a25a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.200+0000 7f8ddb03a740 4 client.16347 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:39:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.200+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2857048066 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8dcc009c20 con 0x56537a257ef0 2024-09-11T23:39:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.201+0000 7f8dd575f640 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56537a257ef0 0x56537a25a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.202+0000 7f8dd575f640 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56537a257ef0 0x56537a25a310 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1683979058991a04 server_cookie=aec2128c539a0290 in_seq=0 out_seq=0 2024-09-11T23:39:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.202+0000 7f8db97fa640 10 client.16347 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.386+0000 7f8db97fa640 1 -- 172.21.15.133:0/2857048066 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8dcc009c20 con 0x56537a257ef0 2024-09-11T23:39:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.386+0000 7f8db97fa640 10 client.16347 handle_command_reply: tid=0 2024-09-11T23:39:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.386+0000 7f8ddb03a740 1 client.16347 shutdown 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.386+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2857048066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 msgr2=0x56537a1d83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.386+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d83c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc024040 tx=0x7f8dcc021e80 comp rx=0 tx=0).stop 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.386+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2857048066 shutdown_connections 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.387+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56537a257ef0 0x56537a25a310 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.387+0000 7f8db97fa640 0 client.16347 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.387+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56537a1cecc0 0x56537a256330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.387+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537a1cb000 0x56537a1d8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.387+0000 7f8ddb03a740 1 --2- 172.21.15.133:0/2857048066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537a1ca660 0x56537a1d83c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.387+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2857048066 >> 172.21.15.133:0/2857048066 conn(0x56537a1bcb10 msgr2=0x56537a1bc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.387+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2857048066 shutdown_connections 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.387+0000 7f8ddb03a740 1 -- 172.21.15.133:0/2857048066 wait complete. 2024-09-11T23:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.387+0000 7f8ddb03a740 20 client.16347 trim_cache size 0 max 16384 2024-09-11T23:39:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:39:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:39:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:39:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:39:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3902943077 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8dbc0a2fd0 msgr2=0x7f8dbc0a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8dbc0a2fd0 0x7f8dbc0a5490 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8dd01a1ad0 tx=0x7f8dcc009250 comp rx=0 tx=0).stop 2024-09-11T23:39:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3902943077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd01138b0 msgr2=0x7f8dd01a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd01138b0 0x7f8dd01a0990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8dc802a960 tx=0x7f8dc8003b90 comp rx=0 tx=0).stop 2024-09-11T23:39:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3902943077 shutdown_connections 2024-09-11T23:39:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8dbc0a2fd0 0x7f8dbc0a5490 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd01138b0 0x7f8dd01a0990 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dd010acb0 0x7f8dd01a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 --2- 172.21.15.133:0/3902943077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8dd010a300 0x7f8dd01a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3902943077 >> 172.21.15.133:0/3902943077 conn(0x7f8dd006ce00 msgr2=0x7f8dd0070850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3902943077 shutdown_connections 2024-09-11T23:39:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:46.388+0000 7f8dd81c4640 1 -- 172.21.15.133:0/3902943077 wait complete. 2024-09-11T23:39:46.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:46 smithi178 ceph-mon[35882]: pgmap v92: 97 pgs: 97 active+clean; 19 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 569 MiB/s wr, 1.28k op/s 2024-09-11T23:39:46.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:46 smithi138 ceph-mon[35142]: pgmap v92: 97 pgs: 97 active+clean; 19 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 569 MiB/s wr, 1.28k op/s 2024-09-11T23:39:46.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:46 smithi133 ceph-mon[31430]: pgmap v92: 97 pgs: 97 active+clean; 19 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 569 MiB/s wr, 1.28k op/s 2024-09-11T23:39:47.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:47 smithi178 ceph-mon[35882]: pgmap v93: 97 pgs: 97 active+clean; 19 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 460 MiB/s wr, 1.06k op/s 2024-09-11T23:39:47.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:47 smithi138 ceph-mon[35142]: pgmap v93: 97 pgs: 97 active+clean; 19 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 460 MiB/s wr, 1.06k op/s 2024-09-11T23:39:47.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:47 smithi133 ceph-mon[31430]: pgmap v93: 97 pgs: 97 active+clean; 19 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 460 MiB/s wr, 1.06k op/s 2024-09-11T23:39:50.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:50 smithi178 ceph-mon[35882]: pgmap v94: 97 pgs: 97 active+clean; 20 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 488 MiB/s wr, 1.08k op/s 2024-09-11T23:39:50.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:50 smithi138 ceph-mon[35142]: pgmap v94: 97 pgs: 97 active+clean; 20 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 488 MiB/s wr, 1.08k op/s 2024-09-11T23:39:50.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:50 smithi133 ceph-mon[31430]: pgmap v94: 97 pgs: 97 active+clean; 20 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 488 MiB/s wr, 1.08k op/s 2024-09-11T23:39:52.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:52 smithi178 ceph-mon[35882]: pgmap v95: 97 pgs: 97 active+clean; 20 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 514 KiB/s rd, 220 MiB/s wr, 288 op/s 2024-09-11T23:39:52.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:52 smithi138 ceph-mon[35142]: pgmap v95: 97 pgs: 97 active+clean; 20 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 514 KiB/s rd, 220 MiB/s wr, 288 op/s 2024-09-11T23:39:52.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:52 smithi133 ceph-mon[31430]: pgmap v95: 97 pgs: 97 active+clean; 20 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 514 KiB/s rd, 220 MiB/s wr, 288 op/s 2024-09-11T23:39:54.500 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:54 smithi138 ceph-mon[35142]: pgmap v96: 97 pgs: 97 active+clean; 20 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 516 KiB/s rd, 199 MiB/s wr, 206 op/s 2024-09-11T23:39:54.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:54 smithi178 ceph-mon[35882]: pgmap v96: 97 pgs: 97 active+clean; 20 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 516 KiB/s rd, 199 MiB/s wr, 206 op/s 2024-09-11T23:39:54.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:54 smithi133 ceph-mon[31430]: pgmap v96: 97 pgs: 97 active+clean; 20 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 516 KiB/s rd, 199 MiB/s wr, 206 op/s 2024-09-11T23:39:56.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:56 smithi178 ceph-mon[35882]: pgmap v97: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 7.8 KiB/s rd, 108 MiB/s wr, 94 op/s 2024-09-11T23:39:56.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:56 smithi138 ceph-mon[35142]: pgmap v97: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 7.8 KiB/s rd, 108 MiB/s wr, 94 op/s 2024-09-11T23:39:56.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:56 smithi133 ceph-mon[31430]: pgmap v97: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 7.8 KiB/s rd, 108 MiB/s wr, 94 op/s 2024-09-11T23:39:58.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:39:58.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:58 smithi178 ceph-mon[35882]: pgmap v98: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 63 MiB/s wr, 48 op/s 2024-09-11T23:39:58.553 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:58 smithi133 ceph-mon[31430]: pgmap v98: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 63 MiB/s wr, 48 op/s 2024-09-11T23:39:58.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:58 smithi138 ceph-mon[35142]: pgmap v98: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 63 MiB/s wr, 48 op/s 2024-09-11T23:39:58.783 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:39:59.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.350+0000 7f365170e640 1 -- 172.21.15.133:0/3614717172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f364c100ca0 msgr2=0x7f364c106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.350+0000 7f365170e640 1 --2- 172.21.15.133:0/3614717172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f364c100ca0 0x7f364c106840 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f363c009930 tx=0x7f363c02a420 comp rx=0 tx=0).stop 2024-09-11T23:39:59.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.350+0000 7f365170e640 1 -- 172.21.15.133:0/3614717172 shutdown_connections 2024-09-11T23:39:59.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.350+0000 7f365170e640 1 --2- 172.21.15.133:0/3614717172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f364c106f50 0x7f364c10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.350+0000 7f365170e640 1 --2- 172.21.15.133:0/3614717172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f364c100ca0 0x7f364c106840 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.350+0000 7f365170e640 1 --2- 172.21.15.133:0/3614717172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f364c1002f0 0x7f364c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.350+0000 7f365170e640 1 -- 172.21.15.133:0/3614717172 >> 172.21.15.133:0/3614717172 conn(0x7f364c077210 msgr2=0x7f364c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.350+0000 7f365170e640 1 -- 172.21.15.133:0/3614717172 shutdown_connections 2024-09-11T23:39:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.350+0000 7f365170e640 1 -- 172.21.15.133:0/3614717172 wait complete. 2024-09-11T23:39:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.350+0000 7f365170e640 1 Processor -- start 2024-09-11T23:39:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f365170e640 1 -- start start 2024-09-11T23:39:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f365170e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f364c1002f0 0x7f364c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f365170e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f364c100ca0 0x7f364c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f365170e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f364c106f50 0x7f364c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f365170e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f364c111fe0 con 0x7f364c100ca0 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f365170e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f364c111e60 con 0x7f364c106f50 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f365170e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f364c112160 con 0x7f364c1002f0 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f364c100ca0 0x7f364c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f364c100ca0 0x7f364c1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34506/0 (socket says 172.21.15.133:34506) 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364a7fc640 1 -- 172.21.15.133:0/2370268082 learned_addr learned my addr 172.21.15.133:0/2370268082 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364b7fe640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f364c106f50 0x7f364c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364affd640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f364c1002f0 0x7f364c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364a7fc640 1 -- 172.21.15.133:0/2370268082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f364c1002f0 msgr2=0x7f364c101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364a7fc640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f364c1002f0 0x7f364c101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364a7fc640 1 -- 172.21.15.133:0/2370268082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f364c106f50 msgr2=0x7f364c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364a7fc640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f364c106f50 0x7f364c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364a7fc640 1 -- 172.21.15.133:0/2370268082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364c1a69c0 con 0x7f364c100ca0 2024-09-11T23:39:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364b7fe640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f364c106f50 0x7f364c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:39:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f364a7fc640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f364c100ca0 0x7f364c1017b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f363c009230 tx=0x7f363c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.351+0000 7f3643fff640 1 -- 172.21.15.133:0/2370268082 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c03f030 con 0x7f364c100ca0 2024-09-11T23:39:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.352+0000 7f365170e640 1 -- 172.21.15.133:0/2370268082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364c1a6c50 con 0x7f364c100ca0 2024-09-11T23:39:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.352+0000 7f3643fff640 1 -- 172.21.15.133:0/2370268082 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363c003ce0 con 0x7f364c100ca0 2024-09-11T23:39:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.352+0000 7f3643fff640 1 -- 172.21.15.133:0/2370268082 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c035070 con 0x7f364c100ca0 2024-09-11T23:39:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.352+0000 7f365170e640 1 -- 172.21.15.133:0/2370268082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364c1a7130 con 0x7f364c100ca0 2024-09-11T23:39:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.353+0000 7f365170e640 1 -- 172.21.15.133:0/2370268082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f364c1006d0 con 0x7f364c100ca0 2024-09-11T23:39:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.354+0000 7f3643fff640 1 -- 172.21.15.133:0/2370268082 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f363c042030 con 0x7f364c100ca0 2024-09-11T23:39:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.355+0000 7f3643fff640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f36380a2cf0 0x7f36380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.355+0000 7f364affd640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f36380a2cf0 0x7f36380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.355+0000 7f3643fff640 1 -- 172.21.15.133:0/2370268082 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f363c0efea0 con 0x7f364c100ca0 2024-09-11T23:39:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.355+0000 7f364affd640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f36380a2cf0 0x7f36380a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3634005e10 tx=0x7f3634005da0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:59.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.358+0000 7f3643fff640 1 -- 172.21.15.133:0/2370268082 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f363c0bcad0 con 0x7f364c100ca0 2024-09-11T23:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.482+0000 7ff325c53640 1 -- 172.21.15.133:0/1857288429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32010c110 msgr2=0x7ff32010e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.482+0000 7ff325c53640 1 --2- 172.21.15.133:0/1857288429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32010c110 0x7ff32010e560 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff318009ca0 tx=0x7ff31802a950 comp rx=0 tx=0).stop 2024-09-11T23:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.482+0000 7ff325c53640 1 -- 172.21.15.133:0/1857288429 shutdown_connections 2024-09-11T23:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.482+0000 7ff325c53640 1 --2- 172.21.15.133:0/1857288429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32010c110 0x7ff32010e560 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.482+0000 7ff325c53640 1 --2- 172.21.15.133:0/1857288429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff320103660 0x7ff32010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.482+0000 7ff325c53640 1 --2- 172.21.15.133:0/1857288429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff320102d40 0x7ff320103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.482+0000 7ff325c53640 1 -- 172.21.15.133:0/1857288429 >> 172.21.15.133:0/1857288429 conn(0x7ff320077990 msgr2=0x7ff3200fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.482+0000 7ff325c53640 1 -- 172.21.15.133:0/1857288429 shutdown_connections 2024-09-11T23:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.482+0000 7ff325c53640 1 -- 172.21.15.133:0/1857288429 wait complete. 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff325c53640 1 Processor -- start 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff325c53640 1 -- start start 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff325c53640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff320102d40 0x7ff320198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff325c53640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff320103660 0x7ff320198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff325c53640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32010c110 0x7ff32019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff325c53640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff320111eb0 con 0x7ff320103660 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff325c53640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff320111d30 con 0x7ff32010c110 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff325c53640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff320112030 con 0x7ff320102d40 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff317fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff320103660 0x7ff320198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff317fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff320103660 0x7ff320198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34538/0 (socket says 172.21.15.133:34538) 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff317fff640 1 -- 172.21.15.133:0/2154364616 learned_addr learned my addr 172.21.15.133:0/2154364616 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff31ffff640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32010c110 0x7ff32019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff31f7fe640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff320102d40 0x7ff320198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff317fff640 1 -- 172.21.15.133:0/2154364616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff320102d40 msgr2=0x7ff320198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff317fff640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff320102d40 0x7ff320198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff317fff640 1 -- 172.21.15.133:0/2154364616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32010c110 msgr2=0x7ff32019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff317fff640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32010c110 0x7ff32019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.483+0000 7ff317fff640 1 -- 172.21.15.133:0/2154364616 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32019d2f0 con 0x7ff320103660 2024-09-11T23:39:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.484+0000 7ff317fff640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff320103660 0x7ff320198840 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e9f0 tx=0x7ff30800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.484+0000 7ff31d7fa640 1 -- 172.21.15.133:0/2154364616 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30800cd80 con 0x7ff320103660 2024-09-11T23:39:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.484+0000 7ff31d7fa640 1 -- 172.21.15.133:0/2154364616 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff308010910 con 0x7ff320103660 2024-09-11T23:39:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.484+0000 7ff325c53640 1 -- 172.21.15.133:0/2154364616 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32019d5e0 con 0x7ff320103660 2024-09-11T23:39:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.484+0000 7ff31d7fa640 1 -- 172.21.15.133:0/2154364616 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308018990 con 0x7ff320103660 2024-09-11T23:39:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.484+0000 7ff325c53640 1 -- 172.21.15.133:0/2154364616 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3201a4e70 con 0x7ff320103660 2024-09-11T23:39:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.486+0000 7ff31d7fa640 1 -- 172.21.15.133:0/2154364616 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff308010ac0 con 0x7ff320103660 2024-09-11T23:39:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.486+0000 7ff3289ec740 1 Processor -- start 2024-09-11T23:39:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.486+0000 7ff3289ec740 1 -- start start 2024-09-11T23:39:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.486+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b7137d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.486+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b7146b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71470c0 0x5595b71494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff3289ec740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5595b709c2f0 con 0x5595b71382d0 2024-09-11T23:39:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff3289ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5595b7044270 con 0x5595b71379b0 2024-09-11T23:39:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff3289ec740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5595b70982b0 con 0x5595b71470c0 2024-09-11T23:39:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31d7fa640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff30c0a2da0 0x7ff30c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b7146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b7146b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34544/0 (socket says 172.21.15.133:34544) 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31f7fe640 1 -- 172.21.15.133:0/1084021928 learned_addr learned my addr 172.21.15.133:0/1084021928 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31ffff640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff30c0a2da0 0x7ff30c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31d7fa640 1 -- 172.21.15.133:0/2154364616 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff3080cf670 con 0x7ff320103660 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff317fff640 1 --2- 172.21.15.133:0/1084021928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71470c0 0x5595b71494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31ffff640 1 --2- 172.21.15.133:0/1084021928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b7137d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31f7fe640 1 -- 172.21.15.133:0/1084021928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71470c0 msgr2=0x5595b71494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31f7fe640 1 --2- 172.21.15.133:0/1084021928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71470c0 0x5595b71494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31f7fe640 1 -- 172.21.15.133:0/1084021928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 msgr2=0x5595b7137d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31f7fe640 1 --2- 172.21.15.133:0/1084021928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b7137d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31f7fe640 1 -- 172.21.15.133:0/1084021928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b7043340 con 0x5595b71382d0 2024-09-11T23:39:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31ffff640 1 --2- 172.21.15.133:0/1084021928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b7137d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31f7fe640 1 --2- 172.21.15.133:0/1084021928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b7146b80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff31000b760 tx=0x7ff31000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f110a3c5b2a130ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff31ffff640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff30c0a2da0 0x7ff30c0a5260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff318005e30 tx=0x7ff318009800 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff315ffb640 1 -- 172.21.15.133:0/1084021928 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310003dd0 con 0x5595b71382d0 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff315ffb640 1 -- 172.21.15.133:0/1084021928 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff310015070 con 0x5595b71382d0 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.487+0000 7ff315ffb640 1 -- 172.21.15.133:0/1084021928 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310011440 con 0x5595b71382d0 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.488+0000 7ff3289ec740 1 -- 172.21.15.133:0/1084021928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 msgr2=0x5595b7146b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.488+0000 7ff3289ec740 1 --2- 172.21.15.133:0/1084021928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b7146b80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff31000b760 tx=0x7ff31000bc20 comp rx=0 tx=0).stop 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.488+0000 7ff3289ec740 1 -- 172.21.15.133:0/1084021928 shutdown_connections 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.488+0000 7ff3289ec740 1 --2- 172.21.15.133:0/1084021928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71470c0 0x5595b71494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.488+0000 7ff3289ec740 1 --2- 172.21.15.133:0/1084021928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b7146b80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.488+0000 7ff3289ec740 1 --2- 172.21.15.133:0/1084021928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b7137d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.488+0000 7ff3289ec740 1 -- 172.21.15.133:0/1084021928 >> 172.21.15.133:0/1084021928 conn(0x5595b7134060 msgr2=0x5595b7134440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.488+0000 7ff3289ec740 1 -- 172.21.15.133:0/1084021928 shutdown_connections 2024-09-11T23:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.488+0000 7ff3289ec740 1 -- 172.21.15.133:0/1084021928 wait complete. 2024-09-11T23:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.488+0000 7ff3289ec740 1 -- start start 2024-09-11T23:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff3289ec740 1 Processor -- start 2024-09-11T23:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71379b0 0x5595b7144060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71382d0 0x5595b71445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b7144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff3289ec740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5595b7149c90 con 0x5595b71470c0 2024-09-11T23:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff3289ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5595b7025290 con 0x5595b71382d0 2024-09-11T23:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff3289ec740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5595b7149e10 con 0x5595b71379b0 2024-09-11T23:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff3289ec740 10 client.0 _ll_register_callbacks cb 0x5595b7056140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff31ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b7144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff31ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b7144ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34548/0 (socket says 172.21.15.133:34548) 2024-09-11T23:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff31ffff640 1 -- 172.21.15.133:0/4093329633 learned_addr learned my addr 172.21.15.133:0/4093329633 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff317fff640 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71382d0 0x5595b71445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff31f7fe640 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71379b0 0x5595b7144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff31ffff640 1 -- 172.21.15.133:0/4093329633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71379b0 msgr2=0x5595b7144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff31ffff640 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71379b0 0x5595b7144060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff31ffff640 1 -- 172.21.15.133:0/4093329633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71382d0 msgr2=0x5595b71445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff31ffff640 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71382d0 0x5595b71445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff31ffff640 1 -- 172.21.15.133:0/4093329633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b6daf210 con 0x5595b71470c0 2024-09-11T23:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff31ffff640 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b7144ae0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff31802fb50 tx=0x7ff3180095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66677aa08de539c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.489+0000 7ff2feffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff2feffd640 1 -- 172.21.15.133:0/4093329633 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318055060 con 0x5595b71470c0 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff2feffd640 1 -- 172.21.15.133:0/4093329633 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff318003d80 con 0x5595b71470c0 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff3289ec740 1 -- 172.21.15.133:0/4093329633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff310015070 con 0x5595b71470c0 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff2feffd640 1 -- 172.21.15.133:0/4093329633 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff318034040 con 0x5595b71470c0 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff3289ec740 10 client.16365 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff3289ec740 1 -- 172.21.15.133:0/4093329633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595b6e7b620 con 0x5595b71470c0 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff2feffd640 1 -- 172.21.15.133:0/4093329633 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff310015070 con 0x5595b71470c0 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff2feffd640 1 -- 172.21.15.133:0/4093329633 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff318058020 con 0x5595b71470c0 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff3289ec740 10 client.16365 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff3289ec740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff3289ec740 20 client.16365 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff3289ec740 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5595b7157080 0x5595b7159490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff3289ec740 4 client.16365 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.490+0000 7ff3289ec740 1 -- 172.21.15.133:0/4093329633 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595b70a9a20 con 0x5595b7157080 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.491+0000 7ff31f7fe640 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5595b7157080 0x5595b7159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.491+0000 7ff31f7fe640 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5595b7157080 0x5595b7159490 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f16aa13659554706 server_cookie=8052a5d962c44ae5 in_seq=0 out_seq=0 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.491+0000 7ff2feffd640 10 client.16365 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff2feffd640 1 -- 172.21.15.133:0/4093329633 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595b70a9a20 con 0x5595b7157080 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff2feffd640 10 client.16365 handle_command_reply: tid=0 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 1 client.16365 shutdown 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 1 -- 172.21.15.133:0/4093329633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 msgr2=0x5595b7144ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b7144ae0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff31802fb50 tx=0x7ff3180095c0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 1 -- 172.21.15.133:0/4093329633 shutdown_connections 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5595b7157080 0x5595b7159490 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b7144ae0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71382d0 0x5595b71445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff2feffd640 0 client.16365 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 1 --2- 172.21.15.133:0/4093329633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71379b0 0x5595b7144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 1 -- 172.21.15.133:0/4093329633 >> 172.21.15.133:0/4093329633 conn(0x5595b7134060 msgr2=0x5595b7139d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 1 -- 172.21.15.133:0/4093329633 shutdown_connections 2024-09-11T23:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 1 -- 172.21.15.133:0/4093329633 wait complete. 2024-09-11T23:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.492+0000 7ff3289ec740 20 client.16365 trim_cache size 0 max 16384 2024-09-11T23:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.506+0000 7ff3289ec740 1 Processor -- start 2024-09-11T23:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.506+0000 7ff3289ec740 1 -- start start 2024-09-11T23:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b7159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b715eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71470c0 0x5595b715f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff3289ec740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff3201ab3a0 con 0x5595b71382d0 2024-09-11T23:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff3289ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3201acee0 con 0x5595b71379b0 2024-09-11T23:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff3289ec740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3201aba70 con 0x5595b71470c0 2024-09-11T23:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff317fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b715eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff317fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b715eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34562/0 (socket says 172.21.15.133:34562) 2024-09-11T23:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff317fff640 1 -- 172.21.15.133:0/1416931589 learned_addr learned my addr 172.21.15.133:0/1416931589 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff317fff640 1 -- 172.21.15.133:0/1416931589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71470c0 msgr2=0x5595b715f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff317fff640 1 --2- 172.21.15.133:0/1416931589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71470c0 0x5595b715f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff317fff640 1 -- 172.21.15.133:0/1416931589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 msgr2=0x5595b7159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff317fff640 1 --2- 172.21.15.133:0/1416931589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b7159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff317fff640 1 -- 172.21.15.133:0/1416931589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b7142ff0 con 0x5595b71382d0 2024-09-11T23:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff317fff640 1 --2- 172.21.15.133:0/1416931589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b715eda0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff30809b9b0 tx=0x7ff3080cf5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf038b8b67a818b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff3157fa640 1 -- 172.21.15.133:0/1416931589 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3080a4870 con 0x5595b71382d0 2024-09-11T23:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff3157fa640 1 -- 172.21.15.133:0/1416931589 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff308099fb0 con 0x5595b71382d0 2024-09-11T23:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.507+0000 7ff3157fa640 1 -- 172.21.15.133:0/1416931589 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308018b30 con 0x5595b71382d0 2024-09-11T23:39:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.508+0000 7ff3289ec740 1 -- 172.21.15.133:0/1416931589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 msgr2=0x5595b715eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.508+0000 7ff3289ec740 1 --2- 172.21.15.133:0/1416931589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b715eda0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff30809b9b0 tx=0x7ff3080cf5a0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.508+0000 7ff3289ec740 1 -- 172.21.15.133:0/1416931589 shutdown_connections 2024-09-11T23:39:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.508+0000 7ff3289ec740 1 --2- 172.21.15.133:0/1416931589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71470c0 0x5595b715f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.508+0000 7ff3289ec740 1 --2- 172.21.15.133:0/1416931589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71382d0 0x5595b715eda0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.508+0000 7ff3289ec740 1 --2- 172.21.15.133:0/1416931589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b7159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.508+0000 7ff3289ec740 1 -- 172.21.15.133:0/1416931589 >> 172.21.15.133:0/1416931589 conn(0x5595b7134060 msgr2=0x5595b7144840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.508+0000 7ff3289ec740 1 -- 172.21.15.133:0/1416931589 shutdown_connections 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.508+0000 7ff3289ec740 1 -- 172.21.15.133:0/1416931589 wait complete. 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.508+0000 7ff3289ec740 1 -- start start 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff3289ec740 1 Processor -- start 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b714f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71382d0 0x5595b714f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff3289ec740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b714c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff3289ec740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff3201ac810 con 0x5595b71470c0 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff3289ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3201ac140 con 0x5595b71379b0 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff3289ec740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3201a93a0 con 0x5595b71382d0 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff3289ec740 10 client.0 _ll_register_callbacks cb 0x7ff318009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:39:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff31ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b714c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff31ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b714c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34576/0 (socket says 172.21.15.133:34576) 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff31ffff640 1 -- 172.21.15.133:0/605711990 learned_addr learned my addr 172.21.15.133:0/605711990 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff317fff640 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71382d0 0x5595b714f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff31f7fe640 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b714f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff31ffff640 1 -- 172.21.15.133:0/605711990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71382d0 msgr2=0x5595b714f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff31ffff640 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71382d0 0x5595b714f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff31ffff640 1 -- 172.21.15.133:0/605711990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 msgr2=0x5595b714f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff31ffff640 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b714f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff31ffff640 1 -- 172.21.15.133:0/605711990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b714c9e0 con 0x5595b71470c0 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff31ffff640 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b714c1b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff318009bf0 tx=0x7ff318009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3488b4e5a4167406 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.509+0000 7ff2feffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff2feffd640 1 -- 172.21.15.133:0/605711990 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318055040 con 0x5595b71470c0 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff2feffd640 1 -- 172.21.15.133:0/605711990 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff31805a650 con 0x5595b71470c0 2024-09-11T23:39:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff3289ec740 1 -- 172.21.15.133:0/605711990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff308099fb0 con 0x5595b71470c0 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff2feffd640 1 -- 172.21.15.133:0/605711990 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff31806a050 con 0x5595b71470c0 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff3289ec740 10 client.16371 fetch_fsmap learned FSMap version 28 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff3289ec740 1 -- 172.21.15.133:0/605711990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595b71c29b0 con 0x5595b71470c0 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff2feffd640 1 -- 172.21.15.133:0/605711990 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff308099fb0 con 0x5595b71470c0 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff2feffd640 1 -- 172.21.15.133:0/605711990 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff318062020 con 0x5595b71470c0 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff3289ec740 10 client.16371 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff3289ec740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff3289ec740 20 client.16371 populate_metadata read hostname 'smithi133' 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff3289ec740 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5595b71c3090 0x5595b71c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff3289ec740 4 client.16371 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff3289ec740 1 -- 172.21.15.133:0/605711990 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595b70a3d70 con 0x5595b71c3090 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.510+0000 7ff31f7fe640 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5595b71c3090 0x5595b71c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:39:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.511+0000 7ff31f7fe640 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5595b71c3090 0x5595b71c54b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4005613b42509452 server_cookie=2f145dbf07dc89cd in_seq=0 out_seq=0 2024-09-11T23:39:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.511+0000 7ff2feffd640 10 client.16371 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff2feffd640 1 -- 172.21.15.133:0/605711990 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595b70a3d70 con 0x5595b71c3090 2024-09-11T23:39:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff2feffd640 10 client.16371 handle_command_reply: tid=0 2024-09-11T23:39:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 1 client.16371 shutdown 2024-09-11T23:39:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 1 -- 172.21.15.133:0/605711990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 msgr2=0x5595b714c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b714c1b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff318009bf0 tx=0x7ff318009a20 comp rx=0 tx=0).stop 2024-09-11T23:39:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 1 -- 172.21.15.133:0/605711990 shutdown_connections 2024-09-11T23:39:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5595b71c3090 0x5595b71c54b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b71470c0 0x5595b714c1b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5595b71382d0 0x5595b714f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff2feffd640 0 client.16371 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:39:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 1 --2- 172.21.15.133:0/605711990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595b71379b0 0x5595b714f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 1 -- 172.21.15.133:0/605711990 >> 172.21.15.133:0/605711990 conn(0x5595b7134060 msgr2=0x5595b71329a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 1 -- 172.21.15.133:0/605711990 shutdown_connections 2024-09-11T23:39:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 1 -- 172.21.15.133:0/605711990 wait complete. 2024-09-11T23:39:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.520+0000 7ff3289ec740 20 client.16371 trim_cache size 0 max 16384 2024-09-11T23:39:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:39:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:39:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:39:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:39:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 -- 172.21.15.133:0/2154364616 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff30c0a2da0 msgr2=0x7ff30c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff30c0a2da0 0x7ff30c0a5260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff318005e30 tx=0x7ff318009800 comp rx=0 tx=0).stop 2024-09-11T23:39:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 -- 172.21.15.133:0/2154364616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff320103660 msgr2=0x7ff320198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff320103660 0x7ff320198840 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e9f0 tx=0x7ff30800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 -- 172.21.15.133:0/2154364616 shutdown_connections 2024-09-11T23:39:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff30c0a2da0 0x7ff30c0a5260 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32010c110 0x7ff32019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff320103660 0x7ff320198840 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 --2- 172.21.15.133:0/2154364616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff320102d40 0x7ff320198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 -- 172.21.15.133:0/2154364616 >> 172.21.15.133:0/2154364616 conn(0x7ff320077990 msgr2=0x7ff3200fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 -- 172.21.15.133:0/2154364616 shutdown_connections 2024-09-11T23:39:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:39:59.521+0000 7ff325c53640 1 -- 172.21.15.133:0/2154364616 wait complete. 2024-09-11T23:39:59.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.568+0000 7f365170e640 1 -- 172.21.15.133:0/2370268082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f364c10a120 con 0x7f364c100ca0 2024-09-11T23:39:59.569 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:39:59 smithi133 ceph-mon[31430]: pgmap v99: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1012 GiB / 1.0 TiB avail; 10 KiB/s rd, 91 MiB/s wr, 68 op/s 2024-09-11T23:39:59.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.569+0000 7f3643fff640 1 -- 172.21.15.133:0/2370268082 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f363c0bc3d0 con 0x7f364c100ca0 2024-09-11T23:39:59.569 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:39:59.569 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:39:59.570 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:39:59.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.571+0000 7f365170e640 1 -- 172.21.15.133:0/2370268082 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f36380a2cf0 msgr2=0x7f36380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.571+0000 7f365170e640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f36380a2cf0 0x7f36380a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3634005e10 tx=0x7f3634005da0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.571+0000 7f365170e640 1 -- 172.21.15.133:0/2370268082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f364c100ca0 msgr2=0x7f364c1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:39:59.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.571+0000 7f365170e640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f364c100ca0 0x7f364c1017b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f363c009230 tx=0x7f363c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.571+0000 7f365170e640 1 -- 172.21.15.133:0/2370268082 shutdown_connections 2024-09-11T23:39:59.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.571+0000 7f365170e640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f36380a2cf0 0x7f36380a51b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.571+0000 7f365170e640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f364c106f50 0x7f364c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.571+0000 7f365170e640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f364c100ca0 0x7f364c1017b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.571+0000 7f365170e640 1 --2- 172.21.15.133:0/2370268082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f364c1002f0 0x7f364c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:39:59.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.571+0000 7f365170e640 1 -- 172.21.15.133:0/2370268082 >> 172.21.15.133:0/2370268082 conn(0x7f364c077210 msgr2=0x7f364c105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:39:59.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.571+0000 7f365170e640 1 -- 172.21.15.133:0/2370268082 shutdown_connections 2024-09-11T23:39:59.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:39:59.572+0000 7f365170e640 1 -- 172.21.15.133:0/2370268082 wait complete. 2024-09-11T23:39:59.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:39:59 smithi138 ceph-mon[35142]: pgmap v99: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1012 GiB / 1.0 TiB avail; 10 KiB/s rd, 91 MiB/s wr, 68 op/s 2024-09-11T23:39:59.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:39:59 smithi178 ceph-mon[35882]: pgmap v99: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1012 GiB / 1.0 TiB avail; 10 KiB/s rd, 91 MiB/s wr, 68 op/s 2024-09-11T23:40:00.295 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id a1b0efb3 --release --await' 2024-09-11T23:40:00.305 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id a1b0efb3 --release --await 2024-09-11T23:40:00.613 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:00.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:00 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2370268082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:00.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:00 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:40:00.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:00 smithi138 ceph-mon[35142]: overall HEALTH_OK 2024-09-11T23:40:00.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:00 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2370268082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:00.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:00 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:40:00.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:00 smithi133 ceph-mon[31430]: overall HEALTH_OK 2024-09-11T23:40:00.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:00 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2370268082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:00.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:00 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:40:00.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:00 smithi178 ceph-mon[35882]: overall HEALTH_OK 2024-09-11T23:40:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.083+0000 7f6043ae5640 1 -- 172.21.15.133:0/3419146191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c102790 msgr2=0x7f603c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.083+0000 7f6043ae5640 1 --2- 172.21.15.133:0/3419146191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c102790 0x7f603c1067e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f602c009900 tx=0x7f602c02a410 comp rx=0 tx=0).stop 2024-09-11T23:40:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.083+0000 7f6043ae5640 1 -- 172.21.15.133:0/3419146191 shutdown_connections 2024-09-11T23:40:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.083+0000 7f6043ae5640 1 --2- 172.21.15.133:0/3419146191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f603c106f10 0x7f603c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.083+0000 7f6043ae5640 1 --2- 172.21.15.133:0/3419146191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c102790 0x7f603c1067e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.083+0000 7f6043ae5640 1 --2- 172.21.15.133:0/3419146191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c101de0 0x7f603c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.083+0000 7f6043ae5640 1 -- 172.21.15.133:0/3419146191 >> 172.21.15.133:0/3419146191 conn(0x7f603c0fd8c0 msgr2=0x7f603c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:01.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.084+0000 7f6043ae5640 1 -- 172.21.15.133:0/3419146191 shutdown_connections 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.084+0000 7f6043ae5640 1 -- 172.21.15.133:0/3419146191 wait complete. 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.084+0000 7f6043ae5640 1 Processor -- start 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.084+0000 7f6043ae5640 1 -- start start 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.084+0000 7f6043ae5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c101de0 0x7f603c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.084+0000 7f6043ae5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c102790 0x7f603c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.084+0000 7f6043ae5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f603c106f10 0x7f603c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.084+0000 7f6043ae5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f603c111f70 con 0x7f603c102790 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.084+0000 7f6043ae5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f603c111df0 con 0x7f603c101de0 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.084+0000 7f6043ae5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1120f0 con 0x7f603c106f10 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041881640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c101de0 0x7f603c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041080640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c102790 0x7f603c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041080640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c102790 0x7f603c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34594/0 (socket says 172.21.15.133:34594) 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041881640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c101de0 0x7f603c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:46212/0 (socket says 172.21.15.133:46212) 2024-09-11T23:40:01.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041080640 1 -- 172.21.15.133:0/2084813796 learned_addr learned my addr 172.21.15.133:0/2084813796 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6042082640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f603c106f10 0x7f603c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041080640 1 -- 172.21.15.133:0/2084813796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f603c106f10 msgr2=0x7f603c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041080640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f603c106f10 0x7f603c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041080640 1 -- 172.21.15.133:0/2084813796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c101de0 msgr2=0x7f603c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041080640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c101de0 0x7f603c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041080640 1 -- 172.21.15.133:0/2084813796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c1a68d0 con 0x7f603c102790 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041881640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c101de0 0x7f603c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6041080640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c102790 0x7f603c074a30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f602c006010 tx=0x7f602c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6032ffd640 1 -- 172.21.15.133:0/2084813796 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c03f030 con 0x7f603c102790 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6032ffd640 1 -- 172.21.15.133:0/2084813796 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602c003ce0 con 0x7f603c102790 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6032ffd640 1 -- 172.21.15.133:0/2084813796 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c035070 con 0x7f603c102790 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6043ae5640 1 -- 172.21.15.133:0/2084813796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c1a6b60 con 0x7f603c102790 2024-09-11T23:40:01.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.085+0000 7f6043ae5640 1 -- 172.21.15.133:0/2084813796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c1a7040 con 0x7f603c102790 2024-09-11T23:40:01.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.087+0000 7f6032ffd640 1 -- 172.21.15.133:0/2084813796 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f602c042030 con 0x7f603c102790 2024-09-11T23:40:01.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.088+0000 7f6045ebe740 1 Processor -- start 2024-09-11T23:40:01.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.088+0000 7f6045ebe740 1 -- start start 2024-09-11T23:40:01.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.088+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64060 0x5622acf64440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.088+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf6ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.088+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf6f240 0x5622acf71630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.088+0000 7f6045ebe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5622acec9620 con 0x5622acf64980 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.088+0000 7f6045ebe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622ace70e50 con 0x5622acf64060 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.088+0000 7f6045ebe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5622acec55e0 con 0x5622acf6f240 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.088+0000 7f6041881640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf6ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6041881640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf6ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34610/0 (socket says 172.21.15.133:34610) 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6041881640 1 -- 172.21.15.133:0/4260738529 learned_addr learned my addr 172.21.15.133:0/4260738529 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6032ffd640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f60280a2cf0 0x7f60280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6042082640 1 --2- 172.21.15.133:0/4260738529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64060 0x5622acf64440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6041080640 1 --2- 172.21.15.133:0/4260738529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf6f240 0x5622acf71630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6042082640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f60280a2cf0 0x7f60280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6042082640 1 -- 172.21.15.133:0/4260738529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf6f240 msgr2=0x5622acf71630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6042082640 1 --2- 172.21.15.133:0/4260738529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf6f240 0x5622acf71630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6042082640 1 -- 172.21.15.133:0/4260738529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 msgr2=0x5622acf6ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6042082640 1 --2- 172.21.15.133:0/4260738529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf6ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6042082640 1 -- 172.21.15.133:0/4260738529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ace61d40 con 0x5622acf64060 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6032ffd640 1 -- 172.21.15.133:0/2084813796 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f602c0ef9a0 con 0x7f603c102790 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.089+0000 7f6042082640 1 --2- 172.21.15.133:0/4260738529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64060 0x5622acf64440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6034014510 tx=0x7f6034009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3599fb814da584 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.090+0000 7f6027fff640 1 -- 172.21.15.133:0/4260738529 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034012bb0 con 0x5622acf64060 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.090+0000 7f6027fff640 1 -- 172.21.15.133:0/4260738529 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6034012d50 con 0x5622acf64060 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.090+0000 7f6027fff640 1 -- 172.21.15.133:0/4260738529 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603400c8d0 con 0x5622acf64060 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.090+0000 7f6045ebe740 1 -- 172.21.15.133:0/4260738529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64060 msgr2=0x5622acf64440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.090+0000 7f6042082640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f60280a2cf0 0x7f60280a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f603401e000 tx=0x7f603400cf00 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.090+0000 7f6045ebe740 1 --2- 172.21.15.133:0/4260738529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64060 0x5622acf64440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6034014510 tx=0x7f6034009f90 comp rx=0 tx=0).stop 2024-09-11T23:40:01.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.090+0000 7f6045ebe740 1 -- 172.21.15.133:0/4260738529 shutdown_connections 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.090+0000 7f6045ebe740 1 --2- 172.21.15.133:0/4260738529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf6f240 0x5622acf71630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.090+0000 7f6045ebe740 1 --2- 172.21.15.133:0/4260738529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf6ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.090+0000 7f6045ebe740 1 --2- 172.21.15.133:0/4260738529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64060 0x5622acf64440 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.090+0000 7f6045ebe740 1 -- 172.21.15.133:0/4260738529 >> 172.21.15.133:0/4260738529 conn(0x5622acf605f0 msgr2=0x5622acf609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.091+0000 7f6045ebe740 1 -- 172.21.15.133:0/4260738529 shutdown_connections 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.091+0000 7f6045ebe740 1 -- 172.21.15.133:0/4260738529 wait complete. 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.091+0000 7f6045ebe740 1 -- start start 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.091+0000 7f6045ebe740 1 Processor -- start 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.091+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64060 0x5622acf6c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.091+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf6c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.091+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf6f240 0x5622acf6cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.091+0000 7f6045ebe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5622acf71e10 con 0x5622acf64980 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.091+0000 7f6045ebe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622ace51f10 con 0x5622acf64060 2024-09-11T23:40:01.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.091+0000 7f6045ebe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5622acf71f90 con 0x5622acf6f240 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.091+0000 7f6045ebe740 10 client.0 _ll_register_callbacks cb 0x5622ace75db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6041080640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf6c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6041080640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf6c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34614/0 (socket says 172.21.15.133:34614) 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6041080640 1 -- 172.21.15.133:0/2086099201 learned_addr learned my addr 172.21.15.133:0/2086099201 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6041881640 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64060 0x5622acf6c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6042082640 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf6f240 0x5622acf6cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6041080640 1 -- 172.21.15.133:0/2086099201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf6f240 msgr2=0x5622acf6cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6041080640 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf6f240 0x5622acf6cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6041080640 1 -- 172.21.15.133:0/2086099201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64060 msgr2=0x5622acf6c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6041080640 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64060 0x5622acf6c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6041080640 1 -- 172.21.15.133:0/2086099201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622acbd8580 con 0x5622acf64980 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6041080640 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf6c600 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f602c000c00 tx=0x7f602c08af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88a8edebf2c33f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f60257fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f60257fa640 1 -- 172.21.15.133:0/2086099201 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c049070 con 0x5622acf64980 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f60257fa640 1 -- 172.21.15.133:0/2086099201 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f602c0bc1f0 con 0x5622acf64980 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6045ebe740 1 -- 172.21.15.133:0/2086099201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6034012d50 con 0x5622acf64980 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f60257fa640 1 -- 172.21.15.133:0/2086099201 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f602c0bbad0 con 0x5622acf64980 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6045ebe740 10 client.16383 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.092+0000 7f6045ebe740 1 -- 172.21.15.133:0/2086099201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622acca2960 con 0x5622acf64980 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.093+0000 7f60257fa640 1 -- 172.21.15.133:0/2086099201 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6034012d50 con 0x5622acf64980 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.093+0000 7f60257fa640 1 -- 172.21.15.133:0/2086099201 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f602c0c4770 con 0x5622acf64980 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.093+0000 7f6045ebe740 10 client.16383 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.093+0000 7f6045ebe740 10 client.16383 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.093+0000 7f6045ebe740 20 client.16383 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.093+0000 7f6045ebe740 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5622acf7ebd0 0x5622acf81020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.093+0000 7f6045ebe740 4 client.16383 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.093+0000 7f6045ebe740 1 -- 172.21.15.133:0/2086099201 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622aced6b90 con 0x5622acf7ebd0 2024-09-11T23:40:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.093+0000 7f6041881640 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5622acf7ebd0 0x5622acf81020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.093+0000 7f60257fa640 10 client.16383 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.093+0000 7f6041881640 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5622acf7ebd0 0x5622acf81020 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=104d10d8a5e5fa74 server_cookie=7f134cbd9bdb3384 in_seq=0 out_seq=0 2024-09-11T23:40:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f60257fa640 1 -- 172.21.15.133:0/2086099201 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622aced6b90 con 0x5622acf7ebd0 2024-09-11T23:40:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f60257fa640 10 client.16383 handle_command_reply: tid=0 2024-09-11T23:40:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 1 client.16383 shutdown 2024-09-11T23:40:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 1 -- 172.21.15.133:0/2086099201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 msgr2=0x5622acf6c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf6c600 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f602c000c00 tx=0x7f602c08af80 comp rx=0 tx=0).stop 2024-09-11T23:40:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 1 -- 172.21.15.133:0/2086099201 shutdown_connections 2024-09-11T23:40:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5622acf7ebd0 0x5622acf81020 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf6f240 0x5622acf6cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf6c600 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 1 --2- 172.21.15.133:0/2086099201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64060 0x5622acf6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 1 -- 172.21.15.133:0/2086099201 >> 172.21.15.133:0/2086099201 conn(0x5622acf605f0 msgr2=0x5622acf5f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f60257fa640 0 client.16383 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 1 -- 172.21.15.133:0/2086099201 shutdown_connections 2024-09-11T23:40:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 1 -- 172.21.15.133:0/2086099201 wait complete. 2024-09-11T23:40:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.094+0000 7f6045ebe740 20 client.16383 trim_cache size 0 max 16384 2024-09-11T23:40:01.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6045ebe740 1 Processor -- start 2024-09-11T23:40:01.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6045ebe740 1 -- start start 2024-09-11T23:40:01.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 0x5622acf86930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf86e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf6f240 0x5622acf69320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6045ebe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1af960 con 0x5622acf64980 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6045ebe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1b14a0 con 0x5622acf6f240 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6045ebe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1b0030 con 0x5622acf64060 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6041080640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf86e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6041080640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf86e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34630/0 (socket says 172.21.15.133:34630) 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6041080640 1 -- 172.21.15.133:0/1268621334 learned_addr learned my addr 172.21.15.133:0/1268621334 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.111+0000 7f6041881640 1 --2- 172.21.15.133:0/1268621334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 0x5622acf86930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.112+0000 7f6041080640 1 -- 172.21.15.133:0/1268621334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 msgr2=0x5622acf86930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.112+0000 7f6041080640 1 --2- 172.21.15.133:0/1268621334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 0x5622acf86930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.112+0000 7f6041080640 1 -- 172.21.15.133:0/1268621334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf6f240 msgr2=0x5622acf69320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:01.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.112+0000 7f6041080640 1 --2- 172.21.15.133:0/1268621334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf6f240 0x5622acf69320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.112+0000 7f6041080640 1 -- 172.21.15.133:0/1268621334 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622acf60090 con 0x5622acf64980 2024-09-11T23:40:01.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.112+0000 7f6041881640 1 --2- 172.21.15.133:0/1268621334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 0x5622acf86930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:40:01.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.112+0000 7f6041080640 1 --2- 172.21.15.133:0/1268621334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf86e70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f602c005940 tx=0x7f602c03b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa720836bf3751a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:01.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.112+0000 7f60277fe640 1 -- 172.21.15.133:0/1268621334 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c049030 con 0x5622acf64980 2024-09-11T23:40:01.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.112+0000 7f60277fe640 1 -- 172.21.15.133:0/1268621334 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602c0c0340 con 0x5622acf64980 2024-09-11T23:40:01.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.112+0000 7f60277fe640 1 -- 172.21.15.133:0/1268621334 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0bb8f0 con 0x5622acf64980 2024-09-11T23:40:01.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 -- 172.21.15.133:0/1268621334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 msgr2=0x5622acf86e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:01.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 --2- 172.21.15.133:0/1268621334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf86e70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f602c005940 tx=0x7f602c03b5d0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 -- 172.21.15.133:0/1268621334 shutdown_connections 2024-09-11T23:40:01.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 --2- 172.21.15.133:0/1268621334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf6f240 0x5622acf69320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 --2- 172.21.15.133:0/1268621334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf64980 0x5622acf86e70 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 --2- 172.21.15.133:0/1268621334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 0x5622acf86930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 -- 172.21.15.133:0/1268621334 >> 172.21.15.133:0/1268621334 conn(0x5622acf605f0 msgr2=0x5622acf6a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:01.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 -- 172.21.15.133:0/1268621334 shutdown_connections 2024-09-11T23:40:01.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 -- 172.21.15.133:0/1268621334 wait complete. 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 -- start start 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 Processor -- start 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 0x5622acf73290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.113+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64980 0x5622acf737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6045ebe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf6f240 0x5622acfe8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6045ebe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1b0dd0 con 0x5622acf6f240 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6045ebe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1b0700 con 0x5622acf64980 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6045ebe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1ad940 con 0x5622acf64060 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6045ebe740 10 client.0 _ll_register_callbacks cb 0x7f602c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6042082640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf6f240 0x5622acfe8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6042082640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf6f240 0x5622acfe8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34642/0 (socket says 172.21.15.133:34642) 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6042082640 1 -- 172.21.15.133:0/519880782 learned_addr learned my addr 172.21.15.133:0/519880782 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6041881640 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 0x5622acf73290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6041080640 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64980 0x5622acf737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6042082640 1 -- 172.21.15.133:0/519880782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 msgr2=0x5622acf73290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6042082640 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 0x5622acf73290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6042082640 1 -- 172.21.15.133:0/519880782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64980 msgr2=0x5622acf737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6042082640 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64980 0x5622acf737d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6042082640 1 -- 172.21.15.133:0/519880782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622acf74080 con 0x5622acf6f240 2024-09-11T23:40:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6041881640 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 0x5622acf73290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:40:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6042082640 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf6f240 0x5622acfe8900 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f603401f040 tx=0x7f60340095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1017b6a4c979cfb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f60257fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:40:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f60257fa640 1 -- 172.21.15.133:0/519880782 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603402c050 con 0x5622acf6f240 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f60257fa640 1 -- 172.21.15.133:0/519880782 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6034004a00 con 0x5622acf6f240 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.114+0000 7f6045ebe740 1 -- 172.21.15.133:0/519880782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602c0c0340 con 0x5622acf6f240 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f60257fa640 1 -- 172.21.15.133:0/519880782 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6034014370 con 0x5622acf6f240 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f6045ebe740 10 client.16389 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f6045ebe740 1 -- 172.21.15.133:0/519880782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622acfe9f60 con 0x5622acf6f240 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f60257fa640 1 -- 172.21.15.133:0/519880782 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602c0c0340 con 0x5622acf6f240 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f60257fa640 1 -- 172.21.15.133:0/519880782 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6034039050 con 0x5622acf6f240 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f6045ebe740 10 client.16389 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f6045ebe740 10 client.16389 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f6045ebe740 20 client.16389 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f6045ebe740 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5622acfea570 0x5622acfec990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f6045ebe740 4 client.16389 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a1b0efb3", "release": true, "await": true}] 2024-09-11T23:40:01.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f6045ebe740 1 -- 172.21.15.133:0/519880782 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a1b0efb3", "release": true, "await": true}) -- 0x5622aced10a0 con 0x5622acfea570 2024-09-11T23:40:01.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.115+0000 7f6041881640 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5622acfea570 0x5622acfec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:01.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.116+0000 7f60257fa640 10 client.16389 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:01.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.116+0000 7f6041881640 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5622acfea570 0x5622acfec990 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e53c4abe9db0d3d7 server_cookie=1c8db7edc61653bd in_seq=0 out_seq=0 2024-09-11T23:40:01.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.119+0000 7f60257fa640 1 -- 172.21.15.133:0/519880782 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5622aced10a0 con 0x5622acfea570 2024-09-11T23:40:01.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.119+0000 7f60257fa640 10 client.16389 handle_command_reply: tid=0 2024-09-11T23:40:01.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.119+0000 7f6045ebe740 1 client.16389 shutdown 2024-09-11T23:40:01.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f6045ebe740 1 -- 172.21.15.133:0/519880782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf6f240 msgr2=0x5622acfe8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:01.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f6045ebe740 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf6f240 0x5622acfe8900 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f603401f040 tx=0x7f60340095d0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f6045ebe740 1 -- 172.21.15.133:0/519880782 shutdown_connections 2024-09-11T23:40:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f6045ebe740 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5622acfea570 0x5622acfec990 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f6045ebe740 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622acf6f240 0x5622acfe8900 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f6045ebe740 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622acf64980 0x5622acf737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f60257fa640 0 client.16389 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f6045ebe740 1 --2- 172.21.15.133:0/519880782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5622acf64060 0x5622acf73290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f6045ebe740 1 -- 172.21.15.133:0/519880782 >> 172.21.15.133:0/519880782 conn(0x5622acf605f0 msgr2=0x5622acf65b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f6045ebe740 1 -- 172.21.15.133:0/519880782 shutdown_connections 2024-09-11T23:40:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f6045ebe740 1 -- 172.21.15.133:0/519880782 wait complete. 2024-09-11T23:40:01.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.120+0000 7f6045ebe740 20 client.16389 trim_cache size 0 max 16384 2024-09-11T23:40:01.124 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:40:01.124 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 28, 2024-09-11T23:40:01.124 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:40:01.124 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 12, 2024-09-11T23:40:01.125 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:40:01.125 INFO:teuthology.orchestra.run.smithi133.stdout: "a1b0efb3": { 2024-09-11T23:40:01.125 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 12, 2024-09-11T23:40:01.125 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:40:01.125 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:40:01.125 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:40:01.125 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:40:01.125 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:40:01.125 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:40:01.125 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 144.7, 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:40:01.126 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:40:01.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 -- 172.21.15.133:0/2084813796 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f60280a2cf0 msgr2=0x7f60280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:01.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f60280a2cf0 0x7f60280a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f603401e000 tx=0x7f603400cf00 comp rx=0 tx=0).stop 2024-09-11T23:40:01.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 -- 172.21.15.133:0/2084813796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c102790 msgr2=0x7f603c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:01.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c102790 0x7f603c074a30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f602c006010 tx=0x7f602c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 -- 172.21.15.133:0/2084813796 shutdown_connections 2024-09-11T23:40:01.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f60280a2cf0 0x7f60280a51b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f603c106f10 0x7f603c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c102790 0x7f603c074a30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 --2- 172.21.15.133:0/2084813796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c101de0 0x7f603c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:01.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 -- 172.21.15.133:0/2084813796 >> 172.21.15.133:0/2084813796 conn(0x7f603c0fd8c0 msgr2=0x7f603c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:01.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 -- 172.21.15.133:0/2084813796 shutdown_connections 2024-09-11T23:40:01.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:01.121+0000 7f6043ae5640 1 -- 172.21.15.133:0/2084813796 wait complete. 2024-09-11T23:40:01.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:01 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:40:01.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:01 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:40:01.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:01 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:40:01.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:01 smithi138 ceph-mon[35142]: pgmap v100: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1012 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 66 MiB/s wr, 46 op/s 2024-09-11T23:40:01.712 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:01 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:40:01.712 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:01 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:40:01.712 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:01 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:40:01.713 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:01 smithi133 ceph-mon[31430]: pgmap v100: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1012 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 66 MiB/s wr, 46 op/s 2024-09-11T23:40:01.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:01 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:40:01.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:01 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:40:01.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:01 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:40:01.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:01 smithi178 ceph-mon[35882]: pgmap v100: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1012 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 66 MiB/s wr, 46 op/s 2024-09-11T23:40:02.469 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a1b0efb3', total seconds elapsed: 28.6 2024-09-11T23:40:03.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:03 smithi138 ceph-mon[35142]: pgmap v101: 97 pgs: 97 active+clean; 20 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 8.3 KiB/s rd, 140 MiB/s wr, 790 op/s 2024-09-11T23:40:03.712 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:03 smithi133 ceph-mon[31430]: pgmap v101: 97 pgs: 97 active+clean; 20 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 8.3 KiB/s rd, 140 MiB/s wr, 790 op/s 2024-09-11T23:40:03.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:03 smithi178 ceph-mon[35882]: pgmap v101: 97 pgs: 97 active+clean; 20 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 8.3 KiB/s rd, 140 MiB/s wr, 790 op/s 2024-09-11T23:40:05.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:05 smithi178 ceph-mon[35882]: pgmap v102: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 16 KiB/s rd, 188 MiB/s wr, 1.31k op/s 2024-09-11T23:40:05.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:05 smithi138 ceph-mon[35142]: pgmap v102: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 16 KiB/s rd, 188 MiB/s wr, 1.31k op/s 2024-09-11T23:40:05.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:05 smithi133 ceph-mon[31430]: pgmap v102: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 16 KiB/s rd, 188 MiB/s wr, 1.31k op/s 2024-09-11T23:40:05.733 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:40:06.579 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:07.600 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:07 smithi133 ceph-mon[31430]: pgmap v103: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 176 MiB/s wr, 1.30k op/s 2024-09-11T23:40:07.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:07 smithi178 ceph-mon[35882]: pgmap v103: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 176 MiB/s wr, 1.30k op/s 2024-09-11T23:40:07.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.823+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1644642893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c110220 msgr2=0x7fa79c06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.823+0000 7fa7a2ee1640 1 --2- 172.21.15.133:0/1644642893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c110220 0x7fa79c06e4f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa790009900 tx=0x7fa79002a440 comp rx=0 tx=0).stop 2024-09-11T23:40:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.824+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1644642893 shutdown_connections 2024-09-11T23:40:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.824+0000 7fa7a2ee1640 1 --2- 172.21.15.133:0/1644642893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa79c11b140 0x7fa79c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.824+0000 7fa7a2ee1640 1 --2- 172.21.15.133:0/1644642893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa79c06ea30 0x7fa79c06ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.824+0000 7fa7a2ee1640 1 --2- 172.21.15.133:0/1644642893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c110220 0x7fa79c06e4f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.824+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1644642893 >> 172.21.15.133:0/1644642893 conn(0x7fa79c073210 msgr2=0x7fa79c073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.826+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1644642893 shutdown_connections 2024-09-11T23:40:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.826+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1644642893 wait complete. 2024-09-11T23:40:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.827+0000 7fa7a2ee1640 1 Processor -- start 2024-09-11T23:40:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.827+0000 7fa7a2ee1640 1 -- start start 2024-09-11T23:40:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.828+0000 7fa7a2ee1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa79c06ea30 0x7fa79c1a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.828+0000 7fa7a2ee1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c110220 0x7fa79c1a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.828+0000 7fa7a2ee1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa79c11b140 0x7fa79c1abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.828+0000 7fa7a2ee1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c120db0 con 0x7fa79c110220 2024-09-11T23:40:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.828+0000 7fa7a2ee1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c120c30 con 0x7fa79c06ea30 2024-09-11T23:40:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.828+0000 7fa7a2ee1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c120f30 con 0x7fa79c11b140 2024-09-11T23:40:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.828+0000 7fa79bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c110220 0x7fa79c1a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.828+0000 7fa79bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c110220 0x7fa79c1a7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52280/0 (socket says 172.21.15.133:52280) 2024-09-11T23:40:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.828+0000 7fa79bfff640 1 -- 172.21.15.133:0/1084443456 learned_addr learned my addr 172.21.15.133:0/1084443456 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:07.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.828+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa79c06ea30 0x7fa79c1a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.828+0000 7fa7a147e640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa79c11b140 0x7fa79c1abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.831+0000 7fa79bfff640 1 -- 172.21.15.133:0/1084443456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa79c11b140 msgr2=0x7fa79c1abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:07.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.831+0000 7fa79bfff640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa79c11b140 0x7fa79c1abac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.831+0000 7fa79bfff640 1 -- 172.21.15.133:0/1084443456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa79c06ea30 msgr2=0x7fa79c1a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:07.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.831+0000 7fa79bfff640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa79c06ea30 0x7fa79c1a71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.831+0000 7fa79bfff640 1 -- 172.21.15.133:0/1084443456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c1ac1a0 con 0x7fa79c110220 2024-09-11T23:40:07.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.835+0000 7fa79bfff640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c110220 0x7fa79c1a7730 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00b450 tx=0x7fa78c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:07.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.836+0000 7fa799ffb640 1 -- 172.21.15.133:0/1084443456 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c015020 con 0x7fa79c110220 2024-09-11T23:40:07.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.836+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1084443456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c1ac490 con 0x7fa79c110220 2024-09-11T23:40:07.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.836+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1084443456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c07a450 con 0x7fa79c110220 2024-09-11T23:40:07.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.841+0000 7fa799ffb640 1 -- 172.21.15.133:0/1084443456 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa78c011d70 con 0x7fa79c110220 2024-09-11T23:40:07.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.841+0000 7fa799ffb640 1 -- 172.21.15.133:0/1084443456 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00f340 con 0x7fa79c110220 2024-09-11T23:40:07.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.846+0000 7fa7a52ba740 1 Processor -- start 2024-09-11T23:40:07.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.847+0000 7fa7a52ba740 1 -- start start 2024-09-11T23:40:07.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.847+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b52da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.847+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b52df90 0x556c5b538310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.847+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b538850 0x556c5b53ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.847+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556c5b4932e0 con 0x556c5b52df90 2024-09-11T23:40:07.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.847+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556c5b43ace0 con 0x556c5b538850 2024-09-11T23:40:07.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.847+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556c5b48f2a0 con 0x556c5b52d670 2024-09-11T23:40:07.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.848+0000 7fa7a147e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b52da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.849+0000 7fa7a147e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b52da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:52452/0 (socket says 172.21.15.133:52452) 2024-09-11T23:40:07.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.849+0000 7fa7a147e640 1 -- 172.21.15.133:0/96273502 learned_addr learned my addr 172.21.15.133:0/96273502 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:07.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.849+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/96273502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b52df90 0x556c5b538310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.849+0000 7fa7a147e640 1 -- 172.21.15.133:0/96273502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b538850 msgr2=0x556c5b53ac40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:40:07.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.849+0000 7fa7a147e640 1 --2- 172.21.15.133:0/96273502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b538850 0x556c5b53ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.849+0000 7fa7a147e640 1 -- 172.21.15.133:0/96273502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b52df90 msgr2=0x556c5b538310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:07.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.849+0000 7fa7a147e640 1 --2- 172.21.15.133:0/96273502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b52df90 0x556c5b538310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.849+0000 7fa7a147e640 1 -- 172.21.15.133:0/96273502 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5b41f440 con 0x556c5b52d670 2024-09-11T23:40:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.850+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/96273502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b52df90 0x556c5b538310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.850+0000 7fa7a147e640 1 --2- 172.21.15.133:0/96273502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b52da50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa79400a500 tx=0x7fa7940095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6a70b0ae7298bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.850+0000 7fa782ffd640 1 -- 172.21.15.133:0/96273502 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794018020 con 0x556c5b52d670 2024-09-11T23:40:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.851+0000 7fa799ffb640 1 -- 172.21.15.133:0/1084443456 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa78c018020 con 0x7fa79c110220 2024-09-11T23:40:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.851+0000 7fa782ffd640 1 -- 172.21.15.133:0/96273502 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa794014cf0 con 0x556c5b52d670 2024-09-11T23:40:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.851+0000 7fa782ffd640 1 -- 172.21.15.133:0/96273502 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79401a2c0 con 0x556c5b52d670 2024-09-11T23:40:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.852+0000 7fa799ffb640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa7840a2ec0 0x7fa7840a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.852+0000 7fa7a52ba740 1 -- 172.21.15.133:0/96273502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 msgr2=0x556c5b52da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.852+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/96273502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b52da50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa79400a500 tx=0x7fa7940095a0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.852+0000 7fa799ffb640 1 -- 172.21.15.133:0/1084443456 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa78c0cea90 con 0x7fa79c110220 2024-09-11T23:40:07.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.852+0000 7fa7a52ba740 1 -- 172.21.15.133:0/96273502 shutdown_connections 2024-09-11T23:40:07.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.852+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/96273502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b538850 0x556c5b53ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.852+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/96273502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b52df90 0x556c5b538310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.852+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/96273502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b52da50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.852+0000 7fa7a52ba740 1 -- 172.21.15.133:0/96273502 >> 172.21.15.133:0/96273502 conn(0x556c5b529c00 msgr2=0x556c5b529fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:07.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.853+0000 7fa7a52ba740 1 -- 172.21.15.133:0/96273502 shutdown_connections 2024-09-11T23:40:07.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.853+0000 7fa7a52ba740 1 -- 172.21.15.133:0/96273502 wait complete. 2024-09-11T23:40:07.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.853+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa7840a2ec0 0x7fa7840a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.854+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa7840a2ec0 0x7fa7840a5380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x556c5b52aa60 tx=0x7fa790009560 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:07.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.854+0000 7fa7a52ba740 1 -- start start 2024-09-11T23:40:07.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.854+0000 7fa7a52ba740 1 Processor -- start 2024-09-11T23:40:07.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.855+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b52d670 0x556c5b5381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.855+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52df90 0x556c5b535470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.855+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b538850 0x556c5b5359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.855+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556c5b53b420 con 0x556c5b52d670 2024-09-11T23:40:07.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.855+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556c5b44e050 con 0x556c5b538850 2024-09-11T23:40:07.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.855+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556c5b53b5a0 con 0x556c5b52df90 2024-09-11T23:40:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.855+0000 7fa7a147e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b538850 0x556c5b5359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.856+0000 7fa7a147e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b538850 0x556c5b5359d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33620/0 (socket says 172.21.15.133:33620) 2024-09-11T23:40:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.856+0000 7fa7a147e640 1 -- 172.21.15.133:0/3596358628 learned_addr learned my addr 172.21.15.133:0/3596358628 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.856+0000 7fa79bfff640 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52df90 0x556c5b535470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.856+0000 7fa7a52ba740 10 client.0 _ll_register_callbacks cb 0x556c5b48e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.856+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b52d670 0x556c5b5381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.856+0000 7fa7a147e640 1 -- 172.21.15.133:0/3596358628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52df90 msgr2=0x556c5b535470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.856+0000 7fa7a147e640 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52df90 0x556c5b535470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.856+0000 7fa7a147e640 1 -- 172.21.15.133:0/3596358628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b52d670 msgr2=0x556c5b5381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:07.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.856+0000 7fa7a147e640 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b52d670 0x556c5b5381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.856+0000 7fa7a147e640 1 -- 172.21.15.133:0/3596358628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5b2043c0 con 0x556c5b538850 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.857+0000 7fa7a147e640 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b538850 0x556c5b5359d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa79400a010 tx=0x7fa794009cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93aa208e3dd8b7b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.857+0000 7fa77bfff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.857+0000 7fa77bfff640 1 -- 172.21.15.133:0/3596358628 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794018030 con 0x556c5b538850 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.857+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3596358628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa794014cf0 con 0x556c5b538850 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.857+0000 7fa77bfff640 1 -- 172.21.15.133:0/3596358628 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa794014aa0 con 0x556c5b538850 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.858+0000 7fa77bfff640 1 -- 172.21.15.133:0/3596358628 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa794023070 con 0x556c5b538850 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.858+0000 7fa7a52ba740 10 client.25832 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.858+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3596358628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c5b546dd0 con 0x556c5b538850 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.858+0000 7fa77bfff640 1 -- 172.21.15.133:0/3596358628 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa794014cf0 con 0x556c5b538850 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.859+0000 7fa77bfff640 1 -- 172.21.15.133:0/3596358628 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa794030450 con 0x556c5b538850 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.859+0000 7fa7a52ba740 10 client.25832 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.859+0000 7fa7a52ba740 10 client.25832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.860+0000 7fa7a52ba740 20 client.25832 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.860+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c5b5481e0 0x556c5b54a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.860+0000 7fa7a52ba740 4 client.25832 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.860+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3596358628 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c5b4a0850 con 0x556c5b5481e0 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.860+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c5b5481e0 0x556c5b54a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.866+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c5b5481e0 0x556c5b54a5d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d056d812a9af530a server_cookie=ecc5ea4238194215 in_seq=0 out_seq=0 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.866+0000 7fa77bfff640 10 client.25832 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.874+0000 7fa77bfff640 1 -- 172.21.15.133:0/3596358628 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c5b4a0850 con 0x556c5b5481e0 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.874+0000 7fa77bfff640 10 client.25832 handle_command_reply: tid=0 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.874+0000 7fa7a52ba740 1 client.25832 shutdown 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.875+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3596358628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b538850 msgr2=0x556c5b5359d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.875+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b538850 0x556c5b5359d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa79400a010 tx=0x7fa794009cf0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.875+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3596358628 shutdown_connections 2024-09-11T23:40:07.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.875+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c5b5481e0 0x556c5b54a5d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.875+0000 7fa77bfff640 0 client.25832 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:07.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.875+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b538850 0x556c5b5359d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.875+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52df90 0x556c5b535470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.875+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/3596358628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b52d670 0x556c5b5381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.875+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3596358628 >> 172.21.15.133:0/3596358628 conn(0x556c5b529c00 msgr2=0x556c5b52f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:07.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.875+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3596358628 shutdown_connections 2024-09-11T23:40:07.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.875+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3596358628 wait complete. 2024-09-11T23:40:07.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.876+0000 7fa7a52ba740 20 client.25832 trim_cache size 0 max 16384 2024-09-11T23:40:07.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.920+0000 7fa7a52ba740 1 Processor -- start 2024-09-11T23:40:07.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.922+0000 7fa7a52ba740 1 -- start start 2024-09-11T23:40:07.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.922+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b544710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.923+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b52df90 0x556c5b54fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.923+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b538850 0x556c5b550280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.923+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa78800e5b0 con 0x556c5b538850 2024-09-11T23:40:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.923+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7880100f0 con 0x556c5b52df90 2024-09-11T23:40:07.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.923+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa78800ec80 con 0x556c5b52d670 2024-09-11T23:40:07.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.923+0000 7fa7a0c7d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b544710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.923+0000 7fa7a0c7d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b544710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:52478/0 (socket says 172.21.15.133:52478) 2024-09-11T23:40:07.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.923+0000 7fa7a0c7d640 1 -- 172.21.15.133:0/3900702954 learned_addr learned my addr 172.21.15.133:0/3900702954 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:07.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.925+0000 7fa7a0c7d640 1 -- 172.21.15.133:0/3900702954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b52df90 msgr2=0x556c5b54fd40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:40:07.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.925+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/3900702954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b52df90 0x556c5b54fd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.925+0000 7fa7a0c7d640 1 -- 172.21.15.133:0/3900702954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b538850 msgr2=0x556c5b550280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:40:07.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.925+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/3900702954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b538850 0x556c5b550280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.925+0000 7fa7a0c7d640 1 -- 172.21.15.133:0/3900702954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5b52a890 con 0x556c5b52d670 2024-09-11T23:40:07.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.927+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/3900702954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b544710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa79002a960 tx=0x7fa79003b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c807fcc668ac028d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:07.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.931+0000 7fa7827fc640 1 -- 172.21.15.133:0/3900702954 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790056070 con 0x556c5b52d670 2024-09-11T23:40:07.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.931+0000 7fa7827fc640 1 -- 172.21.15.133:0/3900702954 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa790032e10 con 0x556c5b52d670 2024-09-11T23:40:07.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.932+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3900702954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 msgr2=0x556c5b544710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:07.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.932+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/3900702954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b544710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa79002a960 tx=0x7fa79003b000 comp rx=0 tx=0).stop 2024-09-11T23:40:07.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.941+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3900702954 shutdown_connections 2024-09-11T23:40:07.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.941+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/3900702954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b538850 0x556c5b550280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.941+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/3900702954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b52df90 0x556c5b54fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.941+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/3900702954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52d670 0x556c5b544710 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.941+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3900702954 >> 172.21.15.133:0/3900702954 conn(0x556c5b529c00 msgr2=0x556c5b534560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:07.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.945+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3900702954 shutdown_connections 2024-09-11T23:40:07.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.945+0000 7fa7a52ba740 1 -- 172.21.15.133:0/3900702954 wait complete. 2024-09-11T23:40:07.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.945+0000 7fa7a52ba740 1 -- start start 2024-09-11T23:40:07.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.945+0000 7fa7a52ba740 1 Processor -- start 2024-09-11T23:40:07.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.946+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b52d670 0x556c5b53bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.946+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52df90 0x556c5b53c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.946+0000 7fa7a52ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b538850 0x556c5b5b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.946+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa78800fa20 con 0x556c5b538850 2024-09-11T23:40:07.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.946+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa78800f350 con 0x556c5b52d670 2024-09-11T23:40:07.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.946+0000 7fa7a52ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa78800c590 con 0x556c5b52df90 2024-09-11T23:40:07.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.946+0000 7fa7a52ba740 10 client.0 _ll_register_callbacks cb 0x7fa794009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:07.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.947+0000 7fa7a0c7d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b52d670 0x556c5b53bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.947+0000 7fa7a0c7d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b52d670 0x556c5b53bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33632/0 (socket says 172.21.15.133:33632) 2024-09-11T23:40:07.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.947+0000 7fa7a0c7d640 1 -- 172.21.15.133:0/118220307 learned_addr learned my addr 172.21.15.133:0/118220307 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:07.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.947+0000 7fa7a147e640 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b538850 0x556c5b5b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.947+0000 7fa7a0c7d640 1 -- 172.21.15.133:0/118220307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52df90 msgr2=0x556c5b53c320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:40:07.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.947+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52df90 0x556c5b53c320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.947+0000 7fa7a0c7d640 1 -- 172.21.15.133:0/118220307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b538850 msgr2=0x556c5b5b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:07.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.947+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b538850 0x556c5b5b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.947+0000 7fa7a0c7d640 1 -- 172.21.15.133:0/118220307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5b5c2250 con 0x556c5b52d670 2024-09-11T23:40:07.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.947+0000 7fa7a147e640 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b538850 0x556c5b5b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:07.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:07 smithi138 ceph-mon[35142]: pgmap v103: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 176 MiB/s wr, 1.30k op/s 2024-09-11T23:40:07.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.948+0000 7fa7a0c7d640 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b52d670 0x556c5b53bde0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa79002a480 tx=0x7fa7900315f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae143f89ae6079ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:07.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.952+0000 7fa77bfff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:07.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.952+0000 7fa77bfff640 1 -- 172.21.15.133:0/118220307 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790056040 con 0x556c5b52d670 2024-09-11T23:40:07.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.952+0000 7fa77bfff640 1 -- 172.21.15.133:0/118220307 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa790004e50 con 0x556c5b52d670 2024-09-11T23:40:07.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.957+0000 7fa7a52ba740 1 -- 172.21.15.133:0/118220307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa790032e10 con 0x556c5b52d670 2024-09-11T23:40:07.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.960+0000 7fa77bfff640 1 -- 172.21.15.133:0/118220307 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa790005050 con 0x556c5b52d670 2024-09-11T23:40:07.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.960+0000 7fa7a52ba740 10 client.25835 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:07.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.960+0000 7fa7a52ba740 1 -- 172.21.15.133:0/118220307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c5b5c34a0 con 0x556c5b52d670 2024-09-11T23:40:07.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.961+0000 7fa77bfff640 1 -- 172.21.15.133:0/118220307 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa790032e10 con 0x556c5b52d670 2024-09-11T23:40:07.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.961+0000 7fa77bfff640 1 -- 172.21.15.133:0/118220307 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa790059070 con 0x556c5b52d670 2024-09-11T23:40:07.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.963+0000 7fa7a52ba740 10 client.25835 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:07.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.963+0000 7fa7a52ba740 10 client.25835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:07.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.963+0000 7fa7a52ba740 20 client.25835 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:07.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.963+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c5b5481e0 0x556c5b5c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:07.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.963+0000 7fa7a52ba740 4 client.25835 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:40:07.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.963+0000 7fa7a52ba740 1 -- 172.21.15.133:0/118220307 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c5b49ad60 con 0x556c5b5481e0 2024-09-11T23:40:07.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.965+0000 7fa79bfff640 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c5b5481e0 0x556c5b5c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:07.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.966+0000 7fa79bfff640 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c5b5481e0 0x556c5b5c5af0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d142da17b59a58 server_cookie=e93052fba2727087 in_seq=0 out_seq=0 2024-09-11T23:40:07.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.966+0000 7fa77bfff640 10 client.25835 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:07.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.968+0000 7fa77bfff640 1 -- 172.21.15.133:0/118220307 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556c5b49ad60 con 0x556c5b5481e0 2024-09-11T23:40:07.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.968+0000 7fa77bfff640 10 client.25835 handle_command_reply: tid=0 2024-09-11T23:40:07.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.968+0000 7fa7a52ba740 1 client.25835 shutdown 2024-09-11T23:40:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.971+0000 7fa7a52ba740 1 -- 172.21.15.133:0/118220307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b52d670 msgr2=0x556c5b53bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.971+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b52d670 0x556c5b53bde0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa79002a480 tx=0x7fa7900315f0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.971+0000 7fa7a52ba740 1 -- 172.21.15.133:0/118220307 shutdown_connections 2024-09-11T23:40:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.971+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c5b5481e0 0x556c5b5c5af0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.971+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c5b538850 0x556c5b5b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.971+0000 7fa77bfff640 0 client.25835 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.971+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c5b52df90 0x556c5b53c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.971+0000 7fa7a52ba740 1 --2- 172.21.15.133:0/118220307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c5b52d670 0x556c5b53bde0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.971+0000 7fa7a52ba740 1 -- 172.21.15.133:0/118220307 >> 172.21.15.133:0/118220307 conn(0x556c5b529c00 msgr2=0x556c5b530240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:07.977 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:40:07.977 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:40:07.977 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:40:07.977 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:40:07.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.972+0000 7fa7a52ba740 1 -- 172.21.15.133:0/118220307 shutdown_connections 2024-09-11T23:40:07.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.973+0000 7fa7a52ba740 1 -- 172.21.15.133:0/118220307 wait complete. 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.973+0000 7fa7a52ba740 20 client.25835 trim_cache size 0 max 16384 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1084443456 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa7840a2ec0 msgr2=0x7fa7840a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa7840a2ec0 0x7fa7840a5380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x556c5b52aa60 tx=0x7fa790009560 comp rx=0 tx=0).stop 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1084443456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c110220 msgr2=0x7fa79c1a7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c110220 0x7fa79c1a7730 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00b450 tx=0x7fa78c00b910 comp rx=0 tx=0).stop 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1084443456 shutdown_connections 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa7840a2ec0 0x7fa7840a5380 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x556c5b52aa60 tx=0x7fa790009560 comp rx=0 tx=0).stop 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa79c11b140 0x7fa79c1abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa79c110220 0x7fa79c1a7730 secure :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00b450 tx=0x7fa78c00b910 comp rx=0 tx=0).stop 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 --2- 172.21.15.133:0/1084443456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa79c06ea30 0x7fa79c1a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1084443456 >> 172.21.15.133:0/1084443456 conn(0x7fa79c073210 msgr2=0x7fa79c10e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1084443456 shutdown_connections 2024-09-11T23:40:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:07.975+0000 7fa7a2ee1640 1 -- 172.21.15.133:0/1084443456 wait complete. 2024-09-11T23:40:09.526 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:40:09.526 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:40:09.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:09 smithi138 ceph-mon[35142]: pgmap v104: 97 pgs: 97 active+clean; 21 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 871 KiB/s rd, 423 MiB/s wr, 2.89k op/s 2024-09-11T23:40:09.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:09 smithi133 ceph-mon[31430]: pgmap v104: 97 pgs: 97 active+clean; 21 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 871 KiB/s rd, 423 MiB/s wr, 2.89k op/s 2024-09-11T23:40:09.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:09 smithi178 ceph-mon[35882]: pgmap v104: 97 pgs: 97 active+clean; 21 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 871 KiB/s rd, 423 MiB/s wr, 2.89k op/s 2024-09-11T23:40:10.716 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:11.606 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:11 smithi133 ceph-mon[31430]: pgmap v105: 97 pgs: 97 active+clean; 21 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 868 KiB/s rd, 432 MiB/s wr, 3.09k op/s 2024-09-11T23:40:11.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.652+0000 7fda4e96a640 1 -- 172.21.15.133:0/1969698887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4806ea30 msgr2=0x7fda4806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:11.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.652+0000 7fda4e96a640 1 --2- 172.21.15.133:0/1969698887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4806ea30 0x7fda4806ee70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fda44009860 tx=0x7fda4402a360 comp rx=0 tx=0).stop 2024-09-11T23:40:11.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.652+0000 7fda4e96a640 1 -- 172.21.15.133:0/1969698887 shutdown_connections 2024-09-11T23:40:11.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.652+0000 7fda4e96a640 1 --2- 172.21.15.133:0/1969698887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda4811b140 0x7fda4811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:11.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.652+0000 7fda4e96a640 1 --2- 172.21.15.133:0/1969698887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4806ea30 0x7fda4806ee70 secure :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fda44009860 tx=0x7fda4402a360 comp rx=0 tx=0).stop 2024-09-11T23:40:11.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.652+0000 7fda4e96a640 1 --2- 172.21.15.133:0/1969698887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda48110190 0x7fda4806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:11.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.652+0000 7fda4e96a640 1 -- 172.21.15.133:0/1969698887 >> 172.21.15.133:0/1969698887 conn(0x7fda48073180 msgr2=0x7fda48073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:11.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.655+0000 7fda4e96a640 1 -- 172.21.15.133:0/1969698887 shutdown_connections 2024-09-11T23:40:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.655+0000 7fda4e96a640 1 -- 172.21.15.133:0/1969698887 wait complete. 2024-09-11T23:40:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.656+0000 7fda4e96a640 1 Processor -- start 2024-09-11T23:40:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.656+0000 7fda4e96a640 1 -- start start 2024-09-11T23:40:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.657+0000 7fda4e96a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48110190 0x7fda481a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.657+0000 7fda4e96a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda4811b140 0x7fda481a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.657+0000 7fda4e96a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda481abbc0 0x7fda481ac070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.657+0000 7fda4e96a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda48120be0 con 0x7fda481abbc0 2024-09-11T23:40:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.657+0000 7fda4e96a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda48120a60 con 0x7fda4811b140 2024-09-11T23:40:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.657+0000 7fda4e96a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda48120d60 con 0x7fda48110190 2024-09-11T23:40:11.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.658+0000 7fda4d968640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48110190 0x7fda481a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.658+0000 7fda4e169640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda481abbc0 0x7fda481ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.658+0000 7fda4e169640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda481abbc0 0x7fda481ac070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52322/0 (socket says 172.21.15.133:52322) 2024-09-11T23:40:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.658+0000 7fda4e169640 1 -- 172.21.15.133:0/3350845091 learned_addr learned my addr 172.21.15.133:0/3350845091 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.658+0000 7fda4d968640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48110190 0x7fda481a72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:52492/0 (socket says 172.21.15.133:52492) 2024-09-11T23:40:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.658+0000 7fda4d167640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda4811b140 0x7fda481a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.659+0000 7fda4d968640 1 -- 172.21.15.133:0/3350845091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda4811b140 msgr2=0x7fda481a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.659+0000 7fda4d968640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda4811b140 0x7fda481a7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.659+0000 7fda4d968640 1 -- 172.21.15.133:0/3350845091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda481abbc0 msgr2=0x7fda481ac070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.659+0000 7fda4d968640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda481abbc0 0x7fda481ac070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.659+0000 7fda4d968640 1 -- 172.21.15.133:0/3350845091 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda481ac710 con 0x7fda48110190 2024-09-11T23:40:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.660+0000 7fda4d968640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48110190 0x7fda481a72f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fda3800c970 tx=0x7fda3800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.661+0000 7fda3effd640 1 -- 172.21.15.133:0/3350845091 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38007bb0 con 0x7fda48110190 2024-09-11T23:40:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.661+0000 7fda4e96a640 1 -- 172.21.15.133:0/3350845091 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda481aca00 con 0x7fda48110190 2024-09-11T23:40:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.661+0000 7fda4e96a640 1 -- 172.21.15.133:0/3350845091 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda48110c60 con 0x7fda48110190 2024-09-11T23:40:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.662+0000 7fda3effd640 1 -- 172.21.15.133:0/3350845091 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda38007d50 con 0x7fda48110190 2024-09-11T23:40:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.662+0000 7fda3effd640 1 -- 172.21.15.133:0/3350845091 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3800f450 con 0x7fda48110190 2024-09-11T23:40:11.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.668+0000 7fda4e96a640 1 -- 172.21.15.133:0/3350845091 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2c005290 con 0x7fda48110190 2024-09-11T23:40:11.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.669+0000 7fda3effd640 1 -- 172.21.15.133:0/3350845091 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fda3800f5f0 con 0x7fda48110190 2024-09-11T23:40:11.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.670+0000 7fda3effd640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fda340a2e80 0x7fda340a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:11.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.671+0000 7fda3effd640 1 -- 172.21.15.133:0/3350845091 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fda380046c0 con 0x7fda48110190 2024-09-11T23:40:11.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.673+0000 7fda4d167640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fda340a2e80 0x7fda340a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:11.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.674+0000 7fda4d167640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fda340a2e80 0x7fda340a5340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda4402a8d0 tx=0x7fda44000bc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:11.677+0000 7fda3effd640 1 -- 172.21.15.133:0/3350845091 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda38014030 con 0x7fda48110190 2024-09-11T23:40:11.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:11 smithi138 ceph-mon[35142]: pgmap v105: 97 pgs: 97 active+clean; 21 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 868 KiB/s rd, 432 MiB/s wr, 3.09k op/s 2024-09-11T23:40:11.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:11 smithi178 ceph-mon[35882]: pgmap v105: 97 pgs: 97 active+clean; 21 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 868 KiB/s rd, 432 MiB/s wr, 3.09k op/s 2024-09-11T23:40:12.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.134+0000 7fda4e96a640 1 -- 172.21.15.133:0/3350845091 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda2c005850 con 0x7fda48110190 2024-09-11T23:40:12.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.135+0000 7fda3effd640 1 -- 172.21.15.133:0/3350845091 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7fda38014030 con 0x7fda48110190 2024-09-11T23:40:12.141 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:40:12.141 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:40:12.141 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:40:12.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.141+0000 7fda4e96a640 1 -- 172.21.15.133:0/3350845091 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fda340a2e80 msgr2=0x7fda340a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:12.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.141+0000 7fda4e96a640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fda340a2e80 0x7fda340a5340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda4402a8d0 tx=0x7fda44000bc0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.141+0000 7fda4e96a640 1 -- 172.21.15.133:0/3350845091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48110190 msgr2=0x7fda481a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:12.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.141+0000 7fda4e96a640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48110190 0x7fda481a72f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fda3800c970 tx=0x7fda3800ce30 comp rx=0 tx=0).stop 2024-09-11T23:40:12.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.143+0000 7fda4e96a640 1 -- 172.21.15.133:0/3350845091 shutdown_connections 2024-09-11T23:40:12.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.143+0000 7fda4e96a640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fda340a2e80 0x7fda340a5340 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.143+0000 7fda4e96a640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda481abbc0 0x7fda481ac070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.143+0000 7fda4e96a640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda4811b140 0x7fda481a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.143+0000 7fda4e96a640 1 --2- 172.21.15.133:0/3350845091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda48110190 0x7fda481a72f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.143+0000 7fda4e96a640 1 -- 172.21.15.133:0/3350845091 >> 172.21.15.133:0/3350845091 conn(0x7fda48073180 msgr2=0x7fda481195d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.149+0000 7fda4e96a640 1 -- 172.21.15.133:0/3350845091 shutdown_connections 2024-09-11T23:40:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:12.151+0000 7fda4e96a640 1 -- 172.21.15.133:0/3350845091 wait complete. 2024-09-11T23:40:12.468 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:12 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3350845091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:12.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:12 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3350845091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.755+0000 7efeeb3e8640 1 -- 172.21.15.133:0/18326937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee4101350 msgr2=0x7efee410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.755+0000 7efeeb3e8640 1 --2- 172.21.15.133:0/18326937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee4101350 0x7efee410d880 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efedc009cd0 tx=0x7efedc02a910 comp rx=0 tx=0).stop 2024-09-11T23:40:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.755+0000 7efeeb3e8640 1 -- 172.21.15.133:0/18326937 shutdown_connections 2024-09-11T23:40:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.755+0000 7efeeb3e8640 1 --2- 172.21.15.133:0/18326937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee4101350 0x7efee410d880 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.755+0000 7efeeb3e8640 1 --2- 172.21.15.133:0/18326937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee41009b0 0x7efee4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.755+0000 7efeeb3e8640 1 --2- 172.21.15.133:0/18326937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee41069c0 0x7efee4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.755+0000 7efeeb3e8640 1 -- 172.21.15.133:0/18326937 >> 172.21.15.133:0/18326937 conn(0x7efee4077990 msgr2=0x7efee40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.756+0000 7efeeb3e8640 1 -- 172.21.15.133:0/18326937 shutdown_connections 2024-09-11T23:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.756+0000 7efeeb3e8640 1 -- 172.21.15.133:0/18326937 wait complete. 2024-09-11T23:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.756+0000 7efeeb3e8640 1 Processor -- start 2024-09-11T23:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.757+0000 7efeeb3e8640 1 -- start start 2024-09-11T23:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.757+0000 7efeeb3e8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee41009b0 0x7efee41a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.757+0000 7efeeb3e8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee4101350 0x7efee41a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.757+0000 7efeeb3e8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41069c0 0x7efee419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.757+0000 7efeeb3e8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efee4110e00 con 0x7efee41069c0 2024-09-11T23:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.757+0000 7efeeb3e8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efee4110c80 con 0x7efee4101350 2024-09-11T23:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.757+0000 7efeeb3e8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efee4110f80 con 0x7efee41009b0 2024-09-11T23:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.757+0000 7efee9985640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41069c0 0x7efee419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.757+0000 7efee9985640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41069c0 0x7efee419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35640/0 (socket says 172.21.15.133:35640) 2024-09-11T23:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.757+0000 7efee9985640 1 -- 172.21.15.133:0/1295149852 learned_addr learned my addr 172.21.15.133:0/1295149852 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.758+0000 7efee8983640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee4101350 0x7efee41a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.758+0000 7efee9985640 1 -- 172.21.15.133:0/1295149852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee41009b0 msgr2=0x7efee41a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.758+0000 7efee9985640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee41009b0 0x7efee41a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.758+0000 7efee9985640 1 -- 172.21.15.133:0/1295149852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee4101350 msgr2=0x7efee41a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.758+0000 7efee9985640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee4101350 0x7efee41a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.758+0000 7efee9985640 1 -- 172.21.15.133:0/1295149852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee419f2b0 con 0x7efee41069c0 2024-09-11T23:40:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.758+0000 7efee9985640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41069c0 0x7efee419eae0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efedc005cc0 tx=0x7efedc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.759+0000 7efeda7fc640 1 -- 172.21.15.133:0/1295149852 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc031660 con 0x7efee41069c0 2024-09-11T23:40:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.759+0000 7efeda7fc640 1 -- 172.21.15.133:0/1295149852 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efedc031800 con 0x7efee41069c0 2024-09-11T23:40:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.759+0000 7efeda7fc640 1 -- 172.21.15.133:0/1295149852 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc02fb50 con 0x7efee41069c0 2024-09-11T23:40:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.759+0000 7efeeb3e8640 1 -- 172.21.15.133:0/1295149852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee419f540 con 0x7efee41069c0 2024-09-11T23:40:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.760+0000 7efeeb3e8640 1 -- 172.21.15.133:0/1295149852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41ab470 con 0x7efee41069c0 2024-09-11T23:40:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.765+0000 7efeee261740 1 Processor -- start 2024-09-11T23:40:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.765+0000 7efeda7fc640 1 -- 172.21.15.133:0/1295149852 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7efedc02fd50 con 0x7efee41069c0 2024-09-11T23:40:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.765+0000 7efeee261740 1 -- start start 2024-09-11T23:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efeee261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 0x561cfc718ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efeee261740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc719410 0x561cfc723770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efeee261740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc723cb0 0x561cfc7260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efeee261740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561cfc67d2f0 con 0x561cfc723cb0 2024-09-11T23:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efeee261740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561cfc625270 con 0x561cfc719410 2024-09-11T23:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efee8983640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 0x561cfc718ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efee9184640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc719410 0x561cfc723770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efee9184640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc719410 0x561cfc723770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48770/0 (socket says 172.21.15.133:48770) 2024-09-11T23:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efeda7fc640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efed00a2e20 0x7efed00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efee8983640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 0x561cfc718ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:39010/0 (socket says 172.21.15.133:39010) 2024-09-11T23:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efee9184640 1 -- 172.21.15.133:0/711459660 learned_addr learned my addr 172.21.15.133:0/711459660 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efeee261740 1 -- 172.21.15.133:0/711459660 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561cfc6792b0 con 0x561cfc718af0 2024-09-11T23:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.766+0000 7efee9985640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efed00a2e20 0x7efed00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efee8983640 1 --2- 172.21.15.133:0/711459660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc723cb0 0x561cfc7260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efeda7fc640 1 -- 172.21.15.133:0/1295149852 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7efedc0f0790 con 0x7efee41069c0 2024-09-11T23:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efee9184640 1 -- 172.21.15.133:0/711459660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 msgr2=0x561cfc718ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efee9184640 1 --2- 172.21.15.133:0/711459660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 0x561cfc718ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efee9184640 1 -- 172.21.15.133:0/711459660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc723cb0 msgr2=0x561cfc7260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efee9184640 1 --2- 172.21.15.133:0/711459660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc723cb0 0x561cfc7260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efee9184640 1 -- 172.21.15.133:0/711459660 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cfc624340 con 0x561cfc719410 2024-09-11T23:40:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efee9184640 1 --2- 172.21.15.133:0/711459660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc719410 0x561cfc723770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efee000bb10 tx=0x7efee000d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68b07cd392e60344 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efee9985640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efed00a2e20 0x7efed00a52e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efedc0ee110 tx=0x7efedc0879b0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efecf7fe640 1 -- 172.21.15.133:0/711459660 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0016070 con 0x561cfc719410 2024-09-11T23:40:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efecf7fe640 1 -- 172.21.15.133:0/711459660 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee0005500 con 0x561cfc719410 2024-09-11T23:40:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.767+0000 7efecf7fe640 1 -- 172.21.15.133:0/711459660 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0010460 con 0x561cfc719410 2024-09-11T23:40:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.768+0000 7efeee261740 1 -- 172.21.15.133:0/711459660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc719410 msgr2=0x561cfc723770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.768+0000 7efeee261740 1 --2- 172.21.15.133:0/711459660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc719410 0x561cfc723770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efee000bb10 tx=0x7efee000d2e0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.768+0000 7efeee261740 1 -- 172.21.15.133:0/711459660 shutdown_connections 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.768+0000 7efeee261740 1 --2- 172.21.15.133:0/711459660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc723cb0 0x561cfc7260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.768+0000 7efeee261740 1 --2- 172.21.15.133:0/711459660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc719410 0x561cfc723770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.768+0000 7efeee261740 1 --2- 172.21.15.133:0/711459660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 0x561cfc718ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.768+0000 7efeee261740 1 -- 172.21.15.133:0/711459660 >> 172.21.15.133:0/711459660 conn(0x561cfc715060 msgr2=0x561cfc715440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.768+0000 7efeee261740 1 -- 172.21.15.133:0/711459660 shutdown_connections 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.769+0000 7efeee261740 1 -- 172.21.15.133:0/711459660 wait complete. 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.769+0000 7efeee261740 1 -- start start 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.769+0000 7efeee261740 1 Processor -- start 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.769+0000 7efeee261740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc718af0 0x561cfc720c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efeee261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc719410 0x561cfc721140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efeee261740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc723cb0 0x561cfc721680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efeee261740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561cfc726880 con 0x561cfc718af0 2024-09-11T23:40:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efeee261740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561cfc606290 con 0x561cfc723cb0 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efeee261740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561cfc726a00 con 0x561cfc719410 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efee8983640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc718af0 0x561cfc720c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efee8983640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc718af0 0x561cfc720c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35666/0 (socket says 172.21.15.133:35666) 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efee8983640 1 -- 172.21.15.133:0/240213559 learned_addr learned my addr 172.21.15.133:0/240213559 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efee9184640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc719410 0x561cfc721140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efee8983640 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc723cb0 0x561cfc721680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efeee261740 10 client.0 _ll_register_callbacks cb 0x561cfc637140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efee9184640 1 -- 172.21.15.133:0/240213559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc723cb0 msgr2=0x561cfc721680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efee9184640 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc723cb0 0x561cfc721680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efee9184640 1 -- 172.21.15.133:0/240213559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc718af0 msgr2=0x561cfc720c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efee9184640 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc718af0 0x561cfc720c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efee9184640 1 -- 172.21.15.133:0/240213559 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cfc390210 con 0x561cfc719410 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.770+0000 7efee8983640 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc723cb0 0x561cfc721680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:40:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.771+0000 7efee9184640 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc719410 0x561cfc721140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efee0000c00 tx=0x7efee000b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff9b4e605cd74 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.771+0000 7efeccff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.771+0000 7efeccff9640 1 -- 172.21.15.133:0/240213559 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0016070 con 0x561cfc719410 2024-09-11T23:40:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.771+0000 7efeccff9640 1 -- 172.21.15.133:0/240213559 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7efee0005020 con 0x561cfc719410 2024-09-11T23:40:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.771+0000 7efeee261740 1 -- 172.21.15.133:0/240213559 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee0005500 con 0x561cfc719410 2024-09-11T23:40:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.771+0000 7efeccff9640 1 -- 172.21.15.133:0/240213559 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efee0005220 con 0x561cfc719410 2024-09-11T23:40:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.771+0000 7efeee261740 10 client.25825 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.771+0000 7efeee261740 1 -- 172.21.15.133:0/240213559 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561cfc45c620 con 0x561cfc719410 2024-09-11T23:40:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.771+0000 7efeccff9640 1 -- 172.21.15.133:0/240213559 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee0004030 con 0x561cfc719410 2024-09-11T23:40:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.772+0000 7efeccff9640 1 -- 172.21.15.133:0/240213559 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efee002c050 con 0x561cfc719410 2024-09-11T23:40:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.772+0000 7efeee261740 10 client.25825 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.772+0000 7efeee261740 10 client.25825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.772+0000 7efeee261740 20 client.25825 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.772+0000 7efeee261740 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561cfc733bf0 0x561cfc735fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.772+0000 7efeee261740 4 client.25825 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.772+0000 7efeee261740 1 -- 172.21.15.133:0/240213559 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efee0005500 con 0x561cfc733bf0 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.772+0000 7efee8983640 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561cfc733bf0 0x561cfc735fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.772+0000 7efee8983640 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561cfc733bf0 0x561cfc735fe0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5149001c5a29f7b7 server_cookie=7a4e41596613835 in_seq=0 out_seq=0 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.772+0000 7efeccff9640 10 client.25825 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.774+0000 7efeccff9640 1 -- 172.21.15.133:0/240213559 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7efee0005500 con 0x561cfc733bf0 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.774+0000 7efeccff9640 10 client.25825 handle_command_reply: tid=0 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.774+0000 7efeee261740 1 client.25825 shutdown 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.774+0000 7efeee261740 1 -- 172.21.15.133:0/240213559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc719410 msgr2=0x561cfc721140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.774+0000 7efeee261740 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc719410 0x561cfc721140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efee0000c00 tx=0x7efee000b6e0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.775+0000 7efeee261740 1 -- 172.21.15.133:0/240213559 shutdown_connections 2024-09-11T23:40:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.775+0000 7efeee261740 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561cfc733bf0 0x561cfc735fe0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.775+0000 7efeee261740 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc723cb0 0x561cfc721680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.775+0000 7efeccff9640 0 client.25825 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.775+0000 7efeee261740 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc719410 0x561cfc721140 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.775+0000 7efeee261740 1 --2- 172.21.15.133:0/240213559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc718af0 0x561cfc720c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.775+0000 7efeee261740 1 -- 172.21.15.133:0/240213559 >> 172.21.15.133:0/240213559 conn(0x561cfc715060 msgr2=0x561cfc71ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.775+0000 7efeee261740 1 -- 172.21.15.133:0/240213559 shutdown_connections 2024-09-11T23:40:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.775+0000 7efeee261740 1 -- 172.21.15.133:0/240213559 wait complete. 2024-09-11T23:40:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.775+0000 7efeee261740 20 client.25825 trim_cache size 0 max 16384 2024-09-11T23:40:12.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:12 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3350845091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efeee261740 1 Processor -- start 2024-09-11T23:40:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efeee261740 1 -- start start 2024-09-11T23:40:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efeee261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 0x561cfc714b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efeee261740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc719410 0x561cfc71b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efeee261740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc723cb0 0x561cfc7214b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efeee261740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efee41b3cf0 con 0x561cfc719410 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efeee261740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efee41b5830 con 0x561cfc723cb0 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efeee261740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efee41b43c0 con 0x561cfc718af0 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efee8983640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 0x561cfc714b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efee9184640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc719410 0x561cfc71b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efee8983640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 0x561cfc714b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:39036/0 (socket says 172.21.15.133:39036) 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efee9184640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc719410 0x561cfc71b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35682/0 (socket says 172.21.15.133:35682) 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efee8983640 1 -- 172.21.15.133:0/1762958318 learned_addr learned my addr 172.21.15.133:0/1762958318 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.793+0000 7efee8983640 1 --2- 172.21.15.133:0/1762958318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc723cb0 0x561cfc7214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.794+0000 7efee8983640 1 -- 172.21.15.133:0/1762958318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc723cb0 msgr2=0x561cfc7214b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.794+0000 7efee8983640 1 --2- 172.21.15.133:0/1762958318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc723cb0 0x561cfc7214b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.794+0000 7efee8983640 1 -- 172.21.15.133:0/1762958318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc719410 msgr2=0x561cfc71b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.794+0000 7efee8983640 1 --2- 172.21.15.133:0/1762958318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc719410 0x561cfc71b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.794+0000 7efee8983640 1 -- 172.21.15.133:0/1762958318 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cfc71fb90 con 0x561cfc718af0 2024-09-11T23:40:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.794+0000 7efee8983640 1 --2- 172.21.15.133:0/1762958318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 0x561cfc714b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13a563354f9363c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.794+0000 7efeceffd640 1 -- 172.21.15.133:0/1762958318 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4025030 con 0x561cfc718af0 2024-09-11T23:40:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.794+0000 7efeceffd640 1 -- 172.21.15.133:0/1762958318 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed401adb0 con 0x561cfc718af0 2024-09-11T23:40:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.794+0000 7efeceffd640 1 -- 172.21.15.133:0/1762958318 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4018720 con 0x561cfc718af0 2024-09-11T23:40:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.795+0000 7efeee261740 1 -- 172.21.15.133:0/1762958318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 msgr2=0x561cfc714b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.795+0000 7efeee261740 1 --2- 172.21.15.133:0/1762958318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 0x561cfc714b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4004750 comp rx=0 tx=0).stop 2024-09-11T23:40:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.795+0000 7efeee261740 1 -- 172.21.15.133:0/1762958318 shutdown_connections 2024-09-11T23:40:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.795+0000 7efeee261740 1 --2- 172.21.15.133:0/1762958318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc723cb0 0x561cfc7214b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.795+0000 7efeee261740 1 --2- 172.21.15.133:0/1762958318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc719410 0x561cfc71b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.795+0000 7efeee261740 1 --2- 172.21.15.133:0/1762958318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc718af0 0x561cfc714b00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.795+0000 7efeee261740 1 -- 172.21.15.133:0/1762958318 >> 172.21.15.133:0/1762958318 conn(0x561cfc715060 msgr2=0x561cfc722c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.796+0000 7efeee261740 1 -- 172.21.15.133:0/1762958318 shutdown_connections 2024-09-11T23:40:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.797+0000 7efeee261740 1 -- 172.21.15.133:0/1762958318 wait complete. 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.797+0000 7efeee261740 1 -- start start 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.797+0000 7efeee261740 1 Processor -- start 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.798+0000 7efeee261740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc718af0 0x561cfc739890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.798+0000 7efeee261740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc719410 0x561cfc736c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.798+0000 7efeee261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc723cb0 0x561cfc737230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.798+0000 7efeee261740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efee41b5160 con 0x561cfc719410 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.798+0000 7efeee261740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efee41b4a90 con 0x561cfc718af0 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.798+0000 7efeee261740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efee41b1cd0 con 0x561cfc723cb0 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.798+0000 7efeee261740 10 client.0 _ll_register_callbacks cb 0x7efee000b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.798+0000 7efee8983640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc718af0 0x561cfc739890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.798+0000 7efee8983640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc718af0 0x561cfc739890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48808/0 (socket says 172.21.15.133:48808) 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.798+0000 7efee8983640 1 -- 172.21.15.133:0/3386893761 learned_addr learned my addr 172.21.15.133:0/3386893761 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.798+0000 7efee8983640 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc723cb0 0x561cfc737230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efee8983640 1 -- 172.21.15.133:0/3386893761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc723cb0 msgr2=0x561cfc737230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efee8983640 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc723cb0 0x561cfc737230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efee8983640 1 -- 172.21.15.133:0/3386893761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc719410 msgr2=0x561cfc736c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efee8983640 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc719410 0x561cfc736c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efee8983640 1 -- 172.21.15.133:0/3386893761 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cfc737a60 con 0x561cfc718af0 2024-09-11T23:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efee8983640 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc718af0 0x561cfc739890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efed4006dc0 tx=0x7efed4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=565d4cb0b5381d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efeccff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efeccff9640 1 -- 172.21.15.133:0/3386893761 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4025040 con 0x561cfc718af0 2024-09-11T23:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efeccff9640 1 -- 172.21.15.133:0/3386893761 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7efed402a410 con 0x561cfc718af0 2024-09-11T23:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efeee261740 1 -- 172.21.15.133:0/3386893761 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed401adb0 con 0x561cfc718af0 2024-09-11T23:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efeccff9640 1 -- 172.21.15.133:0/3386893761 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efed402a610 con 0x561cfc718af0 2024-09-11T23:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efeee261740 10 client.25856 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.799+0000 7efeee261740 1 -- 172.21.15.133:0/3386893761 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561cfc79e670 con 0x561cfc718af0 2024-09-11T23:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.800+0000 7efeccff9640 1 -- 172.21.15.133:0/3386893761 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed402ac90 con 0x561cfc718af0 2024-09-11T23:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.800+0000 7efeccff9640 1 -- 172.21.15.133:0/3386893761 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efed4015040 con 0x561cfc718af0 2024-09-11T23:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.800+0000 7efeee261740 10 client.25856 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.800+0000 7efeee261740 10 client.25856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.800+0000 7efeee261740 20 client.25856 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.800+0000 7efeee261740 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561cfc79ec30 0x561cfc7a1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.800+0000 7efeee261740 4 client.25856 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.800+0000 7efeee261740 1 -- 172.21.15.133:0/3386893761 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7efed401adb0 con 0x561cfc79ec30 2024-09-11T23:40:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.802+0000 7efee9184640 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561cfc79ec30 0x561cfc7a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.803+0000 7efee9184640 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561cfc79ec30 0x561cfc7a1050 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e437bc847e50ab server_cookie=5584c1c4a788ee9e in_seq=0 out_seq=0 2024-09-11T23:40:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:12.803+0000 7efeccff9640 10 client.25856 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:13.671 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:13 smithi133 ceph-mon[31430]: pgmap v106: 97 pgs: 97 active+clean; 23 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 585 MiB/s wr, 3.60k op/s 2024-09-11T23:40:13.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:13 smithi138 ceph-mon[35142]: pgmap v106: 97 pgs: 97 active+clean; 23 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 585 MiB/s wr, 3.60k op/s 2024-09-11T23:40:13.743 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:40:13.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:13 smithi178 ceph-mon[35882]: pgmap v106: 97 pgs: 97 active+clean; 23 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 585 MiB/s wr, 3.60k op/s 2024-09-11T23:40:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.201+0000 7efeccff9640 1 -- 172.21.15.133:0/3386893761 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7efed401adb0 con 0x561cfc79ec30 2024-09-11T23:40:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.201+0000 7efeccff9640 10 client.25856 handle_command_reply: tid=0 2024-09-11T23:40:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.201+0000 7efeee261740 1 client.25856 shutdown 2024-09-11T23:40:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.202+0000 7efeee261740 1 -- 172.21.15.133:0/3386893761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc718af0 msgr2=0x561cfc739890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.202+0000 7efeee261740 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc718af0 0x561cfc739890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efed4006dc0 tx=0x7efed4008040 comp rx=0 tx=0).stop 2024-09-11T23:40:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.202+0000 7efeee261740 1 -- 172.21.15.133:0/3386893761 shutdown_connections 2024-09-11T23:40:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.202+0000 7efeee261740 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561cfc79ec30 0x561cfc7a1050 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.202+0000 7efeee261740 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cfc723cb0 0x561cfc737230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.202+0000 7efeee261740 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cfc719410 0x561cfc736c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.202+0000 7efeee261740 1 --2- 172.21.15.133:0/3386893761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cfc718af0 0x561cfc739890 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.202+0000 7efeee261740 1 -- 172.21.15.133:0/3386893761 >> 172.21.15.133:0/3386893761 conn(0x561cfc715060 msgr2=0x561cfc72ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.205+0000 7efeccff9640 0 client.25856 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:40:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:40:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:40:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:40:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.205+0000 7efeee261740 1 -- 172.21.15.133:0/3386893761 shutdown_connections 2024-09-11T23:40:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.205+0000 7efeee261740 1 -- 172.21.15.133:0/3386893761 wait complete. 2024-09-11T23:40:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.205+0000 7efeee261740 20 client.25856 trim_cache size 0 max 16384 2024-09-11T23:40:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.211+0000 7efeeb3e8640 1 -- 172.21.15.133:0/1295149852 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efed00a2e20 msgr2=0x7efed00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.211+0000 7efeeb3e8640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efed00a2e20 0x7efed00a52e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efedc0ee110 tx=0x7efedc0879b0 comp rx=0 tx=0).stop 2024-09-11T23:40:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.211+0000 7efeeb3e8640 1 -- 172.21.15.133:0/1295149852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41069c0 msgr2=0x7efee419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.211+0000 7efeeb3e8640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41069c0 0x7efee419eae0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efedc005cc0 tx=0x7efedc003f90 comp rx=0 tx=0).stop 2024-09-11T23:40:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.212+0000 7efeeb3e8640 1 -- 172.21.15.133:0/1295149852 shutdown_connections 2024-09-11T23:40:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.212+0000 7efeeb3e8640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efed00a2e20 0x7efed00a52e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.212+0000 7efeeb3e8640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41069c0 0x7efee419eae0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.213+0000 7efeeb3e8640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee4101350 0x7efee41a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.213+0000 7efeeb3e8640 1 --2- 172.21.15.133:0/1295149852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee41009b0 0x7efee41a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.213+0000 7efeeb3e8640 1 -- 172.21.15.133:0/1295149852 >> 172.21.15.133:0/1295149852 conn(0x7efee4077990 msgr2=0x7efee4069900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.217+0000 7efeeb3e8640 1 -- 172.21.15.133:0/1295149852 shutdown_connections 2024-09-11T23:40:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:14.220+0000 7efeeb3e8640 1 -- 172.21.15.133:0/1295149852 wait complete. 2024-09-11T23:40:14.772 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:15.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.764+0000 7f6fe1456640 1 -- 172.21.15.133:0/633680648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc11b140 msgr2=0x7f6fdc11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:15.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.764+0000 7f6fe1456640 1 --2- 172.21.15.133:0/633680648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc11b140 0x7f6fdc11d530 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009ca0 tx=0x7f6fd402a8f0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.764+0000 7f6fe1456640 1 -- 172.21.15.133:0/633680648 shutdown_connections 2024-09-11T23:40:15.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.765+0000 7f6fe1456640 1 --2- 172.21.15.133:0/633680648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc11b140 0x7f6fdc11d530 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.765+0000 7f6fe1456640 1 --2- 172.21.15.133:0/633680648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fdc06ea30 0x7f6fdc06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.765+0000 7f6fe1456640 1 --2- 172.21.15.133:0/633680648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc110190 0x7f6fdc06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.765+0000 7f6fe1456640 1 -- 172.21.15.133:0/633680648 >> 172.21.15.133:0/633680648 conn(0x7f6fdc073180 msgr2=0x7f6fdc073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:15.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.765+0000 7f6fe1456640 1 -- 172.21.15.133:0/633680648 shutdown_connections 2024-09-11T23:40:15.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.765+0000 7f6fe1456640 1 -- 172.21.15.133:0/633680648 wait complete. 2024-09-11T23:40:15.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.771+0000 7f6fe1456640 1 Processor -- start 2024-09-11T23:40:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.772+0000 7f6fe1456640 1 -- start start 2024-09-11T23:40:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.773+0000 7f6fe1456640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fdc06ea30 0x7f6fdc11a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.773+0000 7f6fe1456640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc110190 0x7f6fdc113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.773+0000 7f6fe1456640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc11b140 0x7f6fdc113cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.773+0000 7f6fe1456640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc120b90 con 0x7f6fdc110190 2024-09-11T23:40:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.773+0000 7f6fe1456640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc120a10 con 0x7f6fdc11b140 2024-09-11T23:40:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.773+0000 7f6fe1456640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc120d10 con 0x7f6fdc06ea30 2024-09-11T23:40:15.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.775+0000 7f6fe0c55640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc11b140 0x7f6fdc113cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.775+0000 7f6fe0c55640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc11b140 0x7f6fdc113cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48826/0 (socket says 172.21.15.133:48826) 2024-09-11T23:40:15.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.775+0000 7f6fe0c55640 1 -- 172.21.15.133:0/1336428323 learned_addr learned my addr 172.21.15.133:0/1336428323 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:15.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.776+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fdc06ea30 0x7f6fdc11a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.776+0000 7f6fdb7fe640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc110190 0x7f6fdc113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.776+0000 7f6fe0c55640 1 -- 172.21.15.133:0/1336428323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fdc06ea30 msgr2=0x7f6fdc11a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:15.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.776+0000 7f6fe0c55640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fdc06ea30 0x7f6fdc11a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.777+0000 7f6fe0c55640 1 -- 172.21.15.133:0/1336428323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc110190 msgr2=0x7f6fdc113790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.777+0000 7f6fe0c55640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc110190 0x7f6fdc113790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.777+0000 7f6fe0c55640 1 -- 172.21.15.133:0/1336428323 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc114290 con 0x7f6fdc11b140 2024-09-11T23:40:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.777+0000 7f6fe0c55640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc11b140 0x7f6fdc113cd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4005e30 tx=0x7f6fd4031720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.779+0000 7f6fd97fa640 1 -- 172.21.15.133:0/1336428323 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd40318d0 con 0x7f6fdc11b140 2024-09-11T23:40:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.779+0000 7f6fe1456640 1 -- 172.21.15.133:0/1336428323 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc114520 con 0x7f6fdc11b140 2024-09-11T23:40:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.779+0000 7f6fe1456640 1 -- 172.21.15.133:0/1336428323 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc118510 con 0x7f6fdc11b140 2024-09-11T23:40:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.780+0000 7f6fd97fa640 1 -- 172.21.15.133:0/1336428323 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd402d070 con 0x7f6fdc11b140 2024-09-11T23:40:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.780+0000 7f6fd97fa640 1 -- 172.21.15.133:0/1336428323 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4046480 con 0x7f6fdc11b140 2024-09-11T23:40:15.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.790+0000 7f6fd97fa640 1 -- 172.21.15.133:0/1336428323 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6fd403b400 con 0x7f6fdc11b140 2024-09-11T23:40:15.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.792+0000 7f6fd97fa640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6fc40a2f00 0x7f6fc40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.793+0000 7f6fd97fa640 1 -- 172.21.15.133:0/1336428323 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6fd40f1b30 con 0x7f6fdc11b140 2024-09-11T23:40:15.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.794+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6fc40a2f00 0x7f6fc40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.796+0000 7f6fe4a91740 1 Processor -- start 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.796+0000 7f6fe4a91740 1 -- start start 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.797+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d0f7cf0 0x55f78d0f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.798+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f6780 0x55f78d0f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.798+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d101520 0x55f78d103910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.798+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f78d05c2f0 con 0x55f78d0f7cf0 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.798+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f78d003cf0 con 0x55f78d0f6780 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.798+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f78d0582b0 con 0x55f78d101520 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.798+0000 7f6fdb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f6780 0x55f78d0f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.798+0000 7f6fdb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f6780 0x55f78d0f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48840/0 (socket says 172.21.15.133:48840) 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.798+0000 7f6fdb7fe640 1 -- 172.21.15.133:0/3113687650 learned_addr learned my addr 172.21.15.133:0/3113687650 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.798+0000 7f6fe0c55640 1 --2- 172.21.15.133:0/3113687650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d101520 0x55f78d103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.799+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6fc40a2f00 0x7f6fc40a53c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000bb60 tx=0x7f6fd000b390 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:15.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.799+0000 7f6fe0c55640 1 -- 172.21.15.133:0/3113687650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f6780 msgr2=0x55f78d0f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:15.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.799+0000 7f6fe0c55640 1 --2- 172.21.15.133:0/3113687650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f6780 0x55f78d0f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.799+0000 7f6fe0c55640 1 -- 172.21.15.133:0/3113687650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d0f7cf0 msgr2=0x55f78d0f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:15.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.799+0000 7f6fe0c55640 1 --2- 172.21.15.133:0/3113687650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d0f7cf0 0x55f78d0f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.799+0000 7f6fe0c55640 1 -- 172.21.15.133:0/3113687650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78cfe83a0 con 0x55f78d101520 2024-09-11T23:40:15.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.801+0000 7f6fe0c55640 1 --2- 172.21.15.133:0/3113687650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d101520 0x55f78d103910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40ef3a0 tx=0x7f6fd402a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e872053689a425d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:15.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.804+0000 7f6fc27fc640 1 -- 172.21.15.133:0/3113687650 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd40c2480 con 0x55f78d101520 2024-09-11T23:40:15.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.804+0000 7f6fc27fc640 1 -- 172.21.15.133:0/3113687650 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd4036030 con 0x55f78d101520 2024-09-11T23:40:15.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.805+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3113687650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d101520 msgr2=0x55f78d103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:15.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.805+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3113687650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d101520 0x55f78d103910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40ef3a0 tx=0x7f6fd402a8f0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.806+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3113687650 shutdown_connections 2024-09-11T23:40:15.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.806+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3113687650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d101520 0x55f78d103910 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.806+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3113687650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f6780 0x55f78d0f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.806+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3113687650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d0f7cf0 0x55f78d0f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.806+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3113687650 >> 172.21.15.133:0/3113687650 conn(0x55f78d0bf310 msgr2=0x55f78d0f2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.815+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3113687650 shutdown_connections 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.815+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3113687650 wait complete. 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.816+0000 7f6fe4a91740 1 -- start start 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.816+0000 7f6fe4a91740 1 Processor -- start 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.817+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d1035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.817+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f7cf0 0x55f78d100880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.817+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d100dc0 0x55f78d101240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.817+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f78d103fe0 con 0x55f78d100dc0 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.817+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f78d017060 con 0x55f78d0f7cf0 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.817+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f78d104160 con 0x55f78d0f6780 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.817+0000 7f6fe4a91740 10 client.0 _ll_register_callbacks cb 0x55f78cd6a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.819+0000 7f6fdbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d1035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.819+0000 7f6fdbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d1035d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:39086/0 (socket says 172.21.15.133:39086) 2024-09-11T23:40:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.819+0000 7f6fdbfff640 1 -- 172.21.15.133:0/3706835715 learned_addr learned my addr 172.21.15.133:0/3706835715 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.819+0000 7f6fe0c55640 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d100dc0 0x55f78d101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.828+0000 7f6fdb7fe640 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f7cf0 0x55f78d100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.830+0000 7f6fdbfff640 1 -- 172.21.15.133:0/3706835715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f7cf0 msgr2=0x55f78d100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.830+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f7cf0 0x55f78d100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.830+0000 7f6fdbfff640 1 -- 172.21.15.133:0/3706835715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d100dc0 msgr2=0x55f78d101240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.830+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d100dc0 0x55f78d101240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.830+0000 7f6fdbfff640 1 -- 172.21.15.133:0/3706835715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78cdcd3c0 con 0x55f78d0f6780 2024-09-11T23:40:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.830+0000 7f6fdb7fe640 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f7cf0 0x55f78d100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:40:15.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.832+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d1035d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000c010 tx=0x7f6fd00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=472d15ed2541ed1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:15.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.832+0000 7f6fbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:15.833 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:15 smithi133 ceph-mon[31430]: pgmap v107: 97 pgs: 97 active+clean; 24 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 613 MiB/s wr, 3.17k op/s 2024-09-11T23:40:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.832+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/3706835715 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0005220 con 0x55f78d0f6780 2024-09-11T23:40:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.833+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/3706835715 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6fd00053c0 con 0x55f78d0f6780 2024-09-11T23:40:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.834+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3706835715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f78d069860 con 0x55f78d0f6780 2024-09-11T23:40:15.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.837+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/3706835715 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6fd00055c0 con 0x55f78d0f6780 2024-09-11T23:40:15.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.837+0000 7f6fe4a91740 10 client.25840 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:15.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.837+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3706835715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f78d131cd0 con 0x55f78d0f6780 2024-09-11T23:40:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.839+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/3706835715 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f78d069860 con 0x55f78d0f6780 2024-09-11T23:40:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.839+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/3706835715 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6fd0030420 con 0x55f78d0f6780 2024-09-11T23:40:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.839+0000 7f6fe4a91740 10 client.25840 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.839+0000 7f6fe4a91740 10 client.25840 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:40:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.839+0000 7f6fe4a91740 20 client.25840 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.839+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f78d133660 0x55f78d135ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.839+0000 7f6fe4a91740 4 client.25840 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.839+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3706835715 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f78d063d70 con 0x55f78d133660 2024-09-11T23:40:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.840+0000 7f6fdb7fe640 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f78d133660 0x55f78d135ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.843+0000 7f6fdb7fe640 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f78d133660 0x55f78d135ac0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42f41be7c4d8d2b1 server_cookie=9ab3481675bbd81 in_seq=0 out_seq=0 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.843+0000 7f6fbb7fe640 10 client.25840 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.844+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/3706835715 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f78d063d70 con 0x55f78d133660 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.844+0000 7f6fbb7fe640 10 client.25840 handle_command_reply: tid=0 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.844+0000 7f6fe4a91740 1 client.25840 shutdown 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.845+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3706835715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 msgr2=0x55f78d1035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.845+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d1035d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000c010 tx=0x7f6fd00046c0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.845+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3706835715 shutdown_connections 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.845+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f78d133660 0x55f78d135ac0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.845+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d100dc0 0x55f78d101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.845+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d0f7cf0 0x55f78d100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.845+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3706835715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d1035d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.845+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3706835715 >> 172.21.15.133:0/3706835715 conn(0x55f78d0bf310 msgr2=0x55f78d0f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.845+0000 7f6fbb7fe640 0 client.25840 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:40:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.846+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3706835715 shutdown_connections 2024-09-11T23:40:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.846+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3706835715 wait complete. 2024-09-11T23:40:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.846+0000 7f6fe4a91740 20 client.25840 trim_cache size 0 max 16384 2024-09-11T23:40:15.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.876+0000 7f6fe4a91740 1 Processor -- start 2024-09-11T23:40:15.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.876+0000 7f6fe4a91740 1 -- start start 2024-09-11T23:40:15.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.876+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d0f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.876+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d0f7cf0 0x55f78d0f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.876+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d100dc0 0x55f78d13b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.876+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800e6e0 con 0x55f78d0f7cf0 2024-09-11T23:40:15.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.876+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8010220 con 0x55f78d100dc0 2024-09-11T23:40:15.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.876+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800edb0 con 0x55f78d0f6780 2024-09-11T23:40:15.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.877+0000 7f6fdbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d0f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.877+0000 7f6fdbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d0f1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:39090/0 (socket says 172.21.15.133:39090) 2024-09-11T23:40:15.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.877+0000 7f6fdbfff640 1 -- 172.21.15.133:0/3579739722 learned_addr learned my addr 172.21.15.133:0/3579739722 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:15.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.878+0000 7f6fdb7fe640 1 --2- 172.21.15.133:0/3579739722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d0f7cf0 0x55f78d0f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.878+0000 7f6fe0c55640 1 --2- 172.21.15.133:0/3579739722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d100dc0 0x55f78d13b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.878+0000 7f6fdbfff640 1 -- 172.21.15.133:0/3579739722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d100dc0 msgr2=0x55f78d13b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:15.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.878+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/3579739722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d100dc0 0x55f78d13b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.878+0000 7f6fdbfff640 1 -- 172.21.15.133:0/3579739722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d0f7cf0 msgr2=0x55f78d0f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:15.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.878+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/3579739722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d0f7cf0 0x55f78d0f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.878+0000 7f6fdbfff640 1 -- 172.21.15.133:0/3579739722 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78d103940 con 0x55f78d0f6780 2024-09-11T23:40:15.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.885+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/3579739722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d0f1da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000bfe0 tx=0x7f6fd0005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d9dd112319d0202 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:15.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.886+0000 7f6fc1ffb640 1 -- 172.21.15.133:0/3579739722 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0030620 con 0x55f78d0f6780 2024-09-11T23:40:15.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.886+0000 7f6fc1ffb640 1 -- 172.21.15.133:0/3579739722 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd001e6f0 con 0x55f78d0f6780 2024-09-11T23:40:15.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.888+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3579739722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 msgr2=0x55f78d0f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:15.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.888+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3579739722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d0f1da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000bfe0 tx=0x7f6fd0005870 comp rx=0 tx=0).stop 2024-09-11T23:40:15.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.888+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3579739722 shutdown_connections 2024-09-11T23:40:15.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.888+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3579739722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d100dc0 0x55f78d13b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.888+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3579739722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d0f7cf0 0x55f78d0f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.888+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/3579739722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f6780 0x55f78d0f1da0 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000bfe0 tx=0x7f6fd0005870 comp rx=0 tx=0).stop 2024-09-11T23:40:15.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.888+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3579739722 >> 172.21.15.133:0/3579739722 conn(0x55f78d0bf310 msgr2=0x55f78d0f1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.893+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3579739722 shutdown_connections 2024-09-11T23:40:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.893+0000 7f6fe4a91740 1 -- 172.21.15.133:0/3579739722 wait complete. 2024-09-11T23:40:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.894+0000 7f6fe4a91740 1 -- start start 2024-09-11T23:40:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.894+0000 7f6fe4a91740 1 Processor -- start 2024-09-11T23:40:15.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.895+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f7cf0 0x55f78d106080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.895+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d100dc0 0x55f78d1065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.895+0000 7f6fe4a91740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d106b00 0x55f78d19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.895+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800fb50 con 0x55f78d106b00 2024-09-11T23:40:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.895+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800f480 con 0x55f78d100dc0 2024-09-11T23:40:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.895+0000 7f6fe4a91740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc800c6c0 con 0x55f78d0f7cf0 2024-09-11T23:40:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.895+0000 7f6fe4a91740 10 client.0 _ll_register_callbacks cb 0x7f6fd000b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.898+0000 7f6fe0c55640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d106b00 0x55f78d19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.898+0000 7f6fe0c55640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d106b00 0x55f78d19c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35754/0 (socket says 172.21.15.133:35754) 2024-09-11T23:40:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.898+0000 7f6fe0c55640 1 -- 172.21.15.133:0/599174084 learned_addr learned my addr 172.21.15.133:0/599174084 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:15.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.898+0000 7f6fdb7fe640 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d100dc0 0x55f78d1065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.898+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f7cf0 0x55f78d106080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.903+0000 7f6fdbfff640 1 -- 172.21.15.133:0/599174084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d100dc0 msgr2=0x55f78d1065c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:15.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.903+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d100dc0 0x55f78d1065c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.903+0000 7f6fdbfff640 1 -- 172.21.15.133:0/599174084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d106b00 msgr2=0x55f78d19c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:15.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.903+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d106b00 0x55f78d19c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.903+0000 7f6fdbfff640 1 -- 172.21.15.133:0/599174084 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78d19c8e0 con 0x55f78d0f7cf0 2024-09-11T23:40:15.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.907+0000 7f6fdbfff640 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f7cf0 0x55f78d106080 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00308c0 tx=0x7f6fd0030ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26dac2fbad03ba5a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:15.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.907+0000 7f6fbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.912+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/599174084 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd002fca0 con 0x55f78d0f7cf0 2024-09-11T23:40:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.912+0000 7f6fe4a91740 1 -- 172.21.15.133:0/599174084 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f78cfe0170 con 0x55f78d0f7cf0 2024-09-11T23:40:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.912+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/599174084 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6fd001e6f0 con 0x55f78d0f7cf0 2024-09-11T23:40:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.913+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/599174084 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6fd0030e50 con 0x55f78d0f7cf0 2024-09-11T23:40:15.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.915+0000 7f6fe4a91740 10 client.25846 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:15.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.915+0000 7f6fe4a91740 1 -- 172.21.15.133:0/599174084 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f78d19dae0 con 0x55f78d0f7cf0 2024-09-11T23:40:15.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.916+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/599174084 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f78cfe0170 con 0x55f78d0f7cf0 2024-09-11T23:40:15.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.918+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/599174084 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6fd00437f0 con 0x55f78d0f7cf0 2024-09-11T23:40:15.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.918+0000 7f6fe4a91740 10 client.25846 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:15.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.918+0000 7f6fe4a91740 10 client.25846 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:40:15.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.918+0000 7f6fe4a91740 20 client.25846 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:15.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.918+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f78d19e150 0x55f78d1a0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:15.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.918+0000 7f6fe4a91740 4 client.25846 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:40:15.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.918+0000 7f6fe4a91740 1 -- 172.21.15.133:0/599174084 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f78d02d9b0 con 0x55f78d19e150 2024-09-11T23:40:15.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.924+0000 7f6fdb7fe640 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f78d19e150 0x55f78d1a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:15.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.927+0000 7f6fdb7fe640 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f78d19e150 0x55f78d1a0540 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7422e37fec82025f server_cookie=c9a32160c33f66b6 in_seq=0 out_seq=0 2024-09-11T23:40:15.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.928+0000 7f6fbb7fe640 10 client.25846 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:40:15.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.930+0000 7f6fbb7fe640 1 -- 172.21.15.133:0/599174084 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f78d02d9b0 con 0x55f78d19e150 2024-09-11T23:40:15.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.930+0000 7f6fbb7fe640 10 client.25846 handle_command_reply: tid=0 2024-09-11T23:40:15.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.930+0000 7f6fe4a91740 1 client.25846 shutdown 2024-09-11T23:40:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.936+0000 7f6fe4a91740 1 -- 172.21.15.133:0/599174084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f7cf0 msgr2=0x55f78d106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.936+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f7cf0 0x55f78d106080 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00308c0 tx=0x7f6fd0030ce0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.936+0000 7f6fe4a91740 1 -- 172.21.15.133:0/599174084 shutdown_connections 2024-09-11T23:40:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.936+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f78d19e150 0x55f78d1a0540 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.936+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f78d106b00 0x55f78d19c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.936+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f78d100dc0 0x55f78d1065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.936+0000 7f6fe4a91740 1 --2- 172.21.15.133:0/599174084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f78d0f7cf0 0x55f78d106080 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00308c0 tx=0x7f6fd0030ce0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.936+0000 7f6fe4a91740 1 -- 172.21.15.133:0/599174084 >> 172.21.15.133:0/599174084 conn(0x55f78d0bf310 msgr2=0x55f78d0f1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:15.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.936+0000 7f6fbb7fe640 0 client.25846 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:40:15.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.936+0000 7f6fe4a91740 1 -- 172.21.15.133:0/599174084 shutdown_connections 2024-09-11T23:40:15.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.936+0000 7f6fe4a91740 1 -- 172.21.15.133:0/599174084 wait complete. 2024-09-11T23:40:15.942 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:40:15.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.941+0000 7f6fe4a91740 20 client.25846 trim_cache size 0 max 16384 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.943+0000 7f6fe1456640 1 -- 172.21.15.133:0/1336428323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6fc40a2f00 msgr2=0x7f6fc40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.943+0000 7f6fe1456640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6fc40a2f00 0x7f6fc40a53c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000bb60 tx=0x7f6fd000b390 comp rx=0 tx=0).stop 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.943+0000 7f6fe1456640 1 -- 172.21.15.133:0/1336428323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc11b140 msgr2=0x7f6fdc113cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.943+0000 7f6fe1456640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc11b140 0x7f6fdc113cd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4005e30 tx=0x7f6fd4031720 comp rx=0 tx=0).stop 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.943+0000 7f6fe1456640 1 -- 172.21.15.133:0/1336428323 shutdown_connections 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.943+0000 7f6fe1456640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6fc40a2f00 0x7f6fc40a53c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.943+0000 7f6fe1456640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fdc11b140 0x7f6fdc113cd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.943+0000 7f6fe1456640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc110190 0x7f6fdc113790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.944+0000 7f6fe1456640 1 --2- 172.21.15.133:0/1336428323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fdc06ea30 0x7f6fdc11a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.944+0000 7f6fe1456640 1 -- 172.21.15.133:0/1336428323 >> 172.21.15.133:0/1336428323 conn(0x7f6fdc073180 msgr2=0x7f6fdc11b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.944+0000 7f6fe1456640 1 -- 172.21.15.133:0/1336428323 shutdown_connections 2024-09-11T23:40:15.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:15.944+0000 7f6fe1456640 1 -- 172.21.15.133:0/1336428323 wait complete. 2024-09-11T23:40:15.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:15 smithi138 ceph-mon[35142]: pgmap v107: 97 pgs: 97 active+clean; 24 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 613 MiB/s wr, 3.17k op/s 2024-09-11T23:40:16.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:15 smithi178 ceph-mon[35882]: pgmap v107: 97 pgs: 97 active+clean; 24 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 613 MiB/s wr, 3.17k op/s 2024-09-11T23:40:17.022 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:40:17.842 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:18.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:17 smithi133 ceph-mon[31430]: pgmap v108: 97 pgs: 97 active+clean; 24 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 545 MiB/s wr, 2.63k op/s 2024-09-11T23:40:18.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:17 smithi138 ceph-mon[35142]: pgmap v108: 97 pgs: 97 active+clean; 24 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 545 MiB/s wr, 2.63k op/s 2024-09-11T23:40:18.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:17 smithi178 ceph-mon[35882]: pgmap v108: 97 pgs: 97 active+clean; 24 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 545 MiB/s wr, 2.63k op/s 2024-09-11T23:40:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.467+0000 7f735a928640 1 -- 172.21.15.133:0/3981754305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7354101df0 msgr2=0x7f73541021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.467+0000 7f735a928640 1 --2- 172.21.15.133:0/3981754305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7354101df0 0x7f73541021d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7344009900 tx=0x7f734402a410 comp rx=0 tx=0).stop 2024-09-11T23:40:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.467+0000 7f735a928640 1 -- 172.21.15.133:0/3981754305 shutdown_connections 2024-09-11T23:40:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.467+0000 7f735a928640 1 --2- 172.21.15.133:0/3981754305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7354106f20 0x7f735410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.467+0000 7f735a928640 1 --2- 172.21.15.133:0/3981754305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73541027a0 0x7f73541067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.467+0000 7f735a928640 1 --2- 172.21.15.133:0/3981754305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7354101df0 0x7f73541021d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.467+0000 7f735a928640 1 -- 172.21.15.133:0/3981754305 >> 172.21.15.133:0/3981754305 conn(0x7f73540fd8f0 msgr2=0x7f73540ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.468+0000 7f735a928640 1 -- 172.21.15.133:0/3981754305 shutdown_connections 2024-09-11T23:40:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.468+0000 7f735a928640 1 -- 172.21.15.133:0/3981754305 wait complete. 2024-09-11T23:40:18.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.468+0000 7f735a928640 1 Processor -- start 2024-09-11T23:40:18.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.468+0000 7f735a928640 1 -- start start 2024-09-11T23:40:18.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.469+0000 7f735a928640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7354101df0 0x7f73540744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.469+0000 7f735a928640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73541027a0 0x7f7354074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.469+0000 7f735a928640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7354106f20 0x7f7354074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.469+0000 7f7353fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7354101df0 0x7f73540744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.469+0000 7f7353fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7354101df0 0x7f73540744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35782/0 (socket says 172.21.15.133:35782) 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.469+0000 7f7353fff640 1 -- 172.21.15.133:0/567299982 learned_addr learned my addr 172.21.15.133:0/567299982 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.469+0000 7f735a928640 1 -- 172.21.15.133:0/567299982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7354112040 con 0x7f7354101df0 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.469+0000 7f735a928640 1 -- 172.21.15.133:0/567299982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7354111ec0 con 0x7f7354106f20 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.469+0000 7f735a928640 1 -- 172.21.15.133:0/567299982 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73541121c0 con 0x7f73541027a0 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.470+0000 7f7353fff640 1 -- 172.21.15.133:0/567299982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73541027a0 msgr2=0x7f7354074a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.470+0000 7f7353fff640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73541027a0 0x7f7354074a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.470+0000 7f7353fff640 1 -- 172.21.15.133:0/567299982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7354106f20 msgr2=0x7f7354074f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.470+0000 7f7353fff640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7354106f20 0x7f7354074f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.470+0000 7f7353fff640 1 -- 172.21.15.133:0/567299982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73541a6a80 con 0x7f7354101df0 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.470+0000 7f7353fff640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7354101df0 0x7f73540744f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7344031550 tx=0x7f7344004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.470+0000 7f7351ffb640 1 -- 172.21.15.133:0/567299982 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734403f030 con 0x7f7354101df0 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.470+0000 7f7351ffb640 1 -- 172.21.15.133:0/567299982 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7344003ce0 con 0x7f7354101df0 2024-09-11T23:40:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.470+0000 7f735a928640 1 -- 172.21.15.133:0/567299982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73541a6d10 con 0x7f7354101df0 2024-09-11T23:40:18.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.470+0000 7f7351ffb640 1 -- 172.21.15.133:0/567299982 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344035070 con 0x7f7354101df0 2024-09-11T23:40:18.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.470+0000 7f735a928640 1 -- 172.21.15.133:0/567299982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73541a7180 con 0x7f7354101df0 2024-09-11T23:40:18.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.473+0000 7f7351ffb640 1 -- 172.21.15.133:0/567299982 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7344042030 con 0x7f7354101df0 2024-09-11T23:40:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.474+0000 7f7351ffb640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73400a2d70 0x7f73400a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.474+0000 7f735cd01740 1 Processor -- start 2024-09-11T23:40:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.474+0000 7f735cd01740 1 -- start start 2024-09-11T23:40:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.474+0000 7f7351ffb640 1 -- 172.21.15.133:0/567299982 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f73440efe30 con 0x7f7354101df0 2024-09-11T23:40:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.474+0000 7f734bfff640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73400a2d70 0x7f73400a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.474+0000 7f735cd01740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 0x562c895b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.474+0000 7f735cd01740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895b05d0 0x562c895b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.474+0000 7f735cd01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.474+0000 7f735cd01740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562c895162f0 con 0x562c895b1b40 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.474+0000 7f735cd01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562c894bdcf0 con 0x562c895bb370 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.474+0000 7f735cd01740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c895122b0 con 0x562c895b05d0 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.475+0000 7f7358ec5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.475+0000 7f7358ec5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895bd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48896/0 (socket says 172.21.15.133:48896) 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.475+0000 7f7358ec5640 1 -- 172.21.15.133:0/151645022 learned_addr learned my addr 172.21.15.133:0/151645022 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.475+0000 7f734bfff640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73400a2d70 0x7f73400a5230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7354075f80 tx=0x7f733c006c20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.475+0000 7f7353fff640 1 --2- 172.21.15.133:0/151645022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 0x562c895b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.475+0000 7f7358ec5640 1 -- 172.21.15.133:0/151645022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895b05d0 msgr2=0x562c895b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:18.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.475+0000 7f7358ec5640 1 --2- 172.21.15.133:0/151645022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895b05d0 0x562c895b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.475+0000 7f7358ec5640 1 -- 172.21.15.133:0/151645022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 msgr2=0x562c895b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:18.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.475+0000 7f7358ec5640 1 --2- 172.21.15.133:0/151645022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 0x562c895b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.475+0000 7f7358ec5640 1 -- 172.21.15.133:0/151645022 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c894a23a0 con 0x562c895bb370 2024-09-11T23:40:18.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.476+0000 7f7353fff640 1 --2- 172.21.15.133:0/151645022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 0x562c895b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:18.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.476+0000 7f7358ec5640 1 --2- 172.21.15.133:0/151645022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895bd760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f734c009d20 tx=0x7f734c00b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa93fe0429fce4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:18.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.476+0000 7f734a7fc640 1 -- 172.21.15.133:0/151645022 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c00bd30 con 0x562c895bb370 2024-09-11T23:40:18.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.477+0000 7f734a7fc640 1 -- 172.21.15.133:0/151645022 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f734c028070 con 0x562c895bb370 2024-09-11T23:40:18.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.477+0000 7f734a7fc640 1 -- 172.21.15.133:0/151645022 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c0227a0 con 0x562c895bb370 2024-09-11T23:40:18.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.478+0000 7f735cd01740 1 -- 172.21.15.133:0/151645022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 msgr2=0x562c895bd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:18.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.478+0000 7f735cd01740 1 --2- 172.21.15.133:0/151645022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895bd760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f734c009d20 tx=0x7f734c00b420 comp rx=0 tx=0).stop 2024-09-11T23:40:18.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.478+0000 7f735cd01740 1 -- 172.21.15.133:0/151645022 shutdown_connections 2024-09-11T23:40:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.478+0000 7f735cd01740 1 --2- 172.21.15.133:0/151645022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895bd760 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.478+0000 7f735cd01740 1 --2- 172.21.15.133:0/151645022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895b05d0 0x562c895b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.478+0000 7f735cd01740 1 --2- 172.21.15.133:0/151645022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 0x562c895b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.478+0000 7f735cd01740 1 -- 172.21.15.133:0/151645022 >> 172.21.15.133:0/151645022 conn(0x562c89579310 msgr2=0x562c895acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.478+0000 7f735cd01740 1 -- 172.21.15.133:0/151645022 shutdown_connections 2024-09-11T23:40:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.478+0000 7f735cd01740 1 -- 172.21.15.133:0/151645022 wait complete. 2024-09-11T23:40:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.479+0000 7f735cd01740 1 -- start start 2024-09-11T23:40:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.479+0000 7f735cd01740 1 Processor -- start 2024-09-11T23:40:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.480+0000 7f735cd01740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b05d0 0x562c895bad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.480+0000 7f735cd01740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895b1b40 0x562c895b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.480+0000 7f735cd01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.480+0000 7f7353fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b05d0 0x562c895bad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.480+0000 7f735cd01740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562c895be110 con 0x562c895b05d0 2024-09-11T23:40:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.480+0000 7f735cd01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562c894d1060 con 0x562c895bb370 2024-09-11T23:40:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.480+0000 7f735cd01740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c895be290 con 0x562c895b1b40 2024-09-11T23:40:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.480+0000 7f7353fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b05d0 0x562c895bad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35800/0 (socket says 172.21.15.133:35800) 2024-09-11T23:40:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.480+0000 7f7353fff640 1 -- 172.21.15.133:0/1996206944 learned_addr learned my addr 172.21.15.133:0/1996206944 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.481+0000 7f735cd01740 10 client.0 _ll_register_callbacks cb 0x562c892248c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.481+0000 7f7353fff640 1 -- 172.21.15.133:0/1996206944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895b1b40 msgr2=0x562c895b7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.481+0000 7f7353fff640 1 --2- 172.21.15.133:0/1996206944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895b1b40 0x562c895b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.481+0000 7f7353fff640 1 -- 172.21.15.133:0/1996206944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 msgr2=0x562c895b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.481+0000 7f7353fff640 1 --2- 172.21.15.133:0/1996206944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.481+0000 7f7353fff640 1 -- 172.21.15.133:0/1996206944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c892873c0 con 0x562c895b05d0 2024-09-11T23:40:18.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.481+0000 7f7353fff640 1 --2- 172.21.15.133:0/1996206944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b05d0 0x562c895bad20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f73440b87c0 tx=0x7f73440bc1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b471e3187023f4dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:18.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.481+0000 7f73337fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:40:18.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.481+0000 7f73337fe640 1 -- 172.21.15.133:0/1996206944 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344049040 con 0x562c895b05d0 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.482+0000 7f73337fe640 1 -- 172.21.15.133:0/1996206944 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f73440d5240 con 0x562c895b05d0 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.482+0000 7f735cd01740 1 -- 172.21.15.133:0/1996206944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f734c028070 con 0x562c895b05d0 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.482+0000 7f73337fe640 1 -- 172.21.15.133:0/1996206944 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73440bca60 con 0x562c895b05d0 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.482+0000 7f735cd01740 10 client.16443 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.482+0000 7f735cd01740 1 -- 172.21.15.133:0/1996206944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c895c9b00 con 0x562c895b05d0 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.483+0000 7f73337fe640 1 -- 172.21.15.133:0/1996206944 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f734c028070 con 0x562c895b05d0 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.483+0000 7f73337fe640 1 -- 172.21.15.133:0/1996206944 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f73440edce0 con 0x562c895b05d0 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.483+0000 7f735cd01740 10 client.16443 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.483+0000 7f735cd01740 10 client.16443 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.483+0000 7f735cd01740 20 client.16443 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.483+0000 7f735cd01740 1 --2- 172.21.15.133:0/1996206944 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c895caef0 0x562c895cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.483+0000 7f735cd01740 4 client.16443 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.483+0000 7f735cd01740 1 -- 172.21.15.133:0/1996206944 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c89523860 con 0x562c895caef0 2024-09-11T23:40:18.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.483+0000 7f734bfff640 1 --2- 172.21.15.133:0/1996206944 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c895caef0 0x562c895cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.484+0000 7f734bfff640 1 --2- 172.21.15.133:0/1996206944 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c895caef0 0x562c895cd340 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a717727b4c5ec123 server_cookie=f0d33c4cb4ee5cc6 in_seq=0 out_seq=0 2024-09-11T23:40:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.484+0000 7f73337fe640 10 client.16443 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:40:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.485+0000 7f73337fe640 1 -- 172.21.15.133:0/1996206944 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c89523860 con 0x562c895caef0 2024-09-11T23:40:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.485+0000 7f73337fe640 10 client.16443 handle_command_reply: tid=0 2024-09-11T23:40:18.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.486+0000 7f735cd01740 1 client.16443 shutdown 2024-09-11T23:40:18.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.486+0000 7f735cd01740 1 -- 172.21.15.133:0/1996206944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b05d0 msgr2=0x562c895bad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:18.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.486+0000 7f735cd01740 1 --2- 172.21.15.133:0/1996206944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b05d0 0x562c895bad20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f73440b87c0 tx=0x7f73440bc1f0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.486+0000 7f735cd01740 1 -- 172.21.15.133:0/1996206944 shutdown_connections 2024-09-11T23:40:18.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.486+0000 7f735cd01740 1 --2- 172.21.15.133:0/1996206944 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c895caef0 0x562c895cd340 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.486+0000 7f73337fe640 0 client.16443 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:40:18.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.486+0000 7f735cd01740 1 --2- 172.21.15.133:0/1996206944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.486+0000 7f735cd01740 1 --2- 172.21.15.133:0/1996206944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895b1b40 0x562c895b7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.486+0000 7f735cd01740 1 --2- 172.21.15.133:0/1996206944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b05d0 0x562c895bad20 secure :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f73440b87c0 tx=0x7f73440bc1f0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.486+0000 7f735cd01740 1 -- 172.21.15.133:0/1996206944 >> 172.21.15.133:0/1996206944 conn(0x562c89579310 msgr2=0x562c895b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:18.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.488+0000 7f735cd01740 1 -- 172.21.15.133:0/1996206944 shutdown_connections 2024-09-11T23:40:18.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.490+0000 7f735cd01740 1 -- 172.21.15.133:0/1996206944 wait complete. 2024-09-11T23:40:18.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.490+0000 7f735cd01740 20 client.16443 trim_cache size 0 max 16384 2024-09-11T23:40:18.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.527+0000 7f735cd01740 1 Processor -- start 2024-09-11T23:40:18.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.527+0000 7f735cd01740 1 -- start start 2024-09-11T23:40:18.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.528+0000 7f735cd01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895b1b40 0x562c895ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.529+0000 7f735cd01740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895bb370 0x562c895c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.529+0000 7f735cd01740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895caef0 0x562c895d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.529+0000 7f735cd01740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73541a73b0 con 0x562c895bb370 2024-09-11T23:40:18.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.529+0000 7f735cd01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73541a8e20 con 0x562c895b1b40 2024-09-11T23:40:18.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.529+0000 7f735cd01740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73541a79b0 con 0x562c895caef0 2024-09-11T23:40:18.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.530+0000 7f7353fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895b1b40 0x562c895ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.530+0000 7f734bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895bb370 0x562c895c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.530+0000 7f734bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895bb370 0x562c895c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35814/0 (socket says 172.21.15.133:35814) 2024-09-11T23:40:18.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.530+0000 7f734bfff640 1 -- 172.21.15.133:0/1144671992 learned_addr learned my addr 172.21.15.133:0/1144671992 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:18.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.532+0000 7f7358ec5640 1 --2- 172.21.15.133:0/1144671992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895caef0 0x562c895d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.532+0000 7f734bfff640 1 -- 172.21.15.133:0/1144671992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895caef0 msgr2=0x562c895d2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.532+0000 7f734bfff640 1 --2- 172.21.15.133:0/1144671992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895caef0 0x562c895d2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.532+0000 7f734bfff640 1 -- 172.21.15.133:0/1144671992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895b1b40 msgr2=0x562c895ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.532+0000 7f734bfff640 1 --2- 172.21.15.133:0/1144671992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895b1b40 0x562c895ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.533+0000 7f734bfff640 1 -- 172.21.15.133:0/1144671992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c895b1740 con 0x562c895bb370 2024-09-11T23:40:18.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.535+0000 7f734bfff640 1 --2- 172.21.15.133:0/1144671992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895bb370 0x562c895c73e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f733c006fd0 tx=0x7f733c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b961acf111be1c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:18.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.536+0000 7f7349ffb640 1 -- 172.21.15.133:0/1144671992 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c02c070 con 0x562c895bb370 2024-09-11T23:40:18.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.536+0000 7f7349ffb640 1 -- 172.21.15.133:0/1144671992 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733c0273f0 con 0x562c895bb370 2024-09-11T23:40:18.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.536+0000 7f7349ffb640 1 -- 172.21.15.133:0/1144671992 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c013e60 con 0x562c895bb370 2024-09-11T23:40:18.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.538+0000 7f735cd01740 1 -- 172.21.15.133:0/1144671992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895bb370 msgr2=0x562c895c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:18.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.538+0000 7f735cd01740 1 --2- 172.21.15.133:0/1144671992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895bb370 0x562c895c73e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f733c006fd0 tx=0x7f733c0047d0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.538+0000 7f735cd01740 1 -- 172.21.15.133:0/1144671992 shutdown_connections 2024-09-11T23:40:18.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.538+0000 7f735cd01740 1 --2- 172.21.15.133:0/1144671992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895caef0 0x562c895d2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.538+0000 7f735cd01740 1 --2- 172.21.15.133:0/1144671992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895bb370 0x562c895c73e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.538+0000 7f735cd01740 1 --2- 172.21.15.133:0/1144671992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895b1b40 0x562c895ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.538+0000 7f735cd01740 1 -- 172.21.15.133:0/1144671992 >> 172.21.15.133:0/1144671992 conn(0x562c89579310 msgr2=0x562c895ac700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:18.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.538+0000 7f735cd01740 1 -- 172.21.15.133:0/1144671992 shutdown_connections 2024-09-11T23:40:18.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.540+0000 7f735cd01740 1 -- 172.21.15.133:0/1144671992 wait complete. 2024-09-11T23:40:18.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.541+0000 7f735cd01740 1 -- start start 2024-09-11T23:40:18.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.541+0000 7f735cd01740 1 Processor -- start 2024-09-11T23:40:18.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.542+0000 7f735cd01740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 0x562c895bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.542+0000 7f735cd01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895bf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.542+0000 7f735cd01740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895caef0 0x562c895bff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.542+0000 7f735cd01740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73541a8750 con 0x562c895b1b40 2024-09-11T23:40:18.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.542+0000 7f735cd01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73541a8080 con 0x562c895bb370 2024-09-11T23:40:18.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.542+0000 7f735cd01740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7354072330 con 0x562c895caef0 2024-09-11T23:40:18.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.542+0000 7f735cd01740 10 client.0 _ll_register_callbacks cb 0x7f7344009230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:18.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.543+0000 7f7353fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 0x562c895bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.543+0000 7f7353fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 0x562c895bf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35818/0 (socket says 172.21.15.133:35818) 2024-09-11T23:40:18.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.543+0000 7f7353fff640 1 -- 172.21.15.133:0/2277374325 learned_addr learned my addr 172.21.15.133:0/2277374325 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:18.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.543+0000 7f734bfff640 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895bf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.544+0000 7f7353fff640 1 -- 172.21.15.133:0/2277374325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895caef0 msgr2=0x562c895bff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:18.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.544+0000 7f7353fff640 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895caef0 0x562c895bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.544+0000 7f7353fff640 1 -- 172.21.15.133:0/2277374325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 msgr2=0x562c895bf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:18.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.544+0000 7f7353fff640 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895bf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.544+0000 7f7353fff640 1 -- 172.21.15.133:0/2277374325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8963d630 con 0x562c895b1b40 2024-09-11T23:40:18.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.545+0000 7f7353fff640 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 0x562c895bf490 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f73440b8aa0 tx=0x7f73440058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff93d31885e11f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:18.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.546+0000 7f73337fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:40:18.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.546+0000 7f73337fe640 1 -- 172.21.15.133:0/2277374325 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344049070 con 0x562c895b1b40 2024-09-11T23:40:18.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.546+0000 7f735cd01740 1 -- 172.21.15.133:0/2277374325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f733c0273f0 con 0x562c895b1b40 2024-09-11T23:40:18.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.546+0000 7f73337fe640 1 -- 172.21.15.133:0/2277374325 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f73440bc7d0 con 0x562c895b1b40 2024-09-11T23:40:18.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.546+0000 7f73337fe640 1 -- 172.21.15.133:0/2277374325 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73440bc360 con 0x562c895b1b40 2024-09-11T23:40:18.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.547+0000 7f735cd01740 10 client.16449 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:18.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.547+0000 7f735cd01740 1 -- 172.21.15.133:0/2277374325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c8963e8f0 con 0x562c895b1b40 2024-09-11T23:40:18.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.548+0000 7f73337fe640 1 -- 172.21.15.133:0/2277374325 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733c0273f0 con 0x562c895b1b40 2024-09-11T23:40:18.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.548+0000 7f73337fe640 1 -- 172.21.15.133:0/2277374325 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f734409f020 con 0x562c895b1b40 2024-09-11T23:40:18.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.549+0000 7f735cd01740 10 client.16449 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:18.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.549+0000 7f735cd01740 10 client.16449 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:40:18.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.549+0000 7f735cd01740 20 client.16449 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:18.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.549+0000 7f735cd01740 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c8963ee80 0x562c89641290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:18.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.549+0000 7f735cd01740 4 client.16449 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:40:18.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.549+0000 7f735cd01740 1 -- 172.21.15.133:0/2277374325 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c8951dd70 con 0x562c8963ee80 2024-09-11T23:40:18.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.552+0000 7f734bfff640 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c8963ee80 0x562c89641290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:18.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.555+0000 7f734bfff640 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c8963ee80 0x562c89641290 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3ed0cadf002b854 server_cookie=5f6301cd3c8c1658 in_seq=0 out_seq=0 2024-09-11T23:40:18.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.555+0000 7f73337fe640 10 client.16449 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:40:18.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.558+0000 7f73337fe640 1 -- 172.21.15.133:0/2277374325 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c8951dd70 con 0x562c8963ee80 2024-09-11T23:40:18.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.558+0000 7f73337fe640 10 client.16449 handle_command_reply: tid=0 2024-09-11T23:40:18.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.560+0000 7f735cd01740 1 client.16449 shutdown 2024-09-11T23:40:18.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.560+0000 7f735cd01740 1 -- 172.21.15.133:0/2277374325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 msgr2=0x562c895bf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:18.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.560+0000 7f735cd01740 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 0x562c895bf490 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f73440b8aa0 tx=0x7f73440058b0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.561+0000 7f735cd01740 1 -- 172.21.15.133:0/2277374325 shutdown_connections 2024-09-11T23:40:18.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.561+0000 7f735cd01740 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c8963ee80 0x562c89641290 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.561+0000 7f735cd01740 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c895caef0 0x562c895bff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.561+0000 7f735cd01740 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c895bb370 0x562c895bf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.561+0000 7f735cd01740 1 --2- 172.21.15.133:0/2277374325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c895b1b40 0x562c895bf490 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.561+0000 7f735cd01740 1 -- 172.21.15.133:0/2277374325 >> 172.21.15.133:0/2277374325 conn(0x562c89579310 msgr2=0x562c895b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:18.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.561+0000 7f73337fe640 0 client.16449 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:40:18.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.564+0000 7f735cd01740 1 -- 172.21.15.133:0/2277374325 shutdown_connections 2024-09-11T23:40:18.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.565+0000 7f735cd01740 1 -- 172.21.15.133:0/2277374325 wait complete. 2024-09-11T23:40:18.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.565+0000 7f735cd01740 20 client.16449 trim_cache size 0 max 16384 2024-09-11T23:40:18.569 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:40:18.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.568+0000 7f735a928640 1 -- 172.21.15.133:0/567299982 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73400a2d70 msgr2=0x7f73400a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:18.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.569+0000 7f735a928640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73400a2d70 0x7f73400a5230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7354075f80 tx=0x7f733c006c20 comp rx=0 tx=0).stop 2024-09-11T23:40:18.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.569+0000 7f735a928640 1 -- 172.21.15.133:0/567299982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7354101df0 msgr2=0x7f73540744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:18.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.569+0000 7f735a928640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7354101df0 0x7f73540744f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7344031550 tx=0x7f7344004360 comp rx=0 tx=0).stop 2024-09-11T23:40:18.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.569+0000 7f735a928640 1 -- 172.21.15.133:0/567299982 shutdown_connections 2024-09-11T23:40:18.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.569+0000 7f735a928640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73400a2d70 0x7f73400a5230 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.569+0000 7f735a928640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7354106f20 0x7f7354074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.569+0000 7f735a928640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73541027a0 0x7f7354074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.569+0000 7f735a928640 1 --2- 172.21.15.133:0/567299982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7354101df0 0x7f73540744f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:18.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.569+0000 7f735a928640 1 -- 172.21.15.133:0/567299982 >> 172.21.15.133:0/567299982 conn(0x7f73540fd8f0 msgr2=0x7f73540fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:18.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.571+0000 7f735a928640 1 -- 172.21.15.133:0/567299982 shutdown_connections 2024-09-11T23:40:18.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:18.572+0000 7f735a928640 1 -- 172.21.15.133:0/567299982 wait complete. 2024-09-11T23:40:19.383 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:19 smithi133 ceph-mon[31430]: pgmap v109: 97 pgs: 97 active+clean; 26 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 859 MiB/s wr, 3.56k op/s 2024-09-11T23:40:19.506 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:40:19.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:19 smithi138 ceph-mon[35142]: pgmap v109: 97 pgs: 97 active+clean; 26 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 859 MiB/s wr, 3.56k op/s 2024-09-11T23:40:19.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:19 smithi178 ceph-mon[35882]: pgmap v109: 97 pgs: 97 active+clean; 26 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 859 MiB/s wr, 3.56k op/s 2024-09-11T23:40:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:20.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.553+0000 7fcdbeac2640 1 -- 172.21.15.133:0/898348930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb8109120 msgr2=0x7fcdb8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.553+0000 7fcdbeac2640 1 --2- 172.21.15.133:0/898348930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb8109120 0x7fcdb8110ca0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009ca0 tx=0x7fcdb002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.553+0000 7fcdbeac2640 1 -- 172.21.15.133:0/898348930 shutdown_connections 2024-09-11T23:40:20.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.553+0000 7fcdbeac2640 1 --2- 172.21.15.133:0/898348930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb8109120 0x7fcdb8110ca0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.553+0000 7fcdbeac2640 1 --2- 172.21.15.133:0/898348930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb81049a0 0x7fcdb81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.553+0000 7fcdbeac2640 1 --2- 172.21.15.133:0/898348930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb8103ff0 0x7fcdb81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.553+0000 7fcdbeac2640 1 -- 172.21.15.133:0/898348930 >> 172.21.15.133:0/898348930 conn(0x7fcdb80ffa90 msgr2=0x7fcdb8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:20.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.554+0000 7fcdbeac2640 1 -- 172.21.15.133:0/898348930 shutdown_connections 2024-09-11T23:40:20.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.554+0000 7fcdbeac2640 1 -- 172.21.15.133:0/898348930 wait complete. 2024-09-11T23:40:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdbeac2640 1 Processor -- start 2024-09-11T23:40:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdbeac2640 1 -- start start 2024-09-11T23:40:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdbeac2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb8103ff0 0x7fcdb819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdbeac2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb81049a0 0x7fcdb819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdbeac2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb8109120 0x7fcdb819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdbeac2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8114150 con 0x7fcdb8109120 2024-09-11T23:40:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdbeac2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8113fd0 con 0x7fcdb81049a0 2024-09-11T23:40:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdbeac2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81142d0 con 0x7fcdb8103ff0 2024-09-11T23:40:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdb7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb81049a0 0x7fcdb819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdb7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb81049a0 0x7fcdb819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48934/0 (socket says 172.21.15.133:48934) 2024-09-11T23:40:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdb7fff640 1 -- 172.21.15.133:0/2973308813 learned_addr learned my addr 172.21.15.133:0/2973308813 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb8103ff0 0x7fcdb819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.555+0000 7fcdbd05f640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb8109120 0x7fcdb819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.556+0000 7fcdb7fff640 1 -- 172.21.15.133:0/2973308813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb8103ff0 msgr2=0x7fcdb819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.556+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb8103ff0 0x7fcdb819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.556+0000 7fcdb7fff640 1 -- 172.21.15.133:0/2973308813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb8109120 msgr2=0x7fcdb819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.556+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb8109120 0x7fcdb819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.556+0000 7fcdb7fff640 1 -- 172.21.15.133:0/2973308813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb819f570 con 0x7fcdb81049a0 2024-09-11T23:40:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.556+0000 7fcdbd05f640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb8109120 0x7fcdb819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:40:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.556+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb81049a0 0x7fcdb819ab00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcda400ea60 tx=0x7fcda400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.557+0000 7fcdb5ffb640 1 -- 172.21.15.133:0/2973308813 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda400cde0 con 0x7fcdb81049a0 2024-09-11T23:40:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.557+0000 7fcdb5ffb640 1 -- 172.21.15.133:0/2973308813 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda40035f0 con 0x7fcdb81049a0 2024-09-11T23:40:20.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.557+0000 7fcdb5ffb640 1 -- 172.21.15.133:0/2973308813 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4010670 con 0x7fcdb81049a0 2024-09-11T23:40:20.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.557+0000 7fcdbeac2640 1 -- 172.21.15.133:0/2973308813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb819f860 con 0x7fcdb81049a0 2024-09-11T23:40:20.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.557+0000 7fcdbeac2640 1 -- 172.21.15.133:0/2973308813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb81a7140 con 0x7fcdb81049a0 2024-09-11T23:40:20.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.560+0000 7fcdb5ffb640 1 -- 172.21.15.133:0/2973308813 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcda4004020 con 0x7fcdb81049a0 2024-09-11T23:40:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.561+0000 7fcdc0e9b740 1 Processor -- start 2024-09-11T23:40:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.562+0000 7fcdb5ffb640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcda80a2d70 0x7fcda80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.562+0000 7fcdb5ffb640 1 -- 172.21.15.133:0/2973308813 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fcda40cf5e0 con 0x7fcdb81049a0 2024-09-11T23:40:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.562+0000 7fcdbd05f640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcda80a2d70 0x7fcda80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.563+0000 7fcdc0e9b740 1 -- start start 2024-09-11T23:40:20.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.563+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 0x5607de086090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.563+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de086a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.563+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de09bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.563+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5607ddfec2f0 con 0x5607de099b80 2024-09-11T23:40:20.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.563+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607ddf93cf0 con 0x5607de0865d0 2024-09-11T23:40:20.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.563+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5607ddfe82b0 con 0x5607de087b40 2024-09-11T23:40:20.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.564+0000 7fcdb7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de086a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.564+0000 7fcdb7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de086a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48942/0 (socket says 172.21.15.133:48942) 2024-09-11T23:40:20.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.564+0000 7fcdb7fff640 1 -- 172.21.15.133:0/2163086266 learned_addr learned my addr 172.21.15.133:0/2163086266 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:20.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.564+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/2163086266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 0x5607de086090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.565+0000 7fcdbd05f640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcda80a2d70 0x7fcda80a5230 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0004af0 tx=0x7fcdb0004a60 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:20.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.565+0000 7fcdbd05f640 1 --2- 172.21.15.133:0/2163086266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de09bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.566+0000 7fcdbc85e640 1 -- 172.21.15.133:0/2163086266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 msgr2=0x5607de086a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:20.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.566+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/2163086266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de086a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.566+0000 7fcdbc85e640 1 -- 172.21.15.133:0/2163086266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 msgr2=0x5607de09bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:20.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.566+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/2163086266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de09bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.566+0000 7fcdbc85e640 1 -- 172.21.15.133:0/2163086266 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ddf783a0 con 0x5607de087b40 2024-09-11T23:40:20.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.566+0000 7fcdbd05f640 1 --2- 172.21.15.133:0/2163086266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de09bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:40:20.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.567+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/2163086266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 0x5607de086090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009c00 tx=0x7fcdac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9caa121650a8a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:20.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.568+0000 7fcda2ffd640 1 -- 172.21.15.133:0/2163086266 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac004230 con 0x5607de087b40 2024-09-11T23:40:20.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.568+0000 7fcda2ffd640 1 -- 172.21.15.133:0/2163086266 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdac0043d0 con 0x5607de087b40 2024-09-11T23:40:20.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.568+0000 7fcda2ffd640 1 -- 172.21.15.133:0/2163086266 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac004dd0 con 0x5607de087b40 2024-09-11T23:40:20.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.570+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/2163086266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 msgr2=0x5607de086090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.570+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/2163086266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 0x5607de086090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009c00 tx=0x7fcdac011810 comp rx=0 tx=0).stop 2024-09-11T23:40:20.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.570+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/2163086266 shutdown_connections 2024-09-11T23:40:20.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.570+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/2163086266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de09bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.570+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/2163086266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de086a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.570+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/2163086266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 0x5607de086090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.570+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/2163086266 >> 172.21.15.133:0/2163086266 conn(0x5607de04f310 msgr2=0x5607de082c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:20.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.570+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/2163086266 shutdown_connections 2024-09-11T23:40:20.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.571+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/2163086266 wait complete. 2024-09-11T23:40:20.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.572+0000 7fcdc0e9b740 1 -- start start 2024-09-11T23:40:20.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.572+0000 7fcdc0e9b740 1 Processor -- start 2024-09-11T23:40:20.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.573+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de0934c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.573+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 0x5607de093a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.573+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de0991d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.573+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5607de0896e0 con 0x5607de099b80 2024-09-11T23:40:20.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.573+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607ddfa7060 con 0x5607de0865d0 2024-09-11T23:40:20.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.573+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5607de089860 con 0x5607de087b40 2024-09-11T23:40:20.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.574+0000 7fcdc0e9b740 10 client.0 _ll_register_callbacks cb 0x5607ddcfa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:20.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.574+0000 7fcdb7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 0x5607de093a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.574+0000 7fcdb7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 0x5607de093a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:39218/0 (socket says 172.21.15.133:39218) 2024-09-11T23:40:20.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.574+0000 7fcdb7fff640 1 -- 172.21.15.133:0/333912551 learned_addr learned my addr 172.21.15.133:0/333912551 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:20.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.575+0000 7fcdb7fff640 1 -- 172.21.15.133:0/333912551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 msgr2=0x5607de0934c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:20.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.575+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/333912551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de0934c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.575+0000 7fcdb7fff640 1 -- 172.21.15.133:0/333912551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 msgr2=0x5607de0991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:20.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.575+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/333912551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de0991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.575+0000 7fcdb7fff640 1 -- 172.21.15.133:0/333912551 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607ddd5d3c0 con 0x5607de087b40 2024-09-11T23:40:20.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.576+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/333912551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 0x5607de093a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcda400c760 tx=0x7fcda409ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2029310060acc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:20.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.576+0000 7fcd97fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:20.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.576+0000 7fcd97fff640 1 -- 172.21.15.133:0/333912551 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda40b9060 con 0x5607de087b40 2024-09-11T23:40:20.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.576+0000 7fcd97fff640 1 -- 172.21.15.133:0/333912551 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fcda40981e0 con 0x5607de087b40 2024-09-11T23:40:20.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.576+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/333912551 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdac0043d0 con 0x5607de087b40 2024-09-11T23:40:20.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.576+0000 7fcd97fff640 1 -- 172.21.15.133:0/333912551 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcda40a47e0 con 0x5607de087b40 2024-09-11T23:40:20.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.577+0000 7fcdc0e9b740 10 client.25861 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:20.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.577+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/333912551 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607de0b0b20 con 0x5607de087b40 2024-09-11T23:40:20.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.584+0000 7fcd97fff640 1 -- 172.21.15.133:0/333912551 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda4004310 con 0x5607de087b40 2024-09-11T23:40:20.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.584+0000 7fcd97fff640 1 -- 172.21.15.133:0/333912551 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcda40cf0c0 con 0x5607de087b40 2024-09-11T23:40:20.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.584+0000 7fcdc0e9b740 10 client.25861 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:20.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.584+0000 7fcdc0e9b740 10 client.25861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:40:20.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.584+0000 7fcdc0e9b740 20 client.25861 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:20.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.584+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/333912551 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5607de0b23a0 0x5607de0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.584+0000 7fcdc0e9b740 4 client.25861 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:20.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.584+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/333912551 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcdac0043d0 con 0x5607de0b23a0 2024-09-11T23:40:20.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.585+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/333912551 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5607de0b23a0 0x5607de0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.586+0000 7fcd97fff640 10 client.25861 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:40:20.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.586+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/333912551 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5607de0b23a0 0x5607de0b4830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70a837eca53febaa server_cookie=b4046f394e3558f7 in_seq=0 out_seq=0 2024-09-11T23:40:20.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.587+0000 7fcd97fff640 1 -- 172.21.15.133:0/333912551 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fcdac0043d0 con 0x5607de0b23a0 2024-09-11T23:40:20.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.587+0000 7fcd97fff640 10 client.25861 handle_command_reply: tid=0 2024-09-11T23:40:20.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.587+0000 7fcdc0e9b740 1 client.25861 shutdown 2024-09-11T23:40:20.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.587+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/333912551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 msgr2=0x5607de093a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.587+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/333912551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 0x5607de093a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcda400c760 tx=0x7fcda409ba50 comp rx=0 tx=0).stop 2024-09-11T23:40:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.588+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/333912551 shutdown_connections 2024-09-11T23:40:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.588+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/333912551 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5607de0b23a0 0x5607de0b4830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.588+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/333912551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de0991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.588+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/333912551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de087b40 0x5607de093a00 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcda400c760 tx=0x7fcda409ba50 comp rx=0 tx=0).stop 2024-09-11T23:40:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.588+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/333912551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de0934c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.588+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/333912551 >> 172.21.15.133:0/333912551 conn(0x5607de04f310 msgr2=0x5607de088960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.588+0000 7fcd97fff640 0 client.25861 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:40:20.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.589+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/333912551 shutdown_connections 2024-09-11T23:40:20.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.589+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/333912551 wait complete. 2024-09-11T23:40:20.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.589+0000 7fcdc0e9b740 20 client.25861 trim_cache size 0 max 16384 2024-09-11T23:40:20.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.608+0000 7fcdc0e9b740 1 Processor -- start 2024-09-11T23:40:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.608+0000 7fcdc0e9b740 1 -- start start 2024-09-11T23:40:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.608+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de099140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.608+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de08d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.608+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 0x5607de08d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.608+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81ab510 con 0x5607de099b80 2024-09-11T23:40:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.608+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81ad050 con 0x5607de0865d0 2024-09-11T23:40:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.608+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81abbe0 con 0x5607de08e7b0 2024-09-11T23:40:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.608+0000 7fcdb7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de08d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.608+0000 7fcdb7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de08d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35886/0 (socket says 172.21.15.133:35886) 2024-09-11T23:40:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.608+0000 7fcdb7fff640 1 -- 172.21.15.133:0/2517086681 learned_addr learned my addr 172.21.15.133:0/2517086681 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.609+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/2517086681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de099140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.609+0000 7fcdbd05f640 1 --2- 172.21.15.133:0/2517086681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 0x5607de08d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.609+0000 7fcdb7fff640 1 -- 172.21.15.133:0/2517086681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 msgr2=0x5607de08d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.609+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/2517086681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 0x5607de08d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.609+0000 7fcdb7fff640 1 -- 172.21.15.133:0/2517086681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 msgr2=0x5607de099140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.609+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/2517086681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de099140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.609+0000 7fcdb7fff640 1 -- 172.21.15.133:0/2517086681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607de0997d0 con 0x5607de099b80 2024-09-11T23:40:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.609+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/2517086681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de08d0b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcda40cce50 tx=0x7fcda4003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4236af572310ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.609+0000 7fcda27fc640 1 -- 172.21.15.133:0/2517086681 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda40b9040 con 0x5607de099b80 2024-09-11T23:40:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.609+0000 7fcda27fc640 1 -- 172.21.15.133:0/2517086681 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda4004310 con 0x5607de099b80 2024-09-11T23:40:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.610+0000 7fcda27fc640 1 -- 172.21.15.133:0/2517086681 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda40037a0 con 0x5607de099b80 2024-09-11T23:40:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.610+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/2517086681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 msgr2=0x5607de08d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.610+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/2517086681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de08d0b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcda40cce50 tx=0x7fcda4003f30 comp rx=0 tx=0).stop 2024-09-11T23:40:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.610+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/2517086681 shutdown_connections 2024-09-11T23:40:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.610+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/2517086681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 0x5607de08d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.610+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/2517086681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de099b80 0x5607de08d0b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.610+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/2517086681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de0865d0 0x5607de099140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.610+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/2517086681 >> 172.21.15.133:0/2517086681 conn(0x5607de04f310 msgr2=0x5607de08e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.611+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/2517086681 shutdown_connections 2024-09-11T23:40:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.611+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/2517086681 wait complete. 2024-09-11T23:40:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.611+0000 7fcdc0e9b740 1 -- start start 2024-09-11T23:40:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.611+0000 7fcdc0e9b740 1 Processor -- start 2024-09-11T23:40:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de0865d0 0x5607de0c15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 0x5607de0c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdc0e9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de099b80 0x5607de114f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81ac980 con 0x5607de0865d0 2024-09-11T23:40:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81ac2b0 con 0x5607de099b80 2024-09-11T23:40:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdc0e9b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81a9530 con 0x5607de08e7b0 2024-09-11T23:40:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdc0e9b740 10 client.0 _ll_register_callbacks cb 0x7fcda409b4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdb7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 0x5607de0c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdb7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 0x5607de0c1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:39246/0 (socket says 172.21.15.133:39246) 2024-09-11T23:40:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdb7fff640 1 -- 172.21.15.133:0/72538994 learned_addr learned my addr 172.21.15.133:0/72538994 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdbd05f640 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de099b80 0x5607de114f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.612+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de0865d0 0x5607de0c15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcdb7fff640 1 -- 172.21.15.133:0/72538994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de099b80 msgr2=0x5607de114f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de099b80 0x5607de114f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcdb7fff640 1 -- 172.21.15.133:0/72538994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de0865d0 msgr2=0x5607de0c15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de0865d0 0x5607de0c15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcdb7fff640 1 -- 172.21.15.133:0/72538994 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607de1155f0 con 0x5607de08e7b0 2024-09-11T23:40:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de0865d0 0x5607de0c15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:40:20.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcdb7fff640 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 0x5607de0c1af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcda4098540 tx=0x7fcda40057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74fdc7412e53ddaf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcd97fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcd97fff640 1 -- 172.21.15.133:0/72538994 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda40b9070 con 0x5607de08e7b0 2024-09-11T23:40:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcd97fff640 1 -- 172.21.15.133:0/72538994 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fcda40cf110 con 0x5607de08e7b0 2024-09-11T23:40:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/72538994 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda4004310 con 0x5607de08e7b0 2024-09-11T23:40:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcd97fff640 1 -- 172.21.15.133:0/72538994 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcda409bb70 con 0x5607de08e7b0 2024-09-11T23:40:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcdc0e9b740 10 client.25867 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/72538994 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607de1155f0 con 0x5607de08e7b0 2024-09-11T23:40:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.613+0000 7fcd97fff640 1 -- 172.21.15.133:0/72538994 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda40037a0 con 0x5607de08e7b0 2024-09-11T23:40:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.614+0000 7fcd97fff640 1 -- 172.21.15.133:0/72538994 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcda40a10b0 con 0x5607de08e7b0 2024-09-11T23:40:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.614+0000 7fcdc0e9b740 10 client.25867 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.614+0000 7fcdc0e9b740 10 client.25867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.614+0000 7fcdc0e9b740 20 client.25867 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.614+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5607de11ccb0 0x5607de11f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.614+0000 7fcdc0e9b740 4 client.25867 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.614+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/72538994 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fcda4004310 con 0x5607de11ccb0 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.614+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5607de11ccb0 0x5607de11f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.614+0000 7fcd97fff640 10 client.25867 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcdbc85e640 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5607de11ccb0 0x5607de11f0d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f524d024332aa5a5 server_cookie=ef7bc1f78e0b7fd4 in_seq=0 out_seq=0 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcd97fff640 1 -- 172.21.15.133:0/72538994 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fcda4004310 con 0x5607de11ccb0 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcd97fff640 10 client.25867 handle_command_reply: tid=0 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcdc0e9b740 1 client.25867 shutdown 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/72538994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 msgr2=0x5607de0c1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 0x5607de0c1af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcda4098540 tx=0x7fcda40057e0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/72538994 shutdown_connections 2024-09-11T23:40:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5607de11ccb0 0x5607de11f0d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607de099b80 0x5607de114f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607de08e7b0 0x5607de0c1af0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcd97fff640 0 client.25867 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:40:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcdc0e9b740 1 --2- 172.21.15.133:0/72538994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607de0865d0 0x5607de0c15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.615+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/72538994 >> 172.21.15.133:0/72538994 conn(0x5607de04f310 msgr2=0x5607de08cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.616+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/72538994 shutdown_connections 2024-09-11T23:40:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.616+0000 7fcdc0e9b740 1 -- 172.21.15.133:0/72538994 wait complete. 2024-09-11T23:40:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.616+0000 7fcdc0e9b740 20 client.25867 trim_cache size 0 max 16384 2024-09-11T23:40:20.620 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:40:20.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.617+0000 7fcdbeac2640 1 -- 172.21.15.133:0/2973308813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcda80a2d70 msgr2=0x7fcda80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.617+0000 7fcdbeac2640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcda80a2d70 0x7fcda80a5230 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0004af0 tx=0x7fcdb0004a60 comp rx=0 tx=0).stop 2024-09-11T23:40:20.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.617+0000 7fcdbeac2640 1 -- 172.21.15.133:0/2973308813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb81049a0 msgr2=0x7fcdb819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:20.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.617+0000 7fcdbeac2640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb81049a0 0x7fcdb819ab00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcda400ea60 tx=0x7fcda400ef20 comp rx=0 tx=0).stop 2024-09-11T23:40:20.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.618+0000 7fcdbeac2640 1 -- 172.21.15.133:0/2973308813 shutdown_connections 2024-09-11T23:40:20.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.618+0000 7fcdbeac2640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcda80a2d70 0x7fcda80a5230 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.618+0000 7fcdbeac2640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb8109120 0x7fcdb819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.618+0000 7fcdbeac2640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdb81049a0 0x7fcdb819ab00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.618+0000 7fcdbeac2640 1 --2- 172.21.15.133:0/2973308813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb8103ff0 0x7fcdb819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:20.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.618+0000 7fcdbeac2640 1 -- 172.21.15.133:0/2973308813 >> 172.21.15.133:0/2973308813 conn(0x7fcdb80ffa90 msgr2=0x7fcdb8100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:20.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.618+0000 7fcdbeac2640 1 -- 172.21.15.133:0/2973308813 shutdown_connections 2024-09-11T23:40:20.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:20.618+0000 7fcdbeac2640 1 -- 172.21.15.133:0/2973308813 wait complete. 2024-09-11T23:40:21.584 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:40:21.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:21 smithi178 ceph-mon[35882]: pgmap v110: 97 pgs: 97 active+clean; 27 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 666 MiB/s wr, 2.16k op/s 2024-09-11T23:40:21.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:21 smithi138 ceph-mon[35142]: pgmap v110: 97 pgs: 97 active+clean; 27 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 666 MiB/s wr, 2.16k op/s 2024-09-11T23:40:21.961 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:21 smithi133 ceph-mon[31430]: pgmap v110: 97 pgs: 97 active+clean; 27 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 666 MiB/s wr, 2.16k op/s 2024-09-11T23:40:22.339 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:23.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.218+0000 7f083ce1b640 1 -- 172.21.15.133:0/1541151980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f083811b140 msgr2=0x7f083811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:23.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.219+0000 7f083ce1b640 1 --2- 172.21.15.133:0/1541151980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f083811b140 0x7f083811d530 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0830009ca0 tx=0x7f083002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.220+0000 7f083ce1b640 1 -- 172.21.15.133:0/1541151980 shutdown_connections 2024-09-11T23:40:23.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.221+0000 7f083ce1b640 1 --2- 172.21.15.133:0/1541151980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f083811b140 0x7f083811d530 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.221+0000 7f083ce1b640 1 --2- 172.21.15.133:0/1541151980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0838109830 0x7f0838109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.221+0000 7f083ce1b640 1 --2- 172.21.15.133:0/1541151980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0838109450 0x7f0838110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.221+0000 7f083ce1b640 1 -- 172.21.15.133:0/1541151980 >> 172.21.15.133:0/1541151980 conn(0x7f083806e140 msgr2=0x7f083806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:23.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.228+0000 7f083ce1b640 1 -- 172.21.15.133:0/1541151980 shutdown_connections 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.228+0000 7f083ce1b640 1 -- 172.21.15.133:0/1541151980 wait complete. 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.230+0000 7f083ce1b640 1 Processor -- start 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.230+0000 7f083ce1b640 1 -- start start 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.231+0000 7f083ce1b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838109450 0x7f08381a72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.231+0000 7f083ce1b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0838109830 0x7f08381a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.231+0000 7f083ce1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f083811b140 0x7f08381abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.232+0000 7f083ce1b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0838120e00 con 0x7f083811b140 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.232+0000 7f083ce1b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0838120c80 con 0x7f0838109830 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.232+0000 7f083659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838109450 0x7f08381a72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.232+0000 7f083659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838109450 0x7f08381a72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42112/0 (socket says 172.21.15.133:42112) 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.232+0000 7f083659c640 1 -- 172.21.15.133:0/2432234483 learned_addr learned my addr 172.21.15.133:0/2432234483 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:23.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.232+0000 7f083ce1b640 1 -- 172.21.15.133:0/2432234483 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0838120f80 con 0x7f0838109450 2024-09-11T23:40:23.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.233+0000 7f0836d9d640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f083811b140 0x7f08381abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.235+0000 7f0835d9b640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0838109830 0x7f08381a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.239+0000 7f0836d9d640 1 -- 172.21.15.133:0/2432234483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838109450 msgr2=0x7f08381a72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:23.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.239+0000 7f0836d9d640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838109450 0x7f08381a72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.239+0000 7f0836d9d640 1 -- 172.21.15.133:0/2432234483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0838109830 msgr2=0x7f08381a77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:23.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.239+0000 7f0836d9d640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0838109830 0x7f08381a77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.239+0000 7f0836d9d640 1 -- 172.21.15.133:0/2432234483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08381ac250 con 0x7f083811b140 2024-09-11T23:40:23.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.239+0000 7f0836d9d640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f083811b140 0x7f08381abb70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0830009bf0 tx=0x7f0830032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.244+0000 7f08277fe640 1 -- 172.21.15.133:0/2432234483 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0830003970 con 0x7f083811b140 2024-09-11T23:40:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.244+0000 7f08277fe640 1 -- 172.21.15.133:0/2432234483 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0830030880 con 0x7f083811b140 2024-09-11T23:40:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.244+0000 7f08277fe640 1 -- 172.21.15.133:0/2432234483 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300045c0 con 0x7f083811b140 2024-09-11T23:40:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.244+0000 7f083ce1b640 1 -- 172.21.15.133:0/2432234483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08381ac4e0 con 0x7f083811b140 2024-09-11T23:40:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.245+0000 7f083ce1b640 1 -- 172.21.15.133:0/2432234483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08381ac910 con 0x7f083811b140 2024-09-11T23:40:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.251+0000 7f083f1f4740 1 Processor -- start 2024-09-11T23:40:23.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.278+0000 7f08277fe640 1 -- 172.21.15.133:0/2432234483 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0830030a30 con 0x7f083811b140 2024-09-11T23:40:23.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.279+0000 7f08277fe640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08200a2f00 0x7f08200a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.280+0000 7f0835d9b640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08200a2f00 0x7f08200a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.285+0000 7f08277fe640 1 -- 172.21.15.133:0/2432234483 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f08300f0390 con 0x7f083811b140 2024-09-11T23:40:23.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.294+0000 7f083f1f4740 1 -- start start 2024-09-11T23:40:23.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.301+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f6c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.301+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f6c5d0 0x557c84f6ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.301+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 0x557c84f79760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.301+0000 7f083f1f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557c84ed22f0 con 0x557c84f6c5d0 2024-09-11T23:40:23.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.301+0000 7f083f1f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c84e79cf0 con 0x557c84f77370 2024-09-11T23:40:23.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.302+0000 7f083f1f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557c84ece2b0 con 0x557c84f6db40 2024-09-11T23:40:23.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.302+0000 7f083659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f6c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.305+0000 7f083659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f6c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42116/0 (socket says 172.21.15.133:42116) 2024-09-11T23:40:23.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.305+0000 7f083659c640 1 -- 172.21.15.133:0/2602799611 learned_addr learned my addr 172.21.15.133:0/2602799611 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:23.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.306+0000 7f0836d9d640 1 --2- 172.21.15.133:0/2602799611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 0x557c84f79760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.307+0000 7f0835d9b640 1 --2- 172.21.15.133:0/2602799611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f6c5d0 0x557c84f6ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.308+0000 7f083659c640 1 -- 172.21.15.133:0/2602799611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 msgr2=0x557c84f79760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:23.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.308+0000 7f083659c640 1 --2- 172.21.15.133:0/2602799611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 0x557c84f79760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.308+0000 7f083659c640 1 -- 172.21.15.133:0/2602799611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f6c5d0 msgr2=0x557c84f6ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:23.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.308+0000 7f083659c640 1 --2- 172.21.15.133:0/2602799611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f6c5d0 0x557c84f6ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.308+0000 7f083659c640 1 -- 172.21.15.133:0/2602799611 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c84e5e3a0 con 0x557c84f6db40 2024-09-11T23:40:23.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.308+0000 7f0836d9d640 1 --2- 172.21.15.133:0/2602799611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 0x557c84f79760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:40:23.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.309+0000 7f083659c640 1 --2- 172.21.15.133:0/2602799611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f6c090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0828009e10 tx=0x7f0828011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c43e67f0a944e6a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:23.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.313+0000 7f0824ff9640 1 -- 172.21.15.133:0/2602799611 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082801e070 con 0x557c84f6db40 2024-09-11T23:40:23.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.314+0000 7f0824ff9640 1 -- 172.21.15.133:0/2602799611 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0828019b10 con 0x557c84f6db40 2024-09-11T23:40:23.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.315+0000 7f083f1f4740 1 -- 172.21.15.133:0/2602799611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 msgr2=0x557c84f6c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:23.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.315+0000 7f083f1f4740 1 --2- 172.21.15.133:0/2602799611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f6c090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0828009e10 tx=0x7f0828011810 comp rx=0 tx=0).stop 2024-09-11T23:40:23.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.317+0000 7f083f1f4740 1 -- 172.21.15.133:0/2602799611 shutdown_connections 2024-09-11T23:40:23.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.317+0000 7f0835d9b640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08200a2f00 0x7f08200a53c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f082c006fd0 tx=0x7f082c008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:23.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.317+0000 7f083f1f4740 1 --2- 172.21.15.133:0/2602799611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 0x557c84f79760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.317+0000 7f083f1f4740 1 --2- 172.21.15.133:0/2602799611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f6c5d0 0x557c84f6ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.317+0000 7f083f1f4740 1 --2- 172.21.15.133:0/2602799611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f6c090 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.317+0000 7f083f1f4740 1 -- 172.21.15.133:0/2602799611 >> 172.21.15.133:0/2602799611 conn(0x557c84f35310 msgr2=0x557c84f68c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:23.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.321+0000 7f083f1f4740 1 -- 172.21.15.133:0/2602799611 shutdown_connections 2024-09-11T23:40:23.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.321+0000 7f083f1f4740 1 -- 172.21.15.133:0/2602799611 wait complete. 2024-09-11T23:40:23.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.322+0000 7f083f1f4740 1 -- start start 2024-09-11T23:40:23.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.322+0000 7f083f1f4740 1 Processor -- start 2024-09-11T23:40:23.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.323+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f6c5d0 0x557c84f76c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.323+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f73f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.323+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 0x557c84f744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.323+0000 7f083f1f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557c84f7a0c0 con 0x557c84f77370 2024-09-11T23:40:23.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.323+0000 7f083f1f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c84e8d060 con 0x557c84f6c5d0 2024-09-11T23:40:23.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.323+0000 7f083f1f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557c84f7a240 con 0x557c84f6db40 2024-09-11T23:40:23.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.323+0000 7f083f1f4740 10 client.0 _ll_register_callbacks cb 0x557c84be08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:23.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.327+0000 7f083659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f6c5d0 0x557c84f76c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.327+0000 7f0836d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 0x557c84f744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.327+0000 7f0836d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 0x557c84f744d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59736/0 (socket says 172.21.15.133:59736) 2024-09-11T23:40:23.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.327+0000 7f0836d9d640 1 -- 172.21.15.133:0/788349005 learned_addr learned my addr 172.21.15.133:0/788349005 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:23.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.329+0000 7f0835d9b640 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f73f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.330+0000 7f0836d9d640 1 -- 172.21.15.133:0/788349005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 msgr2=0x557c84f73f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:23.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.330+0000 7f0836d9d640 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f73f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.330+0000 7f0836d9d640 1 -- 172.21.15.133:0/788349005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f6c5d0 msgr2=0x557c84f76c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:23.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.330+0000 7f0836d9d640 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f6c5d0 0x557c84f76c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.330+0000 7f0836d9d640 1 -- 172.21.15.133:0/788349005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c84c433c0 con 0x557c84f77370 2024-09-11T23:40:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.340+0000 7f0836d9d640 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 0x557c84f744d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f08300b93a0 tx=0x7f08300bc3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3c82a3a059149e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.345+0000 7f081dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:40:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.345+0000 7f081dffb640 1 -- 172.21.15.133:0/788349005 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300d4020 con 0x557c84f77370 2024-09-11T23:40:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.345+0000 7f083f1f4740 1 -- 172.21.15.133:0/788349005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0828019b10 con 0x557c84f77370 2024-09-11T23:40:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.345+0000 7f081dffb640 1 -- 172.21.15.133:0/788349005 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f08300b9040 con 0x557c84f77370 2024-09-11T23:40:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.346+0000 7f081dffb640 1 -- 172.21.15.133:0/788349005 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08300c0d80 con 0x557c84f77370 2024-09-11T23:40:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.351+0000 7f083f1f4740 10 client.16473 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.351+0000 7f083f1f4740 1 -- 172.21.15.133:0/788349005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c84f96b30 con 0x557c84f77370 2024-09-11T23:40:23.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.353+0000 7f081dffb640 1 -- 172.21.15.133:0/788349005 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0828019b10 con 0x557c84f77370 2024-09-11T23:40:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.355+0000 7f081dffb640 1 -- 172.21.15.133:0/788349005 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f08300c51b0 con 0x557c84f77370 2024-09-11T23:40:23.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.358+0000 7f083f1f4740 10 client.16473 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:23.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.358+0000 7f083f1f4740 10 client.16473 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:40:23.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.358+0000 7f083f1f4740 20 client.16473 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:23.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.358+0000 7f083f1f4740 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557c84f98040 0x557c84f9a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.358+0000 7f083f1f4740 4 client.16473 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:23.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.358+0000 7f083f1f4740 1 -- 172.21.15.133:0/788349005 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c84edf860 con 0x557c84f98040 2024-09-11T23:40:23.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.359+0000 7f083659c640 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557c84f98040 0x557c84f9a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.365+0000 7f083659c640 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557c84f98040 0x557c84f9a4a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5ed439d003126f server_cookie=96a50187fb7d6f61 in_seq=0 out_seq=0 2024-09-11T23:40:23.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.365+0000 7f081dffb640 10 client.16473 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:40:23.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.367+0000 7f081dffb640 1 -- 172.21.15.133:0/788349005 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c84edf860 con 0x557c84f98040 2024-09-11T23:40:23.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.367+0000 7f081dffb640 10 client.16473 handle_command_reply: tid=0 2024-09-11T23:40:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.367+0000 7f083f1f4740 1 client.16473 shutdown 2024-09-11T23:40:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.368+0000 7f083f1f4740 1 -- 172.21.15.133:0/788349005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 msgr2=0x557c84f744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.368+0000 7f083f1f4740 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 0x557c84f744d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f08300b93a0 tx=0x7f08300bc3f0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.369+0000 7f083f1f4740 1 -- 172.21.15.133:0/788349005 shutdown_connections 2024-09-11T23:40:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.369+0000 7f083f1f4740 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557c84f98040 0x557c84f9a4a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.369+0000 7f083f1f4740 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 0x557c84f744d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.369+0000 7f083f1f4740 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f73f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.369+0000 7f083f1f4740 1 --2- 172.21.15.133:0/788349005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f6c5d0 0x557c84f76c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.369+0000 7f081dffb640 0 client.16473 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:40:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.369+0000 7f083f1f4740 1 -- 172.21.15.133:0/788349005 >> 172.21.15.133:0/788349005 conn(0x557c84f35310 msgr2=0x557c84f6e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.369+0000 7f083f1f4740 1 -- 172.21.15.133:0/788349005 shutdown_connections 2024-09-11T23:40:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.369+0000 7f083f1f4740 1 -- 172.21.15.133:0/788349005 wait complete. 2024-09-11T23:40:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.370+0000 7f083f1f4740 20 client.16473 trim_cache size 0 max 16384 2024-09-11T23:40:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.414+0000 7f083f1f4740 1 Processor -- start 2024-09-11T23:40:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.416+0000 7f083f1f4740 1 -- start start 2024-09-11T23:40:23.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.416+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f6c5d0 0x557c84f6f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.416+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f6f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.416+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 0x557c84f9fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.417+0000 7f083f1f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0838118e90 con 0x557c84f77370 2024-09-11T23:40:23.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.417+0000 7f083f1f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08381b5720 con 0x557c84f6c5d0 2024-09-11T23:40:23.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.417+0000 7f083f1f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08381b42b0 con 0x557c84f6db40 2024-09-11T23:40:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.419+0000 7f083659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f6c5d0 0x557c84f6f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.419+0000 7f0835d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f6f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.419+0000 7f083659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f6c5d0 0x557c84f6f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38034/0 (socket says 172.21.15.133:38034) 2024-09-11T23:40:23.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.419+0000 7f083659c640 1 -- 172.21.15.133:0/3416816197 learned_addr learned my addr 172.21.15.133:0/3416816197 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:23.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.422+0000 7f0836d9d640 1 --2- 172.21.15.133:0/3416816197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 0x557c84f9fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.422+0000 7f0836d9d640 1 -- 172.21.15.133:0/3416816197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 msgr2=0x557c84f6f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:23.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.422+0000 7f0836d9d640 1 --2- 172.21.15.133:0/3416816197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f6f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.423+0000 7f0836d9d640 1 -- 172.21.15.133:0/3416816197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f6c5d0 msgr2=0x557c84f6f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:23.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.423+0000 7f0836d9d640 1 --2- 172.21.15.133:0/3416816197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f6c5d0 0x557c84f6f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.423+0000 7f0836d9d640 1 -- 172.21.15.133:0/3416816197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c84f76fe0 con 0x557c84f77370 2024-09-11T23:40:23.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.423+0000 7f0836d9d640 1 --2- 172.21.15.133:0/3416816197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 0x557c84f9fdb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0830009ca0 tx=0x7f0830004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29d9abca7bfe3fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:23.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.426+0000 7f081ffff640 1 -- 172.21.15.133:0/3416816197 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300d4070 con 0x557c84f77370 2024-09-11T23:40:23.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.427+0000 7f081ffff640 1 -- 172.21.15.133:0/3416816197 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0830036070 con 0x557c84f77370 2024-09-11T23:40:23.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.427+0000 7f081ffff640 1 -- 172.21.15.133:0/3416816197 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300b9040 con 0x557c84f77370 2024-09-11T23:40:23.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.430+0000 7f083f1f4740 1 -- 172.21.15.133:0/3416816197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 msgr2=0x557c84f9fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:23.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.430+0000 7f083f1f4740 1 --2- 172.21.15.133:0/3416816197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 0x557c84f9fdb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0830009ca0 tx=0x7f0830004060 comp rx=0 tx=0).stop 2024-09-11T23:40:23.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.432+0000 7f083f1f4740 1 -- 172.21.15.133:0/3416816197 shutdown_connections 2024-09-11T23:40:23.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.432+0000 7f083f1f4740 1 --2- 172.21.15.133:0/3416816197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f77370 0x557c84f9fdb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.432+0000 7f083f1f4740 1 --2- 172.21.15.133:0/3416816197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6db40 0x557c84f6f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.432+0000 7f083f1f4740 1 --2- 172.21.15.133:0/3416816197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f6c5d0 0x557c84f6f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.432+0000 7f083f1f4740 1 -- 172.21.15.133:0/3416816197 >> 172.21.15.133:0/3416816197 conn(0x557c84f35310 msgr2=0x557c84f75ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:23.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.434+0000 7f083f1f4740 1 -- 172.21.15.133:0/3416816197 shutdown_connections 2024-09-11T23:40:23.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.434+0000 7f083f1f4740 1 -- 172.21.15.133:0/3416816197 wait complete. 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.435+0000 7f083f1f4740 1 -- start start 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.435+0000 7f083f1f4740 1 Processor -- start 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.435+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6c5d0 0x557c84f7b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.435+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f6db40 0x557c84f7b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.436+0000 7f083f1f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 0x557c84f7bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.436+0000 7f083f1f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08381b5050 con 0x557c84f6db40 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.436+0000 7f0836d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 0x557c84f7bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.436+0000 7f0836d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 0x557c84f7bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38044/0 (socket says 172.21.15.133:38044) 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.436+0000 7f0836d9d640 1 -- 172.21.15.133:0/843900602 learned_addr learned my addr 172.21.15.133:0/843900602 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.436+0000 7f083f1f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08381b4980 con 0x557c84f77370 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.436+0000 7f083f1f4740 1 -- 172.21.15.133:0/843900602 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0838116f80 con 0x557c84f6c5d0 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.436+0000 7f083f1f4740 10 client.0 _ll_register_callbacks cb 0x7f08300b8d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:23.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.437+0000 7f0835d9b640 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f6db40 0x557c84f7b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.437+0000 7f083659c640 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6c5d0 0x557c84f7b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.441+0000 7f0836d9d640 1 -- 172.21.15.133:0/843900602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6c5d0 msgr2=0x557c84f7b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:23.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.441+0000 7f0836d9d640 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6c5d0 0x557c84f7b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.441+0000 7f0836d9d640 1 -- 172.21.15.133:0/843900602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f6db40 msgr2=0x557c84f7b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:23.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.441+0000 7f0836d9d640 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f6db40 0x557c84f7b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.441+0000 7f0836d9d640 1 -- 172.21.15.133:0/843900602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c85001e50 con 0x557c84f77370 2024-09-11T23:40:23.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.442+0000 7f0836d9d640 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 0x557c84f7bdd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f08300bac80 tx=0x7f0830009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5307029196588e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:23.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.443+0000 7f081dffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:23.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.443+0000 7f081dffb640 1 -- 172.21.15.133:0/843900602 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300d4040 con 0x557c84f77370 2024-09-11T23:40:23.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.444+0000 7f081dffb640 1 -- 172.21.15.133:0/843900602 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0830004b90 con 0x557c84f77370 2024-09-11T23:40:23.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.444+0000 7f083f1f4740 1 -- 172.21.15.133:0/843900602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0830036070 con 0x557c84f77370 2024-09-11T23:40:23.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.444+0000 7f081dffb640 1 -- 172.21.15.133:0/843900602 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0830004d90 con 0x557c84f77370 2024-09-11T23:40:23.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.444+0000 7f083f1f4740 10 client.25913 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:23.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.444+0000 7f083f1f4740 1 -- 172.21.15.133:0/843900602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c85001e50 con 0x557c84f77370 2024-09-11T23:40:23.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.445+0000 7f081dffb640 1 -- 172.21.15.133:0/843900602 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08300a2070 con 0x557c84f77370 2024-09-11T23:40:23.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.445+0000 7f081dffb640 1 -- 172.21.15.133:0/843900602 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f08300bbfc0 con 0x557c84f77370 2024-09-11T23:40:23.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.445+0000 7f083f1f4740 10 client.25913 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:23.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.445+0000 7f083f1f4740 10 client.25913 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:40:23.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.445+0000 7f083f1f4740 20 client.25913 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:23.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.445+0000 7f083f1f4740 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557c85003430 0x557c85005820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:23.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.445+0000 7f083f1f4740 4 client.25913 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:40:23.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.445+0000 7f083f1f4740 1 -- 172.21.15.133:0/843900602 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0830036070 con 0x557c85003430 2024-09-11T23:40:23.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.448+0000 7f083659c640 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557c85003430 0x557c85005820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:23.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.456+0000 7f083659c640 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557c85003430 0x557c85005820 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94cf0038f0f76b4 server_cookie=2b7475bd27ea6f81 in_seq=0 out_seq=0 2024-09-11T23:40:23.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.456+0000 7f081dffb640 10 client.25913 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:40:23.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.456+0000 7f081dffb640 1 -- 172.21.15.133:0/843900602 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0830036070 con 0x557c85003430 2024-09-11T23:40:23.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.456+0000 7f081dffb640 10 client.25913 handle_command_reply: tid=0 2024-09-11T23:40:23.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.456+0000 7f083f1f4740 1 client.25913 shutdown 2024-09-11T23:40:23.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.459+0000 7f083f1f4740 1 -- 172.21.15.133:0/843900602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 msgr2=0x557c84f7bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:23.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.460+0000 7f083f1f4740 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 0x557c84f7bdd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f08300bac80 tx=0x7f0830009540 comp rx=0 tx=0).stop 2024-09-11T23:40:23.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.460+0000 7f083f1f4740 1 -- 172.21.15.133:0/843900602 shutdown_connections 2024-09-11T23:40:23.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.460+0000 7f083f1f4740 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x557c85003430 0x557c85005820 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.460+0000 7f083f1f4740 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c84f77370 0x557c84f7bdd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.460+0000 7f083f1f4740 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c84f6db40 0x557c84f7b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.460+0000 7f083f1f4740 1 --2- 172.21.15.133:0/843900602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c84f6c5d0 0x557c84f7b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.460+0000 7f081dffb640 0 client.25913 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:40:23.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.460+0000 7f083f1f4740 1 -- 172.21.15.133:0/843900602 >> 172.21.15.133:0/843900602 conn(0x557c84f35310 msgr2=0x557c84f757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:23.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.461+0000 7f083f1f4740 1 -- 172.21.15.133:0/843900602 shutdown_connections 2024-09-11T23:40:23.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.461+0000 7f083f1f4740 1 -- 172.21.15.133:0/843900602 wait complete. 2024-09-11T23:40:23.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.461+0000 7f083f1f4740 20 client.25913 trim_cache size 0 max 16384 2024-09-11T23:40:23.461 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:40:23.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.462+0000 7f083ce1b640 1 -- 172.21.15.133:0/2432234483 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08200a2f00 msgr2=0x7f08200a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:23.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.463+0000 7f083ce1b640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08200a2f00 0x7f08200a53c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f082c006fd0 tx=0x7f082c008040 comp rx=0 tx=0).stop 2024-09-11T23:40:23.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.463+0000 7f083ce1b640 1 -- 172.21.15.133:0/2432234483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f083811b140 msgr2=0x7f08381abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:23.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.463+0000 7f083ce1b640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f083811b140 0x7f08381abb70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0830009bf0 tx=0x7f0830032f40 comp rx=0 tx=0).stop 2024-09-11T23:40:23.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.463+0000 7f083ce1b640 1 -- 172.21.15.133:0/2432234483 shutdown_connections 2024-09-11T23:40:23.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.463+0000 7f083ce1b640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08200a2f00 0x7f08200a53c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.463+0000 7f083ce1b640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f083811b140 0x7f08381abb70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.463+0000 7f083ce1b640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0838109830 0x7f08381a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.463+0000 7f083ce1b640 1 --2- 172.21.15.133:0/2432234483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838109450 0x7f08381a72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:23.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.463+0000 7f083ce1b640 1 -- 172.21.15.133:0/2432234483 >> 172.21.15.133:0/2432234483 conn(0x7f083806e140 msgr2=0x7f083811be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:23.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.468+0000 7f083ce1b640 1 -- 172.21.15.133:0/2432234483 shutdown_connections 2024-09-11T23:40:23.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:23.468+0000 7f083ce1b640 1 -- 172.21.15.133:0/2432234483 wait complete. 2024-09-11T23:40:23.498 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:23 smithi133 ceph-mon[31430]: pgmap v111: 97 pgs: 97 active+clean; 29 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 838 MiB/s wr, 3.08k op/s 2024-09-11T23:40:23.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:23 smithi138 ceph-mon[35142]: pgmap v111: 97 pgs: 97 active+clean; 29 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 838 MiB/s wr, 3.08k op/s 2024-09-11T23:40:23.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:23 smithi178 ceph-mon[35882]: pgmap v111: 97 pgs: 97 active+clean; 29 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 838 MiB/s wr, 3.08k op/s 2024-09-11T23:40:25.155 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:40:26.424 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:26.588 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:26 smithi133 ceph-mon[31430]: pgmap v112: 97 pgs: 97 active+clean; 29 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 902 KiB/s rd, 740 MiB/s wr, 3.23k op/s 2024-09-11T23:40:26.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:26 smithi138 ceph-mon[35142]: pgmap v112: 97 pgs: 97 active+clean; 29 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 902 KiB/s rd, 740 MiB/s wr, 3.23k op/s 2024-09-11T23:40:26.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:26 smithi178 ceph-mon[35882]: pgmap v112: 97 pgs: 97 active+clean; 29 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 902 KiB/s rd, 740 MiB/s wr, 3.23k op/s 2024-09-11T23:40:27.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:27 smithi138 ceph-mon[35142]: pgmap v113: 97 pgs: 97 active+clean; 29 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 560 KiB/s rd, 636 MiB/s wr, 2.91k op/s 2024-09-11T23:40:27.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:27 smithi133 ceph-mon[31430]: pgmap v113: 97 pgs: 97 active+clean; 29 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 560 KiB/s rd, 636 MiB/s wr, 2.91k op/s 2024-09-11T23:40:27.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.744+0000 7f72d89af640 1 -- 172.21.15.133:0/1858541660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72d406ea30 msgr2=0x7f72d406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.744+0000 7f72d89af640 1 --2- 172.21.15.133:0/1858541660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72d406ea30 0x7f72d406ee70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009ca0 tx=0x7f72cc02a8f0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.744+0000 7f72d89af640 1 -- 172.21.15.133:0/1858541660 shutdown_connections 2024-09-11T23:40:27.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.744+0000 7f72d89af640 1 --2- 172.21.15.133:0/1858541660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72d411b140 0x7f72d411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.744+0000 7f72d89af640 1 --2- 172.21.15.133:0/1858541660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72d406ea30 0x7f72d406ee70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.744+0000 7f72d89af640 1 --2- 172.21.15.133:0/1858541660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d4110190 0x7f72d406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.744+0000 7f72d89af640 1 -- 172.21.15.133:0/1858541660 >> 172.21.15.133:0/1858541660 conn(0x7f72d4073180 msgr2=0x7f72d4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.749+0000 7f72d89af640 1 -- 172.21.15.133:0/1858541660 shutdown_connections 2024-09-11T23:40:27.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.749+0000 7f72d89af640 1 -- 172.21.15.133:0/1858541660 wait complete. 2024-09-11T23:40:27.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.750+0000 7f72d89af640 1 Processor -- start 2024-09-11T23:40:27.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.750+0000 7f72d89af640 1 -- start start 2024-09-11T23:40:27.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.751+0000 7f72d89af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d4110190 0x7f72d41a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.751+0000 7f72d89af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72d411b140 0x7f72d41a7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.751+0000 7f72d89af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72d41ab6d0 0x7f72d41abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.751+0000 7f72d89af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f72d4120be0 con 0x7f72d411b140 2024-09-11T23:40:27.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.751+0000 7f72d89af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72d4120a60 con 0x7f72d41ab6d0 2024-09-11T23:40:27.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.751+0000 7f72d89af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72d4120d60 con 0x7f72d4110190 2024-09-11T23:40:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.751+0000 7f72d37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d4110190 0x7f72d41a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.751+0000 7f72d37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d4110190 0x7f72d41a6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42164/0 (socket says 172.21.15.133:42164) 2024-09-11T23:40:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.751+0000 7f72d37fe640 1 -- 172.21.15.133:0/462407728 learned_addr learned my addr 172.21.15.133:0/462407728 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.751+0000 7f72d3fff640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72d41ab6d0 0x7f72d41abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.752+0000 7f72d37fe640 1 -- 172.21.15.133:0/462407728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72d41ab6d0 msgr2=0x7f72d41abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.752+0000 7f72d37fe640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72d41ab6d0 0x7f72d41abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.752+0000 7f72d37fe640 1 -- 172.21.15.133:0/462407728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72d411b140 msgr2=0x7f72d41a7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.752+0000 7f72d37fe640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72d411b140 0x7f72d41a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.752+0000 7f72d37fe640 1 -- 172.21.15.133:0/462407728 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d41ac220 con 0x7f72d4110190 2024-09-11T23:40:27.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.754+0000 7f72d37fe640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d4110190 0x7f72d41a6e00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e9f0 tx=0x7f72c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.755+0000 7f72d0ff9640 1 -- 172.21.15.133:0/462407728 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c400cd80 con 0x7f72d4110190 2024-09-11T23:40:27.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.755+0000 7f72d0ff9640 1 -- 172.21.15.133:0/462407728 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c4010910 con 0x7f72d4110190 2024-09-11T23:40:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.760+0000 7fdd3dffb640 1 -- 172.21.15.133:0/3054035911 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd340319b0 con 0x7fdd40108d10 2024-09-11T23:40:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.761+0000 7fdd459b7640 1 -- 172.21.15.133:0/3054035911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd40108d10 msgr2=0x7fdd4010cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.761+0000 7fdd459b7640 1 --2- 172.21.15.133:0/3054035911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd40108d10 0x7fdd4010cd50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009900 tx=0x7fdd3402a400 comp rx=0 tx=0).stop 2024-09-11T23:40:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.761+0000 7fdd459b7640 1 -- 172.21.15.133:0/3054035911 shutdown_connections 2024-09-11T23:40:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.761+0000 7fdd459b7640 1 --2- 172.21.15.133:0/3054035911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd4010d4b0 0x7fdd40115000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.761+0000 7fdd459b7640 1 --2- 172.21.15.133:0/3054035911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd40108d10 0x7fdd4010cd50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.761+0000 7fdd459b7640 1 --2- 172.21.15.133:0/3054035911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd40108360 0x7fdd40108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.761+0000 7fdd459b7640 1 -- 172.21.15.133:0/3054035911 >> 172.21.15.133:0/3054035911 conn(0x7fdd40077ad0 msgr2=0x7fdd40079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.755+0000 7f72d0ff9640 1 -- 172.21.15.133:0/462407728 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4018990 con 0x7f72d4110190 2024-09-11T23:40:27.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.755+0000 7f72d89af640 1 -- 172.21.15.133:0/462407728 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d41ac510 con 0x7f72d4110190 2024-09-11T23:40:27.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.755+0000 7f72d89af640 1 -- 172.21.15.133:0/462407728 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d4110b80 con 0x7f72d4110190 2024-09-11T23:40:27.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.761+0000 7f72d0ff9640 1 -- 172.21.15.133:0/462407728 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f72c4010ac0 con 0x7f72d4110190 2024-09-11T23:40:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.767+0000 7fdd459b7640 1 -- 172.21.15.133:0/3054035911 shutdown_connections 2024-09-11T23:40:27.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.762+0000 7f72d0ff9640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f72bc0a2db0 0x7f72bc0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.763+0000 7f72d0ff9640 1 -- 172.21.15.133:0/462407728 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f72c40cf7b0 con 0x7f72d4110190 2024-09-11T23:40:27.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.766+0000 7f72dbfea740 1 Processor -- start 2024-09-11T23:40:27.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.766+0000 7f72dbfea740 1 -- start start 2024-09-11T23:40:27.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.766+0000 7f72dbfea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e224d40 0x55563e223290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.766+0000 7f72dbfea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e223c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.766+0000 7f72dbfea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e230960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.766+0000 7f72dbfea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55563e1892f0 con 0x55563e2237d0 2024-09-11T23:40:27.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.766+0000 7f72dbfea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55563e130cf0 con 0x55563e224d40 2024-09-11T23:40:27.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.766+0000 7f72dbfea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55563e1852b0 con 0x55563e22e570 2024-09-11T23:40:27.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.766+0000 7f72d2ffd640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f72bc0a2db0 0x7f72bc0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.767+0000 7f72d37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e224d40 0x55563e223290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.767+0000 7f72d37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e224d40 0x55563e223290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38092/0 (socket says 172.21.15.133:38092) 2024-09-11T23:40:27.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.767+0000 7f72d37fe640 1 -- 172.21.15.133:0/1850833412 learned_addr learned my addr 172.21.15.133:0/1850833412 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:27.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.767+0000 7f72d3fff640 1 --2- 172.21.15.133:0/1850833412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e230960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.767+0000 7f72d3fff640 1 -- 172.21.15.133:0/1850833412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e224d40 msgr2=0x55563e223290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.767+0000 7f72d3fff640 1 --2- 172.21.15.133:0/1850833412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e224d40 0x55563e223290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.767+0000 7f72d3fff640 1 -- 172.21.15.133:0/1850833412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 msgr2=0x55563e223c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.767+0000 7f72d3fff640 1 --2- 172.21.15.133:0/1850833412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e223c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.767+0000 7f72d3fff640 1 -- 172.21.15.133:0/1850833412 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563e1153a0 con 0x55563e22e570 2024-09-11T23:40:27.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.768+0000 7f72d3fff640 1 --2- 172.21.15.133:0/1850833412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e230960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009a30 tx=0x7f72c8009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92179dec50ae43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.768+0000 7f72b9ffb640 1 -- 172.21.15.133:0/1850833412 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8011d40 con 0x55563e22e570 2024-09-11T23:40:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.767+0000 7fdd459b7640 1 -- 172.21.15.133:0/3054035911 wait complete. 2024-09-11T23:40:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.768+0000 7fdd459b7640 1 Processor -- start 2024-09-11T23:40:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.769+0000 7fdd459b7640 1 -- start start 2024-09-11T23:40:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.769+0000 7fdd459b7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd40108360 0x7fdd40073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.769+0000 7fdd459b7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd4010d4b0 0x7fdd40073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.769+0000 7fdd459b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd4006cb20 0x7fdd4006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.769+0000 7fdd459b7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdd40118600 con 0x7fdd4006cb20 2024-09-11T23:40:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.769+0000 7fdd459b7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd40118480 con 0x7fdd4010d4b0 2024-09-11T23:40:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.769+0000 7fdd459b7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd40118780 con 0x7fdd40108360 2024-09-11T23:40:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.769+0000 7fdd3e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd4010d4b0 0x7fdd40073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.770+0000 7fdd3e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd4010d4b0 0x7fdd40073ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38106/0 (socket says 172.21.15.133:38106) 2024-09-11T23:40:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.770+0000 7fdd3e7fc640 1 -- 172.21.15.133:0/3229676973 learned_addr learned my addr 172.21.15.133:0/3229676973 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.770+0000 7fdd3effd640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd40108360 0x7fdd40073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.769+0000 7f72b9ffb640 1 -- 172.21.15.133:0/1850833412 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c8012930 con 0x55563e22e570 2024-09-11T23:40:27.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.770+0000 7f72b9ffb640 1 -- 172.21.15.133:0/1850833412 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c800fa30 con 0x55563e22e570 2024-09-11T23:40:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.770+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd4006cb20 0x7fdd4006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.771+0000 7f72d2ffd640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f72bc0a2db0 0x7f72bc0a5270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55563e220cc0 tx=0x7f72cc004ab0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.772+0000 7f72dbfea740 1 -- 172.21.15.133:0/1850833412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 msgr2=0x55563e230960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.772+0000 7f72dbfea740 1 --2- 172.21.15.133:0/1850833412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e230960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009a30 tx=0x7f72c8009ef0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.773+0000 7f72dbfea740 1 -- 172.21.15.133:0/1850833412 shutdown_connections 2024-09-11T23:40:27.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.773+0000 7f72dbfea740 1 --2- 172.21.15.133:0/1850833412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e230960 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.773+0000 7f72dbfea740 1 --2- 172.21.15.133:0/1850833412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e223c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.773+0000 7f72dbfea740 1 --2- 172.21.15.133:0/1850833412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e224d40 0x55563e223290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.773+0000 7f72dbfea740 1 -- 172.21.15.133:0/1850833412 >> 172.21.15.133:0/1850833412 conn(0x55563e1ec310 msgr2=0x55563e21fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.774+0000 7f72dbfea740 1 -- 172.21.15.133:0/1850833412 shutdown_connections 2024-09-11T23:40:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.776+0000 7fdd3e7fc640 1 -- 172.21.15.133:0/3229676973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd40108360 msgr2=0x7fdd40073990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.775+0000 7f72dbfea740 1 -- 172.21.15.133:0/1850833412 wait complete. 2024-09-11T23:40:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.776+0000 7fdd3e7fc640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd40108360 0x7fdd40073990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.776+0000 7fdd3e7fc640 1 -- 172.21.15.133:0/3229676973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd4006cb20 msgr2=0x7fdd4006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.776+0000 7fdd3e7fc640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd4006cb20 0x7fdd4006cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.776+0000 7fdd3e7fc640 1 -- 172.21.15.133:0/3229676973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd400714b0 con 0x7fdd4010d4b0 2024-09-11T23:40:27.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.775+0000 7f72dbfea740 1 -- start start 2024-09-11T23:40:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.778+0000 7fdd3e7fc640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd4010d4b0 0x7fdd40073ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdd3402a920 tx=0x7fdd34003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.778+0000 7f72dbfea740 1 Processor -- start 2024-09-11T23:40:27.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.779+0000 7f72dbfea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e22df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.780+0000 7f72dbfea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e224d40 0x55563e22b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.780+0000 7f72d37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e22df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.780+0000 7f72d37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e22df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59808/0 (socket says 172.21.15.133:59808) 2024-09-11T23:40:27.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.780+0000 7f72d37fe640 1 -- 172.21.15.133:0/2822556778 learned_addr learned my addr 172.21.15.133:0/2822556778 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:27.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.780+0000 7f72dbfea740 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e22e570 0x55563e22b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.781+0000 7f72d2ffd640 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e224d40 0x55563e22b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.781+0000 7f72dbfea740 1 -- 172.21.15.133:0/2822556778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55563e231270 con 0x55563e2237d0 2024-09-11T23:40:27.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.781+0000 7f72dbfea740 1 -- 172.21.15.133:0/2822556778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55563e144060 con 0x55563e22e570 2024-09-11T23:40:27.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.781+0000 7f72dbfea740 1 -- 172.21.15.133:0/2822556778 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55563e2313f0 con 0x55563e224d40 2024-09-11T23:40:27.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.781+0000 7f72d3fff640 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e22e570 0x55563e22b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.781+0000 7f72dbfea740 10 client.0 _ll_register_callbacks cb 0x55563de978c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.780+0000 7fdd449b5640 1 -- 172.21.15.133:0/3229676973 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3403f070 con 0x7fdd4010d4b0 2024-09-11T23:40:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.780+0000 7fdd449b5640 1 -- 172.21.15.133:0/3229676973 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd34031c90 con 0x7fdd4010d4b0 2024-09-11T23:40:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.780+0000 7fdd449b5640 1 -- 172.21.15.133:0/3229676973 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34035030 con 0x7fdd4010d4b0 2024-09-11T23:40:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.780+0000 7fdd459b7640 1 -- 172.21.15.133:0/3229676973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd40071740 con 0x7fdd4010d4b0 2024-09-11T23:40:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.780+0000 7fdd459b7640 1 -- 172.21.15.133:0/3229676973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd40071be0 con 0x7fdd4010d4b0 2024-09-11T23:40:27.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.782+0000 7f72d3fff640 1 -- 172.21.15.133:0/2822556778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e224d40 msgr2=0x55563e22b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.782+0000 7f72d3fff640 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e224d40 0x55563e22b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.782+0000 7f72d3fff640 1 -- 172.21.15.133:0/2822556778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 msgr2=0x55563e22df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.782+0000 7f72d3fff640 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e22df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.782+0000 7f72d3fff640 1 -- 172.21.15.133:0/2822556778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563defa3c0 con 0x55563e22e570 2024-09-11T23:40:27.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.782+0000 7f72d3fff640 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e22e570 0x55563e22b850 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f72c8000c00 tx=0x7f72c8012680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8812a04cdda336d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.783+0000 7f72b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:27.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.783+0000 7f72b2ffd640 1 -- 172.21.15.133:0/2822556778 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8028020 con 0x55563e22e570 2024-09-11T23:40:27.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.783+0000 7f72dbfea740 1 -- 172.21.15.133:0/2822556778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72c8012930 con 0x55563e22e570 2024-09-11T23:40:27.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.783+0000 7f72b2ffd640 1 -- 172.21.15.133:0/2822556778 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f72c8005070 con 0x55563e22e570 2024-09-11T23:40:27.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.785+0000 7f72b2ffd640 1 -- 172.21.15.133:0/2822556778 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72c8033050 con 0x55563e22e570 2024-09-11T23:40:27.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.785+0000 7f72dbfea740 10 client.25928 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:27.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.785+0000 7f72dbfea740 1 -- 172.21.15.133:0/2822556778 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55563e23cd40 con 0x55563e22e570 2024-09-11T23:40:27.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.786+0000 7f72b2ffd640 1 -- 172.21.15.133:0/2822556778 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c8012930 con 0x55563e22e570 2024-09-11T23:40:27.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.788+0000 7f72b2ffd640 1 -- 172.21.15.133:0/2822556778 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f72c80035f0 con 0x55563e22e570 2024-09-11T23:40:27.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.788+0000 7f72dbfea740 10 client.25928 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.788+0000 7f72dbfea740 10 client.25928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.788+0000 7f72dbfea740 20 client.25928 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.788+0000 7f72dbfea740 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55563e23e510 0x55563e240990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.788+0000 7f72dbfea740 4 client.25928 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.788+0000 7f72dbfea740 1 -- 172.21.15.133:0/2822556778 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55563e196860 con 0x55563e23e510 2024-09-11T23:40:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.787+0000 7fdd449b5640 1 -- 172.21.15.133:0/3229676973 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fdd34042070 con 0x7fdd4010d4b0 2024-09-11T23:40:27.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.789+0000 7f72d37fe640 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55563e23e510 0x55563e240990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.789+0000 7fdd4882c740 1 Processor -- start 2024-09-11T23:40:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.790+0000 7fdd449b5640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdd280a2f80 0x7fdd280a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.790+0000 7fdd449b5640 1 -- 172.21.15.133:0/3229676973 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fdd340f0bd0 con 0x7fdd4010d4b0 2024-09-11T23:40:27.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:27 smithi178 ceph-mon[35882]: pgmap v113: 97 pgs: 97 active+clean; 29 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 560 KiB/s rd, 636 MiB/s wr, 2.91k op/s 2024-09-11T23:40:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.795+0000 7fdd4882c740 1 -- start start 2024-09-11T23:40:27.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.795+0000 7f72b2ffd640 10 client.25928 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.795+0000 7f72d37fe640 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55563e23e510 0x55563e240990 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f6e7116f9f1798 server_cookie=253d67df2fe7ccf5 in_seq=0 out_seq=0 2024-09-11T23:40:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.795+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea2eaf0 0x563feea2eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.795+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea2f410 0x563feea39770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.795+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea39cb0 0x563feea3c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.795+0000 7fdd4882c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563fee9932f0 con 0x563feea2eaf0 2024-09-11T23:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.795+0000 7fdd4882c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563fee93b270 con 0x563feea2f410 2024-09-11T23:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.795+0000 7fdd4882c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563fee98f2b0 con 0x563feea39cb0 2024-09-11T23:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.797+0000 7fdd3e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea2f410 0x563feea39770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.797+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdd280a2f80 0x7fdd280a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.797+0000 7fdd3e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea2f410 0x563feea39770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38122/0 (socket says 172.21.15.133:38122) 2024-09-11T23:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.797+0000 7fdd3e7fc640 1 -- 172.21.15.133:0/1090267459 learned_addr learned my addr 172.21.15.133:0/1090267459 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:27.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.798+0000 7f72b2ffd640 1 -- 172.21.15.133:0/2822556778 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55563e196860 con 0x55563e23e510 2024-09-11T23:40:27.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.798+0000 7f72b2ffd640 10 client.25928 handle_command_reply: tid=0 2024-09-11T23:40:27.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.798+0000 7f72dbfea740 1 client.25928 shutdown 2024-09-11T23:40:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.801+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/1090267459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea39cb0 0x563feea3c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.801+0000 7fdd3f7fe640 1 -- 172.21.15.133:0/1090267459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea2f410 msgr2=0x563feea39770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.801+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/1090267459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea2f410 0x563feea39770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.801+0000 7fdd3f7fe640 1 -- 172.21.15.133:0/1090267459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea2eaf0 msgr2=0x563feea2eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.801+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/1090267459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea2eaf0 0x563feea2eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.801+0000 7fdd3f7fe640 1 -- 172.21.15.133:0/1090267459 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fee93a340 con 0x563feea39cb0 2024-09-11T23:40:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.801+0000 7f72dbfea740 1 -- 172.21.15.133:0/2822556778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e22e570 msgr2=0x55563e22b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.801+0000 7f72dbfea740 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e22e570 0x55563e22b850 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f72c8000c00 tx=0x7f72c8012680 comp rx=0 tx=0).stop 2024-09-11T23:40:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.801+0000 7f72dbfea740 1 -- 172.21.15.133:0/2822556778 shutdown_connections 2024-09-11T23:40:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.801+0000 7f72dbfea740 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55563e23e510 0x55563e240990 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.801+0000 7f72dbfea740 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e22e570 0x55563e22b850 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.801+0000 7f72dbfea740 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e224d40 0x55563e22b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.801+0000 7f72dbfea740 1 --2- 172.21.15.133:0/2822556778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e22df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.801+0000 7f72dbfea740 1 -- 172.21.15.133:0/2822556778 >> 172.21.15.133:0/2822556778 conn(0x55563e1ec310 msgr2=0x55563e225b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.802+0000 7f72b2ffd640 0 client.25928 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.802+0000 7f72dbfea740 1 -- 172.21.15.133:0/2822556778 shutdown_connections 2024-09-11T23:40:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.805+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/1090267459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea39cb0 0x563feea3c0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009d30 tx=0x7fdd38018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5e24cecc0caffcd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.806+0000 7fdd25ffb640 1 -- 172.21.15.133:0/1090267459 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38020070 con 0x563feea39cb0 2024-09-11T23:40:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.807+0000 7fdd25ffb640 1 -- 172.21.15.133:0/1090267459 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd38016040 con 0x563feea39cb0 2024-09-11T23:40:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.807+0000 7fdd25ffb640 1 -- 172.21.15.133:0/1090267459 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38025d00 con 0x563feea39cb0 2024-09-11T23:40:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.808+0000 7fdd4882c740 1 -- 172.21.15.133:0/1090267459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea39cb0 msgr2=0x563feea3c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.809+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1090267459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea39cb0 0x563feea3c0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009d30 tx=0x7fdd38018000 comp rx=0 tx=0).stop 2024-09-11T23:40:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.810+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdd280a2f80 0x7fdd280a5440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800c790 tx=0x7fdd3800cb90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.802+0000 7f72dbfea740 1 -- 172.21.15.133:0/2822556778 wait complete. 2024-09-11T23:40:27.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.802+0000 7f72dbfea740 20 client.25928 trim_cache size 0 max 16384 2024-09-11T23:40:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.811+0000 7fdd4882c740 1 -- 172.21.15.133:0/1090267459 shutdown_connections 2024-09-11T23:40:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.811+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1090267459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea39cb0 0x563feea3c0a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.811+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1090267459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea2f410 0x563feea39770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.811+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1090267459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea2eaf0 0x563feea2eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.811+0000 7fdd4882c740 1 -- 172.21.15.133:0/1090267459 >> 172.21.15.133:0/1090267459 conn(0x563feea2b060 msgr2=0x563feea2b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.814+0000 7fdd4882c740 1 -- 172.21.15.133:0/1090267459 shutdown_connections 2024-09-11T23:40:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.814+0000 7fdd4882c740 1 -- 172.21.15.133:0/1090267459 wait complete. 2024-09-11T23:40:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.815+0000 7fdd4882c740 1 -- start start 2024-09-11T23:40:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.815+0000 7fdd4882c740 1 Processor -- start 2024-09-11T23:40:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.816+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 0x563feea36c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.816+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea2f410 0x563feea37170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.816+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 0x563feea376b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.816+0000 7fdd4882c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563feea3c880 con 0x563feea39cb0 2024-09-11T23:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.816+0000 7fdd4882c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563fee91c290 con 0x563feea2f410 2024-09-11T23:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.816+0000 7fdd4882c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563feea3ca00 con 0x563feea2eaf0 2024-09-11T23:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.816+0000 7fdd4882c740 10 client.0 _ll_register_callbacks cb 0x563fee94d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.820+0000 7fdd3f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 0x563feea376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.820+0000 7fdd3f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 0x563feea376b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59824/0 (socket says 172.21.15.133:59824) 2024-09-11T23:40:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.820+0000 7fdd3f7fe640 1 -- 172.21.15.133:0/3519829286 learned_addr learned my addr 172.21.15.133:0/3519829286 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.821+0000 7fdd3effd640 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 0x563feea36c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.822+0000 7fdd3e7fc640 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea2f410 0x563feea37170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.822+0000 7fdd3effd640 1 -- 172.21.15.133:0/3519829286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea2f410 msgr2=0x563feea37170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.822+0000 7fdd3effd640 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea2f410 0x563feea37170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.822+0000 7fdd3effd640 1 -- 172.21.15.133:0/3519829286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 msgr2=0x563feea376b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.822+0000 7fdd3effd640 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 0x563feea376b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.822+0000 7fdd3effd640 1 -- 172.21.15.133:0/3519829286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fee6a6210 con 0x563feea2eaf0 2024-09-11T23:40:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.825+0000 7fdd3effd640 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 0x563feea36c30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000e450 tx=0x7fdd3000e910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5802366692a246f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.825+0000 7fdd1effd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.828+0000 7fdd1effd640 1 -- 172.21.15.133:0/3519829286 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30017070 con 0x563feea2eaf0 2024-09-11T23:40:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.829+0000 7fdd1effd640 1 -- 172.21.15.133:0/3519829286 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fdd30013680 con 0x563feea2eaf0 2024-09-11T23:40:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.831+0000 7fdd4882c740 1 -- 172.21.15.133:0/3519829286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd38016040 con 0x563feea2eaf0 2024-09-11T23:40:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.833+0000 7fdd1effd640 1 -- 172.21.15.133:0/3519829286 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd30021050 con 0x563feea2eaf0 2024-09-11T23:40:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.833+0000 7fdd4882c740 10 client.25906 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.833+0000 7fdd4882c740 1 -- 172.21.15.133:0/3519829286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563fee772620 con 0x563feea2eaf0 2024-09-11T23:40:27.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.833+0000 7f72dbfea740 1 Processor -- start 2024-09-11T23:40:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.833+0000 7fdd1effd640 1 -- 172.21.15.133:0/3519829286 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd38016040 con 0x563feea2eaf0 2024-09-11T23:40:27.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.833+0000 7f72dbfea740 1 -- start start 2024-09-11T23:40:27.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.834+0000 7f72dbfea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e23a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.834+0000 7f72dbfea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e224d40 0x55563e22ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.834+0000 7f72dbfea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e226320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.834+0000 7f72dbfea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41c4d50 con 0x55563e2237d0 2024-09-11T23:40:27.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.834+0000 7f72dbfea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41c6890 con 0x55563e224d40 2024-09-11T23:40:27.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.834+0000 7f72dbfea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41c5420 con 0x55563e22e570 2024-09-11T23:40:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.835+0000 7fdd1effd640 1 -- 172.21.15.133:0/3519829286 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd3001ca10 con 0x563feea2eaf0 2024-09-11T23:40:27.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.835+0000 7f72d37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e23a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.835+0000 7f72d37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e23a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59828/0 (socket says 172.21.15.133:59828) 2024-09-11T23:40:27.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.835+0000 7f72d37fe640 1 -- 172.21.15.133:0/832469697 learned_addr learned my addr 172.21.15.133:0/832469697 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:27.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.835+0000 7f72d2ffd640 1 --2- 172.21.15.133:0/832469697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e224d40 0x55563e22ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.835+0000 7fdd4882c740 10 client.25906 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.835+0000 7fdd4882c740 10 client.25906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.835+0000 7fdd4882c740 20 client.25906 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:27.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.835+0000 7f72d3fff640 1 --2- 172.21.15.133:0/832469697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e226320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.835+0000 7fdd4882c740 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563feea49d50 0x563feea4c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.835+0000 7fdd4882c740 4 client.25906 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.835+0000 7fdd4882c740 1 -- 172.21.15.133:0/3519829286 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fee9a0a20 con 0x563feea49d50 2024-09-11T23:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.836+0000 7fdd3e7fc640 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563feea49d50 0x563feea4c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.836+0000 7f72d3fff640 1 -- 172.21.15.133:0/832469697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e224d40 msgr2=0x55563e22ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.836+0000 7f72d3fff640 1 --2- 172.21.15.133:0/832469697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e224d40 0x55563e22ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.836+0000 7f72d3fff640 1 -- 172.21.15.133:0/832469697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 msgr2=0x55563e23a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.836+0000 7f72d3fff640 1 --2- 172.21.15.133:0/832469697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e23a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.836+0000 7f72d3fff640 1 -- 172.21.15.133:0/832469697 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563e224960 con 0x55563e22e570 2024-09-11T23:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.837+0000 7fdd3e7fc640 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563feea49d50 0x563feea4c140 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11bfef3a591a375 server_cookie=701b0ad93e5fe8f6 in_seq=0 out_seq=0 2024-09-11T23:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.837+0000 7fdd1effd640 10 client.25906 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.839+0000 7fdd1effd640 1 -- 172.21.15.133:0/3519829286 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fee9a0a20 con 0x563feea49d50 2024-09-11T23:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.839+0000 7fdd1effd640 10 client.25906 handle_command_reply: tid=0 2024-09-11T23:40:27.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.839+0000 7fdd4882c740 1 client.25906 shutdown 2024-09-11T23:40:27.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.842+0000 7f72d3fff640 1 --2- 172.21.15.133:0/832469697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e226320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f72c8003b60 tx=0x7f72c8011fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69da1074af7613ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.843+0000 7fdd4882c740 1 -- 172.21.15.133:0/3519829286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 msgr2=0x563feea36c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.843+0000 7fdd4882c740 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 0x563feea36c30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000e450 tx=0x7fdd3000e910 comp rx=0 tx=0).stop 2024-09-11T23:40:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.844+0000 7fdd4882c740 1 -- 172.21.15.133:0/3519829286 shutdown_connections 2024-09-11T23:40:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.844+0000 7fdd4882c740 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563feea49d50 0x563feea4c140 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.844+0000 7fdd4882c740 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 0x563feea376b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.844+0000 7fdd4882c740 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea2f410 0x563feea37170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.844+0000 7fdd4882c740 1 --2- 172.21.15.133:0/3519829286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 0x563feea36c30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.844+0000 7fdd1effd640 0 client.25906 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.844+0000 7fdd4882c740 1 -- 172.21.15.133:0/3519829286 >> 172.21.15.133:0/3519829286 conn(0x563feea2b060 msgr2=0x563feea30dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.845+0000 7fdd4882c740 1 -- 172.21.15.133:0/3519829286 shutdown_connections 2024-09-11T23:40:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.845+0000 7fdd4882c740 1 -- 172.21.15.133:0/3519829286 wait complete. 2024-09-11T23:40:27.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.846+0000 7f72b97fa640 1 -- 172.21.15.133:0/832469697 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8028070 con 0x55563e22e570 2024-09-11T23:40:27.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.846+0000 7f72b97fa640 1 -- 172.21.15.133:0/832469697 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c8009260 con 0x55563e22e570 2024-09-11T23:40:27.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.847+0000 7f72dbfea740 1 -- 172.21.15.133:0/832469697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 msgr2=0x55563e226320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.847+0000 7f72dbfea740 1 --2- 172.21.15.133:0/832469697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e226320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f72c8003b60 tx=0x7f72c8011fc0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.848+0000 7fdd4882c740 20 client.25906 trim_cache size 0 max 16384 2024-09-11T23:40:27.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.850+0000 7f72dbfea740 1 -- 172.21.15.133:0/832469697 shutdown_connections 2024-09-11T23:40:27.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.850+0000 7f72dbfea740 1 --2- 172.21.15.133:0/832469697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e226320 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.850+0000 7f72dbfea740 1 --2- 172.21.15.133:0/832469697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e224d40 0x55563e22ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.850+0000 7f72dbfea740 1 --2- 172.21.15.133:0/832469697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e2237d0 0x55563e23a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.850+0000 7f72dbfea740 1 -- 172.21.15.133:0/832469697 >> 172.21.15.133:0/832469697 conn(0x55563e1ec310 msgr2=0x55563e240520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.855+0000 7f72dbfea740 1 -- 172.21.15.133:0/832469697 shutdown_connections 2024-09-11T23:40:27.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.855+0000 7f72dbfea740 1 -- 172.21.15.133:0/832469697 wait complete. 2024-09-11T23:40:27.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.855+0000 7f72dbfea740 1 -- start start 2024-09-11T23:40:27.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.856+0000 7f72dbfea740 1 Processor -- start 2024-09-11T23:40:27.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.856+0000 7f72dbfea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e2237d0 0x55563e231640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.856+0000 7f72dbfea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e224d40 0x55563e231b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.856+0000 7f72dbfea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e2320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.856+0000 7f72dbfea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41c61c0 con 0x55563e224d40 2024-09-11T23:40:27.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.857+0000 7f72dbfea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41c5af0 con 0x55563e2237d0 2024-09-11T23:40:27.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.857+0000 7f72dbfea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72d4117580 con 0x55563e22e570 2024-09-11T23:40:27.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.857+0000 7f72dbfea740 10 client.0 _ll_register_callbacks cb 0x7f72c40978c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:27.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.861+0000 7f72d37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e2237d0 0x55563e231640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.861+0000 7f72d3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e2320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.861+0000 7f72d3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e2320c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42240/0 (socket says 172.21.15.133:42240) 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.861+0000 7f72d3fff640 1 -- 172.21.15.133:0/1758442909 learned_addr learned my addr 172.21.15.133:0/1758442909 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.862+0000 7f72d2ffd640 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e224d40 0x55563e231b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.863+0000 7f72d37fe640 1 -- 172.21.15.133:0/1758442909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 msgr2=0x55563e2320c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.863+0000 7f72d37fe640 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e2320c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.863+0000 7f72d37fe640 1 -- 172.21.15.133:0/1758442909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e224d40 msgr2=0x55563e231b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.863+0000 7f72d37fe640 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e224d40 0x55563e231b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.863+0000 7f72d37fe640 1 -- 172.21.15.133:0/1758442909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563e2a74b0 con 0x55563e2237d0 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.863+0000 7f72d37fe640 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e2237d0 0x55563e231640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72c409ba40 tx=0x7f72c409c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81225ea1ea5e4f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.863+0000 7f72b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.863+0000 7f72b2ffd640 1 -- 172.21.15.133:0/1758442909 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c40b8070 con 0x55563e2237d0 2024-09-11T23:40:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.864+0000 7f72dbfea740 1 -- 172.21.15.133:0/1758442909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55563e190d70 con 0x55563e2237d0 2024-09-11T23:40:27.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.864+0000 7f72b2ffd640 1 -- 172.21.15.133:0/1758442909 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f72c40983b0 con 0x55563e2237d0 2024-09-11T23:40:27.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.868+0000 7f72b2ffd640 1 -- 172.21.15.133:0/1758442909 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72c4018b30 con 0x55563e2237d0 2024-09-11T23:40:27.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.868+0000 7f72dbfea740 10 client.25940 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.868+0000 7f72dbfea740 1 -- 172.21.15.133:0/1758442909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55563e2ca890 con 0x55563e2237d0 2024-09-11T23:40:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.871+0000 7f72b2ffd640 1 -- 172.21.15.133:0/1758442909 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55563e190d70 con 0x55563e2237d0 2024-09-11T23:40:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.872+0000 7f72b2ffd640 1 -- 172.21.15.133:0/1758442909 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f72c40ac020 con 0x55563e2237d0 2024-09-11T23:40:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.872+0000 7f72dbfea740 10 client.25940 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.872+0000 7f72dbfea740 10 client.25940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.872+0000 7f72dbfea740 20 client.25940 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.872+0000 7f72dbfea740 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55563e2cb000 0x55563e2cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.872+0000 7f72dbfea740 4 client.25940 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:40:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.872+0000 7f72dbfea740 1 -- 172.21.15.133:0/1758442909 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55563e10d170 con 0x55563e2cb000 2024-09-11T23:40:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.875+0000 7f72d2ffd640 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55563e2cb000 0x55563e2cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.889+0000 7f72b2ffd640 10 client.25940 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:27.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.889+0000 7f72d2ffd640 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55563e2cb000 0x55563e2cd3f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=635091f7effd8628 server_cookie=19d5543fa1c776a7 in_seq=0 out_seq=0 2024-09-11T23:40:27.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.894+0000 7f72b2ffd640 1 -- 172.21.15.133:0/1758442909 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55563e10d170 con 0x55563e2cb000 2024-09-11T23:40:27.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.894+0000 7f72b2ffd640 10 client.25940 handle_command_reply: tid=0 2024-09-11T23:40:27.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.894+0000 7f72dbfea740 1 client.25940 shutdown 2024-09-11T23:40:27.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.898+0000 7f72dbfea740 1 -- 172.21.15.133:0/1758442909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e2237d0 msgr2=0x55563e231640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.898+0000 7f72dbfea740 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e2237d0 0x55563e231640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72c409ba40 tx=0x7f72c409c440 comp rx=0 tx=0).stop 2024-09-11T23:40:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.899+0000 7f72dbfea740 1 -- 172.21.15.133:0/1758442909 shutdown_connections 2024-09-11T23:40:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.899+0000 7f72dbfea740 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55563e2cb000 0x55563e2cd3f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.899+0000 7f72dbfea740 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55563e22e570 0x55563e2320c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.899+0000 7f72dbfea740 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55563e224d40 0x55563e231b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.899+0000 7f72dbfea740 1 --2- 172.21.15.133:0/1758442909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55563e2237d0 0x55563e231640 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.899+0000 7f72dbfea740 1 -- 172.21.15.133:0/1758442909 >> 172.21.15.133:0/1758442909 conn(0x55563e1ec310 msgr2=0x55563e21e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.899+0000 7f72b2ffd640 0 client.25940 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.899+0000 7f72dbfea740 1 -- 172.21.15.133:0/1758442909 shutdown_connections 2024-09-11T23:40:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.899+0000 7f72dbfea740 1 -- 172.21.15.133:0/1758442909 wait complete. 2024-09-11T23:40:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.900+0000 7f72dbfea740 20 client.25940 trim_cache size 0 max 16384 2024-09-11T23:40:27.903 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:40:27.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.908+0000 7f72d89af640 1 -- 172.21.15.133:0/462407728 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f72bc0a2db0 msgr2=0x7f72bc0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.908+0000 7f72d89af640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f72bc0a2db0 0x7f72bc0a5270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55563e220cc0 tx=0x7f72cc004ab0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.908+0000 7f72d89af640 1 -- 172.21.15.133:0/462407728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d4110190 msgr2=0x7f72d41a6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.908+0000 7f72d89af640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d4110190 0x7f72d41a6e00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e9f0 tx=0x7f72c400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.912+0000 7f72d89af640 1 -- 172.21.15.133:0/462407728 shutdown_connections 2024-09-11T23:40:27.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.912+0000 7f72d89af640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f72bc0a2db0 0x7f72bc0a5270 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.912+0000 7f72d89af640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72d41ab6d0 0x7f72d41abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.912+0000 7f72d89af640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72d411b140 0x7f72d41a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.912+0000 7f72d89af640 1 --2- 172.21.15.133:0/462407728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d4110190 0x7f72d41a6e00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.912+0000 7f72d89af640 1 -- 172.21.15.133:0/462407728 >> 172.21.15.133:0/462407728 conn(0x7f72d4073180 msgr2=0x7f72d4119430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.914+0000 7f72d89af640 1 -- 172.21.15.133:0/462407728 shutdown_connections 2024-09-11T23:40:27.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:27.914+0000 7f72d89af640 1 -- 172.21.15.133:0/462407728 wait complete. 2024-09-11T23:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.916+0000 7fdd4882c740 1 Processor -- start 2024-09-11T23:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.917+0000 7fdd4882c740 1 -- start start 2024-09-11T23:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.917+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea2eaf0 0x563feea315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.917+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2f410 0x563feea374e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.917+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea39cb0 0x563feea51a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.917+0000 7fdd4882c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c00e690 con 0x563feea2eaf0 2024-09-11T23:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.917+0000 7fdd4882c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c0101d0 con 0x563feea39cb0 2024-09-11T23:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.917+0000 7fdd4882c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c00ed60 con 0x563feea2f410 2024-09-11T23:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.917+0000 7fdd3e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2f410 0x563feea374e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.917+0000 7fdd3e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2f410 0x563feea374e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42248/0 (socket says 172.21.15.133:42248) 2024-09-11T23:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.917+0000 7fdd3e7fc640 1 -- 172.21.15.133:0/1434657916 learned_addr learned my addr 172.21.15.133:0/1434657916 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.918+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/1434657916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea39cb0 0x563feea51a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.918+0000 7fdd3e7fc640 1 -- 172.21.15.133:0/1434657916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea39cb0 msgr2=0x563feea51a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.918+0000 7fdd3e7fc640 1 --2- 172.21.15.133:0/1434657916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea39cb0 0x563feea51a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.918+0000 7fdd3e7fc640 1 -- 172.21.15.133:0/1434657916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea2eaf0 msgr2=0x563feea315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.918+0000 7fdd3e7fc640 1 --2- 172.21.15.133:0/1434657916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea2eaf0 0x563feea315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.918+0000 7fdd3e7fc640 1 -- 172.21.15.133:0/1434657916 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563feea466c0 con 0x563feea2f410 2024-09-11T23:40:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.920+0000 7fdd3e7fc640 1 --2- 172.21.15.133:0/1434657916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2f410 0x563feea374e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd340b9560 tx=0x7fdd34038000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71a464b7f913d43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.924+0000 7fdd257fa640 1 -- 172.21.15.133:0/1434657916 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd340c14b0 con 0x563feea2f410 2024-09-11T23:40:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.924+0000 7fdd257fa640 1 -- 172.21.15.133:0/1434657916 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd340c5d70 con 0x563feea2f410 2024-09-11T23:40:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.925+0000 7fdd4882c740 1 -- 172.21.15.133:0/1434657916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2f410 msgr2=0x563feea374e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.925+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1434657916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2f410 0x563feea374e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd340b9560 tx=0x7fdd34038000 comp rx=0 tx=0).stop 2024-09-11T23:40:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.926+0000 7fdd4882c740 1 -- 172.21.15.133:0/1434657916 shutdown_connections 2024-09-11T23:40:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.926+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1434657916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea39cb0 0x563feea51a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.926+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1434657916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2f410 0x563feea374e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.926+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1434657916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea2eaf0 0x563feea315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.926+0000 7fdd4882c740 1 -- 172.21.15.133:0/1434657916 >> 172.21.15.133:0/1434657916 conn(0x563feea2b060 msgr2=0x563feea38cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.937+0000 7fdd4882c740 1 -- 172.21.15.133:0/1434657916 shutdown_connections 2024-09-11T23:40:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.937+0000 7fdd4882c740 1 -- 172.21.15.133:0/1434657916 wait complete. 2024-09-11T23:40:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.938+0000 7fdd4882c740 1 -- start start 2024-09-11T23:40:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.939+0000 7fdd4882c740 1 Processor -- start 2024-09-11T23:40:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.939+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 0x563feea42300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.939+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 0x563feea33780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.939+0000 7fdd4882c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea33cc0 0x563feea34140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.939+0000 7fdd4882c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c00fb00 con 0x563feea39cb0 2024-09-11T23:40:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.939+0000 7fdd4882c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c00f430 con 0x563feea33cc0 2024-09-11T23:40:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.939+0000 7fdd4882c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c00c670 con 0x563feea2eaf0 2024-09-11T23:40:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.940+0000 7fdd4882c740 10 client.0 _ll_register_callbacks cb 0x7fdd300093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.940+0000 7fdd3effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 0x563feea42300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.940+0000 7fdd3effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 0x563feea42300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42256/0 (socket says 172.21.15.133:42256) 2024-09-11T23:40:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.940+0000 7fdd3effd640 1 -- 172.21.15.133:0/1271351024 learned_addr learned my addr 172.21.15.133:0/1271351024 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.940+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea33cc0 0x563feea34140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.940+0000 7fdd3f7fe640 1 -- 172.21.15.133:0/1271351024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 msgr2=0x563feea42300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.940+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 0x563feea42300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.940+0000 7fdd3e7fc640 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 0x563feea33780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.940+0000 7fdd3f7fe640 1 -- 172.21.15.133:0/1271351024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 msgr2=0x563feea33780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.940+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 0x563feea33780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.940+0000 7fdd3f7fe640 1 -- 172.21.15.133:0/1271351024 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563feeab3a00 con 0x563feea33cc0 2024-09-11T23:40:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.941+0000 7fdd3e7fc640 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 0x563feea33780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.941+0000 7fdd3f7fe640 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea33cc0 0x563feea34140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdd380070b0 tx=0x7fdd3801bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c7c70eb29c2ee83 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.945+0000 7fdd1effd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.945+0000 7fdd1effd640 1 -- 172.21.15.133:0/1271351024 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38020040 con 0x563feea33cc0 2024-09-11T23:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.946+0000 7fdd1effd640 1 -- 172.21.15.133:0/1271351024 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fdd38025660 con 0x563feea33cc0 2024-09-11T23:40:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.947+0000 7fdd4882c740 1 -- 172.21.15.133:0/1271351024 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563fee99ad70 con 0x563feea33cc0 2024-09-11T23:40:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.948+0000 7fdd1effd640 1 -- 172.21.15.133:0/1271351024 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd38025860 con 0x563feea33cc0 2024-09-11T23:40:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.949+0000 7fdd4882c740 10 client.25946 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.949+0000 7fdd4882c740 1 -- 172.21.15.133:0/1271351024 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563feeab4d90 con 0x563feea33cc0 2024-09-11T23:40:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.950+0000 7fdd1effd640 1 -- 172.21.15.133:0/1271351024 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563fee99ad70 con 0x563feea33cc0 2024-09-11T23:40:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.952+0000 7fdd1effd640 1 -- 172.21.15.133:0/1271351024 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd3801cad0 con 0x563feea33cc0 2024-09-11T23:40:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.952+0000 7fdd4882c740 10 client.25946 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.952+0000 7fdd4882c740 10 client.25946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.952+0000 7fdd4882c740 20 client.25946 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.952+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563feeab5290 0x563feeab76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.953+0000 7fdd4882c740 4 client.25946 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:40:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.953+0000 7fdd4882c740 1 -- 172.21.15.133:0/1271351024 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fee917900 con 0x563feeab5290 2024-09-11T23:40:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.958+0000 7fdd3effd640 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563feeab5290 0x563feeab76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.963+0000 7fdd1effd640 10 client.25946 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:27.963+0000 7fdd3effd640 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563feeab5290 0x563feeab76b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fafcd36a6d486fdf server_cookie=b81051b88e8bd2d3 in_seq=0 out_seq=0 2024-09-11T23:40:29.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:29 smithi178 ceph-mon[35882]: pgmap v114: 97 pgs: 97 active+clean; 29 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 4.11k op/s 2024-09-11T23:40:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.822+0000 7fdd1effd640 1 -- 172.21.15.133:0/1271351024 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563fee917900 con 0x563feeab5290 2024-09-11T23:40:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.822+0000 7fdd1effd640 10 client.25946 handle_command_reply: tid=0 2024-09-11T23:40:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.822+0000 7fdd4882c740 1 client.25946 shutdown 2024-09-11T23:40:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.823+0000 7fdd4882c740 1 -- 172.21.15.133:0/1271351024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea33cc0 msgr2=0x563feea34140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.823+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea33cc0 0x563feea34140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdd380070b0 tx=0x7fdd3801bf20 comp rx=0 tx=0).stop 2024-09-11T23:40:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.823+0000 7fdd4882c740 1 -- 172.21.15.133:0/1271351024 shutdown_connections 2024-09-11T23:40:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.823+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563feeab5290 0x563feeab76b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.823+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563feea33cc0 0x563feea34140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.823+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563feea39cb0 0x563feea33780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.823+0000 7fdd4882c740 1 --2- 172.21.15.133:0/1271351024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563feea2eaf0 0x563feea42300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.823+0000 7fdd4882c740 1 -- 172.21.15.133:0/1271351024 >> 172.21.15.133:0/1271351024 conn(0x563feea2b060 msgr2=0x563feea38490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.823+0000 7fdd1effd640 0 client.25946 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.826+0000 7fdd4882c740 1 -- 172.21.15.133:0/1271351024 shutdown_connections 2024-09-11T23:40:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.827+0000 7fdd4882c740 1 -- 172.21.15.133:0/1271351024 wait complete. 2024-09-11T23:40:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.828+0000 7fdd4882c740 20 client.25946 trim_cache size 0 max 16384 2024-09-11T23:40:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:40:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:40:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:40:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:40:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.839+0000 7fdd459b7640 1 -- 172.21.15.133:0/3229676973 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdd280a2f80 msgr2=0x7fdd280a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.839+0000 7fdd459b7640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdd280a2f80 0x7fdd280a5440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800c790 tx=0x7fdd3800cb90 comp rx=0 tx=0).stop 2024-09-11T23:40:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.840+0000 7fdd459b7640 1 -- 172.21.15.133:0/3229676973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd4010d4b0 msgr2=0x7fdd40073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.840+0000 7fdd459b7640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd4010d4b0 0x7fdd40073ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdd3402a920 tx=0x7fdd34003c00 comp rx=0 tx=0).stop 2024-09-11T23:40:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.840+0000 7fdd459b7640 1 -- 172.21.15.133:0/3229676973 shutdown_connections 2024-09-11T23:40:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.840+0000 7fdd459b7640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fdd280a2f80 0x7fdd280a5440 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.841+0000 7fdd459b7640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd4006cb20 0x7fdd4006cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.841+0000 7fdd459b7640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd4010d4b0 0x7fdd40073ed0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.841+0000 7fdd459b7640 1 --2- 172.21.15.133:0/3229676973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd40108360 0x7fdd40073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.841+0000 7fdd459b7640 1 -- 172.21.15.133:0/3229676973 >> 172.21.15.133:0/3229676973 conn(0x7fdd40077ad0 msgr2=0x7fdd40078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.841+0000 7fdd459b7640 1 -- 172.21.15.133:0/3229676973 shutdown_connections 2024-09-11T23:40:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:29.841+0000 7fdd459b7640 1 -- 172.21.15.133:0/3229676973 wait complete. 2024-09-11T23:40:29.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:29 smithi138 ceph-mon[35142]: pgmap v114: 97 pgs: 97 active+clean; 29 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 4.11k op/s 2024-09-11T23:40:29.962 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:29 smithi133 ceph-mon[31430]: pgmap v114: 97 pgs: 97 active+clean; 29 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 4.11k op/s 2024-09-11T23:40:30.330 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:40:30.330 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:40:31.248 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:31.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:31 smithi133 ceph-mon[31430]: pgmap v115: 97 pgs: 97 active+clean; 29 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 625 MiB/s wr, 3.41k op/s 2024-09-11T23:40:31.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:31 smithi138 ceph-mon[35142]: pgmap v115: 97 pgs: 97 active+clean; 29 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 625 MiB/s wr, 3.41k op/s 2024-09-11T23:40:31.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:31 smithi178 ceph-mon[35882]: pgmap v115: 97 pgs: 97 active+clean; 29 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 625 MiB/s wr, 3.41k op/s 2024-09-11T23:40:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.100+0000 7f0a54c31640 1 -- 172.21.15.133:0/1113427845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a50104e30 msgr2=0x7f0a501052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.100+0000 7f0a54c31640 1 --2- 172.21.15.133:0/1113427845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a50104e30 0x7f0a501052e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009ca0 tx=0x7f0a4802a8d0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.102+0000 7f0a54c31640 1 -- 172.21.15.133:0/1113427845 shutdown_connections 2024-09-11T23:40:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.102+0000 7f0a54c31640 1 --2- 172.21.15.133:0/1113427845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a50104e30 0x7f0a501052e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.102+0000 7f0a54c31640 1 --2- 172.21.15.133:0/1113427845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a50069b50 0x7f0a50100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.102+0000 7f0a54c31640 1 --2- 172.21.15.133:0/1113427845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a500696e0 0x7f0a50100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.102+0000 7f0a54c31640 1 -- 172.21.15.133:0/1113427845 >> 172.21.15.133:0/1113427845 conn(0x7f0a50077230 msgr2=0x7f0a500fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:32.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.105+0000 7f0a54c31640 1 -- 172.21.15.133:0/1113427845 shutdown_connections 2024-09-11T23:40:32.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.105+0000 7f0a54c31640 1 -- 172.21.15.133:0/1113427845 wait complete. 2024-09-11T23:40:32.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.107+0000 7f0a54c31640 1 Processor -- start 2024-09-11T23:40:32.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.107+0000 7f0a54c31640 1 -- start start 2024-09-11T23:40:32.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.108+0000 7f0a54c31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a500696e0 0x7f0a50113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.108+0000 7f0a54c31640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a50069b50 0x7f0a50110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.108+0000 7f0a54c31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a50111520 0x7f0a501119d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.108+0000 7f0a54c31640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5010dd90 con 0x7f0a500696e0 2024-09-11T23:40:32.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.108+0000 7f0a54c31640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5010dc10 con 0x7f0a50111520 2024-09-11T23:40:32.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.109+0000 7f0a54c31640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5010df10 con 0x7f0a50069b50 2024-09-11T23:40:32.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.109+0000 7f0a4dd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a50069b50 0x7f0a50110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.109+0000 7f0a4dd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a50069b50 0x7f0a50110fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:56214/0 (socket says 172.21.15.133:56214) 2024-09-11T23:40:32.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.109+0000 7f0a4dd9b640 1 -- 172.21.15.133:0/2637253134 learned_addr learned my addr 172.21.15.133:0/2637253134 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:32.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.110+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a500696e0 0x7f0a50113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.110+0000 7f0a4dd9b640 1 -- 172.21.15.133:0/2637253134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a50111520 msgr2=0x7f0a501119d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.110+0000 7f0a4dd9b640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a50111520 0x7f0a501119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.110+0000 7f0a4dd9b640 1 -- 172.21.15.133:0/2637253134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a500696e0 msgr2=0x7f0a50113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.110+0000 7f0a4dd9b640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a500696e0 0x7f0a50113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.110+0000 7f0a4dd9b640 1 -- 172.21.15.133:0/2637253134 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a501a0750 con 0x7f0a50069b50 2024-09-11T23:40:32.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.111+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a500696e0 0x7f0a50113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:32.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.111+0000 7f0a4dd9b640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a50069b50 0x7f0a50110fe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000c8c0 tx=0x7f0a4000cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:32.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.112+0000 7f0a3f7fe640 1 -- 172.21.15.133:0/2637253134 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40004220 con 0x7f0a50069b50 2024-09-11T23:40:32.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.113+0000 7f0a3f7fe640 1 -- 172.21.15.133:0/2637253134 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a400043c0 con 0x7f0a50069b50 2024-09-11T23:40:32.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.113+0000 7f0a3f7fe640 1 -- 172.21.15.133:0/2637253134 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40004a00 con 0x7f0a50069b50 2024-09-11T23:40:32.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.116+0000 7f0a54c31640 1 -- 172.21.15.133:0/2637253134 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a501a0a40 con 0x7f0a50069b50 2024-09-11T23:40:32.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.117+0000 7f0a54c31640 1 -- 172.21.15.133:0/2637253134 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a501a0f00 con 0x7f0a50069b50 2024-09-11T23:40:32.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.124+0000 7f0a5700a740 1 Processor -- start 2024-09-11T23:40:32.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.127+0000 7f0a3f7fe640 1 -- 172.21.15.133:0/2637253134 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0a40005120 con 0x7f0a50069b50 2024-09-11T23:40:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.128+0000 7f0a5700a740 1 -- start start 2024-09-11T23:40:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.129+0000 7f0a3f7fe640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0a380a2f00 0x7f0a380a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.130+0000 7f0a3f7fe640 1 -- 172.21.15.133:0/2637253134 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0a400cf400 con 0x7f0a50069b50 2024-09-11T23:40:32.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.131+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b6b60 0x55c3a45b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.131+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 0x55c3a45c1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.132+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.132+0000 7f0a5700a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c3a451c280 con 0x55c3a45b6b60 2024-09-11T23:40:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.132+0000 7f0a5700a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3a44c3b10 con 0x55c3a45c1d40 2024-09-11T23:40:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.132+0000 7f0a5700a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c3a4518240 con 0x55c3a45b7480 2024-09-11T23:40:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.132+0000 7f0a4ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.132+0000 7f0a4ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:57886/0 (socket says 172.21.15.133:57886) 2024-09-11T23:40:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.132+0000 7f0a4ed9d640 1 -- 172.21.15.133:0/3266664434 learned_addr learned my addr 172.21.15.133:0/3266664434 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.133+0000 7f0a4dd9b640 1 --2- 172.21.15.133:0/3266664434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 0x55c3a45c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.133+0000 7f0a4ed9d640 1 -- 172.21.15.133:0/3266664434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 msgr2=0x55c3a45c1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:32.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.133+0000 7f0a4ed9d640 1 --2- 172.21.15.133:0/3266664434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 0x55c3a45c1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.133+0000 7f0a4ed9d640 1 -- 172.21.15.133:0/3266664434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b6b60 msgr2=0x55c3a45b6f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:40:32.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.133+0000 7f0a4ed9d640 1 --2- 172.21.15.133:0/3266664434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b6b60 0x55c3a45b6f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.133+0000 7f0a4ed9d640 1 -- 172.21.15.133:0/3266664434 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a44d01f0 con 0x55c3a45c1d40 2024-09-11T23:40:32.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.135+0000 7f0a4ed9d640 1 --2- 172.21.15.133:0/3266664434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45c4130 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a480097b0 tx=0x7f0a480097e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ac1d1e5ea71c8c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:32.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.137+0000 7f0a4ed9d640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0a380a2f00 0x7f0a380a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.138+0000 7f0a3cff9640 1 -- 172.21.15.133:0/3266664434 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a480307d0 con 0x55c3a45c1d40 2024-09-11T23:40:32.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.138+0000 7f0a3cff9640 1 -- 172.21.15.133:0/3266664434 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a48041030 con 0x55c3a45c1d40 2024-09-11T23:40:32.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.138+0000 7f0a3cff9640 1 -- 172.21.15.133:0/3266664434 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4802fd60 con 0x55c3a45c1d40 2024-09-11T23:40:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.141+0000 7f0a5700a740 1 -- 172.21.15.133:0/3266664434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 msgr2=0x55c3a45c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.141+0000 7f0a5700a740 1 --2- 172.21.15.133:0/3266664434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45c4130 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a480097b0 tx=0x7f0a480097e0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.143+0000 7f0a4ed9d640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0a380a2f00 0x7f0a380a53c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0a48044000 tx=0x7f0a48048040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:32.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.143+0000 7f0a5700a740 1 -- 172.21.15.133:0/3266664434 shutdown_connections 2024-09-11T23:40:32.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.143+0000 7f0a5700a740 1 --2- 172.21.15.133:0/3266664434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45c4130 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.143+0000 7f0a5700a740 1 --2- 172.21.15.133:0/3266664434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 0x55c3a45c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.143+0000 7f0a5700a740 1 --2- 172.21.15.133:0/3266664434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b6b60 0x55c3a45b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.143+0000 7f0a5700a740 1 -- 172.21.15.133:0/3266664434 >> 172.21.15.133:0/3266664434 conn(0x55c3a45b30f0 msgr2=0x55c3a45b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:32.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.149+0000 7f0a5700a740 1 -- 172.21.15.133:0/3266664434 shutdown_connections 2024-09-11T23:40:32.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.153+0000 7f0a5700a740 1 -- 172.21.15.133:0/3266664434 wait complete. 2024-09-11T23:40:32.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.153+0000 7f0a5700a740 1 -- start start 2024-09-11T23:40:32.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.153+0000 7f0a5700a740 1 Processor -- start 2024-09-11T23:40:32.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.157+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b6b60 0x55c3a45bec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.157+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b7480 0x55c3a45bf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.157+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45bf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.157+0000 7f0a5700a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c3a45c4910 con 0x55c3a45b7480 2024-09-11T23:40:32.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.157+0000 7f0a5700a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3a44d6f10 con 0x55c3a45c1d40 2024-09-11T23:40:32.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.157+0000 7f0a5700a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c3a45c4a90 con 0x55c3a45b6b60 2024-09-11T23:40:32.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.157+0000 7f0a5700a740 10 client.0 _ll_register_callbacks cb 0x55c3a449fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:32.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.161+0000 7f0a4dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b7480 0x55c3a45bf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.161+0000 7f0a4dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b7480 0x55c3a45bf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55848/0 (socket says 172.21.15.133:55848) 2024-09-11T23:40:32.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.161+0000 7f0a4dd9b640 1 -- 172.21.15.133:0/1764256650 learned_addr learned my addr 172.21.15.133:0/1764256650 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:32.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.161+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b6b60 0x55c3a45bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.165+0000 7f0a4e59c640 1 -- 172.21.15.133:0/1764256650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 msgr2=0x55c3a45bf680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:40:32.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.165+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45bf680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.165+0000 7f0a4e59c640 1 -- 172.21.15.133:0/1764256650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b7480 msgr2=0x55c3a45bf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:32.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.165+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b7480 0x55c3a45bf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.166+0000 7f0a4e59c640 1 -- 172.21.15.133:0/1764256650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a428d6a0 con 0x55c3a45b6b60 2024-09-11T23:40:32.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.170+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b6b60 0x55c3a45bec00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400afb0 tx=0x7f0a4400c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e69182ec1c125adc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:32.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.170+0000 7f0a35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:32.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.171+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1764256650 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44015070 con 0x55c3a45b6b60 2024-09-11T23:40:32.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.171+0000 7f0a5700a740 1 -- 172.21.15.133:0/1764256650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a48041030 con 0x55c3a45b6b60 2024-09-11T23:40:32.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.171+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1764256650 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0a440115f0 con 0x55c3a45b6b60 2024-09-11T23:40:32.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.172+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1764256650 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a44021050 con 0x55c3a45b6b60 2024-09-11T23:40:32.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.174+0000 7f0a5700a740 10 client.25930 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:32.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.174+0000 7f0a5700a740 1 -- 172.21.15.133:0/1764256650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3a45d0550 con 0x55c3a45b6b60 2024-09-11T23:40:32.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.178+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1764256650 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a48041030 con 0x55c3a45b6b60 2024-09-11T23:40:32.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.178+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1764256650 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0a44019a30 con 0x55c3a45b6b60 2024-09-11T23:40:32.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.178+0000 7f0a5700a740 10 client.25930 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:32.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.178+0000 7f0a5700a740 10 client.25930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:32.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.178+0000 7f0a5700a740 20 client.25930 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:32.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.178+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c3a45d1d90 0x55c3a45d4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.178+0000 7f0a5700a740 4 client.25930 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:32.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.178+0000 7f0a5700a740 1 -- 172.21.15.133:0/1764256650 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3a45297f0 con 0x55c3a45d1d90 2024-09-11T23:40:32.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.182+0000 7f0a4dd9b640 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c3a45d1d90 0x55c3a45d4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.192+0000 7f0a4dd9b640 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c3a45d1d90 0x55c3a45d4250 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93574e16446ff72 server_cookie=e593cbf27e19c939 in_seq=0 out_seq=0 2024-09-11T23:40:32.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.192+0000 7f0a35ffb640 10 client.25930 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:32.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.200+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1764256650 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3a45297f0 con 0x55c3a45d1d90 2024-09-11T23:40:32.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.200+0000 7f0a35ffb640 10 client.25930 handle_command_reply: tid=0 2024-09-11T23:40:32.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.200+0000 7f0a5700a740 1 client.25930 shutdown 2024-09-11T23:40:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.202+0000 7f0a5700a740 1 -- 172.21.15.133:0/1764256650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b6b60 msgr2=0x55c3a45bec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.202+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b6b60 0x55c3a45bec00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400afb0 tx=0x7f0a4400c520 comp rx=0 tx=0).stop 2024-09-11T23:40:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.203+0000 7f0a5700a740 1 -- 172.21.15.133:0/1764256650 shutdown_connections 2024-09-11T23:40:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.203+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c3a45d1d90 0x55c3a45d4250 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.203+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45bf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.203+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b7480 0x55c3a45bf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.203+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1764256650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b6b60 0x55c3a45bec00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.203+0000 7f0a5700a740 1 -- 172.21.15.133:0/1764256650 >> 172.21.15.133:0/1764256650 conn(0x55c3a45b30f0 msgr2=0x55c3a45b8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.203+0000 7f0a35ffb640 0 client.25930 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:32.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.207+0000 7f0a5700a740 1 -- 172.21.15.133:0/1764256650 shutdown_connections 2024-09-11T23:40:32.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.212+0000 7f0a5700a740 1 -- 172.21.15.133:0/1764256650 wait complete. 2024-09-11T23:40:32.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.212+0000 7f0a5700a740 20 client.25930 trim_cache size 0 max 16384 2024-09-11T23:40:32.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.255+0000 7f0a5700a740 1 Processor -- start 2024-09-11T23:40:32.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.255+0000 7f0a5700a740 1 -- start start 2024-09-11T23:40:32.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.255+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b6b60 0x55c3a45d9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.255+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 0x55c3a45da0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.256+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.256+0000 7f0a4dd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 0x55c3a45da0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.256+0000 7f0a4dd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 0x55c3a45da0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:56248/0 (socket says 172.21.15.133:56248) 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.256+0000 7f0a4dd9b640 1 -- 172.21.15.133:0/1851068899 learned_addr learned my addr 172.21.15.133:0/1851068899 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.256+0000 7f0a5700a740 1 -- 172.21.15.133:0/1851068899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501bc7f0 con 0x55c3a45b6b60 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.256+0000 7f0a4ed9d640 1 --2- 172.21.15.133:0/1851068899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.256+0000 7f0a5700a740 1 -- 172.21.15.133:0/1851068899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501be330 con 0x55c3a45c1d40 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.256+0000 7f0a5700a740 1 -- 172.21.15.133:0/1851068899 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501bcec0 con 0x55c3a45b7480 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.256+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/1851068899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b6b60 0x55c3a45d9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.257+0000 7f0a4dd9b640 1 -- 172.21.15.133:0/1851068899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 msgr2=0x55c3a45b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.257+0000 7f0a4dd9b640 1 --2- 172.21.15.133:0/1851068899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.257+0000 7f0a4dd9b640 1 -- 172.21.15.133:0/1851068899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b6b60 msgr2=0x55c3a45d9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.257+0000 7f0a4dd9b640 1 --2- 172.21.15.133:0/1851068899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b6b60 0x55c3a45d9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.257+0000 7f0a4dd9b640 1 -- 172.21.15.133:0/1851068899 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a45b4260 con 0x55c3a45b7480 2024-09-11T23:40:32.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.257+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/1851068899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b6b60 0x55c3a45d9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:40:32.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.257+0000 7f0a4dd9b640 1 --2- 172.21.15.133:0/1851068899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 0x55c3a45da0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a4009b6b0 tx=0x7f0a4001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f4b147f9731df2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:32.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.260+0000 7f0a37fff640 1 -- 172.21.15.133:0/1851068899 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400b8070 con 0x55c3a45b7480 2024-09-11T23:40:32.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.260+0000 7f0a37fff640 1 -- 172.21.15.133:0/1851068899 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a40098020 con 0x55c3a45b7480 2024-09-11T23:40:32.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.260+0000 7f0a37fff640 1 -- 172.21.15.133:0/1851068899 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400b4bb0 con 0x55c3a45b7480 2024-09-11T23:40:32.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.261+0000 7f0a5700a740 1 -- 172.21.15.133:0/1851068899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 msgr2=0x55c3a45da0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.261+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1851068899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 0x55c3a45da0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a4009b6b0 tx=0x7f0a4001a040 comp rx=0 tx=0).stop 2024-09-11T23:40:32.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.262+0000 7f0a5700a740 1 -- 172.21.15.133:0/1851068899 shutdown_connections 2024-09-11T23:40:32.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.262+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1851068899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45c1d40 0x55c3a45b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.262+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1851068899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45b7480 0x55c3a45da0a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.262+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1851068899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b6b60 0x55c3a45d9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.262+0000 7f0a5700a740 1 -- 172.21.15.133:0/1851068899 >> 172.21.15.133:0/1851068899 conn(0x55c3a45b30f0 msgr2=0x55c3a45bdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:32.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.263+0000 7f0a5700a740 1 -- 172.21.15.133:0/1851068899 shutdown_connections 2024-09-11T23:40:32.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.265+0000 7f0a5700a740 1 -- 172.21.15.133:0/1851068899 wait complete. 2024-09-11T23:40:32.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.266+0000 7f0a5700a740 1 -- start start 2024-09-11T23:40:32.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.266+0000 7f0a5700a740 1 Processor -- start 2024-09-11T23:40:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.267+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45b6b60 0x55c3a45ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.267+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b7480 0x55c3a45bb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.267+0000 7f0a5700a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45c1d40 0x55c3a45bbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.267+0000 7f0a5700a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501bdc60 con 0x55c3a45b7480 2024-09-11T23:40:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.267+0000 7f0a5700a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501bd590 con 0x55c3a45b6b60 2024-09-11T23:40:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.267+0000 7f0a5700a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501ba7d0 con 0x55c3a45c1d40 2024-09-11T23:40:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.267+0000 7f0a5700a740 10 client.0 _ll_register_callbacks cb 0x7f0a44006f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.268+0000 7f0a4ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45c1d40 0x55c3a45bbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.268+0000 7f0a4ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45c1d40 0x55c3a45bbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:56262/0 (socket says 172.21.15.133:56262) 2024-09-11T23:40:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.268+0000 7f0a4ed9d640 1 -- 172.21.15.133:0/1218234100 learned_addr learned my addr 172.21.15.133:0/1218234100 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.268+0000 7f0a4dd9b640 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b7480 0x55c3a45bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.268+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45b6b60 0x55c3a45ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.269+0000 7f0a4ed9d640 1 -- 172.21.15.133:0/1218234100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45b6b60 msgr2=0x55c3a45ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.269+0000 7f0a4ed9d640 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45b6b60 0x55c3a45ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.269+0000 7f0a4ed9d640 1 -- 172.21.15.133:0/1218234100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b7480 msgr2=0x55c3a45bb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.269+0000 7f0a4ed9d640 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b7480 0x55c3a45bb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.269+0000 7f0a4ed9d640 1 -- 172.21.15.133:0/1218234100 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a45bc640 con 0x55c3a45c1d40 2024-09-11T23:40:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.269+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45b6b60 0x55c3a45ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.269+0000 7f0a4ed9d640 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45c1d40 0x55c3a45bbe10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55c3a45b4470 tx=0x7f0a480095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea1cac6458bf7cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.269+0000 7f0a35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.270+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1218234100 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4805a070 con 0x55c3a45c1d40 2024-09-11T23:40:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.270+0000 7f0a5700a740 1 -- 172.21.15.133:0/1218234100 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a40098020 con 0x55c3a45c1d40 2024-09-11T23:40:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.270+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1218234100 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0a4802fbf0 con 0x55c3a45c1d40 2024-09-11T23:40:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.270+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1218234100 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a48066050 con 0x55c3a45c1d40 2024-09-11T23:40:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.270+0000 7f0a5700a740 10 client.25936 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.270+0000 7f0a5700a740 1 -- 172.21.15.133:0/1218234100 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3a463cae0 con 0x55c3a45c1d40 2024-09-11T23:40:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.271+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1218234100 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a40098020 con 0x55c3a45c1d40 2024-09-11T23:40:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.271+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1218234100 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0a4803d420 con 0x55c3a45c1d40 2024-09-11T23:40:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.271+0000 7f0a5700a740 10 client.25936 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.271+0000 7f0a5700a740 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.271+0000 7f0a5700a740 20 client.25936 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.271+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c3a463ce40 0x55c3a463f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.271+0000 7f0a5700a740 4 client.25936 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:40:32.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.271+0000 7f0a5700a740 1 -- 172.21.15.133:0/1218234100 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c3a4523d00 con 0x55c3a463ce40 2024-09-11T23:40:32.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.272+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c3a463ce40 0x55c3a463f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:32.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.273+0000 7f0a4e59c640 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c3a463ce40 0x55c3a463f260 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2cbe48fe51c013 server_cookie=e0a38bfc1fec2367 in_seq=0 out_seq=0 2024-09-11T23:40:32.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.273+0000 7f0a35ffb640 10 client.25936 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:32.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.274+0000 7f0a35ffb640 1 -- 172.21.15.133:0/1218234100 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c3a4523d00 con 0x55c3a463ce40 2024-09-11T23:40:32.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.274+0000 7f0a35ffb640 10 client.25936 handle_command_reply: tid=0 2024-09-11T23:40:32.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.274+0000 7f0a5700a740 1 client.25936 shutdown 2024-09-11T23:40:32.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.275+0000 7f0a5700a740 1 -- 172.21.15.133:0/1218234100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45c1d40 msgr2=0x55c3a45bbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.275+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45c1d40 0x55c3a45bbe10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55c3a45b4470 tx=0x7f0a480095c0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.275+0000 7f0a5700a740 1 -- 172.21.15.133:0/1218234100 shutdown_connections 2024-09-11T23:40:32.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.275+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c3a463ce40 0x55c3a463f260 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.275+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3a45c1d40 0x55c3a45bbe10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.275+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3a45b7480 0x55c3a45bb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.275+0000 7f0a5700a740 1 --2- 172.21.15.133:0/1218234100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3a45b6b60 0x55c3a45ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.275+0000 7f0a35ffb640 0 client.25936 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:32.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.275+0000 7f0a5700a740 1 -- 172.21.15.133:0/1218234100 >> 172.21.15.133:0/1218234100 conn(0x55c3a45b30f0 msgr2=0x55c3a45b8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:32.276 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:40:32.276 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:40:32.276 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "382408ee-c55c-4f68-9979-67b0f434861c", 2024-09-11T23:40:32.276 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:40:32.276 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:40:32.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.276+0000 7f0a5700a740 1 -- 172.21.15.133:0/1218234100 shutdown_connections 2024-09-11T23:40:32.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.276+0000 7f0a5700a740 1 -- 172.21.15.133:0/1218234100 wait complete. 2024-09-11T23:40:32.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.276+0000 7f0a5700a740 20 client.25936 trim_cache size 0 max 16384 2024-09-11T23:40:32.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.277+0000 7f0a54c31640 1 -- 172.21.15.133:0/2637253134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0a380a2f00 msgr2=0x7f0a380a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.277+0000 7f0a54c31640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0a380a2f00 0x7f0a380a53c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0a48044000 tx=0x7f0a48048040 comp rx=0 tx=0).stop 2024-09-11T23:40:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.277+0000 7f0a54c31640 1 -- 172.21.15.133:0/2637253134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a50069b50 msgr2=0x7f0a50110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.277+0000 7f0a54c31640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a50069b50 0x7f0a50110fe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000c8c0 tx=0x7f0a4000cd80 comp rx=0 tx=0).stop 2024-09-11T23:40:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.278+0000 7f0a54c31640 1 -- 172.21.15.133:0/2637253134 shutdown_connections 2024-09-11T23:40:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.278+0000 7f0a54c31640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0a380a2f00 0x7f0a380a53c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.278+0000 7f0a54c31640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a50111520 0x7f0a501119d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.278+0000 7f0a54c31640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a50069b50 0x7f0a50110fe0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.278+0000 7f0a54c31640 1 --2- 172.21.15.133:0/2637253134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a500696e0 0x7f0a50113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.278+0000 7f0a54c31640 1 -- 172.21.15.133:0/2637253134 >> 172.21.15.133:0/2637253134 conn(0x7f0a50077230 msgr2=0x7f0a50101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.278+0000 7f0a54c31640 1 -- 172.21.15.133:0/2637253134 shutdown_connections 2024-09-11T23:40:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:32.278+0000 7f0a54c31640 1 -- 172.21.15.133:0/2637253134 wait complete. 2024-09-11T23:40:33.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:32 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:40:33.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:32 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:40:33.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:32 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:40:33.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:32 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:40:33.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:32 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:40:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:32 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:40:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:32 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:40:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:32 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:40:33.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:40:33.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:33 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:40:33.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:33 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:40:33.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:33 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:40:33.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:33 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:40:34.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:34 smithi178 ceph-mon[35882]: pgmap v116: 97 pgs: 97 active+clean; 29 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 765 MiB/s wr, 3.59k op/s 2024-09-11T23:40:34.407 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:34.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:34 smithi138 ceph-mon[35142]: pgmap v116: 97 pgs: 97 active+clean; 29 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 765 MiB/s wr, 3.59k op/s 2024-09-11T23:40:34.465 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:34 smithi133 ceph-mon[31430]: pgmap v116: 97 pgs: 97 active+clean; 29 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 765 MiB/s wr, 3.59k op/s 2024-09-11T23:40:35.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:35 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:40:35.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.223+0000 7f7931161640 1 -- 172.21.15.133:0/3961882441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7924095300 msgr2=0x7f7924004cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:35.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.223+0000 7f7931161640 1 --2- 172.21.15.133:0/3961882441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7924095300 0x7f7924004cb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f792c067960 tx=0x7f792c096dd0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.224+0000 7f7931161640 1 -- 172.21.15.133:0/3961882441 shutdown_connections 2024-09-11T23:40:35.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.224+0000 7f7931161640 1 --2- 172.21.15.133:0/3961882441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7924095300 0x7f7924004cb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.224+0000 7f7931161640 1 --2- 172.21.15.133:0/3961882441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7924094960 0x7f7924094dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.224+0000 7f7931161640 1 --2- 172.21.15.133:0/3961882441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f792409a580 0x7f792409a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.224+0000 7f7931161640 1 -- 172.21.15.133:0/3961882441 >> 172.21.15.133:0/3961882441 conn(0x7f792400a220 msgr2=0x7f792400a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:35.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.224+0000 7f7931161640 1 -- 172.21.15.133:0/3961882441 shutdown_connections 2024-09-11T23:40:35.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.224+0000 7f7931161640 1 -- 172.21.15.133:0/3961882441 wait complete. 2024-09-11T23:40:35.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.226+0000 7f7931161640 1 Processor -- start 2024-09-11T23:40:35.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.229+0000 7f7931161640 1 -- start start 2024-09-11T23:40:35.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.230+0000 7f7931161640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7924094960 0x7f792413efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.230+0000 7f7931161640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7924095300 0x7f792413f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.230+0000 7f7931161640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f792409a580 0x7f7924143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.230+0000 7f7931161640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f79240b8b50 con 0x7f7924095300 2024-09-11T23:40:35.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.230+0000 7f7931161640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79240b89d0 con 0x7f7924094960 2024-09-11T23:40:35.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.230+0000 7f7931161640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f79240b8cd0 con 0x7f792409a580 2024-09-11T23:40:35.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.231+0000 7f792ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7924094960 0x7f792413efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.231+0000 7f792a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7924095300 0x7f792413f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.232+0000 7f792b59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f792409a580 0x7f7924143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.232+0000 7f792b59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f792409a580 0x7f7924143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:56288/0 (socket says 172.21.15.133:56288) 2024-09-11T23:40:35.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.232+0000 7f792b59e640 1 -- 172.21.15.133:0/661915526 learned_addr learned my addr 172.21.15.133:0/661915526 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:35.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.235+0000 7f792b59e640 1 -- 172.21.15.133:0/661915526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7924094960 msgr2=0x7f792413efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:35.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.235+0000 7f792b59e640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7924094960 0x7f792413efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.235+0000 7f792b59e640 1 -- 172.21.15.133:0/661915526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7924095300 msgr2=0x7f792413f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:35.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.235+0000 7f792b59e640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7924095300 0x7f792413f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.235+0000 7f792b59e640 1 -- 172.21.15.133:0/661915526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7924143f70 con 0x7f792409a580 2024-09-11T23:40:35.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.237+0000 7f792b59e640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f792409a580 0x7f7924143890 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f792c065a40 tx=0x7f792c0a09c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:35.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.238+0000 7f7913fff640 1 -- 172.21.15.133:0/661915526 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c0a0c40 con 0x7f792409a580 2024-09-11T23:40:35.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.238+0000 7f7913fff640 1 -- 172.21.15.133:0/661915526 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f792c0a0de0 con 0x7f792409a580 2024-09-11T23:40:35.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.238+0000 7f7913fff640 1 -- 172.21.15.133:0/661915526 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c09e730 con 0x7f792409a580 2024-09-11T23:40:35.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.239+0000 7f7931161640 1 -- 172.21.15.133:0/661915526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7924144200 con 0x7f792409a580 2024-09-11T23:40:35.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.239+0000 7f7931161640 1 -- 172.21.15.133:0/661915526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79241446c0 con 0x7f792409a580 2024-09-11T23:40:35.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.246+0000 7f793353a740 1 Processor -- start 2024-09-11T23:40:35.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.247+0000 7f7913fff640 1 -- 172.21.15.133:0/661915526 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f792c09d030 con 0x7f792409a580 2024-09-11T23:40:35.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.248+0000 7f793353a740 1 -- start start 2024-09-11T23:40:35.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.249+0000 7f7913fff640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79140a2e80 0x7f79140a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.249+0000 7f7913fff640 1 -- 172.21.15.133:0/661915526 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f792c153970 con 0x7f792409a580 2024-09-11T23:40:35.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.250+0000 7f793353a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f7ee80 0x565313f7f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.250+0000 7f793353a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f89b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.250+0000 7f793353a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f8a060 0x565313f8c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.250+0000 7f793353a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x565313edc2e0 con 0x565313f7f7a0 2024-09-11T23:40:35.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.250+0000 7f793353a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565313e83ce0 con 0x565313f7ee80 2024-09-11T23:40:35.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.250+0000 7f793353a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x565313ed82a0 con 0x565313f8a060 2024-09-11T23:40:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.253+0000 7f792a59c640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79140a2e80 0x7f79140a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.253+0000 7f792ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f7ee80 0x565313f7f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.255+0000 7f792b59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f8a060 0x565313f8c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.255+0000 7f792b59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f8a060 0x565313f8c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:56300/0 (socket says 172.21.15.133:56300) 2024-09-11T23:40:35.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.255+0000 7f792b59e640 1 -- 172.21.15.133:0/3830728549 learned_addr learned my addr 172.21.15.133:0/3830728549 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:35.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.256+0000 7f792b59e640 1 -- 172.21.15.133:0/3830728549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f7ee80 msgr2=0x565313f7f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:35.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.256+0000 7f792b59e640 1 --2- 172.21.15.133:0/3830728549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f7ee80 0x565313f7f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.256+0000 7f792b59e640 1 -- 172.21.15.133:0/3830728549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 msgr2=0x565313f89b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:35.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.256+0000 7f792b59e640 1 --2- 172.21.15.133:0/3830728549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f89b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.256+0000 7f792b59e640 1 -- 172.21.15.133:0/3830728549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565313e68440 con 0x565313f8a060 2024-09-11T23:40:35.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.257+0000 7f792a59c640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79140a2e80 0x7f79140a5340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f79240950a0 tx=0x7f7920009510 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:35.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.259+0000 7f792b59e640 1 --2- 172.21.15.133:0/3830728549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f8a060 0x565313f8c450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f792c11fd10 tx=0x7f792c0a7dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85a0ed9c7a1e195a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:35.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.261+0000 7f79117fa640 1 -- 172.21.15.133:0/3830728549 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c120710 con 0x565313f8a060 2024-09-11T23:40:35.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.261+0000 7f79117fa640 1 -- 172.21.15.133:0/3830728549 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f792c09b020 con 0x565313f8a060 2024-09-11T23:40:35.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.261+0000 7f79117fa640 1 -- 172.21.15.133:0/3830728549 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c11c5f0 con 0x565313f8a060 2024-09-11T23:40:35.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.262+0000 7f793353a740 1 -- 172.21.15.133:0/3830728549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f8a060 msgr2=0x565313f8c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:35.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.262+0000 7f793353a740 1 --2- 172.21.15.133:0/3830728549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f8a060 0x565313f8c450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f792c11fd10 tx=0x7f792c0a7dc0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.263+0000 7f793353a740 1 -- 172.21.15.133:0/3830728549 shutdown_connections 2024-09-11T23:40:35.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.263+0000 7f793353a740 1 --2- 172.21.15.133:0/3830728549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f8a060 0x565313f8c450 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.263+0000 7f793353a740 1 --2- 172.21.15.133:0/3830728549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f89b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.263+0000 7f793353a740 1 --2- 172.21.15.133:0/3830728549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f7ee80 0x565313f7f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.263+0000 7f793353a740 1 -- 172.21.15.133:0/3830728549 >> 172.21.15.133:0/3830728549 conn(0x565313f7b410 msgr2=0x565313f7b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:35.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.264+0000 7f793353a740 1 -- 172.21.15.133:0/3830728549 shutdown_connections 2024-09-11T23:40:35.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.266+0000 7f793353a740 1 -- 172.21.15.133:0/3830728549 wait complete. 2024-09-11T23:40:35.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.267+0000 7f793353a740 1 -- start start 2024-09-11T23:40:35.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.268+0000 7f793353a740 1 Processor -- start 2024-09-11T23:40:35.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.273+0000 7f793353a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f89ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.274+0000 7f793353a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f86d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.274+0000 7f793353a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f8a060 0x565313f874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.274+0000 7f793353a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x565313f8c9f0 con 0x565313f7f7a0 2024-09-11T23:40:35.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.274+0000 7f793353a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565313e97050 con 0x565313f8a060 2024-09-11T23:40:35.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.274+0000 7f793353a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x565313f8cb70 con 0x565313f7ee80 2024-09-11T23:40:35.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.274+0000 7f793353a740 10 client.0 _ll_register_callbacks cb 0x565313ed7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:35.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.274+0000 7f792a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f86d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.274+0000 7f792a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f86d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55916/0 (socket says 172.21.15.133:55916) 2024-09-11T23:40:35.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.274+0000 7f792a59c640 1 -- 172.21.15.133:0/2142429522 learned_addr learned my addr 172.21.15.133:0/2142429522 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:35.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.275+0000 7f792b59e640 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f8a060 0x565313f874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.275+0000 7f792ad9d640 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f89ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.276+0000 7f792a59c640 1 -- 172.21.15.133:0/2142429522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 msgr2=0x565313f89ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:35.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.276+0000 7f792a59c640 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f89ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.276+0000 7f792a59c640 1 -- 172.21.15.133:0/2142429522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f8a060 msgr2=0x565313f874d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:35.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.276+0000 7f792a59c640 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f8a060 0x565313f874d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.276+0000 7f792a59c640 1 -- 172.21.15.133:0/2142429522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565313c4d3c0 con 0x565313f7f7a0 2024-09-11T23:40:35.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.277+0000 7f792a59c640 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f86d90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7920009150 tx=0x7f79200047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8b3bf18beea46f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:35.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.277+0000 7f790eff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:40:35.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.281+0000 7f790eff5640 1 -- 172.21.15.133:0/2142429522 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920024020 con 0x565313f7f7a0 2024-09-11T23:40:35.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.282+0000 7f790eff5640 1 -- 172.21.15.133:0/2142429522 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f7920016880 con 0x565313f7f7a0 2024-09-11T23:40:35.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.282+0000 7f793353a740 1 -- 172.21.15.133:0/2142429522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f792c09b020 con 0x565313f7f7a0 2024-09-11T23:40:35.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.282+0000 7f790eff5640 1 -- 172.21.15.133:0/2142429522 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f792001c040 con 0x565313f7f7a0 2024-09-11T23:40:35.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.282+0000 7f793353a740 10 client.16515 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:35.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.282+0000 7f793353a740 1 -- 172.21.15.133:0/2142429522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565313f984b0 con 0x565313f7f7a0 2024-09-11T23:40:35.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.283+0000 7f790eff5640 1 -- 172.21.15.133:0/2142429522 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79200293f0 con 0x565313f7f7a0 2024-09-11T23:40:35.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.283+0000 7f790eff5640 1 -- 172.21.15.133:0/2142429522 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7920031020 con 0x565313f7f7a0 2024-09-11T23:40:35.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.283+0000 7f793353a740 10 client.16515 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:35.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.283+0000 7f793353a740 10 client.16515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:35.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.284+0000 7f793353a740 20 client.16515 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:35.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.284+0000 7f793353a740 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565313f99970 0x565313f9bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.284+0000 7f793353a740 4 client.16515 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:35.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.284+0000 7f793353a740 1 -- 172.21.15.133:0/2142429522 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f792c09b020 con 0x565313f99970 2024-09-11T23:40:35.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.284+0000 7f792ad9d640 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565313f99970 0x565313f9bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.286+0000 7f792ad9d640 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565313f99970 0x565313f9bda0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf2c88538ca53592 server_cookie=1b82317192effed3 in_seq=0 out_seq=0 2024-09-11T23:40:35.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.286+0000 7f790eff5640 10 client.16515 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.289+0000 7f790eff5640 1 -- 172.21.15.133:0/2142429522 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f792c09b020 con 0x565313f99970 2024-09-11T23:40:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.289+0000 7f790eff5640 10 client.16515 handle_command_reply: tid=0 2024-09-11T23:40:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.289+0000 7f793353a740 1 client.16515 shutdown 2024-09-11T23:40:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.290+0000 7f793353a740 1 -- 172.21.15.133:0/2142429522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 msgr2=0x565313f86d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.290+0000 7f793353a740 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f86d90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7920009150 tx=0x7f79200047d0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.291+0000 7f793353a740 1 -- 172.21.15.133:0/2142429522 shutdown_connections 2024-09-11T23:40:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.291+0000 7f793353a740 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565313f99970 0x565313f9bda0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.291+0000 7f793353a740 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f8a060 0x565313f874d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.291+0000 7f793353a740 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f86d90 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.291+0000 7f790eff5640 0 client.16515 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.291+0000 7f793353a740 1 --2- 172.21.15.133:0/2142429522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f89ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.291+0000 7f793353a740 1 -- 172.21.15.133:0/2142429522 >> 172.21.15.133:0/2142429522 conn(0x565313f7b410 msgr2=0x565313f71d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.291+0000 7f793353a740 1 -- 172.21.15.133:0/2142429522 shutdown_connections 2024-09-11T23:40:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.291+0000 7f793353a740 1 -- 172.21.15.133:0/2142429522 wait complete. 2024-09-11T23:40:35.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.292+0000 7f793353a740 20 client.16515 trim_cache size 0 max 16384 2024-09-11T23:40:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.323+0000 7f793353a740 1 Processor -- start 2024-09-11T23:40:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.323+0000 7f793353a740 1 -- start start 2024-09-11T23:40:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.324+0000 7f793353a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f95ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.324+0000 7f793353a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f7f7a0 0x565313fa16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.324+0000 7f792ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f95ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.324+0000 7f792ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f95ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:56326/0 (socket says 172.21.15.133:56326) 2024-09-11T23:40:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.324+0000 7f792ad9d640 1 -- 172.21.15.133:0/1492303156 learned_addr learned my addr 172.21.15.133:0/1492303156 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.324+0000 7f793353a740 1 --2- 172.21.15.133:0/1492303156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f8a060 0x565313fa1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.324+0000 7f792a59c640 1 --2- 172.21.15.133:0/1492303156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f7f7a0 0x565313fa16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.324+0000 7f793353a740 1 -- 172.21.15.133:0/1492303156 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f792414fec0 con 0x565313f8a060 2024-09-11T23:40:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.324+0000 7f793353a740 1 -- 172.21.15.133:0/1492303156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7924151a00 con 0x565313f7f7a0 2024-09-11T23:40:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.324+0000 7f793353a740 1 -- 172.21.15.133:0/1492303156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7924150590 con 0x565313f7ee80 2024-09-11T23:40:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.325+0000 7f792b59e640 1 --2- 172.21.15.133:0/1492303156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f8a060 0x565313fa1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.325+0000 7f792ad9d640 1 -- 172.21.15.133:0/1492303156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f7f7a0 msgr2=0x565313fa16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.325+0000 7f792ad9d640 1 --2- 172.21.15.133:0/1492303156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f7f7a0 0x565313fa16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.325+0000 7f792ad9d640 1 -- 172.21.15.133:0/1492303156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f8a060 msgr2=0x565313fa1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.325+0000 7f792ad9d640 1 --2- 172.21.15.133:0/1492303156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f8a060 0x565313fa1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.325+0000 7f792ad9d640 1 -- 172.21.15.133:0/1492303156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565313f852e0 con 0x565313f7ee80 2024-09-11T23:40:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.325+0000 7f792ad9d640 1 --2- 172.21.15.133:0/1492303156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f95ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f791c00e480 tx=0x7f791c005d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d16f6bfc8ebdc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.325+0000 7f7910ff9640 1 -- 172.21.15.133:0/1492303156 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c01d070 con 0x565313f7ee80 2024-09-11T23:40:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.325+0000 7f7910ff9640 1 -- 172.21.15.133:0/1492303156 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f791c00fd60 con 0x565313f7ee80 2024-09-11T23:40:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.325+0000 7f7910ff9640 1 -- 172.21.15.133:0/1492303156 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c00c7f0 con 0x565313f7ee80 2024-09-11T23:40:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.328+0000 7f793353a740 1 -- 172.21.15.133:0/1492303156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 msgr2=0x565313f95ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.328+0000 7f793353a740 1 --2- 172.21.15.133:0/1492303156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f95ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f791c00e480 tx=0x7f791c005d40 comp rx=0 tx=0).stop 2024-09-11T23:40:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.328+0000 7f793353a740 1 -- 172.21.15.133:0/1492303156 shutdown_connections 2024-09-11T23:40:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.328+0000 7f793353a740 1 --2- 172.21.15.133:0/1492303156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f8a060 0x565313fa1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.329+0000 7f793353a740 1 --2- 172.21.15.133:0/1492303156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f7f7a0 0x565313fa16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.329+0000 7f793353a740 1 --2- 172.21.15.133:0/1492303156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f95ba0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.329+0000 7f793353a740 1 -- 172.21.15.133:0/1492303156 >> 172.21.15.133:0/1492303156 conn(0x565313f7b410 msgr2=0x565313f88e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.329+0000 7f793353a740 1 -- 172.21.15.133:0/1492303156 shutdown_connections 2024-09-11T23:40:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.329+0000 7f793353a740 1 -- 172.21.15.133:0/1492303156 wait complete. 2024-09-11T23:40:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.329+0000 7f793353a740 1 -- start start 2024-09-11T23:40:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f793353a740 1 Processor -- start 2024-09-11T23:40:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f793353a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f793353a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f8d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f793353a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f8a060 0x565313f8dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f793353a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7924151330 con 0x565313f7f7a0 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f793353a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7924150c60 con 0x565313f8a060 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f793353a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f792414df00 con 0x565313f7ee80 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f793353a740 10 client.0 _ll_register_callbacks cb 0x7f7920009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f792a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f8d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f792a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f8d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55944/0 (socket says 172.21.15.133:55944) 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f792a59c640 1 -- 172.21.15.133:0/1530742835 learned_addr learned my addr 172.21.15.133:0/1530742835 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.330+0000 7f792ad9d640 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.331+0000 7f792a59c640 1 -- 172.21.15.133:0/1530742835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 msgr2=0x565313f924e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.331+0000 7f792a59c640 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f924e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.331+0000 7f792a59c640 1 -- 172.21.15.133:0/1530742835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f8a060 msgr2=0x565313f8dd70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.331+0000 7f792a59c640 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f8a060 0x565313f8dd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.331+0000 7f792a59c640 1 -- 172.21.15.133:0/1530742835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565313f8e5a0 con 0x565313f7f7a0 2024-09-11T23:40:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.331+0000 7f792a59c640 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f8d810 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f79200205e0 tx=0x7f79200168b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c954d3501aaa6421 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:35.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.331+0000 7f790eff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:40:35.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.331+0000 7f790eff5640 1 -- 172.21.15.133:0/1530742835 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920039020 con 0x565313f7f7a0 2024-09-11T23:40:35.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.331+0000 7f790eff5640 1 -- 172.21.15.133:0/1530742835 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f79200323f0 con 0x565313f7f7a0 2024-09-11T23:40:35.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.331+0000 7f793353a740 1 -- 172.21.15.133:0/1530742835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f791c00fd60 con 0x565313f7f7a0 2024-09-11T23:40:35.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.332+0000 7f790eff5640 1 -- 172.21.15.133:0/1530742835 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7920044050 con 0x565313f7f7a0 2024-09-11T23:40:35.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.332+0000 7f793353a740 10 client.16521 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:35.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.332+0000 7f793353a740 1 -- 172.21.15.133:0/1530742835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565314004720 con 0x565313f7f7a0 2024-09-11T23:40:35.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.332+0000 7f790eff5640 1 -- 172.21.15.133:0/1530742835 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7920016e30 con 0x565313f7f7a0 2024-09-11T23:40:35.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.333+0000 7f790eff5640 1 -- 172.21.15.133:0/1530742835 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7920031040 con 0x565313f7f7a0 2024-09-11T23:40:35.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.333+0000 7f793353a740 10 client.16521 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:35.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.333+0000 7f793353a740 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:35.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.333+0000 7f793353a740 20 client.16521 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:35.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.333+0000 7f793353a740 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565313f99970 0x565314006f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:35.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.333+0000 7f793353a740 4 client.16521 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:40:35.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.333+0000 7f793353a740 1 -- 172.21.15.133:0/1530742835 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f791c00fd60 con 0x565313f99970 2024-09-11T23:40:35.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.334+0000 7f792ad9d640 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565313f99970 0x565314006f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:35.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.336+0000 7f790eff5640 10 client.16521 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:35.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.336+0000 7f792ad9d640 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565313f99970 0x565314006f30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3314f7c3fed9a25a server_cookie=a837766781a1ea28 in_seq=0 out_seq=0 2024-09-11T23:40:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.338+0000 7f790eff5640 1 -- 172.21.15.133:0/1530742835 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f791c00fd60 con 0x565313f99970 2024-09-11T23:40:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.338+0000 7f790eff5640 10 client.16521 handle_command_reply: tid=0 2024-09-11T23:40:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.338+0000 7f793353a740 1 client.16521 shutdown 2024-09-11T23:40:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.340+0000 7f793353a740 1 -- 172.21.15.133:0/1530742835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 msgr2=0x565313f8d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:35.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.340+0000 7f793353a740 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f8d810 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f79200205e0 tx=0x7f79200168b0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.341+0000 7f793353a740 1 -- 172.21.15.133:0/1530742835 shutdown_connections 2024-09-11T23:40:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.341+0000 7f793353a740 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x565313f99970 0x565314006f30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.341+0000 7f790eff5640 0 client.16521 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.341+0000 7f793353a740 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565313f8a060 0x565313f8dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.341+0000 7f793353a740 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565313f7f7a0 0x565313f8d810 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.341+0000 7f793353a740 1 --2- 172.21.15.133:0/1530742835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565313f7ee80 0x565313f924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.342+0000 7f793353a740 1 -- 172.21.15.133:0/1530742835 >> 172.21.15.133:0/1530742835 conn(0x565313f7b410 msgr2=0x565313f71520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:35.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.343+0000 7f793353a740 1 -- 172.21.15.133:0/1530742835 shutdown_connections 2024-09-11T23:40:35.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.343+0000 7f793353a740 1 -- 172.21.15.133:0/1530742835 wait complete. 2024-09-11T23:40:35.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.343+0000 7f793353a740 20 client.16521 trim_cache size 0 max 16384 2024-09-11T23:40:35.343 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:40:35.343 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:40:35.343 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:40:35.343 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:40:35.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 -- 172.21.15.133:0/661915526 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79140a2e80 msgr2=0x7f79140a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:35.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79140a2e80 0x7f79140a5340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f79240950a0 tx=0x7f7920009510 comp rx=0 tx=0).stop 2024-09-11T23:40:35.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 -- 172.21.15.133:0/661915526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f792409a580 msgr2=0x7f7924143890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:35.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f792409a580 0x7f7924143890 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f792c065a40 tx=0x7f792c0a09c0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 -- 172.21.15.133:0/661915526 shutdown_connections 2024-09-11T23:40:35.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79140a2e80 0x7f79140a5340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f792409a580 0x7f7924143890 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7924095300 0x7f792413f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 --2- 172.21.15.133:0/661915526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7924094960 0x7f792413efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:35.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 -- 172.21.15.133:0/661915526 >> 172.21.15.133:0/661915526 conn(0x7f792400a220 msgr2=0x7f7924096070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:35.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 -- 172.21.15.133:0/661915526 shutdown_connections 2024-09-11T23:40:35.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:35.344+0000 7f7931161640 1 -- 172.21.15.133:0/661915526 wait complete. 2024-09-11T23:40:35.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:35 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:40:35.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:35 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:40:36.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:36 smithi138 ceph-mon[35142]: pgmap v117: 97 pgs: 97 active+clean; 29 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 675 MiB/s wr, 2.82k op/s 2024-09-11T23:40:36.465 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:36 smithi133 ceph-mon[31430]: pgmap v117: 97 pgs: 97 active+clean; 29 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 675 MiB/s wr, 2.82k op/s 2024-09-11T23:40:36.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:36 smithi178 ceph-mon[35882]: pgmap v117: 97 pgs: 97 active+clean; 29 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 675 MiB/s wr, 2.82k op/s 2024-09-11T23:40:36.823 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:40:36.823 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:40:37.575 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:38.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:38 smithi133 ceph-mon[31430]: pgmap v118: 97 pgs: 97 active+clean; 29 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 617 MiB/s wr, 2.16k op/s 2024-09-11T23:40:38.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:38 smithi138 ceph-mon[35142]: pgmap v118: 97 pgs: 97 active+clean; 29 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 617 MiB/s wr, 2.16k op/s 2024-09-11T23:40:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.764+0000 7f10363aa640 1 -- 172.21.15.133:0/3580741576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1030110190 msgr2=0x7f103006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.764+0000 7f10363aa640 1 --2- 172.21.15.133:0/3580741576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1030110190 0x7f103006e4f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1028009ca0 tx=0x7f102802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.765+0000 7f10363aa640 1 -- 172.21.15.133:0/3580741576 shutdown_connections 2024-09-11T23:40:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.765+0000 7f10363aa640 1 --2- 172.21.15.133:0/3580741576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f103011b140 0x7f103011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.765+0000 7f10363aa640 1 --2- 172.21.15.133:0/3580741576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f103006ea30 0x7f103006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.765+0000 7f10363aa640 1 --2- 172.21.15.133:0/3580741576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1030110190 0x7f103006e4f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.765+0000 7f10363aa640 1 -- 172.21.15.133:0/3580741576 >> 172.21.15.133:0/3580741576 conn(0x7f1030073180 msgr2=0x7f1030073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.766+0000 7f10363aa640 1 -- 172.21.15.133:0/3580741576 shutdown_connections 2024-09-11T23:40:38.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.766+0000 7f10363aa640 1 -- 172.21.15.133:0/3580741576 wait complete. 2024-09-11T23:40:38.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.768+0000 7f10363aa640 1 Processor -- start 2024-09-11T23:40:38.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.768+0000 7f10363aa640 1 -- start start 2024-09-11T23:40:38.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.768+0000 7f10363aa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f103006ea30 0x7f10301193d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:38.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.768+0000 7f10363aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f103011b140 0x7f1030119930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.768+0000 7f10363aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10301124d0 0x7f1030112980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.768+0000 7f10363aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1030120c60 con 0x7f10301124d0 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.768+0000 7f10363aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1030120ae0 con 0x7f103011b140 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.768+0000 7f10363aa640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1030120de0 con 0x7f103006ea30 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.769+0000 7f1035ba9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10301124d0 0x7f1030112980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.769+0000 7f1035ba9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10301124d0 0x7f1030112980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55952/0 (socket says 172.21.15.133:55952) 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.769+0000 7f1035ba9640 1 -- 172.21.15.133:0/2069777156 learned_addr learned my addr 172.21.15.133:0/2069777156 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.770+0000 7f10353a8640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f103006ea30 0x7f10301193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.770+0000 7f1034ba7640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f103011b140 0x7f1030119930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.770+0000 7f1035ba9640 1 -- 172.21.15.133:0/2069777156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f103006ea30 msgr2=0x7f10301193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.770+0000 7f1035ba9640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f103006ea30 0x7f10301193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.770+0000 7f1035ba9640 1 -- 172.21.15.133:0/2069777156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f103011b140 msgr2=0x7f1030119930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.770+0000 7f1035ba9640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f103011b140 0x7f1030119930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:38.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.770+0000 7f1035ba9640 1 -- 172.21.15.133:0/2069777156 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1030113180 con 0x7f10301124d0 2024-09-11T23:40:38.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.778+0000 7f1035ba9640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10301124d0 0x7f1030112980 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e970 tx=0x7f102c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:38.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.779+0000 7f10267fc640 1 -- 172.21.15.133:0/2069777156 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c00cab0 con 0x7f10301124d0 2024-09-11T23:40:38.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.779+0000 7f10363aa640 1 -- 172.21.15.133:0/2069777156 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1030116f70 con 0x7f10301124d0 2024-09-11T23:40:38.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.779+0000 7f10363aa640 1 -- 172.21.15.133:0/2069777156 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10301174b0 con 0x7f10301124d0 2024-09-11T23:40:38.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.780+0000 7f10267fc640 1 -- 172.21.15.133:0/2069777156 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102c005500 con 0x7f10301124d0 2024-09-11T23:40:38.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.780+0000 7f10267fc640 1 -- 172.21.15.133:0/2069777156 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c010430 con 0x7f10301124d0 2024-09-11T23:40:38.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:38 smithi178 ceph-mon[35882]: pgmap v118: 97 pgs: 97 active+clean; 29 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 617 MiB/s wr, 2.16k op/s 2024-09-11T23:40:38.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.791+0000 7f10363aa640 1 -- 172.21.15.133:0/2069777156 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10140052c0 con 0x7f10301124d0 2024-09-11T23:40:38.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.797+0000 7f10267fc640 1 -- 172.21.15.133:0/2069777156 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f102c00cc50 con 0x7f10301124d0 2024-09-11T23:40:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.809+0000 7f10267fc640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f101c0a2f00 0x7f101c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.809+0000 7f10267fc640 1 -- 172.21.15.133:0/2069777156 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f102c0cf9f0 con 0x7f10301124d0 2024-09-11T23:40:38.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.812+0000 7f10353a8640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f101c0a2f00 0x7f101c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:38.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.817+0000 7f10353a8640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f101c0a2f00 0x7f101c0a53c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1028000f80 tx=0x7f1028005eb0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:38.819+0000 7f10267fc640 1 -- 172.21.15.133:0/2069777156 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f102c0541b0 con 0x7f10301124d0 2024-09-11T23:40:39.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.389+0000 7f10363aa640 1 -- 172.21.15.133:0/2069777156 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10140055b0 con 0x7f10301124d0 2024-09-11T23:40:39.407 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:40:39.407 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:40:39.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.404+0000 7f10267fc640 1 -- 172.21.15.133:0/2069777156 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f102c09c6a0 con 0x7f10301124d0 2024-09-11T23:40:39.407 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:40:39.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 -- 172.21.15.133:0/2069777156 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f101c0a2f00 msgr2=0x7f101c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:39.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f101c0a2f00 0x7f101c0a53c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1028000f80 tx=0x7f1028005eb0 comp rx=0 tx=0).stop 2024-09-11T23:40:39.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 -- 172.21.15.133:0/2069777156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10301124d0 msgr2=0x7f1030112980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:39.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10301124d0 0x7f1030112980 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e970 tx=0x7f102c00ee30 comp rx=0 tx=0).stop 2024-09-11T23:40:39.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 -- 172.21.15.133:0/2069777156 shutdown_connections 2024-09-11T23:40:39.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f101c0a2f00 0x7f101c0a53c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:39.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10301124d0 0x7f1030112980 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:39.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f103011b140 0x7f1030119930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:39.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 --2- 172.21.15.133:0/2069777156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f103006ea30 0x7f10301193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:39.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 -- 172.21.15.133:0/2069777156 >> 172.21.15.133:0/2069777156 conn(0x7f1030073180 msgr2=0x7f103006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:39.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 -- 172.21.15.133:0/2069777156 shutdown_connections 2024-09-11T23:40:39.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:39.413+0000 7f10363aa640 1 -- 172.21.15.133:0/2069777156 wait complete. 2024-09-11T23:40:39.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:39 smithi138 ceph-mon[35142]: pgmap v119: 97 pgs: 97 active+clean; 30 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 856 MiB/s wr, 2.71k op/s 2024-09-11T23:40:39.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:39 smithi133 ceph-mon[31430]: pgmap v119: 97 pgs: 97 active+clean; 30 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 856 MiB/s wr, 2.71k op/s 2024-09-11T23:40:39.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:39 smithi178 ceph-mon[35882]: pgmap v119: 97 pgs: 97 active+clean; 30 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 856 MiB/s wr, 2.71k op/s 2024-09-11T23:40:40.825 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:40 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2069777156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:40.826 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:40:40.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:40 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2069777156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:41.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:40 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2069777156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:41.639 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:41.888 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:41 smithi133 ceph-mon[31430]: pgmap v120: 97 pgs: 97 active+clean; 30 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 709 MiB/s wr, 1.58k op/s 2024-09-11T23:40:42.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:41 smithi178 ceph-mon[35882]: pgmap v120: 97 pgs: 97 active+clean; 30 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 709 MiB/s wr, 1.58k op/s 2024-09-11T23:40:42.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:41 smithi138 ceph-mon[35142]: pgmap v120: 97 pgs: 97 active+clean; 30 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 709 MiB/s wr, 1.58k op/s 2024-09-11T23:40:42.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.761+0000 7ff2b961d640 1 -- 172.21.15.133:0/947863337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b41068d0 msgr2=0x7ff2b4106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:42.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.761+0000 7ff2b961d640 1 --2- 172.21.15.133:0/947863337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b41068d0 0x7ff2b4106cb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009930 tx=0x7ff2a402a480 comp rx=0 tx=0).stop 2024-09-11T23:40:42.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.765+0000 7ff2b961d640 1 -- 172.21.15.133:0/947863337 shutdown_connections 2024-09-11T23:40:42.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.765+0000 7ff2b961d640 1 --2- 172.21.15.133:0/947863337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b4101270 0x7ff2b410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.765+0000 7ff2b961d640 1 --2- 172.21.15.133:0/947863337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2b41008d0 0x7ff2b4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.765+0000 7ff2b961d640 1 --2- 172.21.15.133:0/947863337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b41068d0 0x7ff2b4106cb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.765+0000 7ff2b961d640 1 -- 172.21.15.133:0/947863337 >> 172.21.15.133:0/947863337 conn(0x7ff2b40fc610 msgr2=0x7ff2b40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:42.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.769+0000 7ff2b961d640 1 -- 172.21.15.133:0/947863337 shutdown_connections 2024-09-11T23:40:42.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.769+0000 7ff2b961d640 1 -- 172.21.15.133:0/947863337 wait complete. 2024-09-11T23:40:42.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.770+0000 7ff2b961d640 1 Processor -- start 2024-09-11T23:40:42.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.770+0000 7ff2b961d640 1 -- start start 2024-09-11T23:40:42.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.771+0000 7ff2b961d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b41008d0 0x7ff2b4073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.771+0000 7ff2b961d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2b4101270 0x7ff2b4073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.771+0000 7ff2b961d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b41068d0 0x7ff2b406cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.771+0000 7ff2b961d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410fcd0 con 0x7ff2b41008d0 2024-09-11T23:40:42.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.771+0000 7ff2b961d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410fb50 con 0x7ff2b4101270 2024-09-11T23:40:42.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.771+0000 7ff2b961d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410fe50 con 0x7ff2b41068d0 2024-09-11T23:40:42.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.772+0000 7ff2b8e1c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b41068d0 0x7ff2b406cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.772+0000 7ff2b8e1c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b41068d0 0x7ff2b406cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57838/0 (socket says 172.21.15.133:57838) 2024-09-11T23:40:42.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.772+0000 7ff2b8e1c640 1 -- 172.21.15.133:0/3472722494 learned_addr learned my addr 172.21.15.133:0/3472722494 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:42.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.773+0000 7ff2b8e1c640 1 -- 172.21.15.133:0/3472722494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2b4101270 msgr2=0x7ff2b4073ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:40:42.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.773+0000 7ff2b8e1c640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2b4101270 0x7ff2b4073ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.773+0000 7ff2b8e1c640 1 -- 172.21.15.133:0/3472722494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b41008d0 msgr2=0x7ff2b4073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:42.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.773+0000 7ff2b8e1c640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b41008d0 0x7ff2b4073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.773+0000 7ff2b8e1c640 1 -- 172.21.15.133:0/3472722494 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b406d4e0 con 0x7ff2b41068d0 2024-09-11T23:40:42.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.774+0000 7ff2b8e1c640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b41068d0 0x7ff2b406cc20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b970 tx=0x7ff2ac00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:42.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.775+0000 7ff2b1ffb640 1 -- 172.21.15.133:0/3472722494 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac019070 con 0x7ff2b41068d0 2024-09-11T23:40:42.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.775+0000 7ff2b1ffb640 1 -- 172.21.15.133:0/3472722494 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ac00cb40 con 0x7ff2b41068d0 2024-09-11T23:40:42.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.775+0000 7ff2b961d640 1 -- 172.21.15.133:0/3472722494 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b406d7d0 con 0x7ff2b41068d0 2024-09-11T23:40:42.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.775+0000 7ff2b961d640 1 -- 172.21.15.133:0/3472722494 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b4071790 con 0x7ff2b41068d0 2024-09-11T23:40:42.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.781+0000 7ff2bcc58740 1 Processor -- start 2024-09-11T23:40:42.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.781+0000 7ff2bcc58740 1 -- start start 2024-09-11T23:40:42.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.782+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65eccd20 0x563f65ecb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.782+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ecbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.782+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65ed6550 0x563f65ed8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.782+0000 7ff2bcc58740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563f65e312f0 con 0x563f65ecb7b0 2024-09-11T23:40:42.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.782+0000 7ff2bcc58740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563f65dd8cf0 con 0x563f65eccd20 2024-09-11T23:40:42.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.782+0000 7ff2bcc58740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563f65e2d2b0 con 0x563f65ed6550 2024-09-11T23:40:42.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.783+0000 7ff2b1ffb640 1 -- 172.21.15.133:0/3472722494 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac0076e0 con 0x7ff2b41068d0 2024-09-11T23:40:42.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.784+0000 7ff2b1ffb640 1 -- 172.21.15.133:0/3472722494 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff2ac01c070 con 0x7ff2b41068d0 2024-09-11T23:40:42.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.786+0000 7ff2b1ffb640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2a00a2dc0 0x7ff2a00a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.787+0000 7ff2b1ffb640 1 -- 172.21.15.133:0/3472722494 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff2ac0cf940 con 0x7ff2b41068d0 2024-09-11T23:40:42.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.793+0000 7ff2b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ecbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.793+0000 7ff2b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ecbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60984/0 (socket says 172.21.15.133:60984) 2024-09-11T23:40:42.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.793+0000 7ff2b3fff640 1 -- 172.21.15.133:0/3954917549 learned_addr learned my addr 172.21.15.133:0/3954917549 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:42.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.793+0000 7ff2abfff640 1 --2- 172.21.15.133:0/3954917549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65eccd20 0x563f65ecb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.794+0000 7ff2abfff640 1 --2- 172.21.15.133:0/3954917549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65ed6550 0x563f65ed8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.794+0000 7ff2b3fff640 1 -- 172.21.15.133:0/3954917549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65ed6550 msgr2=0x563f65ed8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:42.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.794+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/3954917549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65ed6550 0x563f65ed8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.794+0000 7ff2b3fff640 1 -- 172.21.15.133:0/3954917549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65eccd20 msgr2=0x563f65ecb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:42.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.794+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/3954917549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65eccd20 0x563f65ecb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.794+0000 7ff2b3fff640 1 -- 172.21.15.133:0/3954917549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f65dbd3a0 con 0x563f65ecb7b0 2024-09-11T23:40:42.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.794+0000 7ff2b8e1c640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2a00a2dc0 0x7ff2a00a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.795+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/3954917549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ecbbf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009a50 tx=0x7ff2a4004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ab5d886c674dbd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:42.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.796+0000 7ff2aa7fc640 1 -- 172.21.15.133:0/3954917549 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a403f070 con 0x563f65ecb7b0 2024-09-11T23:40:42.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.796+0000 7ff2aa7fc640 1 -- 172.21.15.133:0/3954917549 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a40044b0 con 0x563f65ecb7b0 2024-09-11T23:40:42.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.796+0000 7ff2aa7fc640 1 -- 172.21.15.133:0/3954917549 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4035030 con 0x563f65ecb7b0 2024-09-11T23:40:42.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.796+0000 7ff2b8e1c640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2a00a2dc0 0x7ff2a00a5280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0cd2c0 tx=0x7ff2ac009930 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:42.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.798+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3954917549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 msgr2=0x563f65ecbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:42.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.798+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/3954917549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ecbbf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009a50 tx=0x7ff2a4004310 comp rx=0 tx=0).stop 2024-09-11T23:40:42.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.799+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3954917549 shutdown_connections 2024-09-11T23:40:42.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.799+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/3954917549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65ed6550 0x563f65ed8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.799+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/3954917549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ecbbf0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.799+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/3954917549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65eccd20 0x563f65ecb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.799+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3954917549 >> 172.21.15.133:0/3954917549 conn(0x563f65e94310 msgr2=0x563f65ec7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:42.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.799+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3954917549 shutdown_connections 2024-09-11T23:40:42.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.799+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3954917549 wait complete. 2024-09-11T23:40:42.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.800+0000 7ff2bcc58740 1 -- start start 2024-09-11T23:40:42.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.801+0000 7ff2bcc58740 1 Processor -- start 2024-09-11T23:40:42.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.801+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ed5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.801+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65eccd20 0x563f65ed3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.801+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65ed6550 0x563f65ed3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.801+0000 7ff2bcc58740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563f65ed91f0 con 0x563f65ecb7b0 2024-09-11T23:40:42.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.801+0000 7ff2bcc58740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563f65dec060 con 0x563f65eccd20 2024-09-11T23:40:42.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.801+0000 7ff2bcc58740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563f65ed9370 con 0x563f65ed6550 2024-09-11T23:40:42.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.801+0000 7ff2bcc58740 10 client.0 _ll_register_callbacks cb 0x563f65b3f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:42.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.802+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ed5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.802+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ed5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60996/0 (socket says 172.21.15.133:60996) 2024-09-11T23:40:42.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.802+0000 7ff2abfff640 1 -- 172.21.15.133:0/1240598858 learned_addr learned my addr 172.21.15.133:0/1240598858 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:42.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.802+0000 7ff2abfff640 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65ed6550 0x563f65ed3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.803+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65eccd20 0x563f65ed3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.803+0000 7ff2abfff640 1 -- 172.21.15.133:0/1240598858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65ed6550 msgr2=0x563f65ed3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:42.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.803+0000 7ff2abfff640 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65ed6550 0x563f65ed3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.803+0000 7ff2abfff640 1 -- 172.21.15.133:0/1240598858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65eccd20 msgr2=0x563f65ed3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:42.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.803+0000 7ff2abfff640 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65eccd20 0x563f65ed3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.803+0000 7ff2abfff640 1 -- 172.21.15.133:0/1240598858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f65ba23c0 con 0x563f65ecb7b0 2024-09-11T23:40:42.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.803+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65eccd20 0x563f65ed3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:42.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.803+0000 7ff2abfff640 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ed5ee0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0114b0 tx=0x7ff29c006be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0130cba050893bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:42.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.803+0000 7ff2937fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:40:42.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.803+0000 7ff2937fe640 1 -- 172.21.15.133:0/1240598858 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c01a060 con 0x563f65ecb7b0 2024-09-11T23:40:42.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.803+0000 7ff2937fe640 1 -- 172.21.15.133:0/1240598858 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff29c00f6f0 con 0x563f65ecb7b0 2024-09-11T23:40:42.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.804+0000 7ff2bcc58740 1 -- 172.21.15.133:0/1240598858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a40044b0 con 0x563f65ecb7b0 2024-09-11T23:40:42.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.804+0000 7ff2937fe640 1 -- 172.21.15.133:0/1240598858 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff29c00f8f0 con 0x563f65ecb7b0 2024-09-11T23:40:42.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.804+0000 7ff2bcc58740 10 client.16533 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:42.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.804+0000 7ff2bcc58740 1 -- 172.21.15.133:0/1240598858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f65ee4d10 con 0x563f65ecb7b0 2024-09-11T23:40:42.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.804+0000 7ff2937fe640 1 -- 172.21.15.133:0/1240598858 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff29c010040 con 0x563f65ecb7b0 2024-09-11T23:40:42.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.805+0000 7ff2937fe640 1 -- 172.21.15.133:0/1240598858 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff29c01d020 con 0x563f65ecb7b0 2024-09-11T23:40:42.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.805+0000 7ff2bcc58740 10 client.16533 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:42.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.805+0000 7ff2bcc58740 10 client.16533 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:40:42.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.805+0000 7ff2bcc58740 20 client.16533 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:42.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.805+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563f65ee6220 0x563f65ee8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.805+0000 7ff2bcc58740 4 client.16533 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:42.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.805+0000 7ff2bcc58740 1 -- 172.21.15.133:0/1240598858 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff2a40044b0 con 0x563f65ee6220 2024-09-11T23:40:42.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.806+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563f65ee6220 0x563f65ee8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.807+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563f65ee6220 0x563f65ee8660 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c4e9856c6aca2fb0 server_cookie=8da1b1952e9ef0e4 in_seq=0 out_seq=0 2024-09-11T23:40:42.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.807+0000 7ff2937fe640 10 client.16533 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:40:42.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.807+0000 7ff2937fe640 1 -- 172.21.15.133:0/1240598858 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff2a40044b0 con 0x563f65ee6220 2024-09-11T23:40:42.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.807+0000 7ff2937fe640 10 client.16533 handle_command_reply: tid=0 2024-09-11T23:40:42.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.809+0000 7ff2bcc58740 1 client.16533 shutdown 2024-09-11T23:40:42.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.809+0000 7ff2bcc58740 1 -- 172.21.15.133:0/1240598858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 msgr2=0x563f65ed5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:42.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.809+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ed5ee0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0114b0 tx=0x7ff29c006be0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.809+0000 7ff2bcc58740 1 -- 172.21.15.133:0/1240598858 shutdown_connections 2024-09-11T23:40:42.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.810+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563f65ee6220 0x563f65ee8660 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.810+0000 7ff2937fe640 0 client.16533 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:40:42.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.810+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65ed6550 0x563f65ed3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.810+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65eccd20 0x563f65ed3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.810+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/1240598858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ecb7b0 0x563f65ed5ee0 secure :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0114b0 tx=0x7ff29c006be0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.810+0000 7ff2bcc58740 1 -- 172.21.15.133:0/1240598858 >> 172.21.15.133:0/1240598858 conn(0x563f65e94310 msgr2=0x563f65ecdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:42.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.812+0000 7ff2bcc58740 1 -- 172.21.15.133:0/1240598858 shutdown_connections 2024-09-11T23:40:42.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.812+0000 7ff2bcc58740 1 -- 172.21.15.133:0/1240598858 wait complete. 2024-09-11T23:40:42.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.813+0000 7ff2bcc58740 20 client.16533 trim_cache size 0 max 16384 2024-09-11T23:40:42.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.843+0000 7ff2bcc58740 1 Processor -- start 2024-09-11T23:40:42.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.844+0000 7ff2bcc58740 1 -- start start 2024-09-11T23:40:42.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.844+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 0x563f65ed3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.844+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ed6550 0x563f65ec6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.845+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65ee6220 0x563f65ece300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.845+0000 7ff2bcc58740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff29400e5d0 con 0x563f65ed6550 2024-09-11T23:40:42.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.845+0000 7ff2bcc58740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff294010110 con 0x563f65ee6220 2024-09-11T23:40:42.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.845+0000 7ff2bcc58740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff29400eca0 con 0x563f65eccd20 2024-09-11T23:40:42.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.845+0000 7ff2b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ed6550 0x563f65ec6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.845+0000 7ff2b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ed6550 0x563f65ec6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32780/0 (socket says 172.21.15.133:32780) 2024-09-11T23:40:42.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.845+0000 7ff2b3fff640 1 -- 172.21.15.133:0/2128180112 learned_addr learned my addr 172.21.15.133:0/2128180112 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:42.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.846+0000 7ff2abfff640 1 --2- 172.21.15.133:0/2128180112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 0x563f65ed3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.846+0000 7ff2b3fff640 1 -- 172.21.15.133:0/2128180112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 msgr2=0x563f65ed3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:42.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.846+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/2128180112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 0x563f65ed3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.846+0000 7ff2b3fff640 1 -- 172.21.15.133:0/2128180112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65ee6220 msgr2=0x563f65ece300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:40:42.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.846+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/2128180112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65ee6220 0x563f65ece300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.846+0000 7ff2b3fff640 1 -- 172.21.15.133:0/2128180112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f65ed1760 con 0x563f65ed6550 2024-09-11T23:40:42.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.847+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/2128180112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ed6550 0x563f65ec6e90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4004380 tx=0x7ff2a4004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc452db7d27aaad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:42.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.848+0000 7ff2a9ffb640 1 -- 172.21.15.133:0/2128180112 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4054070 con 0x563f65ed6550 2024-09-11T23:40:42.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.848+0000 7ff2a9ffb640 1 -- 172.21.15.133:0/2128180112 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a4005000 con 0x563f65ed6550 2024-09-11T23:40:42.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.848+0000 7ff2a9ffb640 1 -- 172.21.15.133:0/2128180112 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a403f040 con 0x563f65ed6550 2024-09-11T23:40:42.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.849+0000 7ff2bcc58740 1 -- 172.21.15.133:0/2128180112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ed6550 msgr2=0x563f65ec6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:42.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.850+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/2128180112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ed6550 0x563f65ec6e90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4004380 tx=0x7ff2a4004df0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.851+0000 7ff2bcc58740 1 -- 172.21.15.133:0/2128180112 shutdown_connections 2024-09-11T23:40:42.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.851+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/2128180112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65ee6220 0x563f65ece300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.851+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/2128180112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ed6550 0x563f65ec6e90 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.851+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/2128180112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 0x563f65ed3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.851+0000 7ff2bcc58740 1 -- 172.21.15.133:0/2128180112 >> 172.21.15.133:0/2128180112 conn(0x563f65e94310 msgr2=0x563f65ec6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:42.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.852+0000 7ff2bcc58740 1 -- 172.21.15.133:0/2128180112 shutdown_connections 2024-09-11T23:40:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.852+0000 7ff2bcc58740 1 -- 172.21.15.133:0/2128180112 wait complete. 2024-09-11T23:40:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.853+0000 7ff2bcc58740 1 -- start start 2024-09-11T23:40:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.853+0000 7ff2bcc58740 1 Processor -- start 2024-09-11T23:40:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.854+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 0x563f65ed0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.855+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ed6550 0x563f65ed0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.855+0000 7ff2bcc58740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65ee6220 0x563f65ed0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.855+0000 7ff2bcc58740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff29400fa40 con 0x563f65ed6550 2024-09-11T23:40:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.855+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 0x563f65ed0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.855+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 0x563f65ed0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57862/0 (socket says 172.21.15.133:57862) 2024-09-11T23:40:42.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.855+0000 7ff2abfff640 1 -- 172.21.15.133:0/3566473592 learned_addr learned my addr 172.21.15.133:0/3566473592 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:42.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.855+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3566473592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff29400f370 con 0x563f65ee6220 2024-09-11T23:40:42.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.855+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3566473592 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff29400c5b0 con 0x563f65eccd20 2024-09-11T23:40:42.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.856+0000 7ff2abfff640 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65ee6220 0x563f65ed0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.856+0000 7ff2bcc58740 10 client.0 _ll_register_callbacks cb 0x7ff29c00b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:42.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.856+0000 7ff2abfff640 1 -- 172.21.15.133:0/3566473592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65ee6220 msgr2=0x563f65ed0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:42.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.856+0000 7ff2abfff640 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65ee6220 0x563f65ed0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.856+0000 7ff2abfff640 1 -- 172.21.15.133:0/3566473592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ed6550 msgr2=0x563f65ed0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:42.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.856+0000 7ff2abfff640 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ed6550 0x563f65ed0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.857+0000 7ff2abfff640 1 -- 172.21.15.133:0/3566473592 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f65f4ff90 con 0x563f65eccd20 2024-09-11T23:40:42.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.858+0000 7ff2abfff640 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 0x563f65ed0250 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0111d0 tx=0x7ff29c00bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b508213b7d74a75 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:42.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.859+0000 7ff2937fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:42.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.859+0000 7ff2937fe640 1 -- 172.21.15.133:0/3566473592 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c015a30 con 0x563f65eccd20 2024-09-11T23:40:42.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.860+0000 7ff2937fe640 1 -- 172.21.15.133:0/3566473592 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff29c015bd0 con 0x563f65eccd20 2024-09-11T23:40:42.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.860+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3566473592 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a4005000 con 0x563f65eccd20 2024-09-11T23:40:42.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.861+0000 7ff2937fe640 1 -- 172.21.15.133:0/3566473592 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff29c015dd0 con 0x563f65eccd20 2024-09-11T23:40:42.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.862+0000 7ff2bcc58740 10 client.25963 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:42.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.862+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3566473592 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f65f51250 con 0x563f65eccd20 2024-09-11T23:40:42.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.863+0000 7ff2937fe640 1 -- 172.21.15.133:0/3566473592 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff29c030070 con 0x563f65eccd20 2024-09-11T23:40:42.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.863+0000 7ff2937fe640 1 -- 172.21.15.133:0/3566473592 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff29c004370 con 0x563f65eccd20 2024-09-11T23:40:42.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.863+0000 7ff2bcc58740 10 client.25963 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:42.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.863+0000 7ff2bcc58740 10 client.25963 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:40:42.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.864+0000 7ff2bcc58740 20 client.25963 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:42.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.864+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563f65f51a00 0x563f65f53e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:42.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.864+0000 7ff2bcc58740 4 client.25963 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:40:42.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.864+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3566473592 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff2a4005000 con 0x563f65f51a00 2024-09-11T23:40:42.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.865+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563f65f51a00 0x563f65f53e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:42.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.868+0000 7ff2937fe640 10 client.25963 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:40:42.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.868+0000 7ff2b3fff640 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563f65f51a00 0x563f65f53e40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c47c81d882866c server_cookie=6a01e4dd9cbba0 in_seq=0 out_seq=0 2024-09-11T23:40:42.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.869+0000 7ff2937fe640 1 -- 172.21.15.133:0/3566473592 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff2a4005000 con 0x563f65f51a00 2024-09-11T23:40:42.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.869+0000 7ff2937fe640 10 client.25963 handle_command_reply: tid=0 2024-09-11T23:40:42.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.869+0000 7ff2bcc58740 1 client.25963 shutdown 2024-09-11T23:40:42.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.870+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3566473592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 msgr2=0x563f65ed0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:42.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.870+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 0x563f65ed0250 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0111d0 tx=0x7ff29c00bf50 comp rx=0 tx=0).stop 2024-09-11T23:40:42.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.870+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3566473592 shutdown_connections 2024-09-11T23:40:42.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.870+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563f65f51a00 0x563f65f53e40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.870+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f65ee6220 0x563f65ed0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.870+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f65ed6550 0x563f65ed0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.870+0000 7ff2bcc58740 1 --2- 172.21.15.133:0/3566473592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563f65eccd20 0x563f65ed0250 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.870+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3566473592 >> 172.21.15.133:0/3566473592 conn(0x563f65e94310 msgr2=0x563f65ee1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:42.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.871+0000 7ff2937fe640 0 client.25963 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:40:42.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.873+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3566473592 shutdown_connections 2024-09-11T23:40:42.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.873+0000 7ff2bcc58740 1 -- 172.21.15.133:0/3566473592 wait complete. 2024-09-11T23:40:42.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.873+0000 7ff2bcc58740 20 client.25963 trim_cache size 0 max 16384 2024-09-11T23:40:42.873 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:40:42.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.876+0000 7ff2b961d640 1 -- 172.21.15.133:0/3472722494 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2a00a2dc0 msgr2=0x7ff2a00a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:42.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.876+0000 7ff2b961d640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2a00a2dc0 0x7ff2a00a5280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0cd2c0 tx=0x7ff2ac009930 comp rx=0 tx=0).stop 2024-09-11T23:40:42.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.876+0000 7ff2b961d640 1 -- 172.21.15.133:0/3472722494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b41068d0 msgr2=0x7ff2b406cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:42.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.876+0000 7ff2b961d640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b41068d0 0x7ff2b406cc20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b970 tx=0x7ff2ac00be30 comp rx=0 tx=0).stop 2024-09-11T23:40:42.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.878+0000 7ff2b961d640 1 -- 172.21.15.133:0/3472722494 shutdown_connections 2024-09-11T23:40:42.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.878+0000 7ff2b961d640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2a00a2dc0 0x7ff2a00a5280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.878+0000 7ff2b961d640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b41068d0 0x7ff2b406cc20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.878+0000 7ff2b961d640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2b4101270 0x7ff2b4073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.878+0000 7ff2b961d640 1 --2- 172.21.15.133:0/3472722494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b41008d0 0x7ff2b4073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:42.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.878+0000 7ff2b961d640 1 -- 172.21.15.133:0/3472722494 >> 172.21.15.133:0/3472722494 conn(0x7ff2b40fc610 msgr2=0x7ff2b40fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:42.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.878+0000 7ff2b961d640 1 -- 172.21.15.133:0/3472722494 shutdown_connections 2024-09-11T23:40:42.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:42.878+0000 7ff2b961d640 1 -- 172.21.15.133:0/3472722494 wait complete. 2024-09-11T23:40:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.238+0000 7f4d14e0e640 1 -- 172.21.15.133:0/1192682684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d101095d0 msgr2=0x7f4d10109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.238+0000 7f4d14e0e640 1 --2- 172.21.15.133:0/1192682684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d101095d0 0x7f4d10109a10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009cd0 tx=0x7f4d0802a910 comp rx=0 tx=0).stop 2024-09-11T23:40:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.238+0000 7f4d14e0e640 1 -- 172.21.15.133:0/1192682684 shutdown_connections 2024-09-11T23:40:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.238+0000 7f4d14e0e640 1 --2- 172.21.15.133:0/1192682684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1011b140 0x7f4d1011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.238+0000 7f4d14e0e640 1 --2- 172.21.15.133:0/1192682684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d101095d0 0x7f4d10109a10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.238+0000 7f4d14e0e640 1 --2- 172.21.15.133:0/1192682684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d101091f0 0x7f4d10110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.238+0000 7f4d14e0e640 1 -- 172.21.15.133:0/1192682684 >> 172.21.15.133:0/1192682684 conn(0x7f4d1006ce00 msgr2=0x7f4d1006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.239+0000 7f4d14e0e640 1 -- 172.21.15.133:0/1192682684 shutdown_connections 2024-09-11T23:40:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.239+0000 7f4d14e0e640 1 -- 172.21.15.133:0/1192682684 wait complete. 2024-09-11T23:40:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.240+0000 7f4d14e0e640 1 Processor -- start 2024-09-11T23:40:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.240+0000 7f4d14e0e640 1 -- start start 2024-09-11T23:40:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.241+0000 7f4d14e0e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d101091f0 0x7f4d101a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.241+0000 7f4d14e0e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d101095d0 0x7f4d101a7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.241+0000 7f4d14e0e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1011b140 0x7f4d101abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.241+0000 7f4d14e0e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d10120dd0 con 0x7f4d101091f0 2024-09-11T23:40:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.241+0000 7f4d14e0e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d10120c50 con 0x7f4d1011b140 2024-09-11T23:40:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.241+0000 7f4d14e0e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d10120f50 con 0x7f4d101095d0 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.241+0000 7f4d0dd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d101095d0 0x7f4d101a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.241+0000 7f4d0dd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d101095d0 0x7f4d101a7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57878/0 (socket says 172.21.15.133:57878) 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.241+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/4219057355 learned_addr learned my addr 172.21.15.133:0/4219057355 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.242+0000 7f4d0e59c640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d101091f0 0x7f4d101a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.242+0000 7f4d0ed9d640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1011b140 0x7f4d101abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.242+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/4219057355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1011b140 msgr2=0x7f4d101abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.242+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1011b140 0x7f4d101abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.242+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/4219057355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d101091f0 msgr2=0x7f4d101a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.242+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d101091f0 0x7f4d101a72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.242+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/4219057355 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d101ac280 con 0x7f4d101095d0 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.242+0000 7f4d0ed9d640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1011b140 0x7f4d101abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.242+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d101095d0 0x7f4d101a7800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4d08033040 tx=0x7f4d08031690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.243+0000 7f4cff7fe640 1 -- 172.21.15.133:0/4219057355 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080319b0 con 0x7f4d101095d0 2024-09-11T23:40:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.243+0000 7f4cff7fe640 1 -- 172.21.15.133:0/4219057355 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d08004520 con 0x7f4d101095d0 2024-09-11T23:40:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.243+0000 7f4cff7fe640 1 -- 172.21.15.133:0/4219057355 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0802fd80 con 0x7f4d101095d0 2024-09-11T23:40:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.244+0000 7f4d14e0e640 1 -- 172.21.15.133:0/4219057355 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d101ac510 con 0x7f4d101095d0 2024-09-11T23:40:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.244+0000 7f4d14e0e640 1 -- 172.21.15.133:0/4219057355 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d10110920 con 0x7f4d101095d0 2024-09-11T23:40:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.258+0000 7f4cff7fe640 1 -- 172.21.15.133:0/4219057355 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4d080046d0 con 0x7f4d101095d0 2024-09-11T23:40:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.258+0000 7f4d17c84740 1 Processor -- start 2024-09-11T23:40:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.259+0000 7f4d17c84740 1 -- start start 2024-09-11T23:40:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.260+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0faaf0 0x56378c0faed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.260+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c105770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.260+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 0x56378c1080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.260+0000 7f4d17c84740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56378c05f2f0 con 0x56378c0faaf0 2024-09-11T23:40:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.260+0000 7f4d17c84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56378c007270 con 0x56378c105cb0 2024-09-11T23:40:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.260+0000 7f4d17c84740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56378c05b2b0 con 0x56378c0fb410 2024-09-11T23:40:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.260+0000 7f4d0dd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 0x56378c1080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.260+0000 7f4d0dd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 0x56378c1080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:35858/0 (socket says 172.21.15.133:35858) 2024-09-11T23:40:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.261+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/3250735126 learned_addr learned my addr 172.21.15.133:0/3250735126 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.261+0000 7f4d0e59c640 1 --2- 172.21.15.133:0/3250735126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c105770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.261+0000 7f4d0ed9d640 1 --2- 172.21.15.133:0/3250735126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0faaf0 0x56378c0faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.261+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/3250735126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 msgr2=0x56378c105770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.261+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/3250735126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c105770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.261+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/3250735126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0faaf0 msgr2=0x56378c0faed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.261+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/3250735126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0faaf0 0x56378c0faed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.261+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/3250735126 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378c006340 con 0x56378c105cb0 2024-09-11T23:40:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.262+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/3250735126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 0x56378c1080a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d080ee1e0 tx=0x7f4d080f6ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=359bc418345ff2df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.262+0000 7f4cff7fe640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf80a2ef0 0x7f4cf80a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.263+0000 7f4d0e59c640 1 --2- 172.21.15.133:0/3250735126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.263+0000 7f4cff7fe640 1 -- 172.21.15.133:0/4219057355 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f4d080f0a30 con 0x7f4d101095d0 2024-09-11T23:40:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.264+0000 7f4d0ed9d640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf80a2ef0 0x7f4cf80a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.267+0000 7f4d0ed9d640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf80a2ef0 0x7f4cf80a53b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4d00006fd0 tx=0x7f4d00008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.269+0000 7f4cfcff9640 1 -- 172.21.15.133:0/3250735126 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080ff070 con 0x56378c105cb0 2024-09-11T23:40:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.269+0000 7f4cfcff9640 1 -- 172.21.15.133:0/3250735126 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d08036030 con 0x56378c105cb0 2024-09-11T23:40:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.270+0000 7f4d17c84740 1 -- 172.21.15.133:0/3250735126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 msgr2=0x56378c1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.270+0000 7f4d17c84740 1 --2- 172.21.15.133:0/3250735126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 0x56378c1080a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d080ee1e0 tx=0x7f4d080f6ab0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.275+0000 7f4d17c84740 1 -- 172.21.15.133:0/3250735126 shutdown_connections 2024-09-11T23:40:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.275+0000 7f4d17c84740 1 --2- 172.21.15.133:0/3250735126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 0x56378c1080a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.275+0000 7f4d17c84740 1 --2- 172.21.15.133:0/3250735126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.275+0000 7f4d17c84740 1 --2- 172.21.15.133:0/3250735126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0faaf0 0x56378c0faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.275+0000 7f4d17c84740 1 -- 172.21.15.133:0/3250735126 >> 172.21.15.133:0/3250735126 conn(0x56378c0f7060 msgr2=0x56378c0f7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.279+0000 7f4d17c84740 1 -- 172.21.15.133:0/3250735126 shutdown_connections 2024-09-11T23:40:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.279+0000 7f4d17c84740 1 -- 172.21.15.133:0/3250735126 wait complete. 2024-09-11T23:40:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.280+0000 7f4d17c84740 1 -- start start 2024-09-11T23:40:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.280+0000 7f4d17c84740 1 Processor -- start 2024-09-11T23:40:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.281+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0faaf0 0x56378c102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.281+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c103080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.281+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 0x56378c1035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.281+0000 7f4d17c84740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56378c108640 con 0x56378c0faaf0 2024-09-11T23:40:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.281+0000 7f4d17c84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56378bfe8290 con 0x56378c105cb0 2024-09-11T23:40:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.281+0000 7f4d17c84740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56378c1087c0 con 0x56378c0fb410 2024-09-11T23:40:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.281+0000 7f4d17c84740 10 client.0 _ll_register_callbacks cb 0x56378c019140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.284+0000 7f4d0ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 0x56378c1035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.284+0000 7f4d0dd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.284+0000 7f4d0dd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57890/0 (socket says 172.21.15.133:57890) 2024-09-11T23:40:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.285+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/4263025556 learned_addr learned my addr 172.21.15.133:0/4263025556 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.287+0000 7f4d0e59c640 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0faaf0 0x56378c102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.288+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/4263025556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 msgr2=0x56378c1035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.288+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 0x56378c1035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.288+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/4263025556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0faaf0 msgr2=0x56378c102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.288+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0faaf0 0x56378c102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.288+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/4263025556 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378bd72210 con 0x56378c0fb410 2024-09-11T23:40:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.289+0000 7f4d0ed9d640 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 0x56378c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.289+0000 7f4d0e59c640 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0faaf0 0x56378c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.289+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c103080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d080bcd70 tx=0x7f4d0802f320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa58b684a31d73a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.289+0000 7f4cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.294+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/4263025556 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080ff070 con 0x56378c0fb410 2024-09-11T23:40:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.294+0000 7f4d17c84740 1 -- 172.21.15.133:0/4263025556 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d08036030 con 0x56378c0fb410 2024-09-11T23:40:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.294+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/4263025556 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f4d0802f620 con 0x56378c0fb410 2024-09-11T23:40:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.300+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/4263025556 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d080b9310 con 0x56378c0fb410 2024-09-11T23:40:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.300+0000 7f4d17c84740 10 client.25969 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.300+0000 7f4d17c84740 1 -- 172.21.15.133:0/4263025556 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56378be3e620 con 0x56378c0fb410 2024-09-11T23:40:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.305+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/4263025556 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d08036030 con 0x56378c0fb410 2024-09-11T23:40:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.305+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/4263025556 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4d080b96e0 con 0x56378c0fb410 2024-09-11T23:40:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.305+0000 7f4d17c84740 10 client.25969 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.305+0000 7f4d17c84740 10 client.25969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.305+0000 7f4d17c84740 20 client.25969 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.305+0000 7f4d17c84740 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56378c115830 0x56378c117c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.305+0000 7f4d17c84740 4 client.25969 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.305+0000 7f4d17c84740 1 -- 172.21.15.133:0/4263025556 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56378c06ca20 con 0x56378c115830 2024-09-11T23:40:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.308+0000 7f4d0e59c640 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56378c115830 0x56378c117c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.314+0000 7f4cf5ffb640 10 client.25969 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.314+0000 7f4d0e59c640 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56378c115830 0x56378c117c80 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574b7002d6214a02 server_cookie=6403486316ff4473 in_seq=0 out_seq=0 2024-09-11T23:40:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.316+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/4263025556 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56378c06ca20 con 0x56378c115830 2024-09-11T23:40:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.316+0000 7f4cf5ffb640 10 client.25969 handle_command_reply: tid=0 2024-09-11T23:40:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.316+0000 7f4d17c84740 1 client.25969 shutdown 2024-09-11T23:40:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.317+0000 7f4d17c84740 1 -- 172.21.15.133:0/4263025556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 msgr2=0x56378c103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.317+0000 7f4d17c84740 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c103080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d080bcd70 tx=0x7f4d0802f320 comp rx=0 tx=0).stop 2024-09-11T23:40:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.317+0000 7f4d17c84740 1 -- 172.21.15.133:0/4263025556 shutdown_connections 2024-09-11T23:40:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.317+0000 7f4d17c84740 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56378c115830 0x56378c117c80 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.317+0000 7f4d17c84740 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c105cb0 0x56378c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.317+0000 7f4d17c84740 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c103080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.317+0000 7f4d17c84740 1 --2- 172.21.15.133:0/4263025556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0faaf0 0x56378c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.317+0000 7f4d17c84740 1 -- 172.21.15.133:0/4263025556 >> 172.21.15.133:0/4263025556 conn(0x56378c0f7060 msgr2=0x56378c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.317+0000 7f4cf5ffb640 0 client.25969 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.317+0000 7f4d17c84740 1 -- 172.21.15.133:0/4263025556 shutdown_connections 2024-09-11T23:40:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.317+0000 7f4d17c84740 1 -- 172.21.15.133:0/4263025556 wait complete. 2024-09-11T23:40:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.318+0000 7f4d17c84740 20 client.25969 trim_cache size 0 max 16384 2024-09-11T23:40:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.360+0000 7f4d17c84740 1 Processor -- start 2024-09-11T23:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.360+0000 7f4d17c84740 1 -- start start 2024-09-11T23:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.360+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c0faaf0 0x56378c0fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.360+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c1033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.360+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c105cb0 0x56378c104ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.360+0000 7f4d17c84740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d101c4ec0 con 0x56378c105cb0 2024-09-11T23:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.360+0000 7f4d17c84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d101c6a00 con 0x56378c0faaf0 2024-09-11T23:40:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.360+0000 7f4d17c84740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d101c5590 con 0x56378c0fb410 2024-09-11T23:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.361+0000 7f4d0e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c0faaf0 0x56378c0fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.361+0000 7f4d0e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c0faaf0 0x56378c0fd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:35888/0 (socket says 172.21.15.133:35888) 2024-09-11T23:40:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.361+0000 7f4d0e59c640 1 -- 172.21.15.133:0/4057230083 learned_addr learned my addr 172.21.15.133:0/4057230083 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.362+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/4057230083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c1033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.363+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/4057230083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c0faaf0 msgr2=0x56378c0fd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.363+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/4057230083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c0faaf0 0x56378c0fd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.363+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/4057230083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c105cb0 msgr2=0x56378c104ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.363+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/4057230083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c105cb0 0x56378c104ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.363+0000 7f4d0dd9b640 1 -- 172.21.15.133:0/4057230083 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378c101ad0 con 0x56378c0fb410 2024-09-11T23:40:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.363+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/4057230083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c1033f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4d080bd000 tx=0x7f4d080c5ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e49c4b8e0ffe5f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.367+0000 7f4cf7fff640 1 -- 172.21.15.133:0/4057230083 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080ff070 con 0x56378c0fb410 2024-09-11T23:40:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.367+0000 7f4cf7fff640 1 -- 172.21.15.133:0/4057230083 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d08036020 con 0x56378c0fb410 2024-09-11T23:40:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.367+0000 7f4cf7fff640 1 -- 172.21.15.133:0/4057230083 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080bc9d0 con 0x56378c0fb410 2024-09-11T23:40:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.368+0000 7f4d17c84740 1 -- 172.21.15.133:0/4057230083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 msgr2=0x56378c1033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.368+0000 7f4d17c84740 1 --2- 172.21.15.133:0/4057230083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c1033f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4d080bd000 tx=0x7f4d080c5ce0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.368+0000 7f4d17c84740 1 -- 172.21.15.133:0/4057230083 shutdown_connections 2024-09-11T23:40:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.368+0000 7f4d17c84740 1 --2- 172.21.15.133:0/4057230083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c105cb0 0x56378c104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.368+0000 7f4d17c84740 1 --2- 172.21.15.133:0/4057230083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c0fb410 0x56378c1033f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.368+0000 7f4d17c84740 1 --2- 172.21.15.133:0/4057230083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c0faaf0 0x56378c0fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.369+0000 7f4d17c84740 1 -- 172.21.15.133:0/4057230083 >> 172.21.15.133:0/4057230083 conn(0x56378c0f7060 msgr2=0x56378c1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.379+0000 7f4d17c84740 1 -- 172.21.15.133:0/4057230083 shutdown_connections 2024-09-11T23:40:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.380+0000 7f4d17c84740 1 -- 172.21.15.133:0/4057230083 wait complete. 2024-09-11T23:40:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.380+0000 7f4d17c84740 1 -- start start 2024-09-11T23:40:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.380+0000 7f4d17c84740 1 Processor -- start 2024-09-11T23:40:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.381+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c0faaf0 0x56378c10dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.381+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0fb410 0x56378c10e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.381+0000 7f4d17c84740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c105cb0 0x56378c0ff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.381+0000 7f4d17c84740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d101c6330 con 0x56378c0fb410 2024-09-11T23:40:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.381+0000 7f4d17c84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d101c5c60 con 0x56378c0faaf0 2024-09-11T23:40:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.381+0000 7f4d17c84740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d10117190 con 0x56378c105cb0 2024-09-11T23:40:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.381+0000 7f4d17c84740 10 client.0 _ll_register_callbacks cb 0x7f4d080f6870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4d0ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c105cb0 0x56378c0ff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4d0ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c105cb0 0x56378c0ff350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57916/0 (socket says 172.21.15.133:57916) 2024-09-11T23:40:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4d0ed9d640 1 -- 172.21.15.133:0/85370690 learned_addr learned my addr 172.21.15.133:0/85370690 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4d0e59c640 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c0faaf0 0x56378c10dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4d0dd9b640 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0fb410 0x56378c10e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4d0ed9d640 1 -- 172.21.15.133:0/85370690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c0faaf0 msgr2=0x56378c10dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4d0ed9d640 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c0faaf0 0x56378c10dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4d0ed9d640 1 -- 172.21.15.133:0/85370690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0fb410 msgr2=0x56378c10e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4d0ed9d640 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0fb410 0x56378c10e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4d0ed9d640 1 -- 172.21.15.133:0/85370690 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378c0fb240 con 0x56378c105cb0 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4d0ed9d640 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c105cb0 0x56378c0ff350 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4d000136b0 tx=0x7f4d00013ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbfb7cb547865f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.382+0000 7f4cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.383+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/85370690 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00023020 con 0x56378c105cb0 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.383+0000 7f4d17c84740 1 -- 172.21.15.133:0/85370690 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d08036020 con 0x56378c105cb0 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.383+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/85370690 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f4d000144c0 con 0x56378c105cb0 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.383+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/85370690 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d0002e050 con 0x56378c105cb0 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.383+0000 7f4d17c84740 10 client.25975 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.383+0000 7f4d17c84740 1 -- 172.21.15.133:0/85370690 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56378c17fe80 con 0x56378c105cb0 2024-09-11T23:40:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.383+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/85370690 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d08036020 con 0x56378c105cb0 2024-09-11T23:40:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.385+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/85370690 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4d0001f410 con 0x56378c105cb0 2024-09-11T23:40:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.385+0000 7f4d17c84740 10 client.25975 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.385+0000 7f4d17c84740 10 client.25975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.385+0000 7f4d17c84740 20 client.25975 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.385+0000 7f4d17c84740 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56378c180600 0x56378c182a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.385+0000 7f4d17c84740 4 client.25975 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:40:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.385+0000 7f4d17c84740 1 -- 172.21.15.133:0/85370690 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56378c066d70 con 0x56378c180600 2024-09-11T23:40:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.386+0000 7f4d0e59c640 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56378c180600 0x56378c182a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.387+0000 7f4d0e59c640 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56378c180600 0x56378c182a20 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5635c4dfef4ae7c7 server_cookie=578d0a12faee02e5 in_seq=0 out_seq=0 2024-09-11T23:40:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.387+0000 7f4cf5ffb640 10 client.25975 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.815+0000 7f4cf5ffb640 1 -- 172.21.15.133:0/85370690 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56378c066d70 con 0x56378c180600 2024-09-11T23:40:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.815+0000 7f4cf5ffb640 10 client.25975 handle_command_reply: tid=0 2024-09-11T23:40:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.815+0000 7f4d17c84740 1 client.25975 shutdown 2024-09-11T23:40:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.817+0000 7f4d17c84740 1 -- 172.21.15.133:0/85370690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c105cb0 msgr2=0x56378c0ff350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.817+0000 7f4d17c84740 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c105cb0 0x56378c0ff350 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4d000136b0 tx=0x7f4d00013ab0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.817+0000 7f4d17c84740 1 -- 172.21.15.133:0/85370690 shutdown_connections 2024-09-11T23:40:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.817+0000 7f4d17c84740 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56378c180600 0x56378c182a20 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.817+0000 7f4d17c84740 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56378c105cb0 0x56378c0ff350 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.817+0000 7f4d17c84740 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378c0fb410 0x56378c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.817+0000 7f4d17c84740 1 --2- 172.21.15.133:0/85370690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56378c0faaf0 0x56378c10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.817+0000 7f4d17c84740 1 -- 172.21.15.133:0/85370690 >> 172.21.15.133:0/85370690 conn(0x56378c0f7060 msgr2=0x56378c0f7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.817+0000 7f4cf5ffb640 0 client.25975 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.818+0000 7f4d17c84740 1 -- 172.21.15.133:0/85370690 shutdown_connections 2024-09-11T23:40:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.818+0000 7f4d17c84740 1 -- 172.21.15.133:0/85370690 wait complete. 2024-09-11T23:40:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.818+0000 7f4d17c84740 20 client.25975 trim_cache size 0 max 16384 2024-09-11T23:40:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:40:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:40:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:40:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:40:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.820+0000 7f4d14e0e640 1 -- 172.21.15.133:0/4219057355 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf80a2ef0 msgr2=0x7f4cf80a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.820+0000 7f4d14e0e640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf80a2ef0 0x7f4cf80a53b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4d00006fd0 tx=0x7f4d00008040 comp rx=0 tx=0).stop 2024-09-11T23:40:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.820+0000 7f4d14e0e640 1 -- 172.21.15.133:0/4219057355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d101095d0 msgr2=0x7f4d101a7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.820+0000 7f4d14e0e640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d101095d0 0x7f4d101a7800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4d08033040 tx=0x7f4d08031690 comp rx=0 tx=0).stop 2024-09-11T23:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.821+0000 7f4d14e0e640 1 -- 172.21.15.133:0/4219057355 shutdown_connections 2024-09-11T23:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.821+0000 7f4d14e0e640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf80a2ef0 0x7f4cf80a53b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.821+0000 7f4d14e0e640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1011b140 0x7f4d101abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.821+0000 7f4d14e0e640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d101095d0 0x7f4d101a7800 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.821+0000 7f4d14e0e640 1 --2- 172.21.15.133:0/4219057355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d101091f0 0x7f4d101a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.821+0000 7f4d14e0e640 1 -- 172.21.15.133:0/4219057355 >> 172.21.15.133:0/4219057355 conn(0x7f4d1006ce00 msgr2=0x7f4d1011b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.821+0000 7f4d14e0e640 1 -- 172.21.15.133:0/4219057355 shutdown_connections 2024-09-11T23:40:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:43.821+0000 7f4d14e0e640 1 -- 172.21.15.133:0/4219057355 wait complete. 2024-09-11T23:40:44.004 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:40:44.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:44 smithi138 ceph-mon[35142]: pgmap v121: 97 pgs: 97 active+clean; 30 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 820 MiB/s wr, 1.59k op/s 2024-09-11T23:40:44.712 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:44 smithi133 ceph-mon[31430]: pgmap v121: 97 pgs: 97 active+clean; 30 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 820 MiB/s wr, 1.59k op/s 2024-09-11T23:40:44.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:44 smithi178 ceph-mon[35882]: pgmap v121: 97 pgs: 97 active+clean; 30 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 820 MiB/s wr, 1.59k op/s 2024-09-11T23:40:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:45.623 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:45 smithi133 ceph-mon[31430]: pgmap v122: 97 pgs: 97 active+clean; 30 GiB data, 120 GiB used, 952 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 703 MiB/s wr, 1.33k op/s 2024-09-11T23:40:45.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:45 smithi138 ceph-mon[35142]: pgmap v122: 97 pgs: 97 active+clean; 30 GiB data, 120 GiB used, 952 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 703 MiB/s wr, 1.33k op/s 2024-09-11T23:40:45.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:45 smithi178 ceph-mon[35882]: pgmap v122: 97 pgs: 97 active+clean; 30 GiB data, 120 GiB used, 952 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 703 MiB/s wr, 1.33k op/s 2024-09-11T23:40:46.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.103+0000 7fe7ba00c640 1 -- 172.21.15.133:0/2312119164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7b4108280 msgr2=0x7fe7b4108660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:46.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.103+0000 7fe7ba00c640 1 --2- 172.21.15.133:0/2312119164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7b4108280 0x7fe7b4108660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009900 tx=0x7fe7b002a410 comp rx=0 tx=0).stop 2024-09-11T23:40:46.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.104+0000 7fe7ba00c640 1 -- 172.21.15.133:0/2312119164 shutdown_connections 2024-09-11T23:40:46.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.104+0000 7fe7ba00c640 1 --2- 172.21.15.133:0/2312119164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b410d360 0x7fe7b4114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.104+0000 7fe7ba00c640 1 --2- 172.21.15.133:0/2312119164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7b4108c30 0x7fe7b410cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.104+0000 7fe7ba00c640 1 --2- 172.21.15.133:0/2312119164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7b4108280 0x7fe7b4108660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.104+0000 7fe7ba00c640 1 -- 172.21.15.133:0/2312119164 >> 172.21.15.133:0/2312119164 conn(0x7fe7b40779f0 msgr2=0x7fe7b4079e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:46.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.105+0000 7fe7ba00c640 1 -- 172.21.15.133:0/2312119164 shutdown_connections 2024-09-11T23:40:46.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.105+0000 7fe7ba00c640 1 -- 172.21.15.133:0/2312119164 wait complete. 2024-09-11T23:40:46.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.106+0000 7fe7ba00c640 1 Processor -- start 2024-09-11T23:40:46.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.106+0000 7fe7ba00c640 1 -- start start 2024-09-11T23:40:46.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.106+0000 7fe7ba00c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b4108280 0x7fe7b4073b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.106+0000 7fe7ba00c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7b4108c30 0x7fe7b406cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.106+0000 7fe7ba00c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7b410d360 0x7fe7b406d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.106+0000 7fe7ba00c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41183b0 con 0x7fe7b4108c30 2024-09-11T23:40:46.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.106+0000 7fe7ba00c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4118230 con 0x7fe7b4108280 2024-09-11T23:40:46.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.106+0000 7fe7ba00c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4118530 con 0x7fe7b410d360 2024-09-11T23:40:46.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.108+0000 7fe7b980b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7b410d360 0x7fe7b406d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.108+0000 7fe7b980b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7b410d360 0x7fe7b406d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57926/0 (socket says 172.21.15.133:57926) 2024-09-11T23:40:46.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.108+0000 7fe7b980b640 1 -- 172.21.15.133:0/1670410964 learned_addr learned my addr 172.21.15.133:0/1670410964 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.111+0000 7fe7b8809640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7b4108c30 0x7fe7b406cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.112+0000 7fe7b900a640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b4108280 0x7fe7b4073b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.115+0000 7fe7b8809640 1 -- 172.21.15.133:0/1670410964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7b410d360 msgr2=0x7fe7b406d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.115+0000 7fe7b8809640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7b410d360 0x7fe7b406d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.115+0000 7fe7b8809640 1 -- 172.21.15.133:0/1670410964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b4108280 msgr2=0x7fe7b4073b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.115+0000 7fe7b8809640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b4108280 0x7fe7b4073b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.115+0000 7fe7b8809640 1 -- 172.21.15.133:0/1670410964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b406d810 con 0x7fe7b4108c30 2024-09-11T23:40:46.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.119+0000 7fe7b8809640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7b4108c30 0x7fe7b406cba0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400d6d0 tx=0x7fe7a400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:46.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.122+0000 7fe7aa7fc640 1 -- 172.21.15.133:0/1670410964 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4017070 con 0x7fe7b4108c30 2024-09-11T23:40:46.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.122+0000 7fe7aa7fc640 1 -- 172.21.15.133:0/1670410964 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a4009820 con 0x7fe7b4108c30 2024-09-11T23:40:46.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.124+0000 7fe7aa7fc640 1 -- 172.21.15.133:0/1670410964 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4004d00 con 0x7fe7b4108c30 2024-09-11T23:40:46.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.124+0000 7fe7ba00c640 1 -- 172.21.15.133:0/1670410964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b4071640 con 0x7fe7b4108c30 2024-09-11T23:40:46.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.125+0000 7fe7ba00c640 1 -- 172.21.15.133:0/1670410964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b4071b00 con 0x7fe7b4108c30 2024-09-11T23:40:46.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.139+0000 7fe7aa7fc640 1 -- 172.21.15.133:0/1670410964 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe7a4005220 con 0x7fe7b4108c30 2024-09-11T23:40:46.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.144+0000 7fe7bd647740 1 Processor -- start 2024-09-11T23:40:46.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.145+0000 7fe7bd647740 1 -- start start 2024-09-11T23:40:46.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.145+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec1f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.145+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec1fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.145+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec2a550 0x55967ec2c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.145+0000 7fe7bd647740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55967eb852f0 con 0x55967ec20d20 2024-09-11T23:40:46.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.145+0000 7fe7bd647740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55967eb2ccf0 con 0x55967ec2a550 2024-09-11T23:40:46.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.145+0000 7fe7bd647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55967eb812b0 con 0x55967ec1f7b0 2024-09-11T23:40:46.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.147+0000 7fe7aa7fc640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe7a00a2d60 0x7fe7a00a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.148+0000 7fe7aa7fc640 1 -- 172.21.15.133:0/1670410964 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe7a40cef70 con 0x7fe7b4108c30 2024-09-11T23:40:46.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.150+0000 7fe7b900a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec1fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.151+0000 7fe7b980b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec1f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.151+0000 7fe7b980b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec1f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32868/0 (socket says 172.21.15.133:32868) 2024-09-11T23:40:46.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.151+0000 7fe7b980b640 1 -- 172.21.15.133:0/245396970 learned_addr learned my addr 172.21.15.133:0/245396970 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:46.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.151+0000 7fe7b980b640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe7a00a2d60 0x7fe7a00a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.153+0000 7fe7b8809640 1 --2- 172.21.15.133:0/245396970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec2a550 0x55967ec2c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.155+0000 7fe7b980b640 1 -- 172.21.15.133:0/245396970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 msgr2=0x55967ec1fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.155+0000 7fe7b980b640 1 --2- 172.21.15.133:0/245396970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec1fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.155+0000 7fe7b980b640 1 -- 172.21.15.133:0/245396970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec2a550 msgr2=0x55967ec2c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.155+0000 7fe7b980b640 1 --2- 172.21.15.133:0/245396970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec2a550 0x55967ec2c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.155+0000 7fe7b980b640 1 -- 172.21.15.133:0/245396970 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55967eb113a0 con 0x55967ec20d20 2024-09-11T23:40:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.156+0000 7fe7b980b640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe7a00a2d60 0x7fe7a00a5220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0094d0 tx=0x7fe7ac018000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:46.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.159+0000 7fe7b980b640 1 --2- 172.21.15.133:0/245396970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec1f270 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac013ed0 tx=0x7fe7ac0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9152d7756671cad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:46.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.165+0000 7fe79f7fe640 1 -- 172.21.15.133:0/245396970 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac024070 con 0x55967ec20d20 2024-09-11T23:40:46.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.165+0000 7fe79f7fe640 1 -- 172.21.15.133:0/245396970 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7ac0139e0 con 0x55967ec20d20 2024-09-11T23:40:46.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.165+0000 7fe79f7fe640 1 -- 172.21.15.133:0/245396970 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac005020 con 0x55967ec20d20 2024-09-11T23:40:46.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.167+0000 7fe7bd647740 1 -- 172.21.15.133:0/245396970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 msgr2=0x55967ec1f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:46.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.167+0000 7fe7bd647740 1 --2- 172.21.15.133:0/245396970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec1f270 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac013ed0 tx=0x7fe7ac0117b0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.168+0000 7fe7bd647740 1 -- 172.21.15.133:0/245396970 shutdown_connections 2024-09-11T23:40:46.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.168+0000 7fe7bd647740 1 --2- 172.21.15.133:0/245396970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec2a550 0x55967ec2c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.168+0000 7fe7bd647740 1 --2- 172.21.15.133:0/245396970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec1fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.168+0000 7fe7bd647740 1 --2- 172.21.15.133:0/245396970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec1f270 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.168+0000 7fe7bd647740 1 -- 172.21.15.133:0/245396970 >> 172.21.15.133:0/245396970 conn(0x55967ebe8310 msgr2=0x55967ec1be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.173+0000 7fe7bd647740 1 -- 172.21.15.133:0/245396970 shutdown_connections 2024-09-11T23:40:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.173+0000 7fe7bd647740 1 -- 172.21.15.133:0/245396970 wait complete. 2024-09-11T23:40:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.173+0000 7fe7bd647740 1 -- start start 2024-09-11T23:40:46.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.174+0000 7fe7bd647740 1 Processor -- start 2024-09-11T23:40:46.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.175+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec1f7b0 0x55967ec29f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.175+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec27200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.175+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec2a550 0x55967ec27830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.175+0000 7fe7bd647740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55967ec2d1f0 con 0x55967ec20d20 2024-09-11T23:40:46.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.175+0000 7fe7bd647740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55967eb40060 con 0x55967ec1f7b0 2024-09-11T23:40:46.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.175+0000 7fe7bd647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55967ec2d370 con 0x55967ec2a550 2024-09-11T23:40:46.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.175+0000 7fe7bd647740 10 client.0 _ll_register_callbacks cb 0x55967e8938c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.177+0000 7fe7b8809640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec27200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.178+0000 7fe7b8809640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec27200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32882/0 (socket says 172.21.15.133:32882) 2024-09-11T23:40:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.178+0000 7fe7b8809640 1 -- 172.21.15.133:0/2300651341 learned_addr learned my addr 172.21.15.133:0/2300651341 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.179+0000 7fe7b8809640 1 -- 172.21.15.133:0/2300651341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec2a550 msgr2=0x55967ec27830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:40:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.179+0000 7fe7b8809640 1 --2- 172.21.15.133:0/2300651341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec2a550 0x55967ec27830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.179+0000 7fe7b8809640 1 -- 172.21.15.133:0/2300651341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec1f7b0 msgr2=0x55967ec29f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:40:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.179+0000 7fe7b8809640 1 --2- 172.21.15.133:0/2300651341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec1f7b0 0x55967ec29f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.179+0000 7fe7b8809640 1 -- 172.21.15.133:0/2300651341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55967e8f63c0 con 0x55967ec20d20 2024-09-11T23:40:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.180+0000 7fe7b8809640 1 --2- 172.21.15.133:0/2300651341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec27200 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4097ef0 tx=0x7fe7a409b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fd80be742c0e25 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.181+0000 7fe79cff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:40:46.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.181+0000 7fe79cff9640 1 -- 172.21.15.133:0/2300651341 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40b9060 con 0x55967ec20d20 2024-09-11T23:40:46.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.182+0000 7fe7bd647740 1 -- 172.21.15.133:0/2300651341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7ac0139e0 con 0x55967ec20d20 2024-09-11T23:40:46.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.182+0000 7fe79cff9640 1 -- 172.21.15.133:0/2300651341 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe7a40bb2f0 con 0x55967ec20d20 2024-09-11T23:40:46.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.182+0000 7fe79cff9640 1 -- 172.21.15.133:0/2300651341 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7a40b4070 con 0x55967ec20d20 2024-09-11T23:40:46.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.182+0000 7fe7bd647740 10 client.16557 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:46.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.182+0000 7fe7bd647740 1 -- 172.21.15.133:0/2300651341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55967ec38d30 con 0x55967ec20d20 2024-09-11T23:40:46.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.187+0000 7fe79cff9640 1 -- 172.21.15.133:0/2300651341 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7ac0139e0 con 0x55967ec20d20 2024-09-11T23:40:46.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.187+0000 7fe79cff9640 1 -- 172.21.15.133:0/2300651341 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe7a40b4210 con 0x55967ec20d20 2024-09-11T23:40:46.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.187+0000 7fe7bd647740 10 client.16557 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:46.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.187+0000 7fe7bd647740 10 client.16557 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:40:46.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.187+0000 7fe7bd647740 20 client.16557 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:46.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.187+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2300651341 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55967ec3a680 0x55967ec3caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.187+0000 7fe7bd647740 4 client.16557 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:46.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.187+0000 7fe7bd647740 1 -- 172.21.15.133:0/2300651341 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55967eb92860 con 0x55967ec3a680 2024-09-11T23:40:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.199+0000 7fe7b900a640 1 --2- 172.21.15.133:0/2300651341 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55967ec3a680 0x55967ec3caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.208+0000 7fe7b900a640 1 --2- 172.21.15.133:0/2300651341 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55967ec3a680 0x55967ec3caf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=282bf24219ac7995 server_cookie=38b949a73644973 in_seq=0 out_seq=0 2024-09-11T23:40:46.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.208+0000 7fe79cff9640 10 client.16557 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:40:46.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.210+0000 7fe79cff9640 1 -- 172.21.15.133:0/2300651341 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55967eb92860 con 0x55967ec3a680 2024-09-11T23:40:46.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.211+0000 7fe79cff9640 10 client.16557 handle_command_reply: tid=0 2024-09-11T23:40:46.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.211+0000 7fe7bd647740 1 client.16557 shutdown 2024-09-11T23:40:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.211+0000 7fe7bd647740 1 -- 172.21.15.133:0/2300651341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 msgr2=0x55967ec27200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:46.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.211+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2300651341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec27200 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4097ef0 tx=0x7fe7a409b060 comp rx=0 tx=0).stop 2024-09-11T23:40:46.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.213+0000 7fe7bd647740 1 -- 172.21.15.133:0/2300651341 shutdown_connections 2024-09-11T23:40:46.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.213+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2300651341 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55967ec3a680 0x55967ec3caf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.213+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2300651341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec2a550 0x55967ec27830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.213+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2300651341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec27200 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.213+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2300651341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec1f7b0 0x55967ec29f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.213+0000 7fe79cff9640 0 client.16557 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:40:46.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.213+0000 7fe7bd647740 1 -- 172.21.15.133:0/2300651341 >> 172.21.15.133:0/2300651341 conn(0x55967ebe8310 msgr2=0x55967ec21b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:46.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.216+0000 7fe7bd647740 1 -- 172.21.15.133:0/2300651341 shutdown_connections 2024-09-11T23:40:46.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.216+0000 7fe7bd647740 1 -- 172.21.15.133:0/2300651341 wait complete. 2024-09-11T23:40:46.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.216+0000 7fe7bd647740 20 client.16557 trim_cache size 0 max 16384 2024-09-11T23:40:46.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.259+0000 7fe7bd647740 1 Processor -- start 2024-09-11T23:40:46.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.260+0000 7fe7bd647740 1 -- start start 2024-09-11T23:40:46.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.260+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec3ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.260+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec20d20 0x55967ec36490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.260+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec2a550 0x55967ec22300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.260+0000 7fe7bd647740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41c4990 con 0x55967ec2a550 2024-09-11T23:40:46.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.260+0000 7fe7bd647740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41c64d0 con 0x55967ec20d20 2024-09-11T23:40:46.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.260+0000 7fe7bd647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41c5060 con 0x55967ec1f7b0 2024-09-11T23:40:46.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.264+0000 7fe7b8809640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec20d20 0x55967ec36490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.264+0000 7fe7b900a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec3ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.264+0000 7fe7b900a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec3ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57956/0 (socket says 172.21.15.133:57956) 2024-09-11T23:40:46.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.264+0000 7fe7b900a640 1 -- 172.21.15.133:0/3093794717 learned_addr learned my addr 172.21.15.133:0/3093794717 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:46.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.265+0000 7fe7b900a640 1 -- 172.21.15.133:0/3093794717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec20d20 msgr2=0x55967ec36490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:46.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.265+0000 7fe7b900a640 1 --2- 172.21.15.133:0/3093794717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec20d20 0x55967ec36490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.265+0000 7fe7b900a640 1 -- 172.21.15.133:0/3093794717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec2a550 msgr2=0x55967ec22300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:46.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.265+0000 7fe7b980b640 1 --2- 172.21.15.133:0/3093794717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec2a550 0x55967ec22300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.266+0000 7fe7b900a640 1 --2- 172.21.15.133:0/3093794717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec2a550 0x55967ec22300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.266+0000 7fe7b900a640 1 -- 172.21.15.133:0/3093794717 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55967ec2d360 con 0x55967ec1f7b0 2024-09-11T23:40:46.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.266+0000 7fe7b980b640 1 --2- 172.21.15.133:0/3093794717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec2a550 0x55967ec22300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:46.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.266+0000 7fe7b900a640 1 --2- 172.21.15.133:0/3093794717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec3ca70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009230 tx=0x7fe7b0005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a5321fde66e9324 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:46.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.267+0000 7fe79effd640 1 -- 172.21.15.133:0/3093794717 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0035040 con 0x55967ec1f7b0 2024-09-11T23:40:46.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.267+0000 7fe79effd640 1 -- 172.21.15.133:0/3093794717 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b0005500 con 0x55967ec1f7b0 2024-09-11T23:40:46.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.267+0000 7fe79effd640 1 -- 172.21.15.133:0/3093794717 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0031810 con 0x55967ec1f7b0 2024-09-11T23:40:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.270+0000 7fe7bd647740 1 -- 172.21.15.133:0/3093794717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 msgr2=0x55967ec3ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.270+0000 7fe7bd647740 1 --2- 172.21.15.133:0/3093794717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec3ca70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009230 tx=0x7fe7b0005d50 comp rx=0 tx=0).stop 2024-09-11T23:40:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.270+0000 7fe7bd647740 1 -- 172.21.15.133:0/3093794717 shutdown_connections 2024-09-11T23:40:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.270+0000 7fe7bd647740 1 --2- 172.21.15.133:0/3093794717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec2a550 0x55967ec22300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.270+0000 7fe7bd647740 1 --2- 172.21.15.133:0/3093794717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec20d20 0x55967ec36490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.270+0000 7fe7bd647740 1 --2- 172.21.15.133:0/3093794717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec3ca70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.270+0000 7fe7bd647740 1 -- 172.21.15.133:0/3093794717 >> 172.21.15.133:0/3093794717 conn(0x55967ebe8310 msgr2=0x55967ec16b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:46.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.272+0000 7fe7bd647740 1 -- 172.21.15.133:0/3093794717 shutdown_connections 2024-09-11T23:40:46.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.272+0000 7fe7bd647740 1 -- 172.21.15.133:0/3093794717 wait complete. 2024-09-11T23:40:46.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.272+0000 7fe7bd647740 1 -- start start 2024-09-11T23:40:46.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.273+0000 7fe7bd647740 1 Processor -- start 2024-09-11T23:40:46.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.273+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec32890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.273+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec32dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.274+0000 7fe7bd647740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec2a550 0x55967ec23ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.274+0000 7fe7bd647740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41c5e00 con 0x55967ec20d20 2024-09-11T23:40:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.274+0000 7fe7bd647740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41c5730 con 0x55967ec2a550 2024-09-11T23:40:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.274+0000 7fe7bd647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41c29b0 con 0x55967ec1f7b0 2024-09-11T23:40:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.274+0000 7fe7bd647740 10 client.0 _ll_register_callbacks cb 0x7fe7a409b2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.275+0000 7fe7b900a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec32890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.275+0000 7fe7b900a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec32890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57972/0 (socket says 172.21.15.133:57972) 2024-09-11T23:40:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.275+0000 7fe7b900a640 1 -- 172.21.15.133:0/2932608255 learned_addr learned my addr 172.21.15.133:0/2932608255 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.275+0000 7fe7b900a640 1 -- 172.21.15.133:0/2932608255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec2a550 msgr2=0x55967ec23ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.275+0000 7fe7b900a640 1 --2- 172.21.15.133:0/2932608255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec2a550 0x55967ec23ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.275+0000 7fe7b900a640 1 -- 172.21.15.133:0/2932608255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 msgr2=0x55967ec32dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.275+0000 7fe7b900a640 1 --2- 172.21.15.133:0/2932608255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec32dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.275+0000 7fe7b900a640 1 -- 172.21.15.133:0/2932608255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55967ec24300 con 0x55967ec1f7b0 2024-09-11T23:40:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.275+0000 7fe7b900a640 1 --2- 172.21.15.133:0/2932608255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec32890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009850 tx=0x7fe7b0009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0b2b9c7c8be7260 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:46.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.279+0000 7fe79cff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:46.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.279+0000 7fe79cff9640 1 -- 172.21.15.133:0/2932608255 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0051070 con 0x55967ec1f7b0 2024-09-11T23:40:46.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.279+0000 7fe7bd647740 1 -- 172.21.15.133:0/2932608255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b0005500 con 0x55967ec1f7b0 2024-09-11T23:40:46.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.279+0000 7fe79cff9640 1 -- 172.21.15.133:0/2932608255 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe7b0005020 con 0x55967ec1f7b0 2024-09-11T23:40:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.285+0000 7fe79cff9640 1 -- 172.21.15.133:0/2932608255 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7b0035040 con 0x55967ec1f7b0 2024-09-11T23:40:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.285+0000 7fe7bd647740 10 client.25990 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.285+0000 7fe7bd647740 1 -- 172.21.15.133:0/2932608255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55967eca4e80 con 0x55967ec1f7b0 2024-09-11T23:40:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.289+0000 7fe79cff9640 1 -- 172.21.15.133:0/2932608255 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b0005500 con 0x55967ec1f7b0 2024-09-11T23:40:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.291+0000 7fe79cff9640 1 -- 172.21.15.133:0/2932608255 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe7b0031890 con 0x55967ec1f7b0 2024-09-11T23:40:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.291+0000 7fe7bd647740 10 client.25990 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.291+0000 7fe7bd647740 10 client.25990 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:40:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.291+0000 7fe7bd647740 20 client.25990 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.291+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2932608255 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55967eca54a0 0x55967eca7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.291+0000 7fe7bd647740 4 client.25990 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:40:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.291+0000 7fe7bd647740 1 -- 172.21.15.133:0/2932608255 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55967eb8cd70 con 0x55967eca54a0 2024-09-11T23:40:46.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.292+0000 7fe7b8809640 1 --2- 172.21.15.133:0/2932608255 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55967eca54a0 0x55967eca7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:46.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.296+0000 7fe7b8809640 1 --2- 172.21.15.133:0/2932608255 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55967eca54a0 0x55967eca7940 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d56baf69cc03936 server_cookie=a6a3e12a079a4c43 in_seq=0 out_seq=0 2024-09-11T23:40:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.297+0000 7fe79cff9640 10 client.25990 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:40:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.297+0000 7fe79cff9640 1 -- 172.21.15.133:0/2932608255 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55967eb8cd70 con 0x55967eca54a0 2024-09-11T23:40:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.297+0000 7fe79cff9640 10 client.25990 handle_command_reply: tid=0 2024-09-11T23:40:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.297+0000 7fe7bd647740 1 client.25990 shutdown 2024-09-11T23:40:46.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.300+0000 7fe7bd647740 1 -- 172.21.15.133:0/2932608255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 msgr2=0x55967ec32890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:46.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.300+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2932608255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec32890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009850 tx=0x7fe7b0009680 comp rx=0 tx=0).stop 2024-09-11T23:40:46.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.300+0000 7fe7bd647740 1 -- 172.21.15.133:0/2932608255 shutdown_connections 2024-09-11T23:40:46.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.300+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2932608255 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55967eca54a0 0x55967eca7940 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.300+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2932608255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55967ec2a550 0x55967ec23ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.300+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2932608255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55967ec20d20 0x55967ec32dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.301+0000 7fe7bd647740 1 --2- 172.21.15.133:0/2932608255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55967ec1f7b0 0x55967ec32890 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.301+0000 7fe7bd647740 1 -- 172.21.15.133:0/2932608255 >> 172.21.15.133:0/2932608255 conn(0x55967ebe8310 msgr2=0x55967ec28a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:46.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.301+0000 7fe79cff9640 0 client.25990 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:40:46.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.301+0000 7fe7bd647740 1 -- 172.21.15.133:0/2932608255 shutdown_connections 2024-09-11T23:40:46.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.301+0000 7fe7bd647740 1 -- 172.21.15.133:0/2932608255 wait complete. 2024-09-11T23:40:46.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.302+0000 7fe7bd647740 20 client.25990 trim_cache size 0 max 16384 2024-09-11T23:40:46.302 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:40:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.303+0000 7fe7ba00c640 1 -- 172.21.15.133:0/1670410964 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe7a00a2d60 msgr2=0x7fe7a00a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.303+0000 7fe7ba00c640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe7a00a2d60 0x7fe7a00a5220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0094d0 tx=0x7fe7ac018000 comp rx=0 tx=0).stop 2024-09-11T23:40:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.303+0000 7fe7ba00c640 1 -- 172.21.15.133:0/1670410964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7b4108c30 msgr2=0x7fe7b406cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.303+0000 7fe7ba00c640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7b4108c30 0x7fe7b406cba0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400d6d0 tx=0x7fe7a400db90 comp rx=0 tx=0).stop 2024-09-11T23:40:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.304+0000 7fe7ba00c640 1 -- 172.21.15.133:0/1670410964 shutdown_connections 2024-09-11T23:40:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.304+0000 7fe7ba00c640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe7a00a2d60 0x7fe7a00a5220 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.304+0000 7fe7ba00c640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7b410d360 0x7fe7b406d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.304+0000 7fe7ba00c640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7b4108c30 0x7fe7b406cba0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.304+0000 7fe7ba00c640 1 --2- 172.21.15.133:0/1670410964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b4108280 0x7fe7b4073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.304+0000 7fe7ba00c640 1 -- 172.21.15.133:0/1670410964 >> 172.21.15.133:0/1670410964 conn(0x7fe7b40779f0 msgr2=0x7fe7b4078a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.305+0000 7fe7ba00c640 1 -- 172.21.15.133:0/1670410964 shutdown_connections 2024-09-11T23:40:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:46.305+0000 7fe7ba00c640 1 -- 172.21.15.133:0/1670410964 wait complete. 2024-09-11T23:40:47.371 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:40:47.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:47 smithi138 ceph-mon[35142]: pgmap v123: 97 pgs: 97 active+clean; 30 GiB data, 121 GiB used, 951 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 617 MiB/s wr, 1.02k op/s 2024-09-11T23:40:47.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:47 smithi133 ceph-mon[31430]: pgmap v123: 97 pgs: 97 active+clean; 30 GiB data, 121 GiB used, 951 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 617 MiB/s wr, 1.02k op/s 2024-09-11T23:40:47.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:47 smithi178 ceph-mon[35882]: pgmap v123: 97 pgs: 97 active+clean; 30 GiB data, 121 GiB used, 951 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 617 MiB/s wr, 1.02k op/s 2024-09-11T23:40:48.342 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:40:48.342 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T23:40:48.342 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:40:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:49.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.771+0000 7f095ff64640 1 -- 172.21.15.133:0/2829665403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f095810d520 msgr2=0x7f09581150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:49.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.771+0000 7f095ff64640 1 --2- 172.21.15.133:0/2829665403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f095810d520 0x7f09581150a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0948009860 tx=0x7f094802a380 comp rx=0 tx=0).stop 2024-09-11T23:40:49.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.771+0000 7f095ff64640 1 -- 172.21.15.133:0/2829665403 shutdown_connections 2024-09-11T23:40:49.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.771+0000 7f095ff64640 1 --2- 172.21.15.133:0/2829665403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f095810d520 0x7f09581150a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.771+0000 7f095ff64640 1 --2- 172.21.15.133:0/2829665403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0958108da0 0x7f095810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.771+0000 7f095ff64640 1 --2- 172.21.15.133:0/2829665403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09581083f0 0x7f09581087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.771+0000 7f095ff64640 1 -- 172.21.15.133:0/2829665403 >> 172.21.15.133:0/2829665403 conn(0x7f0958077af0 msgr2=0x7f0958079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.771+0000 7f095ff64640 1 -- 172.21.15.133:0/2829665403 shutdown_connections 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.772+0000 7f095ff64640 1 -- 172.21.15.133:0/2829665403 wait complete. 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.772+0000 7f095ff64640 1 Processor -- start 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.773+0000 7f095ff64640 1 -- start start 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.773+0000 7f095ff64640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09581083f0 0x7f09580739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.773+0000 7f095ff64640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0958108da0 0x7f0958073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.773+0000 7f095ff64640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095810d520 0x7f095806cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.773+0000 7f095ff64640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09581185c0 con 0x7f095810d520 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.773+0000 7f095ff64640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0958118440 con 0x7f09581083f0 2024-09-11T23:40:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.773+0000 7f095ff64640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0958118740 con 0x7f0958108da0 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.774+0000 7f095d4ff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0958108da0 0x7f0958073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.774+0000 7f095d4ff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0958108da0 0x7f0958073ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:57992/0 (socket says 172.21.15.133:57992) 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.774+0000 7f095d4ff640 1 -- 172.21.15.133:0/485964808 learned_addr learned my addr 172.21.15.133:0/485964808 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.774+0000 7f095e501640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095810d520 0x7f095806cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.775+0000 7f095dd00640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09581083f0 0x7f09580739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.775+0000 7f095e501640 1 -- 172.21.15.133:0/485964808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0958108da0 msgr2=0x7f0958073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.775+0000 7f095e501640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0958108da0 0x7f0958073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.775+0000 7f095e501640 1 -- 172.21.15.133:0/485964808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09581083f0 msgr2=0x7f09580739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.775+0000 7f095e501640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09581083f0 0x7f09580739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.775+0000 7f095e501640 1 -- 172.21.15.133:0/485964808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095806d4d0 con 0x7f095810d520 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.776+0000 7f095e501640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095810d520 0x7f095806cc10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f094802a970 tx=0x7f0948033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:49.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.777+0000 7f094effd640 1 -- 172.21.15.133:0/485964808 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948004450 con 0x7f095810d520 2024-09-11T23:40:49.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.777+0000 7f095ff64640 1 -- 172.21.15.133:0/485964808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095806d760 con 0x7f095810d520 2024-09-11T23:40:49.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.777+0000 7f095ff64640 1 -- 172.21.15.133:0/485964808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09580716f0 con 0x7f095810d520 2024-09-11T23:40:49.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.778+0000 7f094effd640 1 -- 172.21.15.133:0/485964808 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0948084070 con 0x7f095810d520 2024-09-11T23:40:49.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.778+0000 7f094effd640 1 -- 172.21.15.133:0/485964808 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948005620 con 0x7f095810d520 2024-09-11T23:40:49.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.782+0000 7f096233d740 1 Processor -- start 2024-09-11T23:40:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.783+0000 7f094effd640 1 -- 172.21.15.133:0/485964808 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f09480305b0 con 0x7f095810d520 2024-09-11T23:40:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.784+0000 7f094effd640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09440a2e40 0x7f09440a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.784+0000 7f094effd640 1 -- 172.21.15.133:0/485964808 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0948132f40 con 0x7f095810d520 2024-09-11T23:40:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.785+0000 7f096233d740 1 -- start start 2024-09-11T23:40:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.785+0000 7f096233d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561282359b40 0x561282358090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.785+0000 7f096233d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612823585d0 0x561282358a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.785+0000 7f096233d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561282374380 0x561282376770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.785+0000 7f096233d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5612822be2f0 con 0x561282374380 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.785+0000 7f096233d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561282265cf0 con 0x561282359b40 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.785+0000 7f096233d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5612822ba2b0 con 0x5612823585d0 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.786+0000 7f095e501640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561282374380 0x561282376770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.786+0000 7f095e501640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561282374380 0x561282376770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32928/0 (socket says 172.21.15.133:32928) 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.786+0000 7f095e501640 1 -- 172.21.15.133:0/3222047067 learned_addr learned my addr 172.21.15.133:0/3222047067 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.786+0000 7f095d4ff640 1 --2- 172.21.15.133:0/3222047067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612823585d0 0x561282358a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.787+0000 7f095dd00640 1 --2- 172.21.15.133:0/3222047067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561282359b40 0x561282358090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.787+0000 7f095e501640 1 -- 172.21.15.133:0/3222047067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612823585d0 msgr2=0x561282358a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.787+0000 7f095e501640 1 --2- 172.21.15.133:0/3222047067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612823585d0 0x561282358a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.787+0000 7f095e501640 1 -- 172.21.15.133:0/3222047067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561282359b40 msgr2=0x561282358090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.787+0000 7f095e501640 1 --2- 172.21.15.133:0/3222047067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561282359b40 0x561282358090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.787+0000 7f095e501640 1 -- 172.21.15.133:0/3222047067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128224a3a0 con 0x561282374380 2024-09-11T23:40:49.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.788+0000 7f095e501640 1 --2- 172.21.15.133:0/3222047067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561282374380 0x561282376770 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0948107cc0 tx=0x7f0948005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e6b4988f1e09e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:49.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.788+0000 7f0943fff640 1 -- 172.21.15.133:0/3222047067 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09480ffc80 con 0x561282374380 2024-09-11T23:40:49.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.790+0000 7f0943fff640 1 -- 172.21.15.133:0/3222047067 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f094807b040 con 0x561282374380 2024-09-11T23:40:49.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.792+0000 7f096233d740 1 -- 172.21.15.133:0/3222047067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561282374380 msgr2=0x561282376770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:49.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.792+0000 7f096233d740 1 --2- 172.21.15.133:0/3222047067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561282374380 0x561282376770 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0948107cc0 tx=0x7f0948005f90 comp rx=0 tx=0).stop 2024-09-11T23:40:49.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.792+0000 7f096233d740 1 -- 172.21.15.133:0/3222047067 shutdown_connections 2024-09-11T23:40:49.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.792+0000 7f096233d740 1 --2- 172.21.15.133:0/3222047067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561282374380 0x561282376770 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.792+0000 7f096233d740 1 --2- 172.21.15.133:0/3222047067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612823585d0 0x561282358a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.792+0000 7f096233d740 1 --2- 172.21.15.133:0/3222047067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561282359b40 0x561282358090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.792+0000 7f096233d740 1 -- 172.21.15.133:0/3222047067 >> 172.21.15.133:0/3222047067 conn(0x561282321310 msgr2=0x561282354c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:49.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.793+0000 7f096233d740 1 -- 172.21.15.133:0/3222047067 shutdown_connections 2024-09-11T23:40:49.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.793+0000 7f096233d740 1 -- 172.21.15.133:0/3222047067 wait complete. 2024-09-11T23:40:49.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.794+0000 7f096233d740 1 -- start start 2024-09-11T23:40:49.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.794+0000 7f096233d740 1 Processor -- start 2024-09-11T23:40:49.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.794+0000 7f095d4ff640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09440a2e40 0x7f09440a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.795+0000 7f096233d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823585d0 0x561282362ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.795+0000 7f096233d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x56128235fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.795+0000 7f096233d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 0x561282360740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.795+0000 7f096233d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561282376e40 con 0x5612823585d0 2024-09-11T23:40:49.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.795+0000 7f096233d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561282279060 con 0x5612823602c0 2024-09-11T23:40:49.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.795+0000 7f096233d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561282376fc0 con 0x561282359b40 2024-09-11T23:40:49.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.795+0000 7f096233d740 10 client.0 _ll_register_callbacks cb 0x561281fcc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:49.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.796+0000 7f095e501640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 0x561282360740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.796+0000 7f095e501640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 0x561282360740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:35986/0 (socket says 172.21.15.133:35986) 2024-09-11T23:40:49.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.796+0000 7f095e501640 1 -- 172.21.15.133:0/355785477 learned_addr learned my addr 172.21.15.133:0/355785477 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:49.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.797+0000 7f095e501640 1 -- 172.21.15.133:0/355785477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 msgr2=0x56128235fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:49.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.797+0000 7f095e501640 1 --2- 172.21.15.133:0/355785477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x56128235fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.797+0000 7f095e501640 1 -- 172.21.15.133:0/355785477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823585d0 msgr2=0x561282362ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:49.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.797+0000 7f095e501640 1 --2- 172.21.15.133:0/355785477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823585d0 0x561282362ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.797+0000 7f095e501640 1 -- 172.21.15.133:0/355785477 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128202f3c0 con 0x5612823602c0 2024-09-11T23:40:49.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.798+0000 7f095e501640 1 --2- 172.21.15.133:0/355785477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 0x561282360740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09480fb190 tx=0x7f09480ff010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b272ea1a6322766 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:49.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.798+0000 7f09417fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:49.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.798+0000 7f09417fa640 1 -- 172.21.15.133:0/355785477 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09480ff9e0 con 0x5612823602c0 2024-09-11T23:40:49.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.799+0000 7f09417fa640 1 -- 172.21.15.133:0/355785477 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0948107d60 con 0x5612823602c0 2024-09-11T23:40:49.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.799+0000 7f096233d740 1 -- 172.21.15.133:0/355785477 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612822cb860 con 0x5612823602c0 2024-09-11T23:40:49.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.799+0000 7f09417fa640 1 -- 172.21.15.133:0/355785477 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f09480ffc80 con 0x5612823602c0 2024-09-11T23:40:49.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.800+0000 7f096233d740 10 client.26000 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:49.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.800+0000 7f096233d740 1 -- 172.21.15.133:0/355785477 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561282376e40 con 0x5612823602c0 2024-09-11T23:40:49.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.800+0000 7f09417fa640 1 -- 172.21.15.133:0/355785477 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5612822cb860 con 0x5612823602c0 2024-09-11T23:40:49.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.802+0000 7f09417fa640 1 -- 172.21.15.133:0/355785477 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f09480fe9d0 con 0x5612823602c0 2024-09-11T23:40:49.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.802+0000 7f096233d740 10 client.26000 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:49.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.802+0000 7f096233d740 10 client.26000 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:40:49.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.802+0000 7f096233d740 20 client.26000 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:49.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.802+0000 7f096233d740 1 --2- 172.21.15.133:0/355785477 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561282383f70 0x561282386410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.802+0000 7f096233d740 4 client.26000 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:49.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.802+0000 7f096233d740 1 -- 172.21.15.133:0/355785477 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612822c5d70 con 0x561282383f70 2024-09-11T23:40:49.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.808+0000 7f095d4ff640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09440a2e40 0x7f09440a5300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x56128235f610 tx=0x7f0950010040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:49.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.818+0000 7f095dd00640 1 --2- 172.21.15.133:0/355785477 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561282383f70 0x561282386410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.825+0000 7f095dd00640 1 --2- 172.21.15.133:0/355785477 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561282383f70 0x561282386410 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c3ae01549b23283 server_cookie=670ce344e207713f in_seq=0 out_seq=0 2024-09-11T23:40:49.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.825+0000 7f09417fa640 10 client.26000 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:40:49.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.828+0000 7f09417fa640 1 -- 172.21.15.133:0/355785477 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612822c5d70 con 0x561282383f70 2024-09-11T23:40:49.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.828+0000 7f09417fa640 10 client.26000 handle_command_reply: tid=0 2024-09-11T23:40:49.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.828+0000 7f096233d740 1 client.26000 shutdown 2024-09-11T23:40:49.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.832+0000 7f096233d740 1 -- 172.21.15.133:0/355785477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 msgr2=0x561282360740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:49.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.832+0000 7f096233d740 1 --2- 172.21.15.133:0/355785477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 0x561282360740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09480fb190 tx=0x7f09480ff010 comp rx=0 tx=0).stop 2024-09-11T23:40:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.834+0000 7f096233d740 1 -- 172.21.15.133:0/355785477 shutdown_connections 2024-09-11T23:40:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.834+0000 7f096233d740 1 --2- 172.21.15.133:0/355785477 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561282383f70 0x561282386410 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.834+0000 7f096233d740 1 --2- 172.21.15.133:0/355785477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 0x561282360740 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.834+0000 7f096233d740 1 --2- 172.21.15.133:0/355785477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x56128235fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.834+0000 7f096233d740 1 --2- 172.21.15.133:0/355785477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823585d0 0x561282362ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.834+0000 7f096233d740 1 -- 172.21.15.133:0/355785477 >> 172.21.15.133:0/355785477 conn(0x561282321310 msgr2=0x56128235a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.834+0000 7f09417fa640 0 client.26000 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:40:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.835+0000 7f096233d740 1 -- 172.21.15.133:0/355785477 shutdown_connections 2024-09-11T23:40:49.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.836+0000 7f096233d740 1 -- 172.21.15.133:0/355785477 wait complete. 2024-09-11T23:40:49.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.836+0000 7f096233d740 20 client.26000 trim_cache size 0 max 16384 2024-09-11T23:40:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.862+0000 7f096233d740 1 Processor -- start 2024-09-11T23:40:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.862+0000 7f096233d740 1 -- start start 2024-09-11T23:40:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.862+0000 7f096233d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823585d0 0x56128235b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.862+0000 7f096233d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x561282354700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.862+0000 7f096233d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823602c0 0x561282354c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f096233d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f093c00e5b0 con 0x5612823602c0 2024-09-11T23:40:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f096233d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f093c0100f0 con 0x5612823585d0 2024-09-11T23:40:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f096233d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f093c00ec80 con 0x561282359b40 2024-09-11T23:40:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095e501640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823602c0 0x561282354c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095d4ff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x561282354700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095d4ff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x561282354700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:58018/0 (socket says 172.21.15.133:58018) 2024-09-11T23:40:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095e501640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823602c0 0x561282354c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32950/0 (socket says 172.21.15.133:32950) 2024-09-11T23:40:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095d4ff640 1 -- 172.21.15.133:0/2631297246 learned_addr learned my addr 172.21.15.133:0/2631297246 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095dd00640 1 --2- 172.21.15.133:0/2631297246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823585d0 0x56128235b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095d4ff640 1 -- 172.21.15.133:0/2631297246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823585d0 msgr2=0x56128235b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095d4ff640 1 --2- 172.21.15.133:0/2631297246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823585d0 0x56128235b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095d4ff640 1 -- 172.21.15.133:0/2631297246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823602c0 msgr2=0x561282354c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095d4ff640 1 --2- 172.21.15.133:0/2631297246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823602c0 0x561282354c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095d4ff640 1 -- 172.21.15.133:0/2631297246 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561282380760 con 0x561282359b40 2024-09-11T23:40:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.863+0000 7f095d4ff640 1 --2- 172.21.15.133:0/2631297246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x561282354700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f095806e1f0 tx=0x7f0950018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba5cedc36c8ad31 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.864+0000 7f09437fe640 1 -- 172.21.15.133:0/2631297246 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950021050 con 0x561282359b40 2024-09-11T23:40:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.864+0000 7f09437fe640 1 -- 172.21.15.133:0/2631297246 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f095002b070 con 0x561282359b40 2024-09-11T23:40:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.864+0000 7f09437fe640 1 -- 172.21.15.133:0/2631297246 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950026e00 con 0x561282359b40 2024-09-11T23:40:49.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.865+0000 7f096233d740 1 -- 172.21.15.133:0/2631297246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 msgr2=0x561282354700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:49.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.865+0000 7f096233d740 1 --2- 172.21.15.133:0/2631297246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x561282354700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f095806e1f0 tx=0x7f0950018000 comp rx=0 tx=0).stop 2024-09-11T23:40:49.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.865+0000 7f096233d740 1 -- 172.21.15.133:0/2631297246 shutdown_connections 2024-09-11T23:40:49.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.865+0000 7f096233d740 1 --2- 172.21.15.133:0/2631297246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823602c0 0x561282354c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.865+0000 7f096233d740 1 --2- 172.21.15.133:0/2631297246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x561282354700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.865+0000 7f096233d740 1 --2- 172.21.15.133:0/2631297246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823585d0 0x56128235b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.865+0000 7f096233d740 1 -- 172.21.15.133:0/2631297246 >> 172.21.15.133:0/2631297246 conn(0x561282321310 msgr2=0x561282376390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:49.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.865+0000 7f096233d740 1 -- 172.21.15.133:0/2631297246 shutdown_connections 2024-09-11T23:40:49.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.865+0000 7f096233d740 1 -- 172.21.15.133:0/2631297246 wait complete. 2024-09-11T23:40:49.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.866+0000 7f096233d740 1 -- start start 2024-09-11T23:40:49.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.866+0000 7f096233d740 1 Processor -- start 2024-09-11T23:40:49.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f096233d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823585d0 0x56128237c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f096233d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x56128237c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f095dd00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823585d0 0x56128237c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f095dd00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823585d0 0x56128237c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32958/0 (socket says 172.21.15.133:32958) 2024-09-11T23:40:49.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f095dd00640 1 -- 172.21.15.133:0/713635280 learned_addr learned my addr 172.21.15.133:0/713635280 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:49.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f096233d740 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 0x56128235c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f096233d740 1 -- 172.21.15.133:0/713635280 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f093c00fa20 con 0x5612823585d0 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f096233d740 1 -- 172.21.15.133:0/713635280 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f093c00f350 con 0x5612823602c0 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f096233d740 1 -- 172.21.15.133:0/713635280 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f093c00c640 con 0x561282359b40 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f095d4ff640 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x56128237c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f095e501640 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 0x56128235c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f096233d740 10 client.0 _ll_register_callbacks cb 0x7f09500099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f095d4ff640 1 -- 172.21.15.133:0/713635280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 msgr2=0x56128235c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.867+0000 7f095d4ff640 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 0x56128235c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f095d4ff640 1 -- 172.21.15.133:0/713635280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823585d0 msgr2=0x56128237c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f095d4ff640 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823585d0 0x56128237c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f095d4ff640 1 -- 172.21.15.133:0/713635280 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128235c710 con 0x561282359b40 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f095e501640 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 0x56128235c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f096233d740 1 -- 172.21.15.133:0/713635280 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f095002b070 con 0x561282359b40 2024-09-11T23:40:49.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f095d4ff640 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x56128237c7f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5612822ba400 tx=0x7f0950027e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8efbdcfc36e574c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f09417fa640 10 client.26005 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f09417fa640 1 -- 172.21.15.133:0/713635280 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950034070 con 0x561282359b40 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f09417fa640 1 -- 172.21.15.133:0/713635280 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0950003970 con 0x561282359b40 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f09417fa640 1 -- 172.21.15.133:0/713635280 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0950003b90 con 0x561282359b40 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f096233d740 10 client.26005 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.868+0000 7f096233d740 1 -- 172.21.15.133:0/713635280 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5612823ee2e0 con 0x561282359b40 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f09417fa640 1 -- 172.21.15.133:0/713635280 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f095000b040 con 0x561282359b40 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f096233d740 10 client.26005 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f096233d740 10 client.26005 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f096233d740 20 client.26005 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f096233d740 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612823ee8a0 0x5612823f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f096233d740 4 client.26005 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:40:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f096233d740 1 -- 172.21.15.133:0/713635280 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561282242170 con 0x5612823ee8a0 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f095dd00640 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612823ee8a0 0x5612823f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f095dd00640 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612823ee8a0 0x5612823f0c90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9f9c5831248b1d server_cookie=e83ee532d17ad875 in_seq=0 out_seq=0 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f09417fa640 10 client.26005 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f09417fa640 1 -- 172.21.15.133:0/713635280 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561282242170 con 0x5612823ee8a0 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f09417fa640 10 client.26005 handle_command_reply: tid=0 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f096233d740 1 client.26005 shutdown 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f096233d740 1 -- 172.21.15.133:0/713635280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 msgr2=0x56128237c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.869+0000 7f096233d740 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x56128237c7f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5612822ba400 tx=0x7f0950027e40 comp rx=0 tx=0).stop 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.870+0000 7f096233d740 1 -- 172.21.15.133:0/713635280 shutdown_connections 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.870+0000 7f096233d740 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612823ee8a0 0x5612823f0c90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.870+0000 7f096233d740 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612823602c0 0x56128235c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.870+0000 7f096233d740 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561282359b40 0x56128237c7f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.870+0000 7f096233d740 1 --2- 172.21.15.133:0/713635280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612823585d0 0x56128237c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.870+0000 7f096233d740 1 -- 172.21.15.133:0/713635280 >> 172.21.15.133:0/713635280 conn(0x561282321310 msgr2=0x561282354700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:49.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.870+0000 7f09417fa640 0 client.26005 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:40:49.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.870+0000 7f096233d740 1 -- 172.21.15.133:0/713635280 shutdown_connections 2024-09-11T23:40:49.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.870+0000 7f096233d740 1 -- 172.21.15.133:0/713635280 wait complete. 2024-09-11T23:40:49.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.870+0000 7f096233d740 20 client.26005 trim_cache size 0 max 16384 2024-09-11T23:40:49.874 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.871+0000 7f095ff64640 1 -- 172.21.15.133:0/485964808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09440a2e40 msgr2=0x7f09440a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.871+0000 7f095ff64640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09440a2e40 0x7f09440a5300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x56128235f610 tx=0x7f0950010040 comp rx=0 tx=0).stop 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.871+0000 7f095ff64640 1 -- 172.21.15.133:0/485964808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095810d520 msgr2=0x7f095806cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.871+0000 7f095ff64640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095810d520 0x7f095806cc10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f094802a970 tx=0x7f0948033d10 comp rx=0 tx=0).stop 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.871+0000 7f095ff64640 1 -- 172.21.15.133:0/485964808 shutdown_connections 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.872+0000 7f095ff64640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09440a2e40 0x7f09440a5300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.872+0000 7f095ff64640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095810d520 0x7f095806cc10 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.872+0000 7f095ff64640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0958108da0 0x7f0958073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.872+0000 7f095ff64640 1 --2- 172.21.15.133:0/485964808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09581083f0 0x7f09580739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.872+0000 7f095ff64640 1 -- 172.21.15.133:0/485964808 >> 172.21.15.133:0/485964808 conn(0x7f0958077af0 msgr2=0x7f0958078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.873+0000 7f095ff64640 1 -- 172.21.15.133:0/485964808 shutdown_connections 2024-09-11T23:40:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:49.873+0000 7f095ff64640 1 -- 172.21.15.133:0/485964808 wait complete. 2024-09-11T23:40:49.879 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:50.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:50 smithi178 ceph-mon[35882]: pgmap v124: 97 pgs: 97 active+clean; 30 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 858 MiB/s wr, 1.33k op/s 2024-09-11T23:40:50.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:50 smithi138 ceph-mon[35142]: pgmap v124: 97 pgs: 97 active+clean; 30 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 858 MiB/s wr, 1.33k op/s 2024-09-11T23:40:50.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:50 smithi133 ceph-mon[31430]: pgmap v124: 97 pgs: 97 active+clean; 30 GiB data, 125 GiB used, 948 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 858 MiB/s wr, 1.33k op/s 2024-09-11T23:40:51.396 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:40:51.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:51 smithi178 ceph-mon[35882]: pgmap v125: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 695 MiB/s wr, 875 op/s 2024-09-11T23:40:51.822 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:51 smithi133 ceph-mon[31430]: pgmap v125: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 695 MiB/s wr, 875 op/s 2024-09-11T23:40:51.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:51 smithi138 ceph-mon[35142]: pgmap v125: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 695 MiB/s wr, 875 op/s 2024-09-11T23:40:52.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.230+0000 7f6b91f77640 1 -- 172.21.15.133:0/3714897750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c110190 msgr2=0x7f6b8c06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:52.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.230+0000 7f6b91f77640 1 --2- 172.21.15.133:0/3714897750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c110190 0x7f6b8c06e4f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009900 tx=0x7f6b7c02a440 comp rx=0 tx=0).stop 2024-09-11T23:40:52.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.231+0000 7f6b91f77640 1 -- 172.21.15.133:0/3714897750 shutdown_connections 2024-09-11T23:40:52.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.231+0000 7f6b91f77640 1 --2- 172.21.15.133:0/3714897750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b8c11b140 0x7f6b8c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:52.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.231+0000 7f6b91f77640 1 --2- 172.21.15.133:0/3714897750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b8c06ea30 0x7f6b8c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:52.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.231+0000 7f6b91f77640 1 --2- 172.21.15.133:0/3714897750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c110190 0x7f6b8c06e4f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:52.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.232+0000 7f6b91f77640 1 -- 172.21.15.133:0/3714897750 >> 172.21.15.133:0/3714897750 conn(0x7f6b8c073180 msgr2=0x7f6b8c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:52.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.232+0000 7f6b91f77640 1 -- 172.21.15.133:0/3714897750 shutdown_connections 2024-09-11T23:40:52.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.232+0000 7f6b91f77640 1 -- 172.21.15.133:0/3714897750 wait complete. 2024-09-11T23:40:52.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.233+0000 7f6b91f77640 1 Processor -- start 2024-09-11T23:40:52.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.233+0000 7f6b91f77640 1 -- start start 2024-09-11T23:40:52.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.234+0000 7f6b91f77640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c06ea30 0x7f6b8c1a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:52.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.234+0000 7f6b91f77640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b8c110190 0x7f6b8c1a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:52.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.234+0000 7f6b91f77640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b8c11b140 0x7f6b8c1ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:52.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.234+0000 7f6b91f77640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c120c40 con 0x7f6b8c11b140 2024-09-11T23:40:52.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.234+0000 7f6b91f77640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c120ac0 con 0x7f6b8c110190 2024-09-11T23:40:52.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.234+0000 7f6b91f77640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c120dc0 con 0x7f6b8c06ea30 2024-09-11T23:40:52.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.236+0000 7f6b8bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b8c110190 0x7f6b8c1a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:52.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.237+0000 7f6b8bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b8c110190 0x7f6b8c1a7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42342/0 (socket says 172.21.15.133:42342) 2024-09-11T23:40:52.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.237+0000 7f6b8bfff640 1 -- 172.21.15.133:0/4161568942 learned_addr learned my addr 172.21.15.133:0/4161568942 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:52.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.239+0000 7f6b90f75640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c06ea30 0x7f6b8c1a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:52.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.241+0000 7f6b91776640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b8c11b140 0x7f6b8c1ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:52.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.241+0000 7f6b8bfff640 1 -- 172.21.15.133:0/4161568942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c06ea30 msgr2=0x7f6b8c1a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:52.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.242+0000 7f6b8bfff640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c06ea30 0x7f6b8c1a70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:52.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.242+0000 7f6b8bfff640 1 -- 172.21.15.133:0/4161568942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b8c11b140 msgr2=0x7f6b8c1ab9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:52.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.242+0000 7f6b8bfff640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b8c11b140 0x7f6b8c1ab9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:52.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.242+0000 7f6b8bfff640 1 -- 172.21.15.133:0/4161568942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c1ac140 con 0x7f6b8c110190 2024-09-11T23:40:52.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.252+0000 7f6b8bfff640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b8c110190 0x7f6b8c1a7630 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400c890 tx=0x7f6b7400cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:52.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.253+0000 7f6b89ffb640 1 -- 172.21.15.133:0/4161568942 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74004220 con 0x7f6b8c110190 2024-09-11T23:40:52.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.253+0000 7f6b91f77640 1 -- 172.21.15.133:0/4161568942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c1ac430 con 0x7f6b8c110190 2024-09-11T23:40:52.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.254+0000 7f6b91f77640 1 -- 172.21.15.133:0/4161568942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c1b3c70 con 0x7f6b8c110190 2024-09-11T23:40:52.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.256+0000 7f6b89ffb640 1 -- 172.21.15.133:0/4161568942 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b740043c0 con 0x7f6b8c110190 2024-09-11T23:40:52.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.256+0000 7f6b89ffb640 1 -- 172.21.15.133:0/4161568942 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b740049e0 con 0x7f6b8c110190 2024-09-11T23:40:52.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.262+0000 7f6b89ffb640 1 -- 172.21.15.133:0/4161568942 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6b74004c60 con 0x7f6b8c110190 2024-09-11T23:40:52.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.262+0000 7f6b91f77640 1 -- 172.21.15.133:0/4161568942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c005290 con 0x7f6b8c110190 2024-09-11T23:40:52.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.263+0000 7f6b89ffb640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b780a2f00 0x7f6b780a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:52.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.264+0000 7f6b89ffb640 1 -- 172.21.15.133:0/4161568942 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6b740cef50 con 0x7f6b8c110190 2024-09-11T23:40:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.269+0000 7f6b90f75640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b780a2f00 0x7f6b780a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.271+0000 7f6b90f75640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b780a2f00 0x7f6b780a53c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c033040 tx=0x7f6b7c000bc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.273+0000 7f6b89ffb640 1 -- 172.21.15.133:0/4161568942 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b74014030 con 0x7f6b8c110190 2024-09-11T23:40:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:52.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.830+0000 7f6b91f77640 1 -- 172.21.15.133:0/4161568942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b6c005850 con 0x7f6b8c110190 2024-09-11T23:40:52.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.834+0000 7f6b89ffb640 1 -- 172.21.15.133:0/4161568942 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f6b7409bb80 con 0x7f6b8c110190 2024-09-11T23:40:52.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:40:52.837 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:40:52.838 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:40:52.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.841+0000 7f6b91f77640 1 -- 172.21.15.133:0/4161568942 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b780a2f00 msgr2=0x7f6b780a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:52.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.841+0000 7f6b91f77640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b780a2f00 0x7f6b780a53c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c033040 tx=0x7f6b7c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:40:52.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.841+0000 7f6b91f77640 1 -- 172.21.15.133:0/4161568942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b8c110190 msgr2=0x7f6b8c1a7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:52.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.841+0000 7f6b91f77640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b8c110190 0x7f6b8c1a7630 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400c890 tx=0x7f6b7400cd50 comp rx=0 tx=0).stop 2024-09-11T23:40:52.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.841+0000 7f6b91f77640 1 -- 172.21.15.133:0/4161568942 shutdown_connections 2024-09-11T23:40:52.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.841+0000 7f6b91f77640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6b780a2f00 0x7f6b780a53c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:52.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.841+0000 7f6b91f77640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b8c11b140 0x7f6b8c1ab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:52.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.841+0000 7f6b91f77640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b8c110190 0x7f6b8c1a7630 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:52.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.841+0000 7f6b91f77640 1 --2- 172.21.15.133:0/4161568942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b8c06ea30 0x7f6b8c1a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:52.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.841+0000 7f6b91f77640 1 -- 172.21.15.133:0/4161568942 >> 172.21.15.133:0/4161568942 conn(0x7f6b8c073180 msgr2=0x7f6b8c1193c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:52.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.843+0000 7f6b91f77640 1 -- 172.21.15.133:0/4161568942 shutdown_connections 2024-09-11T23:40:52.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:52.843+0000 7f6b91f77640 1 -- 172.21.15.133:0/4161568942 wait complete. 2024-09-11T23:40:53.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:52 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/4161568942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:53.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:52 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/4161568942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:53.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:52 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/4161568942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:40:53.889 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T23:40:53.889 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T23:40:54.245 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:53 smithi133 ceph-mon[31430]: pgmap v126: 97 pgs: 97 active+clean; 30 GiB data, 131 GiB used, 942 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 803 MiB/s wr, 1.31k op/s 2024-09-11T23:40:54.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:53 smithi178 ceph-mon[35882]: pgmap v126: 97 pgs: 97 active+clean; 30 GiB data, 131 GiB used, 942 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 803 MiB/s wr, 1.31k op/s 2024-09-11T23:40:54.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:53 smithi138 ceph-mon[35142]: pgmap v126: 97 pgs: 97 active+clean; 30 GiB data, 131 GiB used, 942 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 803 MiB/s wr, 1.31k op/s 2024-09-11T23:40:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.714+0000 7f964869b640 1 -- 172.21.15.133:0/1338246920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f964011b140 msgr2=0x7f964011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.714+0000 7f964869b640 1 --2- 172.21.15.133:0/1338246920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f964011b140 0x7f964011d530 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9638009ca0 tx=0x7f963802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.718+0000 7f964869b640 1 -- 172.21.15.133:0/1338246920 shutdown_connections 2024-09-11T23:40:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.718+0000 7f964869b640 1 --2- 172.21.15.133:0/1338246920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f964011b140 0x7f964011d530 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.718+0000 7f964869b640 1 --2- 172.21.15.133:0/1338246920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9640109830 0x7f9640109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.718+0000 7f964869b640 1 --2- 172.21.15.133:0/1338246920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9640109450 0x7f9640110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.719+0000 7f964869b640 1 -- 172.21.15.133:0/1338246920 >> 172.21.15.133:0/1338246920 conn(0x7f964006e140 msgr2=0x7f964006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:54.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.723+0000 7f964869b640 1 -- 172.21.15.133:0/1338246920 shutdown_connections 2024-09-11T23:40:54.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.729+0000 7f964869b640 1 -- 172.21.15.133:0/1338246920 wait complete. 2024-09-11T23:40:54.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.730+0000 7f964869b640 1 Processor -- start 2024-09-11T23:40:54.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.730+0000 7f964869b640 1 -- start start 2024-09-11T23:40:54.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.731+0000 7f964869b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9640109450 0x7f964011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.731+0000 7f964869b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9640109830 0x7f964011abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.731+0000 7f964869b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f964011b140 0x7f96401138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.731+0000 7f964869b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9640120da0 con 0x7f9640109830 2024-09-11T23:40:54.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.731+0000 7f964869b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9640120c20 con 0x7f9640109450 2024-09-11T23:40:54.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.731+0000 7f964869b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9640120f20 con 0x7f964011b140 2024-09-11T23:40:54.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.732+0000 7f9645c36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9640109830 0x7f964011abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:54.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.732+0000 7f9645c36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9640109830 0x7f964011abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47968/0 (socket says 172.21.15.133:47968) 2024-09-11T23:40:54.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.732+0000 7f9645c36640 1 -- 172.21.15.133:0/745479213 learned_addr learned my addr 172.21.15.133:0/745479213 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:54.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.733+0000 7f9645c36640 1 -- 172.21.15.133:0/745479213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f964011b140 msgr2=0x7f96401138f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:40:54.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.733+0000 7f9645c36640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f964011b140 0x7f96401138f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.733+0000 7f9645c36640 1 -- 172.21.15.133:0/745479213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9640109450 msgr2=0x7f964011a680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:40:54.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.733+0000 7f9645c36640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9640109450 0x7f964011a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.733+0000 7f9645c36640 1 -- 172.21.15.133:0/745479213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96401141b0 con 0x7f9640109830 2024-09-11T23:40:54.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.741+0000 7f9645c36640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9640109830 0x7f964011abc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f963000d950 tx=0x7f963000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:54.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.745+0000 7f96377fe640 1 -- 172.21.15.133:0/745479213 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630017070 con 0x7f9640109830 2024-09-11T23:40:54.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.746+0000 7f964869b640 1 -- 172.21.15.133:0/745479213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96401144a0 con 0x7f9640109830 2024-09-11T23:40:54.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.746+0000 7f964869b640 1 -- 172.21.15.133:0/745479213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9640118440 con 0x7f9640109830 2024-09-11T23:40:54.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.747+0000 7f96377fe640 1 -- 172.21.15.133:0/745479213 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9630009c30 con 0x7f9640109830 2024-09-11T23:40:54.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.747+0000 7f96377fe640 1 -- 172.21.15.133:0/745479213 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963000f620 con 0x7f9640109830 2024-09-11T23:40:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.753+0000 7f964aa74740 1 Processor -- start 2024-09-11T23:40:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.754+0000 7f96377fe640 1 -- 172.21.15.133:0/745479213 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f963000f7c0 con 0x7f9640109830 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.760+0000 7f964aa74740 1 -- start start 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.760+0000 7f964aa74740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65aa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.761+0000 7f964aa74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 0x5610c65aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.761+0000 7f964aa74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65b5370 0x5610c65b7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.761+0000 7f964aa74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5610c65102f0 con 0x5610c65b5370 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.761+0000 7f964aa74740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610c64b7cf0 con 0x5610c65aa5d0 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.761+0000 7f964aa74740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5610c650c2b0 con 0x5610c65abb40 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.761+0000 7f9646c38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.761+0000 7f9646c38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65aa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33046/0 (socket says 172.21.15.133:33046) 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.761+0000 7f9646c38640 1 -- 172.21.15.133:0/31494632 learned_addr learned my addr 172.21.15.133:0/31494632 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.761+0000 7f9646437640 1 --2- 172.21.15.133:0/31494632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 0x5610c65aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.762+0000 7f9646c38640 1 -- 172.21.15.133:0/31494632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 msgr2=0x5610c65aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.762+0000 7f9646c38640 1 --2- 172.21.15.133:0/31494632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 0x5610c65aaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.762+0000 7f9645c36640 1 --2- 172.21.15.133:0/31494632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65b5370 0x5610c65b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.762+0000 7f9646c38640 1 -- 172.21.15.133:0/31494632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65b5370 msgr2=0x5610c65b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.763+0000 7f9646c38640 1 --2- 172.21.15.133:0/31494632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65b5370 0x5610c65b7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.763+0000 7f9646c38640 1 -- 172.21.15.133:0/31494632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c649c3a0 con 0x5610c65abb40 2024-09-11T23:40:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.763+0000 7f96377fe640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f962c0a2e80 0x7f962c0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.763+0000 7f9646c38640 1 --2- 172.21.15.133:0/31494632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65aa090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f963802ae60 tx=0x7f963802f3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810383ca17780e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.764+0000 7f9634ff9640 1 -- 172.21.15.133:0/31494632 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638031aa0 con 0x5610c65abb40 2024-09-11T23:40:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.764+0000 7f9645c36640 1 --2- 172.21.15.133:0/31494632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65b5370 0x5610c65b7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:40:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.764+0000 7f9634ff9640 1 -- 172.21.15.133:0/31494632 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9638036040 con 0x5610c65abb40 2024-09-11T23:40:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.764+0000 7f9634ff9640 1 -- 172.21.15.133:0/31494632 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963802fd80 con 0x5610c65abb40 2024-09-11T23:40:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.765+0000 7f96377fe640 1 -- 172.21.15.133:0/745479213 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f96300cef80 con 0x7f9640109830 2024-09-11T23:40:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.767+0000 7f9646437640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f962c0a2e80 0x7f962c0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.768+0000 7f964aa74740 1 -- 172.21.15.133:0/31494632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 msgr2=0x5610c65aa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.768+0000 7f964aa74740 1 --2- 172.21.15.133:0/31494632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65aa090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f963802ae60 tx=0x7f963802f3d0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.774+0000 7f9646437640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f962c0a2e80 0x7f962c0a5340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x5610c65a7ac0 tx=0x7f963c005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:54.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.775+0000 7f964aa74740 1 -- 172.21.15.133:0/31494632 shutdown_connections 2024-09-11T23:40:54.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.775+0000 7f964aa74740 1 --2- 172.21.15.133:0/31494632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65b5370 0x5610c65b7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.775+0000 7f964aa74740 1 --2- 172.21.15.133:0/31494632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 0x5610c65aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.775+0000 7f964aa74740 1 --2- 172.21.15.133:0/31494632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65aa090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.775+0000 7f964aa74740 1 -- 172.21.15.133:0/31494632 >> 172.21.15.133:0/31494632 conn(0x5610c6573310 msgr2=0x5610c65a6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:54.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.776+0000 7f964aa74740 1 -- 172.21.15.133:0/31494632 shutdown_connections 2024-09-11T23:40:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.780+0000 7f964aa74740 1 -- 172.21.15.133:0/31494632 wait complete. 2024-09-11T23:40:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.780+0000 7f964aa74740 1 -- start start 2024-09-11T23:40:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.781+0000 7f964aa74740 1 Processor -- start 2024-09-11T23:40:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.782+0000 7f964aa74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65aa5d0 0x5610c65b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.782+0000 7f964aa74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65abb40 0x5610c65b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.782+0000 7f964aa74740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65b5370 0x5610c65b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.782+0000 7f964aa74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5610c65b8010 con 0x5610c65aa5d0 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.782+0000 7f964aa74740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610c64cb060 con 0x5610c65abb40 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.782+0000 7f964aa74740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5610c65b8190 con 0x5610c65b5370 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.782+0000 7f9646437640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65aa5d0 0x5610c65b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.782+0000 7f9646437640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65aa5d0 0x5610c65b4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47988/0 (socket says 172.21.15.133:47988) 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.783+0000 7f9646437640 1 -- 172.21.15.133:0/1342631651 learned_addr learned my addr 172.21.15.133:0/1342631651 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.783+0000 7f964aa74740 10 client.0 _ll_register_callbacks cb 0x5610c621e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.783+0000 7f9646437640 1 -- 172.21.15.133:0/1342631651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65b5370 msgr2=0x5610c65b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.783+0000 7f9646437640 1 --2- 172.21.15.133:0/1342631651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65b5370 0x5610c65b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.783+0000 7f9646437640 1 -- 172.21.15.133:0/1342631651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65abb40 msgr2=0x5610c65b2020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.783+0000 7f9646437640 1 --2- 172.21.15.133:0/1342631651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65abb40 0x5610c65b2020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.783+0000 7f9646437640 1 -- 172.21.15.133:0/1342631651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c62813c0 con 0x5610c65aa5d0 2024-09-11T23:40:54.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.783+0000 7f9646437640 1 --2- 172.21.15.133:0/1342631651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65aa5d0 0x5610c65b4d70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f963c0045d0 tx=0x7f963c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18627ec8f359dbc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:54.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.784+0000 7f9629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:40:54.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.787+0000 7f9629ffb640 1 -- 172.21.15.133:0/1342631651 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c01e830 con 0x5610c65aa5d0 2024-09-11T23:40:54.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.787+0000 7f964aa74740 1 -- 172.21.15.133:0/1342631651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9638036040 con 0x5610c65aa5d0 2024-09-11T23:40:54.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.787+0000 7f9629ffb640 1 -- 172.21.15.133:0/1342631651 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f963c01e9d0 con 0x5610c65aa5d0 2024-09-11T23:40:54.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.793+0000 7f9629ffb640 1 -- 172.21.15.133:0/1342631651 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f963c01ebd0 con 0x5610c65aa5d0 2024-09-11T23:40:54.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.793+0000 7f964aa74740 10 client.16581 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:54.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.793+0000 7f964aa74740 1 -- 172.21.15.133:0/1342631651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610c65c3ac0 con 0x5610c65aa5d0 2024-09-11T23:40:54.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.795+0000 7f9629ffb640 1 -- 172.21.15.133:0/1342631651 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f963c01ed70 con 0x5610c65aa5d0 2024-09-11T23:40:54.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.795+0000 7f9629ffb640 1 -- 172.21.15.133:0/1342631651 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f963c02e050 con 0x5610c65aa5d0 2024-09-11T23:40:54.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.797+0000 7f964aa74740 10 client.16581 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:54.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.797+0000 7f964aa74740 10 client.16581 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:40:54.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.798+0000 7f964aa74740 20 client.16581 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:54.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.798+0000 7f964aa74740 1 --2- 172.21.15.133:0/1342631651 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5610c65c5320 0x5610c65c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.798+0000 7f964aa74740 4 client.16581 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:54.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.798+0000 7f964aa74740 1 -- 172.21.15.133:0/1342631651 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9638036040 con 0x5610c65c5320 2024-09-11T23:40:54.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.805+0000 7f9645c36640 1 --2- 172.21.15.133:0/1342631651 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5610c65c5320 0x5610c65c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:54.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.812+0000 7f9629ffb640 10 client.16581 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:40:54.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.812+0000 7f9645c36640 1 --2- 172.21.15.133:0/1342631651 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5610c65c5320 0x5610c65c77c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec5e4d6114dec5a1 server_cookie=a07b298669dc4489 in_seq=0 out_seq=0 2024-09-11T23:40:54.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.813+0000 7f9629ffb640 1 -- 172.21.15.133:0/1342631651 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9638036040 con 0x5610c65c5320 2024-09-11T23:40:54.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.813+0000 7f9629ffb640 10 client.16581 handle_command_reply: tid=0 2024-09-11T23:40:54.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.813+0000 7f964aa74740 1 client.16581 shutdown 2024-09-11T23:40:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.813+0000 7f964aa74740 1 -- 172.21.15.133:0/1342631651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65aa5d0 msgr2=0x5610c65b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.813+0000 7f964aa74740 1 --2- 172.21.15.133:0/1342631651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65aa5d0 0x5610c65b4d70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f963c0045d0 tx=0x7f963c004730 comp rx=0 tx=0).stop 2024-09-11T23:40:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.815+0000 7f964aa74740 1 -- 172.21.15.133:0/1342631651 shutdown_connections 2024-09-11T23:40:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.815+0000 7f964aa74740 1 --2- 172.21.15.133:0/1342631651 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5610c65c5320 0x5610c65c77c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.815+0000 7f964aa74740 1 --2- 172.21.15.133:0/1342631651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65b5370 0x5610c65b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.815+0000 7f9629ffb640 0 client.16581 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:40:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.815+0000 7f964aa74740 1 --2- 172.21.15.133:0/1342631651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65abb40 0x5610c65b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.816+0000 7f964aa74740 1 --2- 172.21.15.133:0/1342631651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65aa5d0 0x5610c65b4d70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.816+0000 7f964aa74740 1 -- 172.21.15.133:0/1342631651 >> 172.21.15.133:0/1342631651 conn(0x5610c6573310 msgr2=0x5610c65ac930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:54.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.816+0000 7f964aa74740 1 -- 172.21.15.133:0/1342631651 shutdown_connections 2024-09-11T23:40:54.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.816+0000 7f964aa74740 1 -- 172.21.15.133:0/1342631651 wait complete. 2024-09-11T23:40:54.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.816+0000 7f964aa74740 20 client.16581 trim_cache size 0 max 16384 2024-09-11T23:40:54.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.851+0000 7f964aa74740 1 Processor -- start 2024-09-11T23:40:54.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.852+0000 7f964aa74740 1 -- start start 2024-09-11T23:40:54.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.852+0000 7f964aa74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65aa5d0 0x5610c65a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.852+0000 7f964aa74740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65a6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.852+0000 7f964aa74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65b5370 0x5610c65ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.852+0000 7f964aa74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f962400e6d0 con 0x5610c65aa5d0 2024-09-11T23:40:54.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.852+0000 7f964aa74740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9624010210 con 0x5610c65b5370 2024-09-11T23:40:54.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.852+0000 7f964aa74740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f962400eda0 con 0x5610c65abb40 2024-09-11T23:40:54.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.853+0000 7f9646c38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65b5370 0x5610c65ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:54.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.853+0000 7f9646c38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65b5370 0x5610c65ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42378/0 (socket says 172.21.15.133:42378) 2024-09-11T23:40:54.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.853+0000 7f9646c38640 1 -- 172.21.15.133:0/3661395451 learned_addr learned my addr 172.21.15.133:0/3661395451 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:54.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.854+0000 7f9646c38640 1 -- 172.21.15.133:0/3661395451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 msgr2=0x5610c65a6130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:40:54.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.854+0000 7f9646c38640 1 --2- 172.21.15.133:0/3661395451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.854+0000 7f9646c38640 1 -- 172.21.15.133:0/3661395451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65aa5d0 msgr2=0x5610c65a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:54.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.854+0000 7f9646c38640 1 --2- 172.21.15.133:0/3661395451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65aa5d0 0x5610c65a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.854+0000 7f9646c38640 1 -- 172.21.15.133:0/3661395451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c65b7ad0 con 0x5610c65b5370 2024-09-11T23:40:54.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.855+0000 7f9646c38640 1 --2- 172.21.15.133:0/3661395451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65b5370 0x5610c65ad120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f96380093a0 tx=0x7f96380093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbc1dc7939df5987 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:54.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.856+0000 7f962bfff640 1 -- 172.21.15.133:0/3661395451 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638031aa0 con 0x5610c65b5370 2024-09-11T23:40:54.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.856+0000 7f962bfff640 1 -- 172.21.15.133:0/3661395451 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9638049070 con 0x5610c65b5370 2024-09-11T23:40:54.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.858+0000 7f964aa74740 1 -- 172.21.15.133:0/3661395451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65b5370 msgr2=0x5610c65ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:54.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.858+0000 7f964aa74740 1 --2- 172.21.15.133:0/3661395451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65b5370 0x5610c65ad120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f96380093a0 tx=0x7f96380093d0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.858+0000 7f964aa74740 1 -- 172.21.15.133:0/3661395451 shutdown_connections 2024-09-11T23:40:54.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.858+0000 7f964aa74740 1 --2- 172.21.15.133:0/3661395451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65b5370 0x5610c65ad120 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.858+0000 7f964aa74740 1 --2- 172.21.15.133:0/3661395451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.858+0000 7f964aa74740 1 --2- 172.21.15.133:0/3661395451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65aa5d0 0x5610c65a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.858+0000 7f964aa74740 1 -- 172.21.15.133:0/3661395451 >> 172.21.15.133:0/3661395451 conn(0x5610c6573310 msgr2=0x5610c65a54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:54.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.858+0000 7f964aa74740 1 -- 172.21.15.133:0/3661395451 shutdown_connections 2024-09-11T23:40:54.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.859+0000 7f964aa74740 1 -- 172.21.15.133:0/3661395451 wait complete. 2024-09-11T23:40:54.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.859+0000 7f964aa74740 1 -- start start 2024-09-11T23:40:54.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.859+0000 7f964aa74740 1 Processor -- start 2024-09-11T23:40:54.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.860+0000 7f964aa74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 0x5610c65bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.860+0000 7f964aa74740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.860+0000 7f964aa74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65b9290 0x5610c65b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.860+0000 7f964aa74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f962400fb40 con 0x5610c65b9290 2024-09-11T23:40:54.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.860+0000 7f964aa74740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f962400f470 con 0x5610c65aa5d0 2024-09-11T23:40:54.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.860+0000 7f964aa74740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f962400c6b0 con 0x5610c65abb40 2024-09-11T23:40:54.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.861+0000 7f964aa74740 10 client.0 _ll_register_callbacks cb 0x7f963c011bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:54.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.862+0000 7f9646437640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 0x5610c65bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:54.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.862+0000 7f9646437640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 0x5610c65bdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42380/0 (socket says 172.21.15.133:42380) 2024-09-11T23:40:54.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.862+0000 7f9646437640 1 -- 172.21.15.133:0/3126192510 learned_addr learned my addr 172.21.15.133:0/3126192510 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:54.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.862+0000 7f9645c36640 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:54.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.862+0000 7f9646437640 1 -- 172.21.15.133:0/3126192510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 msgr2=0x5610c65b8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:54.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.862+0000 7f9646437640 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65b8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.863+0000 7f9646437640 1 -- 172.21.15.133:0/3126192510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65b9290 msgr2=0x5610c65b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:54.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.863+0000 7f9646437640 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65b9290 0x5610c65b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.863+0000 7f9646437640 1 -- 172.21.15.133:0/3126192510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c662f0d0 con 0x5610c65aa5d0 2024-09-11T23:40:54.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.868+0000 7f9646437640 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 0x5610c65bdac0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f963c0118d0 tx=0x7f963c01e860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4fa7e6e7f1dde45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:54.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.868+0000 7f9629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:54.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.868+0000 7f9629ffb640 1 -- 172.21.15.133:0/3126192510 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c030670 con 0x5610c65aa5d0 2024-09-11T23:40:54.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.868+0000 7f964aa74740 1 -- 172.21.15.133:0/3126192510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5610c651d860 con 0x5610c65aa5d0 2024-09-11T23:40:54.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.869+0000 7f9629ffb640 1 -- 172.21.15.133:0/3126192510 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f963c030810 con 0x5610c65aa5d0 2024-09-11T23:40:54.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.869+0000 7f9629ffb640 1 -- 172.21.15.133:0/3126192510 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f963c005850 con 0x5610c65aa5d0 2024-09-11T23:40:54.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.869+0000 7f964aa74740 10 client.26021 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:54.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.869+0000 7f964aa74740 1 -- 172.21.15.133:0/3126192510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610c66303e0 con 0x5610c65aa5d0 2024-09-11T23:40:54.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.870+0000 7f9629ffb640 1 -- 172.21.15.133:0/3126192510 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5610c651d860 con 0x5610c65aa5d0 2024-09-11T23:40:54.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.870+0000 7f9629ffb640 1 -- 172.21.15.133:0/3126192510 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f963c038020 con 0x5610c65aa5d0 2024-09-11T23:40:54.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.873+0000 7f964aa74740 10 client.26021 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:54.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.873+0000 7f964aa74740 10 client.26021 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:40:54.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.873+0000 7f964aa74740 20 client.26021 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:54.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.873+0000 7f964aa74740 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5610c6630b50 0x5610c6632f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:54.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.873+0000 7f964aa74740 4 client.26021 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:40:54.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.873+0000 7f964aa74740 1 -- 172.21.15.133:0/3126192510 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610c6517d70 con 0x5610c6630b50 2024-09-11T23:40:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.874+0000 7f9645c36640 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5610c6630b50 0x5610c6632f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.874+0000 7f9645c36640 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5610c6630b50 0x5610c6632f90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aecfe1ed68b697a8 server_cookie=f33795e5bfc9d21a in_seq=0 out_seq=0 2024-09-11T23:40:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.874+0000 7f9629ffb640 10 client.26021 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:40:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.875+0000 7f9629ffb640 1 -- 172.21.15.133:0/3126192510 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610c6517d70 con 0x5610c6630b50 2024-09-11T23:40:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.875+0000 7f9629ffb640 10 client.26021 handle_command_reply: tid=0 2024-09-11T23:40:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.877+0000 7f964aa74740 1 client.26021 shutdown 2024-09-11T23:40:54.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f964aa74740 1 -- 172.21.15.133:0/3126192510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 msgr2=0x5610c65bdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:54.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f964aa74740 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 0x5610c65bdac0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f963c0118d0 tx=0x7f963c01e860 comp rx=0 tx=0).stop 2024-09-11T23:40:54.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f964aa74740 1 -- 172.21.15.133:0/3126192510 shutdown_connections 2024-09-11T23:40:54.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f964aa74740 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5610c6630b50 0x5610c6632f90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f964aa74740 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610c65b9290 0x5610c65b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f964aa74740 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610c65abb40 0x5610c65b8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f964aa74740 1 --2- 172.21.15.133:0/3126192510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610c65aa5d0 0x5610c65bdac0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f964aa74740 1 -- 172.21.15.133:0/3126192510 >> 172.21.15.133:0/3126192510 conn(0x5610c6573310 msgr2=0x5610c65a5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f9629ffb640 0 client.26021 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:40:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f964aa74740 1 -- 172.21.15.133:0/3126192510 shutdown_connections 2024-09-11T23:40:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f964aa74740 1 -- 172.21.15.133:0/3126192510 wait complete. 2024-09-11T23:40:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.882+0000 7f964aa74740 20 client.26021 trim_cache size 0 max 16384 2024-09-11T23:40:54.884 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:40:54.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.890+0000 7f964869b640 1 -- 172.21.15.133:0/745479213 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f962c0a2e80 msgr2=0x7f962c0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:54.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.890+0000 7f964869b640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f962c0a2e80 0x7f962c0a5340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x5610c65a7ac0 tx=0x7f963c005f90 comp rx=0 tx=0).stop 2024-09-11T23:40:54.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.890+0000 7f964869b640 1 -- 172.21.15.133:0/745479213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9640109830 msgr2=0x7f964011abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:54.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.890+0000 7f964869b640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9640109830 0x7f964011abc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f963000d950 tx=0x7f963000de10 comp rx=0 tx=0).stop 2024-09-11T23:40:54.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.890+0000 7f964869b640 1 -- 172.21.15.133:0/745479213 shutdown_connections 2024-09-11T23:40:54.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.890+0000 7f964869b640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f962c0a2e80 0x7f962c0a5340 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.890+0000 7f964869b640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f964011b140 0x7f96401138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.890+0000 7f964869b640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9640109830 0x7f964011abc0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.890+0000 7f964869b640 1 --2- 172.21.15.133:0/745479213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9640109450 0x7f964011a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:54.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.890+0000 7f964869b640 1 -- 172.21.15.133:0/745479213 >> 172.21.15.133:0/745479213 conn(0x7f964006e140 msgr2=0x7f9640070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:54.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.891+0000 7f964869b640 1 -- 172.21.15.133:0/745479213 shutdown_connections 2024-09-11T23:40:54.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:54.891+0000 7f964869b640 1 -- 172.21.15.133:0/745479213 wait complete. 2024-09-11T23:40:55.406 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:55.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:55 smithi138 ceph-mon[35142]: pgmap v127: 97 pgs: 97 active+clean; 30 GiB data, 134 GiB used, 939 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 690 MiB/s wr, 1.26k op/s 2024-09-11T23:40:55.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:55 smithi133 ceph-mon[31430]: pgmap v127: 97 pgs: 97 active+clean; 30 GiB data, 134 GiB used, 939 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 690 MiB/s wr, 1.26k op/s 2024-09-11T23:40:55.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:55 smithi178 ceph-mon[35882]: pgmap v127: 97 pgs: 97 active+clean; 30 GiB data, 134 GiB used, 939 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 690 MiB/s wr, 1.26k op/s 2024-09-11T23:40:57.211 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:40:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.266+0000 7f6c5d50a640 1 -- 172.21.15.133:0/241021970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c581091f0 msgr2=0x7f6c58110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.266+0000 7f6c5d50a640 1 --2- 172.21.15.133:0/241021970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c581091f0 0x7f6c58110250 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009860 tx=0x7f6c4802a340 comp rx=0 tx=0).stop 2024-09-11T23:40:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.277+0000 7f6c5d50a640 1 -- 172.21.15.133:0/241021970 shutdown_connections 2024-09-11T23:40:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.277+0000 7f6c5d50a640 1 --2- 172.21.15.133:0/241021970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c5811b140 0x7f6c5811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.277+0000 7f6c5d50a640 1 --2- 172.21.15.133:0/241021970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c581095d0 0x7f6c58109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.277+0000 7f6c5d50a640 1 --2- 172.21.15.133:0/241021970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c581091f0 0x7f6c58110250 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.277+0000 7f6c5d50a640 1 -- 172.21.15.133:0/241021970 >> 172.21.15.133:0/241021970 conn(0x7f6c5806cc60 msgr2=0x7f6c5806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.278+0000 7f6c5d50a640 1 -- 172.21.15.133:0/241021970 shutdown_connections 2024-09-11T23:40:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.278+0000 7f6c5d50a640 1 -- 172.21.15.133:0/241021970 wait complete. 2024-09-11T23:40:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.279+0000 7f6c5d50a640 1 Processor -- start 2024-09-11T23:40:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.280+0000 7f6c5d50a640 1 -- start start 2024-09-11T23:40:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.280+0000 7f6c5d50a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c581091f0 0x7f6c581187c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.280+0000 7f6c5d50a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c581095d0 0x7f6c58111890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.280+0000 7f6c5d50a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c5811b140 0x7f6c58111dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.280+0000 7f6c5d50a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58120b10 con 0x7f6c581091f0 2024-09-11T23:40:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.280+0000 7f6c5d50a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58120990 con 0x7f6c5811b140 2024-09-11T23:40:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.280+0000 7f6c5d50a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58120c90 con 0x7f6c581095d0 2024-09-11T23:40:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.281+0000 7f6c56ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c581091f0 0x7f6c581187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.281+0000 7f6c56ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c581091f0 0x7f6c581187c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48014/0 (socket says 172.21.15.133:48014) 2024-09-11T23:40:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.281+0000 7f6c56ffd640 1 -- 172.21.15.133:0/3788804765 learned_addr learned my addr 172.21.15.133:0/3788804765 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.282+0000 7f6c56ffd640 1 -- 172.21.15.133:0/3788804765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c581095d0 msgr2=0x7f6c58111890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:40:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.282+0000 7f6c56ffd640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c581095d0 0x7f6c58111890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.282+0000 7f6c56ffd640 1 -- 172.21.15.133:0/3788804765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c5811b140 msgr2=0x7f6c58111dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.282+0000 7f6c56ffd640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c5811b140 0x7f6c58111dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.282+0000 7f6c56ffd640 1 -- 172.21.15.133:0/3788804765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c58112440 con 0x7f6c581091f0 2024-09-11T23:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.285+0000 7f6c56ffd640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c581091f0 0x7f6c581187c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009980 tx=0x7f6c48005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.286+0000 7f6c3ffff640 1 -- 172.21.15.133:0/3788804765 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48040020 con 0x7f6c581091f0 2024-09-11T23:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.286+0000 7f6c5d50a640 1 -- 172.21.15.133:0/3788804765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c581126d0 con 0x7f6c581091f0 2024-09-11T23:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.287+0000 7f6c5d50a640 1 -- 172.21.15.133:0/3788804765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c58116600 con 0x7f6c581091f0 2024-09-11T23:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.287+0000 7f6c3ffff640 1 -- 172.21.15.133:0/3788804765 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c48005490 con 0x7f6c581091f0 2024-09-11T23:40:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.287+0000 7f6c3ffff640 1 -- 172.21.15.133:0/3788804765 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48037070 con 0x7f6c581091f0 2024-09-11T23:40:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.303+0000 7f6c602a0740 1 Processor -- start 2024-09-11T23:40:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.304+0000 7f6c602a0740 1 -- start start 2024-09-11T23:40:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.304+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f67460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.305+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f67465e0 0x5602f6750940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.305+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6750e80 0x5602f6753270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.305+0000 7f6c602a0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5602f66aa2f0 con 0x5602f67465e0 2024-09-11T23:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.305+0000 7f6c602a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602f6652270 con 0x5602f6745cc0 2024-09-11T23:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.305+0000 7f6c602a0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5602f66a62b0 con 0x5602f6750e80 2024-09-11T23:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.306+0000 7f6c3ffff640 1 -- 172.21.15.133:0/3788804765 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6c48043020 con 0x7f6c581091f0 2024-09-11T23:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.306+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f67460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.306+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f67460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42410/0 (socket says 172.21.15.133:42410) 2024-09-11T23:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.306+0000 7f6c577fe640 1 -- 172.21.15.133:0/4027177009 learned_addr learned my addr 172.21.15.133:0/4027177009 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.306+0000 7f6c567fc640 1 --2- 172.21.15.133:0/4027177009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6750e80 0x5602f6753270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.306+0000 7f6c577fe640 1 -- 172.21.15.133:0/4027177009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6750e80 msgr2=0x5602f6753270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.306+0000 7f6c577fe640 1 --2- 172.21.15.133:0/4027177009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6750e80 0x5602f6753270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.306+0000 7f6c577fe640 1 -- 172.21.15.133:0/4027177009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f67465e0 msgr2=0x5602f6750940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.306+0000 7f6c577fe640 1 --2- 172.21.15.133:0/4027177009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f67465e0 0x5602f6750940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.306+0000 7f6c577fe640 1 -- 172.21.15.133:0/4027177009 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f6651340 con 0x5602f6745cc0 2024-09-11T23:40:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.308+0000 7f6c577fe640 1 --2- 172.21.15.133:0/4027177009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f67460a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009f50 tx=0x7f6c5000e880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91dd38c95e03e7e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.315+0000 7f6c3ffff640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c400a2ec0 0x7f6c400a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.316+0000 7f6c3d7fa640 1 -- 172.21.15.133:0/4027177009 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50003970 con 0x5602f6745cc0 2024-09-11T23:40:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.316+0000 7f6c3d7fa640 1 -- 172.21.15.133:0/4027177009 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c50004020 con 0x5602f6745cc0 2024-09-11T23:40:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.316+0000 7f6c3d7fa640 1 -- 172.21.15.133:0/4027177009 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000cac0 con 0x5602f6745cc0 2024-09-11T23:40:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.317+0000 7f6c602a0740 1 -- 172.21.15.133:0/4027177009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 msgr2=0x5602f67460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.317+0000 7f6c602a0740 1 --2- 172.21.15.133:0/4027177009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f67460a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009f50 tx=0x7f6c5000e880 comp rx=0 tx=0).stop 2024-09-11T23:40:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.318+0000 7f6c3ffff640 1 -- 172.21.15.133:0/3788804765 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6c480eefe0 con 0x7f6c581091f0 2024-09-11T23:40:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.318+0000 7f6c602a0740 1 -- 172.21.15.133:0/4027177009 shutdown_connections 2024-09-11T23:40:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.318+0000 7f6c602a0740 1 --2- 172.21.15.133:0/4027177009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6750e80 0x5602f6753270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.318+0000 7f6c602a0740 1 --2- 172.21.15.133:0/4027177009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f67465e0 0x5602f6750940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.318+0000 7f6c602a0740 1 --2- 172.21.15.133:0/4027177009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f67460a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.318+0000 7f6c602a0740 1 -- 172.21.15.133:0/4027177009 >> 172.21.15.133:0/4027177009 conn(0x5602f6742230 msgr2=0x5602f6742610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.319+0000 7f6c602a0740 1 -- 172.21.15.133:0/4027177009 shutdown_connections 2024-09-11T23:40:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.319+0000 7f6c602a0740 1 -- 172.21.15.133:0/4027177009 wait complete. 2024-09-11T23:40:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.320+0000 7f6c602a0740 1 -- start start 2024-09-11T23:40:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.320+0000 7f6c602a0740 1 Processor -- start 2024-09-11T23:40:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.321+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f6745cc0 0x5602f674de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.321+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f67465e0 0x5602f674e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.321+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6750e80 0x5602f674e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.322+0000 7f6c602a0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5602f6753a50 con 0x5602f6745cc0 2024-09-11T23:40:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.322+0000 7f6c602a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602f6633290 con 0x5602f6750e80 2024-09-11T23:40:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.322+0000 7f6c602a0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5602f6753bd0 con 0x5602f67465e0 2024-09-11T23:40:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.322+0000 7f6c567fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f67465e0 0x5602f674e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.322+0000 7f6c567fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f67465e0 0x5602f674e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33114/0 (socket says 172.21.15.133:33114) 2024-09-11T23:40:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.322+0000 7f6c567fc640 1 -- 172.21.15.133:0/332987238 learned_addr learned my addr 172.21.15.133:0/332987238 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.322+0000 7f6c602a0740 10 client.0 _ll_register_callbacks cb 0x5602f6664140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.323+0000 7f6c577fe640 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6750e80 0x5602f674e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.329+0000 7f6c56ffd640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c400a2ec0 0x7f6c400a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.330+0000 7f6c577fe640 1 -- 172.21.15.133:0/332987238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f67465e0 msgr2=0x5602f674e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.330+0000 7f6c577fe640 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f67465e0 0x5602f674e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.330+0000 7f6c577fe640 1 -- 172.21.15.133:0/332987238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f6745cc0 msgr2=0x5602f674de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.330+0000 7f6c577fe640 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f6745cc0 0x5602f674de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.330+0000 7f6c577fe640 1 -- 172.21.15.133:0/332987238 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f63bd210 con 0x5602f6750e80 2024-09-11T23:40:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.330+0000 7f6c577fe640 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6750e80 0x5602f674e890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009f50 tx=0x7f6c50009d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77f12641124a1510 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.331+0000 7f6c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.334+0000 7f6c56ffd640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c400a2ec0 0x7f6c400a5380 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x5602f6743090 tx=0x7f6c4802a340 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.342+0000 7f6c3aff5640 1 -- 172.21.15.133:0/332987238 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50023030 con 0x5602f6750e80 2024-09-11T23:40:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.343+0000 7f6c602a0740 1 -- 172.21.15.133:0/332987238 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c50004020 con 0x5602f6750e80 2024-09-11T23:40:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.343+0000 7f6c3aff5640 1 -- 172.21.15.133:0/332987238 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6c50013070 con 0x5602f6750e80 2024-09-11T23:40:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.345+0000 7f6c3aff5640 1 -- 172.21.15.133:0/332987238 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c5002d050 con 0x5602f6750e80 2024-09-11T23:40:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.349+0000 7f6c602a0740 10 client.26030 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.349+0000 7f6c602a0740 1 -- 172.21.15.133:0/332987238 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602f6489620 con 0x5602f6750e80 2024-09-11T23:40:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.354+0000 7f6c3aff5640 1 -- 172.21.15.133:0/332987238 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c50004020 con 0x5602f6750e80 2024-09-11T23:40:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.354+0000 7f6c3aff5640 1 -- 172.21.15.133:0/332987238 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6c5000b040 con 0x5602f6750e80 2024-09-11T23:40:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.354+0000 7f6c602a0740 10 client.26030 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.354+0000 7f6c602a0740 10 client.26030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.354+0000 7f6c602a0740 20 client.26030 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.354+0000 7f6c602a0740 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5602f6761270 0x5602f6763690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.354+0000 7f6c602a0740 4 client.26030 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.354+0000 7f6c602a0740 1 -- 172.21.15.133:0/332987238 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602f66b7a20 con 0x5602f6761270 2024-09-11T23:40:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.357+0000 7f6c567fc640 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5602f6761270 0x5602f6763690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.362+0000 7f6c567fc640 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5602f6761270 0x5602f6763690 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fcc1a5bc27e6d84 server_cookie=d07e1134c7427eac in_seq=0 out_seq=0 2024-09-11T23:40:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.362+0000 7f6c3aff5640 10 client.26030 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.371+0000 7f6c3aff5640 1 -- 172.21.15.133:0/332987238 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602f66b7a20 con 0x5602f6761270 2024-09-11T23:40:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.371+0000 7f6c3aff5640 10 client.26030 handle_command_reply: tid=0 2024-09-11T23:40:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.371+0000 7f6c602a0740 1 client.26030 shutdown 2024-09-11T23:40:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.372+0000 7f6c602a0740 1 -- 172.21.15.133:0/332987238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6750e80 msgr2=0x5602f674e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.372+0000 7f6c602a0740 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6750e80 0x5602f674e890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009f50 tx=0x7f6c50009d20 comp rx=0 tx=0).stop 2024-09-11T23:40:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.372+0000 7f6c602a0740 1 -- 172.21.15.133:0/332987238 shutdown_connections 2024-09-11T23:40:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.372+0000 7f6c602a0740 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5602f6761270 0x5602f6763690 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.372+0000 7f6c602a0740 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6750e80 0x5602f674e890 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.372+0000 7f6c602a0740 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f67465e0 0x5602f674e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.372+0000 7f6c602a0740 1 --2- 172.21.15.133:0/332987238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f6745cc0 0x5602f674de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.372+0000 7f6c3aff5640 0 client.26030 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.372+0000 7f6c602a0740 1 -- 172.21.15.133:0/332987238 >> 172.21.15.133:0/332987238 conn(0x5602f6742230 msgr2=0x5602f6747f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.372+0000 7f6c602a0740 1 -- 172.21.15.133:0/332987238 shutdown_connections 2024-09-11T23:40:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.372+0000 7f6c602a0740 1 -- 172.21.15.133:0/332987238 wait complete. 2024-09-11T23:40:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.373+0000 7f6c602a0740 20 client.26030 trim_cache size 0 max 16384 2024-09-11T23:40:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.424+0000 7f6c602a0740 1 Processor -- start 2024-09-11T23:40:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.425+0000 7f6c602a0740 1 -- start start 2024-09-11T23:40:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.426+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f6748780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.426+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f67465e0 0x5602f674e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.427+0000 7f6c567fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f67465e0 0x5602f674e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.427+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f6748780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.427+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f6748780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42426/0 (socket says 172.21.15.133:42426) 2024-09-11T23:40:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.427+0000 7f6c567fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f67465e0 0x5602f674e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48032/0 (socket says 172.21.15.133:48032) 2024-09-11T23:40:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.427+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6750e80 0x5602f6768fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.427+0000 7f6c577fe640 1 -- 172.21.15.133:0/929763841 learned_addr learned my addr 172.21.15.133:0/929763841 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.427+0000 7f6c602a0740 1 -- 172.21.15.133:0/929763841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4400e6e0 con 0x5602f67465e0 2024-09-11T23:40:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.427+0000 7f6c602a0740 1 -- 172.21.15.133:0/929763841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44010220 con 0x5602f6745cc0 2024-09-11T23:40:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.427+0000 7f6c602a0740 1 -- 172.21.15.133:0/929763841 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4400edb0 con 0x5602f6750e80 2024-09-11T23:40:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.429+0000 7f6c567fc640 1 -- 172.21.15.133:0/929763841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6750e80 msgr2=0x5602f6768fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:40:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.429+0000 7f6c567fc640 1 --2- 172.21.15.133:0/929763841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6750e80 0x5602f6768fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.429+0000 7f6c567fc640 1 -- 172.21.15.133:0/929763841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 msgr2=0x5602f6748780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.429+0000 7f6c567fc640 1 --2- 172.21.15.133:0/929763841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f6748780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.429+0000 7f6c567fc640 1 -- 172.21.15.133:0/929763841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f6746410 con 0x5602f67465e0 2024-09-11T23:40:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.429+0000 7f6c577fe640 1 --2- 172.21.15.133:0/929763841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f6748780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:40:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.430+0000 7f6c567fc640 1 --2- 172.21.15.133:0/929763841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f67465e0 0x5602f674e6c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x5602f6763ee0 tx=0x7f6c4c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3add29dab059e9fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.430+0000 7f6c3cff9640 1 -- 172.21.15.133:0/929763841 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c003de0 con 0x5602f67465e0 2024-09-11T23:40:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.431+0000 7f6c3cff9640 1 -- 172.21.15.133:0/929763841 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c4c01d900 con 0x5602f67465e0 2024-09-11T23:40:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.431+0000 7f6c3cff9640 1 -- 172.21.15.133:0/929763841 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c010a90 con 0x5602f67465e0 2024-09-11T23:40:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.433+0000 7f6c602a0740 1 -- 172.21.15.133:0/929763841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f67465e0 msgr2=0x5602f674e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.433+0000 7f6c602a0740 1 --2- 172.21.15.133:0/929763841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f67465e0 0x5602f674e6c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x5602f6763ee0 tx=0x7f6c4c004410 comp rx=0 tx=0).stop 2024-09-11T23:40:57.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.444+0000 7f8f21d51640 1 -- 172.21.15.133:0/1501242777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f1c110190 msgr2=0x7f8f1c06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.444+0000 7f8f21d51640 1 --2- 172.21.15.133:0/1501242777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f1c110190 0x7f8f1c06e4f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009900 tx=0x7f8f0802a440 comp rx=0 tx=0).stop 2024-09-11T23:40:57.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.449+0000 7f8f21d51640 1 -- 172.21.15.133:0/1501242777 shutdown_connections 2024-09-11T23:40:57.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.449+0000 7f8f21d51640 1 --2- 172.21.15.133:0/1501242777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f1c11b140 0x7f8f1c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.449+0000 7f8f21d51640 1 --2- 172.21.15.133:0/1501242777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f1c06ea30 0x7f8f1c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.449+0000 7f8f21d51640 1 --2- 172.21.15.133:0/1501242777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f1c110190 0x7f8f1c06e4f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.449+0000 7f8f21d51640 1 -- 172.21.15.133:0/1501242777 >> 172.21.15.133:0/1501242777 conn(0x7f8f1c073180 msgr2=0x7f8f1c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.453+0000 7f8f21d51640 1 -- 172.21.15.133:0/1501242777 shutdown_connections 2024-09-11T23:40:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.453+0000 7f6c602a0740 1 -- 172.21.15.133:0/929763841 shutdown_connections 2024-09-11T23:40:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.453+0000 7f6c602a0740 1 --2- 172.21.15.133:0/929763841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6750e80 0x5602f6768fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.453+0000 7f6c602a0740 1 --2- 172.21.15.133:0/929763841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f67465e0 0x5602f674e6c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.453+0000 7f6c602a0740 1 --2- 172.21.15.133:0/929763841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f6745cc0 0x5602f6748780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.453+0000 7f6c602a0740 1 -- 172.21.15.133:0/929763841 >> 172.21.15.133:0/929763841 conn(0x5602f6742230 msgr2=0x5602f674fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.453+0000 7f8f21d51640 1 -- 172.21.15.133:0/1501242777 wait complete. 2024-09-11T23:40:57.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.454+0000 7f8f21d51640 1 Processor -- start 2024-09-11T23:40:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.454+0000 7f6c602a0740 1 -- 172.21.15.133:0/929763841 shutdown_connections 2024-09-11T23:40:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.454+0000 7f6c602a0740 1 -- 172.21.15.133:0/929763841 wait complete. 2024-09-11T23:40:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.455+0000 7f6c602a0740 1 -- start start 2024-09-11T23:40:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.455+0000 7f6c602a0740 1 Processor -- start 2024-09-11T23:40:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.456+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6745cc0 0x5602f6759730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.456+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f67465e0 0x5602f6754a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.456+0000 7f6c602a0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f6750e80 0x5602f6755040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.456+0000 7f6c602a0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4400fb50 con 0x5602f6750e80 2024-09-11T23:40:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.456+0000 7f6c602a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4400f480 con 0x5602f67465e0 2024-09-11T23:40:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.456+0000 7f6c602a0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4400c6c0 con 0x5602f6745cc0 2024-09-11T23:40:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.456+0000 7f6c602a0740 10 client.0 _ll_register_callbacks cb 0x7f6c4c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:57.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.454+0000 7f8f21d51640 1 -- start start 2024-09-11T23:40:57.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.454+0000 7f8f21d51640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f1c06ea30 0x7f8f1c119800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.454+0000 7f8f21d51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f1c110190 0x7f8f1c112890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.455+0000 7f8f21d51640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f1c11b140 0x7f8f1c112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.455+0000 7f8f21d51640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c120be0 con 0x7f8f1c11b140 2024-09-11T23:40:57.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.455+0000 7f8f21d51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c120a60 con 0x7f8f1c110190 2024-09-11T23:40:57.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.455+0000 7f8f21d51640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c120d60 con 0x7f8f1c06ea30 2024-09-11T23:40:57.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.456+0000 7f8f20d4f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f1c06ea30 0x7f8f1c119800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.456+0000 7f8f20d4f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f1c06ea30 0x7f8f1c119800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33148/0 (socket says 172.21.15.133:33148) 2024-09-11T23:40:57.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.456+0000 7f8f20d4f640 1 -- 172.21.15.133:0/1496454179 learned_addr learned my addr 172.21.15.133:0/1496454179 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.457+0000 7f6c567fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f67465e0 0x5602f6754a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.457+0000 7f6c567fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f67465e0 0x5602f6754a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42452/0 (socket says 172.21.15.133:42452) 2024-09-11T23:40:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.457+0000 7f6c567fc640 1 -- 172.21.15.133:0/1811721885 learned_addr learned my addr 172.21.15.133:0/1811721885 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.457+0000 7f6c577fe640 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f6750e80 0x5602f6755040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.457+0000 7f8f20d4f640 1 -- 172.21.15.133:0/1496454179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f1c110190 msgr2=0x7f8f1c112890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:40:57.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.457+0000 7f8f20d4f640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f1c110190 0x7f8f1c112890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.457+0000 7f8f20d4f640 1 -- 172.21.15.133:0/1496454179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f1c11b140 msgr2=0x7f8f1c112dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.457+0000 7f8f20d4f640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f1c11b140 0x7f8f1c112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.457+0000 7f8f20d4f640 1 -- 172.21.15.133:0/1496454179 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c113440 con 0x7f8f1c06ea30 2024-09-11T23:40:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.461+0000 7f6c577fe640 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6745cc0 0x5602f6759730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.458+0000 7f8f20d4f640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f1c06ea30 0x7f8f1c119800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f0802a9b0 tx=0x7f8f08031ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.459+0000 7f8f19ffb640 1 -- 172.21.15.133:0/1496454179 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08041020 con 0x7f8f1c06ea30 2024-09-11T23:40:57.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.459+0000 7f8f21d51640 1 -- 172.21.15.133:0/1496454179 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c1136d0 con 0x7f8f1c06ea30 2024-09-11T23:40:57.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.460+0000 7f8f21d51640 1 -- 172.21.15.133:0/1496454179 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1c117690 con 0x7f8f1c06ea30 2024-09-11T23:40:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.462+0000 7f6c567fc640 1 -- 172.21.15.133:0/1811721885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6745cc0 msgr2=0x5602f6759730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.462+0000 7f6c567fc640 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6745cc0 0x5602f6759730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.462+0000 7f6c567fc640 1 -- 172.21.15.133:0/1811721885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f6750e80 msgr2=0x5602f6755040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.462+0000 7f6c567fc640 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f6750e80 0x5602f6755040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.462+0000 7f6c567fc640 1 -- 172.21.15.133:0/1811721885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f6755840 con 0x5602f67465e0 2024-09-11T23:40:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.463+0000 7f6c567fc640 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f67465e0 0x5602f6754a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009c80 tx=0x7f6c4c0099b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7993ff7de23bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.463+0000 7f6c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.464+0000 7f6c3aff5640 1 -- 172.21.15.133:0/1811721885 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c003de0 con 0x5602f67465e0 2024-09-11T23:40:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.464+0000 7f6c602a0740 1 -- 172.21.15.133:0/1811721885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c4c01d900 con 0x5602f67465e0 2024-09-11T23:40:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.464+0000 7f6c3aff5640 1 -- 172.21.15.133:0/1811721885 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6c4c01d420 con 0x5602f67465e0 2024-09-11T23:40:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.465+0000 7f6c3aff5640 1 -- 172.21.15.133:0/1811721885 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c4c01dcc0 con 0x5602f67465e0 2024-09-11T23:40:57.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.471+0000 7f8f2538c740 1 Processor -- start 2024-09-11T23:40:57.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.472+0000 7f8f19ffb640 1 -- 172.21.15.133:0/1496454179 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f08009520 con 0x7f8f1c06ea30 2024-09-11T23:40:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.476+0000 7f6c602a0740 10 client.26039 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.476+0000 7f6c602a0740 1 -- 172.21.15.133:0/1811721885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602f67cc060 con 0x5602f67465e0 2024-09-11T23:40:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.477+0000 7f6c3aff5640 1 -- 172.21.15.133:0/1811721885 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c4c01d900 con 0x5602f67465e0 2024-09-11T23:40:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.477+0000 7f6c3aff5640 1 -- 172.21.15.133:0/1811721885 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6c4c038020 con 0x5602f67465e0 2024-09-11T23:40:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.477+0000 7f6c602a0740 10 client.26039 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.477+0000 7f6c602a0740 10 client.26039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.477+0000 7f6c602a0740 20 client.26039 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.477+0000 7f6c602a0740 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5602f67cc660 0x5602f67cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.477+0000 7f6c602a0740 4 client.26039 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:40:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.477+0000 7f6c602a0740 1 -- 172.21.15.133:0/1811721885 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602f66b1d70 con 0x5602f67cc660 2024-09-11T23:40:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.478+0000 7f6c577fe640 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5602f67cc660 0x5602f67cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.480+0000 7f6c577fe640 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5602f67cc660 0x5602f67cea80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=760042a33e2eaf0c server_cookie=618329c34e4a87a2 in_seq=0 out_seq=0 2024-09-11T23:40:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.480+0000 7f6c3aff5640 10 client.26039 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:57.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.473+0000 7f8f2538c740 1 -- start start 2024-09-11T23:40:57.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.473+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd132e320 0x55fcd132caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.473+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd132d030 0x55fcd132d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.473+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1337990 0x55fcd1339d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.473+0000 7f8f2538c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fcd1291160 con 0x55fcd132d030 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.473+0000 7f8f2538c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fcd1238720 con 0x55fcd1337990 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.473+0000 7f8f2538c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fcd128d120 con 0x55fcd132e320 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.478+0000 7f8f21550640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd132e320 0x55fcd132caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.478+0000 7f8f1bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1337990 0x55fcd1339d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.480+0000 7f8f21550640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd132e320 0x55fcd132caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33164/0 (socket says 172.21.15.133:33164) 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.480+0000 7f8f21550640 1 -- 172.21.15.133:0/2737126640 learned_addr learned my addr 172.21.15.133:0/2737126640 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.483+0000 7f8f21550640 1 -- 172.21.15.133:0/2737126640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1337990 msgr2=0x55fcd1339d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.483+0000 7f8f21550640 1 --2- 172.21.15.133:0/2737126640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1337990 0x55fcd1339d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.483+0000 7f8f21550640 1 -- 172.21.15.133:0/2737126640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd132d030 msgr2=0x55fcd132d470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.483+0000 7f8f21550640 1 --2- 172.21.15.133:0/2737126640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd132d030 0x55fcd132d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.483+0000 7f8f21550640 1 -- 172.21.15.133:0/2737126640 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcd1002ba0 con 0x55fcd132e320 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.489+0000 7f8f21550640 1 --2- 172.21.15.133:0/2737126640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd132e320 0x55fcd132caf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009d60 tx=0x7f8f1400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76ebcd2b8a28534d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.490+0000 7f8f19ffb640 1 -- 172.21.15.133:0/1496454179 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08041020 con 0x7f8f1c06ea30 2024-09-11T23:40:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.491+0000 7f8f21d51640 1 -- 172.21.15.133:0/2737126640 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14038070 con 0x55fcd132e320 2024-09-11T23:40:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.492+0000 7f8f21d51640 1 -- 172.21.15.133:0/2737126640 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f14005020 con 0x55fcd132e320 2024-09-11T23:40:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.494+0000 7f8f2538c740 1 -- 172.21.15.133:0/2737126640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd132e320 msgr2=0x55fcd132caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.494+0000 7f8f2538c740 1 --2- 172.21.15.133:0/2737126640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd132e320 0x55fcd132caf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009d60 tx=0x7f8f1400e6c0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.495+0000 7f8f2538c740 1 -- 172.21.15.133:0/2737126640 shutdown_connections 2024-09-11T23:40:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.495+0000 7f8f2538c740 1 --2- 172.21.15.133:0/2737126640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1337990 0x55fcd1339d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.495+0000 7f8f2538c740 1 --2- 172.21.15.133:0/2737126640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd132d030 0x55fcd132d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.495+0000 7f8f2538c740 1 --2- 172.21.15.133:0/2737126640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd132e320 0x55fcd132caf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.495+0000 7f8f2538c740 1 -- 172.21.15.133:0/2737126640 >> 172.21.15.133:0/2737126640 conn(0x55fcd1329280 msgr2=0x55fcd1329660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.495+0000 7f8f2538c740 1 -- 172.21.15.133:0/2737126640 shutdown_connections 2024-09-11T23:40:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.495+0000 7f8f2538c740 1 -- 172.21.15.133:0/2737126640 wait complete. 2024-09-11T23:40:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.496+0000 7f8f2538c740 1 -- start start 2024-09-11T23:40:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.496+0000 7f8f2538c740 1 Processor -- start 2024-09-11T23:40:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.497+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 0x55fcd1337420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.497+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd1337990 0x55fcd13346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.497+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1334dc0 0x55fcd1343de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.497+0000 7f8f2538c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fcd133a850 con 0x55fcd1337990 2024-09-11T23:40:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.497+0000 7f8f2538c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fcd1219900 con 0x55fcd132d030 2024-09-11T23:40:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.497+0000 7f8f2538c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fcd133a9d0 con 0x55fcd1334dc0 2024-09-11T23:40:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.497+0000 7f8f2538c740 10 client.0 _ll_register_callbacks cb 0x55fcd1217780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.500+0000 7f8f21550640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 0x55fcd1337420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.500+0000 7f8f1bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1334dc0 0x55fcd1343de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.508+0000 7f8f21550640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 0x55fcd1337420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42466/0 (socket says 172.21.15.133:42466) 2024-09-11T23:40:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.508+0000 7f8f21550640 1 -- 172.21.15.133:0/3615046990 learned_addr learned my addr 172.21.15.133:0/3615046990 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.510+0000 7f8f1bfff640 1 -- 172.21.15.133:0/3615046990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 msgr2=0x55fcd1337420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.510+0000 7f8f1bfff640 1 --2- 172.21.15.133:0/3615046990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 0x55fcd1337420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.510+0000 7f8f1bfff640 1 -- 172.21.15.133:0/3615046990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd1337990 msgr2=0x55fcd13346d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:40:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.511+0000 7f8f1bfff640 1 --2- 172.21.15.133:0/3615046990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd1337990 0x55fcd13346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.511+0000 7f8f1bfff640 1 -- 172.21.15.133:0/3615046990 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcd1259c50 con 0x55fcd1334dc0 2024-09-11T23:40:57.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.513+0000 7f8f19ffb640 1 -- 172.21.15.133:0/1496454179 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8f08044020 con 0x7f8f1c06ea30 2024-09-11T23:40:57.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.514+0000 7f8f19ffb640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f0c0a3070 0x7f8f0c0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.515+0000 7f8f19ffb640 1 -- 172.21.15.133:0/1496454179 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8f080051c0 con 0x7f8f1c06ea30 2024-09-11T23:40:57.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.527+0000 7f8f21550640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f0c0a3070 0x7f8f0c0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.528+0000 7f8f21550640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f0c0a3070 0x7f8f0c0a5530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8f140097e0 tx=0x7f8f1403f040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.531+0000 7f8f1bfff640 1 --2- 172.21.15.133:0/3615046990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1334dc0 0x55fcd1343de0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c114050 tx=0x7f8f10003d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b55ec1ef6aa464f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.533+0000 7f8efbfff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:40:57.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.538+0000 7f8efbfff640 1 -- 172.21.15.133:0/3615046990 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10003e30 con 0x55fcd1334dc0 2024-09-11T23:40:57.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.538+0000 7f8f2538c740 1 -- 172.21.15.133:0/3615046990 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fcd129e890 con 0x55fcd1334dc0 2024-09-11T23:40:57.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.539+0000 7f8efbfff640 1 -- 172.21.15.133:0/3615046990 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8f10011440 con 0x55fcd1334dc0 2024-09-11T23:40:57.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.544+0000 7f8efbfff640 1 -- 172.21.15.133:0/3615046990 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f10011640 con 0x55fcd1334dc0 2024-09-11T23:40:57.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.544+0000 7f8f2538c740 10 client.26041 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:57.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.544+0000 7f8f2538c740 1 -- 172.21.15.133:0/3615046990 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fcd1368960 con 0x55fcd1334dc0 2024-09-11T23:40:57.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.547+0000 7f8efbfff640 1 -- 172.21.15.133:0/3615046990 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fcd129e890 con 0x55fcd1334dc0 2024-09-11T23:40:57.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.547+0000 7f8efbfff640 1 -- 172.21.15.133:0/3615046990 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f10022050 con 0x55fcd1334dc0 2024-09-11T23:40:57.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.547+0000 7f8f2538c740 10 client.26041 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:57.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.547+0000 7f8f2538c740 10 client.26041 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:40:57.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.547+0000 7f8f2538c740 20 client.26041 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:57.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.547+0000 7f8f2538c740 1 --2- 172.21.15.133:0/3615046990 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fcd136a1e0 0x55fcd136c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.547+0000 7f8f2538c740 4 client.26041 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:57.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.547+0000 7f8f2538c740 1 -- 172.21.15.133:0/3615046990 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcd1298be0 con 0x55fcd136a1e0 2024-09-11T23:40:57.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.557+0000 7f8f20d4f640 1 --2- 172.21.15.133:0/3615046990 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fcd136a1e0 0x55fcd136c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.565+0000 7f8f20d4f640 1 --2- 172.21.15.133:0/3615046990 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fcd136a1e0 0x55fcd136c660 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d467e682934ddf server_cookie=b328e811b897102b in_seq=0 out_seq=0 2024-09-11T23:40:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.566+0000 7f8efbfff640 10 client.26041 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.568+0000 7f8efbfff640 1 -- 172.21.15.133:0/3615046990 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fcd1298be0 con 0x55fcd136a1e0 2024-09-11T23:40:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.568+0000 7f8efbfff640 10 client.26041 handle_command_reply: tid=0 2024-09-11T23:40:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.568+0000 7f8f2538c740 1 client.26041 shutdown 2024-09-11T23:40:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.568+0000 7f8f2538c740 1 -- 172.21.15.133:0/3615046990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1334dc0 msgr2=0x55fcd1343de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.568+0000 7f8f2538c740 1 --2- 172.21.15.133:0/3615046990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1334dc0 0x55fcd1343de0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c114050 tx=0x7f8f10003d00 comp rx=0 tx=0).stop 2024-09-11T23:40:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.568+0000 7f8f2538c740 1 -- 172.21.15.133:0/3615046990 shutdown_connections 2024-09-11T23:40:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.568+0000 7f8f2538c740 1 --2- 172.21.15.133:0/3615046990 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fcd136a1e0 0x55fcd136c660 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.569+0000 7f8f2538c740 1 --2- 172.21.15.133:0/3615046990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1334dc0 0x55fcd1343de0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.569+0000 7f8f2538c740 1 --2- 172.21.15.133:0/3615046990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd1337990 0x55fcd13346d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.569+0000 7f8f2538c740 1 --2- 172.21.15.133:0/3615046990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 0x55fcd1337420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.569+0000 7f8f2538c740 1 -- 172.21.15.133:0/3615046990 >> 172.21.15.133:0/3615046990 conn(0x55fcd1329280 msgr2=0x55fcd132efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.569+0000 7f8efbfff640 0 client.26041 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.569+0000 7f8f2538c740 1 -- 172.21.15.133:0/3615046990 shutdown_connections 2024-09-11T23:40:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.569+0000 7f8f2538c740 1 -- 172.21.15.133:0/3615046990 wait complete. 2024-09-11T23:40:57.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.569+0000 7f8f2538c740 20 client.26041 trim_cache size 0 max 16384 2024-09-11T23:40:57.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.627+0000 7f8f2538c740 1 Processor -- start 2024-09-11T23:40:57.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.627+0000 7f8f2538c740 1 -- start start 2024-09-11T23:40:57.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.628+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 0x55fcd1328390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.628+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd1334dc0 0x55fcd1343bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.629+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1337990 0x55fcd1344110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.629+0000 7f8f20d4f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 0x55fcd1328390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.629+0000 7f8f20d4f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 0x55fcd1328390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42478/0 (socket says 172.21.15.133:42478) 2024-09-11T23:40:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.629+0000 7f8f20d4f640 1 -- 172.21.15.133:0/205091407 learned_addr learned my addr 172.21.15.133:0/205091407 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.629+0000 7f8f2538c740 1 -- 172.21.15.133:0/205091407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0000e6e0 con 0x55fcd1334dc0 2024-09-11T23:40:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.629+0000 7f8f2538c740 1 -- 172.21.15.133:0/205091407 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f00010220 con 0x55fcd132d030 2024-09-11T23:40:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.629+0000 7f8f2538c740 1 -- 172.21.15.133:0/205091407 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0000edb0 con 0x55fcd1337990 2024-09-11T23:40:57.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.634+0000 7f8f21550640 1 --2- 172.21.15.133:0/205091407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1337990 0x55fcd1344110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.634+0000 7f8f1bfff640 1 --2- 172.21.15.133:0/205091407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd1334dc0 0x55fcd1343bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.635+0000 7f8f21550640 1 -- 172.21.15.133:0/205091407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 msgr2=0x55fcd1328390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.635+0000 7f8f21550640 1 --2- 172.21.15.133:0/205091407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 0x55fcd1328390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.635+0000 7f8f21550640 1 -- 172.21.15.133:0/205091407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd1334dc0 msgr2=0x55fcd1343bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.635+0000 7f8f21550640 1 --2- 172.21.15.133:0/205091407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd1334dc0 0x55fcd1343bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.635+0000 7f8f21550640 1 -- 172.21.15.133:0/205091407 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcd1323f60 con 0x55fcd1337990 2024-09-11T23:40:57.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.636+0000 7f8f21550640 1 --2- 172.21.15.133:0/205091407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1337990 0x55fcd1344110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009d60 tx=0x7f8f14043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71d48272a630099a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.639+0000 7f8f067fc640 1 -- 172.21.15.133:0/205091407 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14004d20 con 0x55fcd1337990 2024-09-11T23:40:57.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.639+0000 7f8f067fc640 1 -- 172.21.15.133:0/205091407 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f14004000 con 0x55fcd1337990 2024-09-11T23:40:57.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.639+0000 7f8f067fc640 1 -- 172.21.15.133:0/205091407 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14005280 con 0x55fcd1337990 2024-09-11T23:40:57.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.640+0000 7f8f2538c740 1 -- 172.21.15.133:0/205091407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1337990 msgr2=0x55fcd1344110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.641+0000 7f8f2538c740 1 --2- 172.21.15.133:0/205091407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1337990 0x55fcd1344110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009d60 tx=0x7f8f14043000 comp rx=0 tx=0).stop 2024-09-11T23:40:57.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.641+0000 7f8f2538c740 1 -- 172.21.15.133:0/205091407 shutdown_connections 2024-09-11T23:40:57.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.641+0000 7f8f2538c740 1 --2- 172.21.15.133:0/205091407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1337990 0x55fcd1344110 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.641+0000 7f8f2538c740 1 --2- 172.21.15.133:0/205091407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd1334dc0 0x55fcd1343bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.641+0000 7f8f2538c740 1 --2- 172.21.15.133:0/205091407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd132d030 0x55fcd1328390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.641+0000 7f8f2538c740 1 -- 172.21.15.133:0/205091407 >> 172.21.15.133:0/205091407 conn(0x55fcd1329280 msgr2=0x55fcd1327c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.644+0000 7f8f2538c740 1 -- 172.21.15.133:0/205091407 shutdown_connections 2024-09-11T23:40:57.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.644+0000 7f8f2538c740 1 -- 172.21.15.133:0/205091407 wait complete. 2024-09-11T23:40:57.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.645+0000 7f8f2538c740 1 -- start start 2024-09-11T23:40:57.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.645+0000 7f8f2538c740 1 Processor -- start 2024-09-11T23:40:57.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.649+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd132d030 0x55fcd1331620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.649+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1334dc0 0x55fcd1331b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.649+0000 7f8f2538c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1337990 0x55fcd13320a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.650+0000 7f8f1bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1334dc0 0x55fcd1331b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.650+0000 7f8f1bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1334dc0 0x55fcd1331b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42490/0 (socket says 172.21.15.133:42490) 2024-09-11T23:40:57.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.650+0000 7f8f20d4f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd132d030 0x55fcd1331620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.650+0000 7f8f20d4f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd132d030 0x55fcd1331620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48082/0 (socket says 172.21.15.133:48082) 2024-09-11T23:40:57.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.650+0000 7f8f1bfff640 1 -- 172.21.15.133:0/555833956 learned_addr learned my addr 172.21.15.133:0/555833956 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:57.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.651+0000 7f8f21550640 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1337990 0x55fcd13320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.651+0000 7f8f2538c740 1 -- 172.21.15.133:0/555833956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0000fb50 con 0x55fcd132d030 2024-09-11T23:40:57.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.651+0000 7f8f2538c740 1 -- 172.21.15.133:0/555833956 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0000f480 con 0x55fcd1334dc0 2024-09-11T23:40:57.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.651+0000 7f8f2538c740 1 -- 172.21.15.133:0/555833956 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0000c6c0 con 0x55fcd1337990 2024-09-11T23:40:57.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.651+0000 7f8f2538c740 10 client.0 _ll_register_callbacks cb 0x7f8f10004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:57.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.651+0000 7f8f1bfff640 1 -- 172.21.15.133:0/555833956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1337990 msgr2=0x55fcd13320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.651+0000 7f8f1bfff640 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1337990 0x55fcd13320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.651+0000 7f8f1bfff640 1 -- 172.21.15.133:0/555833956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd132d030 msgr2=0x55fcd1331620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:57.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.651+0000 7f8f1bfff640 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd132d030 0x55fcd1331620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.651+0000 7f8f1bfff640 1 -- 172.21.15.133:0/555833956 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcd13d3f40 con 0x55fcd1334dc0 2024-09-11T23:40:57.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.652+0000 7f8f20d4f640 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd132d030 0x55fcd1331620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-11T23:40:57.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.653+0000 7f8f1bfff640 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1334dc0 0x55fcd1331b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8f1001ab80 tx=0x7f8f100114d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fbf3663461c6e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:57.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.653+0000 7f8f2538c740 1 -- 172.21.15.133:0/555833956 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f14004000 con 0x55fcd1334dc0 2024-09-11T23:40:57.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.653+0000 7f8efbfff640 10 client.26051 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:57.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.653+0000 7f8efbfff640 1 -- 172.21.15.133:0/555833956 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10029040 con 0x55fcd1334dc0 2024-09-11T23:40:57.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.653+0000 7f8efbfff640 1 -- 172.21.15.133:0/555833956 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8f1001ad70 con 0x55fcd1334dc0 2024-09-11T23:40:57.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.653+0000 7f8efbfff640 1 -- 172.21.15.133:0/555833956 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f1001a420 con 0x55fcd1334dc0 2024-09-11T23:40:57.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.654+0000 7f8f2538c740 10 client.26051 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:57.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.654+0000 7f8f2538c740 1 -- 172.21.15.133:0/555833956 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55fcd13d5010 con 0x55fcd1334dc0 2024-09-11T23:40:57.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.658+0000 7f8efbfff640 1 -- 172.21.15.133:0/555833956 <== mon.2 v2:172.21.15.138:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f10022040 con 0x55fcd1334dc0 2024-09-11T23:40:57.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.658+0000 7f8f2538c740 10 client.26051 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:57.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.658+0000 7f8f2538c740 10 client.26051 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:40:57.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.658+0000 7f8f2538c740 20 client.26051 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:57.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.658+0000 7f8f2538c740 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fcd13d56c0 0x55fcd13d7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:57.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.658+0000 7f8f2538c740 4 client.26051 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T23:40:57.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.658+0000 7f8f2538c740 1 -- 172.21.15.133:0/555833956 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x7f8f14004000 con 0x55fcd13d56c0 2024-09-11T23:40:57.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.660+0000 7f8f20d4f640 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fcd13d56c0 0x55fcd13d7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:57.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.666+0000 7f8efbfff640 10 client.26051 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:57.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.666+0000 7f8f20d4f640 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fcd13d56c0 0x55fcd13d7ae0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46fe579fac5d4206 server_cookie=c8b5471bc7ccfed0 in_seq=0 out_seq=0 2024-09-11T23:40:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.962+0000 7f8efbfff640 1 -- 172.21.15.133:0/555833956 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f8f14004000 con 0x55fcd13d56c0 2024-09-11T23:40:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.962+0000 7f8efbfff640 10 client.26051 handle_command_reply: tid=0 2024-09-11T23:40:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.962+0000 7f8f2538c740 1 client.26051 shutdown 2024-09-11T23:40:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.963+0000 7f8f2538c740 1 -- 172.21.15.133:0/555833956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1334dc0 msgr2=0x55fcd1331b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.963+0000 7f8f2538c740 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1334dc0 0x55fcd1331b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8f1001ab80 tx=0x7f8f100114d0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.963+0000 7f8f2538c740 1 -- 172.21.15.133:0/555833956 shutdown_connections 2024-09-11T23:40:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.963+0000 7f8f2538c740 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fcd13d56c0 0x55fcd13d7ae0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.963+0000 7f8f2538c740 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcd1337990 0x55fcd13320a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.963+0000 7f8f2538c740 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcd1334dc0 0x55fcd1331b60 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8f1001ab80 tx=0x7f8f100114d0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.963+0000 7f8f2538c740 1 --2- 172.21.15.133:0/555833956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fcd132d030 0x55fcd1331620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.963+0000 7f8f2538c740 1 -- 172.21.15.133:0/555833956 >> 172.21.15.133:0/555833956 conn(0x55fcd1329280 msgr2=0x55fcd132efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.963+0000 7f8efbfff640 0 client.26051 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.963+0000 7f8f2538c740 1 -- 172.21.15.133:0/555833956 shutdown_connections 2024-09-11T23:40:57.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.963+0000 7f8f2538c740 1 -- 172.21.15.133:0/555833956 wait complete. 2024-09-11T23:40:57.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.964+0000 7f8f2538c740 20 client.26051 trim_cache size 0 max 16384 2024-09-11T23:40:57.964 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:40:57.964 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 28, 2024-09-11T23:40:57.964 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:40:57.964 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 14, 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "c1232923": { 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 14, 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 140.2, 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:40:57.965 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:40:57.966 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:40:57.966 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:40:57.966 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:40:57.966 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:40:57.966 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:40:57.966 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:40:57.966 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:40:57.966 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:40:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.964+0000 7f8f077fe640 1 -- 172.21.15.133:0/1496454179 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f0c0a3070 msgr2=0x7f8f0c0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.964+0000 7f8f077fe640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f0c0a3070 0x7f8f0c0a5530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8f140097e0 tx=0x7f8f1403f040 comp rx=0 tx=0).stop 2024-09-11T23:40:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.964+0000 7f8f077fe640 1 -- 172.21.15.133:0/1496454179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f1c06ea30 msgr2=0x7f8f1c119800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.964+0000 7f8f077fe640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f1c06ea30 0x7f8f1c119800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f0802a9b0 tx=0x7f8f08031ab0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.965+0000 7f8f077fe640 1 -- 172.21.15.133:0/1496454179 shutdown_connections 2024-09-11T23:40:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.965+0000 7f8f077fe640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f0c0a3070 0x7f8f0c0a5530 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.965+0000 7f8f077fe640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f1c11b140 0x7f8f1c112dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.965+0000 7f8f077fe640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f1c110190 0x7f8f1c112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.965+0000 7f8f077fe640 1 --2- 172.21.15.133:0/1496454179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f1c06ea30 0x7f8f1c119800 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.965+0000 7f8f077fe640 1 -- 172.21.15.133:0/1496454179 >> 172.21.15.133:0/1496454179 conn(0x7f8f1c073180 msgr2=0x7f8f1c06d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.965+0000 7f8f077fe640 1 -- 172.21.15.133:0/1496454179 shutdown_connections 2024-09-11T23:40:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:57.965+0000 7f8f077fe640 1 -- 172.21.15.133:0/1496454179 wait complete. 2024-09-11T23:40:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:40:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.971+0000 7f6c3aff5640 1 -- 172.21.15.133:0/1811721885 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602f66b1d70 con 0x5602f67cc660 2024-09-11T23:40:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.971+0000 7f6c3aff5640 10 client.26039 handle_command_reply: tid=0 2024-09-11T23:40:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.971+0000 7f6c602a0740 1 client.26039 shutdown 2024-09-11T23:40:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.971+0000 7f6c602a0740 1 -- 172.21.15.133:0/1811721885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f67465e0 msgr2=0x5602f6754a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.971+0000 7f6c602a0740 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f67465e0 0x5602f6754a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009c80 tx=0x7f6c4c0099b0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.972+0000 7f6c602a0740 1 -- 172.21.15.133:0/1811721885 shutdown_connections 2024-09-11T23:40:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.972+0000 7f6c602a0740 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5602f67cc660 0x5602f67cea80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.972+0000 7f6c602a0740 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602f6750e80 0x5602f6755040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.972+0000 7f6c602a0740 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f67465e0 0x5602f6754a50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.972+0000 7f6c602a0740 1 --2- 172.21.15.133:0/1811721885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602f6745cc0 0x5602f6759730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.972+0000 7f6c602a0740 1 -- 172.21.15.133:0/1811721885 >> 172.21.15.133:0/1811721885 conn(0x5602f6742230 msgr2=0x5602f6740a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:40:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:40:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:40:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:40:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.972+0000 7f6c3aff5640 0 client.26039 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.972+0000 7f6c602a0740 1 -- 172.21.15.133:0/1811721885 shutdown_connections 2024-09-11T23:40:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.972+0000 7f6c602a0740 1 -- 172.21.15.133:0/1811721885 wait complete. 2024-09-11T23:40:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.972+0000 7f6c602a0740 20 client.26039 trim_cache size 0 max 16384 2024-09-11T23:40:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 -- 172.21.15.133:0/3788804765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c400a2ec0 msgr2=0x7f6c400a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c400a2ec0 0x7f6c400a5380 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x5602f6743090 tx=0x7f6c4802a340 comp rx=0 tx=0).stop 2024-09-11T23:40:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 -- 172.21.15.133:0/3788804765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c581091f0 msgr2=0x7f6c581187c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c581091f0 0x7f6c581187c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009980 tx=0x7f6c48005d80 comp rx=0 tx=0).stop 2024-09-11T23:40:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 -- 172.21.15.133:0/3788804765 shutdown_connections 2024-09-11T23:40:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c400a2ec0 0x7f6c400a5380 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c5811b140 0x7f6c58111dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c581095d0 0x7f6c58111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 --2- 172.21.15.133:0/3788804765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c581091f0 0x7f6c581187c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 -- 172.21.15.133:0/3788804765 >> 172.21.15.133:0/3788804765 conn(0x7f6c5806cc60 msgr2=0x7f6c5811b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 -- 172.21.15.133:0/3788804765 shutdown_connections 2024-09-11T23:40:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:40:57.973+0000 7f6c5d50a640 1 -- 172.21.15.133:0/3788804765 wait complete. 2024-09-11T23:40:58.643 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:58 smithi133 ceph-mon[31430]: pgmap v128: 97 pgs: 97 active+clean; 30 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 650 MiB/s wr, 1.24k op/s 2024-09-11T23:40:58.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:58 smithi138 ceph-mon[35142]: pgmap v128: 97 pgs: 97 active+clean; 30 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 650 MiB/s wr, 1.24k op/s 2024-09-11T23:40:58.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:58 smithi178 ceph-mon[35882]: pgmap v128: 97 pgs: 97 active+clean; 30 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 650 MiB/s wr, 1.24k op/s 2024-09-11T23:40:59.222 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c1232923', quiesce took 10.9 seconds. Will release after: 9.249999999999998 2024-09-11T23:40:59.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.402+0000 7ff098570640 1 -- 172.21.15.133:0/3846187039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0901036e0 msgr2=0x7ff090107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:59.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.402+0000 7ff098570640 1 --2- 172.21.15.133:0/3846187039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0901036e0 0x7ff090107b90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff080009930 tx=0x7ff08002a420 comp rx=0 tx=0).stop 2024-09-11T23:40:59.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.403+0000 7ff098570640 1 -- 172.21.15.133:0/3846187039 shutdown_connections 2024-09-11T23:40:59.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.403+0000 7ff098570640 1 --2- 172.21.15.133:0/3846187039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0901082a0 0x7ff09010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.403+0000 7ff098570640 1 --2- 172.21.15.133:0/3846187039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0901036e0 0x7ff090107b90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.403+0000 7ff098570640 1 --2- 172.21.15.133:0/3846187039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff090102d30 0x7ff090103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.403+0000 7ff098570640 1 -- 172.21.15.133:0/3846187039 >> 172.21.15.133:0/3846187039 conn(0x7ff0900fc7d0 msgr2=0x7ff0900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:59.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.403+0000 7ff098570640 1 -- 172.21.15.133:0/3846187039 shutdown_connections 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.403+0000 7ff098570640 1 -- 172.21.15.133:0/3846187039 wait complete. 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.403+0000 7ff098570640 1 Processor -- start 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.403+0000 7ff098570640 1 -- start start 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.403+0000 7ff098570640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff090102d30 0x7ff090100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.403+0000 7ff098570640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0901036e0 0x7ff090100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff098570640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0901082a0 0x7ff090101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff098570640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901120a0 con 0x7ff090102d30 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff098570640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff090111f20 con 0x7ff0901082a0 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff098570640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff090112220 con 0x7ff0901036e0 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff09630c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff090102d30 0x7ff090100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff09630c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff090102d30 0x7ff090100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48108/0 (socket says 172.21.15.133:48108) 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff09630c640 1 -- 172.21.15.133:0/1629278596 learned_addr learned my addr 172.21.15.133:0/1629278596 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:59.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff095b0b640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0901036e0 0x7ff090100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff096b0d640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0901082a0 0x7ff090101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff09630c640 1 -- 172.21.15.133:0/1629278596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0901036e0 msgr2=0x7ff090100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff09630c640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0901036e0 0x7ff090100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff09630c640 1 -- 172.21.15.133:0/1629278596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0901082a0 msgr2=0x7ff090101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff09630c640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0901082a0 0x7ff090101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff09630c640 1 -- 172.21.15.133:0/1629278596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0901a6b00 con 0x7ff090102d30 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff09630c640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff090102d30 0x7ff090100990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00d950 tx=0x7ff08c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff0877fe640 1 -- 172.21.15.133:0/1629278596 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c017070 con 0x7ff090102d30 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff0877fe640 1 -- 172.21.15.133:0/1629278596 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c009c30 con 0x7ff090102d30 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff098570640 1 -- 172.21.15.133:0/1629278596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0901a6df0 con 0x7ff090102d30 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff0877fe640 1 -- 172.21.15.133:0/1629278596 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00f6b0 con 0x7ff090102d30 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.404+0000 7ff098570640 1 -- 172.21.15.133:0/1629278596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0901a7250 con 0x7ff090102d30 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.407+0000 7ff0877fe640 1 -- 172.21.15.133:0/1629278596 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff08c0109d0 con 0x7ff090102d30 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.407+0000 7ff09a949740 1 Processor -- start 2024-09-11T23:40:59.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09a949740 1 -- start start 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09a949740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 0x560b0b4fc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09a949740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 0x560b0b4fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09a949740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b507370 0x560b0b509760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09a949740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b0b4622f0 con 0x560b0b4fc5d0 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09a949740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560b0b409cf0 con 0x560b0b507370 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09a949740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560b0b45e2b0 con 0x560b0b4fdb40 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09630c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 0x560b0b4fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff096b0d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 0x560b0b4fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09630c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 0x560b0b4fca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48114/0 (socket says 172.21.15.133:48114) 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff096b0d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 0x560b0b4fc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33208/0 (socket says 172.21.15.133:33208) 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09630c640 1 -- 172.21.15.133:0/849735036 learned_addr learned my addr 172.21.15.133:0/849735036 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:59.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff095b0b640 1 --2- 172.21.15.133:0/849735036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b507370 0x560b0b509760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff0877fe640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff07c0a2cf0 0x7ff07c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09630c640 1 -- 172.21.15.133:0/849735036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 msgr2=0x560b0b4fc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09630c640 1 --2- 172.21.15.133:0/849735036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 0x560b0b4fc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09630c640 1 -- 172.21.15.133:0/849735036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b507370 msgr2=0x560b0b509760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09630c640 1 --2- 172.21.15.133:0/849735036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b507370 0x560b0b509760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff09630c640 1 -- 172.21.15.133:0/849735036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b0b3ee3a0 con 0x560b0b4fc5d0 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff096b0d640 1 --2- 172.21.15.133:0/849735036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 0x560b0b4fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.408+0000 7ff096b0d640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff07c0a2cf0 0x7ff07c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.409+0000 7ff0877fe640 1 -- 172.21.15.133:0/1629278596 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff08c0cf010 con 0x7ff090102d30 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.409+0000 7ff09630c640 1 --2- 172.21.15.133:0/849735036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 0x560b0b4fca10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0cc740 tx=0x7ff08c0661d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f5d8896ee0a92e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.409+0000 7ff084ff9640 1 -- 172.21.15.133:0/849735036 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c020070 con 0x560b0b4fc5d0 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.409+0000 7ff084ff9640 1 -- 172.21.15.133:0/849735036 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c09bca0 con 0x560b0b4fc5d0 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.409+0000 7ff084ff9640 1 -- 172.21.15.133:0/849735036 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c09af00 con 0x560b0b4fc5d0 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.409+0000 7ff096b0d640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff07c0a2cf0 0x7ff07c0a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff088009fd0 tx=0x7ff088004a40 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.410+0000 7ff09a949740 1 -- 172.21.15.133:0/849735036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 msgr2=0x560b0b4fca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.410+0000 7ff09a949740 1 --2- 172.21.15.133:0/849735036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 0x560b0b4fca10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0cc740 tx=0x7ff08c0661d0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.410+0000 7ff09a949740 1 -- 172.21.15.133:0/849735036 shutdown_connections 2024-09-11T23:40:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.410+0000 7ff09a949740 1 --2- 172.21.15.133:0/849735036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b507370 0x560b0b509760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.410+0000 7ff09a949740 1 --2- 172.21.15.133:0/849735036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 0x560b0b4fca10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.410+0000 7ff09a949740 1 --2- 172.21.15.133:0/849735036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 0x560b0b4fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.410+0000 7ff09a949740 1 -- 172.21.15.133:0/849735036 >> 172.21.15.133:0/849735036 conn(0x560b0b4c5310 msgr2=0x560b0b4f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.410+0000 7ff09a949740 1 -- 172.21.15.133:0/849735036 shutdown_connections 2024-09-11T23:40:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.410+0000 7ff09a949740 1 -- 172.21.15.133:0/849735036 wait complete. 2024-09-11T23:40:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.410+0000 7ff09a949740 1 -- start start 2024-09-11T23:40:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.410+0000 7ff09a949740 1 Processor -- start 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff09a949740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fc5d0 0x560b0b506d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff09a949740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 0x560b0b504000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff09a949740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 0x560b0b5045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff09a949740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b0b509fb0 con 0x560b0b507370 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff09a949740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560b0b41d060 con 0x560b0b4fdb40 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff09a949740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560b0b50a130 con 0x560b0b4fc5d0 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff09a949740 10 client.0 _ll_register_callbacks cb 0x560b0b1708c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff095b0b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 0x560b0b504000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff096b0d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 0x560b0b5045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff095b0b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 0x560b0b504000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42516/0 (socket says 172.21.15.133:42516) 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff095b0b640 1 -- 172.21.15.133:0/2412818631 learned_addr learned my addr 172.21.15.133:0/2412818631 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff09630c640 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fc5d0 0x560b0b506d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff095b0b640 1 -- 172.21.15.133:0/2412818631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fc5d0 msgr2=0x560b0b506d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff095b0b640 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fc5d0 0x560b0b506d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff095b0b640 1 -- 172.21.15.133:0/2412818631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 msgr2=0x560b0b5045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff095b0b640 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 0x560b0b5045d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff095b0b640 1 -- 172.21.15.133:0/2412818631 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b0b1d33c0 con 0x560b0b4fdb40 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff096b0d640 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 0x560b0b5045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff095b0b640 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 0x560b0b504000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff080000c00 tx=0x7ff08002a9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a103dedd862c76e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff079ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.411+0000 7ff079ffb640 1 -- 172.21.15.133:0/2412818631 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080035040 con 0x560b0b4fdb40 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff09a949740 1 -- 172.21.15.133:0/2412818631 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08c09bca0 con 0x560b0b4fdb40 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff079ffb640 1 -- 172.21.15.133:0/2412818631 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff080004040 con 0x560b0b4fdb40 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff079ffb640 1 -- 172.21.15.133:0/2412818631 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0800319f0 con 0x560b0b4fdb40 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff09a949740 10 client.26063 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff09a949740 1 -- 172.21.15.133:0/2412818631 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b0b515a80 con 0x560b0b4fdb40 2024-09-11T23:40:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff079ffb640 1 -- 172.21.15.133:0/2412818631 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c09bca0 con 0x560b0b4fdb40 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff079ffb640 1 -- 172.21.15.133:0/2412818631 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff08004e050 con 0x560b0b4fdb40 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff09a949740 10 client.26063 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff09a949740 10 client.26063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff09a949740 20 client.26063 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff09a949740 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560b0b516e90 0x560b0b5192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff09a949740 4 client.26063 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff09a949740 1 -- 172.21.15.133:0/2412818631 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b0b46f860 con 0x560b0b516e90 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.412+0000 7ff09630c640 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560b0b516e90 0x560b0b5192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff09630c640 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560b0b516e90 0x560b0b5192d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf3d95a26b996cb server_cookie=7e1be6edbb3e5466 in_seq=0 out_seq=0 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff079ffb640 10 client.26063 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff079ffb640 1 -- 172.21.15.133:0/2412818631 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b0b46f860 con 0x560b0b516e90 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff079ffb640 10 client.26063 handle_command_reply: tid=0 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff09a949740 1 client.26063 shutdown 2024-09-11T23:40:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff09a949740 1 -- 172.21.15.133:0/2412818631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 msgr2=0x560b0b504000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff09a949740 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 0x560b0b504000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff080000c00 tx=0x7ff08002a9c0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff09a949740 1 -- 172.21.15.133:0/2412818631 shutdown_connections 2024-09-11T23:40:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff09a949740 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560b0b516e90 0x560b0b5192d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff09a949740 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 0x560b0b5045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff09a949740 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 0x560b0b504000 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff079ffb640 0 client.26063 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff09a949740 1 --2- 172.21.15.133:0/2412818631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fc5d0 0x560b0b506d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.413+0000 7ff09a949740 1 -- 172.21.15.133:0/2412818631 >> 172.21.15.133:0/2412818631 conn(0x560b0b4c5310 msgr2=0x560b0b4fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.414+0000 7ff09a949740 1 -- 172.21.15.133:0/2412818631 shutdown_connections 2024-09-11T23:40:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.414+0000 7ff09a949740 1 -- 172.21.15.133:0/2412818631 wait complete. 2024-09-11T23:40:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.414+0000 7ff09a949740 20 client.26063 trim_cache size 0 max 16384 2024-09-11T23:40:59.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff09a949740 1 Processor -- start 2024-09-11T23:40:59.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff09a949740 1 -- start start 2024-09-11T23:40:59.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff09a949740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fc5d0 0x560b0b513250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff09a949740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 0x560b0b502fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff09a949740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 0x560b0b51eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff09a949740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901af960 con 0x560b0b507370 2024-09-11T23:40:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff09a949740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901b14a0 con 0x560b0b4fdb40 2024-09-11T23:40:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff09a949740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901b0030 con 0x560b0b4fc5d0 2024-09-11T23:40:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff096b0d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 0x560b0b51eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff096b0d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 0x560b0b51eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48126/0 (socket says 172.21.15.133:48126) 2024-09-11T23:40:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff096b0d640 1 -- 172.21.15.133:0/3786605191 learned_addr learned my addr 172.21.15.133:0/3786605191 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff09630c640 1 --2- 172.21.15.133:0/3786605191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fc5d0 0x560b0b513250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff095b0b640 1 --2- 172.21.15.133:0/3786605191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 0x560b0b502fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff096b0d640 1 -- 172.21.15.133:0/3786605191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fc5d0 msgr2=0x560b0b513250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff096b0d640 1 --2- 172.21.15.133:0/3786605191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fc5d0 0x560b0b513250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff096b0d640 1 -- 172.21.15.133:0/3786605191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 msgr2=0x560b0b502fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:59.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff096b0d640 1 --2- 172.21.15.133:0/3786605191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 0x560b0b502fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff096b0d640 1 -- 172.21.15.133:0/3786605191 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b0b519950 con 0x560b0b507370 2024-09-11T23:40:59.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.427+0000 7ff096b0d640 1 --2- 172.21.15.133:0/3786605191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 0x560b0b51eba0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff0880109d0 tx=0x7ff088016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ae663516ba180b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:59.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff07bfff640 1 -- 172.21.15.133:0/3786605191 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088011940 con 0x560b0b507370 2024-09-11T23:40:59.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff07bfff640 1 -- 172.21.15.133:0/3786605191 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08801b400 con 0x560b0b507370 2024-09-11T23:40:59.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff07bfff640 1 -- 172.21.15.133:0/3786605191 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08801e3f0 con 0x560b0b507370 2024-09-11T23:40:59.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff09a949740 1 -- 172.21.15.133:0/3786605191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 msgr2=0x560b0b51eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:59.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff09a949740 1 --2- 172.21.15.133:0/3786605191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 0x560b0b51eba0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff0880109d0 tx=0x7ff088016000 comp rx=0 tx=0).stop 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff09a949740 1 -- 172.21.15.133:0/3786605191 shutdown_connections 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff09a949740 1 --2- 172.21.15.133:0/3786605191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b507370 0x560b0b51eba0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff09a949740 1 --2- 172.21.15.133:0/3786605191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b4fdb40 0x560b0b502fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff09a949740 1 --2- 172.21.15.133:0/3786605191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fc5d0 0x560b0b513250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff09a949740 1 -- 172.21.15.133:0/3786605191 >> 172.21.15.133:0/3786605191 conn(0x560b0b4c5310 msgr2=0x560b0b5028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff09a949740 1 -- 172.21.15.133:0/3786605191 shutdown_connections 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.428+0000 7ff09a949740 1 -- 172.21.15.133:0/3786605191 wait complete. 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09a949740 1 -- start start 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09a949740 1 Processor -- start 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09a949740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 0x560b0b501160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09a949740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 0x560b0b5016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09a949740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b507370 0x560b0b501be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09a949740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901b0dd0 con 0x560b0b4fc5d0 2024-09-11T23:40:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09a949740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901b0700 con 0x560b0b507370 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09a949740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901ad960 con 0x560b0b4fdb40 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09a949740 10 client.0 _ll_register_callbacks cb 0x7ff080009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09630c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 0x560b0b501160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09630c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 0x560b0b501160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48128/0 (socket says 172.21.15.133:48128) 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff09630c640 1 -- 172.21.15.133:0/689144806 learned_addr learned my addr 172.21.15.133:0/689144806 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff096b0d640 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b507370 0x560b0b501be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.429+0000 7ff095b0b640 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 0x560b0b5016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff096b0d640 1 -- 172.21.15.133:0/689144806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 msgr2=0x560b0b5016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff096b0d640 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 0x560b0b5016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff096b0d640 1 -- 172.21.15.133:0/689144806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 msgr2=0x560b0b501160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff096b0d640 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 0x560b0b501160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff096b0d640 1 -- 172.21.15.133:0/689144806 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b0b580c20 con 0x560b0b507370 2024-09-11T23:40:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff095b0b640 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 0x560b0b5016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff096b0d640 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b507370 0x560b0b501be0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff088011810 tx=0x7ff08801bb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0ee93101a3a64e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff079ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff079ffb640 1 -- 172.21.15.133:0/689144806 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088005b30 con 0x560b0b507370 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff09a949740 1 -- 172.21.15.133:0/689144806 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08801b400 con 0x560b0b507370 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff079ffb640 1 -- 172.21.15.133:0/689144806 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff08802d040 con 0x560b0b507370 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff079ffb640 1 -- 172.21.15.133:0/689144806 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff088011b50 con 0x560b0b507370 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff09a949740 10 client.26069 fetch_fsmap learned FSMap version 28 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff09a949740 1 -- 172.21.15.133:0/689144806 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b0b581f00 con 0x560b0b507370 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff079ffb640 1 -- 172.21.15.133:0/689144806 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08801b400 con 0x560b0b507370 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff079ffb640 1 -- 172.21.15.133:0/689144806 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff088039030 con 0x560b0b507370 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff09a949740 10 client.26069 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff09a949740 10 client.26069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:40:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff09a949740 20 client.26069 populate_metadata read hostname 'smithi133' 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.430+0000 7ff09a949740 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560b0b582620 0x560b0b584a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff09a949740 4 client.26069 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff09a949740 1 -- 172.21.15.133:0/689144806 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b0b469d70 con 0x560b0b582620 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff09630c640 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560b0b582620 0x560b0b584a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff09630c640 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560b0b582620 0x560b0b584a40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4efb068f4918023 server_cookie=ca05bb3ad47a6e1f in_seq=0 out_seq=0 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff079ffb640 10 client.26069 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff079ffb640 1 -- 172.21.15.133:0/689144806 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b0b469d70 con 0x560b0b582620 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff079ffb640 10 client.26069 handle_command_reply: tid=0 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff09a949740 1 client.26069 shutdown 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff09a949740 1 -- 172.21.15.133:0/689144806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b507370 msgr2=0x560b0b501be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff09a949740 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b507370 0x560b0b501be0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff088011810 tx=0x7ff08801bb60 comp rx=0 tx=0).stop 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff09a949740 1 -- 172.21.15.133:0/689144806 shutdown_connections 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.431+0000 7ff09a949740 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560b0b582620 0x560b0b584a40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff09a949740 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b0b507370 0x560b0b501be0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff09a949740 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0b4fdb40 0x560b0b5016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff09a949740 1 --2- 172.21.15.133:0/689144806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b0b4fc5d0 0x560b0b501160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff09a949740 1 -- 172.21.15.133:0/689144806 >> 172.21.15.133:0/689144806 conn(0x560b0b4c5310 msgr2=0x560b0b5030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:59.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff079ffb640 0 client.26069 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:40:59.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff09a949740 1 -- 172.21.15.133:0/689144806 shutdown_connections 2024-09-11T23:40:59.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff09a949740 1 -- 172.21.15.133:0/689144806 wait complete. 2024-09-11T23:40:59.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff09a949740 20 client.26069 trim_cache size 0 max 16384 2024-09-11T23:40:59.442 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:40:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff098570640 1 -- 172.21.15.133:0/1629278596 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff07c0a2cf0 msgr2=0x7ff07c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff098570640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff07c0a2cf0 0x7ff07c0a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff088009fd0 tx=0x7ff088004a40 comp rx=0 tx=0).stop 2024-09-11T23:40:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff098570640 1 -- 172.21.15.133:0/1629278596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff090102d30 msgr2=0x7ff090100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:40:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff098570640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff090102d30 0x7ff090100990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00d950 tx=0x7ff08c00de10 comp rx=0 tx=0).stop 2024-09-11T23:40:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff098570640 1 -- 172.21.15.133:0/1629278596 shutdown_connections 2024-09-11T23:40:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff098570640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff07c0a2cf0 0x7ff07c0a51b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff098570640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0901082a0 0x7ff090101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff098570640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0901036e0 0x7ff090100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff098570640 1 --2- 172.21.15.133:0/1629278596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff090102d30 0x7ff090100990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:40:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff098570640 1 -- 172.21.15.133:0/1629278596 >> 172.21.15.133:0/1629278596 conn(0x7ff0900fc7d0 msgr2=0x7ff090106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:40:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.432+0000 7ff098570640 1 -- 172.21.15.133:0/1629278596 shutdown_connections 2024-09-11T23:40:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:40:59.433+0000 7ff098570640 1 -- 172.21.15.133:0/1629278596 wait complete. 2024-09-11T23:40:59.444 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:40:59 smithi133 ceph-mon[31430]: pgmap v129: 97 pgs: 97 active+clean; 30 GiB data, 138 GiB used, 935 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 838 MiB/s wr, 1.46k op/s 2024-09-11T23:40:59.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:40:59 smithi138 ceph-mon[35142]: pgmap v129: 97 pgs: 97 active+clean; 30 GiB data, 138 GiB used, 935 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 838 MiB/s wr, 1.46k op/s 2024-09-11T23:40:59.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:40:59 smithi178 ceph-mon[35882]: pgmap v129: 97 pgs: 97 active+clean; 30 GiB data, 138 GiB used, 935 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 838 MiB/s wr, 1.46k op/s 2024-09-11T23:41:00.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:00 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:41:00.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:00 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:41:01.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:00 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:41:01.224 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:41:01.225 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:41:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:01.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:01 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:41:01.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:01 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:41:01.869 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:01 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:41:01.869 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:01 smithi133 ceph-mon[31430]: pgmap v130: 97 pgs: 97 active+clean; 30 GiB data, 138 GiB used, 934 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 654 MiB/s wr, 1.21k op/s 2024-09-11T23:41:01.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:01 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:41:01.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:01 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:41:01.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:01 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:41:01.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:01 smithi138 ceph-mon[35142]: pgmap v130: 97 pgs: 97 active+clean; 30 GiB data, 138 GiB used, 934 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 654 MiB/s wr, 1.21k op/s 2024-09-11T23:41:01.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.955+0000 7f4a2f256640 1 -- 172.21.15.133:0/3292396373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a28102790 msgr2=0x7f4a281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:01.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.955+0000 7f4a2f256640 1 --2- 172.21.15.133:0/3292396373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a28102790 0x7f4a281067e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009860 tx=0x7f4a1402a340 comp rx=0 tx=0).stop 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.955+0000 7f4a2f256640 1 -- 172.21.15.133:0/3292396373 shutdown_connections 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.955+0000 7f4a2f256640 1 --2- 172.21.15.133:0/3292396373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a28106f10 0x7f4a2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.955+0000 7f4a2f256640 1 --2- 172.21.15.133:0/3292396373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a28102790 0x7f4a281067e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.955+0000 7f4a2f256640 1 --2- 172.21.15.133:0/3292396373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a28101de0 0x7f4a281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.955+0000 7f4a2f256640 1 -- 172.21.15.133:0/3292396373 >> 172.21.15.133:0/3292396373 conn(0x7f4a280fd8c0 msgr2=0x7f4a280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.955+0000 7f4a2f256640 1 -- 172.21.15.133:0/3292396373 shutdown_connections 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.955+0000 7f4a2f256640 1 -- 172.21.15.133:0/3292396373 wait complete. 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2f256640 1 Processor -- start 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2f256640 1 -- start start 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2f256640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a28101de0 0x7f4a280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2f256640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a28102790 0x7f4a28074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2f256640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a28106f10 0x7f4a28074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2f256640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28111f70 con 0x7f4a28106f10 2024-09-11T23:41:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2f256640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28111df0 con 0x7f4a28101de0 2024-09-11T23:41:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2f256640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281120f0 con 0x7f4a28102790 2024-09-11T23:41:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2d7f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a28106f10 0x7f4a28074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2d7f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a28106f10 0x7f4a28074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48152/0 (socket says 172.21.15.133:48152) 2024-09-11T23:41:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2d7f3640 1 -- 172.21.15.133:0/4247694534 learned_addr learned my addr 172.21.15.133:0/4247694534 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2cff2640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a28101de0 0x7f4a280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a27fff640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a28102790 0x7f4a28074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2d7f3640 1 -- 172.21.15.133:0/4247694534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a28102790 msgr2=0x7f4a28074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2d7f3640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a28102790 0x7f4a28074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2d7f3640 1 -- 172.21.15.133:0/4247694534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a28101de0 msgr2=0x7f4a280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2d7f3640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a28101de0 0x7f4a280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2d7f3640 1 -- 172.21.15.133:0/4247694534 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a281a6930 con 0x7f4a28106f10 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.956+0000 7f4a2d7f3640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a28106f10 0x7f4a28074f70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000efc0 tx=0x7f4a2000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.957+0000 7f4a25ffb640 1 -- 172.21.15.133:0/4247694534 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20003970 con 0x7f4a28106f10 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.957+0000 7f4a25ffb640 1 -- 172.21.15.133:0/4247694534 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a20004020 con 0x7f4a28106f10 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.957+0000 7f4a2f256640 1 -- 172.21.15.133:0/4247694534 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a281a6c20 con 0x7f4a28106f10 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.957+0000 7f4a25ffb640 1 -- 172.21.15.133:0/4247694534 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000b9f0 con 0x7f4a28106f10 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.957+0000 7f4a2f256640 1 -- 172.21.15.133:0/4247694534 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a281a70e0 con 0x7f4a28106f10 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.959+0000 7f4a25ffb640 1 -- 172.21.15.133:0/4247694534 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4a2001d020 con 0x7f4a28106f10 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a3162f740 1 Processor -- start 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a3162f740 1 -- start start 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 0x55cada768f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada773800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada773d40 0x55cada776130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a3162f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cada6ce280 con 0x55cada769480 2024-09-11T23:41:01.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a3162f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cada675b10 con 0x55cada768b60 2024-09-11T23:41:01.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a3162f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cada6ca240 con 0x55cada773d40 2024-09-11T23:41:01.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a2cff2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada773800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a25ffb640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4a180a2cf0 0x7f4a180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a2cff2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada773800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48168/0 (socket says 172.21.15.133:48168) 2024-09-11T23:41:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a2cff2640 1 -- 172.21.15.133:0/10401274 learned_addr learned my addr 172.21.15.133:0/10401274 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a27fff640 1 --2- 172.21.15.133:0/10401274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada773d40 0x55cada776130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a2d7f3640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4a180a2cf0 0x7f4a180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a27fff640 1 --2- 172.21.15.133:0/10401274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 0x55cada768f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a2cff2640 1 -- 172.21.15.133:0/10401274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada773d40 msgr2=0x55cada776130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a2cff2640 1 --2- 172.21.15.133:0/10401274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada773d40 0x55cada776130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a2cff2640 1 -- 172.21.15.133:0/10401274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 msgr2=0x55cada768f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.960+0000 7f4a2cff2640 1 --2- 172.21.15.133:0/10401274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 0x55cada768f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.961+0000 7f4a2cff2640 1 -- 172.21.15.133:0/10401274 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cada6821f0 con 0x55cada769480 2024-09-11T23:41:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.961+0000 7f4a27fff640 1 --2- 172.21.15.133:0/10401274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 0x55cada768f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.961+0000 7f4a25ffb640 1 -- 172.21.15.133:0/4247694534 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f4a200cf300 con 0x7f4a28106f10 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.961+0000 7f4a2cff2640 1 --2- 172.21.15.133:0/10401274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada773800 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009d20 tx=0x7f4a1c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dc73384eb2aef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.961+0000 7f4a2d7f3640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4a180a2cf0 0x7f4a180a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a20097c90 tx=0x7f4a2006e8a0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.961+0000 7f4a12ffd640 1 -- 172.21.15.133:0/10401274 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c01e070 con 0x55cada769480 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.961+0000 7f4a12ffd640 1 -- 172.21.15.133:0/10401274 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a1c019b10 con 0x55cada769480 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.961+0000 7f4a12ffd640 1 -- 172.21.15.133:0/10401274 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00eb80 con 0x55cada769480 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 -- 172.21.15.133:0/10401274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 msgr2=0x55cada773800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 --2- 172.21.15.133:0/10401274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada773800 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009d20 tx=0x7f4a1c011810 comp rx=0 tx=0).stop 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 -- 172.21.15.133:0/10401274 shutdown_connections 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 --2- 172.21.15.133:0/10401274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada773d40 0x55cada776130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 --2- 172.21.15.133:0/10401274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada773800 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 --2- 172.21.15.133:0/10401274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 0x55cada768f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 -- 172.21.15.133:0/10401274 >> 172.21.15.133:0/10401274 conn(0x55cada7650f0 msgr2=0x55cada7654d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 -- 172.21.15.133:0/10401274 shutdown_connections 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 -- 172.21.15.133:0/10401274 wait complete. 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 -- start start 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 Processor -- start 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 0x55cada770ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.962+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada7710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada773d40 0x55cada771620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a3162f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cada776910 con 0x55cada769480 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a3162f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cada688f10 con 0x55cada768b60 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a3162f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cada776a90 con 0x55cada773d40 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a3162f740 10 client.0 _ll_register_callbacks cb 0x55cada651a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a2cff2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada7710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a2cff2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada7710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48182/0 (socket says 172.21.15.133:48182) 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a2cff2640 1 -- 172.21.15.133:0/411373954 learned_addr learned my addr 172.21.15.133:0/411373954 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a27fff640 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 0x55cada770ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a27fff640 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada773d40 0x55cada771620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a2cff2640 1 -- 172.21.15.133:0/411373954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada773d40 msgr2=0x55cada771620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a2cff2640 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada773d40 0x55cada771620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a2cff2640 1 -- 172.21.15.133:0/411373954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 msgr2=0x55cada770ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a2cff2640 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 0x55cada770ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a2cff2640 1 -- 172.21.15.133:0/411373954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cada43f6a0 con 0x55cada769480 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a27fff640 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 0x55cada770ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a2cff2640 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada7710e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00e720 tx=0x7f4a1c019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8509aae9e7433800 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a07fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a07fff640 1 -- 172.21.15.133:0/411373954 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c01e070 con 0x55cada769480 2024-09-11T23:41:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a07fff640 1 -- 172.21.15.133:0/411373954 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f4a1c005250 con 0x55cada769480 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.963+0000 7f4a3162f740 1 -- 172.21.15.133:0/411373954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a1c019b10 con 0x55cada769480 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a07fff640 1 -- 172.21.15.133:0/411373954 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a1c031050 con 0x55cada769480 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a3162f740 10 client.16626 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a3162f740 1 -- 172.21.15.133:0/411373954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cada7824a0 con 0x55cada769480 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a07fff640 1 -- 172.21.15.133:0/411373954 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a1c019b10 con 0x55cada769480 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a07fff640 1 -- 172.21.15.133:0/411373954 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a1c0046c0 con 0x55cada769480 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a3162f740 10 client.16626 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a3162f740 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a3162f740 20 client.16626 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a3162f740 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cada783960 0x55cada785d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a3162f740 4 client.16626 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a3162f740 1 -- 172.21.15.133:0/411373954 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cada6db7f0 con 0x55cada783960 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.964+0000 7f4a27fff640 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cada783960 0x55cada785d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.965+0000 7f4a27fff640 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cada783960 0x55cada785d50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9165288c4afffecc server_cookie=5bb514412bb6c41a in_seq=0 out_seq=0 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.965+0000 7f4a07fff640 10 client.16626 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.965+0000 7f4a07fff640 1 -- 172.21.15.133:0/411373954 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cada6db7f0 con 0x55cada783960 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.965+0000 7f4a07fff640 10 client.16626 handle_command_reply: tid=0 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.965+0000 7f4a3162f740 1 client.16626 shutdown 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a3162f740 1 -- 172.21.15.133:0/411373954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 msgr2=0x55cada7710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a3162f740 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada7710e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00e720 tx=0x7f4a1c019660 comp rx=0 tx=0).stop 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a3162f740 1 -- 172.21.15.133:0/411373954 shutdown_connections 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a3162f740 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cada783960 0x55cada785d50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a3162f740 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada773d40 0x55cada771620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a07fff640 0 client.16626 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a3162f740 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada769480 0x55cada7710e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a3162f740 1 --2- 172.21.15.133:0/411373954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada768b60 0x55cada770ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a3162f740 1 -- 172.21.15.133:0/411373954 >> 172.21.15.133:0/411373954 conn(0x55cada7650f0 msgr2=0x55cada764270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:01.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a3162f740 1 -- 172.21.15.133:0/411373954 shutdown_connections 2024-09-11T23:41:01.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a3162f740 1 -- 172.21.15.133:0/411373954 wait complete. 2024-09-11T23:41:01.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.966+0000 7f4a3162f740 20 client.16626 trim_cache size 0 max 16384 2024-09-11T23:41:01.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a3162f740 1 Processor -- start 2024-09-11T23:41:01.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a3162f740 1 -- start start 2024-09-11T23:41:01.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada78b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada769480 0x55cada78b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada773d40 0x55cada76a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a3162f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281af8e0 con 0x55cada768b60 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a3162f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281b1420 con 0x55cada773d40 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a3162f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281affb0 con 0x55cada769480 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada78b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada78b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48186/0 (socket says 172.21.15.133:48186) 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a27fff640 1 -- 172.21.15.133:0/3266715293 learned_addr learned my addr 172.21.15.133:0/3266715293 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a2cff2640 1 --2- 172.21.15.133:0/3266715293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada769480 0x55cada78b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a27fff640 1 --2- 172.21.15.133:0/3266715293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada773d40 0x55cada76a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a27fff640 1 -- 172.21.15.133:0/3266715293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada769480 msgr2=0x55cada78b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a27fff640 1 --2- 172.21.15.133:0/3266715293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada769480 0x55cada78b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a27fff640 1 -- 172.21.15.133:0/3266715293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada773d40 msgr2=0x55cada76a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.984+0000 7f4a27fff640 1 --2- 172.21.15.133:0/3266715293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada773d40 0x55cada76a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.985+0000 7f4a27fff640 1 -- 172.21.15.133:0/3266715293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cada764f20 con 0x55cada768b60 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.985+0000 7f4a27fff640 1 --2- 172.21.15.133:0/3266715293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada78b490 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009250 tx=0x7f4a140095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8435cd813fb8bd72 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.985+0000 7f4a127fc640 1 -- 172.21.15.133:0/3266715293 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1404f070 con 0x55cada768b60 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.985+0000 7f4a127fc640 1 -- 172.21.15.133:0/3266715293 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a140035f0 con 0x55cada768b60 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.985+0000 7f4a127fc640 1 -- 172.21.15.133:0/3266715293 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14037040 con 0x55cada768b60 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.986+0000 7f4a3162f740 1 -- 172.21.15.133:0/3266715293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 msgr2=0x55cada78b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.986+0000 7f4a3162f740 1 --2- 172.21.15.133:0/3266715293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada78b490 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009250 tx=0x7f4a140095b0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.986+0000 7f4a3162f740 1 -- 172.21.15.133:0/3266715293 shutdown_connections 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.986+0000 7f4a3162f740 1 --2- 172.21.15.133:0/3266715293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada773d40 0x55cada76a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.986+0000 7f4a3162f740 1 --2- 172.21.15.133:0/3266715293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada769480 0x55cada78b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.986+0000 7f4a3162f740 1 --2- 172.21.15.133:0/3266715293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada78b490 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.986+0000 7f4a3162f740 1 -- 172.21.15.133:0/3266715293 >> 172.21.15.133:0/3266715293 conn(0x55cada7650f0 msgr2=0x55cada76fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.986+0000 7f4a3162f740 1 -- 172.21.15.133:0/3266715293 shutdown_connections 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.986+0000 7f4a3162f740 1 -- 172.21.15.133:0/3266715293 wait complete. 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.986+0000 7f4a3162f740 1 -- start start 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.986+0000 7f4a3162f740 1 Processor -- start 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada77c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada769480 0x55cada7776a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a3162f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada773d40 0x55cada777be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a3162f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281b0d50 con 0x55cada768b60 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a3162f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281b0680 con 0x55cada773d40 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a3162f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281ad900 con 0x55cada769480 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a3162f740 10 client.0 _ll_register_callbacks cb 0x7f4a1c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada77c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada77c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48194/0 (socket says 172.21.15.133:48194) 2024-09-11T23:41:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a27fff640 1 -- 172.21.15.133:0/1998564571 learned_addr learned my addr 172.21.15.133:0/1998564571 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a27fff640 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada773d40 0x55cada777be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a2cff2640 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada769480 0x55cada7776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a27fff640 1 -- 172.21.15.133:0/1998564571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada769480 msgr2=0x55cada7776a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a27fff640 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada769480 0x55cada7776a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a27fff640 1 -- 172.21.15.133:0/1998564571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada773d40 msgr2=0x55cada777be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a27fff640 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada773d40 0x55cada777be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a27fff640 1 -- 172.21.15.133:0/1998564571 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cada7783c0 con 0x55cada768b60 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a27fff640 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada77c370 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a1402d020 tx=0x7f4a14003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c64f236fd20d9bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a07fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a07fff640 1 -- 172.21.15.133:0/1998564571 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1405b060 con 0x55cada768b60 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a07fff640 1 -- 172.21.15.133:0/1998564571 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f4a140039a0 con 0x55cada768b60 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.987+0000 7f4a3162f740 1 -- 172.21.15.133:0/1998564571 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a140035f0 con 0x55cada768b60 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a07fff640 1 -- 172.21.15.133:0/1998564571 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a14037030 con 0x55cada768b60 2024-09-11T23:41:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a3162f740 10 client.16632 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a3162f740 1 -- 172.21.15.133:0/1998564571 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cada7ee3f0 con 0x55cada768b60 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a07fff640 1 -- 172.21.15.133:0/1998564571 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a140035f0 con 0x55cada768b60 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a07fff640 1 -- 172.21.15.133:0/1998564571 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a1405e030 con 0x55cada768b60 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a3162f740 10 client.16632 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a3162f740 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a3162f740 20 client.16632 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a3162f740 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cada7eea90 0x55cada7f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a3162f740 4 client.16632 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a3162f740 1 -- 172.21.15.133:0/1998564571 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cada6d5d00 con 0x55cada7eea90 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.988+0000 7f4a2cff2640 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cada7eea90 0x55cada7f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.989+0000 7f4a2cff2640 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cada7eea90 0x55cada7f0eb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ca985cb88b7024 server_cookie=77a76f20c445c585 in_seq=0 out_seq=0 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.989+0000 7f4a07fff640 10 client.16632 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a07fff640 1 -- 172.21.15.133:0/1998564571 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cada6d5d00 con 0x55cada7eea90 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a07fff640 10 client.16632 handle_command_reply: tid=0 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 1 client.16632 shutdown 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 1 -- 172.21.15.133:0/1998564571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 msgr2=0x55cada77c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada77c370 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a1402d020 tx=0x7f4a14003f80 comp rx=0 tx=0).stop 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 1 -- 172.21.15.133:0/1998564571 shutdown_connections 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cada7eea90 0x55cada7f0eb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cada773d40 0x55cada777be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cada769480 0x55cada7776a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a07fff640 0 client.16632 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 1 --2- 172.21.15.133:0/1998564571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cada768b60 0x55cada77c370 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 1 -- 172.21.15.133:0/1998564571 >> 172.21.15.133:0/1998564571 conn(0x55cada7650f0 msgr2=0x55cada763a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 1 -- 172.21.15.133:0/1998564571 shutdown_connections 2024-09-11T23:41:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 1 -- 172.21.15.133:0/1998564571 wait complete. 2024-09-11T23:41:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.990+0000 7f4a3162f740 20 client.16632 trim_cache size 0 max 16384 2024-09-11T23:41:01.999 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:41:01.999 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:41:01.999 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "dc60028c-a513-432c-bf64-ab248f64dfc8", 2024-09-11T23:41:01.999 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:41:01.999 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:41:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 -- 172.21.15.133:0/4247694534 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4a180a2cf0 msgr2=0x7f4a180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4a180a2cf0 0x7f4a180a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a20097c90 tx=0x7f4a2006e8a0 comp rx=0 tx=0).stop 2024-09-11T23:41:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 -- 172.21.15.133:0/4247694534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a28106f10 msgr2=0x7f4a28074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a28106f10 0x7f4a28074f70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000efc0 tx=0x7f4a2000c800 comp rx=0 tx=0).stop 2024-09-11T23:41:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 -- 172.21.15.133:0/4247694534 shutdown_connections 2024-09-11T23:41:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4a180a2cf0 0x7f4a180a51b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a28106f10 0x7f4a28074f70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a28102790 0x7f4a28074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 --2- 172.21.15.133:0/4247694534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a28101de0 0x7f4a280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 -- 172.21.15.133:0/4247694534 >> 172.21.15.133:0/4247694534 conn(0x7f4a280fd8c0 msgr2=0x7f4a280fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 -- 172.21.15.133:0/4247694534 shutdown_connections 2024-09-11T23:41:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:01.991+0000 7f4a2f256640 1 -- 172.21.15.133:0/4247694534 wait complete. 2024-09-11T23:41:02.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:01 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:41:02.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:01 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:41:02.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:01 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:41:02.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:01 smithi178 ceph-mon[35882]: pgmap v130: 97 pgs: 97 active+clean; 30 GiB data, 138 GiB used, 934 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 654 MiB/s wr, 1.21k op/s 2024-09-11T23:41:02.867 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:41:03.119 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:03.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:02 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:41:03.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:02 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:41:03.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:02 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:41:03.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:02 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:41:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:02 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:41:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:02 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:41:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:02 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:41:03.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:02 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:41:03.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:02 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:41:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:02 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:41:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:02 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:41:03.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:02 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:41:03.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 -- 172.21.15.133:0/1371786934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a8108290 msgr2=0x7ff7a810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:03.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 --2- 172.21.15.133:0/1371786934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a8108290 0x7ff7a810ec30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009cd0 tx=0x7ff7a002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 -- 172.21.15.133:0/1371786934 shutdown_connections 2024-09-11T23:41:03.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 --2- 172.21.15.133:0/1371786934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a8108290 0x7ff7a810ec30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 --2- 172.21.15.133:0/1371786934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a8103740 0x7ff7a8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 --2- 172.21.15.133:0/1371786934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a8102d90 0x7ff7a8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 -- 172.21.15.133:0/1371786934 >> 172.21.15.133:0/1371786934 conn(0x7ff7a80fc7d0 msgr2=0x7ff7a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:03.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 -- 172.21.15.133:0/1371786934 shutdown_connections 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 -- 172.21.15.133:0/1371786934 wait complete. 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 Processor -- start 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 -- start start 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a8102d90 0x7ff7a81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.577+0000 7ff7af5b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a8103740 0x7ff7a81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7af5b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a8108290 0x7ff7a819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7af5b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8112090 con 0x7ff7a8103740 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7af5b2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8111f10 con 0x7ff7a8108290 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7af5b2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8112210 con 0x7ff7a8102d90 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7acb4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a8103740 0x7ff7a81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7acb4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a8103740 0x7ff7a81989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33666/0 (socket says 172.21.15.133:33666) 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7acb4d640 1 -- 172.21.15.133:0/2010792540 learned_addr learned my addr 172.21.15.133:0/2010792540 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:03.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7adb4f640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a8108290 0x7ff7a819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7ad34e640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a8102d90 0x7ff7a81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7acb4d640 1 -- 172.21.15.133:0/2010792540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a8102d90 msgr2=0x7ff7a81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a8102d90 0x7ff7a81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7acb4d640 1 -- 172.21.15.133:0/2010792540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a8108290 msgr2=0x7ff7a819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a8108290 0x7ff7a819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7acb4d640 1 -- 172.21.15.133:0/2010792540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a819d450 con 0x7ff7a8103740 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7ad34e640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a8102d90 0x7ff7a81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a8103740 0x7ff7a81989e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e9c0 tx=0x7ff79c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff79a7fc640 1 -- 172.21.15.133:0/2010792540 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c00cd80 con 0x7ff7a8103740 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff79a7fc640 1 -- 172.21.15.133:0/2010792540 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79c005500 con 0x7ff7a8103740 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7af5b2640 1 -- 172.21.15.133:0/2010792540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a819d740 con 0x7ff7a8103740 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff79a7fc640 1 -- 172.21.15.133:0/2010792540 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c010690 con 0x7ff7a8103740 2024-09-11T23:41:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.578+0000 7ff7af5b2640 1 -- 172.21.15.133:0/2010792540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a81a5020 con 0x7ff7a8103740 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff79a7fc640 1 -- 172.21.15.133:0/2010792540 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff79c0046c0 con 0x7ff7a8103740 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff7b198b740 1 Processor -- start 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff7b198b740 1 -- start start 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec37880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec42140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec44a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff7b198b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5565deb9d2e0 con 0x5565dec42680 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff7b198b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5565deb44ce0 con 0x5565dec37dc0 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff7b198b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5565deb992a0 con 0x5565dec374a0 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff79a7fc640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7940a2cf0 0x7ff7940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff7acb4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec44a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.581+0000 7ff7adb4f640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7940a2cf0 0x7ff7940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7acb4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec44a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33670/0 (socket says 172.21.15.133:33670) 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7acb4d640 1 -- 172.21.15.133:0/84730429 learned_addr learned my addr 172.21.15.133:0/84730429 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7ad34e640 1 --2- 172.21.15.133:0/84730429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec42140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7adb4f640 1 --2- 172.21.15.133:0/84730429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec37880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff79a7fc640 1 -- 172.21.15.133:0/2010792540 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff79c0cf7c0 con 0x7ff7a8103740 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7acb4d640 1 -- 172.21.15.133:0/84730429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 msgr2=0x5565dec37880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/84730429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec37880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7acb4d640 1 -- 172.21.15.133:0/84730429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 msgr2=0x5565dec42140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/84730429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec42140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7acb4d640 1 -- 172.21.15.133:0/84730429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565deb29440 con 0x5565dec42680 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7adb4f640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7940a2cf0 0x7ff7940a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009430 tx=0x7ff7a0032fc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/84730429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec44a70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0cce90 tx=0x7ff79c06ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e8188603a453f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7937fe640 1 -- 172.21.15.133:0/84730429 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c09b890 con 0x5565dec42680 2024-09-11T23:41:03.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7937fe640 1 -- 172.21.15.133:0/84730429 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79c09c450 con 0x5565dec42680 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.582+0000 7ff7937fe640 1 -- 172.21.15.133:0/84730429 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c09b6b0 con 0x5565dec42680 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.583+0000 7ff7b198b740 1 -- 172.21.15.133:0/84730429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 msgr2=0x5565dec44a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.583+0000 7ff7b198b740 1 --2- 172.21.15.133:0/84730429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec44a70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0cce90 tx=0x7ff79c06ac80 comp rx=0 tx=0).stop 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.583+0000 7ff7b198b740 1 -- 172.21.15.133:0/84730429 shutdown_connections 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.583+0000 7ff7b198b740 1 --2- 172.21.15.133:0/84730429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec44a70 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.583+0000 7ff7b198b740 1 --2- 172.21.15.133:0/84730429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec42140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.583+0000 7ff7b198b740 1 --2- 172.21.15.133:0/84730429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec37880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.583+0000 7ff7b198b740 1 -- 172.21.15.133:0/84730429 >> 172.21.15.133:0/84730429 conn(0x5565dec33a30 msgr2=0x5565dec33e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.583+0000 7ff7b198b740 1 -- 172.21.15.133:0/84730429 shutdown_connections 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.583+0000 7ff7b198b740 1 -- 172.21.15.133:0/84730429 wait complete. 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.583+0000 7ff7b198b740 1 -- start start 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7b198b740 1 Processor -- start 2024-09-11T23:41:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec374a0 0x5565dec41fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec3f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec42680 0x5565dec3f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7b198b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5565dec45250 con 0x5565dec374a0 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7b198b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5565deb58050 con 0x5565dec37dc0 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7b198b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5565dec453d0 con 0x5565dec42680 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7b198b740 10 client.0 _ll_register_callbacks cb 0x5565deb98160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7ad34e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec374a0 0x5565dec41fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7ad34e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec374a0 0x5565dec41fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33672/0 (socket says 172.21.15.133:33672) 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7ad34e640 1 -- 172.21.15.133:0/2799933177 learned_addr learned my addr 172.21.15.133:0/2799933177 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec3f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7adb4f640 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec42680 0x5565dec3f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7ad34e640 1 -- 172.21.15.133:0/2799933177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec42680 msgr2=0x5565dec3f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7ad34e640 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec42680 0x5565dec3f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7ad34e640 1 -- 172.21.15.133:0/2799933177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 msgr2=0x5565dec3f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7ad34e640 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec3f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7ad34e640 1 -- 172.21.15.133:0/2799933177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565de90e3c0 con 0x5565dec374a0 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec3f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.584+0000 7ff7adb4f640 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec42680 0x5565dec3f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff7ad34e640 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec374a0 0x5565dec41fa0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4010840 tx=0x7ff7a4010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1e0edfb84f5768 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff790ff9640 1 -- 172.21.15.133:0/2799933177 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400eda0 con 0x5565dec374a0 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff790ff9640 1 -- 172.21.15.133:0/2799933177 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff7a4005020 con 0x5565dec374a0 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff7b198b740 1 -- 172.21.15.133:0/2799933177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79c09c450 con 0x5565dec374a0 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff790ff9640 1 -- 172.21.15.133:0/2799933177 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff7a4005260 con 0x5565dec374a0 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff7b198b740 10 client.16644 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff7b198b740 1 -- 172.21.15.133:0/2799933177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565dec50c30 con 0x5565dec374a0 2024-09-11T23:41:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff790ff9640 1 -- 172.21.15.133:0/2799933177 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79c09c450 con 0x5565dec374a0 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff790ff9640 1 -- 172.21.15.133:0/2799933177 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff7a4020030 con 0x5565dec374a0 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff7b198b740 10 client.16644 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff7b198b740 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff7b198b740 20 client.16644 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff7b198b740 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5565dec520f0 0x5565dec54570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff7b198b740 4 client.16644 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.585+0000 7ff7b198b740 1 -- 172.21.15.133:0/2799933177 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565debaa850 con 0x5565dec520f0 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.586+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5565dec520f0 0x5565dec54570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.586+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5565dec520f0 0x5565dec54570 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f11e6529c091859b server_cookie=862d566ffc5af07 in_seq=0 out_seq=0 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.586+0000 7ff790ff9640 10 client.16644 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff790ff9640 1 -- 172.21.15.133:0/2799933177 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565debaa850 con 0x5565dec520f0 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff790ff9640 10 client.16644 handle_command_reply: tid=0 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 1 client.16644 shutdown 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 1 -- 172.21.15.133:0/2799933177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec374a0 msgr2=0x5565dec41fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:03.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec374a0 0x5565dec41fa0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4010840 tx=0x7ff7a4010d00 comp rx=0 tx=0).stop 2024-09-11T23:41:03.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 1 -- 172.21.15.133:0/2799933177 shutdown_connections 2024-09-11T23:41:03.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5565dec520f0 0x5565dec54570 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec42680 0x5565dec3f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff790ff9640 0 client.16644 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:03.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec3f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 1 --2- 172.21.15.133:0/2799933177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec374a0 0x5565dec41fa0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 1 -- 172.21.15.133:0/2799933177 >> 172.21.15.133:0/2799933177 conn(0x5565dec33a30 msgr2=0x5565dec39760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:03.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 1 -- 172.21.15.133:0/2799933177 shutdown_connections 2024-09-11T23:41:03.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 1 -- 172.21.15.133:0/2799933177 wait complete. 2024-09-11T23:41:03.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.587+0000 7ff7b198b740 20 client.16644 trim_cache size 0 max 16384 2024-09-11T23:41:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.604+0000 7ff7b198b740 1 Processor -- start 2024-09-11T23:41:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.604+0000 7ff7b198b740 1 -- start start 2024-09-11T23:41:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec4e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec59e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec5a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7b198b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81ad780 con 0x5565dec42680 2024-09-11T23:41:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7b198b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81af2c0 con 0x5565dec37dc0 2024-09-11T23:41:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7b198b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81ade50 con 0x5565dec374a0 2024-09-11T23:41:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7adb4f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec5a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7adb4f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec5a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33684/0 (socket says 172.21.15.133:33684) 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7adb4f640 1 -- 172.21.15.133:0/3400398085 learned_addr learned my addr 172.21.15.133:0/3400398085 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/3400398085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec59e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7ad34e640 1 --2- 172.21.15.133:0/3400398085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec4e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7adb4f640 1 -- 172.21.15.133:0/3400398085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 msgr2=0x5565dec4e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7adb4f640 1 --2- 172.21.15.133:0/3400398085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec4e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7adb4f640 1 -- 172.21.15.133:0/3400398085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 msgr2=0x5565dec59e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7adb4f640 1 --2- 172.21.15.133:0/3400398085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec59e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7adb4f640 1 -- 172.21.15.133:0/3400398085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565dec37220 con 0x5565dec42680 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff7adb4f640 1 --2- 172.21.15.133:0/3400398085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec5a3d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009cd0 tx=0x7ff7a003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d63785f656f112 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff792ffd640 1 -- 172.21.15.133:0/3400398085 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0004100 con 0x5565dec42680 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff792ffd640 1 -- 172.21.15.133:0/3400398085 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7a0004780 con 0x5565dec42680 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.605+0000 7ff792ffd640 1 -- 172.21.15.133:0/3400398085 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00546e0 con 0x5565dec42680 2024-09-11T23:41:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.606+0000 7ff7b198b740 1 -- 172.21.15.133:0/3400398085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 msgr2=0x5565dec5a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.606+0000 7ff7b198b740 1 --2- 172.21.15.133:0/3400398085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec5a3d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009cd0 tx=0x7ff7a003c000 comp rx=0 tx=0).stop 2024-09-11T23:41:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.606+0000 7ff7b198b740 1 -- 172.21.15.133:0/3400398085 shutdown_connections 2024-09-11T23:41:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.606+0000 7ff7b198b740 1 --2- 172.21.15.133:0/3400398085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec42680 0x5565dec5a3d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.606+0000 7ff7b198b740 1 --2- 172.21.15.133:0/3400398085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec37dc0 0x5565dec59e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.606+0000 7ff7b198b740 1 --2- 172.21.15.133:0/3400398085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec4e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.606+0000 7ff7b198b740 1 -- 172.21.15.133:0/3400398085 >> 172.21.15.133:0/3400398085 conn(0x5565dec33a30 msgr2=0x5565dec3a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.606+0000 7ff7b198b740 1 -- 172.21.15.133:0/3400398085 shutdown_connections 2024-09-11T23:41:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.606+0000 7ff7b198b740 1 -- 172.21.15.133:0/3400398085 wait complete. 2024-09-11T23:41:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7b198b740 1 -- start start 2024-09-11T23:41:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7b198b740 1 Processor -- start 2024-09-11T23:41:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec46580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec37dc0 0x5565dec46ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7b198b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec42680 0x5565dec47000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7b198b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81aebf0 con 0x5565dec37dc0 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7b198b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81ae520 con 0x5565dec42680 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7b198b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81ab760 con 0x5565dec374a0 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7b198b740 10 client.0 _ll_register_callbacks cb 0x7ff7a400b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7acb4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec37dc0 0x5565dec46ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7acb4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec37dc0 0x5565dec46ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33692/0 (socket says 172.21.15.133:33692) 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7acb4d640 1 -- 172.21.15.133:0/207421889 learned_addr learned my addr 172.21.15.133:0/207421889 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.607+0000 7ff7ad34e640 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec46580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff7adb4f640 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec42680 0x5565dec47000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff7acb4d640 1 -- 172.21.15.133:0/207421889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 msgr2=0x5565dec46580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec46580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff7acb4d640 1 -- 172.21.15.133:0/207421889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec42680 msgr2=0x5565dec47000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec42680 0x5565dec47000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff7acb4d640 1 -- 172.21.15.133:0/207421889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565decbc060 con 0x5565dec37dc0 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff7adb4f640 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec42680 0x5565dec47000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff7acb4d640 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec37dc0 0x5565dec46ac0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff79c09ba50 tx=0x7ff79c004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46938a03e67ea44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff790ff9640 1 -- 172.21.15.133:0/207421889 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c09b890 con 0x5565dec37dc0 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff790ff9640 1 -- 172.21.15.133:0/207421889 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff79c0a4530 con 0x5565dec37dc0 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff7b198b740 1 -- 172.21.15.133:0/207421889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a0004780 con 0x5565dec37dc0 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff790ff9640 1 -- 172.21.15.133:0/207421889 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff79c0ba050 con 0x5565dec37dc0 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff7b198b740 10 client.16650 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff7b198b740 1 -- 172.21.15.133:0/207421889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565decbd3b0 con 0x5565dec37dc0 2024-09-11T23:41:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.608+0000 7ff790ff9640 1 -- 172.21.15.133:0/207421889 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7a0004780 con 0x5565dec37dc0 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.609+0000 7ff790ff9640 1 -- 172.21.15.133:0/207421889 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff79c09b1a0 con 0x5565dec37dc0 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.609+0000 7ff7b198b740 10 client.16650 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.609+0000 7ff7b198b740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.609+0000 7ff7b198b740 20 client.16650 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.609+0000 7ff7b198b740 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5565dec520f0 0x5565decbf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.609+0000 7ff7b198b740 4 client.16650 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.609+0000 7ff7b198b740 1 -- 172.21.15.133:0/207421889 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5565deba4d60 con 0x5565dec520f0 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.609+0000 7ff7ad34e640 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5565dec520f0 0x5565decbf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.609+0000 7ff7ad34e640 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5565dec520f0 0x5565decbf750 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95eb105e1f553a7e server_cookie=e67b17621c8fa6e1 in_seq=0 out_seq=0 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.609+0000 7ff790ff9640 10 client.16650 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff790ff9640 1 -- 172.21.15.133:0/207421889 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5565deba4d60 con 0x5565dec520f0 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff790ff9640 10 client.16650 handle_command_reply: tid=0 2024-09-11T23:41:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 1 client.16650 shutdown 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 1 -- 172.21.15.133:0/207421889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec37dc0 msgr2=0x5565dec46ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec37dc0 0x5565dec46ac0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff79c09ba50 tx=0x7ff79c004b30 comp rx=0 tx=0).stop 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 1 -- 172.21.15.133:0/207421889 shutdown_connections 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5565dec520f0 0x5565decbf750 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565dec42680 0x5565dec47000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565dec37dc0 0x5565dec46ac0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 1 --2- 172.21.15.133:0/207421889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565dec374a0 0x5565dec46580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 1 -- 172.21.15.133:0/207421889 >> 172.21.15.133:0/207421889 conn(0x5565dec33a30 msgr2=0x5565dec32350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff790ff9640 0 client.16650 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 1 -- 172.21.15.133:0/207421889 shutdown_connections 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 1 -- 172.21.15.133:0/207421889 wait complete. 2024-09-11T23:41:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.610+0000 7ff7b198b740 20 client.16650 trim_cache size 0 max 16384 2024-09-11T23:41:03.624 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:41:03.624 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:41:03.624 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-09-11T23:41:03.624 INFO:teuthology.orchestra.run.smithi133.stdout: "dc60028c-a513-432c-bf64-ab248f64dfc8": { 2024-09-11T23:41:03.624 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-09-11T23:41:03.624 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "dc60028c-a513-432c-bf64-ab248f64dfc8", 2024-09-11T23:41:03.624 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2024-09-11T23:41:03.624 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:03.624 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:03.624 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:41:03.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 -- 172.21.15.133:0/2010792540 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7940a2cf0 msgr2=0x7ff7940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:03.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7940a2cf0 0x7ff7940a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009430 tx=0x7ff7a0032fc0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 -- 172.21.15.133:0/2010792540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a8103740 msgr2=0x7ff7a81989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:03.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a8103740 0x7ff7a81989e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e9c0 tx=0x7ff79c00ee80 comp rx=0 tx=0).stop 2024-09-11T23:41:03.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 -- 172.21.15.133:0/2010792540 shutdown_connections 2024-09-11T23:41:03.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7940a2cf0 0x7ff7940a51b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a8108290 0x7ff7a819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a8103740 0x7ff7a81989e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 --2- 172.21.15.133:0/2010792540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a8102d90 0x7ff7a81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:03.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 -- 172.21.15.133:0/2010792540 >> 172.21.15.133:0/2010792540 conn(0x7ff7a80fc7d0 msgr2=0x7ff7a8106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:03.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 -- 172.21.15.133:0/2010792540 shutdown_connections 2024-09-11T23:41:03.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:03.611+0000 7ff7af5b2640 1 -- 172.21.15.133:0/2010792540 wait complete. 2024-09-11T23:41:04.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:04 smithi138 ceph-mon[35142]: pgmap v131: 97 pgs: 97 active+clean; 30 GiB data, 135 GiB used, 937 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 595 MiB/s wr, 1.14k op/s 2024-09-11T23:41:04.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:04 smithi133 ceph-mon[31430]: pgmap v131: 97 pgs: 97 active+clean; 30 GiB data, 135 GiB used, 937 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 595 MiB/s wr, 1.14k op/s 2024-09-11T23:41:04.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:04 smithi178 ceph-mon[35882]: pgmap v131: 97 pgs: 97 active+clean; 30 GiB data, 135 GiB used, 937 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 595 MiB/s wr, 1.14k op/s 2024-09-11T23:41:04.594 INFO:tasks.cephfs.filesystem:scrub status for tag:dc60028c-a513-432c-bf64-ab248f64dfc8 - {'path': '/', 'tag': 'dc60028c-a513-432c-bf64-ab248f64dfc8', 'options': 'recursive,force'} 2024-09-11T23:41:05.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:05 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:41:05.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:05 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:41:05.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:05 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:41:06.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:06 smithi138 ceph-mon[35142]: pgmap v132: 97 pgs: 97 active+clean; 30 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 415 MiB/s wr, 656 op/s 2024-09-11T23:41:06.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:06 smithi133 ceph-mon[31430]: pgmap v132: 97 pgs: 97 active+clean; 30 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 415 MiB/s wr, 656 op/s 2024-09-11T23:41:06.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:06 smithi178 ceph-mon[35882]: pgmap v132: 97 pgs: 97 active+clean; 30 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 415 MiB/s wr, 656 op/s 2024-09-11T23:41:08.474 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:41:08.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:08 smithi138 ceph-mon[35142]: pgmap v133: 97 pgs: 97 active+clean; 30 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 341 MiB/s wr, 470 op/s 2024-09-11T23:41:08.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:08 smithi133 ceph-mon[31430]: pgmap v133: 97 pgs: 97 active+clean; 30 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 341 MiB/s wr, 470 op/s 2024-09-11T23:41:08.742 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:08.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:08 smithi178 ceph-mon[35882]: pgmap v133: 97 pgs: 97 active+clean; 30 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 341 MiB/s wr, 470 op/s 2024-09-11T23:41:09.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.402+0000 7f26f73e7640 1 -- 172.21.15.133:0/236841770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26f0101df0 msgr2=0x7f26f01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:09.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.402+0000 7f26f73e7640 1 --2- 172.21.15.133:0/236841770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26f0101df0 0x7f26f01021d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009900 tx=0x7f26ec02a410 comp rx=0 tx=0).stop 2024-09-11T23:41:09.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.402+0000 7f26f73e7640 1 -- 172.21.15.133:0/236841770 shutdown_connections 2024-09-11T23:41:09.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.402+0000 7f26f73e7640 1 --2- 172.21.15.133:0/236841770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f0106f20 0x7f26f010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:09.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.402+0000 7f26f73e7640 1 --2- 172.21.15.133:0/236841770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f01027a0 0x7f26f01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:09.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.402+0000 7f26f73e7640 1 --2- 172.21.15.133:0/236841770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26f0101df0 0x7f26f01021d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:09.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.402+0000 7f26f73e7640 1 -- 172.21.15.133:0/236841770 >> 172.21.15.133:0/236841770 conn(0x7f26f00fd8f0 msgr2=0x7f26f00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:09.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.402+0000 7f26f73e7640 1 -- 172.21.15.133:0/236841770 shutdown_connections 2024-09-11T23:41:09.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.402+0000 7f26f73e7640 1 -- 172.21.15.133:0/236841770 wait complete. 2024-09-11T23:41:09.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.402+0000 7f26f73e7640 1 Processor -- start 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.402+0000 7f26f73e7640 1 -- start start 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f73e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f0101df0 0x7f26f0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f73e7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f01027a0 0x7f26f01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f73e7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26f0106f20 0x7f26f019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f73e7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f26f0111ff0 con 0x7f26f0106f20 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f73e7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26f0111e70 con 0x7f26f01027a0 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f73e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26f0112170 con 0x7f26f0101df0 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f5984640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26f0106f20 0x7f26f019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f5984640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26f0106f20 0x7f26f019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33716/0 (socket says 172.21.15.133:33716) 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f4982640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f01027a0 0x7f26f01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f5984640 1 -- 172.21.15.133:0/2531960548 learned_addr learned my addr 172.21.15.133:0/2531960548 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f5183640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f0101df0 0x7f26f0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f4982640 1 -- 172.21.15.133:0/2531960548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f0101df0 msgr2=0x7f26f0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f4982640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f0101df0 0x7f26f0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:09.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f4982640 1 -- 172.21.15.133:0/2531960548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26f0106f20 msgr2=0x7f26f019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:09.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f4982640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26f0106f20 0x7f26f019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:09.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f4982640 1 -- 172.21.15.133:0/2531960548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f019d440 con 0x7f26f01027a0 2024-09-11T23:41:09.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.403+0000 7f26f5183640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f0101df0 0x7f26f0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:09.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.404+0000 7f26f4982640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f01027a0 0x7f26f01989d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f26e0005fd0 tx=0x7f26e000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:09.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.427+0000 7f26e67fc640 1 -- 172.21.15.133:0/2531960548 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0009c30 con 0x7f26f01027a0 2024-09-11T23:41:09.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.427+0000 7f26e67fc640 1 -- 172.21.15.133:0/2531960548 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e0009dd0 con 0x7f26f01027a0 2024-09-11T23:41:09.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.427+0000 7f26e67fc640 1 -- 172.21.15.133:0/2531960548 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0010460 con 0x7f26f01027a0 2024-09-11T23:41:09.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.427+0000 7f26f73e7640 1 -- 172.21.15.133:0/2531960548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f019d730 con 0x7f26f01027a0 2024-09-11T23:41:09.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.428+0000 7f26f73e7640 1 -- 172.21.15.133:0/2531960548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f01a5010 con 0x7f26f01027a0 2024-09-11T23:41:09.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.429+0000 7f26f73e7640 1 -- 172.21.15.133:0/2531960548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f01021d0 con 0x7f26f01027a0 2024-09-11T23:41:09.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.431+0000 7f26e67fc640 1 -- 172.21.15.133:0/2531960548 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f26e0003f50 con 0x7f26f01027a0 2024-09-11T23:41:09.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.432+0000 7f26e67fc640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f26dc0a2d70 0x7f26dc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:09.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.432+0000 7f26f5183640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f26dc0a2d70 0x7f26dc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:09.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.432+0000 7f26e67fc640 1 -- 172.21.15.133:0/2531960548 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f26e00cef10 con 0x7f26f01027a0 2024-09-11T23:41:09.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.432+0000 7f26f5183640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f26dc0a2d70 0x7f26dc0a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f26ec000c00 tx=0x7f26ec031510 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:09.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.435+0000 7f26e67fc640 1 -- 172.21.15.133:0/2531960548 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26e0017020 con 0x7f26f01027a0 2024-09-11T23:41:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.653+0000 7f26f73e7640 1 -- 172.21.15.133:0/2531960548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26f01997f0 con 0x7f26f01027a0 2024-09-11T23:41:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.654+0000 7f26e67fc640 1 -- 172.21.15.133:0/2531960548 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f26f01997f0 con 0x7f26f01027a0 2024-09-11T23:41:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:41:09.655 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:41:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:41:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 -- 172.21.15.133:0/2531960548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f26dc0a2d70 msgr2=0x7f26dc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f26dc0a2d70 0x7f26dc0a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f26ec000c00 tx=0x7f26ec031510 comp rx=0 tx=0).stop 2024-09-11T23:41:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 -- 172.21.15.133:0/2531960548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f01027a0 msgr2=0x7f26f01989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f01027a0 0x7f26f01989d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f26e0005fd0 tx=0x7f26e000dde0 comp rx=0 tx=0).stop 2024-09-11T23:41:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 -- 172.21.15.133:0/2531960548 shutdown_connections 2024-09-11T23:41:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f26dc0a2d70 0x7f26dc0a5230 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26f0106f20 0x7f26f019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f01027a0 0x7f26f01989d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:09.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 --2- 172.21.15.133:0/2531960548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f0101df0 0x7f26f0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:09.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 -- 172.21.15.133:0/2531960548 >> 172.21.15.133:0/2531960548 conn(0x7f26f00fd8f0 msgr2=0x7f26f00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:09.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 -- 172.21.15.133:0/2531960548 shutdown_connections 2024-09-11T23:41:09.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:09.656+0000 7f26f73e7640 1 -- 172.21.15.133:0/2531960548 wait complete. 2024-09-11T23:41:09.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:09 smithi133 ceph-mon[31430]: pgmap v134: 97 pgs: 97 active+clean; 30 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 328 MiB/s wr, 406 op/s 2024-09-11T23:41:09.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:09 smithi178 ceph-mon[35882]: pgmap v134: 97 pgs: 97 active+clean; 30 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 328 MiB/s wr, 406 op/s 2024-09-11T23:41:09.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:09 smithi138 ceph-mon[35142]: pgmap v134: 97 pgs: 97 active+clean; 30 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 328 MiB/s wr, 406 op/s 2024-09-11T23:41:10.481 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id c1232923 --release --await' 2024-09-11T23:41:10.482 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id c1232923 --release --await 2024-09-11T23:41:10.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:10 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2531960548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:10.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:10 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2531960548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:10 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2531960548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.065+0000 7f025bc82640 1 -- 172.21.15.133:0/3468449468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f025410c0d0 msgr2=0x7f025410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.065+0000 7f025bc82640 1 --2- 172.21.15.133:0/3468449468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f025410c0d0 0x7f025410e550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f024c009ca0 tx=0x7f024c02a950 comp rx=0 tx=0).stop 2024-09-11T23:41:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.065+0000 7f025bc82640 1 -- 172.21.15.133:0/3468449468 shutdown_connections 2024-09-11T23:41:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.065+0000 7f025bc82640 1 --2- 172.21.15.133:0/3468449468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f025410c0d0 0x7f025410e550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.065+0000 7f025bc82640 1 --2- 172.21.15.133:0/3468449468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02541035a0 0x7f025410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.065+0000 7f025bc82640 1 --2- 172.21.15.133:0/3468449468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0254102c80 0x7f0254103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.065+0000 7f025bc82640 1 -- 172.21.15.133:0/3468449468 >> 172.21.15.133:0/3468449468 conn(0x7f0254077990 msgr2=0x7f02540fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.066+0000 7f025bc82640 1 -- 172.21.15.133:0/3468449468 shutdown_connections 2024-09-11T23:41:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.066+0000 7f025bc82640 1 -- 172.21.15.133:0/3468449468 wait complete. 2024-09-11T23:41:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.066+0000 7f025bc82640 1 Processor -- start 2024-09-11T23:41:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.066+0000 7f025bc82640 1 -- start start 2024-09-11T23:41:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025bc82640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0254102c80 0x7f02541008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025bc82640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02541035a0 0x7f0254100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025bc82640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025410c0d0 0x7f0254101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025bc82640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0254111e90 con 0x7f025410c0d0 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025bc82640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0254111d10 con 0x7f02541035a0 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025a21f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025410c0d0 0x7f0254101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025bc82640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0254112010 con 0x7f0254102c80 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025a21f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025410c0d0 0x7f0254101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33738/0 (socket says 172.21.15.133:33738) 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025a21f640 1 -- 172.21.15.133:0/1227858832 learned_addr learned my addr 172.21.15.133:0/1227858832 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025921d640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02541035a0 0x7f0254100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f0259a1e640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0254102c80 0x7f02541008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025a21f640 1 -- 172.21.15.133:0/1227858832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0254102c80 msgr2=0x7f02541008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025a21f640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0254102c80 0x7f02541008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025a21f640 1 -- 172.21.15.133:0/1227858832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02541035a0 msgr2=0x7f0254100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025a21f640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02541035a0 0x7f0254100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025a21f640 1 -- 172.21.15.133:0/1227858832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02541a67e0 con 0x7f025410c0d0 2024-09-11T23:41:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025921d640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02541035a0 0x7f0254100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.067+0000 7f025a21f640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025410c0d0 0x7f0254101350 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f024c004af0 tx=0x7f024c02f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.068+0000 7f024affd640 1 -- 172.21.15.133:0/1227858832 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c034040 con 0x7f025410c0d0 2024-09-11T23:41:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.068+0000 7f025bc82640 1 -- 172.21.15.133:0/1227858832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02541a6a70 con 0x7f025410c0d0 2024-09-11T23:41:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.068+0000 7f024affd640 1 -- 172.21.15.133:0/1227858832 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024c030920 con 0x7f025410c0d0 2024-09-11T23:41:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.068+0000 7f024affd640 1 -- 172.21.15.133:0/1227858832 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c004670 con 0x7f025410c0d0 2024-09-11T23:41:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.068+0000 7f025bc82640 1 -- 172.21.15.133:0/1227858832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02541a6fb0 con 0x7f025410c0d0 2024-09-11T23:41:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.071+0000 7f024affd640 1 -- 172.21.15.133:0/1227858832 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f024c037020 con 0x7f025410c0d0 2024-09-11T23:41:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.072+0000 7f025ea21740 1 Processor -- start 2024-09-11T23:41:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.072+0000 7f024affd640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f02400a2da0 0x7f02400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.072+0000 7f025ea21740 1 -- start start 2024-09-11T23:41:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.072+0000 7f024affd640 1 -- 172.21.15.133:0/1227858832 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f024c0f0430 con 0x7f025410c0d0 2024-09-11T23:41:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025921d640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f02400a2da0 0x7f02400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025ea21740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea967af0 0x5654ea967ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025ea21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea968410 0x5654ea972770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025ea21740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea9750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025ea21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5654ea8cc2f0 con 0x5654ea968410 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025ea21740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654ea874270 con 0x5654ea972cb0 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025ea21740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5654ea8c82b0 con 0x5654ea967af0 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025921d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea968410 0x5654ea972770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025921d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea968410 0x5654ea972770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33742/0 (socket says 172.21.15.133:33742) 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025921d640 1 -- 172.21.15.133:0/1440404528 learned_addr learned my addr 172.21.15.133:0/1440404528 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f0259a1e640 1 --2- 172.21.15.133:0/1440404528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea967af0 0x5654ea967ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025a21f640 1 --2- 172.21.15.133:0/1440404528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea9750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025921d640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f02400a2da0 0x7f02400a5260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0244013870 tx=0x7f0244013c70 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025921d640 1 -- 172.21.15.133:0/1440404528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea967af0 msgr2=0x5654ea967ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025921d640 1 --2- 172.21.15.133:0/1440404528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea967af0 0x5654ea967ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025921d640 1 -- 172.21.15.133:0/1440404528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 msgr2=0x5654ea9750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025921d640 1 --2- 172.21.15.133:0/1440404528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea9750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025921d640 1 -- 172.21.15.133:0/1440404528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ea873340 con 0x5654ea968410 2024-09-11T23:41:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025a21f640 1 --2- 172.21.15.133:0/1440404528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea9750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:41:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f0259a1e640 1 --2- 172.21.15.133:0/1440404528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea967af0 0x5654ea967ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f025921d640 1 --2- 172.21.15.133:0/1440404528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea968410 0x5654ea972770 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f024400fc80 tx=0x7f0244020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61c6a2b8e8eb13f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.073+0000 7f023ffff640 1 -- 172.21.15.133:0/1440404528 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244012110 con 0x5654ea968410 2024-09-11T23:41:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.074+0000 7f023ffff640 1 -- 172.21.15.133:0/1440404528 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0244024070 con 0x5654ea968410 2024-09-11T23:41:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.074+0000 7f023ffff640 1 -- 172.21.15.133:0/1440404528 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244003970 con 0x5654ea968410 2024-09-11T23:41:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.074+0000 7f025ea21740 1 -- 172.21.15.133:0/1440404528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea968410 msgr2=0x5654ea972770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.074+0000 7f025ea21740 1 --2- 172.21.15.133:0/1440404528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea968410 0x5654ea972770 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f024400fc80 tx=0x7f0244020000 comp rx=0 tx=0).stop 2024-09-11T23:41:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.074+0000 7f025ea21740 1 -- 172.21.15.133:0/1440404528 shutdown_connections 2024-09-11T23:41:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.074+0000 7f025ea21740 1 --2- 172.21.15.133:0/1440404528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea9750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.074+0000 7f025ea21740 1 --2- 172.21.15.133:0/1440404528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea968410 0x5654ea972770 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.074+0000 7f025ea21740 1 --2- 172.21.15.133:0/1440404528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea967af0 0x5654ea967ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.074+0000 7f025ea21740 1 -- 172.21.15.133:0/1440404528 >> 172.21.15.133:0/1440404528 conn(0x5654ea964060 msgr2=0x5654ea964440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.075+0000 7f025ea21740 1 -- 172.21.15.133:0/1440404528 shutdown_connections 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.075+0000 7f025ea21740 1 -- 172.21.15.133:0/1440404528 wait complete. 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.075+0000 7f025ea21740 1 -- start start 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.075+0000 7f025ea21740 1 Processor -- start 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.075+0000 7f025ea21740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea967af0 0x5654ea96fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.075+0000 7f025ea21740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea970160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.075+0000 7f025ea21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea972cb0 0x5654ea9706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.075+0000 7f025ea21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5654ea975880 con 0x5654ea972cb0 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.075+0000 7f025ea21740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654ea855290 con 0x5654ea967af0 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.075+0000 7f025ea21740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5654ea975a00 con 0x5654ea968410 2024-09-11T23:41:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.075+0000 7f025ea21740 10 client.0 _ll_register_callbacks cb 0x5654ea886140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025a21f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea972cb0 0x5654ea9706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025a21f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea972cb0 0x5654ea9706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33758/0 (socket says 172.21.15.133:33758) 2024-09-11T23:41:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025a21f640 1 -- 172.21.15.133:0/1964027436 learned_addr learned my addr 172.21.15.133:0/1964027436 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f0259a1e640 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea967af0 0x5654ea96fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025921d640 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea970160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025a21f640 1 -- 172.21.15.133:0/1964027436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 msgr2=0x5654ea970160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025a21f640 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea970160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025a21f640 1 -- 172.21.15.133:0/1964027436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea967af0 msgr2=0x5654ea96fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025a21f640 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea967af0 0x5654ea96fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025a21f640 1 -- 172.21.15.133:0/1964027436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ea5df210 con 0x5654ea972cb0 2024-09-11T23:41:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025a21f640 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea972cb0 0x5654ea9706a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f024c0bca60 tx=0x7f024c0bc430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7536e8a8fb18c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f023d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f023d7fa640 1 -- 172.21.15.133:0/1964027436 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c0da060 con 0x5654ea972cb0 2024-09-11T23:41:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f023d7fa640 1 -- 172.21.15.133:0/1964027436 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f024c0b9140 con 0x5654ea972cb0 2024-09-11T23:41:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025ea21740 1 -- 172.21.15.133:0/1964027436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0244024070 con 0x5654ea972cb0 2024-09-11T23:41:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f023d7fa640 1 -- 172.21.15.133:0/1964027436 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f024c0bcad0 con 0x5654ea972cb0 2024-09-11T23:41:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025ea21740 10 client.16668 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.076+0000 7f025ea21740 1 -- 172.21.15.133:0/1964027436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654ea6ab620 con 0x5654ea972cb0 2024-09-11T23:41:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.077+0000 7f023d7fa640 1 -- 172.21.15.133:0/1964027436 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0244024070 con 0x5654ea972cb0 2024-09-11T23:41:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.077+0000 7f023d7fa640 1 -- 172.21.15.133:0/1964027436 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f024c04d070 con 0x5654ea972cb0 2024-09-11T23:41:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.077+0000 7f025ea21740 10 client.16668 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.077+0000 7f025ea21740 10 client.16668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.077+0000 7f025ea21740 20 client.16668 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.077+0000 7f025ea21740 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5654ea982c10 0x5654ea985000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.077+0000 7f025ea21740 4 client.16668 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.077+0000 7f025ea21740 1 -- 172.21.15.133:0/1964027436 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654ea8d9a20 con 0x5654ea982c10 2024-09-11T23:41:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.077+0000 7f0259a1e640 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5654ea982c10 0x5654ea985000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.078+0000 7f023d7fa640 10 client.16668 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.078+0000 7f0259a1e640 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5654ea982c10 0x5654ea985000 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce38e31cb8e2385 server_cookie=b33ed81cf8c3913 in_seq=0 out_seq=0 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.078+0000 7f023d7fa640 1 -- 172.21.15.133:0/1964027436 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654ea8d9a20 con 0x5654ea982c10 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.078+0000 7f023d7fa640 10 client.16668 handle_command_reply: tid=0 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.078+0000 7f025ea21740 1 client.16668 shutdown 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f025ea21740 1 -- 172.21.15.133:0/1964027436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea972cb0 msgr2=0x5654ea9706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f025ea21740 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea972cb0 0x5654ea9706a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f024c0bca60 tx=0x7f024c0bc430 comp rx=0 tx=0).stop 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f025ea21740 1 -- 172.21.15.133:0/1964027436 shutdown_connections 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f025ea21740 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5654ea982c10 0x5654ea985000 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f025ea21740 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea972cb0 0x5654ea9706a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f025ea21740 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea970160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f023d7fa640 0 client.16668 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f025ea21740 1 --2- 172.21.15.133:0/1964027436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea967af0 0x5654ea96fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f025ea21740 1 -- 172.21.15.133:0/1964027436 >> 172.21.15.133:0/1964027436 conn(0x5654ea964060 msgr2=0x5654ea9631c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f025ea21740 1 -- 172.21.15.133:0/1964027436 shutdown_connections 2024-09-11T23:41:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f025ea21740 1 -- 172.21.15.133:0/1964027436 wait complete. 2024-09-11T23:41:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.079+0000 7f025ea21740 20 client.16668 trim_cache size 0 max 16384 2024-09-11T23:41:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025ea21740 1 Processor -- start 2024-09-11T23:41:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025ea21740 1 -- start start 2024-09-11T23:41:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025ea21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 0x5654ea9704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025ea21740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea98a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025ea21740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea98ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025ea21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f02541af680 con 0x5654ea967af0 2024-09-11T23:41:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025ea21740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02541b11c0 con 0x5654ea972cb0 2024-09-11T23:41:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025ea21740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02541afd50 con 0x5654ea968410 2024-09-11T23:41:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f0259a1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 0x5654ea9704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f0259a1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 0x5654ea9704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33768/0 (socket says 172.21.15.133:33768) 2024-09-11T23:41:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025a21f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea98ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f0259a1e640 1 -- 172.21.15.133:0/525242659 learned_addr learned my addr 172.21.15.133:0/525242659 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025921d640 1 --2- 172.21.15.133:0/525242659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea98a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025a21f640 1 -- 172.21.15.133:0/525242659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 msgr2=0x5654ea98a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025a21f640 1 --2- 172.21.15.133:0/525242659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea98a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025a21f640 1 -- 172.21.15.133:0/525242659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 msgr2=0x5654ea9704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025a21f640 1 --2- 172.21.15.133:0/525242659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 0x5654ea9704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025a21f640 1 -- 172.21.15.133:0/525242659 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ea985660 con 0x5654ea972cb0 2024-09-11T23:41:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.094+0000 7f025a21f640 1 --2- 172.21.15.133:0/525242659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea98ae30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f024c0bc7d0 tx=0x7f024c02f8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f0274269827a4dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.095+0000 7f023f7fe640 1 -- 172.21.15.133:0/525242659 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c0bc8a0 con 0x5654ea972cb0 2024-09-11T23:41:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.095+0000 7f023f7fe640 1 -- 172.21.15.133:0/525242659 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024c0098d0 con 0x5654ea972cb0 2024-09-11T23:41:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.095+0000 7f023f7fe640 1 -- 172.21.15.133:0/525242659 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c0b9140 con 0x5654ea972cb0 2024-09-11T23:41:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.095+0000 7f025ea21740 1 -- 172.21.15.133:0/525242659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 msgr2=0x5654ea98ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.095+0000 7f025ea21740 1 --2- 172.21.15.133:0/525242659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea98ae30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f024c0bc7d0 tx=0x7f024c02f8a0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.095+0000 7f025ea21740 1 -- 172.21.15.133:0/525242659 shutdown_connections 2024-09-11T23:41:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.095+0000 7f025ea21740 1 --2- 172.21.15.133:0/525242659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea98ae30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.095+0000 7f025ea21740 1 --2- 172.21.15.133:0/525242659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea98a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.095+0000 7f025ea21740 1 --2- 172.21.15.133:0/525242659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 0x5654ea9704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.095+0000 7f025ea21740 1 -- 172.21.15.133:0/525242659 >> 172.21.15.133:0/525242659 conn(0x5654ea964060 msgr2=0x5654ea971ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f025ea21740 1 -- 172.21.15.133:0/525242659 shutdown_connections 2024-09-11T23:41:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f025ea21740 1 -- 172.21.15.133:0/525242659 wait complete. 2024-09-11T23:41:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f025ea21740 1 -- start start 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f025ea21740 1 Processor -- start 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f025ea21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 0x5654ea976940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f025ea21740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea976e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f025ea21740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea9773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f025ea21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f02541b0af0 con 0x5654ea967af0 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f025ea21740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02541b0420 con 0x5654ea972cb0 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f025ea21740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02541ad660 con 0x5654ea968410 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f025ea21740 10 client.0 _ll_register_callbacks cb 0x7f024c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f0259a1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 0x5654ea976940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f0259a1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 0x5654ea976940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33784/0 (socket says 172.21.15.133:33784) 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.096+0000 7f0259a1e640 1 -- 172.21.15.133:0/2208080800 learned_addr learned my addr 172.21.15.133:0/2208080800 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f025a21f640 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea9773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f025921d640 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea976e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f0259a1e640 1 -- 172.21.15.133:0/2208080800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 msgr2=0x5654ea976e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f0259a1e640 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea976e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f0259a1e640 1 -- 172.21.15.133:0/2208080800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 msgr2=0x5654ea9773c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f0259a1e640 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea9773c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f0259a1e640 1 -- 172.21.15.133:0/2208080800 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ea95f2d0 con 0x5654ea967af0 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f025a21f640 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea9773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f0259a1e640 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 0x5654ea976940 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0250009d10 tx=0x7f0250004240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ebde7b5f0afd81 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f023d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f023d7fa640 1 -- 172.21.15.133:0/2208080800 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250005020 con 0x5654ea967af0 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f023d7fa640 1 -- 172.21.15.133:0/2208080800 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f02500051c0 con 0x5654ea967af0 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f025ea21740 1 -- 172.21.15.133:0/2208080800 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654ea8d3d70 con 0x5654ea967af0 2024-09-11T23:41:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f023d7fa640 1 -- 172.21.15.133:0/2208080800 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f02500053c0 con 0x5654ea967af0 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f025ea21740 10 client.16674 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.097+0000 7f025ea21740 1 -- 172.21.15.133:0/2208080800 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654ea9ed9a0 con 0x5654ea967af0 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.098+0000 7f023d7fa640 1 -- 172.21.15.133:0/2208080800 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5654ea8d3d70 con 0x5654ea967af0 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.098+0000 7f023d7fa640 1 -- 172.21.15.133:0/2208080800 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f025002f050 con 0x5654ea967af0 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.098+0000 7f025ea21740 10 client.16674 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.098+0000 7f025ea21740 10 client.16674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.098+0000 7f025ea21740 20 client.16674 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.098+0000 7f025ea21740 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5654ea9ee070 0x5654ea9f0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.098+0000 7f025ea21740 4 client.16674 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.098+0000 7f025ea21740 1 -- 172.21.15.133:0/2208080800 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654ea850900 con 0x5654ea9ee070 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.098+0000 7f025921d640 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5654ea9ee070 0x5654ea9f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.099+0000 7f023d7fa640 10 client.16674 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.099+0000 7f025921d640 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5654ea9ee070 0x5654ea9f0460 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff8edbd9fa06927 server_cookie=d3007a1abca285bb in_seq=0 out_seq=0 2024-09-11T23:41:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f023d7fa640 1 -- 172.21.15.133:0/2208080800 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654ea850900 con 0x5654ea9ee070 2024-09-11T23:41:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f023d7fa640 10 client.16674 handle_command_reply: tid=0 2024-09-11T23:41:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 1 client.16674 shutdown 2024-09-11T23:41:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 1 -- 172.21.15.133:0/2208080800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 msgr2=0x5654ea976940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 0x5654ea976940 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0250009d10 tx=0x7f0250004240 comp rx=0 tx=0).stop 2024-09-11T23:41:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 1 -- 172.21.15.133:0/2208080800 shutdown_connections 2024-09-11T23:41:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5654ea9ee070 0x5654ea9f0460 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654ea972cb0 0x5654ea9773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f023d7fa640 0 client.16674 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654ea968410 0x5654ea976e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 1 --2- 172.21.15.133:0/2208080800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5654ea967af0 0x5654ea976940 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 1 -- 172.21.15.133:0/2208080800 >> 172.21.15.133:0/2208080800 conn(0x5654ea964060 msgr2=0x5654ea969550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 1 -- 172.21.15.133:0/2208080800 shutdown_connections 2024-09-11T23:41:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 1 -- 172.21.15.133:0/2208080800 wait complete. 2024-09-11T23:41:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.111+0000 7f025ea21740 20 client.16674 trim_cache size 0 max 16384 2024-09-11T23:41:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:41:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:41:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:41:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:41:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 -- 172.21.15.133:0/1227858832 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f02400a2da0 msgr2=0x7f02400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f02400a2da0 0x7f02400a5260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0244013870 tx=0x7f0244013c70 comp rx=0 tx=0).stop 2024-09-11T23:41:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 -- 172.21.15.133:0/1227858832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025410c0d0 msgr2=0x7f0254101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025410c0d0 0x7f0254101350 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f024c004af0 tx=0x7f024c02f3a0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 -- 172.21.15.133:0/1227858832 shutdown_connections 2024-09-11T23:41:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f02400a2da0 0x7f02400a5260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025410c0d0 0x7f0254101350 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02541035a0 0x7f0254100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 --2- 172.21.15.133:0/1227858832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0254102c80 0x7f02541008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 -- 172.21.15.133:0/1227858832 >> 172.21.15.133:0/1227858832 conn(0x7f0254077990 msgr2=0x7f02540fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 -- 172.21.15.133:0/1227858832 shutdown_connections 2024-09-11T23:41:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:11.112+0000 7f025bc82640 1 -- 172.21.15.133:0/1227858832 wait complete. 2024-09-11T23:41:11.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.260+0000 7f6720117640 1 -- 172.21.15.133:0/1131484857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718101420 msgr2=0x7f671810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.260+0000 7f6720117640 1 --2- 172.21.15.133:0/1131484857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718101420 0x7f671810d950 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6710009cd0 tx=0x7f671002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 -- 172.21.15.133:0/1131484857 shutdown_connections 2024-09-11T23:41:11.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 --2- 172.21.15.133:0/1131484857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718101420 0x7f671810d950 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 --2- 172.21.15.133:0/1131484857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718100a80 0x7f6718100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 --2- 172.21.15.133:0/1131484857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6718106a80 0x7f6718106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 -- 172.21.15.133:0/1131484857 >> 172.21.15.133:0/1131484857 conn(0x7f67180fc7c0 msgr2=0x7f67180febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 -- 172.21.15.133:0/1131484857 shutdown_connections 2024-09-11T23:41:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 -- 172.21.15.133:0/1131484857 wait complete. 2024-09-11T23:41:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 Processor -- start 2024-09-11T23:41:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 -- start start 2024-09-11T23:41:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6718100a80 0x7f6718199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718101420 0x7f67181997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718106a80 0x7f67181a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6718110e80 con 0x7f6718101420 2024-09-11T23:41:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6718110d00 con 0x7f6718100a80 2024-09-11T23:41:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.261+0000 7f6720117640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6718111000 con 0x7f6718106a80 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671d6b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718101420 0x7f67181997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671d6b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718101420 0x7f67181997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33796/0 (socket says 172.21.15.133:33796) 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671d6b2640 1 -- 172.21.15.133:0/517521910 learned_addr learned my addr 172.21.15.133:0/517521910 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671deb3640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6718100a80 0x7f6718199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671e6b4640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718106a80 0x7f67181a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671d6b2640 1 -- 172.21.15.133:0/517521910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718106a80 msgr2=0x7f67181a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671d6b2640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718106a80 0x7f67181a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671d6b2640 1 -- 172.21.15.133:0/517521910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6718100a80 msgr2=0x7f6718199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671d6b2640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6718100a80 0x7f6718199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671d6b2640 1 -- 172.21.15.133:0/517521910 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67181a4830 con 0x7f6718101420 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671e6b4640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718106a80 0x7f67181a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671deb3640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6718100a80 0x7f6718199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:41:11.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f671d6b2640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718101420 0x7f67181997b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f670800e9f0 tx=0x7f670800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f670effd640 1 -- 172.21.15.133:0/517521910 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670800cd80 con 0x7f6718101420 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f6720117640 1 -- 172.21.15.133:0/517521910 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67181a4a00 con 0x7f6718101420 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f670effd640 1 -- 172.21.15.133:0/517521910 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6708005500 con 0x7f6718101420 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f670effd640 1 -- 172.21.15.133:0/517521910 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708010690 con 0x7f6718101420 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.262+0000 7f6720117640 1 -- 172.21.15.133:0/517521910 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67181a4ec0 con 0x7f6718101420 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.265+0000 7f67224f0740 1 Processor -- start 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.265+0000 7f67224f0740 1 -- start start 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.265+0000 7f67224f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556caba99440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.265+0000 7f67224f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556cabaa3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.265+0000 7f67224f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cabaa6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.265+0000 7f67224f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556cab9fe620 con 0x556cabaa4240 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.265+0000 7f67224f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556cab9a5e50 con 0x556caba99060 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.265+0000 7f67224f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556cab9fa5e0 con 0x556caba99980 2024-09-11T23:41:11.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f670effd640 1 -- 172.21.15.133:0/517521910 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f67080046c0 con 0x7f6718101420 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671e6b4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556caba99440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671e6b4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556caba99440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59398/0 (socket says 172.21.15.133:59398) 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671e6b4640 1 -- 172.21.15.133:0/3503432519 learned_addr learned my addr 172.21.15.133:0/3503432519 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671d6b2640 1 --2- 172.21.15.133:0/3503432519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cabaa6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671deb3640 1 --2- 172.21.15.133:0/3503432519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556cabaa3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671e6b4640 1 -- 172.21.15.133:0/3503432519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 msgr2=0x556cabaa3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671e6b4640 1 --2- 172.21.15.133:0/3503432519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556cabaa3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671e6b4640 1 -- 172.21.15.133:0/3503432519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 msgr2=0x556cabaa6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671e6b4640 1 --2- 172.21.15.133:0/3503432519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cabaa6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671e6b4640 1 -- 172.21.15.133:0/3503432519 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cab996d40 con 0x556caba99060 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671deb3640 1 --2- 172.21.15.133:0/3503432519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556cabaa3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671d6b2640 1 --2- 172.21.15.133:0/3503432519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cabaa6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:41:11.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f671e6b4640 1 --2- 172.21.15.133:0/3503432519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556caba99440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6710005e30 tx=0x7f6710009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a9d5254d2c2f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f6703fff640 1 -- 172.21.15.133:0/3503432519 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67100318c0 con 0x556caba99060 2024-09-11T23:41:11.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f6703fff640 1 -- 172.21.15.133:0/3503432519 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6710036040 con 0x556caba99060 2024-09-11T23:41:11.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.266+0000 7f6703fff640 1 -- 172.21.15.133:0/3503432519 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67100044e0 con 0x556caba99060 2024-09-11T23:41:11.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f670effd640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f67040a2cf0 0x7f67040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f671deb3640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f67040a2cf0 0x7f67040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f670effd640 1 -- 172.21.15.133:0/517521910 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f67080cf940 con 0x7f6718101420 2024-09-11T23:41:11.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f67224f0740 1 -- 172.21.15.133:0/3503432519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 msgr2=0x556caba99440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f671deb3640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f67040a2cf0 0x7f67040a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6714005e00 tx=0x7f6714005d90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f67224f0740 1 --2- 172.21.15.133:0/3503432519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556caba99440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6710005e30 tx=0x7f6710009a20 comp rx=0 tx=0).stop 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f67224f0740 1 -- 172.21.15.133:0/3503432519 shutdown_connections 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f67224f0740 1 --2- 172.21.15.133:0/3503432519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cabaa6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f67224f0740 1 --2- 172.21.15.133:0/3503432519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556cabaa3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f67224f0740 1 --2- 172.21.15.133:0/3503432519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556caba99440 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f67224f0740 1 -- 172.21.15.133:0/3503432519 >> 172.21.15.133:0/3503432519 conn(0x556caba955f0 msgr2=0x556caba959d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f67224f0740 1 -- 172.21.15.133:0/3503432519 shutdown_connections 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.267+0000 7f67224f0740 1 -- 172.21.15.133:0/3503432519 wait complete. 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.268+0000 7f67224f0740 1 -- start start 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.268+0000 7f67224f0740 1 Processor -- start 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.268+0000 7f67224f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99060 0x556cabaa1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.268+0000 7f67224f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556caba99980 0x556cabaa16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.268+0000 7f67224f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cabaa4240 0x556cabaa1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.268+0000 7f67224f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556cabaa6e10 con 0x556caba99980 2024-09-11T23:41:11.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.268+0000 7f67224f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556cab986f10 con 0x556cabaa4240 2024-09-11T23:41:11.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.268+0000 7f67224f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556cabaa6f90 con 0x556caba99060 2024-09-11T23:41:11.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f67224f0740 10 client.0 _ll_register_callbacks cb 0x556cab9aadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:11.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f671d6b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556caba99980 0x556cabaa16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f671d6b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556caba99980 0x556cabaa16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33818/0 (socket says 172.21.15.133:33818) 2024-09-11T23:41:11.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f671d6b2640 1 -- 172.21.15.133:0/2138664639 learned_addr learned my addr 172.21.15.133:0/2138664639 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:11.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f671e6b4640 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cabaa4240 0x556cabaa1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f671deb3640 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99060 0x556cabaa1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f671d6b2640 1 -- 172.21.15.133:0/2138664639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99060 msgr2=0x556cabaa1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f671d6b2640 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99060 0x556cabaa1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f671d6b2640 1 -- 172.21.15.133:0/2138664639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cabaa4240 msgr2=0x556cabaa1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f671d6b2640 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cabaa4240 0x556cabaa1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f671d6b2640 1 -- 172.21.15.133:0/2138664639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cab70d580 con 0x556caba99980 2024-09-11T23:41:11.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f671d6b2640 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556caba99980 0x556cabaa16a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f670809bbd0 tx=0x7f670809bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae0354b00173316 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f67017fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:11.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f67017fa640 1 -- 172.21.15.133:0/2138664639 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670809bed0 con 0x556caba99980 2024-09-11T23:41:11.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f67224f0740 1 -- 172.21.15.133:0/2138664639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6710036040 con 0x556caba99980 2024-09-11T23:41:11.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f67017fa640 1 -- 172.21.15.133:0/2138664639 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6708016df0 con 0x556caba99980 2024-09-11T23:41:11.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f67017fa640 1 -- 172.21.15.133:0/2138664639 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67080a0280 con 0x556caba99980 2024-09-11T23:41:11.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f67224f0740 10 client.16686 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:11.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.269+0000 7f67224f0740 1 -- 172.21.15.133:0/2138664639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556cab7d7960 con 0x556caba99980 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.270+0000 7f67017fa640 1 -- 172.21.15.133:0/2138664639 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6710036040 con 0x556caba99980 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.270+0000 7f67017fa640 1 -- 172.21.15.133:0/2138664639 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f670809b320 con 0x556caba99980 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.270+0000 7f67224f0740 10 client.16686 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.270+0000 7f67224f0740 10 client.16686 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.270+0000 7f67224f0740 20 client.16686 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.270+0000 7f67224f0740 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556cabab3e90 0x556cabab62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.270+0000 7f67224f0740 4 client.16686 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.270+0000 7f67224f0740 1 -- 172.21.15.133:0/2138664639 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556caba0bb90 con 0x556cabab3e90 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.270+0000 7f671deb3640 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556cabab3e90 0x556cabab62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.271+0000 7f67017fa640 10 client.16686 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.271+0000 7f671deb3640 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556cabab3e90 0x556cabab62a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce5e0b1fc717c87 server_cookie=d9c4284f5357c558 in_seq=0 out_seq=0 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.271+0000 7f67017fa640 1 -- 172.21.15.133:0/2138664639 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556caba0bb90 con 0x556cabab3e90 2024-09-11T23:41:11.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.271+0000 7f67017fa640 10 client.16686 handle_command_reply: tid=0 2024-09-11T23:41:11.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.271+0000 7f67224f0740 1 client.16686 shutdown 2024-09-11T23:41:11.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.271+0000 7f67224f0740 1 -- 172.21.15.133:0/2138664639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556caba99980 msgr2=0x556cabaa16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.271+0000 7f67224f0740 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556caba99980 0x556cabaa16a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f670809bbd0 tx=0x7f670809bda0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.272+0000 7f67224f0740 1 -- 172.21.15.133:0/2138664639 shutdown_connections 2024-09-11T23:41:11.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.272+0000 7f67224f0740 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556cabab3e90 0x556cabab62a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.272+0000 7f67224f0740 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cabaa4240 0x556cabaa1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.272+0000 7f67017fa640 0 client.16686 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:11.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.272+0000 7f67224f0740 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556caba99980 0x556cabaa16a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.272+0000 7f67224f0740 1 --2- 172.21.15.133:0/2138664639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99060 0x556cabaa1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.272+0000 7f67224f0740 1 -- 172.21.15.133:0/2138664639 >> 172.21.15.133:0/2138664639 conn(0x556caba955f0 msgr2=0x556caba9b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.272+0000 7f67224f0740 1 -- 172.21.15.133:0/2138664639 shutdown_connections 2024-09-11T23:41:11.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.272+0000 7f67224f0740 1 -- 172.21.15.133:0/2138664639 wait complete. 2024-09-11T23:41:11.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.272+0000 7f67224f0740 20 client.16686 trim_cache size 0 max 16384 2024-09-11T23:41:11.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f67224f0740 1 Processor -- start 2024-09-11T23:41:11.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f67224f0740 1 -- start start 2024-09-11T23:41:11.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f67224f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556caba9bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f67224f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556caba94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f67224f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cababbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f67224f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67181ad6d0 con 0x556cabaa4240 2024-09-11T23:41:11.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f67224f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67181af210 con 0x556caba99060 2024-09-11T23:41:11.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f67224f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67181adda0 con 0x556caba99980 2024-09-11T23:41:11.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f671e6b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cababbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f671e6b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cababbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33828/0 (socket says 172.21.15.133:33828) 2024-09-11T23:41:11.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f671e6b4640 1 -- 172.21.15.133:0/2806891873 learned_addr learned my addr 172.21.15.133:0/2806891873 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:11.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f671deb3640 1 --2- 172.21.15.133:0/2806891873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556caba9bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f671d6b2640 1 --2- 172.21.15.133:0/2806891873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556caba94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f671e6b4640 1 -- 172.21.15.133:0/2806891873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 msgr2=0x556caba94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f671e6b4640 1 --2- 172.21.15.133:0/2806891873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556caba94640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f671e6b4640 1 -- 172.21.15.133:0/2806891873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 msgr2=0x556caba9bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f671e6b4640 1 --2- 172.21.15.133:0/2806891873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556caba9bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.289+0000 7f671e6b4640 1 -- 172.21.15.133:0/2806891873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556caba93bb0 con 0x556cabaa4240 2024-09-11T23:41:11.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.290+0000 7f671e6b4640 1 --2- 172.21.15.133:0/2806891873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cababbb20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6710005e30 tx=0x7f6710031e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502110829502e603 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.290+0000 7f67037fe640 1 -- 172.21.15.133:0/2806891873 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710031670 con 0x556cabaa4240 2024-09-11T23:41:11.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.290+0000 7f67037fe640 1 -- 172.21.15.133:0/2806891873 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6710036040 con 0x556cabaa4240 2024-09-11T23:41:11.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.290+0000 7f67037fe640 1 -- 172.21.15.133:0/2806891873 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710003ed0 con 0x556cabaa4240 2024-09-11T23:41:11.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.291+0000 7f67224f0740 1 -- 172.21.15.133:0/2806891873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 msgr2=0x556cababbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.291+0000 7f67224f0740 1 --2- 172.21.15.133:0/2806891873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cababbb20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6710005e30 tx=0x7f6710031e70 comp rx=0 tx=0).stop 2024-09-11T23:41:11.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.291+0000 7f67224f0740 1 -- 172.21.15.133:0/2806891873 shutdown_connections 2024-09-11T23:41:11.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.291+0000 7f67224f0740 1 --2- 172.21.15.133:0/2806891873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cababbb20 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.291+0000 7f67224f0740 1 --2- 172.21.15.133:0/2806891873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556caba94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.291+0000 7f67224f0740 1 --2- 172.21.15.133:0/2806891873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556caba9bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.291+0000 7f67224f0740 1 -- 172.21.15.133:0/2806891873 >> 172.21.15.133:0/2806891873 conn(0x556caba955f0 msgr2=0x556caba93f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.291+0000 7f67224f0740 1 -- 172.21.15.133:0/2806891873 shutdown_connections 2024-09-11T23:41:11.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.291+0000 7f67224f0740 1 -- 172.21.15.133:0/2806891873 wait complete. 2024-09-11T23:41:11.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.291+0000 7f67224f0740 1 -- start start 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f67224f0740 1 Processor -- start 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f67224f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556cabaac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f67224f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556cabaac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f67224f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cabaa91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f67224f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67181aeb40 con 0x556cabaa4240 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f67224f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67181ae470 con 0x556caba99060 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f67224f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67181ab6b0 con 0x556caba99980 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f67224f0740 10 client.0 _ll_register_callbacks cb 0x7f6708098170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f671deb3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556cabaac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f671e6b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cabaa91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f671deb3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556cabaac1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59416/0 (socket says 172.21.15.133:59416) 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f671deb3640 1 -- 172.21.15.133:0/2498615863 learned_addr learned my addr 172.21.15.133:0/2498615863 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:11.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.292+0000 7f671d6b2640 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556cabaac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f671deb3640 1 -- 172.21.15.133:0/2498615863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 msgr2=0x556cabaac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f671deb3640 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556cabaac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f671deb3640 1 -- 172.21.15.133:0/2498615863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 msgr2=0x556cabaa91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f671deb3640 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cabaa91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f671deb3640 1 -- 172.21.15.133:0/2498615863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cabaacde0 con 0x556caba99060 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f671deb3640 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556cabaac1e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f671401e440 tx=0x7f67140046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=622e89d6b1fe4875 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f67017fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f67017fa640 1 -- 172.21.15.133:0/2498615863 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67140049b0 con 0x556caba99060 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f67017fa640 1 -- 172.21.15.133:0/2498615863 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f6714004b50 con 0x556caba99060 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f67224f0740 1 -- 172.21.15.133:0/2498615863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6710036040 con 0x556caba99060 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f67017fa640 1 -- 172.21.15.133:0/2498615863 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6714004d50 con 0x556caba99060 2024-09-11T23:41:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f67224f0740 10 client.26144 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f67224f0740 1 -- 172.21.15.133:0/2498615863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556cabb1f100 con 0x556caba99060 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.293+0000 7f67017fa640 1 -- 172.21.15.133:0/2498615863 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6710036040 con 0x556caba99060 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.294+0000 7f67017fa640 1 -- 172.21.15.133:0/2498615863 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6714037030 con 0x556caba99060 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.294+0000 7f67224f0740 10 client.26144 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.294+0000 7f67224f0740 10 client.26144 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.294+0000 7f67224f0740 20 client.26144 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.294+0000 7f67224f0740 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556cabb1f850 0x556cabb21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.294+0000 7f67224f0740 4 client.26144 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c1232923", "release": true, "await": true}] 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.294+0000 7f67224f0740 1 -- 172.21.15.133:0/2498615863 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c1232923", "release": true, "await": true}) -- 0x556caba060a0 con 0x556cabb1f850 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.294+0000 7f671d6b2640 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556cabb1f850 0x556cabb21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.294+0000 7f671d6b2640 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556cabb1f850 0x556cabb21c70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=244ed536d5e52dcb server_cookie=4b521061441bb2e5 in_seq=0 out_seq=0 2024-09-11T23:41:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.294+0000 7f67017fa640 10 client.26144 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:11.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.298+0000 7f67017fa640 1 -- 172.21.15.133:0/2498615863 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556caba060a0 con 0x556cabb1f850 2024-09-11T23:41:11.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.298+0000 7f67017fa640 10 client.26144 handle_command_reply: tid=0 2024-09-11T23:41:11.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.298+0000 7f67224f0740 1 client.26144 shutdown 2024-09-11T23:41:11.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.298+0000 7f67224f0740 1 -- 172.21.15.133:0/2498615863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 msgr2=0x556cabaac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.298+0000 7f67224f0740 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556cabaac1e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f671401e440 tx=0x7f67140046c0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.299+0000 7f67224f0740 1 -- 172.21.15.133:0/2498615863 shutdown_connections 2024-09-11T23:41:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.299+0000 7f67224f0740 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556cabb1f850 0x556cabb21c70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.299+0000 7f67017fa640 0 client.26144 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.299+0000 7f67224f0740 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cabaa4240 0x556cabaa91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.299+0000 7f67224f0740 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556caba99980 0x556cabaac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.299+0000 7f67224f0740 1 --2- 172.21.15.133:0/2498615863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556caba99060 0x556cabaac1e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.299+0000 7f67224f0740 1 -- 172.21.15.133:0/2498615863 >> 172.21.15.133:0/2498615863 conn(0x556caba955f0 msgr2=0x556cabaa1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.299+0000 7f67224f0740 1 -- 172.21.15.133:0/2498615863 shutdown_connections 2024-09-11T23:41:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.299+0000 7f67224f0740 1 -- 172.21.15.133:0/2498615863 wait complete. 2024-09-11T23:41:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.299+0000 7f67224f0740 20 client.26144 trim_cache size 0 max 16384 2024-09-11T23:41:11.303 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:41:11.303 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 28, 2024-09-11T23:41:11.303 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:41:11.303 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 16, 2024-09-11T23:41:11.303 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:41:11.303 INFO:teuthology.orchestra.run.smithi133.stdout: "c1232923": { 2024-09-11T23:41:11.303 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 16, 2024-09-11T23:41:11.303 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:41:11.303 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:41:11.303 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:41:11.303 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 140.2, 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:11.304 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:11.305 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:11.305 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:41:11.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 -- 172.21.15.133:0/517521910 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f67040a2cf0 msgr2=0x7f67040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f67040a2cf0 0x7f67040a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6714005e00 tx=0x7f6714005d90 comp rx=0 tx=0).stop 2024-09-11T23:41:11.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 -- 172.21.15.133:0/517521910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718101420 msgr2=0x7f67181997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:11.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718101420 0x7f67181997b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f670800e9f0 tx=0x7f670800edf0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 -- 172.21.15.133:0/517521910 shutdown_connections 2024-09-11T23:41:11.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f67040a2cf0 0x7f67040a51b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718106a80 0x7f67181a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718101420 0x7f67181997b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 --2- 172.21.15.133:0/517521910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6718100a80 0x7f6718199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:11.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 -- 172.21.15.133:0/517521910 >> 172.21.15.133:0/517521910 conn(0x7f67180fc7c0 msgr2=0x7f67180fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:11.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 -- 172.21.15.133:0/517521910 shutdown_connections 2024-09-11T23:41:11.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:11.300+0000 7f6720117640 1 -- 172.21.15.133:0/517521910 wait complete. 2024-09-11T23:41:11.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:11 smithi178 ceph-mon[35882]: pgmap v135: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 97 KiB/s rd, 111 MiB/s wr, 143 op/s 2024-09-11T23:41:11.818 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:11 smithi133 ceph-mon[31430]: pgmap v135: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 97 KiB/s rd, 111 MiB/s wr, 143 op/s 2024-09-11T23:41:11.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:11 smithi138 ceph-mon[35142]: pgmap v135: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 97 KiB/s rd, 111 MiB/s wr, 143 op/s 2024-09-11T23:41:12.461 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c1232923', total seconds elapsed: 24.1 2024-09-11T23:41:12.461 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T23:41:12.461 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:41:13.256 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:14.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.184+0000 7fb0313df640 1 -- 172.21.15.133:0/2760176783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02c105d90 msgr2=0x7fb02c106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:14.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.184+0000 7fb0313df640 1 --2- 172.21.15.133:0/2760176783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02c105d90 0x7fb02c106220 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb024009ca0 tx=0x7fb02402a940 comp rx=0 tx=0).stop 2024-09-11T23:41:14.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.185+0000 7fb0313df640 1 -- 172.21.15.133:0/2760176783 shutdown_connections 2024-09-11T23:41:14.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.185+0000 7fb0313df640 1 --2- 172.21.15.133:0/2760176783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02c105d90 0x7fb02c106220 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:14.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.185+0000 7fb0313df640 1 --2- 172.21.15.133:0/2760176783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02c1012a0 0x7fb02c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:14.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.185+0000 7fb0313df640 1 --2- 172.21.15.133:0/2760176783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb02c100980 0x7fb02c100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:14.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.185+0000 7fb0313df640 1 -- 172.21.15.133:0/2760176783 >> 172.21.15.133:0/2760176783 conn(0x7fb02c0fc610 msgr2=0x7fb02c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:14.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.185+0000 7fb0313df640 1 -- 172.21.15.133:0/2760176783 shutdown_connections 2024-09-11T23:41:14.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.185+0000 7fb0313df640 1 -- 172.21.15.133:0/2760176783 wait complete. 2024-09-11T23:41:14.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.186+0000 7fb0313df640 1 Processor -- start 2024-09-11T23:41:14.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.186+0000 7fb0313df640 1 -- start start 2024-09-11T23:41:14.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.187+0000 7fb0313df640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb02c100980 0x7fb02c114aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:14.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.187+0000 7fb0313df640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02c1012a0 0x7fb02c114fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:14.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.187+0000 7fb0313df640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02c105d90 0x7fb02c10dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:14.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.187+0000 7fb0313df640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c118410 con 0x7fb02c1012a0 2024-09-11T23:41:14.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.187+0000 7fb0313df640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c118290 con 0x7fb02c105d90 2024-09-11T23:41:14.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.187+0000 7fb0313df640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c118590 con 0x7fb02c100980 2024-09-11T23:41:14.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.188+0000 7fb02bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb02c100980 0x7fb02c114aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:14.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.188+0000 7fb02bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb02c100980 0x7fb02c114aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:49574/0 (socket says 172.21.15.133:49574) 2024-09-11T23:41:14.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.188+0000 7fb02bfff640 1 -- 172.21.15.133:0/2934969012 learned_addr learned my addr 172.21.15.133:0/2934969012 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:14.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.189+0000 7fb02b7fe640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02c1012a0 0x7fb02c114fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:14.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.189+0000 7fb02b7fe640 1 -- 172.21.15.133:0/2934969012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb02c100980 msgr2=0x7fb02c114aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:14.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.189+0000 7fb02b7fe640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb02c100980 0x7fb02c114aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:14.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.189+0000 7fb02b7fe640 1 -- 172.21.15.133:0/2934969012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02c105d90 msgr2=0x7fb02c10dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:14.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.189+0000 7fb02b7fe640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02c105d90 0x7fb02c10dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:14.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.189+0000 7fb02b7fe640 1 -- 172.21.15.133:0/2934969012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c10e540 con 0x7fb02c1012a0 2024-09-11T23:41:14.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.190+0000 7fb02b7fe640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02c1012a0 0x7fb02c114fe0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb02000d680 tx=0x7fb02000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:14.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.190+0000 7fb0297fa640 1 -- 172.21.15.133:0/2934969012 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020017070 con 0x7fb02c1012a0 2024-09-11T23:41:14.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.190+0000 7fb0313df640 1 -- 172.21.15.133:0/2934969012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02c10e830 con 0x7fb02c1012a0 2024-09-11T23:41:14.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.190+0000 7fb0313df640 1 -- 172.21.15.133:0/2934969012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02c112850 con 0x7fb02c1012a0 2024-09-11T23:41:14.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.192+0000 7fb0313df640 1 -- 172.21.15.133:0/2934969012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02c06a850 con 0x7fb02c1012a0 2024-09-11T23:41:14.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.192+0000 7fb0297fa640 1 -- 172.21.15.133:0/2934969012 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0200097c0 con 0x7fb02c1012a0 2024-09-11T23:41:14.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.192+0000 7fb0297fa640 1 -- 172.21.15.133:0/2934969012 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020020060 con 0x7fb02c1012a0 2024-09-11T23:41:14.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.197+0000 7fb0297fa640 1 -- 172.21.15.133:0/2934969012 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb020004ba0 con 0x7fb02c1012a0 2024-09-11T23:41:14.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.199+0000 7fb0297fa640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb0140a2df0 0x7fb0140a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:14.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.199+0000 7fb0297fa640 1 -- 172.21.15.133:0/2934969012 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fb0200cf5c0 con 0x7fb02c1012a0 2024-09-11T23:41:14.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.201+0000 7fb02bfff640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb0140a2df0 0x7fb0140a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:14.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.204+0000 7fb0297fa640 1 -- 172.21.15.133:0/2934969012 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb02009c1f0 con 0x7fb02c1012a0 2024-09-11T23:41:14.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.212+0000 7fb02bfff640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb0140a2df0 0x7fb0140a52b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00bc40 tx=0x7fb01c005f30 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.655+0000 7fb0313df640 1 -- 172.21.15.133:0/2934969012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb02c10f060 con 0x7fb02c1012a0 2024-09-11T23:41:14.655 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:14 smithi133 ceph-mon[31430]: pgmap v136: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 97 KiB/s rd, 80 MiB/s wr, 152 op/s 2024-09-11T23:41:14.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.658+0000 7fb0297fa640 1 -- 172.21.15.133:0/2934969012 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7fb02c10f060 con 0x7fb02c1012a0 2024-09-11T23:41:14.658 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:41:14.658 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:41:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:41:14.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.666+0000 7fb0313df640 1 -- 172.21.15.133:0/2934969012 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb0140a2df0 msgr2=0x7fb0140a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:14.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.666+0000 7fb0313df640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb0140a2df0 0x7fb0140a52b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00bc40 tx=0x7fb01c005f30 comp rx=0 tx=0).stop 2024-09-11T23:41:14.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.666+0000 7fb0313df640 1 -- 172.21.15.133:0/2934969012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02c1012a0 msgr2=0x7fb02c114fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:14.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.666+0000 7fb0313df640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02c1012a0 0x7fb02c114fe0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb02000d680 tx=0x7fb02000db40 comp rx=0 tx=0).stop 2024-09-11T23:41:14.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.667+0000 7fb0313df640 1 -- 172.21.15.133:0/2934969012 shutdown_connections 2024-09-11T23:41:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.667+0000 7fb0313df640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb0140a2df0 0x7fb0140a52b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.668+0000 7fb0313df640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02c105d90 0x7fb02c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.668+0000 7fb0313df640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02c1012a0 0x7fb02c114fe0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.668+0000 7fb0313df640 1 --2- 172.21.15.133:0/2934969012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb02c100980 0x7fb02c114aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.668+0000 7fb0313df640 1 -- 172.21.15.133:0/2934969012 >> 172.21.15.133:0/2934969012 conn(0x7fb02c0fc610 msgr2=0x7fb02c103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.668+0000 7fb0313df640 1 -- 172.21.15.133:0/2934969012 shutdown_connections 2024-09-11T23:41:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:14.668+0000 7fb0313df640 1 -- 172.21.15.133:0/2934969012 wait complete. 2024-09-11T23:41:14.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:14 smithi138 ceph-mon[35142]: pgmap v136: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 97 KiB/s rd, 80 MiB/s wr, 152 op/s 2024-09-11T23:41:14.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:14 smithi178 ceph-mon[35882]: pgmap v136: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 97 KiB/s rd, 80 MiB/s wr, 152 op/s 2024-09-11T23:41:15.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:15 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2934969012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:15.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:15 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2934969012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:15.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:15 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2934969012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:15.952 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T23:41:15.953 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T23:41:16.625 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:16.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:16 smithi138 ceph-mon[35142]: pgmap v137: 97 pgs: 97 active+clean; 30 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 17 KiB/s rd, 170 MiB/s wr, 584 op/s 2024-09-11T23:41:16.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:16 smithi133 ceph-mon[31430]: pgmap v137: 97 pgs: 97 active+clean; 30 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 17 KiB/s rd, 170 MiB/s wr, 584 op/s 2024-09-11T23:41:16.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:16 smithi178 ceph-mon[35882]: pgmap v137: 97 pgs: 97 active+clean; 30 GiB data, 126 GiB used, 946 GiB / 1.0 TiB avail; 17 KiB/s rd, 170 MiB/s wr, 584 op/s 2024-09-11T23:41:17.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.656+0000 7ff4815a1640 1 -- 172.21.15.133:0/900077864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff47c100de0 msgr2=0x7ff47c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.656+0000 7ff4815a1640 1 --2- 172.21.15.133:0/900077864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff47c100de0 0x7ff47c104e30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff474005fd0 tx=0x7ff47402a6d0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.656+0000 7ff4815a1640 1 -- 172.21.15.133:0/900077864 shutdown_connections 2024-09-11T23:41:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.656+0000 7ff4815a1640 1 --2- 172.21.15.133:0/900077864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff47c105560 0x7ff47c10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.656+0000 7ff4815a1640 1 --2- 172.21.15.133:0/900077864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff47c100de0 0x7ff47c104e30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.656+0000 7ff4815a1640 1 --2- 172.21.15.133:0/900077864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47c100430 0x7ff47c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.656+0000 7ff4815a1640 1 -- 172.21.15.133:0/900077864 >> 172.21.15.133:0/900077864 conn(0x7ff47c077230 msgr2=0x7ff47c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.657+0000 7ff4815a1640 1 -- 172.21.15.133:0/900077864 shutdown_connections 2024-09-11T23:41:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.657+0000 7ff4815a1640 1 -- 172.21.15.133:0/900077864 wait complete. 2024-09-11T23:41:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.658+0000 7ff4815a1640 1 Processor -- start 2024-09-11T23:41:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.659+0000 7ff4815a1640 1 -- start start 2024-09-11T23:41:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.659+0000 7ff4815a1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff47c100430 0x7ff47c073700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.659+0000 7ff4815a1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47c105560 0x7ff47c073c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.659+0000 7ff4815a1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff47c06c790 0x7ff47c06cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.659+0000 7ff4815a1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c10fd90 con 0x7ff47c100430 2024-09-11T23:41:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.659+0000 7ff4815a1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c10fc10 con 0x7ff47c105560 2024-09-11T23:41:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.659+0000 7ff4815a1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c10ff10 con 0x7ff47c06c790 2024-09-11T23:41:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.664+0000 7ff47a7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47c105560 0x7ff47c073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.664+0000 7ff47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff47c100430 0x7ff47c073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.664+0000 7ff47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff47c100430 0x7ff47c073700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50922/0 (socket says 172.21.15.133:50922) 2024-09-11T23:41:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.664+0000 7ff47affd640 1 -- 172.21.15.133:0/1396531573 learned_addr learned my addr 172.21.15.133:0/1396531573 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.664+0000 7ff47b7fe640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff47c06c790 0x7ff47c06cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.665+0000 7ff47affd640 1 -- 172.21.15.133:0/1396531573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff47c06c790 msgr2=0x7ff47c06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.665+0000 7ff47affd640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff47c06c790 0x7ff47c06cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.665+0000 7ff47affd640 1 -- 172.21.15.133:0/1396531573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47c105560 msgr2=0x7ff47c073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.665+0000 7ff47affd640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47c105560 0x7ff47c073c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.665+0000 7ff47affd640 1 -- 172.21.15.133:0/1396531573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c06d410 con 0x7ff47c100430 2024-09-11T23:41:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.666+0000 7ff47affd640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff47c100430 0x7ff47c073700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00bfd0 tx=0x7ff46c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.667+0000 7ff463fff640 1 -- 172.21.15.133:0/1396531573 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c009390 con 0x7ff47c100430 2024-09-11T23:41:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.667+0000 7ff4815a1640 1 -- 172.21.15.133:0/1396531573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47c071230 con 0x7ff47c100430 2024-09-11T23:41:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.667+0000 7ff4815a1640 1 -- 172.21.15.133:0/1396531573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47c071740 con 0x7ff47c100430 2024-09-11T23:41:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.669+0000 7ff463fff640 1 -- 172.21.15.133:0/1396531573 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff46c009a10 con 0x7ff47c100430 2024-09-11T23:41:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.669+0000 7ff463fff640 1 -- 172.21.15.133:0/1396531573 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0138a0 con 0x7ff47c100430 2024-09-11T23:41:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.677+0000 7ff463fff640 1 -- 172.21.15.133:0/1396531573 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff46c013a60 con 0x7ff47c100430 2024-09-11T23:41:17.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.679+0000 7ff463fff640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4640a2e80 0x7ff4640a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.679+0000 7ff48397a740 1 Processor -- start 2024-09-11T23:41:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.679+0000 7ff48397a740 1 -- start start 2024-09-11T23:41:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.679+0000 7ff48397a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.679+0000 7ff48397a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.679+0000 7ff48397a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290baa30 0x55a6290bce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.680+0000 7ff47a7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.680+0000 7ff47a7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290ba4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:49616/0 (socket says 172.21.15.133:49616) 2024-09-11T23:41:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.682+0000 7ff47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.682+0000 7ff47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50934/0 (socket says 172.21.15.133:50934) 2024-09-11T23:41:17.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.683+0000 7ff47a7fc640 1 -- 172.21.15.133:0/755996785 learned_addr learned my addr 172.21.15.133:0/755996785 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:17.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.683+0000 7ff48397a740 1 -- 172.21.15.133:0/755996785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a629014160 con 0x55a6290af850 2024-09-11T23:41:17.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.683+0000 7ff48397a740 1 -- 172.21.15.133:0/755996785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a628fbb720 con 0x55a6290baa30 2024-09-11T23:41:17.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.683+0000 7ff48397a740 1 -- 172.21.15.133:0/755996785 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a629010120 con 0x55a6290b0170 2024-09-11T23:41:17.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.684+0000 7ff47b7fe640 1 --2- 172.21.15.133:0/755996785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290baa30 0x55a6290bce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.686+0000 7ff47b7fe640 1 -- 172.21.15.133:0/755996785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 msgr2=0x55a6290ba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:17.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.686+0000 7ff47b7fe640 1 --2- 172.21.15.133:0/755996785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290ba4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.686+0000 7ff47b7fe640 1 -- 172.21.15.133:0/755996785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 msgr2=0x55a6290afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:17.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.686+0000 7ff47b7fe640 1 --2- 172.21.15.133:0/755996785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.686+0000 7ff47b7fe640 1 -- 172.21.15.133:0/755996785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a628d85ba0 con 0x55a6290baa30 2024-09-11T23:41:17.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.687+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/755996785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:41:17.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.687+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4640a2e80 0x7ff4640a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.687+0000 7ff47affd640 1 --2- 172.21.15.133:0/755996785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:41:17.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.687+0000 7ff463fff640 1 -- 172.21.15.133:0/1396531573 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff46c09ae50 con 0x7ff47c100430 2024-09-11T23:41:17.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.687+0000 7ff47b7fe640 1 --2- 172.21.15.133:0/755996785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290baa30 0x55a6290bce20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff47000dd80 tx=0x7ff470005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=862c2124ddb99738 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:17.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.688+0000 7ff4617fa640 1 -- 172.21.15.133:0/755996785 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4700044d0 con 0x55a6290baa30 2024-09-11T23:41:17.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.688+0000 7ff4617fa640 1 -- 172.21.15.133:0/755996785 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff470003de0 con 0x55a6290baa30 2024-09-11T23:41:17.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.688+0000 7ff4617fa640 1 -- 172.21.15.133:0/755996785 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470004d60 con 0x55a6290baa30 2024-09-11T23:41:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.690+0000 7ff48397a740 1 -- 172.21.15.133:0/755996785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290baa30 msgr2=0x55a6290bce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.690+0000 7ff48397a740 1 --2- 172.21.15.133:0/755996785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290baa30 0x55a6290bce20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff47000dd80 tx=0x7ff470005f90 comp rx=0 tx=0).stop 2024-09-11T23:41:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.691+0000 7ff48397a740 1 -- 172.21.15.133:0/755996785 shutdown_connections 2024-09-11T23:41:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.691+0000 7ff48397a740 1 --2- 172.21.15.133:0/755996785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290baa30 0x55a6290bce20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.691+0000 7ff48397a740 1 --2- 172.21.15.133:0/755996785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.691+0000 7ff48397a740 1 --2- 172.21.15.133:0/755996785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.691+0000 7ff48397a740 1 -- 172.21.15.133:0/755996785 >> 172.21.15.133:0/755996785 conn(0x55a6290abde0 msgr2=0x55a6290ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.694+0000 7ff48397a740 1 -- 172.21.15.133:0/755996785 shutdown_connections 2024-09-11T23:41:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.694+0000 7ff48397a740 1 -- 172.21.15.133:0/755996785 wait complete. 2024-09-11T23:41:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.695+0000 7ff48397a740 1 -- start start 2024-09-11T23:41:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.696+0000 7ff48397a740 1 Processor -- start 2024-09-11T23:41:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.698+0000 7ff48397a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.698+0000 7ff48397a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290b0170 0x55a6290b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.698+0000 7ff48397a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290baa30 0x55a6290b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.698+0000 7ff48397a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a6290bd600 con 0x55a6290af850 2024-09-11T23:41:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.698+0000 7ff48397a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a628f9c900 con 0x55a6290b0170 2024-09-11T23:41:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.698+0000 7ff48397a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a6290bd780 con 0x55a6290baa30 2024-09-11T23:41:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.698+0000 7ff48397a740 10 client.0 _ll_register_callbacks cb 0x55a628f9a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.699+0000 7ff47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.699+0000 7ff47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290b7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50944/0 (socket says 172.21.15.133:50944) 2024-09-11T23:41:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.699+0000 7ff47affd640 1 -- 172.21.15.133:0/89159864 learned_addr learned my addr 172.21.15.133:0/89159864 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.700+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290b0170 0x55a6290b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.700+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4640a2e80 0x7ff4640a5340 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55a6290acc40 tx=0x7ff47402a6d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.700+0000 7ff47b7fe640 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290baa30 0x55a6290b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.701+0000 7ff47a7fc640 1 -- 172.21.15.133:0/89159864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290baa30 msgr2=0x55a6290b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.701+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290baa30 0x55a6290b82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.701+0000 7ff47a7fc640 1 -- 172.21.15.133:0/89159864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 msgr2=0x55a6290b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.701+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290b7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.701+0000 7ff47a7fc640 1 -- 172.21.15.133:0/89159864 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a628fdcc50 con 0x55a6290b0170 2024-09-11T23:41:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.703+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290b0170 0x55a6290b7d70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff47400aa80 tx=0x7ff47400a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc15bd46be39f48 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.704+0000 7ff45eff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:41:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.704+0000 7ff45eff5640 1 -- 172.21.15.133:0/89159864 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47402f920 con 0x55a6290b0170 2024-09-11T23:41:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.704+0000 7ff48397a740 1 -- 172.21.15.133:0/89159864 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff470003de0 con 0x55a6290b0170 2024-09-11T23:41:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.704+0000 7ff45eff5640 1 -- 172.21.15.133:0/89159864 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff47402fac0 con 0x55a6290b0170 2024-09-11T23:41:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.707+0000 7ff45eff5640 1 -- 172.21.15.133:0/89159864 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff47402fcc0 con 0x55a6290b0170 2024-09-11T23:41:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.707+0000 7ff48397a740 10 client.26159 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.707+0000 7ff48397a740 1 -- 172.21.15.133:0/89159864 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6290c9200 con 0x55a6290b0170 2024-09-11T23:41:17.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.709+0000 7ff45eff5640 1 -- 172.21.15.133:0/89159864 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff470003de0 con 0x55a6290b0170 2024-09-11T23:41:17.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.709+0000 7ff45eff5640 1 -- 172.21.15.133:0/89159864 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff474059020 con 0x55a6290b0170 2024-09-11T23:41:17.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.709+0000 7ff48397a740 10 client.26159 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:17.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.709+0000 7ff48397a740 10 client.26159 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:41:17.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.709+0000 7ff48397a740 20 client.26159 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:17.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.709+0000 7ff48397a740 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6290ca710 0x55a6290ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.710+0000 7ff48397a740 4 client.26159 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:17.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.710+0000 7ff48397a740 1 -- 172.21.15.133:0/89159864 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a629021890 con 0x55a6290ca710 2024-09-11T23:41:17.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.712+0000 7ff47affd640 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6290ca710 0x55a6290ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.713+0000 7ff47affd640 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6290ca710 0x55a6290ccb50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=619bbe4a69668bc1 server_cookie=cb4109388887f4cf in_seq=0 out_seq=0 2024-09-11T23:41:17.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.715+0000 7ff45eff5640 10 client.26159 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.716+0000 7ff45eff5640 1 -- 172.21.15.133:0/89159864 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a629021890 con 0x55a6290ca710 2024-09-11T23:41:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.716+0000 7ff45eff5640 10 client.26159 handle_command_reply: tid=0 2024-09-11T23:41:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.716+0000 7ff48397a740 1 client.26159 shutdown 2024-09-11T23:41:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.716+0000 7ff48397a740 1 -- 172.21.15.133:0/89159864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290b0170 msgr2=0x55a6290b7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.717+0000 7ff48397a740 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290b0170 0x55a6290b7d70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff47400aa80 tx=0x7ff47400a750 comp rx=0 tx=0).stop 2024-09-11T23:41:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.717+0000 7ff48397a740 1 -- 172.21.15.133:0/89159864 shutdown_connections 2024-09-11T23:41:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.717+0000 7ff48397a740 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6290ca710 0x55a6290ccb50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.717+0000 7ff48397a740 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290baa30 0x55a6290b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.717+0000 7ff48397a740 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290b0170 0x55a6290b7d70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.717+0000 7ff48397a740 1 --2- 172.21.15.133:0/89159864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290b7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.717+0000 7ff48397a740 1 -- 172.21.15.133:0/89159864 >> 172.21.15.133:0/89159864 conn(0x55a6290abde0 msgr2=0x55a6290b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.717+0000 7ff45eff5640 0 client.26159 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.718+0000 7ff48397a740 1 -- 172.21.15.133:0/89159864 shutdown_connections 2024-09-11T23:41:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.718+0000 7ff48397a740 1 -- 172.21.15.133:0/89159864 wait complete. 2024-09-11T23:41:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.718+0000 7ff48397a740 20 client.26159 trim_cache size 0 max 16384 2024-09-11T23:41:17.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.758+0000 7ff48397a740 1 Processor -- start 2024-09-11T23:41:17.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.758+0000 7ff48397a740 1 -- start start 2024-09-11T23:41:17.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.758+0000 7ff48397a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.759+0000 7ff48397a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290d2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.761 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:17 smithi133 ceph-mon[31430]: pgmap v138: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 198 MiB/s wr, 767 op/s 2024-09-11T23:41:17.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.760+0000 7ff47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.760+0000 7ff47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290b98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50958/0 (socket says 172.21.15.133:50958) 2024-09-11T23:41:17.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.761+0000 7ff47affd640 1 -- 172.21.15.133:0/544118038 learned_addr learned my addr 172.21.15.133:0/544118038 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:17.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.761+0000 7ff48397a740 1 --2- 172.21.15.133:0/544118038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290baa30 0x55a6290d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.761+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/544118038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.762+0000 7ff48397a740 1 -- 172.21.15.133:0/544118038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff46800e6e0 con 0x55a6290af850 2024-09-11T23:41:17.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.762+0000 7ff48397a740 1 -- 172.21.15.133:0/544118038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff468010220 con 0x55a6290baa30 2024-09-11T23:41:17.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.762+0000 7ff48397a740 1 -- 172.21.15.133:0/544118038 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff46800edb0 con 0x55a6290b0170 2024-09-11T23:41:17.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.763+0000 7ff47b7fe640 1 --2- 172.21.15.133:0/544118038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290baa30 0x55a6290d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.764+0000 7ff47a7fc640 1 -- 172.21.15.133:0/544118038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290baa30 msgr2=0x55a6290d29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:17.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.764+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/544118038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290baa30 0x55a6290d29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.764+0000 7ff47a7fc640 1 -- 172.21.15.133:0/544118038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 msgr2=0x55a6290b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:17.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.764+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/544118038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.764+0000 7ff47a7fc640 1 -- 172.21.15.133:0/544118038 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6290af680 con 0x55a6290b0170 2024-09-11T23:41:17.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.766+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/544118038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290d2460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff4740314d0 tx=0x7ff474005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c2f5f3f236fc7e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.767+0000 7ff460ff9640 1 -- 172.21.15.133:0/544118038 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4740318c0 con 0x55a6290b0170 2024-09-11T23:41:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.767+0000 7ff460ff9640 1 -- 172.21.15.133:0/544118038 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff47405f070 con 0x55a6290b0170 2024-09-11T23:41:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.767+0000 7ff460ff9640 1 -- 172.21.15.133:0/544118038 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4740519b0 con 0x55a6290b0170 2024-09-11T23:41:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.771+0000 7ff48397a740 1 -- 172.21.15.133:0/544118038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 msgr2=0x55a6290d2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.771+0000 7ff48397a740 1 --2- 172.21.15.133:0/544118038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290d2460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff4740314d0 tx=0x7ff474005cf0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.771+0000 7ff48397a740 1 -- 172.21.15.133:0/544118038 shutdown_connections 2024-09-11T23:41:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.771+0000 7ff48397a740 1 --2- 172.21.15.133:0/544118038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290baa30 0x55a6290d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.771+0000 7ff48397a740 1 --2- 172.21.15.133:0/544118038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290d2460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.771+0000 7ff48397a740 1 --2- 172.21.15.133:0/544118038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290af850 0x55a6290b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.771+0000 7ff48397a740 1 -- 172.21.15.133:0/544118038 >> 172.21.15.133:0/544118038 conn(0x55a6290abde0 msgr2=0x55a6290b69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.772+0000 7ff48397a740 1 -- 172.21.15.133:0/544118038 shutdown_connections 2024-09-11T23:41:17.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.772+0000 7ff48397a740 1 -- 172.21.15.133:0/544118038 wait complete. 2024-09-11T23:41:17.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.773+0000 7ff48397a740 1 -- start start 2024-09-11T23:41:17.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.773+0000 7ff48397a740 1 Processor -- start 2024-09-11T23:41:17.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.774+0000 7ff48397a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290af850 0x55a6290bea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.774+0000 7ff48397a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290bef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.774+0000 7ff48397a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290baa30 0x55a6290bf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.774+0000 7ff48397a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff46800fb50 con 0x55a6290baa30 2024-09-11T23:41:17.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.774+0000 7ff48397a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff46800f480 con 0x55a6290af850 2024-09-11T23:41:17.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.774+0000 7ff48397a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff46800c6c0 con 0x55a6290b0170 2024-09-11T23:41:17.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.774+0000 7ff48397a740 10 client.0 _ll_register_callbacks cb 0x7ff46c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:17.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.777+0000 7ff47affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290af850 0x55a6290bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.777+0000 7ff47affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290af850 0x55a6290bea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53756/0 (socket says 172.21.15.133:53756) 2024-09-11T23:41:17.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.777+0000 7ff47affd640 1 -- 172.21.15.133:0/1796681081 learned_addr learned my addr 172.21.15.133:0/1796681081 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:17.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.777+0000 7ff47b7fe640 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290baa30 0x55a6290bf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.777+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290bef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.778+0000 7ff47affd640 1 -- 172.21.15.133:0/1796681081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 msgr2=0x55a6290bef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:17.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.778+0000 7ff47affd640 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290bef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.778+0000 7ff47affd640 1 -- 172.21.15.133:0/1796681081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290baa30 msgr2=0x55a6290bf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:17.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.778+0000 7ff47affd640 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290baa30 0x55a6290bf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:17.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.779+0000 7ff47affd640 1 -- 172.21.15.133:0/1796681081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6290b26c0 con 0x55a6290af850 2024-09-11T23:41:17.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.779+0000 7ff47affd640 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290af850 0x55a6290bea50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff46c007b50 tx=0x7ff46c007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9e6f154ea4dd2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:17.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.779+0000 7ff45eff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:41:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.780+0000 7ff45eff5640 1 -- 172.21.15.133:0/1796681081 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0a1160 con 0x55a6290af850 2024-09-11T23:41:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.780+0000 7ff45eff5640 1 -- 172.21.15.133:0/1796681081 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7ff46c013d50 con 0x55a6290af850 2024-09-11T23:41:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.780+0000 7ff48397a740 1 -- 172.21.15.133:0/1796681081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff47405f070 con 0x55a6290af850 2024-09-11T23:41:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.780+0000 7ff45eff5640 1 -- 172.21.15.133:0/1796681081 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff46c0a5a20 con 0x55a6290af850 2024-09-11T23:41:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.780+0000 7ff48397a740 10 client.26165 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.780+0000 7ff48397a740 1 -- 172.21.15.133:0/1796681081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a629135760 con 0x55a6290af850 2024-09-11T23:41:17.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.781+0000 7ff45eff5640 1 -- 172.21.15.133:0/1796681081 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff47405f070 con 0x55a6290af850 2024-09-11T23:41:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.782+0000 7ff45eff5640 1 -- 172.21.15.133:0/1796681081 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff46c026060 con 0x55a6290af850 2024-09-11T23:41:17.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.783+0000 7ff48397a740 10 client.26165 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:17.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.783+0000 7ff48397a740 10 client.26165 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:41:17.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.783+0000 7ff48397a740 20 client.26165 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:17.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.784+0000 7ff48397a740 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a629135ef0 0x55a6291382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:17.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.784+0000 7ff48397a740 4 client.26165 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T23:41:17.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.784+0000 7ff48397a740 1 -- 172.21.15.133:0/1796681081 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x55a62901bbe0 con 0x55a629135ef0 2024-09-11T23:41:17.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.784+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a629135ef0 0x55a6291382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:17.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.784+0000 7ff47a7fc640 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a629135ef0 0x55a6291382e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2cfed4a168b4097 server_cookie=e11783f0195910 in_seq=0 out_seq=0 2024-09-11T23:41:17.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:17.785+0000 7ff45eff5640 10 client.26165 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:17.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:17 smithi178 ceph-mon[35882]: pgmap v138: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 198 MiB/s wr, 767 op/s 2024-09-11T23:41:17.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:17 smithi138 ceph-mon[35142]: pgmap v138: 97 pgs: 97 active+clean; 30 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 198 MiB/s wr, 767 op/s 2024-09-11T23:41:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff45eff5640 1 -- 172.21.15.133:0/1796681081 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a62901bbe0 con 0x55a629135ef0 2024-09-11T23:41:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff45eff5640 10 client.26165 handle_command_reply: tid=0 2024-09-11T23:41:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff48397a740 1 client.26165 shutdown 2024-09-11T23:41:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff48397a740 1 -- 172.21.15.133:0/1796681081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290af850 msgr2=0x55a6290bea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff48397a740 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290af850 0x55a6290bea50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff46c007b50 tx=0x7ff46c007c80 comp rx=0 tx=0).stop 2024-09-11T23:41:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff48397a740 1 -- 172.21.15.133:0/1796681081 shutdown_connections 2024-09-11T23:41:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff48397a740 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a629135ef0 0x55a6291382e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff48397a740 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6290baa30 0x55a6290bf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff45eff5640 0 client.26165 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff48397a740 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6290b0170 0x55a6290bef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff48397a740 1 --2- 172.21.15.133:0/1796681081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6290af850 0x55a6290bea50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff48397a740 1 -- 172.21.15.133:0/1796681081 >> 172.21.15.133:0/1796681081 conn(0x55a6290abde0 msgr2=0x55a6290b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff48397a740 1 -- 172.21.15.133:0/1796681081 shutdown_connections 2024-09-11T23:41:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.034+0000 7ff48397a740 1 -- 172.21.15.133:0/1796681081 wait complete. 2024-09-11T23:41:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff48397a740 20 client.26165 trim_cache size 0 max 16384 2024-09-11T23:41:18.036 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:41:18.036 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 28, 2024-09-11T23:41:18.036 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:41:18.036 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 18, 2024-09-11T23:41:18.037 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:41:18.037 INFO:teuthology.orchestra.run.smithi133.stdout: "14340ff8": { 2024-09-11T23:41:18.037 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 18, 2024-09-11T23:41:18.037 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:41:18.037 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:41:18.037 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:41:18.037 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:41:18.037 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 140.2, 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:18.038 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:41:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff4815a1640 1 -- 172.21.15.133:0/1396531573 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4640a2e80 msgr2=0x7ff4640a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff4815a1640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4640a2e80 0x7ff4640a5340 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55a6290acc40 tx=0x7ff47402a6d0 comp rx=0 tx=0).stop 2024-09-11T23:41:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff4815a1640 1 -- 172.21.15.133:0/1396531573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff47c100430 msgr2=0x7ff47c073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff4815a1640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff47c100430 0x7ff47c073700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00bfd0 tx=0x7ff46c007650 comp rx=0 tx=0).stop 2024-09-11T23:41:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff4815a1640 1 -- 172.21.15.133:0/1396531573 shutdown_connections 2024-09-11T23:41:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff4815a1640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4640a2e80 0x7ff4640a5340 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff4815a1640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff47c06c790 0x7ff47c06cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:18.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff4815a1640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47c105560 0x7ff47c073c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:18.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff4815a1640 1 --2- 172.21.15.133:0/1396531573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff47c100430 0x7ff47c073700 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:18.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff4815a1640 1 -- 172.21.15.133:0/1396531573 >> 172.21.15.133:0/1396531573 conn(0x7ff47c077230 msgr2=0x7ff47c103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:18.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.035+0000 7ff4815a1640 1 -- 172.21.15.133:0/1396531573 shutdown_connections 2024-09-11T23:41:18.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:18.036+0000 7ff4815a1640 1 -- 172.21.15.133:0/1396531573 wait complete. 2024-09-11T23:41:18.855 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '14340ff8', quiesce took 6.4 seconds. Will release after: 13.749999999999998 2024-09-11T23:41:20.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:20 smithi138 ceph-mon[35142]: pgmap v139: 97 pgs: 97 active+clean; 31 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 336 KiB/s rd, 406 MiB/s wr, 2.78k op/s 2024-09-11T23:41:20.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:20 smithi133 ceph-mon[31430]: pgmap v139: 97 pgs: 97 active+clean; 31 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 336 KiB/s rd, 406 MiB/s wr, 2.78k op/s 2024-09-11T23:41:20.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:20 smithi178 ceph-mon[35882]: pgmap v139: 97 pgs: 97 active+clean; 31 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 336 KiB/s rd, 406 MiB/s wr, 2.78k op/s 2024-09-11T23:41:22.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:22 smithi138 ceph-mon[35142]: pgmap v140: 97 pgs: 97 active+clean; 31 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 333 KiB/s rd, 427 MiB/s wr, 3.19k op/s 2024-09-11T23:41:22.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:22 smithi133 ceph-mon[31430]: pgmap v140: 97 pgs: 97 active+clean; 31 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 333 KiB/s rd, 427 MiB/s wr, 3.19k op/s 2024-09-11T23:41:22.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:22 smithi178 ceph-mon[35882]: pgmap v140: 97 pgs: 97 active+clean; 31 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 333 KiB/s rd, 427 MiB/s wr, 3.19k op/s 2024-09-11T23:41:23.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:23 smithi133 ceph-mon[31430]: pgmap v141: 97 pgs: 97 active+clean; 31 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 333 KiB/s rd, 422 MiB/s wr, 3.19k op/s 2024-09-11T23:41:23.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:23 smithi178 ceph-mon[35882]: pgmap v141: 97 pgs: 97 active+clean; 31 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 333 KiB/s rd, 422 MiB/s wr, 3.19k op/s 2024-09-11T23:41:23.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:23 smithi138 ceph-mon[35142]: pgmap v141: 97 pgs: 97 active+clean; 31 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 333 KiB/s rd, 422 MiB/s wr, 3.19k op/s 2024-09-11T23:41:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.208+0000 7f2a9d99f640 1 -- 172.21.15.133:0/1726783598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a98101370 msgr2=0x7f2a9810d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.208+0000 7f2a9d99f640 1 --2- 172.21.15.133:0/1726783598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a98101370 0x7f2a9810d8a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009cd0 tx=0x7f2a9002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.208+0000 7f2a9d99f640 1 -- 172.21.15.133:0/1726783598 shutdown_connections 2024-09-11T23:41:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.208+0000 7f2a9d99f640 1 --2- 172.21.15.133:0/1726783598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a98101370 0x7f2a9810d8a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.208+0000 7f2a9d99f640 1 --2- 172.21.15.133:0/1726783598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a981009d0 0x7f2a98100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.208+0000 7f2a9d99f640 1 --2- 172.21.15.133:0/1726783598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a981069e0 0x7f2a98106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.208+0000 7f2a9d99f640 1 -- 172.21.15.133:0/1726783598 >> 172.21.15.133:0/1726783598 conn(0x7f2a980fc6b0 msgr2=0x7f2a980fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.208+0000 7f2a9d99f640 1 -- 172.21.15.133:0/1726783598 shutdown_connections 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.208+0000 7f2a9d99f640 1 -- 172.21.15.133:0/1726783598 wait complete. 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.208+0000 7f2a9d99f640 1 Processor -- start 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a9d99f640 1 -- start start 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a9d99f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a981009d0 0x7f2a98197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a9d99f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a98101370 0x7f2a98197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a9d99f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a981069e0 0x7f2a9819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a9d99f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98110e20 con 0x7f2a98101370 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a9d99f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98110ca0 con 0x7f2a981069e0 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a9d99f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98110fa0 con 0x7f2a981009d0 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a967fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a98101370 0x7f2a98197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a967fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a98101370 0x7f2a98197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38972/0 (socket says 172.21.15.133:38972) 2024-09-11T23:41:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a967fc640 1 -- 172.21.15.133:0/2897844987 learned_addr learned my addr 172.21.15.133:0/2897844987 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a96ffd640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a981009d0 0x7f2a98197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a977fe640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a981069e0 0x7f2a9819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a967fc640 1 -- 172.21.15.133:0/2897844987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a981009d0 msgr2=0x7f2a98197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a967fc640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a981009d0 0x7f2a98197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a967fc640 1 -- 172.21.15.133:0/2897844987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a981069e0 msgr2=0x7f2a9819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a967fc640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a981069e0 0x7f2a9819baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a967fc640 1 -- 172.21.15.133:0/2897844987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9819c220 con 0x7f2a98101370 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.209+0000 7f2a967fc640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a98101370 0x7f2a98197770 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00e9f0 tx=0x7f2a8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.210+0000 7f2a9c99d640 1 -- 172.21.15.133:0/2897844987 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00cd80 con 0x7f2a98101370 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.210+0000 7f2a9d99f640 1 -- 172.21.15.133:0/2897844987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9819c510 con 0x7f2a98101370 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.210+0000 7f2a9c99d640 1 -- 172.21.15.133:0/2897844987 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c005500 con 0x7f2a98101370 2024-09-11T23:41:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.210+0000 7f2a9c99d640 1 -- 172.21.15.133:0/2897844987 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c010690 con 0x7f2a98101370 2024-09-11T23:41:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.210+0000 7f2a9d99f640 1 -- 172.21.15.133:0/2897844987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a981a3da0 con 0x7f2a98101370 2024-09-11T23:41:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.212+0000 7f2a9c99d640 1 -- 172.21.15.133:0/2897844987 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2a8c0046c0 con 0x7f2a98101370 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.212+0000 7f2aa081b740 1 Processor -- start 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.212+0000 7f2aa081b740 1 -- start start 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.212+0000 7f2a9c99d640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2a800a2da0 0x7f2a800a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611474af0 0x55d611474ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611475410 0x55d61147f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d61147fcb0 0x55d6114820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2aa081b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d6113d92f0 con 0x55d611475410 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2aa081b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d611381270 con 0x55d61147fcb0 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2aa081b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d6113d52b0 con 0x55d611474af0 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a977fe640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2a800a2da0 0x7f2a800a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a967fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611475410 0x55d61147f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a967fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611475410 0x55d61147f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38988/0 (socket says 172.21.15.133:38988) 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a967fc640 1 -- 172.21.15.133:0/953974867 learned_addr learned my addr 172.21.15.133:0/953974867 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a96ffd640 1 --2- 172.21.15.133:0/953974867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611474af0 0x55d611474ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a9c99d640 1 -- 172.21.15.133:0/2897844987 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f2a8c0cf910 con 0x7f2a98101370 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a977fe640 1 --2- 172.21.15.133:0/953974867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d61147fcb0 0x55d6114820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a967fc640 1 -- 172.21.15.133:0/953974867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611474af0 msgr2=0x55d611474ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a967fc640 1 --2- 172.21.15.133:0/953974867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611474af0 0x55d611474ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a967fc640 1 -- 172.21.15.133:0/953974867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d61147fcb0 msgr2=0x55d6114820a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a967fc640 1 --2- 172.21.15.133:0/953974867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d61147fcb0 0x55d6114820a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a967fc640 1 -- 172.21.15.133:0/953974867 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d611380340 con 0x55d611475410 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a96ffd640 1 --2- 172.21.15.133:0/953974867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611474af0 0x55d611474ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a977fe640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2a800a2da0 0x7f2a800a5260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a90005fe0 tx=0x7f2a90005c50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a967fc640 1 --2- 172.21.15.133:0/953974867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611475410 0x55d61147f770 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00a410 tx=0x7f2a8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ee0dc56fdfcdae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a7dffb640 1 -- 172.21.15.133:0/953974867 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c09e070 con 0x55d611475410 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a7dffb640 1 -- 172.21.15.133:0/953974867 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c00a250 con 0x55d611475410 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.213+0000 7f2a7dffb640 1 -- 172.21.15.133:0/953974867 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c06e360 con 0x55d611475410 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.214+0000 7f2aa081b740 1 -- 172.21.15.133:0/953974867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611475410 msgr2=0x55d61147f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.214+0000 7f2aa081b740 1 --2- 172.21.15.133:0/953974867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611475410 0x55d61147f770 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00a410 tx=0x7f2a8c004a40 comp rx=0 tx=0).stop 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.214+0000 7f2aa081b740 1 -- 172.21.15.133:0/953974867 shutdown_connections 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.214+0000 7f2aa081b740 1 --2- 172.21.15.133:0/953974867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d61147fcb0 0x55d6114820a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.214+0000 7f2aa081b740 1 --2- 172.21.15.133:0/953974867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611475410 0x55d61147f770 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.214+0000 7f2aa081b740 1 --2- 172.21.15.133:0/953974867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611474af0 0x55d611474ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.214+0000 7f2aa081b740 1 -- 172.21.15.133:0/953974867 >> 172.21.15.133:0/953974867 conn(0x55d611471060 msgr2=0x55d611471440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.214+0000 7f2aa081b740 1 -- 172.21.15.133:0/953974867 shutdown_connections 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.214+0000 7f2aa081b740 1 -- 172.21.15.133:0/953974867 wait complete. 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.214+0000 7f2aa081b740 1 -- start start 2024-09-11T23:41:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2aa081b740 1 Processor -- start 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d611474af0 0x55d61147cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d61147d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d61147d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2aa081b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d611482880 con 0x55d61147fcb0 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2aa081b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d611362290 con 0x55d611474af0 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2aa081b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d611482a00 con 0x55d611475410 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2aa081b740 10 client.0 _ll_register_callbacks cb 0x55d611393140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d61147d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d61147d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38996/0 (socket says 172.21.15.133:38996) 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2a977fe640 1 -- 172.21.15.133:0/1270863534 learned_addr learned my addr 172.21.15.133:0/1270863534 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2a967fc640 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d61147d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2a96ffd640 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d611474af0 0x55d61147cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2a977fe640 1 -- 172.21.15.133:0/1270863534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 msgr2=0x55d61147d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2a977fe640 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d61147d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2a977fe640 1 -- 172.21.15.133:0/1270863534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d611474af0 msgr2=0x55d61147cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2a977fe640 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d611474af0 0x55d61147cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.215+0000 7f2a977fe640 1 -- 172.21.15.133:0/1270863534 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6110ec210 con 0x55d61147fcb0 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2a977fe640 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d61147d660 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2a90035000 tx=0x7f2a900095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e6ff531f70a174 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2a76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2a76ffd640 1 -- 172.21.15.133:0/1270863534 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90004820 con 0x55d61147fcb0 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2a76ffd640 1 -- 172.21.15.133:0/1270863534 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f2a900305a0 con 0x55d61147fcb0 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2aa081b740 1 -- 172.21.15.133:0/1270863534 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d6113e6a20 con 0x55d61147fcb0 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2a76ffd640 1 -- 172.21.15.133:0/1270863534 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a9005a050 con 0x55d61147fcb0 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2aa081b740 10 client.16725 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2aa081b740 1 -- 172.21.15.133:0/1270863534 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6111b8620 con 0x55d61147fcb0 2024-09-11T23:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2a76ffd640 1 -- 172.21.15.133:0/1270863534 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d6113e6a20 con 0x55d61147fcb0 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2a76ffd640 1 -- 172.21.15.133:0/1270863534 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2a90030070 con 0x55d61147fcb0 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2aa081b740 10 client.16725 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2aa081b740 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2aa081b740 20 client.16725 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2aa081b740 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d61148fc40 0x55d611492070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2aa081b740 4 client.16725 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.216+0000 7f2aa081b740 1 -- 172.21.15.133:0/1270863534 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6113e0d70 con 0x55d61148fc40 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.217+0000 7f2a96ffd640 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d61148fc40 0x55d611492070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.217+0000 7f2a76ffd640 10 client.16725 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.217+0000 7f2a96ffd640 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d61148fc40 0x55d611492070 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ffb56633c3b18 server_cookie=a695a63b3b54f61e in_seq=0 out_seq=0 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2a76ffd640 1 -- 172.21.15.133:0/1270863534 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6113e0d70 con 0x55d61148fc40 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2a76ffd640 10 client.16725 handle_command_reply: tid=0 2024-09-11T23:41:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 1 client.16725 shutdown 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 1 -- 172.21.15.133:0/1270863534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 msgr2=0x55d61147d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d61147d660 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2a90035000 tx=0x7f2a900095c0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 1 -- 172.21.15.133:0/1270863534 shutdown_connections 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d61148fc40 0x55d611492070 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d61147d660 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d61147d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 1 --2- 172.21.15.133:0/1270863534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d611474af0 0x55d61147cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2a76ffd640 0 client.16725 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 1 -- 172.21.15.133:0/1270863534 >> 172.21.15.133:0/1270863534 conn(0x55d611471060 msgr2=0x55d611476dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 1 -- 172.21.15.133:0/1270863534 shutdown_connections 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 1 -- 172.21.15.133:0/1270863534 wait complete. 2024-09-11T23:41:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.218+0000 7f2aa081b740 20 client.16725 trim_cache size 0 max 16384 2024-09-11T23:41:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2aa081b740 1 Processor -- start 2024-09-11T23:41:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2aa081b740 1 -- start start 2024-09-11T23:41:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d611474af0 0x55d611477560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d61147d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d6114979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2aa081b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981ab450 con 0x55d61147fcb0 2024-09-11T23:41:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2aa081b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981acf90 con 0x55d611474af0 2024-09-11T23:41:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2aa081b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981abb20 con 0x55d611475410 2024-09-11T23:41:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d6114979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d6114979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39002/0 (socket says 172.21.15.133:39002) 2024-09-11T23:41:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2a977fe640 1 -- 172.21.15.133:0/1738117465 learned_addr learned my addr 172.21.15.133:0/1738117465 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2a967fc640 1 --2- 172.21.15.133:0/1738117465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d61147d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2a96ffd640 1 --2- 172.21.15.133:0/1738117465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d611474af0 0x55d611477560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2a977fe640 1 -- 172.21.15.133:0/1738117465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 msgr2=0x55d61147d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2a977fe640 1 --2- 172.21.15.133:0/1738117465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d61147d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2a977fe640 1 -- 172.21.15.133:0/1738117465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d611474af0 msgr2=0x55d611477560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2a977fe640 1 --2- 172.21.15.133:0/1738117465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d611474af0 0x55d611477560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2a977fe640 1 -- 172.21.15.133:0/1738117465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6114926f0 con 0x55d61147fcb0 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.232+0000 7f2a96ffd640 1 --2- 172.21.15.133:0/1738117465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d611474af0 0x55d611477560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2a977fe640 1 --2- 172.21.15.133:0/1738117465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d6114979d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2a9002a920 tx=0x7f2a90009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11157ffbf0b54605 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2a7d7fa640 1 -- 172.21.15.133:0/1738117465 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90063070 con 0x55d61147fcb0 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2a7d7fa640 1 -- 172.21.15.133:0/1738117465 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a9004be50 con 0x55d61147fcb0 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2a7d7fa640 1 -- 172.21.15.133:0/1738117465 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90030a80 con 0x55d61147fcb0 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2aa081b740 1 -- 172.21.15.133:0/1738117465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 msgr2=0x55d6114979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2aa081b740 1 --2- 172.21.15.133:0/1738117465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d6114979d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2a9002a920 tx=0x7f2a90009c50 comp rx=0 tx=0).stop 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2aa081b740 1 -- 172.21.15.133:0/1738117465 shutdown_connections 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2aa081b740 1 --2- 172.21.15.133:0/1738117465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d61147fcb0 0x55d6114979d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2aa081b740 1 --2- 172.21.15.133:0/1738117465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d61147d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2aa081b740 1 --2- 172.21.15.133:0/1738117465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d611474af0 0x55d611477560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2aa081b740 1 -- 172.21.15.133:0/1738117465 >> 172.21.15.133:0/1738117465 conn(0x55d611471060 msgr2=0x55d61147ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2aa081b740 1 -- 172.21.15.133:0/1738117465 shutdown_connections 2024-09-11T23:41:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.233+0000 7f2aa081b740 1 -- 172.21.15.133:0/1738117465 wait complete. 2024-09-11T23:41:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2aa081b740 1 -- start start 2024-09-11T23:41:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2aa081b740 1 Processor -- start 2024-09-11T23:41:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611474af0 0x55d611488320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d6114798f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2aa081b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d61147fcb0 0x55d611479e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2aa081b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981ac8c0 con 0x55d611474af0 2024-09-11T23:41:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2aa081b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981ac1f0 con 0x55d61147fcb0 2024-09-11T23:41:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2aa081b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981a9470 con 0x55d611475410 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2aa081b740 10 client.0 _ll_register_callbacks cb 0x7f2a90009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2a96ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611474af0 0x55d611488320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2a96ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611474af0 0x55d611488320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39014/0 (socket says 172.21.15.133:39014) 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2a96ffd640 1 -- 172.21.15.133:0/855845404 learned_addr learned my addr 172.21.15.133:0/855845404 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2a967fc640 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d6114798f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2a977fe640 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d61147fcb0 0x55d611479e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2a96ffd640 1 -- 172.21.15.133:0/855845404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 msgr2=0x55d6114798f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2a96ffd640 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d6114798f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2a96ffd640 1 -- 172.21.15.133:0/855845404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d61147fcb0 msgr2=0x55d611479e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.234+0000 7f2a96ffd640 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d61147fcb0 0x55d611479e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2a96ffd640 1 -- 172.21.15.133:0/855845404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d611471cf0 con 0x55d611474af0 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2a977fe640 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d61147fcb0 0x55d611479e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:41:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2a96ffd640 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611474af0 0x55d611488320 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x55d611492970 tx=0x7f2a8800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67beb893959687cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2a76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2a76ffd640 1 -- 172.21.15.133:0/855845404 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8802a060 con 0x55d611474af0 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2a76ffd640 1 -- 172.21.15.133:0/855845404 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f2a88011440 con 0x55d611474af0 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2aa081b740 1 -- 172.21.15.133:0/855845404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a9004be50 con 0x55d611474af0 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2a76ffd640 1 -- 172.21.15.133:0/855845404 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a88011640 con 0x55d611474af0 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2aa081b740 10 client.16731 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2aa081b740 1 -- 172.21.15.133:0/855845404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6114fa770 con 0x55d611474af0 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2a76ffd640 1 -- 172.21.15.133:0/855845404 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a9004be50 con 0x55d611474af0 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2a76ffd640 1 -- 172.21.15.133:0/855845404 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2a8802d020 con 0x55d611474af0 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2aa081b740 10 client.16731 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2aa081b740 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2aa081b740 20 client.16731 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2aa081b740 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6114faea0 0x55d6114fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2aa081b740 4 client.16731 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:41:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.235+0000 7f2aa081b740 1 -- 172.21.15.133:0/855845404 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d61135d900 con 0x55d6114faea0 2024-09-11T23:41:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.236+0000 7f2a967fc640 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6114faea0 0x55d6114fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.236+0000 7f2a76ffd640 10 client.16731 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.236+0000 7f2a967fc640 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6114faea0 0x55d6114fd2c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8de5c247970befe server_cookie=118c76b1f357b91b in_seq=0 out_seq=0 2024-09-11T23:41:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2a76ffd640 1 -- 172.21.15.133:0/855845404 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d61135d900 con 0x55d6114faea0 2024-09-11T23:41:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2a76ffd640 10 client.16731 handle_command_reply: tid=0 2024-09-11T23:41:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 1 client.16731 shutdown 2024-09-11T23:41:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 1 -- 172.21.15.133:0/855845404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611474af0 msgr2=0x55d611488320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611474af0 0x55d611488320 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x55d611492970 tx=0x7f2a8800be60 comp rx=0 tx=0).stop 2024-09-11T23:41:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 1 -- 172.21.15.133:0/855845404 shutdown_connections 2024-09-11T23:41:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6114faea0 0x55d6114fd2c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d61147fcb0 0x55d611479e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d611475410 0x55d6114798f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2a76ffd640 0 client.16731 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 1 --2- 172.21.15.133:0/855845404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d611474af0 0x55d611488320 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 1 -- 172.21.15.133:0/855845404 >> 172.21.15.133:0/855845404 conn(0x55d611471060 msgr2=0x55d61146f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 1 -- 172.21.15.133:0/855845404 shutdown_connections 2024-09-11T23:41:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 1 -- 172.21.15.133:0/855845404 wait complete. 2024-09-11T23:41:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.248+0000 7f2aa081b740 20 client.16731 trim_cache size 0 max 16384 2024-09-11T23:41:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:41:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:41:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:41:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:41:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 -- 172.21.15.133:0/2897844987 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2a800a2da0 msgr2=0x7f2a800a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2a800a2da0 0x7f2a800a5260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a90005fe0 tx=0x7f2a90005c50 comp rx=0 tx=0).stop 2024-09-11T23:41:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 -- 172.21.15.133:0/2897844987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a98101370 msgr2=0x7f2a98197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a98101370 0x7f2a98197770 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00e9f0 tx=0x7f2a8c00edf0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 -- 172.21.15.133:0/2897844987 shutdown_connections 2024-09-11T23:41:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2a800a2da0 0x7f2a800a5260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a981069e0 0x7f2a9819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a98101370 0x7f2a98197770 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 --2- 172.21.15.133:0/2897844987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a981009d0 0x7f2a98197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 -- 172.21.15.133:0/2897844987 >> 172.21.15.133:0/2897844987 conn(0x7f2a980fc6b0 msgr2=0x7f2a980fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 -- 172.21.15.133:0/2897844987 shutdown_connections 2024-09-11T23:41:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:24.249+0000 7f2a9d99f640 1 -- 172.21.15.133:0/2897844987 wait complete. 2024-09-11T23:41:26.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:26 smithi138 ceph-mon[35142]: pgmap v142: 97 pgs: 97 active+clean; 31 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 652 KiB/s rd, 412 MiB/s wr, 3.24k op/s 2024-09-11T23:41:26.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:26 smithi133 ceph-mon[31430]: pgmap v142: 97 pgs: 97 active+clean; 31 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 652 KiB/s rd, 412 MiB/s wr, 3.24k op/s 2024-09-11T23:41:26.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:26 smithi178 ceph-mon[35882]: pgmap v142: 97 pgs: 97 active+clean; 31 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 652 KiB/s rd, 412 MiB/s wr, 3.24k op/s 2024-09-11T23:41:27.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:27 smithi138 ceph-mon[35142]: pgmap v143: 97 pgs: 97 active+clean; 31 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 648 KiB/s rd, 324 MiB/s wr, 2.92k op/s 2024-09-11T23:41:27.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:27 smithi133 ceph-mon[31430]: pgmap v143: 97 pgs: 97 active+clean; 31 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 648 KiB/s rd, 324 MiB/s wr, 2.92k op/s 2024-09-11T23:41:27.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:27 smithi178 ceph-mon[35882]: pgmap v143: 97 pgs: 97 active+clean; 31 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 648 KiB/s rd, 324 MiB/s wr, 2.92k op/s 2024-09-11T23:41:30.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:30 smithi138 ceph-mon[35142]: pgmap v144: 97 pgs: 97 active+clean; 33 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 654 KiB/s rd, 475 MiB/s wr, 3.87k op/s 2024-09-11T23:41:30.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:30 smithi133 ceph-mon[31430]: pgmap v144: 97 pgs: 97 active+clean; 33 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 654 KiB/s rd, 475 MiB/s wr, 3.87k op/s 2024-09-11T23:41:30.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:30 smithi178 ceph-mon[35882]: pgmap v144: 97 pgs: 97 active+clean; 33 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 654 KiB/s rd, 475 MiB/s wr, 3.87k op/s 2024-09-11T23:41:32.606 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:41:32.634 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:32 smithi133 ceph-mon[31430]: pgmap v145: 97 pgs: 97 active+clean; 33 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 325 KiB/s rd, 283 MiB/s wr, 1.97k op/s 2024-09-11T23:41:32.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:32 smithi138 ceph-mon[35142]: pgmap v145: 97 pgs: 97 active+clean; 33 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 325 KiB/s rd, 283 MiB/s wr, 1.97k op/s 2024-09-11T23:41:32.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:32 smithi178 ceph-mon[35882]: pgmap v145: 97 pgs: 97 active+clean; 33 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 325 KiB/s rd, 283 MiB/s wr, 1.97k op/s 2024-09-11T23:41:32.869 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:33.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.347+0000 7fb8cc444640 1 -- 172.21.15.133:0/2955094383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8c4100a80 msgr2=0x7fb8c4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:33.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.347+0000 7fb8cc444640 1 --2- 172.21.15.133:0/2955094383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8c4100a80 0x7fb8c4100ee0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009930 tx=0x7fb8b402a420 comp rx=0 tx=0).stop 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.347+0000 7fb8cc444640 1 -- 172.21.15.133:0/2955094383 shutdown_connections 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.347+0000 7fb8cc444640 1 --2- 172.21.15.133:0/2955094383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8c4101420 0x7fb8c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.347+0000 7fb8cc444640 1 --2- 172.21.15.133:0/2955094383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8c4100a80 0x7fb8c4100ee0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.347+0000 7fb8cc444640 1 --2- 172.21.15.133:0/2955094383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4106a80 0x7fb8c4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.347+0000 7fb8cc444640 1 -- 172.21.15.133:0/2955094383 >> 172.21.15.133:0/2955094383 conn(0x7fb8c40fc7a0 msgr2=0x7fb8c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.348+0000 7fb8cc444640 1 -- 172.21.15.133:0/2955094383 shutdown_connections 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.348+0000 7fb8cc444640 1 -- 172.21.15.133:0/2955094383 wait complete. 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.348+0000 7fb8cc444640 1 Processor -- start 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.348+0000 7fb8cc444640 1 -- start start 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.348+0000 7fb8cc444640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4100a80 0x7fb8c410f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.348+0000 7fb8cc444640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8c4101420 0x7fb8c410fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:33.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.348+0000 7fb8cc444640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8c4106a80 0x7fb8c4110340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.348+0000 7fb8cc444640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c4111f60 con 0x7fb8c4101420 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.348+0000 7fb8cc444640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c4111de0 con 0x7fb8c4106a80 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.348+0000 7fb8cc444640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c41120e0 con 0x7fb8c4100a80 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.348+0000 7fb8c99df640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8c4101420 0x7fb8c410fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8c99df640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8c4101420 0x7fb8c410fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41726/0 (socket says 172.21.15.133:41726) 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8c99df640 1 -- 172.21.15.133:0/1728667248 learned_addr learned my addr 172.21.15.133:0/1728667248 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8ca1e0640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4100a80 0x7fb8c410f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8ca9e1640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8c4106a80 0x7fb8c4110340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8c99df640 1 -- 172.21.15.133:0/1728667248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4100a80 msgr2=0x7fb8c410f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8c99df640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4100a80 0x7fb8c410f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8c99df640 1 -- 172.21.15.133:0/1728667248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8c4106a80 msgr2=0x7fb8c4110340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8c99df640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8c4106a80 0x7fb8c4110340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8c99df640 1 -- 172.21.15.133:0/1728667248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c41a68c0 con 0x7fb8c4101420 2024-09-11T23:41:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8c99df640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8c4101420 0x7fb8c410fe00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009230 tx=0x7fb8b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8bb7fe640 1 -- 172.21.15.133:0/1728667248 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b403f030 con 0x7fb8c4101420 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8cc444640 1 -- 172.21.15.133:0/1728667248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c41a6b50 con 0x7fb8c4101420 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8bb7fe640 1 -- 172.21.15.133:0/1728667248 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b4003ad0 con 0x7fb8c4101420 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8bb7fe640 1 -- 172.21.15.133:0/1728667248 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4035070 con 0x7fb8c4101420 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.349+0000 7fb8cc444640 1 -- 172.21.15.133:0/1728667248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c41a7080 con 0x7fb8c4101420 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.350+0000 7fb8cc444640 1 -- 172.21.15.133:0/1728667248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c4100ee0 con 0x7fb8c4101420 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.351+0000 7fb8bb7fe640 1 -- 172.21.15.133:0/1728667248 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb8b4042030 con 0x7fb8c4101420 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.352+0000 7fb8bb7fe640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb8b00a2cf0 0x7fb8b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.352+0000 7fb8ca1e0640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb8b00a2cf0 0x7fb8b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.352+0000 7fb8bb7fe640 1 -- 172.21.15.133:0/1728667248 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fb8b40ef8a0 con 0x7fb8c4101420 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.352+0000 7fb8ca1e0640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb8b00a2cf0 0x7fb8b00a51b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000ab60 tx=0x7fb8c000a2d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.356+0000 7fb8bb7fe640 1 -- 172.21.15.133:0/1728667248 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8b40bc530 con 0x7fb8c4101420 2024-09-11T23:41:33.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.560+0000 7fb8cc444640 1 -- 172.21.15.133:0/1728667248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8c402d7c0 con 0x7fb8c4101420 2024-09-11T23:41:33.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.560+0000 7fb8bb7fe640 1 -- 172.21.15.133:0/1728667248 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7fb8b40bbe30 con 0x7fb8c4101420 2024-09-11T23:41:33.560 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:41:33.560 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:41:33.561 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:41:33.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 -- 172.21.15.133:0/1728667248 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb8b00a2cf0 msgr2=0x7fb8b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:33.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb8b00a2cf0 0x7fb8b00a51b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000ab60 tx=0x7fb8c000a2d0 comp rx=0 tx=0).stop 2024-09-11T23:41:33.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 -- 172.21.15.133:0/1728667248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8c4101420 msgr2=0x7fb8c410fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8c4101420 0x7fb8c410fe00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009230 tx=0x7fb8b4004270 comp rx=0 tx=0).stop 2024-09-11T23:41:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 -- 172.21.15.133:0/1728667248 shutdown_connections 2024-09-11T23:41:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb8b00a2cf0 0x7fb8b00a51b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8c4106a80 0x7fb8c4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8c4101420 0x7fb8c410fe00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 --2- 172.21.15.133:0/1728667248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4100a80 0x7fb8c410f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 -- 172.21.15.133:0/1728667248 >> 172.21.15.133:0/1728667248 conn(0x7fb8c40fc7a0 msgr2=0x7fb8c40fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 -- 172.21.15.133:0/1728667248 shutdown_connections 2024-09-11T23:41:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:33.563+0000 7fb8cc444640 1 -- 172.21.15.133:0/1728667248 wait complete. 2024-09-11T23:41:34.362 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id 14340ff8 --release --await' 2024-09-11T23:41:34.362 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id 14340ff8 --release --await 2024-09-11T23:41:34.595 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:41:34.618 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:34 smithi133 ceph-mon[31430]: pgmap v146: 97 pgs: 97 active+clean; 33 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 325 KiB/s rd, 260 MiB/s wr, 1.73k op/s 2024-09-11T23:41:34.618 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:34 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1728667248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:34.653 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:34.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:34 smithi138 ceph-mon[35142]: pgmap v146: 97 pgs: 97 active+clean; 33 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 325 KiB/s rd, 260 MiB/s wr, 1.73k op/s 2024-09-11T23:41:34.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:34 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1728667248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:34.792 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:34 smithi178 ceph-mon[35882]: pgmap v146: 97 pgs: 97 active+clean; 33 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 325 KiB/s rd, 260 MiB/s wr, 1.73k op/s 2024-09-11T23:41:34.792 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:34 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1728667248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:35.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.094+0000 7f8c6f008640 1 -- 172.21.15.133:0/866562281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c68100e20 msgr2=0x7f8c68101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:35.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.094+0000 7f8c6f008640 1 --2- 172.21.15.133:0/866562281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c68100e20 0x7f8c68101200 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009900 tx=0x7f8c5c02a480 comp rx=0 tx=0).stop 2024-09-11T23:41:35.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.094+0000 7f8c6f008640 1 -- 172.21.15.133:0/866562281 shutdown_connections 2024-09-11T23:41:35.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.094+0000 7f8c6f008640 1 --2- 172.21.15.133:0/866562281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c68102ef0 0x7f8c6810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.094+0000 7f8c6f008640 1 --2- 172.21.15.133:0/866562281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c68102570 0x7f8c681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.094+0000 7f8c6f008640 1 --2- 172.21.15.133:0/866562281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c68100e20 0x7f8c68101200 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.094+0000 7f8c6f008640 1 -- 172.21.15.133:0/866562281 >> 172.21.15.133:0/866562281 conn(0x7f8c680fc7d0 msgr2=0x7f8c680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.094+0000 7f8c6f008640 1 -- 172.21.15.133:0/866562281 shutdown_connections 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.094+0000 7f8c6f008640 1 -- 172.21.15.133:0/866562281 wait complete. 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6f008640 1 Processor -- start 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6f008640 1 -- start start 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6f008640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c68100e20 0x7f8c6819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6f008640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c68102570 0x7f8c6819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6f008640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c68102ef0 0x7f8c6819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6f008640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8c68110f50 con 0x7f8c68102ef0 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6f008640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c68110dd0 con 0x7f8c68102570 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6f008640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681110d0 con 0x7f8c68100e20 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6d5a5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c68102ef0 0x7f8c6819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6cda4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c68100e20 0x7f8c6819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6cda4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c68100e20 0x7f8c6819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48684/0 (socket says 172.21.15.133:48684) 2024-09-11T23:41:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6cda4640 1 -- 172.21.15.133:0/75627021 learned_addr learned my addr 172.21.15.133:0/75627021 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.095+0000 7f8c6d5a5640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c68102ef0 0x7f8c6819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41740/0 (socket says 172.21.15.133:41740) 2024-09-11T23:41:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.096+0000 7f8c67fff640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c68102570 0x7f8c6819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.096+0000 7f8c6d5a5640 1 -- 172.21.15.133:0/75627021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c68100e20 msgr2=0x7f8c6819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.096+0000 7f8c6d5a5640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c68100e20 0x7f8c6819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.096+0000 7f8c6d5a5640 1 -- 172.21.15.133:0/75627021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c68102570 msgr2=0x7f8c6819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.096+0000 7f8c6d5a5640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c68102570 0x7f8c6819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.096+0000 7f8c6d5a5640 1 -- 172.21.15.133:0/75627021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6819f670 con 0x7f8c68102ef0 2024-09-11T23:41:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.096+0000 7f8c6d5a5640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c68102ef0 0x7f8c6819ef90 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000cab0 tx=0x7f8c6000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.096+0000 7f8c65ffb640 1 -- 172.21.15.133:0/75627021 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000ae20 con 0x7f8c68102ef0 2024-09-11T23:41:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.096+0000 7f8c6f008640 1 -- 172.21.15.133:0/75627021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6819f960 con 0x7f8c68102ef0 2024-09-11T23:41:35.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.096+0000 7f8c65ffb640 1 -- 172.21.15.133:0/75627021 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c6000f930 con 0x7f8c68102ef0 2024-09-11T23:41:35.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.096+0000 7f8c65ffb640 1 -- 172.21.15.133:0/75627021 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60018b30 con 0x7f8c68102ef0 2024-09-11T23:41:35.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.097+0000 7f8c6f008640 1 -- 172.21.15.133:0/75627021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c681a7240 con 0x7f8c68102ef0 2024-09-11T23:41:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.099+0000 7f8c65ffb640 1 -- 172.21.15.133:0/75627021 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8c6000fae0 con 0x7f8c68102ef0 2024-09-11T23:41:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.099+0000 7f8c713e1740 1 Processor -- start 2024-09-11T23:41:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.099+0000 7f8c713e1740 1 -- start start 2024-09-11T23:41:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.099+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030c6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.099+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 0x5648030d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c65ffb640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8c580a2cf0 0x7f8c580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c713e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56480302b620 con 0x5648030c6060 2024-09-11T23:41:35.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c713e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564802fd2e50 con 0x5648030c6980 2024-09-11T23:41:35.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c713e1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5648030275e0 con 0x5648030d1240 2024-09-11T23:41:35.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c67fff640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8c580a2cf0 0x7f8c580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c6d5a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 0x5648030d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c6cda4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c67fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c6cda4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42536/0 (socket says 172.21.15.133:42536) 2024-09-11T23:41:35.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c6cda4640 1 -- 172.21.15.133:0/1245834794 learned_addr learned my addr 172.21.15.133:0/1245834794 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:35.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c65ffb640 1 -- 172.21.15.133:0/75627021 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8c600cfa30 con 0x7f8c68102ef0 2024-09-11T23:41:35.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c6d5a5640 1 -- 172.21.15.133:0/1245834794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 msgr2=0x5648030d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c6d5a5640 1 --2- 172.21.15.133:0/1245834794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c6d5a5640 1 -- 172.21.15.133:0/1245834794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 msgr2=0x5648030c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c6d5a5640 1 --2- 172.21.15.133:0/1245834794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c6d5a5640 1 -- 172.21.15.133:0/1245834794 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564802fc3d40 con 0x5648030d1240 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c6cda4640 1 --2- 172.21.15.133:0/1245834794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.100+0000 7f8c67fff640 1 --2- 172.21.15.133:0/1245834794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.101+0000 7f8c6d5a5640 1 --2- 172.21.15.133:0/1245834794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 0x5648030d3630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c600cd1f0 tx=0x7f8c60013800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=394552c5076579fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.101+0000 7f8c67fff640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8c580a2cf0 0x7f8c580a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400b620 tx=0x7f8c5400baf0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.101+0000 7f8c52ffd640 1 -- 172.21.15.133:0/1245834794 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6009c6c0 con 0x5648030d1240 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.101+0000 7f8c52ffd640 1 -- 172.21.15.133:0/1245834794 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c6009bfc0 con 0x5648030d1240 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.101+0000 7f8c52ffd640 1 -- 172.21.15.133:0/1245834794 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60098630 con 0x5648030d1240 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.102+0000 7f8c713e1740 1 -- 172.21.15.133:0/1245834794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 msgr2=0x5648030d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:35.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.102+0000 7f8c713e1740 1 --2- 172.21.15.133:0/1245834794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 0x5648030d3630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c600cd1f0 tx=0x7f8c60013800 comp rx=0 tx=0).stop 2024-09-11T23:41:35.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.102+0000 7f8c713e1740 1 -- 172.21.15.133:0/1245834794 shutdown_connections 2024-09-11T23:41:35.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.102+0000 7f8c713e1740 1 --2- 172.21.15.133:0/1245834794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 0x5648030d3630 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.102+0000 7f8c713e1740 1 --2- 172.21.15.133:0/1245834794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.102+0000 7f8c713e1740 1 --2- 172.21.15.133:0/1245834794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.102+0000 7f8c713e1740 1 -- 172.21.15.133:0/1245834794 >> 172.21.15.133:0/1245834794 conn(0x5648030c25f0 msgr2=0x5648030c29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:35.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.102+0000 7f8c713e1740 1 -- 172.21.15.133:0/1245834794 shutdown_connections 2024-09-11T23:41:35.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.102+0000 7f8c713e1740 1 -- 172.21.15.133:0/1245834794 wait complete. 2024-09-11T23:41:35.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c713e1740 1 -- start start 2024-09-11T23:41:35.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c713e1740 1 Processor -- start 2024-09-11T23:41:35.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6060 0x5648030ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030d1240 0x5648030cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c713e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5648030d3e10 con 0x5648030d1240 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c713e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564802fb3f10 con 0x5648030c6980 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c713e1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5648030d3f90 con 0x5648030c6060 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c713e1740 10 client.0 _ll_register_callbacks cb 0x564802fd7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c6cda4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6060 0x5648030ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c6cda4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6060 0x5648030ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48716/0 (socket says 172.21.15.133:48716) 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c6cda4640 1 -- 172.21.15.133:0/3506659543 learned_addr learned my addr 172.21.15.133:0/3506659543 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.103+0000 7f8c6d5a5640 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030d1240 0x5648030cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.104+0000 7f8c67fff640 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.104+0000 7f8c6cda4640 1 -- 172.21.15.133:0/3506659543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 msgr2=0x5648030ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.104+0000 7f8c6cda4640 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.104+0000 7f8c6cda4640 1 -- 172.21.15.133:0/3506659543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030d1240 msgr2=0x5648030cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.104+0000 7f8c6cda4640 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030d1240 0x5648030cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.104+0000 7f8c6cda4640 1 -- 172.21.15.133:0/3506659543 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564802d3a580 con 0x5648030c6060 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.104+0000 7f8c6cda4640 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6060 0x5648030ce1f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009230 tx=0x7f8c5c0044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cacf2d42240f3b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.104+0000 7f8c47fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.104+0000 7f8c47fff640 1 -- 172.21.15.133:0/3506659543 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c035040 con 0x5648030c6060 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.104+0000 7f8c47fff640 1 -- 172.21.15.133:0/3506659543 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8c5c003930 con 0x5648030c6060 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.104+0000 7f8c713e1740 1 -- 172.21.15.133:0/3506659543 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c6009bfc0 con 0x5648030c6060 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.105+0000 7f8c47fff640 1 -- 172.21.15.133:0/3506659543 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c5c003b30 con 0x5648030c6060 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.105+0000 7f8c713e1740 10 client.26176 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.105+0000 7f8c713e1740 1 -- 172.21.15.133:0/3506659543 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564802e04960 con 0x5648030c6060 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.105+0000 7f8c47fff640 1 -- 172.21.15.133:0/3506659543 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c6009bfc0 con 0x5648030c6060 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.105+0000 7f8c47fff640 1 -- 172.21.15.133:0/3506659543 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8c5c04e050 con 0x5648030c6060 2024-09-11T23:41:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.105+0000 7f8c713e1740 10 client.26176 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.105+0000 7f8c713e1740 10 client.26176 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.105+0000 7f8c713e1740 20 client.26176 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.105+0000 7f8c713e1740 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5648030e0dc0 0x5648030e3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.106+0000 7f8c713e1740 4 client.26176 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.106+0000 7f8c713e1740 1 -- 172.21.15.133:0/3506659543 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564803038b90 con 0x5648030e0dc0 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.106+0000 7f8c67fff640 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5648030e0dc0 0x5648030e3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.106+0000 7f8c47fff640 10 client.26176 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.106+0000 7f8c67fff640 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5648030e0dc0 0x5648030e3240 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72cf4ac1c32ea017 server_cookie=bdb1d01c52e78d82 in_seq=0 out_seq=0 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c47fff640 1 -- 172.21.15.133:0/3506659543 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564803038b90 con 0x5648030e0dc0 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c47fff640 10 client.26176 handle_command_reply: tid=0 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c713e1740 1 client.26176 shutdown 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c713e1740 1 -- 172.21.15.133:0/3506659543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6060 msgr2=0x5648030ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c713e1740 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6060 0x5648030ce1f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009230 tx=0x7f8c5c0044d0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c713e1740 1 -- 172.21.15.133:0/3506659543 shutdown_connections 2024-09-11T23:41:35.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c713e1740 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5648030e0dc0 0x5648030e3240 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c47fff640 0 client.26176 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:35.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c713e1740 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030d1240 0x5648030cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c713e1740 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c713e1740 1 --2- 172.21.15.133:0/3506659543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6060 0x5648030ce1f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c713e1740 1 -- 172.21.15.133:0/3506659543 >> 172.21.15.133:0/3506659543 conn(0x5648030c25f0 msgr2=0x5648030c82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:35.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.107+0000 7f8c713e1740 1 -- 172.21.15.133:0/3506659543 shutdown_connections 2024-09-11T23:41:35.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.108+0000 7f8c713e1740 1 -- 172.21.15.133:0/3506659543 wait complete. 2024-09-11T23:41:35.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.108+0000 7f8c713e1740 20 client.26176 trim_cache size 0 max 16384 2024-09-11T23:41:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c713e1740 1 Processor -- start 2024-09-11T23:41:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c713e1740 1 -- start start 2024-09-11T23:41:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6980 0x5648030c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030d1240 0x5648030e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c713e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681af7b0 con 0x5648030c6060 2024-09-11T23:41:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c713e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681b12f0 con 0x5648030d1240 2024-09-11T23:41:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c713e1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681afe80 con 0x5648030c6980 2024-09-11T23:41:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c6cda4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c6cda4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41764/0 (socket says 172.21.15.133:41764) 2024-09-11T23:41:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c6cda4640 1 -- 172.21.15.133:0/559955830 learned_addr learned my addr 172.21.15.133:0/559955830 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.131+0000 7f8c67fff640 1 --2- 172.21.15.133:0/559955830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6980 0x5648030c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.132+0000 7f8c6d5a5640 1 --2- 172.21.15.133:0/559955830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030d1240 0x5648030e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.132+0000 7f8c6cda4640 1 -- 172.21.15.133:0/559955830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6980 msgr2=0x5648030c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.132+0000 7f8c6cda4640 1 --2- 172.21.15.133:0/559955830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6980 0x5648030c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.132+0000 7f8c6cda4640 1 -- 172.21.15.133:0/559955830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030d1240 msgr2=0x5648030e8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.132+0000 7f8c6cda4640 1 --2- 172.21.15.133:0/559955830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030d1240 0x5648030e8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.132+0000 7f8c6cda4640 1 -- 172.21.15.133:0/559955830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648030ce890 con 0x5648030c6060 2024-09-11T23:41:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.132+0000 7f8c6cda4640 1 --2- 172.21.15.133:0/559955830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030c94c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0096b0 tx=0x7f8c5c0040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7c03bac5c225fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.132+0000 7f8c527fc640 1 -- 172.21.15.133:0/559955830 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c000e60 con 0x5648030c6060 2024-09-11T23:41:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.132+0000 7f8c527fc640 1 -- 172.21.15.133:0/559955830 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c5c005850 con 0x5648030c6060 2024-09-11T23:41:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.132+0000 7f8c527fc640 1 -- 172.21.15.133:0/559955830 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c035070 con 0x5648030c6060 2024-09-11T23:41:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.133+0000 7f8c713e1740 1 -- 172.21.15.133:0/559955830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 msgr2=0x5648030c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.133+0000 7f8c713e1740 1 --2- 172.21.15.133:0/559955830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030c94c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0096b0 tx=0x7f8c5c0040d0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.133+0000 7f8c713e1740 1 -- 172.21.15.133:0/559955830 shutdown_connections 2024-09-11T23:41:35.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.133+0000 7f8c713e1740 1 --2- 172.21.15.133:0/559955830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030d1240 0x5648030e8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.133+0000 7f8c713e1740 1 --2- 172.21.15.133:0/559955830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030c6980 0x5648030c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.133+0000 7f8c713e1740 1 --2- 172.21.15.133:0/559955830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030c94c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.133+0000 7f8c713e1740 1 -- 172.21.15.133:0/559955830 >> 172.21.15.133:0/559955830 conn(0x5648030c25f0 msgr2=0x5648030c7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:35.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.133+0000 7f8c713e1740 1 -- 172.21.15.133:0/559955830 shutdown_connections 2024-09-11T23:41:35.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.133+0000 7f8c713e1740 1 -- 172.21.15.133:0/559955830 wait complete. 2024-09-11T23:41:35.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.133+0000 7f8c713e1740 1 -- start start 2024-09-11T23:41:35.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c713e1740 1 Processor -- start 2024-09-11T23:41:35.135 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:35.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030d9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c713e1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 0x5648030d6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c713e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681b0c20 con 0x5648030c6060 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c713e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681b0550 con 0x5648030c6980 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c713e1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681ad7b0 con 0x5648030d1240 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c713e1740 10 client.0 _ll_register_callbacks cb 0x7f8c5c004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c6cda4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030d9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c6cda4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030d9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41778/0 (socket says 172.21.15.133:41778) 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c6cda4640 1 -- 172.21.15.133:0/161751240 learned_addr learned my addr 172.21.15.133:0/161751240 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c67fff640 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c6d5a5640 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 0x5648030d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c6cda4640 1 -- 172.21.15.133:0/161751240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 msgr2=0x5648030d6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:35.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c6cda4640 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 0x5648030d6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c6cda4640 1 -- 172.21.15.133:0/161751240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 msgr2=0x5648030d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.134+0000 7f8c6cda4640 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030d5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.135+0000 7f8c6cda4640 1 -- 172.21.15.133:0/161751240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648030d6680 con 0x5648030c6060 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.135+0000 7f8c6d5a5640 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 0x5648030d6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.135+0000 7f8c6cda4640 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030d9790 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c005770 tx=0x7f8c5c046660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847868c4e86d29b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.135+0000 7f8c47fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.135+0000 7f8c47fff640 1 -- 172.21.15.133:0/161751240 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c035070 con 0x5648030c6060 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.135+0000 7f8c47fff640 1 -- 172.21.15.133:0/161751240 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f8c5c031740 con 0x5648030c6060 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.135+0000 7f8c713e1740 1 -- 172.21.15.133:0/161751240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c5c005850 con 0x5648030c6060 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.135+0000 7f8c47fff640 1 -- 172.21.15.133:0/161751240 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c5c069050 con 0x5648030c6060 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.135+0000 7f8c713e1740 10 client.16755 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.135+0000 7f8c713e1740 1 -- 172.21.15.133:0/161751240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56480314c970 con 0x5648030c6060 2024-09-11T23:41:35.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.136+0000 7f8c47fff640 1 -- 172.21.15.133:0/161751240 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c5c005850 con 0x5648030c6060 2024-09-11T23:41:35.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.136+0000 7f8c47fff640 1 -- 172.21.15.133:0/161751240 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8c5c04e040 con 0x5648030c6060 2024-09-11T23:41:35.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.136+0000 7f8c713e1740 10 client.16755 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:35.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.136+0000 7f8c713e1740 10 client.16755 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:41:35.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.136+0000 7f8c713e1740 20 client.16755 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:35.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.136+0000 7f8c713e1740 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5648030e0dc0 0x56480314ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:35.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.136+0000 7f8c713e1740 4 client.16755 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "14340ff8", "release": true, "await": true}] 2024-09-11T23:41:35.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.136+0000 7f8c713e1740 1 -- 172.21.15.133:0/161751240 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "set_id": "14340ff8", "release": true, "await": true}) -- 0x5648030cf000 con 0x5648030e0dc0 2024-09-11T23:41:35.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.136+0000 7f8c67fff640 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5648030e0dc0 0x56480314ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:35.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.137+0000 7f8c47fff640 10 client.16755 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:35.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.137+0000 7f8c67fff640 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5648030e0dc0 0x56480314ef80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6044d7eb0e9e024 server_cookie=4d5b909737766698 in_seq=0 out_seq=0 2024-09-11T23:41:35.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c47fff640 1 -- 172.21.15.133:0/161751240 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5648030cf000 con 0x5648030e0dc0 2024-09-11T23:41:35.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c47fff640 10 client.16755 handle_command_reply: tid=0 2024-09-11T23:41:35.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c713e1740 1 client.16755 shutdown 2024-09-11T23:41:35.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c713e1740 1 -- 172.21.15.133:0/161751240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 msgr2=0x5648030d9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:35.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c713e1740 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030d9790 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c005770 tx=0x7f8c5c046660 comp rx=0 tx=0).stop 2024-09-11T23:41:35.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c713e1740 1 -- 172.21.15.133:0/161751240 shutdown_connections 2024-09-11T23:41:35.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c713e1740 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5648030e0dc0 0x56480314ef80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c713e1740 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648030d1240 0x5648030d6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c713e1740 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648030c6980 0x5648030d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c713e1740 1 --2- 172.21.15.133:0/161751240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648030c6060 0x5648030d9790 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c713e1740 1 -- 172.21.15.133:0/161751240 >> 172.21.15.133:0/161751240 conn(0x5648030c25f0 msgr2=0x5648030c1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:35.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.142+0000 7f8c47fff640 0 client.16755 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:35.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.143+0000 7f8c713e1740 1 -- 172.21.15.133:0/161751240 shutdown_connections 2024-09-11T23:41:35.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.143+0000 7f8c713e1740 1 -- 172.21.15.133:0/161751240 wait complete. 2024-09-11T23:41:35.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.143+0000 7f8c713e1740 20 client.16755 trim_cache size 0 max 16384 2024-09-11T23:41:35.144 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:41:35.144 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 28, 2024-09-11T23:41:35.144 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:41:35.144 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 20, 2024-09-11T23:41:35.144 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:41:35.144 INFO:teuthology.orchestra.run.smithi133.stdout: "14340ff8": { 2024-09-11T23:41:35.144 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 20, 2024-09-11T23:41:35.144 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:41:35.144 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 140.2, 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:35.145 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:35.146 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:41:35.146 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:41:35.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.147+0000 7f8c6f008640 1 -- 172.21.15.133:0/75627021 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8c580a2cf0 msgr2=0x7f8c580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:35.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.147+0000 7f8c6f008640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8c580a2cf0 0x7f8c580a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400b620 tx=0x7f8c5400baf0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.147+0000 7f8c6f008640 1 -- 172.21.15.133:0/75627021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c68102ef0 msgr2=0x7f8c6819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:35.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.147+0000 7f8c6f008640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c68102ef0 0x7f8c6819ef90 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000cab0 tx=0x7f8c6000cf70 comp rx=0 tx=0).stop 2024-09-11T23:41:35.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.147+0000 7f8c6f008640 1 -- 172.21.15.133:0/75627021 shutdown_connections 2024-09-11T23:41:35.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.147+0000 7f8c6f008640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8c580a2cf0 0x7f8c580a51b0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400b620 tx=0x7f8c5400baf0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.147+0000 7f8c6f008640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c68102ef0 0x7f8c6819ef90 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.147+0000 7f8c6f008640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c68102570 0x7f8c6819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.147+0000 7f8c6f008640 1 --2- 172.21.15.133:0/75627021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c68100e20 0x7f8c6819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:35.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.147+0000 7f8c6f008640 1 -- 172.21.15.133:0/75627021 >> 172.21.15.133:0/75627021 conn(0x7f8c680fc7d0 msgr2=0x7f8c6806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:35.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.149+0000 7f8c6f008640 1 -- 172.21.15.133:0/75627021 shutdown_connections 2024-09-11T23:41:35.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:35.149+0000 7f8c6f008640 1 -- 172.21.15.133:0/75627021 wait complete. 2024-09-11T23:41:36.359 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '14340ff8', total seconds elapsed: 23.9 2024-09-11T23:41:36.463 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:36 smithi133 ceph-mon[31430]: pgmap v147: 97 pgs: 97 active+clean; 34 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 330 KiB/s rd, 283 MiB/s wr, 2.03k op/s 2024-09-11T23:41:36.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:36 smithi138 ceph-mon[35142]: pgmap v147: 97 pgs: 97 active+clean; 34 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 330 KiB/s rd, 283 MiB/s wr, 2.03k op/s 2024-09-11T23:41:36.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:36 smithi178 ceph-mon[35882]: pgmap v147: 97 pgs: 97 active+clean; 34 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 330 KiB/s rd, 283 MiB/s wr, 2.03k op/s 2024-09-11T23:41:36.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.776+0000 7f087f548640 1 -- 172.21.15.133:0/1173880441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0878110190 msgr2=0x7f087806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.776+0000 7f087f548640 1 --2- 172.21.15.133:0/1173880441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0878110190 0x7f087806e4f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f087401a910 tx=0x7f087403b450 comp rx=0 tx=0).stop 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.776+0000 7f087f548640 1 -- 172.21.15.133:0/1173880441 shutdown_connections 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.776+0000 7f087f548640 1 --2- 172.21.15.133:0/1173880441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f087811b140 0x7f087811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.777+0000 7f087f548640 1 --2- 172.21.15.133:0/1173880441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087806ea30 0x7f087806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.777+0000 7f087f548640 1 --2- 172.21.15.133:0/1173880441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0878110190 0x7f087806e4f0 secure :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f087401a910 tx=0x7f087403b450 comp rx=0 tx=0).stop 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.777+0000 7f087f548640 1 -- 172.21.15.133:0/1173880441 >> 172.21.15.133:0/1173880441 conn(0x7f0878073180 msgr2=0x7f0878073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.777+0000 7f087f548640 1 -- 172.21.15.133:0/1173880441 shutdown_connections 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.777+0000 7f087f548640 1 -- 172.21.15.133:0/1173880441 wait complete. 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.779+0000 7f087f548640 1 Processor -- start 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.779+0000 7f087f548640 1 -- start start 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.780+0000 7f087f548640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087806ea30 0x7f0878116cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.780+0000 7f087f548640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087811b140 0x7f0878117210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.780+0000 7f087f548640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0878117750 0x7f08781aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.780+0000 7f087f548640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08781133e0 con 0x7f087811b140 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.780+0000 7f087f548640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0878113260 con 0x7f087806ea30 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.781+0000 7f087f548640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0878113560 con 0x7f0878117750 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.783+0000 7f087ed47640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0878117750 0x7f08781aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.784+0000 7f087e546640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087806ea30 0x7f0878116cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.784+0000 7f087ed47640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0878117750 0x7f08781aaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48760/0 (socket says 172.21.15.133:48760) 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.784+0000 7f087ed47640 1 -- 172.21.15.133:0/2709648014 learned_addr learned my addr 172.21.15.133:0/2709648014 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.785+0000 7f087ed47640 1 -- 172.21.15.133:0/2709648014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087806ea30 msgr2=0x7f0878116cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.785+0000 7f087ed47640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087806ea30 0x7f0878116cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.785+0000 7f087ed47640 1 -- 172.21.15.133:0/2709648014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087811b140 msgr2=0x7f0878117210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.785+0000 7f087ed47640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087811b140 0x7f0878117210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.785+0000 7f087ed47640 1 -- 172.21.15.133:0/2709648014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08781ab4a0 con 0x7f0878117750 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.786+0000 7f087ed47640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0878117750 0x7f08781aaf60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f087000b9e0 tx=0x7f087000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.787+0000 7f086f7fe640 1 -- 172.21.15.133:0/2709648014 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870019070 con 0x7f0878117750 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.787+0000 7f087f548640 1 -- 172.21.15.133:0/2709648014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08781ab670 con 0x7f0878117750 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.787+0000 7f087f548640 1 -- 172.21.15.133:0/2709648014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08781abad0 con 0x7f0878117750 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.794+0000 7f0882b83740 1 Processor -- start 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.794+0000 7f0882b83740 1 -- start start 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.802+0000 7f0882b83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633670 0x5614ad633a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.803+0000 7f0882b83740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad63e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.803+0000 7f0882b83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 0x5614ad640c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.803+0000 7f0882b83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5614ad5992e0 con 0x5614ad63e850 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.803+0000 7f0882b83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5614ad540ce0 con 0x5614ad633670 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.803+0000 7f0882b83740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5614ad5952a0 con 0x5614ad633f90 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.803+0000 7f086f7fe640 1 -- 172.21.15.133:0/2709648014 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870014440 con 0x7f0878117750 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.805+0000 7f087e546640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad63e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.805+0000 7f087e546640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad63e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48762/0 (socket says 172.21.15.133:48762) 2024-09-11T23:41:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.805+0000 7f087e546640 1 -- 172.21.15.133:0/858405471 learned_addr learned my addr 172.21.15.133:0/858405471 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:36.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.806+0000 7f086f7fe640 1 -- 172.21.15.133:0/2709648014 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08700125a0 con 0x7f0878117750 2024-09-11T23:41:36.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.806+0000 7f086f7fe640 1 -- 172.21.15.133:0/2709648014 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f087001c070 con 0x7f0878117750 2024-09-11T23:41:36.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.806+0000 7f087e546640 1 -- 172.21.15.133:0/858405471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633670 msgr2=0x5614ad633a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:41:36.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.806+0000 7f087e546640 1 --2- 172.21.15.133:0/858405471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633670 0x5614ad633a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.806+0000 7f087e546640 1 -- 172.21.15.133:0/858405471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 msgr2=0x5614ad640c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:36.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.806+0000 7f087e546640 1 --2- 172.21.15.133:0/858405471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 0x5614ad640c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.806+0000 7f087e546640 1 -- 172.21.15.133:0/858405471 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ad525440 con 0x5614ad633f90 2024-09-11T23:41:36.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.807+0000 7f087e546640 1 --2- 172.21.15.133:0/858405471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad63e310 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0878109210 tx=0x7f0874000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de8926af827471fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:36.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.817+0000 7f086f7fe640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08640a2f30 0x7f08640a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.817+0000 7f086f7fe640 1 -- 172.21.15.133:0/2709648014 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f08700cf880 con 0x7f0878117750 2024-09-11T23:41:36.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.818+0000 7f086cff9640 1 -- 172.21.15.133:0/858405471 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874046030 con 0x5614ad633f90 2024-09-11T23:41:36.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.818+0000 7f086cff9640 1 -- 172.21.15.133:0/858405471 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0874051070 con 0x5614ad633f90 2024-09-11T23:41:36.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.820+0000 7f0882b83740 1 -- 172.21.15.133:0/858405471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 msgr2=0x5614ad63e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:36.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.820+0000 7f0882b83740 1 --2- 172.21.15.133:0/858405471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad63e310 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0878109210 tx=0x7f0874000bc0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.820+0000 7f0882b83740 1 -- 172.21.15.133:0/858405471 shutdown_connections 2024-09-11T23:41:36.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.820+0000 7f0882b83740 1 --2- 172.21.15.133:0/858405471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 0x5614ad640c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.820+0000 7f0882b83740 1 --2- 172.21.15.133:0/858405471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad63e310 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.821+0000 7f0882b83740 1 --2- 172.21.15.133:0/858405471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633670 0x5614ad633a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.821+0000 7f0882b83740 1 -- 172.21.15.133:0/858405471 >> 172.21.15.133:0/858405471 conn(0x5614ad62fc00 msgr2=0x5614ad62ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:36.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.829+0000 7f087dd45640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08640a2f30 0x7f08640a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.830+0000 7f0882b83740 1 -- 172.21.15.133:0/858405471 shutdown_connections 2024-09-11T23:41:36.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.835+0000 7f0882b83740 1 -- 172.21.15.133:0/858405471 wait complete. 2024-09-11T23:41:36.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.836+0000 7f0882b83740 1 -- start start 2024-09-11T23:41:36.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.836+0000 7f0882b83740 1 Processor -- start 2024-09-11T23:41:36.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.837+0000 7f0882b83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633670 0x5614ad63e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.837+0000 7f0882b83740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad63b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.838+0000 7f0882b83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 0x5614ad63ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.838+0000 7f0882b83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5614ad641470 con 0x5614ad63e850 2024-09-11T23:41:36.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.838+0000 7f0882b83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5614ad554050 con 0x5614ad633670 2024-09-11T23:41:36.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.838+0000 7f0882b83740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5614ad6415f0 con 0x5614ad633f90 2024-09-11T23:41:36.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.838+0000 7f0882b83740 10 client.0 _ll_register_callbacks cb 0x5614ad594160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:36.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.839+0000 7f087e546640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633670 0x5614ad63e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.839+0000 7f087e546640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633670 0x5614ad63e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42618/0 (socket says 172.21.15.133:42618) 2024-09-11T23:41:36.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.839+0000 7f087e546640 1 -- 172.21.15.133:0/3722295537 learned_addr learned my addr 172.21.15.133:0/3722295537 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:36.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.840+0000 7f087ed47640 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 0x5614ad63ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.842+0000 7f087e546640 1 -- 172.21.15.133:0/3722295537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 msgr2=0x5614ad63b480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:41:36.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.842+0000 7f087e546640 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad63b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.842+0000 7f087e546640 1 -- 172.21.15.133:0/3722295537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 msgr2=0x5614ad63ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:36.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.842+0000 7f087e546640 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 0x5614ad63ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.842+0000 7f087e546640 1 -- 172.21.15.133:0/3722295537 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ad30a3c0 con 0x5614ad633670 2024-09-11T23:41:36.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.843+0000 7f087dd45640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08640a2f30 0x7f08640a53f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5614ad630f80 tx=0x7f0868007b40 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:36.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.851+0000 7f087e546640 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633670 0x5614ad63e1d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0874004ef0 tx=0x7f08740048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd10dd41481b488f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:36.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.854+0000 7f0861ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:41:36.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.859+0000 7f0861ffb640 1 -- 172.21.15.133:0/3722295537 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874046070 con 0x5614ad633670 2024-09-11T23:41:36.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.859+0000 7f0882b83740 1 -- 172.21.15.133:0/3722295537 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0874051070 con 0x5614ad633670 2024-09-11T23:41:36.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.859+0000 7f0861ffb640 1 -- 172.21.15.133:0/3722295537 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f0874040510 con 0x5614ad633670 2024-09-11T23:41:36.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.862+0000 7f0861ffb640 1 -- 172.21.15.133:0/3722295537 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0874040710 con 0x5614ad633670 2024-09-11T23:41:36.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.862+0000 7f0882b83740 10 client.26207 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:36.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.862+0000 7f0882b83740 1 -- 172.21.15.133:0/3722295537 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614ad64cf40 con 0x5614ad633670 2024-09-11T23:41:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.865+0000 7f0861ffb640 1 -- 172.21.15.133:0/3722295537 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0874051070 con 0x5614ad633670 2024-09-11T23:41:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.866+0000 7f0861ffb640 1 -- 172.21.15.133:0/3722295537 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0874066020 con 0x5614ad633670 2024-09-11T23:41:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.866+0000 7f0882b83740 10 client.26207 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.866+0000 7f0882b83740 10 client.26207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.866+0000 7f0882b83740 20 client.26207 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.866+0000 7f0882b83740 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5614ad64e3e0 0x5614ad650860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.866+0000 7f0882b83740 4 client.26207 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.866+0000 7f0882b83740 1 -- 172.21.15.133:0/3722295537 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614ad5a6850 con 0x5614ad64e3e0 2024-09-11T23:41:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.868+0000 7f087dd45640 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5614ad64e3e0 0x5614ad650860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.876+0000 7f087dd45640 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5614ad64e3e0 0x5614ad650860 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=149648ca5ab100ce server_cookie=f970b824c8d6cde in_seq=0 out_seq=0 2024-09-11T23:41:36.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.876+0000 7f0861ffb640 10 client.26207 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:36.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.883+0000 7f0861ffb640 1 -- 172.21.15.133:0/3722295537 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614ad5a6850 con 0x5614ad64e3e0 2024-09-11T23:41:36.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.883+0000 7f0861ffb640 10 client.26207 handle_command_reply: tid=0 2024-09-11T23:41:36.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.883+0000 7f0882b83740 1 client.26207 shutdown 2024-09-11T23:41:36.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.883+0000 7f0882b83740 1 -- 172.21.15.133:0/3722295537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633670 msgr2=0x5614ad63e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:36.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.883+0000 7f0882b83740 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633670 0x5614ad63e1d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0874004ef0 tx=0x7f08740048f0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.883+0000 7f0882b83740 1 -- 172.21.15.133:0/3722295537 shutdown_connections 2024-09-11T23:41:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.883+0000 7f0882b83740 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5614ad64e3e0 0x5614ad650860 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.883+0000 7f0882b83740 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 0x5614ad63ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.883+0000 7f0882b83740 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad63b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.883+0000 7f0882b83740 1 --2- 172.21.15.133:0/3722295537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633670 0x5614ad63e1d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.883+0000 7f0882b83740 1 -- 172.21.15.133:0/3722295537 >> 172.21.15.133:0/3722295537 conn(0x5614ad62fc00 msgr2=0x5614ad635930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.884+0000 7f0861ffb640 0 client.26207 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.884+0000 7f0882b83740 1 -- 172.21.15.133:0/3722295537 shutdown_connections 2024-09-11T23:41:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.884+0000 7f0882b83740 1 -- 172.21.15.133:0/3722295537 wait complete. 2024-09-11T23:41:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.887+0000 7f0882b83740 20 client.26207 trim_cache size 0 max 16384 2024-09-11T23:41:36.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.937+0000 7f0882b83740 1 Processor -- start 2024-09-11T23:41:36.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.939+0000 7f0882b83740 1 -- start start 2024-09-11T23:41:36.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.939+0000 7f0882b83740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633670 0x5614ad6503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.939+0000 7f0882b83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633f90 0x5614ad656170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.939+0000 7f0882b83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 0x5614ad6566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.939+0000 7f0882b83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f085c00e730 con 0x5614ad63e850 2024-09-11T23:41:36.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.939+0000 7f0882b83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f085c010270 con 0x5614ad633f90 2024-09-11T23:41:36.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.939+0000 7f0882b83740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f085c00ee00 con 0x5614ad633670 2024-09-11T23:41:36.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.940+0000 7f087e546640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633670 0x5614ad6503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.940+0000 7f087e546640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633670 0x5614ad6503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48778/0 (socket says 172.21.15.133:48778) 2024-09-11T23:41:36.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.940+0000 7f087e546640 1 -- 172.21.15.133:0/538550190 learned_addr learned my addr 172.21.15.133:0/538550190 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:36.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.942+0000 7f087dd45640 1 --2- 172.21.15.133:0/538550190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633f90 0x5614ad656170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.942+0000 7f087ed47640 1 --2- 172.21.15.133:0/538550190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 0x5614ad6566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.943+0000 7f087e546640 1 -- 172.21.15.133:0/538550190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633f90 msgr2=0x5614ad656170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:36.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.943+0000 7f087e546640 1 --2- 172.21.15.133:0/538550190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633f90 0x5614ad656170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.943+0000 7f087e546640 1 -- 172.21.15.133:0/538550190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 msgr2=0x5614ad6566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:36.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.943+0000 7f087e546640 1 --2- 172.21.15.133:0/538550190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 0x5614ad6566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.943+0000 7f087e546640 1 -- 172.21.15.133:0/538550190 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ad63bd70 con 0x5614ad633670 2024-09-11T23:41:36.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.948+0000 7f087e546640 1 --2- 172.21.15.133:0/538550190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633670 0x5614ad6503f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0874004980 tx=0x7f0874005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b708200990c0e82e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:36.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.950+0000 7f0863fff640 1 -- 172.21.15.133:0/538550190 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087406d070 con 0x5614ad633670 2024-09-11T23:41:36.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.950+0000 7f0863fff640 1 -- 172.21.15.133:0/538550190 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0874051070 con 0x5614ad633670 2024-09-11T23:41:36.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.950+0000 7f0863fff640 1 -- 172.21.15.133:0/538550190 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874046040 con 0x5614ad633670 2024-09-11T23:41:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.952+0000 7f0882b83740 1 -- 172.21.15.133:0/538550190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633670 msgr2=0x5614ad6503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.952+0000 7f0882b83740 1 --2- 172.21.15.133:0/538550190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633670 0x5614ad6503f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0874004980 tx=0x7f0874005fd0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.953+0000 7f0882b83740 1 -- 172.21.15.133:0/538550190 shutdown_connections 2024-09-11T23:41:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.953+0000 7f0882b83740 1 --2- 172.21.15.133:0/538550190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad63e850 0x5614ad6566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.953+0000 7f0882b83740 1 --2- 172.21.15.133:0/538550190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad633f90 0x5614ad656170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.953+0000 7f0882b83740 1 --2- 172.21.15.133:0/538550190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633670 0x5614ad6503f0 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0874004980 tx=0x7f0874005fd0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.953+0000 7f0882b83740 1 -- 172.21.15.133:0/538550190 >> 172.21.15.133:0/538550190 conn(0x5614ad62fc00 msgr2=0x5614ad63d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:36.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.958+0000 7f0882b83740 1 -- 172.21.15.133:0/538550190 shutdown_connections 2024-09-11T23:41:36.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.959+0000 7f0882b83740 1 -- 172.21.15.133:0/538550190 wait complete. 2024-09-11T23:41:36.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.959+0000 7f0882b83740 1 -- start start 2024-09-11T23:41:36.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.960+0000 7f0882b83740 1 Processor -- start 2024-09-11T23:41:36.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.960+0000 7f0882b83740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad646ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.960+0000 7f0882b83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad63e850 0x5614ad644180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.960+0000 7f0882b83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad6446c0 0x5614ad644b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.960+0000 7f0882b83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f085c00fba0 con 0x5614ad6446c0 2024-09-11T23:41:36.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.960+0000 7f0882b83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f085c00f4d0 con 0x5614ad63e850 2024-09-11T23:41:36.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.960+0000 7f0882b83740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f085c00c710 con 0x5614ad633f90 2024-09-11T23:41:36.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.960+0000 7f0882b83740 10 client.0 _ll_register_callbacks cb 0x7f0874005170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:36.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.962+0000 7f087ed47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad6446c0 0x5614ad644b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.962+0000 7f087ed47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad6446c0 0x5614ad644b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41828/0 (socket says 172.21.15.133:41828) 2024-09-11T23:41:36.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.962+0000 7f087ed47640 1 -- 172.21.15.133:0/3267370215 learned_addr learned my addr 172.21.15.133:0/3267370215 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:36.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.964+0000 7f087ed47640 1 -- 172.21.15.133:0/3267370215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 msgr2=0x5614ad646ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:41:36.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.964+0000 7f087ed47640 1 --2- 172.21.15.133:0/3267370215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad646ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.964+0000 7f087ed47640 1 -- 172.21.15.133:0/3267370215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad63e850 msgr2=0x5614ad644180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:41:36.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.964+0000 7f087ed47640 1 --2- 172.21.15.133:0/3267370215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad63e850 0x5614ad644180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.964+0000 7f087ed47640 1 -- 172.21.15.133:0/3267370215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ad6b8150 con 0x5614ad6446c0 2024-09-11T23:41:36.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.965+0000 7f087ed47640 1 --2- 172.21.15.133:0/3267370215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad6446c0 0x5614ad644b40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0870098920 tx=0x7f08700a4b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b6edcaae90a817 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:36.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.965+0000 7f0861ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:36.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.966+0000 7f0861ffb640 1 -- 172.21.15.133:0/3267370215 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087009c3c0 con 0x5614ad6446c0 2024-09-11T23:41:36.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.966+0000 7f0882b83740 1 -- 172.21.15.133:0/3267370215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0874051070 con 0x5614ad6446c0 2024-09-11T23:41:36.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.966+0000 7f0861ffb640 1 -- 172.21.15.133:0/3267370215 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f08700a4680 con 0x5614ad6446c0 2024-09-11T23:41:36.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.967+0000 7f0861ffb640 1 -- 172.21.15.133:0/3267370215 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08700a4880 con 0x5614ad6446c0 2024-09-11T23:41:36.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.967+0000 7f0882b83740 10 client.16767 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:36.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.967+0000 7f0882b83740 1 -- 172.21.15.133:0/3267370215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614ad6b9400 con 0x5614ad6446c0 2024-09-11T23:41:36.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.969+0000 7f0861ffb640 1 -- 172.21.15.133:0/3267370215 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0874051070 con 0x5614ad6446c0 2024-09-11T23:41:36.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.969+0000 7f0861ffb640 1 -- 172.21.15.133:0/3267370215 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0870097ca0 con 0x5614ad6446c0 2024-09-11T23:41:36.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.969+0000 7f0882b83740 10 client.16767 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:36.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.969+0000 7f0882b83740 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:36.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.969+0000 7f0882b83740 20 client.16767 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:36.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.969+0000 7f0882b83740 1 --2- 172.21.15.133:0/3267370215 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5614ad6b9b20 0x5614ad6bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:36.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.969+0000 7f0882b83740 4 client.16767 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:41:36.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.969+0000 7f0882b83740 1 -- 172.21.15.133:0/3267370215 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5614ad5a0d60 con 0x5614ad6b9b20 2024-09-11T23:41:36.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.974+0000 7f087e546640 1 --2- 172.21.15.133:0/3267370215 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5614ad6b9b20 0x5614ad6bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:36.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.990+0000 7f087e546640 1 --2- 172.21.15.133:0/3267370215 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5614ad6b9b20 0x5614ad6bbfa0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f5f7a3eed2a023b server_cookie=fc1f2256cf9646e6 in_seq=0 out_seq=0 2024-09-11T23:41:36.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.990+0000 7f0861ffb640 10 client.16767 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:36.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.992+0000 7f0861ffb640 1 -- 172.21.15.133:0/3267370215 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5614ad5a0d60 con 0x5614ad6b9b20 2024-09-11T23:41:36.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.992+0000 7f0861ffb640 10 client.16767 handle_command_reply: tid=0 2024-09-11T23:41:36.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.992+0000 7f0882b83740 1 client.16767 shutdown 2024-09-11T23:41:36.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.993+0000 7f0882b83740 1 -- 172.21.15.133:0/3267370215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad6446c0 msgr2=0x5614ad644b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:36.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.993+0000 7f0882b83740 1 --2- 172.21.15.133:0/3267370215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad6446c0 0x5614ad644b40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0870098920 tx=0x7f08700a4b10 comp rx=0 tx=0).stop 2024-09-11T23:41:36.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.994+0000 7f0882b83740 1 -- 172.21.15.133:0/3267370215 shutdown_connections 2024-09-11T23:41:36.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.994+0000 7f0882b83740 1 --2- 172.21.15.133:0/3267370215 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5614ad6b9b20 0x5614ad6bbfa0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.994+0000 7f0882b83740 1 --2- 172.21.15.133:0/3267370215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614ad6446c0 0x5614ad644b40 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.994+0000 7f0882b83740 1 --2- 172.21.15.133:0/3267370215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614ad63e850 0x5614ad644180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.994+0000 7f0882b83740 1 --2- 172.21.15.133:0/3267370215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614ad633f90 0x5614ad646ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:36.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.994+0000 7f0882b83740 1 -- 172.21.15.133:0/3267370215 >> 172.21.15.133:0/3267370215 conn(0x5614ad62fc00 msgr2=0x5614ad63cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:36.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.994+0000 7f0861ffb640 0 client.16767 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:36.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.996+0000 7f0882b83740 1 -- 172.21.15.133:0/3267370215 shutdown_connections 2024-09-11T23:41:36.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.996+0000 7f0882b83740 1 -- 172.21.15.133:0/3267370215 wait complete. 2024-09-11T23:41:36.997 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:41:36.997 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:41:36.997 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:41:36.997 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:41:36.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.996+0000 7f0882b83740 20 client.16767 trim_cache size 0 max 16384 2024-09-11T23:41:36.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.999+0000 7f087f548640 1 -- 172.21.15.133:0/2709648014 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08640a2f30 msgr2=0x7f08640a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:36.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.999+0000 7f087f548640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08640a2f30 0x7f08640a53f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5614ad630f80 tx=0x7f0868007b40 comp rx=0 tx=0).stop 2024-09-11T23:41:36.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.999+0000 7f087f548640 1 -- 172.21.15.133:0/2709648014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0878117750 msgr2=0x7f08781aaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:36.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:36.999+0000 7f087f548640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0878117750 0x7f08781aaf60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f087000b9e0 tx=0x7f087000bea0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:37.002+0000 7f087f548640 1 -- 172.21.15.133:0/2709648014 shutdown_connections 2024-09-11T23:41:37.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:37.002+0000 7f087f548640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08640a2f30 0x7f08640a53f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:37.002+0000 7f087f548640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0878117750 0x7f08781aaf60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:37.002+0000 7f087f548640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087811b140 0x7f0878117210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:37.002+0000 7f087f548640 1 --2- 172.21.15.133:0/2709648014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087806ea30 0x7f0878116cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:37.002+0000 7f087f548640 1 -- 172.21.15.133:0/2709648014 >> 172.21.15.133:0/2709648014 conn(0x7f0878073180 msgr2=0x7f087811b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:37.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:37.006+0000 7f087f548640 1 -- 172.21.15.133:0/2709648014 shutdown_connections 2024-09-11T23:41:37.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:37.006+0000 7f087f548640 1 -- 172.21.15.133:0/2709648014 wait complete. 2024-09-11T23:41:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.602+0000 7f7f29b49640 1 -- 172.21.15.133:0/2522799221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f241095d0 msgr2=0x7f7f24109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.602+0000 7f7f29b49640 1 --2- 172.21.15.133:0/2522799221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f241095d0 0x7f7f24109a10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009900 tx=0x7f7f1402a420 comp rx=0 tx=0).stop 2024-09-11T23:41:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.603+0000 7f7f29b49640 1 -- 172.21.15.133:0/2522799221 shutdown_connections 2024-09-11T23:41:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.603+0000 7f7f29b49640 1 --2- 172.21.15.133:0/2522799221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f2411b140 0x7f7f2411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.603+0000 7f7f29b49640 1 --2- 172.21.15.133:0/2522799221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f241095d0 0x7f7f24109a10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.603+0000 7f7f29b49640 1 --2- 172.21.15.133:0/2522799221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f241091f0 0x7f7f24110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.603+0000 7f7f29b49640 1 -- 172.21.15.133:0/2522799221 >> 172.21.15.133:0/2522799221 conn(0x7f7f2406cc60 msgr2=0x7f7f2406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.603+0000 7f7f29b49640 1 -- 172.21.15.133:0/2522799221 shutdown_connections 2024-09-11T23:41:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.604+0000 7f7f29b49640 1 -- 172.21.15.133:0/2522799221 wait complete. 2024-09-11T23:41:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.605+0000 7f7f29b49640 1 Processor -- start 2024-09-11T23:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.606+0000 7f7f29b49640 1 -- start start 2024-09-11T23:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.606+0000 7f7f29b49640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f241091f0 0x7f7f24118470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.606+0000 7f7f29b49640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f2411b140 0x7f7f241189b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f29b49640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f241114f0 0x7f7f241119a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f29b49640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24120b60 con 0x7f7f2411b140 2024-09-11T23:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f29b49640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241209e0 con 0x7f7f241114f0 2024-09-11T23:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f29b49640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24120ce0 con 0x7f7f241091f0 2024-09-11T23:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f22ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f2411b140 0x7f7f241189b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f237fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f241091f0 0x7f7f24118470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f22ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f2411b140 0x7f7f241189b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41852/0 (socket says 172.21.15.133:41852) 2024-09-11T23:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f22ffd640 1 -- 172.21.15.133:0/418034645 learned_addr learned my addr 172.21.15.133:0/418034645 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f23fff640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f241114f0 0x7f7f241119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f23fff640 1 -- 172.21.15.133:0/418034645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f241091f0 msgr2=0x7f7f24118470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f23fff640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f241091f0 0x7f7f24118470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f23fff640 1 -- 172.21.15.133:0/418034645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f2411b140 msgr2=0x7f7f241189b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f23fff640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f2411b140 0x7f7f241189b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.607+0000 7f7f23fff640 1 -- 172.21.15.133:0/418034645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f24112170 con 0x7f7f241114f0 2024-09-11T23:41:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.608+0000 7f7f237fe640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f241091f0 0x7f7f24118470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:41:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.608+0000 7f7f23fff640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f241114f0 0x7f7f241119a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7f2406af00 tx=0x7f7f1800cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.608+0000 7f7f20ff9640 1 -- 172.21.15.133:0/418034645 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18004220 con 0x7f7f241114f0 2024-09-11T23:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.609+0000 7f7f29b49640 1 -- 172.21.15.133:0/418034645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f24115f90 con 0x7f7f241114f0 2024-09-11T23:41:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.609+0000 7f7f29b49640 1 -- 172.21.15.133:0/418034645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241164e0 con 0x7f7f241114f0 2024-09-11T23:41:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.612+0000 7f7f20ff9640 1 -- 172.21.15.133:0/418034645 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f180043c0 con 0x7f7f241114f0 2024-09-11T23:41:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.612+0000 7f7f20ff9640 1 -- 172.21.15.133:0/418034645 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18003e50 con 0x7f7f241114f0 2024-09-11T23:41:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.615+0000 7f7f2c9ba740 1 Processor -- start 2024-09-11T23:41:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.619+0000 7f7f20ff9640 1 -- 172.21.15.133:0/418034645 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7f180077a0 con 0x7f7f241114f0 2024-09-11T23:41:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.619+0000 7f7f2c9ba740 1 -- start start 2024-09-11T23:41:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.620+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 0x55fb43d100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.620+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 0x55fb43d1a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.620+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d1ae80 0x55fb43d1d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.620+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fb43c742f0 con 0x55fb43d0fcc0 2024-09-11T23:41:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.620+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fb43c1c270 con 0x55fb43d105e0 2024-09-11T23:41:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.620+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fb43c702b0 con 0x55fb43d1ae80 2024-09-11T23:41:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.620+0000 7f7f237fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 0x55fb43d1a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.620+0000 7f7f237fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 0x55fb43d1a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42674/0 (socket says 172.21.15.133:42674) 2024-09-11T23:41:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.620+0000 7f7f237fe640 1 -- 172.21.15.133:0/2768194471 learned_addr learned my addr 172.21.15.133:0/2768194471 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.621+0000 7f7f237fe640 1 -- 172.21.15.133:0/2768194471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d1ae80 msgr2=0x55fb43d1d270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:41:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.621+0000 7f7f237fe640 1 --2- 172.21.15.133:0/2768194471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d1ae80 0x55fb43d1d270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.621+0000 7f7f237fe640 1 -- 172.21.15.133:0/2768194471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 msgr2=0x55fb43d100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.621+0000 7f7f237fe640 1 --2- 172.21.15.133:0/2768194471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 0x55fb43d100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.621+0000 7f7f237fe640 1 -- 172.21.15.133:0/2768194471 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb43c1b340 con 0x55fb43d105e0 2024-09-11T23:41:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.622+0000 7f7f237fe640 1 --2- 172.21.15.133:0/2768194471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 0x55fb43d1a940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c008f10 tx=0x7f7f1c00d710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883b386aaeff2591 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.622+0000 7f7f20ff9640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7f0c0a30a0 0x7f7f0c0a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.624+0000 7f7f20ff9640 1 -- 172.21.15.133:0/418034645 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f7f180cee90 con 0x7f7f241114f0 2024-09-11T23:41:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.625+0000 7f7f22ffd640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7f0c0a30a0 0x7f7f0c0a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.626+0000 7f7f22ffd640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7f0c0a30a0 0x7f7f0c0a5560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f24112cb0 tx=0x7f7f14031a40 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.627+0000 7f7f09ffb640 1 -- 172.21.15.133:0/2768194471 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00f830 con 0x55fb43d105e0 2024-09-11T23:41:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.627+0000 7f7f09ffb640 1 -- 172.21.15.133:0/2768194471 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f1c00e3b0 con 0x55fb43d105e0 2024-09-11T23:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.629+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/2768194471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 msgr2=0x55fb43d1a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.629+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/2768194471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 0x55fb43d1a940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c008f10 tx=0x7f7f1c00d710 comp rx=0 tx=0).stop 2024-09-11T23:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.631+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/2768194471 shutdown_connections 2024-09-11T23:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.631+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/2768194471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d1ae80 0x55fb43d1d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.631+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/2768194471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 0x55fb43d1a940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.631+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/2768194471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 0x55fb43d100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.631+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/2768194471 >> 172.21.15.133:0/2768194471 conn(0x55fb43d0c230 msgr2=0x55fb43d0c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.631+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/2768194471 shutdown_connections 2024-09-11T23:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.632+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/2768194471 wait complete. 2024-09-11T23:41:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.632+0000 7f7f2c9ba740 1 -- start start 2024-09-11T23:41:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.633+0000 7f7f2c9ba740 1 Processor -- start 2024-09-11T23:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.633+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d0fcc0 0x55fb43d17ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.633+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 0x55fb43d181e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.633+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d1ae80 0x55fb43d18720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.633+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fb43d1daa0 con 0x55fb43d1ae80 2024-09-11T23:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.633+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fb43bfd290 con 0x55fb43d105e0 2024-09-11T23:41:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.633+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fb43d1dc20 con 0x55fb43d0fcc0 2024-09-11T23:41:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.633+0000 7f7f2c9ba740 10 client.0 _ll_register_callbacks cb 0x55fb43c2e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.634+0000 7f7f23fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d1ae80 0x55fb43d18720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.634+0000 7f7f23fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d1ae80 0x55fb43d18720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41874/0 (socket says 172.21.15.133:41874) 2024-09-11T23:41:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.634+0000 7f7f23fff640 1 -- 172.21.15.133:0/637657198 learned_addr learned my addr 172.21.15.133:0/637657198 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.635+0000 7f7f237fe640 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d0fcc0 0x55fb43d17ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.635+0000 7f7f22ffd640 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 0x55fb43d181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.635+0000 7f7f237fe640 1 -- 172.21.15.133:0/637657198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 msgr2=0x55fb43d181e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.635+0000 7f7f237fe640 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 0x55fb43d181e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.635+0000 7f7f237fe640 1 -- 172.21.15.133:0/637657198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d1ae80 msgr2=0x55fb43d18720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.636+0000 7f7f237fe640 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d1ae80 0x55fb43d18720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.636+0000 7f7f237fe640 1 -- 172.21.15.133:0/637657198 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb43987210 con 0x55fb43d0fcc0 2024-09-11T23:41:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.636+0000 7f7f237fe640 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d0fcc0 0x55fb43d17ca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c008ee0 tx=0x7f7f1c008c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54ba01e62cbe9bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.644+0000 7f7f02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:41:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.644+0000 7f7f02ffd640 1 -- 172.21.15.133:0/637657198 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00ba50 con 0x55fb43d0fcc0 2024-09-11T23:41:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.644+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/637657198 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f1c00e3b0 con 0x55fb43d0fcc0 2024-09-11T23:41:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.644+0000 7f7f02ffd640 1 -- 172.21.15.133:0/637657198 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f7f1c00bbf0 con 0x55fb43d0fcc0 2024-09-11T23:41:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.645+0000 7f7f02ffd640 1 -- 172.21.15.133:0/637657198 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f1c00be10 con 0x55fb43d0fcc0 2024-09-11T23:41:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.645+0000 7f7f2c9ba740 10 client.26194 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.645+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/637657198 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb43a53620 con 0x55fb43d0fcc0 2024-09-11T23:41:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.646+0000 7f7f02ffd640 1 -- 172.21.15.133:0/637657198 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f1c00e3b0 con 0x55fb43d0fcc0 2024-09-11T23:41:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.646+0000 7f7f02ffd640 1 -- 172.21.15.133:0/637657198 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f1c02b020 con 0x55fb43d0fcc0 2024-09-11T23:41:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.646+0000 7f7f2c9ba740 10 client.26194 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.646+0000 7f7f2c9ba740 10 client.26194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.646+0000 7f7f2c9ba740 20 client.26194 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.646+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fb43d2b130 0x55fb43d2d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.646+0000 7f7f2c9ba740 4 client.26194 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.646+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/637657198 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb43c81a20 con 0x55fb43d2b130 2024-09-11T23:41:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.647+0000 7f7f22ffd640 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fb43d2b130 0x55fb43d2d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.651+0000 7f7f22ffd640 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fb43d2b130 0x55fb43d2d5b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af03ba997f9485e server_cookie=201ae9268d16bcdc in_seq=0 out_seq=0 2024-09-11T23:41:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.651+0000 7f7f02ffd640 10 client.26194 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.653+0000 7f7f02ffd640 1 -- 172.21.15.133:0/637657198 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb43c81a20 con 0x55fb43d2b130 2024-09-11T23:41:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.653+0000 7f7f02ffd640 10 client.26194 handle_command_reply: tid=0 2024-09-11T23:41:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.653+0000 7f7f2c9ba740 1 client.26194 shutdown 2024-09-11T23:41:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.654+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/637657198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d0fcc0 msgr2=0x55fb43d17ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.654+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d0fcc0 0x55fb43d17ca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c008ee0 tx=0x7f7f1c008c50 comp rx=0 tx=0).stop 2024-09-11T23:41:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.655+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/637657198 shutdown_connections 2024-09-11T23:41:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.655+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fb43d2b130 0x55fb43d2d5b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.655+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d1ae80 0x55fb43d18720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.655+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d105e0 0x55fb43d181e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.655+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/637657198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d0fcc0 0x55fb43d17ca0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.655+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/637657198 >> 172.21.15.133:0/637657198 conn(0x55fb43d0c230 msgr2=0x55fb43d11f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.657+0000 7f7f02ffd640 0 client.26194 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.657+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/637657198 shutdown_connections 2024-09-11T23:41:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.657+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/637657198 wait complete. 2024-09-11T23:41:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.658+0000 7f7f2c9ba740 20 client.26194 trim_cache size 0 max 16384 2024-09-11T23:41:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.689+0000 7f7f2c9ba740 1 Processor -- start 2024-09-11T23:41:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.689+0000 7f7f2c9ba740 1 -- start start 2024-09-11T23:41:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.689+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 0x55fb43d0b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.689+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d105e0 0x55fb43d32ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.689+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 0x55fb43d33400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.689+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1000e700 con 0x55fb43d0fcc0 2024-09-11T23:41:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.689+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f10010240 con 0x55fb43d1ae80 2024-09-11T23:41:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.689+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1000edd0 con 0x55fb43d105e0 2024-09-11T23:41:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.689+0000 7f7f237fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 0x55fb43d0b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.689+0000 7f7f237fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 0x55fb43d0b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41888/0 (socket says 172.21.15.133:41888) 2024-09-11T23:41:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.689+0000 7f7f237fe640 1 -- 172.21.15.133:0/2219556325 learned_addr learned my addr 172.21.15.133:0/2219556325 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.690+0000 7f7f22ffd640 1 --2- 172.21.15.133:0/2219556325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d105e0 0x55fb43d32ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.690+0000 7f7f23fff640 1 --2- 172.21.15.133:0/2219556325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 0x55fb43d33400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.692+0000 7f7f23fff640 1 -- 172.21.15.133:0/2219556325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d105e0 msgr2=0x55fb43d32ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.692+0000 7f7f23fff640 1 --2- 172.21.15.133:0/2219556325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d105e0 0x55fb43d32ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.692+0000 7f7f23fff640 1 -- 172.21.15.133:0/2219556325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 msgr2=0x55fb43d0b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.692+0000 7f7f23fff640 1 --2- 172.21.15.133:0/2219556325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 0x55fb43d0b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.692+0000 7f7f23fff640 1 -- 172.21.15.133:0/2219556325 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb43d0a7b0 con 0x55fb43d1ae80 2024-09-11T23:41:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.693+0000 7f7f23fff640 1 --2- 172.21.15.133:0/2219556325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 0x55fb43d33400 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f18097820 tx=0x7f7f18097040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac5369a27f2d6e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.693+0000 7f7f097fa640 1 -- 172.21.15.133:0/2219556325 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1801d040 con 0x55fb43d1ae80 2024-09-11T23:41:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.693+0000 7f7f097fa640 1 -- 172.21.15.133:0/2219556325 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f180a4030 con 0x55fb43d1ae80 2024-09-11T23:41:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.695+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/2219556325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 msgr2=0x55fb43d33400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.695+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/2219556325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 0x55fb43d33400 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f18097820 tx=0x7f7f18097040 comp rx=0 tx=0).stop 2024-09-11T23:41:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.695+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/2219556325 shutdown_connections 2024-09-11T23:41:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.695+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/2219556325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 0x55fb43d33400 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.695+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/2219556325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d105e0 0x55fb43d32ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.695+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/2219556325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 0x55fb43d0b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.695+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/2219556325 >> 172.21.15.133:0/2219556325 conn(0x55fb43d0c230 msgr2=0x55fb43d0aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.695+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/2219556325 shutdown_connections 2024-09-11T23:41:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.695+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/2219556325 wait complete. 2024-09-11T23:41:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.696+0000 7f7f2c9ba740 1 -- start start 2024-09-11T23:41:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.696+0000 7f7f2c9ba740 1 Processor -- start 2024-09-11T23:41:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.697+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 0x55fb43d23420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.697+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d105e0 0x55fb43d15020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.697+0000 7f7f2c9ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 0x55fb43d15560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.697+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1000fb70 con 0x55fb43d0fcc0 2024-09-11T23:41:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.697+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1000f4a0 con 0x55fb43d1ae80 2024-09-11T23:41:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.697+0000 7f7f2c9ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1000c6e0 con 0x55fb43d105e0 2024-09-11T23:41:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.697+0000 7f7f2c9ba740 10 client.0 _ll_register_callbacks cb 0x7f7f1c008d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.698+0000 7f7f23fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 0x55fb43d15560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.698+0000 7f7f23fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 0x55fb43d15560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42690/0 (socket says 172.21.15.133:42690) 2024-09-11T23:41:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.698+0000 7f7f23fff640 1 -- 172.21.15.133:0/3849174663 learned_addr learned my addr 172.21.15.133:0/3849174663 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.698+0000 7f7f22ffd640 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d105e0 0x55fb43d15020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:37 smithi138 ceph-mon[35142]: pgmap v148: 97 pgs: 97 active+clean; 34 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 10 KiB/s rd, 268 MiB/s wr, 1.90k op/s 2024-09-11T23:41:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.699+0000 7f7f23fff640 1 -- 172.21.15.133:0/3849174663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d105e0 msgr2=0x55fb43d15020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.699+0000 7f7f23fff640 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d105e0 0x55fb43d15020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.699+0000 7f7f23fff640 1 -- 172.21.15.133:0/3849174663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 msgr2=0x55fb43d23420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.699+0000 7f7f23fff640 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 0x55fb43d23420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.699+0000 7f7f23fff640 1 -- 172.21.15.133:0/3849174663 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb43d15b20 con 0x55fb43d1ae80 2024-09-11T23:41:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.700+0000 7f7f23fff640 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 0x55fb43d15560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7f1809b0c0 tx=0x7f7f180cc870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec4cc9d599b957e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.700+0000 7f7f02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:41:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.700+0000 7f7f02ffd640 1 -- 172.21.15.133:0/3849174663 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1801d040 con 0x55fb43d1ae80 2024-09-11T23:41:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.700+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/3849174663 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f180a4030 con 0x55fb43d1ae80 2024-09-11T23:41:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.700+0000 7f7f02ffd640 1 -- 172.21.15.133:0/3849174663 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f7f18097a30 con 0x55fb43d1ae80 2024-09-11T23:41:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.701+0000 7f7f02ffd640 1 -- 172.21.15.133:0/3849174663 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f180ac050 con 0x55fb43d1ae80 2024-09-11T23:41:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.701+0000 7f7f2c9ba740 10 client.26222 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.701+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/3849174663 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb43d94e90 con 0x55fb43d1ae80 2024-09-11T23:41:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.701+0000 7f7f02ffd640 1 -- 172.21.15.133:0/3849174663 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f180a4030 con 0x55fb43d1ae80 2024-09-11T23:41:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.703+0000 7f7f02ffd640 1 -- 172.21.15.133:0/3849174663 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f1809a3a0 con 0x55fb43d1ae80 2024-09-11T23:41:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.703+0000 7f7f2c9ba740 10 client.26222 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.703+0000 7f7f2c9ba740 10 client.26222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.703+0000 7f7f2c9ba740 20 client.26222 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.703+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fb43d95490 0x55fb43d97880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.703+0000 7f7f2c9ba740 4 client.26222 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:41:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.703+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/3849174663 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb43c7bd70 con 0x55fb43d95490 2024-09-11T23:41:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.704+0000 7f7f237fe640 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fb43d95490 0x55fb43d97880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.705+0000 7f7f237fe640 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fb43d95490 0x55fb43d97880 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4ee472d7b50468e server_cookie=b9ffe6bab70d2f09 in_seq=0 out_seq=0 2024-09-11T23:41:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:37.705+0000 7f7f02ffd640 10 client.26222 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:37.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:37 smithi133 ceph-mon[31430]: pgmap v148: 97 pgs: 97 active+clean; 34 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 10 KiB/s rd, 268 MiB/s wr, 1.90k op/s 2024-09-11T23:41:37.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:37 smithi178 ceph-mon[35882]: pgmap v148: 97 pgs: 97 active+clean; 34 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 10 KiB/s rd, 268 MiB/s wr, 1.90k op/s 2024-09-11T23:41:38.626 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:41:38.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:41:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.172+0000 7f7f02ffd640 1 -- 172.21.15.133:0/3849174663 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb43c7bd70 con 0x55fb43d95490 2024-09-11T23:41:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.172+0000 7f7f02ffd640 10 client.26222 handle_command_reply: tid=0 2024-09-11T23:41:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.173+0000 7f7f2c9ba740 1 client.26222 shutdown 2024-09-11T23:41:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.173+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/3849174663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 msgr2=0x55fb43d15560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.173+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 0x55fb43d15560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7f1809b0c0 tx=0x7f7f180cc870 comp rx=0 tx=0).stop 2024-09-11T23:41:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.174+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/3849174663 shutdown_connections 2024-09-11T23:41:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.174+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fb43d95490 0x55fb43d97880 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.174+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb43d1ae80 0x55fb43d15560 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.174+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fb43d105e0 0x55fb43d15020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.174+0000 7f7f2c9ba740 1 --2- 172.21.15.133:0/3849174663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb43d0fcc0 0x55fb43d23420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.174+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/3849174663 >> 172.21.15.133:0/3849174663 conn(0x55fb43d0c230 msgr2=0x55fb43d0b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.174+0000 7f7f02ffd640 0 client.26222 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.174+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/3849174663 shutdown_connections 2024-09-11T23:41:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.174+0000 7f7f2c9ba740 1 -- 172.21.15.133:0/3849174663 wait complete. 2024-09-11T23:41:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.175+0000 7f7f2c9ba740 20 client.26222 trim_cache size 0 max 16384 2024-09-11T23:41:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:41:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:41:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:41:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:41:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.179+0000 7f7f29b49640 1 -- 172.21.15.133:0/418034645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7f0c0a30a0 msgr2=0x7f7f0c0a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.179+0000 7f7f29b49640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7f0c0a30a0 0x7f7f0c0a5560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f24112cb0 tx=0x7f7f14031a40 comp rx=0 tx=0).stop 2024-09-11T23:41:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.179+0000 7f7f29b49640 1 -- 172.21.15.133:0/418034645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f241114f0 msgr2=0x7f7f241119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.179+0000 7f7f29b49640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f241114f0 0x7f7f241119a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7f2406af00 tx=0x7f7f1800cba0 comp rx=0 tx=0).stop 2024-09-11T23:41:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.179+0000 7f7f29b49640 1 -- 172.21.15.133:0/418034645 shutdown_connections 2024-09-11T23:41:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.179+0000 7f7f29b49640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7f0c0a30a0 0x7f7f0c0a5560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.179+0000 7f7f29b49640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f241114f0 0x7f7f241119a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.179+0000 7f7f29b49640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f2411b140 0x7f7f241189b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.179+0000 7f7f29b49640 1 --2- 172.21.15.133:0/418034645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f241091f0 0x7f7f24118470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.179+0000 7f7f29b49640 1 -- 172.21.15.133:0/418034645 >> 172.21.15.133:0/418034645 conn(0x7f7f2406cc60 msgr2=0x7f7f2411b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.180+0000 7f7f29b49640 1 -- 172.21.15.133:0/418034645 shutdown_connections 2024-09-11T23:41:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:39.180+0000 7f7f29b49640 1 -- 172.21.15.133:0/418034645 wait complete. 2024-09-11T23:41:39.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:39 smithi138 ceph-mon[35142]: pgmap v149: 97 pgs: 97 active+clean; 34 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 357 MiB/s wr, 2.73k op/s 2024-09-11T23:41:39.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:39 smithi133 ceph-mon[31430]: pgmap v149: 97 pgs: 97 active+clean; 34 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 357 MiB/s wr, 2.73k op/s 2024-09-11T23:41:39.780 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:39.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:39 smithi178 ceph-mon[35882]: pgmap v149: 97 pgs: 97 active+clean; 34 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 357 MiB/s wr, 2.73k op/s 2024-09-11T23:41:40.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.992+0000 7f46812ba640 1 -- 172.21.15.133:0/2718213156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c10ea80 msgr2=0x7f467c1153d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:40.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.992+0000 7f46812ba640 1 --2- 172.21.15.133:0/2718213156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c10ea80 0x7f467c1153d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4674008840 tx=0x7f467400aec0 comp rx=0 tx=0).stop 2024-09-11T23:41:40.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.992+0000 7f46812ba640 1 -- 172.21.15.133:0/2718213156 shutdown_connections 2024-09-11T23:41:40.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.992+0000 7f46812ba640 1 --2- 172.21.15.133:0/2718213156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c10ea80 0x7f467c1153d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:40.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.992+0000 7f46812ba640 1 --2- 172.21.15.133:0/2718213156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f467c109e00 0x7f467c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:40.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.992+0000 7f46812ba640 1 --2- 172.21.15.133:0/2718213156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f467c109450 0x7f467c109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:40.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.992+0000 7f46812ba640 1 -- 172.21.15.133:0/2718213156 >> 172.21.15.133:0/2718213156 conn(0x7f467c073210 msgr2=0x7f467c073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:40.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.993+0000 7f46812ba640 1 -- 172.21.15.133:0/2718213156 shutdown_connections 2024-09-11T23:41:40.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.996+0000 7f46812ba640 1 -- 172.21.15.133:0/2718213156 wait complete. 2024-09-11T23:41:40.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.997+0000 7f46812ba640 1 Processor -- start 2024-09-11T23:41:40.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.997+0000 7f46812ba640 1 -- start start 2024-09-11T23:41:40.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:40.998+0000 7f46812ba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c109450 0x7f467c1a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:41.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.001+0000 7f46812ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f467c109e00 0x7f467c1a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:41.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.001+0000 7f46812ba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f467c10ea80 0x7f467c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:41.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.001+0000 7f46812ba640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f467c1183d0 con 0x7f467c109e00 2024-09-11T23:41:41.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.001+0000 7f46812ba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f467c118250 con 0x7f467c10ea80 2024-09-11T23:41:41.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.001+0000 7f46812ba640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f467c118550 con 0x7f467c109450 2024-09-11T23:41:41.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.002+0000 7f467affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c109450 0x7f467c1a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:41.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.002+0000 7f467b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f467c10ea80 0x7f467c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:41.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.002+0000 7f467b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f467c10ea80 0x7f467c19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:42702/0 (socket says 172.21.15.133:42702) 2024-09-11T23:41:41.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.002+0000 7f467b7fe640 1 -- 172.21.15.133:0/2971848491 learned_addr learned my addr 172.21.15.133:0/2971848491 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:41.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.003+0000 7f467affd640 1 -- 172.21.15.133:0/2971848491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f467c10ea80 msgr2=0x7f467c19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:41.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.003+0000 7f467affd640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f467c10ea80 0x7f467c19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:41.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.003+0000 7f467affd640 1 -- 172.21.15.133:0/2971848491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f467c109e00 msgr2=0x7f467c1a4ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:41:41.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.003+0000 7f467affd640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f467c109e00 0x7f467c1a4ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:41.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.003+0000 7f467affd640 1 -- 172.21.15.133:0/2971848491 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467c19eeb0 con 0x7f467c109450 2024-09-11T23:41:41.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.003+0000 7f467affd640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c109450 0x7f467c1a4570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f466c00e9f0 tx=0x7f466c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:41.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.004+0000 7f4663fff640 1 -- 172.21.15.133:0/2971848491 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c00ccc0 con 0x7f467c109450 2024-09-11T23:41:41.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.004+0000 7f4663fff640 1 -- 172.21.15.133:0/2971848491 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466c010910 con 0x7f467c109450 2024-09-11T23:41:41.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.004+0000 7f46812ba640 1 -- 172.21.15.133:0/2971848491 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467c19f1a0 con 0x7f467c109450 2024-09-11T23:41:41.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.004+0000 7f4663fff640 1 -- 172.21.15.133:0/2971848491 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c018980 con 0x7f467c109450 2024-09-11T23:41:41.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.004+0000 7f46812ba640 1 -- 172.21.15.133:0/2971848491 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467c1ab4f0 con 0x7f467c109450 2024-09-11T23:41:41.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.006+0000 7f46812ba640 1 -- 172.21.15.133:0/2971848491 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4668005290 con 0x7f467c109450 2024-09-11T23:41:41.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.010+0000 7f4663fff640 1 -- 172.21.15.133:0/2971848491 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f466c010430 con 0x7f467c109450 2024-09-11T23:41:41.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.012+0000 7f4663fff640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46640a2df0 0x7f46640a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:41.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.014+0000 7f4663fff640 1 -- 172.21.15.133:0/2971848491 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f466c0cfac0 con 0x7f467c109450 2024-09-11T23:41:41.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.014+0000 7f4663fff640 1 -- 172.21.15.133:0/2971848491 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f466c09c6f0 con 0x7f467c109450 2024-09-11T23:41:41.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.015+0000 7f467a7fc640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46640a2df0 0x7f46640a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:41.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.033+0000 7f467a7fc640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46640a2df0 0x7f46640a52b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f467c19fe90 tx=0x7f467000d040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:41.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.504+0000 7f46812ba640 1 -- 172.21.15.133:0/2971848491 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4668005580 con 0x7f467c109450 2024-09-11T23:41:41.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.646+0000 7f4663fff640 1 -- 172.21.15.133:0/2971848491 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f466c0968e0 con 0x7f467c109450 2024-09-11T23:41:41.646 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:41:41.647 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:41:41.647 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:41:41.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.652+0000 7f46812ba640 1 -- 172.21.15.133:0/2971848491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46640a2df0 msgr2=0x7f46640a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:41.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.652+0000 7f46812ba640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46640a2df0 0x7f46640a52b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f467c19fe90 tx=0x7f467000d040 comp rx=0 tx=0).stop 2024-09-11T23:41:41.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.652+0000 7f46812ba640 1 -- 172.21.15.133:0/2971848491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c109450 msgr2=0x7f467c1a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:41.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.652+0000 7f46812ba640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c109450 0x7f467c1a4570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f466c00e9f0 tx=0x7f466c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:41:41.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.652+0000 7f46812ba640 1 -- 172.21.15.133:0/2971848491 shutdown_connections 2024-09-11T23:41:41.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.652+0000 7f46812ba640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46640a2df0 0x7f46640a52b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.652+0000 7f46812ba640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f467c10ea80 0x7f467c19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.652+0000 7f46812ba640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f467c109e00 0x7f467c1a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.652+0000 7f46812ba640 1 --2- 172.21.15.133:0/2971848491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c109450 0x7f467c1a4570 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.652+0000 7f46812ba640 1 -- 172.21.15.133:0/2971848491 >> 172.21.15.133:0/2971848491 conn(0x7f467c073210 msgr2=0x7f467c10c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.653+0000 7f46812ba640 1 -- 172.21.15.133:0/2971848491 shutdown_connections 2024-09-11T23:41:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:41.653+0000 7f46812ba640 1 -- 172.21.15.133:0/2971848491 wait complete. 2024-09-11T23:41:41.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:41 smithi133 ceph-mon[31430]: pgmap v150: 97 pgs: 97 active+clean; 34 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 200 MiB/s wr, 1.75k op/s 2024-09-11T23:41:42.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:41 smithi178 ceph-mon[35882]: pgmap v150: 97 pgs: 97 active+clean; 34 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 200 MiB/s wr, 1.75k op/s 2024-09-11T23:41:42.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:41 smithi138 ceph-mon[35142]: pgmap v150: 97 pgs: 97 active+clean; 34 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 200 MiB/s wr, 1.75k op/s 2024-09-11T23:41:42.901 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:41:43.086 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:42 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2971848491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:43.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:42 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2971848491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:43.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:42 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2971848491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:41:43.989 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:44.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:43 smithi133 ceph-mon[31430]: pgmap v151: 97 pgs: 97 active+clean; 35 GiB data, 109 GiB used, 964 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 305 MiB/s wr, 2.08k op/s 2024-09-11T23:41:44.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:43 smithi138 ceph-mon[35142]: pgmap v151: 97 pgs: 97 active+clean; 35 GiB data, 109 GiB used, 964 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 305 MiB/s wr, 2.08k op/s 2024-09-11T23:41:44.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:43 smithi178 ceph-mon[35882]: pgmap v151: 97 pgs: 97 active+clean; 35 GiB data, 109 GiB used, 964 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 305 MiB/s wr, 2.08k op/s 2024-09-11T23:41:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.767+0000 7fee8888d640 1 -- 172.21.15.133:0/2278154888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee841008d0 msgr2=0x7fee84100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.767+0000 7fee8888d640 1 --2- 172.21.15.133:0/2278154888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee841008d0 0x7fee84100d30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009930 tx=0x7fee6c02a420 comp rx=0 tx=0).stop 2024-09-11T23:41:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.767+0000 7fee8888d640 1 -- 172.21.15.133:0/2278154888 shutdown_connections 2024-09-11T23:41:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.767+0000 7fee8888d640 1 --2- 172.21.15.133:0/2278154888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee84101270 0x7fee8410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.767+0000 7fee8888d640 1 --2- 172.21.15.133:0/2278154888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee841008d0 0x7fee84100d30 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.767+0000 7fee8888d640 1 --2- 172.21.15.133:0/2278154888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee841068d0 0x7fee84106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.767+0000 7fee8888d640 1 -- 172.21.15.133:0/2278154888 >> 172.21.15.133:0/2278154888 conn(0x7fee840fc610 msgr2=0x7fee840fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.768+0000 7fee8888d640 1 -- 172.21.15.133:0/2278154888 shutdown_connections 2024-09-11T23:41:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.768+0000 7fee8888d640 1 -- 172.21.15.133:0/2278154888 wait complete. 2024-09-11T23:41:44.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.769+0000 7fee8888d640 1 Processor -- start 2024-09-11T23:41:44.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.769+0000 7fee8888d640 1 -- start start 2024-09-11T23:41:44.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.770+0000 7fee8888d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee84101270 0x7fee84196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.770+0000 7fee8888d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee841068d0 0x7fee841974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.770+0000 7fee8888d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee8419b870 0x7fee8419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.770+0000 7fee8888d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fee84110dc0 con 0x7fee8419b870 2024-09-11T23:41:44.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.771+0000 7fee8888d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee84110c40 con 0x7fee841068d0 2024-09-11T23:41:44.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.771+0000 7fee8888d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee84110f40 con 0x7fee84101270 2024-09-11T23:41:44.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.772+0000 7fee837fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee84101270 0x7fee84196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.772+0000 7fee837fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee84101270 0x7fee84196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40550/0 (socket says 172.21.15.133:40550) 2024-09-11T23:41:44.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.772+0000 7fee837fe640 1 -- 172.21.15.133:0/1864429383 learned_addr learned my addr 172.21.15.133:0/1864429383 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:44.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.772+0000 7fee7bfff640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee841068d0 0x7fee841974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.773+0000 7fee837fe640 1 -- 172.21.15.133:0/1864429383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee841068d0 msgr2=0x7fee841974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:44.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.773+0000 7fee837fe640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee841068d0 0x7fee841974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.773+0000 7fee837fe640 1 -- 172.21.15.133:0/1864429383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee8419b870 msgr2=0x7fee8419bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:44.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.773+0000 7fee837fe640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee8419b870 0x7fee8419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.773+0000 7fee837fe640 1 -- 172.21.15.133:0/1864429383 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8419c3c0 con 0x7fee84101270 2024-09-11T23:41:44.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.774+0000 7fee837fe640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee84101270 0x7fee84196fa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee7400b4a0 tx=0x7fee7400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:44.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.775+0000 7fee817fa640 1 -- 172.21.15.133:0/1864429383 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74014070 con 0x7fee84101270 2024-09-11T23:41:44.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.776+0000 7fee8888d640 1 -- 172.21.15.133:0/1864429383 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8419c6b0 con 0x7fee84101270 2024-09-11T23:41:44.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.776+0000 7fee8888d640 1 -- 172.21.15.133:0/1864429383 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee841a3e60 con 0x7fee84101270 2024-09-11T23:41:44.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.782+0000 7fee8bec8740 1 Processor -- start 2024-09-11T23:41:44.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.782+0000 7fee817fa640 1 -- 172.21.15.133:0/1864429383 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee74010af0 con 0x7fee84101270 2024-09-11T23:41:44.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.782+0000 7fee817fa640 1 -- 172.21.15.133:0/1864429383 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74017d40 con 0x7fee84101270 2024-09-11T23:41:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.788+0000 7fee817fa640 1 -- 172.21.15.133:0/1864429383 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fee74017070 con 0x7fee84101270 2024-09-11T23:41:44.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.791+0000 7fee817fa640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fee700a2f00 0x7fee700a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.792+0000 7fee817fa640 1 -- 172.21.15.133:0/1864429383 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fee740cf680 con 0x7fee84101270 2024-09-11T23:41:44.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.801+0000 7fee8bec8740 1 -- start start 2024-09-11T23:41:44.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.802+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 0x563c79547270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.802+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c795477b0 0x563c79547bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.802+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c79552550 0x563c79554940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.802+0000 7fee8bec8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563c794ad2f0 con 0x563c795477b0 2024-09-11T23:41:44.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.802+0000 7fee8bec8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c79454cf0 con 0x563c79552550 2024-09-11T23:41:44.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.802+0000 7fee8bec8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563c794a92b0 con 0x563c79548d20 2024-09-11T23:41:44.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.806+0000 7fee837fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 0x563c79547270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.806+0000 7fee837fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 0x563c79547270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40564/0 (socket says 172.21.15.133:40564) 2024-09-11T23:41:44.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.806+0000 7fee837fe640 1 -- 172.21.15.133:0/2952936661 learned_addr learned my addr 172.21.15.133:0/2952936661 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:44.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.807+0000 7fee7bfff640 1 --2- 172.21.15.133:0/2952936661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c795477b0 0x563c79547bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.807+0000 7fee83fff640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fee700a2f00 0x7fee700a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.809+0000 7fee83fff640 1 --2- 172.21.15.133:0/2952936661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c79552550 0x563c79554940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.810+0000 7fee837fe640 1 -- 172.21.15.133:0/2952936661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c79552550 msgr2=0x563c79554940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:44.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.810+0000 7fee837fe640 1 --2- 172.21.15.133:0/2952936661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c79552550 0x563c79554940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.810+0000 7fee837fe640 1 -- 172.21.15.133:0/2952936661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c795477b0 msgr2=0x563c79547bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:44.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.810+0000 7fee837fe640 1 --2- 172.21.15.133:0/2952936661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c795477b0 0x563c79547bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.810+0000 7fee837fe640 1 -- 172.21.15.133:0/2952936661 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c794393a0 con 0x563c79548d20 2024-09-11T23:41:44.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.813+0000 7fee837fe640 1 --2- 172.21.15.133:0/2952936661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 0x563c79547270 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fee74017870 tx=0x7fee7409b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69ba6bfcf52762d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:44.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.814+0000 7fee79ffb640 1 -- 172.21.15.133:0/2952936661 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7401e040 con 0x563c79548d20 2024-09-11T23:41:44.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.815+0000 7fee79ffb640 1 -- 172.21.15.133:0/2952936661 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee740b9070 con 0x563c79548d20 2024-09-11T23:41:44.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.816+0000 7fee8bec8740 1 -- 172.21.15.133:0/2952936661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 msgr2=0x563c79547270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:44.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.816+0000 7fee8bec8740 1 --2- 172.21.15.133:0/2952936661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 0x563c79547270 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fee74017870 tx=0x7fee7409b710 comp rx=0 tx=0).stop 2024-09-11T23:41:44.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.817+0000 7fee8bec8740 1 -- 172.21.15.133:0/2952936661 shutdown_connections 2024-09-11T23:41:44.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.817+0000 7fee8bec8740 1 --2- 172.21.15.133:0/2952936661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c79552550 0x563c79554940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.817+0000 7fee8bec8740 1 --2- 172.21.15.133:0/2952936661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c795477b0 0x563c79547bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.817+0000 7fee8bec8740 1 --2- 172.21.15.133:0/2952936661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 0x563c79547270 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.817+0000 7fee8bec8740 1 -- 172.21.15.133:0/2952936661 >> 172.21.15.133:0/2952936661 conn(0x563c79510310 msgr2=0x563c79543e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:44.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.818+0000 7fee8bec8740 1 -- 172.21.15.133:0/2952936661 shutdown_connections 2024-09-11T23:41:44.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.818+0000 7fee8bec8740 1 -- 172.21.15.133:0/2952936661 wait complete. 2024-09-11T23:41:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.818+0000 7fee8bec8740 1 -- start start 2024-09-11T23:41:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.819+0000 7fee8bec8740 1 Processor -- start 2024-09-11T23:41:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.819+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 0x563c79551e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.819+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 0x563c7954f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.820+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79552550 0x563c7954f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.820+0000 7fee8bec8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563c79555290 con 0x563c79548d20 2024-09-11T23:41:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.820+0000 7fee8bec8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c79468060 con 0x563c795477b0 2024-09-11T23:41:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.820+0000 7fee8bec8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563c79555410 con 0x563c79552550 2024-09-11T23:41:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.820+0000 7fee8bec8740 10 client.0 _ll_register_callbacks cb 0x563c791bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:44.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.820+0000 7fee7bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 0x563c7954f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.821+0000 7fee7bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 0x563c7954f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37386/0 (socket says 172.21.15.133:37386) 2024-09-11T23:41:44.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.821+0000 7fee7bfff640 1 -- 172.21.15.133:0/1862283547 learned_addr learned my addr 172.21.15.133:0/1862283547 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:44.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.821+0000 7fee837fe640 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 0x563c79551e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.821+0000 7fee83fff640 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79552550 0x563c7954f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.822+0000 7fee7bfff640 1 -- 172.21.15.133:0/1862283547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79552550 msgr2=0x563c7954f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:44.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.822+0000 7fee7bfff640 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79552550 0x563c7954f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.822+0000 7fee7bfff640 1 -- 172.21.15.133:0/1862283547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 msgr2=0x563c79551e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:44.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.822+0000 7fee7bfff640 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 0x563c79551e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.822+0000 7fee7bfff640 1 -- 172.21.15.133:0/1862283547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7921e3c0 con 0x563c79548d20 2024-09-11T23:41:44.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.822+0000 7fee83fff640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fee700a2f00 0x7fee700a53c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x563c79548250 tx=0x7fee7c00a9f0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:44.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.823+0000 7fee7bfff640 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 0x563c7954f0c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fee6c034040 tx=0x7fee6c004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae3a2b0420ebc59 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:44.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.823+0000 7fee62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:44.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.824+0000 7fee62ffd640 1 -- 172.21.15.133:0/1862283547 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c046060 con 0x563c79548d20 2024-09-11T23:41:44.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.824+0000 7fee8bec8740 1 -- 172.21.15.133:0/1862283547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee740b9070 con 0x563c79548d20 2024-09-11T23:41:44.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.825+0000 7fee62ffd640 1 -- 172.21.15.133:0/1862283547 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fee6c031810 con 0x563c79548d20 2024-09-11T23:41:44.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.825+0000 7fee62ffd640 1 -- 172.21.15.133:0/1862283547 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee6c035040 con 0x563c79548d20 2024-09-11T23:41:44.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.826+0000 7fee8bec8740 10 client.16788 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:44.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.826+0000 7fee8bec8740 1 -- 172.21.15.133:0/1862283547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c79569530 con 0x563c79548d20 2024-09-11T23:41:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.827+0000 7fee62ffd640 1 -- 172.21.15.133:0/1862283547 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee740b9070 con 0x563c79548d20 2024-09-11T23:41:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.828+0000 7fee62ffd640 1 -- 172.21.15.133:0/1862283547 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fee6c049020 con 0x563c79548d20 2024-09-11T23:41:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.829+0000 7fee8bec8740 10 client.16788 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.829+0000 7fee8bec8740 10 client.16788 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:41:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.829+0000 7fee8bec8740 20 client.16788 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.830+0000 7fee8bec8740 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563c7956adb0 0x563c7956d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.830+0000 7fee8bec8740 4 client.16788 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.830+0000 7fee8bec8740 1 -- 172.21.15.133:0/1862283547 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c794ba860 con 0x563c7956adb0 2024-09-11T23:41:44.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.834+0000 7fee837fe640 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563c7956adb0 0x563c7956d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.842+0000 7fee62ffd640 10 client.16788 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:41:44.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.842+0000 7fee837fe640 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563c7956adb0 0x563c7956d1f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dc344a8e694a39f server_cookie=4e2490a3cc971229 in_seq=0 out_seq=0 2024-09-11T23:41:44.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.843+0000 7fee62ffd640 1 -- 172.21.15.133:0/1862283547 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c794ba860 con 0x563c7956adb0 2024-09-11T23:41:44.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.843+0000 7fee62ffd640 10 client.16788 handle_command_reply: tid=0 2024-09-11T23:41:44.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.844+0000 7fee8bec8740 1 client.16788 shutdown 2024-09-11T23:41:44.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.844+0000 7fee8bec8740 1 -- 172.21.15.133:0/1862283547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 msgr2=0x563c7954f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:44.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.844+0000 7fee8bec8740 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 0x563c7954f0c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fee6c034040 tx=0x7fee6c004390 comp rx=0 tx=0).stop 2024-09-11T23:41:44.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.844+0000 7fee8bec8740 1 -- 172.21.15.133:0/1862283547 shutdown_connections 2024-09-11T23:41:44.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.844+0000 7fee8bec8740 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563c7956adb0 0x563c7956d1f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.844+0000 7fee8bec8740 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79552550 0x563c7954f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.844+0000 7fee8bec8740 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 0x563c7954f0c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.844+0000 7fee8bec8740 1 --2- 172.21.15.133:0/1862283547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 0x563c79551e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.844+0000 7fee8bec8740 1 -- 172.21.15.133:0/1862283547 >> 172.21.15.133:0/1862283547 conn(0x563c79510310 msgr2=0x563c79549b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:44.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.845+0000 7fee62ffd640 0 client.16788 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:41:44.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.845+0000 7fee8bec8740 1 -- 172.21.15.133:0/1862283547 shutdown_connections 2024-09-11T23:41:44.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.845+0000 7fee8bec8740 1 -- 172.21.15.133:0/1862283547 wait complete. 2024-09-11T23:41:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.845+0000 7fee8bec8740 20 client.16788 trim_cache size 0 max 16384 2024-09-11T23:41:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.876+0000 7fee8bec8740 1 Processor -- start 2024-09-11T23:41:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.879+0000 7fee8bec8740 1 -- start start 2024-09-11T23:41:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.880+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 0x563c79551030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.880+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 0x563c79572b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.880+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79552550 0x563c79573040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.881+0000 7fee837fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 0x563c79551030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.881+0000 7fee837fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 0x563c79551030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:46366/0 (socket says 172.21.15.133:46366) 2024-09-11T23:41:44.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.881+0000 7fee837fe640 1 -- 172.21.15.133:0/3323856136 learned_addr learned my addr 172.21.15.133:0/3323856136 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:44.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.881+0000 7fee7bfff640 1 --2- 172.21.15.133:0/3323856136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 0x563c79572b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.881+0000 7fee8bec8740 1 -- 172.21.15.133:0/3323856136 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fee6400e690 con 0x563c79552550 2024-09-11T23:41:44.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.882+0000 7fee8bec8740 1 -- 172.21.15.133:0/3323856136 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee640101d0 con 0x563c795477b0 2024-09-11T23:41:44.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.882+0000 7fee8bec8740 1 -- 172.21.15.133:0/3323856136 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee6400ed60 con 0x563c79548d20 2024-09-11T23:41:44.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.883+0000 7fee83fff640 1 --2- 172.21.15.133:0/3323856136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79552550 0x563c79573040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.887+0000 7fee837fe640 1 -- 172.21.15.133:0/3323856136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 msgr2=0x563c79572b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:44.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.887+0000 7fee837fe640 1 --2- 172.21.15.133:0/3323856136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 0x563c79572b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.887+0000 7fee837fe640 1 -- 172.21.15.133:0/3323856136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79552550 msgr2=0x563c79573040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:44.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.887+0000 7fee837fe640 1 --2- 172.21.15.133:0/3323856136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79552550 0x563c79573040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.887+0000 7fee837fe640 1 -- 172.21.15.133:0/3323856136 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c79554cb0 con 0x563c795477b0 2024-09-11T23:41:44.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.890+0000 7fee837fe640 1 --2- 172.21.15.133:0/3323856136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 0x563c79551030 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee74098690 tx=0x7fee7409be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32026f1d3390abfa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:44.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.891+0000 7fee797fa640 1 -- 172.21.15.133:0/3323856136 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee740a9070 con 0x563c795477b0 2024-09-11T23:41:44.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.891+0000 7fee797fa640 1 -- 172.21.15.133:0/3323856136 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee740b9040 con 0x563c795477b0 2024-09-11T23:41:44.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.891+0000 7fee797fa640 1 -- 172.21.15.133:0/3323856136 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7401e040 con 0x563c795477b0 2024-09-11T23:41:44.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.892+0000 7fee8bec8740 1 -- 172.21.15.133:0/3323856136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 msgr2=0x563c79551030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:44.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.892+0000 7fee8bec8740 1 --2- 172.21.15.133:0/3323856136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 0x563c79551030 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee74098690 tx=0x7fee7409be40 comp rx=0 tx=0).stop 2024-09-11T23:41:44.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.892+0000 7fee8bec8740 1 -- 172.21.15.133:0/3323856136 shutdown_connections 2024-09-11T23:41:44.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.892+0000 7fee8bec8740 1 --2- 172.21.15.133:0/3323856136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79552550 0x563c79573040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.892+0000 7fee8bec8740 1 --2- 172.21.15.133:0/3323856136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c79548d20 0x563c79572b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.892+0000 7fee8bec8740 1 --2- 172.21.15.133:0/3323856136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c795477b0 0x563c79551030 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.892+0000 7fee8bec8740 1 -- 172.21.15.133:0/3323856136 >> 172.21.15.133:0/3323856136 conn(0x563c79510310 msgr2=0x563c79550920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:44.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.893+0000 7fee8bec8740 1 -- 172.21.15.133:0/3323856136 shutdown_connections 2024-09-11T23:41:44.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.893+0000 7fee8bec8740 1 -- 172.21.15.133:0/3323856136 wait complete. 2024-09-11T23:41:44.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.894+0000 7fee8bec8740 1 -- start start 2024-09-11T23:41:44.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.895+0000 7fee8bec8740 1 Processor -- start 2024-09-11T23:41:44.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.896+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c795477b0 0x563c795632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.896+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 0x563c79563800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.896+0000 7fee8bec8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c79552550 0x563c7955e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.896+0000 7fee8bec8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fee6400fb00 con 0x563c79548d20 2024-09-11T23:41:44.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.896+0000 7fee8bec8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee6400f430 con 0x563c79552550 2024-09-11T23:41:44.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.896+0000 7fee8bec8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee6400c670 con 0x563c795477b0 2024-09-11T23:41:44.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.896+0000 7fee8bec8740 10 client.0 _ll_register_callbacks cb 0x7fee6c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:44.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.899+0000 7fee837fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c795477b0 0x563c795632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.899+0000 7fee837fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c795477b0 0x563c795632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40592/0 (socket says 172.21.15.133:40592) 2024-09-11T23:41:44.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.899+0000 7fee837fe640 1 -- 172.21.15.133:0/3734475314 learned_addr learned my addr 172.21.15.133:0/3734475314 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:44.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.899+0000 7fee83fff640 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c79552550 0x563c7955e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.900+0000 7fee7bfff640 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 0x563c79563800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.902+0000 7fee837fe640 1 -- 172.21.15.133:0/3734475314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c79552550 msgr2=0x563c7955e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:44.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.902+0000 7fee837fe640 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c79552550 0x563c7955e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.902+0000 7fee837fe640 1 -- 172.21.15.133:0/3734475314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 msgr2=0x563c79563800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.902+0000 7fee837fe640 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 0x563c79563800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.902+0000 7fee837fe640 1 -- 172.21.15.133:0/3734475314 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7955eeb0 con 0x563c795477b0 2024-09-11T23:41:44.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.907+0000 7fee837fe640 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c795477b0 0x563c795632c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fee7409b9c0 tx=0x7fee7409b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d01b6ca8308ac684 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:44.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.907+0000 7fee62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:41:44.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.908+0000 7fee62ffd640 1 -- 172.21.15.133:0/3734475314 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee740a9050 con 0x563c795477b0 2024-09-11T23:41:44.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.908+0000 7fee62ffd640 1 -- 172.21.15.133:0/3734475314 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fee74099fc0 con 0x563c795477b0 2024-09-11T23:41:44.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.908+0000 7fee8bec8740 1 -- 172.21.15.133:0/3734475314 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee740b9040 con 0x563c795477b0 2024-09-11T23:41:44.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.909+0000 7fee62ffd640 1 -- 172.21.15.133:0/3734475314 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee7401e070 con 0x563c795477b0 2024-09-11T23:41:44.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.909+0000 7fee8bec8740 10 client.26221 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:44.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.909+0000 7fee8bec8740 1 -- 172.21.15.133:0/3734475314 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c79556320 con 0x563c795477b0 2024-09-11T23:41:44.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.911+0000 7fee62ffd640 1 -- 172.21.15.133:0/3734475314 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee740cd260 con 0x563c795477b0 2024-09-11T23:41:44.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.911+0000 7fee62ffd640 1 -- 172.21.15.133:0/3734475314 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fee7409b2b0 con 0x563c795477b0 2024-09-11T23:41:44.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.911+0000 7fee8bec8740 10 client.26221 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:44.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.911+0000 7fee8bec8740 10 client.26221 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:41:44.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.911+0000 7fee8bec8740 20 client.26221 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:44.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.911+0000 7fee8bec8740 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563c79556600 0x563c79558a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:44.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.911+0000 7fee8bec8740 4 client.26221 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:41:44.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.911+0000 7fee8bec8740 1 -- 172.21.15.133:0/3734475314 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fee740b9040 con 0x563c79556600 2024-09-11T23:41:44.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.912+0000 7fee7bfff640 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563c79556600 0x563c79558a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:44.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.913+0000 7fee62ffd640 10 client.26221 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:41:44.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.914+0000 7fee7bfff640 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563c79556600 0x563c79558a20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8478c60c1843b6e5 server_cookie=efe3b2f7dc6a6be4 in_seq=0 out_seq=0 2024-09-11T23:41:44.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.917+0000 7fee62ffd640 1 -- 172.21.15.133:0/3734475314 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fee740b9040 con 0x563c79556600 2024-09-11T23:41:44.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.917+0000 7fee62ffd640 10 client.26221 handle_command_reply: tid=0 2024-09-11T23:41:44.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.917+0000 7fee8bec8740 1 client.26221 shutdown 2024-09-11T23:41:44.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.919+0000 7fee8bec8740 1 -- 172.21.15.133:0/3734475314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c795477b0 msgr2=0x563c795632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:44.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.919+0000 7fee8bec8740 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c795477b0 0x563c795632c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fee7409b9c0 tx=0x7fee7409b710 comp rx=0 tx=0).stop 2024-09-11T23:41:44.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.919+0000 7fee8bec8740 1 -- 172.21.15.133:0/3734475314 shutdown_connections 2024-09-11T23:41:44.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.919+0000 7fee8bec8740 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x563c79556600 0x563c79558a20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.920+0000 7fee8bec8740 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c79552550 0x563c7955e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.920+0000 7fee8bec8740 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c79548d20 0x563c79563800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.920+0000 7fee8bec8740 1 --2- 172.21.15.133:0/3734475314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c795477b0 0x563c795632c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.920+0000 7fee8bec8740 1 -- 172.21.15.133:0/3734475314 >> 172.21.15.133:0/3734475314 conn(0x563c79510310 msgr2=0x563c79542780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:44.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.920+0000 7fee62ffd640 0 client.26221 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:41:44.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.923+0000 7fee8bec8740 1 -- 172.21.15.133:0/3734475314 shutdown_connections 2024-09-11T23:41:44.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.923+0000 7fee8bec8740 1 -- 172.21.15.133:0/3734475314 wait complete. 2024-09-11T23:41:44.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.923+0000 7fee8bec8740 20 client.26221 trim_cache size 0 max 16384 2024-09-11T23:41:44.923 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:41:44.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.924+0000 7fee8888d640 1 -- 172.21.15.133:0/1864429383 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fee700a2f00 msgr2=0x7fee700a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:44.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.924+0000 7fee8888d640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fee700a2f00 0x7fee700a53c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x563c79548250 tx=0x7fee7c00a9f0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.924+0000 7fee8888d640 1 -- 172.21.15.133:0/1864429383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee84101270 msgr2=0x7fee84196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:44.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.924+0000 7fee8888d640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee84101270 0x7fee84196fa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee7400b4a0 tx=0x7fee7400b960 comp rx=0 tx=0).stop 2024-09-11T23:41:44.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.925+0000 7fee8888d640 1 -- 172.21.15.133:0/1864429383 shutdown_connections 2024-09-11T23:41:44.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.925+0000 7fee8888d640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fee700a2f00 0x7fee700a53c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.925+0000 7fee8888d640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee8419b870 0x7fee8419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.925+0000 7fee8888d640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee841068d0 0x7fee841974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.925+0000 7fee8888d640 1 --2- 172.21.15.133:0/1864429383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee84101270 0x7fee84196fa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:44.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.925+0000 7fee8888d640 1 -- 172.21.15.133:0/1864429383 >> 172.21.15.133:0/1864429383 conn(0x7fee840fc610 msgr2=0x7fee840fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:44.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.925+0000 7fee8888d640 1 -- 172.21.15.133:0/1864429383 shutdown_connections 2024-09-11T23:41:44.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:44.925+0000 7fee8888d640 1 -- 172.21.15.133:0/1864429383 wait complete. 2024-09-11T23:41:45.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:45 smithi138 ceph-mon[35142]: pgmap v152: 97 pgs: 97 active+clean; 36 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 436 MiB/s wr, 2.28k op/s 2024-09-11T23:41:45.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:45 smithi133 ceph-mon[31430]: pgmap v152: 97 pgs: 97 active+clean; 36 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 436 MiB/s wr, 2.28k op/s 2024-09-11T23:41:45.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:45 smithi178 ceph-mon[35882]: pgmap v152: 97 pgs: 97 active+clean; 36 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 436 MiB/s wr, 2.28k op/s 2024-09-11T23:41:46.214 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:41:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.101+0000 7fc71cdef640 1 -- 172.21.15.133:0/4076509709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc71811b140 msgr2=0x7fc71811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.101+0000 7fc71cdef640 1 --2- 172.21.15.133:0/4076509709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc71811b140 0x7fc71811d530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc710009ca0 tx=0x7fc71002a8f0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.108+0000 7fc71cdef640 1 -- 172.21.15.133:0/4076509709 shutdown_connections 2024-09-11T23:41:48.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.108+0000 7fc71cdef640 1 --2- 172.21.15.133:0/4076509709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc71811b140 0x7fc71811d530 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.108+0000 7fc71cdef640 1 --2- 172.21.15.133:0/4076509709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc718109830 0x7fc718109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.108+0000 7fc71cdef640 1 --2- 172.21.15.133:0/4076509709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc718109450 0x7fc718110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.108+0000 7fc71cdef640 1 -- 172.21.15.133:0/4076509709 >> 172.21.15.133:0/4076509709 conn(0x7fc71806e140 msgr2=0x7fc71806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:48.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.109+0000 7fc71cdef640 1 -- 172.21.15.133:0/4076509709 shutdown_connections 2024-09-11T23:41:48.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.109+0000 7fc71cdef640 1 -- 172.21.15.133:0/4076509709 wait complete. 2024-09-11T23:41:48.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.111+0000 7fc71cdef640 1 Processor -- start 2024-09-11T23:41:48.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.115+0000 7fc71cdef640 1 -- start start 2024-09-11T23:41:48.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.116+0000 7fc71cdef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc718109450 0x7fc71811a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.116+0000 7fc71cdef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc718109830 0x7fc71811ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.116+0000 7fc71cdef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc71811b140 0x7fc7181138a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.116+0000 7fc71cdef640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc718120da0 con 0x7fc718109450 2024-09-11T23:41:48.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.116+0000 7fc71cdef640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc718120c20 con 0x7fc718109830 2024-09-11T23:41:48.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.116+0000 7fc71cdef640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc718120f20 con 0x7fc71811b140 2024-09-11T23:41:48.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.117+0000 7fc716d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc71811b140 0x7fc7181138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.117+0000 7fc71659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc718109450 0x7fc71811a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.117+0000 7fc71659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc718109450 0x7fc71811a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37420/0 (socket says 172.21.15.133:37420) 2024-09-11T23:41:48.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.117+0000 7fc71659c640 1 -- 172.21.15.133:0/2169905502 learned_addr learned my addr 172.21.15.133:0/2169905502 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:48.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.118+0000 7fc715d9b640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc718109830 0x7fc71811ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.118+0000 7fc71659c640 1 -- 172.21.15.133:0/2169905502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc71811b140 msgr2=0x7fc7181138a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:48.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.118+0000 7fc71659c640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc71811b140 0x7fc7181138a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.118+0000 7fc71659c640 1 -- 172.21.15.133:0/2169905502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc718109830 msgr2=0x7fc71811ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:48.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.118+0000 7fc71659c640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc718109830 0x7fc71811ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.118+0000 7fc71659c640 1 -- 172.21.15.133:0/2169905502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7181140d0 con 0x7fc718109450 2024-09-11T23:41:48.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.119+0000 7fc71659c640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc718109450 0x7fc71811a630 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c890 tx=0x7fc70800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:48.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.122+0000 7fc7077fe640 1 -- 172.21.15.133:0/2169905502 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708004220 con 0x7fc718109450 2024-09-11T23:41:48.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.122+0000 7fc7077fe640 1 -- 172.21.15.133:0/2169905502 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7080043c0 con 0x7fc718109450 2024-09-11T23:41:48.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.122+0000 7fc7077fe640 1 -- 172.21.15.133:0/2169905502 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7080049c0 con 0x7fc718109450 2024-09-11T23:41:48.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.122+0000 7fc71cdef640 1 -- 172.21.15.133:0/2169905502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7181143c0 con 0x7fc718109450 2024-09-11T23:41:48.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.122+0000 7fc71cdef640 1 -- 172.21.15.133:0/2169905502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc718118390 con 0x7fc718109450 2024-09-11T23:41:48.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.130+0000 7fc71f1c8740 1 Processor -- start 2024-09-11T23:41:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.130+0000 7fc7077fe640 1 -- 172.21.15.133:0/2169905502 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc708005020 con 0x7fc718109450 2024-09-11T23:41:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.133+0000 7fc71f1c8740 1 -- start start 2024-09-11T23:41:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.133+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 0x562c67fb3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.133+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fb35d0 0x562c67fb3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.133+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fbe370 0x562c67fc0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.133+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562c67f192f0 con 0x562c67fbe370 2024-09-11T23:41:48.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.133+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562c67ec0cf0 con 0x562c67fb4b40 2024-09-11T23:41:48.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.133+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c67f152b0 con 0x562c67fb35d0 2024-09-11T23:41:48.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.134+0000 7fc716d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 0x562c67fb3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.134+0000 7fc716d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 0x562c67fb3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:46402/0 (socket says 172.21.15.133:46402) 2024-09-11T23:41:48.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.134+0000 7fc716d9d640 1 -- 172.21.15.133:0/1486513954 learned_addr learned my addr 172.21.15.133:0/1486513954 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:48.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.136+0000 7fc715d9b640 1 --2- 172.21.15.133:0/1486513954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fbe370 0x562c67fc0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.136+0000 7fc7077fe640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc7000a2f00 0x7fc7000a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.146+0000 7fc71659c640 1 --2- 172.21.15.133:0/1486513954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fb35d0 0x562c67fb3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.146+0000 7fc71659c640 1 -- 172.21.15.133:0/1486513954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 msgr2=0x562c67fb3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:48.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.146+0000 7fc71659c640 1 --2- 172.21.15.133:0/1486513954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 0x562c67fb3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.146+0000 7fc71659c640 1 -- 172.21.15.133:0/1486513954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fbe370 msgr2=0x562c67fc0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.146+0000 7fc71659c640 1 --2- 172.21.15.133:0/1486513954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fbe370 0x562c67fc0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.146+0000 7fc71659c640 1 -- 172.21.15.133:0/1486513954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c67ea53a0 con 0x562c67fb35d0 2024-09-11T23:41:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.147+0000 7fc71659c640 1 --2- 172.21.15.133:0/1486513954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fb35d0 0x562c67fb3a10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc7080ccb10 tx=0x7fc708097450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e643fc56d9caf2d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:48.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.147+0000 7fc704ff9640 1 -- 172.21.15.133:0/1486513954 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70809b420 con 0x562c67fb35d0 2024-09-11T23:41:48.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.147+0000 7fc704ff9640 1 -- 172.21.15.133:0/1486513954 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc70809bfe0 con 0x562c67fb35d0 2024-09-11T23:41:48.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.147+0000 7fc704ff9640 1 -- 172.21.15.133:0/1486513954 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70809b240 con 0x562c67fb35d0 2024-09-11T23:41:48.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.150+0000 7fc71f1c8740 1 -- 172.21.15.133:0/1486513954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fb35d0 msgr2=0x562c67fb3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:48.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.150+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/1486513954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fb35d0 0x562c67fb3a10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc7080ccb10 tx=0x7fc708097450 comp rx=0 tx=0).stop 2024-09-11T23:41:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.153+0000 7fc7077fe640 1 -- 172.21.15.133:0/2169905502 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc7080cf3b0 con 0x7fc718109450 2024-09-11T23:41:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.153+0000 7fc716d9d640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc7000a2f00 0x7fc7000a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.154+0000 7fc71f1c8740 1 -- 172.21.15.133:0/1486513954 shutdown_connections 2024-09-11T23:41:48.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.154+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/1486513954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fbe370 0x562c67fc0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.154+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/1486513954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fb35d0 0x562c67fb3a10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.154+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/1486513954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 0x562c67fb3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.154+0000 7fc71f1c8740 1 -- 172.21.15.133:0/1486513954 >> 172.21.15.133:0/1486513954 conn(0x562c67f7c310 msgr2=0x562c67fafc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:48.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.158+0000 7fc71f1c8740 1 -- 172.21.15.133:0/1486513954 shutdown_connections 2024-09-11T23:41:48.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.158+0000 7fc71f1c8740 1 -- 172.21.15.133:0/1486513954 wait complete. 2024-09-11T23:41:48.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.159+0000 7fc71f1c8740 1 -- start start 2024-09-11T23:41:48.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.159+0000 7fc71f1c8740 1 Processor -- start 2024-09-11T23:41:48.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.161+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb35d0 0x562c67fbdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.161+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 0x562c67fbb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.161+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fbb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.161+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562c67fc1010 con 0x562c67fb35d0 2024-09-11T23:41:48.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.161+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562c67ed4060 con 0x562c67fb4b40 2024-09-11T23:41:48.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.161+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c67fc1190 con 0x562c67fbe370 2024-09-11T23:41:48.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.161+0000 7fc71f1c8740 10 client.0 _ll_register_callbacks cb 0x562c67c278c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:48.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.164+0000 7fc715d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 0x562c67fbb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.164+0000 7fc71659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb35d0 0x562c67fbdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.164+0000 7fc71659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb35d0 0x562c67fbdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37432/0 (socket says 172.21.15.133:37432) 2024-09-11T23:41:48.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.164+0000 7fc71659c640 1 -- 172.21.15.133:0/3491702915 learned_addr learned my addr 172.21.15.133:0/3491702915 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:48.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.165+0000 7fc716d9d640 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fbb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.168+0000 7fc715d9b640 1 -- 172.21.15.133:0/3491702915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 msgr2=0x562c67fbb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:48.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.168+0000 7fc715d9b640 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fbb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.168+0000 7fc715d9b640 1 -- 172.21.15.133:0/3491702915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb35d0 msgr2=0x562c67fbdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:48.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.168+0000 7fc715d9b640 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb35d0 0x562c67fbdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.168+0000 7fc715d9b640 1 -- 172.21.15.133:0/3491702915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c67c8a3c0 con 0x562c67fb4b40 2024-09-11T23:41:48.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.168+0000 7fc716d9d640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc7000a2f00 0x7fc7000a53c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc71002f570 tx=0x7fc71003e040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:48.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.169+0000 7fc71659c640 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb35d0 0x562c67fbdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:48.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.176+0000 7fc715d9b640 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 0x562c67fbb010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc718114e80 tx=0x7fc70c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a07247ff5be2fe7a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:48.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.176+0000 7fc6fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:41:48.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.179+0000 7fc6fdffb640 1 -- 172.21.15.133:0/3491702915 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c004030 con 0x562c67fb4b40 2024-09-11T23:41:48.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.179+0000 7fc6fdffb640 1 -- 172.21.15.133:0/3491702915 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc70c0041d0 con 0x562c67fb4b40 2024-09-11T23:41:48.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.181+0000 7fc71f1c8740 1 -- 172.21.15.133:0/3491702915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70809bfe0 con 0x562c67fb4b40 2024-09-11T23:41:48.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.184+0000 7fc6fdffb640 1 -- 172.21.15.133:0/3491702915 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc70c0043d0 con 0x562c67fb4b40 2024-09-11T23:41:48.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.184+0000 7fc71f1c8740 10 client.26252 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:48.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.184+0000 7fc71f1c8740 1 -- 172.21.15.133:0/3491702915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c67fccb50 con 0x562c67fb4b40 2024-09-11T23:41:48.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.187+0000 7fc6fdffb640 1 -- 172.21.15.133:0/3491702915 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc70809bfe0 con 0x562c67fb4b40 2024-09-11T23:41:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.188+0000 7fc6fdffb640 1 -- 172.21.15.133:0/3491702915 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc70c003c00 con 0x562c67fb4b40 2024-09-11T23:41:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.189+0000 7fc71f1c8740 10 client.26252 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.189+0000 7fc71f1c8740 10 client.26252 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:41:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.189+0000 7fc71f1c8740 20 client.26252 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.189+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c67fce330 0x562c67fd0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.189+0000 7fc71f1c8740 4 client.26252 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.189+0000 7fc71f1c8740 1 -- 172.21.15.133:0/3491702915 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c67f26860 con 0x562c67fce330 2024-09-11T23:41:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.192+0000 7fc71659c640 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c67fce330 0x562c67fd0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.204+0000 7fc71659c640 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c67fce330 0x562c67fd0760 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3696b3d665977acc server_cookie=c2ea6db5cdeaef63 in_seq=0 out_seq=0 2024-09-11T23:41:48.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.205+0000 7fc6fdffb640 10 client.26252 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:41:48.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.205+0000 7fc6fdffb640 1 -- 172.21.15.133:0/3491702915 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c67f26860 con 0x562c67fce330 2024-09-11T23:41:48.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.205+0000 7fc6fdffb640 10 client.26252 handle_command_reply: tid=0 2024-09-11T23:41:48.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.205+0000 7fc71f1c8740 1 client.26252 shutdown 2024-09-11T23:41:48.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.207+0000 7fc71f1c8740 1 -- 172.21.15.133:0/3491702915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 msgr2=0x562c67fbb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:48.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.207+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 0x562c67fbb010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc718114e80 tx=0x7fc70c012b60 comp rx=0 tx=0).stop 2024-09-11T23:41:48.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.209+0000 7fc71f1c8740 1 -- 172.21.15.133:0/3491702915 shutdown_connections 2024-09-11T23:41:48.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.209+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c67fce330 0x562c67fd0760 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.209+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fbb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.209+0000 7fc6fdffb640 0 client.26252 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:41:48.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.209+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb4b40 0x562c67fbb010 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.209+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/3491702915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb35d0 0x562c67fbdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.209+0000 7fc71f1c8740 1 -- 172.21.15.133:0/3491702915 >> 172.21.15.133:0/3491702915 conn(0x562c67f7c310 msgr2=0x562c67fb5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:48.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.209+0000 7fc71f1c8740 1 -- 172.21.15.133:0/3491702915 shutdown_connections 2024-09-11T23:41:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.209+0000 7fc71f1c8740 1 -- 172.21.15.133:0/3491702915 wait complete. 2024-09-11T23:41:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.211+0000 7fc71f1c8740 20 client.26252 trim_cache size 0 max 16384 2024-09-11T23:41:48.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.231+0000 7fc71f1c8740 1 Processor -- start 2024-09-11T23:41:48.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.231+0000 7fc71f1c8740 1 -- start start 2024-09-11T23:41:48.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.231+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 0x562c67fbaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.231+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 0x562c67faebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.231+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fb6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.231+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181c4cf0 con 0x562c67fb4b40 2024-09-11T23:41:48.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.231+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181c6830 con 0x562c67fb35d0 2024-09-11T23:41:48.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.231+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181c53c0 con 0x562c67fbe370 2024-09-11T23:41:48.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc715d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 0x562c67faebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc715d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 0x562c67faebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37442/0 (socket says 172.21.15.133:37442) 2024-09-11T23:41:48.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc715d9b640 1 -- 172.21.15.133:0/1115482922 learned_addr learned my addr 172.21.15.133:0/1115482922 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:48.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc71659c640 1 --2- 172.21.15.133:0/1115482922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 0x562c67fbaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc715d9b640 1 -- 172.21.15.133:0/1115482922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 msgr2=0x562c67fb6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:48.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc715d9b640 1 --2- 172.21.15.133:0/1115482922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fb6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc715d9b640 1 -- 172.21.15.133:0/1115482922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 msgr2=0x562c67fbaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:48.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc715d9b640 1 --2- 172.21.15.133:0/1115482922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 0x562c67fbaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc715d9b640 1 -- 172.21.15.133:0/1115482922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c67fbb990 con 0x562c67fb4b40 2024-09-11T23:41:48.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc715d9b640 1 --2- 172.21.15.133:0/1115482922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 0x562c67faebf0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc70c010430 tx=0x7fc70c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ce67aec4b9e019 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:48.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc6fffff640 1 -- 172.21.15.133:0/1115482922 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c010620 con 0x562c67fb4b40 2024-09-11T23:41:48.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc6fffff640 1 -- 172.21.15.133:0/1115482922 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc70c0107c0 con 0x562c67fb4b40 2024-09-11T23:41:48.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.232+0000 7fc6fffff640 1 -- 172.21.15.133:0/1115482922 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00fe80 con 0x562c67fb4b40 2024-09-11T23:41:48.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.233+0000 7fc71f1c8740 1 -- 172.21.15.133:0/1115482922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 msgr2=0x562c67faebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:48.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.233+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/1115482922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 0x562c67faebf0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc70c010430 tx=0x7fc70c009bf0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.233+0000 7fc71f1c8740 1 -- 172.21.15.133:0/1115482922 shutdown_connections 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.233+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/1115482922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fb6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.233+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/1115482922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 0x562c67faebf0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.233+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/1115482922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 0x562c67fbaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.234+0000 7fc71f1c8740 1 -- 172.21.15.133:0/1115482922 >> 172.21.15.133:0/1115482922 conn(0x562c67f7c310 msgr2=0x562c67fae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.234+0000 7fc71f1c8740 1 -- 172.21.15.133:0/1115482922 shutdown_connections 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.234+0000 7fc71f1c8740 1 -- 172.21.15.133:0/1115482922 wait complete. 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.234+0000 7fc71f1c8740 1 -- start start 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.234+0000 7fc71f1c8740 1 Processor -- start 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 0x562c67fc6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 0x562c67fc6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71f1c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fb7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181c6160 con 0x562c67fb4b40 2024-09-11T23:41:48.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181c5a90 con 0x562c67fb35d0 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71f1c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181c2d80 con 0x562c67fbe370 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71f1c8740 10 client.0 _ll_register_callbacks cb 0x7fc70c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 0x562c67fc6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 0x562c67fc6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:46430/0 (socket says 172.21.15.133:46430) 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71659c640 1 -- 172.21.15.133:0/2739443792 learned_addr learned my addr 172.21.15.133:0/2739443792 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc715d9b640 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 0x562c67fc6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc716d9d640 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fb7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71659c640 1 -- 172.21.15.133:0/2739443792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 msgr2=0x562c67fb7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71659c640 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fb7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71659c640 1 -- 172.21.15.133:0/2739443792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 msgr2=0x562c67fc6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71659c640 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 0x562c67fc6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc71659c640 1 -- 172.21.15.133:0/2739443792 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c67fb82a0 con 0x562c67fb35d0 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.235+0000 7fc716d9d640 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fb7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:41:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc71659c640 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 0x562c67fc6700 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc7080d2960 tx=0x7fc7080bf0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9ece8fa79b75a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:48.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc6fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:41:48.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc6fdffb640 1 -- 172.21.15.133:0/2739443792 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70801d040 con 0x562c67fb35d0 2024-09-11T23:41:48.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc6fdffb640 1 -- 172.21.15.133:0/2739443792 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc708097f50 con 0x562c67fb35d0 2024-09-11T23:41:48.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc71f1c8740 1 -- 172.21.15.133:0/2739443792 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70c0107c0 con 0x562c67fb35d0 2024-09-11T23:41:48.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc6fdffb640 1 -- 172.21.15.133:0/2739443792 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc70809b240 con 0x562c67fb35d0 2024-09-11T23:41:48.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc71f1c8740 10 client.26258 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:48.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc71f1c8740 1 -- 172.21.15.133:0/2739443792 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c68038c40 con 0x562c67fb35d0 2024-09-11T23:41:48.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc6fdffb640 1 -- 172.21.15.133:0/2739443792 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc70809bfe0 con 0x562c67fb35d0 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc6fdffb640 1 -- 172.21.15.133:0/2739443792 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc7080ccd90 con 0x562c67fb35d0 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc71f1c8740 10 client.26258 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc71f1c8740 10 client.26258 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.236+0000 7fc71f1c8740 20 client.26258 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.237+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c68038f30 0x562c6803b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.237+0000 7fc71f1c8740 4 client.26258 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.237+0000 7fc71f1c8740 1 -- 172.21.15.133:0/2739443792 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc70c0107c0 con 0x562c68038f30 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.237+0000 7fc715d9b640 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c68038f30 0x562c6803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.237+0000 7fc6fdffb640 10 client.26258 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.237+0000 7fc715d9b640 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c68038f30 0x562c6803b320 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=899d4472c17eb94c server_cookie=6e0955be9594fdae in_seq=0 out_seq=0 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc6fdffb640 1 -- 172.21.15.133:0/2739443792 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc70c0107c0 con 0x562c68038f30 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc6fdffb640 10 client.26258 handle_command_reply: tid=0 2024-09-11T23:41:48.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc71f1c8740 1 client.26258 shutdown 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc71f1c8740 1 -- 172.21.15.133:0/2739443792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 msgr2=0x562c67fc6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 0x562c67fc6700 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc7080d2960 tx=0x7fc7080bf0d0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc71f1c8740 1 -- 172.21.15.133:0/2739443792 shutdown_connections 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x562c68038f30 0x562c6803b320 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c67fbe370 0x562c67fb7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c67fb4b40 0x562c67fc6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc71f1c8740 1 --2- 172.21.15.133:0/2739443792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562c67fb35d0 0x562c67fc6700 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc71f1c8740 1 -- 172.21.15.133:0/2739443792 >> 172.21.15.133:0/2739443792 conn(0x562c67f7c310 msgr2=0x562c67faed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc6fdffb640 0 client.26258 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc71f1c8740 1 -- 172.21.15.133:0/2739443792 shutdown_connections 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.238+0000 7fc71f1c8740 1 -- 172.21.15.133:0/2739443792 wait complete. 2024-09-11T23:41:48.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.239+0000 7fc71f1c8740 20 client.26258 trim_cache size 0 max 16384 2024-09-11T23:41:48.242 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:41:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.239+0000 7fc71cdef640 1 -- 172.21.15.133:0/2169905502 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc7000a2f00 msgr2=0x7fc7000a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.239+0000 7fc71cdef640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc7000a2f00 0x7fc7000a53c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc71002f570 tx=0x7fc71003e040 comp rx=0 tx=0).stop 2024-09-11T23:41:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.240+0000 7fc71cdef640 1 -- 172.21.15.133:0/2169905502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc718109450 msgr2=0x7fc71811a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.240+0000 7fc71cdef640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc718109450 0x7fc71811a630 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c890 tx=0x7fc70800cd50 comp rx=0 tx=0).stop 2024-09-11T23:41:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.240+0000 7fc71cdef640 1 -- 172.21.15.133:0/2169905502 shutdown_connections 2024-09-11T23:41:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.240+0000 7fc71cdef640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc7000a2f00 0x7fc7000a53c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.240+0000 7fc71cdef640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc71811b140 0x7fc7181138a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.240+0000 7fc71cdef640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc718109830 0x7fc71811ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.240+0000 7fc71cdef640 1 --2- 172.21.15.133:0/2169905502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc718109450 0x7fc71811a630 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.240+0000 7fc71cdef640 1 -- 172.21.15.133:0/2169905502 >> 172.21.15.133:0/2169905502 conn(0x7fc71806e140 msgr2=0x7fc718072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.240+0000 7fc71cdef640 1 -- 172.21.15.133:0/2169905502 shutdown_connections 2024-09-11T23:41:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:48.240+0000 7fc71cdef640 1 -- 172.21.15.133:0/2169905502 wait complete. 2024-09-11T23:41:48.406 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:48 smithi133 ceph-mon[31430]: pgmap v153: 97 pgs: 97 active+clean; 36 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 413 MiB/s wr, 1.98k op/s 2024-09-11T23:41:48.701 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:48 smithi138 ceph-mon[35142]: pgmap v153: 97 pgs: 97 active+clean; 36 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 413 MiB/s wr, 1.98k op/s 2024-09-11T23:41:48.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:48 smithi178 ceph-mon[35882]: pgmap v153: 97 pgs: 97 active+clean; 36 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 413 MiB/s wr, 1.98k op/s 2024-09-11T23:41:49.550 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:41:49.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:49 smithi133 ceph-mon[31430]: pgmap v154: 97 pgs: 97 active+clean; 39 GiB data, 122 GiB used, 950 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 683 MiB/s wr, 2.29k op/s 2024-09-11T23:41:49.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:49 smithi138 ceph-mon[35142]: pgmap v154: 97 pgs: 97 active+clean; 39 GiB data, 122 GiB used, 950 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 683 MiB/s wr, 2.29k op/s 2024-09-11T23:41:49.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:49 smithi178 ceph-mon[35882]: pgmap v154: 97 pgs: 97 active+clean; 39 GiB data, 122 GiB used, 950 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 683 MiB/s wr, 2.29k op/s 2024-09-11T23:41:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:51.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.713+0000 7efd2e1dd640 1 -- 172.21.15.133:0/169427835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd28106a90 msgr2=0x7efd28106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:51.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.713+0000 7efd2e1dd640 1 --2- 172.21.15.133:0/169427835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd28106a90 0x7efd28106e70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7efd18009930 tx=0x7efd1802a440 comp rx=0 tx=0).stop 2024-09-11T23:41:51.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.714+0000 7efd2e1dd640 1 -- 172.21.15.133:0/169427835 shutdown_connections 2024-09-11T23:41:51.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.714+0000 7efd2e1dd640 1 --2- 172.21.15.133:0/169427835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd28101430 0x7efd2810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.714+0000 7efd2e1dd640 1 --2- 172.21.15.133:0/169427835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd28100a90 0x7efd28100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.714+0000 7efd2e1dd640 1 --2- 172.21.15.133:0/169427835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd28106a90 0x7efd28106e70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.714+0000 7efd2e1dd640 1 -- 172.21.15.133:0/169427835 >> 172.21.15.133:0/169427835 conn(0x7efd280fc7d0 msgr2=0x7efd280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:51.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.715+0000 7efd2e1dd640 1 -- 172.21.15.133:0/169427835 shutdown_connections 2024-09-11T23:41:51.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.715+0000 7efd2e1dd640 1 -- 172.21.15.133:0/169427835 wait complete. 2024-09-11T23:41:51.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.716+0000 7efd2e1dd640 1 Processor -- start 2024-09-11T23:41:51.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.716+0000 7efd2e1dd640 1 -- start start 2024-09-11T23:41:51.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.716+0000 7efd2e1dd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd28100a90 0x7efd28197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.716+0000 7efd2e1dd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd28101430 0x7efd28197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.716+0000 7efd2e1dd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd28106a90 0x7efd2819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.717+0000 7efd2e1dd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efd28110ef0 con 0x7efd28101430 2024-09-11T23:41:51.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.717+0000 7efd2e1dd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd28110d70 con 0x7efd28106a90 2024-09-11T23:41:51.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.717+0000 7efd2e1dd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd28111070 con 0x7efd28100a90 2024-09-11T23:41:51.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.717+0000 7efd26ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd28101430 0x7efd28197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.717+0000 7efd26ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd28101430 0x7efd28197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37472/0 (socket says 172.21.15.133:37472) 2024-09-11T23:41:51.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.717+0000 7efd26ffd640 1 -- 172.21.15.133:0/163112806 learned_addr learned my addr 172.21.15.133:0/163112806 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:51.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.717+0000 7efd26ffd640 1 -- 172.21.15.133:0/163112806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd28100a90 msgr2=0x7efd28197430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:41:51.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.717+0000 7efd26ffd640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd28100a90 0x7efd28197430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.717+0000 7efd26ffd640 1 -- 172.21.15.133:0/163112806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd28106a90 msgr2=0x7efd2819bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:51.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.717+0000 7efd26ffd640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd28106a90 0x7efd2819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.717+0000 7efd26ffd640 1 -- 172.21.15.133:0/163112806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2819c3e0 con 0x7efd28101430 2024-09-11T23:41:51.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.718+0000 7efd26ffd640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd28101430 0x7efd28197970 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009860 tx=0x7efd1c009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:51.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.718+0000 7efd24ff9640 1 -- 172.21.15.133:0/163112806 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c005020 con 0x7efd28101430 2024-09-11T23:41:51.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.718+0000 7efd2e1dd640 1 -- 172.21.15.133:0/163112806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2819c6d0 con 0x7efd28101430 2024-09-11T23:41:51.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.719+0000 7efd2e1dd640 1 -- 172.21.15.133:0/163112806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2806a790 con 0x7efd28101430 2024-09-11T23:41:51.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.719+0000 7efd24ff9640 1 -- 172.21.15.133:0/163112806 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd1c0047a0 con 0x7efd28101430 2024-09-11T23:41:51.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.719+0000 7efd24ff9640 1 -- 172.21.15.133:0/163112806 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c007a20 con 0x7efd28101430 2024-09-11T23:41:51.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.722+0000 7efd305b6740 1 Processor -- start 2024-09-11T23:41:51.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.722+0000 7efd305b6740 1 -- start start 2024-09-11T23:41:51.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.723+0000 7efd24ff9640 1 -- 172.21.15.133:0/163112806 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7efd1c004950 con 0x7efd28101430 2024-09-11T23:41:51.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.723+0000 7efd305b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0bab40 0x55736e0b9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.723+0000 7efd305b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0b95d0 0x55736e0b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.723+0000 7efd305b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.723+0000 7efd305b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55736e01f2f0 con 0x55736e0c4370 2024-09-11T23:41:51.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.723+0000 7efd305b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55736dfc6cf0 con 0x55736e0bab40 2024-09-11T23:41:51.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.723+0000 7efd305b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55736e01b2b0 con 0x55736e0b95d0 2024-09-11T23:41:51.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.723+0000 7efd26ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.723+0000 7efd26ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37478/0 (socket says 172.21.15.133:37478) 2024-09-11T23:41:51.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.723+0000 7efd26ffd640 1 -- 172.21.15.133:0/2763337649 learned_addr learned my addr 172.21.15.133:0/2763337649 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.723+0000 7efd27fff640 1 --2- 172.21.15.133:0/2763337649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0bab40 0x55736e0b9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.724+0000 7efd26ffd640 1 -- 172.21.15.133:0/2763337649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0b95d0 msgr2=0x55736e0b9a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:41:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.724+0000 7efd26ffd640 1 --2- 172.21.15.133:0/2763337649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0b95d0 0x55736e0b9a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.724+0000 7efd26ffd640 1 -- 172.21.15.133:0/2763337649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0bab40 msgr2=0x55736e0b9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.724+0000 7efd26ffd640 1 --2- 172.21.15.133:0/2763337649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0bab40 0x55736e0b9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.724+0000 7efd26ffd640 1 -- 172.21.15.133:0/2763337649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55736dfab3a0 con 0x55736e0c4370 2024-09-11T23:41:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.724+0000 7efd26ffd640 1 --2- 172.21.15.133:0/2763337649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c6760 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0ce030 tx=0x7efd1c0d6db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4babfcff622f3493 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.724+0000 7efd0dffb640 1 -- 172.21.15.133:0/2763337649 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c0d8d60 con 0x55736e0c4370 2024-09-11T23:41:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.725+0000 7efd24ff9640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efd100a2d70 0x7efd100a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.725+0000 7efd0dffb640 1 -- 172.21.15.133:0/2763337649 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd1c0dd070 con 0x55736e0c4370 2024-09-11T23:41:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.725+0000 7efd0dffb640 1 -- 172.21.15.133:0/2763337649 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c0d9890 con 0x55736e0c4370 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.725+0000 7efd24ff9640 1 -- 172.21.15.133:0/163112806 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7efd1c0d0900 con 0x7efd28101430 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.725+0000 7efd27fff640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efd100a2d70 0x7efd100a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.726+0000 7efd305b6740 1 -- 172.21.15.133:0/2763337649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 msgr2=0x55736e0c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.726+0000 7efd305b6740 1 --2- 172.21.15.133:0/2763337649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c6760 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0ce030 tx=0x7efd1c0d6db0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.726+0000 7efd305b6740 1 -- 172.21.15.133:0/2763337649 shutdown_connections 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.726+0000 7efd305b6740 1 --2- 172.21.15.133:0/2763337649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c6760 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.726+0000 7efd305b6740 1 --2- 172.21.15.133:0/2763337649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0b95d0 0x55736e0b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.726+0000 7efd305b6740 1 --2- 172.21.15.133:0/2763337649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0bab40 0x55736e0b9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.726+0000 7efd305b6740 1 -- 172.21.15.133:0/2763337649 >> 172.21.15.133:0/2763337649 conn(0x55736e082310 msgr2=0x55736e0b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.726+0000 7efd27fff640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efd100a2d70 0x7efd100a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efd20009e60 tx=0x7efd20009450 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.726+0000 7efd305b6740 1 -- 172.21.15.133:0/2763337649 shutdown_connections 2024-09-11T23:41:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.726+0000 7efd305b6740 1 -- 172.21.15.133:0/2763337649 wait complete. 2024-09-11T23:41:51.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.727+0000 7efd305b6740 1 -- start start 2024-09-11T23:41:51.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.727+0000 7efd305b6740 1 Processor -- start 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.728+0000 7efd305b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0b95d0 0x55736e0c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.728+0000 7efd305b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0bab40 0x55736e0c1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.728+0000 7efd305b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.728+0000 7efd305b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55736e0c7070 con 0x55736e0c4370 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.728+0000 7efd305b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55736dfda060 con 0x55736e0b95d0 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.728+0000 7efd305b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55736e0c71f0 con 0x55736e0bab40 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.728+0000 7efd27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.728+0000 7efd305b6740 10 client.0 _ll_register_callbacks cb 0x55736dd2d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.728+0000 7efd27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37490/0 (socket says 172.21.15.133:37490) 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.728+0000 7efd27fff640 1 -- 172.21.15.133:0/2828393556 learned_addr learned my addr 172.21.15.133:0/2828393556 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.729+0000 7efd27fff640 1 -- 172.21.15.133:0/2828393556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0bab40 msgr2=0x55736e0c1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.729+0000 7efd27fff640 1 --2- 172.21.15.133:0/2828393556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0bab40 0x55736e0c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.729+0000 7efd27fff640 1 -- 172.21.15.133:0/2828393556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0b95d0 msgr2=0x55736e0c3d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:41:51.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.729+0000 7efd27fff640 1 --2- 172.21.15.133:0/2828393556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0b95d0 0x55736e0c3d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.729+0000 7efd27fff640 1 -- 172.21.15.133:0/2828393556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55736dd903c0 con 0x55736e0c4370 2024-09-11T23:41:51.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.729+0000 7efd27fff640 1 --2- 172.21.15.133:0/2828393556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c1650 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7efd20009aa0 tx=0x7efd20004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8029fac507779af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:51.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.729+0000 7efd06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.730+0000 7efd06ffd640 1 -- 172.21.15.133:0/2828393556 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2002b060 con 0x55736e0c4370 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.730+0000 7efd06ffd640 1 -- 172.21.15.133:0/2828393556 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7efd20012030 con 0x55736e0c4370 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.730+0000 7efd305b6740 1 -- 172.21.15.133:0/2828393556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd1c0dd070 con 0x55736e0c4370 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.730+0000 7efd06ffd640 1 -- 172.21.15.133:0/2828393556 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd20016970 con 0x55736e0c4370 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.730+0000 7efd305b6740 10 client.16821 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.730+0000 7efd305b6740 1 -- 172.21.15.133:0/2828393556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55736e0d2c20 con 0x55736e0c4370 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.731+0000 7efd06ffd640 1 -- 172.21.15.133:0/2828393556 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd20030440 con 0x55736e0c4370 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.732+0000 7efd06ffd640 1 -- 172.21.15.133:0/2828393556 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efd2002e020 con 0x55736e0c4370 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.732+0000 7efd305b6740 10 client.16821 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.732+0000 7efd305b6740 10 client.16821 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.732+0000 7efd305b6740 20 client.16821 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.732+0000 7efd305b6740 1 --2- 172.21.15.133:0/2828393556 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55736e0d4130 0x55736e0d65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.732+0000 7efd305b6740 4 client.16821 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:51.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.732+0000 7efd305b6740 1 -- 172.21.15.133:0/2828393556 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efd1c0dd070 con 0x55736e0d4130 2024-09-11T23:41:51.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.735+0000 7efd277fe640 1 --2- 172.21.15.133:0/2828393556 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55736e0d4130 0x55736e0d65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.742+0000 7efd277fe640 1 --2- 172.21.15.133:0/2828393556 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55736e0d4130 0x55736e0d65c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=737f8e42a8ace570 server_cookie=a9d65be849010476 in_seq=0 out_seq=0 2024-09-11T23:41:51.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.747+0000 7efd06ffd640 10 client.16821 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:41:51.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.747+0000 7efd06ffd640 1 -- 172.21.15.133:0/2828393556 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7efd1c0dd070 con 0x55736e0d4130 2024-09-11T23:41:51.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.747+0000 7efd06ffd640 10 client.16821 handle_command_reply: tid=0 2024-09-11T23:41:51.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.747+0000 7efd305b6740 1 client.16821 shutdown 2024-09-11T23:41:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.766+0000 7efd305b6740 1 -- 172.21.15.133:0/2828393556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 msgr2=0x55736e0c1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.767+0000 7efd305b6740 1 --2- 172.21.15.133:0/2828393556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c1650 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7efd20009aa0 tx=0x7efd20004b40 comp rx=0 tx=0).stop 2024-09-11T23:41:51.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.769+0000 7efd305b6740 1 -- 172.21.15.133:0/2828393556 shutdown_connections 2024-09-11T23:41:51.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.769+0000 7efd305b6740 1 --2- 172.21.15.133:0/2828393556 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55736e0d4130 0x55736e0d65c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.769+0000 7efd305b6740 1 --2- 172.21.15.133:0/2828393556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0c4370 0x55736e0c1650 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.769+0000 7efd06ffd640 0 client.16821 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:41:51.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.769+0000 7efd305b6740 1 --2- 172.21.15.133:0/2828393556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0bab40 0x55736e0c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.769+0000 7efd305b6740 1 --2- 172.21.15.133:0/2828393556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0b95d0 0x55736e0c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.769+0000 7efd305b6740 1 -- 172.21.15.133:0/2828393556 >> 172.21.15.133:0/2828393556 conn(0x55736e082310 msgr2=0x55736e0bb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:51.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.774+0000 7efd305b6740 1 -- 172.21.15.133:0/2828393556 shutdown_connections 2024-09-11T23:41:51.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.774+0000 7efd305b6740 1 -- 172.21.15.133:0/2828393556 wait complete. 2024-09-11T23:41:51.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.775+0000 7efd305b6740 20 client.16821 trim_cache size 0 max 16384 2024-09-11T23:41:51.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.867+0000 7efd305b6740 1 Processor -- start 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.867+0000 7efd305b6740 1 -- start start 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.868+0000 7efd305b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0b95d0 0x55736e0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.868+0000 7efd305b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0bab40 0x55736e0bc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.869+0000 7efd305b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0c4370 0x55736e0bc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.869+0000 7efd305b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efd281ab620 con 0x55736e0b95d0 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.869+0000 7efd305b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd281ad160 con 0x55736e0bab40 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.869+0000 7efd305b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd281abcf0 con 0x55736e0c4370 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.873+0000 7efd27fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0c4370 0x55736e0bc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.873+0000 7efd26ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0bab40 0x55736e0bc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.875+0000 7efd277fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0b95d0 0x55736e0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.875+0000 7efd277fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0b95d0 0x55736e0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37502/0 (socket says 172.21.15.133:37502) 2024-09-11T23:41:51.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.875+0000 7efd277fe640 1 -- 172.21.15.133:0/1378907470 learned_addr learned my addr 172.21.15.133:0/1378907470 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:51.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.878+0000 7efd27fff640 1 -- 172.21.15.133:0/1378907470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0bab40 msgr2=0x55736e0bc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:51.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.878+0000 7efd27fff640 1 --2- 172.21.15.133:0/1378907470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0bab40 0x55736e0bc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.878+0000 7efd27fff640 1 -- 172.21.15.133:0/1378907470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0b95d0 msgr2=0x55736e0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:51.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.878+0000 7efd27fff640 1 --2- 172.21.15.133:0/1378907470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0b95d0 0x55736e0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.878+0000 7efd27fff640 1 -- 172.21.15.133:0/1378907470 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55736e0bf5b0 con 0x55736e0c4370 2024-09-11T23:41:51.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.881+0000 7efd27fff640 1 --2- 172.21.15.133:0/1378907470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0c4370 0x55736e0bc660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd20013040 tx=0x7efd2000cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b1db582ea7adb1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:51.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.883+0000 7efd0d7fa640 1 -- 172.21.15.133:0/1378907470 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20026b00 con 0x55736e0c4370 2024-09-11T23:41:51.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.883+0000 7efd0d7fa640 1 -- 172.21.15.133:0/1378907470 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd20026ca0 con 0x55736e0c4370 2024-09-11T23:41:51.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.883+0000 7efd0d7fa640 1 -- 172.21.15.133:0/1378907470 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20039a80 con 0x55736e0c4370 2024-09-11T23:41:51.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.885+0000 7efd305b6740 1 -- 172.21.15.133:0/1378907470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0c4370 msgr2=0x55736e0bc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:51.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.885+0000 7efd305b6740 1 --2- 172.21.15.133:0/1378907470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0c4370 0x55736e0bc660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd20013040 tx=0x7efd2000cf70 comp rx=0 tx=0).stop 2024-09-11T23:41:51.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.886+0000 7efd305b6740 1 -- 172.21.15.133:0/1378907470 shutdown_connections 2024-09-11T23:41:51.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.886+0000 7efd305b6740 1 --2- 172.21.15.133:0/1378907470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0c4370 0x55736e0bc660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.886+0000 7efd305b6740 1 --2- 172.21.15.133:0/1378907470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0bab40 0x55736e0bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.886+0000 7efd305b6740 1 --2- 172.21.15.133:0/1378907470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0b95d0 0x55736e0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.886+0000 7efd305b6740 1 -- 172.21.15.133:0/1378907470 >> 172.21.15.133:0/1378907470 conn(0x55736e082310 msgr2=0x55736e0b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:51.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.886+0000 7efd305b6740 1 -- 172.21.15.133:0/1378907470 shutdown_connections 2024-09-11T23:41:51.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.886+0000 7efd305b6740 1 -- 172.21.15.133:0/1378907470 wait complete. 2024-09-11T23:41:51.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.887+0000 7efd305b6740 1 -- start start 2024-09-11T23:41:51.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.888+0000 7efd305b6740 1 Processor -- start 2024-09-11T23:41:51.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.890+0000 7efd305b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0b95d0 0x55736e0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.890+0000 7efd305b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0bab40 0x55736e0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.890+0000 7efd305b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0c4370 0x55736e0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.891+0000 7efd26ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0bab40 0x55736e0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.891+0000 7efd305b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efd281aca90 con 0x55736e0bab40 2024-09-11T23:41:51.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.891+0000 7efd305b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd281ac3c0 con 0x55736e0b95d0 2024-09-11T23:41:51.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.891+0000 7efd305b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd281a9620 con 0x55736e0c4370 2024-09-11T23:41:51.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.891+0000 7efd305b6740 10 client.0 _ll_register_callbacks cb 0x7efd20009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:51.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.893+0000 7efd277fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0b95d0 0x55736e0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.893+0000 7efd277fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0b95d0 0x55736e0cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:46482/0 (socket says 172.21.15.133:46482) 2024-09-11T23:41:51.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.893+0000 7efd277fe640 1 -- 172.21.15.133:0/694966356 learned_addr learned my addr 172.21.15.133:0/694966356 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:51.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.894+0000 7efd26ffd640 1 -- 172.21.15.133:0/694966356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0c4370 msgr2=0x55736e0c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:51.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.894+0000 7efd26ffd640 1 --2- 172.21.15.133:0/694966356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0c4370 0x55736e0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.894+0000 7efd26ffd640 1 -- 172.21.15.133:0/694966356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0b95d0 msgr2=0x55736e0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:51.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.894+0000 7efd26ffd640 1 --2- 172.21.15.133:0/694966356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0b95d0 0x55736e0cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.894+0000 7efd26ffd640 1 -- 172.21.15.133:0/694966356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55736e0beff0 con 0x55736e0bab40 2024-09-11T23:41:51.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.901+0000 7efd26ffd640 1 --2- 172.21.15.133:0/694966356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0bab40 0x55736e0cc9f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55736e0c2680 tx=0x7efd1c005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6b9ce60867daf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:51.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.902+0000 7efd06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:41:51.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.903+0000 7efd06ffd640 1 -- 172.21.15.133:0/694966356 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c09c7f0 con 0x55736e0bab40 2024-09-11T23:41:51.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.904+0000 7efd305b6740 1 -- 172.21.15.133:0/694966356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd20026ca0 con 0x55736e0bab40 2024-09-11T23:41:51.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.904+0000 7efd06ffd640 1 -- 172.21.15.133:0/694966356 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7efd1c09b190 con 0x55736e0bab40 2024-09-11T23:41:51.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.905+0000 7efd06ffd640 1 -- 172.21.15.133:0/694966356 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd1c09d590 con 0x55736e0bab40 2024-09-11T23:41:51.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.906+0000 7efd305b6740 10 client.16827 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:51.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.906+0000 7efd305b6740 1 -- 172.21.15.133:0/694966356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55736e1503f0 con 0x55736e0bab40 2024-09-11T23:41:51.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.906+0000 7efd06ffd640 1 -- 172.21.15.133:0/694966356 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd1c0dd050 con 0x55736e0bab40 2024-09-11T23:41:51.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.906+0000 7efd06ffd640 1 -- 172.21.15.133:0/694966356 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efd1c099500 con 0x55736e0bab40 2024-09-11T23:41:51.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.906+0000 7efd305b6740 10 client.16827 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:51.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.906+0000 7efd305b6740 10 client.16827 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:41:51.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.906+0000 7efd305b6740 20 client.16827 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:51.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.906+0000 7efd305b6740 1 --2- 172.21.15.133:0/694966356 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55736e0d4130 0x55736e152b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:51.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.907+0000 7efd305b6740 4 client.16827 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:41:51.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.907+0000 7efd305b6740 1 -- 172.21.15.133:0/694966356 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7efd20026ca0 con 0x55736e0d4130 2024-09-11T23:41:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.911+0000 7efd277fe640 1 --2- 172.21.15.133:0/694966356 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55736e0d4130 0x55736e152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:51.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.928+0000 7efd06ffd640 10 client.16827 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:41:51.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.928+0000 7efd277fe640 1 --2- 172.21.15.133:0/694966356 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55736e0d4130 0x55736e152b20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95b5fe66e7cca3e2 server_cookie=b27ef5c8107b2818 in_seq=0 out_seq=0 2024-09-11T23:41:51.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.930+0000 7efd06ffd640 1 -- 172.21.15.133:0/694966356 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7efd20026ca0 con 0x55736e0d4130 2024-09-11T23:41:51.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.930+0000 7efd06ffd640 10 client.16827 handle_command_reply: tid=0 2024-09-11T23:41:51.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.930+0000 7efd305b6740 1 client.16827 shutdown 2024-09-11T23:41:51.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.931+0000 7efd305b6740 1 -- 172.21.15.133:0/694966356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0bab40 msgr2=0x55736e0cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:51.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.931+0000 7efd305b6740 1 --2- 172.21.15.133:0/694966356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0bab40 0x55736e0cc9f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55736e0c2680 tx=0x7efd1c005bb0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.931+0000 7efd305b6740 1 -- 172.21.15.133:0/694966356 shutdown_connections 2024-09-11T23:41:51.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.931+0000 7efd305b6740 1 --2- 172.21.15.133:0/694966356 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55736e0d4130 0x55736e152b20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.931+0000 7efd305b6740 1 --2- 172.21.15.133:0/694966356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55736e0c4370 0x55736e0c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.931+0000 7efd305b6740 1 --2- 172.21.15.133:0/694966356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55736e0bab40 0x55736e0cc9f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.932+0000 7efd06ffd640 0 client.16827 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:41:51.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.932+0000 7efd305b6740 1 --2- 172.21.15.133:0/694966356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55736e0b95d0 0x55736e0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.932+0000 7efd305b6740 1 -- 172.21.15.133:0/694966356 >> 172.21.15.133:0/694966356 conn(0x55736e082310 msgr2=0x55736e0c30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:51.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.933+0000 7efd305b6740 1 -- 172.21.15.133:0/694966356 shutdown_connections 2024-09-11T23:41:51.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.934+0000 7efd305b6740 1 -- 172.21.15.133:0/694966356 wait complete. 2024-09-11T23:41:51.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.935+0000 7efd305b6740 20 client.16827 trim_cache size 0 max 16384 2024-09-11T23:41:51.941 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:41:51.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.943+0000 7efd2e1dd640 1 -- 172.21.15.133:0/163112806 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efd100a2d70 msgr2=0x7efd100a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:51.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.944+0000 7efd2e1dd640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efd100a2d70 0x7efd100a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efd20009e60 tx=0x7efd20009450 comp rx=0 tx=0).stop 2024-09-11T23:41:51.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.944+0000 7efd2e1dd640 1 -- 172.21.15.133:0/163112806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd28101430 msgr2=0x7efd28197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:51.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.944+0000 7efd2e1dd640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd28101430 0x7efd28197970 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009860 tx=0x7efd1c009d20 comp rx=0 tx=0).stop 2024-09-11T23:41:51.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.944+0000 7efd2e1dd640 1 -- 172.21.15.133:0/163112806 shutdown_connections 2024-09-11T23:41:51.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.944+0000 7efd2e1dd640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efd100a2d70 0x7efd100a5230 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.944+0000 7efd2e1dd640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd28106a90 0x7efd2819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.944+0000 7efd2e1dd640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd28101430 0x7efd28197970 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.944+0000 7efd2e1dd640 1 --2- 172.21.15.133:0/163112806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd28100a90 0x7efd28197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:51.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.944+0000 7efd2e1dd640 1 -- 172.21.15.133:0/163112806 >> 172.21.15.133:0/163112806 conn(0x7efd280fc7d0 msgr2=0x7efd280fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:51.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.949+0000 7efd2e1dd640 1 -- 172.21.15.133:0/163112806 shutdown_connections 2024-09-11T23:41:51.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:51.949+0000 7efd2e1dd640 1 -- 172.21.15.133:0/163112806 wait complete. 2024-09-11T23:41:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.572+0000 7fa5a9ffb640 1 -- 172.21.15.133:0/408764300 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0319b0 con 0x7fa5ac112920 2024-09-11T23:41:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.572+0000 7fa5abfff640 1 -- 172.21.15.133:0/408764300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac112920 msgr2=0x7fa5ac114db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.572+0000 7fa5abfff640 1 --2- 172.21.15.133:0/408764300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac112920 0x7fa5ac114db0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009900 tx=0x7fa59c02a440 comp rx=0 tx=0).stop 2024-09-11T23:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.573+0000 7fa5abfff640 1 -- 172.21.15.133:0/408764300 shutdown_connections 2024-09-11T23:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.573+0000 7fa5abfff640 1 --2- 172.21.15.133:0/408764300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac112920 0x7fa5ac114db0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.573+0000 7fa5abfff640 1 --2- 172.21.15.133:0/408764300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac109e10 0x7fa5ac112260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.573+0000 7fa5abfff640 1 --2- 172.21.15.133:0/408764300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac109460 0x7fa5ac109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.573+0000 7fa5abfff640 1 -- 172.21.15.133:0/408764300 >> 172.21.15.133:0/408764300 conn(0x7fa5ac06cbf0 msgr2=0x7fa5ac06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.573+0000 7fa5abfff640 1 -- 172.21.15.133:0/408764300 shutdown_connections 2024-09-11T23:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.573+0000 7fa5abfff640 1 -- 172.21.15.133:0/408764300 wait complete. 2024-09-11T23:41:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.574+0000 7fa5abfff640 1 Processor -- start 2024-09-11T23:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.575+0000 7fa5abfff640 1 -- start start 2024-09-11T23:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.575+0000 7fa5abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac109460 0x7fa5ac19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.575+0000 7fa5abfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac112920 0x7fa5ac19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.575+0000 7fa5abfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac1b0e90 0x7fa5ac1b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.575+0000 7fa5abfff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac118ca0 con 0x7fa5ac109460 2024-09-11T23:41:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.575+0000 7fa5abfff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac118b20 con 0x7fa5ac1b0e90 2024-09-11T23:41:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.575+0000 7fa5abfff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac118e20 con 0x7fa5ac112920 2024-09-11T23:41:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.576+0000 7fa5aaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac109460 0x7fa5ac19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.576+0000 7fa5aa7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac112920 0x7fa5ac19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.576+0000 7fa5aaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac109460 0x7fa5ac19f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56568/0 (socket says 172.21.15.133:56568) 2024-09-11T23:41:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.576+0000 7fa5aaffd640 1 -- 172.21.15.133:0/101520144 learned_addr learned my addr 172.21.15.133:0/101520144 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.576+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac1b0e90 0x7fa5ac1b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.576+0000 7fa5aaffd640 1 -- 172.21.15.133:0/101520144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac112920 msgr2=0x7fa5ac19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.576+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac112920 0x7fa5ac19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.576+0000 7fa5aaffd640 1 -- 172.21.15.133:0/101520144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac1b0e90 msgr2=0x7fa5ac1b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.576+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac1b0e90 0x7fa5ac1b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.577+0000 7fa5aaffd640 1 -- 172.21.15.133:0/101520144 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac1b3890 con 0x7fa5ac109460 2024-09-11T23:41:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.577+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac109460 0x7fa5ac19f120 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000b450 tx=0x7fa5a000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.578+0000 7fa593fff640 1 -- 172.21.15.133:0/101520144 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0015020 con 0x7fa5ac109460 2024-09-11T23:41:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.578+0000 7fa593fff640 1 -- 172.21.15.133:0/101520144 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a0011d70 con 0x7fa5ac109460 2024-09-11T23:41:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.580+0000 7fa5abfff640 1 -- 172.21.15.133:0/101520144 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac1b3b80 con 0x7fa5ac109460 2024-09-11T23:41:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.580+0000 7fa593fff640 1 -- 172.21.15.133:0/101520144 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0011890 con 0x7fa5ac109460 2024-09-11T23:41:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.580+0000 7fa5abfff640 1 -- 172.21.15.133:0/101520144 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac1b40a0 con 0x7fa5ac109460 2024-09-11T23:41:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.586+0000 7fa5b463d740 1 Processor -- start 2024-09-11T23:41:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.586+0000 7fa5b463d740 1 -- start start 2024-09-11T23:41:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.587+0000 7fa5b463d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d81cc0 0x564456d820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.598+0000 7fa5ab7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d81cc0 0x564456d820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.601+0000 7fa593fff640 1 -- 172.21.15.133:0/101520144 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa5a0018020 con 0x7fa5ac109460 2024-09-11T23:41:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.601+0000 7fa5ab7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d81cc0 0x564456d820a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:39914/0 (socket says 172.21.15.133:39914) 2024-09-11T23:41:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.601+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/4293756124 learned_addr learned my addr 172.21.15.133:0/4293756124 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.601+0000 7fa5b463d740 1 --2- 172.21.15.133:0/4293756124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d825e0 0x564456d8c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.601+0000 7fa5b463d740 1 --2- 172.21.15.133:0/4293756124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8ce80 0x564456d8f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.601+0000 7fa5b463d740 1 -- 172.21.15.133:0/4293756124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564456ce62f0 con 0x564456d8ce80 2024-09-11T23:41:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.601+0000 7fa5b463d740 1 -- 172.21.15.133:0/4293756124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564456c8e270 con 0x564456d825e0 2024-09-11T23:41:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.601+0000 7fa5b463d740 1 -- 172.21.15.133:0/4293756124 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564456ce22b0 con 0x564456d81cc0 2024-09-11T23:41:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.602+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/4293756124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8ce80 0x564456d8f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.603+0000 7fa593fff640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5940a2cf0 0x7fa5940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.603+0000 7fa593fff640 1 -- 172.21.15.133:0/101520144 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa5a00ceb30 con 0x7fa5ac109460 2024-09-11T23:41:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.604+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/4293756124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d825e0 0x564456d8c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.607+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/4293756124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d825e0 msgr2=0x564456d8c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.607+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/4293756124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d825e0 0x564456d8c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.607+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/4293756124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8ce80 msgr2=0x564456d8f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.607+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/4293756124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8ce80 0x564456d8f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.607+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/4293756124 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564456c8d340 con 0x564456d81cc0 2024-09-11T23:41:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.608+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5940a2cf0 0x7fa5940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.608+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/4293756124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d81cc0 0x564456d820a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa59c031550 tx=0x7fa59c031800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac15d9a46cb2d4c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.610+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5940a2cf0 0x7fa5940a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa59c042000 tx=0x7fa59c044000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.611+0000 7fa5917fa640 1 -- 172.21.15.133:0/4293756124 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c035040 con 0x564456d81cc0 2024-09-11T23:41:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.611+0000 7fa5917fa640 1 -- 172.21.15.133:0/4293756124 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa59c003e10 con 0x564456d81cc0 2024-09-11T23:41:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.611+0000 7fa5917fa640 1 -- 172.21.15.133:0/4293756124 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c03c430 con 0x564456d81cc0 2024-09-11T23:41:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.627+0000 7fa5b463d740 1 -- 172.21.15.133:0/4293756124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d81cc0 msgr2=0x564456d820a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.627+0000 7fa5b463d740 1 --2- 172.21.15.133:0/4293756124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d81cc0 0x564456d820a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa59c031550 tx=0x7fa59c031800 comp rx=0 tx=0).stop 2024-09-11T23:41:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.632+0000 7fa5b463d740 1 -- 172.21.15.133:0/4293756124 shutdown_connections 2024-09-11T23:41:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.632+0000 7fa5b463d740 1 --2- 172.21.15.133:0/4293756124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8ce80 0x564456d8f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.632+0000 7fa5b463d740 1 --2- 172.21.15.133:0/4293756124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d825e0 0x564456d8c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.632+0000 7fa5b463d740 1 --2- 172.21.15.133:0/4293756124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d81cc0 0x564456d820a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.632+0000 7fa5b463d740 1 -- 172.21.15.133:0/4293756124 >> 172.21.15.133:0/4293756124 conn(0x564456d7e230 msgr2=0x564456d7e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.632+0000 7fa5b463d740 1 -- 172.21.15.133:0/4293756124 shutdown_connections 2024-09-11T23:41:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.633+0000 7fa5b463d740 1 -- 172.21.15.133:0/4293756124 wait complete. 2024-09-11T23:41:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.639+0000 7fa5b463d740 1 -- start start 2024-09-11T23:41:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.640+0000 7fa5b463d740 1 Processor -- start 2024-09-11T23:41:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.641+0000 7fa5b463d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d81cc0 0x564456d89e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.641+0000 7fa5b463d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d825e0 0x564456d8a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.641+0000 7fa5b463d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8ce80 0x564456d8a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.642+0000 7fa5b463d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564456d8fa50 con 0x564456d8ce80 2024-09-11T23:41:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.642+0000 7fa5b463d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564456c6f290 con 0x564456d81cc0 2024-09-11T23:41:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.642+0000 7fa5aa7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d825e0 0x564456d8a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.642+0000 7fa5aa7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d825e0 0x564456d8a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:39922/0 (socket says 172.21.15.133:39922) 2024-09-11T23:41:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.642+0000 7fa5aa7fc640 1 -- 172.21.15.133:0/3129043595 learned_addr learned my addr 172.21.15.133:0/3129043595 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.642+0000 7fa5b463d740 1 -- 172.21.15.133:0/3129043595 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564456d8fbd0 con 0x564456d825e0 2024-09-11T23:41:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.642+0000 7fa5b463d740 10 client.0 _ll_register_callbacks cb 0x564456ca0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.643+0000 7fa5aa7fc640 1 -- 172.21.15.133:0/3129043595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d81cc0 msgr2=0x564456d89e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:41:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.643+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/3129043595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d81cc0 0x564456d89e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.643+0000 7fa5aa7fc640 1 -- 172.21.15.133:0/3129043595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8ce80 msgr2=0x564456d8a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.643+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/3129043595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8ce80 0x564456d8a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.643+0000 7fa5aa7fc640 1 -- 172.21.15.133:0/3129043595 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644569f9210 con 0x564456d825e0 2024-09-11T23:41:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.644+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/3129043595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d825e0 0x564456d8a360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400ab30 tx=0x7fa5a40111b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3f744a18d9c4791 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.663+0000 7fa58eff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:41:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.663+0000 7fa58eff5640 1 -- 172.21.15.133:0/3129043595 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a401a070 con 0x564456d825e0 2024-09-11T23:41:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.663+0000 7fa5b463d740 1 -- 172.21.15.133:0/3129043595 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa59c003e10 con 0x564456d825e0 2024-09-11T23:41:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.663+0000 7fa58eff5640 1 -- 172.21.15.133:0/3129043595 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa5a4016450 con 0x564456d825e0 2024-09-11T23:41:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.664+0000 7fa58eff5640 1 -- 172.21.15.133:0/3129043595 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa5a4026050 con 0x564456d825e0 2024-09-11T23:41:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.666+0000 7fa5b463d740 10 client.26242 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.666+0000 7fa5b463d740 1 -- 172.21.15.133:0/3129043595 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564456ac5620 con 0x564456d825e0 2024-09-11T23:41:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.666+0000 7fa58eff5640 1 -- 172.21.15.133:0/3129043595 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a4011e50 con 0x564456d825e0 2024-09-11T23:41:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.667+0000 7fa58eff5640 1 -- 172.21.15.133:0/3129043595 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa5a401e860 con 0x564456d825e0 2024-09-11T23:41:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.667+0000 7fa5b463d740 10 client.26242 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.667+0000 7fa5b463d740 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.667+0000 7fa5b463d740 20 client.26242 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.667+0000 7fa5b463d740 1 --2- 172.21.15.133:0/3129043595 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x564456d9d390 0x564456d9f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.667+0000 7fa5b463d740 4 client.26242 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.667+0000 7fa5b463d740 1 -- 172.21.15.133:0/3129043595 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa59c003e10 con 0x564456d9d390 2024-09-11T23:41:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.668+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/3129043595 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x564456d9d390 0x564456d9f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.670+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/3129043595 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x564456d9d390 0x564456d9f830 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739855dfddfc3191 server_cookie=3806bffca632b2e7 in_seq=0 out_seq=0 2024-09-11T23:41:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.670+0000 7fa58eff5640 10 client.26242 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.671+0000 7fa58eff5640 1 -- 172.21.15.133:0/3129043595 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa59c003e10 con 0x564456d9d390 2024-09-11T23:41:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.671+0000 7fa58eff5640 10 client.26242 handle_command_reply: tid=0 2024-09-11T23:41:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.671+0000 7fa5b463d740 1 client.26242 shutdown 2024-09-11T23:41:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.672+0000 7fa5b463d740 1 -- 172.21.15.133:0/3129043595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d825e0 msgr2=0x564456d8a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.672+0000 7fa5b463d740 1 --2- 172.21.15.133:0/3129043595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d825e0 0x564456d8a360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400ab30 tx=0x7fa5a40111b0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.672+0000 7fa5b463d740 1 -- 172.21.15.133:0/3129043595 shutdown_connections 2024-09-11T23:41:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.672+0000 7fa5b463d740 1 --2- 172.21.15.133:0/3129043595 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x564456d9d390 0x564456d9f830 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.672+0000 7fa5b463d740 1 --2- 172.21.15.133:0/3129043595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8ce80 0x564456d8a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.672+0000 7fa5b463d740 1 --2- 172.21.15.133:0/3129043595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d825e0 0x564456d8a360 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400ab30 tx=0x7fa5a40111b0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.672+0000 7fa5b463d740 1 --2- 172.21.15.133:0/3129043595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d81cc0 0x564456d89e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.672+0000 7fa5b463d740 1 -- 172.21.15.133:0/3129043595 >> 172.21.15.133:0/3129043595 conn(0x564456d7e230 msgr2=0x564456d83f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.672+0000 7fa58eff5640 0 client.26242 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.676+0000 7fa5b463d740 1 -- 172.21.15.133:0/3129043595 shutdown_connections 2024-09-11T23:41:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.676+0000 7fa5b463d740 1 -- 172.21.15.133:0/3129043595 wait complete. 2024-09-11T23:41:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.677+0000 7fa5b463d740 20 client.26242 trim_cache size 0 max 16384 2024-09-11T23:41:52.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:52 smithi138 ceph-mon[35142]: pgmap v155: 97 pgs: 97 active+clean; 39 GiB data, 123 GiB used, 949 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 604 MiB/s wr, 1.43k op/s 2024-09-11T23:41:52.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:52 smithi133 ceph-mon[31430]: pgmap v155: 97 pgs: 97 active+clean; 39 GiB data, 123 GiB used, 949 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 604 MiB/s wr, 1.43k op/s 2024-09-11T23:41:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.757+0000 7fa5b463d740 1 Processor -- start 2024-09-11T23:41:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.757+0000 7fa5b463d740 1 -- start start 2024-09-11T23:41:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.759+0000 7fa5b463d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d81cc0 0x564456d84780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.763+0000 7fa5aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d81cc0 0x564456d84780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.763+0000 7fa5aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d81cc0 0x564456d84780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40880/0 (socket says 172.21.15.133:40880) 2024-09-11T23:41:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.764+0000 7fa5aaffd640 1 -- 172.21.15.133:0/967702343 learned_addr learned my addr 172.21.15.133:0/967702343 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.764+0000 7fa5b463d740 1 --2- 172.21.15.133:0/967702343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d8ce80 0x564456d886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.764+0000 7fa5b463d740 1 --2- 172.21.15.133:0/967702343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8a6d0 0x564456da5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.764+0000 7fa5b463d740 1 -- 172.21.15.133:0/967702343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1bc650 con 0x564456d8a6d0 2024-09-11T23:41:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.764+0000 7fa5b463d740 1 -- 172.21.15.133:0/967702343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1be190 con 0x564456d81cc0 2024-09-11T23:41:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.764+0000 7fa5b463d740 1 -- 172.21.15.133:0/967702343 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1bcd20 con 0x564456d8ce80 2024-09-11T23:41:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.765+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/967702343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d8ce80 0x564456d886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.766+0000 7fa5aaffd640 1 -- 172.21.15.133:0/967702343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d8ce80 msgr2=0x564456d886d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.766+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/967702343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d8ce80 0x564456d886d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.766+0000 7fa5aaffd640 1 -- 172.21.15.133:0/967702343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8a6d0 msgr2=0x564456da5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.766+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/967702343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8a6d0 0x564456da5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.766+0000 7fa5aaffd640 1 -- 172.21.15.133:0/967702343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564456d88db0 con 0x564456d81cc0 2024-09-11T23:41:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.766+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/967702343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d8ce80 0x564456d886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:41:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.768+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/967702343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d81cc0 0x564456d84780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa5a009b160 tx=0x7fa5a00ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac47d6e68ff9064 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.769+0000 7fa590ff9640 1 -- 172.21.15.133:0/967702343 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00b3070 con 0x564456d81cc0 2024-09-11T23:41:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.770+0000 7fa590ff9640 1 -- 172.21.15.133:0/967702343 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a00a3e40 con 0x564456d81cc0 2024-09-11T23:41:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.773+0000 7fa5b463d740 1 -- 172.21.15.133:0/967702343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d81cc0 msgr2=0x564456d84780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.773+0000 7fa5b463d740 1 --2- 172.21.15.133:0/967702343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d81cc0 0x564456d84780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa5a009b160 tx=0x7fa5a00ce040 comp rx=0 tx=0).stop 2024-09-11T23:41:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.773+0000 7fa5b463d740 1 -- 172.21.15.133:0/967702343 shutdown_connections 2024-09-11T23:41:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.773+0000 7fa5b463d740 1 --2- 172.21.15.133:0/967702343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d8a6d0 0x564456da5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.773+0000 7fa5b463d740 1 --2- 172.21.15.133:0/967702343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d8ce80 0x564456d886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.773+0000 7fa5b463d740 1 --2- 172.21.15.133:0/967702343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d81cc0 0x564456d84780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.773+0000 7fa5b463d740 1 -- 172.21.15.133:0/967702343 >> 172.21.15.133:0/967702343 conn(0x564456d7e230 msgr2=0x564456d8bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.778+0000 7fa5b463d740 1 -- 172.21.15.133:0/967702343 shutdown_connections 2024-09-11T23:41:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.778+0000 7fa5b463d740 1 -- 172.21.15.133:0/967702343 wait complete. 2024-09-11T23:41:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.779+0000 7fa5b463d740 1 -- start start 2024-09-11T23:41:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.779+0000 7fa5b463d740 1 Processor -- start 2024-09-11T23:41:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.779+0000 7fa5b463d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d81cc0 0x564456d94e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.779+0000 7fa5b463d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d8a6d0 0x564456d953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.779+0000 7fa5b463d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d8ce80 0x564456d91e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.779+0000 7fa5b463d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1bdac0 con 0x564456d81cc0 2024-09-11T23:41:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.779+0000 7fa5b463d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1bd3f0 con 0x564456d8ce80 2024-09-11T23:41:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.779+0000 7fa5b463d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1ba650 con 0x564456d8a6d0 2024-09-11T23:41:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.779+0000 7fa5b463d740 10 client.0 _ll_register_callbacks cb 0x7fa5a4009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.780+0000 7fa5ab7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d8ce80 0x564456d91e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.781+0000 7fa5ab7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d8ce80 0x564456d91e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40884/0 (socket says 172.21.15.133:40884) 2024-09-11T23:41:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.781+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/1547884712 learned_addr learned my addr 172.21.15.133:0/1547884712 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.781+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d8a6d0 0x564456d953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.782+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/1547884712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d8a6d0 msgr2=0x564456d953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.782+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d8a6d0 0x564456d953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.782+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/1547884712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d81cc0 msgr2=0x564456d94e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.782+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d81cc0 0x564456d94e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.782+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/1547884712 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564456d82410 con 0x564456d8ce80 2024-09-11T23:41:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.783+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d8ce80 0x564456d91e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa59c003970 tx=0x7fa59c0039a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a0c8767b0ea1ae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:52.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:52 smithi178 ceph-mon[35882]: pgmap v155: 97 pgs: 97 active+clean; 39 GiB data, 123 GiB used, 949 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 604 MiB/s wr, 1.43k op/s 2024-09-11T23:41:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.794+0000 7fa58eff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:41:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.795+0000 7fa58eff5640 1 -- 172.21.15.133:0/1547884712 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c056020 con 0x564456d8ce80 2024-09-11T23:41:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.795+0000 7fa5b463d740 1 -- 172.21.15.133:0/1547884712 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a00a3e40 con 0x564456d8ce80 2024-09-11T23:41:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.795+0000 7fa58eff5640 1 -- 172.21.15.133:0/1547884712 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fa59c005c90 con 0x564456d8ce80 2024-09-11T23:41:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.798+0000 7fa58eff5640 1 -- 172.21.15.133:0/1547884712 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa59c061050 con 0x564456d8ce80 2024-09-11T23:41:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.799+0000 7fa5b463d740 10 client.26276 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.800+0000 7fa5b463d740 1 -- 172.21.15.133:0/1547884712 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564456e08520 con 0x564456d8ce80 2024-09-11T23:41:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.801+0000 7fa58eff5640 1 -- 172.21.15.133:0/1547884712 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a00a3e40 con 0x564456d8ce80 2024-09-11T23:41:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.801+0000 7fa58eff5640 1 -- 172.21.15.133:0/1547884712 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa59c051070 con 0x564456d8ce80 2024-09-11T23:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.805+0000 7fa5b463d740 10 client.26276 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.805+0000 7fa5b463d740 10 client.26276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.805+0000 7fa5b463d740 20 client.26276 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.805+0000 7fa5b463d740 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x564456e087a0 0x564456e0ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.805+0000 7fa5b463d740 4 client.26276 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.805+0000 7fa5b463d740 1 -- 172.21.15.133:0/1547884712 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564456cf3a20 con 0x564456e087a0 2024-09-11T23:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.806+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x564456e087a0 0x564456e0ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.811+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x564456e087a0 0x564456e0ab90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2428875cb0fc1ea0 server_cookie=ef9edb940e4bfcb5 in_seq=0 out_seq=0 2024-09-11T23:41:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:52.812+0000 7fa58eff5640 10 client.26276 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:53.372 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:41:53.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:53 smithi133 ceph-mon[31430]: pgmap v156: 97 pgs: 97 active+clean; 39 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 720 MiB/s wr, 1.63k op/s 2024-09-11T23:41:53.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:53 smithi178 ceph-mon[35882]: pgmap v156: 97 pgs: 97 active+clean; 39 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 720 MiB/s wr, 1.63k op/s 2024-09-11T23:41:53.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:53 smithi138 ceph-mon[35142]: pgmap v156: 97 pgs: 97 active+clean; 39 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 720 MiB/s wr, 1.63k op/s 2024-09-11T23:41:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:54.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.959+0000 7f76d6d2c640 1 -- 172.21.15.133:0/2084398148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d0102d30 msgr2=0x7f76d0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:54.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.959+0000 7f76d6d2c640 1 --2- 172.21.15.133:0/2084398148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d0102d30 0x7f76d0103110 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009890 tx=0x7f76c402a380 comp rx=0 tx=0).stop 2024-09-11T23:41:54.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.960+0000 7f76d6d2c640 1 -- 172.21.15.133:0/2084398148 shutdown_connections 2024-09-11T23:41:54.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.960+0000 7f76d6d2c640 1 --2- 172.21.15.133:0/2084398148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d01082a0 0x7f76d010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.960+0000 7f76d6d2c640 1 --2- 172.21.15.133:0/2084398148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d01036e0 0x7f76d0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.960+0000 7f76d6d2c640 1 --2- 172.21.15.133:0/2084398148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d0102d30 0x7f76d0103110 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.960+0000 7f76d6d2c640 1 -- 172.21.15.133:0/2084398148 >> 172.21.15.133:0/2084398148 conn(0x7f76d00fc7d0 msgr2=0x7f76d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:54.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.962+0000 7f76d6d2c640 1 -- 172.21.15.133:0/2084398148 shutdown_connections 2024-09-11T23:41:54.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.962+0000 7f76d6d2c640 1 -- 172.21.15.133:0/2084398148 wait complete. 2024-09-11T23:41:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.963+0000 7f76d6d2c640 1 Processor -- start 2024-09-11T23:41:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.963+0000 7f76d6d2c640 1 -- start start 2024-09-11T23:41:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.963+0000 7f76d6d2c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d0102d30 0x7f76d0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.963+0000 7f76d6d2c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d01036e0 0x7f76d0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:54.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.964+0000 7f76d4ac8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d0102d30 0x7f76d0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:54.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.964+0000 7f76d4ac8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d0102d30 0x7f76d0198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56630/0 (socket says 172.21.15.133:56630) 2024-09-11T23:41:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.964+0000 7f76d4ac8640 1 -- 172.21.15.133:0/1864425386 learned_addr learned my addr 172.21.15.133:0/1864425386 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.964+0000 7f76cffff640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d01036e0 0x7f76d0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.964+0000 7f76d6d2c640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d01082a0 0x7f76d019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.965+0000 7f76d6d2c640 1 -- 172.21.15.133:0/1864425386 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0112100 con 0x7f76d0102d30 2024-09-11T23:41:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.965+0000 7f76d52c9640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d01082a0 0x7f76d019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.965+0000 7f76d6d2c640 1 -- 172.21.15.133:0/1864425386 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0111f80 con 0x7f76d01082a0 2024-09-11T23:41:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.965+0000 7f76d6d2c640 1 -- 172.21.15.133:0/1864425386 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0112280 con 0x7f76d01036e0 2024-09-11T23:41:54.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.965+0000 7f76cffff640 1 -- 172.21.15.133:0/1864425386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d01082a0 msgr2=0x7f76d019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:54.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.965+0000 7f76cffff640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d01082a0 0x7f76d019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.965+0000 7f76cffff640 1 -- 172.21.15.133:0/1864425386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d0102d30 msgr2=0x7f76d0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.966+0000 7f76cffff640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d0102d30 0x7f76d0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.966+0000 7f76cffff640 1 -- 172.21.15.133:0/1864425386 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d019d5f0 con 0x7f76d01036e0 2024-09-11T23:41:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.966+0000 7f76d52c9640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d01082a0 0x7f76d019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:41:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.966+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d0102d30 0x7f76d0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.966+0000 7f76cffff640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d01036e0 0x7f76d0198b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76d00697d0 tx=0x7f76bc00ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.966+0000 7f76cdffb640 1 -- 172.21.15.133:0/1864425386 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc00cc90 con 0x7f76d01036e0 2024-09-11T23:41:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.966+0000 7f76cdffb640 1 -- 172.21.15.133:0/1864425386 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76bc00ee70 con 0x7f76d01036e0 2024-09-11T23:41:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.967+0000 7f76d6d2c640 1 -- 172.21.15.133:0/1864425386 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d019d8e0 con 0x7f76d01036e0 2024-09-11T23:41:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.967+0000 7f76d6d2c640 1 -- 172.21.15.133:0/1864425386 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d01a51c0 con 0x7f76d01036e0 2024-09-11T23:41:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.967+0000 7f76cdffb640 1 -- 172.21.15.133:0/1864425386 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc004a00 con 0x7f76d01036e0 2024-09-11T23:41:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.970+0000 7f76cdffb640 1 -- 172.21.15.133:0/1864425386 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f76bc004ba0 con 0x7f76d01036e0 2024-09-11T23:41:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76cdffb640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76c00a2e40 0x7f76c00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76d9105740 1 Processor -- start 2024-09-11T23:41:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76d9105740 1 -- start start 2024-09-11T23:41:54.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76d9105740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be06b40 0x56385be05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:54.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76d9105740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be055d0 0x56385be05a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:54.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76d9105740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be14780 0x56385be16b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:54.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76d9105740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56385bd6b2f0 con 0x56385be055d0 2024-09-11T23:41:54.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76d9105740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56385bd12cf0 con 0x56385be14780 2024-09-11T23:41:54.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76d9105740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56385bd672b0 con 0x56385be06b40 2024-09-11T23:41:54.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76d52c9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be14780 0x56385be16b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76d52c9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be14780 0x56385be16b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40912/0 (socket says 172.21.15.133:40912) 2024-09-11T23:41:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.972+0000 7f76d52c9640 1 -- 172.21.15.133:0/953544198 learned_addr learned my addr 172.21.15.133:0/953544198 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.973+0000 7f76cffff640 1 --2- 172.21.15.133:0/953544198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be055d0 0x56385be05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.973+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76c00a2e40 0x7f76c00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.973+0000 7f76cdffb640 1 -- 172.21.15.133:0/1864425386 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f76bc0cf2d0 con 0x7f76d01036e0 2024-09-11T23:41:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.973+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/953544198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be06b40 0x56385be05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.973+0000 7f76d4ac8640 1 -- 172.21.15.133:0/953544198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be14780 msgr2=0x56385be16b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.973+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/953544198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be14780 0x56385be16b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.973+0000 7f76d4ac8640 1 -- 172.21.15.133:0/953544198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be055d0 msgr2=0x56385be05a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.974+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/953544198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be055d0 0x56385be05a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.974+0000 7f76d4ac8640 1 -- 172.21.15.133:0/953544198 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56385bcf73a0 con 0x56385be06b40 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.974+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76c00a2e40 0x7f76c00a5300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f76c4031000 tx=0x7f76c40043f0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.974+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/953544198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be06b40 0x56385be05090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f76c402a970 tx=0x7f76c4045000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a758a7c6bd86c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.974+0000 7f76baffd640 1 -- 172.21.15.133:0/953544198 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4037040 con 0x56385be06b40 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.974+0000 7f76baffd640 1 -- 172.21.15.133:0/953544198 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c40309f0 con 0x56385be06b40 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.974+0000 7f76baffd640 1 -- 172.21.15.133:0/953544198 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c40327a0 con 0x56385be06b40 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.975+0000 7f76d9105740 1 -- 172.21.15.133:0/953544198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be06b40 msgr2=0x56385be05090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.975+0000 7f76d9105740 1 --2- 172.21.15.133:0/953544198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be06b40 0x56385be05090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f76c402a970 tx=0x7f76c4045000 comp rx=0 tx=0).stop 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.975+0000 7f76d9105740 1 -- 172.21.15.133:0/953544198 shutdown_connections 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.975+0000 7f76d9105740 1 --2- 172.21.15.133:0/953544198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be14780 0x56385be16b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.975+0000 7f76d9105740 1 --2- 172.21.15.133:0/953544198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be055d0 0x56385be05a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.975+0000 7f76d9105740 1 --2- 172.21.15.133:0/953544198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be06b40 0x56385be05090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.975+0000 7f76d9105740 1 -- 172.21.15.133:0/953544198 >> 172.21.15.133:0/953544198 conn(0x56385bdce310 msgr2=0x56385be01c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.976+0000 7f76d9105740 1 -- 172.21.15.133:0/953544198 shutdown_connections 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.976+0000 7f76d9105740 1 -- 172.21.15.133:0/953544198 wait complete. 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d9105740 1 -- start start 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d9105740 1 Processor -- start 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d9105740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be0d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d9105740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be06b40 0x56385be0db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d9105740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be14780 0x56385be0e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d9105740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56385be084f0 con 0x56385be14780 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d9105740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56385bd26060 con 0x56385be055d0 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d9105740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56385be08670 con 0x56385be06b40 2024-09-11T23:41:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d9105740 10 client.0 _ll_register_callbacks cb 0x56385ba798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:54.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d4ac8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be0d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:54.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d4ac8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be0d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40914/0 (socket says 172.21.15.133:40914) 2024-09-11T23:41:54.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d4ac8640 1 -- 172.21.15.133:0/1337699196 learned_addr learned my addr 172.21.15.133:0/1337699196 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:54.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.977+0000 7f76d52c9640 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be14780 0x56385be0e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:54.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.978+0000 7f76cffff640 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be06b40 0x56385be0db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:54.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.978+0000 7f76d4ac8640 1 -- 172.21.15.133:0/1337699196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be06b40 msgr2=0x56385be0db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.978+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be06b40 0x56385be0db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.978+0000 7f76d4ac8640 1 -- 172.21.15.133:0/1337699196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be14780 msgr2=0x56385be0e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.978+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be14780 0x56385be0e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.978+0000 7f76d4ac8640 1 -- 172.21.15.133:0/1337699196 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56385badc3c0 con 0x56385be055d0 2024-09-11T23:41:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.978+0000 7f76d52c9640 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be14780 0x56385be0e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:41:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.978+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be0d650 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f76c4030690 tx=0x7f76c4003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea9444a4adffc7ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.978+0000 7f76affff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:41:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.978+0000 7f76affff640 1 -- 172.21.15.133:0/1337699196 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4037040 con 0x56385be055d0 2024-09-11T23:41:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.978+0000 7f76d9105740 1 -- 172.21.15.133:0/1337699196 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56385bd78860 con 0x56385be055d0 2024-09-11T23:41:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.982+0000 7f76affff640 1 -- 172.21.15.133:0/1337699196 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f76c4032930 con 0x56385be055d0 2024-09-11T23:41:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.982+0000 7f76affff640 1 -- 172.21.15.133:0/1337699196 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76c4032b30 con 0x56385be055d0 2024-09-11T23:41:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.983+0000 7f76d9105740 10 client.26285 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.983+0000 7f76d9105740 1 -- 172.21.15.133:0/1337699196 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56385be1eab0 con 0x56385be055d0 2024-09-11T23:41:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.983+0000 7f76affff640 1 -- 172.21.15.133:0/1337699196 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56385bd78860 con 0x56385be055d0 2024-09-11T23:41:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.984+0000 7f76affff640 1 -- 172.21.15.133:0/1337699196 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f76c403d070 con 0x56385be055d0 2024-09-11T23:41:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.985+0000 7f76d9105740 10 client.26285 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.985+0000 7f76d9105740 10 client.26285 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:41:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.985+0000 7f76d9105740 20 client.26285 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.985+0000 7f76d9105740 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56385be1fe00 0x56385be221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.986+0000 7f76d9105740 4 client.26285 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.986+0000 7f76d9105740 1 -- 172.21.15.133:0/1337699196 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56385bd72d70 con 0x56385be1fe00 2024-09-11T23:41:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.987+0000 7f76cffff640 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56385be1fe00 0x56385be221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:54.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.990+0000 7f76affff640 10 client.26285 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:41:54.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.990+0000 7f76cffff640 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56385be1fe00 0x56385be221f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd3d1c241913f16b server_cookie=cf9b4a787cd61 in_seq=0 out_seq=0 2024-09-11T23:41:54.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.991+0000 7f76affff640 1 -- 172.21.15.133:0/1337699196 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56385bd72d70 con 0x56385be1fe00 2024-09-11T23:41:54.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.991+0000 7f76affff640 10 client.26285 handle_command_reply: tid=0 2024-09-11T23:41:54.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.992+0000 7f76d9105740 1 client.26285 shutdown 2024-09-11T23:41:54.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.992+0000 7f76d9105740 1 -- 172.21.15.133:0/1337699196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 msgr2=0x56385be0d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:54.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.992+0000 7f76d9105740 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be0d650 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f76c4030690 tx=0x7f76c4003e80 comp rx=0 tx=0).stop 2024-09-11T23:41:54.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.992+0000 7f76d9105740 1 -- 172.21.15.133:0/1337699196 shutdown_connections 2024-09-11T23:41:54.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.992+0000 7f76d9105740 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56385be1fe00 0x56385be221f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.992+0000 7f76d9105740 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be14780 0x56385be0e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.992+0000 7f76d9105740 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be06b40 0x56385be0db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.992+0000 7f76d9105740 1 --2- 172.21.15.133:0/1337699196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be0d650 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:54.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.992+0000 7f76d9105740 1 -- 172.21.15.133:0/1337699196 >> 172.21.15.133:0/1337699196 conn(0x56385bdce310 msgr2=0x56385be07960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:54.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.992+0000 7f76affff640 0 client.26285 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:41:54.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.993+0000 7f76d9105740 1 -- 172.21.15.133:0/1337699196 shutdown_connections 2024-09-11T23:41:54.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.993+0000 7f76d9105740 1 -- 172.21.15.133:0/1337699196 wait complete. 2024-09-11T23:41:54.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:54.993+0000 7f76d9105740 20 client.26285 trim_cache size 0 max 16384 2024-09-11T23:41:55.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.028+0000 7f76d9105740 1 Processor -- start 2024-09-11T23:41:55.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.029+0000 7f76d9105740 1 -- start start 2024-09-11T23:41:55.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.030+0000 7f76d9105740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be055d0 0x56385be0df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:55.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.030+0000 7f76d9105740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be06b40 0x56385be0a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:55.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.030+0000 7f76d9105740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be14780 0x56385be0ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:55.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.031+0000 7f76d9105740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01bc5d0 con 0x56385be055d0 2024-09-11T23:41:55.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.031+0000 7f76d9105740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01be110 con 0x56385be06b40 2024-09-11T23:41:55.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.031+0000 7f76d9105740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01bcca0 con 0x56385be14780 2024-09-11T23:41:55.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.031+0000 7f76cffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be06b40 0x56385be0a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:55.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.031+0000 7f76cffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be06b40 0x56385be0a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40920/0 (socket says 172.21.15.133:40920) 2024-09-11T23:41:55.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.031+0000 7f76cffff640 1 -- 172.21.15.133:0/3894950182 learned_addr learned my addr 172.21.15.133:0/3894950182 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:55.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.031+0000 7f76d52c9640 1 --2- 172.21.15.133:0/3894950182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be14780 0x56385be0ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:55.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.033+0000 7f76d52c9640 1 -- 172.21.15.133:0/3894950182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be06b40 msgr2=0x56385be0a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:55.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.033+0000 7f76d52c9640 1 --2- 172.21.15.133:0/3894950182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be06b40 0x56385be0a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.033+0000 7f76d52c9640 1 -- 172.21.15.133:0/3894950182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be055d0 msgr2=0x56385be0df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:55.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.033+0000 7f76d52c9640 1 --2- 172.21.15.133:0/3894950182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be055d0 0x56385be0df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.033+0000 7f76d52c9640 1 -- 172.21.15.133:0/3894950182 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56385be02960 con 0x56385be14780 2024-09-11T23:41:55.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.034+0000 7f76d52c9640 1 --2- 172.21.15.133:0/3894950182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be14780 0x56385be0ae30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56385be06070 tx=0x7f76c8009cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c53352dc4454e99 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:55.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.040+0000 7f76ba7fc640 1 -- 172.21.15.133:0/3894950182 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8004740 con 0x56385be14780 2024-09-11T23:41:55.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.040+0000 7f76ba7fc640 1 -- 172.21.15.133:0/3894950182 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c8012930 con 0x56385be14780 2024-09-11T23:41:55.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.040+0000 7f76ba7fc640 1 -- 172.21.15.133:0/3894950182 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8011990 con 0x56385be14780 2024-09-11T23:41:55.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.042+0000 7f76d9105740 1 -- 172.21.15.133:0/3894950182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be14780 msgr2=0x56385be0ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:55.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.042+0000 7f76d9105740 1 --2- 172.21.15.133:0/3894950182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be14780 0x56385be0ae30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56385be06070 tx=0x7f76c8009cb0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.043+0000 7f76d9105740 1 -- 172.21.15.133:0/3894950182 shutdown_connections 2024-09-11T23:41:55.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.043+0000 7f76d9105740 1 --2- 172.21.15.133:0/3894950182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be14780 0x56385be0ae30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.043+0000 7f76d9105740 1 --2- 172.21.15.133:0/3894950182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be06b40 0x56385be0a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.043+0000 7f76d9105740 1 --2- 172.21.15.133:0/3894950182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be055d0 0x56385be0df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.043+0000 7f76d9105740 1 -- 172.21.15.133:0/3894950182 >> 172.21.15.133:0/3894950182 conn(0x56385bdce310 msgr2=0x56385be00d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:55.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.046+0000 7f76d9105740 1 -- 172.21.15.133:0/3894950182 shutdown_connections 2024-09-11T23:41:55.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.048+0000 7f76d9105740 1 -- 172.21.15.133:0/3894950182 wait complete. 2024-09-11T23:41:55.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.049+0000 7f76d9105740 1 -- start start 2024-09-11T23:41:55.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.049+0000 7f76d9105740 1 Processor -- start 2024-09-11T23:41:55.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.055+0000 7f76d9105740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be1c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:55.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.056+0000 7f76d9105740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be06b40 0x56385be19cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:55.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.056+0000 7f76d9105740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be14780 0x56385be1a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:55.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.056+0000 7f76d9105740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01bda40 con 0x56385be06b40 2024-09-11T23:41:55.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.056+0000 7f76d9105740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01bd370 con 0x56385be055d0 2024-09-11T23:41:55.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.056+0000 7f76d9105740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01ba5d0 con 0x56385be14780 2024-09-11T23:41:55.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.056+0000 7f76d9105740 10 client.0 _ll_register_callbacks cb 0x7f76bc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:55.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.058+0000 7f76d4ac8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be1c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:55.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.058+0000 7f76d4ac8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be1c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40928/0 (socket says 172.21.15.133:40928) 2024-09-11T23:41:55.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.058+0000 7f76d4ac8640 1 -- 172.21.15.133:0/341172297 learned_addr learned my addr 172.21.15.133:0/341172297 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:55.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.058+0000 7f76cffff640 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be06b40 0x56385be19cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:55.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.059+0000 7f76d4ac8640 1 -- 172.21.15.133:0/341172297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be14780 msgr2=0x56385be1a210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:41:55.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.059+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be14780 0x56385be1a210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.059+0000 7f76d4ac8640 1 -- 172.21.15.133:0/341172297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be06b40 msgr2=0x56385be19cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:55.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.059+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be06b40 0x56385be19cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.059+0000 7f76d4ac8640 1 -- 172.21.15.133:0/341172297 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56385be1a850 con 0x56385be055d0 2024-09-11T23:41:55.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.060+0000 7f76cffff640 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be06b40 0x56385be19cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:55.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.062+0000 7f76d4ac8640 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be1c980 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f76c40329e0 tx=0x7f76c4032a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ada767b65c791d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:55.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.062+0000 7f76affff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:41:55.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.063+0000 7f76affff640 1 -- 172.21.15.133:0/341172297 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4037040 con 0x56385be055d0 2024-09-11T23:41:55.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.063+0000 7f76affff640 1 -- 172.21.15.133:0/341172297 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f76c4058cd0 con 0x56385be055d0 2024-09-11T23:41:55.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.063+0000 7f76d9105740 1 -- 172.21.15.133:0/341172297 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c8012930 con 0x56385be055d0 2024-09-11T23:41:55.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.064+0000 7f76affff640 1 -- 172.21.15.133:0/341172297 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76c4058080 con 0x56385be055d0 2024-09-11T23:41:55.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.064+0000 7f76d9105740 10 client.26291 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:55.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.064+0000 7f76d9105740 1 -- 172.21.15.133:0/341172297 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56385be8a570 con 0x56385be055d0 2024-09-11T23:41:55.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.067+0000 7f76affff640 1 -- 172.21.15.133:0/341172297 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c8012930 con 0x56385be055d0 2024-09-11T23:41:55.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.067+0000 7f76affff640 1 -- 172.21.15.133:0/341172297 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f76c403d040 con 0x56385be055d0 2024-09-11T23:41:55.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.067+0000 7f76d9105740 10 client.26291 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:55.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.067+0000 7f76d9105740 10 client.26291 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:41:55.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.067+0000 7f76d9105740 20 client.26291 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:55.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.067+0000 7f76d9105740 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56385be8aaa0 0x56385be8ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:55.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.067+0000 7f76d9105740 4 client.26291 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:41:55.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.067+0000 7f76d9105740 1 -- 172.21.15.133:0/341172297 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56385bcef170 con 0x56385be8aaa0 2024-09-11T23:41:55.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.073+0000 7f76cffff640 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56385be8aaa0 0x56385be8ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:55.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.087+0000 7f76affff640 10 client.26291 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:41:55.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.087+0000 7f76cffff640 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56385be8aaa0 0x56385be8ce90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b7fe27123d849a4 server_cookie=233214524ce07d1a in_seq=0 out_seq=0 2024-09-11T23:41:55.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.089+0000 7f76affff640 1 -- 172.21.15.133:0/341172297 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56385bcef170 con 0x56385be8aaa0 2024-09-11T23:41:55.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.089+0000 7f76affff640 10 client.26291 handle_command_reply: tid=0 2024-09-11T23:41:55.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.089+0000 7f76d9105740 1 client.26291 shutdown 2024-09-11T23:41:55.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.091+0000 7f76d9105740 1 -- 172.21.15.133:0/341172297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 msgr2=0x56385be1c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:55.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.091+0000 7f76d9105740 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be1c980 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f76c40329e0 tx=0x7f76c4032a10 comp rx=0 tx=0).stop 2024-09-11T23:41:55.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.092+0000 7f76d9105740 1 -- 172.21.15.133:0/341172297 shutdown_connections 2024-09-11T23:41:55.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.092+0000 7f76d9105740 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56385be8aaa0 0x56385be8ce90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.092+0000 7f76d9105740 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56385be14780 0x56385be1a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.092+0000 7f76d9105740 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56385be06b40 0x56385be19cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.092+0000 7f76d9105740 1 --2- 172.21.15.133:0/341172297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56385be055d0 0x56385be1c980 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.092+0000 7f76d9105740 1 -- 172.21.15.133:0/341172297 >> 172.21.15.133:0/341172297 conn(0x56385bdce310 msgr2=0x56385be0be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:55.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.092+0000 7f76affff640 0 client.26291 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:41:55.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.098+0000 7f76d9105740 1 -- 172.21.15.133:0/341172297 shutdown_connections 2024-09-11T23:41:55.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.098+0000 7f76d9105740 1 -- 172.21.15.133:0/341172297 wait complete. 2024-09-11T23:41:55.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.098+0000 7f76d9105740 20 client.26291 trim_cache size 0 max 16384 2024-09-11T23:41:55.100 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:41:55.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.101+0000 7f76d6d2c640 1 -- 172.21.15.133:0/1864425386 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76c00a2e40 msgr2=0x7f76c00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:55.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.101+0000 7f76d6d2c640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76c00a2e40 0x7f76c00a5300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f76c4031000 tx=0x7f76c40043f0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.101+0000 7f76d6d2c640 1 -- 172.21.15.133:0/1864425386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d01036e0 msgr2=0x7f76d0198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:55.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.101+0000 7f76d6d2c640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d01036e0 0x7f76d0198b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76d00697d0 tx=0x7f76bc00ec20 comp rx=0 tx=0).stop 2024-09-11T23:41:55.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.102+0000 7f76d6d2c640 1 -- 172.21.15.133:0/1864425386 shutdown_connections 2024-09-11T23:41:55.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.102+0000 7f76d6d2c640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76c00a2e40 0x7f76c00a5300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.102+0000 7f76d6d2c640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76d01082a0 0x7f76d019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.102+0000 7f76d6d2c640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d01036e0 0x7f76d0198b80 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76d00697d0 tx=0x7f76bc00ec20 comp rx=0 tx=0).stop 2024-09-11T23:41:55.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.102+0000 7f76d6d2c640 1 --2- 172.21.15.133:0/1864425386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d0102d30 0x7f76d0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:55.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.102+0000 7f76d6d2c640 1 -- 172.21.15.133:0/1864425386 >> 172.21.15.133:0/1864425386 conn(0x7f76d00fc7d0 msgr2=0x7f76d0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:55.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.110+0000 7f76d6d2c640 1 -- 172.21.15.133:0/1864425386 shutdown_connections 2024-09-11T23:41:55.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:55.110+0000 7f76d6d2c640 1 -- 172.21.15.133:0/1864425386 wait complete. 2024-09-11T23:41:56.355 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:41:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.566+0000 7fa58eff5640 1 -- 172.21.15.133:0/1547884712 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564456cf3a20 con 0x564456e087a0 2024-09-11T23:41:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.566+0000 7fa58eff5640 10 client.26276 handle_command_reply: tid=0 2024-09-11T23:41:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.566+0000 7fa5b463d740 1 client.26276 shutdown 2024-09-11T23:41:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.566+0000 7fa5b463d740 1 -- 172.21.15.133:0/1547884712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d8ce80 msgr2=0x564456d91e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.566+0000 7fa5b463d740 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d8ce80 0x564456d91e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa59c003970 tx=0x7fa59c0039a0 comp rx=0 tx=0).stop 2024-09-11T23:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.566+0000 7fa5b463d740 1 -- 172.21.15.133:0/1547884712 shutdown_connections 2024-09-11T23:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.566+0000 7fa5b463d740 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x564456e087a0 0x564456e0ab90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.566+0000 7fa5b463d740 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564456d8ce80 0x564456d91e60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.566+0000 7fa5b463d740 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564456d8a6d0 0x564456d953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.566+0000 7fa5b463d740 1 --2- 172.21.15.133:0/1547884712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564456d81cc0 0x564456d94e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.566+0000 7fa5b463d740 1 -- 172.21.15.133:0/1547884712 >> 172.21.15.133:0/1547884712 conn(0x564456d7e230 msgr2=0x564456d896c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.567+0000 7fa58eff5640 0 client.26276 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.567+0000 7fa5b463d740 1 -- 172.21.15.133:0/1547884712 shutdown_connections 2024-09-11T23:41:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.567+0000 7fa5b463d740 1 -- 172.21.15.133:0/1547884712 wait complete. 2024-09-11T23:41:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.567+0000 7fa5b463d740 20 client.26276 trim_cache size 0 max 16384 2024-09-11T23:41:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:41:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:41:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:41:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:41:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.569+0000 7fa5abfff640 1 -- 172.21.15.133:0/101520144 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5940a2cf0 msgr2=0x7fa5940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.569+0000 7fa5abfff640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5940a2cf0 0x7fa5940a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa59c042000 tx=0x7fa59c044000 comp rx=0 tx=0).stop 2024-09-11T23:41:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.569+0000 7fa5abfff640 1 -- 172.21.15.133:0/101520144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac109460 msgr2=0x7fa5ac19f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.569+0000 7fa5abfff640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac109460 0x7fa5ac19f120 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000b450 tx=0x7fa5a000b910 comp rx=0 tx=0).stop 2024-09-11T23:41:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.570+0000 7fa5abfff640 1 -- 172.21.15.133:0/101520144 shutdown_connections 2024-09-11T23:41:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.570+0000 7fa5abfff640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5940a2cf0 0x7fa5940a51b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.570+0000 7fa5abfff640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac1b0e90 0x7fa5ac1b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.570+0000 7fa5abfff640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac112920 0x7fa5ac19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.570+0000 7fa5abfff640 1 --2- 172.21.15.133:0/101520144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac109460 0x7fa5ac19f120 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.570+0000 7fa5abfff640 1 -- 172.21.15.133:0/101520144 >> 172.21.15.133:0/101520144 conn(0x7fa5ac06cbf0 msgr2=0x7fa5ac070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.574+0000 7fa5abfff640 1 -- 172.21.15.133:0/101520144 shutdown_connections 2024-09-11T23:41:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:41:56.574+0000 7fa5abfff640 1 -- 172.21.15.133:0/101520144 wait complete. 2024-09-11T23:41:56.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:56 smithi138 ceph-mon[35142]: pgmap v157: 97 pgs: 97 active+clean; 40 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 756 MiB/s wr, 1.55k op/s 2024-09-11T23:41:56.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:56 smithi133 ceph-mon[31430]: pgmap v157: 97 pgs: 97 active+clean; 40 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 756 MiB/s wr, 1.55k op/s 2024-09-11T23:41:56.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:56 smithi178 ceph-mon[35882]: pgmap v157: 97 pgs: 97 active+clean; 40 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 756 MiB/s wr, 1.55k op/s 2024-09-11T23:41:57.151 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:41:57.882 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:41:57 smithi133 ceph-mon[31430]: pgmap v158: 97 pgs: 97 active+clean; 40 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 603 MiB/s wr, 1.14k op/s 2024-09-11T23:41:58.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:41:57 smithi178 ceph-mon[35882]: pgmap v158: 97 pgs: 97 active+clean; 40 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 603 MiB/s wr, 1.14k op/s 2024-09-11T23:41:58.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:41:57 smithi138 ceph-mon[35142]: pgmap v158: 97 pgs: 97 active+clean; 40 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 603 MiB/s wr, 1.14k op/s 2024-09-11T23:41:58.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.482+0000 7f31b75fb640 1 -- 172.21.15.133:0/712428742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b010ea80 msgr2=0x7f31b01154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:58.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.482+0000 7f31b75fb640 1 --2- 172.21.15.133:0/712428742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b010ea80 0x7f31b01154f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f31ac008410 tx=0x7f31ac005f90 comp rx=0 tx=0).stop 2024-09-11T23:41:58.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.482+0000 7f31b75fb640 1 -- 172.21.15.133:0/712428742 shutdown_connections 2024-09-11T23:41:58.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.482+0000 7f31b75fb640 1 --2- 172.21.15.133:0/712428742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b010ea80 0x7f31b01154f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.482+0000 7f31b75fb640 1 --2- 172.21.15.133:0/712428742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31b0109e00 0x7f31b010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.482+0000 7f31b75fb640 1 --2- 172.21.15.133:0/712428742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b0109450 0x7f31b0109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.482+0000 7f31b75fb640 1 -- 172.21.15.133:0/712428742 >> 172.21.15.133:0/712428742 conn(0x7f31b006e2d0 msgr2=0x7f31b006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:58.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.485+0000 7f31b75fb640 1 -- 172.21.15.133:0/712428742 shutdown_connections 2024-09-11T23:41:58.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.487+0000 7f31b75fb640 1 -- 172.21.15.133:0/712428742 wait complete. 2024-09-11T23:41:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.489+0000 7f31b75fb640 1 Processor -- start 2024-09-11T23:41:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.489+0000 7f31b75fb640 1 -- start start 2024-09-11T23:41:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.490+0000 7f31b75fb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b0109450 0x7f31b0118dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.490+0000 7f31b75fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b0109e00 0x7f31b0119300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.490+0000 7f31b75fb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31b0119840 0x7f31b01b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.490+0000 7f31b75fb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f31b0118570 con 0x7f31b0109e00 2024-09-11T23:41:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.490+0000 7f31b75fb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01183f0 con 0x7f31b0109450 2024-09-11T23:41:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.490+0000 7f31b75fb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01186f0 con 0x7f31b0119840 2024-09-11T23:41:58.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.497+0000 7f31b5b98640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31b0119840 0x7f31b01b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.497+0000 7f31b4b96640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b0109e00 0x7f31b0119300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.497+0000 7f31b4b96640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b0109e00 0x7f31b0119300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56690/0 (socket says 172.21.15.133:56690) 2024-09-11T23:41:58.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.497+0000 7f31b4b96640 1 -- 172.21.15.133:0/860485277 learned_addr learned my addr 172.21.15.133:0/860485277 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:58.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.497+0000 7f31b5397640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b0109450 0x7f31b0118dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.498+0000 7f31b4b96640 1 -- 172.21.15.133:0/860485277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31b0119840 msgr2=0x7f31b01b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:58.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.498+0000 7f31b4b96640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31b0119840 0x7f31b01b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.498+0000 7f31b4b96640 1 -- 172.21.15.133:0/860485277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b0109450 msgr2=0x7f31b0118dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:58.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.498+0000 7f31b4b96640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b0109450 0x7f31b0118dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.498+0000 7f31b4b96640 1 -- 172.21.15.133:0/860485277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b01a2870 con 0x7f31b0109e00 2024-09-11T23:41:58.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.499+0000 7f31b4b96640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b0109e00 0x7f31b0119300 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f31a00058c0 tx=0x7f31a0009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:58.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.503+0000 7f31a67fc640 1 -- 172.21.15.133:0/860485277 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0009e50 con 0x7f31b0109e00 2024-09-11T23:41:58.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.503+0000 7f31a67fc640 1 -- 172.21.15.133:0/860485277 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a000ed30 con 0x7f31b0109e00 2024-09-11T23:41:58.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.503+0000 7f31a67fc640 1 -- 172.21.15.133:0/860485277 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0004a00 con 0x7f31b0109e00 2024-09-11T23:41:58.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.505+0000 7f31b75fb640 1 -- 172.21.15.133:0/860485277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b01a2b60 con 0x7f31b0109e00 2024-09-11T23:41:58.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.505+0000 7f31b75fb640 1 -- 172.21.15.133:0/860485277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b01a3050 con 0x7f31b0109e00 2024-09-11T23:41:58.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.512+0000 7f31b99d4740 1 Processor -- start 2024-09-11T23:41:58.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.512+0000 7f31b99d4740 1 -- start start 2024-09-11T23:41:58.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.513+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f556b40 0x56252f555090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.513+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f5555d0 0x56252f555a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.513+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 0x56252f562760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.513+0000 7f31b99d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56252f4bb2f0 con 0x56252f560370 2024-09-11T23:41:58.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.513+0000 7f31b99d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56252f462cf0 con 0x56252f556b40 2024-09-11T23:41:58.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.513+0000 7f31b99d4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56252f4b72b0 con 0x56252f5555d0 2024-09-11T23:41:58.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.514+0000 7f31b5397640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f5555d0 0x56252f555a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.514+0000 7f31b5397640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f5555d0 0x56252f555a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40000/0 (socket says 172.21.15.133:40000) 2024-09-11T23:41:58.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.514+0000 7f31b5397640 1 -- 172.21.15.133:0/2799805209 learned_addr learned my addr 172.21.15.133:0/2799805209 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:58.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.515+0000 7f31b5397640 1 -- 172.21.15.133:0/2799805209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f556b40 msgr2=0x56252f555090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:41:58.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.516+0000 7f31b5397640 1 --2- 172.21.15.133:0/2799805209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f556b40 0x56252f555090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.516+0000 7f31b5397640 1 -- 172.21.15.133:0/2799805209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 msgr2=0x56252f562760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:41:58.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.516+0000 7f31b5397640 1 --2- 172.21.15.133:0/2799805209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 0x56252f562760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.516+0000 7f31b5397640 1 -- 172.21.15.133:0/2799805209 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252f4473a0 con 0x56252f5555d0 2024-09-11T23:41:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.517+0000 7f31b5397640 1 --2- 172.21.15.133:0/2799805209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f5555d0 0x56252f555a10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009f80 tx=0x7f31a80108a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb3b184fdc88c45e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.518+0000 7f319b7fe640 1 -- 172.21.15.133:0/2799805209 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a800e8c0 con 0x56252f5555d0 2024-09-11T23:41:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.518+0000 7f319b7fe640 1 -- 172.21.15.133:0/2799805209 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a800c3e0 con 0x56252f5555d0 2024-09-11T23:41:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.518+0000 7f319b7fe640 1 -- 172.21.15.133:0/2799805209 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8019600 con 0x56252f5555d0 2024-09-11T23:41:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.519+0000 7f31b99d4740 1 -- 172.21.15.133:0/2799805209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f5555d0 msgr2=0x56252f555a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.520+0000 7f31b99d4740 1 --2- 172.21.15.133:0/2799805209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f5555d0 0x56252f555a10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009f80 tx=0x7f31a80108a0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.520+0000 7f31b99d4740 1 -- 172.21.15.133:0/2799805209 shutdown_connections 2024-09-11T23:41:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.520+0000 7f31b99d4740 1 --2- 172.21.15.133:0/2799805209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 0x56252f562760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.520+0000 7f31b99d4740 1 --2- 172.21.15.133:0/2799805209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f5555d0 0x56252f555a10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.520+0000 7f31b99d4740 1 --2- 172.21.15.133:0/2799805209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f556b40 0x56252f555090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.520+0000 7f31b99d4740 1 -- 172.21.15.133:0/2799805209 >> 172.21.15.133:0/2799805209 conn(0x56252f51e310 msgr2=0x56252f551c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.527+0000 7f31b99d4740 1 -- 172.21.15.133:0/2799805209 shutdown_connections 2024-09-11T23:41:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.528+0000 7f31a67fc640 1 -- 172.21.15.133:0/860485277 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f31a001b020 con 0x7f31b0109e00 2024-09-11T23:41:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.529+0000 7f31b99d4740 1 -- 172.21.15.133:0/2799805209 wait complete. 2024-09-11T23:41:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.530+0000 7f31a67fc640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f319c0a2fe0 0x7f319c0a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.530+0000 7f31b99d4740 1 -- start start 2024-09-11T23:41:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.530+0000 7f31b99d4740 1 Processor -- start 2024-09-11T23:41:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.530+0000 7f31a67fc640 1 -- 172.21.15.133:0/860485277 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f31a00cea30 con 0x7f31b0109e00 2024-09-11T23:41:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.532+0000 7f31b5b98640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f319c0a2fe0 0x7f319c0a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.532+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f55fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.532+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 0x56252f55e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.533+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 0x56252f55f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.533+0000 7f31b99d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56252f563070 con 0x56252f560370 2024-09-11T23:41:58.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.533+0000 7f31b99d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56252f476060 con 0x56252f5555d0 2024-09-11T23:41:58.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.533+0000 7f31b99d4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56252f5631f0 con 0x56252f556b40 2024-09-11T23:41:58.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.533+0000 7f31b99d4740 10 client.0 _ll_register_callbacks cb 0x56252f1c98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:58.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.533+0000 7f31b4b96640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 0x56252f55e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.533+0000 7f31b4b96640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 0x56252f55e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40010/0 (socket says 172.21.15.133:40010) 2024-09-11T23:41:58.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.533+0000 7f31b4b96640 1 -- 172.21.15.133:0/4166316264 learned_addr learned my addr 172.21.15.133:0/4166316264 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:58.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.535+0000 7f31b4b96640 1 -- 172.21.15.133:0/4166316264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 msgr2=0x56252f55fe20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:41:58.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.535+0000 7f31b4b96640 1 --2- 172.21.15.133:0/4166316264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f55fe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.535+0000 7f31b4b96640 1 -- 172.21.15.133:0/4166316264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 msgr2=0x56252f55f120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:41:58.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.535+0000 7f31b4b96640 1 --2- 172.21.15.133:0/4166316264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 0x56252f55f120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.535+0000 7f31b4b96640 1 -- 172.21.15.133:0/4166316264 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252f22c3c0 con 0x56252f556b40 2024-09-11T23:41:58.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.538+0000 7f31b4b96640 1 --2- 172.21.15.133:0/4166316264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 0x56252f55e990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31a00a3720 tx=0x7f31a009b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=763bbfc6c06a060c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:58.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.538+0000 7f31b5b98640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f319c0a2fe0 0x7f319c0a54a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0083e0 tx=0x7f31ac0326e0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:58.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.538+0000 7f3198ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:41:58.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.538+0000 7f3198ff9640 1 -- 172.21.15.133:0/4166316264 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a00cc610 con 0x56252f556b40 2024-09-11T23:41:58.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.538+0000 7f3198ff9640 1 -- 172.21.15.133:0/4166316264 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f31a00a37c0 con 0x56252f556b40 2024-09-11T23:41:58.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.539+0000 7f31b99d4740 1 -- 172.21.15.133:0/4166316264 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31a800c3e0 con 0x56252f556b40 2024-09-11T23:41:58.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.539+0000 7f3198ff9640 1 -- 172.21.15.133:0/4166316264 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f31a00a39c0 con 0x56252f556b40 2024-09-11T23:41:58.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.539+0000 7f31b99d4740 10 client.26272 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:58.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.539+0000 7f31b99d4740 1 -- 172.21.15.133:0/4166316264 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56252f56ec70 con 0x56252f556b40 2024-09-11T23:41:58.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.540+0000 7f3198ff9640 1 -- 172.21.15.133:0/4166316264 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a800c3e0 con 0x56252f556b40 2024-09-11T23:41:58.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.540+0000 7f3198ff9640 1 -- 172.21.15.133:0/4166316264 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f31a009a430 con 0x56252f556b40 2024-09-11T23:41:58.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.540+0000 7f31b99d4740 10 client.26272 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:58.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.540+0000 7f31b99d4740 10 client.26272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:58.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.540+0000 7f31b99d4740 20 client.26272 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:58.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.540+0000 7f31b99d4740 1 --2- 172.21.15.133:0/4166316264 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56252f570130 0x56252f572520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.540+0000 7f31b99d4740 4 client.26272 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:41:58.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.540+0000 7f31b99d4740 1 -- 172.21.15.133:0/4166316264 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56252f4c8860 con 0x56252f570130 2024-09-11T23:41:58.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.541+0000 7f31b5397640 1 --2- 172.21.15.133:0/4166316264 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56252f570130 0x56252f572520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.542+0000 7f3198ff9640 10 client.26272 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:58.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.542+0000 7f31b5397640 1 --2- 172.21.15.133:0/4166316264 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56252f570130 0x56252f572520 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5af50cc2b05edd44 server_cookie=98fcc47f11a0322e in_seq=0 out_seq=0 2024-09-11T23:41:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.542+0000 7f3198ff9640 1 -- 172.21.15.133:0/4166316264 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56252f4c8860 con 0x56252f570130 2024-09-11T23:41:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.542+0000 7f3198ff9640 10 client.26272 handle_command_reply: tid=0 2024-09-11T23:41:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.542+0000 7f31b99d4740 1 client.26272 shutdown 2024-09-11T23:41:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.543+0000 7f31b99d4740 1 -- 172.21.15.133:0/4166316264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 msgr2=0x56252f55e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.543+0000 7f31b99d4740 1 --2- 172.21.15.133:0/4166316264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 0x56252f55e990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31a00a3720 tx=0x7f31a009b7e0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.543+0000 7f31b99d4740 1 -- 172.21.15.133:0/4166316264 shutdown_connections 2024-09-11T23:41:58.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.543+0000 7f31b99d4740 1 --2- 172.21.15.133:0/4166316264 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56252f570130 0x56252f572520 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.543+0000 7f31b99d4740 1 --2- 172.21.15.133:0/4166316264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 0x56252f55f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.543+0000 7f31b99d4740 1 --2- 172.21.15.133:0/4166316264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 0x56252f55e990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.543+0000 7f3198ff9640 0 client.26272 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:58.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.543+0000 7f31b99d4740 1 --2- 172.21.15.133:0/4166316264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f55fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.543+0000 7f31b99d4740 1 -- 172.21.15.133:0/4166316264 >> 172.21.15.133:0/4166316264 conn(0x56252f51e310 msgr2=0x56252f557960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:58.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.543+0000 7f31b99d4740 1 -- 172.21.15.133:0/4166316264 shutdown_connections 2024-09-11T23:41:58.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.543+0000 7f31b99d4740 1 -- 172.21.15.133:0/4166316264 wait complete. 2024-09-11T23:41:58.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.544+0000 7f31b99d4740 20 client.26272 trim_cache size 0 max 16384 2024-09-11T23:41:58.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.570+0000 7f31b99d4740 1 Processor -- start 2024-09-11T23:41:58.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.570+0000 7f31b99d4740 1 -- start start 2024-09-11T23:41:58.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.570+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f561760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.571+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f556b40 0x56252f561ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.571+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f560370 0x56252f558120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.571+0000 7f31b5397640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f561760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.571+0000 7f31b99d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01c4f50 con 0x56252f556b40 2024-09-11T23:41:58.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.571+0000 7f31b99d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01c6a90 con 0x56252f5555d0 2024-09-11T23:41:58.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.571+0000 7f31b99d4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01c5620 con 0x56252f560370 2024-09-11T23:41:58.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.571+0000 7f31b5397640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f561760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40964/0 (socket says 172.21.15.133:40964) 2024-09-11T23:41:58.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.571+0000 7f31b5397640 1 -- 172.21.15.133:0/2010993796 learned_addr learned my addr 172.21.15.133:0/2010993796 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:58.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.571+0000 7f31b4b96640 1 --2- 172.21.15.133:0/2010993796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f556b40 0x56252f561ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.571+0000 7f31b5b98640 1 --2- 172.21.15.133:0/2010993796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f560370 0x56252f558120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.572+0000 7f31b5397640 1 -- 172.21.15.133:0/2010993796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f560370 msgr2=0x56252f558120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:58.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.572+0000 7f31b5397640 1 --2- 172.21.15.133:0/2010993796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f560370 0x56252f558120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.572+0000 7f31b5397640 1 -- 172.21.15.133:0/2010993796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f556b40 msgr2=0x56252f561ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:58.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.572+0000 7f31b5397640 1 --2- 172.21.15.133:0/2010993796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f556b40 0x56252f561ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.572+0000 7f31b5397640 1 -- 172.21.15.133:0/2010993796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252f55f490 con 0x56252f5555d0 2024-09-11T23:41:58.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.572+0000 7f31b4b96640 1 --2- 172.21.15.133:0/2010993796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f556b40 0x56252f561ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:58.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.572+0000 7f31b5397640 1 --2- 172.21.15.133:0/2010993796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f561760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f31a800ce30 tx=0x7f31a800ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68da35a3f8882134 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:58.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.572+0000 7f319affd640 1 -- 172.21.15.133:0/2010993796 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a802a070 con 0x56252f5555d0 2024-09-11T23:41:58.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.573+0000 7f319affd640 1 -- 172.21.15.133:0/2010993796 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a8003e50 con 0x56252f5555d0 2024-09-11T23:41:58.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.573+0000 7f319affd640 1 -- 172.21.15.133:0/2010993796 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a800e6c0 con 0x56252f5555d0 2024-09-11T23:41:58.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.574+0000 7f31b99d4740 1 -- 172.21.15.133:0/2010993796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 msgr2=0x56252f561760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:58.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.574+0000 7f31b99d4740 1 --2- 172.21.15.133:0/2010993796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f561760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f31a800ce30 tx=0x7f31a800ce60 comp rx=0 tx=0).stop 2024-09-11T23:41:58.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.575+0000 7f31b99d4740 1 -- 172.21.15.133:0/2010993796 shutdown_connections 2024-09-11T23:41:58.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.575+0000 7f31b99d4740 1 --2- 172.21.15.133:0/2010993796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f560370 0x56252f558120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.575+0000 7f31b99d4740 1 --2- 172.21.15.133:0/2010993796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f556b40 0x56252f561ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.575+0000 7f31b99d4740 1 --2- 172.21.15.133:0/2010993796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f561760 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.575+0000 7f31b99d4740 1 -- 172.21.15.133:0/2010993796 >> 172.21.15.133:0/2010993796 conn(0x56252f51e310 msgr2=0x56252f550d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:58.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.575+0000 7f31b99d4740 1 -- 172.21.15.133:0/2010993796 shutdown_connections 2024-09-11T23:41:58.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.575+0000 7f31b99d4740 1 -- 172.21.15.133:0/2010993796 wait complete. 2024-09-11T23:41:58.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.576+0000 7f31b99d4740 1 -- start start 2024-09-11T23:41:58.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.576+0000 7f31b99d4740 1 Processor -- start 2024-09-11T23:41:58.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f55ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 0x56252f568480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b99d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 0x56252f5689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b99d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01c63c0 con 0x56252f560370 2024-09-11T23:41:58.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b99d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01c5cf0 con 0x56252f5555d0 2024-09-11T23:41:58.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b99d4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f31b011a2f0 con 0x56252f556b40 2024-09-11T23:41:58.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b5b98640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 0x56252f5689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b99d4740 10 client.0 _ll_register_callbacks cb 0x7f31a000eee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:41:58.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b4b96640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 0x56252f568480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b5b98640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 0x56252f5689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56708/0 (socket says 172.21.15.133:56708) 2024-09-11T23:41:58.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b5b98640 1 -- 172.21.15.133:0/1390672364 learned_addr learned my addr 172.21.15.133:0/1390672364 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:41:58.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.577+0000 7f31b5397640 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f55ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.578+0000 7f31b4b96640 1 -- 172.21.15.133:0/1390672364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 msgr2=0x56252f55ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:58.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.578+0000 7f31b4b96640 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f55ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.578+0000 7f31b4b96640 1 -- 172.21.15.133:0/1390672364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 msgr2=0x56252f5689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:41:58.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.578+0000 7f31b4b96640 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 0x56252f5689c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.578+0000 7f31b4b96640 1 -- 172.21.15.133:0/1390672364 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252f557af0 con 0x56252f556b40 2024-09-11T23:41:58.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.578+0000 7f31b5397640 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f55ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.578+0000 7f31b4b96640 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 0x56252f568480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31a00979b0 tx=0x7f31a0005540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=599c008dbb623590 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.578+0000 7f3198ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f3198ff9640 1 -- 172.21.15.133:0/1390672364 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0004e80 con 0x56252f556b40 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f31b99d4740 1 -- 172.21.15.133:0/1390672364 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31a8003e50 con 0x56252f556b40 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f3198ff9640 1 -- 172.21.15.133:0/1390672364 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f31a00cc410 con 0x56252f556b40 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f3198ff9640 1 -- 172.21.15.133:0/1390672364 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f31a00cc610 con 0x56252f556b40 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f31b99d4740 10 client.26278 fetch_fsmap learned FSMap version 28 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f31b99d4740 1 -- 172.21.15.133:0/1390672364 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56252f5daec0 con 0x56252f556b40 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f3198ff9640 1 -- 172.21.15.133:0/1390672364 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a8003e50 con 0x56252f556b40 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f3198ff9640 1 -- 172.21.15.133:0/1390672364 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f31a0099fc0 con 0x56252f556b40 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f31b99d4740 10 client.26278 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f31b99d4740 10 client.26278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:41:58.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f31b99d4740 20 client.26278 populate_metadata read hostname 'smithi133' 2024-09-11T23:41:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.579+0000 7f31b99d4740 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56252f5db450 0x56252f5dd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:41:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.580+0000 7f31b99d4740 4 client.26278 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:41:58.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.580+0000 7f31b99d4740 1 -- 172.21.15.133:0/1390672364 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56252f4c2d70 con 0x56252f5db450 2024-09-11T23:41:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.580+0000 7f31b5397640 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56252f5db450 0x56252f5dd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:41:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.580+0000 7f3198ff9640 10 client.26278 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.581+0000 7f31b5397640 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56252f5db450 0x56252f5dd870 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a3f60c064fab95 server_cookie=1f2cdbf8778820f6 in_seq=0 out_seq=0 2024-09-11T23:41:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.581+0000 7f3198ff9640 1 -- 172.21.15.133:0/1390672364 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56252f4c2d70 con 0x56252f5db450 2024-09-11T23:41:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.581+0000 7f3198ff9640 10 client.26278 handle_command_reply: tid=0 2024-09-11T23:41:58.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.581+0000 7f31b99d4740 1 client.26278 shutdown 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.581+0000 7f31b99d4740 1 -- 172.21.15.133:0/1390672364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 msgr2=0x56252f568480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.581+0000 7f31b99d4740 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 0x56252f568480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31a00979b0 tx=0x7f31a0005540 comp rx=0 tx=0).stop 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.581+0000 7f31b99d4740 1 -- 172.21.15.133:0/1390672364 shutdown_connections 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.582+0000 7f31b99d4740 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56252f5db450 0x56252f5dd870 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.582+0000 7f31b99d4740 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56252f560370 0x56252f5689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.582+0000 7f3198ff9640 0 client.26278 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.582+0000 7f31b99d4740 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56252f556b40 0x56252f568480 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.582+0000 7f31b99d4740 1 --2- 172.21.15.133:0/1390672364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56252f5555d0 0x56252f55ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.582+0000 7f31b99d4740 1 -- 172.21.15.133:0/1390672364 >> 172.21.15.133:0/1390672364 conn(0x56252f51e310 msgr2=0x56252f56b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.582+0000 7f31b99d4740 1 -- 172.21.15.133:0/1390672364 shutdown_connections 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.582+0000 7f31b99d4740 1 -- 172.21.15.133:0/1390672364 wait complete. 2024-09-11T23:41:58.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.582+0000 7f31b99d4740 20 client.26278 trim_cache size 0 max 16384 2024-09-11T23:41:58.584 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:41:58.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.583+0000 7f31b75fb640 1 -- 172.21.15.133:0/860485277 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f319c0a2fe0 msgr2=0x7f319c0a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:58.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.583+0000 7f31b75fb640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f319c0a2fe0 0x7f319c0a54a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0083e0 tx=0x7f31ac0326e0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.583+0000 7f31b75fb640 1 -- 172.21.15.133:0/860485277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b0109e00 msgr2=0x7f31b0119300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:41:58.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.583+0000 7f31b75fb640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b0109e00 0x7f31b0119300 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f31a00058c0 tx=0x7f31a0009c60 comp rx=0 tx=0).stop 2024-09-11T23:41:58.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.583+0000 7f31b75fb640 1 -- 172.21.15.133:0/860485277 shutdown_connections 2024-09-11T23:41:58.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.583+0000 7f31b75fb640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f319c0a2fe0 0x7f319c0a54a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.583+0000 7f31b75fb640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31b0119840 0x7f31b01b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.584+0000 7f31b75fb640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b0109e00 0x7f31b0119300 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.584+0000 7f31b75fb640 1 --2- 172.21.15.133:0/860485277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b0109450 0x7f31b0118dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:41:58.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.584+0000 7f31b75fb640 1 -- 172.21.15.133:0/860485277 >> 172.21.15.133:0/860485277 conn(0x7f31b006e2d0 msgr2=0x7f31b0072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:41:58.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.584+0000 7f31b75fb640 1 -- 172.21.15.133:0/860485277 shutdown_connections 2024-09-11T23:41:58.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:41:58.584+0000 7f31b75fb640 1 -- 172.21.15.133:0/860485277 wait complete. 2024-09-11T23:42:00.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:00 smithi138 ceph-mon[35142]: pgmap v159: 97 pgs: 97 active+clean; 40 GiB data, 135 GiB used, 937 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 867 MiB/s wr, 1.90k op/s 2024-09-11T23:42:00.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:00 smithi133 ceph-mon[31430]: pgmap v159: 97 pgs: 97 active+clean; 40 GiB data, 135 GiB used, 937 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 867 MiB/s wr, 1.90k op/s 2024-09-11T23:42:00.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:00 smithi178 ceph-mon[35882]: pgmap v159: 97 pgs: 97 active+clean; 40 GiB data, 135 GiB used, 937 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 867 MiB/s wr, 1.90k op/s 2024-09-11T23:42:00.796 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:42:00.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:42:01.424 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:01.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:01 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:42:01.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:01 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:42:01.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:01 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:42:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.265+0000 7fc26766c640 1 -- 172.21.15.133:0/424060625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc260109830 msgr2=0x7fc260109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:02.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.265+0000 7fc26766c640 1 --2- 172.21.15.133:0/424060625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc260109830 0x7fc260109c90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc258009ca0 tx=0x7fc25802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.274+0000 7fc26766c640 1 -- 172.21.15.133:0/424060625 shutdown_connections 2024-09-11T23:42:02.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.275+0000 7fc26766c640 1 --2- 172.21.15.133:0/424060625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc26011b140 0x7fc26011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.275+0000 7fc26766c640 1 --2- 172.21.15.133:0/424060625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc260109830 0x7fc260109c90 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.275+0000 7fc26766c640 1 --2- 172.21.15.133:0/424060625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc260109450 0x7fc260110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.275+0000 7fc26766c640 1 -- 172.21.15.133:0/424060625 >> 172.21.15.133:0/424060625 conn(0x7fc26006e140 msgr2=0x7fc26006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:02.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.275+0000 7fc26766c640 1 -- 172.21.15.133:0/424060625 shutdown_connections 2024-09-11T23:42:02.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.275+0000 7fc26766c640 1 -- 172.21.15.133:0/424060625 wait complete. 2024-09-11T23:42:02.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.276+0000 7fc26766c640 1 Processor -- start 2024-09-11T23:42:02.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.276+0000 7fc26766c640 1 -- start start 2024-09-11T23:42:02.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.277+0000 7fc26766c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc260109450 0x7fc26011a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.277+0000 7fc26766c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc260109830 0x7fc2601137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.277+0000 7fc26766c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc26011b140 0x7fc260113d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.277+0000 7fc26766c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc260120ed0 con 0x7fc260109450 2024-09-11T23:42:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.277+0000 7fc26766c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc260120d50 con 0x7fc260109830 2024-09-11T23:42:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.277+0000 7fc26766c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc260121050 con 0x7fc26011b140 2024-09-11T23:42:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.278+0000 7fc264c07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc260109830 0x7fc2601137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.278+0000 7fc264c07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc260109830 0x7fc2601137c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51554/0 (socket says 172.21.15.133:51554) 2024-09-11T23:42:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.278+0000 7fc264c07640 1 -- 172.21.15.133:0/1395263578 learned_addr learned my addr 172.21.15.133:0/1395263578 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.278+0000 7fc265c09640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc26011b140 0x7fc260113d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.279+0000 7fc264c07640 1 -- 172.21.15.133:0/1395263578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc26011b140 msgr2=0x7fc260113d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.279+0000 7fc264c07640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc26011b140 0x7fc260113d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.279+0000 7fc264c07640 1 -- 172.21.15.133:0/1395263578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc260109450 msgr2=0x7fc26011a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:02.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.279+0000 7fc264c07640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc260109450 0x7fc26011a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.279+0000 7fc264c07640 1 -- 172.21.15.133:0/1395263578 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2601142f0 con 0x7fc260109830 2024-09-11T23:42:02.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.280+0000 7fc264c07640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc260109830 0x7fc2601137c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc258000f80 tx=0x7fc258031730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:02.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.281+0000 7fc2567fc640 1 -- 172.21.15.133:0/1395263578 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258004240 con 0x7fc260109830 2024-09-11T23:42:02.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.287+0000 7fc26766c640 1 -- 172.21.15.133:0/1395263578 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc260114580 con 0x7fc260109830 2024-09-11T23:42:02.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.288+0000 7fc26766c640 1 -- 172.21.15.133:0/1395263578 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc260118570 con 0x7fc260109830 2024-09-11T23:42:02.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.288+0000 7fc2567fc640 1 -- 172.21.15.133:0/1395263578 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc25802d070 con 0x7fc260109830 2024-09-11T23:42:02.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.288+0000 7fc2567fc640 1 -- 172.21.15.133:0/1395263578 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25803b410 con 0x7fc260109830 2024-09-11T23:42:02.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.298+0000 7fc269a45740 1 Processor -- start 2024-09-11T23:42:02.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.298+0000 7fc269a45740 1 -- start start 2024-09-11T23:42:02.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.298+0000 7fc269a45740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e386b60 0x55f96e386f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.298+0000 7fc269a45740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e387480 0x55f96e391800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.298+0000 7fc269a45740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e394130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.298+0000 7fc269a45740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f96e2ec280 con 0x55f96e387480 2024-09-11T23:42:02.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.298+0000 7fc269a45740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f96e293b10 con 0x55f96e391d40 2024-09-11T23:42:02.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.298+0000 7fc269a45740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f96e2e8240 con 0x55f96e386b60 2024-09-11T23:42:02.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.301+0000 7fc2567fc640 1 -- 172.21.15.133:0/1395263578 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc2580043e0 con 0x7fc260109830 2024-09-11T23:42:02.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.306+0000 7fc264c07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e394130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.306+0000 7fc265408640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e387480 0x55f96e391800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.306+0000 7fc264c07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e394130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51566/0 (socket says 172.21.15.133:51566) 2024-09-11T23:42:02.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.306+0000 7fc264c07640 1 -- 172.21.15.133:0/3567872965 learned_addr learned my addr 172.21.15.133:0/3567872965 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:02.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.307+0000 7fc264c07640 1 -- 172.21.15.133:0/3567872965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e386b60 msgr2=0x55f96e386f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:02.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.307+0000 7fc264c07640 1 --2- 172.21.15.133:0/3567872965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e386b60 0x55f96e386f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.307+0000 7fc264c07640 1 -- 172.21.15.133:0/3567872965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e387480 msgr2=0x55f96e391800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:02.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.307+0000 7fc264c07640 1 --2- 172.21.15.133:0/3567872965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e387480 0x55f96e391800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.307+0000 7fc264c07640 1 -- 172.21.15.133:0/3567872965 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96e2a01f0 con 0x55f96e391d40 2024-09-11T23:42:02.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.308+0000 7fc264c07640 1 --2- 172.21.15.133:0/3567872965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e394130 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc2580ee1f0 tx=0x7fc2580f6ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1add0fade2ecef81 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.313+0000 7fc2567fc640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc24c0a2f00 0x7fc24c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.314+0000 7fc2567fc640 1 -- 172.21.15.133:0/1395263578 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc2580f0a40 con 0x7fc260109830 2024-09-11T23:42:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.315+0000 7fc24b7fe640 1 -- 172.21.15.133:0/3567872965 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2580ff070 con 0x55f96e391d40 2024-09-11T23:42:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.316+0000 7fc24b7fe640 1 -- 172.21.15.133:0/3567872965 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc258036030 con 0x55f96e391d40 2024-09-11T23:42:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.316+0000 7fc24b7fe640 1 -- 172.21.15.133:0/3567872965 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2580fa380 con 0x55f96e391d40 2024-09-11T23:42:02.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.319+0000 7fc269a45740 1 -- 172.21.15.133:0/3567872965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 msgr2=0x55f96e394130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:02.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.319+0000 7fc269a45740 1 --2- 172.21.15.133:0/3567872965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e394130 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc2580ee1f0 tx=0x7fc2580f6ad0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.320+0000 7fc269a45740 1 -- 172.21.15.133:0/3567872965 shutdown_connections 2024-09-11T23:42:02.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.320+0000 7fc269a45740 1 --2- 172.21.15.133:0/3567872965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e394130 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.320+0000 7fc269a45740 1 --2- 172.21.15.133:0/3567872965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e387480 0x55f96e391800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.320+0000 7fc269a45740 1 --2- 172.21.15.133:0/3567872965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e386b60 0x55f96e386f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.320+0000 7fc269a45740 1 -- 172.21.15.133:0/3567872965 >> 172.21.15.133:0/3567872965 conn(0x55f96e3830f0 msgr2=0x55f96e3834d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:02.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.322+0000 7fc269a45740 1 -- 172.21.15.133:0/3567872965 shutdown_connections 2024-09-11T23:42:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.322+0000 7fc265c09640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc24c0a2f00 0x7fc24c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.322+0000 7fc269a45740 1 -- 172.21.15.133:0/3567872965 wait complete. 2024-09-11T23:42:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.323+0000 7fc269a45740 1 -- start start 2024-09-11T23:42:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.323+0000 7fc269a45740 1 Processor -- start 2024-09-11T23:42:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.324+0000 7fc269a45740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e386b60 0x55f96e38ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.324+0000 7fc269a45740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e387480 0x55f96e38f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.324+0000 7fc269a45740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e38f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.324+0000 7fc269a45740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f96e3946d0 con 0x55f96e387480 2024-09-11T23:42:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.324+0000 7fc269a45740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f96e2a6f10 con 0x55f96e391d40 2024-09-11T23:42:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.324+0000 7fc269a45740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f96e394850 con 0x55f96e386b60 2024-09-11T23:42:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.324+0000 7fc269a45740 10 client.0 _ll_register_callbacks cb 0x55f96e26fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.325+0000 7fc265c09640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e38f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.325+0000 7fc265c09640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e38f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51580/0 (socket says 172.21.15.133:51580) 2024-09-11T23:42:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.325+0000 7fc265c09640 1 -- 172.21.15.133:0/386736808 learned_addr learned my addr 172.21.15.133:0/386736808 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.328+0000 7fc265408640 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e386b60 0x55f96e38ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.328+0000 7fc265c09640 1 -- 172.21.15.133:0/386736808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e386b60 msgr2=0x55f96e38ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.329+0000 7fc265c09640 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e386b60 0x55f96e38ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.329+0000 7fc265c09640 1 -- 172.21.15.133:0/386736808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e387480 msgr2=0x55f96e38f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.329+0000 7fc265c09640 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e387480 0x55f96e38f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.329+0000 7fc265c09640 1 -- 172.21.15.133:0/386736808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96e05d6a0 con 0x55f96e391d40 2024-09-11T23:42:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.330+0000 7fc265c09640 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e38f680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc250015e80 tx=0x7fc250013650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cad33896b70a150 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.330+0000 7fc248ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.330+0000 7fc248ff9640 1 -- 172.21.15.133:0/386736808 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25000f040 con 0x55f96e391d40 2024-09-11T23:42:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.330+0000 7fc248ff9640 1 -- 172.21.15.133:0/386736808 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc250017c40 con 0x55f96e391d40 2024-09-11T23:42:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.331+0000 7fc269a45740 1 -- 172.21.15.133:0/386736808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc258036030 con 0x55f96e391d40 2024-09-11T23:42:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.331+0000 7fc248ff9640 1 -- 172.21.15.133:0/386736808 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc250017e40 con 0x55f96e391d40 2024-09-11T23:42:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.331+0000 7fc269a45740 10 client.26312 fetch_fsmap learned FSMap version 28 2024-09-11T23:42:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.331+0000 7fc269a45740 1 -- 172.21.15.133:0/386736808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f96e3c22f0 con 0x55f96e391d40 2024-09-11T23:42:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.333+0000 7fc248ff9640 1 -- 172.21.15.133:0/386736808 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc258036030 con 0x55f96e391d40 2024-09-11T23:42:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.333+0000 7fc248ff9640 1 -- 172.21.15.133:0/386736808 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc250022400 con 0x55f96e391d40 2024-09-11T23:42:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.333+0000 7fc269a45740 10 client.26312 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:42:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.333+0000 7fc269a45740 10 client.26312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.333+0000 7fc269a45740 20 client.26312 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.333+0000 7fc269a45740 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f96e3c3850 0x55f96e3c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.333+0000 7fc269a45740 4 client.26312 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:02.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.333+0000 7fc269a45740 1 -- 172.21.15.133:0/386736808 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f96e2f97f0 con 0x55f96e3c3850 2024-09-11T23:42:02.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.334+0000 7fc265c09640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc24c0a2f00 0x7fc24c0a53c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc250006fd0 tx=0x7fc25000d000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:02.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.335+0000 7fc265408640 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f96e3c3850 0x55f96e3c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.338+0000 7fc248ff9640 10 client.26312 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:02.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.338+0000 7fc265408640 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f96e3c3850 0x55f96e3c5d10 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410775e99d822a3 server_cookie=2da8a7a18088cb51 in_seq=0 out_seq=0 2024-09-11T23:42:02.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.342+0000 7fc248ff9640 1 -- 172.21.15.133:0/386736808 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f96e2f97f0 con 0x55f96e3c3850 2024-09-11T23:42:02.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.342+0000 7fc248ff9640 10 client.26312 handle_command_reply: tid=0 2024-09-11T23:42:02.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.343+0000 7fc269a45740 1 client.26312 shutdown 2024-09-11T23:42:02.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.343+0000 7fc269a45740 1 -- 172.21.15.133:0/386736808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 msgr2=0x55f96e38f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:02.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.343+0000 7fc269a45740 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e38f680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc250015e80 tx=0x7fc250013650 comp rx=0 tx=0).stop 2024-09-11T23:42:02.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.343+0000 7fc269a45740 1 -- 172.21.15.133:0/386736808 shutdown_connections 2024-09-11T23:42:02.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.343+0000 7fc269a45740 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f96e3c3850 0x55f96e3c5d10 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.343+0000 7fc269a45740 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e391d40 0x55f96e38f680 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.343+0000 7fc269a45740 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e387480 0x55f96e38f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.343+0000 7fc269a45740 1 --2- 172.21.15.133:0/386736808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e386b60 0x55f96e38ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.343+0000 7fc269a45740 1 -- 172.21.15.133:0/386736808 >> 172.21.15.133:0/386736808 conn(0x55f96e3830f0 msgr2=0x55f96e388e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:02.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.344+0000 7fc248ff9640 0 client.26312 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:02.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.348+0000 7fc269a45740 1 -- 172.21.15.133:0/386736808 shutdown_connections 2024-09-11T23:42:02.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.351+0000 7fc269a45740 1 -- 172.21.15.133:0/386736808 wait complete. 2024-09-11T23:42:02.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.351+0000 7fc269a45740 20 client.26312 trim_cache size 0 max 16384 2024-09-11T23:42:02.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.406+0000 7fc269a45740 1 Processor -- start 2024-09-11T23:42:02.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.406+0000 7fc269a45740 1 -- start start 2024-09-11T23:42:02.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.407+0000 7fc269a45740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e3830f0 0x55f96e3cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.407+0000 7fc269a45740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e386b60 0x55f96e3cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.407+0000 7fc269a45740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e387480 0x55f96e38be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.407+0000 7fc269a45740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc24400e6d0 con 0x55f96e3830f0 2024-09-11T23:42:02.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.407+0000 7fc269a45740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc244010210 con 0x55f96e386b60 2024-09-11T23:42:02.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.407+0000 7fc269a45740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc24400eda0 con 0x55f96e387480 2024-09-11T23:42:02.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.409+0000 7fc264c07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e386b60 0x55f96e3cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.409+0000 7fc264c07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e386b60 0x55f96e3cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51592/0 (socket says 172.21.15.133:51592) 2024-09-11T23:42:02.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.409+0000 7fc264c07640 1 -- 172.21.15.133:0/740923055 learned_addr learned my addr 172.21.15.133:0/740923055 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.411+0000 7fc265408640 1 --2- 172.21.15.133:0/740923055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e3830f0 0x55f96e3cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.411+0000 7fc264c07640 1 -- 172.21.15.133:0/740923055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e387480 msgr2=0x55f96e38be80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.411+0000 7fc264c07640 1 --2- 172.21.15.133:0/740923055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e387480 0x55f96e38be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.411+0000 7fc264c07640 1 -- 172.21.15.133:0/740923055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e3830f0 msgr2=0x55f96e3cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.411+0000 7fc264c07640 1 --2- 172.21.15.133:0/740923055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e3830f0 0x55f96e3cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.411+0000 7fc264c07640 1 -- 172.21.15.133:0/740923055 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96e3872b0 con 0x55f96e386b60 2024-09-11T23:42:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.418+0000 7fc264c07640 1 --2- 172.21.15.133:0/740923055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e386b60 0x55f96e3cbb20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc2580bd010 tx=0x7fc2580f65a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7607a50607a9870c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.422+0000 7fc24affd640 1 -- 172.21.15.133:0/740923055 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2580ff070 con 0x55f96e386b60 2024-09-11T23:42:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.422+0000 7fc24affd640 1 -- 172.21.15.133:0/740923055 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc258036030 con 0x55f96e386b60 2024-09-11T23:42:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.422+0000 7fc24affd640 1 -- 172.21.15.133:0/740923055 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2580bce50 con 0x55f96e386b60 2024-09-11T23:42:02.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.424+0000 7fc269a45740 1 -- 172.21.15.133:0/740923055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e386b60 msgr2=0x55f96e3cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:02.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.424+0000 7fc269a45740 1 --2- 172.21.15.133:0/740923055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e386b60 0x55f96e3cbb20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc2580bd010 tx=0x7fc2580f65a0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.424+0000 7fc269a45740 1 -- 172.21.15.133:0/740923055 shutdown_connections 2024-09-11T23:42:02.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.424+0000 7fc269a45740 1 --2- 172.21.15.133:0/740923055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e387480 0x55f96e38be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.424+0000 7fc269a45740 1 --2- 172.21.15.133:0/740923055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e386b60 0x55f96e3cbb20 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc2580bd010 tx=0x7fc2580f65a0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.424+0000 7fc269a45740 1 --2- 172.21.15.133:0/740923055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e3830f0 0x55f96e3cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.424+0000 7fc269a45740 1 -- 172.21.15.133:0/740923055 >> 172.21.15.133:0/740923055 conn(0x55f96e383b70 msgr2=0x55f96e38dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:02.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.425+0000 7fc269a45740 1 -- 172.21.15.133:0/740923055 shutdown_connections 2024-09-11T23:42:02.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.425+0000 7fc269a45740 1 -- 172.21.15.133:0/740923055 wait complete. 2024-09-11T23:42:02.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.426+0000 7fc269a45740 1 -- start start 2024-09-11T23:42:02.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.426+0000 7fc269a45740 1 Processor -- start 2024-09-11T23:42:02.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.434+0000 7fc269a45740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e3830f0 0x55f96e399fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.434+0000 7fc269a45740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e387480 0x55f96e396a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.434+0000 7fc269a45740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e391d40 0x55f96e396fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.434+0000 7fc269a45740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc24400fb40 con 0x55f96e3830f0 2024-09-11T23:42:02.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.434+0000 7fc269a45740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc24400f470 con 0x55f96e387480 2024-09-11T23:42:02.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.434+0000 7fc269a45740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc24400c6b0 con 0x55f96e391d40 2024-09-11T23:42:02.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.434+0000 7fc269a45740 10 client.0 _ll_register_callbacks cb 0x7fc260110870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:02.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.435+0000 7fc264c07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e387480 0x55f96e396a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.436+0000 7fc265408640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e3830f0 0x55f96e399fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.436+0000 7fc265408640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e3830f0 0x55f96e399fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42632/0 (socket says 172.21.15.133:42632) 2024-09-11T23:42:02.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.436+0000 7fc265408640 1 -- 172.21.15.133:0/2041679897 learned_addr learned my addr 172.21.15.133:0/2041679897 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:02.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.436+0000 7fc265c09640 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e391d40 0x55f96e396fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.439+0000 7fc265c09640 1 -- 172.21.15.133:0/2041679897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e387480 msgr2=0x55f96e396a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:02.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.439+0000 7fc265c09640 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e387480 0x55f96e396a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.439+0000 7fc265c09640 1 -- 172.21.15.133:0/2041679897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e3830f0 msgr2=0x55f96e399fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:02.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.439+0000 7fc265c09640 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e3830f0 0x55f96e399fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.439+0000 7fc265c09640 1 -- 172.21.15.133:0/2041679897 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96e397570 con 0x55f96e391d40 2024-09-11T23:42:02.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.441+0000 7fc265c09640 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e391d40 0x55f96e396fb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc2500228a0 tx=0x7fc250015d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1492aba7803cabac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:02.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.441+0000 7fc248ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:42:02.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.442+0000 7fc248ff9640 1 -- 172.21.15.133:0/2041679897 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250006d10 con 0x55f96e391d40 2024-09-11T23:42:02.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.442+0000 7fc248ff9640 1 -- 172.21.15.133:0/2041679897 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fc250029e00 con 0x55f96e391d40 2024-09-11T23:42:02.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.442+0000 7fc269a45740 1 -- 172.21.15.133:0/2041679897 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc258036030 con 0x55f96e391d40 2024-09-11T23:42:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.607+0000 7fc248ff9640 1 -- 172.21.15.133:0/2041679897 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc2500225f0 con 0x55f96e391d40 2024-09-11T23:42:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.607+0000 7fc269a45740 10 client.26281 fetch_fsmap learned FSMap version 28 2024-09-11T23:42:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.607+0000 7fc269a45740 1 -- 172.21.15.133:0/2041679897 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f96e42f3b0 con 0x55f96e391d40 2024-09-11T23:42:02.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.608+0000 7fc248ff9640 1 -- 172.21.15.133:0/2041679897 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc258036030 con 0x55f96e391d40 2024-09-11T23:42:02.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.608+0000 7fc248ff9640 1 -- 172.21.15.133:0/2041679897 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc250003860 con 0x55f96e391d40 2024-09-11T23:42:02.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.608+0000 7fc269a45740 10 client.26281 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:42:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.608+0000 7fc269a45740 10 client.26281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.608+0000 7fc269a45740 20 client.26281 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.608+0000 7fc269a45740 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f96e42fb30 0x55f96e431f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.608+0000 7fc269a45740 4 client.26281 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:42:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.608+0000 7fc269a45740 1 -- 172.21.15.133:0/2041679897 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f96e2f3d00 con 0x55f96e42fb30 2024-09-11T23:42:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.609+0000 7fc265408640 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f96e42fb30 0x55f96e431f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.610+0000 7fc248ff9640 10 client.26281 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.610+0000 7fc265408640 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f96e42fb30 0x55f96e431f50 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3a1fa05afc6d1b8 server_cookie=2c40c93ef1b619e7 in_seq=0 out_seq=0 2024-09-11T23:42:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.615+0000 7fc248ff9640 1 -- 172.21.15.133:0/2041679897 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f96e2f3d00 con 0x55f96e42fb30 2024-09-11T23:42:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.615+0000 7fc248ff9640 10 client.26281 handle_command_reply: tid=0 2024-09-11T23:42:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.615+0000 7fc269a45740 1 client.26281 shutdown 2024-09-11T23:42:02.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:02 smithi133 ceph-mon[31430]: pgmap v160: 97 pgs: 97 active+clean; 40 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 635 MiB/s wr, 1.69k op/s 2024-09-11T23:42:02.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:02 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:42:02.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:02 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:42:02.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:02 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:42:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.616+0000 7fc269a45740 1 -- 172.21.15.133:0/2041679897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e391d40 msgr2=0x55f96e396fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.616+0000 7fc269a45740 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e391d40 0x55f96e396fb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc2500228a0 tx=0x7fc250015d60 comp rx=0 tx=0).stop 2024-09-11T23:42:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.616+0000 7fc269a45740 1 -- 172.21.15.133:0/2041679897 shutdown_connections 2024-09-11T23:42:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.616+0000 7fc269a45740 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f96e42fb30 0x55f96e431f50 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.616+0000 7fc269a45740 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f96e391d40 0x55f96e396fb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.616+0000 7fc269a45740 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f96e387480 0x55f96e396a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.616+0000 7fc269a45740 1 --2- 172.21.15.133:0/2041679897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f96e3830f0 0x55f96e399fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.616+0000 7fc269a45740 1 -- 172.21.15.133:0/2041679897 >> 172.21.15.133:0/2041679897 conn(0x55f96e383b70 msgr2=0x55f96e388600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.616+0000 7fc248ff9640 0 client.26281 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.616+0000 7fc269a45740 1 -- 172.21.15.133:0/2041679897 shutdown_connections 2024-09-11T23:42:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.617+0000 7fc269a45740 1 -- 172.21.15.133:0/2041679897 wait complete. 2024-09-11T23:42:02.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.617+0000 7fc269a45740 20 client.26281 trim_cache size 0 max 16384 2024-09-11T23:42:02.620 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:42:02.620 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:42:02.620 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "32bf47be-3ef5-4f30-9bdb-9f0f38f575aa", 2024-09-11T23:42:02.620 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:42:02.620 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:42:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.618+0000 7fc26766c640 1 -- 172.21.15.133:0/1395263578 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc24c0a2f00 msgr2=0x7fc24c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.618+0000 7fc26766c640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc24c0a2f00 0x7fc24c0a53c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc250006fd0 tx=0x7fc25000d000 comp rx=0 tx=0).stop 2024-09-11T23:42:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.618+0000 7fc26766c640 1 -- 172.21.15.133:0/1395263578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc260109830 msgr2=0x7fc2601137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.618+0000 7fc26766c640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc260109830 0x7fc2601137c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc258000f80 tx=0x7fc258031730 comp rx=0 tx=0).stop 2024-09-11T23:42:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.618+0000 7fc26766c640 1 -- 172.21.15.133:0/1395263578 shutdown_connections 2024-09-11T23:42:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.618+0000 7fc26766c640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc24c0a2f00 0x7fc24c0a53c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.618+0000 7fc26766c640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc26011b140 0x7fc260113d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.618+0000 7fc26766c640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc260109830 0x7fc2601137c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.618+0000 7fc26766c640 1 --2- 172.21.15.133:0/1395263578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc260109450 0x7fc26011a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.618+0000 7fc26766c640 1 -- 172.21.15.133:0/1395263578 >> 172.21.15.133:0/1395263578 conn(0x7fc26006e140 msgr2=0x7fc2600723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:02.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.619+0000 7fc26766c640 1 -- 172.21.15.133:0/1395263578 shutdown_connections 2024-09-11T23:42:02.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:02.619+0000 7fc26766c640 1 -- 172.21.15.133:0/1395263578 wait complete. 2024-09-11T23:42:02.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:02 smithi178 ceph-mon[35882]: pgmap v160: 97 pgs: 97 active+clean; 40 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 635 MiB/s wr, 1.69k op/s 2024-09-11T23:42:02.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:02 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:42:02.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:02 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:42:02.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:02 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:42:02.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:02 smithi138 ceph-mon[35142]: pgmap v160: 97 pgs: 97 active+clean; 40 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 635 MiB/s wr, 1.69k op/s 2024-09-11T23:42:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:02 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:42:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:02 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:42:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:02 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:42:03.691 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:42:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:03 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:42:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:03 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:03 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:42:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:03 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:03 smithi178 ceph-mon[35882]: pgmap v161: 97 pgs: 97 active+clean; 40 GiB data, 139 GiB used, 934 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 750 MiB/s wr, 1.90k op/s 2024-09-11T23:42:03.837 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:03 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:42:03.837 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:03 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:03.837 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:03 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:42:03.837 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:03 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:03.837 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:03 smithi133 ceph-mon[31430]: pgmap v161: 97 pgs: 97 active+clean; 40 GiB data, 139 GiB used, 934 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 750 MiB/s wr, 1.90k op/s 2024-09-11T23:42:03.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:03 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:42:03.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:03 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:03 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:42:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:03 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:03 smithi138 ceph-mon[35142]: pgmap v161: 97 pgs: 97 active+clean; 40 GiB data, 139 GiB used, 934 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 750 MiB/s wr, 1.90k op/s 2024-09-11T23:42:04.288 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:04 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:42:05.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.182+0000 7fe428361640 1 -- 172.21.15.133:0/3936426908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe42011b140 msgr2=0x7fe42011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:05.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.182+0000 7fe428361640 1 --2- 172.21.15.133:0/3936426908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe42011b140 0x7fe42011d530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009900 tx=0x7fe41c02a440 comp rx=0 tx=0).stop 2024-09-11T23:42:05.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.184+0000 7fe428361640 1 -- 172.21.15.133:0/3936426908 shutdown_connections 2024-09-11T23:42:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.184+0000 7fe428361640 1 --2- 172.21.15.133:0/3936426908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe42011b140 0x7fe42011d530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.185+0000 7fe428361640 1 --2- 172.21.15.133:0/3936426908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe420109830 0x7fe420109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.185+0000 7fe428361640 1 --2- 172.21.15.133:0/3936426908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420109450 0x7fe420110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.185+0000 7fe428361640 1 -- 172.21.15.133:0/3936426908 >> 172.21.15.133:0/3936426908 conn(0x7fe42006e140 msgr2=0x7fe42006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:05.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.189+0000 7fe428361640 1 -- 172.21.15.133:0/3936426908 shutdown_connections 2024-09-11T23:42:05.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.191+0000 7fe428361640 1 -- 172.21.15.133:0/3936426908 wait complete. 2024-09-11T23:42:05.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.192+0000 7fe428361640 1 Processor -- start 2024-09-11T23:42:05.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.193+0000 7fe428361640 1 -- start start 2024-09-11T23:42:05.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.193+0000 7fe428361640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420109450 0x7fe4201186d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.193+0000 7fe428361640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe420109830 0x7fe420111760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.193+0000 7fe428361640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42011b140 0x7fe420111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.193+0000 7fe428361640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe420120ed0 con 0x7fe42011b140 2024-09-11T23:42:05.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.193+0000 7fe428361640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe420120d50 con 0x7fe420109450 2024-09-11T23:42:05.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.193+0000 7fe428361640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe420121050 con 0x7fe420109830 2024-09-11T23:42:05.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.194+0000 7fe4268fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42011b140 0x7fe420111ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.194+0000 7fe4268fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42011b140 0x7fe420111ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42638/0 (socket says 172.21.15.133:42638) 2024-09-11T23:42:05.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.194+0000 7fe4268fe640 1 -- 172.21.15.133:0/105816298 learned_addr learned my addr 172.21.15.133:0/105816298 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:05.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.194+0000 7fe4260fd640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420109450 0x7fe4201186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.194+0000 7fe4258fc640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe420109830 0x7fe420111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.195+0000 7fe4268fe640 1 -- 172.21.15.133:0/105816298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe420109830 msgr2=0x7fe420111760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:05.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.195+0000 7fe4268fe640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe420109830 0x7fe420111760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.195+0000 7fe4268fe640 1 -- 172.21.15.133:0/105816298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420109450 msgr2=0x7fe4201186d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:05.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.195+0000 7fe4268fe640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420109450 0x7fe4201186d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.195+0000 7fe4268fe640 1 -- 172.21.15.133:0/105816298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe420112260 con 0x7fe42011b140 2024-09-11T23:42:05.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:04 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:42:05.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.195+0000 7fe4268fe640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42011b140 0x7fe420111ca0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe41c004650 tx=0x7fe41c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:05.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.196+0000 7fe4177fe640 1 -- 172.21.15.133:0/105816298 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c03f030 con 0x7fe42011b140 2024-09-11T23:42:05.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.196+0000 7fe428361640 1 -- 172.21.15.133:0/105816298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4201124f0 con 0x7fe42011b140 2024-09-11T23:42:05.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.196+0000 7fe428361640 1 -- 172.21.15.133:0/105816298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4201164e0 con 0x7fe42011b140 2024-09-11T23:42:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.198+0000 7fe4177fe640 1 -- 172.21.15.133:0/105816298 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41c003ad0 con 0x7fe42011b140 2024-09-11T23:42:05.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.199+0000 7fe4177fe640 1 -- 172.21.15.133:0/105816298 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c035070 con 0x7fe42011b140 2024-09-11T23:42:05.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.201+0000 7fe42a73a740 1 Processor -- start 2024-09-11T23:42:05.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.203+0000 7fe4177fe640 1 -- 172.21.15.133:0/105816298 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe41c042030 con 0x7fe42011b140 2024-09-11T23:42:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.203+0000 7fe42a73a740 1 -- start start 2024-09-11T23:42:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.204+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d127204a0 0x558d12720880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.204+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d12720dc0 0x558d1272b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.204+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d1272b680 0x558d1272da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.204+0000 7fe42a73a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558d126862e0 con 0x558d12720dc0 2024-09-11T23:42:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.204+0000 7fe42a73a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d1262dce0 con 0x558d127204a0 2024-09-11T23:42:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.204+0000 7fe42a73a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558d126822a0 con 0x558d1272b680 2024-09-11T23:42:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.205+0000 7fe4258fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d1272b680 0x558d1272da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.205+0000 7fe4258fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d1272b680 0x558d1272da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:51626/0 (socket says 172.21.15.133:51626) 2024-09-11T23:42:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.205+0000 7fe4258fc640 1 -- 172.21.15.133:0/1138222540 learned_addr learned my addr 172.21.15.133:0/1138222540 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.205+0000 7fe4258fc640 1 -- 172.21.15.133:0/1138222540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d127204a0 msgr2=0x558d12720880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.205+0000 7fe4258fc640 1 --2- 172.21.15.133:0/1138222540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d127204a0 0x558d12720880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.205+0000 7fe4258fc640 1 -- 172.21.15.133:0/1138222540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d12720dc0 msgr2=0x558d1272b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.206+0000 7fe4258fc640 1 --2- 172.21.15.133:0/1138222540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d12720dc0 0x558d1272b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.206+0000 7fe4258fc640 1 -- 172.21.15.133:0/1138222540 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d12612440 con 0x558d1272b680 2024-09-11T23:42:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.206+0000 7fe4258fc640 1 --2- 172.21.15.133:0/1138222540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d1272b680 0x558d1272da70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe410009a80 tx=0x7fe410009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78ee6a46754e902b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.206+0000 7fe4177fe640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe40c0a2d70 0x7fe40c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.207+0000 7fe414dea640 1 -- 172.21.15.133:0/1138222540 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410017070 con 0x558d1272b680 2024-09-11T23:42:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.207+0000 7fe414dea640 1 -- 172.21.15.133:0/1138222540 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe410010c20 con 0x558d1272b680 2024-09-11T23:42:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.207+0000 7fe414dea640 1 -- 172.21.15.133:0/1138222540 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41001a5c0 con 0x558d1272b680 2024-09-11T23:42:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.207+0000 7fe4177fe640 1 -- 172.21.15.133:0/105816298 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe41c0ef4c0 con 0x7fe42011b140 2024-09-11T23:42:05.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.208+0000 7fe42a73a740 1 -- 172.21.15.133:0/1138222540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d1272b680 msgr2=0x558d1272da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:05.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.208+0000 7fe42a73a740 1 --2- 172.21.15.133:0/1138222540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d1272b680 0x558d1272da70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe410009a80 tx=0x7fe410009f40 comp rx=0 tx=0).stop 2024-09-11T23:42:05.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.209+0000 7fe4260fd640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe40c0a2d70 0x7fe40c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.210+0000 7fe4260fd640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe40c0a2d70 0x7fe40c0a5230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x558d1271d890 tx=0x7fe418009550 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.211+0000 7fe42a73a740 1 -- 172.21.15.133:0/1138222540 shutdown_connections 2024-09-11T23:42:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.211+0000 7fe42a73a740 1 --2- 172.21.15.133:0/1138222540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d1272b680 0x558d1272da70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.211+0000 7fe42a73a740 1 --2- 172.21.15.133:0/1138222540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d12720dc0 0x558d1272b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.211+0000 7fe42a73a740 1 --2- 172.21.15.133:0/1138222540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d127204a0 0x558d12720880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.211+0000 7fe42a73a740 1 -- 172.21.15.133:0/1138222540 >> 172.21.15.133:0/1138222540 conn(0x558d1271ca30 msgr2=0x558d1271ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.211+0000 7fe42a73a740 1 -- 172.21.15.133:0/1138222540 shutdown_connections 2024-09-11T23:42:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.211+0000 7fe42a73a740 1 -- 172.21.15.133:0/1138222540 wait complete. 2024-09-11T23:42:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.212+0000 7fe42a73a740 1 -- start start 2024-09-11T23:42:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.212+0000 7fe42a73a740 1 Processor -- start 2024-09-11T23:42:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.213+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d127204a0 0x558d1272af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.213+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d12720dc0 0x558d12728210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.213+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d1272b680 0x558d12728750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.213+0000 7fe42a73a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558d1272e250 con 0x558d1272b680 2024-09-11T23:42:05.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.213+0000 7fe42a73a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d12641050 con 0x558d12720dc0 2024-09-11T23:42:05.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.213+0000 7fe42a73a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558d1272e3d0 con 0x558d127204a0 2024-09-11T23:42:05.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.213+0000 7fe42a73a740 10 client.0 _ll_register_callbacks cb 0x558d12681160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:05.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.213+0000 7fe4258fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d12720dc0 0x558d12728210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.213+0000 7fe4258fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d12720dc0 0x558d12728210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51632/0 (socket says 172.21.15.133:51632) 2024-09-11T23:42:05.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.213+0000 7fe4258fc640 1 -- 172.21.15.133:0/3864467981 learned_addr learned my addr 172.21.15.133:0/3864467981 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.214+0000 7fe4268fe640 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d1272b680 0x558d12728750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.214+0000 7fe4258fc640 1 -- 172.21.15.133:0/3864467981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d127204a0 msgr2=0x558d1272af60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.214+0000 7fe4258fc640 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d127204a0 0x558d1272af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.214+0000 7fe4258fc640 1 -- 172.21.15.133:0/3864467981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d1272b680 msgr2=0x558d12728750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.214+0000 7fe4258fc640 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d1272b680 0x558d12728750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.214+0000 7fe4258fc640 1 -- 172.21.15.133:0/3864467981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d123f73c0 con 0x558d12720dc0 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.215+0000 7fe4258fc640 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d12720dc0 0x558d12728210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe41000a010 tx=0x7fe410013480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f5d4255a9fe116 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.215+0000 7fe409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.215+0000 7fe409ffb640 1 -- 172.21.15.133:0/3864467981 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410017070 con 0x558d12720dc0 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.216+0000 7fe42a73a740 1 -- 172.21.15.133:0/3864467981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe410010c20 con 0x558d12720dc0 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.216+0000 7fe409ffb640 1 -- 172.21.15.133:0/3864467981 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe41001c650 con 0x558d12720dc0 2024-09-11T23:42:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.216+0000 7fe409ffb640 1 -- 172.21.15.133:0/3864467981 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe41001c850 con 0x558d12720dc0 2024-09-11T23:42:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.216+0000 7fe42a73a740 10 client.26324 fetch_fsmap learned FSMap version 28 2024-09-11T23:42:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.216+0000 7fe42a73a740 1 -- 172.21.15.133:0/3864467981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d12739c50 con 0x558d12720dc0 2024-09-11T23:42:05.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.218+0000 7fe409ffb640 1 -- 172.21.15.133:0/3864467981 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe410010c20 con 0x558d12720dc0 2024-09-11T23:42:05.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.219+0000 7fe409ffb640 1 -- 172.21.15.133:0/3864467981 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe41002d020 con 0x558d12720dc0 2024-09-11T23:42:05.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.220+0000 7fe42a73a740 10 client.26324 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:42:05.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.220+0000 7fe42a73a740 10 client.26324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:05.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.220+0000 7fe42a73a740 20 client.26324 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:05.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.220+0000 7fe42a73a740 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d1273b310 0x558d1273d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.220+0000 7fe42a73a740 4 client.26324 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:05.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.220+0000 7fe42a73a740 1 -- 172.21.15.133:0/3864467981 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d12693850 con 0x558d1273b310 2024-09-11T23:42:05.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.221+0000 7fe4260fd640 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d1273b310 0x558d1273d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.222+0000 7fe4260fd640 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d1273b310 0x558d1273d790 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1a34a2f48509bb server_cookie=4851ae8fb3f134c2 in_seq=0 out_seq=0 2024-09-11T23:42:05.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.222+0000 7fe409ffb640 10 client.26324 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:05.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.224+0000 7fe409ffb640 1 -- 172.21.15.133:0/3864467981 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d12693850 con 0x558d1273b310 2024-09-11T23:42:05.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.224+0000 7fe409ffb640 10 client.26324 handle_command_reply: tid=0 2024-09-11T23:42:05.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.226+0000 7fe42a73a740 1 client.26324 shutdown 2024-09-11T23:42:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.231+0000 7fe42a73a740 1 -- 172.21.15.133:0/3864467981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d12720dc0 msgr2=0x558d12728210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.231+0000 7fe42a73a740 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d12720dc0 0x558d12728210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe41000a010 tx=0x7fe410013480 comp rx=0 tx=0).stop 2024-09-11T23:42:05.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.235+0000 7fe42a73a740 1 -- 172.21.15.133:0/3864467981 shutdown_connections 2024-09-11T23:42:05.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.235+0000 7fe42a73a740 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d1273b310 0x558d1273d790 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.235+0000 7fe42a73a740 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d1272b680 0x558d12728750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.235+0000 7fe42a73a740 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d12720dc0 0x558d12728210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.235+0000 7fe42a73a740 1 --2- 172.21.15.133:0/3864467981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d127204a0 0x558d1272af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.235+0000 7fe42a73a740 1 -- 172.21.15.133:0/3864467981 >> 172.21.15.133:0/3864467981 conn(0x558d1271ca30 msgr2=0x558d12722760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:05.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.235+0000 7fe409ffb640 0 client.26324 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.236+0000 7fe42a73a740 1 -- 172.21.15.133:0/3864467981 shutdown_connections 2024-09-11T23:42:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.236+0000 7fe42a73a740 1 -- 172.21.15.133:0/3864467981 wait complete. 2024-09-11T23:42:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.236+0000 7fe42a73a740 20 client.26324 trim_cache size 0 max 16384 2024-09-11T23:42:05.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.276+0000 7fe42a73a740 1 Processor -- start 2024-09-11T23:42:05.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.278+0000 7fe42a73a740 1 -- start start 2024-09-11T23:42:05.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.278+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d127204a0 0x558d12737540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.278+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d12720dc0 0x558d1271ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.278+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d1272b680 0x558d127237a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.278+0000 7fe42a73a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe40400e550 con 0x558d127204a0 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.278+0000 7fe42a73a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe404010090 con 0x558d1272b680 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.279+0000 7fe42a73a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe40400ec20 con 0x558d12720dc0 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.285+0000 7fe4268fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d1272b680 0x558d127237a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.285+0000 7fe4268fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d1272b680 0x558d127237a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51634/0 (socket says 172.21.15.133:51634) 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.285+0000 7fe4268fe640 1 -- 172.21.15.133:0/1206487909 learned_addr learned my addr 172.21.15.133:0/1206487909 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.285+0000 7fe4268fe640 1 -- 172.21.15.133:0/1206487909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d12720dc0 msgr2=0x558d1271ce10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.285+0000 7fe4268fe640 1 --2- 172.21.15.133:0/1206487909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d12720dc0 0x558d1271ce10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.285+0000 7fe4268fe640 1 -- 172.21.15.133:0/1206487909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d127204a0 msgr2=0x558d12737540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.285+0000 7fe4268fe640 1 --2- 172.21.15.133:0/1206487909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d127204a0 0x558d12737540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.286+0000 7fe4268fe640 1 -- 172.21.15.133:0/1206487909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d127271e0 con 0x558d1272b680 2024-09-11T23:42:05.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.287+0000 7fe4268fe640 1 --2- 172.21.15.133:0/1206487909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d1272b680 0x558d127237a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0317e0 tx=0x7fe41c031ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc22f7ba46e3e02d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:05.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:04 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:42:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.293+0000 7fe40bfff640 1 -- 172.21.15.133:0/1206487909 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c0d3020 con 0x558d1272b680 2024-09-11T23:42:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.293+0000 7fe40bfff640 1 -- 172.21.15.133:0/1206487909 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41c0bba00 con 0x558d1272b680 2024-09-11T23:42:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.293+0000 7fe40bfff640 1 -- 172.21.15.133:0/1206487909 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c003c80 con 0x558d1272b680 2024-09-11T23:42:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.294+0000 7fe42a73a740 1 -- 172.21.15.133:0/1206487909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d1272b680 msgr2=0x558d127237a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.294+0000 7fe42a73a740 1 --2- 172.21.15.133:0/1206487909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d1272b680 0x558d127237a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0317e0 tx=0x7fe41c031ba0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.294+0000 7fe42a73a740 1 -- 172.21.15.133:0/1206487909 shutdown_connections 2024-09-11T23:42:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.294+0000 7fe42a73a740 1 --2- 172.21.15.133:0/1206487909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d1272b680 0x558d127237a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.294+0000 7fe42a73a740 1 --2- 172.21.15.133:0/1206487909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d12720dc0 0x558d1271ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.294+0000 7fe42a73a740 1 --2- 172.21.15.133:0/1206487909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d127204a0 0x558d12737540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.294+0000 7fe42a73a740 1 -- 172.21.15.133:0/1206487909 >> 172.21.15.133:0/1206487909 conn(0x558d1271ca30 msgr2=0x558d12723090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:05.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.295+0000 7fe42a73a740 1 -- 172.21.15.133:0/1206487909 shutdown_connections 2024-09-11T23:42:05.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.295+0000 7fe42a73a740 1 -- 172.21.15.133:0/1206487909 wait complete. 2024-09-11T23:42:05.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.296+0000 7fe42a73a740 1 -- start start 2024-09-11T23:42:05.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.296+0000 7fe42a73a740 1 Processor -- start 2024-09-11T23:42:05.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.297+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d127204a0 0x558d12733890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.297+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d12720dc0 0x558d12733dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.297+0000 7fe42a73a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d1272b680 0x558d12724a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.297+0000 7fe42a73a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe40400f9c0 con 0x558d1272b680 2024-09-11T23:42:05.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.298+0000 7fe42a73a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe40400f2f0 con 0x558d127204a0 2024-09-11T23:42:05.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.298+0000 7fe42a73a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe40400c5e0 con 0x558d12720dc0 2024-09-11T23:42:05.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.298+0000 7fe42a73a740 10 client.0 _ll_register_callbacks cb 0x7fe410013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:05.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.298+0000 7fe4260fd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d127204a0 0x558d12733890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.298+0000 7fe4260fd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d127204a0 0x558d12733890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51650/0 (socket says 172.21.15.133:51650) 2024-09-11T23:42:05.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.298+0000 7fe4260fd640 1 -- 172.21.15.133:0/915537387 learned_addr learned my addr 172.21.15.133:0/915537387 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:05.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.299+0000 7fe4268fe640 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d1272b680 0x558d12724a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.299+0000 7fe4258fc640 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d12720dc0 0x558d12733dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.299+0000 7fe4268fe640 1 -- 172.21.15.133:0/915537387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d12720dc0 msgr2=0x558d12733dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:05.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.299+0000 7fe4268fe640 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d12720dc0 0x558d12733dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.299+0000 7fe4268fe640 1 -- 172.21.15.133:0/915537387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d127204a0 msgr2=0x558d12733890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.299+0000 7fe4268fe640 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d127204a0 0x558d12733890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.299+0000 7fe4268fe640 1 -- 172.21.15.133:0/915537387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1272b2d0 con 0x558d1272b680 2024-09-11T23:42:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.300+0000 7fe4268fe640 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d1272b680 0x558d12724a40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0317e0 tx=0x7fe41c02a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f523933410a5594 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.300+0000 7fe409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:42:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.301+0000 7fe409ffb640 1 -- 172.21.15.133:0/915537387 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c09e070 con 0x558d1272b680 2024-09-11T23:42:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.301+0000 7fe42a73a740 1 -- 172.21.15.133:0/915537387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe41c0bba00 con 0x558d1272b680 2024-09-11T23:42:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.301+0000 7fe409ffb640 1 -- 172.21.15.133:0/915537387 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7fe41c003c80 con 0x558d1272b680 2024-09-11T23:42:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.301+0000 7fe409ffb640 1 -- 172.21.15.133:0/915537387 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe41c0da050 con 0x558d1272b680 2024-09-11T23:42:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.302+0000 7fe42a73a740 10 client.16884 fetch_fsmap learned FSMap version 28 2024-09-11T23:42:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.302+0000 7fe42a73a740 1 -- 172.21.15.133:0/915537387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d127a5950 con 0x558d1272b680 2024-09-11T23:42:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.309+0000 7fe409ffb640 1 -- 172.21.15.133:0/915537387 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41c0c4670 con 0x558d1272b680 2024-09-11T23:42:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.309+0000 7fe409ffb640 1 -- 172.21.15.133:0/915537387 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe41c0bae50 con 0x558d1272b680 2024-09-11T23:42:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.310+0000 7fe42a73a740 10 client.16884 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:42:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.310+0000 7fe42a73a740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.310+0000 7fe42a73a740 20 client.16884 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.311+0000 7fe42a73a740 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d127a5e90 0x558d127a8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.311+0000 7fe42a73a740 4 client.16884 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:42:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.311+0000 7fe42a73a740 1 -- 172.21.15.133:0/915537387 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe41c0bba00 con 0x558d127a5e90 2024-09-11T23:42:05.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.312+0000 7fe4260fd640 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d127a5e90 0x558d127a8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:05.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.313+0000 7fe4260fd640 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d127a5e90 0x558d127a8280 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f3e733e5d2c90e server_cookie=bf2e44ad708773f2 in_seq=0 out_seq=0 2024-09-11T23:42:05.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.314+0000 7fe409ffb640 10 client.16884 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:05.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.314+0000 7fe409ffb640 1 -- 172.21.15.133:0/915537387 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fe41c0bba00 con 0x558d127a5e90 2024-09-11T23:42:05.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.314+0000 7fe409ffb640 10 client.16884 handle_command_reply: tid=0 2024-09-11T23:42:05.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.314+0000 7fe42a73a740 1 client.16884 shutdown 2024-09-11T23:42:05.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.316+0000 7fe42a73a740 1 -- 172.21.15.133:0/915537387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d1272b680 msgr2=0x558d12724a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:05.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.316+0000 7fe42a73a740 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d1272b680 0x558d12724a40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0317e0 tx=0x7fe41c02a440 comp rx=0 tx=0).stop 2024-09-11T23:42:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.317+0000 7fe42a73a740 1 -- 172.21.15.133:0/915537387 shutdown_connections 2024-09-11T23:42:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.317+0000 7fe42a73a740 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d127a5e90 0x558d127a8280 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.317+0000 7fe409ffb640 0 client.16884 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.317+0000 7fe42a73a740 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d1272b680 0x558d12724a40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.318+0000 7fe42a73a740 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d12720dc0 0x558d12733dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.318+0000 7fe42a73a740 1 --2- 172.21.15.133:0/915537387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d127204a0 0x558d12733890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.318+0000 7fe42a73a740 1 -- 172.21.15.133:0/915537387 >> 172.21.15.133:0/915537387 conn(0x558d1271ca30 msgr2=0x558d127238b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.318+0000 7fe42a73a740 1 -- 172.21.15.133:0/915537387 shutdown_connections 2024-09-11T23:42:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.318+0000 7fe42a73a740 1 -- 172.21.15.133:0/915537387 wait complete. 2024-09-11T23:42:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.318+0000 7fe42a73a740 20 client.16884 trim_cache size 0 max 16384 2024-09-11T23:42:05.320 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:42:05.320 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:42:05.320 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:42:05.320 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:42:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 -- 172.21.15.133:0/105816298 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe40c0a2d70 msgr2=0x7fe40c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe40c0a2d70 0x7fe40c0a5230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x558d1271d890 tx=0x7fe418009550 comp rx=0 tx=0).stop 2024-09-11T23:42:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 -- 172.21.15.133:0/105816298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42011b140 msgr2=0x7fe420111ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42011b140 0x7fe420111ca0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe41c004650 tx=0x7fe41c005e80 comp rx=0 tx=0).stop 2024-09-11T23:42:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 -- 172.21.15.133:0/105816298 shutdown_connections 2024-09-11T23:42:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe40c0a2d70 0x7fe40c0a5230 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42011b140 0x7fe420111ca0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe420109830 0x7fe420111760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 --2- 172.21.15.133:0/105816298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe420109450 0x7fe4201186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 -- 172.21.15.133:0/105816298 >> 172.21.15.133:0/105816298 conn(0x7fe42006e140 msgr2=0x7fe42011bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 -- 172.21.15.133:0/105816298 shutdown_connections 2024-09-11T23:42:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:05.323+0000 7fe428361640 1 -- 172.21.15.133:0/105816298 wait complete. 2024-09-11T23:42:06.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:05 smithi133 ceph-mon[31430]: pgmap v162: 97 pgs: 97 active+clean; 40 GiB data, 142 GiB used, 931 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 747 MiB/s wr, 1.81k op/s 2024-09-11T23:42:06.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:05 smithi178 ceph-mon[35882]: pgmap v162: 97 pgs: 97 active+clean; 40 GiB data, 142 GiB used, 931 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 747 MiB/s wr, 1.81k op/s 2024-09-11T23:42:06.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:05 smithi138 ceph-mon[35142]: pgmap v162: 97 pgs: 97 active+clean; 40 GiB data, 142 GiB used, 931 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 747 MiB/s wr, 1.81k op/s 2024-09-11T23:42:06.465 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:42:06.466 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:42:07.255 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:08.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.274+0000 7faffffff640 1 -- 172.21.15.133:0/1830288393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00006ea30 msgr2=0x7fb00006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:08.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.274+0000 7faffffff640 1 --2- 172.21.15.133:0/1830288393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00006ea30 0x7fb00006ee70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7faff0009900 tx=0x7faff002a410 comp rx=0 tx=0).stop 2024-09-11T23:42:08.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.276+0000 7faffffff640 1 -- 172.21.15.133:0/1830288393 shutdown_connections 2024-09-11T23:42:08.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.276+0000 7faffffff640 1 --2- 172.21.15.133:0/1830288393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00011b140 0x7fb00011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:08.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.276+0000 7faffffff640 1 --2- 172.21.15.133:0/1830288393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00006ea30 0x7fb00006ee70 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:08.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.276+0000 7faffffff640 1 --2- 172.21.15.133:0/1830288393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb000110190 0x7fb00006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:08.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.276+0000 7faffffff640 1 -- 172.21.15.133:0/1830288393 >> 172.21.15.133:0/1830288393 conn(0x7fb000073180 msgr2=0x7fb000073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:08.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.281+0000 7faffffff640 1 -- 172.21.15.133:0/1830288393 shutdown_connections 2024-09-11T23:42:08.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.282+0000 7faffffff640 1 -- 172.21.15.133:0/1830288393 wait complete. 2024-09-11T23:42:08.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.284+0000 7faffffff640 1 Processor -- start 2024-09-11T23:42:08.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.285+0000 7faffffff640 1 -- start start 2024-09-11T23:42:08.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.285+0000 7faffffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00006ea30 0x7fb00011a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:08.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.285+0000 7faffffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb000110190 0x7fb0001149c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:08.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.285+0000 7faffffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00011b140 0x7fb000114f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.286+0000 7faffffff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb000111330 con 0x7fb000110190 2024-09-11T23:42:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.286+0000 7faffffff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0001111b0 con 0x7fb00006ea30 2024-09-11T23:42:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.286+0000 7faffffff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb0001114b0 con 0x7fb00011b140 2024-09-11T23:42:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.286+0000 7faffeffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00006ea30 0x7fb00011a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.286+0000 7faffeffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00006ea30 0x7fb00011a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51658/0 (socket says 172.21.15.133:51658) 2024-09-11T23:42:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.286+0000 7faffeffd640 1 -- 172.21.15.133:0/1269025515 learned_addr learned my addr 172.21.15.133:0/1269025515 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.286+0000 7fafff7fe640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00011b140 0x7fb000114f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:08.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.288+0000 7fafff7fe640 1 -- 172.21.15.133:0/1269025515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00006ea30 msgr2=0x7fb00011a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:08.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.288+0000 7fafff7fe640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00006ea30 0x7fb00011a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:08.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.288+0000 7fafff7fe640 1 -- 172.21.15.133:0/1269025515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb000110190 msgr2=0x7fb0001149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:08.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.288+0000 7fafff7fe640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb000110190 0x7fb0001149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.288+0000 7fafff7fe640 1 -- 172.21.15.133:0/1269025515 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb000115730 con 0x7fb00011b140 2024-09-11T23:42:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.289+0000 7fafff7fe640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00011b140 0x7fb000114f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faff400c710 tx=0x7faff400cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.292+0000 7fafe7fff640 1 -- 172.21.15.133:0/1269025515 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4007bb0 con 0x7fb00011b140 2024-09-11T23:42:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.292+0000 7fafe7fff640 1 -- 172.21.15.133:0/1269025515 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faff400ce20 con 0x7fb00011b140 2024-09-11T23:42:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.292+0000 7fafe7fff640 1 -- 172.21.15.133:0/1269025515 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4018660 con 0x7fb00011b140 2024-09-11T23:42:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.295+0000 7faffffff640 1 -- 172.21.15.133:0/1269025515 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0001bbed0 con 0x7fb00011b140 2024-09-11T23:42:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.295+0000 7faffffff640 1 -- 172.21.15.133:0/1269025515 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0001bc2d0 con 0x7fb00011b140 2024-09-11T23:42:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.298+0000 7faffffff640 1 -- 172.21.15.133:0/1269025515 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0001096f0 con 0x7fb00011b140 2024-09-11T23:42:08.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.309+0000 7fafe7fff640 1 -- 172.21.15.133:0/1269025515 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7faff400f450 con 0x7fb00011b140 2024-09-11T23:42:08.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.310+0000 7fafe7fff640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafe80a2ec0 0x7fafe80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:08.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.312+0000 7fafe7fff640 1 -- 172.21.15.133:0/1269025515 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7faff40cf480 con 0x7fb00011b140 2024-09-11T23:42:08.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.312+0000 7faffeffd640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafe80a2ec0 0x7fafe80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:08.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.313+0000 7fafe7fff640 1 -- 172.21.15.133:0/1269025515 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faff4014030 con 0x7fb00011b140 2024-09-11T23:42:08.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.320+0000 7faffeffd640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafe80a2ec0 0x7fafe80a5380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faff8009fd0 tx=0x7faff8009720 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:08.725 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:42:08.725 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":28,"btime":"2024-09-11T23:36:46:082131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:36:46.031976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/1681049778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1681049778},{"type":"v1","addr":"172.21.15.178:6835","nonce":1681049778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:42:08.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.716+0000 7faffffff640 1 -- 172.21.15.133:0/1269025515 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb00006ee70 con 0x7fb00011b140 2024-09-11T23:42:08.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.721+0000 7fafe7fff640 1 -- 172.21.15.133:0/1269025515 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7faff4014030 con 0x7fb00011b140 2024-09-11T23:42:08.725 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 28 2024-09-11T23:42:08.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.728+0000 7faffffff640 1 -- 172.21.15.133:0/1269025515 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafe80a2ec0 msgr2=0x7fafe80a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:08.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.728+0000 7faffffff640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafe80a2ec0 0x7fafe80a5380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faff8009fd0 tx=0x7faff8009720 comp rx=0 tx=0).stop 2024-09-11T23:42:08.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.728+0000 7faffffff640 1 -- 172.21.15.133:0/1269025515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00011b140 msgr2=0x7fb000114f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:08.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.728+0000 7faffffff640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00011b140 0x7fb000114f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faff400c710 tx=0x7faff400cbd0 comp rx=0 tx=0).stop 2024-09-11T23:42:08.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.729+0000 7faffffff640 1 -- 172.21.15.133:0/1269025515 shutdown_connections 2024-09-11T23:42:08.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.729+0000 7faffffff640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fafe80a2ec0 0x7fafe80a5380 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:08.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.729+0000 7faffffff640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00011b140 0x7fb000114f00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:08.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.729+0000 7faffffff640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb000110190 0x7fb0001149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:08.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.729+0000 7faffffff640 1 --2- 172.21.15.133:0/1269025515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00006ea30 0x7fb00011a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:08.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.729+0000 7faffffff640 1 -- 172.21.15.133:0/1269025515 >> 172.21.15.133:0/1269025515 conn(0x7fb000073180 msgr2=0x7fb00011b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:08.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.730+0000 7faffffff640 1 -- 172.21.15.133:0/1269025515 shutdown_connections 2024-09-11T23:42:08.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:08.731+0000 7faffffff640 1 -- 172.21.15.133:0/1269025515 wait complete. 2024-09-11T23:42:08.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:08 smithi138 ceph-mon[35142]: pgmap v163: 97 pgs: 97 active+clean; 40 GiB data, 143 GiB used, 930 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 619 MiB/s wr, 1.54k op/s 2024-09-11T23:42:08.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:08 smithi133 ceph-mon[31430]: pgmap v163: 97 pgs: 97 active+clean; 40 GiB data, 143 GiB used, 930 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 619 MiB/s wr, 1.54k op/s 2024-09-11T23:42:09.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:08 smithi178 ceph-mon[35882]: pgmap v163: 97 pgs: 97 active+clean; 40 GiB data, 143 GiB used, 930 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 619 MiB/s wr, 1.54k op/s 2024-09-11T23:42:09.604 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:09 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1269025515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:42:09.920 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.920+0000 7f22b51bc640 1 -- 172.21.15.133:0/2065523867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b011b140 msgr2=0x7f22b011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.920+0000 7f22b51bc640 1 --2- 172.21.15.133:0/2065523867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b011b140 0x7f22b011d530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009900 tx=0x7f22a402a440 comp rx=0 tx=0).stop 2024-09-11T23:42:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.920+0000 7f22b51bc640 1 -- 172.21.15.133:0/2065523867 shutdown_connections 2024-09-11T23:42:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.920+0000 7f22b51bc640 1 --2- 172.21.15.133:0/2065523867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b011b140 0x7f22b011d530 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.920+0000 7f22b51bc640 1 --2- 172.21.15.133:0/2065523867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b01095d0 0x7f22b0109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.920+0000 7f22b51bc640 1 --2- 172.21.15.133:0/2065523867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b01091f0 0x7f22b0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.920+0000 7f22b51bc640 1 -- 172.21.15.133:0/2065523867 >> 172.21.15.133:0/2065523867 conn(0x7f22b006cbf0 msgr2=0x7f22b006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.928+0000 7f22b51bc640 1 -- 172.21.15.133:0/2065523867 shutdown_connections 2024-09-11T23:42:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.928+0000 7f22b51bc640 1 -- 172.21.15.133:0/2065523867 wait complete. 2024-09-11T23:42:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.929+0000 7f22b51bc640 1 Processor -- start 2024-09-11T23:42:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.929+0000 7f22b51bc640 1 -- start start 2024-09-11T23:42:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.930+0000 7f22b51bc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b01091f0 0x7f22b01a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.930+0000 7f22b51bc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b01095d0 0x7f22b01a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.930+0000 7f22b51bc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b01ab600 0x7f22b01abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.930+0000 7f22b51bc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22b0120b10 con 0x7f22b01091f0 2024-09-11T23:42:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.930+0000 7f22b51bc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22b0120990 con 0x7f22b01095d0 2024-09-11T23:42:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.930+0000 7f22b51bc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f22b0120c90 con 0x7f22b01ab600 2024-09-11T23:42:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.930+0000 7f22af7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b01095d0 0x7f22b01a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.930+0000 7f22af7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b01095d0 0x7f22b01a7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51674/0 (socket says 172.21.15.133:51674) 2024-09-11T23:42:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.930+0000 7f22af7fe640 1 -- 172.21.15.133:0/3681661892 learned_addr learned my addr 172.21.15.133:0/3681661892 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.930+0000 7f22b49bb640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b01ab600 0x7f22b01abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.931+0000 7f22af7fe640 1 -- 172.21.15.133:0/3681661892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b01ab600 msgr2=0x7f22b01abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.931+0000 7f22af7fe640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b01ab600 0x7f22b01abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.931+0000 7f22af7fe640 1 -- 172.21.15.133:0/3681661892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b01091f0 msgr2=0x7f22b01a6d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:42:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.931+0000 7f22af7fe640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b01091f0 0x7f22b01a6d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.931+0000 7f22af7fe640 1 -- 172.21.15.133:0/3681661892 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b01ac0f0 con 0x7f22b01095d0 2024-09-11T23:42:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.931+0000 7f22af7fe640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b01095d0 0x7f22b01a7280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f22a001fa00 tx=0x7f22a001fec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.940+0000 7f22ad7fa640 1 -- 172.21.15.133:0/3681661892 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a000cd80 con 0x7f22b01095d0 2024-09-11T23:42:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.940+0000 7f22b51bc640 1 -- 172.21.15.133:0/3681661892 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b01ac3e0 con 0x7f22b01095d0 2024-09-11T23:42:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.940+0000 7f22b51bc640 1 -- 172.21.15.133:0/3681661892 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b0110c40 con 0x7f22b01095d0 2024-09-11T23:42:09.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:09 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1269025515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:42:09.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:09 smithi138 ceph-mon[35142]: pgmap v164: 97 pgs: 97 active+clean; 42 GiB data, 151 GiB used, 922 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 905 MiB/s wr, 3.46k op/s 2024-09-11T23:42:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.944+0000 7f22b928d740 1 Processor -- start 2024-09-11T23:42:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.945+0000 7f22b928d740 1 -- start start 2024-09-11T23:42:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.945+0000 7f22b928d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 0x55b61301c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.946+0000 7f22b928d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301c5e0 0x55b613026940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.946+0000 7f22b928d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 0x55b613029270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.946+0000 7f22b928d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b612f802f0 con 0x55b613026e80 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.946+0000 7f22b928d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b612f28270 con 0x55b61301c5e0 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.946+0000 7f22b928d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b612f7c2b0 con 0x55b61301bcc0 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.946+0000 7f22ad7fa640 1 -- 172.21.15.133:0/3681661892 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a0021920 con 0x7f22b01095d0 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.946+0000 7f22ad7fa640 1 -- 172.21.15.133:0/3681661892 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0029990 con 0x7f22b01095d0 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.947+0000 7f22b49bb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 0x55b61301c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.947+0000 7f22b49bb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 0x55b61301c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:51690/0 (socket says 172.21.15.133:51690) 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.947+0000 7f22b49bb640 1 -- 172.21.15.133:0/744708129 learned_addr learned my addr 172.21.15.133:0/744708129 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.948+0000 7f22b49bb640 1 -- 172.21.15.133:0/744708129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301c5e0 msgr2=0x55b613026940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.948+0000 7f22b49bb640 1 --2- 172.21.15.133:0/744708129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301c5e0 0x55b613026940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.948+0000 7f22b49bb640 1 -- 172.21.15.133:0/744708129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 msgr2=0x55b613029270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.948+0000 7f22ad7fa640 1 -- 172.21.15.133:0/3681661892 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f22a0029bf0 con 0x7f22b01095d0 2024-09-11T23:42:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.948+0000 7f22b49bb640 1 --2- 172.21.15.133:0/744708129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 0x55b613029270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.949+0000 7f22ad7fa640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22980a2f00 0x7f22980a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.949+0000 7f22b49bb640 1 -- 172.21.15.133:0/744708129 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b612f27340 con 0x55b61301bcc0 2024-09-11T23:42:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.950+0000 7f22b49bb640 1 --2- 172.21.15.133:0/744708129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 0x55b61301c0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ab0 tx=0x7f22a4009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23b25cf797222df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.950+0000 7f22ad7fa640 1 -- 172.21.15.133:0/3681661892 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f22a00e1710 con 0x7f22b01095d0 2024-09-11T23:42:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.951+0000 7f22affff640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22980a2f00 0x7f22980a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.952+0000 7f22967fc640 1 -- 172.21.15.133:0/744708129 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4005200 con 0x55b61301bcc0 2024-09-11T23:42:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.953+0000 7f22967fc640 1 -- 172.21.15.133:0/744708129 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a4004ba0 con 0x55b61301bcc0 2024-09-11T23:42:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.954+0000 7f22b928d740 1 -- 172.21.15.133:0/744708129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 msgr2=0x55b61301c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.954+0000 7f22b928d740 1 --2- 172.21.15.133:0/744708129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 0x55b61301c0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ab0 tx=0x7f22a4009490 comp rx=0 tx=0).stop 2024-09-11T23:42:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.955+0000 7f22affff640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22980a2f00 0x7f22980a53c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55b613019090 tx=0x7f22a8007570 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:09.961 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:09 smithi133 ceph-mon[31430]: pgmap v164: 97 pgs: 97 active+clean; 42 GiB data, 151 GiB used, 922 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 905 MiB/s wr, 3.46k op/s 2024-09-11T23:42:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.965+0000 7f22b928d740 1 -- 172.21.15.133:0/744708129 shutdown_connections 2024-09-11T23:42:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.965+0000 7f22b928d740 1 --2- 172.21.15.133:0/744708129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 0x55b613029270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.965+0000 7f22b928d740 1 --2- 172.21.15.133:0/744708129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301c5e0 0x55b613026940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.965+0000 7f22b928d740 1 --2- 172.21.15.133:0/744708129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 0x55b61301c0a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.965+0000 7f22b928d740 1 -- 172.21.15.133:0/744708129 >> 172.21.15.133:0/744708129 conn(0x55b613018230 msgr2=0x55b613018610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.965+0000 7f22b928d740 1 -- 172.21.15.133:0/744708129 shutdown_connections 2024-09-11T23:42:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.965+0000 7f22b928d740 1 -- 172.21.15.133:0/744708129 wait complete. 2024-09-11T23:42:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.966+0000 7f22b928d740 1 -- start start 2024-09-11T23:42:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.966+0000 7f22b928d740 1 Processor -- start 2024-09-11T23:42:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.967+0000 7f22b928d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b61301bcc0 0x55b613023d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.967+0000 7f22b928d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301c5e0 0x55b6130242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.967+0000 7f22b928d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b613026e80 0x55b613024810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.967+0000 7f22b928d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b613029aa0 con 0x55b61301bcc0 2024-09-11T23:42:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.967+0000 7f22b928d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b612f09290 con 0x55b613026e80 2024-09-11T23:42:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.967+0000 7f22b928d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b613029c20 con 0x55b61301c5e0 2024-09-11T23:42:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.967+0000 7f22b928d740 10 client.0 _ll_register_callbacks cb 0x55b612f3a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.968+0000 7f22b49bb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b613026e80 0x55b613024810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.968+0000 7f22b49bb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b613026e80 0x55b613024810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51686/0 (socket says 172.21.15.133:51686) 2024-09-11T23:42:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.968+0000 7f22b49bb640 1 -- 172.21.15.133:0/2573226900 learned_addr learned my addr 172.21.15.133:0/2573226900 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.968+0000 7f22af7fe640 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301c5e0 0x55b6130242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.968+0000 7f22b49bb640 1 -- 172.21.15.133:0/2573226900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301c5e0 msgr2=0x55b6130242d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.968+0000 7f22b49bb640 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301c5e0 0x55b6130242d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.968+0000 7f22b49bb640 1 -- 172.21.15.133:0/2573226900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b61301bcc0 msgr2=0x55b613023d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.968+0000 7f22b49bb640 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b61301bcc0 0x55b613023d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.968+0000 7f22b49bb640 1 -- 172.21.15.133:0/2573226900 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b612c93210 con 0x55b613026e80 2024-09-11T23:42:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.969+0000 7f22b49bb640 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b613026e80 0x55b613024810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f22a402aa30 tx=0x7f22a4031be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a282cd93e90b044 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.969+0000 7f228f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.969+0000 7f228f7fe640 1 -- 172.21.15.133:0/2573226900 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4031d80 con 0x55b613026e80 2024-09-11T23:42:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.970+0000 7f228f7fe640 1 -- 172.21.15.133:0/2573226900 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f22a40046c0 con 0x55b613026e80 2024-09-11T23:42:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.970+0000 7f22b928d740 1 -- 172.21.15.133:0/2573226900 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a4004ba0 con 0x55b613026e80 2024-09-11T23:42:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.970+0000 7f228f7fe640 1 -- 172.21.15.133:0/2573226900 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f22a405d050 con 0x55b613026e80 2024-09-11T23:42:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.970+0000 7f22b928d740 10 client.26345 fetch_fsmap learned FSMap version 28 2024-09-11T23:42:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.970+0000 7f22b928d740 1 -- 172.21.15.133:0/2573226900 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b612d5f620 con 0x55b613026e80 2024-09-11T23:42:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.970+0000 7f228f7fe640 1 -- 172.21.15.133:0/2573226900 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a4004ba0 con 0x55b613026e80 2024-09-11T23:42:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.972+0000 7f228f7fe640 1 -- 172.21.15.133:0/2573226900 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f22a4046730 con 0x55b613026e80 2024-09-11T23:42:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.973+0000 7f22b928d740 10 client.26345 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:42:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.973+0000 7f22b928d740 10 client.26345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.973+0000 7f22b928d740 20 client.26345 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.973+0000 7f22b928d740 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b613036d80 0x55b613039170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.973+0000 7f22b928d740 4 client.26345 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.973+0000 7f22b928d740 1 -- 172.21.15.133:0/2573226900 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b612f8da20 con 0x55b613036d80 2024-09-11T23:42:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.975+0000 7f22affff640 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b613036d80 0x55b613039170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.978+0000 7f22affff640 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b613036d80 0x55b613039170 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e42e6099b421cb server_cookie=5fb706495320d686 in_seq=0 out_seq=0 2024-09-11T23:42:09.980 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:42:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.980+0000 7f228f7fe640 10 client.26345 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.987+0000 7f228f7fe640 1 -- 172.21.15.133:0/2573226900 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b612f8da20 con 0x55b613036d80 2024-09-11T23:42:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.987+0000 7f228f7fe640 10 client.26345 handle_command_reply: tid=0 2024-09-11T23:42:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.988+0000 7f22b928d740 1 client.26345 shutdown 2024-09-11T23:42:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.988+0000 7f22b928d740 1 -- 172.21.15.133:0/2573226900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b613026e80 msgr2=0x55b613024810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.988+0000 7f22b928d740 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b613026e80 0x55b613024810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f22a402aa30 tx=0x7f22a4031be0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.988+0000 7f22b928d740 1 -- 172.21.15.133:0/2573226900 shutdown_connections 2024-09-11T23:42:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.988+0000 7f22b928d740 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b613036d80 0x55b613039170 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.988+0000 7f22b928d740 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b613026e80 0x55b613024810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.988+0000 7f22b928d740 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301c5e0 0x55b6130242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.988+0000 7f22b928d740 1 --2- 172.21.15.133:0/2573226900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b61301bcc0 0x55b613023d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.988+0000 7f22b928d740 1 -- 172.21.15.133:0/2573226900 >> 172.21.15.133:0/2573226900 conn(0x55b613018230 msgr2=0x55b61301df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.988+0000 7f228f7fe640 0 client.26345 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.988+0000 7f22b928d740 1 -- 172.21.15.133:0/2573226900 shutdown_connections 2024-09-11T23:42:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.989+0000 7f22b928d740 1 -- 172.21.15.133:0/2573226900 wait complete. 2024-09-11T23:42:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:09.989+0000 7f22b928d740 20 client.26345 trim_cache size 0 max 16384 2024-09-11T23:42:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.028+0000 7f22b928d740 1 Processor -- start 2024-09-11T23:42:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.028+0000 7f22b928d740 1 -- start start 2024-09-11T23:42:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.028+0000 7f22b928d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301bcc0 0x55b613024680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.028+0000 7f22b928d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301c5e0 0x55b613017170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.028+0000 7f22b928d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 0x55b61303c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.028+0000 7f22b928d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f229c00e610 con 0x55b613026e80 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.028+0000 7f22b928d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f229c010150 con 0x55b61301bcc0 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.028+0000 7f22b928d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f229c00ece0 con 0x55b61301c5e0 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.029+0000 7f22b49bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 0x55b61303c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.029+0000 7f22b49bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 0x55b61303c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42742/0 (socket says 172.21.15.133:42742) 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.029+0000 7f22b49bb640 1 -- 172.21.15.133:0/3840050117 learned_addr learned my addr 172.21.15.133:0/3840050117 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.030+0000 7f22af7fe640 1 --2- 172.21.15.133:0/3840050117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301c5e0 0x55b613017170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.031+0000 7f22affff640 1 --2- 172.21.15.133:0/3840050117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301bcc0 0x55b613024680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.032+0000 7f22af7fe640 1 -- 172.21.15.133:0/3840050117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301bcc0 msgr2=0x55b613024680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.032+0000 7f22af7fe640 1 --2- 172.21.15.133:0/3840050117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301bcc0 0x55b613024680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.032+0000 7f22af7fe640 1 -- 172.21.15.133:0/3840050117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 msgr2=0x55b61303c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.032+0000 7f22af7fe640 1 --2- 172.21.15.133:0/3840050117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 0x55b61303c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.032+0000 7f22af7fe640 1 -- 172.21.15.133:0/3840050117 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b613018ec0 con 0x55b61301c5e0 2024-09-11T23:42:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:09 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1269025515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:42:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:09 smithi178 ceph-mon[35882]: pgmap v164: 97 pgs: 97 active+clean; 42 GiB data, 151 GiB used, 922 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 905 MiB/s wr, 3.46k op/s 2024-09-11T23:42:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.033+0000 7f22af7fe640 1 --2- 172.21.15.133:0/3840050117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301c5e0 0x55b613017170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22a00dee70 tx=0x7f22a00a9f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f01388226452c9de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.035+0000 7f2295ffb640 1 -- 172.21.15.133:0/3840050117 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a00b1ff0 con 0x55b61301c5e0 2024-09-11T23:42:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.035+0000 7f2295ffb640 1 -- 172.21.15.133:0/3840050117 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a00b68b0 con 0x55b61301c5e0 2024-09-11T23:42:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.035+0000 7f2295ffb640 1 -- 172.21.15.133:0/3840050117 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a00a9990 con 0x55b61301c5e0 2024-09-11T23:42:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.036+0000 7f22b928d740 1 -- 172.21.15.133:0/3840050117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301c5e0 msgr2=0x55b613017170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.036+0000 7f22b928d740 1 --2- 172.21.15.133:0/3840050117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301c5e0 0x55b613017170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22a00dee70 tx=0x7f22a00a9f90 comp rx=0 tx=0).stop 2024-09-11T23:42:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.044+0000 7f22b928d740 1 -- 172.21.15.133:0/3840050117 shutdown_connections 2024-09-11T23:42:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.044+0000 7f22b928d740 1 --2- 172.21.15.133:0/3840050117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 0x55b61303c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.044+0000 7f22b928d740 1 --2- 172.21.15.133:0/3840050117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301c5e0 0x55b613017170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.044+0000 7f22b928d740 1 --2- 172.21.15.133:0/3840050117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301bcc0 0x55b613024680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.044+0000 7f22b928d740 1 -- 172.21.15.133:0/3840050117 >> 172.21.15.133:0/3840050117 conn(0x55b613018230 msgr2=0x55b613016a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.048+0000 7f22b928d740 1 -- 172.21.15.133:0/3840050117 shutdown_connections 2024-09-11T23:42:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.048+0000 7f22b928d740 1 -- 172.21.15.133:0/3840050117 wait complete. 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.048+0000 7f22b928d740 1 -- start start 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.049+0000 7f22b928d740 1 Processor -- start 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.049+0000 7f22b928d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 0x55b61303c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.049+0000 7f22b928d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301c5e0 0x55b613039a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.049+0000 7f22b928d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 0x55b613039f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.049+0000 7f22b928d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f229c00fa80 con 0x55b613026e80 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.049+0000 7f22b928d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f229c00f3b0 con 0x55b61301c5e0 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.049+0000 7f22b928d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f229c00c5f0 con 0x55b61301bcc0 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.049+0000 7f22b928d740 10 client.0 _ll_register_callbacks cb 0x7f22a40093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.050+0000 7f22affff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 0x55b61303c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.050+0000 7f22affff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 0x55b61303c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:51716/0 (socket says 172.21.15.133:51716) 2024-09-11T23:42:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.050+0000 7f22affff640 1 -- 172.21.15.133:0/3509082939 learned_addr learned my addr 172.21.15.133:0/3509082939 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.050+0000 7f22af7fe640 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301c5e0 0x55b613039a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.050+0000 7f22af7fe640 1 -- 172.21.15.133:0/3509082939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 msgr2=0x55b61303c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.050+0000 7f22af7fe640 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 0x55b61303c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.050+0000 7f22af7fe640 1 -- 172.21.15.133:0/3509082939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 msgr2=0x55b613039f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.050+0000 7f22af7fe640 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 0x55b613039f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.050+0000 7f22af7fe640 1 -- 172.21.15.133:0/3509082939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b61303a560 con 0x55b61301c5e0 2024-09-11T23:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.051+0000 7f22af7fe640 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301c5e0 0x55b613039a30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f22a0029500 tx=0x7f22a000c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b5904839529078 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.052+0000 7f228f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.052+0000 7f228f7fe640 1 -- 172.21.15.133:0/3509082939 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a00b1ff0 con 0x55b61301c5e0 2024-09-11T23:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.052+0000 7f22b928d740 1 -- 172.21.15.133:0/3509082939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a00b68b0 con 0x55b61301c5e0 2024-09-11T23:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.052+0000 7f228f7fe640 1 -- 172.21.15.133:0/3509082939 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8411+0+0 (secure 0 0 0) 0x7f22a0021440 con 0x55b61301c5e0 2024-09-11T23:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f228f7fe640 1 -- 172.21.15.133:0/3509082939 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f22a0021640 con 0x55b61301c5e0 2024-09-11T23:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f22b928d740 10 client.26351 fetch_fsmap learned FSMap version 28 2024-09-11T23:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f22b928d740 1 -- 172.21.15.133:0/3509082939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6130a14d0 con 0x55b61301c5e0 2024-09-11T23:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f228f7fe640 1 -- 172.21.15.133:0/3509082939 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a00b68b0 con 0x55b61301c5e0 2024-09-11T23:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f228f7fe640 1 -- 172.21.15.133:0/3509082939 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f22a0021ad0 con 0x55b61301c5e0 2024-09-11T23:42:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f22b928d740 10 client.26351 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:42:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f22b928d740 10 client.26351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f22b928d740 20 client.26351 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f22b928d740 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b6130a1cd0 0x55b6130a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f22b928d740 4 client.26351 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:42:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f22b928d740 1 -- 172.21.15.133:0/3509082939 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b612f87d70 con 0x55b6130a1cd0 2024-09-11T23:42:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.053+0000 7f22affff640 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b6130a1cd0 0x55b6130a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.055+0000 7f22affff640 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b6130a1cd0 0x55b6130a40f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd4083e75841571 server_cookie=b2aa2d9e3a8c0fc6 in_seq=0 out_seq=0 2024-09-11T23:42:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.055+0000 7f228f7fe640 10 client.26351 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.340+0000 7f228f7fe640 1 -- 172.21.15.133:0/3509082939 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b612f87d70 con 0x55b6130a1cd0 2024-09-11T23:42:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.340+0000 7f228f7fe640 10 client.26351 handle_command_reply: tid=0 2024-09-11T23:42:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.340+0000 7f22b928d740 1 client.26351 shutdown 2024-09-11T23:42:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.340+0000 7f22b928d740 1 -- 172.21.15.133:0/3509082939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301c5e0 msgr2=0x55b613039a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.340+0000 7f22b928d740 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301c5e0 0x55b613039a30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f22a0029500 tx=0x7f22a000c7d0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.341+0000 7f22b928d740 1 -- 172.21.15.133:0/3509082939 shutdown_connections 2024-09-11T23:42:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.341+0000 7f22b928d740 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55b6130a1cd0 0x55b6130a40f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.341+0000 7f22b928d740 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b613026e80 0x55b613039f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.341+0000 7f22b928d740 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b61301c5e0 0x55b613039a30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.341+0000 7f22b928d740 1 --2- 172.21.15.133:0/3509082939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61301bcc0 0x55b61303c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.341+0000 7f22b928d740 1 -- 172.21.15.133:0/3509082939 >> 172.21.15.133:0/3509082939 conn(0x55b613018230 msgr2=0x55b613017260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.341+0000 7f228f7fe640 0 client.26351 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.341+0000 7f22b928d740 1 -- 172.21.15.133:0/3509082939 shutdown_connections 2024-09-11T23:42:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.341+0000 7f22b928d740 1 -- 172.21.15.133:0/3509082939 wait complete. 2024-09-11T23:42:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.341+0000 7f22b928d740 20 client.26351 trim_cache size 0 max 16384 2024-09-11T23:42:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:42:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:42:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:42:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:42:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.344+0000 7f22b51bc640 1 -- 172.21.15.133:0/3681661892 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22980a2f00 msgr2=0x7f22980a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.345+0000 7f22b51bc640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22980a2f00 0x7f22980a53c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55b613019090 tx=0x7f22a8007570 comp rx=0 tx=0).stop 2024-09-11T23:42:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.357+0000 7f22b51bc640 1 -- 172.21.15.133:0/3681661892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b01095d0 msgr2=0x7f22b01a7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.357+0000 7f22b51bc640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b01095d0 0x7f22b01a7280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f22a001fa00 tx=0x7f22a001fec0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.358+0000 7f22b51bc640 1 -- 172.21.15.133:0/3681661892 shutdown_connections 2024-09-11T23:42:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.358+0000 7f22b51bc640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22980a2f00 0x7f22980a53c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.358+0000 7f22b51bc640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b01ab600 0x7f22b01abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.358+0000 7f22b51bc640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b01095d0 0x7f22b01a7280 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.358+0000 7f22b51bc640 1 --2- 172.21.15.133:0/3681661892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b01091f0 0x7f22b01a6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.358+0000 7f22b51bc640 1 -- 172.21.15.133:0/3681661892 >> 172.21.15.133:0/3681661892 conn(0x7f22b006cbf0 msgr2=0x7f22b011b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.360+0000 7f22b51bc640 1 -- 172.21.15.133:0/3681661892 shutdown_connections 2024-09-11T23:42:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:10.360+0000 7f22b51bc640 1 -- 172.21.15.133:0/3681661892 wait complete. 2024-09-11T23:42:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:11.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:10 smithi178 ceph-mon[35882]: daemon mds.f restarted 2024-09-11T23:42:11.040 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:42:10 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: ignoring --setuser ceph since I am not root 2024-09-11T23:42:11.040 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:42:10 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: ignoring --setgroup ceph since I am not root 2024-09-11T23:42:11.040 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:42:10 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: starting mds.f at 2024-09-11T23:42:11.046 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:10 smithi133 ceph-mon[31430]: daemon mds.f restarted 2024-09-11T23:42:11.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:10 smithi138 ceph-mon[35142]: daemon mds.f restarted 2024-09-11T23:42:11.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.750+0000 7f232892b640 1 -- 172.21.15.133:0/1636072803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232406e4f0 msgr2=0x7f232406e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:11.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.750+0000 7f232892b640 1 --2- 172.21.15.133:0/1636072803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232406e4f0 0x7f232406e930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f231800dc70 tx=0x7f2318032900 comp rx=0 tx=0).stop 2024-09-11T23:42:11.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.750+0000 7f232892b640 1 -- 172.21.15.133:0/1636072803 shutdown_connections 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.750+0000 7f232892b640 1 --2- 172.21.15.133:0/1636072803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f232406ee70 0x7f232407a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.750+0000 7f232892b640 1 --2- 172.21.15.133:0/1636072803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232406e4f0 0x7f232406e930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.750+0000 7f232892b640 1 --2- 172.21.15.133:0/1636072803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232410e090 0x7f232410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.750+0000 7f232892b640 1 -- 172.21.15.133:0/1636072803 >> 172.21.15.133:0/1636072803 conn(0x7f2324073180 msgr2=0x7f2324073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.750+0000 7f232892b640 1 -- 172.21.15.133:0/1636072803 shutdown_connections 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.750+0000 7f232892b640 1 -- 172.21.15.133:0/1636072803 wait complete. 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.751+0000 7f232892b640 1 Processor -- start 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.752+0000 7f232892b640 1 -- start start 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.752+0000 7f232892b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232406e4f0 0x7f2324084db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.752+0000 7f232892b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232406ee70 0x7f232407ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.752+0000 7f232892b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f232410e090 0x7f232407f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.752+0000 7f232892b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f232407da70 con 0x7f232406ee70 2024-09-11T23:42:11.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.752+0000 7f232892b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f232407d8f0 con 0x7f232406e4f0 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.752+0000 7f232892b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f232407dbf0 con 0x7f232410e090 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.752+0000 7f23237fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232406e4f0 0x7f2324084db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.753+0000 7f23237fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232406e4f0 0x7f2324084db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51724/0 (socket says 172.21.15.133:51724) 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.753+0000 7f23237fe640 1 -- 172.21.15.133:0/1957891323 learned_addr learned my addr 172.21.15.133:0/1957891323 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.753+0000 7f2323fff640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f232410e090 0x7f232407f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.753+0000 7f23237fe640 1 -- 172.21.15.133:0/1957891323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f232410e090 msgr2=0x7f232407f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.753+0000 7f23237fe640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f232410e090 0x7f232407f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.753+0000 7f23237fe640 1 -- 172.21.15.133:0/1957891323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232406ee70 msgr2=0x7f232407ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.753+0000 7f23237fe640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232406ee70 0x7f232407ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.753+0000 7f23237fe640 1 -- 172.21.15.133:0/1957891323 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2324136e70 con 0x7f232406e4f0 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.754+0000 7f23237fe640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232406e4f0 0x7f2324084db0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f23140078c0 tx=0x7f2314007d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:11.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.759+0000 7f2320ff9640 1 -- 172.21.15.133:0/1957891323 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314052b30 con 0x7f232406e4f0 2024-09-11T23:42:11.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.759+0000 7f2320ff9640 1 -- 172.21.15.133:0/1957891323 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f231405d070 con 0x7f232406e4f0 2024-09-11T23:42:11.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.759+0000 7f2320ff9640 1 -- 172.21.15.133:0/1957891323 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314057640 con 0x7f232406e4f0 2024-09-11T23:42:11.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.760+0000 7f232892b640 1 -- 172.21.15.133:0/1957891323 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2324137100 con 0x7f232406e4f0 2024-09-11T23:42:11.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.761+0000 7f232892b640 1 -- 172.21.15.133:0/1957891323 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23241375c0 con 0x7f232406e4f0 2024-09-11T23:42:11.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.764+0000 7f2320ff9640 1 -- 172.21.15.133:0/1957891323 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2314056050 con 0x7f232406e4f0 2024-09-11T23:42:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.768+0000 7f232bf66740 1 Processor -- start 2024-09-11T23:42:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.769+0000 7f232bf66740 1 -- start start 2024-09-11T23:42:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.769+0000 7f232bf66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd191240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.769+0000 7f232bf66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd191780 0x5641fd191bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.769+0000 7f232bf66740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 0x5641fd19e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.769+0000 7f232bf66740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5641fd0f72f0 con 0x5641fd19c520 2024-09-11T23:42:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.769+0000 7f232bf66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5641fd09ecf0 con 0x5641fd192cf0 2024-09-11T23:42:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.769+0000 7f232bf66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5641fd0f32b0 con 0x5641fd191780 2024-09-11T23:42:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.770+0000 7f2323fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd191240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.770+0000 7f2323fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd191240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51732/0 (socket says 172.21.15.133:51732) 2024-09-11T23:42:11.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.770+0000 7f2323fff640 1 -- 172.21.15.133:0/550364396 learned_addr learned my addr 172.21.15.133:0/550364396 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:11.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.770+0000 7f23237fe640 1 --2- 172.21.15.133:0/550364396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd191780 0x5641fd191bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.770+0000 7f2323fff640 1 -- 172.21.15.133:0/550364396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd191780 msgr2=0x5641fd191bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:11.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.770+0000 7f2323fff640 1 --2- 172.21.15.133:0/550364396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd191780 0x5641fd191bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.770+0000 7f2323fff640 1 -- 172.21.15.133:0/550364396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 msgr2=0x5641fd19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:11.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.770+0000 7f2323fff640 1 --2- 172.21.15.133:0/550364396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 0x5641fd19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.770+0000 7f2323fff640 1 -- 172.21.15.133:0/550364396 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641fd0833a0 con 0x5641fd192cf0 2024-09-11T23:42:11.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.771+0000 7f2323fff640 1 --2- 172.21.15.133:0/550364396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd191240 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f231c00bf00 tx=0x7f231c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d1bc3dcff6946d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:11.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.771+0000 7f2320ff9640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f230c0a2f00 0x7f230c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.772+0000 7f2309ffb640 1 -- 172.21.15.133:0/550364396 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c01a070 con 0x5641fd192cf0 2024-09-11T23:42:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.772+0000 7f2309ffb640 1 -- 172.21.15.133:0/550364396 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f231c012e70 con 0x5641fd192cf0 2024-09-11T23:42:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.772+0000 7f2309ffb640 1 -- 172.21.15.133:0/550364396 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c005020 con 0x5641fd192cf0 2024-09-11T23:42:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.772+0000 7f2320ff9640 1 -- 172.21.15.133:0/1957891323 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f23141129a0 con 0x7f232406e4f0 2024-09-11T23:42:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.776+0000 7f232bf66740 1 -- 172.21.15.133:0/550364396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 msgr2=0x5641fd191240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.776+0000 7f232bf66740 1 --2- 172.21.15.133:0/550364396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd191240 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f231c00bf00 tx=0x7f231c009260 comp rx=0 tx=0).stop 2024-09-11T23:42:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.776+0000 7f232bf66740 1 -- 172.21.15.133:0/550364396 shutdown_connections 2024-09-11T23:42:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.776+0000 7f232bf66740 1 --2- 172.21.15.133:0/550364396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 0x5641fd19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.776+0000 7f232bf66740 1 --2- 172.21.15.133:0/550364396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd191780 0x5641fd191bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.776+0000 7f232bf66740 1 --2- 172.21.15.133:0/550364396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd191240 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.776+0000 7f232bf66740 1 -- 172.21.15.133:0/550364396 >> 172.21.15.133:0/550364396 conn(0x5641fd15a310 msgr2=0x5641fd18de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.776+0000 7f23237fe640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f230c0a2f00 0x7f230c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.777+0000 7f232bf66740 1 -- 172.21.15.133:0/550364396 shutdown_connections 2024-09-11T23:42:11.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.777+0000 7f232bf66740 1 -- 172.21.15.133:0/550364396 wait complete. 2024-09-11T23:42:11.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.778+0000 7f232bf66740 1 -- start start 2024-09-11T23:42:11.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.778+0000 7f232bf66740 1 Processor -- start 2024-09-11T23:42:11.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.779+0000 7f232bf66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd191780 0x5641fd19be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.779+0000 7f232bf66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd192cf0 0x5641fd199150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.779+0000 7f232bf66740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 0x5641fd1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.779+0000 7f232bf66740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5641fd19f220 con 0x5641fd19c520 2024-09-11T23:42:11.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.779+0000 7f232bf66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5641fd0b2060 con 0x5641fd191780 2024-09-11T23:42:11.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.779+0000 7f232bf66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5641fd19f3a0 con 0x5641fd192cf0 2024-09-11T23:42:11.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.779+0000 7f232bf66740 10 client.0 _ll_register_callbacks cb 0x5641fce058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:11.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.782+0000 7f2323fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd191780 0x5641fd19be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.782+0000 7f2323fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 0x5641fd1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.783+0000 7f2323fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 0x5641fd1996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42776/0 (socket says 172.21.15.133:42776) 2024-09-11T23:42:11.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.783+0000 7f2323fff640 1 -- 172.21.15.133:0/2069759685 learned_addr learned my addr 172.21.15.133:0/2069759685 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:11.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.783+0000 7f2322ffd640 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd192cf0 0x5641fd199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.783+0000 7f2323fff640 1 -- 172.21.15.133:0/2069759685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd192cf0 msgr2=0x5641fd199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:11.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.783+0000 7f2323fff640 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd192cf0 0x5641fd199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.783+0000 7f2323fff640 1 -- 172.21.15.133:0/2069759685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 msgr2=0x5641fd1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:11.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.783+0000 7f2323fff640 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 0x5641fd1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.783+0000 7f2323fff640 1 -- 172.21.15.133:0/2069759685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641fce683c0 con 0x5641fd191780 2024-09-11T23:42:11.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.784+0000 7f2323fff640 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd191780 0x5641fd19be50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f231c00bd10 tx=0x7f231c0044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b44c25fd16c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:11.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.785+0000 7f2302ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:11.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.788+0000 7f23237fe640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f230c0a2f00 0x7f230c0a53c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5641fd18ec70 tx=0x7f2314058c50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:11.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.791+0000 7f2302ffd640 1 -- 172.21.15.133:0/2069759685 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c01a040 con 0x5641fd191780 2024-09-11T23:42:11.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.791+0000 7f232bf66740 1 -- 172.21.15.133:0/2069759685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f231c012e70 con 0x5641fd191780 2024-09-11T23:42:11.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.792+0000 7f2302ffd640 1 -- 172.21.15.133:0/2069759685 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f231c005050 con 0x5641fd191780 2024-09-11T23:42:11.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.800+0000 7f2302ffd640 1 -- 172.21.15.133:0/2069759685 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f231c031050 con 0x5641fd191780 2024-09-11T23:42:11.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.801+0000 7f232bf66740 10 client.26369 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:11.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.801+0000 7f232bf66740 1 -- 172.21.15.133:0/2069759685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641fd1aaca0 con 0x5641fd191780 2024-09-11T23:42:11.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.801+0000 7f2302ffd640 1 -- 172.21.15.133:0/2069759685 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f231c012e70 con 0x5641fd191780 2024-09-11T23:42:11.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.801+0000 7f2302ffd640 1 -- 172.21.15.133:0/2069759685 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f231c0167e0 con 0x5641fd191780 2024-09-11T23:42:11.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.803+0000 7f232bf66740 10 client.26369 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:11.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.803+0000 7f232bf66740 10 client.26369 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:42:11.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.803+0000 7f232bf66740 20 client.26369 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:11.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.803+0000 7f232bf66740 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5641fd1ac4e0 0x5641fd1ae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.803+0000 7f232bf66740 4 client.26369 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:11.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.803+0000 7f232bf66740 1 -- 172.21.15.133:0/2069759685 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641fd104860 con 0x5641fd1ac4e0 2024-09-11T23:42:11.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.804+0000 7f2322ffd640 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5641fd1ac4e0 0x5641fd1ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.806+0000 7f2322ffd640 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5641fd1ac4e0 0x5641fd1ae910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70ade4520e1bb187 server_cookie=cbe8c9b31084487 in_seq=0 out_seq=0 2024-09-11T23:42:11.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.806+0000 7f2302ffd640 10 client.26369 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:42:11.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.807+0000 7f2302ffd640 1 -- 172.21.15.133:0/2069759685 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641fd104860 con 0x5641fd1ac4e0 2024-09-11T23:42:11.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.807+0000 7f2302ffd640 10 client.26369 handle_command_reply: tid=0 2024-09-11T23:42:11.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.807+0000 7f232bf66740 1 client.26369 shutdown 2024-09-11T23:42:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f232bf66740 1 -- 172.21.15.133:0/2069759685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd191780 msgr2=0x5641fd19be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f232bf66740 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd191780 0x5641fd19be50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f231c00bd10 tx=0x7f231c0044c0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f232bf66740 1 -- 172.21.15.133:0/2069759685 shutdown_connections 2024-09-11T23:42:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f232bf66740 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5641fd1ac4e0 0x5641fd1ae910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f232bf66740 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 0x5641fd1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f232bf66740 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd192cf0 0x5641fd199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f232bf66740 1 --2- 172.21.15.133:0/2069759685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd191780 0x5641fd19be50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f232bf66740 1 -- 172.21.15.133:0/2069759685 >> 172.21.15.133:0/2069759685 conn(0x5641fd15a310 msgr2=0x5641fd193b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f2302ffd640 0 client.26369 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:42:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f232bf66740 1 -- 172.21.15.133:0/2069759685 shutdown_connections 2024-09-11T23:42:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f232bf66740 1 -- 172.21.15.133:0/2069759685 wait complete. 2024-09-11T23:42:11.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.808+0000 7f232bf66740 20 client.26369 trim_cache size 0 max 16384 2024-09-11T23:42:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.837+0000 7f232bf66740 1 Processor -- start 2024-09-11T23:42:11.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.839+0000 7f232bf66740 1 -- start start 2024-09-11T23:42:11.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.840+0000 7f232bf66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd191780 0x5641fd1a8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.841+0000 7f232bf66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd198da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.841+0000 7f232bf66740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 0x5641fd1942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.841+0000 7f232bf66740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f23241c6ed0 con 0x5641fd19c520 2024-09-11T23:42:11.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.841+0000 7f232bf66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f232413ffb0 con 0x5641fd192cf0 2024-09-11T23:42:11.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.841+0000 7f232bf66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2324141af0 con 0x5641fd191780 2024-09-11T23:42:11.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.842+0000 7f2322ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.842+0000 7f2322ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51746/0 (socket says 172.21.15.133:51746) 2024-09-11T23:42:11.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.842+0000 7f2322ffd640 1 -- 172.21.15.133:0/3769179388 learned_addr learned my addr 172.21.15.133:0/3769179388 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:11.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.842+0000 7f2323fff640 1 --2- 172.21.15.133:0/3769179388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd191780 0x5641fd1a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.849+0000 7f2322ffd640 1 -- 172.21.15.133:0/3769179388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd191780 msgr2=0x5641fd1a8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:11.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.849+0000 7f2322ffd640 1 --2- 172.21.15.133:0/3769179388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd191780 0x5641fd1a8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.849+0000 7f2322ffd640 1 -- 172.21.15.133:0/3769179388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 msgr2=0x5641fd1942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:11.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.849+0000 7f2322ffd640 1 --2- 172.21.15.133:0/3769179388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 0x5641fd1942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.849+0000 7f2322ffd640 1 -- 172.21.15.133:0/3769179388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641fd19c1c0 con 0x5641fd192cf0 2024-09-11T23:42:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.855+0000 7f2322ffd640 1 --2- 172.21.15.133:0/3769179388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd198da0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2318032e70 tx=0x7f2318005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccf169af46e07c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:11.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.859+0000 7f23097fa640 1 -- 172.21.15.133:0/3769179388 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231800d4e0 con 0x5641fd192cf0 2024-09-11T23:42:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.859+0000 7f23097fa640 1 -- 172.21.15.133:0/3769179388 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f231803d040 con 0x5641fd192cf0 2024-09-11T23:42:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.861+0000 7f232bf66740 1 -- 172.21.15.133:0/3769179388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 msgr2=0x5641fd198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.861+0000 7f232bf66740 1 --2- 172.21.15.133:0/3769179388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd198da0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2318032e70 tx=0x7f2318005f50 comp rx=0 tx=0).stop 2024-09-11T23:42:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.861+0000 7f232bf66740 1 -- 172.21.15.133:0/3769179388 shutdown_connections 2024-09-11T23:42:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.861+0000 7f232bf66740 1 --2- 172.21.15.133:0/3769179388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd19c520 0x5641fd1942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.861+0000 7f232bf66740 1 --2- 172.21.15.133:0/3769179388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd192cf0 0x5641fd198da0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.861+0000 7f232bf66740 1 --2- 172.21.15.133:0/3769179388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd191780 0x5641fd1a8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.861+0000 7f232bf66740 1 -- 172.21.15.133:0/3769179388 >> 172.21.15.133:0/3769179388 conn(0x5641fd15a310 msgr2=0x5641fd198690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.861+0000 7f232bf66740 1 -- 172.21.15.133:0/3769179388 shutdown_connections 2024-09-11T23:42:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.861+0000 7f232bf66740 1 -- 172.21.15.133:0/3769179388 wait complete. 2024-09-11T23:42:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.862+0000 7f232bf66740 1 -- start start 2024-09-11T23:42:11.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.863+0000 7f232bf66740 1 Processor -- start 2024-09-11T23:42:11.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.863+0000 7f232bf66740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd191780 0x5641fd1a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.863+0000 7f232bf66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd19c520 0x5641fd19f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.863+0000 7f232bf66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd19fb80 0x5641fd1a0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.863+0000 7f232bf66740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2324140d50 con 0x5641fd191780 2024-09-11T23:42:11.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.863+0000 7f232bf66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2324140680 con 0x5641fd19c520 2024-09-11T23:42:11.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.863+0000 7f232bf66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2324141420 con 0x5641fd19fb80 2024-09-11T23:42:11.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.863+0000 7f232bf66740 10 client.0 _ll_register_callbacks cb 0x7f231c00bdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:11.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.864+0000 7f2322ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd19c520 0x5641fd19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.864+0000 7f2322ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd19c520 0x5641fd19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:51748/0 (socket says 172.21.15.133:51748) 2024-09-11T23:42:11.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.864+0000 7f2322ffd640 1 -- 172.21.15.133:0/3845999344 learned_addr learned my addr 172.21.15.133:0/3845999344 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:11.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.866+0000 7f2323fff640 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd19fb80 0x5641fd1a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.866+0000 7f2322ffd640 1 -- 172.21.15.133:0/3845999344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd19fb80 msgr2=0x5641fd1a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:11.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.866+0000 7f2322ffd640 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd19fb80 0x5641fd1a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.866+0000 7f2322ffd640 1 -- 172.21.15.133:0/3845999344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd191780 msgr2=0x5641fd1a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:11.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.866+0000 7f2322ffd640 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd191780 0x5641fd1a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.866+0000 7f2322ffd640 1 -- 172.21.15.133:0/3845999344 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641fd215290 con 0x5641fd19c520 2024-09-11T23:42:11.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.867+0000 7f2322ffd640 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd19c520 0x5641fd19f640 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f231800f010 tx=0x7f23180383f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87b5a557ae1fe88a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:11.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.867+0000 7f2302ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:11.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.867+0000 7f2302ffd640 1 -- 172.21.15.133:0/3845999344 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231800d4e0 con 0x5641fd19c520 2024-09-11T23:42:11.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.867+0000 7f232bf66740 1 -- 172.21.15.133:0/3845999344 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641fd0fed70 con 0x5641fd19c520 2024-09-11T23:42:11.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.867+0000 7f2302ffd640 1 -- 172.21.15.133:0/3845999344 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f23180163b0 con 0x5641fd19c520 2024-09-11T23:42:11.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.867+0000 7f2302ffd640 1 -- 172.21.15.133:0/3845999344 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f23180165b0 con 0x5641fd19c520 2024-09-11T23:42:11.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.870+0000 7f232bf66740 10 client.26375 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:11.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.870+0000 7f232bf66740 1 -- 172.21.15.133:0/3845999344 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641fd216620 con 0x5641fd19c520 2024-09-11T23:42:11.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.871+0000 7f2302ffd640 1 -- 172.21.15.133:0/3845999344 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5641fd0fed70 con 0x5641fd19c520 2024-09-11T23:42:11.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.872+0000 7f2302ffd640 1 -- 172.21.15.133:0/3845999344 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f23180046c0 con 0x5641fd19c520 2024-09-11T23:42:11.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.873+0000 7f232bf66740 10 client.26375 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:11.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.873+0000 7f232bf66740 10 client.26375 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:42:11.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.873+0000 7f232bf66740 20 client.26375 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:11.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.873+0000 7f232bf66740 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5641fd216d80 0x5641fd2191a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:11.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.873+0000 7f232bf66740 4 client.26375 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:42:11.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.873+0000 7f232bf66740 1 -- 172.21.15.133:0/3845999344 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641fd07b170 con 0x5641fd216d80 2024-09-11T23:42:11.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.874+0000 7f2323fff640 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5641fd216d80 0x5641fd2191a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:11.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.874+0000 7f2323fff640 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5641fd216d80 0x5641fd2191a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=391c58b88398ccb6 server_cookie=f0c75cdf9e2b0038 in_seq=0 out_seq=0 2024-09-11T23:42:11.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.874+0000 7f2302ffd640 10 client.26375 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:42:11.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.875+0000 7f2302ffd640 1 -- 172.21.15.133:0/3845999344 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641fd07b170 con 0x5641fd216d80 2024-09-11T23:42:11.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.875+0000 7f2302ffd640 10 client.26375 handle_command_reply: tid=0 2024-09-11T23:42:11.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.875+0000 7f232bf66740 1 client.26375 shutdown 2024-09-11T23:42:11.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.878+0000 7f232bf66740 1 -- 172.21.15.133:0/3845999344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd19c520 msgr2=0x5641fd19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:11.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.878+0000 7f232bf66740 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd19c520 0x5641fd19f640 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f231800f010 tx=0x7f23180383f0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.878+0000 7f232bf66740 1 -- 172.21.15.133:0/3845999344 shutdown_connections 2024-09-11T23:42:11.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.878+0000 7f232bf66740 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5641fd216d80 0x5641fd2191a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.878+0000 7f232bf66740 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641fd19fb80 0x5641fd1a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.878+0000 7f232bf66740 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641fd19c520 0x5641fd19f640 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.878+0000 7f232bf66740 1 --2- 172.21.15.133:0/3845999344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641fd191780 0x5641fd1a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.878+0000 7f232bf66740 1 -- 172.21.15.133:0/3845999344 >> 172.21.15.133:0/3845999344 conn(0x5641fd15a310 msgr2=0x5641fd19b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:11.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.879+0000 7f2302ffd640 0 client.26375 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:42:11.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.879+0000 7f232bf66740 1 -- 172.21.15.133:0/3845999344 shutdown_connections 2024-09-11T23:42:11.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.879+0000 7f232bf66740 1 -- 172.21.15.133:0/3845999344 wait complete. 2024-09-11T23:42:11.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.879+0000 7f232bf66740 20 client.26375 trim_cache size 0 max 16384 2024-09-11T23:42:11.884 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:42:11.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.883+0000 7f232892b640 1 -- 172.21.15.133:0/1957891323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f230c0a2f00 msgr2=0x7f230c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:11.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.883+0000 7f232892b640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f230c0a2f00 0x7f230c0a53c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5641fd18ec70 tx=0x7f2314058c50 comp rx=0 tx=0).stop 2024-09-11T23:42:11.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.883+0000 7f232892b640 1 -- 172.21.15.133:0/1957891323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232406e4f0 msgr2=0x7f2324084db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:11.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.883+0000 7f232892b640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232406e4f0 0x7f2324084db0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f23140078c0 tx=0x7f2314007d80 comp rx=0 tx=0).stop 2024-09-11T23:42:11.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.883+0000 7f232892b640 1 -- 172.21.15.133:0/1957891323 shutdown_connections 2024-09-11T23:42:11.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.884+0000 7f232892b640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f230c0a2f00 0x7f230c0a53c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.884+0000 7f232892b640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f232410e090 0x7f232407f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.884+0000 7f232892b640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232406ee70 0x7f232407ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.884+0000 7f232892b640 1 --2- 172.21.15.133:0/1957891323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232406e4f0 0x7f2324084db0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:11.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.884+0000 7f232892b640 1 -- 172.21.15.133:0/1957891323 >> 172.21.15.133:0/1957891323 conn(0x7f2324073180 msgr2=0x7f232406cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:11.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.884+0000 7f232892b640 1 -- 172.21.15.133:0/1957891323 shutdown_connections 2024-09-11T23:42:11.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:11.884+0000 7f232892b640 1 -- 172.21.15.133:0/1957891323 wait complete. 2024-09-11T23:42:11.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:11 smithi133 ceph-mon[31430]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:42:11.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:11 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6834/3004728174,v1:172.21.15.178:6835/3004728174] up:boot 2024-09-11T23:42:11.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:11 smithi133 ceph-mon[31430]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T23:42:11.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:11 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:42:11.922 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:11 smithi133 ceph-mon[31430]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:42:11.922 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:11 smithi133 ceph-mon[31430]: pgmap v166: 97 pgs: 97 active+clean; 42 GiB data, 151 GiB used, 922 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 723 MiB/s wr, 3.12k op/s 2024-09-11T23:42:12.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:11 smithi178 ceph-mon[35882]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:42:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:11 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6834/3004728174,v1:172.21.15.178:6835/3004728174] up:boot 2024-09-11T23:42:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:11 smithi178 ceph-mon[35882]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T23:42:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:11 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:42:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:11 smithi178 ceph-mon[35882]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:42:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:11 smithi178 ceph-mon[35882]: pgmap v166: 97 pgs: 97 active+clean; 42 GiB data, 151 GiB used, 922 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 723 MiB/s wr, 3.12k op/s 2024-09-11T23:42:12.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:11 smithi138 ceph-mon[35142]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:42:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:11 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6834/3004728174,v1:172.21.15.178:6835/3004728174] up:boot 2024-09-11T23:42:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:11 smithi138 ceph-mon[35142]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T23:42:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:11 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:42:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:11 smithi138 ceph-mon[35142]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:42:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:11 smithi138 ceph-mon[35142]: pgmap v166: 97 pgs: 97 active+clean; 42 GiB data, 151 GiB used, 922 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 723 MiB/s wr, 3.12k op/s 2024-09-11T23:42:13.246 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:42:14.203 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:14.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:14 smithi138 ceph-mon[35142]: pgmap v167: 97 pgs: 97 active+clean; 43 GiB data, 154 GiB used, 918 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 736 MiB/s wr, 4.20k op/s 2024-09-11T23:42:14.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:14 smithi133 ceph-mon[31430]: pgmap v167: 97 pgs: 97 active+clean; 43 GiB data, 154 GiB used, 918 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 736 MiB/s wr, 4.20k op/s 2024-09-11T23:42:14.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:14 smithi178 ceph-mon[35882]: pgmap v167: 97 pgs: 97 active+clean; 43 GiB data, 154 GiB used, 918 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 736 MiB/s wr, 4.20k op/s 2024-09-11T23:42:15.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.397+0000 7f90ff7fe640 1 -- 172.21.15.133:0/3781419652 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f802fdc0 con 0x7f9100105f50 2024-09-11T23:42:15.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.398+0000 7f9106e15640 1 -- 172.21.15.133:0/3781419652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9100105f50 msgr2=0x7f91001063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:15.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.398+0000 7f9106e15640 1 --2- 172.21.15.133:0/3781419652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9100105f50 0x7f91001063e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009ca0 tx=0x7f90f802a960 comp rx=0 tx=0).stop 2024-09-11T23:42:15.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.399+0000 7f9106e15640 1 -- 172.21.15.133:0/3781419652 shutdown_connections 2024-09-11T23:42:15.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.399+0000 7f9106e15640 1 --2- 172.21.15.133:0/3781419652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9100105f50 0x7f91001063e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.399+0000 7f9106e15640 1 --2- 172.21.15.133:0/3781419652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9100101460 0x7f9100105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.399+0000 7f9106e15640 1 --2- 172.21.15.133:0/3781419652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9100100b40 0x7f9100100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.399+0000 7f9106e15640 1 -- 172.21.15.133:0/3781419652 >> 172.21.15.133:0/3781419652 conn(0x7f91000fc7d0 msgr2=0x7f91000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:15.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.400+0000 7f9106e15640 1 -- 172.21.15.133:0/3781419652 shutdown_connections 2024-09-11T23:42:15.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.400+0000 7f9106e15640 1 -- 172.21.15.133:0/3781419652 wait complete. 2024-09-11T23:42:15.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.402+0000 7f9106e15640 1 Processor -- start 2024-09-11T23:42:15.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.402+0000 7f9106e15640 1 -- start start 2024-09-11T23:42:15.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.402+0000 7f9106e15640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9100100b40 0x7f9100073a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.403+0000 7f9106e15640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9100101460 0x7f910006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.403+0000 7f9106e15640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9100105f50 0x7f910006d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.403+0000 7f9106e15640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91001186c0 con 0x7f9100101460 2024-09-11T23:42:15.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.403+0000 7f9106e15640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9100118540 con 0x7f9100100b40 2024-09-11T23:42:15.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.403+0000 7f9106e15640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9100118840 con 0x7f9100105f50 2024-09-11T23:42:15.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.405+0000 7f90fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9100101460 0x7f910006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.405+0000 7f90fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9100101460 0x7f910006cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36520/0 (socket says 172.21.15.133:36520) 2024-09-11T23:42:15.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.405+0000 7f90fffff640 1 -- 172.21.15.133:0/2132193097 learned_addr learned my addr 172.21.15.133:0/2132193097 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:15.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.405+0000 7f91053b2640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9100105f50 0x7f910006d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.405+0000 7f9104bb1640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9100100b40 0x7f9100073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.407+0000 7f90fffff640 1 -- 172.21.15.133:0/2132193097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9100105f50 msgr2=0x7f910006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:15.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.407+0000 7f90fffff640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9100105f50 0x7f910006d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.407+0000 7f90fffff640 1 -- 172.21.15.133:0/2132193097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9100100b40 msgr2=0x7f9100073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:15.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.407+0000 7f90fffff640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9100100b40 0x7f9100073a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.407+0000 7f90fffff640 1 -- 172.21.15.133:0/2132193097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910006d5f0 con 0x7f9100101460 2024-09-11T23:42:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.408+0000 7f90fffff640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9100101460 0x7f910006cac0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b3c0 tx=0x7f90f400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:15.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.462+0000 7f90fdffb640 1 -- 172.21.15.133:0/2132193097 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4011830 con 0x7f9100101460 2024-09-11T23:42:15.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.462+0000 7f90fdffb640 1 -- 172.21.15.133:0/2132193097 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f4016070 con 0x7f9100101460 2024-09-11T23:42:15.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.462+0000 7f90fdffb640 1 -- 172.21.15.133:0/2132193097 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f400f2e0 con 0x7f9100101460 2024-09-11T23:42:15.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.463+0000 7f9106e15640 1 -- 172.21.15.133:0/2132193097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910006d8e0 con 0x7f9100101460 2024-09-11T23:42:15.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.463+0000 7f9106e15640 1 -- 172.21.15.133:0/2132193097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9100071880 con 0x7f9100101460 2024-09-11T23:42:15.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.470+0000 7f91091ee740 1 Processor -- start 2024-09-11T23:42:15.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.470+0000 7f91091ee740 1 -- start start 2024-09-11T23:42:15.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.471+0000 7f91091ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038211b40 0x561038210090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.471+0000 7f91091ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610382105d0 0x561038210a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.471+0000 7f91091ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103821b370 0x56103821d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.471+0000 7f91091ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5610381762f0 con 0x561038211b40 2024-09-11T23:42:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.471+0000 7f91091ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56103811dcf0 con 0x5610382105d0 2024-09-11T23:42:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.471+0000 7f91091ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5610381722b0 con 0x56103821b370 2024-09-11T23:42:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.471+0000 7f91053b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038211b40 0x561038210090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.471+0000 7f91053b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038211b40 0x561038210090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36530/0 (socket says 172.21.15.133:36530) 2024-09-11T23:42:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.471+0000 7f91053b2640 1 -- 172.21.15.133:0/2692956612 learned_addr learned my addr 172.21.15.133:0/2692956612 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.472+0000 7f9104bb1640 1 --2- 172.21.15.133:0/2692956612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610382105d0 0x561038210a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.472+0000 7f91053b2640 1 -- 172.21.15.133:0/2692956612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103821b370 msgr2=0x56103821d760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:15.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.472+0000 7f91053b2640 1 --2- 172.21.15.133:0/2692956612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103821b370 0x56103821d760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.472+0000 7f91053b2640 1 -- 172.21.15.133:0/2692956612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610382105d0 msgr2=0x561038210a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:15.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.472+0000 7f91053b2640 1 --2- 172.21.15.133:0/2692956612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610382105d0 0x561038210a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.472+0000 7f91053b2640 1 -- 172.21.15.133:0/2692956612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610381023a0 con 0x561038211b40 2024-09-11T23:42:15.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.472+0000 7f9104bb1640 1 --2- 172.21.15.133:0/2692956612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610382105d0 0x561038210a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:42:15.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.472+0000 7f91053b2640 1 --2- 172.21.15.133:0/2692956612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038211b40 0x561038210090 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f90f802f4f0 tx=0x7f90f802f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9007341352b72d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:15.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.473+0000 7f90e6ffd640 1 -- 172.21.15.133:0/2692956612 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8047070 con 0x561038211b40 2024-09-11T23:42:15.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.476+0000 7f90e6ffd640 1 -- 172.21.15.133:0/2692956612 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f8003e50 con 0x561038211b40 2024-09-11T23:42:15.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.477+0000 7f91091ee740 1 -- 172.21.15.133:0/2692956612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038211b40 msgr2=0x561038210090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:15.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.477+0000 7f91091ee740 1 --2- 172.21.15.133:0/2692956612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038211b40 0x561038210090 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f90f802f4f0 tx=0x7f90f802f520 comp rx=0 tx=0).stop 2024-09-11T23:42:15.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.477+0000 7f91091ee740 1 -- 172.21.15.133:0/2692956612 shutdown_connections 2024-09-11T23:42:15.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.477+0000 7f91091ee740 1 --2- 172.21.15.133:0/2692956612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103821b370 0x56103821d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.477+0000 7f91091ee740 1 --2- 172.21.15.133:0/2692956612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610382105d0 0x561038210a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.477+0000 7f91091ee740 1 --2- 172.21.15.133:0/2692956612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038211b40 0x561038210090 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.477+0000 7f91091ee740 1 -- 172.21.15.133:0/2692956612 >> 172.21.15.133:0/2692956612 conn(0x5610381d9310 msgr2=0x56103820cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:15.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.494+0000 7f90fdffb640 1 -- 172.21.15.133:0/2132193097 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f90f40119d0 con 0x7f9100101460 2024-09-11T23:42:15.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.496+0000 7f90fdffb640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f90e80a2dc0 0x7f90e80a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.496+0000 7f9104bb1640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f90e80a2dc0 0x7f90e80a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.496+0000 7f90fdffb640 1 -- 172.21.15.133:0/2132193097 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f90f40ce7a0 con 0x7f9100101460 2024-09-11T23:42:15.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.496+0000 7f91091ee740 1 -- 172.21.15.133:0/2692956612 shutdown_connections 2024-09-11T23:42:15.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.496+0000 7f91091ee740 1 -- 172.21.15.133:0/2692956612 wait complete. 2024-09-11T23:42:15.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.497+0000 7f9104bb1640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f90e80a2dc0 0x7f90e80a5280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x56103820dac0 tx=0x7f90f0008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:15.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.497+0000 7f91091ee740 1 -- start start 2024-09-11T23:42:15.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.497+0000 7f91091ee740 1 Processor -- start 2024-09-11T23:42:15.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.498+0000 7f91091ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x5610382180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.499+0000 7f91091ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561038211b40 0x561038218630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.499+0000 7f91091ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038218b70 0x5610382275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.499+0000 7f91091ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56103821e0b0 con 0x5610382105d0 2024-09-11T23:42:15.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.499+0000 7f91091ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561038131060 con 0x561038218b70 2024-09-11T23:42:15.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.499+0000 7f91091ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56103821e230 con 0x561038211b40 2024-09-11T23:42:15.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.499+0000 7f91091ee740 10 client.0 _ll_register_callbacks cb 0x561037e848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:15.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.500+0000 7f9104bb1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x5610382180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.500+0000 7f9104bb1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x5610382180f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36536/0 (socket says 172.21.15.133:36536) 2024-09-11T23:42:15.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.500+0000 7f9104bb1640 1 -- 172.21.15.133:0/2831247815 learned_addr learned my addr 172.21.15.133:0/2831247815 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:15.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.502+0000 7f9104bb1640 1 -- 172.21.15.133:0/2831247815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561038211b40 msgr2=0x561038218630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:15.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.502+0000 7f9104bb1640 1 --2- 172.21.15.133:0/2831247815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561038211b40 0x561038218630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.502+0000 7f9104bb1640 1 -- 172.21.15.133:0/2831247815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038218b70 msgr2=0x5610382275b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:15.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.502+0000 7f9104bb1640 1 --2- 172.21.15.133:0/2831247815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038218b70 0x5610382275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.502+0000 7f9104bb1640 1 -- 172.21.15.133:0/2831247815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561037ee73c0 con 0x5610382105d0 2024-09-11T23:42:15.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.503+0000 7f9104bb1640 1 --2- 172.21.15.133:0/2831247815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x5610382180f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f90f00146e0 tx=0x7f90f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8107370e7d44a596 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:15.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.507+0000 7f90dffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:42:15.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.507+0000 7f90dffff640 1 -- 172.21.15.133:0/2831247815 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f001b050 con 0x5610382105d0 2024-09-11T23:42:15.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.507+0000 7f90dffff640 1 -- 172.21.15.133:0/2831247815 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f90f0015820 con 0x5610382105d0 2024-09-11T23:42:15.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.508+0000 7f91091ee740 1 -- 172.21.15.133:0/2831247815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90f8003e50 con 0x5610382105d0 2024-09-11T23:42:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.518+0000 7f90dffff640 1 -- 172.21.15.133:0/2831247815 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90f0015a20 con 0x5610382105d0 2024-09-11T23:42:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.518+0000 7f91091ee740 10 client.16911 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.518+0000 7f91091ee740 1 -- 172.21.15.133:0/2831247815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56103822a100 con 0x5610382105d0 2024-09-11T23:42:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.519+0000 7f90dffff640 1 -- 172.21.15.133:0/2831247815 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f8003e50 con 0x5610382105d0 2024-09-11T23:42:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.519+0000 7f90dffff640 1 -- 172.21.15.133:0/2831247815 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f90f00133c0 con 0x5610382105d0 2024-09-11T23:42:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.519+0000 7f91091ee740 10 client.16911 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.519+0000 7f91091ee740 10 client.16911 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:42:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.519+0000 7f91091ee740 20 client.16911 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.519+0000 7f91091ee740 1 --2- 172.21.15.133:0/2831247815 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56103822b9e0 0x56103822dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.520+0000 7f91091ee740 4 client.16911 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.520+0000 7f91091ee740 1 -- 172.21.15.133:0/2831247815 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561038183860 con 0x56103822b9e0 2024-09-11T23:42:15.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.521+0000 7f90fffff640 1 --2- 172.21.15.133:0/2831247815 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56103822b9e0 0x56103822dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.522+0000 7f90fffff640 1 --2- 172.21.15.133:0/2831247815 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56103822b9e0 0x56103822dea0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6a3b5b2114cbefc0 server_cookie=ba342d96866ecf6 in_seq=0 out_seq=0 2024-09-11T23:42:15.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.522+0000 7f90dffff640 10 client.16911 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:42:15.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.523+0000 7f90dffff640 1 -- 172.21.15.133:0/2831247815 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561038183860 con 0x56103822b9e0 2024-09-11T23:42:15.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.523+0000 7f90dffff640 10 client.16911 handle_command_reply: tid=0 2024-09-11T23:42:15.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.523+0000 7f91091ee740 1 client.16911 shutdown 2024-09-11T23:42:15.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.524+0000 7f91091ee740 1 -- 172.21.15.133:0/2831247815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 msgr2=0x5610382180f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:15.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.524+0000 7f91091ee740 1 --2- 172.21.15.133:0/2831247815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x5610382180f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f90f00146e0 tx=0x7f90f00046c0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.526+0000 7f91091ee740 1 -- 172.21.15.133:0/2831247815 shutdown_connections 2024-09-11T23:42:15.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.526+0000 7f91091ee740 1 --2- 172.21.15.133:0/2831247815 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56103822b9e0 0x56103822dea0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.526+0000 7f91091ee740 1 --2- 172.21.15.133:0/2831247815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038218b70 0x5610382275b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.526+0000 7f91091ee740 1 --2- 172.21.15.133:0/2831247815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561038211b40 0x561038218630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.526+0000 7f91091ee740 1 --2- 172.21.15.133:0/2831247815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x5610382180f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.526+0000 7f91091ee740 1 -- 172.21.15.133:0/2831247815 >> 172.21.15.133:0/2831247815 conn(0x5610381d9310 msgr2=0x561038212930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:15.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.526+0000 7f90dffff640 0 client.16911 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:42:15.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.526+0000 7f91091ee740 1 -- 172.21.15.133:0/2831247815 shutdown_connections 2024-09-11T23:42:15.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.526+0000 7f91091ee740 1 -- 172.21.15.133:0/2831247815 wait complete. 2024-09-11T23:42:15.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.529+0000 7f91091ee740 20 client.16911 trim_cache size 0 max 16384 2024-09-11T23:42:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.563+0000 7f91091ee740 1 Processor -- start 2024-09-11T23:42:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.563+0000 7f91091ee740 1 -- start start 2024-09-11T23:42:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.563+0000 7f91091ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x56103820ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.563+0000 7f91091ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561038211b40 0x5610382273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.564+0000 7f91091ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038218b70 0x5610382278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.564+0000 7f91091ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91001269f0 con 0x5610382105d0 2024-09-11T23:42:15.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.564+0000 7f91091ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9100128530 con 0x561038218b70 2024-09-11T23:42:15.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.564+0000 7f91091ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91001270c0 con 0x561038211b40 2024-09-11T23:42:15.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.564+0000 7f9104bb1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x56103820ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.564+0000 7f9104bb1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x56103820ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36542/0 (socket says 172.21.15.133:36542) 2024-09-11T23:42:15.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.564+0000 7f9104bb1640 1 -- 172.21.15.133:0/2481203219 learned_addr learned my addr 172.21.15.133:0/2481203219 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:15.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.565+0000 7f9104bb1640 1 -- 172.21.15.133:0/2481203219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561038211b40 msgr2=0x5610382273a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:15.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.565+0000 7f9104bb1640 1 --2- 172.21.15.133:0/2481203219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561038211b40 0x5610382273a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.565+0000 7f9104bb1640 1 -- 172.21.15.133:0/2481203219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038218b70 msgr2=0x5610382278e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:15.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.565+0000 7f9104bb1640 1 --2- 172.21.15.133:0/2481203219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038218b70 0x5610382278e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.565+0000 7f9104bb1640 1 -- 172.21.15.133:0/2481203219 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561038216640 con 0x5610382105d0 2024-09-11T23:42:15.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.570+0000 7f9104bb1640 1 --2- 172.21.15.133:0/2481203219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x56103820ce10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f90f0004690 tx=0x7f90f000fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1134d6fc03496c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:15.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.571+0000 7f90e67fc640 1 -- 172.21.15.133:0/2481203219 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0038070 con 0x5610382105d0 2024-09-11T23:42:15.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.571+0000 7f90e67fc640 1 -- 172.21.15.133:0/2481203219 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f0025070 con 0x5610382105d0 2024-09-11T23:42:15.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.571+0000 7f90e67fc640 1 -- 172.21.15.133:0/2481203219 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f001b040 con 0x5610382105d0 2024-09-11T23:42:15.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.573+0000 7f91091ee740 1 -- 172.21.15.133:0/2481203219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 msgr2=0x56103820ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:15.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.573+0000 7f91091ee740 1 --2- 172.21.15.133:0/2481203219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x56103820ce10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f90f0004690 tx=0x7f90f000fe40 comp rx=0 tx=0).stop 2024-09-11T23:42:15.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.574+0000 7f91091ee740 1 -- 172.21.15.133:0/2481203219 shutdown_connections 2024-09-11T23:42:15.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.574+0000 7f91091ee740 1 --2- 172.21.15.133:0/2481203219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038218b70 0x5610382278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.574+0000 7f91091ee740 1 --2- 172.21.15.133:0/2481203219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561038211b40 0x5610382273a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.574+0000 7f91091ee740 1 --2- 172.21.15.133:0/2481203219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610382105d0 0x56103820ce10 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.574+0000 7f91091ee740 1 -- 172.21.15.133:0/2481203219 >> 172.21.15.133:0/2481203219 conn(0x5610381d9310 msgr2=0x56103820c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:15.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.575+0000 7f91091ee740 1 -- 172.21.15.133:0/2481203219 shutdown_connections 2024-09-11T23:42:15.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.576+0000 7f91091ee740 1 -- 172.21.15.133:0/2481203219 wait complete. 2024-09-11T23:42:15.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.577+0000 7f91091ee740 1 -- start start 2024-09-11T23:42:15.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.577+0000 7f91091ee740 1 Processor -- start 2024-09-11T23:42:15.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.578+0000 7f91091ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610382105d0 0x56103821e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.578+0000 7f91091ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038211b40 0x56103821ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.578+0000 7f91091ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038218b70 0x56103821ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.578+0000 7f91091ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9100127e60 con 0x561038218b70 2024-09-11T23:42:15.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.578+0000 7f91091ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9100127790 con 0x561038211b40 2024-09-11T23:42:15.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.578+0000 7f91091ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91001249d0 con 0x5610382105d0 2024-09-11T23:42:15.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.579+0000 7f91091ee740 10 client.0 _ll_register_callbacks cb 0x7f90f000b200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:15.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.581+0000 7f91053b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038218b70 0x56103821ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.581+0000 7f91053b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038218b70 0x56103821ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36554/0 (socket says 172.21.15.133:36554) 2024-09-11T23:42:15.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.581+0000 7f91053b2640 1 -- 172.21.15.133:0/981470097 learned_addr learned my addr 172.21.15.133:0/981470097 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:15.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.581+0000 7f90fffff640 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038211b40 0x56103821ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.581+0000 7f9104bb1640 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610382105d0 0x56103821e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.584+0000 7f90fffff640 1 -- 172.21.15.133:0/981470097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610382105d0 msgr2=0x56103821e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:15.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.584+0000 7f90fffff640 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610382105d0 0x56103821e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.584+0000 7f90fffff640 1 -- 172.21.15.133:0/981470097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038218b70 msgr2=0x56103821ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:15.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.584+0000 7f90fffff640 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038218b70 0x56103821ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.584+0000 7f90fffff640 1 -- 172.21.15.133:0/981470097 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561038294770 con 0x561038211b40 2024-09-11T23:42:15.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.585+0000 7f90fffff640 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038211b40 0x56103821ea10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f90f40cc070 tx=0x7f90f4097100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f82469f2609628 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:15.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.585+0000 7f90dffff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:15.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.587+0000 7f90dffff640 1 -- 172.21.15.133:0/981470097 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f409f150 con 0x561038211b40 2024-09-11T23:42:15.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.587+0000 7f91091ee740 1 -- 172.21.15.133:0/981470097 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90f0025070 con 0x561038211b40 2024-09-11T23:42:15.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.588+0000 7f90dffff640 1 -- 172.21.15.133:0/981470097 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f90f40cc1e0 con 0x561038211b40 2024-09-11T23:42:15.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.590+0000 7f90dffff640 1 -- 172.21.15.133:0/981470097 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90f40a3a10 con 0x561038211b40 2024-09-11T23:42:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.593+0000 7f91091ee740 10 client.26384 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.593+0000 7f91091ee740 1 -- 172.21.15.133:0/981470097 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5610382959c0 con 0x561038211b40 2024-09-11T23:42:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.595+0000 7f90dffff640 1 -- 172.21.15.133:0/981470097 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f0025070 con 0x561038211b40 2024-09-11T23:42:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.595+0000 7f90dffff640 1 -- 172.21.15.133:0/981470097 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f90f4079bc0 con 0x561038211b40 2024-09-11T23:42:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.596+0000 7f91091ee740 10 client.26384 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.596+0000 7f91091ee740 10 client.26384 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:42:15.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.596+0000 7f91091ee740 20 client.26384 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:15.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.596+0000 7f91091ee740 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x561038296170 0x561038298590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:15.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.597+0000 7f91091ee740 4 client.26384 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:42:15.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.597+0000 7f91091ee740 1 -- 172.21.15.133:0/981470097 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56103817dd70 con 0x561038296170 2024-09-11T23:42:15.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.597+0000 7f9104bb1640 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x561038296170 0x561038298590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:15.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.598+0000 7f9104bb1640 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x561038296170 0x561038298590 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1bbb4070d64f63c server_cookie=ba37c6f05ef12ad in_seq=0 out_seq=0 2024-09-11T23:42:15.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.601+0000 7f90dffff640 10 client.26384 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:42:15.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.601+0000 7f90dffff640 1 -- 172.21.15.133:0/981470097 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56103817dd70 con 0x561038296170 2024-09-11T23:42:15.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.601+0000 7f90dffff640 10 client.26384 handle_command_reply: tid=0 2024-09-11T23:42:15.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.601+0000 7f91091ee740 1 client.26384 shutdown 2024-09-11T23:42:15.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.605+0000 7f91091ee740 1 -- 172.21.15.133:0/981470097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038211b40 msgr2=0x56103821ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:15.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.605+0000 7f91091ee740 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038211b40 0x56103821ea10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f90f40cc070 tx=0x7f90f4097100 comp rx=0 tx=0).stop 2024-09-11T23:42:15.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.607+0000 7f91091ee740 1 -- 172.21.15.133:0/981470097 shutdown_connections 2024-09-11T23:42:15.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.607+0000 7f91091ee740 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x561038296170 0x561038298590 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.607+0000 7f91091ee740 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561038218b70 0x56103821ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.607+0000 7f91091ee740 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561038211b40 0x56103821ea10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.607+0000 7f91091ee740 1 --2- 172.21.15.133:0/981470097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610382105d0 0x56103821e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.607+0000 7f91091ee740 1 -- 172.21.15.133:0/981470097 >> 172.21.15.133:0/981470097 conn(0x5610381d9310 msgr2=0x56103820b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:15.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.607+0000 7f90dffff640 0 client.26384 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:42:15.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.610+0000 7f91091ee740 1 -- 172.21.15.133:0/981470097 shutdown_connections 2024-09-11T23:42:15.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.611+0000 7f91091ee740 1 -- 172.21.15.133:0/981470097 wait complete. 2024-09-11T23:42:15.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.612+0000 7f91091ee740 20 client.26384 trim_cache size 0 max 16384 2024-09-11T23:42:15.616 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:42:15.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.618+0000 7f9106e15640 1 -- 172.21.15.133:0/2132193097 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f90e80a2dc0 msgr2=0x7f90e80a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:15.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.619+0000 7f9106e15640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f90e80a2dc0 0x7f90e80a5280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x56103820dac0 tx=0x7f90f0008040 comp rx=0 tx=0).stop 2024-09-11T23:42:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.619+0000 7f9106e15640 1 -- 172.21.15.133:0/2132193097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9100101460 msgr2=0x7f910006cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.619+0000 7f9106e15640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9100101460 0x7f910006cac0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b3c0 tx=0x7f90f400b880 comp rx=0 tx=0).stop 2024-09-11T23:42:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.619+0000 7f9106e15640 1 -- 172.21.15.133:0/2132193097 shutdown_connections 2024-09-11T23:42:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.619+0000 7f9106e15640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f90e80a2dc0 0x7f90e80a5280 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x56103820dac0 tx=0x7f90f0008040 comp rx=0 tx=0).stop 2024-09-11T23:42:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.619+0000 7f9106e15640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9100105f50 0x7f910006d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.619+0000 7f9106e15640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9100101460 0x7f910006cac0 secure :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b3c0 tx=0x7f90f400b880 comp rx=0 tx=0).stop 2024-09-11T23:42:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.619+0000 7f9106e15640 1 --2- 172.21.15.133:0/2132193097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9100100b40 0x7f9100073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.619+0000 7f9106e15640 1 -- 172.21.15.133:0/2132193097 >> 172.21.15.133:0/2132193097 conn(0x7f91000fc7d0 msgr2=0x7f9100103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.619+0000 7f9106e15640 1 -- 172.21.15.133:0/2132193097 shutdown_connections 2024-09-11T23:42:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:15.619+0000 7f9106e15640 1 -- 172.21.15.133:0/2132193097 wait complete. 2024-09-11T23:42:15.667 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:15 smithi133 ceph-mon[31430]: pgmap v168: 97 pgs: 97 active+clean; 43 GiB data, 158 GiB used, 915 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 708 MiB/s wr, 4.86k op/s 2024-09-11T23:42:15.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:15 smithi178 ceph-mon[35882]: pgmap v168: 97 pgs: 97 active+clean; 43 GiB data, 158 GiB used, 915 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 708 MiB/s wr, 4.86k op/s 2024-09-11T23:42:15.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:15 smithi138 ceph-mon[35142]: pgmap v168: 97 pgs: 97 active+clean; 43 GiB data, 158 GiB used, 915 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 708 MiB/s wr, 4.86k op/s 2024-09-11T23:42:17.358 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:42:18.279 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:18.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:18 smithi138 ceph-mon[35142]: pgmap v169: 97 pgs: 97 active+clean; 43 GiB data, 159 GiB used, 914 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 705 MiB/s wr, 4.92k op/s 2024-09-11T23:42:18.714 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:18 smithi133 ceph-mon[31430]: pgmap v169: 97 pgs: 97 active+clean; 43 GiB data, 159 GiB used, 914 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 705 MiB/s wr, 4.92k op/s 2024-09-11T23:42:18.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:18 smithi178 ceph-mon[35882]: pgmap v169: 97 pgs: 97 active+clean; 43 GiB data, 159 GiB used, 914 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 705 MiB/s wr, 4.92k op/s 2024-09-11T23:42:19.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.138+0000 7f3af2f81640 1 -- 172.21.15.133:0/2917005635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aec1011e0 msgr2=0x7f3aec105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.138+0000 7f3af2f81640 1 --2- 172.21.15.133:0/2917005635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aec1011e0 0x7f3aec105980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009930 tx=0x7f3ae002a420 comp rx=0 tx=0).stop 2024-09-11T23:42:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.138+0000 7f3af2f81640 1 -- 172.21.15.133:0/2917005635 shutdown_connections 2024-09-11T23:42:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.138+0000 7f3af2f81640 1 --2- 172.21.15.133:0/2917005635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aec105ec0 0x7f3aec106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.138+0000 7f3af2f81640 1 --2- 172.21.15.133:0/2917005635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aec1011e0 0x7f3aec105980 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.138+0000 7f3af2f81640 1 --2- 172.21.15.133:0/2917005635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3aec1008c0 0x7f3aec100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.138+0000 7f3af2f81640 1 -- 172.21.15.133:0/2917005635 >> 172.21.15.133:0/2917005635 conn(0x7f3aec0fc7d0 msgr2=0x7f3aec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.138+0000 7f3af2f81640 1 -- 172.21.15.133:0/2917005635 shutdown_connections 2024-09-11T23:42:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.138+0000 7f3af2f81640 1 -- 172.21.15.133:0/2917005635 wait complete. 2024-09-11T23:42:19.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.139+0000 7f3af2f81640 1 Processor -- start 2024-09-11T23:42:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.140+0000 7f3af2f81640 1 -- start start 2024-09-11T23:42:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.140+0000 7f3af2f81640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aec1008c0 0x7f3aec073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.140+0000 7f3af2f81640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3aec105ec0 0x7f3aec073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.140+0000 7f3af2f81640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aec06c7e0 0x7f3aec06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.140+0000 7f3af2f81640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec118680 con 0x7f3aec105ec0 2024-09-11T23:42:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.140+0000 7f3af2f81640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec118500 con 0x7f3aec06c7e0 2024-09-11T23:42:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.140+0000 7f3af2f81640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec118800 con 0x7f3aec1008c0 2024-09-11T23:42:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.143+0000 7f3af0d1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aec1008c0 0x7f3aec073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.143+0000 7f3af0d1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aec1008c0 0x7f3aec073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:38554/0 (socket says 172.21.15.133:38554) 2024-09-11T23:42:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.143+0000 7f3af0d1d640 1 -- 172.21.15.133:0/2459449801 learned_addr learned my addr 172.21.15.133:0/2459449801 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.143+0000 7f3af151e640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aec06c7e0 0x7f3aec06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.147+0000 7f3aebfff640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3aec105ec0 0x7f3aec073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.157+0000 7f3af0d1d640 1 -- 172.21.15.133:0/2459449801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aec06c7e0 msgr2=0x7f3aec06cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:19.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.157+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aec06c7e0 0x7f3aec06cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.157+0000 7f3af0d1d640 1 -- 172.21.15.133:0/2459449801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3aec105ec0 msgr2=0x7f3aec073c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:19.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.157+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3aec105ec0 0x7f3aec073c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.157+0000 7f3af0d1d640 1 -- 172.21.15.133:0/2459449801 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec06d4b0 con 0x7f3aec1008c0 2024-09-11T23:42:19.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.162+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aec1008c0 0x7f3aec073750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009d50 tx=0x7f3ae400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:19.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.163+0000 7f3ae9ffb640 1 -- 172.21.15.133:0/2459449801 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4003b70 con 0x7f3aec1008c0 2024-09-11T23:42:19.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.163+0000 7f3af2f81640 1 -- 172.21.15.133:0/2459449801 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aec071280 con 0x7f3aec1008c0 2024-09-11T23:42:19.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.163+0000 7f3af2f81640 1 -- 172.21.15.133:0/2459449801 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aec071740 con 0x7f3aec1008c0 2024-09-11T23:42:19.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.164+0000 7f3ae9ffb640 1 -- 172.21.15.133:0/2459449801 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae4003d10 con 0x7f3aec1008c0 2024-09-11T23:42:19.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.164+0000 7f3ae9ffb640 1 -- 172.21.15.133:0/2459449801 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4010a20 con 0x7f3aec1008c0 2024-09-11T23:42:19.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.172+0000 7f3af535a740 1 Processor -- start 2024-09-11T23:42:19.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.173+0000 7f3af535a740 1 -- start start 2024-09-11T23:42:19.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.173+0000 7f3af535a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b8b40 0x561e784b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.173+0000 7f3af535a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b75d0 0x561e784b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.174+0000 7f3af535a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784c2370 0x561e784c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.174+0000 7f3af535a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561e7841d2f0 con 0x561e784b75d0 2024-09-11T23:42:19.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.174+0000 7f3af535a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e783c4cf0 con 0x561e784c2370 2024-09-11T23:42:19.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.174+0000 7f3af535a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e784192b0 con 0x561e784b8b40 2024-09-11T23:42:19.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.174+0000 7f3ae9ffb640 1 -- 172.21.15.133:0/2459449801 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3ae4004000 con 0x7f3aec1008c0 2024-09-11T23:42:19.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.175+0000 7f3aebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784c2370 0x561e784c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.175+0000 7f3aebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784c2370 0x561e784c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:55658/0 (socket says 172.21.15.133:55658) 2024-09-11T23:42:19.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.175+0000 7f3aebfff640 1 -- 172.21.15.133:0/675643213 learned_addr learned my addr 172.21.15.133:0/675643213 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:19.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.175+0000 7f3af151e640 1 --2- 172.21.15.133:0/675643213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b8b40 0x561e784b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.176+0000 7f3ae9ffb640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ad40a2f00 0x7f3ad40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.177+0000 7f3ae9ffb640 1 -- 172.21.15.133:0/2459449801 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3ae40cfa40 con 0x7f3aec1008c0 2024-09-11T23:42:19.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.181+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/675643213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b75d0 0x561e784b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.183+0000 7f3aebfff640 1 -- 172.21.15.133:0/675643213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b8b40 msgr2=0x561e784b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:19.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.183+0000 7f3aebfff640 1 --2- 172.21.15.133:0/675643213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b8b40 0x561e784b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.183+0000 7f3aebfff640 1 -- 172.21.15.133:0/675643213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b75d0 msgr2=0x561e784b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:19.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.183+0000 7f3aebfff640 1 --2- 172.21.15.133:0/675643213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b75d0 0x561e784b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.183+0000 7f3aebfff640 1 -- 172.21.15.133:0/675643213 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e783a93a0 con 0x561e784c2370 2024-09-11T23:42:19.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.186+0000 7f3af151e640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ad40a2f00 0x7f3ad40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.188+0000 7f3aebfff640 1 --2- 172.21.15.133:0/675643213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784c2370 0x561e784c4760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009230 tx=0x7f3ae0009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d76fabd5eb399b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:19.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.191+0000 7f3af151e640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ad40a2f00 0x7f3ad40a53c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009fd0 tx=0x7f3adc004730 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:19.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.192+0000 7f3ad2ffd640 1 -- 172.21.15.133:0/675643213 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae003f070 con 0x561e784c2370 2024-09-11T23:42:19.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.192+0000 7f3ad2ffd640 1 -- 172.21.15.133:0/675643213 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae00316c0 con 0x561e784c2370 2024-09-11T23:42:19.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.193+0000 7f3af535a740 1 -- 172.21.15.133:0/675643213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784c2370 msgr2=0x561e784c4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:19.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.193+0000 7f3af535a740 1 --2- 172.21.15.133:0/675643213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784c2370 0x561e784c4760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009230 tx=0x7f3ae0009460 comp rx=0 tx=0).stop 2024-09-11T23:42:19.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.194+0000 7f3af535a740 1 -- 172.21.15.133:0/675643213 shutdown_connections 2024-09-11T23:42:19.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.194+0000 7f3af535a740 1 --2- 172.21.15.133:0/675643213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784c2370 0x561e784c4760 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.194+0000 7f3af535a740 1 --2- 172.21.15.133:0/675643213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b75d0 0x561e784b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.194+0000 7f3af535a740 1 --2- 172.21.15.133:0/675643213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b8b40 0x561e784b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.194+0000 7f3af535a740 1 -- 172.21.15.133:0/675643213 >> 172.21.15.133:0/675643213 conn(0x561e78480310 msgr2=0x561e784b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:19.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.204+0000 7f3af535a740 1 -- 172.21.15.133:0/675643213 shutdown_connections 2024-09-11T23:42:19.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.204+0000 7f3af535a740 1 -- 172.21.15.133:0/675643213 wait complete. 2024-09-11T23:42:19.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.205+0000 7f3af535a740 1 -- start start 2024-09-11T23:42:19.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.206+0000 7f3af535a740 1 Processor -- start 2024-09-11T23:42:19.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.207+0000 7f3af535a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b75d0 0x561e784c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.207+0000 7f3af535a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b8b40 0x561e784beec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.207+0000 7f3af535a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784c2370 0x561e784bf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.207+0000 7f3af535a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561e784c5060 con 0x561e784b8b40 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.207+0000 7f3af535a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e783d8060 con 0x561e784b75d0 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.207+0000 7f3af535a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e784c51e0 con 0x561e784c2370 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.207+0000 7f3af535a740 10 client.0 _ll_register_callbacks cb 0x561e7812b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.214+0000 7f3af0d1d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b75d0 0x561e784c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.215+0000 7f3aebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b8b40 0x561e784beec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.215+0000 7f3aebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b8b40 0x561e784beec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36596/0 (socket says 172.21.15.133:36596) 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.215+0000 7f3aebfff640 1 -- 172.21.15.133:0/148773269 learned_addr learned my addr 172.21.15.133:0/148773269 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.216+0000 7f3af151e640 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784c2370 0x561e784bf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.217+0000 7f3aebfff640 1 -- 172.21.15.133:0/148773269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784c2370 msgr2=0x561e784bf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.217+0000 7f3aebfff640 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784c2370 0x561e784bf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.217+0000 7f3aebfff640 1 -- 172.21.15.133:0/148773269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b75d0 msgr2=0x561e784c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:19.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.217+0000 7f3aebfff640 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b75d0 0x561e784c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.217+0000 7f3aebfff640 1 -- 172.21.15.133:0/148773269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7818e3c0 con 0x561e784b8b40 2024-09-11T23:42:19.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.226+0000 7f3aebfff640 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b8b40 0x561e784beec0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0006010 tx=0x7f3ae0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9effcf00dd1bfe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.227+0000 7f3acbfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.228+0000 7f3acbfff640 1 -- 172.21.15.133:0/148773269 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae003f070 con 0x561e784b8b40 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.228+0000 7f3af535a740 1 -- 172.21.15.133:0/148773269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae00316c0 con 0x561e784b8b40 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.228+0000 7f3acbfff640 1 -- 172.21.15.133:0/148773269 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3ae00314c0 con 0x561e784b8b40 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.229+0000 7f3acbfff640 1 -- 172.21.15.133:0/148773269 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ae00035f0 con 0x561e784b8b40 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.229+0000 7f3af535a740 10 client.16929 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.229+0000 7f3af535a740 1 -- 172.21.15.133:0/148773269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e784d0a70 con 0x561e784b8b40 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.230+0000 7f3acbfff640 1 -- 172.21.15.133:0/148773269 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae00316c0 con 0x561e784b8b40 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.232+0000 7f3acbfff640 1 -- 172.21.15.133:0/148773269 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3ae0004dc0 con 0x561e784b8b40 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.232+0000 7f3af535a740 10 client.16929 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.232+0000 7f3af535a740 10 client.16929 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:42:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.232+0000 7f3af535a740 20 client.16929 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:19.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.232+0000 7f3af535a740 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561e784d1dc0 0x561e784d41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.232+0000 7f3af535a740 4 client.16929 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:19.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.232+0000 7f3af535a740 1 -- 172.21.15.133:0/148773269 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e7842a860 con 0x561e784d1dc0 2024-09-11T23:42:19.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.238+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561e784d1dc0 0x561e784d41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.258+0000 7f3acbfff640 10 client.16929 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:42:19.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.258+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561e784d1dc0 0x561e784d41b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6fb87a4fb215d2b server_cookie=9676c4cdc50cb36e in_seq=0 out_seq=0 2024-09-11T23:42:19.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.259+0000 7f3acbfff640 1 -- 172.21.15.133:0/148773269 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e7842a860 con 0x561e784d1dc0 2024-09-11T23:42:19.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.260+0000 7f3acbfff640 10 client.16929 handle_command_reply: tid=0 2024-09-11T23:42:19.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.263+0000 7f3af535a740 1 client.16929 shutdown 2024-09-11T23:42:19.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.265+0000 7f3af535a740 1 -- 172.21.15.133:0/148773269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b8b40 msgr2=0x561e784beec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:19.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.265+0000 7f3af535a740 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b8b40 0x561e784beec0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0006010 tx=0x7f3ae0004270 comp rx=0 tx=0).stop 2024-09-11T23:42:19.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.267+0000 7f3af535a740 1 -- 172.21.15.133:0/148773269 shutdown_connections 2024-09-11T23:42:19.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.267+0000 7f3af535a740 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561e784d1dc0 0x561e784d41b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.267+0000 7f3af535a740 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784c2370 0x561e784bf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.267+0000 7f3af535a740 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b8b40 0x561e784beec0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.267+0000 7f3af535a740 1 --2- 172.21.15.133:0/148773269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b75d0 0x561e784c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.267+0000 7f3af535a740 1 -- 172.21.15.133:0/148773269 >> 172.21.15.133:0/148773269 conn(0x561e78480310 msgr2=0x561e784b9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:19.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.267+0000 7f3acbfff640 0 client.16929 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:42:19.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.267+0000 7f3af535a740 1 -- 172.21.15.133:0/148773269 shutdown_connections 2024-09-11T23:42:19.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.267+0000 7f3af535a740 1 -- 172.21.15.133:0/148773269 wait complete. 2024-09-11T23:42:19.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.267+0000 7f3af535a740 20 client.16929 trim_cache size 0 max 16384 2024-09-11T23:42:19.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.293+0000 7f3af535a740 1 Processor -- start 2024-09-11T23:42:19.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.293+0000 7f3af535a740 1 -- start start 2024-09-11T23:42:19.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.294+0000 7f3af535a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b75d0 0x561e784ba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.294+0000 7f3af535a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 0x561e784bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.295+0000 7f3af0d1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b75d0 0x561e784ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.295+0000 7f3af0d1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b75d0 0x561e784ba310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:38600/0 (socket says 172.21.15.133:38600) 2024-09-11T23:42:19.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.295+0000 7f3af0d1d640 1 -- 172.21.15.133:0/2132752520 learned_addr learned my addr 172.21.15.133:0/2132752520 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:19.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.295+0000 7f3aebfff640 1 --2- 172.21.15.133:0/2132752520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 0x561e784bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.296+0000 7f3af535a740 1 --2- 172.21.15.133:0/2132752520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784c2370 0x561e784bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.296+0000 7f3af535a740 1 -- 172.21.15.133:0/2132752520 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad800e5b0 con 0x561e784c2370 2024-09-11T23:42:19.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.296+0000 7f3af151e640 1 --2- 172.21.15.133:0/2132752520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784c2370 0x561e784bdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.297+0000 7f3af535a740 1 -- 172.21.15.133:0/2132752520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad80100f0 con 0x561e784b8b40 2024-09-11T23:42:19.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.297+0000 7f3af535a740 1 -- 172.21.15.133:0/2132752520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad800ec80 con 0x561e784b75d0 2024-09-11T23:42:19.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.298+0000 7f3af0d1d640 1 -- 172.21.15.133:0/2132752520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 msgr2=0x561e784bd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:19.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.298+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/2132752520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 0x561e784bd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.298+0000 7f3af0d1d640 1 -- 172.21.15.133:0/2132752520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784c2370 msgr2=0x561e784bdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:19.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.298+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/2132752520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784c2370 0x561e784bdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.298+0000 7f3af0d1d640 1 -- 172.21.15.133:0/2132752520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e784baae0 con 0x561e784b75d0 2024-09-11T23:42:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.301+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/2132752520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b75d0 0x561e784ba310 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4098940 tx=0x7f3ae4005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56de54db61b239a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.303+0000 7f3ad27fc640 1 -- 172.21.15.133:0/2132752520 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae409bf70 con 0x561e784b75d0 2024-09-11T23:42:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.303+0000 7f3ad27fc640 1 -- 172.21.15.133:0/2132752520 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae40a0320 con 0x561e784b75d0 2024-09-11T23:42:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.303+0000 7f3ad27fc640 1 -- 172.21.15.133:0/2132752520 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40cd420 con 0x561e784b75d0 2024-09-11T23:42:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.305+0000 7f3af535a740 1 -- 172.21.15.133:0/2132752520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b75d0 msgr2=0x561e784ba310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.305+0000 7f3af535a740 1 --2- 172.21.15.133:0/2132752520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b75d0 0x561e784ba310 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4098940 tx=0x7f3ae4005f30 comp rx=0 tx=0).stop 2024-09-11T23:42:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.305+0000 7f3af535a740 1 -- 172.21.15.133:0/2132752520 shutdown_connections 2024-09-11T23:42:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.305+0000 7f3af535a740 1 --2- 172.21.15.133:0/2132752520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784c2370 0x561e784bdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.305+0000 7f3af535a740 1 --2- 172.21.15.133:0/2132752520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 0x561e784bd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.305+0000 7f3af535a740 1 --2- 172.21.15.133:0/2132752520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784b75d0 0x561e784ba310 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.305+0000 7f3af535a740 1 -- 172.21.15.133:0/2132752520 >> 172.21.15.133:0/2132752520 conn(0x561e78480310 msgr2=0x561e784bee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.305+0000 7f3af535a740 1 -- 172.21.15.133:0/2132752520 shutdown_connections 2024-09-11T23:42:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.305+0000 7f3af535a740 1 -- 172.21.15.133:0/2132752520 wait complete. 2024-09-11T23:42:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.306+0000 7f3af535a740 1 -- start start 2024-09-11T23:42:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.306+0000 7f3af535a740 1 Processor -- start 2024-09-11T23:42:19.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.308+0000 7f3af535a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b75d0 0x561e784d7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.308+0000 7f3af535a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 0x561e784d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.308+0000 7f3af535a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784c2370 0x561e784d4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.308+0000 7f3aebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 0x561e784d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.308+0000 7f3aebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 0x561e784d4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:55694/0 (socket says 172.21.15.133:55694) 2024-09-11T23:42:19.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.308+0000 7f3aebfff640 1 -- 172.21.15.133:0/984394784 learned_addr learned my addr 172.21.15.133:0/984394784 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:19.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.308+0000 7f3af535a740 1 -- 172.21.15.133:0/984394784 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad800fa20 con 0x561e784b75d0 2024-09-11T23:42:19.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.309+0000 7f3af151e640 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784c2370 0x561e784d4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.309+0000 7f3af535a740 1 -- 172.21.15.133:0/984394784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad800f350 con 0x561e784b8b40 2024-09-11T23:42:19.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.309+0000 7f3af535a740 1 -- 172.21.15.133:0/984394784 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad800c640 con 0x561e784c2370 2024-09-11T23:42:19.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.309+0000 7f3af535a740 10 client.0 _ll_register_callbacks cb 0x7f3aec074370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:19.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.309+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b75d0 0x561e784d7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.309+0000 7f3aebfff640 1 -- 172.21.15.133:0/984394784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784c2370 msgr2=0x561e784d4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:19.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.309+0000 7f3aebfff640 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784c2370 0x561e784d4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.309+0000 7f3aebfff640 1 -- 172.21.15.133:0/984394784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b75d0 msgr2=0x561e784d7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:19.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.309+0000 7f3aebfff640 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b75d0 0x561e784d7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.309+0000 7f3aebfff640 1 -- 172.21.15.133:0/984394784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e784d5150 con 0x561e784b8b40 2024-09-11T23:42:19.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.310+0000 7f3aebfff640 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 0x561e784d4650 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3ae002aa10 tx=0x7f3ae003b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7910de9620c91f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:19.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.316+0000 7f3acbfff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:19.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.316+0000 7f3acbfff640 1 -- 172.21.15.133:0/984394784 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae003f040 con 0x561e784b8b40 2024-09-11T23:42:19.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.316+0000 7f3acbfff640 1 -- 172.21.15.133:0/984394784 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3ae00055c0 con 0x561e784b8b40 2024-09-11T23:42:19.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.316+0000 7f3af535a740 1 -- 172.21.15.133:0/984394784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae40a0320 con 0x561e784b8b40 2024-09-11T23:42:19.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.318+0000 7f3acbfff640 1 -- 172.21.15.133:0/984394784 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ae0005d50 con 0x561e784b8b40 2024-09-11T23:42:19.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.318+0000 7f3af535a740 10 client.26399 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:19.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.318+0000 7f3af535a740 1 -- 172.21.15.133:0/984394784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e7853c470 con 0x561e784b8b40 2024-09-11T23:42:19.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.319+0000 7f3acbfff640 1 -- 172.21.15.133:0/984394784 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae0003c70 con 0x561e784b8b40 2024-09-11T23:42:19.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.319+0000 7f3acbfff640 1 -- 172.21.15.133:0/984394784 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3ae0004e10 con 0x561e784b8b40 2024-09-11T23:42:19.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.319+0000 7f3af535a740 10 client.26399 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:19.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.319+0000 7f3af535a740 10 client.26399 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:42:19.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.320+0000 7f3af535a740 20 client.26399 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:19.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.320+0000 7f3af535a740 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561e7853cb30 0x561e7853ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:19.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.320+0000 7f3af535a740 4 client.26399 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:42:19.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.320+0000 7f3af535a740 1 -- 172.21.15.133:0/984394784 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3ae40a0320 con 0x561e7853cb30 2024-09-11T23:42:19.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.321+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561e7853cb30 0x561e7853ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:19.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.324+0000 7f3af0d1d640 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561e7853cb30 0x561e7853ef50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3ec9b73f5f4429 server_cookie=a5d142f203c2ede in_seq=0 out_seq=0 2024-09-11T23:42:19.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.324+0000 7f3acbfff640 10 client.26399 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:42:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.325+0000 7f3acbfff640 1 -- 172.21.15.133:0/984394784 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3ae40a0320 con 0x561e7853cb30 2024-09-11T23:42:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.325+0000 7f3acbfff640 10 client.26399 handle_command_reply: tid=0 2024-09-11T23:42:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.325+0000 7f3af535a740 1 client.26399 shutdown 2024-09-11T23:42:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.326+0000 7f3af535a740 1 -- 172.21.15.133:0/984394784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 msgr2=0x561e784d4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.326+0000 7f3af535a740 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 0x561e784d4650 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3ae002aa10 tx=0x7f3ae003b670 comp rx=0 tx=0).stop 2024-09-11T23:42:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.327+0000 7f3af535a740 1 -- 172.21.15.133:0/984394784 shutdown_connections 2024-09-11T23:42:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.327+0000 7f3af535a740 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x561e7853cb30 0x561e7853ef50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.327+0000 7f3af535a740 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e784c2370 0x561e784d4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.327+0000 7f3af535a740 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e784b8b40 0x561e784d4650 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.327+0000 7f3af535a740 1 --2- 172.21.15.133:0/984394784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e784b75d0 0x561e784d7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.327+0000 7f3af535a740 1 -- 172.21.15.133:0/984394784 >> 172.21.15.133:0/984394784 conn(0x561e78480310 msgr2=0x561e784bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.327+0000 7f3acbfff640 0 client.26399 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:42:19.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.329+0000 7f3af535a740 1 -- 172.21.15.133:0/984394784 shutdown_connections 2024-09-11T23:42:19.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.329+0000 7f3af535a740 1 -- 172.21.15.133:0/984394784 wait complete. 2024-09-11T23:42:19.330 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:42:19.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.329+0000 7f3af535a740 20 client.26399 trim_cache size 0 max 16384 2024-09-11T23:42:19.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 -- 172.21.15.133:0/2459449801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ad40a2f00 msgr2=0x7f3ad40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:19.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ad40a2f00 0x7f3ad40a53c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009fd0 tx=0x7f3adc004730 comp rx=0 tx=0).stop 2024-09-11T23:42:19.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 -- 172.21.15.133:0/2459449801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aec1008c0 msgr2=0x7f3aec073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:19.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aec1008c0 0x7f3aec073750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009d50 tx=0x7f3ae400c6a0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 -- 172.21.15.133:0/2459449801 shutdown_connections 2024-09-11T23:42:19.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ad40a2f00 0x7f3ad40a53c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aec06c7e0 0x7f3aec06cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3aec105ec0 0x7f3aec073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 --2- 172.21.15.133:0/2459449801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aec1008c0 0x7f3aec073750 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:19.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 -- 172.21.15.133:0/2459449801 >> 172.21.15.133:0/2459449801 conn(0x7f3aec0fc7d0 msgr2=0x7f3aec1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:19.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 -- 172.21.15.133:0/2459449801 shutdown_connections 2024-09-11T23:42:19.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:19.335+0000 7f3af2f81640 1 -- 172.21.15.133:0/2459449801 wait complete. 2024-09-11T23:42:20.417 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:42:20.649 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:20 smithi133 ceph-mon[31430]: pgmap v170: 97 pgs: 97 active+clean; 45 GiB data, 164 GiB used, 909 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 613 MiB/s wr, 3.71k op/s 2024-09-11T23:42:20.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:20 smithi178 ceph-mon[35882]: pgmap v170: 97 pgs: 97 active+clean; 45 GiB data, 164 GiB used, 909 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 613 MiB/s wr, 3.71k op/s 2024-09-11T23:42:20.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:20 smithi138 ceph-mon[35142]: pgmap v170: 97 pgs: 97 active+clean; 45 GiB data, 164 GiB used, 909 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 613 MiB/s wr, 3.71k op/s 2024-09-11T23:42:21.111 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:21.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:21 smithi133 ceph-mon[31430]: pgmap v171: 97 pgs: 97 active+clean; 45 GiB data, 165 GiB used, 908 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 616 MiB/s wr, 3.72k op/s 2024-09-11T23:42:21.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:21 smithi138 ceph-mon[35142]: pgmap v171: 97 pgs: 97 active+clean; 45 GiB data, 165 GiB used, 908 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 616 MiB/s wr, 3.72k op/s 2024-09-11T23:42:22.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:21 smithi178 ceph-mon[35882]: pgmap v171: 97 pgs: 97 active+clean; 45 GiB data, 165 GiB used, 908 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 616 MiB/s wr, 3.72k op/s 2024-09-11T23:42:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.316+0000 7f70b49f2640 1 -- 172.21.15.133:0/3494088550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0110190 msgr2=0x7f70b006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.316+0000 7f70b49f2640 1 --2- 172.21.15.133:0/3494088550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0110190 0x7f70b006e4f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009860 tx=0x7f70a802a380 comp rx=0 tx=0).stop 2024-09-11T23:42:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.317+0000 7f70b49f2640 1 -- 172.21.15.133:0/3494088550 shutdown_connections 2024-09-11T23:42:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.317+0000 7f70b49f2640 1 --2- 172.21.15.133:0/3494088550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b011b140 0x7f70b011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.317+0000 7f70b49f2640 1 --2- 172.21.15.133:0/3494088550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b006ea30 0x7f70b006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.317+0000 7f70b49f2640 1 --2- 172.21.15.133:0/3494088550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0110190 0x7f70b006e4f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.317+0000 7f70b49f2640 1 -- 172.21.15.133:0/3494088550 >> 172.21.15.133:0/3494088550 conn(0x7f70b0073180 msgr2=0x7f70b0073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.320+0000 7f70b49f2640 1 -- 172.21.15.133:0/3494088550 shutdown_connections 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.322+0000 7f70b49f2640 1 -- 172.21.15.133:0/3494088550 wait complete. 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.323+0000 7f70b49f2640 1 Processor -- start 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.324+0000 7f70b49f2640 1 -- start start 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.324+0000 7f70b49f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b006ea30 0x7f70b011a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.325+0000 7f70b49f2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0110190 0x7f70b01168a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.325+0000 7f70b49f2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b011b140 0x7f70b0116de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.325+0000 7f70b49f2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0113300 con 0x7f70b006ea30 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.325+0000 7f70b49f2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0113180 con 0x7f70b0110190 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.325+0000 7f70b49f2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0113480 con 0x7f70b011b140 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.329+0000 7f70aed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b006ea30 0x7f70b011a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.329+0000 7f70aed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b006ea30 0x7f70b011a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41042/0 (socket says 172.21.15.133:41042) 2024-09-11T23:42:22.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.329+0000 7f70aed9d640 1 -- 172.21.15.133:0/2816401639 learned_addr learned my addr 172.21.15.133:0/2816401639 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:22.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.329+0000 7f70af59e640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b011b140 0x7f70b0116de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.329+0000 7f70aed9d640 1 -- 172.21.15.133:0/2816401639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b011b140 msgr2=0x7f70b0116de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:22.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.329+0000 7f70aed9d640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b011b140 0x7f70b0116de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.330+0000 7f70aed9d640 1 -- 172.21.15.133:0/2816401639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0110190 msgr2=0x7f70b01168a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:22.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.330+0000 7f70aed9d640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0110190 0x7f70b01168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.330+0000 7f70aed9d640 1 -- 172.21.15.133:0/2816401639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b0117510 con 0x7f70b006ea30 2024-09-11T23:42:22.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.331+0000 7f70aed9d640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b006ea30 0x7f70b011a800 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f70a80092e0 tx=0x7f70a8030f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:22.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.331+0000 7f7097fff640 1 -- 172.21.15.133:0/2816401639 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8037040 con 0x7f70b006ea30 2024-09-11T23:42:22.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.332+0000 7f70b49f2640 1 -- 172.21.15.133:0/2816401639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b007a450 con 0x7f70b006ea30 2024-09-11T23:42:22.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.332+0000 7f70b49f2640 1 -- 172.21.15.133:0/2816401639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b007a930 con 0x7f70b006ea30 2024-09-11T23:42:22.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.336+0000 7f70b75cc740 1 Processor -- start 2024-09-11T23:42:22.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.337+0000 7f7097fff640 1 -- 172.21.15.133:0/2816401639 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a8040070 con 0x7f70b006ea30 2024-09-11T23:42:22.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.337+0000 7f7097fff640 1 -- 172.21.15.133:0/2816401639 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8032d60 con 0x7f70b006ea30 2024-09-11T23:42:22.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.341+0000 7f70b75cc740 1 -- start start 2024-09-11T23:42:22.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.342+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e8bcf0 0x55c686e8a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.342+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e8a780 0x55c686e8abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.342+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e95520 0x55c686e97910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.342+0000 7f70b75cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c686df02f0 con 0x55c686e95520 2024-09-11T23:42:22.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.342+0000 7f70b75cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c686d97cf0 con 0x55c686e8bcf0 2024-09-11T23:42:22.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.342+0000 7f70b75cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c686dec2b0 con 0x55c686e8a780 2024-09-11T23:42:22.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.343+0000 7f70aed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e8a780 0x55c686e8abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.348+0000 7f70ae59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e95520 0x55c686e97910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.348+0000 7f70ae59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e95520 0x55c686e97910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41046/0 (socket says 172.21.15.133:41046) 2024-09-11T23:42:22.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.348+0000 7f70ae59c640 1 -- 172.21.15.133:0/2389007687 learned_addr learned my addr 172.21.15.133:0/2389007687 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:22.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.352+0000 7f70ae59c640 1 -- 172.21.15.133:0/2389007687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e8a780 msgr2=0x55c686e8abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:22.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.352+0000 7f70ae59c640 1 --2- 172.21.15.133:0/2389007687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e8a780 0x55c686e8abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.352+0000 7f70ae59c640 1 -- 172.21.15.133:0/2389007687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e8bcf0 msgr2=0x55c686e8a240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:22.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.352+0000 7f70ae59c640 1 --2- 172.21.15.133:0/2389007687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e8bcf0 0x55c686e8a240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.353+0000 7f70ae59c640 1 -- 172.21.15.133:0/2389007687 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c686d7c3a0 con 0x55c686e95520 2024-09-11T23:42:22.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.359+0000 7f7097fff640 1 -- 172.21.15.133:0/2816401639 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f70a8032440 con 0x7f70b006ea30 2024-09-11T23:42:22.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.360+0000 7f7097fff640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70980a2f00 0x7f70980a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.361+0000 7f70ae59c640 1 --2- 172.21.15.133:0/2389007687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e95520 0x55c686e97910 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009da0 tx=0x7f70a000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0245b3ee560e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:22.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.361+0000 7f7097fff640 1 -- 172.21.15.133:0/2816401639 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f70a80f2620 con 0x7f70b006ea30 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.361+0000 7f70957fa640 1 -- 172.21.15.133:0/2389007687 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0016070 con 0x55c686e95520 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.361+0000 7f70957fa640 1 -- 172.21.15.133:0/2389007687 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a0005020 con 0x55c686e95520 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.363+0000 7f70b75cc740 1 -- 172.21.15.133:0/2389007687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e95520 msgr2=0x55c686e97910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.363+0000 7f70b75cc740 1 --2- 172.21.15.133:0/2389007687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e95520 0x55c686e97910 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009da0 tx=0x7f70a000e6c0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.364+0000 7f70b75cc740 1 -- 172.21.15.133:0/2389007687 shutdown_connections 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.364+0000 7f70b75cc740 1 --2- 172.21.15.133:0/2389007687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e95520 0x55c686e97910 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.364+0000 7f70b75cc740 1 --2- 172.21.15.133:0/2389007687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e8a780 0x55c686e8abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.364+0000 7f70b75cc740 1 --2- 172.21.15.133:0/2389007687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e8bcf0 0x55c686e8a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.364+0000 7f70b75cc740 1 -- 172.21.15.133:0/2389007687 >> 172.21.15.133:0/2389007687 conn(0x55c686e53310 msgr2=0x55c686e86e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.365+0000 7f70b75cc740 1 -- 172.21.15.133:0/2389007687 shutdown_connections 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.365+0000 7f70b75cc740 1 -- 172.21.15.133:0/2389007687 wait complete. 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.365+0000 7f70b75cc740 1 -- start start 2024-09-11T23:42:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.366+0000 7f70b75cc740 1 Processor -- start 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.366+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e8a780 0x55c686e94ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.366+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e8bcf0 0x55c686e91fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.366+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e95520 0x55c686e92520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.366+0000 7f70b75cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c686e98270 con 0x55c686e8a780 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.366+0000 7f70b75cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c686dab060 con 0x55c686e95520 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.366+0000 7f70b75cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c686e983f0 con 0x55c686e8bcf0 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.366+0000 7f70b75cc740 10 client.0 _ll_register_callbacks cb 0x55c686afe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.367+0000 7f70aed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e8a780 0x55c686e94ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.367+0000 7f70aed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e8a780 0x55c686e94ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41062/0 (socket says 172.21.15.133:41062) 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.367+0000 7f70aed9d640 1 -- 172.21.15.133:0/4211482902 learned_addr learned my addr 172.21.15.133:0/4211482902 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.367+0000 7f70af59e640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70980a2f00 0x7f70980a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.368+0000 7f70ae59c640 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e8bcf0 0x55c686e91fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.368+0000 7f70af59e640 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e95520 0x55c686e92520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.369+0000 7f70ae59c640 1 -- 172.21.15.133:0/4211482902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e95520 msgr2=0x55c686e92520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.369+0000 7f70ae59c640 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e95520 0x55c686e92520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.369+0000 7f70ae59c640 1 -- 172.21.15.133:0/4211482902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e8a780 msgr2=0x55c686e94ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.369+0000 7f70ae59c640 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e8a780 0x55c686e94ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.369+0000 7f70ae59c640 1 -- 172.21.15.133:0/4211482902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c686b613c0 con 0x55c686e8bcf0 2024-09-11T23:42:22.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.370+0000 7f70ae59c640 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e8bcf0 0x55c686e91fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f70a000ebc0 tx=0x7f70a0003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec2f9d03231fb045 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:22.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.370+0000 7f70af59e640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70980a2f00 0x7f70980a53c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f70a4005fd0 tx=0x7f70a401af40 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:22.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.370+0000 7f7092ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:42:22.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.370+0000 7f7092ff5640 1 -- 172.21.15.133:0/4211482902 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0016070 con 0x55c686e8bcf0 2024-09-11T23:42:22.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.370+0000 7f7092ff5640 1 -- 172.21.15.133:0/4211482902 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f70a0011930 con 0x55c686e8bcf0 2024-09-11T23:42:22.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.372+0000 7f70b75cc740 1 -- 172.21.15.133:0/4211482902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a0005020 con 0x55c686e8bcf0 2024-09-11T23:42:22.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.377+0000 7f7092ff5640 1 -- 172.21.15.133:0/4211482902 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70a0011b30 con 0x55c686e8bcf0 2024-09-11T23:42:22.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.377+0000 7f70b75cc740 10 client.26368 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:22.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.377+0000 7f70b75cc740 1 -- 172.21.15.133:0/4211482902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c686ea3c10 con 0x55c686e8bcf0 2024-09-11T23:42:22.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.384+0000 7f7092ff5640 1 -- 172.21.15.133:0/4211482902 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a0005020 con 0x55c686e8bcf0 2024-09-11T23:42:22.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.385+0000 7f7092ff5640 1 -- 172.21.15.133:0/4211482902 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f70a000f040 con 0x55c686e8bcf0 2024-09-11T23:42:22.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.387+0000 7f70b75cc740 10 client.26368 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:22.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.387+0000 7f70b75cc740 10 client.26368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:42:22.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.387+0000 7f70b75cc740 20 client.26368 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:22.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.387+0000 7f70b75cc740 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55c686ea5010 0x55c686ea7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.387+0000 7f70b75cc740 4 client.26368 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:22.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.387+0000 7f70b75cc740 1 -- 172.21.15.133:0/4211482902 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c686dfd860 con 0x55c686ea5010 2024-09-11T23:42:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.388+0000 7f70aed9d640 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55c686ea5010 0x55c686ea7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.391+0000 7f70aed9d640 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55c686ea5010 0x55c686ea7400 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2079ab38a5ca0c71 server_cookie=fd080eb1c3dd2b23 in_seq=0 out_seq=0 2024-09-11T23:42:22.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.391+0000 7f7092ff5640 10 client.26368 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:42:22.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.393+0000 7f7092ff5640 1 -- 172.21.15.133:0/4211482902 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c686dfd860 con 0x55c686ea5010 2024-09-11T23:42:22.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.393+0000 7f7092ff5640 10 client.26368 handle_command_reply: tid=0 2024-09-11T23:42:22.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.393+0000 7f70b75cc740 1 client.26368 shutdown 2024-09-11T23:42:22.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.394+0000 7f70b75cc740 1 -- 172.21.15.133:0/4211482902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e8bcf0 msgr2=0x55c686e91fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:22.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.394+0000 7f70b75cc740 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e8bcf0 0x55c686e91fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f70a000ebc0 tx=0x7f70a0003ba0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.394+0000 7f70b75cc740 1 -- 172.21.15.133:0/4211482902 shutdown_connections 2024-09-11T23:42:22.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.394+0000 7f70b75cc740 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55c686ea5010 0x55c686ea7400 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.394+0000 7f70b75cc740 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e95520 0x55c686e92520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.394+0000 7f70b75cc740 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e8bcf0 0x55c686e91fe0 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f70a000ebc0 tx=0x7f70a0003ba0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.394+0000 7f70b75cc740 1 --2- 172.21.15.133:0/4211482902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e8a780 0x55c686e94ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.394+0000 7f70b75cc740 1 -- 172.21.15.133:0/4211482902 >> 172.21.15.133:0/4211482902 conn(0x55c686e53310 msgr2=0x55c686e8cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:22.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.394+0000 7f7092ff5640 0 client.26368 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:42:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.396+0000 7f70b75cc740 1 -- 172.21.15.133:0/4211482902 shutdown_connections 2024-09-11T23:42:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.396+0000 7f70b75cc740 1 -- 172.21.15.133:0/4211482902 wait complete. 2024-09-11T23:42:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.400+0000 7f70b75cc740 20 client.26368 trim_cache size 0 max 16384 2024-09-11T23:42:22.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.440+0000 7f70b75cc740 1 Processor -- start 2024-09-11T23:42:22.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.442+0000 7f70b75cc740 1 -- start start 2024-09-11T23:42:22.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.443+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e53310 0x55c686e96910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.443+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e8a780 0x55c686e8c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.443+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e95520 0x55c686e8c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.443+0000 7f70b75cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f709c00e5b0 con 0x55c686e53310 2024-09-11T23:42:22.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.443+0000 7f70b75cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f709c0100f0 con 0x55c686e8a780 2024-09-11T23:42:22.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.443+0000 7f70b75cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f709c00ec80 con 0x55c686e95520 2024-09-11T23:42:22.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.447+0000 7f70ae59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e8a780 0x55c686e8c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.447+0000 7f70ae59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e8a780 0x55c686e8c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:34616/0 (socket says 172.21.15.133:34616) 2024-09-11T23:42:22.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.447+0000 7f70ae59c640 1 -- 172.21.15.133:0/2136528027 learned_addr learned my addr 172.21.15.133:0/2136528027 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:22.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.447+0000 7f70af59e640 1 --2- 172.21.15.133:0/2136528027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e95520 0x55c686e8c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.450+0000 7f70ae59c640 1 -- 172.21.15.133:0/2136528027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e95520 msgr2=0x55c686e8c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.450+0000 7f70ae59c640 1 --2- 172.21.15.133:0/2136528027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e95520 0x55c686e8c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.450+0000 7f70ae59c640 1 -- 172.21.15.133:0/2136528027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e53310 msgr2=0x55c686e96910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.450+0000 7f70ae59c640 1 --2- 172.21.15.133:0/2136528027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e53310 0x55c686e96910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.450+0000 7f70ae59c640 1 -- 172.21.15.133:0/2136528027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c686e95050 con 0x55c686e8a780 2024-09-11T23:42:22.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.455+0000 7f70ae59c640 1 --2- 172.21.15.133:0/2136528027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e8a780 0x55c686e8c2c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f70a0004370 tx=0x7f70a000e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d5fbea7dc47644 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.464+0000 7f7094ff9640 1 -- 172.21.15.133:0/2136528027 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0016040 con 0x55c686e8a780 2024-09-11T23:42:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.464+0000 7f7094ff9640 1 -- 172.21.15.133:0/2136528027 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a0003f20 con 0x55c686e8a780 2024-09-11T23:42:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.466+0000 7f70b75cc740 1 -- 172.21.15.133:0/2136528027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e8a780 msgr2=0x55c686e8c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.466+0000 7f70b75cc740 1 --2- 172.21.15.133:0/2136528027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e8a780 0x55c686e8c2c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f70a0004370 tx=0x7f70a000e6c0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.466+0000 7f70b75cc740 1 -- 172.21.15.133:0/2136528027 shutdown_connections 2024-09-11T23:42:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.466+0000 7f70b75cc740 1 --2- 172.21.15.133:0/2136528027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e95520 0x55c686e8c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.467+0000 7f70b75cc740 1 --2- 172.21.15.133:0/2136528027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e8a780 0x55c686e8c2c0 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f70a0004370 tx=0x7f70a000e6c0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.467+0000 7f70b75cc740 1 --2- 172.21.15.133:0/2136528027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e53310 0x55c686e96910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.467+0000 7f70b75cc740 1 -- 172.21.15.133:0/2136528027 >> 172.21.15.133:0/2136528027 conn(0x55c686e81fa0 msgr2=0x55c686e910d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:22.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.472+0000 7f70b75cc740 1 -- 172.21.15.133:0/2136528027 shutdown_connections 2024-09-11T23:42:22.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.472+0000 7f70b75cc740 1 -- 172.21.15.133:0/2136528027 wait complete. 2024-09-11T23:42:22.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.473+0000 7f70b75cc740 1 -- start start 2024-09-11T23:42:22.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.473+0000 7f70b75cc740 1 Processor -- start 2024-09-11T23:42:22.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.474+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e53310 0x55c686e8fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.474+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e95520 0x55c686e9d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.474+0000 7f70b75cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e9dca0 0x55c686e8e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.474+0000 7f70b75cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f709c00fa20 con 0x55c686e9dca0 2024-09-11T23:42:22.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.474+0000 7f70b75cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f709c00f350 con 0x55c686e53310 2024-09-11T23:42:22.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.474+0000 7f70b75cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f709c00c640 con 0x55c686e95520 2024-09-11T23:42:22.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.475+0000 7f70b75cc740 10 client.0 _ll_register_callbacks cb 0x7f70a0009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:22.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.477+0000 7f70aed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e53310 0x55c686e8fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.477+0000 7f70aed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e53310 0x55c686e8fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:34620/0 (socket says 172.21.15.133:34620) 2024-09-11T23:42:22.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.477+0000 7f70aed9d640 1 -- 172.21.15.133:0/345282 learned_addr learned my addr 172.21.15.133:0/345282 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:22.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.477+0000 7f70ae59c640 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e95520 0x55c686e9d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.477+0000 7f70af59e640 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e9dca0 0x55c686e8e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.478+0000 7f70aed9d640 1 -- 172.21.15.133:0/345282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e95520 msgr2=0x55c686e9d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:22.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.478+0000 7f70aed9d640 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e95520 0x55c686e9d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.478+0000 7f70aed9d640 1 -- 172.21.15.133:0/345282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e9dca0 msgr2=0x55c686e8e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:22.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.478+0000 7f70aed9d640 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e9dca0 0x55c686e8e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.478+0000 7f70aed9d640 1 -- 172.21.15.133:0/345282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c686e9e1d0 con 0x55c686e53310 2024-09-11T23:42:22.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.478+0000 7f70aed9d640 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e53310 0x55c686e8fa90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f70a80be940 tx=0x7f70a80efe40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f521693ef499abe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:22.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.478+0000 7f7092ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.481+0000 7f7092ff5640 1 -- 172.21.15.133:0/345282 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80bf340 con 0x55c686e53310 2024-09-11T23:42:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.481+0000 7f70b75cc740 1 -- 172.21.15.133:0/345282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c686df7d70 con 0x55c686e53310 2024-09-11T23:42:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.482+0000 7f7092ff5640 1 -- 172.21.15.133:0/345282 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f70a80bb2b0 con 0x55c686e53310 2024-09-11T23:42:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.483+0000 7f7092ff5640 1 -- 172.21.15.133:0/345282 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70a80bec40 con 0x55c686e53310 2024-09-11T23:42:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.483+0000 7f70b75cc740 10 client.26411 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.483+0000 7f70b75cc740 1 -- 172.21.15.133:0/345282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c686f10050 con 0x55c686e53310 2024-09-11T23:42:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.484+0000 7f7092ff5640 1 -- 172.21.15.133:0/345282 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c686df7d70 con 0x55c686e53310 2024-09-11T23:42:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.484+0000 7f7092ff5640 1 -- 172.21.15.133:0/345282 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f70a803a070 con 0x55c686e53310 2024-09-11T23:42:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.484+0000 7f70b75cc740 10 client.26411 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.484+0000 7f70b75cc740 10 client.26411 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:42:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.484+0000 7f70b75cc740 20 client.26411 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.484+0000 7f70b75cc740 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55c686f10810 0x55c686f12c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.485+0000 7f70b75cc740 4 client.26411 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:42:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.485+0000 7f70b75cc740 1 -- 172.21.15.133:0/345282 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c686d74170 con 0x55c686f10810 2024-09-11T23:42:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.486+0000 7f70ae59c640 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55c686f10810 0x55c686f12c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.488+0000 7f70ae59c640 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55c686f10810 0x55c686f12c30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb5134e2338d04e6 server_cookie=fa1565c3bd468951 in_seq=0 out_seq=0 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.488+0000 7f7092ff5640 10 client.26411 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.488+0000 7f7092ff5640 1 -- 172.21.15.133:0/345282 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c686d74170 con 0x55c686f10810 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.488+0000 7f7092ff5640 10 client.26411 handle_command_reply: tid=0 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.488+0000 7f70b75cc740 1 client.26411 shutdown 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.490+0000 7f70b75cc740 1 -- 172.21.15.133:0/345282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e53310 msgr2=0x55c686e8fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.490+0000 7f70b75cc740 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e53310 0x55c686e8fa90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f70a80be940 tx=0x7f70a80efe40 comp rx=0 tx=0).stop 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.490+0000 7f70b75cc740 1 -- 172.21.15.133:0/345282 shutdown_connections 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.490+0000 7f70b75cc740 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55c686f10810 0x55c686f12c30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.490+0000 7f70b75cc740 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c686e9dca0 0x55c686e8e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.490+0000 7f70b75cc740 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c686e95520 0x55c686e9d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.490+0000 7f70b75cc740 1 --2- 172.21.15.133:0/345282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c686e53310 0x55c686e8fa90 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f70a80be940 tx=0x7f70a80efe40 comp rx=0 tx=0).stop 2024-09-11T23:42:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.490+0000 7f70b75cc740 1 -- 172.21.15.133:0/345282 >> 172.21.15.133:0/345282 conn(0x55c686e81fa0 msgr2=0x55c686e93840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:22.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.490+0000 7f7092ff5640 0 client.26411 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:42:22.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.492+0000 7f70b75cc740 1 -- 172.21.15.133:0/345282 shutdown_connections 2024-09-11T23:42:22.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.492+0000 7f70b75cc740 1 -- 172.21.15.133:0/345282 wait complete. 2024-09-11T23:42:22.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.492+0000 7f70b75cc740 20 client.26411 trim_cache size 0 max 16384 2024-09-11T23:42:22.501 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:42:22.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.505+0000 7f70b49f2640 1 -- 172.21.15.133:0/2816401639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70980a2f00 msgr2=0x7f70980a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:22.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.505+0000 7f70b49f2640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70980a2f00 0x7f70980a53c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f70a4005fd0 tx=0x7f70a401af40 comp rx=0 tx=0).stop 2024-09-11T23:42:22.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.505+0000 7f70b49f2640 1 -- 172.21.15.133:0/2816401639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b006ea30 msgr2=0x7f70b011a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:22.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.505+0000 7f70b49f2640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b006ea30 0x7f70b011a800 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f70a80092e0 tx=0x7f70a8030f10 comp rx=0 tx=0).stop 2024-09-11T23:42:22.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.508+0000 7f70b49f2640 1 -- 172.21.15.133:0/2816401639 shutdown_connections 2024-09-11T23:42:22.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.508+0000 7f70b49f2640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f70980a2f00 0x7f70980a53c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.508+0000 7f70b49f2640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b011b140 0x7f70b0116de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.508+0000 7f70b49f2640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0110190 0x7f70b01168a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.508+0000 7f70b49f2640 1 --2- 172.21.15.133:0/2816401639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b006ea30 0x7f70b011a800 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:22.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.508+0000 7f70b49f2640 1 -- 172.21.15.133:0/2816401639 >> 172.21.15.133:0/2816401639 conn(0x7f70b0073180 msgr2=0x7f70b011b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:22.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.509+0000 7f70b49f2640 1 -- 172.21.15.133:0/2816401639 shutdown_connections 2024-09-11T23:42:22.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:22.509+0000 7f70b49f2640 1 -- 172.21.15.133:0/2816401639 wait complete. 2024-09-11T23:42:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.675+0000 7f334bfd6640 1 -- 172.21.15.133:0/3980021748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f334410d610 msgr2=0x7f334410d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.675+0000 7f334bfd6640 1 --2- 172.21.15.133:0/3980021748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f334410d610 0x7f334410d9f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3340008570 tx=0x7f334000ac00 comp rx=0 tx=0).stop 2024-09-11T23:42:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.684+0000 7f334bfd6640 1 -- 172.21.15.133:0/3980021748 shutdown_connections 2024-09-11T23:42:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.684+0000 7f334bfd6640 1 --2- 172.21.15.133:0/3980021748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344109b70 0x7f334407b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.684+0000 7f334bfd6640 1 --2- 172.21.15.133:0/3980021748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33441091f0 0x7f3344109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.684+0000 7f334bfd6640 1 --2- 172.21.15.133:0/3980021748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f334410d610 0x7f334410d9f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.684+0000 7f334bfd6640 1 -- 172.21.15.133:0/3980021748 >> 172.21.15.133:0/3980021748 conn(0x7f334406ce00 msgr2=0x7f334406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.689+0000 7f334bfd6640 1 -- 172.21.15.133:0/3980021748 shutdown_connections 2024-09-11T23:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.689+0000 7f334bfd6640 1 -- 172.21.15.133:0/3980021748 wait complete. 2024-09-11T23:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.690+0000 7f334bfd6640 1 Processor -- start 2024-09-11T23:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.691+0000 7f334bfd6640 1 -- start start 2024-09-11T23:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.691+0000 7f334bfd6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33441091f0 0x7f334407ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.691+0000 7f334bfd6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344109b70 0x7f3344075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.691+0000 7f334bfd6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334410d610 0x7f33440762f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.691+0000 7f334bfd6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3344082af0 con 0x7f334410d610 2024-09-11T23:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.691+0000 7f334bfd6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3344082970 con 0x7f3344109b70 2024-09-11T23:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.691+0000 7f334bfd6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3344082c70 con 0x7f33441091f0 2024-09-11T23:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.699+0000 7f3349571640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344109b70 0x7f3344075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.699+0000 7f3349571640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344109b70 0x7f3344075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:34644/0 (socket says 172.21.15.133:34644) 2024-09-11T23:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.699+0000 7f3349571640 1 -- 172.21.15.133:0/2616404002 learned_addr learned my addr 172.21.15.133:0/2616404002 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.699+0000 7f3349d72640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33441091f0 0x7f334407ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.699+0000 7f334a573640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334410d610 0x7f33440762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.709+0000 7f3349571640 1 -- 172.21.15.133:0/2616404002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33441091f0 msgr2=0x7f334407ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.709+0000 7f3349571640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33441091f0 0x7f334407ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.709+0000 7f3349571640 1 -- 172.21.15.133:0/2616404002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334410d610 msgr2=0x7f33440762f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.709+0000 7f3349571640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334410d610 0x7f33440762f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.709+0000 7f3349571640 1 -- 172.21.15.133:0/2616404002 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3344076a20 con 0x7f3344109b70 2024-09-11T23:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.716+0000 7f3349571640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344109b70 0x7f3344075db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f333c00eab0 tx=0x7f333c00ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.718+0000 7f333affd640 1 -- 172.21.15.133:0/2616404002 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c00ce40 con 0x7f3344109b70 2024-09-11T23:42:23.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.718+0000 7f333affd640 1 -- 172.21.15.133:0/2616404002 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f333c011950 con 0x7f3344109b70 2024-09-11T23:42:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.718+0000 7f333affd640 1 -- 172.21.15.133:0/2616404002 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c010b40 con 0x7f3344109b70 2024-09-11T23:42:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.718+0000 7f334bfd6640 1 -- 172.21.15.133:0/2616404002 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3344137290 con 0x7f3344109b70 2024-09-11T23:42:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.718+0000 7f334bfd6640 1 -- 172.21.15.133:0/2616404002 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3344137790 con 0x7f3344109b70 2024-09-11T23:42:23.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.736+0000 7f333affd640 1 -- 172.21.15.133:0/2616404002 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f333c011b00 con 0x7f3344109b70 2024-09-11T23:42:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.739+0000 7f334ee51740 1 Processor -- start 2024-09-11T23:42:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.741+0000 7f333affd640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f33300a2ef0 0x7f33300a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.743+0000 7f333affd640 1 -- 172.21.15.133:0/2616404002 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f333c0cf2e0 con 0x7f3344109b70 2024-09-11T23:42:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.744+0000 7f334ee51740 1 -- start start 2024-09-11T23:42:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.745+0000 7f334ee51740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af2af0 0x55c076af2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.745+0000 7f334ee51740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af3410 0x55c076afd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.745+0000 7f334ee51740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076b000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.745+0000 7f334ee51740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c076a572f0 con 0x55c076af2af0 2024-09-11T23:42:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.745+0000 7f334ee51740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0769ff270 con 0x55c076afdcb0 2024-09-11T23:42:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.745+0000 7f334ee51740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c076a532b0 con 0x55c076af3410 2024-09-11T23:42:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.746+0000 7f334a573640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f33300a2ef0 0x7f33300a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.746+0000 7f3349d72640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af2af0 0x55c076af2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.746+0000 7f3349d72640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af2af0 0x55c076af2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41102/0 (socket says 172.21.15.133:41102) 2024-09-11T23:42:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.746+0000 7f3349d72640 1 -- 172.21.15.133:0/4259646552 learned_addr learned my addr 172.21.15.133:0/4259646552 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.748+0000 7f3349571640 1 --2- 172.21.15.133:0/4259646552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af3410 0x55c076afd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.749+0000 7f3349571640 1 -- 172.21.15.133:0/4259646552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 msgr2=0x55c076b000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.749+0000 7f3349571640 1 --2- 172.21.15.133:0/4259646552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076b000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.749+0000 7f3349571640 1 -- 172.21.15.133:0/4259646552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af2af0 msgr2=0x55c076af2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.749+0000 7f3349571640 1 --2- 172.21.15.133:0/4259646552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af2af0 0x55c076af2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.749+0000 7f3349571640 1 -- 172.21.15.133:0/4259646552 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0769fe340 con 0x55c076af3410 2024-09-11T23:42:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.750+0000 7f3349571640 1 --2- 172.21.15.133:0/4259646552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af3410 0x55c076afd770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f333c0cca40 tx=0x7f333c09b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb997443a0dd669 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.751+0000 7f332ffff640 1 -- 172.21.15.133:0/4259646552 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c0b4070 con 0x55c076af3410 2024-09-11T23:42:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.751+0000 7f332ffff640 1 -- 172.21.15.133:0/4259646552 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f333c09b810 con 0x55c076af3410 2024-09-11T23:42:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.751+0000 7f332ffff640 1 -- 172.21.15.133:0/4259646552 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c0cccc0 con 0x55c076af3410 2024-09-11T23:42:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.753+0000 7f334ee51740 1 -- 172.21.15.133:0/4259646552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af3410 msgr2=0x55c076afd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.753+0000 7f334ee51740 1 --2- 172.21.15.133:0/4259646552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af3410 0x55c076afd770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f333c0cca40 tx=0x7f333c09b6e0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.757+0000 7f334a573640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f33300a2ef0 0x7f33300a53b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55c076af03e0 tx=0x7f333400d040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.771+0000 7f334ee51740 1 -- 172.21.15.133:0/4259646552 shutdown_connections 2024-09-11T23:42:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.771+0000 7f334ee51740 1 --2- 172.21.15.133:0/4259646552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076b000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.771+0000 7f334ee51740 1 --2- 172.21.15.133:0/4259646552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af3410 0x55c076afd770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.771+0000 7f334ee51740 1 --2- 172.21.15.133:0/4259646552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af2af0 0x55c076af2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.771+0000 7f334ee51740 1 -- 172.21.15.133:0/4259646552 >> 172.21.15.133:0/4259646552 conn(0x55c076aef060 msgr2=0x55c076aef440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.777+0000 7f334ee51740 1 -- 172.21.15.133:0/4259646552 shutdown_connections 2024-09-11T23:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.778+0000 7f334ee51740 1 -- 172.21.15.133:0/4259646552 wait complete. 2024-09-11T23:42:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.778+0000 7f334ee51740 1 -- start start 2024-09-11T23:42:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.779+0000 7f334ee51740 1 Processor -- start 2024-09-11T23:42:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.780+0000 7f334ee51740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af2af0 0x55c076afac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.780+0000 7f334ee51740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af3410 0x55c076afb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.780+0000 7f334ee51740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076afb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.780+0000 7f334ee51740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c076b00880 con 0x55c076af2af0 2024-09-11T23:42:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.780+0000 7f334ee51740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0769e0290 con 0x55c076afdcb0 2024-09-11T23:42:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.780+0000 7f334ee51740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c076b00a00 con 0x55c076af3410 2024-09-11T23:42:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.781+0000 7f334ee51740 10 client.0 _ll_register_callbacks cb 0x55c076a11140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.785+0000 7f3349571640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af3410 0x55c076afb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.785+0000 7f334a573640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076afb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.785+0000 7f334a573640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076afb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:34668/0 (socket says 172.21.15.133:34668) 2024-09-11T23:42:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.785+0000 7f334a573640 1 -- 172.21.15.133:0/1286716640 learned_addr learned my addr 172.21.15.133:0/1286716640 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.789+0000 7f334a573640 1 -- 172.21.15.133:0/1286716640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af3410 msgr2=0x55c076afb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.789+0000 7f334a573640 1 --2- 172.21.15.133:0/1286716640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af3410 0x55c076afb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.789+0000 7f334a573640 1 -- 172.21.15.133:0/1286716640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af2af0 msgr2=0x55c076afac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.789+0000 7f334a573640 1 --2- 172.21.15.133:0/1286716640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af2af0 0x55c076afac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.789+0000 7f334a573640 1 -- 172.21.15.133:0/1286716640 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c07676a210 con 0x55c076afdcb0 2024-09-11T23:42:23.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:23 smithi178 ceph-mon[35882]: pgmap v172: 97 pgs: 97 active+clean; 46 GiB data, 169 GiB used, 904 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 668 MiB/s wr, 3.55k op/s 2024-09-11T23:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.792+0000 7f334a573640 1 --2- 172.21.15.133:0/1286716640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076afb6c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f333401c690 tx=0x7f3334004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe8bfce3371414c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.792+0000 7f332d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.793+0000 7f332d7fa640 1 -- 172.21.15.133:0/1286716640 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333402a060 con 0x55c076afdcb0 2024-09-11T23:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.794+0000 7f334ee51740 1 -- 172.21.15.133:0/1286716640 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f333c09b810 con 0x55c076afdcb0 2024-09-11T23:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.794+0000 7f332d7fa640 1 -- 172.21.15.133:0/1286716640 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f333401d7e0 con 0x55c076afdcb0 2024-09-11T23:42:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.795+0000 7f332d7fa640 1 -- 172.21.15.133:0/1286716640 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f333401d9e0 con 0x55c076afdcb0 2024-09-11T23:42:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.796+0000 7f334ee51740 10 client.26420 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.796+0000 7f334ee51740 1 -- 172.21.15.133:0/1286716640 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c076836620 con 0x55c076afdcb0 2024-09-11T23:42:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.798+0000 7f332d7fa640 1 -- 172.21.15.133:0/1286716640 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f333c09b810 con 0x55c076afdcb0 2024-09-11T23:42:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.798+0000 7f332d7fa640 1 -- 172.21.15.133:0/1286716640 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f333402d020 con 0x55c076afdcb0 2024-09-11T23:42:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.799+0000 7f334ee51740 10 client.26420 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.799+0000 7f334ee51740 10 client.26420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.799+0000 7f334ee51740 20 client.26420 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.799+0000 7f334ee51740 1 --2- 172.21.15.133:0/1286716640 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c076b0dd40 0x55c076b10170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.799+0000 7f334ee51740 4 client.26420 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.799+0000 7f334ee51740 1 -- 172.21.15.133:0/1286716640 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c076a64a20 con 0x55c076b0dd40 2024-09-11T23:42:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.805+0000 7f3349d72640 1 --2- 172.21.15.133:0/1286716640 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c076b0dd40 0x55c076b10170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.820+0000 7f3349d72640 1 --2- 172.21.15.133:0/1286716640 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c076b0dd40 0x55c076b10170 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab270cea74ffce1 server_cookie=9f135f85657d5bce in_seq=0 out_seq=0 2024-09-11T23:42:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.820+0000 7f332d7fa640 10 client.26420 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.824+0000 7f332d7fa640 1 -- 172.21.15.133:0/1286716640 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c076a64a20 con 0x55c076b0dd40 2024-09-11T23:42:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.824+0000 7f332d7fa640 10 client.26420 handle_command_reply: tid=0 2024-09-11T23:42:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.824+0000 7f334ee51740 1 client.26420 shutdown 2024-09-11T23:42:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.831+0000 7f334ee51740 1 -- 172.21.15.133:0/1286716640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 msgr2=0x55c076afb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.831+0000 7f334ee51740 1 --2- 172.21.15.133:0/1286716640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076afb6c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f333401c690 tx=0x7f3334004730 comp rx=0 tx=0).stop 2024-09-11T23:42:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.837+0000 7f334ee51740 1 -- 172.21.15.133:0/1286716640 shutdown_connections 2024-09-11T23:42:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.837+0000 7f334ee51740 1 --2- 172.21.15.133:0/1286716640 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c076b0dd40 0x55c076b10170 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.837+0000 7f334ee51740 1 --2- 172.21.15.133:0/1286716640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076afb6c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.837+0000 7f334ee51740 1 --2- 172.21.15.133:0/1286716640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af3410 0x55c076afb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.837+0000 7f334ee51740 1 --2- 172.21.15.133:0/1286716640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af2af0 0x55c076afac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.837+0000 7f334ee51740 1 -- 172.21.15.133:0/1286716640 >> 172.21.15.133:0/1286716640 conn(0x55c076aef060 msgr2=0x55c076af4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.837+0000 7f332d7fa640 0 client.26420 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.837+0000 7f334ee51740 1 -- 172.21.15.133:0/1286716640 shutdown_connections 2024-09-11T23:42:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.837+0000 7f334ee51740 1 -- 172.21.15.133:0/1286716640 wait complete. 2024-09-11T23:42:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.837+0000 7f334ee51740 20 client.26420 trim_cache size 0 max 16384 2024-09-11T23:42:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.894+0000 7f334ee51740 1 Processor -- start 2024-09-11T23:42:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.894+0000 7f334ee51740 1 -- start start 2024-09-11T23:42:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.896+0000 7f334ee51740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076af2af0 0x55c076afcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.897+0000 7f334ee51740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076afb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.897+0000 7f334ee51740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076afdcb0 0x55c076b15a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.898+0000 7f3349571640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076afb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.898+0000 7f3349571640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076afb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41120/0 (socket says 172.21.15.133:41120) 2024-09-11T23:42:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.898+0000 7f3349571640 1 -- 172.21.15.133:0/2258451278 learned_addr learned my addr 172.21.15.133:0/2258451278 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.898+0000 7f334ee51740 1 -- 172.21.15.133:0/2258451278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3344140020 con 0x55c076af3410 2024-09-11T23:42:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.898+0000 7f334ee51740 1 -- 172.21.15.133:0/2258451278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3344141b60 con 0x55c076af2af0 2024-09-11T23:42:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.898+0000 7f334ee51740 1 -- 172.21.15.133:0/2258451278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f33441406f0 con 0x55c076afdcb0 2024-09-11T23:42:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.898+0000 7f3349d72640 1 --2- 172.21.15.133:0/2258451278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076af2af0 0x55c076afcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.901+0000 7f3349571640 1 -- 172.21.15.133:0/2258451278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076afdcb0 msgr2=0x55c076b15a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.901+0000 7f3349571640 1 --2- 172.21.15.133:0/2258451278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076afdcb0 0x55c076b15a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.902+0000 7f3349571640 1 -- 172.21.15.133:0/2258451278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076af2af0 msgr2=0x55c076afcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.902+0000 7f3349571640 1 --2- 172.21.15.133:0/2258451278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076af2af0 0x55c076afcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.902+0000 7f3349571640 1 -- 172.21.15.133:0/2258451278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c076b00400 con 0x55c076af3410 2024-09-11T23:42:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.902+0000 7f3349571640 1 --2- 172.21.15.133:0/2258451278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076afb4f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f333c09b750 tx=0x7f333c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920b220369a200dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.904+0000 7f332f7fe640 1 -- 172.21.15.133:0/2258451278 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c0ba020 con 0x55c076af3410 2024-09-11T23:42:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.904+0000 7f332f7fe640 1 -- 172.21.15.133:0/2258451278 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f333c099b10 con 0x55c076af3410 2024-09-11T23:42:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.904+0000 7f332f7fe640 1 -- 172.21.15.133:0/2258451278 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c0b4040 con 0x55c076af3410 2024-09-11T23:42:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.905+0000 7f334ee51740 1 -- 172.21.15.133:0/2258451278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 msgr2=0x55c076afb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.905+0000 7f334ee51740 1 --2- 172.21.15.133:0/2258451278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076afb4f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f333c09b750 tx=0x7f333c000f40 comp rx=0 tx=0).stop 2024-09-11T23:42:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.907+0000 7f334ee51740 1 -- 172.21.15.133:0/2258451278 shutdown_connections 2024-09-11T23:42:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.907+0000 7f334ee51740 1 --2- 172.21.15.133:0/2258451278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076afdcb0 0x55c076b15a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.907+0000 7f334ee51740 1 --2- 172.21.15.133:0/2258451278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076afb4f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.907+0000 7f334ee51740 1 --2- 172.21.15.133:0/2258451278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076af2af0 0x55c076afcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.907+0000 7f334ee51740 1 -- 172.21.15.133:0/2258451278 >> 172.21.15.133:0/2258451278 conn(0x55c076aef060 msgr2=0x55c076b09330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.915+0000 7f334ee51740 1 -- 172.21.15.133:0/2258451278 shutdown_connections 2024-09-11T23:42:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.915+0000 7f334ee51740 1 -- 172.21.15.133:0/2258451278 wait complete. 2024-09-11T23:42:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.916+0000 7f334ee51740 1 -- start start 2024-09-11T23:42:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.916+0000 7f334ee51740 1 Processor -- start 2024-09-11T23:42:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.917+0000 7f334ee51740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af2af0 0x55c076b019c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.917+0000 7f334ee51740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076b01f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.917+0000 7f334ee51740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076b02440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.917+0000 7f334ee51740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3344141490 con 0x55c076af3410 2024-09-11T23:42:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.917+0000 7f334ee51740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3344140dc0 con 0x55c076afdcb0 2024-09-11T23:42:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.917+0000 7f334ee51740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3344109630 con 0x55c076af2af0 2024-09-11T23:42:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.917+0000 7f334ee51740 10 client.0 _ll_register_callbacks cb 0x7f333400bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.923+0000 7f3349571640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076b01f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.923+0000 7f3349571640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076b01f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41128/0 (socket says 172.21.15.133:41128) 2024-09-11T23:42:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.923+0000 7f3349571640 1 -- 172.21.15.133:0/2139967560 learned_addr learned my addr 172.21.15.133:0/2139967560 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.924+0000 7f3349571640 1 -- 172.21.15.133:0/2139967560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af2af0 msgr2=0x55c076b019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.924+0000 7f3349571640 1 --2- 172.21.15.133:0/2139967560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af2af0 0x55c076b019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.924+0000 7f3349571640 1 -- 172.21.15.133:0/2139967560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 msgr2=0x55c076b02440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.924+0000 7f3349571640 1 --2- 172.21.15.133:0/2139967560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076b02440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.924+0000 7f3349571640 1 -- 172.21.15.133:0/2139967560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c076b77a50 con 0x55c076af3410 2024-09-11T23:42:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.925+0000 7f3349571640 1 --2- 172.21.15.133:0/2139967560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076b01f00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f333c09fd80 tx=0x7f333c0103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a5d01f396f0503 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.925+0000 7f332d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:42:23.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.926+0000 7f332d7fa640 1 -- 172.21.15.133:0/2139967560 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c0b4070 con 0x55c076af3410 2024-09-11T23:42:23.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.926+0000 7f334ee51740 1 -- 172.21.15.133:0/2139967560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f333c099b10 con 0x55c076af3410 2024-09-11T23:42:23.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.926+0000 7f332d7fa640 1 -- 172.21.15.133:0/2139967560 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f333c0a3ff0 con 0x55c076af3410 2024-09-11T23:42:23.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.927+0000 7f332d7fa640 1 -- 172.21.15.133:0/2139967560 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f333c081050 con 0x55c076af3410 2024-09-11T23:42:23.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.927+0000 7f334ee51740 10 client.16965 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:23.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.927+0000 7f334ee51740 1 -- 172.21.15.133:0/2139967560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c076b78d20 con 0x55c076af3410 2024-09-11T23:42:23.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:23 smithi133 ceph-mon[31430]: pgmap v172: 97 pgs: 97 active+clean; 46 GiB data, 169 GiB used, 904 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 668 MiB/s wr, 3.55k op/s 2024-09-11T23:42:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.929+0000 7f332d7fa640 1 -- 172.21.15.133:0/2139967560 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f333c099b10 con 0x55c076af3410 2024-09-11T23:42:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.929+0000 7f332d7fa640 1 -- 172.21.15.133:0/2139967560 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f333c00f040 con 0x55c076af3410 2024-09-11T23:42:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.930+0000 7f334ee51740 10 client.16965 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.930+0000 7f334ee51740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.930+0000 7f334ee51740 20 client.16965 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.930+0000 7f334ee51740 1 --2- 172.21.15.133:0/2139967560 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c076b791f0 0x55c076b7b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.930+0000 7f334ee51740 4 client.16965 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:42:23.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.930+0000 7f334ee51740 1 -- 172.21.15.133:0/2139967560 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c076a5ed70 con 0x55c076b791f0 2024-09-11T23:42:23.931 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:42:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.935+0000 7f3349d72640 1 --2- 172.21.15.133:0/2139967560 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c076b791f0 0x55c076b7b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.939+0000 7f3349d72640 1 --2- 172.21.15.133:0/2139967560 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c076b791f0 0x55c076b7b610 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67bcdcaea186c898 server_cookie=bebdd34545fe736f in_seq=0 out_seq=0 2024-09-11T23:42:23.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:23 smithi138 ceph-mon[35142]: pgmap v172: 97 pgs: 97 active+clean; 46 GiB data, 169 GiB used, 904 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 668 MiB/s wr, 3.55k op/s 2024-09-11T23:42:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:23.954+0000 7f332d7fa640 10 client.16965 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:24.818 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.303+0000 7f332d7fa640 1 -- 172.21.15.133:0/2139967560 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c076a5ed70 con 0x55c076b791f0 2024-09-11T23:42:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.303+0000 7f332d7fa640 10 client.16965 handle_command_reply: tid=0 2024-09-11T23:42:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.304+0000 7f334ee51740 1 client.16965 shutdown 2024-09-11T23:42:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.304+0000 7f334ee51740 1 -- 172.21.15.133:0/2139967560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 msgr2=0x55c076b01f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.304+0000 7f334ee51740 1 --2- 172.21.15.133:0/2139967560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076b01f00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f333c09fd80 tx=0x7f333c0103d0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.305+0000 7f334ee51740 1 -- 172.21.15.133:0/2139967560 shutdown_connections 2024-09-11T23:42:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:42:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:42:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:42:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:42:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.305+0000 7f334ee51740 1 --2- 172.21.15.133:0/2139967560 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c076b791f0 0x55c076b7b610 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.305+0000 7f334ee51740 1 --2- 172.21.15.133:0/2139967560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c076afdcb0 0x55c076b02440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.305+0000 7f334ee51740 1 --2- 172.21.15.133:0/2139967560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c076af3410 0x55c076b01f00 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.305+0000 7f334ee51740 1 --2- 172.21.15.133:0/2139967560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c076af2af0 0x55c076b019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.305+0000 7f334ee51740 1 -- 172.21.15.133:0/2139967560 >> 172.21.15.133:0/2139967560 conn(0x55c076aef060 msgr2=0x55c076aed9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.305+0000 7f332d7fa640 0 client.16965 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.305+0000 7f334ee51740 1 -- 172.21.15.133:0/2139967560 shutdown_connections 2024-09-11T23:42:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.305+0000 7f334ee51740 1 -- 172.21.15.133:0/2139967560 wait complete. 2024-09-11T23:42:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.305+0000 7f334ee51740 20 client.16965 trim_cache size 0 max 16384 2024-09-11T23:42:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.309+0000 7f334bfd6640 1 -- 172.21.15.133:0/2616404002 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f33300a2ef0 msgr2=0x7f33300a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.310+0000 7f334bfd6640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f33300a2ef0 0x7f33300a53b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55c076af03e0 tx=0x7f333400d040 comp rx=0 tx=0).stop 2024-09-11T23:42:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.310+0000 7f334bfd6640 1 -- 172.21.15.133:0/2616404002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344109b70 msgr2=0x7f3344075db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.310+0000 7f334bfd6640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344109b70 0x7f3344075db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f333c00eab0 tx=0x7f333c00ef70 comp rx=0 tx=0).stop 2024-09-11T23:42:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.310+0000 7f334bfd6640 1 -- 172.21.15.133:0/2616404002 shutdown_connections 2024-09-11T23:42:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.310+0000 7f334bfd6640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f33300a2ef0 0x7f33300a53b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.310+0000 7f334bfd6640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334410d610 0x7f33440762f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.310+0000 7f334bfd6640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3344109b70 0x7f3344075db0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.310+0000 7f334bfd6640 1 --2- 172.21.15.133:0/2616404002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33441091f0 0x7f334407ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.310+0000 7f334bfd6640 1 -- 172.21.15.133:0/2616404002 >> 172.21.15.133:0/2616404002 conn(0x7f334406ce00 msgr2=0x7f3344070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.311+0000 7f334bfd6640 1 -- 172.21.15.133:0/2616404002 shutdown_connections 2024-09-11T23:42:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:25.311+0000 7f334bfd6640 1 -- 172.21.15.133:0/2616404002 wait complete. 2024-09-11T23:42:25.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.675+0000 7f4626297640 1 -- 172.21.15.133:0/3882237866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462011b140 msgr2=0x7f462011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:25.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.675+0000 7f4626297640 1 --2- 172.21.15.133:0/3882237866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462011b140 0x7f462011d530 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4618009ca0 tx=0x7f461802a8d0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.677+0000 7f4626297640 1 -- 172.21.15.133:0/3882237866 shutdown_connections 2024-09-11T23:42:25.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.677+0000 7f4626297640 1 --2- 172.21.15.133:0/3882237866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462011b140 0x7f462011d530 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.677+0000 7f4626297640 1 --2- 172.21.15.133:0/3882237866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4620109830 0x7f4620109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.677+0000 7f4626297640 1 --2- 172.21.15.133:0/3882237866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4620109450 0x7f4620110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.677+0000 7f4626297640 1 -- 172.21.15.133:0/3882237866 >> 172.21.15.133:0/3882237866 conn(0x7f462006e140 msgr2=0x7f462006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.682+0000 7f4626297640 1 -- 172.21.15.133:0/3882237866 shutdown_connections 2024-09-11T23:42:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.682+0000 7f4626297640 1 -- 172.21.15.133:0/3882237866 wait complete. 2024-09-11T23:42:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.683+0000 7f4626297640 1 Processor -- start 2024-09-11T23:42:25.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.684+0000 7f4626297640 1 -- start start 2024-09-11T23:42:25.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.684+0000 7f4626297640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4620109450 0x7f46201185d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.684+0000 7f4626297640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4620109830 0x7f4620118b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.684+0000 7f4626297640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462011b140 0x7f4620111760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.684+0000 7f4626297640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4620120e10 con 0x7f4620109450 2024-09-11T23:42:25.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.684+0000 7f4626297640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4620120c90 con 0x7f462011b140 2024-09-11T23:42:25.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.684+0000 7f4626297640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4620120f90 con 0x7f4620109830 2024-09-11T23:42:25.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.685+0000 7f461ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4620109450 0x7f46201185d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.685+0000 7f461ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4620109450 0x7f46201185d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41142/0 (socket says 172.21.15.133:41142) 2024-09-11T23:42:25.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.685+0000 7f461ffff640 1 -- 172.21.15.133:0/2191797004 learned_addr learned my addr 172.21.15.133:0/2191797004 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:25.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.685+0000 7f4624834640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462011b140 0x7f4620111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.685+0000 7f461ffff640 1 -- 172.21.15.133:0/2191797004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4620109830 msgr2=0x7f4620118b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:42:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.685+0000 7f461ffff640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4620109830 0x7f4620118b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.685+0000 7f461ffff640 1 -- 172.21.15.133:0/2191797004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462011b140 msgr2=0x7f4620111760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.685+0000 7f461ffff640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462011b140 0x7f4620111760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.685+0000 7f461ffff640 1 -- 172.21.15.133:0/2191797004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4620112020 con 0x7f4620109450 2024-09-11T23:42:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.686+0000 7f461ffff640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4620109450 0x7f46201185d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f461000d950 tx=0x7f461000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:25.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.688+0000 7f461d7fa640 1 -- 172.21.15.133:0/2191797004 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610017070 con 0x7f4620109450 2024-09-11T23:42:25.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.688+0000 7f461d7fa640 1 -- 172.21.15.133:0/2191797004 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4610009c30 con 0x7f4620109450 2024-09-11T23:42:25.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.688+0000 7f461d7fa640 1 -- 172.21.15.133:0/2191797004 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461000f6b0 con 0x7f4620109450 2024-09-11T23:42:25.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.688+0000 7f4626297640 1 -- 172.21.15.133:0/2191797004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4620112310 con 0x7f4620109450 2024-09-11T23:42:25.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.689+0000 7f4626297640 1 -- 172.21.15.133:0/2191797004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46201162d0 con 0x7f4620109450 2024-09-11T23:42:25.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.698+0000 7f4628670740 1 Processor -- start 2024-09-11T23:42:25.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.699+0000 7f4628670740 1 -- start start 2024-09-11T23:42:25.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.703+0000 7f4628670740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 0x56064c19b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.703+0000 7f4628670740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19b5d0 0x56064c19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.703+0000 7f4628670740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c1a6370 0x56064c1a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.703+0000 7f4628670740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56064c1012f0 con 0x56064c19cb40 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.703+0000 7f4628670740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56064c0a8cf0 con 0x56064c1a6370 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.703+0000 7f4628670740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56064c0fd2b0 con 0x56064c19b5d0 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.704+0000 7f4624834640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 0x56064c19b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.704+0000 7f4624834640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 0x56064c19b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41152/0 (socket says 172.21.15.133:41152) 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.704+0000 7f4624834640 1 -- 172.21.15.133:0/3962001212 learned_addr learned my addr 172.21.15.133:0/3962001212 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.704+0000 7f4624834640 1 -- 172.21.15.133:0/3962001212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19b5d0 msgr2=0x56064c19ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.704+0000 7f4624834640 1 --2- 172.21.15.133:0/3962001212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19b5d0 0x56064c19ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.704+0000 7f4624834640 1 -- 172.21.15.133:0/3962001212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c1a6370 msgr2=0x56064c1a8760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.704+0000 7f4624834640 1 --2- 172.21.15.133:0/3962001212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c1a6370 0x56064c1a8760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.704+0000 7f4624834640 1 -- 172.21.15.133:0/3962001212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56064c08d3a0 con 0x56064c19cb40 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.705+0000 7f461d7fa640 1 -- 172.21.15.133:0/2191797004 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f46100109d0 con 0x7f4620109450 2024-09-11T23:42:25.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.705+0000 7f4624834640 1 --2- 172.21.15.133:0/3962001212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 0x56064c19b090 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4618035000 tx=0x7f461802f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26aab184062e871e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:25.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.706+0000 7f46067fc640 1 -- 172.21.15.133:0/3962001212 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618031d10 con 0x56064c19cb40 2024-09-11T23:42:25.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.706+0000 7f46067fc640 1 -- 172.21.15.133:0/3962001212 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4618036040 con 0x56064c19cb40 2024-09-11T23:42:25.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.706+0000 7f46067fc640 1 -- 172.21.15.133:0/3962001212 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46180043b0 con 0x56064c19cb40 2024-09-11T23:42:25.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.706+0000 7f461d7fa640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46080a2f00 0x7f46080a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.707+0000 7f461d7fa640 1 -- 172.21.15.133:0/2191797004 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f46100cf0a0 con 0x7f4620109450 2024-09-11T23:42:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.708+0000 7f4628670740 1 -- 172.21.15.133:0/3962001212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 msgr2=0x56064c19b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.708+0000 7f4628670740 1 --2- 172.21.15.133:0/3962001212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 0x56064c19b090 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4618035000 tx=0x7f461802f450 comp rx=0 tx=0).stop 2024-09-11T23:42:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.708+0000 7f4628670740 1 -- 172.21.15.133:0/3962001212 shutdown_connections 2024-09-11T23:42:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.708+0000 7f4628670740 1 --2- 172.21.15.133:0/3962001212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c1a6370 0x56064c1a8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.709+0000 7f4628670740 1 --2- 172.21.15.133:0/3962001212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19b5d0 0x56064c19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.709+0000 7f4628670740 1 --2- 172.21.15.133:0/3962001212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 0x56064c19b090 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.709+0000 7f4628670740 1 -- 172.21.15.133:0/3962001212 >> 172.21.15.133:0/3962001212 conn(0x56064c164310 msgr2=0x56064c197c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:25.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.710+0000 7f461ffff640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46080a2f00 0x7f46080a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.711+0000 7f4628670740 1 -- 172.21.15.133:0/3962001212 shutdown_connections 2024-09-11T23:42:25.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.711+0000 7f461ffff640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46080a2f00 0x7f46080a53c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x56064c198ac0 tx=0x7f4610000bc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:25.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.712+0000 7f4628670740 1 -- 172.21.15.133:0/3962001212 wait complete. 2024-09-11T23:42:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.713+0000 7f4628670740 1 -- start start 2024-09-11T23:42:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.713+0000 7f4628670740 1 Processor -- start 2024-09-11T23:42:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.714+0000 7f4628670740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c19b5d0 0x56064c1a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.715+0000 7f4628670740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 0x56064c1a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.715+0000 7f4628670740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c1a6370 0x56064c1a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.715+0000 7f4628670740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56064c1a9010 con 0x56064c1a6370 2024-09-11T23:42:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.715+0000 7f4628670740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56064c0bc060 con 0x56064c19b5d0 2024-09-11T23:42:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.715+0000 7f4624834640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c1a6370 0x56064c1a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.715+0000 7f4624834640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c1a6370 0x56064c1a37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41160/0 (socket says 172.21.15.133:41160) 2024-09-11T23:42:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.715+0000 7f4624834640 1 -- 172.21.15.133:0/2283969518 learned_addr learned my addr 172.21.15.133:0/2283969518 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.722+0000 7f4628670740 1 -- 172.21.15.133:0/2283969518 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56064c1a9190 con 0x56064c19cb40 2024-09-11T23:42:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.722+0000 7f4628670740 10 client.0 _ll_register_callbacks cb 0x56064be0f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.723+0000 7f4624834640 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c19b5d0 0x56064c1a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.723+0000 7f461f7fe640 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 0x56064c1a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.724+0000 7f461f7fe640 1 -- 172.21.15.133:0/2283969518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c19b5d0 msgr2=0x56064c1a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.724+0000 7f461f7fe640 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c19b5d0 0x56064c1a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.724+0000 7f461f7fe640 1 -- 172.21.15.133:0/2283969518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c1a6370 msgr2=0x56064c1a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.724+0000 7f461f7fe640 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c1a6370 0x56064c1a37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.724+0000 7f461f7fe640 1 -- 172.21.15.133:0/2283969518 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56064be723c0 con 0x56064c19cb40 2024-09-11T23:42:25.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.725+0000 7f461f7fe640 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 0x56064c1a3050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4620112e20 tx=0x7f461400b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=450d3664b8a873fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:25.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.725+0000 7f45ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:42:25.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.727+0000 7f45ff7fe640 1 -- 172.21.15.133:0/2283969518 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614011420 con 0x56064c19cb40 2024-09-11T23:42:25.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.727+0000 7f4628670740 1 -- 172.21.15.133:0/2283969518 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4618036040 con 0x56064c19cb40 2024-09-11T23:42:25.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.727+0000 7f45ff7fe640 1 -- 172.21.15.133:0/2283969518 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f461400bda0 con 0x56064c19cb40 2024-09-11T23:42:25.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.727+0000 7f45ff7fe640 1 -- 172.21.15.133:0/2283969518 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f461401f050 con 0x56064c19cb40 2024-09-11T23:42:25.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.727+0000 7f4628670740 10 client.26386 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:25.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.727+0000 7f4628670740 1 -- 172.21.15.133:0/2283969518 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56064c1b4ba0 con 0x56064c19cb40 2024-09-11T23:42:25.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.728+0000 7f45ff7fe640 1 -- 172.21.15.133:0/2283969518 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4618036040 con 0x56064c19cb40 2024-09-11T23:42:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.728+0000 7f45ff7fe640 1 -- 172.21.15.133:0/2283969518 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f461401a790 con 0x56064c19cb40 2024-09-11T23:42:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.728+0000 7f4628670740 10 client.26386 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.728+0000 7f4628670740 10 client.26386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.728+0000 7f4628670740 20 client.26386 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.728+0000 7f4628670740 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56064c1b6470 0x56064c1b8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.728+0000 7f4628670740 4 client.26386 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.728+0000 7f4628670740 1 -- 172.21.15.133:0/2283969518 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56064c10e860 con 0x56064c1b6470 2024-09-11T23:42:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.728+0000 7f4624834640 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56064c1b6470 0x56064c1b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.729+0000 7f45ff7fe640 10 client.26386 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:25.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.729+0000 7f4624834640 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56064c1b6470 0x56064c1b8890 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec70660fdcf5283 server_cookie=a2880eb71ade99ce in_seq=0 out_seq=0 2024-09-11T23:42:25.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.730+0000 7f45ff7fe640 1 -- 172.21.15.133:0/2283969518 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56064c10e860 con 0x56064c1b6470 2024-09-11T23:42:25.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.730+0000 7f45ff7fe640 10 client.26386 handle_command_reply: tid=0 2024-09-11T23:42:25.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.730+0000 7f4628670740 1 client.26386 shutdown 2024-09-11T23:42:25.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f4628670740 1 -- 172.21.15.133:0/2283969518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 msgr2=0x56064c1a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:25.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f4628670740 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 0x56064c1a3050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4620112e20 tx=0x7f461400b4b0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f4628670740 1 -- 172.21.15.133:0/2283969518 shutdown_connections 2024-09-11T23:42:25.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f4628670740 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56064c1b6470 0x56064c1b8890 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f4628670740 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c1a6370 0x56064c1a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f4628670740 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 0x56064c1a3050 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f4628670740 1 --2- 172.21.15.133:0/2283969518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c19b5d0 0x56064c1a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f4628670740 1 -- 172.21.15.133:0/2283969518 >> 172.21.15.133:0/2283969518 conn(0x56064c164310 msgr2=0x56064c19d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:25.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f45ff7fe640 0 client.26386 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:25.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f4628670740 1 -- 172.21.15.133:0/2283969518 shutdown_connections 2024-09-11T23:42:25.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f4628670740 1 -- 172.21.15.133:0/2283969518 wait complete. 2024-09-11T23:42:25.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.731+0000 7f4628670740 20 client.26386 trim_cache size 0 max 16384 2024-09-11T23:42:25.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.748+0000 7f4628670740 1 Processor -- start 2024-09-11T23:42:25.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.749+0000 7f4628670740 1 -- start start 2024-09-11T23:42:25.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.750+0000 7f4628670740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c19b5d0 0x56064c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.750+0000 7f4628670740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 0x56064c1be160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.750+0000 7f4628670740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c1a6370 0x56064c1be6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.750+0000 7f4628670740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46201c4d90 con 0x56064c1a6370 2024-09-11T23:42:25.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.750+0000 7f4628670740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46201c68d0 con 0x56064c19b5d0 2024-09-11T23:42:25.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.750+0000 7f4624834640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c19b5d0 0x56064c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.750+0000 7f4628670740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46201c5460 con 0x56064c19cb40 2024-09-11T23:42:25.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.750+0000 7f4624834640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c19b5d0 0x56064c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:34726/0 (socket says 172.21.15.133:34726) 2024-09-11T23:42:25.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.750+0000 7f4624834640 1 -- 172.21.15.133:0/2103937801 learned_addr learned my addr 172.21.15.133:0/2103937801 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:25.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.750+0000 7f461f7fe640 1 --2- 172.21.15.133:0/2103937801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 0x56064c1be160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.750+0000 7f4624834640 1 --2- 172.21.15.133:0/2103937801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c1a6370 0x56064c1be6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.751+0000 7f461f7fe640 1 -- 172.21.15.133:0/2103937801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c19b5d0 msgr2=0x56064c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:25.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.751+0000 7f461f7fe640 1 --2- 172.21.15.133:0/2103937801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c19b5d0 0x56064c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.751+0000 7f461f7fe640 1 -- 172.21.15.133:0/2103937801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c1a6370 msgr2=0x56064c1be6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:25.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.751+0000 7f461f7fe640 1 --2- 172.21.15.133:0/2103937801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c1a6370 0x56064c1be6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.751+0000 7f461f7fe640 1 -- 172.21.15.133:0/2103937801 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56064c1a3ff0 con 0x56064c19cb40 2024-09-11T23:42:25.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.751+0000 7f461f7fe640 1 --2- 172.21.15.133:0/2103937801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 0x56064c1be160 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4614017df0 tx=0x7f4614009970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e2b5f9daa7840be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:25.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.751+0000 7f4605ffb640 1 -- 172.21.15.133:0/2103937801 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614027070 con 0x56064c19cb40 2024-09-11T23:42:25.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.752+0000 7f4605ffb640 1 -- 172.21.15.133:0/2103937801 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4614016070 con 0x56064c19cb40 2024-09-11T23:42:25.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.753+0000 7f4605ffb640 1 -- 172.21.15.133:0/2103937801 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614011420 con 0x56064c19cb40 2024-09-11T23:42:25.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.753+0000 7f4628670740 1 -- 172.21.15.133:0/2103937801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 msgr2=0x56064c1be160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:25.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.753+0000 7f4628670740 1 --2- 172.21.15.133:0/2103937801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 0x56064c1be160 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4614017df0 tx=0x7f4614009970 comp rx=0 tx=0).stop 2024-09-11T23:42:25.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.753+0000 7f4628670740 1 -- 172.21.15.133:0/2103937801 shutdown_connections 2024-09-11T23:42:25.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.753+0000 7f4628670740 1 --2- 172.21.15.133:0/2103937801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c1a6370 0x56064c1be6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.753+0000 7f4628670740 1 --2- 172.21.15.133:0/2103937801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19cb40 0x56064c1be160 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.753+0000 7f4628670740 1 --2- 172.21.15.133:0/2103937801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c19b5d0 0x56064c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.753+0000 7f4628670740 1 -- 172.21.15.133:0/2103937801 >> 172.21.15.133:0/2103937801 conn(0x56064c164310 msgr2=0x56064c197700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:25.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.754+0000 7f4628670740 1 -- 172.21.15.133:0/2103937801 shutdown_connections 2024-09-11T23:42:25.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.754+0000 7f4628670740 1 -- 172.21.15.133:0/2103937801 wait complete. 2024-09-11T23:42:25.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.754+0000 7f4628670740 1 -- start start 2024-09-11T23:42:25.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.755+0000 7f4628670740 1 Processor -- start 2024-09-11T23:42:25.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4628670740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19b5d0 0x56064c1aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4628670740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 0x56064c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4628670740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c1a6370 0x56064c1a0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4628670740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46201c6200 con 0x56064c19cb40 2024-09-11T23:42:25.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4628670740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46201c5b30 con 0x56064c1a6370 2024-09-11T23:42:25.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4624834640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19b5d0 0x56064c1aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f461f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 0x56064c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4624834640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19b5d0 0x56064c1aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:45370/0 (socket says 172.21.15.133:45370) 2024-09-11T23:42:25.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4624834640 1 -- 172.21.15.133:0/4153868356 learned_addr learned my addr 172.21.15.133:0/4153868356 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:25.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4628670740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46201c2d70 con 0x56064c19b5d0 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4624834640 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c1a6370 0x56064c1a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4628670740 10 client.0 _ll_register_callbacks cb 0x7f4614006e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4624834640 1 -- 172.21.15.133:0/4153868356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c1a6370 msgr2=0x56064c1a0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4624834640 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c1a6370 0x56064c1a0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4624834640 1 -- 172.21.15.133:0/4153868356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 msgr2=0x56064c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4624834640 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 0x56064c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.756+0000 7f4624834640 1 -- 172.21.15.133:0/4153868356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56064c19e120 con 0x56064c19b5d0 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.757+0000 7f4624834640 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19b5d0 0x56064c1aea30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4618003d80 tx=0x7f4618047040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43a3bc1856ca434 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.757+0000 7f45ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.757+0000 7f45ff7fe640 1 -- 172.21.15.133:0/4153868356 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461804a040 con 0x56064c19b5d0 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.757+0000 7f45ff7fe640 1 -- 172.21.15.133:0/4153868356 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f4618031460 con 0x56064c19b5d0 2024-09-11T23:42:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.757+0000 7f4628670740 1 -- 172.21.15.133:0/4153868356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4614016070 con 0x56064c19b5d0 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.757+0000 7f45ff7fe640 1 -- 172.21.15.133:0/4153868356 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4618031d60 con 0x56064c19b5d0 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.757+0000 7f4628670740 10 client.26392 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.757+0000 7f4628670740 1 -- 172.21.15.133:0/4153868356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56064c1a9180 con 0x56064c19b5d0 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.757+0000 7f45ff7fe640 1 -- 172.21.15.133:0/4153868356 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4618036070 con 0x56064c19b5d0 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.758+0000 7f45ff7fe640 1 -- 172.21.15.133:0/4153868356 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4618050040 con 0x56064c19b5d0 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.758+0000 7f4628670740 10 client.26392 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.758+0000 7f4628670740 10 client.26392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.758+0000 7f4628670740 20 client.26392 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.758+0000 7f4628670740 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56064c220f30 0x56064c223320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.758+0000 7f4628670740 4 client.26392 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.758+0000 7f4628670740 1 -- 172.21.15.133:0/4153868356 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4614016070 con 0x56064c220f30 2024-09-11T23:42:25.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.758+0000 7f461f7fe640 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56064c220f30 0x56064c223320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:25.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.758+0000 7f45ff7fe640 10 client.26392 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:25.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.758+0000 7f461f7fe640 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56064c220f30 0x56064c223320 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d4d96fdccd8c88 server_cookie=d1b66588f4b1dc34 in_seq=0 out_seq=0 2024-09-11T23:42:25.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.759+0000 7f45ff7fe640 1 -- 172.21.15.133:0/4153868356 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4614016070 con 0x56064c220f30 2024-09-11T23:42:25.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.759+0000 7f45ff7fe640 10 client.26392 handle_command_reply: tid=0 2024-09-11T23:42:25.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.759+0000 7f4628670740 1 client.26392 shutdown 2024-09-11T23:42:25.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.759+0000 7f4628670740 1 -- 172.21.15.133:0/4153868356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19b5d0 msgr2=0x56064c1aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:25.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.759+0000 7f4628670740 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19b5d0 0x56064c1aea30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4618003d80 tx=0x7f4618047040 comp rx=0 tx=0).stop 2024-09-11T23:42:25.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.759+0000 7f4628670740 1 -- 172.21.15.133:0/4153868356 shutdown_connections 2024-09-11T23:42:25.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.759+0000 7f4628670740 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56064c220f30 0x56064c223320 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.760+0000 7f4628670740 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064c1a6370 0x56064c1a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.760+0000 7f4628670740 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56064c19cb40 0x56064c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.760+0000 7f4628670740 1 --2- 172.21.15.133:0/4153868356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56064c19b5d0 0x56064c1aea30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.760+0000 7f4628670740 1 -- 172.21.15.133:0/4153868356 >> 172.21.15.133:0/4153868356 conn(0x56064c164310 msgr2=0x56064c1a2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:25.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.760+0000 7f45ff7fe640 0 client.26392 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:25.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.760+0000 7f4628670740 1 -- 172.21.15.133:0/4153868356 shutdown_connections 2024-09-11T23:42:25.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.761+0000 7f4628670740 1 -- 172.21.15.133:0/4153868356 wait complete. 2024-09-11T23:42:25.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.761+0000 7f4628670740 20 client.26392 trim_cache size 0 max 16384 2024-09-11T23:42:25.776 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:42:25.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.763+0000 7f4626297640 1 -- 172.21.15.133:0/2191797004 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46080a2f00 msgr2=0x7f46080a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:25.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.763+0000 7f4626297640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46080a2f00 0x7f46080a53c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x56064c198ac0 tx=0x7f4610000bc0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.763+0000 7f4626297640 1 -- 172.21.15.133:0/2191797004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4620109450 msgr2=0x7f46201185d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:25.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.763+0000 7f4626297640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4620109450 0x7f46201185d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f461000d950 tx=0x7f461000de10 comp rx=0 tx=0).stop 2024-09-11T23:42:25.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.763+0000 7f4626297640 1 -- 172.21.15.133:0/2191797004 shutdown_connections 2024-09-11T23:42:25.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.763+0000 7f4626297640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f46080a2f00 0x7f46080a53c0 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x56064c198ac0 tx=0x7f4610000bc0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.763+0000 7f4626297640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462011b140 0x7f4620111760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.763+0000 7f4626297640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4620109830 0x7f4620118b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:25.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.763+0000 7f4626297640 1 --2- 172.21.15.133:0/2191797004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4620109450 0x7f46201185d0 secure :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f461000d950 tx=0x7f461000de10 comp rx=0 tx=0).stop 2024-09-11T23:42:25.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.763+0000 7f4626297640 1 -- 172.21.15.133:0/2191797004 >> 172.21.15.133:0/2191797004 conn(0x7f462006e140 msgr2=0x7f462011bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:25.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.764+0000 7f4626297640 1 -- 172.21.15.133:0/2191797004 shutdown_connections 2024-09-11T23:42:25.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:25.764+0000 7f4626297640 1 -- 172.21.15.133:0/2191797004 wait complete. 2024-09-11T23:42:26.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:26 smithi178 ceph-mon[35882]: pgmap v173: 97 pgs: 97 active+clean; 47 GiB data, 173 GiB used, 900 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 680 MiB/s wr, 2.56k op/s 2024-09-11T23:42:26.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:26 smithi138 ceph-mon[35142]: pgmap v173: 97 pgs: 97 active+clean; 47 GiB data, 173 GiB used, 900 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 680 MiB/s wr, 2.56k op/s 2024-09-11T23:42:26.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:26 smithi133 ceph-mon[31430]: pgmap v173: 97 pgs: 97 active+clean; 47 GiB data, 173 GiB used, 900 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 680 MiB/s wr, 2.56k op/s 2024-09-11T23:42:27.720 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:42:27.720 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:42:27.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:27 smithi178 ceph-mon[35882]: pgmap v174: 97 pgs: 97 active+clean; 47 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 605 MiB/s wr, 1.77k op/s 2024-09-11T23:42:27.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:27 smithi138 ceph-mon[35142]: pgmap v174: 97 pgs: 97 active+clean; 47 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 605 MiB/s wr, 1.77k op/s 2024-09-11T23:42:27.948 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:27 smithi133 ceph-mon[31430]: pgmap v174: 97 pgs: 97 active+clean; 47 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 605 MiB/s wr, 1.77k op/s 2024-09-11T23:42:29.032 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:30.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.351+0000 7f35c1a0e640 1 -- 172.21.15.133:0/3084297875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc101270 msgr2=0x7f35bc10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:30.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.351+0000 7f35c1a0e640 1 --2- 172.21.15.133:0/3084297875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc101270 0x7f35bc10d7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f35b001a910 tx=0x7f35b003b470 comp rx=0 tx=0).stop 2024-09-11T23:42:30.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.355+0000 7f35c1a0e640 1 -- 172.21.15.133:0/3084297875 shutdown_connections 2024-09-11T23:42:30.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.355+0000 7f35c1a0e640 1 --2- 172.21.15.133:0/3084297875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc101270 0x7f35bc10d7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.355+0000 7f35c1a0e640 1 --2- 172.21.15.133:0/3084297875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35bc1008d0 0x7f35bc100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.355+0000 7f35c1a0e640 1 --2- 172.21.15.133:0/3084297875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35bc1068d0 0x7f35bc106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.355+0000 7f35c1a0e640 1 -- 172.21.15.133:0/3084297875 >> 172.21.15.133:0/3084297875 conn(0x7f35bc0fc610 msgr2=0x7f35bc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:30.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.357+0000 7f35c1a0e640 1 -- 172.21.15.133:0/3084297875 shutdown_connections 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.358+0000 7f35c1a0e640 1 -- 172.21.15.133:0/3084297875 wait complete. 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.359+0000 7f35c1a0e640 1 Processor -- start 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35c1a0e640 1 -- start start 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35c1a0e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35bc1008d0 0x7f35bc10d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35c1a0e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc101270 0x7f35bc10d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35c1a0e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35bc1068d0 0x7f35bc108480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35c1a0e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc06e570 con 0x7f35bc1008d0 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35c1a0e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc06e3f0 con 0x7f35bc101270 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35c1a0e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc06e6f0 con 0x7f35bc1068d0 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35bbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc101270 0x7f35bc10d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35bbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc101270 0x7f35bc10d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:34760/0 (socket says 172.21.15.133:34760) 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35bbfff640 1 -- 172.21.15.133:0/772094278 learned_addr learned my addr 172.21.15.133:0/772094278 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35c0a0c640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35bc1008d0 0x7f35bc10d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.360+0000 7f35c120d640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35bc1068d0 0x7f35bc108480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.361+0000 7f35bbfff640 1 -- 172.21.15.133:0/772094278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35bc1068d0 msgr2=0x7f35bc108480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:30.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.361+0000 7f35bbfff640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35bc1068d0 0x7f35bc108480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.361+0000 7f35bbfff640 1 -- 172.21.15.133:0/772094278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35bc1008d0 msgr2=0x7f35bc10d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:30.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.361+0000 7f35bbfff640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35bc1008d0 0x7f35bc10d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.361+0000 7f35bbfff640 1 -- 172.21.15.133:0/772094278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35bc108d40 con 0x7f35bc101270 2024-09-11T23:42:30.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.361+0000 7f35bbfff640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc101270 0x7f35bc10d750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00b4a0 tx=0x7f35ac00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:30.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.362+0000 7f35b9ffb640 1 -- 172.21.15.133:0/772094278 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac014070 con 0x7f35bc101270 2024-09-11T23:42:30.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.363+0000 7f35b9ffb640 1 -- 172.21.15.133:0/772094278 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35ac010af0 con 0x7f35bc101270 2024-09-11T23:42:30.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.364+0000 7f35b9ffb640 1 -- 172.21.15.133:0/772094278 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac017d20 con 0x7f35bc101270 2024-09-11T23:42:30.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.364+0000 7f35c1a0e640 1 -- 172.21.15.133:0/772094278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35bc109030 con 0x7f35bc101270 2024-09-11T23:42:30.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.365+0000 7f35c1a0e640 1 -- 172.21.15.133:0/772094278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35bc19a2b0 con 0x7f35bc101270 2024-09-11T23:42:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.369+0000 7f35c5049740 1 Processor -- start 2024-09-11T23:42:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.372+0000 7f35c5049740 1 -- start start 2024-09-11T23:42:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.372+0000 7f35c5049740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 0x562cbb3a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.372+0000 7f35c5049740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3a8970 0x562cbb3b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.372+0000 7f35c5049740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3b3230 0x562cbb3b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.372+0000 7f35c5049740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562cbb30d280 con 0x562cbb3b3230 2024-09-11T23:42:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.372+0000 7f35c5049740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562cbb2b4b10 con 0x562cbb3a8970 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.372+0000 7f35c5049740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562cbb309240 con 0x562cbb3a8050 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.373+0000 7f35b9ffb640 1 -- 172.21.15.133:0/772094278 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f35ac017070 con 0x7f35bc101270 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.373+0000 7f35c120d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 0x562cbb3a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.373+0000 7f35c120d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 0x562cbb3a8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:45394/0 (socket says 172.21.15.133:45394) 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.373+0000 7f35c120d640 1 -- 172.21.15.133:0/3119781243 learned_addr learned my addr 172.21.15.133:0/3119781243 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.374+0000 7f35c0a0c640 1 --2- 172.21.15.133:0/3119781243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3a8970 0x562cbb3b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.374+0000 7f35c120d640 1 -- 172.21.15.133:0/3119781243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3a8970 msgr2=0x562cbb3b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.374+0000 7f35c120d640 1 --2- 172.21.15.133:0/3119781243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3a8970 0x562cbb3b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.374+0000 7f35c120d640 1 -- 172.21.15.133:0/3119781243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3b3230 msgr2=0x562cbb3b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.374+0000 7f35c120d640 1 --2- 172.21.15.133:0/3119781243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3b3230 0x562cbb3b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.374+0000 7f35c120d640 1 -- 172.21.15.133:0/3119781243 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbb2c11f0 con 0x562cbb3a8050 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.374+0000 7f35c120d640 1 --2- 172.21.15.133:0/3119781243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 0x562cbb3a8430 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f35b0043000 tx=0x7f35b0004d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4edcf71cde549ce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.375+0000 7f35a27fc640 1 -- 172.21.15.133:0/3119781243 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0051030 con 0x562cbb3a8050 2024-09-11T23:42:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.376+0000 7f35a27fc640 1 -- 172.21.15.133:0/3119781243 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b00046c0 con 0x562cbb3a8050 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.376+0000 7f35b9ffb640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35a40a2d60 0x7f35a40a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.377+0000 7f35b9ffb640 1 -- 172.21.15.133:0/772094278 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f35ac0ce5c0 con 0x7f35bc101270 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.377+0000 7f35c0a0c640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35a40a2d60 0x7f35a40a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.377+0000 7f35a27fc640 1 -- 172.21.15.133:0/3119781243 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0005570 con 0x562cbb3a8050 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.377+0000 7f35c5049740 1 -- 172.21.15.133:0/3119781243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 msgr2=0x562cbb3a8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.377+0000 7f35c5049740 1 --2- 172.21.15.133:0/3119781243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 0x562cbb3a8430 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f35b0043000 tx=0x7f35b0004d40 comp rx=0 tx=0).stop 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.378+0000 7f35c5049740 1 -- 172.21.15.133:0/3119781243 shutdown_connections 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.378+0000 7f35c5049740 1 --2- 172.21.15.133:0/3119781243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3b3230 0x562cbb3b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.378+0000 7f35c5049740 1 --2- 172.21.15.133:0/3119781243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3a8970 0x562cbb3b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.378+0000 7f35c5049740 1 --2- 172.21.15.133:0/3119781243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 0x562cbb3a8430 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.378+0000 7f35c5049740 1 -- 172.21.15.133:0/3119781243 >> 172.21.15.133:0/3119781243 conn(0x562cbb3a45e0 msgr2=0x562cbb3a49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.378+0000 7f35c5049740 1 -- 172.21.15.133:0/3119781243 shutdown_connections 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.378+0000 7f35c5049740 1 -- 172.21.15.133:0/3119781243 wait complete. 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.378+0000 7f35c0a0c640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35a40a2d60 0x7f35a40a5220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x562cbb3a5440 tx=0x7f35b4009500 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.378+0000 7f35c5049740 1 -- start start 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.379+0000 7f35c5049740 1 Processor -- start 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.379+0000 7f35c5049740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3a8050 0x562cbb3b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.379+0000 7f35c5049740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8970 0x562cbb3aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.379+0000 7f35c5049740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3b3230 0x562cbb3b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.380+0000 7f35c5049740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562cbb3b5d70 con 0x562cbb3a8050 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.380+0000 7f35c5049740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562cbb2c7f10 con 0x562cbb3b3230 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.380+0000 7f35c5049740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562cbb3b5ef0 con 0x562cbb3a8970 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.380+0000 7f35c5049740 10 client.0 _ll_register_callbacks cb 0x562cbb290a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.383+0000 7f35bbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8970 0x562cbb3aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.383+0000 7f35c120d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3b3230 0x562cbb3b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.388+0000 7f35bbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8970 0x562cbb3aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:45408/0 (socket says 172.21.15.133:45408) 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.388+0000 7f35bbfff640 1 -- 172.21.15.133:0/4223692255 learned_addr learned my addr 172.21.15.133:0/4223692255 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.388+0000 7f35bbfff640 1 -- 172.21.15.133:0/4223692255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3b3230 msgr2=0x562cbb3b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.388+0000 7f35bbfff640 1 --2- 172.21.15.133:0/4223692255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3b3230 0x562cbb3b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.388+0000 7f35bbfff640 1 -- 172.21.15.133:0/4223692255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3a8050 msgr2=0x562cbb3b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.388+0000 7f35bbfff640 1 --2- 172.21.15.133:0/4223692255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3a8050 0x562cbb3b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.388+0000 7f35bbfff640 1 -- 172.21.15.133:0/4223692255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbb07e6a0 con 0x562cbb3a8970 2024-09-11T23:42:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.394+0000 7f35bbfff640 1 --2- 172.21.15.133:0/4223692255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8970 0x562cbb3aff20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f35ac097060 tx=0x7f35ac09b300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a3aab50faa251bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.395+0000 7f359bfff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:42:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.399+0000 7f359bfff640 1 -- 172.21.15.133:0/4223692255 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac0b1060 con 0x562cbb3a8970 2024-09-11T23:42:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.399+0000 7f35c5049740 1 -- 172.21.15.133:0/4223692255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b00046c0 con 0x562cbb3a8970 2024-09-11T23:42:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.399+0000 7f359bfff640 1 -- 172.21.15.133:0/4223692255 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f35ac017360 con 0x562cbb3a8970 2024-09-11T23:42:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.401+0000 7f359bfff640 1 -- 172.21.15.133:0/4223692255 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35ac0a3870 con 0x562cbb3a8970 2024-09-11T23:42:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.401+0000 7f35c5049740 10 client.26404 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.401+0000 7f35c5049740 1 -- 172.21.15.133:0/4223692255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562cbb3c18b0 con 0x562cbb3a8970 2024-09-11T23:42:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.402+0000 7f359bfff640 1 -- 172.21.15.133:0/4223692255 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b00046c0 con 0x562cbb3a8970 2024-09-11T23:42:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.403+0000 7f359bfff640 1 -- 172.21.15.133:0/4223692255 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f35ac09a1e0 con 0x562cbb3a8970 2024-09-11T23:42:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.404+0000 7f35c5049740 10 client.26404 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.404+0000 7f35c5049740 10 client.26404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.404+0000 7f35c5049740 20 client.26404 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.404+0000 7f35c5049740 1 --2- 172.21.15.133:0/4223692255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562cbb3c31f0 0x562cbb3c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.404+0000 7f35c5049740 4 client.26404 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.404+0000 7f35c5049740 1 -- 172.21.15.133:0/4223692255 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cbb31a7f0 con 0x562cbb3c31f0 2024-09-11T23:42:30.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.405+0000 7f35c0a0c640 1 --2- 172.21.15.133:0/4223692255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562cbb3c31f0 0x562cbb3c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.410+0000 7f35c0a0c640 1 --2- 172.21.15.133:0/4223692255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562cbb3c31f0 0x562cbb3c5610 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3eddcdd9ac9e5de server_cookie=86cf377b014c5061 in_seq=0 out_seq=0 2024-09-11T23:42:30.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.410+0000 7f359bfff640 10 client.26404 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:30.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.412+0000 7f359bfff640 1 -- 172.21.15.133:0/4223692255 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cbb31a7f0 con 0x562cbb3c31f0 2024-09-11T23:42:30.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.412+0000 7f359bfff640 10 client.26404 handle_command_reply: tid=0 2024-09-11T23:42:30.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.412+0000 7f35c5049740 1 client.26404 shutdown 2024-09-11T23:42:30.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.414+0000 7f35c5049740 1 -- 172.21.15.133:0/4223692255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8970 msgr2=0x562cbb3aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:30.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.414+0000 7f35c5049740 1 --2- 172.21.15.133:0/4223692255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8970 0x562cbb3aff20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f35ac097060 tx=0x7f35ac09b300 comp rx=0 tx=0).stop 2024-09-11T23:42:30.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.414+0000 7f35c5049740 1 -- 172.21.15.133:0/4223692255 shutdown_connections 2024-09-11T23:42:30.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.414+0000 7f35c5049740 1 --2- 172.21.15.133:0/4223692255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562cbb3c31f0 0x562cbb3c5610 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.414+0000 7f35c5049740 1 --2- 172.21.15.133:0/4223692255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3b3230 0x562cbb3b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.414+0000 7f35c5049740 1 --2- 172.21.15.133:0/4223692255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8970 0x562cbb3aff20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.414+0000 7f35c5049740 1 --2- 172.21.15.133:0/4223692255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3a8050 0x562cbb3b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.414+0000 7f35c5049740 1 -- 172.21.15.133:0/4223692255 >> 172.21.15.133:0/4223692255 conn(0x562cbb3a45e0 msgr2=0x562cbb3aa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:30.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.415+0000 7f359bfff640 0 client.26404 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:30.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.417+0000 7f35c5049740 1 -- 172.21.15.133:0/4223692255 shutdown_connections 2024-09-11T23:42:30.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.418+0000 7f35c5049740 1 -- 172.21.15.133:0/4223692255 wait complete. 2024-09-11T23:42:30.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.421+0000 7f35c5049740 20 client.26404 trim_cache size 0 max 16384 2024-09-11T23:42:30.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.463+0000 7f35c5049740 1 Processor -- start 2024-09-11T23:42:30.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.464+0000 7f35c5049740 1 -- start start 2024-09-11T23:42:30.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.464+0000 7f35c5049740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 0x562cbb3bf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.464+0000 7f35c5049740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3a8970 0x562cbb3b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.464+0000 7f35c5049740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3b3230 0x562cbb3caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.464+0000 7f35c5049740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1b3830 con 0x562cbb3a8970 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.464+0000 7f35c5049740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1b5370 con 0x562cbb3b3230 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.464+0000 7f35c5049740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1b3f00 con 0x562cbb3a8050 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.464+0000 7f35bbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3a8970 0x562cbb3b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.464+0000 7f35bbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3a8970 0x562cbb3b1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41222/0 (socket says 172.21.15.133:41222) 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.464+0000 7f35bbfff640 1 -- 172.21.15.133:0/2463376061 learned_addr learned my addr 172.21.15.133:0/2463376061 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.465+0000 7f35c120d640 1 --2- 172.21.15.133:0/2463376061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3b3230 0x562cbb3caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.465+0000 7f35c0a0c640 1 --2- 172.21.15.133:0/2463376061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 0x562cbb3bf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.465+0000 7f35bbfff640 1 -- 172.21.15.133:0/2463376061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 msgr2=0x562cbb3bf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.465+0000 7f35bbfff640 1 --2- 172.21.15.133:0/2463376061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 0x562cbb3bf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.465+0000 7f35bbfff640 1 -- 172.21.15.133:0/2463376061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3b3230 msgr2=0x562cbb3caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:30.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.465+0000 7f35bbfff640 1 --2- 172.21.15.133:0/2463376061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3b3230 0x562cbb3caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.465+0000 7f35bbfff640 1 -- 172.21.15.133:0/2463376061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbb37e4e0 con 0x562cbb3a8970 2024-09-11T23:42:30.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.465+0000 7f35c120d640 1 --2- 172.21.15.133:0/2463376061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3b3230 0x562cbb3caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:42:30.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.465+0000 7f35bbfff640 1 --2- 172.21.15.133:0/2463376061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3a8970 0x562cbb3b1d80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0a3340 tx=0x7f35ac0cbe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49288581e2096f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.486+0000 7f35a1ffb640 1 -- 172.21.15.133:0/2463376061 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac0a8070 con 0x562cbb3a8970 2024-09-11T23:42:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.486+0000 7f35a1ffb640 1 -- 172.21.15.133:0/2463376061 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35ac0cc160 con 0x562cbb3a8970 2024-09-11T23:42:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.488+0000 7f35c5049740 1 -- 172.21.15.133:0/2463376061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3a8970 msgr2=0x562cbb3b1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.488+0000 7f35c5049740 1 --2- 172.21.15.133:0/2463376061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3a8970 0x562cbb3b1d80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0a3340 tx=0x7f35ac0cbe00 comp rx=0 tx=0).stop 2024-09-11T23:42:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.489+0000 7f35c5049740 1 -- 172.21.15.133:0/2463376061 shutdown_connections 2024-09-11T23:42:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.489+0000 7f35c5049740 1 --2- 172.21.15.133:0/2463376061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3b3230 0x562cbb3caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.489+0000 7f35c5049740 1 --2- 172.21.15.133:0/2463376061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3a8970 0x562cbb3b1d80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.489+0000 7f35c5049740 1 --2- 172.21.15.133:0/2463376061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8050 0x562cbb3bf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.489+0000 7f35c5049740 1 -- 172.21.15.133:0/2463376061 >> 172.21.15.133:0/2463376061 conn(0x562cbb3a45e0 msgr2=0x562cbb3b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:30.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.489+0000 7f35c5049740 1 -- 172.21.15.133:0/2463376061 shutdown_connections 2024-09-11T23:42:30.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.489+0000 7f35c5049740 1 -- 172.21.15.133:0/2463376061 wait complete. 2024-09-11T23:42:30.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.490+0000 7f35c5049740 1 -- start start 2024-09-11T23:42:30.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.490+0000 7f35c5049740 1 Processor -- start 2024-09-11T23:42:30.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.491+0000 7f35c5049740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3a8050 0x562cbb3b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.491+0000 7f35c5049740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8970 0x562cbb3b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.491+0000 7f35c5049740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3b3230 0x562cbb3b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.491+0000 7f35c5049740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1b4ca0 con 0x562cbb3b3230 2024-09-11T23:42:30.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.491+0000 7f35c5049740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1b45d0 con 0x562cbb3a8050 2024-09-11T23:42:30.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.491+0000 7f35c5049740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1b1870 con 0x562cbb3a8970 2024-09-11T23:42:30.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.491+0000 7f35c5049740 10 client.0 _ll_register_callbacks cb 0x7f35ac09a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.492+0000 7f35c120d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3b3230 0x562cbb3b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.492+0000 7f35c0a0c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3a8050 0x562cbb3b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.492+0000 7f35c120d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3b3230 0x562cbb3b7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41228/0 (socket says 172.21.15.133:41228) 2024-09-11T23:42:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.492+0000 7f35c0a0c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3a8050 0x562cbb3b7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:34792/0 (socket says 172.21.15.133:34792) 2024-09-11T23:42:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.492+0000 7f35c0a0c640 1 -- 172.21.15.133:0/1510504787 learned_addr learned my addr 172.21.15.133:0/1510504787 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.493+0000 7f35c120d640 1 -- 172.21.15.133:0/1510504787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8970 msgr2=0x562cbb3b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.493+0000 7f35c120d640 1 --2- 172.21.15.133:0/1510504787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8970 0x562cbb3b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.493+0000 7f35c120d640 1 -- 172.21.15.133:0/1510504787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3a8050 msgr2=0x562cbb3b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.493+0000 7f35c120d640 1 --2- 172.21.15.133:0/1510504787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3a8050 0x562cbb3b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.493+0000 7f35c120d640 1 -- 172.21.15.133:0/1510504787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbb42d050 con 0x562cbb3b3230 2024-09-11T23:42:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.493+0000 7f35c120d640 1 --2- 172.21.15.133:0/1510504787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3b3230 0x562cbb3b7ae0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f35b001a860 tx=0x7f35b0004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd5a4f9e73103f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.493+0000 7f359bfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:42:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.494+0000 7f359bfff640 1 -- 172.21.15.133:0/1510504787 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0012020 con 0x562cbb3b3230 2024-09-11T23:42:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.494+0000 7f359bfff640 1 -- 172.21.15.133:0/1510504787 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f35b0019040 con 0x562cbb3b3230 2024-09-11T23:42:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.494+0000 7f35c5049740 1 -- 172.21.15.133:0/1510504787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35ac0cc160 con 0x562cbb3b3230 2024-09-11T23:42:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.494+0000 7f359bfff640 1 -- 172.21.15.133:0/1510504787 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35b005b050 con 0x562cbb3b3230 2024-09-11T23:42:30.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.495+0000 7f35c5049740 10 client.16989 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:30.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.495+0000 7f35c5049740 1 -- 172.21.15.133:0/1510504787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562cbb42e350 con 0x562cbb3b3230 2024-09-11T23:42:30.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.495+0000 7f359bfff640 1 -- 172.21.15.133:0/1510504787 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35ac0cc160 con 0x562cbb3b3230 2024-09-11T23:42:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.497+0000 7f359bfff640 1 -- 172.21.15.133:0/1510504787 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f35b0040d20 con 0x562cbb3b3230 2024-09-11T23:42:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.497+0000 7f35c5049740 10 client.16989 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.497+0000 7f35c5049740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.497+0000 7f35c5049740 20 client.16989 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.497+0000 7f35c5049740 1 --2- 172.21.15.133:0/1510504787 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562cbb42ea60 0x562cbb430e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:30.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.497+0000 7f35c5049740 4 client.16989 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:42:30.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.497+0000 7f35c5049740 1 -- 172.21.15.133:0/1510504787 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562cbb39fd30 con 0x562cbb42ea60 2024-09-11T23:42:30.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.498+0000 7f35c0a0c640 1 --2- 172.21.15.133:0/1510504787 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562cbb42ea60 0x562cbb430e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:30.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.499+0000 7f35c0a0c640 1 --2- 172.21.15.133:0/1510504787 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562cbb42ea60 0x562cbb430e90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1a66d45840d461 server_cookie=26f0edf08b6ad9c in_seq=0 out_seq=0 2024-09-11T23:42:30.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.500+0000 7f359bfff640 10 client.16989 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:30.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.504+0000 7f359bfff640 1 -- 172.21.15.133:0/1510504787 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562cbb39fd30 con 0x562cbb42ea60 2024-09-11T23:42:30.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.504+0000 7f359bfff640 10 client.16989 handle_command_reply: tid=0 2024-09-11T23:42:30.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.504+0000 7f35c5049740 1 client.16989 shutdown 2024-09-11T23:42:30.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.505+0000 7f35c5049740 1 -- 172.21.15.133:0/1510504787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3b3230 msgr2=0x562cbb3b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:30.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.505+0000 7f35c5049740 1 --2- 172.21.15.133:0/1510504787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3b3230 0x562cbb3b7ae0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f35b001a860 tx=0x7f35b0004bb0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.506+0000 7f35c5049740 1 -- 172.21.15.133:0/1510504787 shutdown_connections 2024-09-11T23:42:30.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.506+0000 7f35c5049740 1 --2- 172.21.15.133:0/1510504787 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562cbb42ea60 0x562cbb430e90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.506+0000 7f35c5049740 1 --2- 172.21.15.133:0/1510504787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cbb3b3230 0x562cbb3b7ae0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.506+0000 7f35c5049740 1 --2- 172.21.15.133:0/1510504787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562cbb3a8970 0x562cbb3b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.506+0000 7f35c5049740 1 --2- 172.21.15.133:0/1510504787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562cbb3a8050 0x562cbb3b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.506+0000 7f35c5049740 1 -- 172.21.15.133:0/1510504787 >> 172.21.15.133:0/1510504787 conn(0x562cbb3a45e0 msgr2=0x562cbb3aa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:30.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.508+0000 7f359bfff640 0 client.16989 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:30.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.508+0000 7f35c5049740 1 -- 172.21.15.133:0/1510504787 shutdown_connections 2024-09-11T23:42:30.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.510+0000 7f35c5049740 1 -- 172.21.15.133:0/1510504787 wait complete. 2024-09-11T23:42:30.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.511+0000 7f35c5049740 20 client.16989 trim_cache size 0 max 16384 2024-09-11T23:42:30.511 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:42:30.511 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:42:30.511 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "11e70c4b-b2c2-49ad-8876-d3cf45d75e0e", 2024-09-11T23:42:30.512 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:42:30.512 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:42:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 -- 172.21.15.133:0/772094278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35a40a2d60 msgr2=0x7f35a40a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35a40a2d60 0x7f35a40a5220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x562cbb3a5440 tx=0x7f35b4009500 comp rx=0 tx=0).stop 2024-09-11T23:42:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 -- 172.21.15.133:0/772094278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc101270 msgr2=0x7f35bc10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc101270 0x7f35bc10d750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00b4a0 tx=0x7f35ac00b960 comp rx=0 tx=0).stop 2024-09-11T23:42:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 -- 172.21.15.133:0/772094278 shutdown_connections 2024-09-11T23:42:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35a40a2d60 0x7f35a40a5220 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35bc1068d0 0x7f35bc108480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc101270 0x7f35bc10d750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 --2- 172.21.15.133:0/772094278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35bc1008d0 0x7f35bc10d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:30.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 -- 172.21.15.133:0/772094278 >> 172.21.15.133:0/772094278 conn(0x7f35bc0fc610 msgr2=0x7f35bc0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:30.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 -- 172.21.15.133:0/772094278 shutdown_connections 2024-09-11T23:42:30.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:30.513+0000 7f35a37fe640 1 -- 172.21.15.133:0/772094278 wait complete. 2024-09-11T23:42:30.711 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:30 smithi133 ceph-mon[31430]: pgmap v175: 97 pgs: 97 active+clean; 48 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 813 MiB/s wr, 2.30k op/s 2024-09-11T23:42:30.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:30 smithi178 ceph-mon[35882]: pgmap v175: 97 pgs: 97 active+clean; 48 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 813 MiB/s wr, 2.30k op/s 2024-09-11T23:42:30.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:30 smithi138 ceph-mon[35142]: pgmap v175: 97 pgs: 97 active+clean; 48 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 813 MiB/s wr, 2.30k op/s 2024-09-11T23:42:31.863 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:31 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:42:31.863 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:31 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:31.863 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:31 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:42:31.863 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:31 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:31.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:31 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:42:31.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:31 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:31.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:31 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:42:31.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:31 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:31.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:42:32.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:31 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:42:32.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:31 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:32.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:31 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:42:32.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:31 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:42:32.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:32 smithi138 ceph-mon[35142]: pgmap v176: 97 pgs: 97 active+clean; 48 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 644 MiB/s wr, 1.53k op/s 2024-09-11T23:42:32.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:32 smithi133 ceph-mon[31430]: pgmap v176: 97 pgs: 97 active+clean; 48 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 644 MiB/s wr, 1.53k op/s 2024-09-11T23:42:33.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:32 smithi178 ceph-mon[35882]: pgmap v176: 97 pgs: 97 active+clean; 48 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 644 MiB/s wr, 1.53k op/s 2024-09-11T23:42:33.213 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:33.936 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:33 smithi133 ceph-mon[31430]: pgmap v177: 97 pgs: 97 active+clean; 49 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 838 MiB/s wr, 1.65k op/s 2024-09-11T23:42:33.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:33 smithi138 ceph-mon[35142]: pgmap v177: 97 pgs: 97 active+clean; 49 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 838 MiB/s wr, 1.65k op/s 2024-09-11T23:42:34.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:33 smithi178 ceph-mon[35882]: pgmap v177: 97 pgs: 97 active+clean; 49 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 838 MiB/s wr, 1.65k op/s 2024-09-11T23:42:34.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.466+0000 7f306be11640 1 -- 172.21.15.133:0/2403602168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064100e20 msgr2=0x7f3064101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:34.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.466+0000 7f306be11640 1 --2- 172.21.15.133:0/2403602168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064100e20 0x7f3064101200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3060009890 tx=0x7f306002a410 comp rx=0 tx=0).stop 2024-09-11T23:42:34.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.467+0000 7f306be11640 1 -- 172.21.15.133:0/2403602168 shutdown_connections 2024-09-11T23:42:34.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.467+0000 7f306be11640 1 --2- 172.21.15.133:0/2403602168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3064102ef0 0x7f306410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.467+0000 7f306be11640 1 --2- 172.21.15.133:0/2403602168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3064102570 0x7f30641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.467+0000 7f306be11640 1 --2- 172.21.15.133:0/2403602168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064100e20 0x7f3064101200 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.467+0000 7f306be11640 1 -- 172.21.15.133:0/2403602168 >> 172.21.15.133:0/2403602168 conn(0x7f30640fc7d0 msgr2=0x7f30640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:34.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.467+0000 7f306be11640 1 -- 172.21.15.133:0/2403602168 shutdown_connections 2024-09-11T23:42:34.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.467+0000 7f306be11640 1 -- 172.21.15.133:0/2403602168 wait complete. 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.467+0000 7f306be11640 1 Processor -- start 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.467+0000 7f306be11640 1 -- start start 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306be11640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3064100e20 0x7f3064197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306be11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3064102570 0x7f30641979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306be11640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064102ef0 0x7f306419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306be11640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3064110f80 con 0x7f3064102570 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306be11640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3064110e00 con 0x7f3064102ef0 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306be11640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3064111100 con 0x7f3064100e20 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f30693ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3064102570 0x7f30641979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f3069bad640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3064100e20 0x7f3064197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f30693ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3064102570 0x7f30641979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42434/0 (socket says 172.21.15.133:42434) 2024-09-11T23:42:34.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f30693ac640 1 -- 172.21.15.133:0/49007808 learned_addr learned my addr 172.21.15.133:0/49007808 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306a3ae640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064102ef0 0x7f306419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306a3ae640 1 -- 172.21.15.133:0/49007808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3064100e20 msgr2=0x7f3064197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306a3ae640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3064100e20 0x7f3064197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306a3ae640 1 -- 172.21.15.133:0/49007808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3064102570 msgr2=0x7f30641979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306a3ae640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3064102570 0x7f30641979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306a3ae640 1 -- 172.21.15.133:0/49007808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306419c410 con 0x7f3064102ef0 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.468+0000 7f306a3ae640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064102ef0 0x7f306419bd30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f305c009fd0 tx=0x7f305c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.469+0000 7f305affd640 1 -- 172.21.15.133:0/49007808 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c00ce20 con 0x7f3064102ef0 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.469+0000 7f305affd640 1 -- 172.21.15.133:0/49007808 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c005020 con 0x7f3064102ef0 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.469+0000 7f306be11640 1 -- 172.21.15.133:0/49007808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306419c700 con 0x7f3064102ef0 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.469+0000 7f305affd640 1 -- 172.21.15.133:0/49007808 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c010720 con 0x7f3064102ef0 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.469+0000 7f306be11640 1 -- 172.21.15.133:0/49007808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3064069f70 con 0x7f3064102ef0 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.472+0000 7f305affd640 1 -- 172.21.15.133:0/49007808 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f305c004b20 con 0x7f3064102ef0 2024-09-11T23:42:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.472+0000 7f306e1ea740 1 Processor -- start 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.473+0000 7f306e1ea740 1 -- start start 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.473+0000 7f305affd640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30500a2d70 0x7f30500a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.473+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 0x55d22402d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.473+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d22402ddc0 0x55d224038140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.473+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 0x55d22403aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.473+0000 7f306e1ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d223f932e0 con 0x55d22402ddc0 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.473+0000 7f306e1ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d223f3ace0 con 0x55d224038680 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.473+0000 7f306e1ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d223f8f2a0 con 0x55d22402d4a0 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.473+0000 7f305affd640 1 -- 172.21.15.133:0/49007808 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f305c0cf9d0 con 0x7f3064102ef0 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.473+0000 7f30693ac640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30500a2d70 0x7f30500a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f306a3ae640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 0x55d22403aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f306a3ae640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 0x55d22403aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:56964/0 (socket says 172.21.15.133:56964) 2024-09-11T23:42:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f306a3ae640 1 -- 172.21.15.133:0/3917381305 learned_addr learned my addr 172.21.15.133:0/3917381305 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f3069bad640 1 --2- 172.21.15.133:0/3917381305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 0x55d22402d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f306a3ae640 1 -- 172.21.15.133:0/3917381305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 msgr2=0x55d22402d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f306a3ae640 1 --2- 172.21.15.133:0/3917381305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 0x55d22402d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f306a3ae640 1 -- 172.21.15.133:0/3917381305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d22402ddc0 msgr2=0x55d224038140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f306a3ae640 1 --2- 172.21.15.133:0/3917381305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d22402ddc0 0x55d224038140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f306a3ae640 1 -- 172.21.15.133:0/3917381305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d223f1f440 con 0x55d224038680 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f30693ac640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30500a2d70 0x7f30500a5230 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f30540098b0 tx=0x7f3054009840 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f306a3ae640 1 --2- 172.21.15.133:0/3917381305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 0x55d22403aa70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f305c0983c0 tx=0x7f305c09bbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96110635171b4fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f304ffff640 1 -- 172.21.15.133:0/3917381305 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c0a0310 con 0x55d224038680 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f304ffff640 1 -- 172.21.15.133:0/3917381305 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c0a4bd0 con 0x55d224038680 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.474+0000 7f304ffff640 1 -- 172.21.15.133:0/3917381305 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c05f0a0 con 0x55d224038680 2024-09-11T23:42:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.476+0000 7f306e1ea740 1 -- 172.21.15.133:0/3917381305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 msgr2=0x55d22403aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.476+0000 7f306e1ea740 1 --2- 172.21.15.133:0/3917381305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 0x55d22403aa70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f305c0983c0 tx=0x7f305c09bbd0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.476+0000 7f306e1ea740 1 -- 172.21.15.133:0/3917381305 shutdown_connections 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.476+0000 7f306e1ea740 1 --2- 172.21.15.133:0/3917381305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 0x55d22403aa70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.476+0000 7f306e1ea740 1 --2- 172.21.15.133:0/3917381305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d22402ddc0 0x55d224038140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.476+0000 7f306e1ea740 1 --2- 172.21.15.133:0/3917381305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 0x55d22402d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.476+0000 7f306e1ea740 1 -- 172.21.15.133:0/3917381305 >> 172.21.15.133:0/3917381305 conn(0x55d224029a30 msgr2=0x55d224029e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.476+0000 7f306e1ea740 1 -- 172.21.15.133:0/3917381305 shutdown_connections 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.477+0000 7f306e1ea740 1 -- 172.21.15.133:0/3917381305 wait complete. 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.477+0000 7f306e1ea740 1 -- start start 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.477+0000 7f306e1ea740 1 Processor -- start 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.478+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 0x55d224037fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.478+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d22402ddc0 0x55d2240352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.478+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 0x55d224035880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.478+0000 7f3069bad640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 0x55d224037fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.478+0000 7f306e1ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d22403b250 con 0x55d22402ddc0 2024-09-11T23:42:34.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.478+0000 7f3069bad640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 0x55d224037fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:43798/0 (socket says 172.21.15.133:43798) 2024-09-11T23:42:34.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.478+0000 7f306e1ea740 1 -- 172.21.15.133:0/2224019057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d223f4e050 con 0x55d224038680 2024-09-11T23:42:34.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.478+0000 7f3069bad640 1 -- 172.21.15.133:0/2224019057 learned_addr learned my addr 172.21.15.133:0/2224019057 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:34.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.478+0000 7f306e1ea740 1 -- 172.21.15.133:0/2224019057 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d22403b3d0 con 0x55d22402d4a0 2024-09-11T23:42:34.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.478+0000 7f30693ac640 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d22402ddc0 0x55d2240352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.478+0000 7f306a3ae640 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 0x55d224035880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f306e1ea740 10 client.0 _ll_register_callbacks cb 0x55d223f8e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:34.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f3069bad640 1 -- 172.21.15.133:0/2224019057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 msgr2=0x55d224035880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:34.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f3069bad640 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 0x55d224035880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f3069bad640 1 -- 172.21.15.133:0/2224019057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d22402ddc0 msgr2=0x55d2240352e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:34.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f3069bad640 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d22402ddc0 0x55d2240352e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f3069bad640 1 -- 172.21.15.133:0/2224019057 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d223d043c0 con 0x55d22402d4a0 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f306a3ae640 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 0x55d224035880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f306e1ea740 1 -- 172.21.15.133:0/2224019057 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f305c0a4bd0 con 0x55d22402d4a0 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f3069bad640 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 0x55d224037fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f30600094c0 tx=0x7f30600094f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b1d76bc8a487e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f304d7fa640 10 client.26419 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f304d7fa640 1 -- 172.21.15.133:0/2224019057 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306004c050 con 0x55d22402d4a0 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f304d7fa640 1 -- 172.21.15.133:0/2224019057 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3060004300 con 0x55d22402d4a0 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f304d7fa640 1 -- 172.21.15.133:0/2224019057 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3060030420 con 0x55d22402d4a0 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f306e1ea740 10 client.26419 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.479+0000 7f306e1ea740 1 -- 172.21.15.133:0/2224019057 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55d224046b30 con 0x55d22402d4a0 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.480+0000 7f304d7fa640 1 -- 172.21.15.133:0/2224019057 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f30600305c0 con 0x55d22402d4a0 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.480+0000 7f306e1ea740 10 client.26419 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:34.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.480+0000 7f306e1ea740 10 client.26419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.480+0000 7f306e1ea740 20 client.26419 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.480+0000 7f306e1ea740 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d224048130 0x55d22404a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.480+0000 7f306e1ea740 4 client.26419 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.480+0000 7f306e1ea740 1 -- 172.21.15.133:0/2224019057 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d223fa0850 con 0x55d224048130 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.480+0000 7f30693ac640 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d224048130 0x55d22404a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.481+0000 7f30693ac640 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d224048130 0x55d22404a5d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a28613df97059e server_cookie=a1bd0ea67f2822b9 in_seq=0 out_seq=0 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.481+0000 7f304d7fa640 10 client.26419 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.482+0000 7f304d7fa640 1 -- 172.21.15.133:0/2224019057 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d223fa0850 con 0x55d224048130 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.482+0000 7f304d7fa640 10 client.26419 handle_command_reply: tid=0 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.482+0000 7f306e1ea740 1 client.26419 shutdown 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f306e1ea740 1 -- 172.21.15.133:0/2224019057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 msgr2=0x55d224037fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f306e1ea740 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 0x55d224037fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f30600094c0 tx=0x7f30600094f0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f306e1ea740 1 -- 172.21.15.133:0/2224019057 shutdown_connections 2024-09-11T23:42:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f306e1ea740 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d224048130 0x55d22404a5d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f306e1ea740 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d224038680 0x55d224035880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f306e1ea740 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d22402ddc0 0x55d2240352e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f304d7fa640 0 client.26419 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f306e1ea740 1 --2- 172.21.15.133:0/2224019057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402d4a0 0x55d224037fe0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f306e1ea740 1 -- 172.21.15.133:0/2224019057 >> 172.21.15.133:0/2224019057 conn(0x55d224029a30 msgr2=0x55d22402f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f306e1ea740 1 -- 172.21.15.133:0/2224019057 shutdown_connections 2024-09-11T23:42:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f306e1ea740 1 -- 172.21.15.133:0/2224019057 wait complete. 2024-09-11T23:42:34.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.483+0000 7f306e1ea740 20 client.26419 trim_cache size 0 max 16384 2024-09-11T23:42:34.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f306e1ea740 1 Processor -- start 2024-09-11T23:42:34.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f306e1ea740 1 -- start start 2024-09-11T23:42:34.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d22402d4a0 0x55d224028e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 0x55d224034bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d224038680 0x55d22404fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f306e1ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f30641af9d0 con 0x55d224038680 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f306e1ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30641b1510 con 0x55d22402d4a0 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f306e1ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30641b00a0 con 0x55d22402ddc0 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f30693ac640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 0x55d224034bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f306a3ae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d224038680 0x55d22404fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f30693ac640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 0x55d224034bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:43800/0 (socket says 172.21.15.133:43800) 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f306a3ae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d224038680 0x55d22404fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42468/0 (socket says 172.21.15.133:42468) 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f30693ac640 1 -- 172.21.15.133:0/3385133769 learned_addr learned my addr 172.21.15.133:0/3385133769 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.501+0000 7f3069bad640 1 --2- 172.21.15.133:0/3385133769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d22402d4a0 0x55d224028e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.502+0000 7f3069bad640 1 -- 172.21.15.133:0/3385133769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 msgr2=0x55d224034bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.502+0000 7f3069bad640 1 --2- 172.21.15.133:0/3385133769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 0x55d224034bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.502+0000 7f3069bad640 1 -- 172.21.15.133:0/3385133769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d224038680 msgr2=0x55d22404fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.502+0000 7f3069bad640 1 --2- 172.21.15.133:0/3385133769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d224038680 0x55d22404fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.502+0000 7f3069bad640 1 -- 172.21.15.133:0/3385133769 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d22402abc0 con 0x55d22402d4a0 2024-09-11T23:42:34.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.502+0000 7f3069bad640 1 --2- 172.21.15.133:0/3385133769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d22402d4a0 0x55d224028e20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f306002a930 tx=0x7f3060038000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a1199a46ab7f038 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:34.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.502+0000 7f304f7fe640 1 -- 172.21.15.133:0/3385133769 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306004c070 con 0x55d22402d4a0 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.502+0000 7f304f7fe640 1 -- 172.21.15.133:0/3385133769 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c0a4bd0 con 0x55d22402d4a0 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.502+0000 7f304f7fe640 1 -- 172.21.15.133:0/3385133769 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060003ba0 con 0x55d22402d4a0 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.503+0000 7f306e1ea740 1 -- 172.21.15.133:0/3385133769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d22402d4a0 msgr2=0x55d224028e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.503+0000 7f306e1ea740 1 --2- 172.21.15.133:0/3385133769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d22402d4a0 0x55d224028e20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f306002a930 tx=0x7f3060038000 comp rx=0 tx=0).stop 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.503+0000 7f306e1ea740 1 -- 172.21.15.133:0/3385133769 shutdown_connections 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.503+0000 7f306e1ea740 1 --2- 172.21.15.133:0/3385133769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d224038680 0x55d22404fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.503+0000 7f306e1ea740 1 --2- 172.21.15.133:0/3385133769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 0x55d224034bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.503+0000 7f306e1ea740 1 --2- 172.21.15.133:0/3385133769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d22402d4a0 0x55d224028e20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.503+0000 7f306e1ea740 1 -- 172.21.15.133:0/3385133769 >> 172.21.15.133:0/3385133769 conn(0x55d224029a30 msgr2=0x55d22402f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.503+0000 7f306e1ea740 1 -- 172.21.15.133:0/3385133769 shutdown_connections 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.503+0000 7f306e1ea740 1 -- 172.21.15.133:0/3385133769 wait complete. 2024-09-11T23:42:34.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.504+0000 7f306e1ea740 1 -- start start 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.504+0000 7f306e1ea740 1 Processor -- start 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.504+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d22402d4a0 0x55d22403cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.504+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 0x55d22403d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.504+0000 7f306e1ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d224038680 0x55d2240abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.504+0000 7f306e1ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f30641b0e40 con 0x55d224038680 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.504+0000 7f306e1ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30641b0770 con 0x55d22402d4a0 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.504+0000 7f306e1ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30641ad9d0 con 0x55d22402ddc0 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.504+0000 7f306e1ea740 10 client.0 _ll_register_callbacks cb 0x7f305c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f30693ac640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 0x55d22403d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f3069bad640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d22402d4a0 0x55d22403cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f30693ac640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 0x55d22403d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:43810/0 (socket says 172.21.15.133:43810) 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f30693ac640 1 -- 172.21.15.133:0/1952695987 learned_addr learned my addr 172.21.15.133:0/1952695987 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:34.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f306a3ae640 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d224038680 0x55d2240abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f30693ac640 1 -- 172.21.15.133:0/1952695987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d22402d4a0 msgr2=0x55d22403cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f30693ac640 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d22402d4a0 0x55d22403cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f30693ac640 1 -- 172.21.15.133:0/1952695987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d224038680 msgr2=0x55d2240abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f30693ac640 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d224038680 0x55d2240abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f30693ac640 1 -- 172.21.15.133:0/1952695987 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2240ac360 con 0x55d22402ddc0 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f30693ac640 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 0x55d22403d1a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f30540096c0 tx=0x7f30540046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64c10c39e910a4b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.505+0000 7f304d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f304d7fa640 1 -- 172.21.15.133:0/1952695987 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054036060 con 0x55d22402ddc0 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f304d7fa640 1 -- 172.21.15.133:0/1952695987 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3054017070 con 0x55d22402ddc0 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f306e1ea740 1 -- 172.21.15.133:0/1952695987 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f305c0a4bd0 con 0x55d22402ddc0 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f304d7fa640 1 -- 172.21.15.133:0/1952695987 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3054004de0 con 0x55d22402ddc0 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f306e1ea740 10 client.26425 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f306e1ea740 1 -- 172.21.15.133:0/1952695987 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d2240b3770 con 0x55d22402ddc0 2024-09-11T23:42:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f304d7fa640 1 -- 172.21.15.133:0/1952695987 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c0a4bd0 con 0x55d22402ddc0 2024-09-11T23:42:34.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f304d7fa640 1 -- 172.21.15.133:0/1952695987 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3054031850 con 0x55d22402ddc0 2024-09-11T23:42:34.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f306e1ea740 10 client.26425 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:34.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f306e1ea740 10 client.26425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:34.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f306e1ea740 20 client.26425 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:34.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f306e1ea740 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d2240b3e80 0x55d2240b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:34.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f306e1ea740 4 client.26425 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.506+0000 7f306e1ea740 1 -- 172.21.15.133:0/1952695987 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d223f9ad60 con 0x55d2240b3e80 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.507+0000 7f3069bad640 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d2240b3e80 0x55d2240b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.507+0000 7f3069bad640 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d2240b3e80 0x55d2240b62a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=165f1f9324a7b217 server_cookie=45335e384a1131c4 in_seq=0 out_seq=0 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.507+0000 7f304d7fa640 10 client.26425 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f304d7fa640 1 -- 172.21.15.133:0/1952695987 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d223f9ad60 con 0x55d2240b3e80 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f304d7fa640 10 client.26425 handle_command_reply: tid=0 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f306e1ea740 1 client.26425 shutdown 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f306e1ea740 1 -- 172.21.15.133:0/1952695987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 msgr2=0x55d22403d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f306e1ea740 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 0x55d22403d1a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f30540096c0 tx=0x7f30540046c0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f306e1ea740 1 -- 172.21.15.133:0/1952695987 shutdown_connections 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f306e1ea740 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d2240b3e80 0x55d2240b62a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f306e1ea740 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d224038680 0x55d2240abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f306e1ea740 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d22402ddc0 0x55d22403d1a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f304d7fa640 0 client.26425 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:34.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f306e1ea740 1 --2- 172.21.15.133:0/1952695987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d22402d4a0 0x55d22403cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f306e1ea740 1 -- 172.21.15.133:0/1952695987 >> 172.21.15.133:0/1952695987 conn(0x55d224029a30 msgr2=0x55d224036b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:34.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f306e1ea740 1 -- 172.21.15.133:0/1952695987 shutdown_connections 2024-09-11T23:42:34.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.509+0000 7f306e1ea740 1 -- 172.21.15.133:0/1952695987 wait complete. 2024-09-11T23:42:34.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.510+0000 7f306e1ea740 20 client.26425 trim_cache size 0 max 16384 2024-09-11T23:42:34.512 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:42:34.512 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:42:34.512 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-09-11T23:42:34.512 INFO:teuthology.orchestra.run.smithi133.stdout: "11e70c4b-b2c2-49ad-8876-d3cf45d75e0e": { 2024-09-11T23:42:34.512 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-09-11T23:42:34.513 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "11e70c4b-b2c2-49ad-8876-d3cf45d75e0e", 2024-09-11T23:42:34.513 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2024-09-11T23:42:34.513 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:42:34.513 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:42:34.513 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:42:34.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.510+0000 7f306be11640 1 -- 172.21.15.133:0/49007808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30500a2d70 msgr2=0x7f30500a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:34.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.510+0000 7f306be11640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30500a2d70 0x7f30500a5230 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f30540098b0 tx=0x7f3054009840 comp rx=0 tx=0).stop 2024-09-11T23:42:34.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.510+0000 7f306be11640 1 -- 172.21.15.133:0/49007808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064102ef0 msgr2=0x7f306419bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:34.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.510+0000 7f306be11640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064102ef0 0x7f306419bd30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f305c009fd0 tx=0x7f305c00ef10 comp rx=0 tx=0).stop 2024-09-11T23:42:34.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.510+0000 7f306be11640 1 -- 172.21.15.133:0/49007808 shutdown_connections 2024-09-11T23:42:34.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.510+0000 7f306be11640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f30500a2d70 0x7f30500a5230 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.510+0000 7f306be11640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064102ef0 0x7f306419bd30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.510+0000 7f306be11640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3064102570 0x7f30641979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.510+0000 7f306be11640 1 --2- 172.21.15.133:0/49007808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3064100e20 0x7f3064197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:34.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.510+0000 7f306be11640 1 -- 172.21.15.133:0/49007808 >> 172.21.15.133:0/49007808 conn(0x7f30640fc7d0 msgr2=0x7f3064104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:34.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.511+0000 7f306be11640 1 -- 172.21.15.133:0/49007808 shutdown_connections 2024-09-11T23:42:34.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:34.511+0000 7f306be11640 1 -- 172.21.15.133:0/49007808 wait complete. 2024-09-11T23:42:35.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:34 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:42:35.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:34 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:42:35.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:34 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:42:36.038 INFO:tasks.cephfs.filesystem:scrub status for tag:11e70c4b-b2c2-49ad-8876-d3cf45d75e0e - {'path': '/', 'tag': '11e70c4b-b2c2-49ad-8876-d3cf45d75e0e', 'options': 'recursive,force'} 2024-09-11T23:42:36.215 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:36 smithi133 ceph-mon[31430]: pgmap v178: 97 pgs: 97 active+clean; 50 GiB data, 190 GiB used, 883 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 805 MiB/s wr, 1.50k op/s 2024-09-11T23:42:36.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:36 smithi178 ceph-mon[35882]: pgmap v178: 97 pgs: 97 active+clean; 50 GiB data, 190 GiB used, 883 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 805 MiB/s wr, 1.50k op/s 2024-09-11T23:42:36.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:36 smithi138 ceph-mon[35142]: pgmap v178: 97 pgs: 97 active+clean; 50 GiB data, 190 GiB used, 883 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 805 MiB/s wr, 1.50k op/s 2024-09-11T23:42:38.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:38 smithi178 ceph-mon[35882]: pgmap v179: 97 pgs: 97 active+clean; 50 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 672 MiB/s wr, 1.34k op/s 2024-09-11T23:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.875+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2241517047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2c811b140 msgr2=0x7fa2c811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.875+0000 7fa2c7fff640 1 --2- 172.21.15.133:0/2241517047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2c811b140 0x7fa2c811d530 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009ca0 tx=0x7fa2b802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.882+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2241517047 shutdown_connections 2024-09-11T23:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.883+0000 7fa2c7fff640 1 --2- 172.21.15.133:0/2241517047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2c811b140 0x7fa2c811d530 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.883+0000 7fa2c7fff640 1 --2- 172.21.15.133:0/2241517047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c81095d0 0x7fa2c8109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.883+0000 7fa2c7fff640 1 --2- 172.21.15.133:0/2241517047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c81091f0 0x7fa2c8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.883+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2241517047 >> 172.21.15.133:0/2241517047 conn(0x7fa2c806cbf0 msgr2=0x7fa2c806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.889+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2241517047 shutdown_connections 2024-09-11T23:42:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.889+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2241517047 wait complete. 2024-09-11T23:42:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.890+0000 7fa2c7fff640 1 Processor -- start 2024-09-11T23:42:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.894+0000 7fa2c7fff640 1 -- start start 2024-09-11T23:42:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.895+0000 7fa2c7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2c81091f0 0x7fa2c811a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.895+0000 7fa2c7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c81095d0 0x7fa2c81149a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.895+0000 7fa2c7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c811b140 0x7fa2c8114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.895+0000 7fa2c7fff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c8111370 con 0x7fa2c81091f0 2024-09-11T23:42:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.895+0000 7fa2c7fff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c81111f0 con 0x7fa2c81095d0 2024-09-11T23:42:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.895+0000 7fa2c7fff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c81114f0 con 0x7fa2c811b140 2024-09-11T23:42:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.898+0000 7fa2c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2c81091f0 0x7fa2c811a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.898+0000 7fa2c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2c81091f0 0x7fa2c811a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42492/0 (socket says 172.21.15.133:42492) 2024-09-11T23:42:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.898+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/2512587245 learned_addr learned my addr 172.21.15.133:0/2512587245 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.898+0000 7fa2c67fc640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c81095d0 0x7fa2c81149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.899+0000 7fa2c77fe640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c811b140 0x7fa2c8114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.904+0000 7fa2c67fc640 1 -- 172.21.15.133:0/2512587245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c811b140 msgr2=0x7fa2c8114ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.904+0000 7fa2c67fc640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c811b140 0x7fa2c8114ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.904+0000 7fa2c67fc640 1 -- 172.21.15.133:0/2512587245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2c81091f0 msgr2=0x7fa2c811a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.904+0000 7fa2c67fc640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2c81091f0 0x7fa2c811a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.904+0000 7fa2c67fc640 1 -- 172.21.15.133:0/2512587245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c81156b0 con 0x7fa2c81095d0 2024-09-11T23:42:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.910+0000 7fa2c67fc640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c81095d0 0x7fa2c81149a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000d6d0 tx=0x7fa2b000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.913+0000 7fa2bffff640 1 -- 172.21.15.133:0/2512587245 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0017070 con 0x7fa2c81095d0 2024-09-11T23:42:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.913+0000 7fa2bffff640 1 -- 172.21.15.133:0/2512587245 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b0009820 con 0x7fa2c81095d0 2024-09-11T23:42:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.913+0000 7fa2bffff640 1 -- 172.21.15.133:0/2512587245 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0004d00 con 0x7fa2c81095d0 2024-09-11T23:42:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.915+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2512587245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c81bbe20 con 0x7fa2c81095d0 2024-09-11T23:42:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.915+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2512587245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c81bc230 con 0x7fa2c81095d0 2024-09-11T23:42:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.924+0000 7fa2d0284740 1 Processor -- start 2024-09-11T23:42:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.927+0000 7fa2d0284740 1 -- start start 2024-09-11T23:42:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.928+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941b9cc0 0x55cf941ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.928+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941ba5e0 0x55cf941c4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.928+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941c4e80 0x55cf941c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.928+0000 7fa2d0284740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cf9411e2f0 con 0x55cf941ba5e0 2024-09-11T23:42:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.928+0000 7fa2d0284740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf940c6270 con 0x55cf941b9cc0 2024-09-11T23:42:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.928+0000 7fa2d0284740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cf9411a2b0 con 0x55cf941c4e80 2024-09-11T23:42:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.929+0000 7fa2c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941ba5e0 0x55cf941c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.929+0000 7fa2c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941ba5e0 0x55cf941c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42504/0 (socket says 172.21.15.133:42504) 2024-09-11T23:42:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.930+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1276703424 learned_addr learned my addr 172.21.15.133:0/1276703424 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.930+0000 7fa2c67fc640 1 --2- 172.21.15.133:0/1276703424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941c4e80 0x55cf941c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.930+0000 7fa2c77fe640 1 --2- 172.21.15.133:0/1276703424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941b9cc0 0x55cf941ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.931+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1276703424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941c4e80 msgr2=0x55cf941c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.931+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/1276703424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941c4e80 0x55cf941c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.931+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1276703424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941b9cc0 msgr2=0x55cf941ba0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.931+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/1276703424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941b9cc0 0x55cf941ba0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.931+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1276703424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf940c5340 con 0x55cf941ba5e0 2024-09-11T23:42:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.932+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/1276703424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941ba5e0 0x55cf941c4940 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009b10 tx=0x7fa2c000e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822e05dcae4fd850 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.932+0000 7fa2bffff640 1 -- 172.21.15.133:0/2512587245 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa2b0005220 con 0x7fa2c81095d0 2024-09-11T23:42:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.934+0000 7fa2bffff640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa2b40a2ec0 0x7fa2b40a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.935+0000 7fa2bffff640 1 -- 172.21.15.133:0/2512587245 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fa2b00ceaf0 con 0x7fa2c81095d0 2024-09-11T23:42:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.936+0000 7fa2bd7fa640 1 -- 172.21.15.133:0/1276703424 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0017070 con 0x55cf941ba5e0 2024-09-11T23:42:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.936+0000 7fa2bd7fa640 1 -- 172.21.15.133:0/1276703424 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c000ebe0 con 0x55cf941ba5e0 2024-09-11T23:42:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.936+0000 7fa2c77fe640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa2b40a2ec0 0x7fa2b40a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.938+0000 7fa2d0284740 1 -- 172.21.15.133:0/1276703424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941ba5e0 msgr2=0x55cf941c4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.938+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1276703424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941ba5e0 0x55cf941c4940 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009b10 tx=0x7fa2c000e1d0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.938+0000 7fa2c77fe640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa2b40a2ec0 0x7fa2b40a5380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8005e30 tx=0x7fa2b8005c50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.939+0000 7fa2d0284740 1 -- 172.21.15.133:0/1276703424 shutdown_connections 2024-09-11T23:42:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.939+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1276703424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941c4e80 0x55cf941c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.939+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1276703424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941ba5e0 0x55cf941c4940 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.939+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1276703424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941b9cc0 0x55cf941ba0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.939+0000 7fa2d0284740 1 -- 172.21.15.133:0/1276703424 >> 172.21.15.133:0/1276703424 conn(0x55cf941b6230 msgr2=0x55cf941b6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:38.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:38 smithi138 ceph-mon[35142]: pgmap v179: 97 pgs: 97 active+clean; 50 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 672 MiB/s wr, 1.34k op/s 2024-09-11T23:42:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.949+0000 7fa2d0284740 1 -- 172.21.15.133:0/1276703424 shutdown_connections 2024-09-11T23:42:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.949+0000 7fa2d0284740 1 -- 172.21.15.133:0/1276703424 wait complete. 2024-09-11T23:42:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.949+0000 7fa2d0284740 1 -- start start 2024-09-11T23:42:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.954+0000 7fa2d0284740 1 Processor -- start 2024-09-11T23:42:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.955+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941b9cc0 0x55cf941c1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.955+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941ba5e0 0x55cf941c2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.955+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941c4e80 0x55cf941c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.955+0000 7fa2d0284740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cf941c7aa0 con 0x55cf941ba5e0 2024-09-11T23:42:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.955+0000 7fa2d0284740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf940a7290 con 0x55cf941c4e80 2024-09-11T23:42:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.955+0000 7fa2d0284740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cf941c7c20 con 0x55cf941b9cc0 2024-09-11T23:42:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.955+0000 7fa2d0284740 10 client.0 _ll_register_callbacks cb 0x55cf940d8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.955+0000 7fa2c6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941b9cc0 0x55cf941c1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.956+0000 7fa2c6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941b9cc0 0x55cf941c1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:43856/0 (socket says 172.21.15.133:43856) 2024-09-11T23:42:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.956+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1085317168 learned_addr learned my addr 172.21.15.133:0/1085317168 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.956+0000 7fa2c77fe640 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941c4e80 0x55cf941c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.958+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1085317168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941c4e80 msgr2=0x55cf941c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.958+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941c4e80 0x55cf941c2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.958+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1085317168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941ba5e0 msgr2=0x55cf941c2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.958+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941ba5e0 0x55cf941c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.958+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1085317168 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf93e31210 con 0x55cf941b9cc0 2024-09-11T23:42:38.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:38 smithi133 ceph-mon[31430]: pgmap v179: 97 pgs: 97 active+clean; 50 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 672 MiB/s wr, 1.34k op/s 2024-09-11T23:42:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.965+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941b9cc0 0x55cf941c1e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000ef50 tx=0x7fa2c000ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=219e90b25be18f3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.965+0000 7fa2ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:42:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.965+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/1085317168 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0017050 con 0x55cf941b9cc0 2024-09-11T23:42:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.966+0000 7fa2d0284740 1 -- 172.21.15.133:0/1085317168 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cf9412ba20 con 0x55cf941b9cc0 2024-09-11T23:42:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.966+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/1085317168 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fa2c0012de0 con 0x55cf941b9cc0 2024-09-11T23:42:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.966+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/1085317168 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa2c0013470 con 0x55cf941b9cc0 2024-09-11T23:42:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.969+0000 7fa2d0284740 10 client.26434 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.969+0000 7fa2d0284740 1 -- 172.21.15.133:0/1085317168 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cf93efd620 con 0x55cf941b9cc0 2024-09-11T23:42:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.971+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/1085317168 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cf9412ba20 con 0x55cf941b9cc0 2024-09-11T23:42:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.971+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/1085317168 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa2c002e050 con 0x55cf941b9cc0 2024-09-11T23:42:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.972+0000 7fa2d0284740 10 client.26434 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.972+0000 7fa2d0284740 10 client.26434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.972+0000 7fa2d0284740 20 client.26434 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.972+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf941d53c0 0x55cf941d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.972+0000 7fa2d0284740 4 client.26434 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.972+0000 7fa2d0284740 1 -- 172.21.15.133:0/1085317168 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf94125d70 con 0x55cf941d53c0 2024-09-11T23:42:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.976+0000 7fa2c67fc640 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf941d53c0 0x55cf941d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.982+0000 7fa2c67fc640 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf941d53c0 0x55cf941d7860 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87d26355ec08be server_cookie=b2697a5d8c1dfbb9 in_seq=0 out_seq=0 2024-09-11T23:42:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.982+0000 7fa2ae7fc640 10 client.26434 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.984+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/1085317168 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf94125d70 con 0x55cf941d53c0 2024-09-11T23:42:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.985+0000 7fa2ae7fc640 10 client.26434 handle_command_reply: tid=0 2024-09-11T23:42:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.985+0000 7fa2d0284740 1 client.26434 shutdown 2024-09-11T23:42:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.985+0000 7fa2d0284740 1 -- 172.21.15.133:0/1085317168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941b9cc0 msgr2=0x55cf941c1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.985+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941b9cc0 0x55cf941c1e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000ef50 tx=0x7fa2c000ef80 comp rx=0 tx=0).stop 2024-09-11T23:42:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.985+0000 7fa2d0284740 1 -- 172.21.15.133:0/1085317168 shutdown_connections 2024-09-11T23:42:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.985+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf941d53c0 0x55cf941d7860 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.985+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941c4e80 0x55cf941c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.985+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941ba5e0 0x55cf941c2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.985+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1085317168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941b9cc0 0x55cf941c1e00 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000ef50 tx=0x7fa2c000ef80 comp rx=0 tx=0).stop 2024-09-11T23:42:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.985+0000 7fa2d0284740 1 -- 172.21.15.133:0/1085317168 >> 172.21.15.133:0/1085317168 conn(0x55cf941b6230 msgr2=0x55cf941bbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.985+0000 7fa2ae7fc640 0 client.26434 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.989+0000 7fa2d0284740 1 -- 172.21.15.133:0/1085317168 shutdown_connections 2024-09-11T23:42:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.990+0000 7fa2d0284740 1 -- 172.21.15.133:0/1085317168 wait complete. 2024-09-11T23:42:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:38.990+0000 7fa2d0284740 20 client.26434 trim_cache size 0 max 16384 2024-09-11T23:42:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.031+0000 7fa2d0284740 1 Processor -- start 2024-09-11T23:42:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.031+0000 7fa2d0284740 1 -- start start 2024-09-11T23:42:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.032+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941ba5e0 0x55cf941c2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.032+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941c4e80 0x55cf941dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.032+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941c4670 0x55cf941dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.033+0000 7fa2d0284740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a800e740 con 0x55cf941c4e80 2024-09-11T23:42:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.033+0000 7fa2d0284740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8010280 con 0x55cf941c4670 2024-09-11T23:42:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.033+0000 7fa2d0284740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a800ee10 con 0x55cf941ba5e0 2024-09-11T23:42:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.033+0000 7fa2c6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941ba5e0 0x55cf941c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.033+0000 7fa2c6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941ba5e0 0x55cf941c2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:43860/0 (socket says 172.21.15.133:43860) 2024-09-11T23:42:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.033+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1261800870 learned_addr learned my addr 172.21.15.133:0/1261800870 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.033+0000 7fa2c77fe640 1 --2- 172.21.15.133:0/1261800870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941c4670 0x55cf941dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.034+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1261800870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941c4670 msgr2=0x55cf941dd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.034+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/1261800870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941c4670 0x55cf941dd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.034+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1261800870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941c4e80 msgr2=0x55cf941dd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.034+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/1261800870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941c4e80 0x55cf941dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.034+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/1261800870 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf941d0df0 con 0x55cf941ba5e0 2024-09-11T23:42:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.034+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/1261800870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941ba5e0 0x55cf941c2680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009a60 tx=0x7fa2c0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=575b8e1e9a321085 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.035+0000 7fa2bcff9640 1 -- 172.21.15.133:0/1261800870 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0017040 con 0x55cf941ba5e0 2024-09-11T23:42:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.035+0000 7fa2bcff9640 1 -- 172.21.15.133:0/1261800870 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c0034030 con 0x55cf941ba5e0 2024-09-11T23:42:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.035+0000 7fa2bcff9640 1 -- 172.21.15.133:0/1261800870 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0013af0 con 0x55cf941ba5e0 2024-09-11T23:42:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.036+0000 7fa2d0284740 1 -- 172.21.15.133:0/1261800870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941ba5e0 msgr2=0x55cf941c2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.036+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1261800870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941ba5e0 0x55cf941c2680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009a60 tx=0x7fa2c0000bc0 comp rx=0 tx=0).stop 2024-09-11T23:42:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.037+0000 7fa2d0284740 1 -- 172.21.15.133:0/1261800870 shutdown_connections 2024-09-11T23:42:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.037+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1261800870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941c4670 0x55cf941dd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.037+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1261800870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941c4e80 0x55cf941dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.037+0000 7fa2d0284740 1 --2- 172.21.15.133:0/1261800870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941ba5e0 0x55cf941c2680 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.037+0000 7fa2d0284740 1 -- 172.21.15.133:0/1261800870 >> 172.21.15.133:0/1261800870 conn(0x55cf941b6230 msgr2=0x55cf941c06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.037+0000 7fa2d0284740 1 -- 172.21.15.133:0/1261800870 shutdown_connections 2024-09-11T23:42:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.038+0000 7fa2d0284740 1 -- 172.21.15.133:0/1261800870 wait complete. 2024-09-11T23:42:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.038+0000 7fa2d0284740 1 -- start start 2024-09-11T23:42:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.039+0000 7fa2d0284740 1 Processor -- start 2024-09-11T23:42:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.039+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941ba5e0 0x55cf941c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.039+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941c4670 0x55cf941c8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.040+0000 7fa2d0284740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941c4e80 0x55cf941c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.040+0000 7fa2d0284740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a800fbb0 con 0x55cf941c4e80 2024-09-11T23:42:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.040+0000 7fa2d0284740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a800f4e0 con 0x55cf941ba5e0 2024-09-11T23:42:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.040+0000 7fa2d0284740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a800c720 con 0x55cf941c4670 2024-09-11T23:42:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.040+0000 7fa2c6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941ba5e0 0x55cf941c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.040+0000 7fa2c6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941ba5e0 0x55cf941c7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:57044/0 (socket says 172.21.15.133:57044) 2024-09-11T23:42:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.040+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/358023995 learned_addr learned my addr 172.21.15.133:0/358023995 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.040+0000 7fa2d0284740 10 client.0 _ll_register_callbacks cb 0x7fa2c0009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.041+0000 7fa2c77fe640 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941c4e80 0x55cf941c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.041+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/358023995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941c4670 msgr2=0x55cf941c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.041+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941c4670 0x55cf941c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.041+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/358023995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941c4e80 msgr2=0x55cf941c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.041+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941c4e80 0x55cf941c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.041+0000 7fa2c6ffd640 1 -- 172.21.15.133:0/358023995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9423e1b0 con 0x55cf941ba5e0 2024-09-11T23:42:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.042+0000 7fa2c6ffd640 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941ba5e0 0x55cf941c7d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2c002f430 tx=0x7fa2c001ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c726b12e5df63fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.042+0000 7fa2ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.043+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/358023995 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0017040 con 0x55cf941ba5e0 2024-09-11T23:42:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.043+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/358023995 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fa2c0013820 con 0x55cf941ba5e0 2024-09-11T23:42:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.043+0000 7fa2d0284740 1 -- 172.21.15.133:0/358023995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c0034030 con 0x55cf941ba5e0 2024-09-11T23:42:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.043+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/358023995 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa2c0012be0 con 0x55cf941ba5e0 2024-09-11T23:42:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.043+0000 7fa2d0284740 10 client.26489 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.044+0000 7fa2d0284740 1 -- 172.21.15.133:0/358023995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cf9423f480 con 0x55cf941ba5e0 2024-09-11T23:42:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.044+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/358023995 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c0034030 con 0x55cf941ba5e0 2024-09-11T23:42:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.044+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/358023995 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa2c002e070 con 0x55cf941ba5e0 2024-09-11T23:42:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.044+0000 7fa2d0284740 10 client.26489 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.044+0000 7fa2d0284740 10 client.26489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.044+0000 7fa2d0284740 20 client.26489 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.044+0000 7fa2d0284740 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf9423f870 0x55cf94241cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.045+0000 7fa2d0284740 4 client.26489 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:42:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.045+0000 7fa2d0284740 1 -- 172.21.15.133:0/358023995 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf940a2900 con 0x55cf9423f870 2024-09-11T23:42:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.045+0000 7fa2c67fc640 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf9423f870 0x55cf94241cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.046+0000 7fa2c67fc640 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf9423f870 0x55cf94241cd0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35aef04e898a5c8e server_cookie=6c23de259d0d1e82 in_seq=0 out_seq=0 2024-09-11T23:42:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:39.046+0000 7fa2ae7fc640 10 client.26489 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:39.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:39 smithi138 ceph-mon[35142]: pgmap v180: 97 pgs: 97 active+clean; 50 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 872 MiB/s wr, 1.85k op/s 2024-09-11T23:42:39.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:39 smithi133 ceph-mon[31430]: pgmap v180: 97 pgs: 97 active+clean; 50 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 872 MiB/s wr, 1.85k op/s 2024-09-11T23:42:40.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:39 smithi178 ceph-mon[35882]: pgmap v180: 97 pgs: 97 active+clean; 50 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 872 MiB/s wr, 1.85k op/s 2024-09-11T23:42:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.376+0000 7fa2ae7fc640 1 -- 172.21.15.133:0/358023995 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf940a2900 con 0x55cf9423f870 2024-09-11T23:42:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.376+0000 7fa2ae7fc640 10 client.26489 handle_command_reply: tid=0 2024-09-11T23:42:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.377+0000 7fa2d0284740 1 client.26489 shutdown 2024-09-11T23:42:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.384+0000 7fa2d0284740 1 -- 172.21.15.133:0/358023995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941ba5e0 msgr2=0x55cf941c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.384+0000 7fa2d0284740 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941ba5e0 0x55cf941c7d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2c002f430 tx=0x7fa2c001ca10 comp rx=0 tx=0).stop 2024-09-11T23:42:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.388+0000 7fa2d0284740 1 -- 172.21.15.133:0/358023995 shutdown_connections 2024-09-11T23:42:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.388+0000 7fa2d0284740 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf9423f870 0x55cf94241cd0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.388+0000 7fa2d0284740 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf941c4e80 0x55cf941c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.388+0000 7fa2d0284740 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf941c4670 0x55cf941c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.388+0000 7fa2d0284740 1 --2- 172.21.15.133:0/358023995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf941ba5e0 0x55cf941c7d00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.388+0000 7fa2d0284740 1 -- 172.21.15.133:0/358023995 >> 172.21.15.133:0/358023995 conn(0x55cf941b6230 msgr2=0x55cf941b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.388+0000 7fa2ae7fc640 0 client.26489 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.389+0000 7fa2d0284740 1 -- 172.21.15.133:0/358023995 shutdown_connections 2024-09-11T23:42:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.390+0000 7fa2d0284740 1 -- 172.21.15.133:0/358023995 wait complete. 2024-09-11T23:42:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.390+0000 7fa2d0284740 20 client.26489 trim_cache size 0 max 16384 2024-09-11T23:42:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:42:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:42:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:42:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:42:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.396+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2512587245 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa2b40a2ec0 msgr2=0x7fa2b40a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.396+0000 7fa2c7fff640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa2b40a2ec0 0x7fa2b40a5380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8005e30 tx=0x7fa2b8005c50 comp rx=0 tx=0).stop 2024-09-11T23:42:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.396+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2512587245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c81095d0 msgr2=0x7fa2c81149a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.397+0000 7fa2c7fff640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c81095d0 0x7fa2c81149a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000d6d0 tx=0x7fa2b000db90 comp rx=0 tx=0).stop 2024-09-11T23:42:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.399+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2512587245 shutdown_connections 2024-09-11T23:42:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.399+0000 7fa2c7fff640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa2b40a2ec0 0x7fa2b40a5380 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.399+0000 7fa2c7fff640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2c811b140 0x7fa2c8114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.399+0000 7fa2c7fff640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c81095d0 0x7fa2c81149a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.399+0000 7fa2c7fff640 1 --2- 172.21.15.133:0/2512587245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2c81091f0 0x7fa2c811a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.400+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2512587245 >> 172.21.15.133:0/2512587245 conn(0x7fa2c806cbf0 msgr2=0x7fa2c80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.400+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2512587245 shutdown_connections 2024-09-11T23:42:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:40.400+0000 7fa2c7fff640 1 -- 172.21.15.133:0/2512587245 wait complete. 2024-09-11T23:42:41.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:41 smithi138 ceph-mon[35142]: pgmap v181: 97 pgs: 97 active+clean; 50 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 669 MiB/s wr, 1.29k op/s 2024-09-11T23:42:41.972 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:41 smithi133 ceph-mon[31430]: pgmap v181: 97 pgs: 97 active+clean; 50 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 669 MiB/s wr, 1.29k op/s 2024-09-11T23:42:42.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:41 smithi178 ceph-mon[35882]: pgmap v181: 97 pgs: 97 active+clean; 50 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 669 MiB/s wr, 1.29k op/s 2024-09-11T23:42:44.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:44 smithi138 ceph-mon[35142]: pgmap v182: 97 pgs: 97 active+clean; 50 GiB data, 198 GiB used, 874 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 857 MiB/s wr, 1.92k op/s 2024-09-11T23:42:44.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:44 smithi133 ceph-mon[31430]: pgmap v182: 97 pgs: 97 active+clean; 50 GiB data, 198 GiB used, 874 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 857 MiB/s wr, 1.92k op/s 2024-09-11T23:42:45.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:44 smithi178 ceph-mon[35882]: pgmap v182: 97 pgs: 97 active+clean; 50 GiB data, 198 GiB used, 874 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 857 MiB/s wr, 1.92k op/s 2024-09-11T23:42:46.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:45 smithi178 ceph-mon[35882]: pgmap v183: 97 pgs: 97 active+clean; 50 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 714 MiB/s wr, 1.79k op/s 2024-09-11T23:42:46.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:45 smithi138 ceph-mon[35142]: pgmap v183: 97 pgs: 97 active+clean; 50 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 714 MiB/s wr, 1.79k op/s 2024-09-11T23:42:46.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:45 smithi133 ceph-mon[31430]: pgmap v183: 97 pgs: 97 active+clean; 50 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 714 MiB/s wr, 1.79k op/s 2024-09-11T23:42:48.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:48 smithi178 ceph-mon[35882]: pgmap v184: 97 pgs: 97 active+clean; 50 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 654 MiB/s wr, 1.74k op/s 2024-09-11T23:42:48.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:48 smithi138 ceph-mon[35142]: pgmap v184: 97 pgs: 97 active+clean; 50 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 654 MiB/s wr, 1.74k op/s 2024-09-11T23:42:48.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:48 smithi133 ceph-mon[31430]: pgmap v184: 97 pgs: 97 active+clean; 50 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 654 MiB/s wr, 1.74k op/s 2024-09-11T23:42:49.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:49 smithi138 ceph-mon[35142]: pgmap v185: 97 pgs: 97 active+clean; 50 GiB data, 206 GiB used, 866 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 837 MiB/s wr, 1.88k op/s 2024-09-11T23:42:49.961 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:49 smithi133 ceph-mon[31430]: pgmap v185: 97 pgs: 97 active+clean; 50 GiB data, 206 GiB used, 866 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 837 MiB/s wr, 1.88k op/s 2024-09-11T23:42:50.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:49 smithi178 ceph-mon[35882]: pgmap v185: 97 pgs: 97 active+clean; 50 GiB data, 206 GiB used, 866 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 837 MiB/s wr, 1.88k op/s 2024-09-11T23:42:51.202 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-11T23:42:51.203 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:42:52.176 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:52.661 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:52 smithi133 ceph-mon[31430]: pgmap v186: 97 pgs: 97 active+clean; 50 GiB data, 208 GiB used, 865 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 646 MiB/s wr, 1.38k op/s 2024-09-11T23:42:52.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:52 smithi138 ceph-mon[35142]: pgmap v186: 97 pgs: 97 active+clean; 50 GiB data, 208 GiB used, 865 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 646 MiB/s wr, 1.38k op/s 2024-09-11T23:42:53.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:52 smithi178 ceph-mon[35882]: pgmap v186: 97 pgs: 97 active+clean; 50 GiB data, 208 GiB used, 865 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 646 MiB/s wr, 1.38k op/s 2024-09-11T23:42:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.226+0000 7f6dc914e640 1 -- 172.21.15.133:0/1438666391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dc406ea30 msgr2=0x7f6dc406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.227+0000 7f6dc914e640 1 --2- 172.21.15.133:0/1438666391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dc406ea30 0x7f6dc406ee70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009900 tx=0x7f6db802a440 comp rx=0 tx=0).stop 2024-09-11T23:42:53.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.229+0000 7f6dc914e640 1 -- 172.21.15.133:0/1438666391 shutdown_connections 2024-09-11T23:42:53.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.229+0000 7f6dc914e640 1 --2- 172.21.15.133:0/1438666391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc411b140 0x7f6dc411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.229+0000 7f6dc914e640 1 --2- 172.21.15.133:0/1438666391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dc406ea30 0x7f6dc406ee70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.229+0000 7f6dc914e640 1 --2- 172.21.15.133:0/1438666391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dc4110190 0x7f6dc406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.229+0000 7f6dc914e640 1 -- 172.21.15.133:0/1438666391 >> 172.21.15.133:0/1438666391 conn(0x7f6dc4073180 msgr2=0x7f6dc4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.229+0000 7f6dc914e640 1 -- 172.21.15.133:0/1438666391 shutdown_connections 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.229+0000 7f6dc914e640 1 -- 172.21.15.133:0/1438666391 wait complete. 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.230+0000 7f6dc914e640 1 Processor -- start 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.230+0000 7f6dc914e640 1 -- start start 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.231+0000 7f6dc914e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dc406ea30 0x7f6dc41a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.231+0000 7f6dc914e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dc4110190 0x7f6dc41a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.231+0000 7f6dc914e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc411b140 0x7f6dc41ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.231+0000 7f6dc914e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4120be0 con 0x7f6dc4110190 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.231+0000 7f6dc914e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4120a60 con 0x7f6dc406ea30 2024-09-11T23:42:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.231+0000 7f6dc914e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4120d60 con 0x7f6dc411b140 2024-09-11T23:42:53.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.234+0000 7f6dc3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dc406ea30 0x7f6dc41a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.234+0000 7f6dc3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dc406ea30 0x7f6dc41a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33050/0 (socket says 172.21.15.133:33050) 2024-09-11T23:42:53.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.234+0000 7f6dc3fff640 1 -- 172.21.15.133:0/3513078572 learned_addr learned my addr 172.21.15.133:0/3513078572 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:53.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.234+0000 7f6dc37fe640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dc4110190 0x7f6dc41a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.234+0000 7f6dc894d640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc411b140 0x7f6dc41ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.236+0000 7f6dc3fff640 1 -- 172.21.15.133:0/3513078572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc411b140 msgr2=0x7f6dc41ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.236+0000 7f6dc3fff640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc411b140 0x7f6dc41ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.236+0000 7f6dc3fff640 1 -- 172.21.15.133:0/3513078572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dc4110190 msgr2=0x7f6dc41a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.236+0000 7f6dc3fff640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dc4110190 0x7f6dc41a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.236+0000 7f6dc3fff640 1 -- 172.21.15.133:0/3513078572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc41ac050 con 0x7f6dc406ea30 2024-09-11T23:42:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.237+0000 7f6dc3fff640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dc406ea30 0x7f6dc41a7080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e9f0 tx=0x7f6db400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.237+0000 7f6dc17fa640 1 -- 172.21.15.133:0/3513078572 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db400cd80 con 0x7f6dc406ea30 2024-09-11T23:42:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.238+0000 7f6dc914e640 1 -- 172.21.15.133:0/3513078572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc41ac340 con 0x7f6dc406ea30 2024-09-11T23:42:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.238+0000 7f6dc914e640 1 -- 172.21.15.133:0/3513078572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc4110920 con 0x7f6dc406ea30 2024-09-11T23:42:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.240+0000 7f6dc914e640 1 -- 172.21.15.133:0/3513078572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc406ee70 con 0x7f6dc406ea30 2024-09-11T23:42:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.240+0000 7f6dc17fa640 1 -- 172.21.15.133:0/3513078572 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db4010910 con 0x7f6dc406ea30 2024-09-11T23:42:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.241+0000 7f6dc17fa640 1 -- 172.21.15.133:0/3513078572 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4018990 con 0x7f6dc406ea30 2024-09-11T23:42:53.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.247+0000 7f6dc17fa640 1 -- 172.21.15.133:0/3513078572 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6db4010ac0 con 0x7f6dc406ea30 2024-09-11T23:42:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.254+0000 7f6dc17fa640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6dac0a2e30 0x7f6dac0a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.255+0000 7f6dc17fa640 1 -- 172.21.15.133:0/3513078572 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f6db40cff60 con 0x7f6dc406ea30 2024-09-11T23:42:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.255+0000 7f6dc17fa640 1 -- 172.21.15.133:0/3513078572 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6db40d0380 con 0x7f6dc406ea30 2024-09-11T23:42:53.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.270+0000 7f6dc37fe640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6dac0a2e30 0x7f6dac0a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.279+0000 7f6dc37fe640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6dac0a2e30 0x7f6dac0a52f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6db8033040 tx=0x7f6db8005850 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.722+0000 7f55d2653640 1 -- 172.21.15.133:0/2152569250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c40b7570 msgr2=0x7f55c40b9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.722+0000 7f55d2653640 1 --2- 172.21.15.133:0/2152569250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c40b7570 0x7f55c40b9960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009900 tx=0x7f55c002a4b0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.738+0000 7f55d2653640 1 -- 172.21.15.133:0/2152569250 shutdown_connections 2024-09-11T23:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.738+0000 7f55d2653640 1 --2- 172.21.15.133:0/2152569250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c40b7570 0x7f55c40b9960 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.738+0000 7f55d2653640 1 --2- 172.21.15.133:0/2152569250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c40a5170 0x7f55c40b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.738+0000 7f55d2653640 1 --2- 172.21.15.133:0/2152569250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40a4850 0x7f55c40a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.738+0000 7f55d2653640 1 -- 172.21.15.133:0/2152569250 >> 172.21.15.133:0/2152569250 conn(0x7f55c401a140 msgr2=0x7f55c401a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:53.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.738+0000 7f6dc914e640 1 -- 172.21.15.133:0/3513078572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dc41a83e0 con 0x7f6dc406ea30 2024-09-11T23:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.739+0000 7f55d2653640 1 -- 172.21.15.133:0/2152569250 shutdown_connections 2024-09-11T23:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.739+0000 7f55d2653640 1 -- 172.21.15.133:0/2152569250 wait complete. 2024-09-11T23:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.740+0000 7f55d2653640 1 Processor -- start 2024-09-11T23:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.740+0000 7f55d2653640 1 -- start start 2024-09-11T23:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.741+0000 7f55d2653640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c40a4850 0x7f55c4143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.741+0000 7f55d2653640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40a5170 0x7f55c4143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.741+0000 7f55d2653640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c40b7570 0x7f55c4147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.741+0000 7f55d2653640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55c40bd080 con 0x7f55c40a5170 2024-09-11T23:42:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.741+0000 7f55d2653640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55c40bcf00 con 0x7f55c40a4850 2024-09-11T23:42:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.741+0000 7f55d2653640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55c40bd200 con 0x7f55c40b7570 2024-09-11T23:42:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.742+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c40a4850 0x7f55c4143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.742+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c40a4850 0x7f55c4143530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33072/0 (socket says 172.21.15.133:33072) 2024-09-11T23:42:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.742+0000 7f55cbfff640 1 -- 172.21.15.133:0/1437475045 learned_addr learned my addr 172.21.15.133:0/1437475045 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:53.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.742+0000 7f6dc17fa640 1 -- 172.21.15.133:0/3513078572 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7f6db409cb90 con 0x7f6dc406ea30 2024-09-11T23:42:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.746+0000 7f55cb7fe640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40a5170 0x7f55c4143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.746+0000 7f55d0bf0640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c40b7570 0x7f55c4147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.751 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:42:53.751 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:42:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:42:53.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.754+0000 7f6dc914e640 1 -- 172.21.15.133:0/3513078572 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6dac0a2e30 msgr2=0x7f6dac0a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:53.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.754+0000 7f6dc914e640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6dac0a2e30 0x7f6dac0a52f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6db8033040 tx=0x7f6db8005850 comp rx=0 tx=0).stop 2024-09-11T23:42:53.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.754+0000 7f6dc914e640 1 -- 172.21.15.133:0/3513078572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dc406ea30 msgr2=0x7f6dc41a7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:53.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.754+0000 7f6dc914e640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dc406ea30 0x7f6dc41a7080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e9f0 tx=0x7f6db400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.750+0000 7f55cbfff640 1 -- 172.21.15.133:0/1437475045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c40b7570 msgr2=0x7f55c4147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:53.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f6dc914e640 1 -- 172.21.15.133:0/3513078572 shutdown_connections 2024-09-11T23:42:53.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f6dc914e640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6dac0a2e30 0x7f6dac0a52f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f6dc914e640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc411b140 0x7f6dc41ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f6dc914e640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6dc4110190 0x7f6dc41a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f6dc914e640 1 --2- 172.21.15.133:0/3513078572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dc406ea30 0x7f6dc41a7080 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f6dc914e640 1 -- 172.21.15.133:0/3513078572 >> 172.21.15.133:0/3513078572 conn(0x7f6dc4073180 msgr2=0x7f6dc410e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.750+0000 7f55cbfff640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c40b7570 0x7f55c4147df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.750+0000 7f55cbfff640 1 -- 172.21.15.133:0/1437475045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40a5170 msgr2=0x7f55c4143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.766+0000 7f6dc914e640 1 -- 172.21.15.133:0/3513078572 shutdown_connections 2024-09-11T23:42:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:53.769+0000 7f6dc914e640 1 -- 172.21.15.133:0/3513078572 wait complete. 2024-09-11T23:42:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.750+0000 7f55cbfff640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40a5170 0x7f55c4143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.750+0000 7f55cbfff640 1 -- 172.21.15.133:0/1437475045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c4148520 con 0x7f55c40a4850 2024-09-11T23:42:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.751+0000 7f55cbfff640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c40a4850 0x7f55c4143530 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f55b800b4a0 tx=0x7f55b800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.751+0000 7f55c97fa640 1 -- 172.21.15.133:0/1437475045 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8014070 con 0x7f55c40a4850 2024-09-11T23:42:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.751+0000 7f55c97fa640 1 -- 172.21.15.133:0/1437475045 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b8010af0 con 0x7f55c40a4850 2024-09-11T23:42:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.751+0000 7f55d2653640 1 -- 172.21.15.133:0/1437475045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c4148810 con 0x7f55c40a4850 2024-09-11T23:42:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.751+0000 7f55d2653640 1 -- 172.21.15.133:0/1437475045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c400b810 con 0x7f55c40a4850 2024-09-11T23:42:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.756+0000 7f55cb7fe640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40a5170 0x7f55c4143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.756+0000 7f55d0bf0640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c40b7570 0x7f55c4147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.757+0000 7f55d53ee740 1 Processor -- start 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.757+0000 7f55d53ee740 1 -- start start 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.758+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 0x5621d44cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.758+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 0x5621d44c0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.759+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.759+0000 7f55d53ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5621d44212f0 con 0x5621d44cdbd0 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.759+0000 7f55d53ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5621d43c9270 con 0x5621d44ce4f0 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.759+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 0x5621d44c0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.759+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 0x5621d44c0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33080/0 (socket says 172.21.15.133:33080) 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.759+0000 7f55cbfff640 1 -- 172.21.15.133:0/3137685170 learned_addr learned my addr 172.21.15.133:0/3137685170 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.759+0000 7f55d53ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5621d441d2b0 con 0x5621d44c1060 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.760+0000 7f55c97fa640 1 -- 172.21.15.133:0/1437475045 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8017d50 con 0x7f55c40a4850 2024-09-11T23:42:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.761+0000 7f55cb7fe640 1 --2- 172.21.15.133:0/3137685170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f55c97fa640 1 -- 172.21.15.133:0/1437475045 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f55b8017070 con 0x7f55c40a4850 2024-09-11T23:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f55cbfff640 1 -- 172.21.15.133:0/3137685170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 msgr2=0x5621d44c34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f55cbfff640 1 --2- 172.21.15.133:0/3137685170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44c34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f55cbfff640 1 -- 172.21.15.133:0/3137685170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 msgr2=0x5621d44cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f55cbfff640 1 --2- 172.21.15.133:0/3137685170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 0x5621d44cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.762+0000 7f55cbfff640 1 -- 172.21.15.133:0/3137685170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d43c8340 con 0x5621d44ce4f0 2024-09-11T23:42:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.763+0000 7f55c97fa640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55bc0a2e40 0x7f55bc0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.768+0000 7f55c97fa640 1 -- 172.21.15.133:0/1437475045 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f55b80d5380 con 0x7f55c40a4850 2024-09-11T23:42:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.772+0000 7f55cbfff640 1 --2- 172.21.15.133:0/3137685170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 0x5621d44c0b20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55b80d2bc0 tx=0x7f55b8011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eefabe8164112b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.782+0000 7f55b67fc640 1 -- 172.21.15.133:0/3137685170 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b801e070 con 0x5621d44ce4f0 2024-09-11T23:42:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.782+0000 7f55b67fc640 1 -- 172.21.15.133:0/3137685170 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b80a20c0 con 0x5621d44ce4f0 2024-09-11T23:42:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.782+0000 7f55b67fc640 1 -- 172.21.15.133:0/3137685170 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80a1500 con 0x5621d44ce4f0 2024-09-11T23:42:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.783+0000 7f55d0bf0640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55bc0a2e40 0x7f55bc0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.787+0000 7f55d53ee740 1 -- 172.21.15.133:0/3137685170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 msgr2=0x5621d44c0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.787+0000 7f55d53ee740 1 --2- 172.21.15.133:0/3137685170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 0x5621d44c0b20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55b80d2bc0 tx=0x7f55b8011000 comp rx=0 tx=0).stop 2024-09-11T23:42:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.795+0000 7f55d53ee740 1 -- 172.21.15.133:0/3137685170 shutdown_connections 2024-09-11T23:42:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.795+0000 7f55d53ee740 1 --2- 172.21.15.133:0/3137685170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44c34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.795+0000 7f55d53ee740 1 --2- 172.21.15.133:0/3137685170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 0x5621d44c0b20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.795+0000 7f55d53ee740 1 --2- 172.21.15.133:0/3137685170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 0x5621d44cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.795+0000 7f55d53ee740 1 -- 172.21.15.133:0/3137685170 >> 172.21.15.133:0/3137685170 conn(0x5621d44ca240 msgr2=0x5621d44ca620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.795+0000 7f55d53ee740 1 -- 172.21.15.133:0/3137685170 shutdown_connections 2024-09-11T23:42:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.795+0000 7f55d53ee740 1 -- 172.21.15.133:0/3137685170 wait complete. 2024-09-11T23:42:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.795+0000 7f55d53ee740 1 -- start start 2024-09-11T23:42:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.796+0000 7f55d53ee740 1 Processor -- start 2024-09-11T23:42:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.797+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44c0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.797+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44cdbd0 0x5621d44c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.797+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44ce4f0 0x5621d44b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.798+0000 7f55d53ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5621d44c3d10 con 0x5621d44ce4f0 2024-09-11T23:42:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.798+0000 7f55d53ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5621d43aa290 con 0x5621d44cdbd0 2024-09-11T23:42:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.798+0000 7f55d53ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5621d44c3e90 con 0x5621d44c1060 2024-09-11T23:42:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.798+0000 7f55d53ee740 10 client.0 _ll_register_callbacks cb 0x5621d43db140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.798+0000 7f55cb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44cdbd0 0x5621d44c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.798+0000 7f55cb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44cdbd0 0x5621d44c0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33096/0 (socket says 172.21.15.133:33096) 2024-09-11T23:42:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.798+0000 7f55cb7fe640 1 -- 172.21.15.133:0/4077049204 learned_addr learned my addr 172.21.15.133:0/4077049204 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.804+0000 7f55cbfff640 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.804+0000 7f55cb7fe640 1 -- 172.21.15.133:0/4077049204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 msgr2=0x5621d44c0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.804+0000 7f55cb7fe640 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44c0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.804+0000 7f55cb7fe640 1 -- 172.21.15.133:0/4077049204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44ce4f0 msgr2=0x5621d44b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.804+0000 7f55cb7fe640 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44ce4f0 0x5621d44b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.804+0000 7f55cb7fe640 1 -- 172.21.15.133:0/4077049204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d4134210 con 0x5621d44cdbd0 2024-09-11T23:42:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.807+0000 7f55cb7fe640 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44cdbd0 0x5621d44c0a80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f55cc067960 tx=0x7f55cc074730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f897aa41f7face35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.811+0000 7f55af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.811+0000 7f55af7fe640 1 -- 172.21.15.133:0/4077049204 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc076640 con 0x5621d44cdbd0 2024-09-11T23:42:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.811+0000 7f55af7fe640 1 -- 172.21.15.133:0/4077049204 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f55cc0767e0 con 0x5621d44cdbd0 2024-09-11T23:42:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.813+0000 7f55d53ee740 1 -- 172.21.15.133:0/4077049204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b80a20c0 con 0x5621d44cdbd0 2024-09-11T23:42:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.814+0000 7f55af7fe640 1 -- 172.21.15.133:0/4077049204 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f55cc068b70 con 0x5621d44cdbd0 2024-09-11T23:42:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.814+0000 7f55d53ee740 10 client.26507 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.814+0000 7f55d53ee740 1 -- 172.21.15.133:0/4077049204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621d4200620 con 0x5621d44cdbd0 2024-09-11T23:42:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.814+0000 7f55af7fe640 1 -- 172.21.15.133:0/4077049204 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b80a20c0 con 0x5621d44cdbd0 2024-09-11T23:42:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.815+0000 7f55af7fe640 1 -- 172.21.15.133:0/4077049204 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f55cc0694b0 con 0x5621d44cdbd0 2024-09-11T23:42:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.815+0000 7f55d53ee740 10 client.26507 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.815+0000 7f55d53ee740 10 client.26507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.815+0000 7f55d53ee740 20 client.26507 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.815+0000 7f55d53ee740 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5621d44fa550 0x5621d44fc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.815+0000 7f55d53ee740 4 client.26507 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.815+0000 7f55d53ee740 1 -- 172.21.15.133:0/4077049204 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621d442ea20 con 0x5621d44fa550 2024-09-11T23:42:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.815+0000 7f55d0bf0640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55bc0a2e40 0x7f55bc0a5300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f55c0033040 tx=0x7f55c0005d50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.817+0000 7f55cbfff640 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5621d44fa550 0x5621d44fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.822+0000 7f55cbfff640 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5621d44fa550 0x5621d44fc970 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d07c38edd521a5c server_cookie=e3f843ce707a012b in_seq=0 out_seq=0 2024-09-11T23:42:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.822+0000 7f55af7fe640 10 client.26507 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.828+0000 7f55af7fe640 1 -- 172.21.15.133:0/4077049204 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621d442ea20 con 0x5621d44fa550 2024-09-11T23:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.828+0000 7f55af7fe640 10 client.26507 handle_command_reply: tid=0 2024-09-11T23:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.828+0000 7f55d53ee740 1 client.26507 shutdown 2024-09-11T23:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.828+0000 7f55d53ee740 1 -- 172.21.15.133:0/4077049204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44cdbd0 msgr2=0x5621d44c0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.828+0000 7f55d53ee740 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44cdbd0 0x5621d44c0a80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f55cc067960 tx=0x7f55cc074730 comp rx=0 tx=0).stop 2024-09-11T23:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.834+0000 7f55d53ee740 1 -- 172.21.15.133:0/4077049204 shutdown_connections 2024-09-11T23:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.834+0000 7f55d53ee740 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5621d44fa550 0x5621d44fc970 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.834+0000 7f55d53ee740 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44ce4f0 0x5621d44b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.834+0000 7f55d53ee740 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44cdbd0 0x5621d44c0a80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.834+0000 7f55d53ee740 1 --2- 172.21.15.133:0/4077049204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.834+0000 7f55d53ee740 1 -- 172.21.15.133:0/4077049204 >> 172.21.15.133:0/4077049204 conn(0x5621d44ca240 msgr2=0x5621d44cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.834+0000 7f55af7fe640 0 client.26507 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.834+0000 7f55d53ee740 1 -- 172.21.15.133:0/4077049204 shutdown_connections 2024-09-11T23:42:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.834+0000 7f55d53ee740 1 -- 172.21.15.133:0/4077049204 wait complete. 2024-09-11T23:42:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.837+0000 7f55d53ee740 20 client.26507 trim_cache size 0 max 16384 2024-09-11T23:42:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.883+0000 7f55d53ee740 1 Processor -- start 2024-09-11T23:42:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.883+0000 7f55d53ee740 1 -- start start 2024-09-11T23:42:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.884+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.884+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 0x5621d44c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.884+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 0x5621d44c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.884+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.885+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:37478/0 (socket says 172.21.15.133:37478) 2024-09-11T23:42:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.885+0000 7f55cbfff640 1 -- 172.21.15.133:0/796721731 learned_addr learned my addr 172.21.15.133:0/796721731 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.885+0000 7f55d53ee740 1 -- 172.21.15.133:0/796721731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55b000e680 con 0x5621d44cdbd0 2024-09-11T23:42:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.885+0000 7f55d53ee740 1 -- 172.21.15.133:0/796721731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55b00101c0 con 0x5621d44ce4f0 2024-09-11T23:42:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.885+0000 7f55d53ee740 1 -- 172.21.15.133:0/796721731 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55b000ed50 con 0x5621d44c1060 2024-09-11T23:42:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.885+0000 7f55d0bf0640 1 --2- 172.21.15.133:0/796721731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 0x5621d44c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.887+0000 7f55d0bf0640 1 -- 172.21.15.133:0/796721731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 msgr2=0x5621d44b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.887+0000 7f55d0bf0640 1 --2- 172.21.15.133:0/796721731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.887+0000 7f55d0bf0640 1 -- 172.21.15.133:0/796721731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 msgr2=0x5621d44c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.887+0000 7f55d0bf0640 1 --2- 172.21.15.133:0/796721731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 0x5621d44c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.887+0000 7f55d0bf0640 1 -- 172.21.15.133:0/796721731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d44ce320 con 0x5621d44ce4f0 2024-09-11T23:42:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.887+0000 7f55d0bf0640 1 --2- 172.21.15.133:0/796721731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 0x5621d44c60a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009900 tx=0x7f55c0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=571d856549ccdfcc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.889+0000 7f55cbfff640 1 --2- 172.21.15.133:0/796721731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:42:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.891+0000 7f55b5ffb640 1 -- 172.21.15.133:0/796721731 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c004d070 con 0x5621d44ce4f0 2024-09-11T23:42:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.892+0000 7f55b5ffb640 1 -- 172.21.15.133:0/796721731 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c00054e0 con 0x5621d44ce4f0 2024-09-11T23:42:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.892+0000 7f55b5ffb640 1 -- 172.21.15.133:0/796721731 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0049e00 con 0x5621d44ce4f0 2024-09-11T23:42:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.894+0000 7f55d53ee740 1 -- 172.21.15.133:0/796721731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 msgr2=0x5621d44c60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.894+0000 7f55d53ee740 1 --2- 172.21.15.133:0/796721731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 0x5621d44c60a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009900 tx=0x7f55c0005fd0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.903+0000 7f55d53ee740 1 -- 172.21.15.133:0/796721731 shutdown_connections 2024-09-11T23:42:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.903+0000 7f55d53ee740 1 --2- 172.21.15.133:0/796721731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44ce4f0 0x5621d44c60a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.903+0000 7f55d53ee740 1 --2- 172.21.15.133:0/796721731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 0x5621d44c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.903+0000 7f55d53ee740 1 --2- 172.21.15.133:0/796721731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44c1060 0x5621d44b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.903+0000 7f55d53ee740 1 -- 172.21.15.133:0/796721731 >> 172.21.15.133:0/796721731 conn(0x5621d44ca240 msgr2=0x5621d44c9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.905+0000 7f55d53ee740 1 -- 172.21.15.133:0/796721731 shutdown_connections 2024-09-11T23:42:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.905+0000 7f55d53ee740 1 -- 172.21.15.133:0/796721731 wait complete. 2024-09-11T23:42:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.906+0000 7f55d53ee740 1 -- start start 2024-09-11T23:42:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.907+0000 7f55d53ee740 1 Processor -- start 2024-09-11T23:42:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.908+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44c1060 0x5621d450b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.909+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 0x5621d450b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.910+0000 7f55d53ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44ce4f0 0x5621d4563090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.910+0000 7f55d53ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55b000faf0 con 0x5621d44cdbd0 2024-09-11T23:42:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.910+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44c1060 0x5621d450b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.910+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44c1060 0x5621d450b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33114/0 (socket says 172.21.15.133:33114) 2024-09-11T23:42:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.910+0000 7f55cbfff640 1 -- 172.21.15.133:0/969785789 learned_addr learned my addr 172.21.15.133:0/969785789 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.910+0000 7f55d0bf0640 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44ce4f0 0x5621d4563090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.911+0000 7f55d53ee740 1 -- 172.21.15.133:0/969785789 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55b000f420 con 0x5621d44c1060 2024-09-11T23:42:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.911+0000 7f55d53ee740 1 -- 172.21.15.133:0/969785789 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55b000c660 con 0x5621d44ce4f0 2024-09-11T23:42:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.911+0000 7f55cb7fe640 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 0x5621d450b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.911+0000 7f55d53ee740 10 client.0 _ll_register_callbacks cb 0x7f55cc068670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.912+0000 7f55cbfff640 1 -- 172.21.15.133:0/969785789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44ce4f0 msgr2=0x5621d4563090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.912+0000 7f55cbfff640 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44ce4f0 0x5621d4563090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.912+0000 7f55cbfff640 1 -- 172.21.15.133:0/969785789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 msgr2=0x5621d450b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.912+0000 7f55cbfff640 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 0x5621d450b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.912+0000 7f55cbfff640 1 -- 172.21.15.133:0/969785789 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d45635d0 con 0x5621d44c1060 2024-09-11T23:42:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.913+0000 7f55cbfff640 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44c1060 0x5621d450b3e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f55b80aa100 tx=0x7f55b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d05ef168d0fde994 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.913+0000 7f55af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.914+0000 7f55af7fe640 1 -- 172.21.15.133:0/969785789 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b801e040 con 0x5621d44c1060 2024-09-11T23:42:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.914+0000 7f55af7fe640 1 -- 172.21.15.133:0/969785789 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f55b80173f0 con 0x5621d44c1060 2024-09-11T23:42:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.915+0000 7f55d53ee740 1 -- 172.21.15.133:0/969785789 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c00054e0 con 0x5621d44c1060 2024-09-11T23:42:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.915+0000 7f55af7fe640 1 -- 172.21.15.133:0/969785789 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f55b809fb00 con 0x5621d44c1060 2024-09-11T23:42:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.916+0000 7f55d53ee740 10 client.26513 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.916+0000 7f55d53ee740 1 -- 172.21.15.133:0/969785789 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621d45647e0 con 0x5621d44c1060 2024-09-11T23:42:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.921+0000 7f55af7fe640 1 -- 172.21.15.133:0/969785789 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b80a20c0 con 0x5621d44c1060 2024-09-11T23:42:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.921+0000 7f55af7fe640 1 -- 172.21.15.133:0/969785789 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f55b808c0b0 con 0x5621d44c1060 2024-09-11T23:42:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.922+0000 7f55d53ee740 10 client.26513 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.922+0000 7f55d53ee740 10 client.26513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:42:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.922+0000 7f55d53ee740 20 client.26513 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.922+0000 7f55d53ee740 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5621d4564b20 0x5621d4566f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.923+0000 7f55d53ee740 4 client.26513 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:42:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.923+0000 7f55d53ee740 1 -- 172.21.15.133:0/969785789 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f55c00054e0 con 0x5621d4564b20 2024-09-11T23:42:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.923+0000 7f55cb7fe640 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5621d4564b20 0x5621d4566f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.927+0000 7f55cb7fe640 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5621d4564b20 0x5621d4566f10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63ea58d64d34de97 server_cookie=f3d142560f72fb17 in_seq=0 out_seq=0 2024-09-11T23:42:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:53.929+0000 7f55af7fe640 10 client.26513 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:53.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:53 smithi138 ceph-mon[35142]: pgmap v187: 97 pgs: 97 active+clean; 50 GiB data, 213 GiB used, 859 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 866 MiB/s wr, 1.72k op/s 2024-09-11T23:42:53.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:53 smithi133 ceph-mon[31430]: pgmap v187: 97 pgs: 97 active+clean; 50 GiB data, 213 GiB used, 859 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 866 MiB/s wr, 1.72k op/s 2024-09-11T23:42:54.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:53 smithi178 ceph-mon[35882]: pgmap v187: 97 pgs: 97 active+clean; 50 GiB data, 213 GiB used, 859 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 866 MiB/s wr, 1.72k op/s 2024-09-11T23:42:54.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:54 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3513078572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:42:54.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:54 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3513078572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:42:55.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:54 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3513078572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:42:55.068 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-11T23:42:55.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-11T23:42:55.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:55 smithi133 ceph-mon[31430]: pgmap v188: 97 pgs: 97 active+clean; 50 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 723 MiB/s wr, 1.14k op/s 2024-09-11T23:42:56.011 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:42:56.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:55 smithi178 ceph-mon[35882]: pgmap v188: 97 pgs: 97 active+clean; 50 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 723 MiB/s wr, 1.14k op/s 2024-09-11T23:42:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.125+0000 7f55af7fe640 1 -- 172.21.15.133:0/969785789 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f55c00054e0 con 0x5621d4564b20 2024-09-11T23:42:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.125+0000 7f55af7fe640 10 client.26513 handle_command_reply: tid=0 2024-09-11T23:42:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.131+0000 7f55d53ee740 1 client.26513 shutdown 2024-09-11T23:42:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.133+0000 7f55d53ee740 1 -- 172.21.15.133:0/969785789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44c1060 msgr2=0x5621d450b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.133+0000 7f55d53ee740 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44c1060 0x5621d450b3e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f55b80aa100 tx=0x7f55b80046c0 comp rx=0 tx=0).stop 2024-09-11T23:42:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.133+0000 7f55d53ee740 1 -- 172.21.15.133:0/969785789 shutdown_connections 2024-09-11T23:42:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.133+0000 7f55d53ee740 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5621d4564b20 0x5621d4566f10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.133+0000 7f55d53ee740 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5621d44ce4f0 0x5621d4563090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.133+0000 7f55d53ee740 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d44cdbd0 0x5621d450b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.133+0000 7f55d53ee740 1 --2- 172.21.15.133:0/969785789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621d44c1060 0x5621d450b3e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.133+0000 7f55d53ee740 1 -- 172.21.15.133:0/969785789 >> 172.21.15.133:0/969785789 conn(0x5621d44ca240 msgr2=0x5621d44c0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.133+0000 7f55af7fe640 0 client.26513 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.133+0000 7f55d53ee740 1 -- 172.21.15.133:0/969785789 shutdown_connections 2024-09-11T23:42:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.133+0000 7f55d53ee740 1 -- 172.21.15.133:0/969785789 wait complete. 2024-09-11T23:42:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.134+0000 7f55d53ee740 20 client.26513 trim_cache size 0 max 16384 2024-09-11T23:42:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:42:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:42:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:42:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:42:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.139+0000 7f55d2653640 1 -- 172.21.15.133:0/1437475045 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55bc0a2e40 msgr2=0x7f55bc0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.139+0000 7f55d2653640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55bc0a2e40 0x7f55bc0a5300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f55c0033040 tx=0x7f55c0005d50 comp rx=0 tx=0).stop 2024-09-11T23:42:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.139+0000 7f55d2653640 1 -- 172.21.15.133:0/1437475045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c40a4850 msgr2=0x7f55c4143530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.139+0000 7f55d2653640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c40a4850 0x7f55c4143530 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f55b800b4a0 tx=0x7f55b800b960 comp rx=0 tx=0).stop 2024-09-11T23:42:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.141+0000 7f55d2653640 1 -- 172.21.15.133:0/1437475045 shutdown_connections 2024-09-11T23:42:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.141+0000 7f55d2653640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f55bc0a2e40 0x7f55bc0a5300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.141+0000 7f55d2653640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55c40b7570 0x7f55c4147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.141+0000 7f55d2653640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40a5170 0x7f55c4143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.141+0000 7f55d2653640 1 --2- 172.21.15.133:0/1437475045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c40a4850 0x7f55c4143530 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.141+0000 7f55d2653640 1 -- 172.21.15.133:0/1437475045 >> 172.21.15.133:0/1437475045 conn(0x7f55c401a140 msgr2=0x7f55c40b5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.145+0000 7f55d2653640 1 -- 172.21.15.133:0/1437475045 shutdown_connections 2024-09-11T23:42:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:42:56.145+0000 7f55d2653640 1 -- 172.21.15.133:0/1437475045 wait complete. 2024-09-11T23:42:56.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:55 smithi138 ceph-mon[35142]: pgmap v188: 97 pgs: 97 active+clean; 50 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 723 MiB/s wr, 1.14k op/s 2024-09-11T23:42:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.065+0000 7f7d5d306640 1 -- 172.21.15.133:0/2044584377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d58109b10 msgr2=0x7f7d58114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.065+0000 7f7d5d306640 1 --2- 172.21.15.133:0/2044584377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d58109b10 0x7f7d58114570 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009860 tx=0x7f7d4802a360 comp rx=0 tx=0).stop 2024-09-11T23:42:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.066+0000 7f7d5d306640 1 -- 172.21.15.133:0/2044584377 shutdown_connections 2024-09-11T23:42:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.066+0000 7f7d5d306640 1 --2- 172.21.15.133:0/2044584377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d58114ab0 0x7f7d58116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.066+0000 7f7d5d306640 1 --2- 172.21.15.133:0/2044584377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d58109b10 0x7f7d58114570 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.066+0000 7f7d5d306640 1 --2- 172.21.15.133:0/2044584377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d581091f0 0x7f7d581095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.066+0000 7f7d5d306640 1 -- 172.21.15.133:0/2044584377 >> 172.21.15.133:0/2044584377 conn(0x7f7d58073660 msgr2=0x7f7d58073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.071+0000 7f7d5d306640 1 -- 172.21.15.133:0/2044584377 shutdown_connections 2024-09-11T23:42:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.078+0000 7f7d5d306640 1 -- 172.21.15.133:0/2044584377 wait complete. 2024-09-11T23:42:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.079+0000 7f7d5d306640 1 Processor -- start 2024-09-11T23:42:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.079+0000 7f7d5d306640 1 -- start start 2024-09-11T23:42:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.079+0000 7f7d5d306640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d581091f0 0x7f7d581117e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.080+0000 7f7d5d306640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d58109b10 0x7f7d58111d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.080+0000 7f7d5d306640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d58114ab0 0x7f7d58112260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.080+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d581091f0 0x7f7d581117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.080+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d581091f0 0x7f7d581117e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:37510/0 (socket says 172.21.15.133:37510) 2024-09-11T23:42:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.080+0000 7f7d57fff640 1 -- 172.21.15.133:0/2919593730 learned_addr learned my addr 172.21.15.133:0/2919593730 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.080+0000 7f7d5cb05640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d58114ab0 0x7f7d58112260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.080+0000 7f7d5d306640 1 -- 172.21.15.133:0/2919593730 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5810bfe0 con 0x7f7d58109b10 2024-09-11T23:42:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.080+0000 7f7d5d306640 1 -- 172.21.15.133:0/2919593730 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5810be60 con 0x7f7d58114ab0 2024-09-11T23:42:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.080+0000 7f7d5d306640 1 -- 172.21.15.133:0/2919593730 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5810c160 con 0x7f7d581091f0 2024-09-11T23:42:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.081+0000 7f7d577fe640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d58109b10 0x7f7d58111d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.081+0000 7f7d577fe640 1 -- 172.21.15.133:0/2919593730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d581091f0 msgr2=0x7f7d581117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:57.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.081+0000 7f7d577fe640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d581091f0 0x7f7d581117e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.082+0000 7f7d577fe640 1 -- 172.21.15.133:0/2919593730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d58114ab0 msgr2=0x7f7d58112260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:57.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.082+0000 7f7d577fe640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d58114ab0 0x7f7d58112260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.082+0000 7f7d577fe640 1 -- 172.21.15.133:0/2919593730 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d581b33e0 con 0x7f7d58109b10 2024-09-11T23:42:57.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.082+0000 7f7d577fe640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d58109b10 0x7f7d58111d20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7d4802a880 tx=0x7f7d48033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.083+0000 7f7d557fa640 1 -- 172.21.15.133:0/2919593730 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48037040 con 0x7f7d58109b10 2024-09-11T23:42:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.084+0000 7f7d557fa640 1 -- 172.21.15.133:0/2919593730 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d48042070 con 0x7f7d58109b10 2024-09-11T23:42:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.084+0000 7f7d5d306640 1 -- 172.21.15.133:0/2919593730 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5811cd60 con 0x7f7d58109b10 2024-09-11T23:42:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.084+0000 7f7d5d306640 1 -- 172.21.15.133:0/2919593730 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5811d210 con 0x7f7d58109b10 2024-09-11T23:42:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.084+0000 7f7d557fa640 1 -- 172.21.15.133:0/2919593730 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480038c0 con 0x7f7d58109b10 2024-09-11T23:42:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.095+0000 7f7d557fa640 1 -- 172.21.15.133:0/2919593730 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7d48003a60 con 0x7f7d58109b10 2024-09-11T23:42:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.096+0000 7f7d557fa640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7d400a2ec0 0x7f7d400a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.097+0000 7f7d557fa640 1 -- 172.21.15.133:0/2919593730 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f7d480ef3d0 con 0x7f7d58109b10 2024-09-11T23:42:57.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.098+0000 7f7d57fff640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7d400a2ec0 0x7f7d400a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.100+0000 7f7d57fff640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7d400a2ec0 0x7f7d400a5380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c005fd0 tx=0x7f7d4c00b480 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.108+0000 7f7d60941740 1 Processor -- start 2024-09-11T23:42:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.110+0000 7f7d60941740 1 -- start start 2024-09-11T23:42:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.110+0000 7f7d60941740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a278ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.110+0000 7f7d60941740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a278f220 0x55d6a27995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.110+0000 7f7d60941740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a2799ae0 0x55d6a279bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.110+0000 7f7d60941740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d6a26f42a0 con 0x55d6a278f220 2024-09-11T23:42:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.110+0000 7f7d60941740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d6a269bb10 con 0x55d6a278e900 2024-09-11T23:42:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.110+0000 7f7d60941740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d6a26f0260 con 0x55d6a2799ae0 2024-09-11T23:42:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.111+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a278ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.111+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a278ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33140/0 (socket says 172.21.15.133:33140) 2024-09-11T23:42:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.111+0000 7f7d57fff640 1 -- 172.21.15.133:0/4097038695 learned_addr learned my addr 172.21.15.133:0/4097038695 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.111+0000 7f7d5cb05640 1 --2- 172.21.15.133:0/4097038695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a2799ae0 0x55d6a279bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.112+0000 7f7d57fff640 1 -- 172.21.15.133:0/4097038695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a2799ae0 msgr2=0x55d6a279bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.112+0000 7f7d57fff640 1 --2- 172.21.15.133:0/4097038695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a2799ae0 0x55d6a279bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.112+0000 7f7d57fff640 1 -- 172.21.15.133:0/4097038695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a278f220 msgr2=0x55d6a27995a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:42:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.112+0000 7f7d57fff640 1 --2- 172.21.15.133:0/4097038695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a278f220 0x55d6a27995a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.112+0000 7f7d57fff640 1 -- 172.21.15.133:0/4097038695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6a2465b90 con 0x55d6a278e900 2024-09-11T23:42:57.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.117+0000 7f7d57fff640 1 --2- 172.21.15.133:0/4097038695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a278ece0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c011010 tx=0x7f7d4c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fc1c4a5f5a584dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.117+0000 7f7d3e7fc640 1 -- 172.21.15.133:0/4097038695 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c005220 con 0x55d6a278e900 2024-09-11T23:42:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.117+0000 7f7d3e7fc640 1 -- 172.21.15.133:0/4097038695 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c004c80 con 0x55d6a278e900 2024-09-11T23:42:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.119+0000 7f7d60941740 1 -- 172.21.15.133:0/4097038695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 msgr2=0x55d6a278ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.119+0000 7f7d60941740 1 --2- 172.21.15.133:0/4097038695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a278ece0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c011010 tx=0x7f7d4c013000 comp rx=0 tx=0).stop 2024-09-11T23:42:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.119+0000 7f7d60941740 1 -- 172.21.15.133:0/4097038695 shutdown_connections 2024-09-11T23:42:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.119+0000 7f7d60941740 1 --2- 172.21.15.133:0/4097038695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a2799ae0 0x55d6a279bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.120+0000 7f7d60941740 1 --2- 172.21.15.133:0/4097038695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a278f220 0x55d6a27995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.120+0000 7f7d60941740 1 --2- 172.21.15.133:0/4097038695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a278ece0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.120+0000 7f7d60941740 1 -- 172.21.15.133:0/4097038695 >> 172.21.15.133:0/4097038695 conn(0x55d6a278ae90 msgr2=0x55d6a278b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.121+0000 7f7d60941740 1 -- 172.21.15.133:0/4097038695 shutdown_connections 2024-09-11T23:42:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.123+0000 7f7d60941740 1 -- 172.21.15.133:0/4097038695 wait complete. 2024-09-11T23:42:57.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.124+0000 7f7d60941740 1 -- start start 2024-09-11T23:42:57.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.125+0000 7f7d60941740 1 Processor -- start 2024-09-11T23:42:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.126+0000 7f7d60941740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a27969d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.126+0000 7f7d60941740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278f220 0x55d6a2796f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.126+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a27969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.126+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a27969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33156/0 (socket says 172.21.15.133:33156) 2024-09-11T23:42:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.127+0000 7f7d60941740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a2799ae0 0x55d6a2797450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.127+0000 7f7d57fff640 1 -- 172.21.15.133:0/2343566764 learned_addr learned my addr 172.21.15.133:0/2343566764 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.127+0000 7f7d60941740 1 -- 172.21.15.133:0/2343566764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d6a279c700 con 0x55d6a2799ae0 2024-09-11T23:42:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.127+0000 7f7d60941740 1 -- 172.21.15.133:0/2343566764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d6a267c6e0 con 0x55d6a278e900 2024-09-11T23:42:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.127+0000 7f7d60941740 1 -- 172.21.15.133:0/2343566764 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d6a279c880 con 0x55d6a278f220 2024-09-11T23:42:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.127+0000 7f7d60941740 10 client.0 _ll_register_callbacks cb 0x55d6a26a0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.128+0000 7f7d57fff640 1 -- 172.21.15.133:0/2343566764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278f220 msgr2=0x55d6a2796f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:42:57.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.128+0000 7f7d57fff640 1 --2- 172.21.15.133:0/2343566764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278f220 0x55d6a2796f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.128+0000 7f7d57fff640 1 -- 172.21.15.133:0/2343566764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a2799ae0 msgr2=0x55d6a2797450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:57.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.128+0000 7f7d57fff640 1 --2- 172.21.15.133:0/2343566764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a2799ae0 0x55d6a2797450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.128+0000 7f7d57fff640 1 -- 172.21.15.133:0/2343566764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6a27a5ca0 con 0x55d6a278e900 2024-09-11T23:42:57.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.134+0000 7f7d57fff640 1 --2- 172.21.15.133:0/2343566764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a27969d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c01fea0 tx=0x7f7d4c01fed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3447fbd5f2daa5a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:57.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.134+0000 7f7d377fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:42:57.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.136+0000 7f7d377fe640 1 -- 172.21.15.133:0/2343566764 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c004e30 con 0x55d6a278e900 2024-09-11T23:42:57.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.136+0000 7f7d377fe640 1 -- 172.21.15.133:0/2343566764 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f7d4c004fd0 con 0x55d6a278e900 2024-09-11T23:42:57.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.136+0000 7f7d60941740 1 -- 172.21.15.133:0/2343566764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d4c004c80 con 0x55d6a278e900 2024-09-11T23:42:57.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.137+0000 7f7d377fe640 1 -- 172.21.15.133:0/2343566764 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d4c0051d0 con 0x55d6a278e900 2024-09-11T23:42:57.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.140+0000 7f7d60941740 10 client.26525 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:57.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.140+0000 7f7d60941740 1 -- 172.21.15.133:0/2343566764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d6a27a86c0 con 0x55d6a278e900 2024-09-11T23:42:57.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.143+0000 7f7d377fe640 1 -- 172.21.15.133:0/2343566764 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c005370 con 0x55d6a278e900 2024-09-11T23:42:57.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.143+0000 7f7d377fe640 1 -- 172.21.15.133:0/2343566764 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7d4c03c420 con 0x55d6a278e900 2024-09-11T23:42:57.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.143+0000 7f7d60941740 10 client.26525 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:57.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.143+0000 7f7d60941740 10 client.26525 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:42:57.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.143+0000 7f7d60941740 20 client.26525 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:57.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.143+0000 7f7d60941740 1 --2- 172.21.15.133:0/2343566764 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6a27a9b80 0x55d6a27abf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.143+0000 7f7d60941740 4 client.26525 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:42:57.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.143+0000 7f7d60941740 1 -- 172.21.15.133:0/2343566764 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7d4c004c80 con 0x55d6a27a9b80 2024-09-11T23:42:57.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.145+0000 7f7d577fe640 1 --2- 172.21.15.133:0/2343566764 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6a27a9b80 0x55d6a27abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.148+0000 7f7d577fe640 1 --2- 172.21.15.133:0/2343566764 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6a27a9b80 0x55d6a27abf70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b45e88848348597 server_cookie=3c9a09bf57849151 in_seq=0 out_seq=0 2024-09-11T23:42:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.148+0000 7f7d377fe640 10 client.26525 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.150+0000 7f7d377fe640 1 -- 172.21.15.133:0/2343566764 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f7d4c004c80 con 0x55d6a27a9b80 2024-09-11T23:42:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.150+0000 7f7d377fe640 10 client.26525 handle_command_reply: tid=0 2024-09-11T23:42:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.150+0000 7f7d60941740 1 client.26525 shutdown 2024-09-11T23:42:57.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.153+0000 7f7d60941740 1 -- 172.21.15.133:0/2343566764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 msgr2=0x55d6a27969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:57.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.153+0000 7f7d60941740 1 --2- 172.21.15.133:0/2343566764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a27969d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c01fea0 tx=0x7f7d4c01fed0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.154+0000 7f7d60941740 1 -- 172.21.15.133:0/2343566764 shutdown_connections 2024-09-11T23:42:57.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.154+0000 7f7d60941740 1 --2- 172.21.15.133:0/2343566764 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6a27a9b80 0x55d6a27abf70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.154+0000 7f7d60941740 1 --2- 172.21.15.133:0/2343566764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a2799ae0 0x55d6a2797450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.154+0000 7f7d60941740 1 --2- 172.21.15.133:0/2343566764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278f220 0x55d6a2796f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.154+0000 7f7d60941740 1 --2- 172.21.15.133:0/2343566764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278e900 0x55d6a27969d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.154+0000 7f7d60941740 1 -- 172.21.15.133:0/2343566764 >> 172.21.15.133:0/2343566764 conn(0x55d6a278ae90 msgr2=0x55d6a2790bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:57.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.154+0000 7f7d377fe640 0 client.26525 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:57.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.155+0000 7f7d60941740 1 -- 172.21.15.133:0/2343566764 shutdown_connections 2024-09-11T23:42:57.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.155+0000 7f7d60941740 1 -- 172.21.15.133:0/2343566764 wait complete. 2024-09-11T23:42:57.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.156+0000 7f7d60941740 20 client.26525 trim_cache size 0 max 16384 2024-09-11T23:42:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.187+0000 7f7d60941740 1 Processor -- start 2024-09-11T23:42:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.187+0000 7f7d60941740 1 -- start start 2024-09-11T23:42:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.187+0000 7f7d60941740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a2789e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.187+0000 7f7d60941740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278f220 0x55d6a27913b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.187+0000 7f7d60941740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a2799ae0 0x55d6a27952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.187+0000 7f7d60941740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581b3af0 con 0x55d6a2799ae0 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.187+0000 7f7d60941740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581b5630 con 0x55d6a278f220 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.187+0000 7f7d60941740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581b41c0 con 0x55d6a278e900 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278f220 0x55d6a27913b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a2789e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a2789e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:37534/0 (socket says 172.21.15.133:37534) 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278f220 0x55d6a27913b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33164/0 (socket says 172.21.15.133:33164) 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d57fff640 1 -- 172.21.15.133:0/3239595364 learned_addr learned my addr 172.21.15.133:0/3239595364 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d5cb05640 1 --2- 172.21.15.133:0/3239595364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a2799ae0 0x55d6a27952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d57fff640 1 -- 172.21.15.133:0/3239595364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278f220 msgr2=0x55d6a27913b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d57fff640 1 --2- 172.21.15.133:0/3239595364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278f220 0x55d6a27913b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d57fff640 1 -- 172.21.15.133:0/3239595364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a2799ae0 msgr2=0x55d6a27952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d57fff640 1 --2- 172.21.15.133:0/3239595364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a2799ae0 0x55d6a27952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d57fff640 1 -- 172.21.15.133:0/3239595364 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6a279c590 con 0x55d6a278e900 2024-09-11T23:42:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d5cb05640 1 --2- 172.21.15.133:0/3239595364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a2799ae0 0x55d6a27952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:42:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d577fe640 1 --2- 172.21.15.133:0/3239595364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278f220 0x55d6a27913b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:42:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.188+0000 7f7d57fff640 1 --2- 172.21.15.133:0/3239595364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a2789e50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c03c9b0 tx=0x7f7d4c01e660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=493edbdcd6aeb4f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.189+0000 7f7d3dffb640 1 -- 172.21.15.133:0/3239595364 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c02a040 con 0x55d6a278e900 2024-09-11T23:42:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.189+0000 7f7d3dffb640 1 -- 172.21.15.133:0/3239595364 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c03c630 con 0x55d6a278e900 2024-09-11T23:42:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.189+0000 7f7d3dffb640 1 -- 172.21.15.133:0/3239595364 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c01ebb0 con 0x55d6a278e900 2024-09-11T23:42:57.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.190+0000 7f7d60941740 1 -- 172.21.15.133:0/3239595364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 msgr2=0x55d6a2789e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:57.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.190+0000 7f7d60941740 1 --2- 172.21.15.133:0/3239595364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a2789e50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c03c9b0 tx=0x7f7d4c01e660 comp rx=0 tx=0).stop 2024-09-11T23:42:57.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.190+0000 7f7d60941740 1 -- 172.21.15.133:0/3239595364 shutdown_connections 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.190+0000 7f7d60941740 1 --2- 172.21.15.133:0/3239595364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a2799ae0 0x55d6a27952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.190+0000 7f7d60941740 1 --2- 172.21.15.133:0/3239595364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a278f220 0x55d6a27913b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.190+0000 7f7d60941740 1 --2- 172.21.15.133:0/3239595364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a2789e50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.190+0000 7f7d60941740 1 -- 172.21.15.133:0/3239595364 >> 172.21.15.133:0/3239595364 conn(0x55d6a278ae90 msgr2=0x55d6a27971b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.190+0000 7f7d60941740 1 -- 172.21.15.133:0/3239595364 shutdown_connections 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.190+0000 7f7d60941740 1 -- 172.21.15.133:0/3239595364 wait complete. 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.191+0000 7f7d60941740 1 -- start start 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.191+0000 7f7d60941740 1 Processor -- start 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.191+0000 7f7d60941740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a27a2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d60941740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a278f220 0x55d6a27938c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d60941740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a2799ae0 0x55d6a2794050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a27a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a27a2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:37550/0 (socket says 172.21.15.133:37550) 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d57fff640 1 -- 172.21.15.133:0/2616239472 learned_addr learned my addr 172.21.15.133:0/2616239472 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d577fe640 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a278f220 0x55d6a27938c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d60941740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581b4f60 con 0x55d6a278f220 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d60941740 1 -- 172.21.15.133:0/2616239472 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581b4890 con 0x55d6a2799ae0 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d60941740 1 -- 172.21.15.133:0/2616239472 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58112d10 con 0x55d6a278e900 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d5cb05640 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a2799ae0 0x55d6a2794050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d60941740 10 client.0 _ll_register_callbacks cb 0x7f7d4c00be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d57fff640 1 -- 172.21.15.133:0/2616239472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a2799ae0 msgr2=0x55d6a2794050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d57fff640 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a2799ae0 0x55d6a2794050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d57fff640 1 -- 172.21.15.133:0/2616239472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a278f220 msgr2=0x55d6a27938c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d57fff640 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a278f220 0x55d6a27938c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d57fff640 1 -- 172.21.15.133:0/2616239472 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6a28136b0 con 0x55d6a278e900 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.192+0000 7f7d57fff640 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a27a2440 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c011510 tx=0x7f7d4c010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0dac523b2217d12 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.193+0000 7f7d377fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:42:57.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.193+0000 7f7d377fe640 1 -- 172.21.15.133:0/2616239472 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c02a040 con 0x55d6a278e900 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.193+0000 7f7d60941740 1 -- 172.21.15.133:0/2616239472 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d6a27019d0 con 0x55d6a278e900 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.193+0000 7f7d377fe640 1 -- 172.21.15.133:0/2616239472 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f7d4c004af0 con 0x55d6a278e900 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.193+0000 7f7d377fe640 1 -- 172.21.15.133:0/2616239472 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d4c056050 con 0x55d6a278e900 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.193+0000 7f7d60941740 10 client.26467 fetch_fsmap learned FSMap version 30 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.193+0000 7f7d60941740 1 -- 172.21.15.133:0/2616239472 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d6a2814830 con 0x55d6a278e900 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.193+0000 7f7d377fe640 1 -- 172.21.15.133:0/2616239472 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d6a27019d0 con 0x55d6a278e900 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.193+0000 7f7d377fe640 1 -- 172.21.15.133:0/2616239472 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7d4c032930 con 0x55d6a278e900 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.194+0000 7f7d60941740 10 client.26467 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.194+0000 7f7d60941740 10 client.26467 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.194+0000 7f7d60941740 20 client.26467 populate_metadata read hostname 'smithi133' 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.194+0000 7f7d60941740 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6a2814ea0 0x55d6a28172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:42:57.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.194+0000 7f7d60941740 4 client.26467 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-11T23:42:57.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.194+0000 7f7d60941740 1 -- 172.21.15.133:0/2616239472 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x55d6a26fbd20 con 0x55d6a2814ea0 2024-09-11T23:42:57.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.194+0000 7f7d577fe640 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6a2814ea0 0x55d6a28172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:42:57.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.194+0000 7f7d377fe640 10 client.26467 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:57.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.194+0000 7f7d577fe640 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6a2814ea0 0x55d6a28172c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1f0122b48da7c7 server_cookie=f3859785412ae905 in_seq=0 out_seq=0 2024-09-11T23:42:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.482+0000 7f7d377fe640 1 -- 172.21.15.133:0/2616239472 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d6a26fbd20 con 0x55d6a2814ea0 2024-09-11T23:42:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.482+0000 7f7d377fe640 10 client.26467 handle_command_reply: tid=0 2024-09-11T23:42:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.482+0000 7f7d60941740 1 client.26467 shutdown 2024-09-11T23:42:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d60941740 1 -- 172.21.15.133:0/2616239472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 msgr2=0x55d6a27a2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d60941740 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a27a2440 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c011510 tx=0x7f7d4c010fd0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d60941740 1 -- 172.21.15.133:0/2616239472 shutdown_connections 2024-09-11T23:42:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d60941740 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d6a2814ea0 0x55d6a28172c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d60941740 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6a2799ae0 0x55d6a2794050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d60941740 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6a278f220 0x55d6a27938c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d377fe640 0 client.26467 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:42:57.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d60941740 1 --2- 172.21.15.133:0/2616239472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6a278e900 0x55d6a27a2440 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d60941740 1 -- 172.21.15.133:0/2616239472 >> 172.21.15.133:0/2616239472 conn(0x55d6a278ae90 msgr2=0x55d6a27a5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:57.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d60941740 1 -- 172.21.15.133:0/2616239472 shutdown_connections 2024-09-11T23:42:57.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d60941740 1 -- 172.21.15.133:0/2616239472 wait complete. 2024-09-11T23:42:57.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.483+0000 7f7d60941740 20 client.26467 trim_cache size 0 max 16384 2024-09-11T23:42:57.485 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:42:57.485 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 22, 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "e33d2d6a": { 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 22, 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 140.1, 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:42:57.486 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:42:57.487 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:42:57.487 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:42:57.487 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:42:57.487 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:42:57.487 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:42:57.487 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:42:57.487 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:42:57.487 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:42:57.487 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:42:57.487 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:42:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 -- 172.21.15.133:0/2919593730 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7d400a2ec0 msgr2=0x7f7d400a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7d400a2ec0 0x7f7d400a5380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c005fd0 tx=0x7f7d4c00b480 comp rx=0 tx=0).stop 2024-09-11T23:42:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 -- 172.21.15.133:0/2919593730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d58109b10 msgr2=0x7f7d58111d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:42:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d58109b10 0x7f7d58111d20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7d4802a880 tx=0x7f7d48033d10 comp rx=0 tx=0).stop 2024-09-11T23:42:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 -- 172.21.15.133:0/2919593730 shutdown_connections 2024-09-11T23:42:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7d400a2ec0 0x7f7d400a5380 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d58114ab0 0x7f7d58112260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d58109b10 0x7f7d58111d20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 --2- 172.21.15.133:0/2919593730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d581091f0 0x7f7d581117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:42:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 -- 172.21.15.133:0/2919593730 >> 172.21.15.133:0/2919593730 conn(0x7f7d58073660 msgr2=0x7f7d5806b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:42:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 -- 172.21.15.133:0/2919593730 shutdown_connections 2024-09-11T23:42:57.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:42:57.484+0000 7f7d5d306640 1 -- 172.21.15.133:0/2919593730 wait complete. 2024-09-11T23:42:58.296 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e33d2d6a', quiesce took 7.1 seconds. Will release after: 13.000000000000002 2024-09-11T23:42:58.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:42:58 smithi133 ceph-mon[31430]: pgmap v189: 97 pgs: 97 active+clean; 50 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 670 MiB/s wr, 1.06k op/s 2024-09-11T23:42:58.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:42:58 smithi178 ceph-mon[35882]: pgmap v189: 97 pgs: 97 active+clean; 50 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 670 MiB/s wr, 1.06k op/s 2024-09-11T23:42:58.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:42:58 smithi138 ceph-mon[35142]: pgmap v189: 97 pgs: 97 active+clean; 50 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 670 MiB/s wr, 1.06k op/s 2024-09-11T23:43:00.749 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:00 smithi138 ceph-mon[35142]: pgmap v190: 97 pgs: 97 active+clean; 50 GiB data, 218 GiB used, 854 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 815 MiB/s wr, 1.66k op/s 2024-09-11T23:43:00.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:00 smithi178 ceph-mon[35882]: pgmap v190: 97 pgs: 97 active+clean; 50 GiB data, 218 GiB used, 854 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 815 MiB/s wr, 1.66k op/s 2024-09-11T23:43:00.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:00 smithi133 ceph-mon[31430]: pgmap v190: 97 pgs: 97 active+clean; 50 GiB data, 218 GiB used, 854 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 815 MiB/s wr, 1.66k op/s 2024-09-11T23:43:02.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:02 smithi178 ceph-mon[35882]: pgmap v191: 97 pgs: 97 active+clean; 50 GiB data, 219 GiB used, 854 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 614 MiB/s wr, 1.53k op/s 2024-09-11T23:43:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:02 smithi138 ceph-mon[35142]: pgmap v191: 97 pgs: 97 active+clean; 50 GiB data, 219 GiB used, 854 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 614 MiB/s wr, 1.53k op/s 2024-09-11T23:43:02.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:02 smithi133 ceph-mon[31430]: pgmap v191: 97 pgs: 97 active+clean; 50 GiB data, 219 GiB used, 854 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 614 MiB/s wr, 1.53k op/s 2024-09-11T23:43:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:03 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:43:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:03 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:03 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:43:03.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:03 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:43:03.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:03 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:43:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:03 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:03 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:43:03.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:03 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:43:03.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:03 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:43:03.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:03 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:43:03.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:03 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:43:03.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:03 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:43:04.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:04 smithi178 ceph-mon[35882]: pgmap v192: 97 pgs: 97 active+clean; 50 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 592 MiB/s wr, 1.52k op/s 2024-09-11T23:43:04.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:04 smithi138 ceph-mon[35142]: pgmap v192: 97 pgs: 97 active+clean; 50 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 592 MiB/s wr, 1.52k op/s 2024-09-11T23:43:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:04 smithi133 ceph-mon[31430]: pgmap v192: 97 pgs: 97 active+clean; 50 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 592 MiB/s wr, 1.52k op/s 2024-09-11T23:43:06.039 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:43:06.303 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:06.705 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:06 smithi133 ceph-mon[31430]: pgmap v193: 97 pgs: 97 active+clean; 50 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 340 MiB/s wr, 1.09k op/s 2024-09-11T23:43:06.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:06 smithi178 ceph-mon[35882]: pgmap v193: 97 pgs: 97 active+clean; 50 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 340 MiB/s wr, 1.09k op/s 2024-09-11T23:43:06.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.789+0000 7f7967482640 1 -- 172.21.15.133:0/2397199362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7960101420 msgr2=0x7f796010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.789+0000 7f7967482640 1 --2- 172.21.15.133:0/2397199362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7960101420 0x7f796010d950 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7958009930 tx=0x7f795802a420 comp rx=0 tx=0).stop 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.789+0000 7f7967482640 1 -- 172.21.15.133:0/2397199362 shutdown_connections 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.789+0000 7f7967482640 1 --2- 172.21.15.133:0/2397199362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7960101420 0x7f796010d950 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.789+0000 7f7967482640 1 --2- 172.21.15.133:0/2397199362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7960100a80 0x7f7960100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.789+0000 7f7967482640 1 --2- 172.21.15.133:0/2397199362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7960106a80 0x7f7960106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.789+0000 7f7967482640 1 -- 172.21.15.133:0/2397199362 >> 172.21.15.133:0/2397199362 conn(0x7f79600fc7a0 msgr2=0x7f79600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.789+0000 7f7967482640 1 -- 172.21.15.133:0/2397199362 shutdown_connections 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.789+0000 7f7967482640 1 -- 172.21.15.133:0/2397199362 wait complete. 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7967482640 1 Processor -- start 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7967482640 1 -- start start 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7967482640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7960100a80 0x7f796010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7967482640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7960101420 0x7f796010e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7967482640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7960106a80 0x7f796010ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7967482640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7960111fa0 con 0x7f7960106a80 2024-09-11T23:43:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7967482640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7960111e20 con 0x7f7960100a80 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7967482640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7960112120 con 0x7f7960101420 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7965a1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7960106a80 0x7f796010ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7965a1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7960106a80 0x7f796010ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41816/0 (socket says 172.21.15.133:41816) 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7965a1f640 1 -- 172.21.15.133:0/3674910564 learned_addr learned my addr 172.21.15.133:0/3674910564 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f7964a1d640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7960101420 0x7f796010e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.790+0000 7f796521e640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7960100a80 0x7f796010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.791+0000 7f7965a1f640 1 -- 172.21.15.133:0/3674910564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7960101420 msgr2=0x7f796010e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.791+0000 7f7965a1f640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7960101420 0x7f796010e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.791+0000 7f7965a1f640 1 -- 172.21.15.133:0/3674910564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7960100a80 msgr2=0x7f796010e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.791+0000 7f7965a1f640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7960100a80 0x7f796010e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.791+0000 7f7965a1f640 1 -- 172.21.15.133:0/3674910564 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79601a6900 con 0x7f7960106a80 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.791+0000 7f7965a1f640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7960106a80 0x7f796010ee60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7958005ab0 tx=0x7f79580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.791+0000 7f795e7fc640 1 -- 172.21.15.133:0/3674910564 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795803f030 con 0x7f7960106a80 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.791+0000 7f7967482640 1 -- 172.21.15.133:0/3674910564 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79601a6b90 con 0x7f7960106a80 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.791+0000 7f795e7fc640 1 -- 172.21.15.133:0/3674910564 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7958003ce0 con 0x7f7960106a80 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.791+0000 7f795e7fc640 1 -- 172.21.15.133:0/3674910564 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958035070 con 0x7f7960106a80 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.791+0000 7f7967482640 1 -- 172.21.15.133:0/3674910564 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79601a70c0 con 0x7f7960106a80 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.793+0000 7f795e7fc640 1 -- 172.21.15.133:0/3674910564 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7958042030 con 0x7f7960106a80 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796985b740 1 Processor -- start 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796985b740 1 -- start start 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796985b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d474a0 0x559435d47880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796985b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d47dc0 0x559435d52140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796985b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d52680 0x559435d54a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796985b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559435cad2e0 con 0x559435d47dc0 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796985b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559435c54ce0 con 0x559435d52680 2024-09-11T23:43:06.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796985b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559435ca92a0 con 0x559435d474a0 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796521e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d47dc0 0x559435d52140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796521e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d47dc0 0x559435d52140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41820/0 (socket says 172.21.15.133:41820) 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796521e640 1 -- 172.21.15.133:0/1340598704 learned_addr learned my addr 172.21.15.133:0/1340598704 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f795e7fc640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79480a2cf0 0x7f79480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1340598704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d474a0 0x559435d47880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1340598704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d52680 0x559435d54a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f7965a1f640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79480a2cf0 0x7f79480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796521e640 1 -- 172.21.15.133:0/1340598704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d474a0 msgr2=0x559435d47880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796521e640 1 --2- 172.21.15.133:0/1340598704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d474a0 0x559435d47880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796521e640 1 -- 172.21.15.133:0/1340598704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d52680 msgr2=0x559435d54a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796521e640 1 --2- 172.21.15.133:0/1340598704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d52680 0x559435d54a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f796521e640 1 -- 172.21.15.133:0/1340598704 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559435c39440 con 0x559435d47dc0 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1340598704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d52680 0x559435d54a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.794+0000 7f795e7fc640 1 -- 172.21.15.133:0/3674910564 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f79580ef790 con 0x7f7960106a80 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.795+0000 7f796521e640 1 --2- 172.21.15.133:0/1340598704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d47dc0 0x559435d52140 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7950010370 tx=0x7f7950010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147574ff04d2a559 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.795+0000 7f79477fe640 1 -- 172.21.15.133:0/1340598704 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950016070 con 0x559435d47dc0 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.795+0000 7f7965a1f640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79480a2cf0 0x7f79480a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f79580ed550 tx=0x7f79580b1260 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.795+0000 7f79477fe640 1 -- 172.21.15.133:0/1340598704 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7950004d20 con 0x559435d47dc0 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.795+0000 7f79477fe640 1 -- 172.21.15.133:0/1340598704 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950005480 con 0x559435d47dc0 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.796+0000 7f796985b740 1 -- 172.21.15.133:0/1340598704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d47dc0 msgr2=0x559435d52140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.796+0000 7f796985b740 1 --2- 172.21.15.133:0/1340598704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d47dc0 0x559435d52140 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7950010370 tx=0x7f7950010830 comp rx=0 tx=0).stop 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.796+0000 7f796985b740 1 -- 172.21.15.133:0/1340598704 shutdown_connections 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.796+0000 7f796985b740 1 --2- 172.21.15.133:0/1340598704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d52680 0x559435d54a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.796+0000 7f796985b740 1 --2- 172.21.15.133:0/1340598704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d47dc0 0x559435d52140 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.796+0000 7f796985b740 1 --2- 172.21.15.133:0/1340598704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d474a0 0x559435d47880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.796+0000 7f796985b740 1 -- 172.21.15.133:0/1340598704 >> 172.21.15.133:0/1340598704 conn(0x559435d43a30 msgr2=0x559435d43e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.796+0000 7f796985b740 1 -- 172.21.15.133:0/1340598704 shutdown_connections 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.796+0000 7f796985b740 1 -- 172.21.15.133:0/1340598704 wait complete. 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.796+0000 7f796985b740 1 -- start start 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.796+0000 7f796985b740 1 Processor -- start 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f796985b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d474a0 0x559435d51f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f796985b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d47dc0 0x559435d4f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f796985b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d52680 0x559435d4f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f796985b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559435d55250 con 0x559435d52680 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f796985b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559435c68050 con 0x559435d47dc0 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f796985b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559435d553d0 con 0x559435d474a0 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f796985b740 10 client.0 _ll_register_callbacks cb 0x559435ca8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f7964a1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d52680 0x559435d4f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f7964a1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d52680 0x559435d4f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41834/0 (socket says 172.21.15.133:41834) 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f7964a1d640 1 -- 172.21.15.133:0/1743350735 learned_addr learned my addr 172.21.15.133:0/1743350735 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d474a0 0x559435d51f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f796521e640 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d47dc0 0x559435d4f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f7964a1d640 1 -- 172.21.15.133:0/1743350735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d474a0 msgr2=0x559435d51f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d474a0 0x559435d51f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f7964a1d640 1 -- 172.21.15.133:0/1743350735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d47dc0 msgr2=0x559435d4f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d47dc0 0x559435d4f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f7964a1d640 1 -- 172.21.15.133:0/1743350735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559435a1e3c0 con 0x559435d52680 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f796521e640 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d47dc0 0x559435d4f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d52680 0x559435d4f710 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7954013bf0 tx=0x7f79540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9ce7940ddd6783 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.797+0000 7f7944ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f7944ff9640 1 -- 172.21.15.133:0/1743350735 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954011d30 con 0x559435d52680 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f7944ff9640 1 -- 172.21.15.133:0/1743350735 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f7954013050 con 0x559435d52680 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f796985b740 1 -- 172.21.15.133:0/1743350735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7950004d20 con 0x559435d52680 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f7944ff9640 1 -- 172.21.15.133:0/1743350735 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7954013250 con 0x559435d52680 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f796985b740 10 client.17049 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f796985b740 1 -- 172.21.15.133:0/1743350735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559435d60ba0 con 0x559435d52680 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f7944ff9640 1 -- 172.21.15.133:0/1743350735 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7950004d20 con 0x559435d52680 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f7944ff9640 1 -- 172.21.15.133:0/1743350735 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7954004a70 con 0x559435d52680 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f796985b740 10 client.17049 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f796985b740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f796985b740 20 client.17049 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f796985b740 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559435d62060 0x559435d64450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f796985b740 4 client.17049 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.798+0000 7f796985b740 1 -- 172.21.15.133:0/1743350735 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559435cba850 con 0x559435d62060 2024-09-11T23:43:06.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.799+0000 7f796521e640 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559435d62060 0x559435d64450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.799+0000 7f796521e640 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559435d62060 0x559435d64450 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c445c9da727a827a server_cookie=1a6d7c52aa44c5c1 in_seq=0 out_seq=0 2024-09-11T23:43:06.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.799+0000 7f7944ff9640 10 client.17049 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f7944ff9640 1 -- 172.21.15.133:0/1743350735 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559435cba850 con 0x559435d62060 2024-09-11T23:43:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f7944ff9640 10 client.17049 handle_command_reply: tid=0 2024-09-11T23:43:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 1 client.17049 shutdown 2024-09-11T23:43:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 1 -- 172.21.15.133:0/1743350735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d52680 msgr2=0x559435d4f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d52680 0x559435d4f710 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7954013bf0 tx=0x7f79540046c0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 1 -- 172.21.15.133:0/1743350735 shutdown_connections 2024-09-11T23:43:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559435d62060 0x559435d64450 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d52680 0x559435d4f710 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d47dc0 0x559435d4f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f7944ff9640 0 client.17049 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 1 --2- 172.21.15.133:0/1743350735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d474a0 0x559435d51f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 1 -- 172.21.15.133:0/1743350735 >> 172.21.15.133:0/1743350735 conn(0x559435d43a30 msgr2=0x559435d49710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 1 -- 172.21.15.133:0/1743350735 shutdown_connections 2024-09-11T23:43:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 1 -- 172.21.15.133:0/1743350735 wait complete. 2024-09-11T23:43:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.800+0000 7f796985b740 20 client.17049 trim_cache size 0 max 16384 2024-09-11T23:43:06.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796985b740 1 Processor -- start 2024-09-11T23:43:06.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796985b740 1 -- start start 2024-09-11T23:43:06.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796985b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d43a30 0x559435d51140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796985b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d474a0 0x559435d69c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796985b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d47dc0 0x559435d6a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796985b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f79601af960 con 0x559435d474a0 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796985b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79601b14a0 con 0x559435d43a30 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796985b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f79601b0030 con 0x559435d47dc0 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796521e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d474a0 0x559435d69c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796521e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d474a0 0x559435d69c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41838/0 (socket says 172.21.15.133:41838) 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796521e640 1 -- 172.21.15.133:0/1184996569 learned_addr learned my addr 172.21.15.133:0/1184996569 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1184996569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d47dc0 0x559435d6a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796521e640 1 -- 172.21.15.133:0/1184996569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d47dc0 msgr2=0x559435d6a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796521e640 1 --2- 172.21.15.133:0/1184996569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d47dc0 0x559435d6a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796521e640 1 -- 172.21.15.133:0/1184996569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d43a30 msgr2=0x559435d51140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:06.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796521e640 1 --2- 172.21.15.133:0/1184996569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d43a30 0x559435d51140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f796521e640 1 -- 172.21.15.133:0/1184996569 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559435d47bf0 con 0x559435d474a0 2024-09-11T23:43:06.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.818+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1184996569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d47dc0 0x559435d6a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:43:06.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.819+0000 7f796521e640 1 --2- 172.21.15.133:0/1184996569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d474a0 0x559435d69c40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x559435d44890 tx=0x7f7950010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2082bf10cd9150ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:06.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.819+0000 7f7946ffd640 1 -- 172.21.15.133:0/1184996569 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950016050 con 0x559435d474a0 2024-09-11T23:43:06.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.819+0000 7f7946ffd640 1 -- 172.21.15.133:0/1184996569 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7950003b70 con 0x559435d474a0 2024-09-11T23:43:06.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.819+0000 7f7946ffd640 1 -- 172.21.15.133:0/1184996569 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950012510 con 0x559435d474a0 2024-09-11T23:43:06.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.819+0000 7f796985b740 1 -- 172.21.15.133:0/1184996569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d474a0 msgr2=0x559435d69c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.819+0000 7f796985b740 1 --2- 172.21.15.133:0/1184996569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d474a0 0x559435d69c40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x559435d44890 tx=0x7f7950010fa0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.819+0000 7f796985b740 1 -- 172.21.15.133:0/1184996569 shutdown_connections 2024-09-11T23:43:06.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.819+0000 7f796985b740 1 --2- 172.21.15.133:0/1184996569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d47dc0 0x559435d6a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.819+0000 7f796985b740 1 --2- 172.21.15.133:0/1184996569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d474a0 0x559435d69c40 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.819+0000 7f796985b740 1 --2- 172.21.15.133:0/1184996569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d43a30 0x559435d51140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.819+0000 7f796985b740 1 -- 172.21.15.133:0/1184996569 >> 172.21.15.133:0/1184996569 conn(0x559435d4f870 msgr2=0x559435d50a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:06.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f796985b740 1 -- 172.21.15.133:0/1184996569 shutdown_connections 2024-09-11T23:43:06.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f796985b740 1 -- 172.21.15.133:0/1184996569 wait complete. 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f796985b740 1 -- start start 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f796985b740 1 Processor -- start 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f796985b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d43a30 0x559435d58730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f796985b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d474a0 0x559435d58c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f796985b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d47dc0 0x559435d591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f796985b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f79601b0dd0 con 0x559435d43a30 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f796985b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79601b0700 con 0x559435d474a0 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f796985b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f79601ad960 con 0x559435d47dc0 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f796985b740 10 client.0 _ll_register_callbacks cb 0x7f795400be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.820+0000 7f7964a1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d43a30 0x559435d58730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7964a1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d43a30 0x559435d58730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41844/0 (socket says 172.21.15.133:41844) 2024-09-11T23:43:06.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7964a1d640 1 -- 172.21.15.133:0/1638961035 learned_addr learned my addr 172.21.15.133:0/1638961035 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f796521e640 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d474a0 0x559435d58c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d47dc0 0x559435d591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7964a1d640 1 -- 172.21.15.133:0/1638961035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d47dc0 msgr2=0x559435d591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d47dc0 0x559435d591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7964a1d640 1 -- 172.21.15.133:0/1638961035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d474a0 msgr2=0x559435d58c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d474a0 0x559435d58c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7964a1d640 1 -- 172.21.15.133:0/1638961035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559435dcbf10 con 0x559435d43a30 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7964a1d640 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d43a30 0x559435d58730 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f795400b620 tx=0x7f7954001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc26358df099da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7944ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7944ff9640 1 -- 172.21.15.133:0/1638961035 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79540046c0 con 0x559435d43a30 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7944ff9640 1 -- 172.21.15.133:0/1638961035 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f7954004860 con 0x559435d43a30 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f796985b740 1 -- 172.21.15.133:0/1638961035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7950003b70 con 0x559435d43a30 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f7944ff9640 1 -- 172.21.15.133:0/1638961035 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f795403a050 con 0x559435d43a30 2024-09-11T23:43:06.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f796985b740 10 client.17055 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.821+0000 7f796985b740 1 -- 172.21.15.133:0/1638961035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559435dcd190 con 0x559435d43a30 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.822+0000 7f7944ff9640 1 -- 172.21.15.133:0/1638961035 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7950003b70 con 0x559435d43a30 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.822+0000 7f7944ff9640 1 -- 172.21.15.133:0/1638961035 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7954021460 con 0x559435d43a30 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.822+0000 7f796985b740 10 client.17055 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.822+0000 7f796985b740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.822+0000 7f796985b740 20 client.17055 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.822+0000 7f796985b740 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559435d52680 0x559435dcf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.822+0000 7f796985b740 4 client.17055 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.822+0000 7f796985b740 1 -- 172.21.15.133:0/1638961035 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559435cb4d60 con 0x559435d52680 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.822+0000 7f796521e640 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559435d52680 0x559435dcf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.822+0000 7f7944ff9640 10 client.17055 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.822+0000 7f796521e640 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559435d52680 0x559435dcf950 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6546edf392461304 server_cookie=79ff063a43e981cb in_seq=0 out_seq=0 2024-09-11T23:43:06.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f7944ff9640 1 -- 172.21.15.133:0/1638961035 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559435cb4d60 con 0x559435d52680 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f7944ff9640 10 client.17055 handle_command_reply: tid=0 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 1 client.17055 shutdown 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 1 -- 172.21.15.133:0/1638961035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d43a30 msgr2=0x559435d58730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d43a30 0x559435d58730 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f795400b620 tx=0x7f7954001e50 comp rx=0 tx=0).stop 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 1 -- 172.21.15.133:0/1638961035 shutdown_connections 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559435d52680 0x559435dcf950 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559435d47dc0 0x559435d591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559435d474a0 0x559435d58c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f7944ff9640 0 client.17055 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 1 --2- 172.21.15.133:0/1638961035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559435d43a30 0x559435d58730 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 1 -- 172.21.15.133:0/1638961035 >> 172.21.15.133:0/1638961035 conn(0x559435d4f870 msgr2=0x559435d51250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 1 -- 172.21.15.133:0/1638961035 shutdown_connections 2024-09-11T23:43:06.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 1 -- 172.21.15.133:0/1638961035 wait complete. 2024-09-11T23:43:06.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.823+0000 7f796985b740 20 client.17055 trim_cache size 0 max 16384 2024-09-11T23:43:06.829 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:43:06.829 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:43:06.829 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:43:06.829 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 -- 172.21.15.133:0/3674910564 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79480a2cf0 msgr2=0x7f79480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79480a2cf0 0x7f79480a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f79580ed550 tx=0x7f79580b1260 comp rx=0 tx=0).stop 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 -- 172.21.15.133:0/3674910564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7960106a80 msgr2=0x7f796010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7960106a80 0x7f796010ee60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7958005ab0 tx=0x7f79580042f0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 -- 172.21.15.133:0/3674910564 shutdown_connections 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79480a2cf0 0x7f79480a51b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7960106a80 0x7f796010ee60 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7960101420 0x7f796010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 --2- 172.21.15.133:0/3674910564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7960100a80 0x7f796010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 -- 172.21.15.133:0/3674910564 >> 172.21.15.133:0/3674910564 conn(0x7f79600fc7a0 msgr2=0x7f79600fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 -- 172.21.15.133:0/3674910564 shutdown_connections 2024-09-11T23:43:06.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:06.824+0000 7f7967482640 1 -- 172.21.15.133:0/3674910564 wait complete. 2024-09-11T23:43:06.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:06 smithi138 ceph-mon[35142]: pgmap v193: 97 pgs: 97 active+clean; 50 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 340 MiB/s wr, 1.09k op/s 2024-09-11T23:43:07.628 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:43:07.629 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:43:08.124 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.564+0000 7f0158722640 1 -- 172.21.15.133:0/2726615356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0150101de0 msgr2=0x7f01501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.564+0000 7f0158722640 1 --2- 172.21.15.133:0/2726615356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0150101de0 0x7f01501021c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f014c009900 tx=0x7f014c02a410 comp rx=0 tx=0).stop 2024-09-11T23:43:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.564+0000 7f0158722640 1 -- 172.21.15.133:0/2726615356 shutdown_connections 2024-09-11T23:43:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.564+0000 7f0158722640 1 --2- 172.21.15.133:0/2726615356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0150106f10 0x7f015010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.564+0000 7f0158722640 1 --2- 172.21.15.133:0/2726615356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150102790 0x7f01501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.564+0000 7f0158722640 1 --2- 172.21.15.133:0/2726615356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0150101de0 0x7f01501021c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.564+0000 7f0158722640 1 -- 172.21.15.133:0/2726615356 >> 172.21.15.133:0/2726615356 conn(0x7f01500fd8c0 msgr2=0x7f01500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.564+0000 7f0158722640 1 -- 172.21.15.133:0/2726615356 shutdown_connections 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.564+0000 7f0158722640 1 -- 172.21.15.133:0/2726615356 wait complete. 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0158722640 1 Processor -- start 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0158722640 1 -- start start 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0158722640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150101de0 0x7f015019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0158722640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0150102790 0x7f015019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0158722640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0150106f10 0x7f015019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0158722640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0150111f70 con 0x7f0150106f10 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0158722640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0150111df0 con 0x7f0150101de0 2024-09-11T23:43:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0158722640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f01501120f0 con 0x7f0150102790 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0156cbf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0150106f10 0x7f015019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0156cbf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0150106f10 0x7f015019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41870/0 (socket says 172.21.15.133:41870) 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0156cbf640 1 -- 172.21.15.133:0/2755367495 learned_addr learned my addr 172.21.15.133:0/2755367495 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f01564be640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150101de0 0x7f015019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0155cbd640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0150102790 0x7f015019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0156cbf640 1 -- 172.21.15.133:0/2755367495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0150102790 msgr2=0x7f015019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0156cbf640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0150102790 0x7f015019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0156cbf640 1 -- 172.21.15.133:0/2755367495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150101de0 msgr2=0x7f015019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0156cbf640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150101de0 0x7f015019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0156cbf640 1 -- 172.21.15.133:0/2755367495 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015019f620 con 0x7f0150106f10 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.565+0000 7f0156cbf640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0150106f10 0x7f015019ef40 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0148009fd0 tx=0x7f014800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.566+0000 7f01477fe640 1 -- 172.21.15.133:0/2755367495 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148017070 con 0x7f0150106f10 2024-09-11T23:43:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.566+0000 7f0158722640 1 -- 172.21.15.133:0/2755367495 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015019f910 con 0x7f0150106f10 2024-09-11T23:43:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.566+0000 7f01477fe640 1 -- 172.21.15.133:0/2755367495 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0148003e50 con 0x7f0150106f10 2024-09-11T23:43:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.566+0000 7f01477fe640 1 -- 172.21.15.133:0/2755367495 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01480106c0 con 0x7f0150106f10 2024-09-11T23:43:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.566+0000 7f0158722640 1 -- 172.21.15.133:0/2755367495 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01501a71f0 con 0x7f0150106f10 2024-09-11T23:43:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.567+0000 7f0158722640 1 -- 172.21.15.133:0/2755367495 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01501021c0 con 0x7f0150106f10 2024-09-11T23:43:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.568+0000 7f01477fe640 1 -- 172.21.15.133:0/2755367495 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0148003970 con 0x7f0150106f10 2024-09-11T23:43:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.569+0000 7f01477fe640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f013c0a2cf0 0x7f013c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.569+0000 7f01564be640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f013c0a2cf0 0x7f013c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.569+0000 7f01477fe640 1 -- 172.21.15.133:0/2755367495 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f01480cf1a0 con 0x7f0150106f10 2024-09-11T23:43:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.569+0000 7f01564be640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f013c0a2cf0 0x7f013c0a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f014c005ab0 tx=0x7f014c005a20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.572+0000 7f01477fe640 1 -- 172.21.15.133:0/2755367495 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f014809be30 con 0x7f0150106f10 2024-09-11T23:43:08.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.780+0000 7f0158722640 1 -- 172.21.15.133:0/2755367495 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01501a7420 con 0x7f0150106f10 2024-09-11T23:43:08.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:08 smithi133 ceph-mon[31430]: pgmap v194: 97 pgs: 97 active+clean; 50 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 259 MiB/s wr, 903 op/s 2024-09-11T23:43:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.781+0000 7f01477fe640 1 -- 172.21.15.133:0/2755367495 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7f014809b730 con 0x7f0150106f10 2024-09-11T23:43:08.781 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:43:08.781 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:43:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:43:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 -- 172.21.15.133:0/2755367495 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f013c0a2cf0 msgr2=0x7f013c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f013c0a2cf0 0x7f013c0a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f014c005ab0 tx=0x7f014c005a20 comp rx=0 tx=0).stop 2024-09-11T23:43:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 -- 172.21.15.133:0/2755367495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0150106f10 msgr2=0x7f015019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0150106f10 0x7f015019ef40 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0148009fd0 tx=0x7f014800b260 comp rx=0 tx=0).stop 2024-09-11T23:43:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 -- 172.21.15.133:0/2755367495 shutdown_connections 2024-09-11T23:43:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f013c0a2cf0 0x7f013c0a51b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0150106f10 0x7f015019ef40 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0150102790 0x7f015019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 --2- 172.21.15.133:0/2755367495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150101de0 0x7f015019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 -- 172.21.15.133:0/2755367495 >> 172.21.15.133:0/2755367495 conn(0x7f01500fd8c0 msgr2=0x7f01500fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:08.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 -- 172.21.15.133:0/2755367495 shutdown_connections 2024-09-11T23:43:08.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:08.783+0000 7f0158722640 1 -- 172.21.15.133:0/2755367495 wait complete. 2024-09-11T23:43:08.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:08 smithi178 ceph-mon[35882]: pgmap v194: 97 pgs: 97 active+clean; 50 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 259 MiB/s wr, 903 op/s 2024-09-11T23:43:08.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:08 smithi138 ceph-mon[35142]: pgmap v194: 97 pgs: 97 active+clean; 50 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 259 MiB/s wr, 903 op/s 2024-09-11T23:43:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.264+0000 7fb70dab7640 1 -- 172.21.15.133:0/3552784217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7081069c0 msgr2=0x7fb708106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.264+0000 7fb70dab7640 1 --2- 172.21.15.133:0/3552784217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7081069c0 0x7fb708106da0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009930 tx=0x7fb6f802a420 comp rx=0 tx=0).stop 2024-09-11T23:43:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.264+0000 7fb70dab7640 1 -- 172.21.15.133:0/3552784217 shutdown_connections 2024-09-11T23:43:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.264+0000 7fb70dab7640 1 --2- 172.21.15.133:0/3552784217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb708101350 0x7fb70810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.264+0000 7fb70dab7640 1 --2- 172.21.15.133:0/3552784217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7081009b0 0x7fb708100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.264+0000 7fb70dab7640 1 --2- 172.21.15.133:0/3552784217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7081069c0 0x7fb708106da0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.264+0000 7fb70dab7640 1 -- 172.21.15.133:0/3552784217 >> 172.21.15.133:0/3552784217 conn(0x7fb708077990 msgr2=0x7fb7080fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.264+0000 7fb70dab7640 1 -- 172.21.15.133:0/3552784217 shutdown_connections 2024-09-11T23:43:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.264+0000 7fb70dab7640 1 -- 172.21.15.133:0/3552784217 wait complete. 2024-09-11T23:43:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.264+0000 7fb70dab7640 1 Processor -- start 2024-09-11T23:43:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb70dab7640 1 -- start start 2024-09-11T23:43:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb70dab7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7081009b0 0x7fb7081991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb70dab7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb708101350 0x7fb708199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb70dab7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7081069c0 0x7fb7081a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb70dab7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb708110e00 con 0x7fb7081069c0 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb70dab7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb708110c80 con 0x7fb7081009b0 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb70dab7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb708110f80 con 0x7fb708101350 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb707fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7081069c0 0x7fb7081a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb707fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7081069c0 0x7fb7081a4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41884/0 (socket says 172.21.15.133:41884) 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb707fff640 1 -- 172.21.15.133:0/3669830252 learned_addr learned my addr 172.21.15.133:0/3669830252 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb7077fe640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7081009b0 0x7fb7081991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb706ffd640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb708101350 0x7fb708199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb707fff640 1 -- 172.21.15.133:0/3669830252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb708101350 msgr2=0x7fb708199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb707fff640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb708101350 0x7fb708199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb707fff640 1 -- 172.21.15.133:0/3669830252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7081009b0 msgr2=0x7fb7081991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb707fff640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7081009b0 0x7fb7081991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.265+0000 7fb707fff640 1 -- 172.21.15.133:0/3669830252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7081a4790 con 0x7fb7081069c0 2024-09-11T23:43:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.266+0000 7fb707fff640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7081069c0 0x7fb7081a4250 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb70000eaa0 tx=0x7fb70000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.266+0000 7fb704ff9640 1 -- 172.21.15.133:0/3669830252 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70000cdb0 con 0x7fb7081069c0 2024-09-11T23:43:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.266+0000 7fb70dab7640 1 -- 172.21.15.133:0/3669830252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7081a4960 con 0x7fb7081069c0 2024-09-11T23:43:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.266+0000 7fb704ff9640 1 -- 172.21.15.133:0/3669830252 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb700005020 con 0x7fb7081069c0 2024-09-11T23:43:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.266+0000 7fb704ff9640 1 -- 172.21.15.133:0/3669830252 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7000116b0 con 0x7fb7081069c0 2024-09-11T23:43:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.266+0000 7fb70dab7640 1 -- 172.21.15.133:0/3669830252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081a4e80 con 0x7fb7081069c0 2024-09-11T23:43:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.268+0000 7fb704ff9640 1 -- 172.21.15.133:0/3669830252 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb700005560 con 0x7fb7081069c0 2024-09-11T23:43:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.268+0000 7fb710851740 1 Processor -- start 2024-09-11T23:43:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.268+0000 7fb710851740 1 -- start start 2024-09-11T23:43:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb710851740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a379d0 0x5558d7a37db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb710851740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a382f0 0x5558d7a46b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb710851740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a470c0 0x5558d7a494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb710851740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5558d799c2f0 con 0x5558d7a379d0 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb710851740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5558d7944270 con 0x5558d7a470c0 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb710851740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5558d79982b0 con 0x5558d7a382f0 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a379d0 0x5558d7a37db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a379d0 0x5558d7a37db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41886/0 (socket says 172.21.15.133:41886) 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb706ffd640 1 -- 172.21.15.133:0/1355048334 learned_addr learned my addr 172.21.15.133:0/1355048334 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb704ff9640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6f00a2da0 0x7fb6f00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1355048334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a470c0 0x5558d7a494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb7077fe640 1 --2- 172.21.15.133:0/1355048334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a382f0 0x5558d7a46b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb706ffd640 1 -- 172.21.15.133:0/1355048334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a382f0 msgr2=0x5558d7a46b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1355048334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a382f0 0x5558d7a46b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb706ffd640 1 -- 172.21.15.133:0/1355048334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a470c0 msgr2=0x5558d7a494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1355048334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a470c0 0x5558d7a494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb706ffd640 1 -- 172.21.15.133:0/1355048334 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558d7943340 con 0x5558d7a379d0 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb707fff640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6f00a2da0 0x7fb6f00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb704ff9640 1 -- 172.21.15.133:0/3669830252 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fb7000cf5c0 con 0x7fb7081069c0 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1355048334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a379d0 0x5558d7a37db0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0135a0 tx=0x7fb6fc00b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff704efa4ee9fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb6edffb640 1 -- 172.21.15.133:0/1355048334 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc019030 con 0x5558d7a379d0 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb6edffb640 1 -- 172.21.15.133:0/1355048334 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc0119a0 con 0x5558d7a379d0 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.269+0000 7fb6edffb640 1 -- 172.21.15.133:0/1355048334 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc01d880 con 0x5558d7a379d0 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.270+0000 7fb710851740 1 -- 172.21.15.133:0/1355048334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a379d0 msgr2=0x5558d7a37db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.270+0000 7fb710851740 1 --2- 172.21.15.133:0/1355048334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a379d0 0x5558d7a37db0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0135a0 tx=0x7fb6fc00b6b0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.270+0000 7fb707fff640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6f00a2da0 0x7fb6f00a5260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb7000cce30 tx=0x7fb70006b9b0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.270+0000 7fb710851740 1 -- 172.21.15.133:0/1355048334 shutdown_connections 2024-09-11T23:43:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.270+0000 7fb710851740 1 --2- 172.21.15.133:0/1355048334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a470c0 0x5558d7a494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.270+0000 7fb710851740 1 --2- 172.21.15.133:0/1355048334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a382f0 0x5558d7a46b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.270+0000 7fb710851740 1 --2- 172.21.15.133:0/1355048334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a379d0 0x5558d7a37db0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.270+0000 7fb710851740 1 -- 172.21.15.133:0/1355048334 >> 172.21.15.133:0/1355048334 conn(0x5558d7a34060 msgr2=0x5558d7a34440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.270+0000 7fb710851740 1 -- 172.21.15.133:0/1355048334 shutdown_connections 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.270+0000 7fb710851740 1 -- 172.21.15.133:0/1355048334 wait complete. 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.270+0000 7fb710851740 1 -- start start 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb710851740 1 Processor -- start 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb710851740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a43ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb710851740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a382f0 0x5558d7a44530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb710851740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a470c0 0x5558d7a44a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb710851740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5558d7a49cf0 con 0x5558d7a470c0 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb710851740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5558d7925290 con 0x5558d7a379d0 2024-09-11T23:43:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb710851740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5558d7a49e70 con 0x5558d7a382f0 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb710851740 10 client.0 _ll_register_callbacks cb 0x5558d7956140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a470c0 0x5558d7a44a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a470c0 0x5558d7a44a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41894/0 (socket says 172.21.15.133:41894) 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb706ffd640 1 -- 172.21.15.133:0/1482310166 learned_addr learned my addr 172.21.15.133:0/1482310166 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a43ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb7077fe640 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a382f0 0x5558d7a44530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb706ffd640 1 -- 172.21.15.133:0/1482310166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a382f0 msgr2=0x5558d7a44530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a382f0 0x5558d7a44530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb706ffd640 1 -- 172.21.15.133:0/1482310166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 msgr2=0x5558d7a43ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a43ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.271+0000 7fb706ffd640 1 -- 172.21.15.133:0/1482310166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558d76af210 con 0x5558d7a470c0 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a470c0 0x5558d7a44a70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc011b50 tx=0x7fb6fc01d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48098677b2bcac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1482310166 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc019040 con 0x5558d7a470c0 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1482310166 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fb6fc011bb0 con 0x5558d7a470c0 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb710851740 1 -- 172.21.15.133:0/1482310166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6fc0119a0 con 0x5558d7a470c0 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1482310166 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb6fc011db0 con 0x5558d7a470c0 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb710851740 10 client.17073 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb710851740 1 -- 172.21.15.133:0/1482310166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5558d777b620 con 0x5558d7a470c0 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1482310166 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc0119a0 con 0x5558d7a470c0 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1482310166 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb6fc003bf0 con 0x5558d7a470c0 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb710851740 10 client.17073 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb710851740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb710851740 20 client.17073 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb710851740 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558d7a57000 0x5558d7a593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb710851740 4 client.17073 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.272+0000 7fb710851740 1 -- 172.21.15.133:0/1482310166 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558d79a9a20 con 0x5558d7a57000 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.273+0000 7fb7077fe640 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558d7a57000 0x5558d7a593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.273+0000 7fb7077fe640 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558d7a57000 0x5558d7a593f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434fe7f9d8cf0643 server_cookie=e7a2a3caaed7b7c5 in_seq=0 out_seq=0 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.273+0000 7fb6e6ffd640 10 client.17073 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1482310166 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558d79a9a20 con 0x5558d7a57000 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb6e6ffd640 10 client.17073 handle_command_reply: tid=0 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 1 client.17073 shutdown 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 1 -- 172.21.15.133:0/1482310166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a470c0 msgr2=0x5558d7a44a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a470c0 0x5558d7a44a70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc011b50 tx=0x7fb6fc01d460 comp rx=0 tx=0).stop 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 1 -- 172.21.15.133:0/1482310166 shutdown_connections 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558d7a57000 0x5558d7a593f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a470c0 0x5558d7a44a70 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a382f0 0x5558d7a44530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb6e6ffd640 0 client.17073 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 1 --2- 172.21.15.133:0/1482310166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a43ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 1 -- 172.21.15.133:0/1482310166 >> 172.21.15.133:0/1482310166 conn(0x5558d7a34060 msgr2=0x5558d7a39dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 1 -- 172.21.15.133:0/1482310166 shutdown_connections 2024-09-11T23:43:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 1 -- 172.21.15.133:0/1482310166 wait complete. 2024-09-11T23:43:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.274+0000 7fb710851740 20 client.17073 trim_cache size 0 max 16384 2024-09-11T23:43:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.288+0000 7fb710851740 1 Processor -- start 2024-09-11T23:43:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.288+0000 7fb710851740 1 -- start start 2024-09-11T23:43:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb710851740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a45f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb710851740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a5ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb710851740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a470c0 0x5558d7a5ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb710851740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081ab560 con 0x5558d7a382f0 2024-09-11T23:43:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb710851740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081ad0a0 con 0x5558d7a379d0 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb710851740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081abc30 con 0x5558d7a470c0 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a5ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a470c0 0x5558d7a5ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a5ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41904/0 (socket says 172.21.15.133:41904) 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb7077fe640 1 -- 172.21.15.133:0/513943004 learned_addr learned my addr 172.21.15.133:0/513943004 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb706ffd640 1 --2- 172.21.15.133:0/513943004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a45f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb7077fe640 1 -- 172.21.15.133:0/513943004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a470c0 msgr2=0x5558d7a5ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb7077fe640 1 --2- 172.21.15.133:0/513943004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a470c0 0x5558d7a5ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb7077fe640 1 -- 172.21.15.133:0/513943004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 msgr2=0x5558d7a45f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb7077fe640 1 --2- 172.21.15.133:0/513943004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a45f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb7077fe640 1 -- 172.21.15.133:0/513943004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558d7a33e90 con 0x5558d7a382f0 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb7077fe640 1 --2- 172.21.15.133:0/513943004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a5ea50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb6f802a940 tx=0x7fb6f80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29b7d0011ecdc94 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb6ed7fa640 1 -- 172.21.15.133:0/513943004 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8035040 con 0x5558d7a382f0 2024-09-11T23:43:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb6ed7fa640 1 -- 172.21.15.133:0/513943004 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6f8004240 con 0x5558d7a382f0 2024-09-11T23:43:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.289+0000 7fb6ed7fa640 1 -- 172.21.15.133:0/513943004 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8005640 con 0x5558d7a382f0 2024-09-11T23:43:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.290+0000 7fb710851740 1 -- 172.21.15.133:0/513943004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 msgr2=0x5558d7a5ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.290+0000 7fb710851740 1 --2- 172.21.15.133:0/513943004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a5ea50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb6f802a940 tx=0x7fb6f80058d0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.290+0000 7fb710851740 1 -- 172.21.15.133:0/513943004 shutdown_connections 2024-09-11T23:43:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.290+0000 7fb710851740 1 --2- 172.21.15.133:0/513943004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a470c0 0x5558d7a5ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.290+0000 7fb710851740 1 --2- 172.21.15.133:0/513943004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a5ea50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.290+0000 7fb710851740 1 --2- 172.21.15.133:0/513943004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a45f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.290+0000 7fb710851740 1 -- 172.21.15.133:0/513943004 >> 172.21.15.133:0/513943004 conn(0x5558d7a34060 msgr2=0x5558d7a45850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.290+0000 7fb710851740 1 -- 172.21.15.133:0/513943004 shutdown_connections 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.290+0000 7fb710851740 1 -- 172.21.15.133:0/513943004 wait complete. 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.290+0000 7fb710851740 1 -- start start 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb710851740 1 Processor -- start 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb710851740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a4f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb710851740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a40bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb710851740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a470c0 0x5558d7a41100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb710851740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081ac9d0 con 0x5558d7a382f0 2024-09-11T23:43:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb710851740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081ac300 con 0x5558d7a379d0 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb710851740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081a9580 con 0x5558d7a470c0 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb710851740 10 client.0 _ll_register_callbacks cb 0x7fb6fc013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a40bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a40bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41910/0 (socket says 172.21.15.133:41910) 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb7077fe640 1 -- 172.21.15.133:0/1178600746 learned_addr learned my addr 172.21.15.133:0/1178600746 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a4f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb7077fe640 1 -- 172.21.15.133:0/1178600746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a470c0 msgr2=0x5558d7a41100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a470c0 0x5558d7a41100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb7077fe640 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a470c0 0x5558d7a41100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb7077fe640 1 -- 172.21.15.133:0/1178600746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 msgr2=0x5558d7a4f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb7077fe640 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a4f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb7077fe640 1 -- 172.21.15.133:0/1178600746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558d7a43dc0 con 0x5558d7a382f0 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a4f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb7077fe640 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a40bc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009230 tx=0x7fb6f8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baf813eefe04fea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.291+0000 7fb6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1178600746 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8035030 con 0x5558d7a382f0 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1178600746 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fb6f8042040 con 0x5558d7a382f0 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb710851740 1 -- 172.21.15.133:0/1178600746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6f8004240 con 0x5558d7a382f0 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1178600746 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb6f80043f0 con 0x5558d7a382f0 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb710851740 10 client.17079 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb710851740 1 -- 172.21.15.133:0/1178600746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5558d7ac1710 con 0x5558d7a382f0 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1178600746 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6f8004240 con 0x5558d7a382f0 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1178600746 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb6f8004930 con 0x5558d7a382f0 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb710851740 10 client.17079 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb710851740 10 client.17079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb710851740 20 client.17079 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb710851740 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558d7ac1e30 0x5558d7ac4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb710851740 4 client.17079 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:43:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb710851740 1 -- 172.21.15.133:0/1178600746 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558d79a3d70 con 0x5558d7ac1e30 2024-09-11T23:43:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.292+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558d7ac1e30 0x5558d7ac4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.293+0000 7fb706ffd640 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558d7ac1e30 0x5558d7ac4250 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4b75d1a91ae675 server_cookie=60d46070aac1ae2 in_seq=0 out_seq=0 2024-09-11T23:43:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.293+0000 7fb6e6ffd640 10 client.17079 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb6e6ffd640 1 -- 172.21.15.133:0/1178600746 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558d79a3d70 con 0x5558d7ac1e30 2024-09-11T23:43:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb6e6ffd640 10 client.17079 handle_command_reply: tid=0 2024-09-11T23:43:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb710851740 1 client.17079 shutdown 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb710851740 1 -- 172.21.15.133:0/1178600746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 msgr2=0x5558d7a40bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb710851740 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a40bc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009230 tx=0x7fb6f8009520 comp rx=0 tx=0).stop 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb710851740 1 -- 172.21.15.133:0/1178600746 shutdown_connections 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb710851740 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558d7ac1e30 0x5558d7ac4250 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb6e6ffd640 0 client.17079 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb710851740 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558d7a470c0 0x5558d7a41100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb710851740 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558d7a382f0 0x5558d7a40bc0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb710851740 1 --2- 172.21.15.133:0/1178600746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558d7a379d0 0x5558d7a4f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.311+0000 7fb710851740 1 -- 172.21.15.133:0/1178600746 >> 172.21.15.133:0/1178600746 conn(0x5558d7a34060 msgr2=0x5558d7a331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb710851740 1 -- 172.21.15.133:0/1178600746 shutdown_connections 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb710851740 1 -- 172.21.15.133:0/1178600746 wait complete. 2024-09-11T23:43:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb710851740 20 client.17079 trim_cache size 0 max 16384 2024-09-11T23:43:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:43:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:43:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:43:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:43:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 -- 172.21.15.133:0/3669830252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6f00a2da0 msgr2=0x7fb6f00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6f00a2da0 0x7fb6f00a5260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb7000cce30 tx=0x7fb70006b9b0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 -- 172.21.15.133:0/3669830252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7081069c0 msgr2=0x7fb7081a4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7081069c0 0x7fb7081a4250 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb70000eaa0 tx=0x7fb70000ef60 comp rx=0 tx=0).stop 2024-09-11T23:43:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 -- 172.21.15.133:0/3669830252 shutdown_connections 2024-09-11T23:43:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6f00a2da0 0x7fb6f00a5260 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7081069c0 0x7fb7081a4250 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb708101350 0x7fb708199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 --2- 172.21.15.133:0/3669830252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7081009b0 0x7fb7081991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 -- 172.21.15.133:0/3669830252 >> 172.21.15.133:0/3669830252 conn(0x7fb708077990 msgr2=0x7fb7080ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 -- 172.21.15.133:0/3669830252 shutdown_connections 2024-09-11T23:43:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:09.312+0000 7fb70dab7640 1 -- 172.21.15.133:0/3669830252 wait complete. 2024-09-11T23:43:09.471 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:43:09.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:09 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2755367495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:09.726 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:09.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:09 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2755367495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:09.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:09 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2755367495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:10.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.223+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/465073400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b4102ef0 msgr2=0x7fb6b410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:10.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.224+0000 7fb6b9ba3640 1 --2- 172.21.15.133:0/465073400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b4102ef0 0x7fb6b410da40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009ca0 tx=0x7fb6ac02a950 comp rx=0 tx=0).stop 2024-09-11T23:43:10.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.224+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/465073400 shutdown_connections 2024-09-11T23:43:10.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.224+0000 7fb6b9ba3640 1 --2- 172.21.15.133:0/465073400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b4102ef0 0x7fb6b410da40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.224+0000 7fb6b9ba3640 1 --2- 172.21.15.133:0/465073400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b4102570 0x7fb6b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.224+0000 7fb6b9ba3640 1 --2- 172.21.15.133:0/465073400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4100e20 0x7fb6b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.224+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/465073400 >> 172.21.15.133:0/465073400 conn(0x7fb6b40fc7d0 msgr2=0x7fb6b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:10.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.224+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/465073400 shutdown_connections 2024-09-11T23:43:10.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.224+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/465073400 wait complete. 2024-09-11T23:43:10.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.224+0000 7fb6b9ba3640 1 Processor -- start 2024-09-11T23:43:10.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.224+0000 7fb6b9ba3640 1 -- start start 2024-09-11T23:43:10.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b9ba3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4100e20 0x7fb6b419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b9ba3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b4102570 0x7fb6b419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b9ba3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b4102ef0 0x7fb6b419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b9ba3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b4110f80 con 0x7fb6b4102ef0 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b9ba3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b4110e00 con 0x7fb6b4102570 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b9ba3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b4111100 con 0x7fb6b4100e20 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b4102ef0 0x7fb6b419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b4102ef0 0x7fb6b419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41928/0 (socket says 172.21.15.133:41928) 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b3fff640 1 -- 172.21.15.133:0/3687185630 learned_addr learned my addr 172.21.15.133:0/3687185630 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b37fe640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4100e20 0x7fb6b419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b4102570 0x7fb6b419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b3fff640 1 -- 172.21.15.133:0/3687185630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4100e20 msgr2=0x7fb6b419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b3fff640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4100e20 0x7fb6b419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b3fff640 1 -- 172.21.15.133:0/3687185630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b4102570 msgr2=0x7fb6b419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b3fff640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b4102570 0x7fb6b419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b3fff640 1 -- 172.21.15.133:0/3687185630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b419f5f0 con 0x7fb6b4102ef0 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b4102570 0x7fb6b419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:10.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b3fff640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b4102ef0 0x7fb6b419ef10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac02ae70 tx=0x7fb6ac02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:10.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.225+0000 7fb6b0ff9640 1 -- 172.21.15.133:0/3687185630 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac034040 con 0x7fb6b4102ef0 2024-09-11T23:43:10.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.226+0000 7fb6b0ff9640 1 -- 172.21.15.133:0/3687185630 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6ac030920 con 0x7fb6b4102ef0 2024-09-11T23:43:10.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.226+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/3687185630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b419f880 con 0x7fb6b4102ef0 2024-09-11T23:43:10.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.226+0000 7fb6b0ff9640 1 -- 172.21.15.133:0/3687185630 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac045480 con 0x7fb6b4102ef0 2024-09-11T23:43:10.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.226+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/3687185630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b41a71c0 con 0x7fb6b4102ef0 2024-09-11T23:43:10.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.228+0000 7fb6b0ff9640 1 -- 172.21.15.133:0/3687185630 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb6ac037020 con 0x7fb6b4102ef0 2024-09-11T23:43:10.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.228+0000 7fb6bbf7c740 1 Processor -- start 2024-09-11T23:43:10.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.228+0000 7fb6bbf7c740 1 -- start start 2024-09-11T23:43:10.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.228+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 0x560087a32090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.228+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a325d0 0x560087a32a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.228+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a3f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.228+0000 7fb6b0ff9640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb69c0a2cf0 0x7fb69c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.228+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5600879982f0 con 0x560087a3d370 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.228+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56008793fcf0 con 0x560087a33b40 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.228+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5600879942b0 con 0x560087a325d0 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a3f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 0x560087a32090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 0x560087a32090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:50420/0 (socket says 172.21.15.133:50420) 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b2ffd640 1 -- 172.21.15.133:0/3936325953 learned_addr learned my addr 172.21.15.133:0/3936325953 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b37fe640 1 --2- 172.21.15.133:0/3936325953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a325d0 0x560087a32a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb69c0a2cf0 0x7fb69c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b0ff9640 1 -- 172.21.15.133:0/3687185630 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fb6ac0f01e0 con 0x7fb6b4102ef0 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b2ffd640 1 -- 172.21.15.133:0/3936325953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a325d0 msgr2=0x560087a32a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/3936325953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a325d0 0x560087a32a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b2ffd640 1 -- 172.21.15.133:0/3936325953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 msgr2=0x560087a3f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/3936325953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a3f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b2ffd640 1 -- 172.21.15.133:0/3936325953 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600879243a0 con 0x560087a33b40 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b3fff640 1 --2- 172.21.15.133:0/3936325953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a3f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/3936325953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 0x560087a32090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8014ce0 tx=0x7fb6a80117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f2d7728c720dbac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb69c0a2cf0 0x7fb69c0a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800bfd0 tx=0x7fb6a8016000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb699ffb640 1 -- 172.21.15.133:0/3936325953 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a800cb30 con 0x560087a33b40 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb699ffb640 1 -- 172.21.15.133:0/3936325953 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6a800ba70 con 0x560087a33b40 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.229+0000 7fb699ffb640 1 -- 172.21.15.133:0/3936325953 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8005020 con 0x560087a33b40 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.230+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/3936325953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 msgr2=0x560087a32090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.230+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/3936325953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 0x560087a32090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8014ce0 tx=0x7fb6a80117b0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.230+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/3936325953 shutdown_connections 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.230+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/3936325953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a3f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.230+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/3936325953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a325d0 0x560087a32a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.230+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/3936325953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 0x560087a32090 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.230+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/3936325953 >> 172.21.15.133:0/3936325953 conn(0x5600879fb310 msgr2=0x560087a2ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.230+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/3936325953 shutdown_connections 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.230+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/3936325953 wait complete. 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6bbf7c740 1 -- start start 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6bbf7c740 1 Processor -- start 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a325d0 0x560087a3cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a33b40 0x560087a3a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a3d370 0x560087a3a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560087a3ffb0 con 0x560087a33b40 2024-09-11T23:43:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560087953060 con 0x560087a325d0 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560087a40130 con 0x560087a3d370 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6bbf7c740 10 client.0 _ll_register_callbacks cb 0x5600876a68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6b2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a33b40 0x560087a3a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6b2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a33b40 0x560087a3a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41956/0 (socket says 172.21.15.133:41956) 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6b37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a325d0 0x560087a3cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6b2ffd640 1 -- 172.21.15.133:0/4206523047 learned_addr learned my addr 172.21.15.133:0/4206523047 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.231+0000 7fb6b3fff640 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a3d370 0x560087a3a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb6b37fe640 1 -- 172.21.15.133:0/4206523047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a3d370 msgr2=0x560087a3a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb6b37fe640 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a3d370 0x560087a3a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb6b37fe640 1 -- 172.21.15.133:0/4206523047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a33b40 msgr2=0x560087a3a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb6b37fe640 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a33b40 0x560087a3a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb6b37fe640 1 -- 172.21.15.133:0/4206523047 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600877093c0 con 0x560087a325d0 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb6b37fe640 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a325d0 0x560087a3cd60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4005fd0 tx=0x7fb6a400bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a69b04b77c4578a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb692ffd640 1 -- 172.21.15.133:0/4206523047 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4003e20 con 0x560087a325d0 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb692ffd640 1 -- 172.21.15.133:0/4206523047 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fb6a4005020 con 0x560087a325d0 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/4206523047 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a800ba70 con 0x560087a325d0 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb692ffd640 1 -- 172.21.15.133:0/4206523047 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb6a4005220 con 0x560087a325d0 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb6bbf7c740 10 client.26579 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/4206523047 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560087a4baf0 con 0x560087a325d0 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.232+0000 7fb692ffd640 1 -- 172.21.15.133:0/4206523047 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6a800ba70 con 0x560087a325d0 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.233+0000 7fb692ffd640 1 -- 172.21.15.133:0/4206523047 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb6a4023020 con 0x560087a325d0 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.233+0000 7fb6bbf7c740 10 client.26579 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.233+0000 7fb6bbf7c740 10 client.26579 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.233+0000 7fb6bbf7c740 20 client.26579 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.233+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560087a4cf00 0x560087a4f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.233+0000 7fb6bbf7c740 4 client.26579 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.233+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/4206523047 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600879a5860 con 0x560087a4cf00 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.233+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560087a4cf00 0x560087a4f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.233+0000 7fb692ffd640 10 client.26579 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.233+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560087a4cf00 0x560087a4f350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ee67808de0baef3 server_cookie=c909a1ee8aa93925 in_seq=0 out_seq=0 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb692ffd640 1 -- 172.21.15.133:0/4206523047 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600879a5860 con 0x560087a4cf00 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb692ffd640 10 client.26579 handle_command_reply: tid=0 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 1 client.26579 shutdown 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/4206523047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a325d0 msgr2=0x560087a3cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a325d0 0x560087a3cd60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4005fd0 tx=0x7fb6a400bd30 comp rx=0 tx=0).stop 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/4206523047 shutdown_connections 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560087a4cf00 0x560087a4f350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a3d370 0x560087a3a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a33b40 0x560087a3a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/4206523047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a325d0 0x560087a3cd60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb692ffd640 0 client.26579 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:43:10.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/4206523047 >> 172.21.15.133:0/4206523047 conn(0x5600879fb310 msgr2=0x560087a34960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:10.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/4206523047 shutdown_connections 2024-09-11T23:43:10.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/4206523047 wait complete. 2024-09-11T23:43:10.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.234+0000 7fb6bbf7c740 20 client.26579 trim_cache size 0 max 16384 2024-09-11T23:43:10.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6bbf7c740 1 Processor -- start 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6bbf7c740 1 -- start start 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a325d0 0x560087a3bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 0x560087a35120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a35660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b41af760 con 0x560087a3d370 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b41b12a0 con 0x560087a33b40 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b41afe30 con 0x560087a325d0 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a35660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a35660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41968/0 (socket says 172.21.15.133:41968) 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6b3fff640 1 -- 172.21.15.133:0/1965351409 learned_addr learned my addr 172.21.15.133:0/1965351409 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6b37fe640 1 --2- 172.21.15.133:0/1965351409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a325d0 0x560087a3bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/1965351409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 0x560087a35120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.248+0000 7fb6b3fff640 1 -- 172.21.15.133:0/1965351409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a325d0 msgr2=0x560087a3bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:10.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.249+0000 7fb6b3fff640 1 --2- 172.21.15.133:0/1965351409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a325d0 0x560087a3bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.249+0000 7fb6b3fff640 1 -- 172.21.15.133:0/1965351409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 msgr2=0x560087a35120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:10.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.249+0000 7fb6b3fff640 1 --2- 172.21.15.133:0/1965351409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 0x560087a35120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.249+0000 7fb6b3fff640 1 -- 172.21.15.133:0/1965351409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560087a2f960 con 0x560087a3d370 2024-09-11T23:43:10.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.249+0000 7fb6b37fe640 1 --2- 172.21.15.133:0/1965351409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a325d0 0x560087a3bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:10.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.249+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/1965351409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 0x560087a35120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:10.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.249+0000 7fb6b3fff640 1 --2- 172.21.15.133:0/1965351409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a35660 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0bc580 tx=0x7fb6ac0b4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d079f654ac56345 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:10.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.249+0000 7fb6997fa640 1 -- 172.21.15.133:0/1965351409 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0bc880 con 0x560087a3d370 2024-09-11T23:43:10.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.249+0000 7fb6997fa640 1 -- 172.21.15.133:0/1965351409 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6ac0c0c30 con 0x560087a3d370 2024-09-11T23:43:10.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.249+0000 7fb6997fa640 1 -- 172.21.15.133:0/1965351409 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0edc20 con 0x560087a3d370 2024-09-11T23:43:10.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.250+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/1965351409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 msgr2=0x560087a35660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:10.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.250+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/1965351409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a35660 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0bc580 tx=0x7fb6ac0b4040 comp rx=0 tx=0).stop 2024-09-11T23:43:10.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.250+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/1965351409 shutdown_connections 2024-09-11T23:43:10.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.250+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/1965351409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a35660 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.250+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/1965351409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a33b40 0x560087a35120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.250+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/1965351409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a325d0 0x560087a3bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.250+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/1965351409 >> 172.21.15.133:0/1965351409 conn(0x5600879fb310 msgr2=0x560087a3b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.250+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/1965351409 shutdown_connections 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.250+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/1965351409 wait complete. 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.250+0000 7fb6bbf7c740 1 -- start start 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.250+0000 7fb6bbf7c740 1 Processor -- start 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a325d0 0x560087a45bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a33b40 0x560087a36e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6bbf7c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b41b0bd0 con 0x560087a3d370 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b41b0500 con 0x560087a325d0 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6bbf7c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b41ad740 con 0x560087a33b40 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6bbf7c740 10 client.0 _ll_register_callbacks cb 0x7fb6a40042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a373a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41974/0 (socket says 172.21.15.133:41974) 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6b3fff640 1 -- 172.21.15.133:0/766848295 learned_addr learned my addr 172.21.15.133:0/766848295 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6b37fe640 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a325d0 0x560087a45bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6b2ffd640 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a33b40 0x560087a36e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6b3fff640 1 -- 172.21.15.133:0/766848295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a33b40 msgr2=0x560087a36e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6b3fff640 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a33b40 0x560087a36e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6b3fff640 1 -- 172.21.15.133:0/766848295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a325d0 msgr2=0x560087a45bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6b3fff640 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a325d0 0x560087a45bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6b3fff640 1 -- 172.21.15.133:0/766848295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560087a37bd0 con 0x560087a3d370 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6b3fff640 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a373a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0edb60 tx=0x7fb6ac0c3b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96709c839ca6ee5e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb692ffd640 1 -- 172.21.15.133:0/766848295 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0d4070 con 0x560087a3d370 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb692ffd640 1 -- 172.21.15.133:0/766848295 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fb6ac0bab80 con 0x560087a3d370 2024-09-11T23:43:10.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/766848295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6ac0c0c30 con 0x560087a3d370 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.251+0000 7fb692ffd640 1 -- 172.21.15.133:0/766848295 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb6ac0bc880 con 0x560087a3d370 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.252+0000 7fb6bbf7c740 10 client.17097 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.252+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/766848295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560087ab7c90 con 0x560087a3d370 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.252+0000 7fb692ffd640 1 -- 172.21.15.133:0/766848295 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6ac0c0c30 con 0x560087a3d370 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.252+0000 7fb692ffd640 1 -- 172.21.15.133:0/766848295 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb6ac04d030 con 0x560087a3d370 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.252+0000 7fb6bbf7c740 10 client.17097 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.252+0000 7fb6bbf7c740 10 client.17097 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.252+0000 7fb6bbf7c740 20 client.17097 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.252+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560087ab8300 0x560087aba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.252+0000 7fb6bbf7c740 4 client.17097 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.252+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/766848295 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56008799fd70 con 0x560087ab8300 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.252+0000 7fb6b37fe640 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560087ab8300 0x560087aba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb692ffd640 10 client.17097 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:43:10.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6b37fe640 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560087ab8300 0x560087aba6f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef4820191349d9ce server_cookie=8ef2161a334c0faf in_seq=0 out_seq=0 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb692ffd640 1 -- 172.21.15.133:0/766848295 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56008799fd70 con 0x560087ab8300 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb692ffd640 10 client.17097 handle_command_reply: tid=0 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 1 client.17097 shutdown 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/766848295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 msgr2=0x560087a373a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a373a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0edb60 tx=0x7fb6ac0c3b60 comp rx=0 tx=0).stop 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/766848295 shutdown_connections 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560087ab8300 0x560087aba6f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560087a3d370 0x560087a373a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560087a33b40 0x560087a36e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 1 --2- 172.21.15.133:0/766848295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560087a325d0 0x560087a45bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/766848295 >> 172.21.15.133:0/766848295 conn(0x5600879fb310 msgr2=0x560087a3c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb692ffd640 0 client.17097 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/766848295 shutdown_connections 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 1 -- 172.21.15.133:0/766848295 wait complete. 2024-09-11T23:43:10.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.253+0000 7fb6bbf7c740 20 client.17097 trim_cache size 0 max 16384 2024-09-11T23:43:10.259 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:43:10.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/3687185630 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb69c0a2cf0 msgr2=0x7fb69c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:10.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb69c0a2cf0 0x7fb69c0a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800bfd0 tx=0x7fb6a8016000 comp rx=0 tx=0).stop 2024-09-11T23:43:10.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/3687185630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b4102ef0 msgr2=0x7fb6b419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:10.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b4102ef0 0x7fb6b419ef10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac02ae70 tx=0x7fb6ac02f5d0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/3687185630 shutdown_connections 2024-09-11T23:43:10.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb69c0a2cf0 0x7fb69c0a51b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b4102ef0 0x7fb6b419ef10 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b4102570 0x7fb6b419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 --2- 172.21.15.133:0/3687185630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4100e20 0x7fb6b419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:10.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/3687185630 >> 172.21.15.133:0/3687185630 conn(0x7fb6b40fc7d0 msgr2=0x7fb6b406a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:10.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/3687185630 shutdown_connections 2024-09-11T23:43:10.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:10.254+0000 7fb6b9ba3640 1 -- 172.21.15.133:0/3687185630 wait complete. 2024-09-11T23:43:10.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:10 smithi178 ceph-mon[35882]: pgmap v195: 97 pgs: 97 active+clean; 50 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 250 MiB/s wr, 827 op/s 2024-09-11T23:43:10.873 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:10 smithi133 ceph-mon[31430]: pgmap v195: 97 pgs: 97 active+clean; 50 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 250 MiB/s wr, 827 op/s 2024-09-11T23:43:10.937 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:43:10.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:10 smithi138 ceph-mon[35142]: pgmap v195: 97 pgs: 97 active+clean; 50 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 250 MiB/s wr, 827 op/s 2024-09-11T23:43:11.194 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:11.297 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:43:11.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.742+0000 7ff63a184640 1 -- 172.21.15.133:0/721269859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6341082a0 msgr2=0x7ff63410ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:11.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.742+0000 7ff63a184640 1 --2- 172.21.15.133:0/721269859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6341082a0 0x7ff63410ec10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009cd0 tx=0x7ff62c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.742+0000 7ff63a184640 1 -- 172.21.15.133:0/721269859 shutdown_connections 2024-09-11T23:43:11.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.742+0000 7ff63a184640 1 --2- 172.21.15.133:0/721269859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6341082a0 0x7ff63410ec10 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.742+0000 7ff63a184640 1 --2- 172.21.15.133:0/721269859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff634103720 0x7ff634107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.742+0000 7ff63a184640 1 --2- 172.21.15.133:0/721269859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff634102d70 0x7ff634103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.742+0000 7ff63a184640 1 -- 172.21.15.133:0/721269859 >> 172.21.15.133:0/721269859 conn(0x7ff6340fc7d0 msgr2=0x7ff6340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:11.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.742+0000 7ff63a184640 1 -- 172.21.15.133:0/721269859 shutdown_connections 2024-09-11T23:43:11.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.742+0000 7ff63a184640 1 -- 172.21.15.133:0/721269859 wait complete. 2024-09-11T23:43:11.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.742+0000 7ff63a184640 1 Processor -- start 2024-09-11T23:43:11.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.743+0000 7ff63a184640 1 -- start start 2024-09-11T23:43:11.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.743+0000 7ff63a184640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff634102d70 0x7ff634198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.743+0000 7ff63a184640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff634103720 0x7ff634198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.743+0000 7ff63a184640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6341082a0 0x7ff63419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.743+0000 7ff63a184640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341120c0 con 0x7ff6341082a0 2024-09-11T23:43:11.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.743+0000 7ff63a184640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff634111f40 con 0x7ff634102d70 2024-09-11T23:43:11.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.743+0000 7ff63a184640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff634112240 con 0x7ff634103720 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.743+0000 7ff633fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6341082a0 0x7ff63419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.743+0000 7ff633fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6341082a0 0x7ff63419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42004/0 (socket says 172.21.15.133:42004) 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.743+0000 7ff633fff640 1 -- 172.21.15.133:0/3003928672 learned_addr learned my addr 172.21.15.133:0/3003928672 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.743+0000 7ff6337fe640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff634102d70 0x7ff634198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.744+0000 7ff633fff640 1 -- 172.21.15.133:0/3003928672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff634103720 msgr2=0x7ff634198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.744+0000 7ff633fff640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff634103720 0x7ff634198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.744+0000 7ff633fff640 1 -- 172.21.15.133:0/3003928672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff634102d70 msgr2=0x7ff634198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.744+0000 7ff633fff640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff634102d70 0x7ff634198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.744+0000 7ff633fff640 1 -- 172.21.15.133:0/3003928672 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63419d530 con 0x7ff6341082a0 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.744+0000 7ff633fff640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6341082a0 0x7ff63419ce50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff62c005cc0 tx=0x7ff62c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.744+0000 7ff630ff9640 1 -- 172.21.15.133:0/3003928672 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c004040 con 0x7ff6341082a0 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.744+0000 7ff630ff9640 1 -- 172.21.15.133:0/3003928672 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff62c0046c0 con 0x7ff6341082a0 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.744+0000 7ff630ff9640 1 -- 172.21.15.133:0/3003928672 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c030e00 con 0x7ff6341082a0 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.744+0000 7ff63a184640 1 -- 172.21.15.133:0/3003928672 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63419d7c0 con 0x7ff6341082a0 2024-09-11T23:43:11.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.744+0000 7ff63a184640 1 -- 172.21.15.133:0/3003928672 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63419dc90 con 0x7ff6341082a0 2024-09-11T23:43:11.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.747+0000 7ff630ff9640 1 -- 172.21.15.133:0/3003928672 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff62c0305a0 con 0x7ff6341082a0 2024-09-11T23:43:11.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.747+0000 7ff63c55d740 1 Processor -- start 2024-09-11T23:43:11.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.747+0000 7ff630ff9640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff61c0a2d70 0x7ff61c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.747+0000 7ff63c55d740 1 -- start start 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff632ffd640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff61c0a2d70 0x7ff61c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff630ff9640 1 -- 172.21.15.133:0/3003928672 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7ff62c02e030 con 0x7ff6341082a0 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409dab40 0x55a7409d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409e4370 0x55a7409e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff63c55d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a74093f2f0 con 0x55a7409d95d0 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff63c55d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7408e6cf0 con 0x55a7409e4370 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff63c55d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a74093b2b0 con 0x55a7409dab40 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409d9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42006/0 (socket says 172.21.15.133:42006) 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff632ffd640 1 -- 172.21.15.133:0/3986301702 learned_addr learned my addr 172.21.15.133:0/3986301702 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff633fff640 1 --2- 172.21.15.133:0/3986301702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409e4370 0x55a7409e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff6337fe640 1 --2- 172.21.15.133:0/3986301702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409dab40 0x55a7409d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff632ffd640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff61c0a2d70 0x7ff61c0a5230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff628009eb0 tx=0x7ff6280127c0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:11.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff632ffd640 1 -- 172.21.15.133:0/3986301702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409dab40 msgr2=0x55a7409d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:11.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff632ffd640 1 --2- 172.21.15.133:0/3986301702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409dab40 0x55a7409d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff632ffd640 1 -- 172.21.15.133:0/3986301702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409e4370 msgr2=0x55a7409e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:11.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff632ffd640 1 --2- 172.21.15.133:0/3986301702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409e4370 0x55a7409e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.748+0000 7ff632ffd640 1 -- 172.21.15.133:0/3986301702 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7408cb3a0 con 0x55a7409d95d0 2024-09-11T23:43:11.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.749+0000 7ff632ffd640 1 --2- 172.21.15.133:0/3986301702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409d9a10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff62801d610 tx=0x7ff62801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d8412881ddeebd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:11.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.749+0000 7ff619ffb640 1 -- 172.21.15.133:0/3986301702 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628005220 con 0x55a7409d95d0 2024-09-11T23:43:11.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.749+0000 7ff619ffb640 1 -- 172.21.15.133:0/3986301702 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff628004c80 con 0x55a7409d95d0 2024-09-11T23:43:11.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.749+0000 7ff619ffb640 1 -- 172.21.15.133:0/3986301702 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62801ea90 con 0x55a7409d95d0 2024-09-11T23:43:11.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.750+0000 7ff63c55d740 1 -- 172.21.15.133:0/3986301702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 msgr2=0x55a7409d9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:11.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.750+0000 7ff63c55d740 1 --2- 172.21.15.133:0/3986301702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409d9a10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff62801d610 tx=0x7ff62801d000 comp rx=0 tx=0).stop 2024-09-11T23:43:11.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.750+0000 7ff63c55d740 1 -- 172.21.15.133:0/3986301702 shutdown_connections 2024-09-11T23:43:11.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.750+0000 7ff63c55d740 1 --2- 172.21.15.133:0/3986301702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409e4370 0x55a7409e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.750+0000 7ff63c55d740 1 --2- 172.21.15.133:0/3986301702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409d9a10 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.750+0000 7ff63c55d740 1 --2- 172.21.15.133:0/3986301702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409dab40 0x55a7409d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.750+0000 7ff63c55d740 1 -- 172.21.15.133:0/3986301702 >> 172.21.15.133:0/3986301702 conn(0x55a7409a2310 msgr2=0x55a7409d5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:11.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.750+0000 7ff63c55d740 1 -- 172.21.15.133:0/3986301702 shutdown_connections 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.750+0000 7ff63c55d740 1 -- 172.21.15.133:0/3986301702 wait complete. 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.750+0000 7ff63c55d740 1 -- start start 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff63c55d740 1 Processor -- start 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409d95d0 0x55a7409e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409dab40 0x55a7409e1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409e4370 0x55a7409e17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff63c55d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a7409eb420 con 0x55a7409e4370 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff63c55d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7408fa060 con 0x55a7409d95d0 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff63c55d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a7409eb5a0 con 0x55a7409dab40 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff63c55d740 10 client.0 _ll_register_callbacks cb 0x55a74064d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409d95d0 0x55a7409e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff633fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409e4370 0x55a7409e17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409d95d0 0x55a7409e3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:50490/0 (socket says 172.21.15.133:50490) 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff6337fe640 1 -- 172.21.15.133:0/1325173664 learned_addr learned my addr 172.21.15.133:0/1325173664 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff632ffd640 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409dab40 0x55a7409e1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff6337fe640 1 -- 172.21.15.133:0/1325173664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409dab40 msgr2=0x55a7409e1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff6337fe640 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409dab40 0x55a7409e1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff6337fe640 1 -- 172.21.15.133:0/1325173664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409e4370 msgr2=0x55a7409e17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff6337fe640 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409e4370 0x55a7409e17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.751+0000 7ff6337fe640 1 -- 172.21.15.133:0/1325173664 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7406b03c0 con 0x55a7409d95d0 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff6337fe640 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409d95d0 0x55a7409e3da0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff62400b920 tx=0x7ff62400bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c92cd67c18bad2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff612ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff612ffd640 1 -- 172.21.15.133:0/1325173664 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62401b440 con 0x55a7409d95d0 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff63c55d740 1 -- 172.21.15.133:0/1325173664 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff628004c80 con 0x55a7409d95d0 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff612ffd640 1 -- 172.21.15.133:0/1325173664 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7ff624011440 con 0x55a7409d95d0 2024-09-11T23:43:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff612ffd640 1 -- 172.21.15.133:0/1325173664 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff624011640 con 0x55a7409d95d0 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff63c55d740 10 client.26597 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff63c55d740 1 -- 172.21.15.133:0/1325173664 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a7409f6f60 con 0x55a7409d95d0 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff612ffd640 1 -- 172.21.15.133:0/1325173664 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff628004c80 con 0x55a7409d95d0 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff612ffd640 1 -- 172.21.15.133:0/1325173664 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff624019020 con 0x55a7409d95d0 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff63c55d740 10 client.26597 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff63c55d740 10 client.26597 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff63c55d740 20 client.26597 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff63c55d740 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a7409f8480 0x55a7409fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff63c55d740 4 client.26597 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.752+0000 7ff63c55d740 1 -- 172.21.15.133:0/1325173664 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a74094c860 con 0x55a7409f8480 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.753+0000 7ff632ffd640 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a7409f8480 0x55a7409fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.753+0000 7ff612ffd640 10 client.26597 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.753+0000 7ff632ffd640 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a7409f8480 0x55a7409fa8b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45b6c7eeaf26248c server_cookie=7c83c47beba407cf in_seq=0 out_seq=0 2024-09-11T23:43:11.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff612ffd640 1 -- 172.21.15.133:0/1325173664 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a74094c860 con 0x55a7409f8480 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff612ffd640 10 client.26597 handle_command_reply: tid=0 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 1 client.26597 shutdown 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 1 -- 172.21.15.133:0/1325173664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409d95d0 msgr2=0x55a7409e3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409d95d0 0x55a7409e3da0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff62400b920 tx=0x7ff62400bd20 comp rx=0 tx=0).stop 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 1 -- 172.21.15.133:0/1325173664 shutdown_connections 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a7409f8480 0x55a7409fa8b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409e4370 0x55a7409e17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409dab40 0x55a7409e1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff612ffd640 0 client.26597 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 1 --2- 172.21.15.133:0/1325173664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409d95d0 0x55a7409e3da0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 1 -- 172.21.15.133:0/1325173664 >> 172.21.15.133:0/1325173664 conn(0x55a7409a2310 msgr2=0x55a7409db960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 1 -- 172.21.15.133:0/1325173664 shutdown_connections 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 1 -- 172.21.15.133:0/1325173664 wait complete. 2024-09-11T23:43:11.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.754+0000 7ff63c55d740 20 client.26597 trim_cache size 0 max 16384 2024-09-11T23:43:11.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.767+0000 7ff63c55d740 1 Processor -- start 2024-09-11T23:43:11.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.767+0000 7ff63c55d740 1 -- start start 2024-09-11T23:43:11.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409dc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409dab40 0x55a7409dc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409e4370 0x55a740a00050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff63c55d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341ab5c0 con 0x55a7409d95d0 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff63c55d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341ad100 con 0x55a7409dab40 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff63c55d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341abc90 con 0x55a7409e4370 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409dc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409dc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42028/0 (socket says 172.21.15.133:42028) 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff6337fe640 1 -- 172.21.15.133:0/1133602787 learned_addr learned my addr 172.21.15.133:0/1133602787 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff632ffd640 1 --2- 172.21.15.133:0/1133602787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409dab40 0x55a7409dc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff6337fe640 1 -- 172.21.15.133:0/1133602787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409e4370 msgr2=0x55a740a00050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff6337fe640 1 --2- 172.21.15.133:0/1133602787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409e4370 0x55a740a00050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff6337fe640 1 -- 172.21.15.133:0/1133602787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409dab40 msgr2=0x55a7409dc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff6337fe640 1 --2- 172.21.15.133:0/1133602787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409dab40 0x55a7409dc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.768+0000 7ff6337fe640 1 -- 172.21.15.133:0/1133602787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7409df5a0 con 0x55a7409d95d0 2024-09-11T23:43:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.769+0000 7ff6337fe640 1 --2- 172.21.15.133:0/1133602787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409dc120 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c010 tx=0x7ff6240244f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfe770482787477 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.769+0000 7ff6197fa640 1 -- 172.21.15.133:0/1133602787 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624003de0 con 0x55a7409d95d0 2024-09-11T23:43:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.769+0000 7ff6197fa640 1 -- 172.21.15.133:0/1133602787 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff624005040 con 0x55a7409d95d0 2024-09-11T23:43:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.769+0000 7ff6197fa640 1 -- 172.21.15.133:0/1133602787 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624011a20 con 0x55a7409d95d0 2024-09-11T23:43:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.770+0000 7ff63c55d740 1 -- 172.21.15.133:0/1133602787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 msgr2=0x55a7409dc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.770+0000 7ff63c55d740 1 --2- 172.21.15.133:0/1133602787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409dc120 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c010 tx=0x7ff6240244f0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.770+0000 7ff63c55d740 1 -- 172.21.15.133:0/1133602787 shutdown_connections 2024-09-11T23:43:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.770+0000 7ff63c55d740 1 --2- 172.21.15.133:0/1133602787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409e4370 0x55a740a00050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.770+0000 7ff63c55d740 1 --2- 172.21.15.133:0/1133602787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409dab40 0x55a7409dc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.770+0000 7ff63c55d740 1 --2- 172.21.15.133:0/1133602787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409d95d0 0x55a7409dc120 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.770+0000 7ff63c55d740 1 -- 172.21.15.133:0/1133602787 >> 172.21.15.133:0/1133602787 conn(0x55a7409a2310 msgr2=0x55a7409d4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.770+0000 7ff63c55d740 1 -- 172.21.15.133:0/1133602787 shutdown_connections 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.770+0000 7ff63c55d740 1 -- 172.21.15.133:0/1133602787 wait complete. 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.770+0000 7ff63c55d740 1 -- start start 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.770+0000 7ff63c55d740 1 Processor -- start 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409d95d0 0x55a7409f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409dab40 0x55a7409f0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff63c55d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409e4370 0x55a7409ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff63c55d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341aca30 con 0x55a7409dab40 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff63c55d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341ac360 con 0x55a7409e4370 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff63c55d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a95c0 con 0x55a7409d95d0 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff63c55d740 10 client.0 _ll_register_callbacks cb 0x7ff624004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409d95d0 0x55a7409f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409d95d0 0x55a7409f0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48750/0 (socket says 172.21.15.133:48750) 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff6337fe640 1 -- 172.21.15.133:0/3293913751 learned_addr learned my addr 172.21.15.133:0/3293913751 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff633fff640 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409e4370 0x55a7409ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff632ffd640 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409dab40 0x55a7409f0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff6337fe640 1 -- 172.21.15.133:0/3293913751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409e4370 msgr2=0x55a7409ed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff6337fe640 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409e4370 0x55a7409ed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff6337fe640 1 -- 172.21.15.133:0/3293913751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409dab40 msgr2=0x55a7409f0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff6337fe640 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409dab40 0x55a7409f0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff6337fe640 1 -- 172.21.15.133:0/3293913751 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7409edef0 con 0x55a7409d95d0 2024-09-11T23:43:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.771+0000 7ff632ffd640 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409dab40 0x55a7409f0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff6337fe640 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409d95d0 0x55a7409f0a10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff62401bc20 tx=0x7ff624005cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7636f90c5167c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff612ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:43:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff612ffd640 1 -- 172.21.15.133:0/3293913751 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62403a060 con 0x55a7409d95d0 2024-09-11T23:43:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff612ffd640 1 -- 172.21.15.133:0/3293913751 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7ff624005400 con 0x55a7409d95d0 2024-09-11T23:43:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff63c55d740 1 -- 172.21.15.133:0/3293913751 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff624005040 con 0x55a7409d95d0 2024-09-11T23:43:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff612ffd640 1 -- 172.21.15.133:0/3293913751 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff624005600 con 0x55a7409d95d0 2024-09-11T23:43:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff63c55d740 10 client.26518 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff63c55d740 1 -- 172.21.15.133:0/3293913751 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a740a63990 con 0x55a7409d95d0 2024-09-11T23:43:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff612ffd640 1 -- 172.21.15.133:0/3293913751 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff624005040 con 0x55a7409d95d0 2024-09-11T23:43:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff612ffd640 1 -- 172.21.15.133:0/3293913751 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff624019030 con 0x55a7409d95d0 2024-09-11T23:43:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff63c55d740 10 client.26518 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff63c55d740 10 client.26518 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:43:11.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff63c55d740 20 client.26518 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.772+0000 7ff63c55d740 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a740a64040 0x55a740a66430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.773+0000 7ff63c55d740 4 client.26518 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.773+0000 7ff63c55d740 1 -- 172.21.15.133:0/3293913751 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a740946d70 con 0x55a740a64040 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.773+0000 7ff632ffd640 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a740a64040 0x55a740a66430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.773+0000 7ff612ffd640 10 client.26518 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.773+0000 7ff632ffd640 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a740a64040 0x55a740a66430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=75d527bd22f4f4be server_cookie=18398d2073dac765 in_seq=0 out_seq=0 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.773+0000 7ff612ffd640 1 -- 172.21.15.133:0/3293913751 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a740946d70 con 0x55a740a64040 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.773+0000 7ff612ffd640 10 client.26518 handle_command_reply: tid=0 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.773+0000 7ff63c55d740 1 client.26518 shutdown 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff63c55d740 1 -- 172.21.15.133:0/3293913751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409d95d0 msgr2=0x55a7409f0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff63c55d740 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409d95d0 0x55a7409f0a10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff62401bc20 tx=0x7ff624005cb0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff63c55d740 1 -- 172.21.15.133:0/3293913751 shutdown_connections 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff63c55d740 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55a740a64040 0x55a740a66430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff63c55d740 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7409e4370 0x55a7409ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff63c55d740 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7409dab40 0x55a7409f0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff612ffd640 0 client.26518 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:43:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff63c55d740 1 --2- 172.21.15.133:0/3293913751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7409d95d0 0x55a7409f0a10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff63c55d740 1 -- 172.21.15.133:0/3293913751 >> 172.21.15.133:0/3293913751 conn(0x55a7409a2310 msgr2=0x55a7409f36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff63c55d740 1 -- 172.21.15.133:0/3293913751 shutdown_connections 2024-09-11T23:43:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff63c55d740 1 -- 172.21.15.133:0/3293913751 wait complete. 2024-09-11T23:43:11.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:11 smithi133 ceph-mon[31430]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:43:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.774+0000 7ff63c55d740 20 client.26518 trim_cache size 0 max 16384 2024-09-11T23:43:11.780 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:43:11.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.775+0000 7ff63a184640 1 -- 172.21.15.133:0/3003928672 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff61c0a2d70 msgr2=0x7ff61c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:11.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.775+0000 7ff63a184640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff61c0a2d70 0x7ff61c0a5230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff628009eb0 tx=0x7ff6280127c0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.775+0000 7ff63a184640 1 -- 172.21.15.133:0/3003928672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6341082a0 msgr2=0x7ff63419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:11.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.775+0000 7ff63a184640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6341082a0 0x7ff63419ce50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff62c005cc0 tx=0x7ff62c032f40 comp rx=0 tx=0).stop 2024-09-11T23:43:11.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.775+0000 7ff63a184640 1 -- 172.21.15.133:0/3003928672 shutdown_connections 2024-09-11T23:43:11.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.775+0000 7ff63a184640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff61c0a2d70 0x7ff61c0a5230 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.775+0000 7ff63a184640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6341082a0 0x7ff63419ce50 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.775+0000 7ff63a184640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff634103720 0x7ff634198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.775+0000 7ff63a184640 1 --2- 172.21.15.133:0/3003928672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff634102d70 0x7ff634198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:11.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.775+0000 7ff63a184640 1 -- 172.21.15.133:0/3003928672 >> 172.21.15.133:0/3003928672 conn(0x7ff6340fc7d0 msgr2=0x7ff634106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:11.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.776+0000 7ff63a184640 1 -- 172.21.15.133:0/3003928672 shutdown_connections 2024-09-11T23:43:11.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:11.776+0000 7ff63a184640 1 -- 172.21.15.133:0/3003928672 wait complete. 2024-09-11T23:43:11.782 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:11.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:11 smithi178 ceph-mon[35882]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:43:11.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:11 smithi138 ceph-mon[35142]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:43:12.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:12 smithi178 ceph-mon[35882]: pgmap v197: 97 pgs: 97 active+clean; 50 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 179 KiB/s rd, 76 MiB/s wr, 126 op/s 2024-09-11T23:43:12.815 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:12 smithi133 ceph-mon[31430]: pgmap v197: 97 pgs: 97 active+clean; 50 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 179 KiB/s rd, 76 MiB/s wr, 126 op/s 2024-09-11T23:43:12.817 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:43:12.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:12 smithi138 ceph-mon[35142]: pgmap v197: 97 pgs: 97 active+clean; 50 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 179 KiB/s rd, 76 MiB/s wr, 126 op/s 2024-09-11T23:43:13.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.226+0000 7f8cba57f640 1 -- 172.21.15.133:0/2988225607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cb41068d0 msgr2=0x7f8cb4106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:13.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.226+0000 7f8cba57f640 1 --2- 172.21.15.133:0/2988225607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cb41068d0 0x7f8cb4106cb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8cb001a9c0 tx=0x7f8cb003d680 comp rx=0 tx=0).stop 2024-09-11T23:43:13.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.227+0000 7f8cba57f640 1 -- 172.21.15.133:0/2988225607 shutdown_connections 2024-09-11T23:43:13.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.227+0000 7f8cba57f640 1 --2- 172.21.15.133:0/2988225607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cb4101270 0x7f8cb410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:13.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.227+0000 7f8cba57f640 1 --2- 172.21.15.133:0/2988225607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cb41008d0 0x7f8cb4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:13.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.227+0000 7f8cba57f640 1 --2- 172.21.15.133:0/2988225607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cb41068d0 0x7f8cb4106cb0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:13.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.227+0000 7f8cba57f640 1 -- 172.21.15.133:0/2988225607 >> 172.21.15.133:0/2988225607 conn(0x7f8cb40fc610 msgr2=0x7f8cb40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:13.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.227+0000 7f8cba57f640 1 -- 172.21.15.133:0/2988225607 shutdown_connections 2024-09-11T23:43:13.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.227+0000 7f8cba57f640 1 -- 172.21.15.133:0/2988225607 wait complete. 2024-09-11T23:43:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.228+0000 7f8cba57f640 1 Processor -- start 2024-09-11T23:43:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.228+0000 7f8cba57f640 1 -- start start 2024-09-11T23:43:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.228+0000 7f8cba57f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cb41008d0 0x7f8cb410c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.228+0000 7f8cba57f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cb4101270 0x7f8cb410ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.228+0000 7f8cba57f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cb41068d0 0x7f8cb4196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.228+0000 7f8cba57f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb40718d0 con 0x7f8cb41008d0 2024-09-11T23:43:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.228+0000 7f8cba57f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb4071750 con 0x7f8cb4101270 2024-09-11T23:43:13.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.228+0000 7f8cba57f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb4071a50 con 0x7f8cb41068d0 2024-09-11T23:43:13.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.229+0000 7f8cb957d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cb41008d0 0x7f8cb410c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:13.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.229+0000 7f8cb957d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cb41008d0 0x7f8cb410c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46582/0 (socket says 172.21.15.133:46582) 2024-09-11T23:43:13.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.229+0000 7f8cb957d640 1 -- 172.21.15.133:0/2651291707 learned_addr learned my addr 172.21.15.133:0/2651291707 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:13.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.229+0000 7f8cb8d7c640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cb4101270 0x7f8cb410ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:13.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.230+0000 7f8cb957d640 1 -- 172.21.15.133:0/2651291707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cb41068d0 msgr2=0x7f8cb4196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:13.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.230+0000 7f8cb957d640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cb41068d0 0x7f8cb4196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:13.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.230+0000 7f8cb957d640 1 -- 172.21.15.133:0/2651291707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cb4101270 msgr2=0x7f8cb410ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:13.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.230+0000 7f8cb957d640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cb4101270 0x7f8cb410ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:13.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.230+0000 7f8cb957d640 1 -- 172.21.15.133:0/2651291707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb4196cf0 con 0x7f8cb41008d0 2024-09-11T23:43:13.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.230+0000 7f8cb957d640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cb41008d0 0x7f8cb410c930 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f8cb001f4c0 tx=0x7f8cb00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:13.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.231+0000 7f8caa7fc640 1 -- 172.21.15.133:0/2651291707 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0050020 con 0x7f8cb41008d0 2024-09-11T23:43:13.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.231+0000 7f8caa7fc640 1 -- 172.21.15.133:0/2651291707 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb0004ca0 con 0x7f8cb41008d0 2024-09-11T23:43:13.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.231+0000 7f8caa7fc640 1 -- 172.21.15.133:0/2651291707 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0048030 con 0x7f8cb41008d0 2024-09-11T23:43:13.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.231+0000 7f8cba57f640 1 -- 172.21.15.133:0/2651291707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb4196f80 con 0x7f8cb41008d0 2024-09-11T23:43:13.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.232+0000 7f8cba57f640 1 -- 172.21.15.133:0/2651291707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb4197350 con 0x7f8cb41008d0 2024-09-11T23:43:13.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.233+0000 7f8cba57f640 1 -- 172.21.15.133:0/2651291707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb4100d30 con 0x7f8cb41008d0 2024-09-11T23:43:13.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.235+0000 7f8caa7fc640 1 -- 172.21.15.133:0/2651291707 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8cb0053020 con 0x7f8cb41008d0 2024-09-11T23:43:13.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.236+0000 7f8caa7fc640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ca00a2e40 0x7f8ca00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:13.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.236+0000 7f8cb8d7c640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ca00a2e40 0x7f8ca00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:13.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.236+0000 7f8caa7fc640 1 -- 172.21.15.133:0/2651291707 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7f8cb0100b00 con 0x7f8cb41008d0 2024-09-11T23:43:13.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.237+0000 7f8cb8d7c640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ca00a2e40 0x7f8ca00a5300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40fef90 tx=0x7f8cac0094f0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.246+0000 7f8caa7fc640 1 -- 172.21.15.133:0/2651291707 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cb005a020 con 0x7f8cb41008d0 2024-09-11T23:43:13.317 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:13.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.474+0000 7f8cba57f640 1 -- 172.21.15.133:0/2651291707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cb419ea80 con 0x7f8cb41008d0 2024-09-11T23:43:13.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.475+0000 7f8caa7fc640 1 -- 172.21.15.133:0/2651291707 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7f8cb00cd6d0 con 0x7f8cb41008d0 2024-09-11T23:43:13.475 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:43:13.475 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:43:13.475 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:43:13.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.477+0000 7f8cba57f640 1 -- 172.21.15.133:0/2651291707 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ca00a2e40 msgr2=0x7f8ca00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:13.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.477+0000 7f8cba57f640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ca00a2e40 0x7f8ca00a5300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40fef90 tx=0x7f8cac0094f0 comp rx=0 tx=0).stop 2024-09-11T23:43:13.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.477+0000 7f8cba57f640 1 -- 172.21.15.133:0/2651291707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cb41008d0 msgr2=0x7f8cb410c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:13.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.477+0000 7f8cba57f640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cb41008d0 0x7f8cb410c930 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f8cb001f4c0 tx=0x7f8cb00048f0 comp rx=0 tx=0).stop 2024-09-11T23:43:13.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.477+0000 7f8cba57f640 1 -- 172.21.15.133:0/2651291707 shutdown_connections 2024-09-11T23:43:13.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.477+0000 7f8cba57f640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8ca00a2e40 0x7f8ca00a5300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:13.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.477+0000 7f8cba57f640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cb41068d0 0x7f8cb4196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:13.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.477+0000 7f8cba57f640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cb4101270 0x7f8cb410ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:13.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.477+0000 7f8cba57f640 1 --2- 172.21.15.133:0/2651291707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cb41008d0 0x7f8cb410c930 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:13.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.477+0000 7f8cba57f640 1 -- 172.21.15.133:0/2651291707 >> 172.21.15.133:0/2651291707 conn(0x7f8cb40fc610 msgr2=0x7f8cb40725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:13.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.478+0000 7f8cba57f640 1 -- 172.21.15.133:0/2651291707 shutdown_connections 2024-09-11T23:43:13.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:13.478+0000 7f8cba57f640 1 -- 172.21.15.133:0/2651291707 wait complete. 2024-09-11T23:43:13.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:13 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2651291707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:13.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:13 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2651291707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:13.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:13 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2651291707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:14.354 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id e33d2d6a --release --await' 2024-09-11T23:43:14.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id e33d2d6a --release --await 2024-09-11T23:43:14.602 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:14 smithi133 ceph-mon[31430]: pgmap v198: 97 pgs: 97 active+clean; 50 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 23 KiB/s rd, 71 MiB/s wr, 127 op/s 2024-09-11T23:43:14.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.602+0000 7f45fb64a640 1 -- 172.21.15.133:0/1052137290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45f4102d40 msgr2=0x7f45f4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.602+0000 7f45fb64a640 1 --2- 172.21.15.133:0/1052137290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45f4102d40 0x7f45f4103120 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009900 tx=0x7f45f002a480 comp rx=0 tx=0).stop 2024-09-11T23:43:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.603+0000 7f45fb64a640 1 -- 172.21.15.133:0/1052137290 shutdown_connections 2024-09-11T23:43:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.603+0000 7f45fb64a640 1 --2- 172.21.15.133:0/1052137290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45f410c170 0x7f45f410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.603+0000 7f45fb64a640 1 --2- 172.21.15.133:0/1052137290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f4103660 0x7f45f410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.603+0000 7f45fb64a640 1 --2- 172.21.15.133:0/1052137290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45f4102d40 0x7f45f4103120 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.603+0000 7f45fb64a640 1 -- 172.21.15.133:0/1052137290 >> 172.21.15.133:0/1052137290 conn(0x7f45f40fc7a0 msgr2=0x7f45f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.603+0000 7f45fb64a640 1 -- 172.21.15.133:0/1052137290 shutdown_connections 2024-09-11T23:43:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.603+0000 7f45fb64a640 1 -- 172.21.15.133:0/1052137290 wait complete. 2024-09-11T23:43:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45fb64a640 1 Processor -- start 2024-09-11T23:43:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45fb64a640 1 -- start start 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45fb64a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45f4102d40 0x7f45f419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45fb64a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45f4103660 0x7f45f419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45fb64a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f410c170 0x7f45f419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45fb64a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45f4111fc0 con 0x7f45f4102d40 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45fb64a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45f4111e40 con 0x7f45f4103660 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45fb64a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45f4112140 con 0x7f45f410c170 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45f93e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45f4102d40 0x7f45f419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45f93e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45f4102d40 0x7f45f419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46610/0 (socket says 172.21.15.133:46610) 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45f93e6640 1 -- 172.21.15.133:0/657327278 learned_addr learned my addr 172.21.15.133:0/657327278 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45f8be5640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45f4103660 0x7f45f419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.604+0000 7f45f9be7640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f410c170 0x7f45f419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45f93e6640 1 -- 172.21.15.133:0/657327278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f410c170 msgr2=0x7f45f419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45f93e6640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f410c170 0x7f45f419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45f93e6640 1 -- 172.21.15.133:0/657327278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45f4103660 msgr2=0x7f45f419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45f93e6640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45f4103660 0x7f45f419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45f93e6640 1 -- 172.21.15.133:0/657327278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f419f570 con 0x7f45f4102d40 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45f9be7640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f410c170 0x7f45f419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45f93e6640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45f4102d40 0x7f45f419a5c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f45f0004770 tx=0x7f45f0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45ea7fc640 1 -- 172.21.15.133:0/657327278 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f003f070 con 0x7f45f4102d40 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45ea7fc640 1 -- 172.21.15.133:0/657327278 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f0031ca0 con 0x7f45f4102d40 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45fb64a640 1 -- 172.21.15.133:0/657327278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f419f800 con 0x7f45f4102d40 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45ea7fc640 1 -- 172.21.15.133:0/657327278 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0035030 con 0x7f45f4102d40 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.605+0000 7f45fb64a640 1 -- 172.21.15.133:0/657327278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f41a7140 con 0x7f45f4102d40 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.607+0000 7f45ea7fc640 1 -- 172.21.15.133:0/657327278 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f45f0042070 con 0x7f45f4102d40 2024-09-11T23:43:14.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45fda23740 1 Processor -- start 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45fda23740 1 -- start start 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45fda23740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9feab40 0x5612c9fe9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45fda23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9fe9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45fda23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9ff4370 0x5612c9ff6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45fda23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5612c9f4f2f0 con 0x5612c9fe95d0 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45fda23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5612c9ef6cf0 con 0x5612c9ff4370 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45fda23740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5612c9f4b2b0 con 0x5612c9feab40 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45ea7fc640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f45e00a2cf0 0x7f45e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45f93e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9fe9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45f9be7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9feab40 0x5612c9fe9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45f93e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9fe9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46616/0 (socket says 172.21.15.133:46616) 2024-09-11T23:43:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45f93e6640 1 -- 172.21.15.133:0/2917511977 learned_addr learned my addr 172.21.15.133:0/2917511977 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.608+0000 7f45f9be7640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f45e00a2cf0 0x7f45e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45f8be5640 1 --2- 172.21.15.133:0/2917511977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9ff4370 0x5612c9ff6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45ea7fc640 1 -- 172.21.15.133:0/657327278 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7f45f00efc30 con 0x7f45f4102d40 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45f93e6640 1 -- 172.21.15.133:0/2917511977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9feab40 msgr2=0x5612c9fe9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45f93e6640 1 --2- 172.21.15.133:0/2917511977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9feab40 0x5612c9fe9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45f93e6640 1 -- 172.21.15.133:0/2917511977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9ff4370 msgr2=0x5612c9ff6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45f93e6640 1 --2- 172.21.15.133:0/2917511977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9ff4370 0x5612c9ff6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45f93e6640 1 -- 172.21.15.133:0/2917511977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c9edb3a0 con 0x5612c9fe95d0 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45f9be7640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f45e00a2cf0 0x7f45e00a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f45ec005c40 tx=0x7f45ec009a20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45f93e6640 1 --2- 172.21.15.133:0/2917511977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9fe9a10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f45f0000c00 tx=0x7f45f0034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39488c60af03457 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45df7fe640 1 -- 172.21.15.133:0/2917511977 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0049070 con 0x5612c9fe95d0 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45df7fe640 1 -- 172.21.15.133:0/2917511977 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f00bc160 con 0x5612c9fe95d0 2024-09-11T23:43:14.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.609+0000 7f45df7fe640 1 -- 172.21.15.133:0/2917511977 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f00b87d0 con 0x5612c9fe95d0 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.610+0000 7f45fda23740 1 -- 172.21.15.133:0/2917511977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 msgr2=0x5612c9fe9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.610+0000 7f45fda23740 1 --2- 172.21.15.133:0/2917511977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9fe9a10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f45f0000c00 tx=0x7f45f0034000 comp rx=0 tx=0).stop 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.610+0000 7f45fda23740 1 -- 172.21.15.133:0/2917511977 shutdown_connections 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.610+0000 7f45fda23740 1 --2- 172.21.15.133:0/2917511977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9ff4370 0x5612c9ff6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.610+0000 7f45fda23740 1 --2- 172.21.15.133:0/2917511977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9fe9a10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.610+0000 7f45fda23740 1 --2- 172.21.15.133:0/2917511977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9feab40 0x5612c9fe9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.610+0000 7f45fda23740 1 -- 172.21.15.133:0/2917511977 >> 172.21.15.133:0/2917511977 conn(0x5612c9fb2310 msgr2=0x5612c9fe5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.610+0000 7f45fda23740 1 -- 172.21.15.133:0/2917511977 shutdown_connections 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.610+0000 7f45fda23740 1 -- 172.21.15.133:0/2917511977 wait complete. 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.610+0000 7f45fda23740 1 -- start start 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.610+0000 7f45fda23740 1 Processor -- start 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45fda23740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9fe95d0 0x5612c9ff3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45fda23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9feab40 0x5612c9ff1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45fda23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9ff4370 0x5612c9ff17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45fda23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5612c9ff6e30 con 0x5612c9ff4370 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45fda23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5612c9f0a060 con 0x5612c9feab40 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45fda23740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5612c9ff6fb0 con 0x5612c9fe95d0 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45fda23740 10 client.0 _ll_register_callbacks cb 0x5612c9c5d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45f9be7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9ff4370 0x5612c9ff17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45f8be5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9feab40 0x5612c9ff1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45f9be7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9ff4370 0x5612c9ff17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46626/0 (socket says 172.21.15.133:46626) 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45f9be7640 1 -- 172.21.15.133:0/262288083 learned_addr learned my addr 172.21.15.133:0/262288083 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45f93e6640 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9fe95d0 0x5612c9ff3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45f9be7640 1 -- 172.21.15.133:0/262288083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9fe95d0 msgr2=0x5612c9ff3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45f9be7640 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9fe95d0 0x5612c9ff3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45f9be7640 1 -- 172.21.15.133:0/262288083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9feab40 msgr2=0x5612c9ff1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:14.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45f9be7640 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9feab40 0x5612c9ff1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.611+0000 7f45f9be7640 1 -- 172.21.15.133:0/262288083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612c9cc03c0 con 0x5612c9ff4370 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45f9be7640 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9ff4370 0x5612c9ff17e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f45ec010da0 tx=0x7f45ec009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4852b8e6bc88b6e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45dcff9640 1 -- 172.21.15.133:0/262288083 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec010e90 con 0x5612c9ff4370 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45dcff9640 1 -- 172.21.15.133:0/262288083 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7f45ec004b20 con 0x5612c9ff4370 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45fda23740 1 -- 172.21.15.133:0/262288083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612c9f5c860 con 0x5612c9ff4370 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45dcff9640 1 -- 172.21.15.133:0/262288083 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f45ec004d20 con 0x5612c9ff4370 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45fda23740 10 client.17133 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45fda23740 1 -- 172.21.15.133:0/262288083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5612ca002850 con 0x5612c9ff4370 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45dcff9640 1 -- 172.21.15.133:0/262288083 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5612c9f5c860 con 0x5612c9ff4370 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45dcff9640 1 -- 172.21.15.133:0/262288083 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f45ec01f070 con 0x5612c9ff4370 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45fda23740 10 client.17133 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:14.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45fda23740 10 client.17133 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45fda23740 20 client.17133 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45fda23740 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612ca003cb0 0x5612ca0060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45fda23740 4 client.17133 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.612+0000 7f45fda23740 1 -- 172.21.15.133:0/262288083 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612c9f56d70 con 0x5612ca003cb0 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.613+0000 7f45f93e6640 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612ca003cb0 0x5612ca0060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.613+0000 7f45f93e6640 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612ca003cb0 0x5612ca0060f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c1cb3c4d04a246f server_cookie=d83f3471da1eed42 in_seq=0 out_seq=0 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.613+0000 7f45dcff9640 10 client.17133 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45dcff9640 1 -- 172.21.15.133:0/262288083 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612c9f56d70 con 0x5612ca003cb0 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45dcff9640 10 client.17133 handle_command_reply: tid=0 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 1 client.17133 shutdown 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 1 -- 172.21.15.133:0/262288083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9ff4370 msgr2=0x5612c9ff17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9ff4370 0x5612c9ff17e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f45ec010da0 tx=0x7f45ec009700 comp rx=0 tx=0).stop 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 1 -- 172.21.15.133:0/262288083 shutdown_connections 2024-09-11T23:43:14.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612ca003cb0 0x5612ca0060f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9ff4370 0x5612c9ff17e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9feab40 0x5612c9ff1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 1 --2- 172.21.15.133:0/262288083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9fe95d0 0x5612c9ff3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 1 -- 172.21.15.133:0/262288083 >> 172.21.15.133:0/262288083 conn(0x5612c9fb2310 msgr2=0x5612c9feb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45dcff9640 0 client.17133 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 1 -- 172.21.15.133:0/262288083 shutdown_connections 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 1 -- 172.21.15.133:0/262288083 wait complete. 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.614+0000 7f45fda23740 20 client.17133 trim_cache size 0 max 16384 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.627+0000 7f45fda23740 1 Processor -- start 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.627+0000 7f45fda23740 1 -- start start 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45fda23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9fe95d0 0x5612c9fe5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45fda23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9feab40 0x5612c9fe5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45fda23740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9ff4370 0x5612ca00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45fda23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41af9f0 con 0x5612c9feab40 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45fda23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41b1530 con 0x5612c9fe95d0 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45fda23740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41b00c0 con 0x5612c9ff4370 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45f8be5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9feab40 0x5612c9fe5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45f8be5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9feab40 0x5612c9fe5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46632/0 (socket says 172.21.15.133:46632) 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45f8be5640 1 -- 172.21.15.133:0/4143571170 learned_addr learned my addr 172.21.15.133:0/4143571170 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45f93e6640 1 --2- 172.21.15.133:0/4143571170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9fe95d0 0x5612c9fe5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45f9be7640 1 --2- 172.21.15.133:0/4143571170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9ff4370 0x5612ca00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45f8be5640 1 -- 172.21.15.133:0/4143571170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9ff4370 msgr2=0x5612ca00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45f8be5640 1 --2- 172.21.15.133:0/4143571170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9ff4370 0x5612ca00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45f8be5640 1 -- 172.21.15.133:0/4143571170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9fe95d0 msgr2=0x5612c9fe5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45f8be5640 1 --2- 172.21.15.133:0/4143571170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9fe95d0 0x5612c9fe5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45f8be5640 1 -- 172.21.15.133:0/4143571170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c9fe54f0 con 0x5612c9feab40 2024-09-11T23:43:14.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45f8be5640 1 --2- 172.21.15.133:0/4143571170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9feab40 0x5612c9fe5c40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f45e400ad20 tx=0x7f45e400a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d39ba5c7d0fa10e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:14.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45deffd640 1 -- 172.21.15.133:0/4143571170 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e401e030 con 0x5612c9feab40 2024-09-11T23:43:14.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45deffd640 1 -- 172.21.15.133:0/4143571170 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e400ae10 con 0x5612c9feab40 2024-09-11T23:43:14.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.628+0000 7f45deffd640 1 -- 172.21.15.133:0/4143571170 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e401a870 con 0x5612c9feab40 2024-09-11T23:43:14.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.629+0000 7f45fda23740 1 -- 172.21.15.133:0/4143571170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9feab40 msgr2=0x5612c9fe5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.629+0000 7f45fda23740 1 --2- 172.21.15.133:0/4143571170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9feab40 0x5612c9fe5c40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f45e400ad20 tx=0x7f45e400a250 comp rx=0 tx=0).stop 2024-09-11T23:43:14.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.629+0000 7f45fda23740 1 -- 172.21.15.133:0/4143571170 shutdown_connections 2024-09-11T23:43:14.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.629+0000 7f45fda23740 1 --2- 172.21.15.133:0/4143571170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9ff4370 0x5612ca00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.629+0000 7f45fda23740 1 --2- 172.21.15.133:0/4143571170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9feab40 0x5612c9fe5c40 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.629+0000 7f45fda23740 1 --2- 172.21.15.133:0/4143571170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9fe95d0 0x5612c9fe5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.629+0000 7f45fda23740 1 -- 172.21.15.133:0/4143571170 >> 172.21.15.133:0/4143571170 conn(0x5612c9fb2310 msgr2=0x5612c9fec260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.629+0000 7f45fda23740 1 -- 172.21.15.133:0/4143571170 shutdown_connections 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.629+0000 7f45fda23740 1 -- 172.21.15.133:0/4143571170 wait complete. 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.629+0000 7f45fda23740 1 -- start start 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45fda23740 1 Processor -- start 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45fda23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9ffcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45fda23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9feab40 0x5612c9fedee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45fda23740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9ff4370 0x5612c9fee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45fda23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41b0e60 con 0x5612c9fe95d0 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45fda23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41b0790 con 0x5612c9feab40 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45fda23740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41ad9f0 con 0x5612c9ff4370 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45fda23740 10 client.0 _ll_register_callbacks cb 0x7f45ec009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:14.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45f93e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9ffcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45f8be5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9feab40 0x5612c9fedee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45f93e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9ffcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46648/0 (socket says 172.21.15.133:46648) 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45f93e6640 1 -- 172.21.15.133:0/4130640801 learned_addr learned my addr 172.21.15.133:0/4130640801 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45f9be7640 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9ff4370 0x5612c9fee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45f8be5640 1 -- 172.21.15.133:0/4130640801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9ff4370 msgr2=0x5612c9fee670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45f8be5640 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9ff4370 0x5612c9fee670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45f8be5640 1 -- 172.21.15.133:0/4130640801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 msgr2=0x5612c9ffcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.630+0000 7f45f8be5640 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9ffcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45f8be5640 1 -- 172.21.15.133:0/4130640801 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612ca06d990 con 0x5612c9feab40 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45f93e6640 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9ffcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45f8be5640 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9feab40 0x5612c9fedee0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5612c9ff7100 tx=0x7f45e40048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69706a0380c023e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:14.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45dcff9640 1 -- 172.21.15.133:0/4130640801 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4005430 con 0x5612c9feab40 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45fda23740 1 -- 172.21.15.133:0/4130640801 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e400ae10 con 0x5612c9feab40 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45dcff9640 1 -- 172.21.15.133:0/4130640801 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7f45e40055d0 con 0x5612c9feab40 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45dcff9640 1 -- 172.21.15.133:0/4130640801 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f45e401a610 con 0x5612c9feab40 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45fda23740 10 client.26624 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45fda23740 1 -- 172.21.15.133:0/4130640801 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5612ca06eae0 con 0x5612c9feab40 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45dcff9640 1 -- 172.21.15.133:0/4130640801 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e400ae10 con 0x5612c9feab40 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45dcff9640 1 -- 172.21.15.133:0/4130640801 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f45e402c020 con 0x5612c9feab40 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45fda23740 10 client.26624 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45fda23740 10 client.26624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.631+0000 7f45fda23740 20 client.26624 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612ca06f130 0x5612ca071550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 4 client.26624 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:43:14.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 1 -- 172.21.15.133:0/4130640801 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612c9ed3170 con 0x5612ca06f130 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45f93e6640 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612ca06f130 0x5612ca071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45f93e6640 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612ca06f130 0x5612ca071550 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9171e8ca550f4ac5 server_cookie=494a22979f22d811 in_seq=0 out_seq=0 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45dcff9640 10 client.26624 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45dcff9640 1 -- 172.21.15.133:0/4130640801 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612c9ed3170 con 0x5612ca06f130 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45dcff9640 10 client.26624 handle_command_reply: tid=0 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 1 client.26624 shutdown 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 1 -- 172.21.15.133:0/4130640801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9feab40 msgr2=0x5612c9fedee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9feab40 0x5612c9fedee0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5612c9ff7100 tx=0x7f45e40048f0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 1 -- 172.21.15.133:0/4130640801 shutdown_connections 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5612ca06f130 0x5612ca071550 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612c9ff4370 0x5612c9fee670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45dcff9640 0 client.26624 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:43:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612c9feab40 0x5612c9fedee0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 1 --2- 172.21.15.133:0/4130640801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612c9fe95d0 0x5612c9ffcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.632+0000 7f45fda23740 1 -- 172.21.15.133:0/4130640801 >> 172.21.15.133:0/4130640801 conn(0x5612c9fb2310 msgr2=0x5612c9fef510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:14.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fda23740 1 -- 172.21.15.133:0/4130640801 shutdown_connections 2024-09-11T23:43:14.658 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:43:14.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fda23740 1 -- 172.21.15.133:0/4130640801 wait complete. 2024-09-11T23:43:14.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fda23740 20 client.26624 trim_cache size 0 max 16384 2024-09-11T23:43:14.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 -- 172.21.15.133:0/657327278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f45e00a2cf0 msgr2=0x7f45e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f45e00a2cf0 0x7f45e00a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f45ec005c40 tx=0x7f45ec009a20 comp rx=0 tx=0).stop 2024-09-11T23:43:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 -- 172.21.15.133:0/657327278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45f4102d40 msgr2=0x7f45f419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45f4102d40 0x7f45f419a5c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f45f0004770 tx=0x7f45f0003c10 comp rx=0 tx=0).stop 2024-09-11T23:43:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 -- 172.21.15.133:0/657327278 shutdown_connections 2024-09-11T23:43:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f45e00a2cf0 0x7f45e00a51b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f410c170 0x7f45f419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45f4103660 0x7f45f419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 --2- 172.21.15.133:0/657327278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45f4102d40 0x7f45f419a5c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 -- 172.21.15.133:0/657327278 >> 172.21.15.133:0/657327278 conn(0x7f45f40fc7a0 msgr2=0x7f45f40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 -- 172.21.15.133:0/657327278 shutdown_connections 2024-09-11T23:43:14.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:14.633+0000 7f45fb64a640 1 -- 172.21.15.133:0/657327278 wait complete. 2024-09-11T23:43:14.789 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:14 smithi178 ceph-mon[35882]: pgmap v198: 97 pgs: 97 active+clean; 50 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 23 KiB/s rd, 71 MiB/s wr, 127 op/s 2024-09-11T23:43:14.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:14 smithi138 ceph-mon[35142]: pgmap v198: 97 pgs: 97 active+clean; 50 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 23 KiB/s rd, 71 MiB/s wr, 127 op/s 2024-09-11T23:43:15.415 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:43:15.468 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.902+0000 7ff0810b9640 1 -- 172.21.15.133:0/1148869983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff07c101df0 msgr2=0x7ff07c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.902+0000 7ff0810b9640 1 --2- 172.21.15.133:0/1148869983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff07c101df0 0x7ff07c1021d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009900 tx=0x7ff06c02a410 comp rx=0 tx=0).stop 2024-09-11T23:43:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.903+0000 7ff0810b9640 1 -- 172.21.15.133:0/1148869983 shutdown_connections 2024-09-11T23:43:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.903+0000 7ff0810b9640 1 --2- 172.21.15.133:0/1148869983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff07c106f20 0x7ff07c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.903+0000 7ff0810b9640 1 --2- 172.21.15.133:0/1148869983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c1027a0 0x7ff07c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.903+0000 7ff0810b9640 1 --2- 172.21.15.133:0/1148869983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff07c101df0 0x7ff07c1021d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.903+0000 7ff0810b9640 1 -- 172.21.15.133:0/1148869983 >> 172.21.15.133:0/1148869983 conn(0x7ff07c0fd8f0 msgr2=0x7ff07c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.903+0000 7ff0810b9640 1 -- 172.21.15.133:0/1148869983 shutdown_connections 2024-09-11T23:43:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.903+0000 7ff0810b9640 1 -- 172.21.15.133:0/1148869983 wait complete. 2024-09-11T23:43:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.903+0000 7ff0810b9640 1 Processor -- start 2024-09-11T23:43:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff0810b9640 1 -- start start 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff0810b9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c101df0 0x7ff07c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff0810b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff07c1027a0 0x7ff07c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff0810b9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff07c106f20 0x7ff07c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff0810b9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c111fa0 con 0x7ff07c1027a0 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff0810b9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c111e20 con 0x7ff07c101df0 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff0810b9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c112120 con 0x7ff07c106f20 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff07a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff07c1027a0 0x7ff07c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff07a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff07c1027a0 0x7ff07c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46678/0 (socket says 172.21.15.133:46678) 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff07ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c101df0 0x7ff07c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff07a59c640 1 -- 172.21.15.133:0/2477566279 learned_addr learned my addr 172.21.15.133:0/2477566279 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff07a59c640 1 -- 172.21.15.133:0/2477566279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff07c106f20 msgr2=0x7ff07c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff07a59c640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff07c106f20 0x7ff07c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff07a59c640 1 -- 172.21.15.133:0/2477566279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c101df0 msgr2=0x7ff07c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff07a59c640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c101df0 0x7ff07c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff07a59c640 1 -- 172.21.15.133:0/2477566279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c19d430 con 0x7ff07c1027a0 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff07a59c640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff07c1027a0 0x7ff07c1989c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff07000d950 tx=0x7ff07000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.904+0000 7ff063fff640 1 -- 172.21.15.133:0/2477566279 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070017070 con 0x7ff07c1027a0 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.905+0000 7ff0810b9640 1 -- 172.21.15.133:0/2477566279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07c19d720 con 0x7ff07c1027a0 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.905+0000 7ff063fff640 1 -- 172.21.15.133:0/2477566279 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff070009c30 con 0x7ff07c1027a0 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.905+0000 7ff063fff640 1 -- 172.21.15.133:0/2477566279 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07000f6b0 con 0x7ff07c1027a0 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.905+0000 7ff0810b9640 1 -- 172.21.15.133:0/2477566279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07c1a5000 con 0x7ff07c1027a0 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.907+0000 7ff063fff640 1 -- 172.21.15.133:0/2477566279 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff0700109d0 con 0x7ff07c1027a0 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff083492740 1 Processor -- start 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff083492740 1 -- start start 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff083492740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28af40 0x560bca28b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff083492740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 0x560bca29a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff083492740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff083492740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560bca1f0620 con 0x560bca29a650 2024-09-11T23:43:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff083492740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560bca197e50 con 0x560bca28b860 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff083492740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560bca1ec5e0 con 0x560bca28af40 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46690/0 (socket says 172.21.15.133:46690) 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07a59c640 1 -- 172.21.15.133:0/1081972203 learned_addr learned my addr 172.21.15.133:0/1081972203 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07ad9d640 1 --2- 172.21.15.133:0/1081972203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 0x560bca29a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff063fff640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0640a2cf0 0x7ff0640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07b59e640 1 --2- 172.21.15.133:0/1081972203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28af40 0x560bca28b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07a59c640 1 -- 172.21.15.133:0/1081972203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28af40 msgr2=0x560bca28b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07a59c640 1 --2- 172.21.15.133:0/1081972203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28af40 0x560bca28b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07a59c640 1 -- 172.21.15.133:0/1081972203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 msgr2=0x560bca29a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07a59c640 1 --2- 172.21.15.133:0/1081972203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 0x560bca29a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07a59c640 1 -- 172.21.15.133:0/1081972203 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bca188d40 con 0x560bca29a650 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07ad9d640 1 --2- 172.21.15.133:0/1081972203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 0x560bca29a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:43:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff063fff640 1 -- 172.21.15.133:0/2477566279 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7ff0700cf0b0 con 0x7ff07c1027a0 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.908+0000 7ff07b59e640 1 --2- 172.21.15.133:0/1081972203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28af40 0x560bca28b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.909+0000 7ff07b59e640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0640a2cf0 0x7ff0640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.909+0000 7ff07a59c640 1 --2- 172.21.15.133:0/1081972203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29ca40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff0700cc780 tx=0x7ff070020fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36975b5fbb83450c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.909+0000 7ff0617fa640 1 -- 172.21.15.133:0/1081972203 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070020070 con 0x560bca29a650 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.909+0000 7ff0617fa640 1 -- 172.21.15.133:0/1081972203 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff070009de0 con 0x560bca29a650 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.909+0000 7ff07b59e640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0640a2cf0 0x7ff0640a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff074009f20 tx=0x7ff074004b50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.909+0000 7ff0617fa640 1 -- 172.21.15.133:0/1081972203 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07009b5e0 con 0x560bca29a650 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 -- 172.21.15.133:0/1081972203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 msgr2=0x560bca29ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 --2- 172.21.15.133:0/1081972203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29ca40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff0700cc780 tx=0x7ff070020fc0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 -- 172.21.15.133:0/1081972203 shutdown_connections 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 --2- 172.21.15.133:0/1081972203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29ca40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 --2- 172.21.15.133:0/1081972203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 0x560bca29a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 --2- 172.21.15.133:0/1081972203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28af40 0x560bca28b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 -- 172.21.15.133:0/1081972203 >> 172.21.15.133:0/1081972203 conn(0x560bca2875f0 msgr2=0x560bca2879d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 -- 172.21.15.133:0/1081972203 shutdown_connections 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 -- 172.21.15.133:0/1081972203 wait complete. 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 -- start start 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 Processor -- start 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28af40 0x560bca2974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 0x560bca297a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca297f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560bca29d1c0 con 0x560bca29a650 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560bca178f10 con 0x560bca28b860 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.910+0000 7ff083492740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560bca29d340 con 0x560bca28af40 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff083492740 10 client.0 _ll_register_callbacks cb 0x560bca19cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07b59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca297f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07b59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca297f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46698/0 (socket says 172.21.15.133:46698) 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07b59e640 1 -- 172.21.15.133:0/640550510 learned_addr learned my addr 172.21.15.133:0/640550510 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07a59c640 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 0x560bca297a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07ad9d640 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28af40 0x560bca2974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07b59e640 1 -- 172.21.15.133:0/640550510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28af40 msgr2=0x560bca2974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07b59e640 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28af40 0x560bca2974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07b59e640 1 -- 172.21.15.133:0/640550510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 msgr2=0x560bca297a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07b59e640 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 0x560bca297a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07b59e640 1 -- 172.21.15.133:0/640550510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bc9eff580 con 0x560bca29a650 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07a59c640 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 0x560bca297a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff07b59e640 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca297f40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff074017af0 tx=0x7ff074009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7897ec3c9d0e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff05eff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff05eff5640 1 -- 172.21.15.133:0/640550510 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074025060 con 0x560bca29a650 2024-09-11T23:43:15.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff05eff5640 1 -- 172.21.15.133:0/640550510 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7ff074012050 con 0x560bca29a650 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff083492740 1 -- 172.21.15.133:0/640550510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff070009de0 con 0x560bca29a650 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff05eff5640 1 -- 172.21.15.133:0/640550510 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff07400aa80 con 0x560bca29a650 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff083492740 10 client.17151 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.911+0000 7ff083492740 1 -- 172.21.15.133:0/640550510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560bc9fc9960 con 0x560bca29a650 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.912+0000 7ff05eff5640 1 -- 172.21.15.133:0/640550510 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff070009de0 con 0x560bca29a650 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.912+0000 7ff05eff5640 1 -- 172.21.15.133:0/640550510 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff074014070 con 0x560bca29a650 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.912+0000 7ff083492740 10 client.17151 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.912+0000 7ff083492740 10 client.17151 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.912+0000 7ff083492740 20 client.17151 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.912+0000 7ff083492740 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560bca2aa170 0x560bca2ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.912+0000 7ff083492740 4 client.17151 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.912+0000 7ff083492740 1 -- 172.21.15.133:0/640550510 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bca1fdb90 con 0x560bca2aa170 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.912+0000 7ff07ad9d640 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560bca2aa170 0x560bca2ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.913+0000 7ff05eff5640 10 client.17151 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.913+0000 7ff07ad9d640 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560bca2aa170 0x560bca2ac560 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe41d00f2c847a7 server_cookie=7875d3419d2b593d in_seq=0 out_seq=0 2024-09-11T23:43:15.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.913+0000 7ff05eff5640 1 -- 172.21.15.133:0/640550510 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bca1fdb90 con 0x560bca2aa170 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.913+0000 7ff05eff5640 10 client.17151 handle_command_reply: tid=0 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.913+0000 7ff083492740 1 client.17151 shutdown 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.913+0000 7ff083492740 1 -- 172.21.15.133:0/640550510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 msgr2=0x560bca297f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.913+0000 7ff083492740 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca297f40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff074017af0 tx=0x7ff074009540 comp rx=0 tx=0).stop 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.913+0000 7ff083492740 1 -- 172.21.15.133:0/640550510 shutdown_connections 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.913+0000 7ff083492740 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560bca2aa170 0x560bca2ac560 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.914+0000 7ff083492740 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca297f40 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.914+0000 7ff083492740 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28b860 0x560bca297a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.914+0000 7ff05eff5640 0 client.17151 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.914+0000 7ff083492740 1 --2- 172.21.15.133:0/640550510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28af40 0x560bca2974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.914+0000 7ff083492740 1 -- 172.21.15.133:0/640550510 >> 172.21.15.133:0/640550510 conn(0x560bca2875f0 msgr2=0x560bca28d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:15.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.914+0000 7ff083492740 1 -- 172.21.15.133:0/640550510 shutdown_connections 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.914+0000 7ff083492740 1 -- 172.21.15.133:0/640550510 wait complete. 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.914+0000 7ff083492740 20 client.17151 trim_cache size 0 max 16384 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.930+0000 7ff083492740 1 Processor -- start 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.930+0000 7ff083492740 1 -- start start 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.930+0000 7ff083492740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28af40 0x560bca299390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.930+0000 7ff083492740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28b860 0x560bca2b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff083492740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca2b2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff083492740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1ab630 con 0x560bca29a650 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff083492740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1ad170 con 0x560bca28af40 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff083492740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1abd00 con 0x560bca28b860 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07b59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca2b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07b59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca2b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46708/0 (socket says 172.21.15.133:46708) 2024-09-11T23:43:15.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07b59e640 1 -- 172.21.15.133:0/2103529411 learned_addr learned my addr 172.21.15.133:0/2103529411 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07ad9d640 1 --2- 172.21.15.133:0/2103529411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28af40 0x560bca299390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07a59c640 1 --2- 172.21.15.133:0/2103529411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28b860 0x560bca2b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07b59e640 1 -- 172.21.15.133:0/2103529411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28b860 msgr2=0x560bca2b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07b59e640 1 --2- 172.21.15.133:0/2103529411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28b860 0x560bca2b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07b59e640 1 -- 172.21.15.133:0/2103529411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28af40 msgr2=0x560bca299390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07b59e640 1 --2- 172.21.15.133:0/2103529411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28af40 0x560bca299390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07b59e640 1 -- 172.21.15.133:0/2103529411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bca28b690 con 0x560bca29a650 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07a59c640 1 --2- 172.21.15.133:0/2103529411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28b860 0x560bca2b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff07b59e640 1 --2- 172.21.15.133:0/2103529411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca2b2350 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff074009620 tx=0x7ff07400add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3795273c6432094d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff060ff9640 1 -- 172.21.15.133:0/2103529411 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074025070 con 0x560bca29a650 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff060ff9640 1 -- 172.21.15.133:0/2103529411 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff074015d80 con 0x560bca29a650 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.931+0000 7ff060ff9640 1 -- 172.21.15.133:0/2103529411 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0740203e0 con 0x560bca29a650 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.932+0000 7ff083492740 1 -- 172.21.15.133:0/2103529411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 msgr2=0x560bca2b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.932+0000 7ff083492740 1 --2- 172.21.15.133:0/2103529411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca2b2350 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff074009620 tx=0x7ff07400add0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.932+0000 7ff083492740 1 -- 172.21.15.133:0/2103529411 shutdown_connections 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.932+0000 7ff083492740 1 --2- 172.21.15.133:0/2103529411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca2b2350 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.932+0000 7ff083492740 1 --2- 172.21.15.133:0/2103529411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28b860 0x560bca2b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.932+0000 7ff083492740 1 --2- 172.21.15.133:0/2103529411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28af40 0x560bca299390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.932+0000 7ff083492740 1 -- 172.21.15.133:0/2103529411 >> 172.21.15.133:0/2103529411 conn(0x560bca2875f0 msgr2=0x560bca286750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.932+0000 7ff083492740 1 -- 172.21.15.133:0/2103529411 shutdown_connections 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.932+0000 7ff083492740 1 -- 172.21.15.133:0/2103529411 wait complete. 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.932+0000 7ff083492740 1 -- start start 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.932+0000 7ff083492740 1 Processor -- start 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff083492740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28af40 0x560bca2a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff083492740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28b860 0x560bca2a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff083492740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff083492740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1acaa0 con 0x560bca29a650 2024-09-11T23:43:15.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff083492740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1ac3d0 con 0x560bca28af40 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff083492740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1a9630 con 0x560bca28b860 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07b59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff083492740 10 client.0 _ll_register_callbacks cb 0x7ff074017770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07b59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46720/0 (socket says 172.21.15.133:46720) 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07b59e640 1 -- 172.21.15.133:0/72423735 learned_addr learned my addr 172.21.15.133:0/72423735 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07ad9d640 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28af40 0x560bca2a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07a59c640 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28b860 0x560bca2a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07b59e640 1 -- 172.21.15.133:0/72423735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28b860 msgr2=0x560bca2a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07b59e640 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28b860 0x560bca2a2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07b59e640 1 -- 172.21.15.133:0/72423735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28af40 msgr2=0x560bca2a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07b59e640 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28af40 0x560bca2a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07b59e640 1 -- 172.21.15.133:0/72423735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bca29fb90 con 0x560bca29a650 2024-09-11T23:43:15.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07ad9d640 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28af40 0x560bca2a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff07b59e640 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29f5a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff0740070c0 tx=0x7ff0740155f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618da19d2ca5bab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff05eff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff05eff5640 1 -- 172.21.15.133:0/72423735 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074025040 con 0x560bca29a650 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff05eff5640 1 -- 172.21.15.133:0/72423735 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7ff074012040 con 0x560bca29a650 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.933+0000 7ff083492740 1 -- 172.21.15.133:0/72423735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff074015d80 con 0x560bca29a650 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff05eff5640 1 -- 172.21.15.133:0/72423735 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff074043050 con 0x560bca29a650 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff083492740 10 client.17157 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff083492740 1 -- 172.21.15.133:0/72423735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560bca315680 con 0x560bca29a650 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff05eff5640 1 -- 172.21.15.133:0/72423735 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff074015d80 con 0x560bca29a650 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff05eff5640 1 -- 172.21.15.133:0/72423735 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff074014040 con 0x560bca29a650 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff083492740 10 client.17157 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff083492740 10 client.17157 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff083492740 20 client.17157 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:15.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff083492740 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560bca315cd0 0x560bca3180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:15.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff083492740 4 client.17157 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e33d2d6a", "release": true, "await": true}] 2024-09-11T23:43:15.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff083492740 1 -- 172.21.15.133:0/72423735 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e33d2d6a", "release": true, "await": true}) -- 0x560bca1f80a0 con 0x560bca315cd0 2024-09-11T23:43:15.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.934+0000 7ff07ad9d640 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560bca315cd0 0x560bca3180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:15.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.935+0000 7ff05eff5640 10 client.17157 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:15.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.935+0000 7ff07ad9d640 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560bca315cd0 0x560bca3180f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee8bb770739562 server_cookie=2672fe34f553450f in_seq=0 out_seq=0 2024-09-11T23:43:15.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.938+0000 7ff05eff5640 1 -- 172.21.15.133:0/72423735 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560bca1f80a0 con 0x560bca315cd0 2024-09-11T23:43:15.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.938+0000 7ff05eff5640 10 client.17157 handle_command_reply: tid=0 2024-09-11T23:43:15.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.938+0000 7ff083492740 1 client.17157 shutdown 2024-09-11T23:43:15.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.938+0000 7ff083492740 1 -- 172.21.15.133:0/72423735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 msgr2=0x560bca29f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:15.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.938+0000 7ff083492740 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29f5a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff0740070c0 tx=0x7ff0740155f0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.939+0000 7ff083492740 1 -- 172.21.15.133:0/72423735 shutdown_connections 2024-09-11T23:43:15.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.939+0000 7ff083492740 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560bca315cd0 0x560bca3180f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.939+0000 7ff083492740 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bca29a650 0x560bca29f5a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.939+0000 7ff083492740 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bca28b860 0x560bca2a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.939+0000 7ff083492740 1 --2- 172.21.15.133:0/72423735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bca28af40 0x560bca2a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.939+0000 7ff083492740 1 -- 172.21.15.133:0/72423735 >> 172.21.15.133:0/72423735 conn(0x560bca2875f0 msgr2=0x560bca286750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:15.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.939+0000 7ff05eff5640 0 client.17157 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:15.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.939+0000 7ff083492740 1 -- 172.21.15.133:0/72423735 shutdown_connections 2024-09-11T23:43:15.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.939+0000 7ff083492740 1 -- 172.21.15.133:0/72423735 wait complete. 2024-09-11T23:43:15.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.939+0000 7ff083492740 20 client.17157 trim_cache size 0 max 16384 2024-09-11T23:43:15.976 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:43:15.976 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:43:15.976 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:43:15.976 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 24, 2024-09-11T23:43:15.976 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:43:15.977 INFO:teuthology.orchestra.run.smithi133.stdout: "e33d2d6a": { 2024-09-11T23:43:15.977 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 24, 2024-09-11T23:43:15.977 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:43:15.977 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:43:15.977 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:43:15.977 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:43:15.977 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:43:15.977 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:43:15.977 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 140.1, 2024-09-11T23:43:15.977 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:43:15.977 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:43:15.978 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:43:15.978 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:43:15.978 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:43:15.978 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:43:15.978 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:43:15.978 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:43:15.978 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:43:15.978 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:43:15.978 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:43:15.978 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:43:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 -- 172.21.15.133:0/2477566279 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0640a2cf0 msgr2=0x7ff0640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0640a2cf0 0x7ff0640a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff074009f20 tx=0x7ff074004b50 comp rx=0 tx=0).stop 2024-09-11T23:43:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 -- 172.21.15.133:0/2477566279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff07c1027a0 msgr2=0x7ff07c1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff07c1027a0 0x7ff07c1989c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff07000d950 tx=0x7ff07000de10 comp rx=0 tx=0).stop 2024-09-11T23:43:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 -- 172.21.15.133:0/2477566279 shutdown_connections 2024-09-11T23:43:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0640a2cf0 0x7ff0640a51b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff07c106f20 0x7ff07c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff07c1027a0 0x7ff07c1989c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 --2- 172.21.15.133:0/2477566279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff07c101df0 0x7ff07c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 -- 172.21.15.133:0/2477566279 >> 172.21.15.133:0/2477566279 conn(0x7ff07c0fd8f0 msgr2=0x7ff07c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:15.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 -- 172.21.15.133:0/2477566279 shutdown_connections 2024-09-11T23:43:15.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:15.940+0000 7ff0810b9640 1 -- 172.21.15.133:0/2477566279 wait complete. 2024-09-11T23:43:16.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:16 smithi133 ceph-mon[31430]: pgmap v199: 97 pgs: 97 active+clean; 50 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 12 KiB/s rd, 74 MiB/s wr, 117 op/s 2024-09-11T23:43:16.917 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e33d2d6a', total seconds elapsed: 25.7 2024-09-11T23:43:16.926 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:16.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:16 smithi138 ceph-mon[35142]: pgmap v199: 97 pgs: 97 active+clean; 50 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 12 KiB/s rd, 74 MiB/s wr, 117 op/s 2024-09-11T23:43:17.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:16 smithi178 ceph-mon[35882]: pgmap v199: 97 pgs: 97 active+clean; 50 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 12 KiB/s rd, 74 MiB/s wr, 117 op/s 2024-09-11T23:43:17.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.738+0000 7f042205e640 1 -- 172.21.15.133:0/1894543658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c102790 msgr2=0x7f041c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.738+0000 7f042205e640 1 --2- 172.21.15.133:0/1894543658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c102790 0x7f041c1067e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0404009900 tx=0x7f040402e800 comp rx=0 tx=0).stop 2024-09-11T23:43:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.738+0000 7f042205e640 1 -- 172.21.15.133:0/1894543658 shutdown_connections 2024-09-11T23:43:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.738+0000 7f042205e640 1 --2- 172.21.15.133:0/1894543658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c106f10 0x7f041c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.738+0000 7f042205e640 1 --2- 172.21.15.133:0/1894543658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c102790 0x7f041c1067e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.738+0000 7f042205e640 1 --2- 172.21.15.133:0/1894543658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c101de0 0x7f041c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.738+0000 7f042205e640 1 -- 172.21.15.133:0/1894543658 >> 172.21.15.133:0/1894543658 conn(0x7f041c0fd880 msgr2=0x7f041c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.738+0000 7f042205e640 1 -- 172.21.15.133:0/1894543658 shutdown_connections 2024-09-11T23:43:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.738+0000 7f042205e640 1 -- 172.21.15.133:0/1894543658 wait complete. 2024-09-11T23:43:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.740+0000 7f042205e640 1 Processor -- start 2024-09-11T23:43:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.740+0000 7f042205e640 1 -- start start 2024-09-11T23:43:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.740+0000 7f042205e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c101de0 0x7f041c196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.741+0000 7f042205e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102790 0x7f041c1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.741+0000 7f042205e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c106f10 0x7f041c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.741+0000 7f042205e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10fe40 con 0x7f041c106f10 2024-09-11T23:43:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.741+0000 7f042205e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10fcc0 con 0x7f041c102790 2024-09-11T23:43:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.741+0000 7f042205e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10ffc0 con 0x7f041c101de0 2024-09-11T23:43:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.741+0000 7f041b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c101de0 0x7f041c196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.741+0000 7f041b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c101de0 0x7f041c196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48708/0 (socket says 172.21.15.133:48708) 2024-09-11T23:43:17.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.741+0000 7f041b7fe640 1 -- 172.21.15.133:0/2776218871 learned_addr learned my addr 172.21.15.133:0/2776218871 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:17.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.741+0000 7f0413fff640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102790 0x7f041c1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.742+0000 7f041bfff640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c106f10 0x7f041c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.742+0000 7f0413fff640 1 -- 172.21.15.133:0/2776218871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c101de0 msgr2=0x7f041c196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.742+0000 7f0413fff640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c101de0 0x7f041c196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.742+0000 7f0413fff640 1 -- 172.21.15.133:0/2776218871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c106f10 msgr2=0x7f041c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.742+0000 7f0413fff640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c106f10 0x7f041c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.742+0000 7f0413fff640 1 -- 172.21.15.133:0/2776218871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c19b330 con 0x7f041c102790 2024-09-11T23:43:17.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.743+0000 7f0413fff640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102790 0x7f041c1968c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0404009230 tx=0x7f0404003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:17.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.744+0000 7f04197fa640 1 -- 172.21.15.133:0/2776218871 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404043070 con 0x7f041c102790 2024-09-11T23:43:17.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.745+0000 7f04197fa640 1 -- 172.21.15.133:0/2776218871 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0404036c90 con 0x7f041c102790 2024-09-11T23:43:17.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.745+0000 7f04197fa640 1 -- 172.21.15.133:0/2776218871 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040403a030 con 0x7f041c102790 2024-09-11T23:43:17.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.746+0000 7f042205e640 1 -- 172.21.15.133:0/2776218871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c19b5c0 con 0x7f041c102790 2024-09-11T23:43:17.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.746+0000 7f042205e640 1 -- 172.21.15.133:0/2776218871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c19b960 con 0x7f041c102790 2024-09-11T23:43:17.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.752+0000 7f04197fa640 1 -- 172.21.15.133:0/2776218871 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0404046070 con 0x7f041c102790 2024-09-11T23:43:17.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.753+0000 7f0424437740 1 Processor -- start 2024-09-11T23:43:17.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.753+0000 7f04197fa640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04080a2e40 0x7f04080a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.753+0000 7f0424437740 1 -- start start 2024-09-11T23:43:17.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.754+0000 7f04197fa640 1 -- 172.21.15.133:0/2776218871 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7f04040f37a0 con 0x7f041c102790 2024-09-11T23:43:17.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.754+0000 7f041bfff640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04080a2e40 0x7f04080a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.756+0000 7f0424437740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b4453090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.756+0000 7f0424437740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 0x5556b4453a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.756+0000 7f0424437740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b445e370 0x5556b4460760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.756+0000 7f0424437740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5556b43b92f0 con 0x5556b4454b40 2024-09-11T23:43:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.756+0000 7f0424437740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5556b4360cf0 con 0x5556b44535d0 2024-09-11T23:43:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.756+0000 7f0424437740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5556b43b52b0 con 0x5556b445e370 2024-09-11T23:43:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.756+0000 7f041bfff640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04080a2e40 0x7f04080a5300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f041400b830 tx=0x7f0414008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.757+0000 7f041b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b4453090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.757+0000 7f041b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b4453090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46764/0 (socket says 172.21.15.133:46764) 2024-09-11T23:43:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.757+0000 7f041b7fe640 1 -- 172.21.15.133:0/263708737 learned_addr learned my addr 172.21.15.133:0/263708737 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:17.764 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:17 smithi133 ceph-mon[31430]: pgmap v200: 97 pgs: 97 active+clean; 50 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 12 KiB/s rd, 85 MiB/s wr, 135 op/s 2024-09-11T23:43:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.757+0000 7f0413fff640 1 --2- 172.21.15.133:0/263708737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 0x5556b4453a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.757+0000 7f041bfff640 1 --2- 172.21.15.133:0/263708737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b445e370 0x5556b4460760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.758+0000 7f041b7fe640 1 -- 172.21.15.133:0/263708737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b445e370 msgr2=0x5556b4460760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:17.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.758+0000 7f041b7fe640 1 --2- 172.21.15.133:0/263708737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b445e370 0x5556b4460760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.758+0000 7f041b7fe640 1 -- 172.21.15.133:0/263708737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 msgr2=0x5556b4453a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:17.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.758+0000 7f041b7fe640 1 --2- 172.21.15.133:0/263708737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 0x5556b4453a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.758+0000 7f041b7fe640 1 -- 172.21.15.133:0/263708737 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556b43453a0 con 0x5556b4454b40 2024-09-11T23:43:17.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.758+0000 7f0413fff640 1 --2- 172.21.15.133:0/263708737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 0x5556b4453a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:43:17.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.758+0000 7f041b7fe640 1 --2- 172.21.15.133:0/263708737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b4453090 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f040c007420 tx=0x7f040c0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967c72c82eb82d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:17.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.759+0000 7f0411ffb640 1 -- 172.21.15.133:0/263708737 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c019070 con 0x5556b4454b40 2024-09-11T23:43:17.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.759+0000 7f0411ffb640 1 -- 172.21.15.133:0/263708737 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f040c012bf0 con 0x5556b4454b40 2024-09-11T23:43:17.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.759+0000 7f0411ffb640 1 -- 172.21.15.133:0/263708737 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c014db0 con 0x5556b4454b40 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.760+0000 7f0424437740 1 -- 172.21.15.133:0/263708737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 msgr2=0x5556b4453090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.760+0000 7f0424437740 1 --2- 172.21.15.133:0/263708737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b4453090 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f040c007420 tx=0x7f040c0078e0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.761+0000 7f0424437740 1 -- 172.21.15.133:0/263708737 shutdown_connections 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.761+0000 7f0424437740 1 --2- 172.21.15.133:0/263708737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b445e370 0x5556b4460760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.761+0000 7f0424437740 1 --2- 172.21.15.133:0/263708737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 0x5556b4453a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.761+0000 7f0424437740 1 --2- 172.21.15.133:0/263708737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b4453090 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.761+0000 7f0424437740 1 -- 172.21.15.133:0/263708737 >> 172.21.15.133:0/263708737 conn(0x5556b441c310 msgr2=0x5556b444fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.762+0000 7f0424437740 1 -- 172.21.15.133:0/263708737 shutdown_connections 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.762+0000 7f0424437740 1 -- 172.21.15.133:0/263708737 wait complete. 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.762+0000 7f0424437740 1 -- start start 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.763+0000 7f0424437740 1 Processor -- start 2024-09-11T23:43:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.764+0000 7f0424437740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b44535d0 0x5556b445dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.764+0000 7f0424437740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b4454b40 0x5556b445afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.764+0000 7f0424437740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b445e370 0x5556b445b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.764+0000 7f0424437740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5556b4461070 con 0x5556b445e370 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.765+0000 7f0424437740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5556b4374060 con 0x5556b4454b40 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.765+0000 7f0424437740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5556b44611f0 con 0x5556b44535d0 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.765+0000 7f0424437740 10 client.0 _ll_register_callbacks cb 0x5556b40c78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.765+0000 7f041bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b445e370 0x5556b445b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.765+0000 7f041bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b445e370 0x5556b445b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46772/0 (socket says 172.21.15.133:46772) 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.765+0000 7f041bfff640 1 -- 172.21.15.133:0/1450237875 learned_addr learned my addr 172.21.15.133:0/1450237875 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.766+0000 7f041bfff640 1 -- 172.21.15.133:0/1450237875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b44535d0 msgr2=0x5556b445dd30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.766+0000 7f041bfff640 1 --2- 172.21.15.133:0/1450237875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b44535d0 0x5556b445dd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.766+0000 7f041bfff640 1 -- 172.21.15.133:0/1450237875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b4454b40 msgr2=0x5556b445afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.766+0000 7f041bfff640 1 --2- 172.21.15.133:0/1450237875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b4454b40 0x5556b445afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.766+0000 7f041bfff640 1 -- 172.21.15.133:0/1450237875 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556b412a3c0 con 0x5556b445e370 2024-09-11T23:43:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.767+0000 7f041bfff640 1 --2- 172.21.15.133:0/1450237875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b445e370 0x5556b445b5d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f041400ded0 tx=0x7f04140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d61628d53964bc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.767+0000 7f03faffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.768+0000 7f03faffd640 1 -- 172.21.15.133:0/1450237875 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414014e40 con 0x5556b445e370 2024-09-11T23:43:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.769+0000 7f03faffd640 1 -- 172.21.15.133:0/1450237875 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7f041400f040 con 0x5556b445e370 2024-09-11T23:43:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.769+0000 7f0424437740 1 -- 172.21.15.133:0/1450237875 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f040c012bf0 con 0x5556b445e370 2024-09-11T23:43:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.770+0000 7f03faffd640 1 -- 172.21.15.133:0/1450237875 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04140059c0 con 0x5556b445e370 2024-09-11T23:43:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.770+0000 7f0424437740 10 client.17169 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:17.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.770+0000 7f0424437740 1 -- 172.21.15.133:0/1450237875 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556b446cb60 con 0x5556b445e370 2024-09-11T23:43:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.772+0000 7f03faffd640 1 -- 172.21.15.133:0/1450237875 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0414025070 con 0x5556b445e370 2024-09-11T23:43:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.772+0000 7f03faffd640 1 -- 172.21.15.133:0/1450237875 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0414013070 con 0x5556b445e370 2024-09-11T23:43:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.773+0000 7f0424437740 10 client.17169 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.774+0000 7f0424437740 10 client.17169 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:43:17.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.774+0000 7f0424437740 20 client.17169 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:17.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.774+0000 7f0424437740 1 --2- 172.21.15.133:0/1450237875 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5556b446e110 0x5556b44705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.774+0000 7f0424437740 4 client.17169 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:17.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.774+0000 7f0424437740 1 -- 172.21.15.133:0/1450237875 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f040c012bf0 con 0x5556b446e110 2024-09-11T23:43:17.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.780+0000 7f041b7fe640 1 --2- 172.21.15.133:0/1450237875 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5556b446e110 0x5556b44705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.814+0000 7f041b7fe640 1 --2- 172.21.15.133:0/1450237875 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5556b446e110 0x5556b44705d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee27a621217caef6 server_cookie=6c88126f00c5d8a8 in_seq=0 out_seq=0 2024-09-11T23:43:17.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.814+0000 7f03faffd640 10 client.17169 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:43:17.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.820+0000 7f03faffd640 1 -- 172.21.15.133:0/1450237875 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f040c012bf0 con 0x5556b446e110 2024-09-11T23:43:17.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.820+0000 7f03faffd640 10 client.17169 handle_command_reply: tid=0 2024-09-11T23:43:17.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.820+0000 7f0424437740 1 client.17169 shutdown 2024-09-11T23:43:17.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.822+0000 7f0424437740 1 -- 172.21.15.133:0/1450237875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b445e370 msgr2=0x5556b445b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.822+0000 7f0424437740 1 --2- 172.21.15.133:0/1450237875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b445e370 0x5556b445b5d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f041400ded0 tx=0x7f04140046c0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.823+0000 7f0424437740 1 -- 172.21.15.133:0/1450237875 shutdown_connections 2024-09-11T23:43:17.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.823+0000 7f0424437740 1 --2- 172.21.15.133:0/1450237875 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5556b446e110 0x5556b44705d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.823+0000 7f03faffd640 0 client.17169 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:43:17.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.823+0000 7f0424437740 1 --2- 172.21.15.133:0/1450237875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b445e370 0x5556b445b5d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.823+0000 7f0424437740 1 --2- 172.21.15.133:0/1450237875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b4454b40 0x5556b445afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.823+0000 7f0424437740 1 --2- 172.21.15.133:0/1450237875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b44535d0 0x5556b445dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.823+0000 7f0424437740 1 -- 172.21.15.133:0/1450237875 >> 172.21.15.133:0/1450237875 conn(0x5556b441c310 msgr2=0x5556b4455930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:17.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.823+0000 7f0424437740 1 -- 172.21.15.133:0/1450237875 shutdown_connections 2024-09-11T23:43:17.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.823+0000 7f0424437740 1 -- 172.21.15.133:0/1450237875 wait complete. 2024-09-11T23:43:17.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.823+0000 7f0424437740 20 client.17169 trim_cache size 0 max 16384 2024-09-11T23:43:17.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.854+0000 7f0424437740 1 Processor -- start 2024-09-11T23:43:17.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.854+0000 7f0424437740 1 -- start start 2024-09-11T23:43:17.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.855+0000 7f0424437740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 0x5556b446a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.855+0000 7f0424437740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b445a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.856+0000 7f041b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 0x5556b446a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.856+0000 7f041b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 0x5556b446a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:36988/0 (socket says 172.21.15.133:36988) 2024-09-11T23:43:17.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.857+0000 7f0413fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b445a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.857+0000 7f0413fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b445a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46778/0 (socket says 172.21.15.133:46778) 2024-09-11T23:43:17.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.857+0000 7f041b7fe640 1 -- 172.21.15.133:0/92792779 learned_addr learned my addr 172.21.15.133:0/92792779 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:17.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.857+0000 7f0424437740 1 --2- 172.21.15.133:0/92792779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b445e370 0x5556b4475ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.858+0000 7f0424437740 1 -- 172.21.15.133:0/92792779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1ab720 con 0x5556b4454b40 2024-09-11T23:43:17.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.858+0000 7f0424437740 1 -- 172.21.15.133:0/92792779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1ad260 con 0x5556b44535d0 2024-09-11T23:43:17.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.858+0000 7f0424437740 1 -- 172.21.15.133:0/92792779 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1abdf0 con 0x5556b445e370 2024-09-11T23:43:17.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.858+0000 7f041bfff640 1 --2- 172.21.15.133:0/92792779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b445e370 0x5556b4475ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.860+0000 7f0413fff640 1 -- 172.21.15.133:0/92792779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b445e370 msgr2=0x5556b4475ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:17.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.860+0000 7f0413fff640 1 --2- 172.21.15.133:0/92792779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b445e370 0x5556b4475ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.860+0000 7f0413fff640 1 -- 172.21.15.133:0/92792779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 msgr2=0x5556b446a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:17.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.860+0000 7f0413fff640 1 --2- 172.21.15.133:0/92792779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 0x5556b446a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.860+0000 7f0413fff640 1 -- 172.21.15.133:0/92792779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556b4459530 con 0x5556b4454b40 2024-09-11T23:43:17.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.862+0000 7f0413fff640 1 --2- 172.21.15.133:0/92792779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b445a520 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f04040f0ed0 tx=0x7f04040bf900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70787516a6be179d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:17.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.863+0000 7f04117fa640 1 -- 172.21.15.133:0/92792779 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040404d070 con 0x5556b4454b40 2024-09-11T23:43:17.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.864+0000 7f04117fa640 1 -- 172.21.15.133:0/92792779 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04040bfcd0 con 0x5556b4454b40 2024-09-11T23:43:17.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.865+0000 7f0424437740 1 -- 172.21.15.133:0/92792779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 msgr2=0x5556b445a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.866+0000 7f0424437740 1 --2- 172.21.15.133:0/92792779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b445a520 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f04040f0ed0 tx=0x7f04040bf900 comp rx=0 tx=0).stop 2024-09-11T23:43:17.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.879+0000 7f0424437740 1 -- 172.21.15.133:0/92792779 shutdown_connections 2024-09-11T23:43:17.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.879+0000 7f0424437740 1 --2- 172.21.15.133:0/92792779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b445e370 0x5556b4475ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.879+0000 7f0424437740 1 --2- 172.21.15.133:0/92792779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b4454b40 0x5556b445a520 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.879+0000 7f0424437740 1 --2- 172.21.15.133:0/92792779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b44535d0 0x5556b446a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.879+0000 7f0424437740 1 -- 172.21.15.133:0/92792779 >> 172.21.15.133:0/92792779 conn(0x5556b441c310 msgr2=0x5556b44598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:17.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.880+0000 7f0424437740 1 -- 172.21.15.133:0/92792779 shutdown_connections 2024-09-11T23:43:17.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.880+0000 7f0424437740 1 -- 172.21.15.133:0/92792779 wait complete. 2024-09-11T23:43:17.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.881+0000 7f0424437740 1 -- start start 2024-09-11T23:43:17.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.882+0000 7f0424437740 1 Processor -- start 2024-09-11T23:43:17.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.888+0000 7f0424437740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b44535d0 0x5556b4466d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.888+0000 7f0424437740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b445e370 0x5556b4461fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.888+0000 7f0424437740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b446e110 0x5556b4462770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.888+0000 7f0424437740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1acb90 con 0x5556b44535d0 2024-09-11T23:43:17.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.888+0000 7f0424437740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1ac4c0 con 0x5556b445e370 2024-09-11T23:43:17.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.888+0000 7f0424437740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f041c1a9720 con 0x5556b446e110 2024-09-11T23:43:17.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.888+0000 7f0424437740 10 client.0 _ll_register_callbacks cb 0x7f041400b4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.889+0000 7f0413fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b445e370 0x5556b4461fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.890+0000 7f0413fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b445e370 0x5556b4461fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:36998/0 (socket says 172.21.15.133:36998) 2024-09-11T23:43:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.890+0000 7f0413fff640 1 -- 172.21.15.133:0/4022708499 learned_addr learned my addr 172.21.15.133:0/4022708499 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:17.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.891+0000 7f0413fff640 1 -- 172.21.15.133:0/4022708499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b446e110 msgr2=0x5556b4462770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:17.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.891+0000 7f0413fff640 1 --2- 172.21.15.133:0/4022708499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b446e110 0x5556b4462770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.891+0000 7f0413fff640 1 -- 172.21.15.133:0/4022708499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b44535d0 msgr2=0x5556b4466d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:17.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.891+0000 7f0413fff640 1 --2- 172.21.15.133:0/4022708499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b44535d0 0x5556b4466d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.891+0000 7f0413fff640 1 -- 172.21.15.133:0/4022708499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556b44e8e80 con 0x5556b445e370 2024-09-11T23:43:17.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.892+0000 7f0413fff640 1 --2- 172.21.15.133:0/4022708499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b445e370 0x5556b4461fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04040bf9d0 tx=0x7f040403f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5092507a297d17c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:17.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.892+0000 7f03faffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:17.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.893+0000 7f03faffd640 1 -- 172.21.15.133:0/4022708499 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04040dc020 con 0x5556b445e370 2024-09-11T23:43:17.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.893+0000 7f0424437740 1 -- 172.21.15.133:0/4022708499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556b43c6860 con 0x5556b445e370 2024-09-11T23:43:17.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.894+0000 7f03faffd640 1 -- 172.21.15.133:0/4022708499 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8499+0+0 (secure 0 0 0) 0x7f04040c8d90 con 0x5556b445e370 2024-09-11T23:43:17.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.894+0000 7f03faffd640 1 -- 172.21.15.133:0/4022708499 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f040404d070 con 0x5556b445e370 2024-09-11T23:43:17.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.894+0000 7f0424437740 10 client.26651 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:17.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.894+0000 7f0424437740 1 -- 172.21.15.133:0/4022708499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556b44ea090 con 0x5556b445e370 2024-09-11T23:43:17.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.895+0000 7f03faffd640 1 -- 172.21.15.133:0/4022708499 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5556b43c6860 con 0x5556b445e370 2024-09-11T23:43:17.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.895+0000 7f03faffd640 1 -- 172.21.15.133:0/4022708499 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0404035070 con 0x5556b445e370 2024-09-11T23:43:17.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.896+0000 7f0424437740 10 client.26651 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:17.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.896+0000 7f0424437740 10 client.26651 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:43:17.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.896+0000 7f0424437740 20 client.26651 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:17.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.896+0000 7f0424437740 1 --2- 172.21.15.133:0/4022708499 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5556b44ea740 0x5556b44ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:17.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.896+0000 7f0424437740 4 client.26651 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:43:17.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.896+0000 7f0424437740 1 -- 172.21.15.133:0/4022708499 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556b43c0d70 con 0x5556b44ea740 2024-09-11T23:43:17.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.898+0000 7f041b7fe640 1 --2- 172.21.15.133:0/4022708499 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5556b44ea740 0x5556b44ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:17.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.903+0000 7f041b7fe640 1 --2- 172.21.15.133:0/4022708499 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5556b44ea740 0x5556b44ecba0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=618304553b186bd6 server_cookie=7ce02f20d0ed4ae0 in_seq=0 out_seq=0 2024-09-11T23:43:17.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.903+0000 7f03faffd640 10 client.26651 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:43:17.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.903+0000 7f03faffd640 1 -- 172.21.15.133:0/4022708499 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556b43c0d70 con 0x5556b44ea740 2024-09-11T23:43:17.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.903+0000 7f03faffd640 10 client.26651 handle_command_reply: tid=0 2024-09-11T23:43:17.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.903+0000 7f0424437740 1 client.26651 shutdown 2024-09-11T23:43:17.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.906+0000 7f0424437740 1 -- 172.21.15.133:0/4022708499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b445e370 msgr2=0x5556b4461fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.906+0000 7f0424437740 1 --2- 172.21.15.133:0/4022708499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b445e370 0x5556b4461fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04040bf9d0 tx=0x7f040403f790 comp rx=0 tx=0).stop 2024-09-11T23:43:17.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.906+0000 7f0424437740 1 -- 172.21.15.133:0/4022708499 shutdown_connections 2024-09-11T23:43:17.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.906+0000 7f0424437740 1 --2- 172.21.15.133:0/4022708499 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5556b44ea740 0x5556b44ecba0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.906+0000 7f0424437740 1 --2- 172.21.15.133:0/4022708499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556b446e110 0x5556b4462770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.906+0000 7f0424437740 1 --2- 172.21.15.133:0/4022708499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556b445e370 0x5556b4461fe0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.906+0000 7f0424437740 1 --2- 172.21.15.133:0/4022708499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556b44535d0 0x5556b4466d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.906+0000 7f0424437740 1 -- 172.21.15.133:0/4022708499 >> 172.21.15.133:0/4022708499 conn(0x5556b441c310 msgr2=0x5556b444e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:17.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.907+0000 7f03faffd640 0 client.26651 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:43:17.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.916+0000 7f0424437740 1 -- 172.21.15.133:0/4022708499 shutdown_connections 2024-09-11T23:43:17.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.916+0000 7f0424437740 1 -- 172.21.15.133:0/4022708499 wait complete. 2024-09-11T23:43:17.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.918+0000 7f0424437740 20 client.26651 trim_cache size 0 max 16384 2024-09-11T23:43:17.919 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:43:17.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.934+0000 7f042205e640 1 -- 172.21.15.133:0/2776218871 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04080a2e40 msgr2=0x7f04080a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.934+0000 7f042205e640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04080a2e40 0x7f04080a5300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f041400b830 tx=0x7f0414008040 comp rx=0 tx=0).stop 2024-09-11T23:43:17.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.934+0000 7f042205e640 1 -- 172.21.15.133:0/2776218871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102790 msgr2=0x7f041c1968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:17.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.934+0000 7f042205e640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102790 0x7f041c1968c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0404009230 tx=0x7f0404003c10 comp rx=0 tx=0).stop 2024-09-11T23:43:17.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.935+0000 7f042205e640 1 -- 172.21.15.133:0/2776218871 shutdown_connections 2024-09-11T23:43:17.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.935+0000 7f042205e640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f04080a2e40 0x7f04080a5300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.935+0000 7f042205e640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f041c106f10 0x7f041c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.935+0000 7f042205e640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f041c102790 0x7f041c1968c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.935+0000 7f042205e640 1 --2- 172.21.15.133:0/2776218871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f041c101de0 0x7f041c196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:17.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.935+0000 7f042205e640 1 -- 172.21.15.133:0/2776218871 >> 172.21.15.133:0/2776218871 conn(0x7f041c0fd880 msgr2=0x7f041c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:17.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.938+0000 7f042205e640 1 -- 172.21.15.133:0/2776218871 shutdown_connections 2024-09-11T23:43:17.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:17.938+0000 7f042205e640 1 -- 172.21.15.133:0/2776218871 wait complete. 2024-09-11T23:43:17.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:17 smithi138 ceph-mon[35142]: pgmap v200: 97 pgs: 97 active+clean; 50 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 12 KiB/s rd, 85 MiB/s wr, 135 op/s 2024-09-11T23:43:18.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:17 smithi178 ceph-mon[35882]: pgmap v200: 97 pgs: 97 active+clean; 50 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 12 KiB/s rd, 85 MiB/s wr, 135 op/s 2024-09-11T23:43:18.824 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:43:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:19.993 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:19 smithi133 ceph-mon[31430]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:43:19.993 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:19 smithi133 ceph-mon[31430]: pgmap v202: 97 pgs: 97 active+clean; 50 GiB data, 169 GiB used, 904 GiB / 1.0 TiB avail; 301 KiB/s rd, 175 MiB/s wr, 376 op/s 2024-09-11T23:43:20.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:19 smithi178 ceph-mon[35882]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:43:20.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:19 smithi178 ceph-mon[35882]: pgmap v202: 97 pgs: 97 active+clean; 50 GiB data, 169 GiB used, 904 GiB / 1.0 TiB avail; 301 KiB/s rd, 175 MiB/s wr, 376 op/s 2024-09-11T23:43:20.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:19 smithi138 ceph-mon[35142]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:43:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:19 smithi138 ceph-mon[35142]: pgmap v202: 97 pgs: 97 active+clean; 50 GiB data, 169 GiB used, 904 GiB / 1.0 TiB avail; 301 KiB/s rd, 175 MiB/s wr, 376 op/s 2024-09-11T23:43:20.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.273+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/2014632996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4106a80 msgr2=0x7f8fd4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.273+0000 7f8fd9ab2640 1 --2- 172.21.15.133:0/2014632996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4106a80 0x7f8fd4106e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009900 tx=0x7f8fc402a410 comp rx=0 tx=0).stop 2024-09-11T23:43:20.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.273+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/2014632996 shutdown_connections 2024-09-11T23:43:20.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.273+0000 7f8fd9ab2640 1 --2- 172.21.15.133:0/2014632996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd4101420 0x7f8fd406a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.273+0000 7f8fd9ab2640 1 --2- 172.21.15.133:0/2014632996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd4100a80 0x7f8fd4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.273+0000 7f8fd9ab2640 1 --2- 172.21.15.133:0/2014632996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4106a80 0x7f8fd4106e60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.273+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/2014632996 >> 172.21.15.133:0/2014632996 conn(0x7f8fd40fc7a0 msgr2=0x7f8fd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:20.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.274+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/2014632996 shutdown_connections 2024-09-11T23:43:20.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.274+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/2014632996 wait complete. 2024-09-11T23:43:20.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.275+0000 7f8fd9ab2640 1 Processor -- start 2024-09-11T23:43:20.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.275+0000 7f8fd9ab2640 1 -- start start 2024-09-11T23:43:20.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.275+0000 7f8fd9ab2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd4100a80 0x7f8fd4195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.275+0000 7f8fd9ab2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4101420 0x7f8fd4196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.275+0000 7f8fd9ab2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd4106a80 0x7f8fd419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd9ab2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd410fde0 con 0x7f8fd4106a80 2024-09-11T23:43:20.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd9ab2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd410fc60 con 0x7f8fd4100a80 2024-09-11T23:43:20.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd9ab2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd410ff60 con 0x7f8fd4101420 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4101420 0x7f8fd4196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd4106a80 0x7f8fd419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4101420 0x7f8fd4196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48744/0 (socket says 172.21.15.133:48744) 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/3463434333 learned_addr learned my addr 172.21.15.133:0/3463434333 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd37fe640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd4100a80 0x7f8fd4195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd3fff640 1 -- 172.21.15.133:0/3463434333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4101420 msgr2=0x7f8fd4196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd3fff640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4101420 0x7f8fd4196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd3fff640 1 -- 172.21.15.133:0/3463434333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd4100a80 msgr2=0x7f8fd4195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd3fff640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd4100a80 0x7f8fd4195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.276+0000 7f8fd3fff640 1 -- 172.21.15.133:0/3463434333 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd419af00 con 0x7f8fd4106a80 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.277+0000 7f8fd3fff640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd4106a80 0x7f8fd419a820 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc005fd0 tx=0x7f8fcc016850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.277+0000 7f8fd0ff9640 1 -- 172.21.15.133:0/3463434333 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc020020 con 0x7f8fd4106a80 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.277+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/3463434333 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd419b1f0 con 0x7f8fd4106a80 2024-09-11T23:43:20.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.277+0000 7f8fd0ff9640 1 -- 172.21.15.133:0/3463434333 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fcc00fde0 con 0x7f8fd4106a80 2024-09-11T23:43:20.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.277+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/3463434333 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd41a2ad0 con 0x7f8fd4106a80 2024-09-11T23:43:20.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.277+0000 7f8fd0ff9640 1 -- 172.21.15.133:0/3463434333 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0115e0 con 0x7f8fd4106a80 2024-09-11T23:43:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.281+0000 7f8fdbe8b740 1 Processor -- start 2024-09-11T23:43:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.282+0000 7f8fd0ff9640 1 -- 172.21.15.133:0/3463434333 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8fcc01a050 con 0x7f8fd4106a80 2024-09-11T23:43:20.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.282+0000 7f8fdbe8b740 1 -- start start 2024-09-11T23:43:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.282+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 0x55a2326e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.282+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 0x55a2326e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.282+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326f1370 0x55a2326f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.282+0000 7f8fdbe8b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a23264c2f0 con 0x55a2326f1370 2024-09-11T23:43:20.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fdbe8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a2325f3cf0 con 0x55a2326e7b40 2024-09-11T23:43:20.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fdbe8b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a2326482b0 con 0x55a2326e65d0 2024-09-11T23:43:20.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 0x55a2326e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 0x55a2326e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 0x55a2326e6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37024/0 (socket says 172.21.15.133:37024) 2024-09-11T23:43:20.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/3505053308 learned_addr learned my addr 172.21.15.133:0/3505053308 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:20.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd0ff9640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8fbc0a2cf0 0x7f8fbc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/3505053308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326f1370 0x55a2326f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/3505053308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 msgr2=0x55a2326e6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:20.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/3505053308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 0x55a2326e6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/3505053308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326f1370 msgr2=0x55a2326f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:20.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/3505053308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326f1370 0x55a2326f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/3505053308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2325d83a0 con 0x55a2326e7b40 2024-09-11T23:43:20.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.283+0000 7f8fd0ff9640 1 -- 172.21.15.133:0/3463434333 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f8fcc0d7420 con 0x7f8fd4106a80 2024-09-11T23:43:20.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.284+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/3505053308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 0x55a2326e6090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800a670 tx=0x7f8fc800a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b86d9b32661c7e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:20.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.284+0000 7f8fb9ffb640 1 -- 172.21.15.133:0/3505053308 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8005220 con 0x55a2326e7b40 2024-09-11T23:43:20.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.284+0000 7f8fd3fff640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8fbc0a2cf0 0x7f8fbc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.284+0000 7f8fb9ffb640 1 -- 172.21.15.133:0/3505053308 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc8004c80 con 0x55a2326e7b40 2024-09-11T23:43:20.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.284+0000 7f8fb9ffb640 1 -- 172.21.15.133:0/3505053308 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc800fd60 con 0x55a2326e7b40 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.285+0000 7f8fd3fff640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8fbc0a2cf0 0x7f8fbc0a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0d4c60 tx=0x7f8fcc00ff90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.285+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/3505053308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 msgr2=0x55a2326e6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.285+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/3505053308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 0x55a2326e6090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800a670 tx=0x7f8fc800a8c0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.285+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/3505053308 shutdown_connections 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.285+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/3505053308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326f1370 0x55a2326f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.285+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/3505053308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 0x55a2326e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.285+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/3505053308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 0x55a2326e6090 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.285+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/3505053308 >> 172.21.15.133:0/3505053308 conn(0x55a2326af310 msgr2=0x55a2326e2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.285+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/3505053308 shutdown_connections 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.285+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/3505053308 wait complete. 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.286+0000 7f8fdbe8b740 1 -- start start 2024-09-11T23:43:20.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.286+0000 7f8fdbe8b740 1 Processor -- start 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.287+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 0x55a2326f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.287+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 0x55a2326ee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.287+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326f1370 0x55a2326ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.287+0000 7f8fd2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 0x55a2326f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.287+0000 7f8fd2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 0x55a2326f0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48758/0 (socket says 172.21.15.133:48758) 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.287+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/1057288328 learned_addr learned my addr 172.21.15.133:0/1057288328 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.287+0000 7f8fd37fe640 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 0x55a2326ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.287+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/1057288328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a2326f4070 con 0x55a2326f1370 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.287+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/1057288328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a232607060 con 0x55a2326e7b40 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.287+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/1057288328 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a2326f41f0 con 0x55a2326e65d0 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.287+0000 7f8fdbe8b740 10 client.0 _ll_register_callbacks cb 0x55a23235a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326f1370 0x55a2326ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fd37fe640 1 -- 172.21.15.133:0/1057288328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 msgr2=0x55a2326f0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:20.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fd37fe640 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 0x55a2326f0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fd37fe640 1 -- 172.21.15.133:0/1057288328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326f1370 msgr2=0x55a2326ee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:20.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fd37fe640 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326f1370 0x55a2326ee7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fd37fe640 1 -- 172.21.15.133:0/1057288328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2323bd3c0 con 0x55a2326e7b40 2024-09-11T23:43:20.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 0x55a2326f0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:43:20.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fd37fe640 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 0x55a2326ee060 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8fc402a930 tx=0x7f8fc4031440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7c5f1fe2a109ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:20.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:20.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/1057288328 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4035030 con 0x55a2326e7b40 2024-09-11T23:43:20.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/1057288328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc8004c80 con 0x55a2326e7b40 2024-09-11T23:43:20.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.288+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/1057288328 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f8fc4004460 con 0x55a2326e7b40 2024-09-11T23:43:20.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/1057288328 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8fc400b0b0 con 0x55a2326e7b40 2024-09-11T23:43:20.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fdbe8b740 10 client.26660 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:20.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/1057288328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a2326ffb20 con 0x55a2326e7b40 2024-09-11T23:43:20.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/1057288328 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc8004c80 con 0x55a2326e7b40 2024-09-11T23:43:20.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/1057288328 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8fc404e7d0 con 0x55a2326e7b40 2024-09-11T23:43:20.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fdbe8b740 10 client.26660 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fdbe8b740 10 client.26660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fdbe8b740 20 client.26660 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a232700f80 0x55a232703370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fdbe8b740 4 client.26660 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/1057288328 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a232659860 con 0x55a232700f80 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.289+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a232700f80 0x55a232703370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.290+0000 7f8fb2ffd640 10 client.26660 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.290+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a232700f80 0x55a232703370 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2911581dc00d11 server_cookie=afed2ebcb97cb9a2 in_seq=0 out_seq=0 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.290+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/1057288328 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a232659860 con 0x55a232700f80 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.290+0000 7f8fb2ffd640 10 client.26660 handle_command_reply: tid=0 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 1 client.26660 shutdown 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/1057288328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 msgr2=0x55a2326ee060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 0x55a2326ee060 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8fc402a930 tx=0x7f8fc4031440 comp rx=0 tx=0).stop 2024-09-11T23:43:20.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/1057288328 shutdown_connections 2024-09-11T23:43:20.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a232700f80 0x55a232703370 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326f1370 0x55a2326ee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e7b40 0x55a2326ee060 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/1057288328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326e65d0 0x55a2326f0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/1057288328 >> 172.21.15.133:0/1057288328 conn(0x55a2326af310 msgr2=0x55a2326e8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:20.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fb2ffd640 0 client.26660 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:20.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/1057288328 shutdown_connections 2024-09-11T23:43:20.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/1057288328 wait complete. 2024-09-11T23:43:20.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.291+0000 7f8fdbe8b740 20 client.26660 trim_cache size 0 max 16384 2024-09-11T23:43:20.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.305+0000 7f8fdbe8b740 1 Processor -- start 2024-09-11T23:43:20.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fdbe8b740 1 -- start start 2024-09-11T23:43:20.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 0x55a232708c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 0x55a232709160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326f1370 0x55a2326eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fdbe8b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41ab4f0 con 0x55a2326e7b40 2024-09-11T23:43:20.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fdbe8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41ad030 con 0x55a2326e65d0 2024-09-11T23:43:20.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fdbe8b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41abbc0 con 0x55a2326f1370 2024-09-11T23:43:20.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 0x55a232709160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 0x55a232709160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46842/0 (socket says 172.21.15.133:46842) 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd37fe640 1 -- 172.21.15.133:0/3000911134 learned_addr learned my addr 172.21.15.133:0/3000911134 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/3000911134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 0x55a232708c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/3000911134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326f1370 0x55a2326eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/3000911134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326f1370 msgr2=0x55a2326eb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/3000911134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326f1370 0x55a2326eb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/3000911134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 msgr2=0x55a232709160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/3000911134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 0x55a232709160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/3000911134 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2326e2510 con 0x55a2326e65d0 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd37fe640 1 --2- 172.21.15.133:0/3000911134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 0x55a232709160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.306+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/3000911134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 0x55a232708c20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55a232703b90 tx=0x7f8fc801e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b713765e61355bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.307+0000 7f8fb97fa640 1 -- 172.21.15.133:0/3000911134 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8004af0 con 0x55a2326e65d0 2024-09-11T23:43:20.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.307+0000 7f8fb97fa640 1 -- 172.21.15.133:0/3000911134 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc8004c90 con 0x55a2326e65d0 2024-09-11T23:43:20.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.307+0000 7f8fb97fa640 1 -- 172.21.15.133:0/3000911134 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc802a3f0 con 0x55a2326e65d0 2024-09-11T23:43:20.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.307+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/3000911134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 msgr2=0x55a232708c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.307+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/3000911134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 0x55a232708c20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55a232703b90 tx=0x7f8fc801e040 comp rx=0 tx=0).stop 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.308+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/3000911134 shutdown_connections 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.308+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/3000911134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326f1370 0x55a2326eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.308+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/3000911134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 0x55a232709160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.308+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/3000911134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 0x55a232708c20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.308+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/3000911134 >> 172.21.15.133:0/3000911134 conn(0x55a2326af310 msgr2=0x55a2326ec930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.308+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/3000911134 shutdown_connections 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.308+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/3000911134 wait complete. 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.308+0000 7f8fdbe8b740 1 -- start start 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.308+0000 7f8fdbe8b740 1 Processor -- start 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 0x55a2326f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 0x55a2326f5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fdbe8b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326f1370 0x55a2326f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fdbe8b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41ac960 con 0x55a2326e7b40 2024-09-11T23:43:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fdbe8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41ac290 con 0x55a2326e65d0 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fdbe8b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41a94f0 con 0x55a2326f1370 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fdbe8b740 10 client.0 _ll_register_callbacks cb 0x7f8fc4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fd2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 0x55a2326f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fd37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 0x55a2326f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fd2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 0x55a2326f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37056/0 (socket says 172.21.15.133:37056) 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/2690641511 learned_addr learned my addr 172.21.15.133:0/2690641511 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326f1370 0x55a2326f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/2690641511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326f1370 msgr2=0x55a2326f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326f1370 0x55a2326f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/2690641511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 msgr2=0x55a2326f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 0x55a2326f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fd2ffd640 1 -- 172.21.15.133:0/2690641511 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a23276abf0 con 0x55a2326e65d0 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fd2ffd640 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 0x55a2326f51d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8004ea0 tx=0x7f8fc802d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdde16dc27e89340 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:20.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.309+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/2690641511 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8041060 con 0x55a2326e65d0 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/2690641511 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc8004c90 con 0x55a2326e65d0 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/2690641511 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f8fc800f910 con 0x55a2326e65d0 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/2690641511 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8fc8005120 con 0x55a2326e65d0 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fdbe8b740 10 client.26666 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/2690641511 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a23276bed0 con 0x55a2326e65d0 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/2690641511 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc8004c90 con 0x55a2326e65d0 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/2690641511 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8fc8039040 con 0x55a2326e65d0 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fdbe8b740 10 client.26666 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fdbe8b740 10 client.26666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fdbe8b740 20 client.26666 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a23276c320 0x55a23276e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fdbe8b740 4 client.26666 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:43:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/2690641511 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a232653d70 con 0x55a23276c320 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.310+0000 7f8fd37fe640 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a23276c320 0x55a23276e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fb2ffd640 10 client.26666 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fd37fe640 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a23276c320 0x55a23276e740 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9226b7ac0f30bee server_cookie=5bffcce4fdcbce29 in_seq=0 out_seq=0 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fb2ffd640 1 -- 172.21.15.133:0/2690641511 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a232653d70 con 0x55a23276c320 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fb2ffd640 10 client.26666 handle_command_reply: tid=0 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fdbe8b740 1 client.26666 shutdown 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/2690641511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 msgr2=0x55a2326f51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 0x55a2326f51d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8004ea0 tx=0x7f8fc802d040 comp rx=0 tx=0).stop 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/2690641511 shutdown_connections 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a23276c320 0x55a23276e740 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2326f1370 0x55a2326f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fb2ffd640 0 client.26666 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2326e7b40 0x55a2326f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fdbe8b740 1 --2- 172.21.15.133:0/2690641511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2326e65d0 0x55a2326f51d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/2690641511 >> 172.21.15.133:0/2690641511 conn(0x55a2326af310 msgr2=0x55a2326e14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:20.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/2690641511 shutdown_connections 2024-09-11T23:43:20.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.311+0000 7f8fdbe8b740 1 -- 172.21.15.133:0/2690641511 wait complete. 2024-09-11T23:43:20.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.312+0000 7f8fdbe8b740 20 client.26666 trim_cache size 0 max 16384 2024-09-11T23:43:20.318 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:43:20.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.312+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/3463434333 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8fbc0a2cf0 msgr2=0x7f8fbc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.312+0000 7f8fd9ab2640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8fbc0a2cf0 0x7f8fbc0a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0d4c60 tx=0x7f8fcc00ff90 comp rx=0 tx=0).stop 2024-09-11T23:43:20.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.312+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/3463434333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd4106a80 msgr2=0x7f8fd419a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:20.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.312+0000 7f8fd9ab2640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd4106a80 0x7f8fd419a820 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc005fd0 tx=0x7f8fcc016850 comp rx=0 tx=0).stop 2024-09-11T23:43:20.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.312+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/3463434333 shutdown_connections 2024-09-11T23:43:20.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.312+0000 7f8fd9ab2640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8fbc0a2cf0 0x7f8fbc0a51b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.312+0000 7f8fd9ab2640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd4106a80 0x7f8fd419a820 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.312+0000 7f8fd9ab2640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd4101420 0x7f8fd4196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.312+0000 7f8fd9ab2640 1 --2- 172.21.15.133:0/3463434333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd4100a80 0x7f8fd4195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:20.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.312+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/3463434333 >> 172.21.15.133:0/3463434333 conn(0x7f8fd40fc7a0 msgr2=0x7f8fd4107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:20.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.313+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/3463434333 shutdown_connections 2024-09-11T23:43:20.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:20.313+0000 7f8fd9ab2640 1 -- 172.21.15.133:0/3463434333 wait complete. 2024-09-11T23:43:22.380 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:43:22.381 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:43:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.499+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1947001748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a201091f0 msgr2=0x7f5a201095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.499+0000 7f5a25d8b640 1 --2- 172.21.15.133:0/1947001748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a201091f0 0x7f5a201095d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009900 tx=0x7f5a1402a3f0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.499+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1947001748 shutdown_connections 2024-09-11T23:43:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.499+0000 7f5a25d8b640 1 --2- 172.21.15.133:0/1947001748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a20114ab0 0x7f5a20116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.499+0000 7f5a25d8b640 1 --2- 172.21.15.133:0/1947001748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a20109b10 0x7f5a20114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.499+0000 7f5a25d8b640 1 --2- 172.21.15.133:0/1947001748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a201091f0 0x7f5a201095d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.499+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1947001748 >> 172.21.15.133:0/1947001748 conn(0x7f5a2006cbf0 msgr2=0x7f5a2006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.499+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1947001748 shutdown_connections 2024-09-11T23:43:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.499+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1947001748 wait complete. 2024-09-11T23:43:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.500+0000 7f5a25d8b640 1 Processor -- start 2024-09-11T23:43:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.500+0000 7f5a25d8b640 1 -- start start 2024-09-11T23:43:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.500+0000 7f5a25d8b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a201091f0 0x7f5a2010a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.500+0000 7f5a25d8b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a20109b10 0x7f5a2010a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.500+0000 7f5a25d8b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a20114ab0 0x7f5a2010ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.500+0000 7f5a25d8b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2011a4f0 con 0x7f5a20114ab0 2024-09-11T23:43:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.500+0000 7f5a25d8b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2011a370 con 0x7f5a20109b10 2024-09-11T23:43:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.500+0000 7f5a25d8b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2011a670 con 0x7f5a201091f0 2024-09-11T23:43:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.501+0000 7f5a2558a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a20114ab0 0x7f5a2010ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.501+0000 7f5a1ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a20109b10 0x7f5a2010a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.501+0000 7f5a2558a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a20114ab0 0x7f5a2010ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44912/0 (socket says 172.21.15.133:44912) 2024-09-11T23:43:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.501+0000 7f5a2558a640 1 -- 172.21.15.133:0/1145378404 learned_addr learned my addr 172.21.15.133:0/1145378404 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.501+0000 7f5a24d89640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a201091f0 0x7f5a2010a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.501+0000 7f5a1ffff640 1 -- 172.21.15.133:0/1145378404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a201091f0 msgr2=0x7f5a2010a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.501+0000 7f5a1ffff640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a201091f0 0x7f5a2010a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.501+0000 7f5a1ffff640 1 -- 172.21.15.133:0/1145378404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a20114ab0 msgr2=0x7f5a2010ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.501+0000 7f5a1ffff640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a20114ab0 0x7f5a2010ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.501+0000 7f5a1ffff640 1 -- 172.21.15.133:0/1145378404 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a201b3400 con 0x7f5a20109b10 2024-09-11T23:43:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.502+0000 7f5a1ffff640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a20109b10 0x7f5a2010a910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800e9f0 tx=0x7f5a1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.502+0000 7f5a1dffb640 1 -- 172.21.15.133:0/1145378404 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1800cd80 con 0x7f5a20109b10 2024-09-11T23:43:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.502+0000 7f5a1dffb640 1 -- 172.21.15.133:0/1145378404 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a18011950 con 0x7f5a20109b10 2024-09-11T23:43:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.502+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1145378404 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a201b3630 con 0x7f5a20109b10 2024-09-11T23:43:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.502+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1145378404 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a201b3bd0 con 0x7f5a20109b10 2024-09-11T23:43:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.502+0000 7f5a1dffb640 1 -- 172.21.15.133:0/1145378404 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18010a40 con 0x7f5a20109b10 2024-09-11T23:43:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.506+0000 7f5a1dffb640 1 -- 172.21.15.133:0/1145378404 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5a18010be0 con 0x7f5a20109b10 2024-09-11T23:43:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.506+0000 7f5a29e5d740 1 Processor -- start 2024-09-11T23:43:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.506+0000 7f5a29e5d740 1 -- start start 2024-09-11T23:43:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.506+0000 7f5a29e5d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x56005325c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.506+0000 7f5a29e5d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325c5e0 0x560053266940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.506+0000 7f5a29e5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053269270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.506+0000 7f5a29e5d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5600531c02f0 con 0x560053266e80 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.506+0000 7f5a29e5d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560053168270 con 0x56005325bcc0 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.506+0000 7f5a29e5d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5600531bc2b0 con 0x56005325c5e0 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.507+0000 7f5a1dffb640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5a080a2dc0 0x7f5a080a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.507+0000 7f5a2558a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x56005325c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.508+0000 7f5a2558a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x56005325c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40398/0 (socket says 172.21.15.133:40398) 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.508+0000 7f5a2558a640 1 -- 172.21.15.133:0/84756788 learned_addr learned my addr 172.21.15.133:0/84756788 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.508+0000 7f5a1dffb640 1 -- 172.21.15.133:0/1145378404 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f5a180cf820 con 0x7f5a20109b10 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.508+0000 7f5a2558a640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5a080a2dc0 0x7f5a080a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.508+0000 7f5a2558a640 1 -- 172.21.15.133:0/84756788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325c5e0 msgr2=0x560053266940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.508+0000 7f5a2558a640 1 --2- 172.21.15.133:0/84756788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325c5e0 0x560053266940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.508+0000 7f5a2558a640 1 -- 172.21.15.133:0/84756788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 msgr2=0x560053269270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.508+0000 7f5a2558a640 1 --2- 172.21.15.133:0/84756788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053269270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.508+0000 7f5a2558a640 1 -- 172.21.15.133:0/84756788 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560053167340 con 0x56005325bcc0 2024-09-11T23:43:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.509+0000 7f5a2558a640 1 --2- 172.21.15.133:0/84756788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x56005325c0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a10011680 tx=0x7f5a10011b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49aa87c10ec452f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.509+0000 7f5a2558a640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5a080a2dc0 0x7f5a080a5280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a1001c000 tx=0x7f5a1001e000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.509+0000 7f5a06ffd640 1 -- 172.21.15.133:0/84756788 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1001d7b0 con 0x56005325bcc0 2024-09-11T23:43:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.509+0000 7f5a06ffd640 1 -- 172.21.15.133:0/84756788 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a10022020 con 0x56005325bcc0 2024-09-11T23:43:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.509+0000 7f5a06ffd640 1 -- 172.21.15.133:0/84756788 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1001e980 con 0x56005325bcc0 2024-09-11T23:43:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.511+0000 7f5a29e5d740 1 -- 172.21.15.133:0/84756788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 msgr2=0x56005325c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.511+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/84756788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x56005325c0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a10011680 tx=0x7f5a10011b40 comp rx=0 tx=0).stop 2024-09-11T23:43:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.512+0000 7f5a29e5d740 1 -- 172.21.15.133:0/84756788 shutdown_connections 2024-09-11T23:43:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.512+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/84756788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053269270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.512+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/84756788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325c5e0 0x560053266940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.512+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/84756788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x56005325c0a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.512+0000 7f5a29e5d740 1 -- 172.21.15.133:0/84756788 >> 172.21.15.133:0/84756788 conn(0x560053258230 msgr2=0x560053258610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.512+0000 7f5a29e5d740 1 -- 172.21.15.133:0/84756788 shutdown_connections 2024-09-11T23:43:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.512+0000 7f5a29e5d740 1 -- 172.21.15.133:0/84756788 wait complete. 2024-09-11T23:43:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.513+0000 7f5a29e5d740 1 -- start start 2024-09-11T23:43:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.514+0000 7f5a29e5d740 1 Processor -- start 2024-09-11T23:43:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.515+0000 7f5a29e5d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325bcc0 0x560053263e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.515+0000 7f5a29e5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053264360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.515+0000 7f5a1ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053264360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.515+0000 7f5a1ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053264360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44932/0 (socket says 172.21.15.133:44932) 2024-09-11T23:43:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.515+0000 7f5a1ffff640 1 -- 172.21.15.133:0/2868925569 learned_addr learned my addr 172.21.15.133:0/2868925569 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.515+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600532648a0 0x560053272f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.515+0000 7f5a29e5d740 1 -- 172.21.15.133:0/2868925569 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560053269a50 con 0x560053266e80 2024-09-11T23:43:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.515+0000 7f5a29e5d740 1 -- 172.21.15.133:0/2868925569 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560053149290 con 0x5600532648a0 2024-09-11T23:43:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.515+0000 7f5a29e5d740 1 -- 172.21.15.133:0/2868925569 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560053269bd0 con 0x56005325bcc0 2024-09-11T23:43:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.516+0000 7f5a2558a640 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600532648a0 0x560053272f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.516+0000 7f5a29e5d740 10 client.0 _ll_register_callbacks cb 0x56005317a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.516+0000 7f5a2558a640 1 -- 172.21.15.133:0/2868925569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325bcc0 msgr2=0x560053263e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:43:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.516+0000 7f5a2558a640 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325bcc0 0x560053263e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.516+0000 7f5a2558a640 1 -- 172.21.15.133:0/2868925569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 msgr2=0x560053264360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.516+0000 7f5a2558a640 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053264360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.516+0000 7f5a2558a640 1 -- 172.21.15.133:0/2868925569 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560052ed3210 con 0x5600532648a0 2024-09-11T23:43:22.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.517+0000 7f5a2558a640 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600532648a0 0x560053272f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a10000c00 tx=0x7f5a1001e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f530d0cedafa167a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.524+0000 7f59fffff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.524+0000 7f59fffff640 1 -- 172.21.15.133:0/2868925569 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1001ee20 con 0x5600532648a0 2024-09-11T23:43:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.524+0000 7f5a29e5d740 1 -- 172.21.15.133:0/2868925569 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a10022020 con 0x5600532648a0 2024-09-11T23:43:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.524+0000 7f59fffff640 1 -- 172.21.15.133:0/2868925569 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f5a10005050 con 0x5600532648a0 2024-09-11T23:43:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.524+0000 7f59fffff640 1 -- 172.21.15.133:0/2868925569 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a10005250 con 0x5600532648a0 2024-09-11T23:43:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.524+0000 7f5a29e5d740 10 client.26678 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.524+0000 7f5a29e5d740 1 -- 172.21.15.133:0/2868925569 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560052f9f620 con 0x5600532648a0 2024-09-11T23:43:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.527+0000 7f59fffff640 1 -- 172.21.15.133:0/2868925569 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a10022020 con 0x5600532648a0 2024-09-11T23:43:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.528+0000 7f59fffff640 1 -- 172.21.15.133:0/2868925569 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5a100047b0 con 0x5600532648a0 2024-09-11T23:43:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.528+0000 7f5a29e5d740 10 client.26678 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.528+0000 7f5a29e5d740 10 client.26678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.528+0000 7f5a29e5d740 20 client.26678 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.528+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560053277620 0x560053279ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.528+0000 7f5a29e5d740 4 client.26678 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.528+0000 7f5a29e5d740 1 -- 172.21.15.133:0/2868925569 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600531cda20 con 0x560053277620 2024-09-11T23:43:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.530+0000 7f5a24d89640 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560053277620 0x560053279ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.533+0000 7f59fffff640 10 client.26678 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.533+0000 7f5a24d89640 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560053277620 0x560053279ae0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1a9bb2395d5f5c7 server_cookie=7824c16c099bf0b3 in_seq=0 out_seq=0 2024-09-11T23:43:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.535+0000 7f59fffff640 1 -- 172.21.15.133:0/2868925569 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600531cda20 con 0x560053277620 2024-09-11T23:43:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.536+0000 7f59fffff640 10 client.26678 handle_command_reply: tid=0 2024-09-11T23:43:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.536+0000 7f5a29e5d740 1 client.26678 shutdown 2024-09-11T23:43:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.537+0000 7f5a29e5d740 1 -- 172.21.15.133:0/2868925569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600532648a0 msgr2=0x560053272f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.537+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600532648a0 0x560053272f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a10000c00 tx=0x7f5a1001e6c0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.543+0000 7f5a29e5d740 1 -- 172.21.15.133:0/2868925569 shutdown_connections 2024-09-11T23:43:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.543+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560053277620 0x560053279ae0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.543+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600532648a0 0x560053272f70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.543+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053264360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.543+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/2868925569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325bcc0 0x560053263e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.543+0000 7f5a29e5d740 1 -- 172.21.15.133:0/2868925569 >> 172.21.15.133:0/2868925569 conn(0x560053258230 msgr2=0x56005325df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.544+0000 7f59fffff640 0 client.26678 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.544+0000 7f5a29e5d740 1 -- 172.21.15.133:0/2868925569 shutdown_connections 2024-09-11T23:43:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.544+0000 7f5a29e5d740 1 -- 172.21.15.133:0/2868925569 wait complete. 2024-09-11T23:43:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.548+0000 7f5a29e5d740 20 client.26678 trim_cache size 0 max 16384 2024-09-11T23:43:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.608+0000 7f5a29e5d740 1 Processor -- start 2024-09-11T23:43:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.608+0000 7f5a29e5d740 1 -- start start 2024-09-11T23:43:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.608+0000 7f5a29e5d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325bcc0 0x560053264350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.608+0000 7f5a29e5d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600532648a0 0x560053252ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.608+0000 7f5a29e5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053272d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.609+0000 7f5a29e5d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201bc5a0 con 0x560053266e80 2024-09-11T23:43:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.609+0000 7f5a29e5d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201be0e0 con 0x5600532648a0 2024-09-11T23:43:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.609+0000 7f5a29e5d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201bcc70 con 0x56005325bcc0 2024-09-11T23:43:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.609+0000 7f5a2558a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053272d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.610+0000 7f5a2558a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053272d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44938/0 (socket says 172.21.15.133:44938) 2024-09-11T23:43:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.610+0000 7f5a2558a640 1 -- 172.21.15.133:0/825024777 learned_addr learned my addr 172.21.15.133:0/825024777 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.610+0000 7f5a2558a640 1 -- 172.21.15.133:0/825024777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325bcc0 msgr2=0x560053264350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.610+0000 7f5a2558a640 1 --2- 172.21.15.133:0/825024777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325bcc0 0x560053264350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.610+0000 7f5a2558a640 1 -- 172.21.15.133:0/825024777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600532648a0 msgr2=0x560053252ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.610+0000 7f5a2558a640 1 --2- 172.21.15.133:0/825024777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600532648a0 0x560053252ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.610+0000 7f5a2558a640 1 -- 172.21.15.133:0/825024777 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560053262db0 con 0x560053266e80 2024-09-11T23:43:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.611+0000 7f5a2558a640 1 --2- 172.21.15.133:0/825024777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053272d60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5a10004b80 tx=0x7f5a1001e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a2ea0f3cdf5af8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.612+0000 7f5a067fc640 1 -- 172.21.15.133:0/825024777 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10027420 con 0x560053266e80 2024-09-11T23:43:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.612+0000 7f5a067fc640 1 -- 172.21.15.133:0/825024777 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a1005e020 con 0x560053266e80 2024-09-11T23:43:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.612+0000 7f5a067fc640 1 -- 172.21.15.133:0/825024777 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10004680 con 0x560053266e80 2024-09-11T23:43:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.618+0000 7f5a29e5d740 1 -- 172.21.15.133:0/825024777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 msgr2=0x560053272d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.618+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/825024777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053272d60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5a10004b80 tx=0x7f5a1001e650 comp rx=0 tx=0).stop 2024-09-11T23:43:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.622+0000 7f5a29e5d740 1 -- 172.21.15.133:0/825024777 shutdown_connections 2024-09-11T23:43:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.622+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/825024777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560053266e80 0x560053272d60 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.622+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/825024777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600532648a0 0x560053252ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.622+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/825024777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56005325bcc0 0x560053264350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.623+0000 7f5a29e5d740 1 -- 172.21.15.133:0/825024777 >> 172.21.15.133:0/825024777 conn(0x560053258230 msgr2=0x560053256a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.630+0000 7f5a29e5d740 1 -- 172.21.15.133:0/825024777 shutdown_connections 2024-09-11T23:43:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.630+0000 7f5a29e5d740 1 -- 172.21.15.133:0/825024777 wait complete. 2024-09-11T23:43:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.631+0000 7f5a29e5d740 1 -- start start 2024-09-11T23:43:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.631+0000 7f5a29e5d740 1 Processor -- start 2024-09-11T23:43:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.632+0000 7f5a29e5d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x560053261750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.632+0000 7f5a29e5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600532648a0 0x56005326eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.633+0000 7f5a24d89640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x560053261750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.633+0000 7f5a24d89640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x560053261750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40420/0 (socket says 172.21.15.133:40420) 2024-09-11T23:43:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.634+0000 7f5a1ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600532648a0 0x56005326eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.635+0000 7f5a24d89640 1 -- 172.21.15.133:0/4102420708 learned_addr learned my addr 172.21.15.133:0/4102420708 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.635+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560053266e80 0x56005326f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.635+0000 7f5a29e5d740 1 -- 172.21.15.133:0/4102420708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201bda10 con 0x5600532648a0 2024-09-11T23:43:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.635+0000 7f5a29e5d740 1 -- 172.21.15.133:0/4102420708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201bd340 con 0x56005325bcc0 2024-09-11T23:43:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.635+0000 7f5a29e5d740 1 -- 172.21.15.133:0/4102420708 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201ba5a0 con 0x560053266e80 2024-09-11T23:43:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.636+0000 7f5a29e5d740 10 client.0 _ll_register_callbacks cb 0x7f5a100114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.636+0000 7f5a2558a640 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560053266e80 0x56005326f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.637+0000 7f5a24d89640 1 -- 172.21.15.133:0/4102420708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560053266e80 msgr2=0x56005326f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.637+0000 7f5a24d89640 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560053266e80 0x56005326f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.637+0000 7f5a24d89640 1 -- 172.21.15.133:0/4102420708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600532648a0 msgr2=0x56005326eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.637+0000 7f5a24d89640 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600532648a0 0x56005326eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.637+0000 7f5a24d89640 1 -- 172.21.15.133:0/4102420708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005326fa50 con 0x56005325bcc0 2024-09-11T23:43:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.638+0000 7f5a24d89640 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x560053261750 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a140096c0 tx=0x7f5a140041b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e4ca310199a0c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.642+0000 7f59fffff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.642+0000 7f59fffff640 1 -- 172.21.15.133:0/4102420708 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14035040 con 0x56005325bcc0 2024-09-11T23:43:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.642+0000 7f59fffff640 1 -- 172.21.15.133:0/4102420708 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f5a14005040 con 0x56005325bcc0 2024-09-11T23:43:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.642+0000 7f5a29e5d740 1 -- 172.21.15.133:0/4102420708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1005e020 con 0x56005325bcc0 2024-09-11T23:43:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.643+0000 7f59fffff640 1 -- 172.21.15.133:0/4102420708 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a14050050 con 0x56005325bcc0 2024-09-11T23:43:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.647+0000 7f5a29e5d740 10 client.26681 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.647+0000 7f5a29e5d740 1 -- 172.21.15.133:0/4102420708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600532e1420 con 0x56005325bcc0 2024-09-11T23:43:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.648+0000 7f59fffff640 1 -- 172.21.15.133:0/4102420708 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a140046c0 con 0x56005325bcc0 2024-09-11T23:43:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.650+0000 7f59fffff640 1 -- 172.21.15.133:0/4102420708 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5a14005240 con 0x56005325bcc0 2024-09-11T23:43:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.650+0000 7f5a29e5d740 10 client.26681 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.650+0000 7f5a29e5d740 10 client.26681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.650+0000 7f5a29e5d740 20 client.26681 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.650+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5600532e1b20 0x5600532e3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.650+0000 7f5a29e5d740 4 client.26681 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:43:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.650+0000 7f5a29e5d740 1 -- 172.21.15.133:0/4102420708 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5a1005e020 con 0x5600532e1b20 2024-09-11T23:43:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.654+0000 7f5a1ffff640 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5600532e1b20 0x5600532e3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.680+0000 7f5a1ffff640 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5600532e1b20 0x5600532e3f40 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57577565005bfe62 server_cookie=59e3ac84fd19503f in_seq=0 out_seq=0 2024-09-11T23:43:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:22.680+0000 7f59fffff640 10 client.26681 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:22.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:22 smithi133 ceph-mon[31430]: pgmap v203: 97 pgs: 97 active+clean; 50 GiB data, 167 GiB used, 906 GiB / 1.0 TiB avail; 269 KiB/s rd, 235 MiB/s wr, 598 op/s 2024-09-11T23:43:22.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:22 smithi178 ceph-mon[35882]: pgmap v203: 97 pgs: 97 active+clean; 50 GiB data, 167 GiB used, 906 GiB / 1.0 TiB avail; 269 KiB/s rd, 235 MiB/s wr, 598 op/s 2024-09-11T23:43:22.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:22 smithi138 ceph-mon[35142]: pgmap v203: 97 pgs: 97 active+clean; 50 GiB data, 167 GiB used, 906 GiB / 1.0 TiB avail; 269 KiB/s rd, 235 MiB/s wr, 598 op/s 2024-09-11T23:43:23.048 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:24.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.064+0000 7f1bc110e640 1 -- 172.21.15.133:0/1563750426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bbc109830 msgr2=0x7f1bbc109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.064+0000 7f1bc110e640 1 --2- 172.21.15.133:0/1563750426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bbc109830 0x7f1bbc109c90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009900 tx=0x7f1bb002a440 comp rx=0 tx=0).stop 2024-09-11T23:43:24.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.065+0000 7f1bc110e640 1 -- 172.21.15.133:0/1563750426 shutdown_connections 2024-09-11T23:43:24.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.065+0000 7f1bc110e640 1 --2- 172.21.15.133:0/1563750426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bbc11b140 0x7f1bbc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.065+0000 7f1bc110e640 1 --2- 172.21.15.133:0/1563750426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bbc109830 0x7f1bbc109c90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.065+0000 7f1bc110e640 1 --2- 172.21.15.133:0/1563750426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bbc109450 0x7f1bbc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.065+0000 7f1bc110e640 1 -- 172.21.15.133:0/1563750426 >> 172.21.15.133:0/1563750426 conn(0x7f1bbc06e140 msgr2=0x7f1bbc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:24.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.066+0000 7f1bc110e640 1 -- 172.21.15.133:0/1563750426 shutdown_connections 2024-09-11T23:43:24.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.066+0000 7f1bc110e640 1 -- 172.21.15.133:0/1563750426 wait complete. 2024-09-11T23:43:24.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.068+0000 7f1bc110e640 1 Processor -- start 2024-09-11T23:43:24.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.068+0000 7f1bc110e640 1 -- start start 2024-09-11T23:43:24.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.068+0000 7f1bc110e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bbc109450 0x7f1bbc1a7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.068+0000 7f1bc110e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bbc109830 0x7f1bbc1a7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.068+0000 7f1bc110e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bbc11b140 0x7f1bbc1abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.068+0000 7f1bc110e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc120df0 con 0x7f1bbc109830 2024-09-11T23:43:24.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.068+0000 7f1bc110e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc120c70 con 0x7f1bbc109450 2024-09-11T23:43:24.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.068+0000 7f1bc110e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc120f70 con 0x7f1bbc11b140 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.069+0000 7f1bbad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bbc109450 0x7f1bbc1a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.069+0000 7f1bbad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bbc109450 0x7f1bbc1a7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40442/0 (socket says 172.21.15.133:40442) 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.069+0000 7f1bbad9d640 1 -- 172.21.15.133:0/340788316 learned_addr learned my addr 172.21.15.133:0/340788316 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.069+0000 7f1bbb59e640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bbc11b140 0x7f1bbc1abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.070+0000 7f1bba59c640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bbc109830 0x7f1bbc1a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.070+0000 7f1bbad9d640 1 -- 172.21.15.133:0/340788316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bbc11b140 msgr2=0x7f1bbc1abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.070+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bbc11b140 0x7f1bbc1abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.070+0000 7f1bbad9d640 1 -- 172.21.15.133:0/340788316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bbc109830 msgr2=0x7f1bbc1a7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.070+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bbc109830 0x7f1bbc1a7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.070+0000 7f1bbad9d640 1 -- 172.21.15.133:0/340788316 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc1ac220 con 0x7f1bbc109450 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.070+0000 7f1bba59c640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bbc109830 0x7f1bbc1a7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:24.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.070+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bbc109450 0x7f1bbc1a7250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00d950 tx=0x7f1bac00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:24.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.073+0000 7f1ba3fff640 1 -- 172.21.15.133:0/340788316 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac017070 con 0x7f1bbc109450 2024-09-11T23:43:24.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.073+0000 7f1bc110e640 1 -- 172.21.15.133:0/340788316 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc1ac510 con 0x7f1bbc109450 2024-09-11T23:43:24.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.074+0000 7f1bc110e640 1 -- 172.21.15.133:0/340788316 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc110920 con 0x7f1bbc109450 2024-09-11T23:43:24.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.079+0000 7f1bc34e7740 1 Processor -- start 2024-09-11T23:43:24.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.079+0000 7f1bc34e7740 1 -- start start 2024-09-11T23:43:24.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.079+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e1cb60 0x5631a8e1cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.079+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1d480 0x5631a8e27800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.079+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e27d40 0x5631a8e2a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5631a8d82280 con 0x5631a8e27d40 2024-09-11T23:43:24.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5631a8d29b10 con 0x5631a8e1d480 2024-09-11T23:43:24.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5631a8d7e240 con 0x5631a8e1cb60 2024-09-11T23:43:24.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1ba3fff640 1 -- 172.21.15.133:0/340788316 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac009c30 con 0x7f1bbc109450 2024-09-11T23:43:24.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1ba3fff640 1 -- 172.21.15.133:0/340788316 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac00f6b0 con 0x7f1bbc109450 2024-09-11T23:43:24.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1bba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e27d40 0x5631a8e2a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1bba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e27d40 0x5631a8e2a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44992/0 (socket says 172.21.15.133:44992) 2024-09-11T23:43:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1bba59c640 1 -- 172.21.15.133:0/3678810426 learned_addr learned my addr 172.21.15.133:0/3678810426 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1bba59c640 1 -- 172.21.15.133:0/3678810426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e1cb60 msgr2=0x5631a8e1cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1bba59c640 1 --2- 172.21.15.133:0/3678810426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e1cb60 0x5631a8e1cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1bba59c640 1 -- 172.21.15.133:0/3678810426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1d480 msgr2=0x5631a8e27800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1bba59c640 1 --2- 172.21.15.133:0/3678810426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1d480 0x5631a8e27800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.080+0000 7f1bba59c640 1 -- 172.21.15.133:0/3678810426 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a8d361f0 con 0x5631a8e27d40 2024-09-11T23:43:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.081+0000 7f1bba59c640 1 --2- 172.21.15.133:0/3678810426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e27d40 0x5631a8e2a130 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0006010 tx=0x7f1bb0003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e477ff49f25665e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.081+0000 7f1ba17fa640 1 -- 172.21.15.133:0/3678810426 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0041070 con 0x5631a8e27d40 2024-09-11T23:43:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.081+0000 7f1ba17fa640 1 -- 172.21.15.133:0/3678810426 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb0005000 con 0x5631a8e27d40 2024-09-11T23:43:24.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.081+0000 7f1ba17fa640 1 -- 172.21.15.133:0/3678810426 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0031900 con 0x5631a8e27d40 2024-09-11T23:43:24.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.083+0000 7f1bc34e7740 1 -- 172.21.15.133:0/3678810426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e27d40 msgr2=0x5631a8e2a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.083+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/3678810426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e27d40 0x5631a8e2a130 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0006010 tx=0x7f1bb0003800 comp rx=0 tx=0).stop 2024-09-11T23:43:24.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.083+0000 7f1bc34e7740 1 -- 172.21.15.133:0/3678810426 shutdown_connections 2024-09-11T23:43:24.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.083+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/3678810426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e27d40 0x5631a8e2a130 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.083+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/3678810426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1d480 0x5631a8e27800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.083+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/3678810426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e1cb60 0x5631a8e1cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.083+0000 7f1bc34e7740 1 -- 172.21.15.133:0/3678810426 >> 172.21.15.133:0/3678810426 conn(0x5631a8e190f0 msgr2=0x5631a8e194d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.084+0000 7f1ba3fff640 1 -- 172.21.15.133:0/340788316 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1bac00f8d0 con 0x7f1bbc109450 2024-09-11T23:43:24.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.086+0000 7f1ba3fff640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1ba40a2f00 0x7f1ba40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.087+0000 7f1ba3fff640 1 -- 172.21.15.133:0/340788316 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f1bac0cff80 con 0x7f1bbc109450 2024-09-11T23:43:24.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.089+0000 7f1bba59c640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1ba40a2f00 0x7f1ba40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.090+0000 7f1bc34e7740 1 -- 172.21.15.133:0/3678810426 shutdown_connections 2024-09-11T23:43:24.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.090+0000 7f1bba59c640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1ba40a2f00 0x7f1ba40a53c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00037d0 tx=0x7f1bb0000bc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:24.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.096+0000 7f1bc34e7740 1 -- 172.21.15.133:0/3678810426 wait complete. 2024-09-11T23:43:24.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.097+0000 7f1bc34e7740 1 -- start start 2024-09-11T23:43:24.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.097+0000 7f1bc34e7740 1 Processor -- start 2024-09-11T23:43:24.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.099+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e24c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.099+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e1d480 0x5631a8e25160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.099+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e27d40 0x5631a8e256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.099+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5631a8e2a910 con 0x5631a8e27d40 2024-09-11T23:43:24.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.099+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5631a8d3cf10 con 0x5631a8e1cb60 2024-09-11T23:43:24.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.099+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5631a8e2aa90 con 0x5631a8e1d480 2024-09-11T23:43:24.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.099+0000 7f1bc34e7740 10 client.0 _ll_register_callbacks cb 0x5631a8d05a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.099+0000 7f1bbad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e24c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.099+0000 7f1bbad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e24c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40462/0 (socket says 172.21.15.133:40462) 2024-09-11T23:43:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.099+0000 7f1bbad9d640 1 -- 172.21.15.133:0/4172432281 learned_addr learned my addr 172.21.15.133:0/4172432281 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.100+0000 7f1bba59c640 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e1d480 0x5631a8e25160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.100+0000 7f1bbad9d640 1 -- 172.21.15.133:0/4172432281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e1d480 msgr2=0x5631a8e25160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.100+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e1d480 0x5631a8e25160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.100+0000 7f1bbad9d640 1 -- 172.21.15.133:0/4172432281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e27d40 msgr2=0x5631a8e256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.100+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e27d40 0x5631a8e256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.100+0000 7f1bbad9d640 1 -- 172.21.15.133:0/4172432281 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a8af36a0 con 0x5631a8e1cb60 2024-09-11T23:43:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.101+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e24c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0a4c60 tx=0x7f1bac09bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17081f69d06cabeb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.101+0000 7f1b9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.101+0000 7f1b9eff5640 1 -- 172.21.15.133:0/4172432281 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac020050 con 0x5631a8e1cb60 2024-09-11T23:43:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.101+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4172432281 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bb0005000 con 0x5631a8e1cb60 2024-09-11T23:43:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.101+0000 7f1b9eff5640 1 -- 172.21.15.133:0/4172432281 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f1bac0109d0 con 0x5631a8e1cb60 2024-09-11T23:43:24.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.102+0000 7f1b9eff5640 1 -- 172.21.15.133:0/4172432281 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1bac09c520 con 0x5631a8e1cb60 2024-09-11T23:43:24.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.103+0000 7f1bc34e7740 10 client.26690 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:24.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.103+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4172432281 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5631a8e365c0 con 0x5631a8e1cb60 2024-09-11T23:43:24.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.103+0000 7f1b9eff5640 1 -- 172.21.15.133:0/4172432281 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb0005000 con 0x5631a8e1cb60 2024-09-11T23:43:24.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.104+0000 7f1b9eff5640 1 -- 172.21.15.133:0/4172432281 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1bac0bc310 con 0x5631a8e1cb60 2024-09-11T23:43:24.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.105+0000 7f1bc34e7740 10 client.26690 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:24.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.105+0000 7f1bc34e7740 10 client.26690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.105+0000 7f1bc34e7740 20 client.26690 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.105+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5631a8e37df0 0x5631a8e3a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.105+0000 7f1bc34e7740 4 client.26690 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.105+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4172432281 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631a8d8f7f0 con 0x5631a8e37df0 2024-09-11T23:43:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.105+0000 7f1bba59c640 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5631a8e37df0 0x5631a8e3a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.105+0000 7f1bba59c640 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5631a8e37df0 0x5631a8e3a260 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b72bf376709874 server_cookie=8d5698152add1197 in_seq=0 out_seq=0 2024-09-11T23:43:24.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.105+0000 7f1b9eff5640 10 client.26690 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:24.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1b9eff5640 1 -- 172.21.15.133:0/4172432281 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631a8d8f7f0 con 0x5631a8e37df0 2024-09-11T23:43:24.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1b9eff5640 10 client.26690 handle_command_reply: tid=0 2024-09-11T23:43:24.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1bc34e7740 1 client.26690 shutdown 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4172432281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 msgr2=0x5631a8e24c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e24c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0a4c60 tx=0x7f1bac09bf00 comp rx=0 tx=0).stop 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4172432281 shutdown_connections 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5631a8e37df0 0x5631a8e3a260 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e27d40 0x5631a8e256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e1d480 0x5631a8e25160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/4172432281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e24c20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4172432281 >> 172.21.15.133:0/4172432281 conn(0x5631a8e190f0 msgr2=0x5631a8e18270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.107+0000 7f1b9eff5640 0 client.26690 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.108+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4172432281 shutdown_connections 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.108+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4172432281 wait complete. 2024-09-11T23:43:24.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.108+0000 7f1bc34e7740 20 client.26690 trim_cache size 0 max 16384 2024-09-11T23:43:24.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.127+0000 7f1bc34e7740 1 Processor -- start 2024-09-11T23:43:24.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.127+0000 7f1bc34e7740 1 -- start start 2024-09-11T23:43:24.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e26a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e1d480 0x5631a8e3fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e27d40 0x5631a8e400b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1c4de0 con 0x5631a8e1d480 2024-09-11T23:43:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1c6920 con 0x5631a8e1cb60 2024-09-11T23:43:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1c54b0 con 0x5631a8e27d40 2024-09-11T23:43:24.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bbad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e26a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bbad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e26a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40466/0 (socket says 172.21.15.133:40466) 2024-09-11T23:43:24.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bbad9d640 1 -- 172.21.15.133:0/4293981878 learned_addr learned my addr 172.21.15.133:0/4293981878 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:24.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bbb59e640 1 --2- 172.21.15.133:0/4293981878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e27d40 0x5631a8e400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bba59c640 1 --2- 172.21.15.133:0/4293981878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e1d480 0x5631a8e3fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bbad9d640 1 -- 172.21.15.133:0/4293981878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e27d40 msgr2=0x5631a8e400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:24.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/4293981878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e27d40 0x5631a8e400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bbad9d640 1 -- 172.21.15.133:0/4293981878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e1d480 msgr2=0x5631a8e3fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:24.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/4293981878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e1d480 0x5631a8e3fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.128+0000 7f1bbad9d640 1 -- 172.21.15.133:0/4293981878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a8e135f0 con 0x5631a8e1cb60 2024-09-11T23:43:24.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.129+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/4293981878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e26a80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1bac09c220 tx=0x7f1bac00bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=541115988afea469 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:24.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.129+0000 7f1ba0ff9640 1 -- 172.21.15.133:0/4293981878 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac020040 con 0x5631a8e1cb60 2024-09-11T23:43:24.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.129+0000 7f1ba0ff9640 1 -- 172.21.15.133:0/4293981878 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac0a5190 con 0x5631a8e1cb60 2024-09-11T23:43:24.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.129+0000 7f1ba0ff9640 1 -- 172.21.15.133:0/4293981878 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac0b52d0 con 0x5631a8e1cb60 2024-09-11T23:43:24.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.130+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4293981878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 msgr2=0x5631a8e26a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.130+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/4293981878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e26a80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1bac09c220 tx=0x7f1bac00bf60 comp rx=0 tx=0).stop 2024-09-11T23:43:24.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.131+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4293981878 shutdown_connections 2024-09-11T23:43:24.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.131+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/4293981878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e27d40 0x5631a8e400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.131+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/4293981878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e1d480 0x5631a8e3fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.131+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/4293981878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e26a80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.131+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4293981878 >> 172.21.15.133:0/4293981878 conn(0x5631a8e190f0 msgr2=0x5631a8e26370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.131+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4293981878 shutdown_connections 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.132+0000 7f1bc34e7740 1 -- 172.21.15.133:0/4293981878 wait complete. 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.132+0000 7f1bc34e7740 1 -- start start 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.132+0000 7f1bc34e7740 1 Processor -- start 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.133+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e2df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.133+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e1d480 0x5631a8e2e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.133+0000 7f1bc34e7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e27d40 0x5631a8ea1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.133+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1c6250 con 0x5631a8e1d480 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.133+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1c5b80 con 0x5631a8e1cb60 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.133+0000 7f1bc34e7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc117130 con 0x5631a8e27d40 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.133+0000 7f1bc34e7740 10 client.0 _ll_register_callbacks cb 0x7f1bac00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.134+0000 7f1bbb59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e27d40 0x5631a8ea1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.134+0000 7f1bbb59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e27d40 0x5631a8ea1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:54900/0 (socket says 172.21.15.133:54900) 2024-09-11T23:43:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.134+0000 7f1bbb59e640 1 -- 172.21.15.133:0/2965634735 learned_addr learned my addr 172.21.15.133:0/2965634735 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.134+0000 7f1bba59c640 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e1d480 0x5631a8e2e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.134+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e2df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.134+0000 7f1bba59c640 1 -- 172.21.15.133:0/2965634735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e27d40 msgr2=0x5631a8ea1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.134+0000 7f1bba59c640 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e27d40 0x5631a8ea1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.134+0000 7f1bba59c640 1 -- 172.21.15.133:0/2965634735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 msgr2=0x5631a8e2df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.134+0000 7f1bba59c640 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e2df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.134+0000 7f1bba59c640 1 -- 172.21.15.133:0/2965634735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a8ea2090 con 0x5631a8e1d480 2024-09-11T23:43:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.138+0000 7f1bba59c640 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e1d480 0x5631a8e2e480 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0004fd0 tx=0x7f1bb0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf099fff25c4b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.138+0000 7f1b9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.139+0000 7f1b9eff5640 1 -- 172.21.15.133:0/2965634735 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0041070 con 0x5631a8e1d480 2024-09-11T23:43:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.139+0000 7f1b9eff5640 1 -- 172.21.15.133:0/2965634735 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f1bb0031750 con 0x5631a8e1d480 2024-09-11T23:43:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.139+0000 7f1bc34e7740 1 -- 172.21.15.133:0/2965634735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bac0a5190 con 0x5631a8e1d480 2024-09-11T23:43:24.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.139+0000 7f1b9eff5640 1 -- 172.21.15.133:0/2965634735 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1bb0031950 con 0x5631a8e1d480 2024-09-11T23:43:24.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.139+0000 7f1bc34e7740 10 client.17217 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:24.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.139+0000 7f1bc34e7740 1 -- 172.21.15.133:0/2965634735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5631a8ea3270 con 0x5631a8e1d480 2024-09-11T23:43:24.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.140+0000 7f1b9eff5640 1 -- 172.21.15.133:0/2965634735 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac0a5190 con 0x5631a8e1d480 2024-09-11T23:43:24.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.141+0000 7f1b9eff5640 1 -- 172.21.15.133:0/2965634735 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1bb0068080 con 0x5631a8e1d480 2024-09-11T23:43:24.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.141+0000 7f1bc34e7740 10 client.17217 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.141+0000 7f1bc34e7740 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.141+0000 7f1bc34e7740 20 client.17217 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.141+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5631a8ea34f0 0x5631a8ea58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.141+0000 7f1bc34e7740 4 client.17217 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:43:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.141+0000 7f1bc34e7740 1 -- 172.21.15.133:0/2965634735 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5631a8d89d00 con 0x5631a8ea34f0 2024-09-11T23:43:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.141+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5631a8ea34f0 0x5631a8ea58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:24.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.143+0000 7f1b9eff5640 10 client.17217 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:24.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.143+0000 7f1bbad9d640 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5631a8ea34f0 0x5631a8ea58e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df6101c135f9e20 server_cookie=d0e6a57a695cebae in_seq=0 out_seq=0 2024-09-11T23:43:24.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.212+0000 7f1b9eff5640 1 -- 172.21.15.133:0/2965634735 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5631a8d89d00 con 0x5631a8ea34f0 2024-09-11T23:43:24.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.212+0000 7f1b9eff5640 10 client.17217 handle_command_reply: tid=0 2024-09-11T23:43:24.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.212+0000 7f1bc34e7740 1 client.17217 shutdown 2024-09-11T23:43:24.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.213+0000 7f1bc34e7740 1 -- 172.21.15.133:0/2965634735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e1d480 msgr2=0x5631a8e2e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.213+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e1d480 0x5631a8e2e480 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0004fd0 tx=0x7f1bb0009260 comp rx=0 tx=0).stop 2024-09-11T23:43:24.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.213+0000 7f1bc34e7740 1 -- 172.21.15.133:0/2965634735 shutdown_connections 2024-09-11T23:43:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.213+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5631a8ea34f0 0x5631a8ea58e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.213+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5631a8e27d40 0x5631a8ea1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.213+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5631a8e1d480 0x5631a8e2e480 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.213+0000 7f1bc34e7740 1 --2- 172.21.15.133:0/2965634735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a8e1cb60 0x5631a8e2df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.213+0000 7f1b9eff5640 0 client.17217 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.213+0000 7f1bc34e7740 1 -- 172.21.15.133:0/2965634735 >> 172.21.15.133:0/2965634735 conn(0x5631a8e190f0 msgr2=0x5631a8e18270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.213+0000 7f1bc34e7740 1 -- 172.21.15.133:0/2965634735 shutdown_connections 2024-09-11T23:43:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.213+0000 7f1bc34e7740 1 -- 172.21.15.133:0/2965634735 wait complete. 2024-09-11T23:43:24.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.214+0000 7f1bc34e7740 20 client.17217 trim_cache size 0 max 16384 2024-09-11T23:43:24.214 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:43:24.215 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:43:24.215 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "e2ccb98d-db32-4f68-befd-f658aab0a814", 2024-09-11T23:43:24.215 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:43:24.215 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:43:24.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 -- 172.21.15.133:0/340788316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1ba40a2f00 msgr2=0x7f1ba40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1ba40a2f00 0x7f1ba40a53c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00037d0 tx=0x7f1bb0000bc0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 -- 172.21.15.133:0/340788316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bbc109450 msgr2=0x7f1bbc1a7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bbc109450 0x7f1bbc1a7250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00d950 tx=0x7f1bac00de10 comp rx=0 tx=0).stop 2024-09-11T23:43:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 -- 172.21.15.133:0/340788316 shutdown_connections 2024-09-11T23:43:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1ba40a2f00 0x7f1ba40a53c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bbc11b140 0x7f1bbc1abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bbc109830 0x7f1bbc1a7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 --2- 172.21.15.133:0/340788316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bbc109450 0x7f1bbc1a7250 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 -- 172.21.15.133:0/340788316 >> 172.21.15.133:0/340788316 conn(0x7f1bbc06e140 msgr2=0x7f1bbc11c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 -- 172.21.15.133:0/340788316 shutdown_connections 2024-09-11T23:43:24.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:24.215+0000 7f1bc110e640 1 -- 172.21.15.133:0/340788316 wait complete. 2024-09-11T23:43:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.222+0000 7f59fffff640 1 -- 172.21.15.133:0/4102420708 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5a1005e020 con 0x5600532e1b20 2024-09-11T23:43:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.222+0000 7f59fffff640 10 client.26681 handle_command_reply: tid=0 2024-09-11T23:43:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.222+0000 7f5a29e5d740 1 client.26681 shutdown 2024-09-11T23:43:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.223+0000 7f5a29e5d740 1 -- 172.21.15.133:0/4102420708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 msgr2=0x560053261750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.223+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x560053261750 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a140096c0 tx=0x7f5a140041b0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.223+0000 7f5a29e5d740 1 -- 172.21.15.133:0/4102420708 shutdown_connections 2024-09-11T23:43:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.223+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5600532e1b20 0x5600532e3f40 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.223+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560053266e80 0x56005326f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.223+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600532648a0 0x56005326eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.223+0000 7f5a29e5d740 1 --2- 172.21.15.133:0/4102420708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005325bcc0 0x560053261750 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.223+0000 7f5a29e5d740 1 -- 172.21.15.133:0/4102420708 >> 172.21.15.133:0/4102420708 conn(0x560053258230 msgr2=0x560053257260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.223+0000 7f59fffff640 0 client.26681 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.223+0000 7f5a29e5d740 1 -- 172.21.15.133:0/4102420708 shutdown_connections 2024-09-11T23:43:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.223+0000 7f5a29e5d740 1 -- 172.21.15.133:0/4102420708 wait complete. 2024-09-11T23:43:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.224+0000 7f5a29e5d740 20 client.26681 trim_cache size 0 max 16384 2024-09-11T23:43:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:43:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:43:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:43:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:43:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.227+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1145378404 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5a080a2dc0 msgr2=0x7f5a080a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.227+0000 7f5a25d8b640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5a080a2dc0 0x7f5a080a5280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a1001c000 tx=0x7f5a1001e000 comp rx=0 tx=0).stop 2024-09-11T23:43:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.227+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1145378404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a20109b10 msgr2=0x7f5a2010a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.227+0000 7f5a25d8b640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a20109b10 0x7f5a2010a910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800e9f0 tx=0x7f5a1800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.236+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1145378404 shutdown_connections 2024-09-11T23:43:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.236+0000 7f5a25d8b640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5a080a2dc0 0x7f5a080a5280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.236+0000 7f5a25d8b640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a20114ab0 0x7f5a2010ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.236+0000 7f5a25d8b640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a20109b10 0x7f5a2010a910 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.236+0000 7f5a25d8b640 1 --2- 172.21.15.133:0/1145378404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a201091f0 0x7f5a2010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.236+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1145378404 >> 172.21.15.133:0/1145378404 conn(0x7f5a2006cbf0 msgr2=0x7f5a200707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.239+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1145378404 shutdown_connections 2024-09-11T23:43:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:24.239+0000 7f5a25d8b640 1 -- 172.21.15.133:0/1145378404 wait complete. 2024-09-11T23:43:24.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:24 smithi138 ceph-mon[35142]: pgmap v204: 97 pgs: 97 active+clean; 51 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 441 MiB/s wr, 1.99k op/s 2024-09-11T23:43:24.963 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:24 smithi133 ceph-mon[31430]: pgmap v204: 97 pgs: 97 active+clean; 51 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 441 MiB/s wr, 1.99k op/s 2024-09-11T23:43:25.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:24 smithi178 ceph-mon[35882]: pgmap v204: 97 pgs: 97 active+clean; 51 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 441 MiB/s wr, 1.99k op/s 2024-09-11T23:43:25.096 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:43:25.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:25 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:43:25.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:25 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:25.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:25 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:43:25.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:25 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:25.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:25 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:43:25.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:25 smithi133 ceph-mon[31430]: pgmap v205: 97 pgs: 97 active+clean; 51 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 506 MiB/s wr, 2.31k op/s 2024-09-11T23:43:25.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:25 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:43:25.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:25 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:25.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:25 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:43:25.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:25 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:25.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:25 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:43:25.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:25 smithi138 ceph-mon[35142]: pgmap v205: 97 pgs: 97 active+clean; 51 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 506 MiB/s wr, 2.31k op/s 2024-09-11T23:43:25.964 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:26.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:25 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:43:26.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:25 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:26.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:25 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:43:26.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:25 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:26.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:25 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:43:26.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:25 smithi178 ceph-mon[35882]: pgmap v205: 97 pgs: 97 active+clean; 51 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 506 MiB/s wr, 2.31k op/s 2024-09-11T23:43:27.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.157+0000 7ff2f3576640 1 -- 172.21.15.133:0/1010788727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec113940 msgr2=0x7ff2ec115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:27.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.157+0000 7ff2f3576640 1 --2- 172.21.15.133:0/1010788727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec113940 0x7ff2ec115d30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009900 tx=0x7ff2e802a420 comp rx=0 tx=0).stop 2024-09-11T23:43:27.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.157+0000 7ff2f3576640 1 -- 172.21.15.133:0/1010788727 shutdown_connections 2024-09-11T23:43:27.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.157+0000 7ff2f3576640 1 --2- 172.21.15.133:0/1010788727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec113940 0x7ff2ec115d30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.157+0000 7ff2f3576640 1 --2- 172.21.15.133:0/1010788727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2ec10ae80 0x7ff2ec113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.157+0000 7ff2f3576640 1 --2- 172.21.15.133:0/1010788727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2ec10a560 0x7ff2ec10a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.157+0000 7ff2f3576640 1 -- 172.21.15.133:0/1010788727 >> 172.21.15.133:0/1010788727 conn(0x7ff2ec06e140 msgr2=0x7ff2ec06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:27.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.157+0000 7ff2f3576640 1 -- 172.21.15.133:0/1010788727 shutdown_connections 2024-09-11T23:43:27.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.158+0000 7ff2f3576640 1 -- 172.21.15.133:0/1010788727 wait complete. 2024-09-11T23:43:27.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.159+0000 7ff2f3576640 1 Processor -- start 2024-09-11T23:43:27.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.164+0000 7ff2f3576640 1 -- start start 2024-09-11T23:43:27.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.165+0000 7ff2f3576640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2ec10a560 0x7ff2ec1a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.166+0000 7ff2f3576640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2ec10ae80 0x7ff2ec1a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.166+0000 7ff2f3576640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec113940 0x7ff2ec19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.166+0000 7ff2f3576640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1196b0 con 0x7ff2ec10ae80 2024-09-11T23:43:27.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.166+0000 7ff2f3576640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec119530 con 0x7ff2ec10a560 2024-09-11T23:43:27.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.166+0000 7ff2f3576640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec119830 con 0x7ff2ec113940 2024-09-11T23:43:27.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.167+0000 7ff2f1312640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2ec10a560 0x7ff2ec1a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.167+0000 7ff2f0b11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2ec10ae80 0x7ff2ec1a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.167+0000 7ff2f0b11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2ec10ae80 0x7ff2ec1a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45042/0 (socket says 172.21.15.133:45042) 2024-09-11T23:43:27.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.167+0000 7ff2f0b11640 1 -- 172.21.15.133:0/3010477116 learned_addr learned my addr 172.21.15.133:0/3010477116 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:27.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.167+0000 7ff2f1b13640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec113940 0x7ff2ec19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.167+0000 7ff2f0b11640 1 -- 172.21.15.133:0/3010477116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec113940 msgr2=0x7ff2ec19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:27.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.167+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec113940 0x7ff2ec19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.167+0000 7ff2f0b11640 1 -- 172.21.15.133:0/3010477116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2ec10a560 msgr2=0x7ff2ec1a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:27.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.167+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2ec10a560 0x7ff2ec1a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.167+0000 7ff2f0b11640 1 -- 172.21.15.133:0/3010477116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec19f4c0 con 0x7ff2ec10ae80 2024-09-11T23:43:27.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.167+0000 7ff2f1b13640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec113940 0x7ff2ec19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:27.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.169+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2ec10ae80 0x7ff2ec1a6df0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00d950 tx=0x7ff2dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:27.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.169+0000 7ff2e27fc640 1 -- 172.21.15.133:0/3010477116 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc017070 con 0x7ff2ec10ae80 2024-09-11T23:43:27.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.170+0000 7ff2f3576640 1 -- 172.21.15.133:0/3010477116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ec19f7b0 con 0x7ff2ec10ae80 2024-09-11T23:43:27.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.170+0000 7ff2f3576640 1 -- 172.21.15.133:0/3010477116 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ec1bc590 con 0x7ff2ec10ae80 2024-09-11T23:43:27.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.172+0000 7ff2e27fc640 1 -- 172.21.15.133:0/3010477116 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2dc009c30 con 0x7ff2ec10ae80 2024-09-11T23:43:27.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.172+0000 7ff2e27fc640 1 -- 172.21.15.133:0/3010477116 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc020060 con 0x7ff2ec10ae80 2024-09-11T23:43:27.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.176+0000 7ff2f594f740 1 Processor -- start 2024-09-11T23:43:27.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.176+0000 7ff2f594f740 1 -- start start 2024-09-11T23:43:27.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.176+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af6fa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.176+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af6fadc0 0x5587af705140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.177+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af705680 0x5587af707a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.177+0000 7ff2f594f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5587af6602e0 con 0x5587af6fadc0 2024-09-11T23:43:27.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.177+0000 7ff2f594f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5587af607ce0 con 0x5587af705680 2024-09-11T23:43:27.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.177+0000 7ff2f594f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5587af65c2a0 con 0x5587af6fa4a0 2024-09-11T23:43:27.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.177+0000 7ff2f0b11640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af705680 0x5587af707a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.177+0000 7ff2f0b11640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af705680 0x5587af707a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40494/0 (socket says 172.21.15.133:40494) 2024-09-11T23:43:27.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.177+0000 7ff2f0b11640 1 -- 172.21.15.133:0/2010225758 learned_addr learned my addr 172.21.15.133:0/2010225758 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:27.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.178+0000 7ff2f0b11640 1 -- 172.21.15.133:0/2010225758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 msgr2=0x5587af6fa880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:27.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.178+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/2010225758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af6fa880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.179+0000 7ff2f0b11640 1 -- 172.21.15.133:0/2010225758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af6fadc0 msgr2=0x5587af705140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:27.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.179+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/2010225758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af6fadc0 0x5587af705140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.179+0000 7ff2f0b11640 1 -- 172.21.15.133:0/2010225758 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587af5ec440 con 0x5587af705680 2024-09-11T23:43:27.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.179+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/2010225758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af705680 0x5587af707a70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005340 tx=0x7ff2dc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b91426d4d32fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:27.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.180+0000 7ff2d77fe640 1 -- 172.21.15.133:0/2010225758 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc004ad0 con 0x5587af705680 2024-09-11T23:43:27.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.180+0000 7ff2d77fe640 1 -- 172.21.15.133:0/2010225758 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2dc029070 con 0x5587af705680 2024-09-11T23:43:27.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.180+0000 7ff2d77fe640 1 -- 172.21.15.133:0/2010225758 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc004ad0 con 0x5587af705680 2024-09-11T23:43:27.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.182+0000 7ff2f594f740 1 -- 172.21.15.133:0/2010225758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af705680 msgr2=0x5587af707a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:27.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.182+0000 7ff2f594f740 1 --2- 172.21.15.133:0/2010225758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af705680 0x5587af707a70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005340 tx=0x7ff2dc005740 comp rx=0 tx=0).stop 2024-09-11T23:43:27.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.182+0000 7ff2f594f740 1 -- 172.21.15.133:0/2010225758 shutdown_connections 2024-09-11T23:43:27.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.182+0000 7ff2f594f740 1 --2- 172.21.15.133:0/2010225758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af705680 0x5587af707a70 secure :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005340 tx=0x7ff2dc005740 comp rx=0 tx=0).stop 2024-09-11T23:43:27.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.182+0000 7ff2f594f740 1 --2- 172.21.15.133:0/2010225758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af6fadc0 0x5587af705140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.182+0000 7ff2f594f740 1 --2- 172.21.15.133:0/2010225758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af6fa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.182+0000 7ff2f594f740 1 -- 172.21.15.133:0/2010225758 >> 172.21.15.133:0/2010225758 conn(0x5587af6f6a30 msgr2=0x5587af6f6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.189+0000 7ff2e27fc640 1 -- 172.21.15.133:0/3010477116 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff2dc02b030 con 0x7ff2ec10ae80 2024-09-11T23:43:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.191+0000 7ff2e27fc640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2d80a2dc0 0x7ff2d80a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.192+0000 7ff2e27fc640 1 -- 172.21.15.133:0/3010477116 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7ff2dc0048d0 con 0x7ff2ec10ae80 2024-09-11T23:43:27.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.194+0000 7ff2f1312640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2d80a2dc0 0x7ff2d80a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.196+0000 7ff2f594f740 1 -- 172.21.15.133:0/2010225758 shutdown_connections 2024-09-11T23:43:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.198+0000 7ff2f594f740 1 -- 172.21.15.133:0/2010225758 wait complete. 2024-09-11T23:43:27.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.199+0000 7ff2f594f740 1 -- start start 2024-09-11T23:43:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.199+0000 7ff2f594f740 1 Processor -- start 2024-09-11T23:43:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.200+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af704eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.200+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fadc0 0x5587af703a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.200+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af703f60 0x5587af7043e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.200+0000 7ff2f594f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5587af7081f0 con 0x5587af703f60 2024-09-11T23:43:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.200+0000 7ff2f1312640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2d80a2dc0 0x7ff2d80a5280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5587af6f7890 tx=0x7ff2e4007250 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.200+0000 7ff2f594f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5587af61b050 con 0x5587af6fadc0 2024-09-11T23:43:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.200+0000 7ff2f594f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5587af708370 con 0x5587af6fa4a0 2024-09-11T23:43:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.200+0000 7ff2f594f740 10 client.0 _ll_register_callbacks cb 0x5587af65b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.201+0000 7ff2f1312640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af704eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.201+0000 7ff2f1312640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af704eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:54946/0 (socket says 172.21.15.133:54946) 2024-09-11T23:43:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.201+0000 7ff2f1312640 1 -- 172.21.15.133:0/4155537996 learned_addr learned my addr 172.21.15.133:0/4155537996 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.201+0000 7ff2f1b13640 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af703f60 0x5587af7043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.201+0000 7ff2f1312640 1 -- 172.21.15.133:0/4155537996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fadc0 msgr2=0x5587af703a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.201+0000 7ff2f1312640 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fadc0 0x5587af703a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.201+0000 7ff2f1312640 1 -- 172.21.15.133:0/4155537996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af703f60 msgr2=0x5587af7043e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:27.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.201+0000 7ff2f1312640 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af703f60 0x5587af7043e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.201+0000 7ff2f1312640 1 -- 172.21.15.133:0/4155537996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587af3d13c0 con 0x5587af6fa4a0 2024-09-11T23:43:27.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.202+0000 7ff2f1b13640 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af703f60 0x5587af7043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:43:27.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.203+0000 7ff2f1312640 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af704eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40139e0 tx=0x7ff2e4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c11fbbcf104a8d6c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:27.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.203+0000 7ff2d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:43:27.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.203+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4155537996 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4023060 con 0x5587af6fa4a0 2024-09-11T23:43:27.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.203+0000 7ff2f594f740 1 -- 172.21.15.133:0/4155537996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2dc029070 con 0x5587af6fa4a0 2024-09-11T23:43:27.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.204+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4155537996 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7ff2e4011d20 con 0x5587af6fa4a0 2024-09-11T23:43:27.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.204+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4155537996 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff2e4028470 con 0x5587af6fa4a0 2024-09-11T23:43:27.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.204+0000 7ff2f594f740 10 client.26599 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:27.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.204+0000 7ff2f594f740 1 -- 172.21.15.133:0/4155537996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587af713ff0 con 0x5587af6fa4a0 2024-09-11T23:43:27.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.208+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4155537996 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2dc029070 con 0x5587af6fa4a0 2024-09-11T23:43:27.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.209+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4155537996 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff2e401d530 con 0x5587af6fa4a0 2024-09-11T23:43:27.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.209+0000 7ff2f594f740 10 client.26599 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:27.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.209+0000 7ff2f594f740 10 client.26599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:27.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.209+0000 7ff2f594f740 20 client.26599 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:27.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.209+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587af715950 0x5587af717df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.209+0000 7ff2f594f740 4 client.26599 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:27.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.209+0000 7ff2f594f740 1 -- 172.21.15.133:0/4155537996 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587af66d850 con 0x5587af715950 2024-09-11T23:43:27.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.213+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587af715950 0x5587af717df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.223+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587af715950 0x5587af717df0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695b6a3a3bca1714 server_cookie=e254b0b0c2e91ed0 in_seq=0 out_seq=0 2024-09-11T23:43:27.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.224+0000 7ff2d4ff9640 10 client.26599 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.225+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4155537996 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587af66d850 con 0x5587af715950 2024-09-11T23:43:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.225+0000 7ff2d4ff9640 10 client.26599 handle_command_reply: tid=0 2024-09-11T23:43:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.225+0000 7ff2f594f740 1 client.26599 shutdown 2024-09-11T23:43:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.225+0000 7ff2f594f740 1 -- 172.21.15.133:0/4155537996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 msgr2=0x5587af704eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.225+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af704eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40139e0 tx=0x7ff2e4004a40 comp rx=0 tx=0).stop 2024-09-11T23:43:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.227+0000 7ff2f594f740 1 -- 172.21.15.133:0/4155537996 shutdown_connections 2024-09-11T23:43:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.227+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587af715950 0x5587af717df0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.227+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af703f60 0x5587af7043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.227+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fadc0 0x5587af703a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.227+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4155537996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af704eb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.227+0000 7ff2f594f740 1 -- 172.21.15.133:0/4155537996 >> 172.21.15.133:0/4155537996 conn(0x5587af6f6a30 msgr2=0x5587af6fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.228+0000 7ff2d4ff9640 0 client.26599 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:27.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.229+0000 7ff2f594f740 1 -- 172.21.15.133:0/4155537996 shutdown_connections 2024-09-11T23:43:27.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.229+0000 7ff2f594f740 1 -- 172.21.15.133:0/4155537996 wait complete. 2024-09-11T23:43:27.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.230+0000 7ff2f594f740 20 client.26599 trim_cache size 0 max 16384 2024-09-11T23:43:27.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.273+0000 7ff2f594f740 1 Processor -- start 2024-09-11T23:43:27.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.273+0000 7ff2f594f740 1 -- start start 2024-09-11T23:43:27.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.273+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af702a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.274+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fadc0 0x5587af71d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.274+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af703f60 0x5587af71dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.274+0000 7ff2f594f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d000e770 con 0x5587af703f60 2024-09-11T23:43:27.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.274+0000 7ff2f594f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d00102b0 con 0x5587af6fadc0 2024-09-11T23:43:27.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.274+0000 7ff2f594f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d000ee40 con 0x5587af6fa4a0 2024-09-11T23:43:27.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.276+0000 7ff2f0b11640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fadc0 0x5587af71d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.276+0000 7ff2f0b11640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fadc0 0x5587af71d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40512/0 (socket says 172.21.15.133:40512) 2024-09-11T23:43:27.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.276+0000 7ff2f0b11640 1 -- 172.21.15.133:0/631400644 learned_addr learned my addr 172.21.15.133:0/631400644 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:27.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.276+0000 7ff2f1312640 1 --2- 172.21.15.133:0/631400644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af702a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.276+0000 7ff2f1b13640 1 --2- 172.21.15.133:0/631400644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af703f60 0x5587af71dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.276+0000 7ff2f0b11640 1 -- 172.21.15.133:0/631400644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 msgr2=0x5587af702a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.276+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/631400644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af702a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.276+0000 7ff2f0b11640 1 -- 172.21.15.133:0/631400644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af703f60 msgr2=0x5587af71dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.276+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/631400644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af703f60 0x5587af71dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.276+0000 7ff2f0b11640 1 -- 172.21.15.133:0/631400644 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587af701170 con 0x5587af6fadc0 2024-09-11T23:43:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.277+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/631400644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fadc0 0x5587af71d6c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc09b590 tx=0x7ff2dc0a26e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a213f9a8928a17a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.277+0000 7ff2d6ffd640 1 -- 172.21.15.133:0/631400644 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc09ab70 con 0x5587af6fadc0 2024-09-11T23:43:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.277+0000 7ff2d6ffd640 1 -- 172.21.15.133:0/631400644 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2dc029040 con 0x5587af6fadc0 2024-09-11T23:43:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.277+0000 7ff2d6ffd640 1 -- 172.21.15.133:0/631400644 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc09ad10 con 0x5587af6fadc0 2024-09-11T23:43:27.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.279+0000 7ff2f594f740 1 -- 172.21.15.133:0/631400644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fadc0 msgr2=0x5587af71d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:27.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.279+0000 7ff2f594f740 1 --2- 172.21.15.133:0/631400644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fadc0 0x5587af71d6c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc09b590 tx=0x7ff2dc0a26e0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.279+0000 7ff2f594f740 1 -- 172.21.15.133:0/631400644 shutdown_connections 2024-09-11T23:43:27.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.279+0000 7ff2f594f740 1 --2- 172.21.15.133:0/631400644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af703f60 0x5587af71dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.279+0000 7ff2f594f740 1 --2- 172.21.15.133:0/631400644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fadc0 0x5587af71d6c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.279+0000 7ff2f594f740 1 --2- 172.21.15.133:0/631400644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af6fa4a0 0x5587af702a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.279+0000 7ff2f594f740 1 -- 172.21.15.133:0/631400644 >> 172.21.15.133:0/631400644 conn(0x5587af6f6a30 msgr2=0x5587af6fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:27.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.279+0000 7ff2f594f740 1 -- 172.21.15.133:0/631400644 shutdown_connections 2024-09-11T23:43:27.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.279+0000 7ff2f594f740 1 -- 172.21.15.133:0/631400644 wait complete. 2024-09-11T23:43:27.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.279+0000 7ff2f594f740 1 -- start start 2024-09-11T23:43:27.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.280+0000 7ff2f594f740 1 Processor -- start 2024-09-11T23:43:27.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.281+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fa4a0 0x5587af70d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.281+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af6fadc0 0x5587af70df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.281+0000 7ff2f594f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af703f60 0x5587af708f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.281+0000 7ff2f594f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d000fbe0 con 0x5587af6fadc0 2024-09-11T23:43:27.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.282+0000 7ff2f594f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d000f510 con 0x5587af6fa4a0 2024-09-11T23:43:27.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.282+0000 7ff2f594f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d000c770 con 0x5587af703f60 2024-09-11T23:43:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.282+0000 7ff2f0b11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af6fadc0 0x5587af70df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.282+0000 7ff2f594f740 10 client.0 _ll_register_callbacks cb 0x7ff2e4008040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.282+0000 7ff2f0b11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af6fadc0 0x5587af70df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45076/0 (socket says 172.21.15.133:45076) 2024-09-11T23:43:27.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.282+0000 7ff2f0b11640 1 -- 172.21.15.133:0/4174074562 learned_addr learned my addr 172.21.15.133:0/4174074562 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:27.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.282+0000 7ff2f0b11640 1 -- 172.21.15.133:0/4174074562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af703f60 msgr2=0x5587af708f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:27.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.283+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/4174074562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af703f60 0x5587af708f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.283+0000 7ff2f0b11640 1 -- 172.21.15.133:0/4174074562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fa4a0 msgr2=0x5587af70d9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:27.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.283+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/4174074562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fa4a0 0x5587af70d9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.283+0000 7ff2f0b11640 1 -- 172.21.15.133:0/4174074562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587af7097c0 con 0x5587af6fadc0 2024-09-11T23:43:27.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.283+0000 7ff2f0b11640 1 --2- 172.21.15.133:0/4174074562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af6fadc0 0x5587af70df00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc09b050 tx=0x7ff2dc00dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc0005719ada26f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:27.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.283+0000 7ff2d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:27.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.283+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4174074562 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc09aeb0 con 0x5587af6fadc0 2024-09-11T23:43:27.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.284+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4174074562 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7ff2dc0cf450 con 0x5587af6fadc0 2024-09-11T23:43:27.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.284+0000 7ff2f594f740 1 -- 172.21.15.133:0/4174074562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2dc029040 con 0x5587af6fadc0 2024-09-11T23:43:27.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.284+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4174074562 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff2dc09b080 con 0x5587af6fadc0 2024-09-11T23:43:27.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.284+0000 7ff2f594f740 10 client.17226 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:27.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.284+0000 7ff2f594f740 1 -- 172.21.15.133:0/4174074562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587af7802e0 con 0x5587af6fadc0 2024-09-11T23:43:27.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.284+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4174074562 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2dc027ec0 con 0x5587af6fadc0 2024-09-11T23:43:27.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.285+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4174074562 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff2dc09d620 con 0x5587af6fadc0 2024-09-11T23:43:27.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.285+0000 7ff2f594f740 10 client.17226 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.285+0000 7ff2f594f740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.285+0000 7ff2f594f740 20 client.17226 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.285+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4174074562 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587af780570 0x5587af7829e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.285+0000 7ff2f594f740 4 client.17226 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:43:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.286+0000 7ff2f594f740 1 -- 172.21.15.133:0/4174074562 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff2dc029040 con 0x5587af780570 2024-09-11T23:43:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.287+0000 7ff2f1312640 1 --2- 172.21.15.133:0/4174074562 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587af780570 0x5587af7829e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:27.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.287+0000 7ff2f1312640 1 --2- 172.21.15.133:0/4174074562 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587af780570 0x5587af7829e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7fe8e445265abc2 server_cookie=dcbebd2d69f46a92 in_seq=0 out_seq=0 2024-09-11T23:43:27.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.287+0000 7ff2d4ff9640 10 client.17226 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.287+0000 7ff2d4ff9640 1 -- 172.21.15.133:0/4174074562 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7ff2dc029040 con 0x5587af780570 2024-09-11T23:43:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.287+0000 7ff2d4ff9640 10 client.17226 handle_command_reply: tid=0 2024-09-11T23:43:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.287+0000 7ff2f594f740 1 client.17226 shutdown 2024-09-11T23:43:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.288+0000 7ff2f594f740 1 -- 172.21.15.133:0/4174074562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af6fadc0 msgr2=0x5587af70df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.288+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4174074562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af6fadc0 0x5587af70df00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc09b050 tx=0x7ff2dc00dfd0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.288+0000 7ff2f594f740 1 -- 172.21.15.133:0/4174074562 shutdown_connections 2024-09-11T23:43:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.288+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4174074562 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587af780570 0x5587af7829e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.288+0000 7ff2d4ff9640 0 client.17226 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:27.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.288+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4174074562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587af703f60 0x5587af708f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.288+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4174074562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587af6fadc0 0x5587af70df00 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.288+0000 7ff2f594f740 1 --2- 172.21.15.133:0/4174074562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587af6fa4a0 0x5587af70d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.289+0000 7ff2f594f740 1 -- 172.21.15.133:0/4174074562 >> 172.21.15.133:0/4174074562 conn(0x5587af6f6a30 msgr2=0x5587af6fbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:27.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.289+0000 7ff2f594f740 1 -- 172.21.15.133:0/4174074562 shutdown_connections 2024-09-11T23:43:27.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.289+0000 7ff2f594f740 1 -- 172.21.15.133:0/4174074562 wait complete. 2024-09-11T23:43:27.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.289+0000 7ff2f594f740 20 client.17226 trim_cache size 0 max 16384 2024-09-11T23:43:27.290 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:43:27.291 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:43:27.291 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:43:27.291 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:43:27.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.292+0000 7ff2f3576640 1 -- 172.21.15.133:0/3010477116 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2d80a2dc0 msgr2=0x7ff2d80a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:27.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.292+0000 7ff2f3576640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2d80a2dc0 0x7ff2d80a5280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5587af6f7890 tx=0x7ff2e4007250 comp rx=0 tx=0).stop 2024-09-11T23:43:27.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.292+0000 7ff2f3576640 1 -- 172.21.15.133:0/3010477116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2ec10ae80 msgr2=0x7ff2ec1a6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:27.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.292+0000 7ff2f3576640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2ec10ae80 0x7ff2ec1a6df0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00d950 tx=0x7ff2dc00de10 comp rx=0 tx=0).stop 2024-09-11T23:43:27.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.294+0000 7ff2f3576640 1 -- 172.21.15.133:0/3010477116 shutdown_connections 2024-09-11T23:43:27.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.295+0000 7ff2f3576640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff2d80a2dc0 0x7ff2d80a5280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.295+0000 7ff2f3576640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec113940 0x7ff2ec19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.295+0000 7ff2f3576640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2ec10ae80 0x7ff2ec1a6df0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.295+0000 7ff2f3576640 1 --2- 172.21.15.133:0/3010477116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2ec10a560 0x7ff2ec1a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:27.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.295+0000 7ff2f3576640 1 -- 172.21.15.133:0/3010477116 >> 172.21.15.133:0/3010477116 conn(0x7ff2ec06e140 msgr2=0x7ff2ec109290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:27.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.300+0000 7ff2f3576640 1 -- 172.21.15.133:0/3010477116 shutdown_connections 2024-09-11T23:43:27.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:27.300+0000 7ff2f3576640 1 -- 172.21.15.133:0/3010477116 wait complete. 2024-09-11T23:43:28.214 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:43:28.214 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:43:28.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:28 smithi178 ceph-mon[35882]: pgmap v206: 97 pgs: 97 active+clean; 51 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 490 MiB/s wr, 2.29k op/s 2024-09-11T23:43:28.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:28 smithi138 ceph-mon[35142]: pgmap v206: 97 pgs: 97 active+clean; 51 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 490 MiB/s wr, 2.29k op/s 2024-09-11T23:43:28.961 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:28 smithi133 ceph-mon[31430]: pgmap v206: 97 pgs: 97 active+clean; 51 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 490 MiB/s wr, 2.29k op/s 2024-09-11T23:43:29.084 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:29.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.721+0000 7fc705d9b640 1 -- 172.21.15.133:0/702582977 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc030950 con 0x7fc7081091f0 2024-09-11T23:43:29.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.721+0000 7fc70d1ee640 1 -- 172.21.15.133:0/702582977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7081091f0 msgr2=0x7fc7081095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:29.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.721+0000 7fc70d1ee640 1 --2- 172.21.15.133:0/702582977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7081091f0 0x7fc7081095d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009860 tx=0x7fc6fc02a340 comp rx=0 tx=0).stop 2024-09-11T23:43:29.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.723+0000 7fc70d1ee640 1 -- 172.21.15.133:0/702582977 shutdown_connections 2024-09-11T23:43:29.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.724+0000 7fc70d1ee640 1 --2- 172.21.15.133:0/702582977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc708116ea0 0x7fc708119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:29.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.724+0000 7fc70d1ee640 1 --2- 172.21.15.133:0/702582977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc708109b10 0x7fc708116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:29.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.724+0000 7fc70d1ee640 1 --2- 172.21.15.133:0/702582977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7081091f0 0x7fc7081095d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:29.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.724+0000 7fc70d1ee640 1 -- 172.21.15.133:0/702582977 >> 172.21.15.133:0/702582977 conn(0x7fc70806e140 msgr2=0x7fc70806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:29.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.724+0000 7fc70d1ee640 1 -- 172.21.15.133:0/702582977 shutdown_connections 2024-09-11T23:43:29.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.725+0000 7fc70d1ee640 1 -- 172.21.15.133:0/702582977 wait complete. 2024-09-11T23:43:29.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.726+0000 7fc70d1ee640 1 Processor -- start 2024-09-11T23:43:29.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.727+0000 7fc70d1ee640 1 -- start start 2024-09-11T23:43:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.727+0000 7fc70d1ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7081091f0 0x7fc708122ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.727+0000 7fc70d1ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc708109b10 0x7fc7081231e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.727+0000 7fc70d1ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc708116ea0 0x7fc708123720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.727+0000 7fc70d1ee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc70810d6b0 con 0x7fc7081091f0 2024-09-11T23:43:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.727+0000 7fc70d1ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc70810d530 con 0x7fc708109b10 2024-09-11T23:43:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.727+0000 7fc70d1ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc70810d830 con 0x7fc708116ea0 2024-09-11T23:43:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.728+0000 7fc70659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc708109b10 0x7fc7081231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.728+0000 7fc70659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc708109b10 0x7fc7081231e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:40530/0 (socket says 172.21.15.133:40530) 2024-09-11T23:43:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.728+0000 7fc70659c640 1 -- 172.21.15.133:0/819125970 learned_addr learned my addr 172.21.15.133:0/819125970 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.728+0000 7fc70659c640 1 -- 172.21.15.133:0/819125970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc708116ea0 msgr2=0x7fc708123720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.728+0000 7fc70759e640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc708116ea0 0x7fc708123720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:29.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.728+0000 7fc706d9d640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7081091f0 0x7fc708122ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:29.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.728+0000 7fc70659c640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc708116ea0 0x7fc708123720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:29.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.728+0000 7fc70659c640 1 -- 172.21.15.133:0/819125970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7081091f0 msgr2=0x7fc708122ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:29.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.728+0000 7fc70659c640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7081091f0 0x7fc708122ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:29.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.728+0000 7fc70659c640 1 -- 172.21.15.133:0/819125970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7081bbda0 con 0x7fc708109b10 2024-09-11T23:43:29.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.728+0000 7fc70659c640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc708109b10 0x7fc7081231e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800d950 tx=0x7fc6f800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:29.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.729+0000 7fc70759e640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc708116ea0 0x7fc708123720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:43:29.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.729+0000 7fc706d9d640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7081091f0 0x7fc708122ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:43:29.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.729+0000 7fc6effff640 1 -- 172.21.15.133:0/819125970 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8017070 con 0x7fc708109b10 2024-09-11T23:43:29.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.729+0000 7fc70d1ee640 1 -- 172.21.15.133:0/819125970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7081bc090 con 0x7fc708109b10 2024-09-11T23:43:29.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.729+0000 7fc70d1ee640 1 -- 172.21.15.133:0/819125970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7081bc5f0 con 0x7fc708109b10 2024-09-11T23:43:29.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.729+0000 7fc6effff640 1 -- 172.21.15.133:0/819125970 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f8009c30 con 0x7fc708109b10 2024-09-11T23:43:29.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.729+0000 7fc6effff640 1 -- 172.21.15.133:0/819125970 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800f690 con 0x7fc708109b10 2024-09-11T23:43:29.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.733+0000 7fc6effff640 1 -- 172.21.15.133:0/819125970 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc6f800f830 con 0x7fc708109b10 2024-09-11T23:43:29.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.734+0000 7fc6effff640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6f00a2df0 0x7fc6f00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:29.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.735+0000 7fc6effff640 1 -- 172.21.15.133:0/819125970 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fc6f80cec30 con 0x7fc708109b10 2024-09-11T23:43:29.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.735+0000 7fc70d1ee640 1 -- 172.21.15.133:0/819125970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f4005290 con 0x7fc708109b10 2024-09-11T23:43:29.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.736+0000 7fc706d9d640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6f00a2df0 0x7fc6f00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:29.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.737+0000 7fc706d9d640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6f00a2df0 0x7fc6f00a52b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc005b60 tx=0x7fc6fc000bc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:29.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:29.741+0000 7fc6effff640 1 -- 172.21.15.133:0/819125970 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6f809b830 con 0x7fc708109b10 2024-09-11T23:43:30.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.196+0000 7fc70d1ee640 1 -- 172.21.15.133:0/819125970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6f4005850 con 0x7fc708109b10 2024-09-11T23:43:30.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.207+0000 7fc6effff640 1 -- 172.21.15.133:0/819125970 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7fc6f809b130 con 0x7fc708109b10 2024-09-11T23:43:30.207 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:43:30.207 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:43:30.208 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:43:30.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.212+0000 7fc70d1ee640 1 -- 172.21.15.133:0/819125970 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6f00a2df0 msgr2=0x7fc6f00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:30.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.212+0000 7fc70d1ee640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6f00a2df0 0x7fc6f00a52b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc005b60 tx=0x7fc6fc000bc0 comp rx=0 tx=0).stop 2024-09-11T23:43:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.212+0000 7fc70d1ee640 1 -- 172.21.15.133:0/819125970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc708109b10 msgr2=0x7fc7081231e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.212+0000 7fc70d1ee640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc708109b10 0x7fc7081231e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800d950 tx=0x7fc6f800de10 comp rx=0 tx=0).stop 2024-09-11T23:43:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.212+0000 7fc70d1ee640 1 -- 172.21.15.133:0/819125970 shutdown_connections 2024-09-11T23:43:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.212+0000 7fc70d1ee640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc6f00a2df0 0x7fc6f00a52b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.212+0000 7fc70d1ee640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc708116ea0 0x7fc708123720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.212+0000 7fc70d1ee640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc708109b10 0x7fc7081231e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.212+0000 7fc70d1ee640 1 --2- 172.21.15.133:0/819125970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7081091f0 0x7fc708122ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.212+0000 7fc70d1ee640 1 -- 172.21.15.133:0/819125970 >> 172.21.15.133:0/819125970 conn(0x7fc70806e140 msgr2=0x7fc7081174c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.213+0000 7fc70d1ee640 1 -- 172.21.15.133:0/819125970 shutdown_connections 2024-09-11T23:43:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:30.213+0000 7fc70d1ee640 1 -- 172.21.15.133:0/819125970 wait complete. 2024-09-11T23:43:30.758 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:30 smithi133 ceph-mon[31430]: pgmap v207: 97 pgs: 97 active+clean; 53 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 570 MiB/s wr, 2.55k op/s 2024-09-11T23:43:30.758 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:30 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/819125970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:31.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:30 smithi178 ceph-mon[35882]: pgmap v207: 97 pgs: 97 active+clean; 53 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 570 MiB/s wr, 2.55k op/s 2024-09-11T23:43:31.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:30 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/819125970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:31.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:30 smithi138 ceph-mon[35142]: pgmap v207: 97 pgs: 97 active+clean; 53 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 570 MiB/s wr, 2.55k op/s 2024-09-11T23:43:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:30 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/819125970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:31.225 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:43:32.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:31 smithi178 ceph-mon[35882]: pgmap v208: 97 pgs: 97 active+clean; 54 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 545 MiB/s wr, 2.43k op/s 2024-09-11T23:43:32.090 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:32.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:31 smithi138 ceph-mon[35142]: pgmap v208: 97 pgs: 97 active+clean; 54 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 545 MiB/s wr, 2.43k op/s 2024-09-11T23:43:32.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:31 smithi133 ceph-mon[31430]: pgmap v208: 97 pgs: 97 active+clean; 54 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 545 MiB/s wr, 2.43k op/s 2024-09-11T23:43:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.322+0000 7fa0aacdb640 1 -- 172.21.15.133:0/8153426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a406b8a0 msgr2=0x7fa0a406bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.322+0000 7fa0aacdb640 1 --2- 172.21.15.133:0/8153426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a406b8a0 0x7fa0a406bc80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa094009890 tx=0x7fa09402a380 comp rx=0 tx=0).stop 2024-09-11T23:43:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.323+0000 7fa0aacdb640 1 -- 172.21.15.133:0/8153426 shutdown_connections 2024-09-11T23:43:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.323+0000 7fa0aacdb640 1 --2- 172.21.15.133:0/8153426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0a411b380 0x7fa0a411d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.323+0000 7fa0aacdb640 1 --2- 172.21.15.133:0/8153426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0a406c1c0 0x7fa0a411ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.323+0000 7fa0aacdb640 1 --2- 172.21.15.133:0/8153426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a406b8a0 0x7fa0a406bc80 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.323+0000 7fa0aacdb640 1 -- 172.21.15.133:0/8153426 >> 172.21.15.133:0/8153426 conn(0x7fa0a4071390 msgr2=0x7fa0a40737b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.324+0000 7fa0aacdb640 1 -- 172.21.15.133:0/8153426 shutdown_connections 2024-09-11T23:43:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.324+0000 7fa0aacdb640 1 -- 172.21.15.133:0/8153426 wait complete. 2024-09-11T23:43:33.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.325+0000 7fa0aacdb640 1 Processor -- start 2024-09-11T23:43:33.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.325+0000 7fa0aacdb640 1 -- start start 2024-09-11T23:43:33.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.325+0000 7fa0aacdb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0a406b8a0 0x7fa0a411a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.326+0000 7fa0aacdb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a406c1c0 0x7fa0a41139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.326+0000 7fa0aacdb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0a411b380 0x7fa0a4113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.326+0000 7fa0aacdb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a4120c30 con 0x7fa0a406c1c0 2024-09-11T23:43:33.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.326+0000 7fa0aacdb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a4120ab0 con 0x7fa0a411b380 2024-09-11T23:43:33.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.326+0000 7fa0aacdb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a4120db0 con 0x7fa0a406b8a0 2024-09-11T23:43:33.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.328+0000 7fa0a9cd9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0a406b8a0 0x7fa0a411a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.328+0000 7fa0a9cd9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0a406b8a0 0x7fa0a411a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:55338/0 (socket says 172.21.15.133:55338) 2024-09-11T23:43:33.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.328+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/2747223143 learned_addr learned my addr 172.21.15.133:0/2747223143 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:33.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.328+0000 7fa0aa4da640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0a411b380 0x7fa0a4113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.329+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/2747223143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0a411b380 msgr2=0x7fa0a4113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:33.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.329+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0a411b380 0x7fa0a4113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.329+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/2747223143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a406c1c0 msgr2=0x7fa0a41139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:33.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.329+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a406c1c0 0x7fa0a41139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.329+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/2747223143 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a41144f0 con 0x7fa0a406b8a0 2024-09-11T23:43:33.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.335+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0a406b8a0 0x7fa0a411a960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa094035040 tx=0x7fa094033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:33.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.336+0000 7fa09affd640 1 -- 172.21.15.133:0/2747223143 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094037040 con 0x7fa0a406b8a0 2024-09-11T23:43:33.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.336+0000 7fa09affd640 1 -- 172.21.15.133:0/2747223143 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa094040070 con 0x7fa0a406b8a0 2024-09-11T23:43:33.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.337+0000 7fa09affd640 1 -- 172.21.15.133:0/2747223143 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094003ad0 con 0x7fa0a406b8a0 2024-09-11T23:43:33.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.337+0000 7fa0aacdb640 1 -- 172.21.15.133:0/2747223143 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a4114780 con 0x7fa0a406b8a0 2024-09-11T23:43:33.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.337+0000 7fa0aacdb640 1 -- 172.21.15.133:0/2747223143 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a4118660 con 0x7fa0a406b8a0 2024-09-11T23:43:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.346+0000 7fa09affd640 1 -- 172.21.15.133:0/2747223143 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa094004250 con 0x7fa0a406b8a0 2024-09-11T23:43:33.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.361+0000 7fa09affd640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa0900a2ce0 0x7fa0900a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.361+0000 7fa09affd640 1 -- 172.21.15.133:0/2747223143 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fa0940eebf0 con 0x7fa0a406b8a0 2024-09-11T23:43:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.365+0000 7fa0ae316740 1 Processor -- start 2024-09-11T23:43:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.365+0000 7fa0ae316740 1 -- start start 2024-09-11T23:43:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.365+0000 7fa0ae316740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 0x560cc8466240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.366+0000 7fa0ae316740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8466780 0x560cc8466bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.366+0000 7fa0ae316740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8471520 0x560cc8473910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.366+0000 7fa0ae316740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560cc83cc2f0 con 0x560cc8467cf0 2024-09-11T23:43:33.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.366+0000 7fa0ae316740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560cc8373cf0 con 0x560cc8471520 2024-09-11T23:43:33.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.366+0000 7fa0ae316740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560cc83c82b0 con 0x560cc8466780 2024-09-11T23:43:33.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.366+0000 7fa0a9cd9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 0x560cc8466240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.367+0000 7fa0a94d8640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa0900a2ce0 0x7fa0900a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.367+0000 7fa0a9cd9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 0x560cc8466240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49982/0 (socket says 172.21.15.133:49982) 2024-09-11T23:43:33.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.367+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/567568217 learned_addr learned my addr 172.21.15.133:0/567568217 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:33.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.367+0000 7fa0aa4da640 1 --2- 172.21.15.133:0/567568217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8471520 0x560cc8473910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.369+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/567568217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8466780 msgr2=0x560cc8466bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:33.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.369+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/567568217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8466780 0x560cc8466bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.369+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/567568217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8471520 msgr2=0x560cc8473910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:33.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.369+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/567568217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8471520 0x560cc8473910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.369+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/567568217 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc83583a0 con 0x560cc8467cf0 2024-09-11T23:43:33.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.370+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/567568217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 0x560cc8466240 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa0940c3940 tx=0x7fa094005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b67bd6261e779eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:33.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.370+0000 7fa08ffff640 1 -- 172.21.15.133:0/567568217 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0940bb290 con 0x560cc8467cf0 2024-09-11T23:43:33.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.374+0000 7fa0a94d8640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa0900a2ce0 0x7fa0900a51a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0005d00 tx=0x7fa0a0005e50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:33.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.375+0000 7fa08ffff640 1 -- 172.21.15.133:0/567568217 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0940bf640 con 0x560cc8467cf0 2024-09-11T23:43:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.375+0000 7fa08ffff640 1 -- 172.21.15.133:0/567568217 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0940ec5a0 con 0x560cc8467cf0 2024-09-11T23:43:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.381+0000 7fa0ae316740 1 -- 172.21.15.133:0/567568217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 msgr2=0x560cc8466240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.385+0000 7fa0ae316740 1 --2- 172.21.15.133:0/567568217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 0x560cc8466240 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa0940c3940 tx=0x7fa094005f90 comp rx=0 tx=0).stop 2024-09-11T23:43:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.385+0000 7fa0ae316740 1 -- 172.21.15.133:0/567568217 shutdown_connections 2024-09-11T23:43:33.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.388+0000 7fa0ae316740 1 --2- 172.21.15.133:0/567568217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8471520 0x560cc8473910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.388+0000 7fa0ae316740 1 --2- 172.21.15.133:0/567568217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8466780 0x560cc8466bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.388+0000 7fa0ae316740 1 --2- 172.21.15.133:0/567568217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 0x560cc8466240 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.388+0000 7fa0ae316740 1 -- 172.21.15.133:0/567568217 >> 172.21.15.133:0/567568217 conn(0x560cc842f310 msgr2=0x560cc8462e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:33.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.393+0000 7fa0ae316740 1 -- 172.21.15.133:0/567568217 shutdown_connections 2024-09-11T23:43:33.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.393+0000 7fa0ae316740 1 -- 172.21.15.133:0/567568217 wait complete. 2024-09-11T23:43:33.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.394+0000 7fa0ae316740 1 -- start start 2024-09-11T23:43:33.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.395+0000 7fa0ae316740 1 Processor -- start 2024-09-11T23:43:33.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.396+0000 7fa0ae316740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8466780 0x560cc8470f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.396+0000 7fa0ae316740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 0x560cc846e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.397+0000 7fa0ae316740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8471520 0x560cc846e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.397+0000 7fa0ae316740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560cc8473fe0 con 0x560cc8467cf0 2024-09-11T23:43:33.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.397+0000 7fa0ae316740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560cc8387060 con 0x560cc8466780 2024-09-11T23:43:33.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.397+0000 7fa0ae316740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560cc8474160 con 0x560cc8471520 2024-09-11T23:43:33.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.397+0000 7fa0ae316740 10 client.0 _ll_register_callbacks cb 0x560cc80da8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:33.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.402+0000 7fa0aa4da640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8471520 0x560cc846e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.402+0000 7fa0a94d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 0x560cc846e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.402+0000 7fa0a94d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 0x560cc846e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49998/0 (socket says 172.21.15.133:49998) 2024-09-11T23:43:33.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.402+0000 7fa0a94d8640 1 -- 172.21.15.133:0/1398175822 learned_addr learned my addr 172.21.15.133:0/1398175822 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:33.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.402+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8466780 0x560cc8470f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.403+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/1398175822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8471520 msgr2=0x560cc846e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:33.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.403+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8471520 0x560cc846e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.403+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/1398175822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 msgr2=0x560cc846e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:33.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.403+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 0x560cc846e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.403+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/1398175822 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc813d3c0 con 0x560cc8466780 2024-09-11T23:43:33.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.404+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8466780 0x560cc8470f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa094009860 tx=0x7fa0940c3f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f3146adfb6b4302 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:33.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.404+0000 7fa08d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:33.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.404+0000 7fa08d7fa640 1 -- 172.21.15.133:0/1398175822 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0940d9060 con 0x560cc8466780 2024-09-11T23:43:33.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.405+0000 7fa0ae316740 1 -- 172.21.15.133:0/1398175822 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0940bf640 con 0x560cc8466780 2024-09-11T23:43:33.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.405+0000 7fa08d7fa640 1 -- 172.21.15.133:0/1398175822 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fa0940ec5a0 con 0x560cc8466780 2024-09-11T23:43:33.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.405+0000 7fa08d7fa640 1 -- 172.21.15.133:0/1398175822 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa0940ec7a0 con 0x560cc8466780 2024-09-11T23:43:33.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.405+0000 7fa0ae316740 10 client.26717 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:33.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.405+0000 7fa0ae316740 1 -- 172.21.15.133:0/1398175822 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560cc8473fe0 con 0x560cc8466780 2024-09-11T23:43:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.406+0000 7fa08d7fa640 1 -- 172.21.15.133:0/1398175822 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0940bb4f0 con 0x560cc8466780 2024-09-11T23:43:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.408+0000 7fa08d7fa640 1 -- 172.21.15.133:0/1398175822 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa09403a070 con 0x560cc8466780 2024-09-11T23:43:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.408+0000 7fa0ae316740 10 client.26717 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.408+0000 7fa0ae316740 10 client.26717 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:43:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.408+0000 7fa0ae316740 20 client.26717 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.408+0000 7fa0ae316740 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560cc8480dc0 0x560cc84831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.409+0000 7fa0ae316740 4 client.26717 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.409+0000 7fa0ae316740 1 -- 172.21.15.133:0/1398175822 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa0940bf640 con 0x560cc8480dc0 2024-09-11T23:43:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.409+0000 7fa0a94d8640 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560cc8480dc0 0x560cc84831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.411+0000 7fa0a94d8640 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560cc8480dc0 0x560cc84831e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d68b9b8574fdb1f server_cookie=5d29c0b8cbb1c6a3 in_seq=0 out_seq=0 2024-09-11T23:43:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.411+0000 7fa08d7fa640 10 client.26717 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:43:33.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.413+0000 7fa08d7fa640 1 -- 172.21.15.133:0/1398175822 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa0940bf640 con 0x560cc8480dc0 2024-09-11T23:43:33.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.413+0000 7fa08d7fa640 10 client.26717 handle_command_reply: tid=0 2024-09-11T23:43:33.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.413+0000 7fa0ae316740 1 client.26717 shutdown 2024-09-11T23:43:33.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.413+0000 7fa0ae316740 1 -- 172.21.15.133:0/1398175822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8466780 msgr2=0x560cc8470f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:33.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.413+0000 7fa0ae316740 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8466780 0x560cc8470f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa094009860 tx=0x7fa0940c3f60 comp rx=0 tx=0).stop 2024-09-11T23:43:33.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.418+0000 7fa0ae316740 1 -- 172.21.15.133:0/1398175822 shutdown_connections 2024-09-11T23:43:33.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.418+0000 7fa0ae316740 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560cc8480dc0 0x560cc84831e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.419+0000 7fa0ae316740 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8471520 0x560cc846e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.419+0000 7fa08d7fa640 0 client.26717 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:43:33.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.419+0000 7fa0ae316740 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8467cf0 0x560cc846e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.419+0000 7fa0ae316740 1 --2- 172.21.15.133:0/1398175822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8466780 0x560cc8470f30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.419+0000 7fa0ae316740 1 -- 172.21.15.133:0/1398175822 >> 172.21.15.133:0/1398175822 conn(0x560cc842f310 msgr2=0x560cc8468b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:33.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.420+0000 7fa0ae316740 1 -- 172.21.15.133:0/1398175822 shutdown_connections 2024-09-11T23:43:33.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.420+0000 7fa0ae316740 1 -- 172.21.15.133:0/1398175822 wait complete. 2024-09-11T23:43:33.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.426+0000 7fa0ae316740 20 client.26717 trim_cache size 0 max 16384 2024-09-11T23:43:33.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.455+0000 7fa0ae316740 1 Processor -- start 2024-09-11T23:43:33.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.457+0000 7fa0ae316740 1 -- start start 2024-09-11T23:43:33.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.457+0000 7fa0ae316740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 0x560cc847d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.458+0000 7fa0ae316740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8467cf0 0x560cc84692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.458+0000 7fa0ae316740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8471520 0x560cc8469810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.458+0000 7fa0ae316740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41c4850 con 0x560cc8466780 2024-09-11T23:43:33.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.458+0000 7fa0ae316740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41c6390 con 0x560cc8467cf0 2024-09-11T23:43:33.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.458+0000 7fa0ae316740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41c4f20 con 0x560cc8471520 2024-09-11T23:43:33.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.459+0000 7fa0a94d8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8467cf0 0x560cc84692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.459+0000 7fa0a94d8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8467cf0 0x560cc84692d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38840/0 (socket says 172.21.15.133:38840) 2024-09-11T23:43:33.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.459+0000 7fa0a94d8640 1 -- 172.21.15.133:0/516887341 learned_addr learned my addr 172.21.15.133:0/516887341 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:33.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.459+0000 7fa0aa4da640 1 --2- 172.21.15.133:0/516887341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8471520 0x560cc8469810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.460+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/516887341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 0x560cc847d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.461+0000 7fa0a94d8640 1 -- 172.21.15.133:0/516887341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8471520 msgr2=0x560cc8469810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:33.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.461+0000 7fa0a94d8640 1 --2- 172.21.15.133:0/516887341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8471520 0x560cc8469810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.461+0000 7fa0a94d8640 1 -- 172.21.15.133:0/516887341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 msgr2=0x560cc847d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:33.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.461+0000 7fa0a94d8640 1 --2- 172.21.15.133:0/516887341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 0x560cc847d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.461+0000 7fa0a94d8640 1 -- 172.21.15.133:0/516887341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc8463b10 con 0x560cc8467cf0 2024-09-11T23:43:33.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.462+0000 7fa0a94d8640 1 --2- 172.21.15.133:0/516887341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8467cf0 0x560cc84692d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000d720 tx=0x7fa0a0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba0f8a77bc5c3abb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:33.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.465+0000 7fa08f7fe640 1 -- 172.21.15.133:0/516887341 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00262d0 con 0x560cc8467cf0 2024-09-11T23:43:33.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.465+0000 7fa08f7fe640 1 -- 172.21.15.133:0/516887341 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a0017040 con 0x560cc8467cf0 2024-09-11T23:43:33.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.465+0000 7fa08f7fe640 1 -- 172.21.15.133:0/516887341 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a001f5f0 con 0x560cc8467cf0 2024-09-11T23:43:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.466+0000 7fa0ae316740 1 -- 172.21.15.133:0/516887341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8467cf0 msgr2=0x560cc84692d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:33.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.466+0000 7fa0ae316740 1 --2- 172.21.15.133:0/516887341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8467cf0 0x560cc84692d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000d720 tx=0x7fa0a0014000 comp rx=0 tx=0).stop 2024-09-11T23:43:33.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.467+0000 7fa0ae316740 1 -- 172.21.15.133:0/516887341 shutdown_connections 2024-09-11T23:43:33.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.467+0000 7fa0ae316740 1 --2- 172.21.15.133:0/516887341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8471520 0x560cc8469810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.467+0000 7fa0ae316740 1 --2- 172.21.15.133:0/516887341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8467cf0 0x560cc84692d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.467+0000 7fa0ae316740 1 --2- 172.21.15.133:0/516887341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 0x560cc847d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.467+0000 7fa0ae316740 1 -- 172.21.15.133:0/516887341 >> 172.21.15.133:0/516887341 conn(0x560cc842f310 msgr2=0x560cc846d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:33.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.467+0000 7fa0ae316740 1 -- 172.21.15.133:0/516887341 shutdown_connections 2024-09-11T23:43:33.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.467+0000 7fa0ae316740 1 -- 172.21.15.133:0/516887341 wait complete. 2024-09-11T23:43:33.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.468+0000 7fa0ae316740 1 -- start start 2024-09-11T23:43:33.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.469+0000 7fa0ae316740 1 Processor -- start 2024-09-11T23:43:33.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.471+0000 7fa0ae316740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 0x560cc8479480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.472+0000 7fa0ae316740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8467cf0 0x560cc84799c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.475+0000 7fa0a9cd9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 0x560cc8479480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.475+0000 7fa0a9cd9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 0x560cc8479480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50030/0 (socket says 172.21.15.133:50030) 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.476+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/170153055 learned_addr learned my addr 172.21.15.133:0/170153055 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.476+0000 7fa0ae316740 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8471520 0x560cc8474910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.477+0000 7fa0ae316740 1 -- 172.21.15.133:0/170153055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41c5cc0 con 0x560cc8466780 2024-09-11T23:43:33.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.477+0000 7fa0ae316740 1 -- 172.21.15.133:0/170153055 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41c55f0 con 0x560cc8471520 2024-09-11T23:43:33.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.477+0000 7fa0ae316740 1 -- 172.21.15.133:0/170153055 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41c28e0 con 0x560cc8467cf0 2024-09-11T23:43:33.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.477+0000 7fa0ae316740 10 client.0 _ll_register_callbacks cb 0x7fa0940bb060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.478+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/170153055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8467cf0 msgr2=0x560cc84799c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.478+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8467cf0 0x560cc84799c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.478+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/170153055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8471520 msgr2=0x560cc8474910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.478+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8471520 0x560cc8474910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.478+0000 7fa0a9cd9640 1 -- 172.21.15.133:0/170153055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc8475140 con 0x560cc8466780 2024-09-11T23:43:33.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.480+0000 7fa0a9cd9640 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 0x560cc8479480 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa0940b76f0 tx=0x7fa09402a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9b8b2df33226a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:33.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.480+0000 7fa08d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:33.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.481+0000 7fa08d7fa640 1 -- 172.21.15.133:0/170153055 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0940c39e0 con 0x560cc8466780 2024-09-11T23:43:33.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.482+0000 7fa08d7fa640 1 -- 172.21.15.133:0/170153055 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fa0940bb700 con 0x560cc8466780 2024-09-11T23:43:33.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.482+0000 7fa0ae316740 1 -- 172.21.15.133:0/170153055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a0017040 con 0x560cc8466780 2024-09-11T23:43:33.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.483+0000 7fa08d7fa640 1 -- 172.21.15.133:0/170153055 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa0940bf640 con 0x560cc8466780 2024-09-11T23:43:33.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.484+0000 7fa0ae316740 10 client.17244 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:33.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.484+0000 7fa0ae316740 1 -- 172.21.15.133:0/170153055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560cc8475140 con 0x560cc8466780 2024-09-11T23:43:33.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.486+0000 7fa08d7fa640 1 -- 172.21.15.133:0/170153055 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa09409c050 con 0x560cc8466780 2024-09-11T23:43:33.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.486+0000 7fa08d7fa640 1 -- 172.21.15.133:0/170153055 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa0940ba6e0 con 0x560cc8466780 2024-09-11T23:43:33.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.487+0000 7fa0ae316740 10 client.17244 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:33.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.487+0000 7fa0ae316740 10 client.17244 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:43:33.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.487+0000 7fa0ae316740 20 client.17244 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:33.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.487+0000 7fa0ae316740 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560cc8480dc0 0x560cc84ed8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:33.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.487+0000 7fa0ae316740 4 client.17244 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:43:33.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.487+0000 7fa0ae316740 1 -- 172.21.15.133:0/170153055 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa0a0017040 con 0x560cc8480dc0 2024-09-11T23:43:33.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.488+0000 7fa0a94d8640 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560cc8480dc0 0x560cc84ed8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:33.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.489+0000 7fa0a94d8640 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560cc8480dc0 0x560cc84ed8f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=33725870e2501bbc server_cookie=31f4028dcdd81d96 in_seq=0 out_seq=0 2024-09-11T23:43:33.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.489+0000 7fa08d7fa640 10 client.17244 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:43:33.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.490+0000 7fa08d7fa640 1 -- 172.21.15.133:0/170153055 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa0a0017040 con 0x560cc8480dc0 2024-09-11T23:43:33.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.490+0000 7fa08d7fa640 10 client.17244 handle_command_reply: tid=0 2024-09-11T23:43:33.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.490+0000 7fa0ae316740 1 client.17244 shutdown 2024-09-11T23:43:33.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.491+0000 7fa0ae316740 1 -- 172.21.15.133:0/170153055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 msgr2=0x560cc8479480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:33.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.491+0000 7fa0ae316740 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 0x560cc8479480 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa0940b76f0 tx=0x7fa09402a380 comp rx=0 tx=0).stop 2024-09-11T23:43:33.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.491+0000 7fa0ae316740 1 -- 172.21.15.133:0/170153055 shutdown_connections 2024-09-11T23:43:33.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.491+0000 7fa0ae316740 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x560cc8480dc0 0x560cc84ed8f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.491+0000 7fa0ae316740 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cc8471520 0x560cc8474910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.491+0000 7fa0ae316740 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560cc8467cf0 0x560cc84799c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.491+0000 7fa0ae316740 1 --2- 172.21.15.133:0/170153055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cc8466780 0x560cc8479480 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.491+0000 7fa0ae316740 1 -- 172.21.15.133:0/170153055 >> 172.21.15.133:0/170153055 conn(0x560cc842f310 msgr2=0x560cc8461690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:33.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.492+0000 7fa08d7fa640 0 client.17244 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:43:33.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.492+0000 7fa0ae316740 1 -- 172.21.15.133:0/170153055 shutdown_connections 2024-09-11T23:43:33.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.492+0000 7fa0ae316740 1 -- 172.21.15.133:0/170153055 wait complete. 2024-09-11T23:43:33.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.495+0000 7fa0ae316740 20 client.17244 trim_cache size 0 max 16384 2024-09-11T23:43:33.497 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:43:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.505+0000 7fa0aacdb640 1 -- 172.21.15.133:0/2747223143 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa0900a2ce0 msgr2=0x7fa0900a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.505+0000 7fa0aacdb640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa0900a2ce0 0x7fa0900a51a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0005d00 tx=0x7fa0a0005e50 comp rx=0 tx=0).stop 2024-09-11T23:43:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.505+0000 7fa0aacdb640 1 -- 172.21.15.133:0/2747223143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0a406b8a0 msgr2=0x7fa0a411a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.505+0000 7fa0aacdb640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0a406b8a0 0x7fa0a411a960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa094035040 tx=0x7fa094033d10 comp rx=0 tx=0).stop 2024-09-11T23:43:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.506+0000 7fa0aacdb640 1 -- 172.21.15.133:0/2747223143 shutdown_connections 2024-09-11T23:43:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.506+0000 7fa0aacdb640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa0900a2ce0 0x7fa0900a51a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.506+0000 7fa0aacdb640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0a411b380 0x7fa0a4113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.506+0000 7fa0aacdb640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a406c1c0 0x7fa0a41139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.506+0000 7fa0aacdb640 1 --2- 172.21.15.133:0/2747223143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0a406b8a0 0x7fa0a411a960 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.506+0000 7fa0aacdb640 1 -- 172.21.15.133:0/2747223143 >> 172.21.15.133:0/2747223143 conn(0x7fa0a4071390 msgr2=0x7fa0a4071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.506+0000 7fa0aacdb640 1 -- 172.21.15.133:0/2747223143 shutdown_connections 2024-09-11T23:43:33.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:33.507+0000 7fa0aacdb640 1 -- 172.21.15.133:0/2747223143 wait complete. 2024-09-11T23:43:34.383 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:43:34.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:34 smithi138 ceph-mon[35142]: pgmap v209: 97 pgs: 97 active+clean; 56 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 694 MiB/s wr, 3.15k op/s 2024-09-11T23:43:34.964 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:34 smithi133 ceph-mon[31430]: pgmap v209: 97 pgs: 97 active+clean; 56 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 694 MiB/s wr, 3.15k op/s 2024-09-11T23:43:35.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:34 smithi178 ceph-mon[35882]: pgmap v209: 97 pgs: 97 active+clean; 56 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 694 MiB/s wr, 3.15k op/s 2024-09-11T23:43:35.062 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:35.821 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:35 smithi133 ceph-mon[31430]: pgmap v210: 97 pgs: 97 active+clean; 57 GiB data, 185 GiB used, 888 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 617 MiB/s wr, 2.42k op/s 2024-09-11T23:43:35.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.950+0000 7f4d10bc8640 1 -- 172.21.15.133:0/1390171757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d0c101420 msgr2=0x7f4d0c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:35.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.950+0000 7f4d10bc8640 1 --2- 172.21.15.133:0/1390171757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d0c101420 0x7f4d0c10d950 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009900 tx=0x7f4d0002a440 comp rx=0 tx=0).stop 2024-09-11T23:43:35.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.951+0000 7f4d10bc8640 1 -- 172.21.15.133:0/1390171757 shutdown_connections 2024-09-11T23:43:35.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.951+0000 7f4d10bc8640 1 --2- 172.21.15.133:0/1390171757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d0c101420 0x7f4d0c10d950 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:35.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.951+0000 7f4d10bc8640 1 --2- 172.21.15.133:0/1390171757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d0c100a80 0x7f4d0c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:35.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.951+0000 7f4d10bc8640 1 --2- 172.21.15.133:0/1390171757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c106a80 0x7f4d0c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:35.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.951+0000 7f4d10bc8640 1 -- 172.21.15.133:0/1390171757 >> 172.21.15.133:0/1390171757 conn(0x7f4d0c0fc7a0 msgr2=0x7f4d0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:35.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.960+0000 7f4d10bc8640 1 -- 172.21.15.133:0/1390171757 shutdown_connections 2024-09-11T23:43:35.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.960+0000 7f4d10bc8640 1 -- 172.21.15.133:0/1390171757 wait complete. 2024-09-11T23:43:35.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.963+0000 7f4d10bc8640 1 Processor -- start 2024-09-11T23:43:35.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.964+0000 7f4d10bc8640 1 -- start start 2024-09-11T23:43:35.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.965+0000 7f4d10bc8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d0c100a80 0x7f4d0c1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:35.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.965+0000 7f4d10bc8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c101420 0x7f4d0c116610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:35.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.965+0000 7f4d10bc8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d0c106a80 0x7f4d0c1111f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:35.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.965+0000 7f4d10bc8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c10fe30 con 0x7f4d0c100a80 2024-09-11T23:43:35.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.966+0000 7f4d10bc8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c10fcb0 con 0x7f4d0c106a80 2024-09-11T23:43:35.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.966+0000 7f4d10bc8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c10ffb0 con 0x7f4d0c101420 2024-09-11T23:43:35.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.966+0000 7f4d0ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d0c106a80 0x7f4d0c1111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:35.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.966+0000 7f4d0ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d0c106a80 0x7f4d0c1111f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38868/0 (socket says 172.21.15.133:38868) 2024-09-11T23:43:35.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.966+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/3910113543 learned_addr learned my addr 172.21.15.133:0/3910113543 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:35.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.967+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/3910113543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c101420 msgr2=0x7f4d0c116610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:43:35.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.967+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c101420 0x7f4d0c116610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:35.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.967+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/3910113543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d0c100a80 msgr2=0x7f4d0c1160d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:43:35.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.967+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d0c100a80 0x7f4d0c1160d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:35.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.967+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/3910113543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c111ab0 con 0x7f4d0c106a80 2024-09-11T23:43:35.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.968+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d0c106a80 0x7f4d0c1111f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4d0002a9b0 tx=0x7f4d00030940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:35.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.969+0000 7f4cfb7fe640 1 -- 172.21.15.133:0/3910113543 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0003f020 con 0x7f4d0c106a80 2024-09-11T23:43:35.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.969+0000 7f4d10bc8640 1 -- 172.21.15.133:0/3910113543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0c111d40 con 0x7f4d0c106a80 2024-09-11T23:43:35.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.970+0000 7f4d10bc8640 1 -- 172.21.15.133:0/3910113543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d0c11bfc0 con 0x7f4d0c106a80 2024-09-11T23:43:35.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.970+0000 7f4cfb7fe640 1 -- 172.21.15.133:0/3910113543 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d00004050 con 0x7f4d0c106a80 2024-09-11T23:43:35.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.972+0000 7f4cfb7fe640 1 -- 172.21.15.133:0/3910113543 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00035070 con 0x7f4d0c106a80 2024-09-11T23:43:35.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.979+0000 7f4cfb7fe640 1 -- 172.21.15.133:0/3910113543 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4d00042020 con 0x7f4d0c106a80 2024-09-11T23:43:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.981+0000 7f4d12fa1740 1 Processor -- start 2024-09-11T23:43:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.982+0000 7f4cfb7fe640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf40a2c20 0x7f4cf40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:35.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.983+0000 7f4cfb7fe640 1 -- 172.21.15.133:0/3910113543 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f4d000f0220 con 0x7f4d0c106a80 2024-09-11T23:43:35.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.982+0000 7f4d12fa1740 1 -- start start 2024-09-11T23:43:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.985+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf8b40 0x5591bfbf7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.985+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf75d0 0x5591bfbf7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.985+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfc02370 0x5591bfc04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.985+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5591bfb5d2f0 con 0x5591bfbf8b40 2024-09-11T23:43:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.985+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5591bfb04cf0 con 0x5591bfc02370 2024-09-11T23:43:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.985+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5591bfb592b0 con 0x5591bfbf75d0 2024-09-11T23:43:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.988+0000 7f4d09d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf75d0 0x5591bfbf7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.988+0000 7f4d09d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf75d0 0x5591bfbf7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:55380/0 (socket says 172.21.15.133:55380) 2024-09-11T23:43:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.988+0000 7f4d09d9b640 1 -- 172.21.15.133:0/445527679 learned_addr learned my addr 172.21.15.133:0/445527679 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.988+0000 7f4d09d9b640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf40a2c20 0x7f4cf40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.988+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/445527679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfc02370 0x5591bfc04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.988+0000 7f4d0a59c640 1 --2- 172.21.15.133:0/445527679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf8b40 0x5591bfbf7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:35.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.993+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/445527679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf75d0 msgr2=0x5591bfbf7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:35.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.993+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/445527679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf75d0 0x5591bfbf7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:35.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.993+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/445527679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf8b40 msgr2=0x5591bfbf7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:35.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.993+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/445527679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf8b40 0x5591bfbf7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:35.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.993+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/445527679 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591bfae93a0 con 0x5591bfc02370 2024-09-11T23:43:35.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:35.996+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/445527679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfc02370 0x5591bfc04760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4d000ed9b0 tx=0x7f4d0002a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2347240b9a02f54f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:36.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.001+0000 7f4d09d9b640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf40a2c20 0x7f4cf40a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00bfd0 tx=0x7f4cfc00b330 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:36.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.006+0000 7f4cf8ff9640 1 -- 172.21.15.133:0/445527679 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d000d5070 con 0x5591bfc02370 2024-09-11T23:43:36.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.006+0000 7f4cf8ff9640 1 -- 172.21.15.133:0/445527679 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0000b0b0 con 0x5591bfc02370 2024-09-11T23:43:36.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.006+0000 7f4cf8ff9640 1 -- 172.21.15.133:0/445527679 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d000edc30 con 0x5591bfc02370 2024-09-11T23:43:36.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.008+0000 7f4d12fa1740 1 -- 172.21.15.133:0/445527679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfc02370 msgr2=0x5591bfc04760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:36.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.008+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/445527679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfc02370 0x5591bfc04760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4d000ed9b0 tx=0x7f4d0002a440 comp rx=0 tx=0).stop 2024-09-11T23:43:36.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.008+0000 7f4d12fa1740 1 -- 172.21.15.133:0/445527679 shutdown_connections 2024-09-11T23:43:36.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.008+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/445527679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfc02370 0x5591bfc04760 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.008+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/445527679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf75d0 0x5591bfbf7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.008+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/445527679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf8b40 0x5591bfbf7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.008+0000 7f4d12fa1740 1 -- 172.21.15.133:0/445527679 >> 172.21.15.133:0/445527679 conn(0x5591bfbc0310 msgr2=0x5591bfbf3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:36.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.010+0000 7f4d12fa1740 1 -- 172.21.15.133:0/445527679 shutdown_connections 2024-09-11T23:43:36.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.010+0000 7f4d12fa1740 1 -- 172.21.15.133:0/445527679 wait complete. 2024-09-11T23:43:36.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.011+0000 7f4d12fa1740 1 -- start start 2024-09-11T23:43:36.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.011+0000 7f4d12fa1740 1 Processor -- start 2024-09-11T23:43:36.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.012+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf75d0 0x5591bfc01d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.012+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 0x5591bfbfefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.012+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfc02370 0x5591bfbff530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.012+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5591bfc04e30 con 0x5591bfbf75d0 2024-09-11T23:43:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.012+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5591bfb18060 con 0x5591bfc02370 2024-09-11T23:43:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.012+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5591bfc04fb0 con 0x5591bfbf8b40 2024-09-11T23:43:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.012+0000 7f4d12fa1740 10 client.0 _ll_register_callbacks cb 0x5591bf86b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:36.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.016+0000 7f4d0a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf75d0 0x5591bfc01d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:36.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.016+0000 7f4d0a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf75d0 0x5591bfc01d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50056/0 (socket says 172.21.15.133:50056) 2024-09-11T23:43:36.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.016+0000 7f4d0a59c640 1 -- 172.21.15.133:0/3515758106 learned_addr learned my addr 172.21.15.133:0/3515758106 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:36.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.017+0000 7f4d0a59c640 1 -- 172.21.15.133:0/3515758106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 msgr2=0x5591bfbfefb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:36.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.017+0000 7f4d0a59c640 1 --2- 172.21.15.133:0/3515758106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 0x5591bfbfefb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.017+0000 7f4d0a59c640 1 -- 172.21.15.133:0/3515758106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfc02370 msgr2=0x5591bfbff530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:36.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.017+0000 7f4d0a59c640 1 --2- 172.21.15.133:0/3515758106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfc02370 0x5591bfbff530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.017+0000 7f4d0a59c640 1 -- 172.21.15.133:0/3515758106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591bf8ce3c0 con 0x5591bfbf75d0 2024-09-11T23:43:36.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.019+0000 7f4d0a59c640 1 --2- 172.21.15.133:0/3515758106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf75d0 0x5591bfc01d00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400aad0 tx=0x7f4d0400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a920a69c9a5f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:36.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.019+0000 7f4cf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:36.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.020+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/3515758106 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04018020 con 0x5591bfbf75d0 2024-09-11T23:43:36.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.021+0000 7f4d12fa1740 1 -- 172.21.15.133:0/3515758106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d0000b0b0 con 0x5591bfbf75d0 2024-09-11T23:43:36.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.021+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/3515758106 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f4d0400ec30 con 0x5591bfbf75d0 2024-09-11T23:43:36.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.022+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/3515758106 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d04023050 con 0x5591bfbf75d0 2024-09-11T23:43:36.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.023+0000 7f4d12fa1740 10 client.17250 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:36.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.023+0000 7f4d12fa1740 1 -- 172.21.15.133:0/3515758106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591bfc21880 con 0x5591bfbf75d0 2024-09-11T23:43:36.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.026+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/3515758106 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0000b0b0 con 0x5591bfbf75d0 2024-09-11T23:43:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.028+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/3515758106 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4d0401c5b0 con 0x5591bfbf75d0 2024-09-11T23:43:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.028+0000 7f4d12fa1740 10 client.17250 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.028+0000 7f4d12fa1740 10 client.17250 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:43:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.028+0000 7f4d12fa1740 20 client.17250 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.028+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/3515758106 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591bfc22e30 0x5591bfc252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.028+0000 7f4d12fa1740 4 client.17250 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.028+0000 7f4d12fa1740 1 -- 172.21.15.133:0/3515758106 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591bfb6a860 con 0x5591bfc22e30 2024-09-11T23:43:36.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.029+0000 7f4d09d9b640 1 --2- 172.21.15.133:0/3515758106 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591bfc22e30 0x5591bfc252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:36.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.033+0000 7f4d09d9b640 1 --2- 172.21.15.133:0/3515758106 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591bfc22e30 0x5591bfc252c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f1e86ee55df3491d server_cookie=887fb9cbd1012ee1 in_seq=0 out_seq=0 2024-09-11T23:43:36.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.034+0000 7f4cf1ffb640 10 client.17250 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:43:36.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.035+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/3515758106 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591bfb6a860 con 0x5591bfc22e30 2024-09-11T23:43:36.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.035+0000 7f4cf1ffb640 10 client.17250 handle_command_reply: tid=0 2024-09-11T23:43:36.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.036+0000 7f4d12fa1740 1 client.17250 shutdown 2024-09-11T23:43:36.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.036+0000 7f4d12fa1740 1 -- 172.21.15.133:0/3515758106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf75d0 msgr2=0x5591bfc01d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:36.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.036+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/3515758106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf75d0 0x5591bfc01d00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400aad0 tx=0x7f4d0400af90 comp rx=0 tx=0).stop 2024-09-11T23:43:36.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:35 smithi178 ceph-mon[35882]: pgmap v210: 97 pgs: 97 active+clean; 57 GiB data, 185 GiB used, 888 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 617 MiB/s wr, 2.42k op/s 2024-09-11T23:43:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.040+0000 7f4d12fa1740 1 -- 172.21.15.133:0/3515758106 shutdown_connections 2024-09-11T23:43:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.040+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/3515758106 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591bfc22e30 0x5591bfc252c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.040+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/3515758106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfc02370 0x5591bfbff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.040+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/3515758106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 0x5591bfbfefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.040+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/3515758106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfbf75d0 0x5591bfc01d00 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.040+0000 7f4cf1ffb640 0 client.17250 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:43:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.040+0000 7f4d12fa1740 1 -- 172.21.15.133:0/3515758106 >> 172.21.15.133:0/3515758106 conn(0x5591bfbc0310 msgr2=0x5591bfbf99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.041+0000 7f4d12fa1740 1 -- 172.21.15.133:0/3515758106 shutdown_connections 2024-09-11T23:43:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.041+0000 7f4d12fa1740 1 -- 172.21.15.133:0/3515758106 wait complete. 2024-09-11T23:43:36.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.041+0000 7f4d12fa1740 20 client.17250 trim_cache size 0 max 16384 2024-09-11T23:43:36.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.100+0000 7f4d12fa1740 1 Processor -- start 2024-09-11T23:43:36.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.101+0000 7f4d12fa1740 1 -- start start 2024-09-11T23:43:36.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.102+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfbf75d0 0x5591bfbf3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:36.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.102+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 0x5591bfbfe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:36.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.102+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc2ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:36.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.102+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec00e5b0 con 0x5591bfc02370 2024-09-11T23:43:36.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.102+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec0100f0 con 0x5591bfbf75d0 2024-09-11T23:43:36.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.102+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec00ec80 con 0x5591bfbf8b40 2024-09-11T23:43:36.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.104+0000 7f4d09d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 0x5591bfbfe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:36.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.104+0000 7f4d0ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc2ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:36.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.104+0000 7f4d0ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc2ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50068/0 (socket says 172.21.15.133:50068) 2024-09-11T23:43:36.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.104+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/2383384617 learned_addr learned my addr 172.21.15.133:0/2383384617 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:36.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.106+0000 7f4d0a59c640 1 --2- 172.21.15.133:0/2383384617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfbf75d0 0x5591bfbf3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:36.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.107+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/2383384617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 msgr2=0x5591bfbfe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:36.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.107+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/2383384617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 0x5591bfbfe870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.107+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/2383384617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfbf75d0 msgr2=0x5591bfbf3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:36.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.107+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/2383384617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfbf75d0 0x5591bfbf3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.107+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/2383384617 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591bfbfdf60 con 0x5591bfc02370 2024-09-11T23:43:36.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.111+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/2383384617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc2ab90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4d000baa30 tx=0x7f4d000f01b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624b4245ecac65a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:36.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.115+0000 7f4cf3fff640 1 -- 172.21.15.133:0/2383384617 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d000d5070 con 0x5591bfc02370 2024-09-11T23:43:36.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.115+0000 7f4cf3fff640 1 -- 172.21.15.133:0/2383384617 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d000bce80 con 0x5591bfc02370 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.117+0000 7f4d12fa1740 1 -- 172.21.15.133:0/2383384617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 msgr2=0x5591bfc2ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.117+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/2383384617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc2ab90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4d000baa30 tx=0x7f4d000f01b0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.120+0000 7f4d12fa1740 1 -- 172.21.15.133:0/2383384617 shutdown_connections 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.120+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/2383384617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc2ab90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.120+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/2383384617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 0x5591bfbfe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.120+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/2383384617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfbf75d0 0x5591bfbf3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.120+0000 7f4d12fa1740 1 -- 172.21.15.133:0/2383384617 >> 172.21.15.133:0/2383384617 conn(0x5591bfbc0310 msgr2=0x5591bfbf3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.120+0000 7f4d12fa1740 1 -- 172.21.15.133:0/2383384617 shutdown_connections 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.120+0000 7f4d12fa1740 1 -- 172.21.15.133:0/2383384617 wait complete. 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.121+0000 7f4d12fa1740 1 -- start start 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.121+0000 7f4d12fa1740 1 Processor -- start 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.122+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfbf75d0 0x5591bfc0a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:36.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.122+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 0x5591bfc05c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.122+0000 7f4d12fa1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc06160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.122+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec00fa20 con 0x5591bfc02370 2024-09-11T23:43:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.122+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec00f350 con 0x5591bfbf75d0 2024-09-11T23:43:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.122+0000 7f4d12fa1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec00c640 con 0x5591bfbf8b40 2024-09-11T23:43:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.122+0000 7f4d12fa1740 10 client.0 _ll_register_callbacks cb 0x7f4d0400a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.125+0000 7f4d0ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc06160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.126+0000 7f4d0ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc06160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50072/0 (socket says 172.21.15.133:50072) 2024-09-11T23:43:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.126+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/4148423504 learned_addr learned my addr 172.21.15.133:0/4148423504 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.126+0000 7f4d09d9b640 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 0x5591bfc05c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:36.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.126+0000 7f4d0a59c640 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfbf75d0 0x5591bfc0a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:36.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.127+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/4148423504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 msgr2=0x5591bfc05c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:36.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.127+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 0x5591bfc05c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.127+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/4148423504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfbf75d0 msgr2=0x5591bfc0a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:36.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.127+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfbf75d0 0x5591bfc0a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.127+0000 7f4d0ad9d640 1 -- 172.21.15.133:0/4148423504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591bfc06990 con 0x5591bfc02370 2024-09-11T23:43:36.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.128+0000 7f4d0ad9d640 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc06160 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4d000bc480 tx=0x7f4d000b9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e922be42ab6ac7ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:36.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.128+0000 7f4cf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:36.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.129+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/4148423504 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d000db050 con 0x5591bfc02370 2024-09-11T23:43:36.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.129+0000 7f4d12fa1740 1 -- 172.21.15.133:0/4148423504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d000bce80 con 0x5591bfc02370 2024-09-11T23:43:36.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.129+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/4148423504 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f4d000bc9e0 con 0x5591bfc02370 2024-09-11T23:43:36.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.130+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/4148423504 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d000d5040 con 0x5591bfc02370 2024-09-11T23:43:36.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.131+0000 7f4d12fa1740 10 client.17256 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:36.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.131+0000 7f4d12fa1740 1 -- 172.21.15.133:0/4148423504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591bfc8dad0 con 0x5591bfc02370 2024-09-11T23:43:36.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.133+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/4148423504 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d000bce80 con 0x5591bfc02370 2024-09-11T23:43:36.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.133+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/4148423504 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4d000b8040 con 0x5591bfc02370 2024-09-11T23:43:36.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.133+0000 7f4d12fa1740 10 client.17256 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:36.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.133+0000 7f4d12fa1740 10 client.17256 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:43:36.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.133+0000 7f4d12fa1740 20 client.17256 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:36.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.133+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591bfc8dd20 0x5591bfc90140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:36.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.133+0000 7f4d12fa1740 4 client.17256 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:43:36.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.133+0000 7f4d12fa1740 1 -- 172.21.15.133:0/4148423504 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591bfb64d70 con 0x5591bfc8dd20 2024-09-11T23:43:36.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.135+0000 7f4d0a59c640 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591bfc8dd20 0x5591bfc90140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:36.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.139+0000 7f4d0a59c640 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591bfc8dd20 0x5591bfc90140 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2f3dbdae3fe9920d server_cookie=1f383ec0ce5f4877 in_seq=0 out_seq=0 2024-09-11T23:43:36.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.139+0000 7f4cf1ffb640 10 client.17256 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:43:36.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.140+0000 7f4cf1ffb640 1 -- 172.21.15.133:0/4148423504 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591bfb64d70 con 0x5591bfc8dd20 2024-09-11T23:43:36.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.140+0000 7f4cf1ffb640 10 client.17256 handle_command_reply: tid=0 2024-09-11T23:43:36.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.140+0000 7f4d12fa1740 1 client.17256 shutdown 2024-09-11T23:43:36.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.140+0000 7f4d12fa1740 1 -- 172.21.15.133:0/4148423504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 msgr2=0x5591bfc06160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:36.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.140+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc06160 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4d000bc480 tx=0x7f4d000b9000 comp rx=0 tx=0).stop 2024-09-11T23:43:36.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.145+0000 7f4d12fa1740 1 -- 172.21.15.133:0/4148423504 shutdown_connections 2024-09-11T23:43:36.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.145+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591bfc8dd20 0x5591bfc90140 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.145+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591bfc02370 0x5591bfc06160 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.145+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591bfbf8b40 0x5591bfc05c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.145+0000 7f4d12fa1740 1 --2- 172.21.15.133:0/4148423504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591bfbf75d0 0x5591bfc0a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.145+0000 7f4d12fa1740 1 -- 172.21.15.133:0/4148423504 >> 172.21.15.133:0/4148423504 conn(0x5591bfbc0310 msgr2=0x5591bfc0d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:36.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.145+0000 7f4cf1ffb640 0 client.17256 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:43:36.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.145+0000 7f4d12fa1740 1 -- 172.21.15.133:0/4148423504 shutdown_connections 2024-09-11T23:43:36.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.146+0000 7f4d12fa1740 1 -- 172.21.15.133:0/4148423504 wait complete. 2024-09-11T23:43:36.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.146+0000 7f4d12fa1740 20 client.17256 trim_cache size 0 max 16384 2024-09-11T23:43:36.148 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:43:36.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.151+0000 7f4d10bc8640 1 -- 172.21.15.133:0/3910113543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf40a2c20 msgr2=0x7f4cf40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:36.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.151+0000 7f4d10bc8640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf40a2c20 0x7f4cf40a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00bfd0 tx=0x7f4cfc00b330 comp rx=0 tx=0).stop 2024-09-11T23:43:36.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.151+0000 7f4d10bc8640 1 -- 172.21.15.133:0/3910113543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d0c106a80 msgr2=0x7f4d0c1111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:36.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.151+0000 7f4d10bc8640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d0c106a80 0x7f4d0c1111f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4d0002a9b0 tx=0x7f4d00030940 comp rx=0 tx=0).stop 2024-09-11T23:43:36.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.155+0000 7f4d10bc8640 1 -- 172.21.15.133:0/3910113543 shutdown_connections 2024-09-11T23:43:36.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.155+0000 7f4d10bc8640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4cf40a2c20 0x7f4cf40a50e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.155+0000 7f4d10bc8640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d0c106a80 0x7f4d0c1111f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.155+0000 7f4d10bc8640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c101420 0x7f4d0c116610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.155+0000 7f4d10bc8640 1 --2- 172.21.15.133:0/3910113543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d0c100a80 0x7f4d0c1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:36.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.155+0000 7f4d10bc8640 1 -- 172.21.15.133:0/3910113543 >> 172.21.15.133:0/3910113543 conn(0x7f4d0c0fc7a0 msgr2=0x7f4d0c0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:36.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.157+0000 7f4d10bc8640 1 -- 172.21.15.133:0/3910113543 shutdown_connections 2024-09-11T23:43:36.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:36.157+0000 7f4d10bc8640 1 -- 172.21.15.133:0/3910113543 wait complete. 2024-09-11T23:43:36.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:35 smithi138 ceph-mon[35142]: pgmap v210: 97 pgs: 97 active+clean; 57 GiB data, 185 GiB used, 888 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 617 MiB/s wr, 2.42k op/s 2024-09-11T23:43:37.039 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:43:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.765+0000 7f5b558d9640 1 -- 172.21.15.133:0/1912236515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b50100b20 msgr2=0x7f5b5010cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.765+0000 7f5b558d9640 1 --2- 172.21.15.133:0/1912236515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b50100b20 0x7f5b5010cde0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800bd70 tx=0x7f5b4802c450 comp rx=0 tx=0).stop 2024-09-11T23:43:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.766+0000 7f5b558d9640 1 -- 172.21.15.133:0/1912236515 shutdown_connections 2024-09-11T23:43:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.766+0000 7f5b558d9640 1 --2- 172.21.15.133:0/1912236515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b50100b20 0x7f5b5010cde0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.766+0000 7f5b558d9640 1 --2- 172.21.15.133:0/1912236515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b50100180 0x7f5b501005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.766+0000 7f5b558d9640 1 --2- 172.21.15.133:0/1912236515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b501060d0 0x7f5b501064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.766+0000 7f5b558d9640 1 -- 172.21.15.133:0/1912236515 >> 172.21.15.133:0/1912236515 conn(0x7f5b50077200 msgr2=0x7f5b50077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.767+0000 7f5b558d9640 1 -- 172.21.15.133:0/1912236515 shutdown_connections 2024-09-11T23:43:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.767+0000 7f5b558d9640 1 -- 172.21.15.133:0/1912236515 wait complete. 2024-09-11T23:43:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.768+0000 7f5b558d9640 1 Processor -- start 2024-09-11T23:43:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.768+0000 7f5b558d9640 1 -- start start 2024-09-11T23:43:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b558d9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50100180 0x7f5b50073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b558d9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b50100b20 0x7f5b5006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b558d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b501060d0 0x7f5b5006e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b558d9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5006c640 con 0x7f5b501060d0 2024-09-11T23:43:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b558d9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5006c4c0 con 0x7f5b50100180 2024-09-11T23:43:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b558d9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5006c7c0 con 0x7f5b50100b20 2024-09-11T23:43:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b4effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50100180 0x7f5b50073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b4effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50100180 0x7f5b50073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38926/0 (socket says 172.21.15.133:38926) 2024-09-11T23:43:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b4effd640 1 -- 172.21.15.133:0/3116445415 learned_addr learned my addr 172.21.15.133:0/3116445415 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b4e7fc640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b50100b20 0x7f5b5006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b4effd640 1 -- 172.21.15.133:0/3116445415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b50100b20 msgr2=0x7f5b5006dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b4effd640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b50100b20 0x7f5b5006dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b4effd640 1 -- 172.21.15.133:0/3116445415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b501060d0 msgr2=0x7f5b5006e180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:43:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b4effd640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b501060d0 0x7f5b5006e180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.769+0000 7f5b4effd640 1 -- 172.21.15.133:0/3116445415 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5006e920 con 0x7f5b50100180 2024-09-11T23:43:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.770+0000 7f5b4effd640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50100180 0x7f5b50073bd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000fb00 tx=0x7f5b40009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.770+0000 7f5b548d7640 1 -- 172.21.15.133:0/3116445415 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000dc90 con 0x7f5b50100180 2024-09-11T23:43:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.770+0000 7f5b558d9640 1 -- 172.21.15.133:0/3116445415 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b501a4e10 con 0x7f5b50100180 2024-09-11T23:43:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.771+0000 7f5b558d9640 1 -- 172.21.15.133:0/3116445415 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b501a5360 con 0x7f5b50100180 2024-09-11T23:43:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.771+0000 7f5b548d7640 1 -- 172.21.15.133:0/3116445415 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4000de30 con 0x7f5b50100180 2024-09-11T23:43:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.771+0000 7f5b548d7640 1 -- 172.21.15.133:0/3116445415 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40017410 con 0x7f5b50100180 2024-09-11T23:43:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.777+0000 7f5b548d7640 1 -- 172.21.15.133:0/3116445415 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5b40017670 con 0x7f5b50100180 2024-09-11T23:43:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.779+0000 7f5b548d7640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b380a2f00 0x7f5b380a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.780+0000 7f5b548d7640 1 -- 172.21.15.133:0/3116445415 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f5b400d0190 con 0x7f5b50100180 2024-09-11T23:43:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.781+0000 7f5b4e7fc640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b380a2f00 0x7f5b380a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.783+0000 7f5b5866f740 1 Processor -- start 2024-09-11T23:43:37.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.783+0000 7f5b5866f740 1 -- start start 2024-09-11T23:43:37.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.784+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a310cc0 0x556c7a3110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.784+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 0x556c7a31b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.784+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a31be80 0x556c7a31e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.785+0000 7f5b4effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a310cc0 0x556c7a3110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.785+0000 7f5b4effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a310cc0 0x556c7a3110a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50076/0 (socket says 172.21.15.133:50076) 2024-09-11T23:43:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.785+0000 7f5b4e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 0x556c7a31b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.785+0000 7f5b4e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 0x556c7a31b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:55432/0 (socket says 172.21.15.133:55432) 2024-09-11T23:43:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.785+0000 7f5b4f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a31be80 0x556c7a31e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.785+0000 7f5b4f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a31be80 0x556c7a31e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38934/0 (socket says 172.21.15.133:38934) 2024-09-11T23:43:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.785+0000 7f5b4effd640 1 -- 172.21.15.133:0/3667660851 learned_addr learned my addr 172.21.15.133:0/3667660851 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.785+0000 7f5b5866f740 1 -- 172.21.15.133:0/3667660851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556c7a2752f0 con 0x556c7a310cc0 2024-09-11T23:43:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.785+0000 7f5b5866f740 1 -- 172.21.15.133:0/3667660851 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556c7a21d270 con 0x556c7a31be80 2024-09-11T23:43:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.785+0000 7f5b5866f740 1 -- 172.21.15.133:0/3667660851 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556c7a2712b0 con 0x556c7a3115e0 2024-09-11T23:43:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.787+0000 7f5b4effd640 1 -- 172.21.15.133:0/3667660851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 msgr2=0x556c7a31b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.787+0000 7f5b4effd640 1 --2- 172.21.15.133:0/3667660851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 0x556c7a31b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.787+0000 7f5b4effd640 1 -- 172.21.15.133:0/3667660851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a31be80 msgr2=0x556c7a31e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.787+0000 7f5b4effd640 1 --2- 172.21.15.133:0/3667660851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a31be80 0x556c7a31e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.787+0000 7f5b4effd640 1 -- 172.21.15.133:0/3667660851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7a21c340 con 0x556c7a310cc0 2024-09-11T23:43:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.788+0000 7f5b4effd640 1 --2- 172.21.15.133:0/3667660851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a310cc0 0x556c7a3110a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5b400a4dd0 tx=0x7f5b4009ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf34e4be9685600 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.801+0000 7f5b35ffb640 1 -- 172.21.15.133:0/3667660851 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b400b4020 con 0x556c7a310cc0 2024-09-11T23:43:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.802+0000 7f5b35ffb640 1 -- 172.21.15.133:0/3667660851 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b400a5300 con 0x556c7a310cc0 2024-09-11T23:43:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.803+0000 7f5b5866f740 1 -- 172.21.15.133:0/3667660851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a310cc0 msgr2=0x556c7a3110a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.803+0000 7f5b5866f740 1 --2- 172.21.15.133:0/3667660851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a310cc0 0x556c7a3110a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5b400a4dd0 tx=0x7f5b4009ce50 comp rx=0 tx=0).stop 2024-09-11T23:43:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.806+0000 7f5b5866f740 1 -- 172.21.15.133:0/3667660851 shutdown_connections 2024-09-11T23:43:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.806+0000 7f5b5866f740 1 --2- 172.21.15.133:0/3667660851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a31be80 0x556c7a31e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.806+0000 7f5b5866f740 1 --2- 172.21.15.133:0/3667660851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 0x556c7a31b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.806+0000 7f5b5866f740 1 --2- 172.21.15.133:0/3667660851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a310cc0 0x556c7a3110a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.806+0000 7f5b5866f740 1 -- 172.21.15.133:0/3667660851 >> 172.21.15.133:0/3667660851 conn(0x556c7a30d230 msgr2=0x556c7a30d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.808+0000 7f5b5866f740 1 -- 172.21.15.133:0/3667660851 shutdown_connections 2024-09-11T23:43:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.808+0000 7f5b5866f740 1 -- 172.21.15.133:0/3667660851 wait complete. 2024-09-11T23:43:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.809+0000 7f5b5866f740 1 -- start start 2024-09-11T23:43:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.809+0000 7f5b5866f740 1 Processor -- start 2024-09-11T23:43:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.810+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 0x556c7a318c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.810+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a3115e0 0x556c7a3191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.810+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 0x556c7a3196f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.810+0000 7f5b5866f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556c7a31eaa0 con 0x556c7a31be80 2024-09-11T23:43:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.810+0000 7f5b5866f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556c7a1fe290 con 0x556c7a3115e0 2024-09-11T23:43:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.810+0000 7f5b5866f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556c7a31ec20 con 0x556c7a310cc0 2024-09-11T23:43:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.810+0000 7f5b5866f740 10 client.0 _ll_register_callbacks cb 0x556c7a22f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.810+0000 7f5b4e7fc640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b380a2f00 0x7f5b380a53c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006fd0 tx=0x7f5b44014660 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.811+0000 7f5b4effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 0x556c7a318c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.811+0000 7f5b4effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 0x556c7a318c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:55434/0 (socket says 172.21.15.133:55434) 2024-09-11T23:43:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.811+0000 7f5b4effd640 1 -- 172.21.15.133:0/1621669675 learned_addr learned my addr 172.21.15.133:0/1621669675 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.812+0000 7f5b4e7fc640 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a3115e0 0x556c7a3191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.813+0000 7f5b4e7fc640 1 -- 172.21.15.133:0/1621669675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 msgr2=0x556c7a318c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.813+0000 7f5b4e7fc640 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 0x556c7a318c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.813+0000 7f5b4e7fc640 1 -- 172.21.15.133:0/1621669675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 msgr2=0x556c7a3196f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.813+0000 7f5b4e7fc640 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 0x556c7a3196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.813+0000 7f5b4e7fc640 1 -- 172.21.15.133:0/1621669675 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c79f88210 con 0x556c7a3115e0 2024-09-11T23:43:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.814+0000 7f5b4e7fc640 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a3115e0 0x556c7a3191b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b5006f490 tx=0x7f5b440046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d84a02f2684b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.816+0000 7f5b2effd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.816+0000 7f5b2effd640 1 -- 172.21.15.133:0/1621669675 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44028060 con 0x556c7a3115e0 2024-09-11T23:43:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.816+0000 7f5b5866f740 1 -- 172.21.15.133:0/1621669675 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b400a5300 con 0x556c7a3115e0 2024-09-11T23:43:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.817+0000 7f5b2effd640 1 -- 172.21.15.133:0/1621669675 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f5b44017d00 con 0x556c7a3115e0 2024-09-11T23:43:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.817+0000 7f5b2effd640 1 -- 172.21.15.133:0/1621669675 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b4401f040 con 0x556c7a3115e0 2024-09-11T23:43:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.817+0000 7f5b5866f740 10 client.26744 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.817+0000 7f5b5866f740 1 -- 172.21.15.133:0/1621669675 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c7a054620 con 0x556c7a3115e0 2024-09-11T23:43:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.817+0000 7f5b2effd640 1 -- 172.21.15.133:0/1621669675 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b400a5300 con 0x556c7a3115e0 2024-09-11T23:43:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.817+0000 7f5b2effd640 1 -- 172.21.15.133:0/1621669675 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5b44023420 con 0x556c7a3115e0 2024-09-11T23:43:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.818+0000 7f5b5866f740 10 client.26744 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.818+0000 7f5b5866f740 10 client.26744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.818+0000 7f5b5866f740 20 client.26744 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.818+0000 7f5b5866f740 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c7a32c2f0 0x556c7a32e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.818+0000 7f5b5866f740 4 client.26744 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.818+0000 7f5b5866f740 1 -- 172.21.15.133:0/1621669675 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c7a282a20 con 0x556c7a32c2f0 2024-09-11T23:43:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.820+0000 7f5b4effd640 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c7a32c2f0 0x556c7a32e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.836+0000 7f5b4effd640 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c7a32c2f0 0x556c7a32e790 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6922ee7b4424dcf server_cookie=860bf256dc7bb96 in_seq=0 out_seq=0 2024-09-11T23:43:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.839+0000 7f5b2effd640 10 client.26744 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.839+0000 7f5b2effd640 1 -- 172.21.15.133:0/1621669675 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c7a282a20 con 0x556c7a32c2f0 2024-09-11T23:43:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.839+0000 7f5b2effd640 10 client.26744 handle_command_reply: tid=0 2024-09-11T23:43:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.840+0000 7f5b5866f740 1 client.26744 shutdown 2024-09-11T23:43:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.840+0000 7f5b5866f740 1 -- 172.21.15.133:0/1621669675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a3115e0 msgr2=0x556c7a3191b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.840+0000 7f5b5866f740 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a3115e0 0x556c7a3191b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b5006f490 tx=0x7f5b440046e0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.840+0000 7f5b5866f740 1 -- 172.21.15.133:0/1621669675 shutdown_connections 2024-09-11T23:43:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.840+0000 7f5b5866f740 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c7a32c2f0 0x556c7a32e790 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.840+0000 7f5b5866f740 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 0x556c7a3196f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.840+0000 7f5b5866f740 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a3115e0 0x556c7a3191b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.840+0000 7f5b5866f740 1 --2- 172.21.15.133:0/1621669675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 0x556c7a318c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.840+0000 7f5b5866f740 1 -- 172.21.15.133:0/1621669675 >> 172.21.15.133:0/1621669675 conn(0x556c7a30d230 msgr2=0x556c7a312f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.840+0000 7f5b2effd640 0 client.26744 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.841+0000 7f5b5866f740 1 -- 172.21.15.133:0/1621669675 shutdown_connections 2024-09-11T23:43:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.841+0000 7f5b5866f740 1 -- 172.21.15.133:0/1621669675 wait complete. 2024-09-11T23:43:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.841+0000 7f5b5866f740 20 client.26744 trim_cache size 0 max 16384 2024-09-11T23:43:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.914+0000 7f5b5866f740 1 Processor -- start 2024-09-11T23:43:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.918+0000 7f5b5866f740 1 -- start start 2024-09-11T23:43:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.918+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 0x556c7a319520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.918+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a3115e0 0x556c7a334060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.918+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 0x556c7a3345a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.919+0000 7f5b5866f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c00e790 con 0x556c7a31be80 2024-09-11T23:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.919+0000 7f5b5866f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c0102d0 con 0x556c7a3115e0 2024-09-11T23:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.919+0000 7f5b5866f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c00ee60 con 0x556c7a310cc0 2024-09-11T23:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.921+0000 7f5b4e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a3115e0 0x556c7a334060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.921+0000 7f5b4f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 0x556c7a3345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.921+0000 7f5b4f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 0x556c7a3345a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50094/0 (socket says 172.21.15.133:50094) 2024-09-11T23:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.921+0000 7f5b4f7fe640 1 -- 172.21.15.133:0/2159238661 learned_addr learned my addr 172.21.15.133:0/2159238661 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.921+0000 7f5b4effd640 1 --2- 172.21.15.133:0/2159238661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 0x556c7a319520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.930+0000 7f5b4effd640 1 -- 172.21.15.133:0/2159238661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a3115e0 msgr2=0x556c7a334060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.930+0000 7f5b4effd640 1 --2- 172.21.15.133:0/2159238661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a3115e0 0x556c7a334060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.930+0000 7f5b4effd640 1 -- 172.21.15.133:0/2159238661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 msgr2=0x556c7a3345a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.930+0000 7f5b4effd640 1 --2- 172.21.15.133:0/2159238661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 0x556c7a3345a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.930+0000 7f5b4effd640 1 -- 172.21.15.133:0/2159238661 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7a311e60 con 0x556c7a310cc0 2024-09-11T23:43:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.931+0000 7f5b4effd640 1 --2- 172.21.15.133:0/2159238661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 0x556c7a319520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000fad0 tx=0x7f5b4009ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d780ee7145535b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.941+0000 7f5b357fa640 1 -- 172.21.15.133:0/2159238661 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b400b4040 con 0x556c7a310cc0 2024-09-11T23:43:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.941+0000 7f5b357fa640 1 -- 172.21.15.133:0/2159238661 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b40098d00 con 0x556c7a310cc0 2024-09-11T23:43:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.941+0000 7f5b357fa640 1 -- 172.21.15.133:0/2159238661 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40098640 con 0x556c7a310cc0 2024-09-11T23:43:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.943+0000 7f5b5866f740 1 -- 172.21.15.133:0/2159238661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 msgr2=0x556c7a319520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.943+0000 7f5b5866f740 1 --2- 172.21.15.133:0/2159238661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 0x556c7a319520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000fad0 tx=0x7f5b4009ce50 comp rx=0 tx=0).stop 2024-09-11T23:43:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.957+0000 7f5b5866f740 1 -- 172.21.15.133:0/2159238661 shutdown_connections 2024-09-11T23:43:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.957+0000 7f5b5866f740 1 --2- 172.21.15.133:0/2159238661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 0x556c7a3345a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.957+0000 7f5b5866f740 1 --2- 172.21.15.133:0/2159238661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a3115e0 0x556c7a334060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.957+0000 7f5b5866f740 1 --2- 172.21.15.133:0/2159238661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a310cc0 0x556c7a319520 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.957+0000 7f5b5866f740 1 -- 172.21.15.133:0/2159238661 >> 172.21.15.133:0/2159238661 conn(0x556c7a30d230 msgr2=0x556c7a317d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.965+0000 7f5b5866f740 1 -- 172.21.15.133:0/2159238661 shutdown_connections 2024-09-11T23:43:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.965+0000 7f5b5866f740 1 -- 172.21.15.133:0/2159238661 wait complete. 2024-09-11T23:43:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.966+0000 7f5b5866f740 1 -- start start 2024-09-11T23:43:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.966+0000 7f5b5866f740 1 Processor -- start 2024-09-11T23:43:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.976+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a310cc0 0x556c7a315290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.977+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 0x556c7a3157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.977+0000 7f5b5866f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 0x556c7a315d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.977+0000 7f5b5866f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c00fc00 con 0x556c7a31be80 2024-09-11T23:43:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.977+0000 7f5b5866f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c00f530 con 0x556c7a310cc0 2024-09-11T23:43:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.977+0000 7f5b4e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 0x556c7a3157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.977+0000 7f5b4e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 0x556c7a3157d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:55456/0 (socket says 172.21.15.133:55456) 2024-09-11T23:43:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.977+0000 7f5b4e7fc640 1 -- 172.21.15.133:0/4254362823 learned_addr learned my addr 172.21.15.133:0/4254362823 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.978+0000 7f5b5866f740 1 -- 172.21.15.133:0/4254362823 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c00c790 con 0x556c7a3115e0 2024-09-11T23:43:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.981+0000 7f5b4effd640 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a310cc0 0x556c7a315290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.982+0000 7f5b5866f740 10 client.0 _ll_register_callbacks cb 0x7f5b4401c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.982+0000 7f5b4e7fc640 1 -- 172.21.15.133:0/4254362823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a310cc0 msgr2=0x556c7a315290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.982+0000 7f5b4e7fc640 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a310cc0 0x556c7a315290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.982+0000 7f5b4e7fc640 1 -- 172.21.15.133:0/4254362823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 msgr2=0x556c7a315d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.982+0000 7f5b4e7fc640 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 0x556c7a315d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.982+0000 7f5b4e7fc640 1 -- 172.21.15.133:0/4254362823 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c7a323210 con 0x556c7a3115e0 2024-09-11T23:43:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.987+0000 7f5b4e7fc640 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 0x556c7a3157d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x556c7a31a2b0 tx=0x7f5b44023c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=408bd83ab813b937 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.987+0000 7f5b2effd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:43:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.990+0000 7f5b2effd640 1 -- 172.21.15.133:0/4254362823 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44028040 con 0x556c7a3115e0 2024-09-11T23:43:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.991+0000 7f5b5866f740 1 -- 172.21.15.133:0/4254362823 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b40098d00 con 0x556c7a3115e0 2024-09-11T23:43:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.991+0000 7f5b2effd640 1 -- 172.21.15.133:0/4254362823 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f5b44004020 con 0x556c7a3115e0 2024-09-11T23:43:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.996+0000 7f5b2effd640 1 -- 172.21.15.133:0/4254362823 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b44009880 con 0x556c7a3115e0 2024-09-11T23:43:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.996+0000 7f5b5866f740 10 client.26638 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:37.996+0000 7f5b5866f740 1 -- 172.21.15.133:0/4254362823 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c7a3244a0 con 0x556c7a3115e0 2024-09-11T23:43:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:38.000+0000 7f5b2effd640 1 -- 172.21.15.133:0/4254362823 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b40098d00 con 0x556c7a3115e0 2024-09-11T23:43:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:38.000+0000 7f5b2effd640 1 -- 172.21.15.133:0/4254362823 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5b44017ce0 con 0x556c7a3115e0 2024-09-11T23:43:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:38.000+0000 7f5b5866f740 10 client.26638 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:38.000+0000 7f5b5866f740 10 client.26638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:38.000+0000 7f5b5866f740 20 client.26638 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:38.000+0000 7f5b5866f740 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c7a397030 0x556c7a399420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:38.000+0000 7f5b5866f740 4 client.26638 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:43:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:38.000+0000 7f5b5866f740 1 -- 172.21.15.133:0/4254362823 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c7a27cd70 con 0x556c7a397030 2024-09-11T23:43:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:38.005+0000 7f5b4effd640 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c7a397030 0x556c7a399420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:38.010+0000 7f5b4effd640 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c7a397030 0x556c7a399420 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c40dadf1ac06436 server_cookie=dc81e1dd34231534 in_seq=0 out_seq=0 2024-09-11T23:43:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:38.015+0000 7f5b2effd640 10 client.26638 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:38.230 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:38.713 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:38 smithi133 ceph-mon[31430]: pgmap v211: 97 pgs: 97 active+clean; 58 GiB data, 180 GiB used, 893 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 663 MiB/s wr, 2.45k op/s 2024-09-11T23:43:38.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:38 smithi138 ceph-mon[35142]: pgmap v211: 97 pgs: 97 active+clean; 58 GiB data, 180 GiB used, 893 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 663 MiB/s wr, 2.45k op/s 2024-09-11T23:43:39.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:38 smithi178 ceph-mon[35882]: pgmap v211: 97 pgs: 97 active+clean; 58 GiB data, 180 GiB used, 893 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 663 MiB/s wr, 2.45k op/s 2024-09-11T23:43:39.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.670+0000 7fed94769640 1 -- 172.21.15.133:0/841218957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed8c112a30 msgr2=0x7fed8c114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:39.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.670+0000 7fed94769640 1 --2- 172.21.15.133:0/841218957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed8c112a30 0x7fed8c114e20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fed8401acb0 tx=0x7fed8403b920 comp rx=0 tx=0).stop 2024-09-11T23:43:39.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.670+0000 7fed94769640 1 -- 172.21.15.133:0/841218957 shutdown_connections 2024-09-11T23:43:39.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.670+0000 7fed94769640 1 --2- 172.21.15.133:0/841218957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed8c112a30 0x7fed8c114e20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.670+0000 7fed94769640 1 --2- 172.21.15.133:0/841218957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed8c077040 0x7fed8c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.670+0000 7fed94769640 1 --2- 172.21.15.133:0/841218957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed8c076720 0x7fed8c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.670+0000 7fed94769640 1 -- 172.21.15.133:0/841218957 >> 172.21.15.133:0/841218957 conn(0x7fed8c0ffab0 msgr2=0x7fed8c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:39.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.670+0000 7fed94769640 1 -- 172.21.15.133:0/841218957 shutdown_connections 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.670+0000 7fed94769640 1 -- 172.21.15.133:0/841218957 wait complete. 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.672+0000 7fed94769640 1 Processor -- start 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.672+0000 7fed94769640 1 -- start start 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.672+0000 7fed94769640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed8c076720 0x7fed8c10cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.672+0000 7fed94769640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed8c077040 0x7fed8c10d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.672+0000 7fed94769640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed8c108080 0x7fed8c114c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.672+0000 7fed94769640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1066e0 con 0x7fed8c077040 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.672+0000 7fed94769640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c106560 con 0x7fed8c108080 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.672+0000 7fed94769640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c106860 con 0x7fed8c076720 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.673+0000 7fed92505640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed8c076720 0x7fed8c10cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.673+0000 7fed92505640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed8c076720 0x7fed8c10cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:55474/0 (socket says 172.21.15.133:55474) 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.673+0000 7fed92505640 1 -- 172.21.15.133:0/1565292807 learned_addr learned my addr 172.21.15.133:0/1565292807 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:39.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.673+0000 7fed92d06640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed8c108080 0x7fed8c114c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.673+0000 7fed91d04640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed8c077040 0x7fed8c10d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.674+0000 7fed92d06640 1 -- 172.21.15.133:0/1565292807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed8c076720 msgr2=0x7fed8c10cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.674+0000 7fed92d06640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed8c076720 0x7fed8c10cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.674+0000 7fed92d06640 1 -- 172.21.15.133:0/1565292807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed8c077040 msgr2=0x7fed8c10d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.674+0000 7fed92d06640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed8c077040 0x7fed8c10d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.674+0000 7fed92d06640 1 -- 172.21.15.133:0/1565292807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c108700 con 0x7fed8c108080 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.674+0000 7fed92d06640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed8c108080 0x7fed8c114c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed8401ac00 tx=0x7fed84040860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.674+0000 7fed837fe640 1 -- 172.21.15.133:0/1565292807 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84045070 con 0x7fed8c108080 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.675+0000 7fed94769640 1 -- 172.21.15.133:0/1565292807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c108990 con 0x7fed8c108080 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.675+0000 7fed94769640 1 -- 172.21.15.133:0/1565292807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c108e70 con 0x7fed8c108080 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.675+0000 7fed837fe640 1 -- 172.21.15.133:0/1565292807 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed84004730 con 0x7fed8c108080 2024-09-11T23:43:39.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.675+0000 7fed837fe640 1 -- 172.21.15.133:0/1565292807 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84041da0 con 0x7fed8c108080 2024-09-11T23:43:39.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.678+0000 7fed837fe640 1 -- 172.21.15.133:0/1565292807 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fed84048020 con 0x7fed8c108080 2024-09-11T23:43:39.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.681+0000 7fed837fe640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed780a2f00 0x7fed780a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.681+0000 7fed837fe640 1 -- 172.21.15.133:0/1565292807 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fed84101640 con 0x7fed8c108080 2024-09-11T23:43:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.682+0000 7fed92505640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed780a2f00 0x7fed780a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.686+0000 7fed92505640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed780a2f00 0x7fed780a53c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fed7c04d9b0 tx=0x7fed7c04d1a0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.688+0000 7fed96b42740 1 Processor -- start 2024-09-11T23:43:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.688+0000 7fed96b42740 1 -- start start 2024-09-11T23:43:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.689+0000 7fed96b42740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fdb40 0x558a6e2fc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.689+0000 7fed96b42740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e2fc5d0 0x558a6e2fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.689+0000 7fed96b42740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 0x558a6e309760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.689+0000 7fed96b42740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a6e2622f0 con 0x558a6e2fc5d0 2024-09-11T23:43:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.689+0000 7fed96b42740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a6e209cf0 con 0x558a6e2fdb40 2024-09-11T23:43:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.689+0000 7fed96b42740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558a6e25e2b0 con 0x558a6e307370 2024-09-11T23:43:39.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.689+0000 7fed92d06640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 0x558a6e309760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.689+0000 7fed92d06640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 0x558a6e309760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:55480/0 (socket says 172.21.15.133:55480) 2024-09-11T23:43:39.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.689+0000 7fed92d06640 1 -- 172.21.15.133:0/2978764158 learned_addr learned my addr 172.21.15.133:0/2978764158 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:39.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.689+0000 7fed91d04640 1 --2- 172.21.15.133:0/2978764158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e2fc5d0 0x558a6e2fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.690+0000 7fed92d06640 1 -- 172.21.15.133:0/2978764158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fdb40 msgr2=0x558a6e2fc090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:39.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.690+0000 7fed92d06640 1 --2- 172.21.15.133:0/2978764158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fdb40 0x558a6e2fc090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.690+0000 7fed92d06640 1 -- 172.21.15.133:0/2978764158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e2fc5d0 msgr2=0x558a6e2fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:39.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.690+0000 7fed92d06640 1 --2- 172.21.15.133:0/2978764158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e2fc5d0 0x558a6e2fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.690+0000 7fed92d06640 1 -- 172.21.15.133:0/2978764158 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a6e1ee3a0 con 0x558a6e307370 2024-09-11T23:43:39.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.692+0000 7fed92d06640 1 --2- 172.21.15.133:0/2978764158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 0x558a6e309760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fed8401acb0 tx=0x7fed840cd680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e035f9f9ee88a6bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:39.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.693+0000 7fed80ff9640 1 -- 172.21.15.133:0/2978764158 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed840ce240 con 0x558a6e307370 2024-09-11T23:43:39.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.693+0000 7fed80ff9640 1 -- 172.21.15.133:0/2978764158 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed840cdb40 con 0x558a6e307370 2024-09-11T23:43:39.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.694+0000 7fed96b42740 1 -- 172.21.15.133:0/2978764158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 msgr2=0x558a6e309760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:39.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.696+0000 7fed96b42740 1 --2- 172.21.15.133:0/2978764158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 0x558a6e309760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fed8401acb0 tx=0x7fed840cd680 comp rx=0 tx=0).stop 2024-09-11T23:43:39.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.696+0000 7fed96b42740 1 -- 172.21.15.133:0/2978764158 shutdown_connections 2024-09-11T23:43:39.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.696+0000 7fed96b42740 1 --2- 172.21.15.133:0/2978764158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 0x558a6e309760 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.697+0000 7fed96b42740 1 --2- 172.21.15.133:0/2978764158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e2fc5d0 0x558a6e2fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.697+0000 7fed96b42740 1 --2- 172.21.15.133:0/2978764158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fdb40 0x558a6e2fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.697+0000 7fed96b42740 1 -- 172.21.15.133:0/2978764158 >> 172.21.15.133:0/2978764158 conn(0x558a6e2c5310 msgr2=0x558a6e2f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:39.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.703+0000 7fed96b42740 1 -- 172.21.15.133:0/2978764158 shutdown_connections 2024-09-11T23:43:39.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.707+0000 7fed96b42740 1 -- 172.21.15.133:0/2978764158 wait complete. 2024-09-11T23:43:39.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.707+0000 7fed96b42740 1 -- start start 2024-09-11T23:43:39.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.707+0000 7fed96b42740 1 Processor -- start 2024-09-11T23:43:39.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.709+0000 7fed96b42740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 0x558a6e306d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.709+0000 7fed96b42740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fdb40 0x558a6e303fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.709+0000 7fed96b42740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e307370 0x558a6e304570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.709+0000 7fed96b42740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a6e309e30 con 0x558a6e307370 2024-09-11T23:43:39.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.709+0000 7fed96b42740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a6e21d060 con 0x558a6e2fdb40 2024-09-11T23:43:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.709+0000 7fed96b42740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558a6e309fb0 con 0x558a6e2fc5d0 2024-09-11T23:43:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.709+0000 7fed96b42740 10 client.0 _ll_register_callbacks cb 0x558a6df708c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.710+0000 7fed92505640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 0x558a6e306d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.710+0000 7fed92505640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 0x558a6e306d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:55494/0 (socket says 172.21.15.133:55494) 2024-09-11T23:43:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.710+0000 7fed92505640 1 -- 172.21.15.133:0/79345916 learned_addr learned my addr 172.21.15.133:0/79345916 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.710+0000 7fed91d04640 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fdb40 0x558a6e303fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.713+0000 7fed91d04640 1 -- 172.21.15.133:0/79345916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 msgr2=0x558a6e306d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.713+0000 7fed91d04640 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 0x558a6e306d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.713+0000 7fed91d04640 1 -- 172.21.15.133:0/79345916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e307370 msgr2=0x558a6e304570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:39.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.713+0000 7fed91d04640 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e307370 0x558a6e304570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.713+0000 7fed91d04640 1 -- 172.21.15.133:0/79345916 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a6dfd33c0 con 0x558a6e2fdb40 2024-09-11T23:43:39.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.713+0000 7fed91d04640 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fdb40 0x558a6e303fd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fed8c109840 tx=0x7fed88012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e56684afc59ee0b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:39.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.714+0000 7fed75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:39.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.714+0000 7fed75ffb640 1 -- 172.21.15.133:0/79345916 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed880035f0 con 0x558a6e2fdb40 2024-09-11T23:43:39.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.714+0000 7fed96b42740 1 -- 172.21.15.133:0/79345916 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed840cdb40 con 0x558a6e2fdb40 2024-09-11T23:43:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.714+0000 7fed75ffb640 1 -- 172.21.15.133:0/79345916 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fed88003790 con 0x558a6e2fdb40 2024-09-11T23:43:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.714+0000 7fed75ffb640 1 -- 172.21.15.133:0/79345916 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed88003990 con 0x558a6e2fdb40 2024-09-11T23:43:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.715+0000 7fed96b42740 10 client.26759 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.715+0000 7fed96b42740 1 -- 172.21.15.133:0/79345916 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a6e315970 con 0x558a6e2fdb40 2024-09-11T23:43:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.718+0000 7fed75ffb640 1 -- 172.21.15.133:0/79345916 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed840cdb40 con 0x558a6e2fdb40 2024-09-11T23:43:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.718+0000 7fed75ffb640 1 -- 172.21.15.133:0/79345916 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed88019020 con 0x558a6e2fdb40 2024-09-11T23:43:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.718+0000 7fed96b42740 10 client.26759 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.718+0000 7fed96b42740 10 client.26759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:43:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.718+0000 7fed96b42740 20 client.26759 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:39.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.718+0000 7fed96b42740 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558a6e3172b0 0x558a6e3196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.718+0000 7fed96b42740 4 client.26759 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:39.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.718+0000 7fed96b42740 1 -- 172.21.15.133:0/79345916 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a6e26f860 con 0x558a6e3172b0 2024-09-11T23:43:39.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.719+0000 7fed92505640 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558a6e3172b0 0x558a6e3196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.719+0000 7fed75ffb640 10 client.26759 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:43:39.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.720+0000 7fed92505640 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558a6e3172b0 0x558a6e3196d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d679968a3748a2c server_cookie=7231bb02f514553f in_seq=0 out_seq=0 2024-09-11T23:43:39.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.724+0000 7fed75ffb640 1 -- 172.21.15.133:0/79345916 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a6e26f860 con 0x558a6e3172b0 2024-09-11T23:43:39.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.724+0000 7fed75ffb640 10 client.26759 handle_command_reply: tid=0 2024-09-11T23:43:39.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.724+0000 7fed96b42740 1 client.26759 shutdown 2024-09-11T23:43:39.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.725+0000 7fed96b42740 1 -- 172.21.15.133:0/79345916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fdb40 msgr2=0x558a6e303fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:39.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.725+0000 7fed96b42740 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fdb40 0x558a6e303fd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fed8c109840 tx=0x7fed88012820 comp rx=0 tx=0).stop 2024-09-11T23:43:39.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.725+0000 7fed96b42740 1 -- 172.21.15.133:0/79345916 shutdown_connections 2024-09-11T23:43:39.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.725+0000 7fed96b42740 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558a6e3172b0 0x558a6e3196d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.725+0000 7fed96b42740 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e307370 0x558a6e304570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.725+0000 7fed96b42740 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fdb40 0x558a6e303fd0 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fed8c109840 tx=0x7fed88012820 comp rx=0 tx=0).stop 2024-09-11T23:43:39.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.725+0000 7fed96b42740 1 --2- 172.21.15.133:0/79345916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 0x558a6e306d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.725+0000 7fed96b42740 1 -- 172.21.15.133:0/79345916 >> 172.21.15.133:0/79345916 conn(0x558a6e2c5310 msgr2=0x558a6e2fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:39.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.725+0000 7fed75ffb640 0 client.26759 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:43:39.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.725+0000 7fed96b42740 1 -- 172.21.15.133:0/79345916 shutdown_connections 2024-09-11T23:43:39.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.726+0000 7fed96b42740 1 -- 172.21.15.133:0/79345916 wait complete. 2024-09-11T23:43:39.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.726+0000 7fed96b42740 20 client.26759 trim_cache size 0 max 16384 2024-09-11T23:43:39.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.757+0000 7fed96b42740 1 Processor -- start 2024-09-11T23:43:39.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.757+0000 7fed96b42740 1 -- start start 2024-09-11T23:43:39.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.758+0000 7fed96b42740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fc5d0 0x558a6e303fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.758+0000 7fed96b42740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 0x558a6e313130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.758+0000 7fed96b42740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e31efe0 0x558a6e31f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.758+0000 7fed92505640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fc5d0 0x558a6e303fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.758+0000 7fed92505640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fc5d0 0x558a6e303fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:39000/0 (socket says 172.21.15.133:39000) 2024-09-11T23:43:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.758+0000 7fed92505640 1 -- 172.21.15.133:0/581226362 learned_addr learned my addr 172.21.15.133:0/581226362 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.758+0000 7fed91d04640 1 --2- 172.21.15.133:0/581226362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 0x558a6e313130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.758+0000 7fed96b42740 1 -- 172.21.15.133:0/581226362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed7000e780 con 0x558a6e31efe0 2024-09-11T23:43:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.759+0000 7fed92d06640 1 --2- 172.21.15.133:0/581226362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e31efe0 0x558a6e31f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.759+0000 7fed96b42740 1 -- 172.21.15.133:0/581226362 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed700102c0 con 0x558a6e2fc5d0 2024-09-11T23:43:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.759+0000 7fed96b42740 1 -- 172.21.15.133:0/581226362 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed7000ee50 con 0x558a6e307370 2024-09-11T23:43:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.760+0000 7fed91d04640 1 -- 172.21.15.133:0/581226362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fc5d0 msgr2=0x558a6e303fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.760+0000 7fed91d04640 1 --2- 172.21.15.133:0/581226362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fc5d0 0x558a6e303fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.760+0000 7fed91d04640 1 -- 172.21.15.133:0/581226362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e31efe0 msgr2=0x558a6e31f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.760+0000 7fed91d04640 1 --2- 172.21.15.133:0/581226362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e31efe0 0x558a6e31f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.760+0000 7fed91d04640 1 -- 172.21.15.133:0/581226362 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a6e307090 con 0x558a6e307370 2024-09-11T23:43:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.761+0000 7fed91d04640 1 --2- 172.21.15.133:0/581226362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 0x558a6e313130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x558a6e305580 tx=0x7fed88004590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ac90242da43aff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:39.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.762+0000 7fed77fff640 1 -- 172.21.15.133:0/581226362 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88031070 con 0x558a6e307370 2024-09-11T23:43:39.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.762+0000 7fed77fff640 1 -- 172.21.15.133:0/581226362 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed88005510 con 0x558a6e307370 2024-09-11T23:43:39.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.762+0000 7fed77fff640 1 -- 172.21.15.133:0/581226362 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88028040 con 0x558a6e307370 2024-09-11T23:43:39.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.763+0000 7fed96b42740 1 -- 172.21.15.133:0/581226362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 msgr2=0x558a6e313130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:39.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.764+0000 7fed96b42740 1 --2- 172.21.15.133:0/581226362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 0x558a6e313130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x558a6e305580 tx=0x7fed88004590 comp rx=0 tx=0).stop 2024-09-11T23:43:39.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.764+0000 7fed96b42740 1 -- 172.21.15.133:0/581226362 shutdown_connections 2024-09-11T23:43:39.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.764+0000 7fed96b42740 1 --2- 172.21.15.133:0/581226362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e31efe0 0x558a6e31f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.764+0000 7fed96b42740 1 --2- 172.21.15.133:0/581226362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e307370 0x558a6e313130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.764+0000 7fed96b42740 1 --2- 172.21.15.133:0/581226362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e2fc5d0 0x558a6e303fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.764+0000 7fed96b42740 1 -- 172.21.15.133:0/581226362 >> 172.21.15.133:0/581226362 conn(0x558a6e2c5310 msgr2=0x558a6e303890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:39.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.772+0000 7fed96b42740 1 -- 172.21.15.133:0/581226362 shutdown_connections 2024-09-11T23:43:39.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.772+0000 7fed96b42740 1 -- 172.21.15.133:0/581226362 wait complete. 2024-09-11T23:43:39.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.773+0000 7fed96b42740 1 -- start start 2024-09-11T23:43:39.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.774+0000 7fed96b42740 1 Processor -- start 2024-09-11T23:43:39.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.775+0000 7fed96b42740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 0x558a6e30b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.775+0000 7fed96b42740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e307370 0x558a6e30b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.775+0000 7fed92505640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 0x558a6e30b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.776+0000 7fed92505640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 0x558a6e30b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:55514/0 (socket says 172.21.15.133:55514) 2024-09-11T23:43:39.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.776+0000 7fed92505640 1 -- 172.21.15.133:0/1560810700 learned_addr learned my addr 172.21.15.133:0/1560810700 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:39.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.776+0000 7fed96b42740 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e31efe0 0x558a6e30bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.777+0000 7fed96b42740 1 -- 172.21.15.133:0/1560810700 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed7000fbf0 con 0x558a6e31efe0 2024-09-11T23:43:39.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.777+0000 7fed96b42740 1 -- 172.21.15.133:0/1560810700 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed7000f520 con 0x558a6e307370 2024-09-11T23:43:39.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.777+0000 7fed96b42740 1 -- 172.21.15.133:0/1560810700 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed7000c780 con 0x558a6e2fc5d0 2024-09-11T23:43:39.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.777+0000 7fed96b42740 10 client.0 _ll_register_callbacks cb 0x7fed8800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:39.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.777+0000 7fed92d06640 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e31efe0 0x558a6e30bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.778+0000 7fed92d06640 1 -- 172.21.15.133:0/1560810700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 msgr2=0x558a6e30b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:39.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.778+0000 7fed92d06640 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 0x558a6e30b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.778+0000 7fed92d06640 1 -- 172.21.15.133:0/1560810700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e307370 msgr2=0x558a6e30b7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:39.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.778+0000 7fed92d06640 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e307370 0x558a6e30b7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.778+0000 7fed92d06640 1 -- 172.21.15.133:0/1560810700 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a6e381200 con 0x558a6e31efe0 2024-09-11T23:43:39.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.779+0000 7fed92d06640 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e31efe0 0x558a6e30bcf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x558a6e2fd070 tx=0x7fed8403c020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6215ad0f721a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:39.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.779+0000 7fed75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:39.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.782+0000 7fed75ffb640 1 -- 172.21.15.133:0/1560810700 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed840cdb40 con 0x558a6e31efe0 2024-09-11T23:43:39.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.782+0000 7fed75ffb640 1 -- 172.21.15.133:0/1560810700 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fed84041620 con 0x558a6e31efe0 2024-09-11T23:43:39.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.782+0000 7fed96b42740 1 -- 172.21.15.133:0/1560810700 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed88005510 con 0x558a6e31efe0 2024-09-11T23:43:39.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.785+0000 7fed75ffb640 1 -- 172.21.15.133:0/1560810700 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed840ce240 con 0x558a6e31efe0 2024-09-11T23:43:39.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.786+0000 7fed96b42740 10 client.17280 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:39.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.786+0000 7fed96b42740 1 -- 172.21.15.133:0/1560810700 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a6e382500 con 0x558a6e31efe0 2024-09-11T23:43:39.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.787+0000 7fed75ffb640 1 -- 172.21.15.133:0/1560810700 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed840d1ef0 con 0x558a6e31efe0 2024-09-11T23:43:39.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.787+0000 7fed75ffb640 1 -- 172.21.15.133:0/1560810700 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed8405a030 con 0x558a6e31efe0 2024-09-11T23:43:39.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.788+0000 7fed96b42740 10 client.17280 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:39.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.788+0000 7fed96b42740 10 client.17280 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:43:39.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.788+0000 7fed96b42740 20 client.17280 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:39.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.788+0000 7fed96b42740 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558a6e382790 0x558a6e384b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:39.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.788+0000 7fed96b42740 4 client.17280 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:43:39.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.788+0000 7fed96b42740 1 -- 172.21.15.133:0/1560810700 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fed88005510 con 0x558a6e382790 2024-09-11T23:43:39.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.789+0000 7fed92505640 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558a6e382790 0x558a6e384b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:39.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.791+0000 7fed75ffb640 10 client.17280 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:43:39.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.791+0000 7fed92505640 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558a6e382790 0x558a6e384b80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=325d5239500de815 server_cookie=d06e33bca4570b6 in_seq=0 out_seq=0 2024-09-11T23:43:39.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.791+0000 7fed75ffb640 1 -- 172.21.15.133:0/1560810700 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fed88005510 con 0x558a6e382790 2024-09-11T23:43:39.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.791+0000 7fed75ffb640 10 client.17280 handle_command_reply: tid=0 2024-09-11T23:43:39.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.791+0000 7fed96b42740 1 client.17280 shutdown 2024-09-11T23:43:39.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.792+0000 7fed96b42740 1 -- 172.21.15.133:0/1560810700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e31efe0 msgr2=0x558a6e30bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:39.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.792+0000 7fed96b42740 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e31efe0 0x558a6e30bcf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x558a6e2fd070 tx=0x7fed8403c020 comp rx=0 tx=0).stop 2024-09-11T23:43:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.792+0000 7fed96b42740 1 -- 172.21.15.133:0/1560810700 shutdown_connections 2024-09-11T23:43:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.792+0000 7fed96b42740 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558a6e382790 0x558a6e384b80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.792+0000 7fed75ffb640 0 client.17280 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:43:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.792+0000 7fed96b42740 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a6e31efe0 0x558a6e30bcf0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.792+0000 7fed96b42740 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a6e307370 0x558a6e30b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.792+0000 7fed96b42740 1 --2- 172.21.15.133:0/1560810700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a6e2fc5d0 0x558a6e30b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.792+0000 7fed96b42740 1 -- 172.21.15.133:0/1560810700 >> 172.21.15.133:0/1560810700 conn(0x558a6e2c5310 msgr2=0x558a6e302460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.793+0000 7fed96b42740 1 -- 172.21.15.133:0/1560810700 shutdown_connections 2024-09-11T23:43:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.793+0000 7fed96b42740 1 -- 172.21.15.133:0/1560810700 wait complete. 2024-09-11T23:43:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.793+0000 7fed96b42740 20 client.17280 trim_cache size 0 max 16384 2024-09-11T23:43:39.794 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:43:39.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.795+0000 7fed94769640 1 -- 172.21.15.133:0/1565292807 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed780a2f00 msgr2=0x7fed780a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:39.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.795+0000 7fed94769640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed780a2f00 0x7fed780a53c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fed7c04d9b0 tx=0x7fed7c04d1a0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.795+0000 7fed94769640 1 -- 172.21.15.133:0/1565292807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed8c108080 msgr2=0x7fed8c114c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:39.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.796+0000 7fed94769640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed8c108080 0x7fed8c114c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed8401ac00 tx=0x7fed84040860 comp rx=0 tx=0).stop 2024-09-11T23:43:39.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.796+0000 7fed94769640 1 -- 172.21.15.133:0/1565292807 shutdown_connections 2024-09-11T23:43:39.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.796+0000 7fed94769640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed780a2f00 0x7fed780a53c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.796+0000 7fed94769640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed8c108080 0x7fed8c114c90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.796+0000 7fed94769640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed8c077040 0x7fed8c10d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.796+0000 7fed94769640 1 --2- 172.21.15.133:0/1565292807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed8c076720 0x7fed8c10cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:39.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.796+0000 7fed94769640 1 -- 172.21.15.133:0/1565292807 >> 172.21.15.133:0/1565292807 conn(0x7fed8c0ffab0 msgr2=0x7fed8c101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:39.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.797+0000 7fed94769640 1 -- 172.21.15.133:0/1565292807 shutdown_connections 2024-09-11T23:43:39.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:39.797+0000 7fed94769640 1 -- 172.21.15.133:0/1565292807 wait complete. 2024-09-11T23:43:39.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:39 smithi138 ceph-mon[35142]: pgmap v212: 97 pgs: 97 active+clean; 59 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 883 MiB/s wr, 3.15k op/s 2024-09-11T23:43:39.962 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:39 smithi133 ceph-mon[31430]: pgmap v212: 97 pgs: 97 active+clean; 59 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 883 MiB/s wr, 3.15k op/s 2024-09-11T23:43:40.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:39 smithi178 ceph-mon[35882]: pgmap v212: 97 pgs: 97 active+clean; 59 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 883 MiB/s wr, 3.15k op/s 2024-09-11T23:43:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.334+0000 7f5b2effd640 1 -- 172.21.15.133:0/4254362823 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c7a27cd70 con 0x556c7a397030 2024-09-11T23:43:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.334+0000 7f5b2effd640 10 client.26638 handle_command_reply: tid=0 2024-09-11T23:43:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.334+0000 7f5b5866f740 1 client.26638 shutdown 2024-09-11T23:43:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.336+0000 7f5b5866f740 1 -- 172.21.15.133:0/4254362823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 msgr2=0x556c7a3157d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.336+0000 7f5b5866f740 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 0x556c7a3157d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x556c7a31a2b0 tx=0x7f5b44023c90 comp rx=0 tx=0).stop 2024-09-11T23:43:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.336+0000 7f5b5866f740 1 -- 172.21.15.133:0/4254362823 shutdown_connections 2024-09-11T23:43:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.336+0000 7f5b5866f740 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556c7a397030 0x556c7a399420 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.336+0000 7f5b5866f740 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556c7a31be80 0x556c7a315d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.336+0000 7f5b5866f740 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c7a3115e0 0x556c7a3157d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.336+0000 7f5b5866f740 1 --2- 172.21.15.133:0/4254362823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c7a310cc0 0x556c7a315290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.336+0000 7f5b5866f740 1 -- 172.21.15.133:0/4254362823 >> 172.21.15.133:0/4254362823 conn(0x556c7a30d230 msgr2=0x556c7a30ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.336+0000 7f5b2effd640 0 client.26638 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.340+0000 7f5b5866f740 1 -- 172.21.15.133:0/4254362823 shutdown_connections 2024-09-11T23:43:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.340+0000 7f5b5866f740 1 -- 172.21.15.133:0/4254362823 wait complete. 2024-09-11T23:43:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.340+0000 7f5b5866f740 20 client.26638 trim_cache size 0 max 16384 2024-09-11T23:43:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:43:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:43:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:43:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:43:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.343+0000 7f5b558d9640 1 -- 172.21.15.133:0/3116445415 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b380a2f00 msgr2=0x7f5b380a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.343+0000 7f5b558d9640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b380a2f00 0x7f5b380a53c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006fd0 tx=0x7f5b44014660 comp rx=0 tx=0).stop 2024-09-11T23:43:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.343+0000 7f5b558d9640 1 -- 172.21.15.133:0/3116445415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50100180 msgr2=0x7f5b50073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.343+0000 7f5b558d9640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50100180 0x7f5b50073bd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000fb00 tx=0x7f5b40009e70 comp rx=0 tx=0).stop 2024-09-11T23:43:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.344+0000 7f5b558d9640 1 -- 172.21.15.133:0/3116445415 shutdown_connections 2024-09-11T23:43:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.344+0000 7f5b558d9640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5b380a2f00 0x7f5b380a53c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.344+0000 7f5b558d9640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b501060d0 0x7f5b5006e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.344+0000 7f5b558d9640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b50100b20 0x7f5b5006dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.344+0000 7f5b558d9640 1 --2- 172.21.15.133:0/3116445415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50100180 0x7f5b50073bd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.344+0000 7f5b558d9640 1 -- 172.21.15.133:0/3116445415 >> 172.21.15.133:0/3116445415 conn(0x7f5b50077200 msgr2=0x7f5b500fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.345+0000 7f5b558d9640 1 -- 172.21.15.133:0/3116445415 shutdown_connections 2024-09-11T23:43:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:40.345+0000 7f5b558d9640 1 -- 172.21.15.133:0/3116445415 wait complete. 2024-09-11T23:43:41.031 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:43:41.962 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:42.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:41 smithi178 ceph-mon[35882]: pgmap v213: 97 pgs: 97 active+clean; 59 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 711 MiB/s wr, 2.63k op/s 2024-09-11T23:43:42.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:41 smithi133 ceph-mon[31430]: pgmap v213: 97 pgs: 97 active+clean; 59 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 711 MiB/s wr, 2.63k op/s 2024-09-11T23:43:42.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:41 smithi138 ceph-mon[35142]: pgmap v213: 97 pgs: 97 active+clean; 59 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 711 MiB/s wr, 2.63k op/s 2024-09-11T23:43:42.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.981+0000 7f2db0de4640 1 -- 172.21.15.133:0/831435413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dac076720 msgr2=0x7f2dac076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:42.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.981+0000 7f2db0de4640 1 --2- 172.21.15.133:0/831435413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dac076720 0x7f2dac076b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009be0 tx=0x7f2da402a6f0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.982+0000 7f2db0de4640 1 -- 172.21.15.133:0/831435413 shutdown_connections 2024-09-11T23:43:42.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.982+0000 7f2db0de4640 1 --2- 172.21.15.133:0/831435413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dac112a30 0x7f2dac114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.982+0000 7f2db0de4640 1 --2- 172.21.15.133:0/831435413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac077040 0x7f2dac1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.982+0000 7f2db0de4640 1 --2- 172.21.15.133:0/831435413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dac076720 0x7f2dac076b00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.982+0000 7f2db0de4640 1 -- 172.21.15.133:0/831435413 >> 172.21.15.133:0/831435413 conn(0x7f2dac0ffa70 msgr2=0x7f2dac101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:42.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.982+0000 7f2db0de4640 1 -- 172.21.15.133:0/831435413 shutdown_connections 2024-09-11T23:43:42.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.982+0000 7f2db0de4640 1 -- 172.21.15.133:0/831435413 wait complete. 2024-09-11T23:43:42.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.983+0000 7f2db0de4640 1 Processor -- start 2024-09-11T23:43:42.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.984+0000 7f2db0de4640 1 -- start start 2024-09-11T23:43:42.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.984+0000 7f2db0de4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac076720 0x7f2dac0733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:42.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.984+0000 7f2db0de4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dac077040 0x7f2dac073930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:42.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.984+0000 7f2db0de4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dac112a30 0x7f2dac1073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:42.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.984+0000 7f2db0de4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac1185f0 con 0x7f2dac076720 2024-09-11T23:43:42.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.984+0000 7f2db0de4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac118470 con 0x7f2dac077040 2024-09-11T23:43:42.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.984+0000 7f2db0de4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac118770 con 0x7f2dac112a30 2024-09-11T23:43:42.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.984+0000 7f2da9d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dac077040 0x7f2dac073930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:42.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.986+0000 7f2daa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac076720 0x7f2dac0733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:42.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.986+0000 7f2daa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac076720 0x7f2dac0733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56182/0 (socket says 172.21.15.133:56182) 2024-09-11T23:43:42.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.986+0000 7f2daa59c640 1 -- 172.21.15.133:0/3367624371 learned_addr learned my addr 172.21.15.133:0/3367624371 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.986+0000 7f2daad9d640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dac112a30 0x7f2dac1073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.986+0000 7f2da9d9b640 1 -- 172.21.15.133:0/3367624371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dac112a30 msgr2=0x7f2dac1073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.986+0000 7f2da9d9b640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dac112a30 0x7f2dac1073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.987+0000 7f2da9d9b640 1 -- 172.21.15.133:0/3367624371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac076720 msgr2=0x7f2dac0733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.987+0000 7f2da9d9b640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac076720 0x7f2dac0733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.987+0000 7f2da9d9b640 1 -- 172.21.15.133:0/3367624371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac107a80 con 0x7f2dac077040 2024-09-11T23:43:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.987+0000 7f2daa59c640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac076720 0x7f2dac0733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.987+0000 7f2daad9d640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dac112a30 0x7f2dac1073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:43:42.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.989+0000 7f2da9d9b640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dac077040 0x7f2dac073930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00e790 tx=0x7f2d9c00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:42.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.990+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/3367624371 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00eea0 con 0x7f2dac077040 2024-09-11T23:43:42.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.990+0000 7f2db0de4640 1 -- 172.21.15.133:0/3367624371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dac107d70 con 0x7f2dac077040 2024-09-11T23:43:42.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.990+0000 7f2db0de4640 1 -- 172.21.15.133:0/3367624371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dac10f690 con 0x7f2dac077040 2024-09-11T23:43:42.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.994+0000 7f2db31bd740 1 Processor -- start 2024-09-11T23:43:42.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.995+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/3367624371 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d9c00ccc0 con 0x7f2dac077040 2024-09-11T23:43:42.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:42.995+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/3367624371 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c018780 con 0x7f2dac077040 2024-09-11T23:43:43.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.014+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/3367624371 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2d9c010430 con 0x7f2dac077040 2024-09-11T23:43:43.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.015+0000 7f2db31bd740 1 -- start start 2024-09-11T23:43:43.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.015+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 0x561586624090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.015+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615866245d0 0x561586624a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.015+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56158662f370 0x561586631760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.015+0000 7f2db31bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56158658a2f0 con 0x56158662f370 2024-09-11T23:43:43.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.015+0000 7f2db31bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561586531cf0 con 0x561586625b40 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.015+0000 7f2db31bd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5615865862b0 con 0x5615866245d0 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.016+0000 7f2daa59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615866245d0 0x561586624a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.016+0000 7f2daa59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615866245d0 0x561586624a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42814/0 (socket says 172.21.15.133:42814) 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.017+0000 7f2daa59c640 1 -- 172.21.15.133:0/626956590 learned_addr learned my addr 172.21.15.133:0/626956590 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.017+0000 7f2d9b7fe640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d940a2dc0 0x7f2d940a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.018+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/3367624371 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f2d9c0cf660 con 0x7f2dac077040 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.018+0000 7f2daad9d640 1 --2- 172.21.15.133:0/626956590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 0x561586624090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.018+0000 7f2daa59c640 1 -- 172.21.15.133:0/626956590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 msgr2=0x561586624090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.018+0000 7f2daa59c640 1 --2- 172.21.15.133:0/626956590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 0x561586624090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.018+0000 7f2daa59c640 1 -- 172.21.15.133:0/626956590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56158662f370 msgr2=0x561586631760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.018+0000 7f2daa59c640 1 --2- 172.21.15.133:0/626956590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56158662f370 0x561586631760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.018+0000 7f2daa59c640 1 -- 172.21.15.133:0/626956590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615865163a0 con 0x5615866245d0 2024-09-11T23:43:43.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.019+0000 7f2daad9d640 1 --2- 172.21.15.133:0/626956590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 0x561586624090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:43:43.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.019+0000 7f2daad9d640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d940a2dc0 0x7f2d940a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:43.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.019+0000 7f2daa59c640 1 --2- 172.21.15.133:0/626956590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615866245d0 0x561586624a10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2da402ac60 tx=0x7f2da402f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1da613a48e0f7cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:43.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.020+0000 7f2d98ff9640 1 -- 172.21.15.133:0/626956590 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da4040030 con 0x5615866245d0 2024-09-11T23:43:43.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.020+0000 7f2d98ff9640 1 -- 172.21.15.133:0/626956590 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da4003970 con 0x5615866245d0 2024-09-11T23:43:43.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.021+0000 7f2db31bd740 1 -- 172.21.15.133:0/626956590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615866245d0 msgr2=0x561586624a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:43.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.021+0000 7f2d98ff9640 1 -- 172.21.15.133:0/626956590 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da4040030 con 0x5615866245d0 2024-09-11T23:43:43.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.021+0000 7f2db31bd740 1 --2- 172.21.15.133:0/626956590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615866245d0 0x561586624a10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2da402ac60 tx=0x7f2da402f410 comp rx=0 tx=0).stop 2024-09-11T23:43:43.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.025+0000 7f2daad9d640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d940a2dc0 0x7f2d940a5280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2da00058c0 tx=0x7f2da0005850 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:43.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.025+0000 7f2db31bd740 1 -- 172.21.15.133:0/626956590 shutdown_connections 2024-09-11T23:43:43.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.025+0000 7f2db31bd740 1 --2- 172.21.15.133:0/626956590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56158662f370 0x561586631760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.025+0000 7f2db31bd740 1 --2- 172.21.15.133:0/626956590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615866245d0 0x561586624a10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.025+0000 7f2db31bd740 1 --2- 172.21.15.133:0/626956590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 0x561586624090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.025+0000 7f2db31bd740 1 -- 172.21.15.133:0/626956590 >> 172.21.15.133:0/626956590 conn(0x5615865ed310 msgr2=0x561586620c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:43.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.025+0000 7f2db31bd740 1 -- 172.21.15.133:0/626956590 shutdown_connections 2024-09-11T23:43:43.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.025+0000 7f2db31bd740 1 -- 172.21.15.133:0/626956590 wait complete. 2024-09-11T23:43:43.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.026+0000 7f2db31bd740 1 -- start start 2024-09-11T23:43:43.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.026+0000 7f2db31bd740 1 Processor -- start 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.027+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615866245d0 0x56158662eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.027+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561586625b40 0x56158662bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.027+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56158662f370 0x56158662c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.027+0000 7f2db31bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561586632060 con 0x561586625b40 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.027+0000 7f2db31bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561586545060 con 0x5615866245d0 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.027+0000 7f2db31bd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5615866321e0 con 0x56158662f370 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.027+0000 7f2db31bd740 10 client.0 _ll_register_callbacks cb 0x5615862988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.028+0000 7f2daa59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615866245d0 0x56158662eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.028+0000 7f2daa59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615866245d0 0x56158662eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58338/0 (socket says 172.21.15.133:58338) 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.028+0000 7f2daa59c640 1 -- 172.21.15.133:0/1456077027 learned_addr learned my addr 172.21.15.133:0/1456077027 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.028+0000 7f2daad9d640 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56158662f370 0x56158662c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.029+0000 7f2daa59c640 1 -- 172.21.15.133:0/1456077027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56158662f370 msgr2=0x56158662c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.029+0000 7f2daa59c640 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56158662f370 0x56158662c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.029+0000 7f2daa59c640 1 -- 172.21.15.133:0/1456077027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561586625b40 msgr2=0x56158662bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:43.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.029+0000 7f2daa59c640 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561586625b40 0x56158662bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.029+0000 7f2daa59c640 1 -- 172.21.15.133:0/1456077027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615862fb3c0 con 0x5615866245d0 2024-09-11T23:43:43.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.029+0000 7f2daa59c640 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615866245d0 0x56158662eb80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2da4004af0 tx=0x7f2da4030420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cd9330b1be91a13 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:43.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.029+0000 7f2d91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:43.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.030+0000 7f2d91ffb640 1 -- 172.21.15.133:0/1456077027 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da4040040 con 0x5615866245d0 2024-09-11T23:43:43.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.030+0000 7f2db31bd740 1 -- 172.21.15.133:0/1456077027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2da4003970 con 0x5615866245d0 2024-09-11T23:43:43.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.030+0000 7f2d91ffb640 1 -- 172.21.15.133:0/1456077027 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f2da4003d20 con 0x5615866245d0 2024-09-11T23:43:43.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.031+0000 7f2d91ffb640 1 -- 172.21.15.133:0/1456077027 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2da4036040 con 0x5615866245d0 2024-09-11T23:43:43.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.031+0000 7f2db31bd740 10 client.26771 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:43.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.031+0000 7f2db31bd740 1 -- 172.21.15.133:0/1456077027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56158663da10 con 0x5615866245d0 2024-09-11T23:43:43.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.031+0000 7f2d91ffb640 1 -- 172.21.15.133:0/1456077027 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da4003970 con 0x5615866245d0 2024-09-11T23:43:43.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.032+0000 7f2d91ffb640 1 -- 172.21.15.133:0/1456077027 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2da402f600 con 0x5615866245d0 2024-09-11T23:43:43.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.032+0000 7f2db31bd740 10 client.26771 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:43.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.032+0000 7f2db31bd740 10 client.26771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:43:43.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.032+0000 7f2db31bd740 20 client.26771 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:43.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.032+0000 7f2db31bd740 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56158663ef60 0x5615866413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.032+0000 7f2db31bd740 4 client.26771 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:43.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.032+0000 7f2db31bd740 1 -- 172.21.15.133:0/1456077027 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561586597860 con 0x56158663ef60 2024-09-11T23:43:43.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.032+0000 7f2da9d9b640 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56158663ef60 0x5615866413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:43.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.033+0000 7f2da9d9b640 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56158663ef60 0x5615866413e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bdb3ff62f1ba4b65 server_cookie=1a6fa39644fba9c6 in_seq=0 out_seq=0 2024-09-11T23:43:43.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.034+0000 7f2d91ffb640 10 client.26771 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:43:43.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.034+0000 7f2d91ffb640 1 -- 172.21.15.133:0/1456077027 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561586597860 con 0x56158663ef60 2024-09-11T23:43:43.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.034+0000 7f2d91ffb640 10 client.26771 handle_command_reply: tid=0 2024-09-11T23:43:43.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.034+0000 7f2db31bd740 1 client.26771 shutdown 2024-09-11T23:43:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.038+0000 7f2db31bd740 1 -- 172.21.15.133:0/1456077027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615866245d0 msgr2=0x56158662eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.038+0000 7f2db31bd740 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615866245d0 0x56158662eb80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2da4004af0 tx=0x7f2da4030420 comp rx=0 tx=0).stop 2024-09-11T23:43:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.038+0000 7f2db31bd740 1 -- 172.21.15.133:0/1456077027 shutdown_connections 2024-09-11T23:43:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.038+0000 7f2db31bd740 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x56158663ef60 0x5615866413e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.039+0000 7f2db31bd740 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56158662f370 0x56158662c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.039+0000 7f2db31bd740 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561586625b40 0x56158662bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.039+0000 7f2db31bd740 1 --2- 172.21.15.133:0/1456077027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615866245d0 0x56158662eb80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.039+0000 7f2db31bd740 1 -- 172.21.15.133:0/1456077027 >> 172.21.15.133:0/1456077027 conn(0x5615865ed310 msgr2=0x561586626960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:43.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.041+0000 7f2d91ffb640 0 client.26771 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:43:43.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.041+0000 7f2db31bd740 1 -- 172.21.15.133:0/1456077027 shutdown_connections 2024-09-11T23:43:43.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.041+0000 7f2db31bd740 1 -- 172.21.15.133:0/1456077027 wait complete. 2024-09-11T23:43:43.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.043+0000 7f2db31bd740 20 client.26771 trim_cache size 0 max 16384 2024-09-11T23:43:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.123+0000 7f2db31bd740 1 Processor -- start 2024-09-11T23:43:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.124+0000 7f2db31bd740 1 -- start start 2024-09-11T23:43:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.124+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615866245d0 0x561586620e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.124+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 0x561586627120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.125+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56158662f370 0x561586627660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.125+0000 7f2db31bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c00e780 con 0x5615866245d0 2024-09-11T23:43:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.125+0000 7f2db31bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c0102c0 con 0x561586625b40 2024-09-11T23:43:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.125+0000 7f2db31bd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c00ee50 con 0x56158662f370 2024-09-11T23:43:43.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.128+0000 7f2da9d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 0x561586627120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:43.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.129+0000 7f2da9d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 0x561586627120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58342/0 (socket says 172.21.15.133:58342) 2024-09-11T23:43:43.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.129+0000 7f2da9d9b640 1 -- 172.21.15.133:0/2945307705 learned_addr learned my addr 172.21.15.133:0/2945307705 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:43.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.129+0000 7f2daad9d640 1 --2- 172.21.15.133:0/2945307705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56158662f370 0x561586627660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:43.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.130+0000 7f2da9d9b640 1 -- 172.21.15.133:0/2945307705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56158662f370 msgr2=0x561586627660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:43.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.130+0000 7f2da9d9b640 1 --2- 172.21.15.133:0/2945307705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56158662f370 0x561586627660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.130+0000 7f2da9d9b640 1 -- 172.21.15.133:0/2945307705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615866245d0 msgr2=0x561586620e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:43.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.130+0000 7f2da9d9b640 1 --2- 172.21.15.133:0/2945307705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615866245d0 0x561586620e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.130+0000 7f2da9d9b640 1 -- 172.21.15.133:0/2945307705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561586625740 con 0x561586625b40 2024-09-11T23:43:43.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.131+0000 7f2da9d9b640 1 --2- 172.21.15.133:0/2945307705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 0x561586627120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c098530 tx=0x7f2d9c0cf8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a610c70096b17f2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:43.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.132+0000 7f2d93fff640 1 -- 172.21.15.133:0/2945307705 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0b3070 con 0x561586625b40 2024-09-11T23:43:43.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.132+0000 7f2d93fff640 1 -- 172.21.15.133:0/2945307705 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d9c0a47d0 con 0x561586625b40 2024-09-11T23:43:43.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.132+0000 7f2d93fff640 1 -- 172.21.15.133:0/2945307705 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c09b210 con 0x561586625b40 2024-09-11T23:43:43.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.135+0000 7f2db31bd740 1 -- 172.21.15.133:0/2945307705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 msgr2=0x561586627120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:43.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.135+0000 7f2db31bd740 1 --2- 172.21.15.133:0/2945307705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 0x561586627120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c098530 tx=0x7f2d9c0cf8e0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.136+0000 7f2db31bd740 1 -- 172.21.15.133:0/2945307705 shutdown_connections 2024-09-11T23:43:43.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.136+0000 7f2db31bd740 1 --2- 172.21.15.133:0/2945307705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56158662f370 0x561586627660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.136+0000 7f2db31bd740 1 --2- 172.21.15.133:0/2945307705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561586625b40 0x561586627120 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.136+0000 7f2db31bd740 1 --2- 172.21.15.133:0/2945307705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615866245d0 0x561586620e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.136+0000 7f2db31bd740 1 -- 172.21.15.133:0/2945307705 >> 172.21.15.133:0/2945307705 conn(0x5615865ed310 msgr2=0x561586620700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:43.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.142+0000 7f2db31bd740 1 -- 172.21.15.133:0/2945307705 shutdown_connections 2024-09-11T23:43:43.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.142+0000 7f2db31bd740 1 -- 172.21.15.133:0/2945307705 wait complete. 2024-09-11T23:43:43.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.143+0000 7f2db31bd740 1 -- start start 2024-09-11T23:43:43.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.143+0000 7f2db31bd740 1 Processor -- start 2024-09-11T23:43:43.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.144+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615866245d0 0x561586637550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.144+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561586625b40 0x561586637a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.145+0000 7f2db31bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56158662f370 0x5615866282c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.145+0000 7f2db31bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c00fbf0 con 0x5615866245d0 2024-09-11T23:43:43.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.145+0000 7f2db31bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c00f520 con 0x56158662f370 2024-09-11T23:43:43.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.145+0000 7f2db31bd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c00c780 con 0x561586625b40 2024-09-11T23:43:43.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.145+0000 7f2db31bd740 10 client.0 _ll_register_callbacks cb 0x7f2d9c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:43.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.145+0000 7f2da9d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561586625b40 0x561586637a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:43.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.145+0000 7f2da9d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561586625b40 0x561586637a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42844/0 (socket says 172.21.15.133:42844) 2024-09-11T23:43:43.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.145+0000 7f2da9d9b640 1 -- 172.21.15.133:0/3959929043 learned_addr learned my addr 172.21.15.133:0/3959929043 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:43.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.145+0000 7f2daad9d640 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56158662f370 0x5615866282c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:43.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.150+0000 7f2daad9d640 1 -- 172.21.15.133:0/3959929043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561586625b40 msgr2=0x561586637a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:43.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.150+0000 7f2daad9d640 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561586625b40 0x561586637a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.150+0000 7f2daad9d640 1 -- 172.21.15.133:0/3959929043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615866245d0 msgr2=0x561586637550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:43.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.150+0000 7f2daad9d640 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615866245d0 0x561586637550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.150+0000 7f2daad9d640 1 -- 172.21.15.133:0/3959929043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561586628800 con 0x56158662f370 2024-09-11T23:43:43.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.154+0000 7f2daad9d640 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56158662f370 0x5615866282c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x56158652ce20 tx=0x7f2da00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dc64d19b4c9da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:43.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.154+0000 7f2d91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:43.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.155+0000 7f2d91ffb640 1 -- 172.21.15.133:0/3959929043 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da00207b0 con 0x56158662f370 2024-09-11T23:43:43.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.156+0000 7f2db31bd740 1 -- 172.21.15.133:0/3959929043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d9c0a47d0 con 0x56158662f370 2024-09-11T23:43:43.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.156+0000 7f2d91ffb640 1 -- 172.21.15.133:0/3959929043 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f2da0020950 con 0x56158662f370 2024-09-11T23:43:43.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.156+0000 7f2d91ffb640 1 -- 172.21.15.133:0/3959929043 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2da0020b50 con 0x56158662f370 2024-09-11T23:43:43.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.158+0000 7f2db31bd740 10 client.26777 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:43.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.158+0000 7f2db31bd740 1 -- 172.21.15.133:0/3959929043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5615866a9460 con 0x56158662f370 2024-09-11T23:43:43.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.161+0000 7f2d91ffb640 1 -- 172.21.15.133:0/3959929043 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da0020cf0 con 0x56158662f370 2024-09-11T23:43:43.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.162+0000 7f2d91ffb640 1 -- 172.21.15.133:0/3959929043 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2da0011400 con 0x56158662f370 2024-09-11T23:43:43.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.163+0000 7f2db31bd740 10 client.26777 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:43.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.163+0000 7f2db31bd740 10 client.26777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:43:43.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.163+0000 7f2db31bd740 20 client.26777 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:43.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.163+0000 7f2db31bd740 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5615866a9bb0 0x5615866abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:43.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.163+0000 7f2db31bd740 4 client.26777 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:43:43.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.163+0000 7f2db31bd740 1 -- 172.21.15.133:0/3959929043 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2d9c0a47d0 con 0x5615866a9bb0 2024-09-11T23:43:43.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.165+0000 7f2daa59c640 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5615866a9bb0 0x5615866abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:43.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.166+0000 7f2daa59c640 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5615866a9bb0 0x5615866abfd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=134ef03ef9c204ab server_cookie=8f75c8fabde6a69b in_seq=0 out_seq=0 2024-09-11T23:43:43.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.167+0000 7f2d91ffb640 10 client.26777 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:43:43.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.167+0000 7f2d91ffb640 1 -- 172.21.15.133:0/3959929043 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2d9c0a47d0 con 0x5615866a9bb0 2024-09-11T23:43:43.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.167+0000 7f2d91ffb640 10 client.26777 handle_command_reply: tid=0 2024-09-11T23:43:43.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.167+0000 7f2db31bd740 1 client.26777 shutdown 2024-09-11T23:43:43.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.168+0000 7f2db31bd740 1 -- 172.21.15.133:0/3959929043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56158662f370 msgr2=0x5615866282c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:43.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.168+0000 7f2db31bd740 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56158662f370 0x5615866282c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x56158652ce20 tx=0x7f2da00046c0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.168+0000 7f2db31bd740 1 -- 172.21.15.133:0/3959929043 shutdown_connections 2024-09-11T23:43:43.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.168+0000 7f2db31bd740 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5615866a9bb0 0x5615866abfd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.168+0000 7f2db31bd740 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56158662f370 0x5615866282c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.168+0000 7f2db31bd740 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561586625b40 0x561586637a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.168+0000 7f2db31bd740 1 --2- 172.21.15.133:0/3959929043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615866245d0 0x561586637550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.168+0000 7f2db31bd740 1 -- 172.21.15.133:0/3959929043 >> 172.21.15.133:0/3959929043 conn(0x5615865ed310 msgr2=0x56158661f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:43.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.168+0000 7f2d91ffb640 0 client.26777 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:43:43.174 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:43:43.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.172+0000 7f2db31bd740 1 -- 172.21.15.133:0/3959929043 shutdown_connections 2024-09-11T23:43:43.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.172+0000 7f2db31bd740 1 -- 172.21.15.133:0/3959929043 wait complete. 2024-09-11T23:43:43.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.172+0000 7f2db31bd740 20 client.26777 trim_cache size 0 max 16384 2024-09-11T23:43:43.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.176+0000 7f2db0de4640 1 -- 172.21.15.133:0/3367624371 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d940a2dc0 msgr2=0x7f2d940a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:43.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.176+0000 7f2db0de4640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d940a2dc0 0x7f2d940a5280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2da00058c0 tx=0x7f2da0005850 comp rx=0 tx=0).stop 2024-09-11T23:43:43.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.176+0000 7f2db0de4640 1 -- 172.21.15.133:0/3367624371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dac077040 msgr2=0x7f2dac073930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:43.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.176+0000 7f2db0de4640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dac077040 0x7f2dac073930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00e790 tx=0x7f2d9c00ec50 comp rx=0 tx=0).stop 2024-09-11T23:43:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.178+0000 7f2db0de4640 1 -- 172.21.15.133:0/3367624371 shutdown_connections 2024-09-11T23:43:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.178+0000 7f2db0de4640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d940a2dc0 0x7f2d940a5280 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.178+0000 7f2db0de4640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dac112a30 0x7f2dac1073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.178+0000 7f2db0de4640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dac077040 0x7f2dac073930 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.178+0000 7f2db0de4640 1 --2- 172.21.15.133:0/3367624371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac076720 0x7f2dac0733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.178+0000 7f2db0de4640 1 -- 172.21.15.133:0/3367624371 >> 172.21.15.133:0/3367624371 conn(0x7f2dac0ffa70 msgr2=0x7f2dac113290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.178+0000 7f2db0de4640 1 -- 172.21.15.133:0/3367624371 shutdown_connections 2024-09-11T23:43:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:43.178+0000 7f2db0de4640 1 -- 172.21.15.133:0/3367624371 wait complete. 2024-09-11T23:43:44.412 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:43:44.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:44 smithi138 ceph-mon[35142]: pgmap v214: 97 pgs: 97 active+clean; 59 GiB data, 190 GiB used, 882 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 858 MiB/s wr, 3.23k op/s 2024-09-11T23:43:44.963 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:44 smithi133 ceph-mon[31430]: pgmap v214: 97 pgs: 97 active+clean; 59 GiB data, 190 GiB used, 882 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 858 MiB/s wr, 3.23k op/s 2024-09-11T23:43:45.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:44 smithi178 ceph-mon[35882]: pgmap v214: 97 pgs: 97 active+clean; 59 GiB data, 190 GiB used, 882 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 858 MiB/s wr, 3.23k op/s 2024-09-11T23:43:45.263 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:45.715 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:45 smithi133 ceph-mon[31430]: pgmap v215: 97 pgs: 97 active+clean; 59 GiB data, 192 GiB used, 880 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 728 MiB/s wr, 2.59k op/s 2024-09-11T23:43:46.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:45 smithi178 ceph-mon[35882]: pgmap v215: 97 pgs: 97 active+clean; 59 GiB data, 192 GiB used, 880 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 728 MiB/s wr, 2.59k op/s 2024-09-11T23:43:46.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:45 smithi138 ceph-mon[35142]: pgmap v215: 97 pgs: 97 active+clean; 59 GiB data, 192 GiB used, 880 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 728 MiB/s wr, 2.59k op/s 2024-09-11T23:43:46.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.235+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/4072503327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b981068d0 msgr2=0x7f9b98106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:46.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.235+0000 7f9b9dfdc640 1 --2- 172.21.15.133:0/4072503327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b981068d0 0x7f9b98106cb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009930 tx=0x7f9b8c02a480 comp rx=0 tx=0).stop 2024-09-11T23:43:46.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.235+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/4072503327 shutdown_connections 2024-09-11T23:43:46.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.235+0000 7f9b9dfdc640 1 --2- 172.21.15.133:0/4072503327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b98101270 0x7f9b9810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.235+0000 7f9b9dfdc640 1 --2- 172.21.15.133:0/4072503327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b981008d0 0x7f9b98100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.235+0000 7f9b9dfdc640 1 --2- 172.21.15.133:0/4072503327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b981068d0 0x7f9b98106cb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.235+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/4072503327 >> 172.21.15.133:0/4072503327 conn(0x7f9b980fc610 msgr2=0x7f9b980fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:46.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.236+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/4072503327 shutdown_connections 2024-09-11T23:43:46.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.236+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/4072503327 wait complete. 2024-09-11T23:43:46.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.237+0000 7f9b9dfdc640 1 Processor -- start 2024-09-11T23:43:46.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.238+0000 7f9b9dfdc640 1 -- start start 2024-09-11T23:43:46.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.238+0000 7f9b9dfdc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b981008d0 0x7f9b98116010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.239+0000 7f9b9dfdc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b98101270 0x7f9b98116550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.239+0000 7f9b9dfdc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b981068d0 0x7f9b98111130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.239+0000 7f9b9dfdc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9810fc50 con 0x7f9b981008d0 2024-09-11T23:43:46.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.239+0000 7f9b9dfdc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9810fad0 con 0x7f9b98101270 2024-09-11T23:43:46.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.239+0000 7f9b9dfdc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9810fdd0 con 0x7f9b981068d0 2024-09-11T23:43:46.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.241+0000 7f9b9d7db640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b981068d0 0x7f9b98111130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.241+0000 7f9b9d7db640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b981068d0 0x7f9b98111130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42852/0 (socket says 172.21.15.133:42852) 2024-09-11T23:43:46.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.241+0000 7f9b9d7db640 1 -- 172.21.15.133:0/3293041877 learned_addr learned my addr 172.21.15.133:0/3293041877 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:46.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.241+0000 7f9b97fff640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b98101270 0x7f9b98116550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.245+0000 7f9b97fff640 1 -- 172.21.15.133:0/3293041877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b981068d0 msgr2=0x7f9b98111130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:46.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.245+0000 7f9b97fff640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b981068d0 0x7f9b98111130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.245+0000 7f9b97fff640 1 -- 172.21.15.133:0/3293041877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b981008d0 msgr2=0x7f9b98116010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:46.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.245+0000 7f9b97fff640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b981008d0 0x7f9b98116010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.245+0000 7f9b97fff640 1 -- 172.21.15.133:0/3293041877 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b98111940 con 0x7f9b98101270 2024-09-11T23:43:46.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.245+0000 7f9b9d7db640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b981068d0 0x7f9b98111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:43:46.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.247+0000 7f9b97fff640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b98101270 0x7f9b98116550 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800cce0 tx=0x7f9b880075f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:46.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.247+0000 7f9b95ffb640 1 -- 172.21.15.133:0/3293041877 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88004b20 con 0x7f9b98101270 2024-09-11T23:43:46.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.248+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/3293041877 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b98111c30 con 0x7f9b98101270 2024-09-11T23:43:46.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.248+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/3293041877 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9806bde0 con 0x7f9b98101270 2024-09-11T23:43:46.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.249+0000 7f9b95ffb640 1 -- 172.21.15.133:0/3293041877 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b88004cc0 con 0x7f9b98101270 2024-09-11T23:43:46.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.251+0000 7f9b95ffb640 1 -- 172.21.15.133:0/3293041877 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800f6f0 con 0x7f9b98101270 2024-09-11T23:43:46.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.256+0000 7f9ba1617740 1 Processor -- start 2024-09-11T23:43:46.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.258+0000 7f9ba1617740 1 -- start start 2024-09-11T23:43:46.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.258+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce9d20 0x556018ce8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.258+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018ce8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.258+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018cf5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.258+0000 7f9ba1617740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556018c4e2f0 con 0x556018ce87b0 2024-09-11T23:43:46.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.258+0000 7f9ba1617740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556018bf5cf0 con 0x556018ce9d20 2024-09-11T23:43:46.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.258+0000 7f9ba1617740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556018c4a2b0 con 0x556018cf3550 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.258+0000 7f9b95ffb640 1 -- 172.21.15.133:0/3293041877 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9b88005020 con 0x7f9b98101270 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.259+0000 7f9b9d7db640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce9d20 0x556018ce8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.259+0000 7f9b9d7db640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce9d20 0x556018ce8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58374/0 (socket says 172.21.15.133:58374) 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.259+0000 7f9b9d7db640 1 -- 172.21.15.133:0/3018295183 learned_addr learned my addr 172.21.15.133:0/3018295183 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.259+0000 7f9b97fff640 1 --2- 172.21.15.133:0/3018295183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018cf5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.260+0000 7f9b9d7db640 1 -- 172.21.15.133:0/3018295183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 msgr2=0x556018cf5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.260+0000 7f9b9d7db640 1 --2- 172.21.15.133:0/3018295183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018cf5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.260+0000 7f9b9d7db640 1 -- 172.21.15.133:0/3018295183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 msgr2=0x556018ce8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.260+0000 7f9b9d7db640 1 --2- 172.21.15.133:0/3018295183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018ce8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.260+0000 7f9b9d7db640 1 -- 172.21.15.133:0/3018295183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556018bda3a0 con 0x556018ce9d20 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.261+0000 7f9b95ffb640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9b800a2e30 0x7f9b800a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.261+0000 7f9b9d7db640 1 --2- 172.21.15.133:0/3018295183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce9d20 0x556018ce8270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9b900103d0 tx=0x7f9b90010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc22d3b3b4691c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:46.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.261+0000 7f9b95ffb640 1 -- 172.21.15.133:0/3293041877 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f9b880cf750 con 0x7f9b98101270 2024-09-11T23:43:46.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.263+0000 7f9b9cfda640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9b800a2e30 0x7f9b800a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.265+0000 7f9b7effd640 1 -- 172.21.15.133:0/3018295183 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900337d0 con 0x556018ce9d20 2024-09-11T23:43:46.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.266+0000 7f9b7effd640 1 -- 172.21.15.133:0/3018295183 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b90033e50 con 0x556018ce9d20 2024-09-11T23:43:46.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.268+0000 7f9ba1617740 1 -- 172.21.15.133:0/3018295183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce9d20 msgr2=0x556018ce8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:46.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.268+0000 7f9ba1617740 1 --2- 172.21.15.133:0/3018295183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce9d20 0x556018ce8270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9b900103d0 tx=0x7f9b90010890 comp rx=0 tx=0).stop 2024-09-11T23:43:46.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.275+0000 7f9ba1617740 1 -- 172.21.15.133:0/3018295183 shutdown_connections 2024-09-11T23:43:46.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.275+0000 7f9ba1617740 1 --2- 172.21.15.133:0/3018295183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018cf5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.275+0000 7f9ba1617740 1 --2- 172.21.15.133:0/3018295183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018ce8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.275+0000 7f9ba1617740 1 --2- 172.21.15.133:0/3018295183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce9d20 0x556018ce8270 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.275+0000 7f9ba1617740 1 -- 172.21.15.133:0/3018295183 >> 172.21.15.133:0/3018295183 conn(0x556018cb1310 msgr2=0x556018ce4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:46.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.280+0000 7f9ba1617740 1 -- 172.21.15.133:0/3018295183 shutdown_connections 2024-09-11T23:43:46.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.280+0000 7f9b9cfda640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9b800a2e30 0x7f9b800a52f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c033040 tx=0x7f9b8c0095a0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.280+0000 7f9ba1617740 1 -- 172.21.15.133:0/3018295183 wait complete. 2024-09-11T23:43:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.281+0000 7f9ba1617740 1 -- start start 2024-09-11T23:43:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.282+0000 7f9ba1617740 1 Processor -- start 2024-09-11T23:43:46.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.283+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce87b0 0x556018cf2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.284+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018cf3550 0x556018ceff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.284+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf0470 0x556018cf08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.284+0000 7f9ba1617740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556018cf62a0 con 0x556018cf3550 2024-09-11T23:43:46.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.284+0000 7f9ba1617740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556018c09060 con 0x556018ce87b0 2024-09-11T23:43:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.284+0000 7f9ba1617740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556018cf6420 con 0x556018cf0470 2024-09-11T23:43:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.284+0000 7f9ba1617740 10 client.0 _ll_register_callbacks cb 0x55601895c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.286+0000 7f9b97fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018cf3550 0x556018ceff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.286+0000 7f9b97fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018cf3550 0x556018ceff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56254/0 (socket says 172.21.15.133:56254) 2024-09-11T23:43:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.286+0000 7f9b97fff640 1 -- 172.21.15.133:0/2021199845 learned_addr learned my addr 172.21.15.133:0/2021199845 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.287+0000 7f9b9cfda640 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce87b0 0x556018cf2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.288+0000 7f9b97fff640 1 -- 172.21.15.133:0/2021199845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf0470 msgr2=0x556018cf08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.288+0000 7f9b97fff640 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf0470 0x556018cf08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.288+0000 7f9b97fff640 1 -- 172.21.15.133:0/2021199845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce87b0 msgr2=0x556018cf2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.288+0000 7f9b97fff640 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce87b0 0x556018cf2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.288+0000 7f9b97fff640 1 -- 172.21.15.133:0/2021199845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560189bf3c0 con 0x556018cf3550 2024-09-11T23:43:46.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.293+0000 7f9b97fff640 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018cf3550 0x556018ceff30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9b88007af0 tx=0x7f9b8800a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effaf3b249429165 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:46.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.293+0000 7f9b77fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.298+0000 7f9b77fff640 1 -- 172.21.15.133:0/2021199845 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8801d040 con 0x556018cf3550 2024-09-11T23:43:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.298+0000 7f9b77fff640 1 -- 172.21.15.133:0/2021199845 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f9b880a4630 con 0x556018cf3550 2024-09-11T23:43:46.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.300+0000 7f9ba1617740 1 -- 172.21.15.133:0/2021199845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556018c5b860 con 0x556018cf3550 2024-09-11T23:43:46.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.302+0000 7f9b77fff640 1 -- 172.21.15.133:0/2021199845 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b880b1020 con 0x556018cf3550 2024-09-11T23:43:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.306+0000 7f9ba1617740 10 client.17292 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.306+0000 7f9ba1617740 1 -- 172.21.15.133:0/2021199845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556018d02020 con 0x556018cf3550 2024-09-11T23:43:46.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.315+0000 7f9b77fff640 1 -- 172.21.15.133:0/2021199845 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b880b4e50 con 0x556018cf3550 2024-09-11T23:43:46.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.315+0000 7f9b77fff640 1 -- 172.21.15.133:0/2021199845 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9b880cfb80 con 0x556018cf3550 2024-09-11T23:43:46.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.315+0000 7f9ba1617740 10 client.17292 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:46.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.315+0000 7f9ba1617740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:46.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.315+0000 7f9ba1617740 20 client.17292 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.315+0000 7f9ba1617740 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556018d03430 0x556018d058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.315+0000 7f9ba1617740 4 client.17292 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.315+0000 7f9ba1617740 1 -- 172.21.15.133:0/2021199845 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556018c55d70 con 0x556018d03430 2024-09-11T23:43:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.316+0000 7f9b9cfda640 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556018d03430 0x556018d058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.320+0000 7f9b9cfda640 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556018d03430 0x556018d058c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db589b6b3f28610f server_cookie=bd63fbd968c9e8d4 in_seq=0 out_seq=0 2024-09-11T23:43:46.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.320+0000 7f9b77fff640 10 client.17292 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.325+0000 7f9b77fff640 1 -- 172.21.15.133:0/2021199845 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556018c55d70 con 0x556018d03430 2024-09-11T23:43:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.325+0000 7f9b77fff640 10 client.17292 handle_command_reply: tid=0 2024-09-11T23:43:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.325+0000 7f9ba1617740 1 client.17292 shutdown 2024-09-11T23:43:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.326+0000 7f9ba1617740 1 -- 172.21.15.133:0/2021199845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018cf3550 msgr2=0x556018ceff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.326+0000 7f9ba1617740 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018cf3550 0x556018ceff30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9b88007af0 tx=0x7f9b8800a730 comp rx=0 tx=0).stop 2024-09-11T23:43:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.326+0000 7f9ba1617740 1 -- 172.21.15.133:0/2021199845 shutdown_connections 2024-09-11T23:43:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.326+0000 7f9ba1617740 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556018d03430 0x556018d058c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.326+0000 7f9ba1617740 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf0470 0x556018cf08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.326+0000 7f9ba1617740 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018cf3550 0x556018ceff30 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.326+0000 7f9ba1617740 1 --2- 172.21.15.133:0/2021199845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018ce87b0 0x556018cf2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.326+0000 7f9ba1617740 1 -- 172.21.15.133:0/2021199845 >> 172.21.15.133:0/2021199845 conn(0x556018cb1310 msgr2=0x556018ceab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.326+0000 7f9b77fff640 0 client.17292 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.330+0000 7f9ba1617740 1 -- 172.21.15.133:0/2021199845 shutdown_connections 2024-09-11T23:43:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.330+0000 7f9ba1617740 1 -- 172.21.15.133:0/2021199845 wait complete. 2024-09-11T23:43:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.330+0000 7f9ba1617740 20 client.17292 trim_cache size 0 max 16384 2024-09-11T23:43:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.374+0000 7f9ba1617740 1 Processor -- start 2024-09-11T23:43:46.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.379+0000 7f9ba1617740 1 -- start start 2024-09-11T23:43:46.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.384+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018ce3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.384+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018cf0470 0x556018ceb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.385+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018d0b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.385+0000 7f9ba1617740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8400e730 con 0x556018ce87b0 2024-09-11T23:43:46.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.385+0000 7f9ba1617740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84010270 con 0x556018cf0470 2024-09-11T23:43:46.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.385+0000 7f9ba1617740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8400ee00 con 0x556018cf3550 2024-09-11T23:43:46.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.386+0000 7f9b97fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018cf0470 0x556018ceb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.387+0000 7f9b9d7db640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018d0b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.388+0000 7f9b9d7db640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018d0b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42874/0 (socket says 172.21.15.133:42874) 2024-09-11T23:43:46.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.388+0000 7f9b9d7db640 1 -- 172.21.15.133:0/1793063498 learned_addr learned my addr 172.21.15.133:0/1793063498 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:46.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.391+0000 7f9b9cfda640 1 --2- 172.21.15.133:0/1793063498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018ce3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.394+0000 7f9b97fff640 1 -- 172.21.15.133:0/1793063498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 msgr2=0x556018d0b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:46.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.394+0000 7f9b97fff640 1 --2- 172.21.15.133:0/1793063498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018d0b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.394+0000 7f9b97fff640 1 -- 172.21.15.133:0/1793063498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 msgr2=0x556018ce3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:46.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.394+0000 7f9b97fff640 1 --2- 172.21.15.133:0/1793063498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018ce3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.394+0000 7f9b97fff640 1 -- 172.21.15.133:0/1793063498 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556018cf0c60 con 0x556018cf0470 2024-09-11T23:43:46.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.396+0000 7f9b97fff640 1 --2- 172.21.15.133:0/1793063498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018cf0470 0x556018ceb300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x556018cf14e0 tx=0x7f9b88007c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a74d700afcc688d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:46.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.397+0000 7f9b7e7fc640 1 -- 172.21.15.133:0/1793063498 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8801d070 con 0x556018cf0470 2024-09-11T23:43:46.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.397+0000 7f9b7e7fc640 1 -- 172.21.15.133:0/1793063498 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556018c5b860 con 0x556018cf0470 2024-09-11T23:43:46.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.397+0000 7f9b7e7fc640 1 -- 172.21.15.133:0/1793063498 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b880cfcf0 con 0x556018cf0470 2024-09-11T23:43:46.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.399+0000 7f9ba1617740 1 -- 172.21.15.133:0/1793063498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018cf0470 msgr2=0x556018ceb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:46.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.399+0000 7f9ba1617740 1 --2- 172.21.15.133:0/1793063498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018cf0470 0x556018ceb300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x556018cf14e0 tx=0x7f9b88007c40 comp rx=0 tx=0).stop 2024-09-11T23:43:46.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.405+0000 7f9ba1617740 1 -- 172.21.15.133:0/1793063498 shutdown_connections 2024-09-11T23:43:46.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.405+0000 7f9ba1617740 1 --2- 172.21.15.133:0/1793063498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018d0b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.405+0000 7f9ba1617740 1 --2- 172.21.15.133:0/1793063498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018cf0470 0x556018ceb300 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.405+0000 7f9ba1617740 1 --2- 172.21.15.133:0/1793063498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018ce3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.405+0000 7f9ba1617740 1 -- 172.21.15.133:0/1793063498 >> 172.21.15.133:0/1793063498 conn(0x556018cb1310 msgr2=0x556018ce3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:46.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.413+0000 7f9ba1617740 1 -- 172.21.15.133:0/1793063498 shutdown_connections 2024-09-11T23:43:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.413+0000 7f9ba1617740 1 -- 172.21.15.133:0/1793063498 wait complete. 2024-09-11T23:43:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.414+0000 7f9ba1617740 1 -- start start 2024-09-11T23:43:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.414+0000 7f9ba1617740 1 Processor -- start 2024-09-11T23:43:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.416+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018cf7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.416+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018cf0470 0x556018cf7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.416+0000 7f9ba1617740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018d6d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.416+0000 7f9ba1617740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8400fba0 con 0x556018ce87b0 2024-09-11T23:43:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.416+0000 7f9ba1617740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8400f4d0 con 0x556018cf0470 2024-09-11T23:43:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.416+0000 7f9ba1617740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8400c730 con 0x556018cf3550 2024-09-11T23:43:46.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.417+0000 7f9ba1617740 10 client.0 _ll_register_callbacks cb 0x7f9b88005b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:46.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.420+0000 7f9b9d7db640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018d6d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.420+0000 7f9b97fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018cf0470 0x556018cf7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.421+0000 7f9b9cfda640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018cf7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.422+0000 7f9b9cfda640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018cf7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56272/0 (socket says 172.21.15.133:56272) 2024-09-11T23:43:46.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.422+0000 7f9b9cfda640 1 -- 172.21.15.133:0/1275869929 learned_addr learned my addr 172.21.15.133:0/1275869929 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:46.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.422+0000 7f9b9cfda640 1 -- 172.21.15.133:0/1275869929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 msgr2=0x556018d6d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:46.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.422+0000 7f9b9cfda640 1 --2- 172.21.15.133:0/1275869929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018d6d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.422+0000 7f9b9cfda640 1 -- 172.21.15.133:0/1275869929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018cf0470 msgr2=0x556018cf7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:46.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.422+0000 7f9b9cfda640 1 --2- 172.21.15.133:0/1275869929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018cf0470 0x556018cf7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.422+0000 7f9b9cfda640 1 -- 172.21.15.133:0/1275869929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556018cf86c0 con 0x556018ce87b0 2024-09-11T23:43:46.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.423+0000 7f9b9cfda640 1 --2- 172.21.15.133:0/1275869929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018cf7880 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009930 tx=0x7f9b8c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d884198faf478235 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:46.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.423+0000 7f9b77fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:46.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.425+0000 7f9b77fff640 1 -- 172.21.15.133:0/1275869929 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c005410 con 0x556018ce87b0 2024-09-11T23:43:46.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.425+0000 7f9b77fff640 1 -- 172.21.15.133:0/1275869929 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f9b8c0055b0 con 0x556018ce87b0 2024-09-11T23:43:46.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.425+0000 7f9ba1617740 1 -- 172.21.15.133:0/1275869929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556018c5b860 con 0x556018ce87b0 2024-09-11T23:43:46.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.426+0000 7f9b77fff640 1 -- 172.21.15.133:0/1275869929 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b8c03c440 con 0x556018ce87b0 2024-09-11T23:43:46.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.426+0000 7f9ba1617740 10 client.17298 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:46.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.426+0000 7f9ba1617740 1 -- 172.21.15.133:0/1275869929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556018d6e870 con 0x556018ce87b0 2024-09-11T23:43:46.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.427+0000 7f9b77fff640 1 -- 172.21.15.133:0/1275869929 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8c042030 con 0x556018ce87b0 2024-09-11T23:43:46.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.428+0000 7f9b77fff640 1 -- 172.21.15.133:0/1275869929 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9b8c0317c0 con 0x556018ce87b0 2024-09-11T23:43:46.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.428+0000 7f9ba1617740 10 client.17298 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:46.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.428+0000 7f9ba1617740 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:46.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.428+0000 7f9ba1617740 20 client.17298 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:46.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.428+0000 7f9ba1617740 1 --2- 172.21.15.133:0/1275869929 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556018d6efc0 0x556018d713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:46.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.428+0000 7f9ba1617740 4 client.17298 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:43:46.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.428+0000 7f9ba1617740 1 -- 172.21.15.133:0/1275869929 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556018c5b860 con 0x556018d6efc0 2024-09-11T23:43:46.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.428+0000 7f9b97fff640 1 --2- 172.21.15.133:0/1275869929 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556018d6efc0 0x556018d713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:46.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.431+0000 7f9b77fff640 10 client.17298 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:46.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.431+0000 7f9b97fff640 1 --2- 172.21.15.133:0/1275869929 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556018d6efc0 0x556018d713e0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac191bea188d8d5c server_cookie=e16a925331b27e59 in_seq=0 out_seq=0 2024-09-11T23:43:46.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.435+0000 7f9b77fff640 1 -- 172.21.15.133:0/1275869929 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556018c5b860 con 0x556018d6efc0 2024-09-11T23:43:46.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.435+0000 7f9b77fff640 10 client.17298 handle_command_reply: tid=0 2024-09-11T23:43:46.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.435+0000 7f9ba1617740 1 client.17298 shutdown 2024-09-11T23:43:46.452 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:43:46.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.442+0000 7f9ba1617740 1 -- 172.21.15.133:0/1275869929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 msgr2=0x556018cf7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:46.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.442+0000 7f9ba1617740 1 --2- 172.21.15.133:0/1275869929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018cf7880 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009930 tx=0x7f9b8c005fd0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.442+0000 7f9ba1617740 1 -- 172.21.15.133:0/1275869929 shutdown_connections 2024-09-11T23:43:46.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.442+0000 7f9ba1617740 1 --2- 172.21.15.133:0/1275869929 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556018d6efc0 0x556018d713e0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.442+0000 7f9b77fff640 0 client.17298 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.442+0000 7f9ba1617740 1 --2- 172.21.15.133:0/1275869929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556018cf3550 0x556018d6d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.442+0000 7f9ba1617740 1 --2- 172.21.15.133:0/1275869929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556018cf0470 0x556018cf7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.442+0000 7f9ba1617740 1 --2- 172.21.15.133:0/1275869929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556018ce87b0 0x556018cf7880 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.442+0000 7f9ba1617740 1 -- 172.21.15.133:0/1275869929 >> 172.21.15.133:0/1275869929 conn(0x556018cb1310 msgr2=0x556018ce3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.444+0000 7f9ba1617740 1 -- 172.21.15.133:0/1275869929 shutdown_connections 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.444+0000 7f9ba1617740 1 -- 172.21.15.133:0/1275869929 wait complete. 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.444+0000 7f9ba1617740 20 client.17298 trim_cache size 0 max 16384 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.446+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/3293041877 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9b800a2e30 msgr2=0x7f9b800a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.446+0000 7f9b9dfdc640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9b800a2e30 0x7f9b800a52f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c033040 tx=0x7f9b8c0095a0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.446+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/3293041877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b98101270 msgr2=0x7f9b98116550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.446+0000 7f9b9dfdc640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b98101270 0x7f9b98116550 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800cce0 tx=0x7f9b880075f0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.447+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/3293041877 shutdown_connections 2024-09-11T23:43:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.447+0000 7f9b9dfdc640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9b800a2e30 0x7f9b800a52f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.447+0000 7f9b9dfdc640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b981068d0 0x7f9b98111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.447+0000 7f9b9dfdc640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b98101270 0x7f9b98116550 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.447+0000 7f9b9dfdc640 1 --2- 172.21.15.133:0/3293041877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b981008d0 0x7f9b98116010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:46.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.447+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/3293041877 >> 172.21.15.133:0/3293041877 conn(0x7f9b980fc610 msgr2=0x7f9b980fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:46.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.447+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/3293041877 shutdown_connections 2024-09-11T23:43:46.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:46.447+0000 7f9b9dfdc640 1 -- 172.21.15.133:0/3293041877 wait complete. 2024-09-11T23:43:48.460 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:43:48.460 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:43:48.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:48 smithi138 ceph-mon[35142]: pgmap v216: 97 pgs: 97 active+clean; 59 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 685 MiB/s wr, 2.23k op/s 2024-09-11T23:43:48.962 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:48 smithi133 ceph-mon[31430]: pgmap v216: 97 pgs: 97 active+clean; 59 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 685 MiB/s wr, 2.23k op/s 2024-09-11T23:43:49.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:48 smithi178 ceph-mon[35882]: pgmap v216: 97 pgs: 97 active+clean; 59 GiB data, 194 GiB used, 879 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 685 MiB/s wr, 2.23k op/s 2024-09-11T23:43:49.390 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:50.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.432+0000 7f8811a21640 1 -- 172.21.15.133:0/3685109376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f880c1012a0 msgr2=0x7f880c105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:50.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.432+0000 7f8811a21640 1 --2- 172.21.15.133:0/3685109376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f880c1012a0 0x7f880c105770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ca0 tx=0x7f880402a980 comp rx=0 tx=0).stop 2024-09-11T23:43:50.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.433+0000 7f8811a21640 1 -- 172.21.15.133:0/3685109376 shutdown_connections 2024-09-11T23:43:50.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.433+0000 7f8811a21640 1 --2- 172.21.15.133:0/3685109376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f880c105d90 0x7f880c106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.433+0000 7f8811a21640 1 --2- 172.21.15.133:0/3685109376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f880c1012a0 0x7f880c105770 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.433+0000 7f8811a21640 1 --2- 172.21.15.133:0/3685109376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f880c100980 0x7f880c100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.433+0000 7f8811a21640 1 -- 172.21.15.133:0/3685109376 >> 172.21.15.133:0/3685109376 conn(0x7f880c0fc610 msgr2=0x7f880c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.433+0000 7f8811a21640 1 -- 172.21.15.133:0/3685109376 shutdown_connections 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.433+0000 7f8811a21640 1 -- 172.21.15.133:0/3685109376 wait complete. 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.434+0000 7f8811a21640 1 Processor -- start 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.434+0000 7f8811a21640 1 -- start start 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.434+0000 7f8811a21640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f880c100980 0x7f880c071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.434+0000 7f8811a21640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f880c1012a0 0x7f880c0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.434+0000 7f8811a21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f880c105d90 0x7f880c0733d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.434+0000 7f8811a21640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f880c107350 con 0x7f880c105d90 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.434+0000 7f8811a21640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1071d0 con 0x7f880c1012a0 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.434+0000 7f8811a21640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1074d0 con 0x7f880c100980 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.435+0000 7f8811220640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f880c105d90 0x7f880c0733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.435+0000 7f8811220640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f880c105d90 0x7f880c0733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56280/0 (socket says 172.21.15.133:56280) 2024-09-11T23:43:50.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.435+0000 7f8811220640 1 -- 172.21.15.133:0/2238403845 learned_addr learned my addr 172.21.15.133:0/2238403845 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:50.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.436+0000 7f8811220640 1 -- 172.21.15.133:0/2238403845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f880c100980 msgr2=0x7f880c071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:50.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.436+0000 7f8811220640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f880c100980 0x7f880c071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.436+0000 7f8811220640 1 -- 172.21.15.133:0/2238403845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f880c1012a0 msgr2=0x7f880c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:50.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.436+0000 7f8811220640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f880c1012a0 0x7f880c0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.436+0000 7f8811220640 1 -- 172.21.15.133:0/2238403845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c073c60 con 0x7f880c105d90 2024-09-11T23:43:50.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.436+0000 7f8811220640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f880c105d90 0x7f880c0733d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f87f800b3f0 tx=0x7f87f800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:50.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.437+0000 7f8809ffb640 1 -- 172.21.15.133:0/2238403845 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8015020 con 0x7f880c105d90 2024-09-11T23:43:50.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.437+0000 7f8809ffb640 1 -- 172.21.15.133:0/2238403845 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f8011d10 con 0x7f880c105d90 2024-09-11T23:43:50.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.437+0000 7f8809ffb640 1 -- 172.21.15.133:0/2238403845 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8015020 con 0x7f880c105d90 2024-09-11T23:43:50.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.438+0000 7f8811a21640 1 -- 172.21.15.133:0/2238403845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c073f50 con 0x7f880c105d90 2024-09-11T23:43:50.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.439+0000 7f8811a21640 1 -- 172.21.15.133:0/2238403845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c113070 con 0x7f880c105d90 2024-09-11T23:43:50.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.446+0000 7f881505c740 1 Processor -- start 2024-09-11T23:43:50.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.446+0000 7f881505c740 1 -- start start 2024-09-11T23:43:50.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.446+0000 7f881505c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee065d70 0x55cbee066150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.446+0000 7f881505c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee070a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.446+0000 7f881505c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee070f50 0x55cbee073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.446+0000 7f881505c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cbedfcb280 con 0x55cbee070f50 2024-09-11T23:43:50.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.446+0000 7f881505c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbedf72b10 con 0x55cbee066690 2024-09-11T23:43:50.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.446+0000 7f881505c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cbedfc7240 con 0x55cbee065d70 2024-09-11T23:43:50.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.448+0000 7f880bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee070f50 0x55cbee073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.448+0000 7f880bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee070f50 0x55cbee073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56294/0 (socket says 172.21.15.133:56294) 2024-09-11T23:43:50.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.448+0000 7f880bfff640 1 -- 172.21.15.133:0/1275235694 learned_addr learned my addr 172.21.15.133:0/1275235694 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:50.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.449+0000 7f8810a1f640 1 --2- 172.21.15.133:0/1275235694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.450+0000 7f880bfff640 1 --2- 172.21.15.133:0/1275235694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee065d70 0x55cbee066150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.453+0000 7f8810a1f640 1 -- 172.21.15.133:0/1275235694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee065d70 msgr2=0x55cbee066150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:50.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.453+0000 7f8810a1f640 1 --2- 172.21.15.133:0/1275235694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee065d70 0x55cbee066150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.453+0000 7f8810a1f640 1 -- 172.21.15.133:0/1275235694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee070f50 msgr2=0x55cbee073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:50.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.453+0000 7f8810a1f640 1 --2- 172.21.15.133:0/1275235694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee070f50 0x55cbee073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.453+0000 7f8810a1f640 1 -- 172.21.15.133:0/1275235694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbedf7f1f0 con 0x55cbee066690 2024-09-11T23:43:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.454+0000 7f8810a1f640 1 --2- 172.21.15.133:0/1275235694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee070a10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009750 tx=0x7f87fc009c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=217f6bf06311a908 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.455+0000 7f8809ffb640 1 -- 172.21.15.133:0/2238403845 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f87f8018020 con 0x7f880c105d90 2024-09-11T23:43:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.456+0000 7f8802ffd640 1 -- 172.21.15.133:0/1275235694 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc015020 con 0x55cbee066690 2024-09-11T23:43:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.456+0000 7f8802ffd640 1 -- 172.21.15.133:0/1275235694 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87fc011420 con 0x55cbee066690 2024-09-11T23:43:50.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.456+0000 7f8809ffb640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87f00a3130 0x7f87f00a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.457+0000 7f880bfff640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87f00a3130 0x7f87f00a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.457+0000 7f8809ffb640 1 -- 172.21.15.133:0/2238403845 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f87f8011630 con 0x7f880c105d90 2024-09-11T23:43:50.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.457+0000 7f881505c740 1 -- 172.21.15.133:0/1275235694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 msgr2=0x55cbee070a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:50.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.457+0000 7f881505c740 1 --2- 172.21.15.133:0/1275235694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee070a10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009750 tx=0x7f87fc009c10 comp rx=0 tx=0).stop 2024-09-11T23:43:50.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.458+0000 7f881505c740 1 -- 172.21.15.133:0/1275235694 shutdown_connections 2024-09-11T23:43:50.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.458+0000 7f881505c740 1 --2- 172.21.15.133:0/1275235694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee070f50 0x55cbee073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.458+0000 7f881505c740 1 --2- 172.21.15.133:0/1275235694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee070a10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.458+0000 7f881505c740 1 --2- 172.21.15.133:0/1275235694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee065d70 0x55cbee066150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.458+0000 7f881505c740 1 -- 172.21.15.133:0/1275235694 >> 172.21.15.133:0/1275235694 conn(0x55cbee062300 msgr2=0x55cbee0626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:50.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.459+0000 7f881505c740 1 -- 172.21.15.133:0/1275235694 shutdown_connections 2024-09-11T23:43:50.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.459+0000 7f881505c740 1 -- 172.21.15.133:0/1275235694 wait complete. 2024-09-11T23:43:50.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.459+0000 7f880bfff640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87f00a3130 0x7f87f00a55f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f880402af70 tx=0x7f8804000f40 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:50.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.459+0000 7f881505c740 1 -- start start 2024-09-11T23:43:50.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.459+0000 7f881505c740 1 Processor -- start 2024-09-11T23:43:50.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.460+0000 7f881505c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee070970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.460+0000 7f881505c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee06dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.460+0000 7f881505c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.460+0000 7f881505c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cbee073b70 con 0x55cbee065d70 2024-09-11T23:43:50.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.460+0000 7f881505c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbedf85f10 con 0x55cbee066690 2024-09-11T23:43:50.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.460+0000 7f881505c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cbee073cf0 con 0x55cbee070f50 2024-09-11T23:43:50.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.460+0000 7f881505c740 10 client.0 _ll_register_callbacks cb 0x55cbedf4ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:50.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.461+0000 7f8810a1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.461+0000 7f8810a1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee070970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56296/0 (socket says 172.21.15.133:56296) 2024-09-11T23:43:50.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.461+0000 7f8810a1f640 1 -- 172.21.15.133:0/1751909210 learned_addr learned my addr 172.21.15.133:0/1751909210 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:50.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.461+0000 7f880bfff640 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee06dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.462+0000 7f880bfff640 1 -- 172.21.15.133:0/1751909210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 msgr2=0x55cbee06e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:50.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.462+0000 7f880bfff640 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.462+0000 7f880bfff640 1 -- 172.21.15.133:0/1751909210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 msgr2=0x55cbee070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:50.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.462+0000 7f880bfff640 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee070970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.462+0000 7f880bfff640 1 -- 172.21.15.133:0/1751909210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbedd3c6a0 con 0x55cbee066690 2024-09-11T23:43:50.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.462+0000 7f880bfff640 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee06dcc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55cbee063680 tx=0x7f88040095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa720c0ca1249d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:50.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.462+0000 7f87effff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:50.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.463+0000 7f87effff640 1 -- 172.21.15.133:0/1751909210 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804034040 con 0x55cbee066690 2024-09-11T23:43:50.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.463+0000 7f881505c740 1 -- 172.21.15.133:0/1751909210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87fc011420 con 0x55cbee066690 2024-09-11T23:43:50.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.463+0000 7f87effff640 1 -- 172.21.15.133:0/1751909210 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f8804055470 con 0x55cbee066690 2024-09-11T23:43:50.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.464+0000 7f87effff640 1 -- 172.21.15.133:0/1751909210 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8804055670 con 0x55cbee066690 2024-09-11T23:43:50.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.464+0000 7f881505c740 10 client.26798 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:50.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.464+0000 7f881505c740 1 -- 172.21.15.133:0/1751909210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cbee07f6d0 con 0x55cbee066690 2024-09-11T23:43:50.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.465+0000 7f87effff640 1 -- 172.21.15.133:0/1751909210 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87fc011420 con 0x55cbee066690 2024-09-11T23:43:50.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.465+0000 7f87effff640 1 -- 172.21.15.133:0/1751909210 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f880404c540 con 0x55cbee066690 2024-09-11T23:43:50.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.465+0000 7f881505c740 10 client.26798 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:50.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.465+0000 7f881505c740 10 client.26798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:50.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.465+0000 7f881505c740 20 client.26798 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:50.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.465+0000 7f881505c740 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cbee080f20 0x55cbee0833e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.465+0000 7f881505c740 4 client.26798 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:50.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.465+0000 7f881505c740 1 -- 172.21.15.133:0/1751909210 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbedfd87f0 con 0x55cbee080f20 2024-09-11T23:43:50.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.469+0000 7f8810a1f640 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cbee080f20 0x55cbee0833e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.476+0000 7f87effff640 10 client.26798 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:50.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.476+0000 7f8810a1f640 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cbee080f20 0x55cbee0833e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb69e3289c147a8 server_cookie=65406787b999f3c5 in_seq=0 out_seq=0 2024-09-11T23:43:50.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.478+0000 7f87effff640 1 -- 172.21.15.133:0/1751909210 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbedfd87f0 con 0x55cbee080f20 2024-09-11T23:43:50.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.478+0000 7f87effff640 10 client.26798 handle_command_reply: tid=0 2024-09-11T23:43:50.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.478+0000 7f881505c740 1 client.26798 shutdown 2024-09-11T23:43:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.478+0000 7f881505c740 1 -- 172.21.15.133:0/1751909210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 msgr2=0x55cbee06dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.478+0000 7f881505c740 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee06dcc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55cbee063680 tx=0x7f88040095c0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.478+0000 7f881505c740 1 -- 172.21.15.133:0/1751909210 shutdown_connections 2024-09-11T23:43:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.478+0000 7f881505c740 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cbee080f20 0x55cbee0833e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.478+0000 7f881505c740 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee06e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.478+0000 7f881505c740 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee06dcc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.478+0000 7f881505c740 1 --2- 172.21.15.133:0/1751909210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee070970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.479+0000 7f881505c740 1 -- 172.21.15.133:0/1751909210 >> 172.21.15.133:0/1751909210 conn(0x55cbee062300 msgr2=0x55cbee067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.479+0000 7f87effff640 0 client.26798 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.479+0000 7f881505c740 1 -- 172.21.15.133:0/1751909210 shutdown_connections 2024-09-11T23:43:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.479+0000 7f881505c740 1 -- 172.21.15.133:0/1751909210 wait complete. 2024-09-11T23:43:50.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.479+0000 7f881505c740 20 client.26798 trim_cache size 0 max 16384 2024-09-11T23:43:50.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.508+0000 7f881505c740 1 Processor -- start 2024-09-11T23:43:50.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.509+0000 7f881505c740 1 -- start start 2024-09-11T23:43:50.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.509+0000 7f881505c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee07ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.509+0000 7f881505c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee06fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.509+0000 7f881505c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee088cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.509+0000 7f881505c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1b3d50 con 0x55cbee065d70 2024-09-11T23:43:50.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.509+0000 7f881505c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1b5890 con 0x55cbee066690 2024-09-11T23:43:50.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.509+0000 7f881505c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1b4420 con 0x55cbee070f50 2024-09-11T23:43:50.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.510+0000 7f880bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee06fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.510+0000 7f880bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee06fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58442/0 (socket says 172.21.15.133:58442) 2024-09-11T23:43:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.510+0000 7f880bfff640 1 -- 172.21.15.133:0/3269421249 learned_addr learned my addr 172.21.15.133:0/3269421249 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.510+0000 7f8810a1f640 1 --2- 172.21.15.133:0/3269421249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee07ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.511+0000 7f8810a1f640 1 -- 172.21.15.133:0/3269421249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 msgr2=0x55cbee088cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.511+0000 7f8810a1f640 1 --2- 172.21.15.133:0/3269421249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee088cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.511+0000 7f8810a1f640 1 -- 172.21.15.133:0/3269421249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 msgr2=0x55cbee06fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:50.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.511+0000 7f8810a1f640 1 --2- 172.21.15.133:0/3269421249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee06fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.511+0000 7f8810a1f640 1 -- 172.21.15.133:0/3269421249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbee06cc70 con 0x55cbee065d70 2024-09-11T23:43:50.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.512+0000 7f8810a1f640 1 --2- 172.21.15.133:0/3269421249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee07ce50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f87fc011ec0 tx=0x7f87fc011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555c5fe1f1c5bb7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:50.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.512+0000 7f88027fc640 1 -- 172.21.15.133:0/3269421249 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc015040 con 0x55cbee065d70 2024-09-11T23:43:50.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.513+0000 7f88027fc640 1 -- 172.21.15.133:0/3269421249 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87fc00bdb0 con 0x55cbee065d70 2024-09-11T23:43:50.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.513+0000 7f88027fc640 1 -- 172.21.15.133:0/3269421249 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc010a90 con 0x55cbee065d70 2024-09-11T23:43:50.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.514+0000 7f881505c740 1 -- 172.21.15.133:0/3269421249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 msgr2=0x55cbee07ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:50.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.514+0000 7f881505c740 1 --2- 172.21.15.133:0/3269421249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee07ce50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f87fc011ec0 tx=0x7f87fc011f90 comp rx=0 tx=0).stop 2024-09-11T23:43:50.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.515+0000 7f881505c740 1 -- 172.21.15.133:0/3269421249 shutdown_connections 2024-09-11T23:43:50.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.515+0000 7f881505c740 1 --2- 172.21.15.133:0/3269421249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee088cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.515+0000 7f881505c740 1 --2- 172.21.15.133:0/3269421249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee06fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.515+0000 7f881505c740 1 --2- 172.21.15.133:0/3269421249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee07ce50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.515+0000 7f881505c740 1 -- 172.21.15.133:0/3269421249 >> 172.21.15.133:0/3269421249 conn(0x55cbee062300 msgr2=0x55cbee06f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:50.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.515+0000 7f881505c740 1 -- 172.21.15.133:0/3269421249 shutdown_connections 2024-09-11T23:43:50.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.515+0000 7f881505c740 1 -- 172.21.15.133:0/3269421249 wait complete. 2024-09-11T23:43:50.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.516+0000 7f881505c740 1 -- start start 2024-09-11T23:43:50.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.516+0000 7f881505c740 1 Processor -- start 2024-09-11T23:43:50.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.517+0000 7f881505c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee078fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.517+0000 7f881505c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee079510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.517+0000 7f881505c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee073f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.517+0000 7f881505c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1b51c0 con 0x55cbee065d70 2024-09-11T23:43:50.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.517+0000 7f881505c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1b4af0 con 0x55cbee066690 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.517+0000 7f881505c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1b1d70 con 0x55cbee070f50 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.517+0000 7f881505c740 10 client.0 _ll_register_callbacks cb 0x7f880402e3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.517+0000 7f8811220640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.518+0000 7f880bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee079510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.518+0000 7f880bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee079510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58456/0 (socket says 172.21.15.133:58456) 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.518+0000 7f8811220640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee073f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42954/0 (socket says 172.21.15.133:42954) 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.518+0000 7f880bfff640 1 -- 172.21.15.133:0/2045898447 learned_addr learned my addr 172.21.15.133:0/2045898447 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.518+0000 7f8810a1f640 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee078fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.518+0000 7f8811220640 1 -- 172.21.15.133:0/2045898447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 msgr2=0x55cbee079510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.518+0000 7f8811220640 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee079510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.518+0000 7f8811220640 1 -- 172.21.15.133:0/2045898447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 msgr2=0x55cbee078fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.518+0000 7f8811220640 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee078fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.518+0000 7f8811220640 1 -- 172.21.15.133:0/2045898447 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbee06e640 con 0x55cbee070f50 2024-09-11T23:43:50.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.519+0000 7f8811220640 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee073f80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f87f80cbde0 tx=0x7f87f809b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e299e2cc3f8dad8a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:50.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.519+0000 7f87effff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:43:50.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.519+0000 7f87effff640 1 -- 172.21.15.133:0/2045898447 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8015050 con 0x55cbee070f50 2024-09-11T23:43:50.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.519+0000 7f87effff640 1 -- 172.21.15.133:0/2045898447 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f87f8097220 con 0x55cbee070f50 2024-09-11T23:43:50.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.519+0000 7f881505c740 1 -- 172.21.15.133:0/2045898447 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87fc00bdb0 con 0x55cbee070f50 2024-09-11T23:43:50.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.519+0000 7f87effff640 1 -- 172.21.15.133:0/2045898447 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87f80a3820 con 0x55cbee070f50 2024-09-11T23:43:50.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.520+0000 7f881505c740 10 client.26683 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:50.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.520+0000 7f881505c740 1 -- 172.21.15.133:0/2045898447 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cbee0ec2f0 con 0x55cbee070f50 2024-09-11T23:43:50.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.520+0000 7f87effff640 1 -- 172.21.15.133:0/2045898447 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87fc00bdb0 con 0x55cbee070f50 2024-09-11T23:43:50.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.520+0000 7f87effff640 1 -- 172.21.15.133:0/2045898447 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f87f8022060 con 0x55cbee070f50 2024-09-11T23:43:50.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.520+0000 7f881505c740 10 client.26683 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:50.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.520+0000 7f881505c740 10 client.26683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:50.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.520+0000 7f881505c740 20 client.26683 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:50.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.520+0000 7f881505c740 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cbee0eca10 0x55cbee0eee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:50.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.520+0000 7f881505c740 4 client.26683 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:43:50.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.520+0000 7f881505c740 1 -- 172.21.15.133:0/2045898447 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cbedfd2d00 con 0x55cbee0eca10 2024-09-11T23:43:50.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.521+0000 7f8810a1f640 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cbee0eca10 0x55cbee0eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:50.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.521+0000 7f8810a1f640 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cbee0eca10 0x55cbee0eee30 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9821a9eeb3b2d3ae server_cookie=a950146aeb9b62cb in_seq=0 out_seq=0 2024-09-11T23:43:50.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.521+0000 7f87effff640 10 client.26683 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:50.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.522+0000 7f87effff640 1 -- 172.21.15.133:0/2045898447 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cbedfd2d00 con 0x55cbee0eca10 2024-09-11T23:43:50.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.522+0000 7f87effff640 10 client.26683 handle_command_reply: tid=0 2024-09-11T23:43:50.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.522+0000 7f881505c740 1 client.26683 shutdown 2024-09-11T23:43:50.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.523+0000 7f881505c740 1 -- 172.21.15.133:0/2045898447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 msgr2=0x55cbee073f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:50.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.523+0000 7f881505c740 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee073f80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f87f80cbde0 tx=0x7f87f809b370 comp rx=0 tx=0).stop 2024-09-11T23:43:50.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.523+0000 7f881505c740 1 -- 172.21.15.133:0/2045898447 shutdown_connections 2024-09-11T23:43:50.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.523+0000 7f881505c740 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cbee0eca10 0x55cbee0eee30 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.523+0000 7f881505c740 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cbee070f50 0x55cbee073f80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.523+0000 7f881505c740 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbee066690 0x55cbee079510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.523+0000 7f881505c740 1 --2- 172.21.15.133:0/2045898447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cbee065d70 0x55cbee078fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.523+0000 7f881505c740 1 -- 172.21.15.133:0/2045898447 >> 172.21.15.133:0/2045898447 conn(0x55cbee062300 msgr2=0x55cbee061410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:50.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.523+0000 7f87effff640 0 client.26683 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:50.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.523+0000 7f881505c740 1 -- 172.21.15.133:0/2045898447 shutdown_connections 2024-09-11T23:43:50.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.523+0000 7f881505c740 1 -- 172.21.15.133:0/2045898447 wait complete. 2024-09-11T23:43:50.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.524+0000 7f881505c740 20 client.26683 trim_cache size 0 max 16384 2024-09-11T23:43:50.528 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:43:50.528 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:43:50.528 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "72a3ff9d-7381-4834-a989-6d6181550c26", 2024-09-11T23:43:50.528 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:43:50.528 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:43:50.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.530+0000 7f8811a21640 1 -- 172.21.15.133:0/2238403845 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87f00a3130 msgr2=0x7f87f00a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:50.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.530+0000 7f8811a21640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87f00a3130 0x7f87f00a55f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f880402af70 tx=0x7f8804000f40 comp rx=0 tx=0).stop 2024-09-11T23:43:50.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.530+0000 7f8811a21640 1 -- 172.21.15.133:0/2238403845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f880c105d90 msgr2=0x7f880c0733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:50.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.530+0000 7f8811a21640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f880c105d90 0x7f880c0733d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f87f800b3f0 tx=0x7f87f800b8b0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.534+0000 7f8811a21640 1 -- 172.21.15.133:0/2238403845 shutdown_connections 2024-09-11T23:43:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.534+0000 7f8811a21640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87f00a3130 0x7f87f00a55f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.534+0000 7f8811a21640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f880c105d90 0x7f880c0733d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.534+0000 7f8811a21640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f880c1012a0 0x7f880c0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.534+0000 7f8811a21640 1 --2- 172.21.15.133:0/2238403845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f880c100980 0x7f880c071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.534+0000 7f8811a21640 1 -- 172.21.15.133:0/2238403845 >> 172.21.15.133:0/2238403845 conn(0x7f880c0fc610 msgr2=0x7f880c107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.535+0000 7f8811a21640 1 -- 172.21.15.133:0/2238403845 shutdown_connections 2024-09-11T23:43:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:50.535+0000 7f8811a21640 1 -- 172.21.15.133:0/2238403845 wait complete. 2024-09-11T23:43:50.789 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:50 smithi133 ceph-mon[31430]: pgmap v217: 97 pgs: 97 active+clean; 60 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 807 MiB/s wr, 2.46k op/s 2024-09-11T23:43:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:50 smithi138 ceph-mon[35142]: pgmap v217: 97 pgs: 97 active+clean; 60 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 807 MiB/s wr, 2.46k op/s 2024-09-11T23:43:51.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:50 smithi178 ceph-mon[35882]: pgmap v217: 97 pgs: 97 active+clean; 60 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 807 MiB/s wr, 2.46k op/s 2024-09-11T23:43:51.637 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:43:51.765 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:51 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:43:51.765 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:51 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:51.765 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:51 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:43:51.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:51 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:43:51.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:51 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:51.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:51 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:43:52.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:51 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:43:52.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:51 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:52.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:51 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:43:52.444 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:52.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:52 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:52.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:52 smithi133 ceph-mon[31430]: pgmap v218: 97 pgs: 97 active+clean; 60 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 597 MiB/s wr, 1.80k op/s 2024-09-11T23:43:53.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:52 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:53.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:52 smithi178 ceph-mon[35882]: pgmap v218: 97 pgs: 97 active+clean; 60 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 597 MiB/s wr, 1.80k op/s 2024-09-11T23:43:53.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:52 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:43:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:52 smithi138 ceph-mon[35142]: pgmap v218: 97 pgs: 97 active+clean; 60 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 597 MiB/s wr, 1.80k op/s 2024-09-11T23:43:53.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.332+0000 7fe09ae7e640 1 -- 172.21.15.133:0/2675283431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0941093c0 msgr2=0x7fe0940742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.332+0000 7fe0937fe640 1 -- 172.21.15.133:0/2675283431 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084035020 con 0x7fe0941093c0 2024-09-11T23:43:53.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.332+0000 7fe09ae7e640 1 --2- 172.21.15.133:0/2675283431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0941093c0 0x7fe0940742d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe08400a910 tx=0x7fe08402a600 comp rx=0 tx=0).stop 2024-09-11T23:43:53.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.333+0000 7fe09ae7e640 1 -- 172.21.15.133:0/2675283431 shutdown_connections 2024-09-11T23:43:53.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.333+0000 7fe09ae7e640 1 --2- 172.21.15.133:0/2675283431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094079440 0x7fe09407b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.333+0000 7fe09ae7e640 1 --2- 172.21.15.133:0/2675283431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe094074810 0x7fe094074c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.333+0000 7fe09ae7e640 1 --2- 172.21.15.133:0/2675283431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0941093c0 0x7fe0940742d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.333+0000 7fe09ae7e640 1 -- 172.21.15.133:0/2675283431 >> 172.21.15.133:0/2675283431 conn(0x7fe09406e140 msgr2=0x7fe09406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:53.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.337+0000 7fe09ae7e640 1 -- 172.21.15.133:0/2675283431 shutdown_connections 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.337+0000 7fe09ae7e640 1 -- 172.21.15.133:0/2675283431 wait complete. 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.338+0000 7fe09ae7e640 1 Processor -- start 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.338+0000 7fe09ae7e640 1 -- start start 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.339+0000 7fe09ae7e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094074810 0x7fe0940804c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.339+0000 7fe09ae7e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe094079440 0x7fe094078080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.339+0000 7fe09ae7e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0941093c0 0x7fe0940785c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.339+0000 7fe09ae7e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe094082b40 con 0x7fe094074810 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.339+0000 7fe09ae7e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0940829c0 con 0x7fe0941093c0 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.339+0000 7fe09ae7e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe094082cc0 con 0x7fe094079440 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.340+0000 7fe098c1a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094074810 0x7fe0940804c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.340+0000 7fe098c1a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094074810 0x7fe0940804c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32978/0 (socket says 172.21.15.133:32978) 2024-09-11T23:43:53.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.340+0000 7fe098c1a640 1 -- 172.21.15.133:0/1190890683 learned_addr learned my addr 172.21.15.133:0/1190890683 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:53.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.342+0000 7fe09941b640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0941093c0 0x7fe0940785c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.342+0000 7fe098c1a640 1 -- 172.21.15.133:0/1190890683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe094079440 msgr2=0x7fe094078080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:43:53.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.342+0000 7fe098c1a640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe094079440 0x7fe094078080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.342+0000 7fe098c1a640 1 -- 172.21.15.133:0/1190890683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0941093c0 msgr2=0x7fe0940785c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.342+0000 7fe098c1a640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0941093c0 0x7fe0940785c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.343+0000 7fe098c1a640 1 -- 172.21.15.133:0/1190890683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe094078d40 con 0x7fe094074810 2024-09-11T23:43:53.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.343+0000 7fe098c1a640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094074810 0x7fe0940804c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe08402abf0 tx=0x7fe0840316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:53.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.344+0000 7fe091ffb640 1 -- 172.21.15.133:0/1190890683 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084040070 con 0x7fe094074810 2024-09-11T23:43:53.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.344+0000 7fe091ffb640 1 -- 172.21.15.133:0/1190890683 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe084031ce0 con 0x7fe094074810 2024-09-11T23:43:53.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.344+0000 7fe09ae7e640 1 -- 172.21.15.133:0/1190890683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe094137290 con 0x7fe094074810 2024-09-11T23:43:53.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.345+0000 7fe091ffb640 1 -- 172.21.15.133:0/1190890683 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084035030 con 0x7fe094074810 2024-09-11T23:43:53.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.345+0000 7fe09ae7e640 1 -- 172.21.15.133:0/1190890683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0941376b0 con 0x7fe094074810 2024-09-11T23:43:53.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.353+0000 7fe09d257740 1 Processor -- start 2024-09-11T23:43:53.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.354+0000 7fe09d257740 1 -- start start 2024-09-11T23:43:53.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.354+0000 7fe09d257740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0f4c0 0x558164f0f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.355+0000 7fe09d257740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 0x558164f1a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.355+0000 7fe09d257740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f1a6a0 0x558164f1ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.355+0000 7fe09d257740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558164e752e0 con 0x558164f1a6a0 2024-09-11T23:43:53.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.355+0000 7fe09d257740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558164e1cce0 con 0x558164f0fde0 2024-09-11T23:43:53.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.355+0000 7fe09d257740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558164e712a0 con 0x558164f0f4c0 2024-09-11T23:43:53.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.357+0000 7fe091ffb640 1 -- 172.21.15.133:0/1190890683 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe084043070 con 0x7fe094074810 2024-09-11T23:43:53.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.359+0000 7fe091ffb640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe07c0a2f00 0x7fe07c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.359+0000 7fe091ffb640 1 -- 172.21.15.133:0/1190890683 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fe0840f0140 con 0x7fe094074810 2024-09-11T23:43:53.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.360+0000 7fe098c1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 0x558164f1a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.360+0000 7fe098c1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 0x558164f1a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48060/0 (socket says 172.21.15.133:48060) 2024-09-11T23:43:53.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.360+0000 7fe098c1a640 1 -- 172.21.15.133:0/3252533309 learned_addr learned my addr 172.21.15.133:0/3252533309 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:53.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.360+0000 7fe09941b640 1 --2- 172.21.15.133:0/3252533309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0f4c0 0x558164f0f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.362+0000 7fe09941b640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe07c0a2f00 0x7fe07c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.364+0000 7fe098c1a640 1 -- 172.21.15.133:0/3252533309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0f4c0 msgr2=0x558164f0f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.364+0000 7fe098c1a640 1 --2- 172.21.15.133:0/3252533309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0f4c0 0x558164f0f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.364+0000 7fe098c1a640 1 -- 172.21.15.133:0/3252533309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f1a6a0 msgr2=0x558164f1ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.364+0000 7fe098c1a640 1 --2- 172.21.15.133:0/3252533309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f1a6a0 0x558164f1ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.364+0000 7fe098c1a640 1 -- 172.21.15.133:0/3252533309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558164e01440 con 0x558164f0fde0 2024-09-11T23:43:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.366+0000 7fe098c1a640 1 --2- 172.21.15.133:0/3252533309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 0x558164f1a160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0840b9010 tx=0x7fe08400a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=329b639b24e3d303 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.367+0000 7fe07affd640 1 -- 172.21.15.133:0/3252533309 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840bcd40 con 0x558164f0fde0 2024-09-11T23:43:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.367+0000 7fe07affd640 1 -- 172.21.15.133:0/3252533309 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0840bc640 con 0x558164f0fde0 2024-09-11T23:43:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.367+0000 7fe07affd640 1 -- 172.21.15.133:0/3252533309 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840b8e40 con 0x558164f0fde0 2024-09-11T23:43:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.368+0000 7fe09d257740 1 -- 172.21.15.133:0/3252533309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 msgr2=0x558164f1a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.368+0000 7fe09d257740 1 --2- 172.21.15.133:0/3252533309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 0x558164f1a160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0840b9010 tx=0x7fe08400a7a0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.368+0000 7fe09d257740 1 -- 172.21.15.133:0/3252533309 shutdown_connections 2024-09-11T23:43:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.368+0000 7fe09d257740 1 --2- 172.21.15.133:0/3252533309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f1a6a0 0x558164f1ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.368+0000 7fe09d257740 1 --2- 172.21.15.133:0/3252533309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 0x558164f1a160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.368+0000 7fe09d257740 1 --2- 172.21.15.133:0/3252533309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0f4c0 0x558164f0f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.368+0000 7fe09d257740 1 -- 172.21.15.133:0/3252533309 >> 172.21.15.133:0/3252533309 conn(0x558164f0ba50 msgr2=0x558164f0be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.368+0000 7fe09941b640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe07c0a2f00 0x7fe07c0a53c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe094080c80 tx=0x7fe08c0098a0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.370+0000 7fe09d257740 1 -- 172.21.15.133:0/3252533309 shutdown_connections 2024-09-11T23:43:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.371+0000 7fe09d257740 1 -- 172.21.15.133:0/3252533309 wait complete. 2024-09-11T23:43:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.372+0000 7fe09d257740 1 -- start start 2024-09-11T23:43:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.372+0000 7fe09d257740 1 Processor -- start 2024-09-11T23:43:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.373+0000 7fe09d257740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0f4c0 0x558164f1a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.373+0000 7fe09d257740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0fde0 0x558164f172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.373+0000 7fe09d257740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f1a6a0 0x558164f17820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.373+0000 7fe09d257740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558164f1d030 con 0x558164f0fde0 2024-09-11T23:43:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.373+0000 7fe09d257740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558164e30050 con 0x558164f0f4c0 2024-09-11T23:43:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.373+0000 7fe09d257740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558164f1d1b0 con 0x558164f1a6a0 2024-09-11T23:43:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.373+0000 7fe09d257740 10 client.0 _ll_register_callbacks cb 0x558164e70160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:53.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.379+0000 7fe098c1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0f4c0 0x558164f1a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.379+0000 7fe098c1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0f4c0 0x558164f1a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48074/0 (socket says 172.21.15.133:48074) 2024-09-11T23:43:53.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.379+0000 7fe098c1a640 1 -- 172.21.15.133:0/3977526252 learned_addr learned my addr 172.21.15.133:0/3977526252 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:53.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.379+0000 7fe09941b640 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f1a6a0 0x558164f17820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.383+0000 7fe093fff640 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0fde0 0x558164f172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.384+0000 7fe093fff640 1 -- 172.21.15.133:0/3977526252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f1a6a0 msgr2=0x558164f17820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.384+0000 7fe093fff640 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f1a6a0 0x558164f17820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.384+0000 7fe093fff640 1 -- 172.21.15.133:0/3977526252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0f4c0 msgr2=0x558164f1a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.384+0000 7fe093fff640 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0f4c0 0x558164f1a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.384+0000 7fe093fff640 1 -- 172.21.15.133:0/3977526252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558164be63c0 con 0x558164f0fde0 2024-09-11T23:43:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.384+0000 7fe093fff640 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0fde0 0x558164f172c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe0880057c0 tx=0x7fe08800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe7946e51e947f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.384+0000 7fe073fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:53.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.385+0000 7fe073fff640 1 -- 172.21.15.133:0/3977526252 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088004020 con 0x558164f0fde0 2024-09-11T23:43:53.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.385+0000 7fe073fff640 1 -- 172.21.15.133:0/3977526252 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fe0880041c0 con 0x558164f0fde0 2024-09-11T23:43:53.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.385+0000 7fe09d257740 1 -- 172.21.15.133:0/3977526252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0840bc640 con 0x558164f0fde0 2024-09-11T23:43:53.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.386+0000 7fe073fff640 1 -- 172.21.15.133:0/3977526252 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0880043c0 con 0x558164f0fde0 2024-09-11T23:43:53.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.386+0000 7fe09d257740 10 client.17322 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:53.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.386+0000 7fe09d257740 1 -- 172.21.15.133:0/3977526252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558164f289d0 con 0x558164f0fde0 2024-09-11T23:43:53.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.390+0000 7fe073fff640 1 -- 172.21.15.133:0/3977526252 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0840bc640 con 0x558164f0fde0 2024-09-11T23:43:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.390+0000 7fe073fff640 1 -- 172.21.15.133:0/3977526252 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe0880035f0 con 0x558164f0fde0 2024-09-11T23:43:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.390+0000 7fe09d257740 10 client.17322 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.390+0000 7fe09d257740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.390+0000 7fe09d257740 20 client.17322 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.390+0000 7fe09d257740 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558164f29ee0 0x558164f2c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.390+0000 7fe09d257740 4 client.17322 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.390+0000 7fe09d257740 1 -- 172.21.15.133:0/3977526252 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558164e82850 con 0x558164f29ee0 2024-09-11T23:43:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.391+0000 7fe098c1a640 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558164f29ee0 0x558164f2c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.391+0000 7fe073fff640 10 client.17322 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.391+0000 7fe098c1a640 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558164f29ee0 0x558164f2c360 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a765686b47c1a8d3 server_cookie=8de3ea86a4e6fb77 in_seq=0 out_seq=0 2024-09-11T23:43:53.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.392+0000 7fe073fff640 1 -- 172.21.15.133:0/3977526252 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558164e82850 con 0x558164f29ee0 2024-09-11T23:43:53.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.392+0000 7fe073fff640 10 client.17322 handle_command_reply: tid=0 2024-09-11T23:43:53.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.392+0000 7fe09d257740 1 client.17322 shutdown 2024-09-11T23:43:53.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe09d257740 1 -- 172.21.15.133:0/3977526252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0fde0 msgr2=0x558164f172c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe09d257740 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0fde0 0x558164f172c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe0880057c0 tx=0x7fe08800bf90 comp rx=0 tx=0).stop 2024-09-11T23:43:53.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe09d257740 1 -- 172.21.15.133:0/3977526252 shutdown_connections 2024-09-11T23:43:53.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe09d257740 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558164f29ee0 0x558164f2c360 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe09d257740 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f1a6a0 0x558164f17820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe09d257740 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0fde0 0x558164f172c0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe09d257740 1 --2- 172.21.15.133:0/3977526252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0f4c0 0x558164f1a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe09d257740 1 -- 172.21.15.133:0/3977526252 >> 172.21.15.133:0/3977526252 conn(0x558164f0ba50 msgr2=0x558164f0ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:53.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe073fff640 0 client.17322 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:53.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe09d257740 1 -- 172.21.15.133:0/3977526252 shutdown_connections 2024-09-11T23:43:53.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe09d257740 1 -- 172.21.15.133:0/3977526252 wait complete. 2024-09-11T23:43:53.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.393+0000 7fe09d257740 20 client.17322 trim_cache size 0 max 16384 2024-09-11T23:43:53.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.462+0000 7fe09d257740 1 Processor -- start 2024-09-11T23:43:53.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.462+0000 7fe09d257740 1 -- start start 2024-09-11T23:43:53.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.463+0000 7fe09d257740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 0x558164f26140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.464+0000 7fe09d257740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0fde0 0x558164f31c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.464+0000 7fe09d257740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f1a6a0 0x558164f32170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.464+0000 7fe09d257740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941d5ce0 con 0x558164f0f4c0 2024-09-11T23:43:53.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.464+0000 7fe09d257740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941d7820 con 0x558164f1a6a0 2024-09-11T23:43:53.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.464+0000 7fe09d257740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941d63b0 con 0x558164f0fde0 2024-09-11T23:43:53.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.466+0000 7fe093fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0fde0 0x558164f31c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.466+0000 7fe093fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0fde0 0x558164f31c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:41910/0 (socket says 172.21.15.133:41910) 2024-09-11T23:43:53.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.466+0000 7fe093fff640 1 -- 172.21.15.133:0/2345004407 learned_addr learned my addr 172.21.15.133:0/2345004407 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:53.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.467+0000 7fe09941b640 1 --2- 172.21.15.133:0/2345004407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f1a6a0 0x558164f32170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.467+0000 7fe098c1a640 1 --2- 172.21.15.133:0/2345004407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 0x558164f26140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.468+0000 7fe093fff640 1 -- 172.21.15.133:0/2345004407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f1a6a0 msgr2=0x558164f32170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.468+0000 7fe093fff640 1 --2- 172.21.15.133:0/2345004407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f1a6a0 0x558164f32170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.468+0000 7fe093fff640 1 -- 172.21.15.133:0/2345004407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 msgr2=0x558164f26140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.468+0000 7fe093fff640 1 --2- 172.21.15.133:0/2345004407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 0x558164f26140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.468+0000 7fe093fff640 1 -- 172.21.15.133:0/2345004407 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558164f0f2f0 con 0x558164f0fde0 2024-09-11T23:43:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.470+0000 7fe093fff640 1 --2- 172.21.15.133:0/2345004407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0fde0 0x558164f31c30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe08800b670 tx=0x7fe088005cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2c3b6caf31fce4d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.470+0000 7fe07a7fc640 1 -- 172.21.15.133:0/2345004407 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088004210 con 0x558164f0fde0 2024-09-11T23:43:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.470+0000 7fe07a7fc640 1 -- 172.21.15.133:0/2345004407 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0880054e0 con 0x558164f0fde0 2024-09-11T23:43:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.471+0000 7fe098c1a640 1 --2- 172.21.15.133:0/2345004407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 0x558164f26140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:43:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.471+0000 7fe07a7fc640 1 -- 172.21.15.133:0/2345004407 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0880049d0 con 0x558164f0fde0 2024-09-11T23:43:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.472+0000 7fe09d257740 1 -- 172.21.15.133:0/2345004407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0fde0 msgr2=0x558164f31c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.472+0000 7fe09d257740 1 --2- 172.21.15.133:0/2345004407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0fde0 0x558164f31c30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe08800b670 tx=0x7fe088005cd0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.472+0000 7fe09d257740 1 -- 172.21.15.133:0/2345004407 shutdown_connections 2024-09-11T23:43:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.472+0000 7fe09d257740 1 --2- 172.21.15.133:0/2345004407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f1a6a0 0x558164f32170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.472+0000 7fe09d257740 1 --2- 172.21.15.133:0/2345004407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f0fde0 0x558164f31c30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.472+0000 7fe09d257740 1 --2- 172.21.15.133:0/2345004407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 0x558164f26140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.472+0000 7fe09d257740 1 -- 172.21.15.133:0/2345004407 >> 172.21.15.133:0/2345004407 conn(0x558164f0ba50 msgr2=0x558164f12920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.475+0000 7fe09d257740 1 -- 172.21.15.133:0/2345004407 shutdown_connections 2024-09-11T23:43:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.475+0000 7fe09d257740 1 -- 172.21.15.133:0/2345004407 wait complete. 2024-09-11T23:43:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.476+0000 7fe09d257740 1 -- start start 2024-09-11T23:43:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.476+0000 7fe09d257740 1 Processor -- start 2024-09-11T23:43:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.477+0000 7fe09d257740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 0x558164f22490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.478+0000 7fe09d257740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 0x558164f229d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.478+0000 7fe09d257740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f1a6a0 0x558164f13ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.487+0000 7fe09941b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f1a6a0 0x558164f13ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.487+0000 7fe098c1a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 0x558164f22490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.488+0000 7fe098c1a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 0x558164f22490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33014/0 (socket says 172.21.15.133:33014) 2024-09-11T23:43:53.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.488+0000 7fe098c1a640 1 -- 172.21.15.133:0/2230055231 learned_addr learned my addr 172.21.15.133:0/2230055231 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:53.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.487+0000 7fe093fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 0x558164f229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.488+0000 7fe09d257740 1 -- 172.21.15.133:0/2230055231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941d7150 con 0x558164f0f4c0 2024-09-11T23:43:53.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.488+0000 7fe09d257740 1 -- 172.21.15.133:0/2230055231 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941d6a80 con 0x558164f0fde0 2024-09-11T23:43:53.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.488+0000 7fe09d257740 1 -- 172.21.15.133:0/2230055231 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941d3ce0 con 0x558164f1a6a0 2024-09-11T23:43:53.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.488+0000 7fe09d257740 10 client.0 _ll_register_callbacks cb 0x7fe08800b5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:53.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.491+0000 7fe098c1a640 1 -- 172.21.15.133:0/2230055231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f1a6a0 msgr2=0x558164f13ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.491+0000 7fe098c1a640 1 --2- 172.21.15.133:0/2230055231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f1a6a0 0x558164f13ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.491+0000 7fe098c1a640 1 -- 172.21.15.133:0/2230055231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 msgr2=0x558164f229d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.491+0000 7fe098c1a640 1 --2- 172.21.15.133:0/2230055231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 0x558164f229d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.491+0000 7fe098c1a640 1 -- 172.21.15.133:0/2230055231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558164f141c0 con 0x558164f0f4c0 2024-09-11T23:43:53.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.494+0000 7fe098c1a640 1 --2- 172.21.15.133:0/2230055231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 0x558164f22490 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe0840c4d80 tx=0x7fe0840b8220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d69426cdd6c60ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:53.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.494+0000 7fe073fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:53.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.495+0000 7fe073fff640 1 -- 172.21.15.133:0/2230055231 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840da060 con 0x558164f0f4c0 2024-09-11T23:43:53.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.496+0000 7fe09d257740 1 -- 172.21.15.133:0/2230055231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0880054e0 con 0x558164f0f4c0 2024-09-11T23:43:53.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.497+0000 7fe073fff640 1 -- 172.21.15.133:0/2230055231 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fe0840edaf0 con 0x558164f0f4c0 2024-09-11T23:43:53.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.499+0000 7fe073fff640 1 -- 172.21.15.133:0/2230055231 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0840ca050 con 0x558164f0f4c0 2024-09-11T23:43:53.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.499+0000 7fe09d257740 10 client.17325 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:53.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.499+0000 7fe09d257740 1 -- 172.21.15.133:0/2230055231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558164f945c0 con 0x558164f0f4c0 2024-09-11T23:43:53.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.501+0000 7fe073fff640 1 -- 172.21.15.133:0/2230055231 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0880054e0 con 0x558164f0f4c0 2024-09-11T23:43:53.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.502+0000 7fe073fff640 1 -- 172.21.15.133:0/2230055231 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe0840cd020 con 0x558164f0f4c0 2024-09-11T23:43:53.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.503+0000 7fe09d257740 10 client.17325 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:53.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.503+0000 7fe09d257740 10 client.17325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:53.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.503+0000 7fe09d257740 20 client.17325 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:53.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.503+0000 7fe09d257740 1 --2- 172.21.15.133:0/2230055231 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558164f94cc0 0x558164f970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.503+0000 7fe09d257740 4 client.17325 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:43:53.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.503+0000 7fe09d257740 1 -- 172.21.15.133:0/2230055231 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558164e7cd60 con 0x558164f94cc0 2024-09-11T23:43:53.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.506+0000 7fe093fff640 1 --2- 172.21.15.133:0/2230055231 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558164f94cc0 0x558164f970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.514+0000 7fe093fff640 1 --2- 172.21.15.133:0/2230055231 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558164f94cc0 0x558164f970e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=683823cdda8e350c server_cookie=5267f157028ca346 in_seq=0 out_seq=0 2024-09-11T23:43:53.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.514+0000 7fe073fff640 10 client.17325 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:53.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.515+0000 7fe073fff640 1 -- 172.21.15.133:0/2230055231 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558164e7cd60 con 0x558164f94cc0 2024-09-11T23:43:53.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.515+0000 7fe073fff640 10 client.17325 handle_command_reply: tid=0 2024-09-11T23:43:53.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.515+0000 7fe09d257740 1 client.17325 shutdown 2024-09-11T23:43:53.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.516+0000 7fe09d257740 1 -- 172.21.15.133:0/2230055231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 msgr2=0x558164f22490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.516+0000 7fe09d257740 1 --2- 172.21.15.133:0/2230055231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 0x558164f22490 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe0840c4d80 tx=0x7fe0840b8220 comp rx=0 tx=0).stop 2024-09-11T23:43:53.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.516+0000 7fe09d257740 1 -- 172.21.15.133:0/2230055231 shutdown_connections 2024-09-11T23:43:53.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.516+0000 7fe09d257740 1 --2- 172.21.15.133:0/2230055231 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558164f94cc0 0x558164f970e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.516+0000 7fe09d257740 1 --2- 172.21.15.133:0/2230055231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558164f1a6a0 0x558164f13ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.516+0000 7fe09d257740 1 --2- 172.21.15.133:0/2230055231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164f0fde0 0x558164f229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.516+0000 7fe09d257740 1 --2- 172.21.15.133:0/2230055231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558164f0f4c0 0x558164f22490 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.516+0000 7fe09d257740 1 -- 172.21.15.133:0/2230055231 >> 172.21.15.133:0/2230055231 conn(0x558164f0ba50 msgr2=0x558164f18b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:53.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.516+0000 7fe073fff640 0 client.17325 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:53.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.516+0000 7fe09d257740 1 -- 172.21.15.133:0/2230055231 shutdown_connections 2024-09-11T23:43:53.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.517+0000 7fe09d257740 1 -- 172.21.15.133:0/2230055231 wait complete. 2024-09-11T23:43:53.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.517+0000 7fe09d257740 20 client.17325 trim_cache size 0 max 16384 2024-09-11T23:43:53.519 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:43:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:43:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-09-11T23:43:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: "72a3ff9d-7381-4834-a989-6d6181550c26": { 2024-09-11T23:43:53.519 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-09-11T23:43:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "72a3ff9d-7381-4834-a989-6d6181550c26", 2024-09-11T23:43:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2024-09-11T23:43:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:43:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:43:53.520 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:43:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.520+0000 7fe09ae7e640 1 -- 172.21.15.133:0/1190890683 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe07c0a2f00 msgr2=0x7fe07c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.520+0000 7fe09ae7e640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe07c0a2f00 0x7fe07c0a53c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe094080c80 tx=0x7fe08c0098a0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.520+0000 7fe09ae7e640 1 -- 172.21.15.133:0/1190890683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094074810 msgr2=0x7fe0940804c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.520+0000 7fe09ae7e640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094074810 0x7fe0940804c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe08402abf0 tx=0x7fe0840316b0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.521+0000 7fe09ae7e640 1 -- 172.21.15.133:0/1190890683 shutdown_connections 2024-09-11T23:43:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.521+0000 7fe09ae7e640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe07c0a2f00 0x7fe07c0a53c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.521+0000 7fe09ae7e640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0941093c0 0x7fe0940785c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.521+0000 7fe09ae7e640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe094079440 0x7fe094078080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.521+0000 7fe09ae7e640 1 --2- 172.21.15.133:0/1190890683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094074810 0x7fe0940804c0 secure :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe08402abf0 tx=0x7fe0840316b0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.521+0000 7fe09ae7e640 1 -- 172.21.15.133:0/1190890683 >> 172.21.15.133:0/1190890683 conn(0x7fe09406e140 msgr2=0x7fe09407acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:53.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.521+0000 7fe09ae7e640 1 -- 172.21.15.133:0/1190890683 shutdown_connections 2024-09-11T23:43:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:53.528+0000 7fe09ae7e640 1 -- 172.21.15.133:0/1190890683 wait complete. 2024-09-11T23:43:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.846+0000 7f57a1d9b640 1 -- 172.21.15.133:0/1859744405 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c031db0 con 0x7f57a41091f0 2024-09-11T23:43:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.848+0000 7f57a918d640 1 -- 172.21.15.133:0/1859744405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57a41091f0 msgr2=0x7f57a4110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.848+0000 7f57a918d640 1 --2- 172.21.15.133:0/1859744405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57a41091f0 0x7f57a4110250 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f579c009ca0 tx=0x7f579c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.848+0000 7f57a918d640 1 -- 172.21.15.133:0/1859744405 shutdown_connections 2024-09-11T23:43:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.848+0000 7f57a918d640 1 --2- 172.21.15.133:0/1859744405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a411b140 0x7f57a411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.848+0000 7f57a918d640 1 --2- 172.21.15.133:0/1859744405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a41095d0 0x7f57a4109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.848+0000 7f57a918d640 1 --2- 172.21.15.133:0/1859744405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57a41091f0 0x7f57a4110250 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.848+0000 7f57a918d640 1 -- 172.21.15.133:0/1859744405 >> 172.21.15.133:0/1859744405 conn(0x7f57a406ce00 msgr2=0x7f57a406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.848+0000 7f57a918d640 1 -- 172.21.15.133:0/1859744405 shutdown_connections 2024-09-11T23:43:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.848+0000 7f57a918d640 1 -- 172.21.15.133:0/1859744405 wait complete. 2024-09-11T23:43:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.849+0000 7f57a918d640 1 Processor -- start 2024-09-11T23:43:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.850+0000 7f57a918d640 1 -- start start 2024-09-11T23:43:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.850+0000 7f57a918d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a41091f0 0x7f57a41a7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.850+0000 7f57a918d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57a41095d0 0x7f57a41a7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.850+0000 7f57a918d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a411b140 0x7f57a41ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.850+0000 7f57a918d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4120cf0 con 0x7f57a41095d0 2024-09-11T23:43:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.850+0000 7f57a918d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4120b70 con 0x7f57a41091f0 2024-09-11T23:43:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.851+0000 7f57a918d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4120e70 con 0x7f57a411b140 2024-09-11T23:43:53.854 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:53 smithi133 ceph-mon[31430]: pgmap v219: 97 pgs: 97 active+clean; 60 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 778 MiB/s wr, 2.59k op/s 2024-09-11T23:43:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.853+0000 7f57a359e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a411b140 0x7f57a41ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.853+0000 7f57a359e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a411b140 0x7f57a41ab9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:41924/0 (socket says 172.21.15.133:41924) 2024-09-11T23:43:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.853+0000 7f57a359e640 1 -- 172.21.15.133:0/3783782121 learned_addr learned my addr 172.21.15.133:0/3783782121 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.854+0000 7f57a359e640 1 -- 172.21.15.133:0/3783782121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a41091f0 msgr2=0x7f57a41a7100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:43:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.854+0000 7f57a359e640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a41091f0 0x7f57a41a7100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.854+0000 7f57a359e640 1 -- 172.21.15.133:0/3783782121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57a41095d0 msgr2=0x7f57a41a7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.854+0000 7f57a359e640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57a41095d0 0x7f57a41a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.855+0000 7f57a359e640 1 -- 172.21.15.133:0/3783782121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a41ac0c0 con 0x7f57a411b140 2024-09-11T23:43:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.856+0000 7f57a359e640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a411b140 0x7f57a41ab9c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f579800d780 tx=0x7f579800dc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.861+0000 7f578bfff640 1 -- 172.21.15.133:0/3783782121 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798017070 con 0x7f57a411b140 2024-09-11T23:43:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.861+0000 7f578bfff640 1 -- 172.21.15.133:0/3783782121 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579800de90 con 0x7f57a411b140 2024-09-11T23:43:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.863+0000 7f578bfff640 1 -- 172.21.15.133:0/3783782121 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798004d60 con 0x7f57a411b140 2024-09-11T23:43:53.867 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.864+0000 7f57a918d640 1 -- 172.21.15.133:0/3783782121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a41ac3b0 con 0x7f57a411b140 2024-09-11T23:43:53.867 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.864+0000 7f57a918d640 1 -- 172.21.15.133:0/3783782121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a4110920 con 0x7f57a411b140 2024-09-11T23:43:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.869+0000 7f57ac004740 1 Processor -- start 2024-09-11T23:43:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.870+0000 7f57ac004740 1 -- start start 2024-09-11T23:43:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.882+0000 7f57ac004740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e05af0 0x558e88e05ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.882+0000 7f57ac004740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e06410 0x558e88e10770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.882+0000 7f57ac004740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e10cb0 0x558e88e130a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.882+0000 7f57ac004740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558e88d6a2f0 con 0x558e88e05af0 2024-09-11T23:43:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.882+0000 7f57ac004740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558e88d12270 con 0x558e88e10cb0 2024-09-11T23:43:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.882+0000 7f57ac004740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558e88d662b0 con 0x558e88e06410 2024-09-11T23:43:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.883+0000 7f578bfff640 1 -- 172.21.15.133:0/3783782121 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5798009bd0 con 0x7f57a411b140 2024-09-11T23:43:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.885+0000 7f578bfff640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f578c0a2e20 0x7f578c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.885+0000 7f578bfff640 1 -- 172.21.15.133:0/3783782121 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f57980cf0c0 con 0x7f57a411b140 2024-09-11T23:43:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.887+0000 7f57a359e640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f578c0a2e20 0x7f578c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.888+0000 7f57a359e640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f578c0a2e20 0x7f578c0a52e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5798005c20 tx=0x7f5798004680 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.890+0000 7f57a259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e05af0 0x558e88e05ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.890+0000 7f57a259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e05af0 0x558e88e05ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33032/0 (socket says 172.21.15.133:33032) 2024-09-11T23:43:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.890+0000 7f57a259c640 1 -- 172.21.15.133:0/2225110276 learned_addr learned my addr 172.21.15.133:0/2225110276 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.890+0000 7f57a259c640 1 --2- 172.21.15.133:0/2225110276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e10cb0 0x558e88e130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.891+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/2225110276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e06410 0x558e88e10770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.892+0000 7f57a2d9d640 1 -- 172.21.15.133:0/2225110276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e10cb0 msgr2=0x558e88e130a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.892+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/2225110276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e10cb0 0x558e88e130a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.892+0000 7f57a2d9d640 1 -- 172.21.15.133:0/2225110276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e05af0 msgr2=0x558e88e05ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.892+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/2225110276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e05af0 0x558e88e05ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.892+0000 7f57a2d9d640 1 -- 172.21.15.133:0/2225110276 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e88d11340 con 0x558e88e06410 2024-09-11T23:43:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.893+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/2225110276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e06410 0x558e88e10770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f579c000f80 tx=0x7f579c0099b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=172c2bb24d23e31 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.893+0000 7f57897fa640 1 -- 172.21.15.133:0/2225110276 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c0316f0 con 0x558e88e06410 2024-09-11T23:43:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.894+0000 7f57897fa640 1 -- 172.21.15.133:0/2225110276 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579c036050 con 0x558e88e06410 2024-09-11T23:43:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.895+0000 7f57ac004740 1 -- 172.21.15.133:0/2225110276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e06410 msgr2=0x558e88e10770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.895+0000 7f57897fa640 1 -- 172.21.15.133:0/2225110276 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c044410 con 0x558e88e06410 2024-09-11T23:43:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.896+0000 7f57ac004740 1 --2- 172.21.15.133:0/2225110276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e06410 0x558e88e10770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f579c000f80 tx=0x7f579c0099b0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.896+0000 7f57ac004740 1 -- 172.21.15.133:0/2225110276 shutdown_connections 2024-09-11T23:43:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.896+0000 7f57ac004740 1 --2- 172.21.15.133:0/2225110276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e10cb0 0x558e88e130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.896+0000 7f57ac004740 1 --2- 172.21.15.133:0/2225110276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e06410 0x558e88e10770 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f579c000f80 tx=0x7f579c0099b0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.896+0000 7f57ac004740 1 --2- 172.21.15.133:0/2225110276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e05af0 0x558e88e05ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.896+0000 7f57ac004740 1 -- 172.21.15.133:0/2225110276 >> 172.21.15.133:0/2225110276 conn(0x558e88e02060 msgr2=0x558e88e02440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.897+0000 7f57ac004740 1 -- 172.21.15.133:0/2225110276 shutdown_connections 2024-09-11T23:43:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.897+0000 7f57ac004740 1 -- 172.21.15.133:0/2225110276 wait complete. 2024-09-11T23:43:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.897+0000 7f57ac004740 1 -- start start 2024-09-11T23:43:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.898+0000 7f57ac004740 1 Processor -- start 2024-09-11T23:43:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.898+0000 7f57ac004740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0df40 0x558e88e0e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.898+0000 7f57ac004740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e0e860 0x558e88e1ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.898+0000 7f57ac004740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e1b340 0x558e88e1d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.898+0000 7f57ac004740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558e88e138d0 con 0x558e88e0df40 2024-09-11T23:43:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.898+0000 7f57ac004740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558e88cf3290 con 0x558e88e0e860 2024-09-11T23:43:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.898+0000 7f57ac004740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558e88e13a50 con 0x558e88e1b340 2024-09-11T23:43:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.898+0000 7f57ac004740 10 client.0 _ll_register_callbacks cb 0x558e88d24140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.900+0000 7f57a2d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e0e860 0x558e88e1ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.900+0000 7f57a2d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e0e860 0x558e88e1ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48116/0 (socket says 172.21.15.133:48116) 2024-09-11T23:43:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.900+0000 7f57a2d9d640 1 -- 172.21.15.133:0/1482710369 learned_addr learned my addr 172.21.15.133:0/1482710369 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.900+0000 7f57a259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e1b340 0x558e88e1d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.900+0000 7f57a259c640 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0df40 0x558e88e0e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.903+0000 7f57a2d9d640 1 -- 172.21.15.133:0/1482710369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e1b340 msgr2=0x558e88e1d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.904+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e1b340 0x558e88e1d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.904+0000 7f57a2d9d640 1 -- 172.21.15.133:0/1482710369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0df40 msgr2=0x558e88e0e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.904+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0df40 0x558e88e0e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.904+0000 7f57a2d9d640 1 -- 172.21.15.133:0/1482710369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e88a7d210 con 0x558e88e0e860 2024-09-11T23:43:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.904+0000 7f57a259c640 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0df40 0x558e88e0e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.906+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e0e860 0x558e88e1ae00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f579c030a60 tx=0x7f579c030810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae31fbea6e5e9128 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.906+0000 7f5786ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:43:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.909+0000 7f5786ff5640 1 -- 172.21.15.133:0/1482710369 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c052060 con 0x558e88e0e860 2024-09-11T23:43:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.910+0000 7f5786ff5640 1 -- 172.21.15.133:0/1482710369 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f579c0319b0 con 0x558e88e0e860 2024-09-11T23:43:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.910+0000 7f57ac004740 1 -- 172.21.15.133:0/1482710369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f579c036050 con 0x558e88e0e860 2024-09-11T23:43:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.910+0000 7f5786ff5640 1 -- 172.21.15.133:0/1482710369 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f579c031bb0 con 0x558e88e0e860 2024-09-11T23:43:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.911+0000 7f57ac004740 10 client.26828 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.911+0000 7f57ac004740 1 -- 172.21.15.133:0/1482710369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e88b49620 con 0x558e88e0e860 2024-09-11T23:43:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.912+0000 7f5786ff5640 1 -- 172.21.15.133:0/1482710369 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579c036050 con 0x558e88e0e860 2024-09-11T23:43:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.912+0000 7f5786ff5640 1 -- 172.21.15.133:0/1482710369 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f579c0043e0 con 0x558e88e0e860 2024-09-11T23:43:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.912+0000 7f57ac004740 10 client.26828 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.912+0000 7f57ac004740 10 client.26828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.912+0000 7f57ac004740 20 client.26828 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.912+0000 7f57ac004740 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558e88e21aa0 0x558e88e23e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.912+0000 7f57ac004740 4 client.26828 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.912+0000 7f57ac004740 1 -- 172.21.15.133:0/1482710369 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e88d77a20 con 0x558e88e21aa0 2024-09-11T23:43:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.916+0000 7f57a259c640 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558e88e21aa0 0x558e88e23e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.917+0000 7f57a259c640 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558e88e21aa0 0x558e88e23e90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff29203fcd6f5a3 server_cookie=683f685514b95abf in_seq=0 out_seq=0 2024-09-11T23:43:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.917+0000 7f5786ff5640 10 client.26828 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.923+0000 7f5786ff5640 1 -- 172.21.15.133:0/1482710369 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e88d77a20 con 0x558e88e21aa0 2024-09-11T23:43:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.924+0000 7f5786ff5640 10 client.26828 handle_command_reply: tid=0 2024-09-11T23:43:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.924+0000 7f57ac004740 1 client.26828 shutdown 2024-09-11T23:43:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.930+0000 7f57ac004740 1 -- 172.21.15.133:0/1482710369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e0e860 msgr2=0x558e88e1ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.930+0000 7f57ac004740 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e0e860 0x558e88e1ae00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f579c030a60 tx=0x7f579c030810 comp rx=0 tx=0).stop 2024-09-11T23:43:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.930+0000 7f57ac004740 1 -- 172.21.15.133:0/1482710369 shutdown_connections 2024-09-11T23:43:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.930+0000 7f57ac004740 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558e88e21aa0 0x558e88e23e90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.930+0000 7f57ac004740 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e1b340 0x558e88e1d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.930+0000 7f57ac004740 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e0e860 0x558e88e1ae00 secure :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f579c030a60 tx=0x7f579c030810 comp rx=0 tx=0).stop 2024-09-11T23:43:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.930+0000 7f57ac004740 1 --2- 172.21.15.133:0/1482710369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0df40 0x558e88e0e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.930+0000 7f57ac004740 1 -- 172.21.15.133:0/1482710369 >> 172.21.15.133:0/1482710369 conn(0x558e88e02060 msgr2=0x558e88e07dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.932+0000 7f5786ff5640 0 client.26828 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.941+0000 7f57ac004740 1 -- 172.21.15.133:0/1482710369 shutdown_connections 2024-09-11T23:43:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.947+0000 7f57ac004740 1 -- 172.21.15.133:0/1482710369 wait complete. 2024-09-11T23:43:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:53.947+0000 7f57ac004740 20 client.26828 trim_cache size 0 max 16384 2024-09-11T23:43:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.055+0000 7f57ac004740 1 Processor -- start 2024-09-11T23:43:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.056+0000 7f57ac004740 1 -- start start 2024-09-11T23:43:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.056+0000 7f57ac004740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e0df40 0x558e88e0b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.056+0000 7f57ac004740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e1b340 0x558e88e0b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.057+0000 7f57ac004740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 0x558e88e0f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.057+0000 7f57ac004740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4118f50 con 0x558e88e0bc60 2024-09-11T23:43:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.057+0000 7f57ac004740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41c65f0 con 0x558e88e1b340 2024-09-11T23:43:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.057+0000 7f57ac004740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41c5180 con 0x558e88e0df40 2024-09-11T23:43:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.061+0000 7f57a2d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e1b340 0x558e88e0b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.061+0000 7f57a259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e0df40 0x558e88e0b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.062+0000 7f57a259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 0x558e88e0f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.062+0000 7f57a259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 0x558e88e0f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33040/0 (socket says 172.21.15.133:33040) 2024-09-11T23:43:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.062+0000 7f57a259c640 1 -- 172.21.15.133:0/718454151 learned_addr learned my addr 172.21.15.133:0/718454151 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.067+0000 7f57a2d9d640 1 -- 172.21.15.133:0/718454151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e0df40 msgr2=0x558e88e0b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.067+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/718454151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e0df40 0x558e88e0b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.067+0000 7f57a2d9d640 1 -- 172.21.15.133:0/718454151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 msgr2=0x558e88e0f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.067+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/718454151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 0x558e88e0f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.067+0000 7f57a2d9d640 1 -- 172.21.15.133:0/718454151 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e88e0e690 con 0x558e88e1b340 2024-09-11T23:43:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.071+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/718454151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e1b340 0x558e88e0b720 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f579c03b000 tx=0x7f579c009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b512e815045cebc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.091+0000 7f5788ff9640 1 -- 172.21.15.133:0/718454151 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c052040 con 0x558e88e1b340 2024-09-11T23:43:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.091+0000 7f5788ff9640 1 -- 172.21.15.133:0/718454151 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579c04a070 con 0x558e88e1b340 2024-09-11T23:43:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.091+0000 7f5788ff9640 1 -- 172.21.15.133:0/718454151 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c05c080 con 0x558e88e1b340 2024-09-11T23:43:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.094+0000 7f57ac004740 1 -- 172.21.15.133:0/718454151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e1b340 msgr2=0x558e88e0b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.095+0000 7f57ac004740 1 --2- 172.21.15.133:0/718454151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e1b340 0x558e88e0b720 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f579c03b000 tx=0x7f579c009890 comp rx=0 tx=0).stop 2024-09-11T23:43:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.095+0000 7f57ac004740 1 -- 172.21.15.133:0/718454151 shutdown_connections 2024-09-11T23:43:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.095+0000 7f57ac004740 1 --2- 172.21.15.133:0/718454151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 0x558e88e0f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.095+0000 7f57ac004740 1 --2- 172.21.15.133:0/718454151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e1b340 0x558e88e0b720 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.095+0000 7f57ac004740 1 --2- 172.21.15.133:0/718454151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e0df40 0x558e88e0b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.095+0000 7f57ac004740 1 -- 172.21.15.133:0/718454151 >> 172.21.15.133:0/718454151 conn(0x558e88e02060 msgr2=0x558e88e09110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.098+0000 7f57ac004740 1 -- 172.21.15.133:0/718454151 shutdown_connections 2024-09-11T23:43:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.099+0000 7f57ac004740 1 -- 172.21.15.133:0/718454151 wait complete. 2024-09-11T23:43:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.100+0000 7f57ac004740 1 -- start start 2024-09-11T23:43:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.100+0000 7f57ac004740 1 Processor -- start 2024-09-11T23:43:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.101+0000 7f57ac004740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 0x558e88e184e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.101+0000 7f57ac004740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e0df40 0x558e88e18a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.102+0000 7f57ac004740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e1b340 0x558e88e18f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.102+0000 7f57ac004740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41c5f20 con 0x558e88e0bc60 2024-09-11T23:43:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.102+0000 7f57ac004740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41c5850 con 0x558e88e1b340 2024-09-11T23:43:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.102+0000 7f57ac004740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4116f50 con 0x558e88e0df40 2024-09-11T23:43:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.102+0000 7f57ac004740 10 client.0 _ll_register_callbacks cb 0x7f579c03e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.102+0000 7f57a259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 0x558e88e184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.102+0000 7f57a2d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e0df40 0x558e88e18a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.102+0000 7f57a259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 0x558e88e184e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33044/0 (socket says 172.21.15.133:33044) 2024-09-11T23:43:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.102+0000 7f57a259c640 1 -- 172.21.15.133:0/3731260162 learned_addr learned my addr 172.21.15.133:0/3731260162 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.103+0000 7f57a259c640 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e1b340 0x558e88e18f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.105+0000 7f57a259c640 1 -- 172.21.15.133:0/3731260162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e0df40 msgr2=0x558e88e18a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.105+0000 7f57a259c640 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e0df40 0x558e88e18a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.105+0000 7f57a259c640 1 -- 172.21.15.133:0/3731260162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e1b340 msgr2=0x558e88e18f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.105+0000 7f57a259c640 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e1b340 0x558e88e18f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.105+0000 7f57a259c640 1 -- 172.21.15.133:0/3731260162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e88e06240 con 0x558e88e0bc60 2024-09-11T23:43:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.106+0000 7f57a259c640 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 0x558e88e184e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f579400b950 tx=0x7f5794005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4b7573d6eafe37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.129+0000 7f5786ff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.129+0000 7f5786ff5640 1 -- 172.21.15.133:0/3731260162 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794004250 con 0x558e88e0bc60 2024-09-11T23:43:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.129+0000 7f5786ff5640 1 -- 172.21.15.133:0/3731260162 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f57940043f0 con 0x558e88e0bc60 2024-09-11T23:43:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.131+0000 7f57ac004740 1 -- 172.21.15.133:0/3731260162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f579c04a070 con 0x558e88e0bc60 2024-09-11T23:43:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.132+0000 7f5786ff5640 1 -- 172.21.15.133:0/3731260162 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5794021770 con 0x558e88e0bc60 2024-09-11T23:43:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.136+0000 7f57ac004740 10 client.17340 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.136+0000 7f57ac004740 1 -- 172.21.15.133:0/3731260162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e88e8c800 con 0x558e88e0bc60 2024-09-11T23:43:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.139+0000 7f5786ff5640 1 -- 172.21.15.133:0/3731260162 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f579c04a070 con 0x558e88e0bc60 2024-09-11T23:43:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.141+0000 7f5786ff5640 1 -- 172.21.15.133:0/3731260162 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5794003ae0 con 0x558e88e0bc60 2024-09-11T23:43:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.145+0000 7f57ac004740 10 client.17340 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.145+0000 7f57ac004740 10 client.17340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:43:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.145+0000 7f57ac004740 20 client.17340 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.145+0000 7f57ac004740 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558e88e8ced0 0x558e88e8f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.145+0000 7f57ac004740 4 client.17340 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:43:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.145+0000 7f57ac004740 1 -- 172.21.15.133:0/3731260162 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e88d71d70 con 0x558e88e8ced0 2024-09-11T23:43:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.148+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558e88e8ced0 0x558e88e8f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.157+0000 7f57a2d9d640 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558e88e8ced0 0x558e88e8f2f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca564980a4ca0d0 server_cookie=2d2be1d983424701 in_seq=0 out_seq=0 2024-09-11T23:43:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.157+0000 7f5786ff5640 10 client.17340 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:54.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:53 smithi138 ceph-mon[35142]: pgmap v219: 97 pgs: 97 active+clean; 60 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 778 MiB/s wr, 2.59k op/s 2024-09-11T23:43:54.246 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-11T23:43:54.246 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:43:54.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:53 smithi178 ceph-mon[35882]: pgmap v219: 97 pgs: 97 active+clean; 60 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 778 MiB/s wr, 2.59k op/s 2024-09-11T23:43:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.526+0000 7f5786ff5640 1 -- 172.21.15.133:0/3731260162 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e88d71d70 con 0x558e88e8ced0 2024-09-11T23:43:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.527+0000 7f5786ff5640 10 client.17340 handle_command_reply: tid=0 2024-09-11T23:43:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.527+0000 7f57ac004740 1 client.17340 shutdown 2024-09-11T23:43:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.529+0000 7f57ac004740 1 -- 172.21.15.133:0/3731260162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 msgr2=0x558e88e184e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.529+0000 7f57ac004740 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 0x558e88e184e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f579400b950 tx=0x7f5794005850 comp rx=0 tx=0).stop 2024-09-11T23:43:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.529+0000 7f57ac004740 1 -- 172.21.15.133:0/3731260162 shutdown_connections 2024-09-11T23:43:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.529+0000 7f57ac004740 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558e88e8ced0 0x558e88e8f2f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.529+0000 7f57ac004740 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e88e1b340 0x558e88e18f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.529+0000 7f57ac004740 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e88e0df40 0x558e88e18a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.529+0000 7f57ac004740 1 --2- 172.21.15.133:0/3731260162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e88e0bc60 0x558e88e184e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.529+0000 7f57ac004740 1 -- 172.21.15.133:0/3731260162 >> 172.21.15.133:0/3731260162 conn(0x558e88e02060 msgr2=0x558e88e0f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.529+0000 7f5786ff5640 0 client.17340 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.530+0000 7f57ac004740 1 -- 172.21.15.133:0/3731260162 shutdown_connections 2024-09-11T23:43:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.530+0000 7f57ac004740 1 -- 172.21.15.133:0/3731260162 wait complete. 2024-09-11T23:43:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.530+0000 7f57ac004740 20 client.17340 trim_cache size 0 max 16384 2024-09-11T23:43:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:43:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:43:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:43:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:43:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.538+0000 7f57a918d640 1 -- 172.21.15.133:0/3783782121 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f578c0a2e20 msgr2=0x7f578c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.538+0000 7f57a918d640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f578c0a2e20 0x7f578c0a52e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5798005c20 tx=0x7f5798004680 comp rx=0 tx=0).stop 2024-09-11T23:43:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.538+0000 7f57a918d640 1 -- 172.21.15.133:0/3783782121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a411b140 msgr2=0x7f57a41ab9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.538+0000 7f57a918d640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a411b140 0x7f57a41ab9c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f579800d780 tx=0x7f579800dc40 comp rx=0 tx=0).stop 2024-09-11T23:43:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.544+0000 7f57a918d640 1 -- 172.21.15.133:0/3783782121 shutdown_connections 2024-09-11T23:43:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.545+0000 7f57a918d640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f578c0a2e20 0x7f578c0a52e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.545+0000 7f57a918d640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a411b140 0x7f57a41ab9c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.545+0000 7f57a918d640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57a41095d0 0x7f57a41a7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.545+0000 7f57a918d640 1 --2- 172.21.15.133:0/3783782121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a41091f0 0x7f57a41a7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.545+0000 7f57a918d640 1 -- 172.21.15.133:0/3783782121 >> 172.21.15.133:0/3783782121 conn(0x7f57a406ce00 msgr2=0x7f57a411b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.548+0000 7f57a918d640 1 -- 172.21.15.133:0/3783782121 shutdown_connections 2024-09-11T23:43:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:43:54.549+0000 7f57a918d640 1 -- 172.21.15.133:0/3783782121 wait complete. 2024-09-11T23:43:54.810 INFO:tasks.cephfs.filesystem:scrub status for tag:72a3ff9d-7381-4834-a989-6d6181550c26 - {'path': '/', 'tag': '72a3ff9d-7381-4834-a989-6d6181550c26', 'options': 'recursive,force'} 2024-09-11T23:43:54.919 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:54 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:43:55.119 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:55.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:54 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:43:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:54 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:43:56.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.117+0000 7fbb9b83c640 1 -- 172.21.15.133:0/3114027562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb94109450 msgr2=0x7fbb94110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:56.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.117+0000 7fbb9b83c640 1 --2- 172.21.15.133:0/3114027562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb94109450 0x7fbb94110250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009ca0 tx=0x7fbb8c02a8f0 comp rx=0 tx=0).stop 2024-09-11T23:43:56.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.118+0000 7fbb9b83c640 1 -- 172.21.15.133:0/3114027562 shutdown_connections 2024-09-11T23:43:56.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.118+0000 7fbb9b83c640 1 --2- 172.21.15.133:0/3114027562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb9411b140 0x7fbb9411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:56.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.118+0000 7fbb9b83c640 1 --2- 172.21.15.133:0/3114027562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb94109830 0x7fbb94109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:56.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.118+0000 7fbb9b83c640 1 --2- 172.21.15.133:0/3114027562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb94109450 0x7fbb94110250 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:56.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.118+0000 7fbb9b83c640 1 -- 172.21.15.133:0/3114027562 >> 172.21.15.133:0/3114027562 conn(0x7fbb9406e140 msgr2=0x7fbb9406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:56.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.118+0000 7fbb9b83c640 1 -- 172.21.15.133:0/3114027562 shutdown_connections 2024-09-11T23:43:56.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.118+0000 7fbb9b83c640 1 -- 172.21.15.133:0/3114027562 wait complete. 2024-09-11T23:43:56.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.119+0000 7fbb9b83c640 1 Processor -- start 2024-09-11T23:43:56.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.119+0000 7fbb9b83c640 1 -- start start 2024-09-11T23:43:56.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.120+0000 7fbb9b83c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb94109450 0x7fbb941a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:56.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.120+0000 7fbb9b83c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb94109830 0x7fbb941a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:56.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.120+0000 7fbb9b83c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb9411b140 0x7fbb941abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:56.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.120+0000 7fbb9b83c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94120e10 con 0x7fbb94109830 2024-09-11T23:43:56.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.120+0000 7fbb9b83c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94120c90 con 0x7fbb9411b140 2024-09-11T23:43:56.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.120+0000 7fbb9b83c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94120f90 con 0x7fbb94109450 2024-09-11T23:43:56.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.120+0000 7fbb98dd7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb94109830 0x7fbb941a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:56.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.120+0000 7fbb98dd7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb94109830 0x7fbb941a77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33062/0 (socket says 172.21.15.133:33062) 2024-09-11T23:43:56.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.120+0000 7fbb98dd7640 1 -- 172.21.15.133:0/4289738743 learned_addr learned my addr 172.21.15.133:0/4289738743 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:56.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.121+0000 7fbb98dd7640 1 -- 172.21.15.133:0/4289738743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb94109450 msgr2=0x7fbb941a72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:56.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.121+0000 7fbb98dd7640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb94109450 0x7fbb941a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:56.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.121+0000 7fbb98dd7640 1 -- 172.21.15.133:0/4289738743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb9411b140 msgr2=0x7fbb941abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:56.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.121+0000 7fbb98dd7640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb9411b140 0x7fbb941abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:56.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.121+0000 7fbb98dd7640 1 -- 172.21.15.133:0/4289738743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb941ac260 con 0x7fbb94109830 2024-09-11T23:43:56.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.122+0000 7fbb98dd7640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb94109830 0x7fbb941a77f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000e9f0 tx=0x7fbb9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:56.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.122+0000 7fbb8a7fc640 1 -- 172.21.15.133:0/4289738743 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000cd80 con 0x7fbb94109830 2024-09-11T23:43:56.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.122+0000 7fbb8a7fc640 1 -- 172.21.15.133:0/4289738743 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb90010910 con 0x7fbb94109830 2024-09-11T23:43:56.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.122+0000 7fbb9b83c640 1 -- 172.21.15.133:0/4289738743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb941ac550 con 0x7fbb94109830 2024-09-11T23:43:56.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.123+0000 7fbb9b83c640 1 -- 172.21.15.133:0/4289738743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941b3e30 con 0x7fbb94109830 2024-09-11T23:43:56.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.123+0000 7fbb8a7fc640 1 -- 172.21.15.133:0/4289738743 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb900188b0 con 0x7fbb94109830 2024-09-11T23:43:56.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.127+0000 7fbb9b83c640 1 -- 172.21.15.133:0/4289738743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb78005290 con 0x7fbb94109830 2024-09-11T23:43:56.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.131+0000 7fbb8a7fc640 1 -- 172.21.15.133:0/4289738743 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbb90018b50 con 0x7fbb94109830 2024-09-11T23:43:56.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.133+0000 7fbb8a7fc640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbb800a2f00 0x7fbb800a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:56.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.134+0000 7fbb8a7fc640 1 -- 172.21.15.133:0/4289738743 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fbb900cf700 con 0x7fbb94109830 2024-09-11T23:43:56.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.136+0000 7fbb995d8640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbb800a2f00 0x7fbb800a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:56.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.143+0000 7fbb8a7fc640 1 -- 172.21.15.133:0/4289738743 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb9009c300 con 0x7fbb94109830 2024-09-11T23:43:56.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.149+0000 7fbb995d8640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbb800a2f00 0x7fbb800a53c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009bf0 tx=0x7fbb8c005f80 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:56 smithi178 ceph-mon[35882]: pgmap v220: 97 pgs: 97 active+clean; 60 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 679 MiB/s wr, 2.18k op/s 2024-09-11T23:43:56.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:56 smithi138 ceph-mon[35142]: pgmap v220: 97 pgs: 97 active+clean; 60 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 679 MiB/s wr, 2.18k op/s 2024-09-11T23:43:56.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:56 smithi133 ceph-mon[31430]: pgmap v220: 97 pgs: 97 active+clean; 60 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 679 MiB/s wr, 2.18k op/s 2024-09-11T23:43:56.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.699+0000 7fbb9b83c640 1 -- 172.21.15.133:0/4289738743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb78005850 con 0x7fbb94109830 2024-09-11T23:43:56.723 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:43:56.723 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:43:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.713+0000 7fbb8a7fc640 1 -- 172.21.15.133:0/4289738743 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7fbb9009bc00 con 0x7fbb94109830 2024-09-11T23:43:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:43:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.720+0000 7fbb9b83c640 1 -- 172.21.15.133:0/4289738743 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbb800a2f00 msgr2=0x7fbb800a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.720+0000 7fbb9b83c640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbb800a2f00 0x7fbb800a53c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009bf0 tx=0x7fbb8c005f80 comp rx=0 tx=0).stop 2024-09-11T23:43:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.720+0000 7fbb9b83c640 1 -- 172.21.15.133:0/4289738743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb94109830 msgr2=0x7fbb941a77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.720+0000 7fbb9b83c640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb94109830 0x7fbb941a77f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000e9f0 tx=0x7fbb9000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:43:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.721+0000 7fbb9b83c640 1 -- 172.21.15.133:0/4289738743 shutdown_connections 2024-09-11T23:43:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.721+0000 7fbb9b83c640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbb800a2f00 0x7fbb800a53c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.721+0000 7fbb9b83c640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb9411b140 0x7fbb941abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.721+0000 7fbb9b83c640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb94109830 0x7fbb941a77f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:56.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.721+0000 7fbb9b83c640 1 --2- 172.21.15.133:0/4289738743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb94109450 0x7fbb941a72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:56.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.721+0000 7fbb9b83c640 1 -- 172.21.15.133:0/4289738743 >> 172.21.15.133:0/4289738743 conn(0x7fbb9406e140 msgr2=0x7fbb9411bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:56.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.721+0000 7fbb9b83c640 1 -- 172.21.15.133:0/4289738743 shutdown_connections 2024-09-11T23:43:56.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:56.721+0000 7fbb9b83c640 1 -- 172.21.15.133:0/4289738743 wait complete. 2024-09-11T23:43:57.245 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:57 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/4289738743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:57.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:57 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/4289738743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:57.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:57 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/4289738743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:43:57.852 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-11T23:43:57.852 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-11T23:43:58.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:43:58 smithi178 ceph-mon[35882]: pgmap v221: 97 pgs: 97 active+clean; 60 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 664 MiB/s wr, 2.01k op/s 2024-09-11T23:43:58.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:43:58 smithi138 ceph-mon[35142]: pgmap v221: 97 pgs: 97 active+clean; 60 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 664 MiB/s wr, 2.01k op/s 2024-09-11T23:43:58.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:43:58 smithi133 ceph-mon[31430]: pgmap v221: 97 pgs: 97 active+clean; 60 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 664 MiB/s wr, 2.01k op/s 2024-09-11T23:43:58.762 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:43:59.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.635+0000 7f7670333640 1 -- 172.21.15.133:0/635784466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76681091f0 msgr2=0x7f76681095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:59.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.635+0000 7f7670333640 1 --2- 172.21.15.133:0/635784466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76681091f0 0x7f76681095d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7660009ca0 tx=0x7f766002a980 comp rx=0 tx=0).stop 2024-09-11T23:43:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.642+0000 7f7670333640 1 -- 172.21.15.133:0/635784466 shutdown_connections 2024-09-11T23:43:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.642+0000 7f7670333640 1 --2- 172.21.15.133:0/635784466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7668114ab0 0x7f7668116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.642+0000 7f7670333640 1 --2- 172.21.15.133:0/635784466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7668109b10 0x7f7668114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.642+0000 7f7670333640 1 --2- 172.21.15.133:0/635784466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76681091f0 0x7f76681095d0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.642+0000 7f7670333640 1 -- 172.21.15.133:0/635784466 >> 172.21.15.133:0/635784466 conn(0x7f766806e140 msgr2=0x7f766806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.646+0000 7f7670333640 1 -- 172.21.15.133:0/635784466 shutdown_connections 2024-09-11T23:43:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.646+0000 7f7670333640 1 -- 172.21.15.133:0/635784466 wait complete. 2024-09-11T23:43:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.648+0000 7f7670333640 1 Processor -- start 2024-09-11T23:43:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.650+0000 7f7670333640 1 -- start start 2024-09-11T23:43:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.650+0000 7f7670333640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76681091f0 0x7f7668117860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.650+0000 7f7670333640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7668109b10 0x7f7668117da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.650+0000 7f7670333640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7668114ab0 0x7f76681182e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.650+0000 7f7670333640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f766811c9f0 con 0x7f76681091f0 2024-09-11T23:43:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.650+0000 7f7670333640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f766811c870 con 0x7f7668114ab0 2024-09-11T23:43:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.650+0000 7f7670333640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f766811cb70 con 0x7f7668109b10 2024-09-11T23:43:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.654+0000 7f766d8ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7668109b10 0x7f7668117da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.654+0000 7f766e0cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76681091f0 0x7f7668117860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.654+0000 7f766e0cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76681091f0 0x7f7668117860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33076/0 (socket says 172.21.15.133:33076) 2024-09-11T23:43:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.654+0000 7f766e0cf640 1 -- 172.21.15.133:0/3650835252 learned_addr learned my addr 172.21.15.133:0/3650835252 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:59.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.655+0000 7f766d8ce640 1 -- 172.21.15.133:0/3650835252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7668114ab0 msgr2=0x7f76681182e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:43:59.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.655+0000 7f766d8ce640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7668114ab0 0x7f76681182e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.655+0000 7f766d8ce640 1 -- 172.21.15.133:0/3650835252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76681091f0 msgr2=0x7f7668117860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:59.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.655+0000 7f766d8ce640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76681091f0 0x7f7668117860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.655+0000 7f766d8ce640 1 -- 172.21.15.133:0/3650835252 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76681a2cf0 con 0x7f7668109b10 2024-09-11T23:43:59.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.661+0000 7f766d8ce640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7668109b10 0x7f7668117da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f766400d700 tx=0x7f766400dbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:59.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.662+0000 7f765f7fe640 1 -- 172.21.15.133:0/3650835252 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664017070 con 0x7f7668109b10 2024-09-11T23:43:59.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.662+0000 7f765f7fe640 1 -- 172.21.15.133:0/3650835252 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7664009820 con 0x7f7668109b10 2024-09-11T23:43:59.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.662+0000 7f765f7fe640 1 -- 172.21.15.133:0/3650835252 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664004ce0 con 0x7f7668109b10 2024-09-11T23:43:59.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.662+0000 7f7670333640 1 -- 172.21.15.133:0/3650835252 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76681a2fe0 con 0x7f7668109b10 2024-09-11T23:43:59.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.663+0000 7f7670333640 1 -- 172.21.15.133:0/3650835252 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76681a34d0 con 0x7f7668109b10 2024-09-11T23:43:59.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.668+0000 7f767270c740 1 Processor -- start 2024-09-11T23:43:59.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.669+0000 7f767270c740 1 -- start start 2024-09-11T23:43:59.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.669+0000 7f767270c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cd4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.669+0000 7f767270c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cdf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.669+0000 7f767270c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249ce1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.669+0000 7f767270c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d249c3a2a0 con 0x55d249cd5080 2024-09-11T23:43:59.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.669+0000 7f767270c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d249be1b10 con 0x55d249cd4760 2024-09-11T23:43:59.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.669+0000 7f767270c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d249c36260 con 0x55d249cdf940 2024-09-11T23:43:59.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.672+0000 7f765f7fe640 1 -- 172.21.15.133:0/3650835252 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7664005220 con 0x7f7668109b10 2024-09-11T23:43:59.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.673+0000 7f765f7fe640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76540a2ec0 0x7f76540a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.673+0000 7f765f7fe640 1 -- 172.21.15.133:0/3650835252 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f76640ceca0 con 0x7f7668109b10 2024-09-11T23:43:59.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.674+0000 7f766d8ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249ce1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.675+0000 7f766d8ce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249ce1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42002/0 (socket says 172.21.15.133:42002) 2024-09-11T23:43:59.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.675+0000 7f766d8ce640 1 -- 172.21.15.133:0/4103554978 learned_addr learned my addr 172.21.15.133:0/4103554978 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:59.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.675+0000 7f766e0cf640 1 --2- 172.21.15.133:0/4103554978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cdf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.675+0000 7f766e8d0640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76540a2ec0 0x7f76540a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.675+0000 7f766e8d0640 1 --2- 172.21.15.133:0/4103554978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cd4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.676+0000 7f766e0cf640 1 -- 172.21.15.133:0/4103554978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 msgr2=0x55d249ce1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:59.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.676+0000 7f766e0cf640 1 --2- 172.21.15.133:0/4103554978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249ce1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.676+0000 7f766e0cf640 1 -- 172.21.15.133:0/4103554978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 msgr2=0x55d249cd4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:59.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.676+0000 7f766e0cf640 1 --2- 172.21.15.133:0/4103554978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cd4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.676+0000 7f766e0cf640 1 -- 172.21.15.133:0/4103554978 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2499abb90 con 0x55d249cd5080 2024-09-11T23:43:59.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.676+0000 7f766e8d0640 1 --2- 172.21.15.133:0/4103554978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cd4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:43:59.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.677+0000 7f766e8d0640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76540a2ec0 0x7f76540a5380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f766806b430 tx=0x7f765800b5f0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:59.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.677+0000 7f766e0cf640 1 --2- 172.21.15.133:0/4103554978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cdf400 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f766002aea0 tx=0x7f7660009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e097480855ad5b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:59.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.678+0000 7f765cdea640 1 -- 172.21.15.133:0/4103554978 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660034020 con 0x55d249cd5080 2024-09-11T23:43:59.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.678+0000 7f765cdea640 1 -- 172.21.15.133:0/4103554978 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7660030880 con 0x55d249cd5080 2024-09-11T23:43:59.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.678+0000 7f765cdea640 1 -- 172.21.15.133:0/4103554978 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766002f6d0 con 0x55d249cd5080 2024-09-11T23:43:59.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.679+0000 7f767270c740 1 -- 172.21.15.133:0/4103554978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 msgr2=0x55d249cdf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:59.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.679+0000 7f767270c740 1 --2- 172.21.15.133:0/4103554978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cdf400 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f766002aea0 tx=0x7f7660009890 comp rx=0 tx=0).stop 2024-09-11T23:43:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.680+0000 7f767270c740 1 -- 172.21.15.133:0/4103554978 shutdown_connections 2024-09-11T23:43:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.680+0000 7f767270c740 1 --2- 172.21.15.133:0/4103554978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249ce1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.680+0000 7f767270c740 1 --2- 172.21.15.133:0/4103554978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cdf400 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.680+0000 7f767270c740 1 --2- 172.21.15.133:0/4103554978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cd4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.680+0000 7f767270c740 1 -- 172.21.15.133:0/4103554978 >> 172.21.15.133:0/4103554978 conn(0x55d249cd0cf0 msgr2=0x55d249cd10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.681+0000 7f767270c740 1 -- 172.21.15.133:0/4103554978 shutdown_connections 2024-09-11T23:43:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.681+0000 7f767270c740 1 -- 172.21.15.133:0/4103554978 wait complete. 2024-09-11T23:43:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.681+0000 7f767270c740 1 -- start start 2024-09-11T23:43:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.682+0000 7f767270c740 1 Processor -- start 2024-09-11T23:43:59.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.682+0000 7f767270c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cdc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.682+0000 7f767270c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cdccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.682+0000 7f767270c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249cdd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.682+0000 7f767270c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d249ce2510 con 0x55d249cd5080 2024-09-11T23:43:59.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.682+0000 7f767270c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d249bc26e0 con 0x55d249cd4760 2024-09-11T23:43:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.682+0000 7f767270c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d249ce2690 con 0x55d249cdf940 2024-09-11T23:43:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.682+0000 7f767270c740 10 client.0 _ll_register_callbacks cb 0x55d249be6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.683+0000 7f766d8ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cdccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.683+0000 7f766d8ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cdccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33096/0 (socket says 172.21.15.133:33096) 2024-09-11T23:43:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.683+0000 7f766d8ce640 1 -- 172.21.15.133:0/4214743011 learned_addr learned my addr 172.21.15.133:0/4214743011 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.684+0000 7f766e0cf640 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cdc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.684+0000 7f766d8ce640 1 -- 172.21.15.133:0/4214743011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 msgr2=0x55d249cdd1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:43:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.684+0000 7f766d8ce640 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249cdd1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.684+0000 7f766d8ce640 1 -- 172.21.15.133:0/4214743011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 msgr2=0x55d249cdc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.684+0000 7f766d8ce640 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cdc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.684+0000 7f766d8ce640 1 -- 172.21.15.133:0/4214743011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d249ceb9e0 con 0x55d249cd5080 2024-09-11T23:43:59.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.685+0000 7f766d8ce640 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cdccb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7664000c00 tx=0x7f766409b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee630f0e195a4ab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:59.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.685+0000 7f7651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:43:59.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.686+0000 7f7651ffb640 1 -- 172.21.15.133:0/4214743011 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664020040 con 0x55d249cd5080 2024-09-11T23:43:59.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.686+0000 7f7651ffb640 1 -- 172.21.15.133:0/4214743011 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f76640bb0b0 con 0x55d249cd5080 2024-09-11T23:43:59.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.686+0000 7f767270c740 1 -- 172.21.15.133:0/4214743011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7660030880 con 0x55d249cd5080 2024-09-11T23:43:59.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.687+0000 7f7651ffb640 1 -- 172.21.15.133:0/4214743011 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f76640a3e10 con 0x55d249cd5080 2024-09-11T23:43:59.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.687+0000 7f767270c740 10 client.17355 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:59.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.687+0000 7f767270c740 1 -- 172.21.15.133:0/4214743011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d249cee3a0 con 0x55d249cd5080 2024-09-11T23:43:59.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.688+0000 7f7651ffb640 1 -- 172.21.15.133:0/4214743011 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7660030880 con 0x55d249cd5080 2024-09-11T23:43:59.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.688+0000 7f7651ffb640 1 -- 172.21.15.133:0/4214743011 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f766409f0d0 con 0x55d249cd5080 2024-09-11T23:43:59.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.688+0000 7f767270c740 10 client.17355 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:59.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.688+0000 7f767270c740 10 client.17355 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:43:59.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.688+0000 7f767270c740 20 client.17355 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:59.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.688+0000 7f767270c740 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d249cef900 0x55d249cf1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.688+0000 7f767270c740 4 client.17355 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:43:59.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.688+0000 7f767270c740 1 -- 172.21.15.133:0/4214743011 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d249c479d0 con 0x55d249cef900 2024-09-11T23:43:59.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.690+0000 7f766e0cf640 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d249cef900 0x55d249cf1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.696+0000 7f766e0cf640 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d249cef900 0x55d249cf1d40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da4041967826695d server_cookie=cb447b50cce937c2 in_seq=0 out_seq=0 2024-09-11T23:43:59.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.696+0000 7f7651ffb640 10 client.17355 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.701+0000 7f7651ffb640 1 -- 172.21.15.133:0/4214743011 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d249c479d0 con 0x55d249cef900 2024-09-11T23:43:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.701+0000 7f7651ffb640 10 client.17355 handle_command_reply: tid=0 2024-09-11T23:43:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.701+0000 7f767270c740 1 client.17355 shutdown 2024-09-11T23:43:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.704+0000 7f767270c740 1 -- 172.21.15.133:0/4214743011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 msgr2=0x55d249cdccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.704+0000 7f767270c740 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cdccb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7664000c00 tx=0x7f766409b960 comp rx=0 tx=0).stop 2024-09-11T23:43:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.705+0000 7f767270c740 1 -- 172.21.15.133:0/4214743011 shutdown_connections 2024-09-11T23:43:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.705+0000 7f767270c740 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d249cef900 0x55d249cf1d40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.705+0000 7f767270c740 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249cdd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.705+0000 7f767270c740 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cdccb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.705+0000 7f767270c740 1 --2- 172.21.15.133:0/4214743011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cdc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.705+0000 7f767270c740 1 -- 172.21.15.133:0/4214743011 >> 172.21.15.133:0/4214743011 conn(0x55d249cd0cf0 msgr2=0x55d249ccfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.705+0000 7f7651ffb640 0 client.17355 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:43:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.706+0000 7f767270c740 1 -- 172.21.15.133:0/4214743011 shutdown_connections 2024-09-11T23:43:59.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.707+0000 7f767270c740 1 -- 172.21.15.133:0/4214743011 wait complete. 2024-09-11T23:43:59.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.707+0000 7f767270c740 20 client.17355 trim_cache size 0 max 16384 2024-09-11T23:43:59.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.745+0000 7f767270c740 1 Processor -- start 2024-09-11T23:43:59.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.745+0000 7f767270c740 1 -- start start 2024-09-11T23:43:59.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.746+0000 7f767270c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cf7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.746+0000 7f767270c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cf7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.746+0000 7f767270c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249cd9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.747+0000 7f766d8ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cf7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.747+0000 7f766d8ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cf7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33100/0 (socket says 172.21.15.133:33100) 2024-09-11T23:43:59.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.747+0000 7f766e0cf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cf7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.747+0000 7f766e0cf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cf7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:48178/0 (socket says 172.21.15.133:48178) 2024-09-11T23:43:59.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.747+0000 7f766d8ce640 1 -- 172.21.15.133:0/3760409014 learned_addr learned my addr 172.21.15.133:0/3760409014 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:59.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.748+0000 7f767270c740 1 -- 172.21.15.133:0/3760409014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f764c00e720 con 0x55d249cd5080 2024-09-11T23:43:59.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.748+0000 7f767270c740 1 -- 172.21.15.133:0/3760409014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f764c010260 con 0x55d249cd4760 2024-09-11T23:43:59.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.748+0000 7f767270c740 1 -- 172.21.15.133:0/3760409014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f764c00edf0 con 0x55d249cdf940 2024-09-11T23:43:59.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.748+0000 7f766e8d0640 1 --2- 172.21.15.133:0/3760409014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249cd9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.749+0000 7f766d8ce640 1 -- 172.21.15.133:0/3760409014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 msgr2=0x55d249cd9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:59.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.749+0000 7f766d8ce640 1 --2- 172.21.15.133:0/3760409014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249cd9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.749+0000 7f766d8ce640 1 -- 172.21.15.133:0/3760409014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 msgr2=0x55d249cf7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:59.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.749+0000 7f766d8ce640 1 --2- 172.21.15.133:0/3760409014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cf7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.749+0000 7f766d8ce640 1 -- 172.21.15.133:0/3760409014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d249cd0b20 con 0x55d249cd5080 2024-09-11T23:43:59.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.751+0000 7f766d8ce640 1 --2- 172.21.15.133:0/3760409014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cf7b90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f766409a770 tx=0x7f766409f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e1d19c1f8cc9a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:59.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.751+0000 7f7653fff640 1 -- 172.21.15.133:0/3760409014 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664015070 con 0x55d249cd5080 2024-09-11T23:43:59.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.752+0000 7f7653fff640 1 -- 172.21.15.133:0/3760409014 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76640b4070 con 0x55d249cd5080 2024-09-11T23:43:59.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.753+0000 7f767270c740 1 -- 172.21.15.133:0/3760409014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 msgr2=0x55d249cf7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:43:59.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.753+0000 7f767270c740 1 --2- 172.21.15.133:0/3760409014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cf7b90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f766409a770 tx=0x7f766409f6d0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.753+0000 7f767270c740 1 -- 172.21.15.133:0/3760409014 shutdown_connections 2024-09-11T23:43:59.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.753+0000 7f767270c740 1 --2- 172.21.15.133:0/3760409014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249cdf940 0x55d249cd9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.753+0000 7f767270c740 1 --2- 172.21.15.133:0/3760409014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd5080 0x55d249cf7b90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.753+0000 7f767270c740 1 --2- 172.21.15.133:0/3760409014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cd4760 0x55d249cf7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.754+0000 7f767270c740 1 -- 172.21.15.133:0/3760409014 >> 172.21.15.133:0/3760409014 conn(0x55d249cd0cf0 msgr2=0x55d249cd69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:43:59.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.754+0000 7f767270c740 1 -- 172.21.15.133:0/3760409014 shutdown_connections 2024-09-11T23:43:59.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.754+0000 7f767270c740 1 -- 172.21.15.133:0/3760409014 wait complete. 2024-09-11T23:43:59.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.754+0000 7f767270c740 1 -- start start 2024-09-11T23:43:59.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.755+0000 7f767270c740 1 Processor -- start 2024-09-11T23:43:59.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.755+0000 7f767270c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd4760 0x55d249ce7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.755+0000 7f767270c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cdf940 0x55d249ce4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.755+0000 7f767270c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249ce4e50 0x55d249ce52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.755+0000 7f767270c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f764c00fb90 con 0x55d249cd4760 2024-09-11T23:43:59.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.755+0000 7f767270c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f764c00f4c0 con 0x55d249cdf940 2024-09-11T23:43:59.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.755+0000 7f767270c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f764c00c720 con 0x55d249ce4e50 2024-09-11T23:43:59.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.755+0000 7f767270c740 10 client.0 _ll_register_callbacks cb 0x7f7664004e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:43:59.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.756+0000 7f766e0cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd4760 0x55d249ce7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.756+0000 7f766e0cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd4760 0x55d249ce7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33116/0 (socket says 172.21.15.133:33116) 2024-09-11T23:43:59.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.756+0000 7f766e0cf640 1 -- 172.21.15.133:0/3261795868 learned_addr learned my addr 172.21.15.133:0/3261795868 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:43:59.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.756+0000 7f766e8d0640 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249ce4e50 0x55d249ce52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.756+0000 7f766d8ce640 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cdf940 0x55d249ce4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.757+0000 7f766e8d0640 1 -- 172.21.15.133:0/3261795868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cdf940 msgr2=0x55d249ce4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:59.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.757+0000 7f766e8d0640 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cdf940 0x55d249ce4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.757+0000 7f766e8d0640 1 -- 172.21.15.133:0/3261795868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd4760 msgr2=0x55d249ce7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:43:59.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.757+0000 7f766e8d0640 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd4760 0x55d249ce7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:43:59.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.757+0000 7f766e8d0640 1 -- 172.21.15.133:0/3261795868 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d249ccc260 con 0x55d249ce4e50 2024-09-11T23:43:59.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.758+0000 7f766e8d0640 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249ce4e50 0x55d249ce52d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f76580062e0 tx=0x7f7658015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1436e4f9ef0a74a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:43:59.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.762+0000 7f7651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:43:59.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.762+0000 7f7651ffb640 1 -- 172.21.15.133:0/3261795868 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658026060 con 0x55d249ce4e50 2024-09-11T23:43:59.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.762+0000 7f7651ffb640 1 -- 172.21.15.133:0/3261795868 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f7658004e00 con 0x55d249ce4e50 2024-09-11T23:43:59.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.764+0000 7f767270c740 1 -- 172.21.15.133:0/3261795868 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d249c41d20 con 0x55d249ce4e50 2024-09-11T23:43:59.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.765+0000 7f7651ffb640 1 -- 172.21.15.133:0/3261795868 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7658011040 con 0x55d249ce4e50 2024-09-11T23:43:59.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.765+0000 7f767270c740 10 client.26728 fetch_fsmap learned FSMap version 30 2024-09-11T23:43:59.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.765+0000 7f767270c740 1 -- 172.21.15.133:0/3261795868 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d249d5b5b0 con 0x55d249ce4e50 2024-09-11T23:43:59.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.767+0000 7f7651ffb640 1 -- 172.21.15.133:0/3261795868 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d249c41d20 con 0x55d249ce4e50 2024-09-11T23:43:59.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.769+0000 7f7651ffb640 1 -- 172.21.15.133:0/3261795868 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7658021790 con 0x55d249ce4e50 2024-09-11T23:43:59.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.769+0000 7f767270c740 10 client.26728 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:43:59.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.769+0000 7f767270c740 10 client.26728 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:43:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.769+0000 7f767270c740 20 client.26728 populate_metadata read hostname 'smithi133' 2024-09-11T23:43:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.769+0000 7f767270c740 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d249d5bd80 0x55d249d5e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:43:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.769+0000 7f767270c740 4 client.26728 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-11T23:43:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.769+0000 7f767270c740 1 -- 172.21.15.133:0/3261795868 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x55d249bbdec0 con 0x55d249d5bd80 2024-09-11T23:43:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.772+0000 7f766e0cf640 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d249d5bd80 0x55d249d5e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:43:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.781+0000 7f766e0cf640 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d249d5bd80 0x55d249d5e1a0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b90ad7fa0af57a server_cookie=e39f8858eee66eb3 in_seq=0 out_seq=0 2024-09-11T23:43:59.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:43:59.781+0000 7f7651ffb640 10 client.26728 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.033+0000 7f7651ffb640 1 -- 172.21.15.133:0/3261795868 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d249bbdec0 con 0x55d249d5bd80 2024-09-11T23:44:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.033+0000 7f7651ffb640 10 client.26728 handle_command_reply: tid=0 2024-09-11T23:44:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.033+0000 7f767270c740 1 client.26728 shutdown 2024-09-11T23:44:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f767270c740 1 -- 172.21.15.133:0/3261795868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249ce4e50 msgr2=0x55d249ce52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f767270c740 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249ce4e50 0x55d249ce52d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f76580062e0 tx=0x7f7658015000 comp rx=0 tx=0).stop 2024-09-11T23:44:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f767270c740 1 -- 172.21.15.133:0/3261795868 shutdown_connections 2024-09-11T23:44:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f767270c740 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d249d5bd80 0x55d249d5e1a0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f767270c740 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d249ce4e50 0x55d249ce52d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f767270c740 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d249cdf940 0x55d249ce4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f7651ffb640 0 client.26728 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f767270c740 1 --2- 172.21.15.133:0/3261795868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d249cd4760 0x55d249ce7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f767270c740 1 -- 172.21.15.133:0/3261795868 >> 172.21.15.133:0/3261795868 conn(0x55d249cd0cf0 msgr2=0x55d249cddfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:00.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f767270c740 1 -- 172.21.15.133:0/3261795868 shutdown_connections 2024-09-11T23:44:00.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f767270c740 1 -- 172.21.15.133:0/3261795868 wait complete. 2024-09-11T23:44:00.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.034+0000 7f767270c740 20 client.26728 trim_cache size 0 max 16384 2024-09-11T23:44:00.035 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:44:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:44:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:44:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 26, 2024-09-11T23:44:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:44:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "d5d9c530": { 2024-09-11T23:44:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 26, 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 136.6, 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:44:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:00.037 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:00.037 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:00.037 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:00.037 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:00.037 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 -- 172.21.15.133:0/3650835252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76540a2ec0 msgr2=0x7f76540a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76540a2ec0 0x7f76540a5380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f766806b430 tx=0x7f765800b5f0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 -- 172.21.15.133:0/3650835252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7668109b10 msgr2=0x7f7668117da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7668109b10 0x7f7668117da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f766400d700 tx=0x7f766400dbc0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 -- 172.21.15.133:0/3650835252 shutdown_connections 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76540a2ec0 0x7f76540a5380 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7668114ab0 0x7f76681182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7668109b10 0x7f7668117da0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 --2- 172.21.15.133:0/3650835252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76681091f0 0x7f7668117860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 -- 172.21.15.133:0/3650835252 >> 172.21.15.133:0/3650835252 conn(0x7f766806e140 msgr2=0x7f7668070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 -- 172.21.15.133:0/3650835252 shutdown_connections 2024-09-11T23:44:00.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:00.035+0000 7f7670333640 1 -- 172.21.15.133:0/3650835252 wait complete. 2024-09-11T23:44:00.712 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:00 smithi133 ceph-mon[31430]: pgmap v222: 97 pgs: 97 active+clean; 60 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 814 MiB/s wr, 2.06k op/s 2024-09-11T23:44:00.843 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd5d9c530', quiesce took 6.6 seconds. Will release after: 10.000000000000002 2024-09-11T23:44:00.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:00 smithi138 ceph-mon[35142]: pgmap v222: 97 pgs: 97 active+clean; 60 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 814 MiB/s wr, 2.06k op/s 2024-09-11T23:44:01.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:00 smithi178 ceph-mon[35882]: pgmap v222: 97 pgs: 97 active+clean; 60 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 814 MiB/s wr, 2.06k op/s 2024-09-11T23:44:02.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:02 smithi138 ceph-mon[35142]: pgmap v223: 97 pgs: 97 active+clean; 60 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 598 MiB/s wr, 1.53k op/s 2024-09-11T23:44:02.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:02 smithi133 ceph-mon[31430]: pgmap v223: 97 pgs: 97 active+clean; 60 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 598 MiB/s wr, 1.53k op/s 2024-09-11T23:44:03.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:02 smithi178 ceph-mon[35882]: pgmap v223: 97 pgs: 97 active+clean; 60 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 598 MiB/s wr, 1.53k op/s 2024-09-11T23:44:03.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:03 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:03.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:03 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:04.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:03 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:44:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:04 smithi138 ceph-mon[35142]: pgmap v224: 97 pgs: 97 active+clean; 60 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 669 MiB/s wr, 1.63k op/s 2024-09-11T23:44:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:04 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:04 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:04 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:44:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:04 smithi133 ceph-mon[31430]: pgmap v224: 97 pgs: 97 active+clean; 60 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 669 MiB/s wr, 1.63k op/s 2024-09-11T23:44:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:04 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:04 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:04 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:44:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:04 smithi178 ceph-mon[35882]: pgmap v224: 97 pgs: 97 active+clean; 60 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 669 MiB/s wr, 1.63k op/s 2024-09-11T23:44:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:04 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:44:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:04 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:44:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:04 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:44:06.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:06 smithi138 ceph-mon[35142]: pgmap v225: 97 pgs: 97 active+clean; 60 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 495 MiB/s wr, 827 op/s 2024-09-11T23:44:06.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:06 smithi133 ceph-mon[31430]: pgmap v225: 97 pgs: 97 active+clean; 60 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 495 MiB/s wr, 827 op/s 2024-09-11T23:44:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:06 smithi178 ceph-mon[35882]: pgmap v225: 97 pgs: 97 active+clean; 60 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 495 MiB/s wr, 827 op/s 2024-09-11T23:44:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.719+0000 7f5f2f03c640 1 -- 172.21.15.133:0/2495885665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f28102ca0 msgr2=0x7f5f28103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.719+0000 7f5f2f03c640 1 --2- 172.21.15.133:0/2495885665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f28102ca0 0x7f5f28103080 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009900 tx=0x7f5f2402a410 comp rx=0 tx=0).stop 2024-09-11T23:44:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.719+0000 7f5f2f03c640 1 -- 172.21.15.133:0/2495885665 shutdown_connections 2024-09-11T23:44:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.719+0000 7f5f2f03c640 1 --2- 172.21.15.133:0/2495885665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f2810c0d0 0x7f5f2810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.719+0000 7f5f2f03c640 1 --2- 172.21.15.133:0/2495885665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f281035c0 0x7f5f2810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.719+0000 7f5f2f03c640 1 --2- 172.21.15.133:0/2495885665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f28102ca0 0x7f5f28103080 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.719+0000 7f5f2f03c640 1 -- 172.21.15.133:0/2495885665 >> 172.21.15.133:0/2495885665 conn(0x7f5f280fc6b0 msgr2=0x7f5f280fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.719+0000 7f5f2f03c640 1 -- 172.21.15.133:0/2495885665 shutdown_connections 2024-09-11T23:44:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.719+0000 7f5f2f03c640 1 -- 172.21.15.133:0/2495885665 wait complete. 2024-09-11T23:44:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.719+0000 7f5f2f03c640 1 Processor -- start 2024-09-11T23:44:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2f03c640 1 -- start start 2024-09-11T23:44:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2f03c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f28102ca0 0x7f5f281008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2f03c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f281035c0 0x7f5f28100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2f03c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2810c0d0 0x7f5f28101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2f03c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28111ed0 con 0x7f5f2810c0d0 2024-09-11T23:44:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2f03c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28111d50 con 0x7f5f281035c0 2024-09-11T23:44:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2f03c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28112050 con 0x7f5f28102ca0 2024-09-11T23:44:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2d5d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2810c0d0 0x7f5f28101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f1ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f281035c0 0x7f5f28100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2d5d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2810c0d0 0x7f5f28101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50378/0 (socket says 172.21.15.133:50378) 2024-09-11T23:44:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1712561946 learned_addr learned my addr 172.21.15.133:0/1712561946 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f28102ca0 0x7f5f281008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1712561946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f28102ca0 msgr2=0x7f5f281008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f28102ca0 0x7f5f281008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1712561946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f281035c0 msgr2=0x7f5f28100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f281035c0 0x7f5f28100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1712561946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f281a6830 con 0x7f5f2810c0d0 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f1ffff640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f281035c0 0x7f5f28100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f28102ca0 0x7f5f281008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.720+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2810c0d0 0x7f5f28101370 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000eaa0 tx=0x7f5f2000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.721+0000 7f5f1dffb640 1 -- 172.21.15.133:0/1712561946 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000cdb0 con 0x7f5f2810c0d0 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.721+0000 7f5f1dffb640 1 -- 172.21.15.133:0/1712561946 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f20005020 con 0x7f5f2810c0d0 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.721+0000 7f5f2f03c640 1 -- 172.21.15.133:0/1712561946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f281a6b20 con 0x7f5f2810c0d0 2024-09-11T23:44:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.721+0000 7f5f1dffb640 1 -- 172.21.15.133:0/1712561946 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f200116b0 con 0x7f5f2810c0d0 2024-09-11T23:44:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.721+0000 7f5f2f03c640 1 -- 172.21.15.133:0/1712561946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f281a7070 con 0x7f5f2810c0d0 2024-09-11T23:44:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.723+0000 7f5f1dffb640 1 -- 172.21.15.133:0/1712561946 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5f20005560 con 0x7f5f2810c0d0 2024-09-11T23:44:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f31dd5740 1 Processor -- start 2024-09-11T23:44:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f31dd5740 1 -- start start 2024-09-11T23:44:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f1dffb640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5f100a2da0 0x7f5f100a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 0x5641f3819ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f381a410 0x5641f3824770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f38270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5641f377e2f0 con 0x5641f3824cb0 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5641f3726270 con 0x5641f3819af0 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5641f377a2b0 con 0x5641f381a410 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f2cdd8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 0x5641f3819ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f1ffff640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5f100a2da0 0x7f5f100a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f2cdd8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 0x5641f3819ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52468/0 (socket says 172.21.15.133:52468) 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f1dffb640 1 -- 172.21.15.133:0/1712561946 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f5f200cf5f0 con 0x7f5f2810c0d0 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f2cdd8640 1 -- 172.21.15.133:0/601028117 learned_addr learned my addr 172.21.15.133:0/601028117 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/601028117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f38270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f1ffff640 1 --2- 172.21.15.133:0/601028117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f381a410 0x5641f3824770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f2cdd8640 1 -- 172.21.15.133:0/601028117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f381a410 msgr2=0x5641f3824770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/601028117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f381a410 0x5641f3824770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f2cdd8640 1 -- 172.21.15.133:0/601028117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 msgr2=0x5641f38270a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/601028117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f38270a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.724+0000 7f5f2cdd8640 1 -- 172.21.15.133:0/601028117 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f3725340 con 0x5641f3819af0 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.725+0000 7f5f1ffff640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5f100a2da0 0x7f5f100a5260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5f28102380 tx=0x7f5f1800bf90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.725+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/601028117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 0x5641f3819ed0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f24033000 tx=0x7f5f24032c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa5be92c470158b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.725+0000 7f5f0effd640 1 -- 172.21.15.133:0/601028117 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2403f070 con 0x5641f3819af0 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.725+0000 7f5f0effd640 1 -- 172.21.15.133:0/601028117 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f240317b0 con 0x5641f3819af0 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.725+0000 7f5f0effd640 1 -- 172.21.15.133:0/601028117 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24035070 con 0x5641f3819af0 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.725+0000 7f5f31dd5740 1 -- 172.21.15.133:0/601028117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 msgr2=0x5641f3819ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.725+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/601028117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 0x5641f3819ed0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f24033000 tx=0x7f5f24032c70 comp rx=0 tx=0).stop 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 -- 172.21.15.133:0/601028117 shutdown_connections 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/601028117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f38270a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/601028117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f381a410 0x5641f3824770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/601028117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 0x5641f3819ed0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 -- 172.21.15.133:0/601028117 >> 172.21.15.133:0/601028117 conn(0x5641f3816060 msgr2=0x5641f3816440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 -- 172.21.15.133:0/601028117 shutdown_connections 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 -- 172.21.15.133:0/601028117 wait complete. 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 -- start start 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 Processor -- start 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 0x5641f3821b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f381a410 0x5641f38220b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f38225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5641f3827880 con 0x5641f3824cb0 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5641f3707290 con 0x5641f3819af0 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5641f3827a00 con 0x5641f381a410 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.726+0000 7f5f31dd5740 10 client.0 _ll_register_callbacks cb 0x5641f3738140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f2d5d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f38225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f2d5d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f38225f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50408/0 (socket says 172.21.15.133:50408) 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1036909230 learned_addr learned my addr 172.21.15.133:0/1036909230 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 0x5641f3821b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f1ffff640 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f381a410 0x5641f38220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1036909230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f381a410 msgr2=0x5641f38220b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f381a410 0x5641f38220b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1036909230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 msgr2=0x5641f3821b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 0x5641f3821b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1036909230 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f3491210 con 0x5641f3824cb0 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 0x5641f3821b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f38225f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5f2009bae0 tx=0x7f5f200791f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c895000d3a1c08fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f07fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f07fff640 1 -- 172.21.15.133:0/1036909230 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2001f040 con 0x5641f3824cb0 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f07fff640 1 -- 172.21.15.133:0/1036909230 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f5f200051d0 con 0x5641f3824cb0 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1036909230 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f240317b0 con 0x5641f3824cb0 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f07fff640 1 -- 172.21.15.133:0/1036909230 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f2009bb50 con 0x5641f3824cb0 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f31dd5740 10 client.17373 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.727+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1036909230 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641f355d620 con 0x5641f3824cb0 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.728+0000 7f5f07fff640 1 -- 172.21.15.133:0/1036909230 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f240317b0 con 0x5641f3824cb0 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.728+0000 7f5f07fff640 1 -- 172.21.15.133:0/1036909230 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5f2000f040 con 0x5641f3824cb0 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.728+0000 7f5f31dd5740 10 client.17373 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.728+0000 7f5f31dd5740 10 client.17373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.728+0000 7f5f31dd5740 20 client.17373 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.728+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5641f3834c30 0x5641f3837040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.728+0000 7f5f31dd5740 4 client.17373 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.728+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1036909230 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641f378ba20 con 0x5641f3834c30 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.728+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5641f3834c30 0x5641f3837040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.729+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5641f3834c30 0x5641f3837040 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd1d2415342107d server_cookie=c90a0764beb1ef41 in_seq=0 out_seq=0 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.729+0000 7f5f07fff640 10 client.17373 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.729+0000 7f5f07fff640 1 -- 172.21.15.133:0/1036909230 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641f378ba20 con 0x5641f3834c30 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.729+0000 7f5f07fff640 10 client.17373 handle_command_reply: tid=0 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.729+0000 7f5f31dd5740 1 client.17373 shutdown 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.729+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1036909230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 msgr2=0x5641f38225f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.729+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f38225f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5f2009bae0 tx=0x7f5f200791f0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.729+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1036909230 shutdown_connections 2024-09-11T23:44:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.729+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5641f3834c30 0x5641f3837040 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.729+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f38225f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.730+0000 7f5f07fff640 0 client.17373 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.730+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f381a410 0x5641f38220b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.730+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1036909230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f3819af0 0x5641f3821b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.730+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1036909230 >> 172.21.15.133:0/1036909230 conn(0x5641f3816060 msgr2=0x5641f381bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.730+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1036909230 shutdown_connections 2024-09-11T23:44:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.730+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1036909230 wait complete. 2024-09-11T23:44:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.730+0000 7f5f31dd5740 20 client.17373 trim_cache size 0 max 16384 2024-09-11T23:44:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f31dd5740 1 Processor -- start 2024-09-11T23:44:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f31dd5740 1 -- start start 2024-09-11T23:44:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f3819af0 0x5641f3822420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 0x5641f383c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f383ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281af8f0 con 0x5641f3824cb0 2024-09-11T23:44:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281b1430 con 0x5641f381a410 2024-09-11T23:44:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281affc0 con 0x5641f3819af0 2024-09-11T23:44:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f2d5d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f383ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f2d5d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f383ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50422/0 (socket says 172.21.15.133:50422) 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/44465617 learned_addr learned my addr 172.21.15.133:0/44465617 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.744+0000 7f5f1ffff640 1 --2- 172.21.15.133:0/44465617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 0x5641f383c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.745+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/44465617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f3819af0 0x5641f3822420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.745+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/44465617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f3819af0 msgr2=0x5641f3822420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.745+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/44465617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f3819af0 0x5641f3822420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.745+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/44465617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 msgr2=0x5641f383c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.745+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/44465617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 0x5641f383c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.745+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/44465617 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f381a240 con 0x5641f3824cb0 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.745+0000 7f5f1ffff640 1 --2- 172.21.15.133:0/44465617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 0x5641f383c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.745+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/44465617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f383ce90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5f200a4290 tx=0x7f5f20000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aceb7e3c7dc9a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.745+0000 7f5f0e7fc640 1 -- 172.21.15.133:0/44465617 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f200b9070 con 0x5641f3824cb0 2024-09-11T23:44:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.745+0000 7f5f0e7fc640 1 -- 172.21.15.133:0/44465617 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f200a47c0 con 0x5641f3824cb0 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.745+0000 7f5f0e7fc640 1 -- 172.21.15.133:0/44465617 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2001f040 con 0x5641f3824cb0 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 -- 172.21.15.133:0/44465617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 msgr2=0x5641f383ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/44465617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f383ce90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5f200a4290 tx=0x7f5f20000f40 comp rx=0 tx=0).stop 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 -- 172.21.15.133:0/44465617 shutdown_connections 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/44465617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f383ce90 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/44465617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 0x5641f383c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/44465617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f3819af0 0x5641f3822420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 -- 172.21.15.133:0/44465617 >> 172.21.15.133:0/44465617 conn(0x5641f3816060 msgr2=0x5641f3823bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 -- 172.21.15.133:0/44465617 shutdown_connections 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 -- 172.21.15.133:0/44465617 wait complete. 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 -- start start 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 Processor -- start 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f3819af0 0x5641f382d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 0x5641f381e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f381eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281b0d60 con 0x5641f3824cb0 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281b0690 con 0x5641f381a410 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.746+0000 7f5f31dd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281ad8f0 con 0x5641f3819af0 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f31dd5740 10 client.0 _ll_register_callbacks cb 0x7f5f2009b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f1ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 0x5641f381e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f2d5d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f381eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f2d5d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f381eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50430/0 (socket says 172.21.15.133:50430) 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f1ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 0x5641f381e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52502/0 (socket says 172.21.15.133:52502) 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1572632692 learned_addr learned my addr 172.21.15.133:0/1572632692 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f3819af0 0x5641f382d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1572632692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f3819af0 msgr2=0x5641f382d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f3819af0 0x5641f382d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1572632692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 msgr2=0x5641f381e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 0x5641f381e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f2d5d9640 1 -- 172.21.15.133:0/1572632692 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f381f210 con 0x5641f3824cb0 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f3819af0 0x5641f382d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f1ffff640 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 0x5641f381e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f2d5d9640 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f381eba0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5f20051300 tx=0x7f5f2009ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a64c0e42fd612df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f07fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f07fff640 1 -- 172.21.15.133:0/1572632692 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20080060 con 0x5641f3824cb0 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f07fff640 1 -- 172.21.15.133:0/1572632692 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f5f200a4530 con 0x5641f3824cb0 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1572632692 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f200a47c0 con 0x5641f3824cb0 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f07fff640 1 -- 172.21.15.133:0/1572632692 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f200b9070 con 0x5641f3824cb0 2024-09-11T23:44:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f31dd5740 10 client.17379 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.747+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1572632692 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641f381f210 con 0x5641f3824cb0 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.748+0000 7f5f07fff640 1 -- 172.21.15.133:0/1572632692 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f200a47c0 con 0x5641f3824cb0 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.748+0000 7f5f07fff640 1 -- 172.21.15.133:0/1572632692 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5f200978a0 con 0x5641f3824cb0 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.748+0000 7f5f31dd5740 10 client.17379 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.748+0000 7f5f31dd5740 10 client.17379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.748+0000 7f5f31dd5740 20 client.17379 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.748+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5641f389fa40 0x5641f38a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.748+0000 7f5f31dd5740 4 client.17379 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.748+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1572632692 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641f3785d70 con 0x5641f389fa40 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.748+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5641f389fa40 0x5641f38a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.749+0000 7f5f07fff640 10 client.17379 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.749+0000 7f5f2cdd8640 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5641f389fa40 0x5641f38a1e30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad523a8d2dd2d9f server_cookie=dacdabc7fe5e70bd in_seq=0 out_seq=0 2024-09-11T23:44:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f07fff640 1 -- 172.21.15.133:0/1572632692 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641f3785d70 con 0x5641f389fa40 2024-09-11T23:44:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f07fff640 10 client.17379 handle_command_reply: tid=0 2024-09-11T23:44:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f31dd5740 1 client.17379 shutdown 2024-09-11T23:44:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1572632692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 msgr2=0x5641f381eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f381eba0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5f20051300 tx=0x7f5f2009ff80 comp rx=0 tx=0).stop 2024-09-11T23:44:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1572632692 shutdown_connections 2024-09-11T23:44:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5641f389fa40 0x5641f38a1e30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641f3824cb0 0x5641f381eba0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641f381a410 0x5641f381e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f31dd5740 1 --2- 172.21.15.133:0/1572632692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641f3819af0 0x5641f382d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f07fff640 0 client.17379 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1572632692 >> 172.21.15.133:0/1572632692 conn(0x5641f3816060 msgr2=0x5641f38149a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1572632692 shutdown_connections 2024-09-11T23:44:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.761+0000 7f5f31dd5740 1 -- 172.21.15.133:0/1572632692 wait complete. 2024-09-11T23:44:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f31dd5740 20 client.17379 trim_cache size 0 max 16384 2024-09-11T23:44:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:44:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:44:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:44:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 -- 172.21.15.133:0/1712561946 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5f100a2da0 msgr2=0x7f5f100a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5f100a2da0 0x7f5f100a5260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5f28102380 tx=0x7f5f1800bf90 comp rx=0 tx=0).stop 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 -- 172.21.15.133:0/1712561946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2810c0d0 msgr2=0x7f5f28101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2810c0d0 0x7f5f28101370 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000eaa0 tx=0x7f5f2000ef60 comp rx=0 tx=0).stop 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 -- 172.21.15.133:0/1712561946 shutdown_connections 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5f100a2da0 0x7f5f100a5260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2810c0d0 0x7f5f28101370 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f281035c0 0x7f5f28100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 --2- 172.21.15.133:0/1712561946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f28102ca0 0x7f5f281008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 -- 172.21.15.133:0/1712561946 >> 172.21.15.133:0/1712561946 conn(0x7f5f280fc6b0 msgr2=0x7f5f280fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 -- 172.21.15.133:0/1712561946 shutdown_connections 2024-09-11T23:44:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:07.762+0000 7f5f2f03c640 1 -- 172.21.15.133:0/1712561946 wait complete. 2024-09-11T23:44:08.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:08 smithi138 ceph-mon[35142]: pgmap v226: 97 pgs: 97 active+clean; 60 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 417 MiB/s wr, 491 op/s 2024-09-11T23:44:08.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:08 smithi133 ceph-mon[31430]: pgmap v226: 97 pgs: 97 active+clean; 60 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 417 MiB/s wr, 491 op/s 2024-09-11T23:44:09.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:08 smithi178 ceph-mon[35882]: pgmap v226: 97 pgs: 97 active+clean; 60 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 417 MiB/s wr, 491 op/s 2024-09-11T23:44:10.844 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:44:10.896 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:10 smithi133 ceph-mon[31430]: pgmap v227: 97 pgs: 97 active+clean; 60 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 369 MiB/s wr, 403 op/s 2024-09-11T23:44:10.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:10 smithi138 ceph-mon[35142]: pgmap v227: 97 pgs: 97 active+clean; 60 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 369 MiB/s wr, 403 op/s 2024-09-11T23:44:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:10 smithi178 ceph-mon[35882]: pgmap v227: 97 pgs: 97 active+clean; 60 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 369 MiB/s wr, 403 op/s 2024-09-11T23:44:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:11.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.922+0000 7f9a6cb44640 1 -- 172.21.15.133:0/3878069580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a68106f20 msgr2=0x7f9a6810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:11.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.922+0000 7f9a6cb44640 1 --2- 172.21.15.133:0/3878069580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a68106f20 0x7f9a6810eaa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009ca0 tx=0x7f9a6002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:44:11.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.922+0000 7f9a6cb44640 1 -- 172.21.15.133:0/3878069580 shutdown_connections 2024-09-11T23:44:11.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.922+0000 7f9a6cb44640 1 --2- 172.21.15.133:0/3878069580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a68106f20 0x7f9a6810eaa0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:11.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.922+0000 7f9a6cb44640 1 --2- 172.21.15.133:0/3878069580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a681027a0 0x7f9a681067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:11.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.922+0000 7f9a6cb44640 1 --2- 172.21.15.133:0/3878069580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a68101df0 0x7f9a681021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:11.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.922+0000 7f9a6cb44640 1 -- 172.21.15.133:0/3878069580 >> 172.21.15.133:0/3878069580 conn(0x7f9a680fd8f0 msgr2=0x7f9a680ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:11.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.922+0000 7f9a6cb44640 1 -- 172.21.15.133:0/3878069580 shutdown_connections 2024-09-11T23:44:11.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.922+0000 7f9a6cb44640 1 -- 172.21.15.133:0/3878069580 wait complete. 2024-09-11T23:44:11.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.923+0000 7f9a6cb44640 1 Processor -- start 2024-09-11T23:44:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.923+0000 7f9a6cb44640 1 -- start start 2024-09-11T23:44:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a6cb44640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a68101df0 0x7f9a68198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a6cb44640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a681027a0 0x7f9a68198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a6cb44640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a68106f20 0x7f9a6819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a6cb44640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68111f80 con 0x7f9a68106f20 2024-09-11T23:44:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a6cb44640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68111e00 con 0x7f9a681027a0 2024-09-11T23:44:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a6cb44640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68112100 con 0x7f9a68101df0 2024-09-11T23:44:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a66d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a68106f20 0x7f9a6819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a66d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a68106f20 0x7f9a6819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50452/0 (socket says 172.21.15.133:50452) 2024-09-11T23:44:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a66d9d640 1 -- 172.21.15.133:0/432546007 learned_addr learned my addr 172.21.15.133:0/432546007 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a5dd9b640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a681027a0 0x7f9a68198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a6659c640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a68101df0 0x7f9a68198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a5dd9b640 1 -- 172.21.15.133:0/432546007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a68101df0 msgr2=0x7f9a68198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a5dd9b640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a68101df0 0x7f9a68198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a5dd9b640 1 -- 172.21.15.133:0/432546007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a68106f20 msgr2=0x7f9a6819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a5dd9b640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a68106f20 0x7f9a6819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a5dd9b640 1 -- 172.21.15.133:0/432546007 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6819d3b0 con 0x7f9a681027a0 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a66d9d640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a68106f20 0x7f9a6819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a6659c640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a68101df0 0x7f9a68198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.924+0000 7f9a5dd9b640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a681027a0 0x7f9a68198940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000e9f0 tx=0x7f9a5000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.925+0000 7f9a5ffff640 1 -- 172.21.15.133:0/432546007 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5000ccc0 con 0x7f9a681027a0 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.925+0000 7f9a5ffff640 1 -- 172.21.15.133:0/432546007 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a50010910 con 0x7f9a681027a0 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.925+0000 7f9a6cb44640 1 -- 172.21.15.133:0/432546007 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6819d6a0 con 0x7f9a681027a0 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.925+0000 7f9a5ffff640 1 -- 172.21.15.133:0/432546007 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50018960 con 0x7f9a681027a0 2024-09-11T23:44:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.925+0000 7f9a6cb44640 1 -- 172.21.15.133:0/432546007 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a681a4f80 con 0x7f9a681027a0 2024-09-11T23:44:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.926+0000 7f9a6cb44640 1 -- 172.21.15.133:0/432546007 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a681021d0 con 0x7f9a681027a0 2024-09-11T23:44:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.927+0000 7f9a5ffff640 1 -- 172.21.15.133:0/432546007 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9a50010ac0 con 0x7f9a681027a0 2024-09-11T23:44:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.927+0000 7f9a5ffff640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a540a2cf0 0x7f9a540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.927+0000 7f9a6659c640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a540a2cf0 0x7f9a540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.928+0000 7f9a5ffff640 1 -- 172.21.15.133:0/432546007 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f9a500cfa90 con 0x7f9a681027a0 2024-09-11T23:44:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.928+0000 7f9a6659c640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a540a2cf0 0x7f9a540a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009fc0 tx=0x7f9a58005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:11.931+0000 7f9a5ffff640 1 -- 172.21.15.133:0/432546007 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a5009c070 con 0x7f9a681027a0 2024-09-11T23:44:12.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:11 smithi133 ceph-mon[31430]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:44:12.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.143+0000 7f9a6cb44640 1 -- 172.21.15.133:0/432546007 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a6802d7c0 con 0x7f9a681027a0 2024-09-11T23:44:12.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.144+0000 7f9a5ffff640 1 -- 172.21.15.133:0/432546007 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7f9a500a0370 con 0x7f9a681027a0 2024-09-11T23:44:12.145 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:44:12.145 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:44:12.146 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:44:12.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 -- 172.21.15.133:0/432546007 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a540a2cf0 msgr2=0x7f9a540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:12.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a540a2cf0 0x7f9a540a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009fc0 tx=0x7f9a58005f90 comp rx=0 tx=0).stop 2024-09-11T23:44:12.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 -- 172.21.15.133:0/432546007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a681027a0 msgr2=0x7f9a68198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a681027a0 0x7f9a68198940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000e9f0 tx=0x7f9a5000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:44:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 -- 172.21.15.133:0/432546007 shutdown_connections 2024-09-11T23:44:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a540a2cf0 0x7f9a540a51b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a68106f20 0x7f9a6819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a681027a0 0x7f9a68198940 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 --2- 172.21.15.133:0/432546007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a68101df0 0x7f9a68198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 -- 172.21.15.133:0/432546007 >> 172.21.15.133:0/432546007 conn(0x7f9a680fd8f0 msgr2=0x7f9a680fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 -- 172.21.15.133:0/432546007 shutdown_connections 2024-09-11T23:44:12.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:12.147+0000 7f9a6cb44640 1 -- 172.21.15.133:0/432546007 wait complete. 2024-09-11T23:44:12.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:11 smithi138 ceph-mon[35142]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:44:12.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:11 smithi178 ceph-mon[35882]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:44:12.839 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id d5d9c530 --release --await' 2024-09-11T23:44:12.839 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id d5d9c530 --release --await 2024-09-11T23:44:13.101 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:12 smithi138 ceph-mon[35142]: pgmap v229: 97 pgs: 97 active+clean; 60 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 673 KiB/s rd, 161 MiB/s wr, 282 op/s 2024-09-11T23:44:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:12 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/432546007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:44:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:12 smithi133 ceph-mon[31430]: pgmap v229: 97 pgs: 97 active+clean; 60 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 673 KiB/s rd, 161 MiB/s wr, 282 op/s 2024-09-11T23:44:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:12 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/432546007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:44:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:12 smithi178 ceph-mon[35882]: pgmap v229: 97 pgs: 97 active+clean; 60 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 673 KiB/s rd, 161 MiB/s wr, 282 op/s 2024-09-11T23:44:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:12 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/432546007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:44:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.571+0000 7f8106e9d640 1 -- 172.21.15.133:0/1847252327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8100102570 msgr2=0x7f81001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.571+0000 7f8106e9d640 1 --2- 172.21.15.133:0/1847252327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8100102570 0x7f81001029b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009900 tx=0x7f80ec02a480 comp rx=0 tx=0).stop 2024-09-11T23:44:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.571+0000 7f8106e9d640 1 -- 172.21.15.133:0/1847252327 shutdown_connections 2024-09-11T23:44:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.571+0000 7f8106e9d640 1 --2- 172.21.15.133:0/1847252327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102ef0 0x7f810010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.571+0000 7f8106e9d640 1 --2- 172.21.15.133:0/1847252327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8100102570 0x7f81001029b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.571+0000 7f8106e9d640 1 --2- 172.21.15.133:0/1847252327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8100100e20 0x7f8100101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.571+0000 7f8106e9d640 1 -- 172.21.15.133:0/1847252327 >> 172.21.15.133:0/1847252327 conn(0x7f81000fc7d0 msgr2=0x7f81000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.571+0000 7f8106e9d640 1 -- 172.21.15.133:0/1847252327 shutdown_connections 2024-09-11T23:44:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.571+0000 7f8106e9d640 1 -- 172.21.15.133:0/1847252327 wait complete. 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f8106e9d640 1 Processor -- start 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f8106e9d640 1 -- start start 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f8106e9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8100100e20 0x7f8100199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f8106e9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8100102570 0x7f8100199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f8106e9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102ef0 0x7f81001a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f8106e9d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8100110fa0 con 0x7f8100100e20 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f8106e9d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8100110e20 con 0x7f8100102ef0 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f8106e9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8100111120 con 0x7f8100102570 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f8104c39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8100100e20 0x7f8100199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f8104c39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8100100e20 0x7f8100199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45800/0 (socket says 172.21.15.133:45800) 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f8104c39640 1 -- 172.21.15.133:0/825235111 learned_addr learned my addr 172.21.15.133:0/825235111 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f810543a640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102ef0 0x7f81001a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.572+0000 7f80fffff640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8100102570 0x7f8100199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.573+0000 7f8104c39640 1 -- 172.21.15.133:0/825235111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8100102570 msgr2=0x7f8100199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.573+0000 7f8104c39640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8100102570 0x7f8100199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.573+0000 7f8104c39640 1 -- 172.21.15.133:0/825235111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102ef0 msgr2=0x7f81001a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.573+0000 7f8104c39640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102ef0 0x7f81001a44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.573+0000 7f8104c39640 1 -- 172.21.15.133:0/825235111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81001a49f0 con 0x7f8100100e20 2024-09-11T23:44:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.573+0000 7f8104c39640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8100100e20 0x7f8100199430 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f80f400b4a0 tx=0x7f80f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.573+0000 7f80fdffb640 1 -- 172.21.15.133:0/825235111 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4014070 con 0x7f8100100e20 2024-09-11T23:44:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.573+0000 7f80fdffb640 1 -- 172.21.15.133:0/825235111 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f4010af0 con 0x7f8100100e20 2024-09-11T23:44:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.573+0000 7f8106e9d640 1 -- 172.21.15.133:0/825235111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81001a4bc0 con 0x7f8100100e20 2024-09-11T23:44:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.573+0000 7f80fdffb640 1 -- 172.21.15.133:0/825235111 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4017d10 con 0x7f8100100e20 2024-09-11T23:44:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.573+0000 7f8106e9d640 1 -- 172.21.15.133:0/825235111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81001a50d0 con 0x7f8100100e20 2024-09-11T23:44:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.575+0000 7f80fdffb640 1 -- 172.21.15.133:0/825235111 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f80f4017070 con 0x7f8100100e20 2024-09-11T23:44:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f8109276740 1 Processor -- start 2024-09-11T23:44:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f8109276740 1 -- start start 2024-09-11T23:44:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f80fdffb640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80f00a2cf0 0x7f80f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f8109276740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed365300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f8109276740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed365840 0x5601ed374110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f8109276740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed374650 0x5601ed376a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f8109276740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5601ed2ca620 con 0x5601ed374650 2024-09-11T23:44:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f8109276740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5601ed271e50 con 0x5601ed365840 2024-09-11T23:44:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f8109276740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5601ed2c65e0 con 0x5601ed364f20 2024-09-11T23:44:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f810543a640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80f00a2cf0 0x7f80f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f80fdffb640 1 -- 172.21.15.133:0/825235111 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f80f40ce6a0 con 0x7f8100100e20 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f80fffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed365840 0x5601ed374110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f810543a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed374650 0x5601ed376a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f8104c39640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed365300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f810543a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed374650 0x5601ed376a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45802/0 (socket says 172.21.15.133:45802) 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f80fffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed365840 0x5601ed374110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:60764/0 (socket says 172.21.15.133:60764) 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.576+0000 7f810543a640 1 -- 172.21.15.133:0/652770589 learned_addr learned my addr 172.21.15.133:0/652770589 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.577+0000 7f80fffff640 1 -- 172.21.15.133:0/652770589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 msgr2=0x5601ed365300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.577+0000 7f80fffff640 1 --2- 172.21.15.133:0/652770589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed365300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.577+0000 7f80fffff640 1 -- 172.21.15.133:0/652770589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed374650 msgr2=0x5601ed376a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.577+0000 7f80fffff640 1 --2- 172.21.15.133:0/652770589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed374650 0x5601ed376a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.577+0000 7f80fffff640 1 -- 172.21.15.133:0/652770589 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601ed262d40 con 0x5601ed365840 2024-09-11T23:44:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.577+0000 7f810543a640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80f00a2cf0 0x7f80f00a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80f8012c80 tx=0x7f80f8009e80 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.577+0000 7f80fffff640 1 --2- 172.21.15.133:0/652770589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed365840 0x5601ed374110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009230 tx=0x7f80ec004190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ece4630558b6c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.577+0000 7f80eaffd640 1 -- 172.21.15.133:0/652770589 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec035030 con 0x5601ed365840 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.577+0000 7f80eaffd640 1 -- 172.21.15.133:0/652770589 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec005520 con 0x5601ed365840 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.577+0000 7f80eaffd640 1 -- 172.21.15.133:0/652770589 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec0035f0 con 0x5601ed365840 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.578+0000 7f8109276740 1 -- 172.21.15.133:0/652770589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed365840 msgr2=0x5601ed374110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.578+0000 7f8109276740 1 --2- 172.21.15.133:0/652770589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed365840 0x5601ed374110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009230 tx=0x7f80ec004190 comp rx=0 tx=0).stop 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.578+0000 7f8109276740 1 -- 172.21.15.133:0/652770589 shutdown_connections 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.578+0000 7f8109276740 1 --2- 172.21.15.133:0/652770589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed374650 0x5601ed376a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.578+0000 7f8109276740 1 --2- 172.21.15.133:0/652770589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed365840 0x5601ed374110 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.578+0000 7f8109276740 1 --2- 172.21.15.133:0/652770589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed365300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.578+0000 7f8109276740 1 -- 172.21.15.133:0/652770589 >> 172.21.15.133:0/652770589 conn(0x5601ed3615f0 msgr2=0x5601ed3619d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.578+0000 7f8109276740 1 -- 172.21.15.133:0/652770589 shutdown_connections 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.578+0000 7f8109276740 1 -- 172.21.15.133:0/652770589 wait complete. 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.578+0000 7f8109276740 1 -- start start 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.578+0000 7f8109276740 1 Processor -- start 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f8109276740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed3715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f8109276740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed371ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f8109276740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 0x5601ed372020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f8109276740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5601ed3771c0 con 0x5601ed365840 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f8109276740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5601ed252f10 con 0x5601ed374650 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f8109276740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5601ed377340 con 0x5601ed364f20 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f8109276740 10 client.0 _ll_register_callbacks cb 0x5601ed276db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f80fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed371ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f80fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed371ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45818/0 (socket says 172.21.15.133:45818) 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f80fffff640 1 -- 172.21.15.133:0/3671286446 learned_addr learned my addr 172.21.15.133:0/3671286446 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f810543a640 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 0x5601ed372020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f8104c39640 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed3715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f810543a640 1 -- 172.21.15.133:0/3671286446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 msgr2=0x5601ed3715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f810543a640 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed3715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f810543a640 1 -- 172.21.15.133:0/3671286446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 msgr2=0x5601ed371ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f810543a640 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed371ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f810543a640 1 -- 172.21.15.133:0/3671286446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601ecfd9580 con 0x5601ed374650 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f80fffff640 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed371ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f810543a640 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 0x5601ed372020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f80f80098b0 tx=0x7f80f8009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58a5a32ee2639a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.579+0000 7f80dffff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:44:13.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f80dffff640 1 -- 172.21.15.133:0/3671286446 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f802a410 con 0x5601ed374650 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f80dffff640 1 -- 172.21.15.133:0/3671286446 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f80f8020400 con 0x5601ed374650 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f8109276740 1 -- 172.21.15.133:0/3671286446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ec005520 con 0x5601ed374650 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f80dffff640 1 -- 172.21.15.133:0/3671286446 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f80f8020600 con 0x5601ed374650 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f8109276740 10 client.26888 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f8109276740 1 -- 172.21.15.133:0/3671286446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601ed0a3960 con 0x5601ed374650 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f80dffff640 1 -- 172.21.15.133:0/3671286446 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec005520 con 0x5601ed374650 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f80dffff640 1 -- 172.21.15.133:0/3671286446 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f80f801f070 con 0x5601ed374650 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f8109276740 10 client.26888 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f8109276740 10 client.26888 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f8109276740 20 client.26888 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f8109276740 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5601ed384150 0x5601ed386560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f8109276740 4 client.26888 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:13.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.580+0000 7f8109276740 1 -- 172.21.15.133:0/3671286446 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601ed2d7b90 con 0x5601ed384150 2024-09-11T23:44:13.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.581+0000 7f8104c39640 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5601ed384150 0x5601ed386560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.581+0000 7f8104c39640 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5601ed384150 0x5601ed386560 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ded6bd064cd9e97 server_cookie=bca3971d3da095b7 in_seq=0 out_seq=0 2024-09-11T23:44:13.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.581+0000 7f80dffff640 10 client.26888 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:13.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f80dffff640 1 -- 172.21.15.133:0/3671286446 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601ed2d7b90 con 0x5601ed384150 2024-09-11T23:44:13.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f80dffff640 10 client.26888 handle_command_reply: tid=0 2024-09-11T23:44:13.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 1 client.26888 shutdown 2024-09-11T23:44:13.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 1 -- 172.21.15.133:0/3671286446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 msgr2=0x5601ed372020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:13.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 0x5601ed372020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f80f80098b0 tx=0x7f80f8009600 comp rx=0 tx=0).stop 2024-09-11T23:44:13.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 1 -- 172.21.15.133:0/3671286446 shutdown_connections 2024-09-11T23:44:13.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5601ed384150 0x5601ed386560 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 0x5601ed372020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed371ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 1 --2- 172.21.15.133:0/3671286446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed3715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f80dffff640 0 client.26888 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:13.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 1 -- 172.21.15.133:0/3671286446 >> 172.21.15.133:0/3671286446 conn(0x5601ed3615f0 msgr2=0x5601ed367350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:13.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 1 -- 172.21.15.133:0/3671286446 shutdown_connections 2024-09-11T23:44:13.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 1 -- 172.21.15.133:0/3671286446 wait complete. 2024-09-11T23:44:13.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.582+0000 7f8109276740 20 client.26888 trim_cache size 0 max 16384 2024-09-11T23:44:13.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.599+0000 7f8109276740 1 Processor -- start 2024-09-11T23:44:13.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.599+0000 7f8109276740 1 -- start start 2024-09-11T23:44:13.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.599+0000 7f8109276740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed364f20 0x5601ed38be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8109276740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed38c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8109276740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed374650 0x5601ed36e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8109276740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81001ab610 con 0x5601ed365840 2024-09-11T23:44:13.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8109276740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81001ad150 con 0x5601ed364f20 2024-09-11T23:44:13.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8109276740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81001abce0 con 0x5601ed374650 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f80fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed38c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8104c39640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed364f20 0x5601ed38be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f80fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed38c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45832/0 (socket says 172.21.15.133:45832) 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f80fffff640 1 -- 172.21.15.133:0/3411127274 learned_addr learned my addr 172.21.15.133:0/3411127274 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f810543a640 1 --2- 172.21.15.133:0/3411127274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed374650 0x5601ed36e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8104c39640 1 -- 172.21.15.133:0/3411127274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed374650 msgr2=0x5601ed36e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8104c39640 1 --2- 172.21.15.133:0/3411127274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed374650 0x5601ed36e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8104c39640 1 -- 172.21.15.133:0/3411127274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 msgr2=0x5601ed38c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8104c39640 1 --2- 172.21.15.133:0/3411127274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed38c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8104c39640 1 -- 172.21.15.133:0/3411127274 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601ed361420 con 0x5601ed364f20 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f80fffff640 1 --2- 172.21.15.133:0/3411127274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed38c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f8104c39640 1 --2- 172.21.15.133:0/3411127274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed364f20 0x5601ed38be30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f80f40cbf10 tx=0x7f80f4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc4e339bb31717d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:13.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f80ea7fc640 1 -- 172.21.15.133:0/3411127274 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f401e040 con 0x5601ed364f20 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f80ea7fc640 1 -- 172.21.15.133:0/3411127274 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f40bb0a0 con 0x5601ed364f20 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.600+0000 7f80ea7fc640 1 -- 172.21.15.133:0/3411127274 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4097350 con 0x5601ed364f20 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.601+0000 7f8109276740 1 -- 172.21.15.133:0/3411127274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed364f20 msgr2=0x5601ed38be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.601+0000 7f8109276740 1 --2- 172.21.15.133:0/3411127274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed364f20 0x5601ed38be30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f80f40cbf10 tx=0x7f80f4000bc0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.601+0000 7f8109276740 1 -- 172.21.15.133:0/3411127274 shutdown_connections 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.601+0000 7f8109276740 1 --2- 172.21.15.133:0/3411127274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed374650 0x5601ed36e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.601+0000 7f8109276740 1 --2- 172.21.15.133:0/3411127274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed38c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.601+0000 7f8109276740 1 --2- 172.21.15.133:0/3411127274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed364f20 0x5601ed38be30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.601+0000 7f8109276740 1 -- 172.21.15.133:0/3411127274 >> 172.21.15.133:0/3411127274 conn(0x5601ed3615f0 msgr2=0x5601ed373620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.601+0000 7f8109276740 1 -- 172.21.15.133:0/3411127274 shutdown_connections 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.601+0000 7f8109276740 1 -- 172.21.15.133:0/3411127274 wait complete. 2024-09-11T23:44:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.601+0000 7f8109276740 1 -- start start 2024-09-11T23:44:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f8109276740 1 Processor -- start 2024-09-11T23:44:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f8109276740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed3785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f8109276740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed378b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f8109276740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 0x5601ed379050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f8109276740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81001aca80 con 0x5601ed365840 2024-09-11T23:44:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f8109276740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81001ac3b0 con 0x5601ed374650 2024-09-11T23:44:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f8109276740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81001a9610 con 0x5601ed364f20 2024-09-11T23:44:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f8109276740 10 client.0 _ll_register_callbacks cb 0x7f80f8009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f80fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed378b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f80fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed378b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45848/0 (socket says 172.21.15.133:45848) 2024-09-11T23:44:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f80fffff640 1 -- 172.21.15.133:0/3403666149 learned_addr learned my addr 172.21.15.133:0/3403666149 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f8104c39640 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed3785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f810543a640 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 0x5601ed379050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f80fffff640 1 -- 172.21.15.133:0/3403666149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 msgr2=0x5601ed3785d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f80fffff640 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed3785d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f80fffff640 1 -- 172.21.15.133:0/3403666149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 msgr2=0x5601ed379050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f80fffff640 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 0x5601ed379050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f80fffff640 1 -- 172.21.15.133:0/3403666149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601ed3ee0a0 con 0x5601ed365840 2024-09-11T23:44:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.602+0000 7f810543a640 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 0x5601ed379050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:44:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.603+0000 7f80fffff640 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed378b10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f80ec0096c0 tx=0x7f80ec005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5eb41469874ca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.603+0000 7f80dffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.603+0000 7f80dffff640 1 -- 172.21.15.133:0/3403666149 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec053060 con 0x5601ed365840 2024-09-11T23:44:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.603+0000 7f80dffff640 1 -- 172.21.15.133:0/3403666149 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f80ec004c10 con 0x5601ed365840 2024-09-11T23:44:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.603+0000 7f8109276740 1 -- 172.21.15.133:0/3403666149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f40bb0a0 con 0x5601ed365840 2024-09-11T23:44:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.603+0000 7f80dffff640 1 -- 172.21.15.133:0/3403666149 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f80ec035040 con 0x5601ed365840 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.603+0000 7f8109276740 10 client.17403 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.603+0000 7f8109276740 1 -- 172.21.15.133:0/3403666149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601ed3ef3c0 con 0x5601ed365840 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.603+0000 7f80dffff640 1 -- 172.21.15.133:0/3403666149 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f40bb0a0 con 0x5601ed365840 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.603+0000 7f80dffff640 1 -- 172.21.15.133:0/3403666149 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f80ec031640 con 0x5601ed365840 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.604+0000 7f8109276740 10 client.17403 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.604+0000 7f8109276740 10 client.17403 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.604+0000 7f8109276740 20 client.17403 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.604+0000 7f8109276740 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5601ed3ef6a0 0x5601ed3f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.604+0000 7f8109276740 4 client.17403 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d5d9c530", "release": true, "await": true}] 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.604+0000 7f8109276740 1 -- 172.21.15.133:0/3403666149 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d5d9c530", "release": true, "await": true}) -- 0x5601ed2d20a0 con 0x5601ed3ef6a0 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.604+0000 7f8104c39640 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5601ed3ef6a0 0x5601ed3f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.604+0000 7f80dffff640 10 client.17403 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:13.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.604+0000 7f8104c39640 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5601ed3ef6a0 0x5601ed3f1a90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1be6d5212fff756 server_cookie=f7b17f55ed7ba111 in_seq=0 out_seq=0 2024-09-11T23:44:13.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.608+0000 7f80dffff640 1 -- 172.21.15.133:0/3403666149 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5601ed2d20a0 con 0x5601ed3ef6a0 2024-09-11T23:44:13.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.608+0000 7f80dffff640 10 client.17403 handle_command_reply: tid=0 2024-09-11T23:44:13.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.608+0000 7f8109276740 1 client.17403 shutdown 2024-09-11T23:44:13.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.608+0000 7f8109276740 1 -- 172.21.15.133:0/3403666149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 msgr2=0x5601ed378b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:13.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.608+0000 7f8109276740 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed378b10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f80ec0096c0 tx=0x7f80ec005650 comp rx=0 tx=0).stop 2024-09-11T23:44:13.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.608+0000 7f8109276740 1 -- 172.21.15.133:0/3403666149 shutdown_connections 2024-09-11T23:44:13.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.608+0000 7f8109276740 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5601ed3ef6a0 0x5601ed3f1a90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.608+0000 7f8109276740 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601ed374650 0x5601ed379050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.608+0000 7f8109276740 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601ed365840 0x5601ed378b10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.608+0000 7f8109276740 1 --2- 172.21.15.133:0/3403666149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601ed364f20 0x5601ed3785d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.608+0000 7f8109276740 1 -- 172.21.15.133:0/3403666149 >> 172.21.15.133:0/3403666149 conn(0x5601ed3615f0 msgr2=0x5601ed366a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:13.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.609+0000 7f80dffff640 0 client.17403 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:13.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.609+0000 7f8109276740 1 -- 172.21.15.133:0/3403666149 shutdown_connections 2024-09-11T23:44:13.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.609+0000 7f8109276740 1 -- 172.21.15.133:0/3403666149 wait complete. 2024-09-11T23:44:13.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.609+0000 7f8109276740 20 client.17403 trim_cache size 0 max 16384 2024-09-11T23:44:13.613 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:44:13.613 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:44:13.613 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:44:13.613 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 28, 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "d5d9c530": { 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 28, 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 136.6, 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:44:13.614 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:44:13.615 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:44:13.615 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:13.615 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:13.615 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:13.615 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:13.615 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:13.615 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:44:13.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 -- 172.21.15.133:0/825235111 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80f00a2cf0 msgr2=0x7f80f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:13.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80f00a2cf0 0x7f80f00a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80f8012c80 tx=0x7f80f8009e80 comp rx=0 tx=0).stop 2024-09-11T23:44:13.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 -- 172.21.15.133:0/825235111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8100100e20 msgr2=0x7f8100199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:13.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8100100e20 0x7f8100199430 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f80f400b4a0 tx=0x7f80f400b960 comp rx=0 tx=0).stop 2024-09-11T23:44:13.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 -- 172.21.15.133:0/825235111 shutdown_connections 2024-09-11T23:44:13.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80f00a2cf0 0x7f80f00a51b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102ef0 0x7f81001a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8100102570 0x7f8100199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 --2- 172.21.15.133:0/825235111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8100100e20 0x7f8100199430 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:13.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 -- 172.21.15.133:0/825235111 >> 172.21.15.133:0/825235111 conn(0x7f81000fc7d0 msgr2=0x7f81000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:13.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 -- 172.21.15.133:0/825235111 shutdown_connections 2024-09-11T23:44:13.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:13.610+0000 7f8106e9d640 1 -- 172.21.15.133:0/825235111 wait complete. 2024-09-11T23:44:14.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:13 smithi133 ceph-mon[31430]: pgmap v230: 97 pgs: 97 active+clean; 60 GiB data, 206 GiB used, 866 GiB / 1.0 TiB avail; 433 KiB/s rd, 120 MiB/s wr, 336 op/s 2024-09-11T23:44:14.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:13 smithi178 ceph-mon[35882]: pgmap v230: 97 pgs: 97 active+clean; 60 GiB data, 206 GiB used, 866 GiB / 1.0 TiB avail; 433 KiB/s rd, 120 MiB/s wr, 336 op/s 2024-09-11T23:44:14.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:13 smithi138 ceph-mon[35142]: pgmap v230: 97 pgs: 97 active+clean; 60 GiB data, 206 GiB used, 866 GiB / 1.0 TiB avail; 433 KiB/s rd, 120 MiB/s wr, 336 op/s 2024-09-11T23:44:14.598 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd5d9c530', total seconds elapsed: 20.4 2024-09-11T23:44:16.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:16 smithi138 ceph-mon[35142]: pgmap v231: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 13 KiB/s rd, 123 MiB/s wr, 393 op/s 2024-09-11T23:44:16.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:16 smithi133 ceph-mon[31430]: pgmap v231: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 13 KiB/s rd, 123 MiB/s wr, 393 op/s 2024-09-11T23:44:17.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:16 smithi178 ceph-mon[35882]: pgmap v231: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 13 KiB/s rd, 123 MiB/s wr, 393 op/s 2024-09-11T23:44:19.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:18 smithi178 ceph-mon[35882]: pgmap v232: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 13 KiB/s rd, 143 MiB/s wr, 484 op/s 2024-09-11T23:44:19.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:18 smithi138 ceph-mon[35142]: pgmap v232: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 13 KiB/s rd, 143 MiB/s wr, 484 op/s 2024-09-11T23:44:19.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:18 smithi133 ceph-mon[31430]: pgmap v232: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 13 KiB/s rd, 143 MiB/s wr, 484 op/s 2024-09-11T23:44:20.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:19 smithi178 ceph-mon[35882]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:44:20.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:19 smithi178 ceph-mon[35882]: pgmap v234: 97 pgs: 97 active+clean; 61 GiB data, 209 GiB used, 863 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 448 MiB/s wr, 2.25k op/s 2024-09-11T23:44:20.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:19 smithi138 ceph-mon[35142]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:44:20.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:19 smithi138 ceph-mon[35142]: pgmap v234: 97 pgs: 97 active+clean; 61 GiB data, 209 GiB used, 863 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 448 MiB/s wr, 2.25k op/s 2024-09-11T23:44:20.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:19 smithi133 ceph-mon[31430]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:44:20.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:19 smithi133 ceph-mon[31430]: pgmap v234: 97 pgs: 97 active+clean; 61 GiB data, 209 GiB used, 863 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 448 MiB/s wr, 2.25k op/s 2024-09-11T23:44:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.005+0000 7ff703c01640 1 -- 172.21.15.133:0/238404466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc11b140 msgr2=0x7ff6fc11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.005+0000 7ff703c01640 1 --2- 172.21.15.133:0/238404466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc11b140 0x7ff6fc11d530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009ca0 tx=0x7ff6f402a8f0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.007+0000 7ff703c01640 1 -- 172.21.15.133:0/238404466 shutdown_connections 2024-09-11T23:44:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.007+0000 7ff703c01640 1 --2- 172.21.15.133:0/238404466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc11b140 0x7ff6fc11d530 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.007+0000 7ff703c01640 1 --2- 172.21.15.133:0/238404466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6fc1095d0 0x7ff6fc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.007+0000 7ff703c01640 1 --2- 172.21.15.133:0/238404466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6fc1091f0 0x7ff6fc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.007+0000 7ff703c01640 1 -- 172.21.15.133:0/238404466 >> 172.21.15.133:0/238404466 conn(0x7ff6fc06cc60 msgr2=0x7ff6fc06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.007+0000 7ff703c01640 1 -- 172.21.15.133:0/238404466 shutdown_connections 2024-09-11T23:44:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.007+0000 7ff703c01640 1 -- 172.21.15.133:0/238404466 wait complete. 2024-09-11T23:44:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.008+0000 7ff703c01640 1 Processor -- start 2024-09-11T23:44:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.008+0000 7ff703c01640 1 -- start start 2024-09-11T23:44:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.008+0000 7ff703c01640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc1091f0 0x7ff6fc118640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.008+0000 7ff703c01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6fc1095d0 0x7ff6fc118b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.009+0000 7ff703c01640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6fc11b140 0x7ff6fc1118a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.009+0000 7ff703c01640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc120b80 con 0x7ff6fc11b140 2024-09-11T23:44:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.009+0000 7ff703c01640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc120a00 con 0x7ff6fc1095d0 2024-09-11T23:44:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.009+0000 7ff703c01640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc120d00 con 0x7ff6fc1091f0 2024-09-11T23:44:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.009+0000 7ff70119c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6fc1095d0 0x7ff6fc118b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.009+0000 7ff70119c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6fc1095d0 0x7ff6fc118b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:60816/0 (socket says 172.21.15.133:60816) 2024-09-11T23:44:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.010+0000 7ff70119c640 1 -- 172.21.15.133:0/1120547645 learned_addr learned my addr 172.21.15.133:0/1120547645 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.010+0000 7ff70219e640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6fc11b140 0x7ff6fc1118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.010+0000 7ff70199d640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc1091f0 0x7ff6fc118640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.010+0000 7ff70119c640 1 -- 172.21.15.133:0/1120547645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc1091f0 msgr2=0x7ff6fc118640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.010+0000 7ff70119c640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc1091f0 0x7ff6fc118640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.010+0000 7ff70119c640 1 -- 172.21.15.133:0/1120547645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6fc11b140 msgr2=0x7ff6fc1118a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.010+0000 7ff70119c640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6fc11b140 0x7ff6fc1118a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.010+0000 7ff70119c640 1 -- 172.21.15.133:0/1120547645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6fc112160 con 0x7ff6fc1095d0 2024-09-11T23:44:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.011+0000 7ff70119c640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6fc1095d0 0x7ff6fc118b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800b4a0 tx=0x7ff6f800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.015+0000 7ff6f2ffd640 1 -- 172.21.15.133:0/1120547645 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8014070 con 0x7ff6fc1095d0 2024-09-11T23:44:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.015+0000 7ff6f2ffd640 1 -- 172.21.15.133:0/1120547645 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6f8010af0 con 0x7ff6fc1095d0 2024-09-11T23:44:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.015+0000 7ff6f2ffd640 1 -- 172.21.15.133:0/1120547645 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8017d10 con 0x7ff6fc1095d0 2024-09-11T23:44:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.015+0000 7ff703c01640 1 -- 172.21.15.133:0/1120547645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6fc112450 con 0x7ff6fc1095d0 2024-09-11T23:44:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.016+0000 7ff703c01640 1 -- 172.21.15.133:0/1120547645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6fc1163a0 con 0x7ff6fc1095d0 2024-09-11T23:44:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.023+0000 7ff6f2ffd640 1 -- 172.21.15.133:0/1120547645 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff6f8017070 con 0x7ff6fc1095d0 2024-09-11T23:44:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.031+0000 7ff706a72740 1 Processor -- start 2024-09-11T23:44:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.031+0000 7ff706a72740 1 -- start start 2024-09-11T23:44:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.033+0000 7ff706a72740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 0x55bcac3d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.033+0000 7ff706a72740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3d65e0 0x55bcac3e0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.033+0000 7ff706a72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3e0e80 0x55bcac3e3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.033+0000 7ff706a72740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bcac33a2f0 con 0x55bcac3d5cc0 2024-09-11T23:44:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.033+0000 7ff706a72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bcac2e2270 con 0x55bcac3e0e80 2024-09-11T23:44:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.033+0000 7ff706a72740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bcac3362b0 con 0x55bcac3d65e0 2024-09-11T23:44:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.033+0000 7ff6f2ffd640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6e80a2f00 0x7ff6e80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.034+0000 7ff70199d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3d65e0 0x55bcac3e0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.034+0000 7ff70199d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3d65e0 0x55bcac3e0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40770/0 (socket says 172.21.15.133:40770) 2024-09-11T23:44:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.034+0000 7ff70199d640 1 -- 172.21.15.133:0/1170682696 learned_addr learned my addr 172.21.15.133:0/1170682696 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.035+0000 7ff70219e640 1 --2- 172.21.15.133:0/1170682696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 0x55bcac3d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.035+0000 7ff70199d640 1 -- 172.21.15.133:0/1170682696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3e0e80 msgr2=0x55bcac3e3270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:44:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.035+0000 7ff70199d640 1 --2- 172.21.15.133:0/1170682696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3e0e80 0x55bcac3e3270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.036+0000 7ff70199d640 1 -- 172.21.15.133:0/1170682696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 msgr2=0x55bcac3d60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.036+0000 7ff70199d640 1 --2- 172.21.15.133:0/1170682696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 0x55bcac3d60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.036+0000 7ff70199d640 1 -- 172.21.15.133:0/1170682696 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcac2e1340 con 0x55bcac3d65e0 2024-09-11T23:44:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.036+0000 7ff70219e640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6e80a2f00 0x7ff6e80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.037+0000 7ff70199d640 1 --2- 172.21.15.133:0/1170682696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3d65e0 0x55bcac3e0940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00cbf0 tx=0x7ff6ec00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd47b99a8ab91937 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.038+0000 7ff6f2ffd640 1 -- 172.21.15.133:0/1120547645 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7ff6f80cf6b0 con 0x7ff6fc1095d0 2024-09-11T23:44:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.038+0000 7ff6e7fff640 1 -- 172.21.15.133:0/1170682696 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec010630 con 0x55bcac3d65e0 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.038+0000 7ff6e7fff640 1 -- 172.21.15.133:0/1170682696 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ec010cb0 con 0x55bcac3d65e0 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.038+0000 7ff6e7fff640 1 -- 172.21.15.133:0/1170682696 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec019cb0 con 0x55bcac3d65e0 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.039+0000 7ff706a72740 1 -- 172.21.15.133:0/1170682696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3d65e0 msgr2=0x55bcac3e0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.039+0000 7ff706a72740 1 --2- 172.21.15.133:0/1170682696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3d65e0 0x55bcac3e0940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00cbf0 tx=0x7ff6ec00e720 comp rx=0 tx=0).stop 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.039+0000 7ff706a72740 1 -- 172.21.15.133:0/1170682696 shutdown_connections 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.039+0000 7ff706a72740 1 --2- 172.21.15.133:0/1170682696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3e0e80 0x55bcac3e3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.039+0000 7ff706a72740 1 --2- 172.21.15.133:0/1170682696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3d65e0 0x55bcac3e0940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.039+0000 7ff706a72740 1 --2- 172.21.15.133:0/1170682696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 0x55bcac3d60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.040+0000 7ff706a72740 1 -- 172.21.15.133:0/1170682696 >> 172.21.15.133:0/1170682696 conn(0x55bcac3d2230 msgr2=0x55bcac3d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.041+0000 7ff706a72740 1 -- 172.21.15.133:0/1170682696 shutdown_connections 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.041+0000 7ff706a72740 1 -- 172.21.15.133:0/1170682696 wait complete. 2024-09-11T23:44:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.042+0000 7ff706a72740 1 -- start start 2024-09-11T23:44:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.042+0000 7ff706a72740 1 Processor -- start 2024-09-11T23:44:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.044+0000 7ff706a72740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 0x55bcac3dddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.044+0000 7ff706a72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 0x55bcac3de330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.044+0000 7ff706a72740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3e0e80 0x55bcac3de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.044+0000 7ff706a72740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bcac3e3a50 con 0x55bcac3d5cc0 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.044+0000 7ff706a72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bcac2c3290 con 0x55bcac3d65e0 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.044+0000 7ff706a72740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bcac3e3bd0 con 0x55bcac3e0e80 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.044+0000 7ff706a72740 10 client.0 _ll_register_callbacks cb 0x55bcac2f4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.044+0000 7ff70119c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 0x55bcac3de330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.045+0000 7ff70119c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 0x55bcac3de330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:60832/0 (socket says 172.21.15.133:60832) 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.045+0000 7ff70119c640 1 -- 172.21.15.133:0/1881970807 learned_addr learned my addr 172.21.15.133:0/1881970807 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.046+0000 7ff70119c640 1 -- 172.21.15.133:0/1881970807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3e0e80 msgr2=0x55bcac3de870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.046+0000 7ff70119c640 1 --2- 172.21.15.133:0/1881970807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3e0e80 0x55bcac3de870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.046+0000 7ff70119c640 1 -- 172.21.15.133:0/1881970807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 msgr2=0x55bcac3dddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.046+0000 7ff70119c640 1 --2- 172.21.15.133:0/1881970807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 0x55bcac3dddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.046+0000 7ff70119c640 1 -- 172.21.15.133:0/1881970807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcac04d210 con 0x55bcac3d65e0 2024-09-11T23:44:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.046+0000 7ff70219e640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6e80a2f00 0x7ff6e80a53c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009a60 tx=0x7ff6f40095c0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.048+0000 7ff70119c640 1 --2- 172.21.15.133:0/1881970807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 0x55bcac3de330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8000c00 tx=0x7ff6f809b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=309816c1df9c8f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.052+0000 7ff6e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:44:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.052+0000 7ff6e57fa640 1 -- 172.21.15.133:0/1881970807 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f801e070 con 0x55bcac3d65e0 2024-09-11T23:44:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.052+0000 7ff706a72740 1 -- 172.21.15.133:0/1881970807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ec010cb0 con 0x55bcac3d65e0 2024-09-11T23:44:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.052+0000 7ff6e57fa640 1 -- 172.21.15.133:0/1881970807 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7ff6f8010ca0 con 0x55bcac3d65e0 2024-09-11T23:44:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.053+0000 7ff6e57fa640 1 -- 172.21.15.133:0/1881970807 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6f80a4930 con 0x55bcac3d65e0 2024-09-11T23:44:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.053+0000 7ff706a72740 10 client.26900 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.053+0000 7ff706a72740 1 -- 172.21.15.133:0/1881970807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bcac119620 con 0x55bcac3d65e0 2024-09-11T23:44:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.054+0000 7ff6e57fa640 1 -- 172.21.15.133:0/1881970807 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ec010cb0 con 0x55bcac3d65e0 2024-09-11T23:44:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.056+0000 7ff6e57fa640 1 -- 172.21.15.133:0/1881970807 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff6f809b110 con 0x55bcac3d65e0 2024-09-11T23:44:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.056+0000 7ff706a72740 10 client.26900 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.056+0000 7ff706a72740 10 client.26900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.056+0000 7ff706a72740 20 client.26900 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.056+0000 7ff706a72740 1 --2- 172.21.15.133:0/1881970807 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bcac3f12a0 0x55bcac3f3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.056+0000 7ff706a72740 4 client.26900 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.056+0000 7ff706a72740 1 -- 172.21.15.133:0/1881970807 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcac347a20 con 0x55bcac3f12a0 2024-09-11T23:44:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.057+0000 7ff70199d640 1 --2- 172.21.15.133:0/1881970807 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bcac3f12a0 0x55bcac3f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.057+0000 7ff70199d640 1 --2- 172.21.15.133:0/1881970807 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bcac3f12a0 0x55bcac3f3740 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17029dddfd8a956d server_cookie=dca715a7c864ad63 in_seq=0 out_seq=0 2024-09-11T23:44:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.058+0000 7ff6e57fa640 10 client.26900 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.059+0000 7ff6e57fa640 1 -- 172.21.15.133:0/1881970807 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcac347a20 con 0x55bcac3f12a0 2024-09-11T23:44:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.059+0000 7ff6e57fa640 10 client.26900 handle_command_reply: tid=0 2024-09-11T23:44:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.059+0000 7ff706a72740 1 client.26900 shutdown 2024-09-11T23:44:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.060+0000 7ff706a72740 1 -- 172.21.15.133:0/1881970807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 msgr2=0x55bcac3de330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.060+0000 7ff706a72740 1 --2- 172.21.15.133:0/1881970807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 0x55bcac3de330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8000c00 tx=0x7ff6f809b710 comp rx=0 tx=0).stop 2024-09-11T23:44:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.060+0000 7ff706a72740 1 -- 172.21.15.133:0/1881970807 shutdown_connections 2024-09-11T23:44:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.060+0000 7ff706a72740 1 --2- 172.21.15.133:0/1881970807 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bcac3f12a0 0x55bcac3f3740 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.060+0000 7ff706a72740 1 --2- 172.21.15.133:0/1881970807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3e0e80 0x55bcac3de870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.060+0000 7ff706a72740 1 --2- 172.21.15.133:0/1881970807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 0x55bcac3de330 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.060+0000 7ff706a72740 1 --2- 172.21.15.133:0/1881970807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 0x55bcac3dddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.060+0000 7ff706a72740 1 -- 172.21.15.133:0/1881970807 >> 172.21.15.133:0/1881970807 conn(0x55bcac3d2230 msgr2=0x55bcac3d7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.060+0000 7ff6e57fa640 0 client.26900 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.060+0000 7ff706a72740 1 -- 172.21.15.133:0/1881970807 shutdown_connections 2024-09-11T23:44:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.060+0000 7ff706a72740 1 -- 172.21.15.133:0/1881970807 wait complete. 2024-09-11T23:44:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.061+0000 7ff706a72740 20 client.26900 trim_cache size 0 max 16384 2024-09-11T23:44:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.139+0000 7ff706a72740 1 Processor -- start 2024-09-11T23:44:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.143+0000 7ff706a72740 1 -- start start 2024-09-11T23:44:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.143+0000 7ff706a72740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 0x55bcac3d8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.144+0000 7ff706a72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 0x55bcac3de6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.144+0000 7ff706a72740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3e0e80 0x55bcac3f9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.144+0000 7ff706a72740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1b3a30 con 0x55bcac3d5cc0 2024-09-11T23:44:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.144+0000 7ff706a72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1b5570 con 0x55bcac3d65e0 2024-09-11T23:44:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.144+0000 7ff706a72740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1b4100 con 0x55bcac3e0e80 2024-09-11T23:44:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.144+0000 7ff70219e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3e0e80 0x55bcac3f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.144+0000 7ff70219e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3e0e80 0x55bcac3f9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40784/0 (socket says 172.21.15.133:40784) 2024-09-11T23:44:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.144+0000 7ff70219e640 1 -- 172.21.15.133:0/3689518057 learned_addr learned my addr 172.21.15.133:0/3689518057 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.145+0000 7ff70199d640 1 --2- 172.21.15.133:0/3689518057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 0x55bcac3d8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.145+0000 7ff70119c640 1 --2- 172.21.15.133:0/3689518057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 0x55bcac3de6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.145+0000 7ff70219e640 1 -- 172.21.15.133:0/3689518057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 msgr2=0x55bcac3de6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.145+0000 7ff70219e640 1 --2- 172.21.15.133:0/3689518057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 0x55bcac3de6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.145+0000 7ff70219e640 1 -- 172.21.15.133:0/3689518057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 msgr2=0x55bcac3d8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.145+0000 7ff70219e640 1 --2- 172.21.15.133:0/3689518057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 0x55bcac3d8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.145+0000 7ff70219e640 1 -- 172.21.15.133:0/3689518057 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcac3d33a0 con 0x55bcac3e0e80 2024-09-11T23:44:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.147+0000 7ff70219e640 1 --2- 172.21.15.133:0/3689518057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3e0e80 0x55bcac3f9050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4032e90 tx=0x7ff6f403b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e4ee39c0148fc9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.148+0000 7ff6e77fe640 1 -- 172.21.15.133:0/3689518057 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4003970 con 0x55bcac3e0e80 2024-09-11T23:44:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.148+0000 7ff6e77fe640 1 -- 172.21.15.133:0/3689518057 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6f404e070 con 0x55bcac3e0e80 2024-09-11T23:44:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.148+0000 7ff6e77fe640 1 -- 172.21.15.133:0/3689518057 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f402f8a0 con 0x55bcac3e0e80 2024-09-11T23:44:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.151+0000 7ff706a72740 1 -- 172.21.15.133:0/3689518057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3e0e80 msgr2=0x55bcac3f9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.151+0000 7ff706a72740 1 --2- 172.21.15.133:0/3689518057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3e0e80 0x55bcac3f9050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4032e90 tx=0x7ff6f403b000 comp rx=0 tx=0).stop 2024-09-11T23:44:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.151+0000 7ff706a72740 1 -- 172.21.15.133:0/3689518057 shutdown_connections 2024-09-11T23:44:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.151+0000 7ff706a72740 1 --2- 172.21.15.133:0/3689518057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3e0e80 0x55bcac3f9050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.151+0000 7ff706a72740 1 --2- 172.21.15.133:0/3689518057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d65e0 0x55bcac3de6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.151+0000 7ff706a72740 1 --2- 172.21.15.133:0/3689518057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3d5cc0 0x55bcac3d8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.151+0000 7ff706a72740 1 -- 172.21.15.133:0/3689518057 >> 172.21.15.133:0/3689518057 conn(0x55bcac3d2230 msgr2=0x55bcac3d9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.161+0000 7ff706a72740 1 -- 172.21.15.133:0/3689518057 shutdown_connections 2024-09-11T23:44:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.161+0000 7ff706a72740 1 -- 172.21.15.133:0/3689518057 wait complete. 2024-09-11T23:44:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.162+0000 7ff706a72740 1 -- start start 2024-09-11T23:44:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.162+0000 7ff706a72740 1 Processor -- start 2024-09-11T23:44:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.164+0000 7ff706a72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d5cc0 0x55bcac3e8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.164+0000 7ff706a72740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3d65e0 0x55bcac3e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.164+0000 7ff706a72740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3e0e80 0x55bcac3da040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.164+0000 7ff706a72740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1b4ea0 con 0x55bcac3e0e80 2024-09-11T23:44:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.164+0000 7ff706a72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1b47d0 con 0x55bcac3d5cc0 2024-09-11T23:44:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.164+0000 7ff706a72740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1b1ac0 con 0x55bcac3d65e0 2024-09-11T23:44:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.164+0000 7ff706a72740 10 client.0 _ll_register_callbacks cb 0x7ff6f8017eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.164+0000 7ff70219e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3e0e80 0x55bcac3da040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.164+0000 7ff70219e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3e0e80 0x55bcac3da040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45926/0 (socket says 172.21.15.133:45926) 2024-09-11T23:44:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.165+0000 7ff70219e640 1 -- 172.21.15.133:0/2173044242 learned_addr learned my addr 172.21.15.133:0/2173044242 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.165+0000 7ff70219e640 1 -- 172.21.15.133:0/2173044242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3d65e0 msgr2=0x55bcac3e93a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.165+0000 7ff70219e640 1 --2- 172.21.15.133:0/2173044242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3d65e0 0x55bcac3e93a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.165+0000 7ff70219e640 1 -- 172.21.15.133:0/2173044242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d5cc0 msgr2=0x55bcac3e8e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.165+0000 7ff70219e640 1 --2- 172.21.15.133:0/2173044242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d5cc0 0x55bcac3e8e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.165+0000 7ff70219e640 1 -- 172.21.15.133:0/2173044242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcac3dbd40 con 0x55bcac3e0e80 2024-09-11T23:44:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.167+0000 7ff70219e640 1 --2- 172.21.15.133:0/2173044242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3e0e80 0x55bcac3da040 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4031850 tx=0x7ff6f402adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825d88520bcdaef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.167+0000 7ff6e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.168+0000 7ff6e57fa640 1 -- 172.21.15.133:0/2173044242 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f405b020 con 0x55bcac3e0e80 2024-09-11T23:44:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.169+0000 7ff706a72740 1 -- 172.21.15.133:0/2173044242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6f404e070 con 0x55bcac3e0e80 2024-09-11T23:44:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.169+0000 7ff6e57fa640 1 -- 172.21.15.133:0/2173044242 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7ff6f4003b80 con 0x55bcac3e0e80 2024-09-11T23:44:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.170+0000 7ff6e57fa640 1 -- 172.21.15.133:0/2173044242 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6f4066050 con 0x55bcac3e0e80 2024-09-11T23:44:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.171+0000 7ff706a72740 10 client.17412 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.171+0000 7ff706a72740 1 -- 172.21.15.133:0/2173044242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bcac45b520 con 0x55bcac3e0e80 2024-09-11T23:44:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.173+0000 7ff6e57fa640 1 -- 172.21.15.133:0/2173044242 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6f4036040 con 0x55bcac3e0e80 2024-09-11T23:44:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.173+0000 7ff6e57fa640 1 -- 172.21.15.133:0/2173044242 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff6f40305f0 con 0x55bcac3e0e80 2024-09-11T23:44:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.173+0000 7ff706a72740 10 client.17412 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.173+0000 7ff706a72740 10 client.17412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.173+0000 7ff706a72740 20 client.17412 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.173+0000 7ff706a72740 1 --2- 172.21.15.133:0/2173044242 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bcac45bb90 0x55bcac45dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.174+0000 7ff706a72740 4 client.17412 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:44:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.174+0000 7ff706a72740 1 -- 172.21.15.133:0/2173044242 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff6f404e070 con 0x55bcac45bb90 2024-09-11T23:44:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.188+0000 7ff70199d640 1 --2- 172.21.15.133:0/2173044242 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bcac45bb90 0x55bcac45dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.194+0000 7ff70199d640 1 --2- 172.21.15.133:0/2173044242 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bcac45bb90 0x55bcac45dfb0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb9caccf47df49d server_cookie=91968cbb1f8abfef in_seq=0 out_seq=0 2024-09-11T23:44:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:21.194+0000 7ff6e57fa640 10 client.17412 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:22.946 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:22 smithi133 ceph-mon[31430]: pgmap v235: 97 pgs: 97 active+clean; 61 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 411 MiB/s wr, 2.09k op/s 2024-09-11T23:44:22.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:22 smithi138 ceph-mon[35142]: pgmap v235: 97 pgs: 97 active+clean; 61 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 411 MiB/s wr, 2.09k op/s 2024-09-11T23:44:23.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:22 smithi178 ceph-mon[35882]: pgmap v235: 97 pgs: 97 active+clean; 61 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 411 MiB/s wr, 2.09k op/s 2024-09-11T23:44:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.790+0000 7ff6e57fa640 1 -- 172.21.15.133:0/2173044242 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff6f404e070 con 0x55bcac45bb90 2024-09-11T23:44:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.790+0000 7ff6e57fa640 10 client.17412 handle_command_reply: tid=0 2024-09-11T23:44:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.790+0000 7ff706a72740 1 client.17412 shutdown 2024-09-11T23:44:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.790+0000 7ff706a72740 1 -- 172.21.15.133:0/2173044242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3e0e80 msgr2=0x55bcac3da040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.790+0000 7ff706a72740 1 --2- 172.21.15.133:0/2173044242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3e0e80 0x55bcac3da040 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4031850 tx=0x7ff6f402adf0 comp rx=0 tx=0).stop 2024-09-11T23:44:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.791+0000 7ff706a72740 1 -- 172.21.15.133:0/2173044242 shutdown_connections 2024-09-11T23:44:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.791+0000 7ff706a72740 1 --2- 172.21.15.133:0/2173044242 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bcac45bb90 0x55bcac45dfb0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.791+0000 7ff706a72740 1 --2- 172.21.15.133:0/2173044242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcac3e0e80 0x55bcac3da040 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.791+0000 7ff706a72740 1 --2- 172.21.15.133:0/2173044242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcac3d65e0 0x55bcac3e93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.791+0000 7ff706a72740 1 --2- 172.21.15.133:0/2173044242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcac3d5cc0 0x55bcac3e8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.791+0000 7ff706a72740 1 -- 172.21.15.133:0/2173044242 >> 172.21.15.133:0/2173044242 conn(0x55bcac3d2230 msgr2=0x55bcac3d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.791+0000 7ff6e57fa640 0 client.17412 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.791+0000 7ff706a72740 1 -- 172.21.15.133:0/2173044242 shutdown_connections 2024-09-11T23:44:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.791+0000 7ff706a72740 1 -- 172.21.15.133:0/2173044242 wait complete. 2024-09-11T23:44:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.791+0000 7ff706a72740 20 client.17412 trim_cache size 0 max 16384 2024-09-11T23:44:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:44:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:44:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:44:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:44:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 -- 172.21.15.133:0/1120547645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6e80a2f00 msgr2=0x7ff6e80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6e80a2f00 0x7ff6e80a53c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009a60 tx=0x7ff6f40095c0 comp rx=0 tx=0).stop 2024-09-11T23:44:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 -- 172.21.15.133:0/1120547645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6fc1095d0 msgr2=0x7ff6fc118b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6fc1095d0 0x7ff6fc118b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800b4a0 tx=0x7ff6f800b960 comp rx=0 tx=0).stop 2024-09-11T23:44:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 -- 172.21.15.133:0/1120547645 shutdown_connections 2024-09-11T23:44:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff6e80a2f00 0x7ff6e80a53c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6fc11b140 0x7ff6fc1118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6fc1095d0 0x7ff6fc118b80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 --2- 172.21.15.133:0/1120547645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6fc1091f0 0x7ff6fc118640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 -- 172.21.15.133:0/1120547645 >> 172.21.15.133:0/1120547645 conn(0x7ff6fc06cc60 msgr2=0x7ff6fc11b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 -- 172.21.15.133:0/1120547645 shutdown_connections 2024-09-11T23:44:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:23.793+0000 7ff703c01640 1 -- 172.21.15.133:0/1120547645 wait complete. 2024-09-11T23:44:24.812 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:44:24.939 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:24 smithi133 ceph-mon[31430]: pgmap v236: 97 pgs: 97 active+clean; 62 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 511 MiB/s wr, 3.10k op/s 2024-09-11T23:44:24.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:24 smithi138 ceph-mon[35142]: pgmap v236: 97 pgs: 97 active+clean; 62 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 511 MiB/s wr, 3.10k op/s 2024-09-11T23:44:25.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:24 smithi178 ceph-mon[35882]: pgmap v236: 97 pgs: 97 active+clean; 62 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 511 MiB/s wr, 3.10k op/s 2024-09-11T23:44:26.009 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:26.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:26 smithi178 ceph-mon[35882]: pgmap v237: 97 pgs: 97 active+clean; 63 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 639 MiB/s wr, 3.99k op/s 2024-09-11T23:44:26.334 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:26 smithi133 ceph-mon[31430]: pgmap v237: 97 pgs: 97 active+clean; 63 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 639 MiB/s wr, 3.99k op/s 2024-09-11T23:44:26.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:26 smithi138 ceph-mon[35142]: pgmap v237: 97 pgs: 97 active+clean; 63 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 639 MiB/s wr, 3.99k op/s 2024-09-11T23:44:27.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.056+0000 7fcf68f63640 1 -- 172.21.15.133:0/2061778504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf6411b140 msgr2=0x7fcf6411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:27.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.056+0000 7fcf68f63640 1 --2- 172.21.15.133:0/2061778504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf6411b140 0x7fcf6411d530 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009be0 tx=0x7fcf5402a6c0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.057+0000 7fcf68f63640 1 -- 172.21.15.133:0/2061778504 shutdown_connections 2024-09-11T23:44:27.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.057+0000 7fcf68f63640 1 --2- 172.21.15.133:0/2061778504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf6411b140 0x7fcf6411d530 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.057+0000 7fcf68f63640 1 --2- 172.21.15.133:0/2061778504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf6406ea30 0x7fcf6406ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.057+0000 7fcf68f63640 1 --2- 172.21.15.133:0/2061778504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf64110220 0x7fcf6406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.057+0000 7fcf68f63640 1 -- 172.21.15.133:0/2061778504 >> 172.21.15.133:0/2061778504 conn(0x7fcf64073210 msgr2=0x7fcf64073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:27.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.058+0000 7fcf68f63640 1 -- 172.21.15.133:0/2061778504 shutdown_connections 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.058+0000 7fcf68f63640 1 -- 172.21.15.133:0/2061778504 wait complete. 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.059+0000 7fcf68f63640 1 Processor -- start 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.059+0000 7fcf68f63640 1 -- start start 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.059+0000 7fcf68f63640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf6406ea30 0x7fcf641a7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.059+0000 7fcf68f63640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf64110220 0x7fcf641a7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.059+0000 7fcf68f63640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf6411b140 0x7fcf641aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.059+0000 7fcf68f63640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf64120d10 con 0x7fcf6406ea30 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.059+0000 7fcf68f63640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf64120b90 con 0x7fcf6411b140 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.059+0000 7fcf68f63640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf64120e90 con 0x7fcf64110220 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.060+0000 7fcf62d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf6411b140 0x7fcf641aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.060+0000 7fcf62d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf6411b140 0x7fcf641aba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:41266/0 (socket says 172.21.15.133:41266) 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.060+0000 7fcf62d9d640 1 -- 172.21.15.133:0/1677899256 learned_addr learned my addr 172.21.15.133:0/1677899256 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.061+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf64110220 0x7fcf641a7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.062+0000 7fcf62d9d640 1 -- 172.21.15.133:0/1677899256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf64110220 msgr2=0x7fcf641a7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.062+0000 7fcf62d9d640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf64110220 0x7fcf641a7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.062+0000 7fcf62d9d640 1 -- 172.21.15.133:0/1677899256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf6406ea30 msgr2=0x7fcf641a7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.062+0000 7fcf62d9d640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf6406ea30 0x7fcf641a7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.062+0000 7fcf62d9d640 1 -- 172.21.15.133:0/1677899256 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf641ac100 con 0x7fcf6411b140 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.062+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf64110220 0x7fcf641a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.062+0000 7fcf62d9d640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf6411b140 0x7fcf641aba00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009b30 tx=0x7fcf540305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.063+0000 7fcf5b7fe640 1 -- 172.21.15.133:0/1677899256 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54041020 con 0x7fcf6411b140 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.063+0000 7fcf5b7fe640 1 -- 172.21.15.133:0/1677899256 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf54003e50 con 0x7fcf6411b140 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.063+0000 7fcf68f63640 1 -- 172.21.15.133:0/1677899256 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf641ac390 con 0x7fcf6411b140 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.063+0000 7fcf68f63640 1 -- 172.21.15.133:0/1677899256 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf64110920 con 0x7fcf6411b140 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.065+0000 7fcf5b7fe640 1 -- 172.21.15.133:0/1677899256 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54041020 con 0x7fcf6411b140 2024-09-11T23:44:27.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.069+0000 7fcf6b33c740 1 Processor -- start 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.072+0000 7fcf5b7fe640 1 -- 172.21.15.133:0/1677899256 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcf54044020 con 0x7fcf6411b140 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.073+0000 7fcf5b7fe640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf500a30f0 0x7fcf500a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.074+0000 7fcf5b7fe640 1 -- 172.21.15.133:0/1677899256 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fcf54030770 con 0x7fcf6411b140 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.074+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf500a30f0 0x7fcf500a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.075+0000 7fcf6b33c740 1 -- start start 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.076+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x56082733ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.076+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56082733bf90 0x5608273464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.076+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 0x560827348e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.076+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5608272a12e0 con 0x560827346a30 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.076+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560827248ce0 con 0x56082733bf90 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.076+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf500a30f0 0x7fcf500a55b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c006fd0 tx=0x7fcf4c008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.076+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56082729d2a0 con 0x56082733b670 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.078+0000 7fcf62d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 0x560827348e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.078+0000 7fcf61d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56082733bf90 0x5608273464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.078+0000 7fcf62d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 0x560827348e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53594/0 (socket says 172.21.15.133:53594) 2024-09-11T23:44:27.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.078+0000 7fcf62d9d640 1 -- 172.21.15.133:0/1012154483 learned_addr learned my addr 172.21.15.133:0/1012154483 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:27.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.081+0000 7fcf61d9b640 1 -- 172.21.15.133:0/1012154483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 msgr2=0x56082733ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:27.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.081+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/1012154483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x56082733ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.081+0000 7fcf61d9b640 1 -- 172.21.15.133:0/1012154483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 msgr2=0x560827348e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:27.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.081+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/1012154483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 0x560827348e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.081+0000 7fcf61d9b640 1 -- 172.21.15.133:0/1012154483 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082722d440 con 0x56082733bf90 2024-09-11T23:44:27.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.084+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/1012154483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56082733bf90 0x5608273464f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00ca30 tx=0x7fcf4c00f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=139c62b6b1b883ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:27.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.086+0000 7fcf68f63640 1 -- 172.21.15.133:0/1012154483 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c023070 con 0x56082733bf90 2024-09-11T23:44:27.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.086+0000 7fcf68f63640 1 -- 172.21.15.133:0/1012154483 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf4c013db0 con 0x56082733bf90 2024-09-11T23:44:27.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.087+0000 7fcf6b33c740 1 -- 172.21.15.133:0/1012154483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56082733bf90 msgr2=0x5608273464f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:27.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.087+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/1012154483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56082733bf90 0x5608273464f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00ca30 tx=0x7fcf4c00f000 comp rx=0 tx=0).stop 2024-09-11T23:44:27.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.089+0000 7fcf6b33c740 1 -- 172.21.15.133:0/1012154483 shutdown_connections 2024-09-11T23:44:27.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.089+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/1012154483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 0x560827348e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.089+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/1012154483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56082733bf90 0x5608273464f0 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00ca30 tx=0x7fcf4c00f000 comp rx=0 tx=0).stop 2024-09-11T23:44:27.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.089+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/1012154483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x56082733ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.090+0000 7fcf6b33c740 1 -- 172.21.15.133:0/1012154483 >> 172.21.15.133:0/1012154483 conn(0x560827337de0 msgr2=0x5608273381c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:27.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.091+0000 7fcf6b33c740 1 -- 172.21.15.133:0/1012154483 shutdown_connections 2024-09-11T23:44:27.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.093+0000 7fcf6b33c740 1 -- 172.21.15.133:0/1012154483 wait complete. 2024-09-11T23:44:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.094+0000 7fcf6b33c740 1 -- start start 2024-09-11T23:44:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.094+0000 7fcf6b33c740 1 Processor -- start 2024-09-11T23:44:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.095+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x5608273437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.095+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 0x560827343d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.095+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827352980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.095+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560827349480 con 0x560827346a30 2024-09-11T23:44:27.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.095+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56082725c050 con 0x560827344250 2024-09-11T23:44:27.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.095+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560827349600 con 0x56082733b670 2024-09-11T23:44:27.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.095+0000 7fcf6b33c740 10 client.0 _ll_register_callbacks cb 0x56082729c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:27.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.099+0000 7fcf62d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827352980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.099+0000 7fcf62d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827352980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:41284/0 (socket says 172.21.15.133:41284) 2024-09-11T23:44:27.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.099+0000 7fcf62d9d640 1 -- 172.21.15.133:0/4089415428 learned_addr learned my addr 172.21.15.133:0/4089415428 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.100+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 0x560827343d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.100+0000 7fcf62d9d640 1 -- 172.21.15.133:0/4089415428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 msgr2=0x5608273437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.100+0000 7fcf62d9d640 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x5608273437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.100+0000 7fcf62d9d640 1 -- 172.21.15.133:0/4089415428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 msgr2=0x560827343d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.100+0000 7fcf62d9d640 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 0x560827343d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.100+0000 7fcf62d9d640 1 -- 172.21.15.133:0/4089415428 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608270123c0 con 0x560827344250 2024-09-11T23:44:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.100+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 0x560827343d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:44:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.102+0000 7fcf62d9d640 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827352980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcf540b7f20 tx=0x7fcf540baf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127119bbb4dcb744 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.102+0000 7fcf49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:44:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.104+0000 7fcf49ffb640 1 -- 172.21.15.133:0/4089415428 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54041040 con 0x560827344250 2024-09-11T23:44:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.104+0000 7fcf6b33c740 1 -- 172.21.15.133:0/4089415428 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5608272ae850 con 0x560827344250 2024-09-11T23:44:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.104+0000 7fcf49ffb640 1 -- 172.21.15.133:0/4089415428 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fcf540b7bc0 con 0x560827344250 2024-09-11T23:44:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.104+0000 7fcf49ffb640 1 -- 172.21.15.133:0/4089415428 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf540bf900 con 0x560827344250 2024-09-11T23:44:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.105+0000 7fcf6b33c740 10 client.26915 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.105+0000 7fcf6b33c740 1 -- 172.21.15.133:0/4089415428 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560827349480 con 0x560827344250 2024-09-11T23:44:27.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.107+0000 7fcf49ffb640 1 -- 172.21.15.133:0/4089415428 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5608272ae850 con 0x560827344250 2024-09-11T23:44:27.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.108+0000 7fcf49ffb640 1 -- 172.21.15.133:0/4089415428 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcf540c3ca0 con 0x560827344250 2024-09-11T23:44:27.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.108+0000 7fcf6b33c740 10 client.26915 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:27.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.108+0000 7fcf6b33c740 10 client.26915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:27.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.109+0000 7fcf6b33c740 20 client.26915 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:27.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.109+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560827356990 0x560827358e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.109+0000 7fcf6b33c740 4 client.26915 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:27.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.109+0000 7fcf6b33c740 1 -- 172.21.15.133:0/4089415428 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608272a8d60 con 0x560827356990 2024-09-11T23:44:27.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.111+0000 7fcf6259c640 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560827356990 0x560827358e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.116+0000 7fcf6259c640 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560827356990 0x560827358e00 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe21f8aed8174666 server_cookie=495c9ede1a12d805 in_seq=0 out_seq=0 2024-09-11T23:44:27.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.116+0000 7fcf49ffb640 10 client.26915 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:27.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.119+0000 7fcf49ffb640 1 -- 172.21.15.133:0/4089415428 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608272a8d60 con 0x560827356990 2024-09-11T23:44:27.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.119+0000 7fcf49ffb640 10 client.26915 handle_command_reply: tid=0 2024-09-11T23:44:27.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.120+0000 7fcf6b33c740 1 client.26915 shutdown 2024-09-11T23:44:27.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.121+0000 7fcf6b33c740 1 -- 172.21.15.133:0/4089415428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 msgr2=0x560827352980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:27.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.121+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827352980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcf540b7f20 tx=0x7fcf540baf70 comp rx=0 tx=0).stop 2024-09-11T23:44:27.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.122+0000 7fcf6b33c740 1 -- 172.21.15.133:0/4089415428 shutdown_connections 2024-09-11T23:44:27.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.122+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560827356990 0x560827358e00 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.122+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827352980 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.122+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827346a30 0x560827343d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.122+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/4089415428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x5608273437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.122+0000 7fcf6b33c740 1 -- 172.21.15.133:0/4089415428 >> 172.21.15.133:0/4089415428 conn(0x560827337de0 msgr2=0x56082733d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:27.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.125+0000 7fcf49ffb640 0 client.26915 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:27.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.126+0000 7fcf6b33c740 1 -- 172.21.15.133:0/4089415428 shutdown_connections 2024-09-11T23:44:27.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.126+0000 7fcf6b33c740 1 -- 172.21.15.133:0/4089415428 wait complete. 2024-09-11T23:44:27.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.127+0000 7fcf6b33c740 20 client.26915 trim_cache size 0 max 16384 2024-09-11T23:44:27.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.182+0000 7fcf6b33c740 1 Processor -- start 2024-09-11T23:44:27.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.183+0000 7fcf6b33c740 1 -- start start 2024-09-11T23:44:27.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.183+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560827337de0 0x560827337300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.183+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56082733b670 0x560827342710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.183+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827343bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.183+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf64118e60 con 0x56082733b670 2024-09-11T23:44:27.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.183+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf641c6540 con 0x560827344250 2024-09-11T23:44:27.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.183+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf641c50d0 con 0x560827337de0 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.183+0000 7fcf62d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827343bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.184+0000 7fcf62d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827343bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:41296/0 (socket says 172.21.15.133:41296) 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.184+0000 7fcf62d9d640 1 -- 172.21.15.133:0/3761577809 learned_addr learned my addr 172.21.15.133:0/3761577809 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.184+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/3761577809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56082733b670 0x560827342710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.184+0000 7fcf62d9d640 1 -- 172.21.15.133:0/3761577809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560827337de0 msgr2=0x560827337300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.184+0000 7fcf62d9d640 1 --2- 172.21.15.133:0/3761577809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560827337de0 0x560827337300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.184+0000 7fcf62d9d640 1 -- 172.21.15.133:0/3761577809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56082733b670 msgr2=0x560827342710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.184+0000 7fcf62d9d640 1 --2- 172.21.15.133:0/3761577809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56082733b670 0x560827342710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.184+0000 7fcf62d9d640 1 -- 172.21.15.133:0/3761577809 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608273593e0 con 0x560827344250 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.184+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/3761577809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56082733b670 0x560827342710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.184+0000 7fcf62d9d640 1 --2- 172.21.15.133:0/3761577809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827343bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcf540bfac0 tx=0x7fcf540b9a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b7ed4ecf9c7c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.185+0000 7fcf4bfff640 1 -- 172.21.15.133:0/3761577809 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54041070 con 0x560827344250 2024-09-11T23:44:27.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.185+0000 7fcf4bfff640 1 -- 172.21.15.133:0/3761577809 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf540d4070 con 0x560827344250 2024-09-11T23:44:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.186+0000 7fcf6b33c740 1 -- 172.21.15.133:0/3761577809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 msgr2=0x560827343bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.186+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/3761577809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827343bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcf540bfac0 tx=0x7fcf540b9a60 comp rx=0 tx=0).stop 2024-09-11T23:44:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.191+0000 7fcf6b33c740 1 -- 172.21.15.133:0/3761577809 shutdown_connections 2024-09-11T23:44:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.191+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/3761577809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x560827343bf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.191+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/3761577809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56082733b670 0x560827342710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.191+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/3761577809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560827337de0 0x560827337300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.191+0000 7fcf6b33c740 1 -- 172.21.15.133:0/3761577809 >> 172.21.15.133:0/3761577809 conn(0x560827338860 msgr2=0x560827345030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:27.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.191+0000 7fcf6b33c740 1 -- 172.21.15.133:0/3761577809 shutdown_connections 2024-09-11T23:44:27.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.192+0000 7fcf6b33c740 1 -- 172.21.15.133:0/3761577809 wait complete. 2024-09-11T23:44:27.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.192+0000 7fcf6b33c740 1 -- start start 2024-09-11T23:44:27.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.192+0000 7fcf6b33c740 1 Processor -- start 2024-09-11T23:44:27.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.193+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827337de0 0x56082734beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.193+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x56082734c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.193+0000 7fcf6b33c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x5608273bf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.193+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf641c5e70 con 0x560827337de0 2024-09-11T23:44:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.193+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf641c57a0 con 0x560827344250 2024-09-11T23:44:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.193+0000 7fcf6b33c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6406ee90 con 0x56082733b670 2024-09-11T23:44:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.193+0000 7fcf6b33c740 10 client.0 _ll_register_callbacks cb 0x7fcf540bb3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.194+0000 7fcf61d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x56082734c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.194+0000 7fcf61d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x56082734c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:54630/0 (socket says 172.21.15.133:54630) 2024-09-11T23:44:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.194+0000 7fcf61d9b640 1 -- 172.21.15.133:0/566652845 learned_addr learned my addr 172.21.15.133:0/566652845 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.195+0000 7fcf61d9b640 1 -- 172.21.15.133:0/566652845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 msgr2=0x5608273bf640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.195+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/566652845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x5608273bf640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.195+0000 7fcf61d9b640 1 -- 172.21.15.133:0/566652845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827337de0 msgr2=0x56082734beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.195+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/566652845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827337de0 0x56082734beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.195+0000 7fcf61d9b640 1 -- 172.21.15.133:0/566652845 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608273bfc50 con 0x56082733b670 2024-09-11T23:44:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.200+0000 7fcf61d9b640 1 --2- 172.21.15.133:0/566652845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x56082734c3f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00ca00 tx=0x7fcf4c00af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4d5a4c67916f1be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.200+0000 7fcf49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:44:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.202+0000 7fcf49ffb640 1 -- 172.21.15.133:0/566652845 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c02e070 con 0x56082733b670 2024-09-11T23:44:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.202+0000 7fcf6b33c740 1 -- 172.21.15.133:0/566652845 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf540d4070 con 0x56082733b670 2024-09-11T23:44:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.203+0000 7fcf49ffb640 1 -- 172.21.15.133:0/566652845 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fcf4c011930 con 0x56082733b670 2024-09-11T23:44:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.203+0000 7fcf49ffb640 1 -- 172.21.15.133:0/566652845 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf4c03a050 con 0x56082733b670 2024-09-11T23:44:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.203+0000 7fcf6b33c740 10 client.26776 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.203+0000 7fcf6b33c740 1 -- 172.21.15.133:0/566652845 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5608273c0ec0 con 0x56082733b670 2024-09-11T23:44:27.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.205+0000 7fcf49ffb640 1 -- 172.21.15.133:0/566652845 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf540d4070 con 0x56082733b670 2024-09-11T23:44:27.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.206+0000 7fcf49ffb640 1 -- 172.21.15.133:0/566652845 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcf4c02ac20 con 0x56082733b670 2024-09-11T23:44:27.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.206+0000 7fcf6b33c740 10 client.26776 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:27.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.206+0000 7fcf6b33c740 10 client.26776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:27.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.206+0000 7fcf6b33c740 20 client.26776 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:27.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.206+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/566652845 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560827346a30 0x5608273c3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:27.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.206+0000 7fcf6b33c740 4 client.26776 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:44:27.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.206+0000 7fcf6b33c740 1 -- 172.21.15.133:0/566652845 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560827225210 con 0x560827346a30 2024-09-11T23:44:27.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.207+0000 7fcf6259c640 1 --2- 172.21.15.133:0/566652845 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560827346a30 0x5608273c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:27.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.212+0000 7fcf6259c640 1 --2- 172.21.15.133:0/566652845 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560827346a30 0x5608273c3560 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f2bd46190643e5 server_cookie=b3cfe87ede14f758 in_seq=0 out_seq=0 2024-09-11T23:44:27.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.212+0000 7fcf49ffb640 10 client.26776 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:27.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.215+0000 7fcf49ffb640 1 -- 172.21.15.133:0/566652845 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560827225210 con 0x560827346a30 2024-09-11T23:44:27.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.215+0000 7fcf49ffb640 10 client.26776 handle_command_reply: tid=0 2024-09-11T23:44:27.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.215+0000 7fcf6b33c740 1 client.26776 shutdown 2024-09-11T23:44:27.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.219+0000 7fcf6b33c740 1 -- 172.21.15.133:0/566652845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 msgr2=0x56082734c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:27.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.219+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/566652845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x56082734c3f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00ca00 tx=0x7fcf4c00af90 comp rx=0 tx=0).stop 2024-09-11T23:44:27.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.220+0000 7fcf6b33c740 1 -- 172.21.15.133:0/566652845 shutdown_connections 2024-09-11T23:44:27.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.220+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/566652845 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560827346a30 0x5608273c3560 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.220+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/566652845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560827344250 0x5608273bf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.220+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/566652845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082733b670 0x56082734c3f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.220+0000 7fcf6b33c740 1 --2- 172.21.15.133:0/566652845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560827337de0 0x56082734beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.220+0000 7fcf6b33c740 1 -- 172.21.15.133:0/566652845 >> 172.21.15.133:0/566652845 conn(0x560827338860 msgr2=0x560827345850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:27.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.220+0000 7fcf49ffb640 0 client.26776 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:27.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.220+0000 7fcf6b33c740 1 -- 172.21.15.133:0/566652845 shutdown_connections 2024-09-11T23:44:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.220+0000 7fcf6b33c740 1 -- 172.21.15.133:0/566652845 wait complete. 2024-09-11T23:44:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.221+0000 7fcf6b33c740 20 client.26776 trim_cache size 0 max 16384 2024-09-11T23:44:27.225 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:44:27.225 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:44:27.225 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:44:27.225 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:44:27.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.226+0000 7fcf597fa640 1 -- 172.21.15.133:0/1677899256 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf500a30f0 msgr2=0x7fcf500a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:27.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.226+0000 7fcf597fa640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf500a30f0 0x7fcf500a55b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c006fd0 tx=0x7fcf4c008040 comp rx=0 tx=0).stop 2024-09-11T23:44:27.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.226+0000 7fcf597fa640 1 -- 172.21.15.133:0/1677899256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf6411b140 msgr2=0x7fcf641aba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:27.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.226+0000 7fcf597fa640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf6411b140 0x7fcf641aba00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009b30 tx=0x7fcf540305d0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.227+0000 7fcf597fa640 1 -- 172.21.15.133:0/1677899256 shutdown_connections 2024-09-11T23:44:27.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.227+0000 7fcf597fa640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf500a30f0 0x7fcf500a55b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.227+0000 7fcf597fa640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf6411b140 0x7fcf641aba00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.227+0000 7fcf597fa640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf64110220 0x7fcf641a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.227+0000 7fcf597fa640 1 --2- 172.21.15.133:0/1677899256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf6406ea30 0x7fcf641a7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.227+0000 7fcf597fa640 1 -- 172.21.15.133:0/1677899256 >> 172.21.15.133:0/1677899256 conn(0x7fcf64073210 msgr2=0x7fcf64119130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.228+0000 7fcf597fa640 1 -- 172.21.15.133:0/1677899256 shutdown_connections 2024-09-11T23:44:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:27.229+0000 7fcf597fa640 1 -- 172.21.15.133:0/1677899256 wait complete. 2024-09-11T23:44:28.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:28 smithi138 ceph-mon[35142]: pgmap v238: 97 pgs: 97 active+clean; 63 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 643 MiB/s wr, 3.95k op/s 2024-09-11T23:44:28.961 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:28 smithi133 ceph-mon[31430]: pgmap v238: 97 pgs: 97 active+clean; 63 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 643 MiB/s wr, 3.95k op/s 2024-09-11T23:44:29.015 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:44:29.015 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:44:29.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:28 smithi178 ceph-mon[35882]: pgmap v238: 97 pgs: 97 active+clean; 63 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 643 MiB/s wr, 3.95k op/s 2024-09-11T23:44:30.102 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:30.961 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:30 smithi133 ceph-mon[31430]: pgmap v239: 97 pgs: 97 active+clean; 65 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 658 MiB/s wr, 3.20k op/s 2024-09-11T23:44:31.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:30 smithi178 ceph-mon[35882]: pgmap v239: 97 pgs: 97 active+clean; 65 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 658 MiB/s wr, 3.20k op/s 2024-09-11T23:44:31.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.104+0000 7fa3b924e640 1 -- 172.21.15.133:0/688329528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b41008c0 msgr2=0x7fa3b4100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:31.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.104+0000 7fa3b1d9b640 1 -- 172.21.15.133:0/688329528 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8035020 con 0x7fa3b41008c0 2024-09-11T23:44:31.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.104+0000 7fa3b924e640 1 --2- 172.21.15.133:0/688329528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b41008c0 0x7fa3b4100ca0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009930 tx=0x7fa3a802a440 comp rx=0 tx=0).stop 2024-09-11T23:44:31.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.110+0000 7fa3b924e640 1 -- 172.21.15.133:0/688329528 shutdown_connections 2024-09-11T23:44:31.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.110+0000 7fa3b924e640 1 --2- 172.21.15.133:0/688329528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b4105ec0 0x7fa3b4106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:31.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.110+0000 7fa3b924e640 1 --2- 172.21.15.133:0/688329528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b41011e0 0x7fa3b4105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:31.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.110+0000 7fa3b924e640 1 --2- 172.21.15.133:0/688329528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b41008c0 0x7fa3b4100ca0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:31.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.110+0000 7fa3b924e640 1 -- 172.21.15.133:0/688329528 >> 172.21.15.133:0/688329528 conn(0x7fa3b40fc7d0 msgr2=0x7fa3b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:31.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.115+0000 7fa3b924e640 1 -- 172.21.15.133:0/688329528 shutdown_connections 2024-09-11T23:44:31.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.115+0000 7fa3b924e640 1 -- 172.21.15.133:0/688329528 wait complete. 2024-09-11T23:44:31.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.116+0000 7fa3b924e640 1 Processor -- start 2024-09-11T23:44:31.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.117+0000 7fa3b924e640 1 -- start start 2024-09-11T23:44:31.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.117+0000 7fa3b924e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b41008c0 0x7fa3b4073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:31.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.117+0000 7fa3b924e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b41011e0 0x7fa3b406cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:31.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.117+0000 7fa3b924e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b4105ec0 0x7fa3b406d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:31.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.117+0000 7fa3b924e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41186e0 con 0x7fa3b4105ec0 2024-09-11T23:44:31.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.117+0000 7fa3b924e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4118560 con 0x7fa3b41008c0 2024-09-11T23:44:31.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.117+0000 7fa3b924e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4118860 con 0x7fa3b41011e0 2024-09-11T23:44:31.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.122+0000 7fa3b359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b4105ec0 0x7fa3b406d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:31.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.122+0000 7fa3b359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b4105ec0 0x7fa3b406d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53650/0 (socket says 172.21.15.133:53650) 2024-09-11T23:44:31.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.122+0000 7fa3b359e640 1 -- 172.21.15.133:0/434480264 learned_addr learned my addr 172.21.15.133:0/434480264 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:31.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.122+0000 7fa3b259c640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b41011e0 0x7fa3b406cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:31.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.122+0000 7fa3b2d9d640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b41008c0 0x7fa3b4073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:31.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.131+0000 7fa3b359e640 1 -- 172.21.15.133:0/434480264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b41011e0 msgr2=0x7fa3b406cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:31.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.131+0000 7fa3b359e640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b41011e0 0x7fa3b406cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:31.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.131+0000 7fa3b359e640 1 -- 172.21.15.133:0/434480264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b41008c0 msgr2=0x7fa3b4073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:31.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.131+0000 7fa3b359e640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b41008c0 0x7fa3b4073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:31.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.131+0000 7fa3b359e640 1 -- 172.21.15.133:0/434480264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b406d680 con 0x7fa3b4105ec0 2024-09-11T23:44:31.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.136+0000 7fa3b359e640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b4105ec0 0x7fa3b406d040 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00eb10 tx=0x7fa3ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:31.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.158+0000 7fa39bfff640 1 -- 172.21.15.133:0/434480264 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac004b20 con 0x7fa3b4105ec0 2024-09-11T23:44:31.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.158+0000 7fa39bfff640 1 -- 172.21.15.133:0/434480264 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3ac004cc0 con 0x7fa3b4105ec0 2024-09-11T23:44:31.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.158+0000 7fa39bfff640 1 -- 172.21.15.133:0/434480264 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac010720 con 0x7fa3b4105ec0 2024-09-11T23:44:31.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.158+0000 7fa3b924e640 1 -- 172.21.15.133:0/434480264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b40715a0 con 0x7fa3b4105ec0 2024-09-11T23:44:31.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.159+0000 7fa3b924e640 1 -- 172.21.15.133:0/434480264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b4071a90 con 0x7fa3b4105ec0 2024-09-11T23:44:31.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.161+0000 7fa3b924e640 1 -- 172.21.15.133:0/434480264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b406d910 con 0x7fa3b4105ec0 2024-09-11T23:44:31.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.186+0000 7fa39bfff640 1 -- 172.21.15.133:0/434480264 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa3ac0108c0 con 0x7fa3b4105ec0 2024-09-11T23:44:31.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.189+0000 7fa39bfff640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa39c0a2db0 0x7fa39c0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:31.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.189+0000 7fa3b2d9d640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa39c0a2db0 0x7fa39c0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:31.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.191+0000 7fa39bfff640 1 -- 172.21.15.133:0/434480264 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fa3ac0cfc40 con 0x7fa3b4105ec0 2024-09-11T23:44:31.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.191+0000 7fa39bfff640 1 -- 172.21.15.133:0/434480264 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3ac0d0160 con 0x7fa3b4105ec0 2024-09-11T23:44:31.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:30 smithi138 ceph-mon[35142]: pgmap v239: 97 pgs: 97 active+clean; 65 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 658 MiB/s wr, 3.20k op/s 2024-09-11T23:44:31.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.224+0000 7fa3b2d9d640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa39c0a2db0 0x7fa39c0a5270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8031440 tx=0x7fa3a802a940 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:31.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.676+0000 7fa3b924e640 1 -- 172.21.15.133:0/434480264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3b406e030 con 0x7fa3b4105ec0 2024-09-11T23:44:31.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.681+0000 7fa39bfff640 1 -- 172.21.15.133:0/434480264 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7fa3ac09c8a0 con 0x7fa3b4105ec0 2024-09-11T23:44:31.681 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:44:31.682 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:44:31.682 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:44:31.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.691+0000 7fa399ffb640 1 -- 172.21.15.133:0/434480264 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa39c0a2db0 msgr2=0x7fa39c0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:31.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.692+0000 7fa399ffb640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa39c0a2db0 0x7fa39c0a5270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8031440 tx=0x7fa3a802a940 comp rx=0 tx=0).stop 2024-09-11T23:44:31.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.692+0000 7fa399ffb640 1 -- 172.21.15.133:0/434480264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b4105ec0 msgr2=0x7fa3b406d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:31.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.692+0000 7fa399ffb640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b4105ec0 0x7fa3b406d040 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00eb10 tx=0x7fa3ac00c2d0 comp rx=0 tx=0).stop 2024-09-11T23:44:31.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.692+0000 7fa399ffb640 1 -- 172.21.15.133:0/434480264 shutdown_connections 2024-09-11T23:44:31.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.692+0000 7fa399ffb640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa39c0a2db0 0x7fa39c0a5270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:31.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.692+0000 7fa399ffb640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b4105ec0 0x7fa3b406d040 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:31.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.692+0000 7fa399ffb640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3b41011e0 0x7fa3b406cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:31.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.692+0000 7fa399ffb640 1 --2- 172.21.15.133:0/434480264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3b41008c0 0x7fa3b4073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:31.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.692+0000 7fa399ffb640 1 -- 172.21.15.133:0/434480264 >> 172.21.15.133:0/434480264 conn(0x7fa3b40fc7d0 msgr2=0x7fa3b4103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:31.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.694+0000 7fa399ffb640 1 -- 172.21.15.133:0/434480264 shutdown_connections 2024-09-11T23:44:31.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:31.695+0000 7fa399ffb640 1 -- 172.21.15.133:0/434480264 wait complete. 2024-09-11T23:44:31.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:31 smithi133 ceph-mon[31430]: pgmap v240: 97 pgs: 97 active+clean; 65 GiB data, 218 GiB used, 854 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 576 MiB/s wr, 2.69k op/s 2024-09-11T23:44:32.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:31 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/434480264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:44:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:31 smithi178 ceph-mon[35882]: pgmap v240: 97 pgs: 97 active+clean; 65 GiB data, 218 GiB used, 854 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 576 MiB/s wr, 2.69k op/s 2024-09-11T23:44:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:31 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/434480264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:44:32.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:31 smithi138 ceph-mon[35142]: pgmap v240: 97 pgs: 97 active+clean; 65 GiB data, 218 GiB used, 854 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 576 MiB/s wr, 2.69k op/s 2024-09-11T23:44:32.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:31 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/434480264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:44:32.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:44:33.569 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:34.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.565+0000 7ff07a6ad640 1 -- 172.21.15.133:0/1304768170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff074105d90 msgr2=0x7ff074106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:34.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.565+0000 7ff07a6ad640 1 --2- 172.21.15.133:0/1304768170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff074105d90 0x7ff074106220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff064009900 tx=0x7ff06402a490 comp rx=0 tx=0).stop 2024-09-11T23:44:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.569+0000 7ff07a6ad640 1 -- 172.21.15.133:0/1304768170 shutdown_connections 2024-09-11T23:44:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.569+0000 7ff07a6ad640 1 --2- 172.21.15.133:0/1304768170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff074105d90 0x7ff074106220 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.569+0000 7ff07a6ad640 1 --2- 172.21.15.133:0/1304768170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0741012a0 0x7ff074105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.569+0000 7ff07a6ad640 1 --2- 172.21.15.133:0/1304768170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff074100980 0x7ff074100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.569+0000 7ff07a6ad640 1 -- 172.21.15.133:0/1304768170 >> 172.21.15.133:0/1304768170 conn(0x7ff0740fc610 msgr2=0x7ff0740fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:34.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.570+0000 7ff07a6ad640 1 -- 172.21.15.133:0/1304768170 shutdown_connections 2024-09-11T23:44:34.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.571+0000 7ff07a6ad640 1 -- 172.21.15.133:0/1304768170 wait complete. 2024-09-11T23:44:34.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.572+0000 7ff07a6ad640 1 Processor -- start 2024-09-11T23:44:34.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.572+0000 7ff07a6ad640 1 -- start start 2024-09-11T23:44:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.572+0000 7ff07a6ad640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff074100980 0x7ff0740739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.572+0000 7ff07a6ad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0741012a0 0x7ff074073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff07a6ad640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff074105d90 0x7ff07406cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff07a6ad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741184e0 con 0x7ff0741012a0 2024-09-11T23:44:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff07a6ad640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff074118360 con 0x7ff074100980 2024-09-11T23:44:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff07a6ad640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff074118660 con 0x7ff074105d90 2024-09-11T23:44:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff0796ab640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff074100980 0x7ff0740739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff079eac640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff074105d90 0x7ff07406cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff079eac640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff074105d90 0x7ff07406cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35182/0 (socket says 172.21.15.133:35182) 2024-09-11T23:44:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff079eac640 1 -- 172.21.15.133:0/2514736313 learned_addr learned my addr 172.21.15.133:0/2514736313 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:34.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff079eac640 1 -- 172.21.15.133:0/2514736313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff074100980 msgr2=0x7ff0740739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:34.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff079eac640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff074100980 0x7ff0740739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff079eac640 1 -- 172.21.15.133:0/2514736313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0741012a0 msgr2=0x7ff074073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:34.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff079eac640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0741012a0 0x7ff074073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.573+0000 7ff079eac640 1 -- 172.21.15.133:0/2514736313 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07406d4d0 con 0x7ff074105d90 2024-09-11T23:44:34.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.574+0000 7ff079eac640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff074105d90 0x7ff07406cc10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff06402aa80 tx=0x7ff064032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:34.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.574+0000 7ff06a7fc640 1 -- 172.21.15.133:0/2514736313 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06403f030 con 0x7ff074105d90 2024-09-11T23:44:34.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.574+0000 7ff07a6ad640 1 -- 172.21.15.133:0/2514736313 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07406d760 con 0x7ff074105d90 2024-09-11T23:44:34.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.574+0000 7ff07a6ad640 1 -- 172.21.15.133:0/2514736313 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff074071750 con 0x7ff074105d90 2024-09-11T23:44:34.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.575+0000 7ff06a7fc640 1 -- 172.21.15.133:0/2514736313 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff064003ad0 con 0x7ff074105d90 2024-09-11T23:44:34.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.575+0000 7ff06a7fc640 1 -- 172.21.15.133:0/2514736313 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064035070 con 0x7ff074105d90 2024-09-11T23:44:34.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.580+0000 7ff06a7fc640 1 -- 172.21.15.133:0/2514736313 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff064042030 con 0x7ff074105d90 2024-09-11T23:44:34.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.580+0000 7ff07dce8740 1 Processor -- start 2024-09-11T23:44:34.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.581+0000 7ff07dce8740 1 -- start start 2024-09-11T23:44:34.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.581+0000 7ff06a7fc640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0600a2df0 0x7ff0600a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.582+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c5ecf0 0x5649c3c5d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.582+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5d780 0x5649c3c5dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.582+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c68520 0x5649c3c6a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.582+0000 7ff07dce8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5649c3bc32f0 con 0x5649c3c68520 2024-09-11T23:44:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.582+0000 7ff07dce8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5649c3b6acf0 con 0x5649c3c5ecf0 2024-09-11T23:44:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.582+0000 7ff07dce8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5649c3bbf2b0 con 0x5649c3c5d780 2024-09-11T23:44:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.582+0000 7ff0796ab640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c5ecf0 0x5649c3c5d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.582+0000 7ff0796ab640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c5ecf0 0x5649c3c5d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47200/0 (socket says 172.21.15.133:47200) 2024-09-11T23:44:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.582+0000 7ff0796ab640 1 -- 172.21.15.133:0/2992421944 learned_addr learned my addr 172.21.15.133:0/2992421944 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:34.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.582+0000 7ff06a7fc640 1 -- 172.21.15.133:0/2514736313 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7ff0640efda0 con 0x7ff074105d90 2024-09-11T23:44:34.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.583+0000 7ff0796ab640 1 -- 172.21.15.133:0/2992421944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5d780 msgr2=0x5649c3c5dbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:34.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.583+0000 7ff0796ab640 1 --2- 172.21.15.133:0/2992421944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5d780 0x5649c3c5dbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.583+0000 7ff0796ab640 1 -- 172.21.15.133:0/2992421944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c68520 msgr2=0x5649c3c6a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:34.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.583+0000 7ff0796ab640 1 --2- 172.21.15.133:0/2992421944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c68520 0x5649c3c6a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.583+0000 7ff0796ab640 1 -- 172.21.15.133:0/2992421944 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c3b4f3a0 con 0x5649c3c5ecf0 2024-09-11T23:44:34.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.583+0000 7ff0796ab640 1 --2- 172.21.15.133:0/2992421944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c5ecf0 0x5649c3c5d240 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00a4b0 tx=0x7ff06c0096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c1bba679068e19d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:34.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.584+0000 7ff05f7fe640 1 -- 172.21.15.133:0/2992421944 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c018020 con 0x5649c3c5ecf0 2024-09-11T23:44:34.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.584+0000 7ff05f7fe640 1 -- 172.21.15.133:0/2992421944 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff06c014cf0 con 0x5649c3c5ecf0 2024-09-11T23:44:34.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.585+0000 7ff078eaa640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0600a2df0 0x7ff0600a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.585+0000 7ff05f7fe640 1 -- 172.21.15.133:0/2992421944 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c013dc0 con 0x5649c3c5ecf0 2024-09-11T23:44:34.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.585+0000 7ff07dce8740 1 -- 172.21.15.133:0/2992421944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c5ecf0 msgr2=0x5649c3c5d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:34.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.585+0000 7ff07dce8740 1 --2- 172.21.15.133:0/2992421944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c5ecf0 0x5649c3c5d240 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00a4b0 tx=0x7ff06c0096d0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.585+0000 7ff078eaa640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0600a2df0 0x7ff0600a52b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x5649c3c5ac70 tx=0x7ff070006df0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.585+0000 7ff07dce8740 1 -- 172.21.15.133:0/2992421944 shutdown_connections 2024-09-11T23:44:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.585+0000 7ff07dce8740 1 --2- 172.21.15.133:0/2992421944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c68520 0x5649c3c6a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.585+0000 7ff07dce8740 1 --2- 172.21.15.133:0/2992421944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5d780 0x5649c3c5dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.585+0000 7ff07dce8740 1 --2- 172.21.15.133:0/2992421944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c5ecf0 0x5649c3c5d240 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.585+0000 7ff07dce8740 1 -- 172.21.15.133:0/2992421944 >> 172.21.15.133:0/2992421944 conn(0x5649c3c26310 msgr2=0x5649c3c59e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.586+0000 7ff07dce8740 1 -- 172.21.15.133:0/2992421944 shutdown_connections 2024-09-11T23:44:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.586+0000 7ff07dce8740 1 -- 172.21.15.133:0/2992421944 wait complete. 2024-09-11T23:44:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.586+0000 7ff07dce8740 1 -- start start 2024-09-11T23:44:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.587+0000 7ff07dce8740 1 Processor -- start 2024-09-11T23:44:34.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.587+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5d780 0x5649c3c67de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.587+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5ecf0 0x5649c3c65090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.587+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c68520 0x5649c3c655d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.588+0000 7ff07dce8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5649c3c6b270 con 0x5649c3c5d780 2024-09-11T23:44:34.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.588+0000 7ff07dce8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5649c3b7e060 con 0x5649c3c68520 2024-09-11T23:44:34.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.588+0000 7ff07dce8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5649c3c6b3f0 con 0x5649c3c5ecf0 2024-09-11T23:44:34.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.588+0000 7ff079eac640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c68520 0x5649c3c655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.588+0000 7ff079eac640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c68520 0x5649c3c655d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47206/0 (socket says 172.21.15.133:47206) 2024-09-11T23:44:34.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.588+0000 7ff079eac640 1 -- 172.21.15.133:0/2254691449 learned_addr learned my addr 172.21.15.133:0/2254691449 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:34.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.588+0000 7ff0796ab640 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5d780 0x5649c3c67de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.589+0000 7ff078eaa640 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5ecf0 0x5649c3c65090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.588+0000 7ff07dce8740 10 client.0 _ll_register_callbacks cb 0x5649c38d18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:34.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.589+0000 7ff079eac640 1 -- 172.21.15.133:0/2254691449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5ecf0 msgr2=0x5649c3c65090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:34.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.589+0000 7ff079eac640 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5ecf0 0x5649c3c65090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.589+0000 7ff079eac640 1 -- 172.21.15.133:0/2254691449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5d780 msgr2=0x5649c3c67de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:34.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.589+0000 7ff079eac640 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5d780 0x5649c3c67de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.589+0000 7ff079eac640 1 -- 172.21.15.133:0/2254691449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c39343c0 con 0x5649c3c68520 2024-09-11T23:44:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.589+0000 7ff079eac640 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c68520 0x5649c3c655d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff0640bc230 tx=0x7ff064005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5566c48e479839 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.589+0000 7ff05cff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:44:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.590+0000 7ff05cff9640 1 -- 172.21.15.133:0/2254691449 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064004980 con 0x5649c3c68520 2024-09-11T23:44:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.590+0000 7ff07dce8740 1 -- 172.21.15.133:0/2254691449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06c014cf0 con 0x5649c3c68520 2024-09-11T23:44:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.590+0000 7ff05cff9640 1 -- 172.21.15.133:0/2254691449 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7ff0640b8910 con 0x5649c3c68520 2024-09-11T23:44:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.591+0000 7ff05cff9640 1 -- 172.21.15.133:0/2254691449 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff0640047a0 con 0x5649c3c68520 2024-09-11T23:44:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.591+0000 7ff07dce8740 10 client.26933 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.591+0000 7ff07dce8740 1 -- 172.21.15.133:0/2254691449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5649c3c76ce0 con 0x5649c3c68520 2024-09-11T23:44:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.591+0000 7ff05cff9640 1 -- 172.21.15.133:0/2254691449 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff06c014cf0 con 0x5649c3c68520 2024-09-11T23:44:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.592+0000 7ff05cff9640 1 -- 172.21.15.133:0/2254691449 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff0640c4a80 con 0x5649c3c68520 2024-09-11T23:44:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.592+0000 7ff07dce8740 10 client.26933 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.592+0000 7ff07dce8740 10 client.26933 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:44:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.592+0000 7ff07dce8740 20 client.26933 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.592+0000 7ff07dce8740 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5649c3c78390 0x5649c3c7a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.592+0000 7ff07dce8740 4 client.26933 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.592+0000 7ff07dce8740 1 -- 172.21.15.133:0/2254691449 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649c3bd0860 con 0x5649c3c78390 2024-09-11T23:44:34.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.593+0000 7ff0796ab640 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5649c3c78390 0x5649c3c7a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.597+0000 7ff0796ab640 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5649c3c78390 0x5649c3c7a7f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72140a65bb1ef0bf server_cookie=e385ef6bce3ef317 in_seq=0 out_seq=0 2024-09-11T23:44:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.597+0000 7ff05cff9640 10 client.26933 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:44:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.598+0000 7ff05cff9640 1 -- 172.21.15.133:0/2254691449 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649c3bd0860 con 0x5649c3c78390 2024-09-11T23:44:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.598+0000 7ff05cff9640 10 client.26933 handle_command_reply: tid=0 2024-09-11T23:44:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.598+0000 7ff07dce8740 1 client.26933 shutdown 2024-09-11T23:44:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.599+0000 7ff07dce8740 1 -- 172.21.15.133:0/2254691449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c68520 msgr2=0x5649c3c655d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.599+0000 7ff07dce8740 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c68520 0x5649c3c655d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff0640bc230 tx=0x7ff064005d80 comp rx=0 tx=0).stop 2024-09-11T23:44:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.599+0000 7ff07dce8740 1 -- 172.21.15.133:0/2254691449 shutdown_connections 2024-09-11T23:44:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.599+0000 7ff07dce8740 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5649c3c78390 0x5649c3c7a7f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.599+0000 7ff07dce8740 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c68520 0x5649c3c655d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.599+0000 7ff07dce8740 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5ecf0 0x5649c3c65090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.599+0000 7ff07dce8740 1 --2- 172.21.15.133:0/2254691449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5d780 0x5649c3c67de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.599+0000 7ff07dce8740 1 -- 172.21.15.133:0/2254691449 >> 172.21.15.133:0/2254691449 conn(0x5649c3c26310 msgr2=0x5649c3c5fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.600+0000 7ff05cff9640 0 client.26933 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:44:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.600+0000 7ff07dce8740 1 -- 172.21.15.133:0/2254691449 shutdown_connections 2024-09-11T23:44:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.600+0000 7ff07dce8740 1 -- 172.21.15.133:0/2254691449 wait complete. 2024-09-11T23:44:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.600+0000 7ff07dce8740 20 client.26933 trim_cache size 0 max 16384 2024-09-11T23:44:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.632+0000 7ff07dce8740 1 Processor -- start 2024-09-11T23:44:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.632+0000 7ff07dce8740 1 -- start start 2024-09-11T23:44:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.632+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c5d780 0x5649c3c74630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.632+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c69910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.632+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c68520 0x5649c3c69e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.632+0000 7ff07dce8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff05800e630 con 0x5649c3c5ecf0 2024-09-11T23:44:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.632+0000 7ff07dce8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff058010170 con 0x5649c3c5d780 2024-09-11T23:44:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.632+0000 7ff07dce8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff05800ed00 con 0x5649c3c68520 2024-09-11T23:44:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.633+0000 7ff078eaa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c69910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.633+0000 7ff079eac640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c68520 0x5649c3c69e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.633+0000 7ff078eaa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c69910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43668/0 (socket says 172.21.15.133:43668) 2024-09-11T23:44:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.633+0000 7ff078eaa640 1 -- 172.21.15.133:0/1084941739 learned_addr learned my addr 172.21.15.133:0/1084941739 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.634+0000 7ff078eaa640 1 -- 172.21.15.133:0/1084941739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c68520 msgr2=0x5649c3c69e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.634+0000 7ff078eaa640 1 --2- 172.21.15.133:0/1084941739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c68520 0x5649c3c69e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.634+0000 7ff078eaa640 1 -- 172.21.15.133:0/1084941739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c5d780 msgr2=0x5649c3c74630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.634+0000 7ff078eaa640 1 --2- 172.21.15.133:0/1084941739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c5d780 0x5649c3c74630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.634+0000 7ff078eaa640 1 -- 172.21.15.133:0/1084941739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c3c60c90 con 0x5649c3c5ecf0 2024-09-11T23:44:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.634+0000 7ff078eaa640 1 --2- 172.21.15.133:0/1084941739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c69910 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff07000b980 tx=0x7ff07000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a92a2f7180516e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.636+0000 7ff05effd640 1 -- 172.21.15.133:0/1084941739 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070018050 con 0x5649c3c5ecf0 2024-09-11T23:44:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.636+0000 7ff05effd640 1 -- 172.21.15.133:0/1084941739 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff070022070 con 0x5649c3c5ecf0 2024-09-11T23:44:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.636+0000 7ff05effd640 1 -- 172.21.15.133:0/1084941739 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070010680 con 0x5649c3c5ecf0 2024-09-11T23:44:34.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.637+0000 7ff07dce8740 1 -- 172.21.15.133:0/1084941739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 msgr2=0x5649c3c69910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:34.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.638+0000 7ff07dce8740 1 --2- 172.21.15.133:0/1084941739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c69910 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff07000b980 tx=0x7ff07000e000 comp rx=0 tx=0).stop 2024-09-11T23:44:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.638+0000 7ff07dce8740 1 -- 172.21.15.133:0/1084941739 shutdown_connections 2024-09-11T23:44:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.638+0000 7ff07dce8740 1 --2- 172.21.15.133:0/1084941739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c68520 0x5649c3c69e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.638+0000 7ff07dce8740 1 --2- 172.21.15.133:0/1084941739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c69910 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.638+0000 7ff07dce8740 1 --2- 172.21.15.133:0/1084941739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c5d780 0x5649c3c74630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.638+0000 7ff07dce8740 1 -- 172.21.15.133:0/1084941739 >> 172.21.15.133:0/1084941739 conn(0x5649c3c26310 msgr2=0x5649c3c64180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.638+0000 7ff07dce8740 1 -- 172.21.15.133:0/1084941739 shutdown_connections 2024-09-11T23:44:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.638+0000 7ff07dce8740 1 -- 172.21.15.133:0/1084941739 wait complete. 2024-09-11T23:44:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.639+0000 7ff07dce8740 1 -- start start 2024-09-11T23:44:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.639+0000 7ff07dce8740 1 Processor -- start 2024-09-11T23:44:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.640+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5d780 0x5649c3c70ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.640+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c70ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.640+0000 7ff07dce8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c68520 0x5649c3c7f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.640+0000 7ff078eaa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c70ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.640+0000 7ff078eaa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c70ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43678/0 (socket says 172.21.15.133:43678) 2024-09-11T23:44:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.640+0000 7ff078eaa640 1 -- 172.21.15.133:0/938590935 learned_addr learned my addr 172.21.15.133:0/938590935 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.641+0000 7ff07dce8740 1 -- 172.21.15.133:0/938590935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff05800faa0 con 0x5649c3c5ecf0 2024-09-11T23:44:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.641+0000 7ff07dce8740 1 -- 172.21.15.133:0/938590935 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff05800f3d0 con 0x5649c3c68520 2024-09-11T23:44:34.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.641+0000 7ff07dce8740 1 -- 172.21.15.133:0/938590935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff05800c630 con 0x5649c3c5d780 2024-09-11T23:44:34.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.641+0000 7ff07dce8740 10 client.0 _ll_register_callbacks cb 0x7ff064005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:34.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.641+0000 7ff078eaa640 1 -- 172.21.15.133:0/938590935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5d780 msgr2=0x5649c3c70ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:34.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.641+0000 7ff078eaa640 1 --2- 172.21.15.133:0/938590935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5d780 0x5649c3c70ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.641+0000 7ff078eaa640 1 -- 172.21.15.133:0/938590935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c68520 msgr2=0x5649c3c7f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:34.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.641+0000 7ff078eaa640 1 --2- 172.21.15.133:0/938590935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c68520 0x5649c3c7f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.641+0000 7ff078eaa640 1 -- 172.21.15.133:0/938590935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c3c7ff20 con 0x5649c3c5ecf0 2024-09-11T23:44:34.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.641+0000 7ff07dce8740 1 -- 172.21.15.133:0/938590935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff070022070 con 0x5649c3c5ecf0 2024-09-11T23:44:34.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.646+0000 7ff078eaa640 1 --2- 172.21.15.133:0/938590935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c70ff0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff070009790 tx=0x7ff070012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ca6d91ba97c34b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:34.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.646+0000 7ff05cff9640 10 client.17427 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:34.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.646+0000 7ff05cff9640 1 -- 172.21.15.133:0/938590935 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070004220 con 0x5649c3c5ecf0 2024-09-11T23:44:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.647+0000 7ff05cff9640 1 -- 172.21.15.133:0/938590935 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7ff0700043c0 con 0x5649c3c5ecf0 2024-09-11T23:44:34.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.647+0000 7ff05cff9640 1 -- 172.21.15.133:0/938590935 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff07001d440 con 0x5649c3c5ecf0 2024-09-11T23:44:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.647+0000 7ff07dce8740 10 client.17427 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.647+0000 7ff07dce8740 1 -- 172.21.15.133:0/938590935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5649c3c702f0 con 0x5649c3c5ecf0 2024-09-11T23:44:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.648+0000 7ff05cff9640 1 -- 172.21.15.133:0/938590935 <== mon.0 v2:172.21.15.133:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff07001d5e0 con 0x5649c3c5ecf0 2024-09-11T23:44:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.648+0000 7ff07dce8740 10 client.17427 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.648+0000 7ff07dce8740 10 client.17427 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:44:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.648+0000 7ff07dce8740 20 client.17427 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:34.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.648+0000 7ff07dce8740 1 --2- 172.21.15.133:0/938590935 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5649c3ce30e0 0x5649c3ce5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.648+0000 7ff07dce8740 4 client.17427 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:44:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.648+0000 7ff07dce8740 1 -- 172.21.15.133:0/938590935 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649c3bcad70 con 0x5649c3ce30e0 2024-09-11T23:44:34.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.657+0000 7ff0796ab640 1 --2- 172.21.15.133:0/938590935 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5649c3ce30e0 0x5649c3ce5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:34.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.660+0000 7ff0796ab640 1 --2- 172.21.15.133:0/938590935 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5649c3ce30e0 0x5649c3ce5500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=be65714bf24f7475 server_cookie=8f19d314947cdfc1 in_seq=0 out_seq=0 2024-09-11T23:44:34.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.660+0000 7ff05cff9640 10 client.17427 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:44:34.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.660+0000 7ff05cff9640 1 -- 172.21.15.133:0/938590935 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649c3bcad70 con 0x5649c3ce30e0 2024-09-11T23:44:34.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.660+0000 7ff05cff9640 10 client.17427 handle_command_reply: tid=0 2024-09-11T23:44:34.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.660+0000 7ff07dce8740 1 client.17427 shutdown 2024-09-11T23:44:34.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.661+0000 7ff07dce8740 1 -- 172.21.15.133:0/938590935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 msgr2=0x5649c3c70ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:34.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.661+0000 7ff07dce8740 1 --2- 172.21.15.133:0/938590935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c70ff0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff070009790 tx=0x7ff070012f40 comp rx=0 tx=0).stop 2024-09-11T23:44:34.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.661+0000 7ff07dce8740 1 -- 172.21.15.133:0/938590935 shutdown_connections 2024-09-11T23:44:34.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.661+0000 7ff07dce8740 1 --2- 172.21.15.133:0/938590935 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x5649c3ce30e0 0x5649c3ce5500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.662+0000 7ff05cff9640 0 client.17427 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:44:34.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.662+0000 7ff07dce8740 1 --2- 172.21.15.133:0/938590935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649c3c68520 0x5649c3c7f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.662+0000 7ff07dce8740 1 --2- 172.21.15.133:0/938590935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649c3c5ecf0 0x5649c3c70ff0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.662+0000 7ff07dce8740 1 --2- 172.21.15.133:0/938590935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649c3c5d780 0x5649c3c70ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.662+0000 7ff07dce8740 1 -- 172.21.15.133:0/938590935 >> 172.21.15.133:0/938590935 conn(0x5649c3c26310 msgr2=0x5649c3c58690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:34.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.663+0000 7ff07dce8740 1 -- 172.21.15.133:0/938590935 shutdown_connections 2024-09-11T23:44:34.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.663+0000 7ff07dce8740 1 -- 172.21.15.133:0/938590935 wait complete. 2024-09-11T23:44:34.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.663+0000 7ff07dce8740 20 client.17427 trim_cache size 0 max 16384 2024-09-11T23:44:34.664 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:44:34.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.664+0000 7ff07a6ad640 1 -- 172.21.15.133:0/2514736313 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0600a2df0 msgr2=0x7ff0600a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:34.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.665+0000 7ff07a6ad640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0600a2df0 0x7ff0600a52b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x5649c3c5ac70 tx=0x7ff070006df0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.665+0000 7ff07a6ad640 1 -- 172.21.15.133:0/2514736313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff074105d90 msgr2=0x7ff07406cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:34.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.665+0000 7ff07a6ad640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff074105d90 0x7ff07406cc10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff06402aa80 tx=0x7ff064032b90 comp rx=0 tx=0).stop 2024-09-11T23:44:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.665+0000 7ff07a6ad640 1 -- 172.21.15.133:0/2514736313 shutdown_connections 2024-09-11T23:44:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.665+0000 7ff07a6ad640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff0600a2df0 0x7ff0600a52b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.665+0000 7ff07a6ad640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff074105d90 0x7ff07406cc10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.665+0000 7ff07a6ad640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0741012a0 0x7ff074073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.665+0000 7ff07a6ad640 1 --2- 172.21.15.133:0/2514736313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff074100980 0x7ff0740739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.666+0000 7ff07a6ad640 1 -- 172.21.15.133:0/2514736313 >> 172.21.15.133:0/2514736313 conn(0x7ff0740fc610 msgr2=0x7ff074103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.666+0000 7ff07a6ad640 1 -- 172.21.15.133:0/2514736313 shutdown_connections 2024-09-11T23:44:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:34.666+0000 7ff07a6ad640 1 -- 172.21.15.133:0/2514736313 wait complete. 2024-09-11T23:44:34.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:34 smithi133 ceph-mon[31430]: pgmap v241: 97 pgs: 97 active+clean; 66 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 710 MiB/s wr, 3.05k op/s 2024-09-11T23:44:35.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:34 smithi178 ceph-mon[35882]: pgmap v241: 97 pgs: 97 active+clean; 66 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 710 MiB/s wr, 3.05k op/s 2024-09-11T23:44:35.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:34 smithi138 ceph-mon[35142]: pgmap v241: 97 pgs: 97 active+clean; 66 GiB data, 223 GiB used, 850 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 710 MiB/s wr, 3.05k op/s 2024-09-11T23:44:35.699 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:44:36.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:35 smithi138 ceph-mon[35142]: pgmap v242: 97 pgs: 97 active+clean; 68 GiB data, 227 GiB used, 846 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 728 MiB/s wr, 2.28k op/s 2024-09-11T23:44:36.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:35 smithi133 ceph-mon[31430]: pgmap v242: 97 pgs: 97 active+clean; 68 GiB data, 227 GiB used, 846 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 728 MiB/s wr, 2.28k op/s 2024-09-11T23:44:36.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:35 smithi178 ceph-mon[35882]: pgmap v242: 97 pgs: 97 active+clean; 68 GiB data, 227 GiB used, 846 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 728 MiB/s wr, 2.28k op/s 2024-09-11T23:44:36.726 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.231+0000 7f6002e77640 1 -- 172.21.15.133:0/1197848680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc11b140 msgr2=0x7f5ffc11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.231+0000 7f6002e77640 1 --2- 172.21.15.133:0/1197848680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc11b140 0x7f5ffc11d530 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5fec01a940 tx=0x7f5fec03b450 comp rx=0 tx=0).stop 2024-09-11T23:44:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.231+0000 7f6002e77640 1 -- 172.21.15.133:0/1197848680 shutdown_connections 2024-09-11T23:44:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.231+0000 7f6002e77640 1 --2- 172.21.15.133:0/1197848680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc11b140 0x7f5ffc11d530 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.231+0000 7f6002e77640 1 --2- 172.21.15.133:0/1197848680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ffc1095d0 0x7f5ffc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.231+0000 7f6002e77640 1 --2- 172.21.15.133:0/1197848680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ffc1091f0 0x7f5ffc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.231+0000 7f6002e77640 1 -- 172.21.15.133:0/1197848680 >> 172.21.15.133:0/1197848680 conn(0x7f5ffc06ce00 msgr2=0x7f5ffc06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.231+0000 7f6002e77640 1 -- 172.21.15.133:0/1197848680 shutdown_connections 2024-09-11T23:44:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.231+0000 7f6002e77640 1 -- 172.21.15.133:0/1197848680 wait complete. 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.232+0000 7f6002e77640 1 Processor -- start 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.232+0000 7f6002e77640 1 -- start start 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.233+0000 7f6002e77640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc1091f0 0x7f5ffc11a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.233+0000 7f6002e77640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ffc1095d0 0x7f5ffc116800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.233+0000 7f6002e77640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ffc11b140 0x7f5ffc116d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.233+0000 7f6002e77640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc113300 con 0x7f5ffc11b140 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.233+0000 7f6002e77640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc113180 con 0x7f5ffc1095d0 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.233+0000 7f6002e77640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc113480 con 0x7f5ffc1091f0 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.233+0000 7f6001414640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ffc11b140 0x7f5ffc116d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.233+0000 7f6001414640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ffc11b140 0x7f5ffc116d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43696/0 (socket says 172.21.15.133:43696) 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.233+0000 7f6001414640 1 -- 172.21.15.133:0/603624612 learned_addr learned my addr 172.21.15.133:0/603624612 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.233+0000 7f6000c13640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc1091f0 0x7f5ffc11a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.233+0000 7f5ffbfff640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ffc1095d0 0x7f5ffc116800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.234+0000 7f6001414640 1 -- 172.21.15.133:0/603624612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc1091f0 msgr2=0x7f5ffc11a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.234+0000 7f6001414640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc1091f0 0x7f5ffc11a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.234+0000 7f6001414640 1 -- 172.21.15.133:0/603624612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ffc1095d0 msgr2=0x7f5ffc116800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.234+0000 7f6001414640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ffc1095d0 0x7f5ffc116800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.234+0000 7f6001414640 1 -- 172.21.15.133:0/603624612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc117300 con 0x7f5ffc11b140 2024-09-11T23:44:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.234+0000 7f6001414640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ffc11b140 0x7f5ffc116d40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0054d0 tx=0x7f5fec004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.235+0000 7f5ff9ffb640 1 -- 172.21.15.133:0/603624612 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec046070 con 0x7f5ffc11b140 2024-09-11T23:44:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.235+0000 7f5ff9ffb640 1 -- 172.21.15.133:0/603624612 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fec050070 con 0x7f5ffc11b140 2024-09-11T23:44:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.235+0000 7f5ff9ffb640 1 -- 172.21.15.133:0/603624612 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec040770 con 0x7f5ffc11b140 2024-09-11T23:44:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.235+0000 7f6002e77640 1 -- 172.21.15.133:0/603624612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ffc117590 con 0x7f5ffc11b140 2024-09-11T23:44:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.235+0000 7f6002e77640 1 -- 172.21.15.133:0/603624612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ffc1ab250 con 0x7f5ffc11b140 2024-09-11T23:44:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.241+0000 7f5ff9ffb640 1 -- 172.21.15.133:0/603624612 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5fec0035f0 con 0x7f5ffc11b140 2024-09-11T23:44:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.243+0000 7f5ff9ffb640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5fe40a2f40 0x7f5fe40a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.243+0000 7f5ff9ffb640 1 -- 172.21.15.133:0/603624612 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f5fec100510 con 0x7f5ffc11b140 2024-09-11T23:44:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.243+0000 7f6000c13640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5fe40a2f40 0x7f5fe40a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.244+0000 7f6000c13640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5fe40a2f40 0x7f5fe40a5400 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4005fd0 tx=0x7f5ff4009680 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.246+0000 7f6005c0d740 1 Processor -- start 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.247+0000 7f6005c0d740 1 -- start start 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.248+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9bded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.248+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9be410 0x55e7ec9c8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.248+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.248+0000 7f6005c0d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e7ec9222f0 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.248+0000 7f6005c0d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e7ec8ca270 con 0x55e7ec9be410 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.248+0000 7f6005c0d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e7ec91e2b0 con 0x55e7ec9bdaf0 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.248+0000 7f6001414640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.249+0000 7f6001414640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43708/0 (socket says 172.21.15.133:43708) 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.249+0000 7f6001414640 1 -- 172.21.15.133:0/2157039026 learned_addr learned my addr 172.21.15.133:0/2157039026 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.249+0000 7f5ffbfff640 1 --2- 172.21.15.133:0/2157039026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9be410 0x55e7ec9c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.249+0000 7f6000c13640 1 --2- 172.21.15.133:0/2157039026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9bded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.249+0000 7f6001414640 1 -- 172.21.15.133:0/2157039026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9bdaf0 msgr2=0x55e7ec9bded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.249+0000 7f6001414640 1 --2- 172.21.15.133:0/2157039026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9bded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.249+0000 7f6001414640 1 -- 172.21.15.133:0/2157039026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9be410 msgr2=0x55e7ec9c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.249+0000 7f6001414640 1 --2- 172.21.15.133:0/2157039026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9be410 0x55e7ec9c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.249+0000 7f6001414640 1 -- 172.21.15.133:0/2157039026 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ec8c9340 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.250+0000 7f6001414640 1 --2- 172.21.15.133:0/2157039026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9cb0a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0d5150 tx=0x7f5fec01a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2678dc7177be7812 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.250+0000 7f5fe2ffd640 1 -- 172.21.15.133:0/2157039026 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0e5070 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.250+0000 7f5fe2ffd640 1 -- 172.21.15.133:0/2157039026 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fec0cd110 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.250+0000 7f5fe2ffd640 1 -- 172.21.15.133:0/2157039026 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0c9080 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.252+0000 7f6005c0d740 1 -- 172.21.15.133:0/2157039026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9c8cb0 msgr2=0x55e7ec9cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.252+0000 7f6005c0d740 1 --2- 172.21.15.133:0/2157039026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9cb0a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0d5150 tx=0x7f5fec01a270 comp rx=0 tx=0).stop 2024-09-11T23:44:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.252+0000 7f6005c0d740 1 -- 172.21.15.133:0/2157039026 shutdown_connections 2024-09-11T23:44:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.252+0000 7f6005c0d740 1 --2- 172.21.15.133:0/2157039026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9cb0a0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.252+0000 7f6005c0d740 1 --2- 172.21.15.133:0/2157039026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9be410 0x55e7ec9c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.252+0000 7f6005c0d740 1 --2- 172.21.15.133:0/2157039026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9bded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.253+0000 7f6005c0d740 1 -- 172.21.15.133:0/2157039026 >> 172.21.15.133:0/2157039026 conn(0x55e7ec9ba060 msgr2=0x55e7ec9ba440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.253+0000 7f6005c0d740 1 -- 172.21.15.133:0/2157039026 shutdown_connections 2024-09-11T23:44:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.253+0000 7f6005c0d740 1 -- 172.21.15.133:0/2157039026 wait complete. 2024-09-11T23:44:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.253+0000 7f6005c0d740 1 -- start start 2024-09-11T23:44:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.254+0000 7f6005c0d740 1 Processor -- start 2024-09-11T23:44:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.254+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.254+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9be410 0x55e7ec9c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.254+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.254+0000 7f6005c0d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e7ec9cb640 con 0x55e7ec9be410 2024-09-11T23:44:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.254+0000 7f6005c0d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e7ec8ab290 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.254+0000 7f6005c0d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e7ec9cb7c0 con 0x55e7ec9bdaf0 2024-09-11T23:44:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.254+0000 7f6005c0d740 10 client.0 _ll_register_callbacks cb 0x55e7ec8dc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.255+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9be410 0x55e7ec9c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.255+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9be410 0x55e7ec9c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43710/0 (socket says 172.21.15.133:43710) 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.255+0000 7f5ffbfff640 1 -- 172.21.15.133:0/3316968631 learned_addr learned my addr 172.21.15.133:0/3316968631 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.255+0000 7f6001414640 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.256+0000 7f5ffbfff640 1 -- 172.21.15.133:0/3316968631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9bdaf0 msgr2=0x55e7ec9c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.256+0000 7f5ffbfff640 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.256+0000 7f5ffbfff640 1 -- 172.21.15.133:0/3316968631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9c8cb0 msgr2=0x55e7ec9c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.256+0000 7f5ffbfff640 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.256+0000 7f5ffbfff640 1 -- 172.21.15.133:0/3316968631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ec635210 con 0x55e7ec9be410 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.256+0000 7f5fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.256+0000 7f5ffbfff640 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9be410 0x55e7ec9c6190 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000bad0 tx=0x7f5ff000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497f3607d76cc6a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.256+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3316968631 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0003de0 con 0x55e7ec9be410 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.256+0000 7f6005c0d740 1 -- 172.21.15.133:0/3316968631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fec0cd110 con 0x55e7ec9be410 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.257+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3316968631 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f5ff0011440 con 0x55e7ec9be410 2024-09-11T23:44:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.257+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3316968631 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5ff0011640 con 0x55e7ec9be410 2024-09-11T23:44:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.257+0000 7f6005c0d740 10 client.17439 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.257+0000 7f6005c0d740 1 -- 172.21.15.133:0/3316968631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e7ec701620 con 0x55e7ec9be410 2024-09-11T23:44:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.257+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3316968631 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fec0cd110 con 0x55e7ec9be410 2024-09-11T23:44:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.257+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3316968631 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5ff001b9d0 con 0x55e7ec9be410 2024-09-11T23:44:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.258+0000 7f6005c0d740 10 client.17439 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.258+0000 7f6005c0d740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.258+0000 7f6005c0d740 20 client.17439 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.258+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7ec9d8ab0 0x55e7ec9daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.258+0000 7f6005c0d740 4 client.17439 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.258+0000 7f6005c0d740 1 -- 172.21.15.133:0/3316968631 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ec92fa20 con 0x55e7ec9d8ab0 2024-09-11T23:44:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.258+0000 7f6000c13640 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7ec9d8ab0 0x55e7ec9daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.259+0000 7f6000c13640 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7ec9d8ab0 0x55e7ec9daf20 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db316d42a7364fb2 server_cookie=38f70f7b645769ee in_seq=0 out_seq=0 2024-09-11T23:44:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.259+0000 7f5fdbfff640 10 client.17439 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.260+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3316968631 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7ec92fa20 con 0x55e7ec9d8ab0 2024-09-11T23:44:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.260+0000 7f5fdbfff640 10 client.17439 handle_command_reply: tid=0 2024-09-11T23:44:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.260+0000 7f6005c0d740 1 client.17439 shutdown 2024-09-11T23:44:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f6005c0d740 1 -- 172.21.15.133:0/3316968631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9be410 msgr2=0x55e7ec9c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9be410 0x55e7ec9c6190 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000bad0 tx=0x7f5ff000bf90 comp rx=0 tx=0).stop 2024-09-11T23:44:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f6005c0d740 1 -- 172.21.15.133:0/3316968631 shutdown_connections 2024-09-11T23:44:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7ec9d8ab0 0x55e7ec9daf20 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9be410 0x55e7ec9c6190 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f5fdbfff640 0 client.17439 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3316968631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f6005c0d740 1 -- 172.21.15.133:0/3316968631 >> 172.21.15.133:0/3316968631 conn(0x55e7ec9ba060 msgr2=0x55e7ec9bfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f6005c0d740 1 -- 172.21.15.133:0/3316968631 shutdown_connections 2024-09-11T23:44:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f6005c0d740 1 -- 172.21.15.133:0/3316968631 wait complete. 2024-09-11T23:44:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.261+0000 7f6005c0d740 20 client.17439 trim_cache size 0 max 16384 2024-09-11T23:44:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.291+0000 7f6005c0d740 1 Processor -- start 2024-09-11T23:44:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.304+0000 7f6005c0d740 1 -- start start 2024-09-11T23:44:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.304+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9c6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.305+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9be410 0x55e7ec9e0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.305+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9e0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.305+0000 7f6005c0d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1b3c50 con 0x55e7ec9bdaf0 2024-09-11T23:44:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.305+0000 7f6005c0d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1b5790 con 0x55e7ec9be410 2024-09-11T23:44:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.305+0000 7f6005c0d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1b4320 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.305+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9be410 0x55e7ec9e0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.305+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9be410 0x55e7ec9e0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47248/0 (socket says 172.21.15.133:47248) 2024-09-11T23:44:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.305+0000 7f5ffbfff640 1 -- 172.21.15.133:0/3269314537 learned_addr learned my addr 172.21.15.133:0/3269314537 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.305+0000 7f6000c13640 1 --2- 172.21.15.133:0/3269314537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.306+0000 7f6000c13640 1 -- 172.21.15.133:0/3269314537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9c8cb0 msgr2=0x55e7ec9e0d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.306+0000 7f6000c13640 1 --2- 172.21.15.133:0/3269314537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9e0d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.306+0000 7f6000c13640 1 -- 172.21.15.133:0/3269314537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9be410 msgr2=0x55e7ec9e0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.306+0000 7f6000c13640 1 --2- 172.21.15.133:0/3269314537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9be410 0x55e7ec9e0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.306+0000 7f6000c13640 1 -- 172.21.15.133:0/3269314537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ec9db5a0 con 0x55e7ec9bdaf0 2024-09-11T23:44:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.308+0000 7f6000c13640 1 --2- 172.21.15.133:0/3269314537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9c6500 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4029fd0 tx=0x7f5ff4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55b3d1a9ca70060 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.309+0000 7f5fe27fc640 1 -- 172.21.15.133:0/3269314537 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff402f070 con 0x55e7ec9bdaf0 2024-09-11T23:44:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.309+0000 7f5fe27fc640 1 -- 172.21.15.133:0/3269314537 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ff4029a90 con 0x55e7ec9bdaf0 2024-09-11T23:44:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.311+0000 7f6005c0d740 1 -- 172.21.15.133:0/3269314537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9bdaf0 msgr2=0x55e7ec9c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.311+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3269314537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9c6500 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4029fd0 tx=0x7f5ff4013000 comp rx=0 tx=0).stop 2024-09-11T23:44:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.311+0000 7f6005c0d740 1 -- 172.21.15.133:0/3269314537 shutdown_connections 2024-09-11T23:44:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.311+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3269314537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9e0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.311+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3269314537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9be410 0x55e7ec9e0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.311+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3269314537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9bdaf0 0x55e7ec9c6500 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.311+0000 7f6005c0d740 1 -- 172.21.15.133:0/3269314537 >> 172.21.15.133:0/3269314537 conn(0x55e7ec9ba060 msgr2=0x55e7ec9c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.311+0000 7f6005c0d740 1 -- 172.21.15.133:0/3269314537 shutdown_connections 2024-09-11T23:44:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.312+0000 7f6005c0d740 1 -- 172.21.15.133:0/3269314537 wait complete. 2024-09-11T23:44:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.312+0000 7f6005c0d740 1 -- start start 2024-09-11T23:44:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.312+0000 7f6005c0d740 1 Processor -- start 2024-09-11T23:44:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.313+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9be410 0x55e7ec9d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.313+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.313+0000 7f6005c0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9d1490 0x55e7ec9cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.313+0000 7f6005c0d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1b50c0 con 0x55e7ec9be410 2024-09-11T23:44:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.313+0000 7f6005c0d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1b49f0 con 0x55e7ec9d1490 2024-09-11T23:44:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.313+0000 7f6005c0d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1b1c50 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.313+0000 7f6005c0d740 10 client.0 _ll_register_callbacks cb 0x7f5ff0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.314+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.314+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35300/0 (socket says 172.21.15.133:35300) 2024-09-11T23:44:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.314+0000 7f5ffbfff640 1 -- 172.21.15.133:0/3239084750 learned_addr learned my addr 172.21.15.133:0/3239084750 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.318+0000 7f6001414640 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9d1490 0x55e7ec9cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.318+0000 7f5ffbfff640 1 -- 172.21.15.133:0/3239084750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9d1490 msgr2=0x55e7ec9cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.318+0000 7f5ffbfff640 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9d1490 0x55e7ec9cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.318+0000 7f5ffbfff640 1 -- 172.21.15.133:0/3239084750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9be410 msgr2=0x55e7ec9d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.318+0000 7f5ffbfff640 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9be410 0x55e7ec9d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.318+0000 7f5ffbfff640 1 -- 172.21.15.133:0/3239084750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ec9c6a40 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.319+0000 7f5ffbfff640 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9d0f50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff0005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85287deab625b96a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.322+0000 7f5fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:44:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.322+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3239084750 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0028070 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.322+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3239084750 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f5ff0011440 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.322+0000 7f6005c0d740 1 -- 172.21.15.133:0/3239084750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e7ec929d70 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.324+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3239084750 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5ff00117e0 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.324+0000 7f6005c0d740 10 client.26803 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.324+0000 7f6005c0d740 1 -- 172.21.15.133:0/3239084750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e7ec9d7600 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.325+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3239084750 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e7ec929d70 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.327+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3239084750 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5ff0004630 con 0x55e7ec9c8cb0 2024-09-11T23:44:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.327+0000 7f6005c0d740 10 client.26803 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.327+0000 7f6005c0d740 10 client.26803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.327+0000 7f6005c0d740 20 client.26803 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.327+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7eca44680 0x55e7eca46aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.327+0000 7f6005c0d740 4 client.26803 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:44:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.327+0000 7f6005c0d740 1 -- 172.21.15.133:0/3239084750 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7ec8a6900 con 0x55e7eca44680 2024-09-11T23:44:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.329+0000 7f6000c13640 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7eca44680 0x55e7eca46aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.335+0000 7f6000c13640 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7eca44680 0x55e7eca46aa0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5203238a50198042 server_cookie=b4de50458b598b4e in_seq=0 out_seq=0 2024-09-11T23:44:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:37.335+0000 7f5fdbfff640 10 client.26803 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:37.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.782+0000 7f7bb91bc640 1 -- 172.21.15.133:0/136808645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb406e4f0 msgr2=0x7f7bb406e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.782+0000 7f7bb91bc640 1 --2- 172.21.15.133:0/136808645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb406e4f0 0x7f7bb406e930 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00afd0 tx=0x7f7bac02b430 comp rx=0 tx=0).stop 2024-09-11T23:44:37.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.782+0000 7f7bb91bc640 1 -- 172.21.15.133:0/136808645 shutdown_connections 2024-09-11T23:44:37.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.782+0000 7f7bb91bc640 1 --2- 172.21.15.133:0/136808645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb406ee70 0x7f7bb407a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.782+0000 7f7bb91bc640 1 --2- 172.21.15.133:0/136808645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb406e4f0 0x7f7bb406e930 secure :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00afd0 tx=0x7f7bac02b430 comp rx=0 tx=0).stop 2024-09-11T23:44:37.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.782+0000 7f7bb91bc640 1 --2- 172.21.15.133:0/136808645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bb410e090 0x7f7bb410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.782+0000 7f7bb91bc640 1 -- 172.21.15.133:0/136808645 >> 172.21.15.133:0/136808645 conn(0x7f7bb4073180 msgr2=0x7f7bb4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.783+0000 7f7bb91bc640 1 -- 172.21.15.133:0/136808645 shutdown_connections 2024-09-11T23:44:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.783+0000 7f7bb91bc640 1 -- 172.21.15.133:0/136808645 wait complete. 2024-09-11T23:44:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.784+0000 7f7bb91bc640 1 Processor -- start 2024-09-11T23:44:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.784+0000 7f7bb91bc640 1 -- start start 2024-09-11T23:44:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.785+0000 7f7bb91bc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb406ee70 0x7f7bb4074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.785+0000 7f7bb91bc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410e090 0x7f7bb40748d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.785+0000 7f7bb91bc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bb4074e10 0x7f7bb41c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.785+0000 7f7bb91bc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb407dad0 con 0x7f7bb406ee70 2024-09-11T23:44:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.785+0000 7f7bb91bc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb407d950 con 0x7f7bb410e090 2024-09-11T23:44:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.785+0000 7f7bb91bc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb407dc50 con 0x7f7bb4074e10 2024-09-11T23:44:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.785+0000 7f7bb37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410e090 0x7f7bb40748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.786+0000 7f7bb37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410e090 0x7f7bb40748d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47284/0 (socket says 172.21.15.133:47284) 2024-09-11T23:44:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.786+0000 7f7bb37fe640 1 -- 172.21.15.133:0/3408633936 learned_addr learned my addr 172.21.15.133:0/3408633936 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:37.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.787+0000 7f7bb89bb640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bb4074e10 0x7f7bb41c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.787+0000 7f7bb37fe640 1 -- 172.21.15.133:0/3408633936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bb4074e10 msgr2=0x7f7bb41c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.787+0000 7f7bb37fe640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bb4074e10 0x7f7bb41c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.787+0000 7f7bb37fe640 1 -- 172.21.15.133:0/3408633936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb406ee70 msgr2=0x7f7bb4074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.787+0000 7f7bb37fe640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb406ee70 0x7f7bb4074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.787+0000 7f7bb37fe640 1 -- 172.21.15.133:0/3408633936 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb41c48d0 con 0x7f7bb410e090 2024-09-11T23:44:37.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.788+0000 7f7bb89bb640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bb4074e10 0x7f7bb41c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:44:37.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.792+0000 7f7bb37fe640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410e090 0x7f7bb40748d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7bac02ba20 tx=0x7f7bac032880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.796+0000 7f7bb17fa640 1 -- 172.21.15.133:0/3408633936 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac041070 con 0x7f7bb410e090 2024-09-11T23:44:37.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.797+0000 7f7bb17fa640 1 -- 172.21.15.133:0/3408633936 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bac02fbe0 con 0x7f7bb410e090 2024-09-11T23:44:37.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.797+0000 7f7bb17fa640 1 -- 172.21.15.133:0/3408633936 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac037040 con 0x7f7bb410e090 2024-09-11T23:44:37.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.797+0000 7f7bb91bc640 1 -- 172.21.15.133:0/3408633936 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb41c4b60 con 0x7f7bb410e090 2024-09-11T23:44:37.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.797+0000 7f7bb91bc640 1 -- 172.21.15.133:0/3408633936 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb41c50a0 con 0x7f7bb410e090 2024-09-11T23:44:37.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.803+0000 7f7bbc7f7740 1 Processor -- start 2024-09-11T23:44:37.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.804+0000 7f7bbc7f7740 1 -- start start 2024-09-11T23:44:37.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.804+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f85e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.805+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85e780 0x56305f85ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.805+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f869520 0x56305f86b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.805+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56305f7c42f0 con 0x56305f869520 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.805+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56305f76bcf0 con 0x56305f85e780 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.805+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56305f7c02b0 con 0x56305f85fcf0 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.806+0000 7f7bb89bb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f85e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.806+0000 7f7bb89bb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f85e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35336/0 (socket says 172.21.15.133:35336) 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.806+0000 7f7bb89bb640 1 -- 172.21.15.133:0/2377138177 learned_addr learned my addr 172.21.15.133:0/2377138177 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.808+0000 7f7bb3fff640 1 --2- 172.21.15.133:0/2377138177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85e780 0x56305f85ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.808+0000 7f7bb89bb640 1 -- 172.21.15.133:0/2377138177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85e780 msgr2=0x56305f85ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.808+0000 7f7bb89bb640 1 --2- 172.21.15.133:0/2377138177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85e780 0x56305f85ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.808+0000 7f7bb89bb640 1 -- 172.21.15.133:0/2377138177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f869520 msgr2=0x56305f86b910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.808+0000 7f7bb89bb640 1 --2- 172.21.15.133:0/2377138177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f869520 0x56305f86b910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.809+0000 7f7bb89bb640 1 -- 172.21.15.133:0/2377138177 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56305f7503a0 con 0x56305f85fcf0 2024-09-11T23:44:37.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.809+0000 7f7bb17fa640 1 -- 172.21.15.133:0/3408633936 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7bac03a070 con 0x7f7bb410e090 2024-09-11T23:44:37.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.810+0000 7f7bb89bb640 1 --2- 172.21.15.133:0/2377138177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f85e240 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800bb70 tx=0x7f7ba800d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25fce8376616e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.811+0000 7f7bb17fa640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b9c0a2e80 0x7f7b9c0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.811+0000 7f7bb3fff640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b9c0a2e80 0x7f7b9c0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.811+0000 7f7b9a7fc640 1 -- 172.21.15.133:0/2377138177 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8017620 con 0x56305f85fcf0 2024-09-11T23:44:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.811+0000 7f7b9a7fc640 1 -- 172.21.15.133:0/2377138177 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba8017ca0 con 0x56305f85fcf0 2024-09-11T23:44:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.813+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2377138177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 msgr2=0x56305f85e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.813+0000 7f7bb3fff640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b9c0a2e80 0x7f7b9c0a5340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4005fd0 tx=0x7f7ba4005e00 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.813+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/2377138177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f85e240 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800bb70 tx=0x7f7ba800d710 comp rx=0 tx=0).stop 2024-09-11T23:44:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.814+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2377138177 shutdown_connections 2024-09-11T23:44:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.814+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/2377138177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f869520 0x56305f86b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.814+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/2377138177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85e780 0x56305f85ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.814+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/2377138177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f85e240 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.814+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2377138177 >> 172.21.15.133:0/2377138177 conn(0x56305f827310 msgr2=0x56305f85ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:37.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.814+0000 7f7bb17fa640 1 -- 172.21.15.133:0/3408633936 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f7bac0b94f0 con 0x7f7bb410e090 2024-09-11T23:44:37.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.817+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2377138177 shutdown_connections 2024-09-11T23:44:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.817+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2377138177 wait complete. 2024-09-11T23:44:37.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.818+0000 7f7bbc7f7740 1 -- start start 2024-09-11T23:44:37.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.818+0000 7f7bbc7f7740 1 Processor -- start 2024-09-11T23:44:37.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.819+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85e780 0x56305f868e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.819+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f8660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.819+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f8665f0 0x56305f866a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.819+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56305f86c2c0 con 0x56305f8665f0 2024-09-11T23:44:37.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.819+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56305f77f060 con 0x56305f85e780 2024-09-11T23:44:37.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.819+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56305f86c440 con 0x56305f85fcf0 2024-09-11T23:44:37.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.819+0000 7f7bbc7f7740 10 client.0 _ll_register_callbacks cb 0x56305f4d28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:37.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.820+0000 7f7bb89bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f8665f0 0x56305f866a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.821+0000 7f7bb89bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f8665f0 0x56305f866a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43764/0 (socket says 172.21.15.133:43764) 2024-09-11T23:44:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.821+0000 7f7bb89bb640 1 -- 172.21.15.133:0/4099521431 learned_addr learned my addr 172.21.15.133:0/4099521431 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.821+0000 7f7bb89bb640 1 -- 172.21.15.133:0/4099521431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 msgr2=0x56305f8660b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.821+0000 7f7bb89bb640 1 --2- 172.21.15.133:0/4099521431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f8660b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.821+0000 7f7bb89bb640 1 -- 172.21.15.133:0/4099521431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85e780 msgr2=0x56305f868e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.821+0000 7f7bb89bb640 1 --2- 172.21.15.133:0/4099521431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85e780 0x56305f868e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.821+0000 7f7bb89bb640 1 -- 172.21.15.133:0/4099521431 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56305f5353c0 con 0x56305f8665f0 2024-09-11T23:44:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.822+0000 7f7bb89bb640 1 --2- 172.21.15.133:0/4099521431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f8665f0 0x56305f866a70 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800f870 tx=0x7f7ba80176d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804f93c5cfa116bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.822+0000 7f7b937fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.822+0000 7f7b937fe640 1 -- 172.21.15.133:0/4099521431 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8025020 con 0x56305f8665f0 2024-09-11T23:44:37.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.823+0000 7f7b937fe640 1 -- 172.21.15.133:0/4099521431 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f7ba8004680 con 0x56305f8665f0 2024-09-11T23:44:37.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.823+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/4099521431 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ba8017ca0 con 0x56305f8665f0 2024-09-11T23:44:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.824+0000 7f7b937fe640 1 -- 172.21.15.133:0/4099521431 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ba8017e50 con 0x56305f8665f0 2024-09-11T23:44:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.824+0000 7f7bbc7f7740 10 client.17448 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.824+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/4099521431 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56305f878110 con 0x56305f8665f0 2024-09-11T23:44:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.824+0000 7f7b937fe640 1 -- 172.21.15.133:0/4099521431 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba8017ca0 con 0x56305f8665f0 2024-09-11T23:44:37.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.826+0000 7f7b937fe640 1 -- 172.21.15.133:0/4099521431 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7ba8021450 con 0x56305f8665f0 2024-09-11T23:44:37.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.826+0000 7f7bbc7f7740 10 client.17448 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:37.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.826+0000 7f7bbc7f7740 10 client.17448 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:44:37.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.826+0000 7f7bbc7f7740 20 client.17448 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:37.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.826+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/4099521431 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56305f879840 0x56305f87bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.826+0000 7f7bbc7f7740 4 client.17448 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:37.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.826+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/4099521431 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56305f7d1860 con 0x56305f879840 2024-09-11T23:44:37.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.829+0000 7f7bb3fff640 1 --2- 172.21.15.133:0/4099521431 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56305f879840 0x56305f87bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.831+0000 7f7bb3fff640 1 --2- 172.21.15.133:0/4099521431 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56305f879840 0x56305f87bce0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45270305f3d43f44 server_cookie=917b263c122c63d in_seq=0 out_seq=0 2024-09-11T23:44:37.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.831+0000 7f7b937fe640 10 client.17448 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:44:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.833+0000 7f7b937fe640 1 -- 172.21.15.133:0/4099521431 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56305f7d1860 con 0x56305f879840 2024-09-11T23:44:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.833+0000 7f7b937fe640 10 client.17448 handle_command_reply: tid=0 2024-09-11T23:44:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.833+0000 7f7bbc7f7740 1 client.17448 shutdown 2024-09-11T23:44:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.833+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/4099521431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f8665f0 msgr2=0x56305f866a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.833+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/4099521431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f8665f0 0x56305f866a70 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800f870 tx=0x7f7ba80176d0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.837+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/4099521431 shutdown_connections 2024-09-11T23:44:37.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.837+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/4099521431 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56305f879840 0x56305f87bce0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.837+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/4099521431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f8665f0 0x56305f866a70 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.837+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/4099521431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f8660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.837+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/4099521431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85e780 0x56305f868e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.837+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/4099521431 >> 172.21.15.133:0/4099521431 conn(0x56305f827310 msgr2=0x56305f860b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:37.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.837+0000 7f7b937fe640 0 client.17448 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:44:37.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.837+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/4099521431 shutdown_connections 2024-09-11T23:44:37.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.837+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/4099521431 wait complete. 2024-09-11T23:44:37.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.838+0000 7f7bbc7f7740 20 client.17448 trim_cache size 0 max 16384 2024-09-11T23:44:37.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.882+0000 7f7bbc7f7740 1 Processor -- start 2024-09-11T23:44:37.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.885+0000 7f7bbc7f7740 1 -- start start 2024-09-11T23:44:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.887+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f85e780 0x56305f865970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.891+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f861290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.891+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f8665f0 0x56305f8617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.891+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba000e660 con 0x56305f85e780 2024-09-11T23:44:37.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.891+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba00101a0 con 0x56305f8665f0 2024-09-11T23:44:37.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.891+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba000ed30 con 0x56305f85fcf0 2024-09-11T23:44:37.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.894+0000 7f7bb3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f85e780 0x56305f865970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.902+0000 7f7bb37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f861290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.902+0000 7f7bb37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f861290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35350/0 (socket says 172.21.15.133:35350) 2024-09-11T23:44:37.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.902+0000 7f7bb37fe640 1 -- 172.21.15.133:0/965042022 learned_addr learned my addr 172.21.15.133:0/965042022 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:37.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.902+0000 7f7bb89bb640 1 --2- 172.21.15.133:0/965042022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f8665f0 0x56305f8617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.904+0000 7f7bb37fe640 1 -- 172.21.15.133:0/965042022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f8665f0 msgr2=0x56305f8617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.904+0000 7f7bb37fe640 1 --2- 172.21.15.133:0/965042022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f8665f0 0x56305f8617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.904+0000 7f7bb37fe640 1 -- 172.21.15.133:0/965042022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f85e780 msgr2=0x56305f865970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.904+0000 7f7bb37fe640 1 --2- 172.21.15.133:0/965042022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f85e780 0x56305f865970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.904+0000 7f7bb37fe640 1 -- 172.21.15.133:0/965042022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56305f869170 con 0x56305f85fcf0 2024-09-11T23:44:37.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.906+0000 7f7bb37fe640 1 --2- 172.21.15.133:0/965042022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f861290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00d090 tx=0x7f7bac0c0920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1341996222a8e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.907+0000 7f7b99ffb640 1 -- 172.21.15.133:0/965042022 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0b7e80 con 0x56305f85fcf0 2024-09-11T23:44:37.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.907+0000 7f7b99ffb640 1 -- 172.21.15.133:0/965042022 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bac0059a0 con 0x56305f85fcf0 2024-09-11T23:44:37.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.907+0000 7f7b99ffb640 1 -- 172.21.15.133:0/965042022 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0b8020 con 0x56305f85fcf0 2024-09-11T23:44:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.909+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/965042022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 msgr2=0x56305f861290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.909+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/965042022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f861290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00d090 tx=0x7f7bac0c0920 comp rx=0 tx=0).stop 2024-09-11T23:44:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.909+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/965042022 shutdown_connections 2024-09-11T23:44:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.909+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/965042022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f8665f0 0x56305f8617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.909+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/965042022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f85fcf0 0x56305f861290 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.909+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/965042022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f85e780 0x56305f865970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.909+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/965042022 >> 172.21.15.133:0/965042022 conn(0x56305f827310 msgr2=0x56305f864980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.910+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/965042022 shutdown_connections 2024-09-11T23:44:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.910+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/965042022 wait complete. 2024-09-11T23:44:37.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.911+0000 7f7bbc7f7740 1 -- start start 2024-09-11T23:44:37.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.911+0000 7f7bbc7f7740 1 Processor -- start 2024-09-11T23:44:37.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.912+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f85e780 0x56305f8dc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.913+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85fcf0 0x56305f8dce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.913+0000 7f7bbc7f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f8665f0 0x56305f86cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.913+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba000fad0 con 0x56305f85e780 2024-09-11T23:44:37.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.913+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba000f400 con 0x56305f85fcf0 2024-09-11T23:44:37.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.913+0000 7f7bbc7f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba000c660 con 0x56305f8665f0 2024-09-11T23:44:37.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.914+0000 7f7bbc7f7740 10 client.0 _ll_register_callbacks cb 0x7f7ba800b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:37.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.914+0000 7f7bb37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85fcf0 0x56305f8dce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.914+0000 7f7bb37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85fcf0 0x56305f8dce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47332/0 (socket says 172.21.15.133:47332) 2024-09-11T23:44:37.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.914+0000 7f7bb37fe640 1 -- 172.21.15.133:0/2597276293 learned_addr learned my addr 172.21.15.133:0/2597276293 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:37.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.914+0000 7f7bb89bb640 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f8665f0 0x56305f86cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.915+0000 7f7bb3fff640 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f85e780 0x56305f8dc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.915+0000 7f7bb37fe640 1 -- 172.21.15.133:0/2597276293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f8665f0 msgr2=0x56305f86cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.915+0000 7f7bb37fe640 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f8665f0 0x56305f86cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.915+0000 7f7bb37fe640 1 -- 172.21.15.133:0/2597276293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f85e780 msgr2=0x56305f8dc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:37.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.915+0000 7f7bb37fe640 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f85e780 0x56305f8dc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.916+0000 7f7bb37fe640 1 -- 172.21.15.133:0/2597276293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56305f86d550 con 0x56305f85fcf0 2024-09-11T23:44:37.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.917+0000 7f7bb37fe640 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85fcf0 0x56305f8dce20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0b86d0 tx=0x7f7bac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e443386c61671ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:37.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.918+0000 7f7b937fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:44:37.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.918+0000 7f7b937fe640 1 -- 172.21.15.133:0/2597276293 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0b81c0 con 0x56305f85fcf0 2024-09-11T23:44:37.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.918+0000 7f7b937fe640 1 -- 172.21.15.133:0/2597276293 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f7bac0c4060 con 0x56305f85fcf0 2024-09-11T23:44:37.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.918+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2597276293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bac0059a0 con 0x56305f85fcf0 2024-09-11T23:44:37.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.919+0000 7f7b937fe640 1 -- 172.21.15.133:0/2597276293 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7bac0b8390 con 0x56305f85fcf0 2024-09-11T23:44:37.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.926+0000 7f7bbc7f7740 10 client.26963 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:37.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.926+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2597276293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56305f8e4080 con 0x56305f85fcf0 2024-09-11T23:44:37.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.928+0000 7f7b937fe640 1 -- 172.21.15.133:0/2597276293 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bac032c90 con 0x56305f85fcf0 2024-09-11T23:44:37.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.928+0000 7f7b937fe640 1 -- 172.21.15.133:0/2597276293 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7bac0ba8a0 con 0x56305f85fcf0 2024-09-11T23:44:37.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.931+0000 7f7bbc7f7740 10 client.26963 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:37.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.931+0000 7f7bbc7f7740 10 client.26963 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:44:37.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.931+0000 7f7bbc7f7740 20 client.26963 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:37.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.931+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56305f8e4780 0x56305f8e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:37.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.931+0000 7f7bbc7f7740 4 client.26963 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:44:37.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.931+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2597276293 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7bac0059a0 con 0x56305f8e4780 2024-09-11T23:44:37.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.933+0000 7f7bb3fff640 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56305f8e4780 0x56305f8e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:37.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.941+0000 7f7b937fe640 10 client.26963 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:44:37.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.941+0000 7f7bb3fff640 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56305f8e4780 0x56305f8e6ba0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=33136f34b586fc03 server_cookie=6d4c38b4190e3863 in_seq=0 out_seq=0 2024-09-11T23:44:37.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.942+0000 7f7b937fe640 1 -- 172.21.15.133:0/2597276293 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7bac0059a0 con 0x56305f8e4780 2024-09-11T23:44:37.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.942+0000 7f7b937fe640 10 client.26963 handle_command_reply: tid=0 2024-09-11T23:44:37.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.942+0000 7f7bbc7f7740 1 client.26963 shutdown 2024-09-11T23:44:37.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.943+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2597276293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85fcf0 msgr2=0x56305f8dce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.943+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85fcf0 0x56305f8dce20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0b86d0 tx=0x7f7bac004a40 comp rx=0 tx=0).stop 2024-09-11T23:44:37.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.944+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2597276293 shutdown_connections 2024-09-11T23:44:37.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.944+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x56305f8e4780 0x56305f8e6ba0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.944+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56305f8665f0 0x56305f86cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.944+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56305f85fcf0 0x56305f8dce20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.944+0000 7f7bbc7f7740 1 --2- 172.21.15.133:0/2597276293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56305f85e780 0x56305f8dc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.944+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2597276293 >> 172.21.15.133:0/2597276293 conn(0x56305f827310 msgr2=0x56305f867910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:37.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.944+0000 7f7b937fe640 0 client.26963 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:44:37.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.945+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2597276293 shutdown_connections 2024-09-11T23:44:37.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.945+0000 7f7bbc7f7740 1 -- 172.21.15.133:0/2597276293 wait complete. 2024-09-11T23:44:37.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.947+0000 7f7bbc7f7740 20 client.26963 trim_cache size 0 max 16384 2024-09-11T23:44:37.953 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:44:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.957+0000 7f7bb91bc640 1 -- 172.21.15.133:0/3408633936 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b9c0a2e80 msgr2=0x7f7b9c0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.957+0000 7f7bb91bc640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b9c0a2e80 0x7f7b9c0a5340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4005fd0 tx=0x7f7ba4005e00 comp rx=0 tx=0).stop 2024-09-11T23:44:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.957+0000 7f7bb91bc640 1 -- 172.21.15.133:0/3408633936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410e090 msgr2=0x7f7bb40748d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.957+0000 7f7bb91bc640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410e090 0x7f7bb40748d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7bac02ba20 tx=0x7f7bac032880 comp rx=0 tx=0).stop 2024-09-11T23:44:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.957+0000 7f7bb91bc640 1 -- 172.21.15.133:0/3408633936 shutdown_connections 2024-09-11T23:44:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.957+0000 7f7bb91bc640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b9c0a2e80 0x7f7b9c0a5340 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.957+0000 7f7bb91bc640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bb4074e10 0x7f7bb41c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.957+0000 7f7bb91bc640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410e090 0x7f7bb40748d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.957+0000 7f7bb91bc640 1 --2- 172.21.15.133:0/3408633936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb406ee70 0x7f7bb4074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.957+0000 7f7bb91bc640 1 -- 172.21.15.133:0/3408633936 >> 172.21.15.133:0/3408633936 conn(0x7f7bb4073180 msgr2=0x7f7bb406cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.958+0000 7f7bb91bc640 1 -- 172.21.15.133:0/3408633936 shutdown_connections 2024-09-11T23:44:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:37.958+0000 7f7bb91bc640 1 -- 172.21.15.133:0/3408633936 wait complete. 2024-09-11T23:44:38.943 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:38 smithi133 ceph-mon[31430]: pgmap v243: 97 pgs: 97 active+clean; 68 GiB data, 227 GiB used, 846 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 626 MiB/s wr, 1.50k op/s 2024-09-11T23:44:38.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:38 smithi138 ceph-mon[35142]: pgmap v243: 97 pgs: 97 active+clean; 68 GiB data, 227 GiB used, 846 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 626 MiB/s wr, 1.50k op/s 2024-09-11T23:44:39.024 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:44:39.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:38 smithi178 ceph-mon[35882]: pgmap v243: 97 pgs: 97 active+clean; 68 GiB data, 227 GiB used, 846 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 626 MiB/s wr, 1.50k op/s 2024-09-11T23:44:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f5fdbfff640 1 -- 172.21.15.133:0/3239084750 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7ec8a6900 con 0x55e7eca44680 2024-09-11T23:44:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f5fdbfff640 10 client.26803 handle_command_reply: tid=0 2024-09-11T23:44:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f6005c0d740 1 client.26803 shutdown 2024-09-11T23:44:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f6005c0d740 1 -- 172.21.15.133:0/3239084750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9c8cb0 msgr2=0x55e7ec9d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9d0f50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff0005870 comp rx=0 tx=0).stop 2024-09-11T23:44:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f6005c0d740 1 -- 172.21.15.133:0/3239084750 shutdown_connections 2024-09-11T23:44:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e7eca44680 0x55e7eca46aa0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7ec9d1490 0x55e7ec9cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7ec9c8cb0 0x55e7ec9d0f50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f6005c0d740 1 --2- 172.21.15.133:0/3239084750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7ec9be410 0x55e7ec9d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f6005c0d740 1 -- 172.21.15.133:0/3239084750 >> 172.21.15.133:0/3239084750 conn(0x55e7ec9ba060 msgr2=0x55e7ec9bf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.613+0000 7f5fdbfff640 0 client.26803 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.614+0000 7f6005c0d740 1 -- 172.21.15.133:0/3239084750 shutdown_connections 2024-09-11T23:44:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.614+0000 7f6005c0d740 1 -- 172.21.15.133:0/3239084750 wait complete. 2024-09-11T23:44:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.614+0000 7f6005c0d740 20 client.26803 trim_cache size 0 max 16384 2024-09-11T23:44:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:44:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:44:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:44:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:44:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 -- 172.21.15.133:0/603624612 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5fe40a2f40 msgr2=0x7f5fe40a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5fe40a2f40 0x7f5fe40a5400 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4005fd0 tx=0x7f5ff4009680 comp rx=0 tx=0).stop 2024-09-11T23:44:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 -- 172.21.15.133:0/603624612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ffc11b140 msgr2=0x7f5ffc116d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ffc11b140 0x7f5ffc116d40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0054d0 tx=0x7f5fec004ac0 comp rx=0 tx=0).stop 2024-09-11T23:44:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 -- 172.21.15.133:0/603624612 shutdown_connections 2024-09-11T23:44:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5fe40a2f40 0x7f5fe40a5400 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ffc11b140 0x7f5ffc116d40 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ffc1095d0 0x7f5ffc116800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 --2- 172.21.15.133:0/603624612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc1091f0 0x7f5ffc11a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 -- 172.21.15.133:0/603624612 >> 172.21.15.133:0/603624612 conn(0x7f5ffc06ce00 msgr2=0x7f5ffc0707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 -- 172.21.15.133:0/603624612 shutdown_connections 2024-09-11T23:44:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:39.615+0000 7f6002e77640 1 -- 172.21.15.133:0/603624612 wait complete. 2024-09-11T23:44:39.627 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:40.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.336+0000 7f8223383640 1 -- 172.21.15.133:0/1741044034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c11b140 msgr2=0x7f821c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:40.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.336+0000 7f8223383640 1 --2- 172.21.15.133:0/1741044034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c11b140 0x7f821c11d530 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f821401acb0 tx=0x7f821403b920 comp rx=0 tx=0).stop 2024-09-11T23:44:40.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.336+0000 7f8223383640 1 -- 172.21.15.133:0/1741044034 shutdown_connections 2024-09-11T23:44:40.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.336+0000 7f8223383640 1 --2- 172.21.15.133:0/1741044034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c11b140 0x7f821c11d530 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.336+0000 7f8223383640 1 --2- 172.21.15.133:0/1741044034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f821c109830 0x7f821c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.336+0000 7f8223383640 1 --2- 172.21.15.133:0/1741044034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c109450 0x7f821c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.336+0000 7f8223383640 1 -- 172.21.15.133:0/1741044034 >> 172.21.15.133:0/1741044034 conn(0x7f821c06e140 msgr2=0x7f821c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:40.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.339+0000 7f8223383640 1 -- 172.21.15.133:0/1741044034 shutdown_connections 2024-09-11T23:44:40.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.341+0000 7f8223383640 1 -- 172.21.15.133:0/1741044034 wait complete. 2024-09-11T23:44:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.343+0000 7f8223383640 1 Processor -- start 2024-09-11T23:44:40.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.350+0000 7f8223383640 1 -- start start 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.350+0000 7f8223383640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c109450 0x7f821c11a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.350+0000 7f8223383640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f821c109830 0x7f821c1168f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.350+0000 7f8223383640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c11b140 0x7f821c116e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.350+0000 7f8223383640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1132f0 con 0x7f821c11b140 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.350+0000 7f8223383640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f821c113170 con 0x7f821c109450 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.350+0000 7f8223383640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f821c113470 con 0x7f821c109830 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.350+0000 7f822091e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f821c109830 0x7f821c1168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.350+0000 7f822091e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f821c109830 0x7f821c1168f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35382/0 (socket says 172.21.15.133:35382) 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.351+0000 7f822091e640 1 -- 172.21.15.133:0/304050072 learned_addr learned my addr 172.21.15.133:0/304050072 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.353+0000 7f8221920640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c11b140 0x7f821c116e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.353+0000 7f822091e640 1 -- 172.21.15.133:0/304050072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c109450 msgr2=0x7f821c11a850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.353+0000 7f822091e640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c109450 0x7f821c11a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.353+0000 7f822091e640 1 -- 172.21.15.133:0/304050072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c11b140 msgr2=0x7f821c116e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:40.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.353+0000 7f822091e640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c11b140 0x7f821c116e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.353+0000 7f822091e640 1 -- 172.21.15.133:0/304050072 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c117560 con 0x7f821c109830 2024-09-11T23:44:40.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.362+0000 7f822091e640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f821c109830 0x7f821c1168f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f820c00e9f0 tx=0x7f820c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:40.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.364+0000 7f82127fc640 1 -- 172.21.15.133:0/304050072 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c00cd80 con 0x7f821c109830 2024-09-11T23:44:40.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.364+0000 7f82127fc640 1 -- 172.21.15.133:0/304050072 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820c010910 con 0x7f821c109830 2024-09-11T23:44:40.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.364+0000 7f82127fc640 1 -- 172.21.15.133:0/304050072 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c018990 con 0x7f821c109830 2024-09-11T23:44:40.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.367+0000 7f8223383640 1 -- 172.21.15.133:0/304050072 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821c1ab170 con 0x7f821c109830 2024-09-11T23:44:40.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.367+0000 7f8223383640 1 -- 172.21.15.133:0/304050072 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821c1ab570 con 0x7f821c109830 2024-09-11T23:44:40.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.376+0000 7f822575c740 1 Processor -- start 2024-09-11T23:44:40.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.379+0000 7f822575c740 1 -- start start 2024-09-11T23:44:40.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.379+0000 7f822575c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c919b40 0x55d41c918090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.379+0000 7f822575c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c9185d0 0x55d41c918a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.379+0000 7f822575c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c923370 0x55d41c925760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.379+0000 7f822575c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d41c87e2f0 con 0x55d41c9185d0 2024-09-11T23:44:40.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.379+0000 7f822575c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d41c825cf0 con 0x55d41c923370 2024-09-11T23:44:40.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.379+0000 7f822575c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d41c87a2b0 con 0x55d41c919b40 2024-09-11T23:44:40.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.380+0000 7f82127fc640 1 -- 172.21.15.133:0/304050072 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f820c010ac0 con 0x7f821c109830 2024-09-11T23:44:40.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.381+0000 7f822111f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c9185d0 0x55d41c918a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.381+0000 7f822111f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c9185d0 0x55d41c918a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43808/0 (socket says 172.21.15.133:43808) 2024-09-11T23:44:40.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.381+0000 7f822111f640 1 -- 172.21.15.133:0/2889234021 learned_addr learned my addr 172.21.15.133:0/2889234021 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.381+0000 7f82127fc640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82080a2f00 0x7f82080a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.381+0000 7f82127fc640 1 -- 172.21.15.133:0/304050072 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f820c0cf920 con 0x7f821c109830 2024-09-11T23:44:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.385+0000 7f822091e640 1 --2- 172.21.15.133:0/2889234021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c923370 0x55d41c925760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.386+0000 7f822111f640 1 -- 172.21.15.133:0/2889234021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c919b40 msgr2=0x55d41c918090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:44:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.386+0000 7f822111f640 1 --2- 172.21.15.133:0/2889234021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c919b40 0x55d41c918090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.386+0000 7f822111f640 1 -- 172.21.15.133:0/2889234021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c923370 msgr2=0x55d41c925760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.386+0000 7f822111f640 1 --2- 172.21.15.133:0/2889234021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c923370 0x55d41c925760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.386+0000 7f822111f640 1 -- 172.21.15.133:0/2889234021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d41c80a3a0 con 0x55d41c9185d0 2024-09-11T23:44:40.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.393+0000 7f822111f640 1 --2- 172.21.15.133:0/2889234021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c9185d0 0x55d41c918a10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8218009770 tx=0x7f8218009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d6e918676abbf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:40.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.402+0000 7f82077fe640 1 -- 172.21.15.133:0/2889234021 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218015020 con 0x55d41c9185d0 2024-09-11T23:44:40.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.402+0000 7f82077fe640 1 -- 172.21.15.133:0/2889234021 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8218011420 con 0x55d41c9185d0 2024-09-11T23:44:40.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.402+0000 7f82077fe640 1 -- 172.21.15.133:0/2889234021 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180105e0 con 0x55d41c9185d0 2024-09-11T23:44:40.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.404+0000 7f822575c740 1 -- 172.21.15.133:0/2889234021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c9185d0 msgr2=0x55d41c918a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:40.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.404+0000 7f822575c740 1 --2- 172.21.15.133:0/2889234021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c9185d0 0x55d41c918a10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8218009770 tx=0x7f8218009c30 comp rx=0 tx=0).stop 2024-09-11T23:44:40.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.404+0000 7f822575c740 1 -- 172.21.15.133:0/2889234021 shutdown_connections 2024-09-11T23:44:40.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.404+0000 7f822575c740 1 --2- 172.21.15.133:0/2889234021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c923370 0x55d41c925760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.404+0000 7f822575c740 1 --2- 172.21.15.133:0/2889234021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c9185d0 0x55d41c918a10 secure :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8218009770 tx=0x7f8218009c30 comp rx=0 tx=0).stop 2024-09-11T23:44:40.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.404+0000 7f822575c740 1 --2- 172.21.15.133:0/2889234021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c919b40 0x55d41c918090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.404+0000 7f822575c740 1 -- 172.21.15.133:0/2889234021 >> 172.21.15.133:0/2889234021 conn(0x55d41c8e1310 msgr2=0x55d41c914c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:40.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.406+0000 7f822575c740 1 -- 172.21.15.133:0/2889234021 shutdown_connections 2024-09-11T23:44:40.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.409+0000 7f822575c740 1 -- 172.21.15.133:0/2889234021 wait complete. 2024-09-11T23:44:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.409+0000 7f822575c740 1 -- start start 2024-09-11T23:44:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.409+0000 7f822575c740 1 Processor -- start 2024-09-11T23:44:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.410+0000 7f822575c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c919b40 0x55d41c9202e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.410+0000 7f822575c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c923370 0x55d41c920820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.410+0000 7f822575c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c920d60 0x55d41c92f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.410+0000 7f822575c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d41c926070 con 0x55d41c923370 2024-09-11T23:44:40.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.410+0000 7f822575c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d41c839060 con 0x55d41c920d60 2024-09-11T23:44:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.410+0000 7f822575c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d41c9261f0 con 0x55d41c919b40 2024-09-11T23:44:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.410+0000 7f822575c740 10 client.0 _ll_register_callbacks cb 0x55d41c58c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.411+0000 7f822111f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c919b40 0x55d41c9202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.411+0000 7f822111f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c919b40 0x55d41c9202e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35384/0 (socket says 172.21.15.133:35384) 2024-09-11T23:44:40.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.411+0000 7f822111f640 1 -- 172.21.15.133:0/3771157136 learned_addr learned my addr 172.21.15.133:0/3771157136 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:40.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.415+0000 7f8221920640 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c920d60 0x55d41c92f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.415+0000 7f8221920640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82080a2f00 0x7f82080a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.419+0000 7f822111f640 1 -- 172.21.15.133:0/3771157136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c920d60 msgr2=0x55d41c92f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.419+0000 7f822111f640 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c920d60 0x55d41c92f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.419+0000 7f822111f640 1 -- 172.21.15.133:0/3771157136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c923370 msgr2=0x55d41c920820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.419+0000 7f822111f640 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c923370 0x55d41c920820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.419+0000 7f822111f640 1 -- 172.21.15.133:0/3771157136 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d41c5ef3c0 con 0x55d41c919b40 2024-09-11T23:44:40.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.428+0000 7f822111f640 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c919b40 0x55d41c9202e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8218011e20 tx=0x7f8218009fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9a01d1f174655d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:40.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.428+0000 7f8204ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:44:40.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.429+0000 7f8221920640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82080a2f00 0x7f82080a53c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8214005e30 tx=0x7f8214005d80 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:40.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.432+0000 7f8204ff9640 1 -- 172.21.15.133:0/3771157136 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218015050 con 0x55d41c919b40 2024-09-11T23:44:40.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.432+0000 7f822575c740 1 -- 172.21.15.133:0/3771157136 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8218011420 con 0x55d41c919b40 2024-09-11T23:44:40.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.432+0000 7f8204ff9640 1 -- 172.21.15.133:0/3771157136 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f821800b8d0 con 0x55d41c919b40 2024-09-11T23:44:40.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.438+0000 7f8204ff9640 1 -- 172.21.15.133:0/3771157136 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f821800bad0 con 0x55d41c919b40 2024-09-11T23:44:40.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.438+0000 7f822575c740 10 client.26821 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.438+0000 7f822575c740 1 -- 172.21.15.133:0/3771157136 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d41c9541c0 con 0x55d41c919b40 2024-09-11T23:44:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.440+0000 7f8204ff9640 1 -- 172.21.15.133:0/3771157136 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8218011420 con 0x55d41c919b40 2024-09-11T23:44:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.440+0000 7f8204ff9640 1 -- 172.21.15.133:0/3771157136 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f82180039f0 con 0x55d41c919b40 2024-09-11T23:44:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.441+0000 7f822575c740 10 client.26821 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.441+0000 7f822575c740 10 client.26821 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:44:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.441+0000 7f822575c740 20 client.26821 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.441+0000 7f822575c740 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55d41c9558f0 0x55d41c957d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.441+0000 7f822575c740 4 client.26821 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.441+0000 7f822575c740 1 -- 172.21.15.133:0/3771157136 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d41c88b860 con 0x55d41c9558f0 2024-09-11T23:44:40.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.444+0000 7f822091e640 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55d41c9558f0 0x55d41c957d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.445+0000 7f8204ff9640 10 client.26821 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:44:40.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.445+0000 7f822091e640 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55d41c9558f0 0x55d41c957d60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9f2ddf5c7cc7b29 server_cookie=253ec9b8656020b0 in_seq=0 out_seq=0 2024-09-11T23:44:40.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.446+0000 7f8204ff9640 1 -- 172.21.15.133:0/3771157136 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d41c88b860 con 0x55d41c9558f0 2024-09-11T23:44:40.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.447+0000 7f8204ff9640 10 client.26821 handle_command_reply: tid=0 2024-09-11T23:44:40.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.447+0000 7f822575c740 1 client.26821 shutdown 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.452+0000 7f822575c740 1 -- 172.21.15.133:0/3771157136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c919b40 msgr2=0x55d41c9202e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.452+0000 7f822575c740 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c919b40 0x55d41c9202e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8218011e20 tx=0x7f8218009fc0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.452+0000 7f822575c740 1 -- 172.21.15.133:0/3771157136 shutdown_connections 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.452+0000 7f822575c740 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55d41c9558f0 0x55d41c957d60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.452+0000 7f822575c740 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c920d60 0x55d41c92f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.452+0000 7f822575c740 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c923370 0x55d41c920820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.452+0000 7f822575c740 1 --2- 172.21.15.133:0/3771157136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c919b40 0x55d41c9202e0 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8218011e20 tx=0x7f8218009fc0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.452+0000 7f822575c740 1 -- 172.21.15.133:0/3771157136 >> 172.21.15.133:0/3771157136 conn(0x55d41c8e1310 msgr2=0x55d41c91a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.452+0000 7f8204ff9640 0 client.26821 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.452+0000 7f822575c740 1 -- 172.21.15.133:0/3771157136 shutdown_connections 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.452+0000 7f822575c740 1 -- 172.21.15.133:0/3771157136 wait complete. 2024-09-11T23:44:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.453+0000 7f822575c740 20 client.26821 trim_cache size 0 max 16384 2024-09-11T23:44:40.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.497+0000 7f822575c740 1 Processor -- start 2024-09-11T23:44:40.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.497+0000 7f822575c740 1 -- start start 2024-09-11T23:44:40.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.498+0000 7f822575c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c8e1310 0x55d41c9202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.498+0000 7f822575c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c920d60 0x55d41c92f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.498+0000 7f822575c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c923370 0x55d41c91f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.498+0000 7f822575c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b3fc0 con 0x55d41c923370 2024-09-11T23:44:40.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.498+0000 7f822575c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b5b00 con 0x55d41c8e1310 2024-09-11T23:44:40.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.499+0000 7f822575c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b4690 con 0x55d41c920d60 2024-09-11T23:44:40.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.499+0000 7f822111f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c8e1310 0x55d41c9202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.499+0000 7f822111f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c8e1310 0x55d41c9202a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:47374/0 (socket says 172.21.15.133:47374) 2024-09-11T23:44:40.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.499+0000 7f822111f640 1 -- 172.21.15.133:0/2757729984 learned_addr learned my addr 172.21.15.133:0/2757729984 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:40.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.499+0000 7f822091e640 1 --2- 172.21.15.133:0/2757729984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c920d60 0x55d41c92f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.501+0000 7f8221920640 1 --2- 172.21.15.133:0/2757729984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c923370 0x55d41c91f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.502+0000 7f8221920640 1 -- 172.21.15.133:0/2757729984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c920d60 msgr2=0x55d41c92f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.502+0000 7f8221920640 1 --2- 172.21.15.133:0/2757729984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c920d60 0x55d41c92f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.502+0000 7f8221920640 1 -- 172.21.15.133:0/2757729984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c8e1310 msgr2=0x55d41c9202a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.502+0000 7f8221920640 1 --2- 172.21.15.133:0/2757729984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c8e1310 0x55d41c9202a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.502+0000 7f8221920640 1 -- 172.21.15.133:0/2757729984 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d41c915960 con 0x55d41c923370 2024-09-11T23:44:40.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.504+0000 7f8221920640 1 --2- 172.21.15.133:0/2757729984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c923370 0x55d41c91f2c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f821401ac00 tx=0x7f8214009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3bde6d9821a6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:40.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.505+0000 7f8206ffd640 1 -- 172.21.15.133:0/2757729984 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821404e070 con 0x55d41c923370 2024-09-11T23:44:40.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.505+0000 7f8206ffd640 1 -- 172.21.15.133:0/2757729984 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8214041c10 con 0x55d41c923370 2024-09-11T23:44:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.508+0000 7f822575c740 1 -- 172.21.15.133:0/2757729984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c923370 msgr2=0x55d41c91f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.508+0000 7f822575c740 1 --2- 172.21.15.133:0/2757729984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c923370 0x55d41c91f2c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f821401ac00 tx=0x7f8214009000 comp rx=0 tx=0).stop 2024-09-11T23:44:40.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.514+0000 7f822575c740 1 -- 172.21.15.133:0/2757729984 shutdown_connections 2024-09-11T23:44:40.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.514+0000 7f822575c740 1 --2- 172.21.15.133:0/2757729984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c923370 0x55d41c91f2c0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.514+0000 7f822575c740 1 --2- 172.21.15.133:0/2757729984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c920d60 0x55d41c92f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.514+0000 7f822575c740 1 --2- 172.21.15.133:0/2757729984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c8e1310 0x55d41c9202a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.514+0000 7f822575c740 1 -- 172.21.15.133:0/2757729984 >> 172.21.15.133:0/2757729984 conn(0x55d41c920980 msgr2=0x55d41c91ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:40.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.525+0000 7f822575c740 1 -- 172.21.15.133:0/2757729984 shutdown_connections 2024-09-11T23:44:40.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.527+0000 7f822575c740 1 -- 172.21.15.133:0/2757729984 wait complete. 2024-09-11T23:44:40.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.527+0000 7f822575c740 1 -- start start 2024-09-11T23:44:40.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.527+0000 7f822575c740 1 Processor -- start 2024-09-11T23:44:40.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.528+0000 7f822575c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c8e1310 0x55d41c923750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.528+0000 7f822575c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c920d60 0x55d41c923c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.528+0000 7f822575c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c923370 0x55d41c9241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.528+0000 7f822575c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b5430 con 0x55d41c8e1310 2024-09-11T23:44:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.528+0000 7f822575c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b4d60 con 0x55d41c920d60 2024-09-11T23:44:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.528+0000 7f822575c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b1fc0 con 0x55d41c923370 2024-09-11T23:44:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.528+0000 7f822575c740 10 client.0 _ll_register_callbacks cb 0x7f82180095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:40.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.530+0000 7f8221920640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c923370 0x55d41c9241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.532+0000 7f8221920640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c923370 0x55d41c9241d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35392/0 (socket says 172.21.15.133:35392) 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.532+0000 7f8221920640 1 -- 172.21.15.133:0/83432225 learned_addr learned my addr 172.21.15.133:0/83432225 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.532+0000 7f822111f640 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c8e1310 0x55d41c923750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.532+0000 7f822111f640 1 -- 172.21.15.133:0/83432225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c923370 msgr2=0x55d41c9241d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.532+0000 7f822111f640 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c923370 0x55d41c9241d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.532+0000 7f822111f640 1 -- 172.21.15.133:0/83432225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c920d60 msgr2=0x55d41c923c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.532+0000 7f822111f640 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c920d60 0x55d41c923c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.532+0000 7f822111f640 1 -- 172.21.15.133:0/83432225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d41c92a6f0 con 0x55d41c8e1310 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.533+0000 7f822111f640 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c8e1310 0x55d41c923750 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8218003df0 tx=0x7f8218009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcef531054c59fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.533+0000 7f8204ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.539+0000 7f8204ff9640 1 -- 172.21.15.133:0/83432225 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218032070 con 0x55d41c8e1310 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.540+0000 7f8204ff9640 1 -- 172.21.15.133:0/83432225 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f821800b8d0 con 0x55d41c8e1310 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.540+0000 7f822575c740 1 -- 172.21.15.133:0/83432225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8214041c10 con 0x55d41c8e1310 2024-09-11T23:44:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.544+0000 7f8204ff9640 1 -- 172.21.15.133:0/83432225 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8218015040 con 0x55d41c8e1310 2024-09-11T23:44:40.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.545+0000 7f822575c740 10 client.17469 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:40.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.545+0000 7f822575c740 1 -- 172.21.15.133:0/83432225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d41c92b9f0 con 0x55d41c8e1310 2024-09-11T23:44:40.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.551+0000 7f8204ff9640 1 -- 172.21.15.133:0/83432225 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8214041c10 con 0x55d41c8e1310 2024-09-11T23:44:40.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.552+0000 7f8204ff9640 1 -- 172.21.15.133:0/83432225 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f82180038f0 con 0x55d41c8e1310 2024-09-11T23:44:40.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.552+0000 7f822575c740 10 client.17469 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:40.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.552+0000 7f822575c740 10 client.17469 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:44:40.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.552+0000 7f822575c740 20 client.17469 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:40.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.552+0000 7f822575c740 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55d41c9c05d0 0x55d41c9c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:40.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.552+0000 7f822575c740 4 client.17469 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:44:40.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.552+0000 7f822575c740 1 -- 172.21.15.133:0/83432225 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d41c885d70 con 0x55d41c9c05d0 2024-09-11T23:44:40.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.560+0000 7f822091e640 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55d41c9c05d0 0x55d41c9c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:40.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.567+0000 7f822091e640 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55d41c9c05d0 0x55d41c9c29c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee5ff68882e8e39b server_cookie=e05681841d4bb468 in_seq=0 out_seq=0 2024-09-11T23:44:40.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.567+0000 7f8204ff9640 10 client.17469 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:44:40.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.567+0000 7f8204ff9640 1 -- 172.21.15.133:0/83432225 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d41c885d70 con 0x55d41c9c05d0 2024-09-11T23:44:40.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.567+0000 7f8204ff9640 10 client.17469 handle_command_reply: tid=0 2024-09-11T23:44:40.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.568+0000 7f822575c740 1 client.17469 shutdown 2024-09-11T23:44:40.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.568+0000 7f822575c740 1 -- 172.21.15.133:0/83432225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c8e1310 msgr2=0x55d41c923750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:40.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.568+0000 7f822575c740 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c8e1310 0x55d41c923750 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8218003df0 tx=0x7f8218009720 comp rx=0 tx=0).stop 2024-09-11T23:44:40.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.573+0000 7f822575c740 1 -- 172.21.15.133:0/83432225 shutdown_connections 2024-09-11T23:44:40.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.573+0000 7f822575c740 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55d41c9c05d0 0x55d41c9c29c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.573+0000 7f822575c740 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d41c923370 0x55d41c9241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.573+0000 7f822575c740 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d41c920d60 0x55d41c923c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.573+0000 7f822575c740 1 --2- 172.21.15.133:0/83432225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41c8e1310 0x55d41c923750 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.573+0000 7f822575c740 1 -- 172.21.15.133:0/83432225 >> 172.21.15.133:0/83432225 conn(0x55d41c920980 msgr2=0x55d41c91ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:40.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.573+0000 7f8204ff9640 0 client.17469 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:44:40.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.577+0000 7f822575c740 1 -- 172.21.15.133:0/83432225 shutdown_connections 2024-09-11T23:44:40.582 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:44:40.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.582+0000 7f822575c740 1 -- 172.21.15.133:0/83432225 wait complete. 2024-09-11T23:44:40.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.582+0000 7f822575c740 20 client.17469 trim_cache size 0 max 16384 2024-09-11T23:44:40.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 -- 172.21.15.133:0/304050072 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82080a2f00 msgr2=0x7f82080a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:40.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82080a2f00 0x7f82080a53c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8214005e30 tx=0x7f8214005d80 comp rx=0 tx=0).stop 2024-09-11T23:44:40.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 -- 172.21.15.133:0/304050072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f821c109830 msgr2=0x7f821c1168f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:40.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f821c109830 0x7f821c1168f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f820c00e9f0 tx=0x7f820c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 -- 172.21.15.133:0/304050072 shutdown_connections 2024-09-11T23:44:40.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82080a2f00 0x7f82080a53c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c11b140 0x7f821c116e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f821c109830 0x7f821c1168f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 --2- 172.21.15.133:0/304050072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c109450 0x7f821c11a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:40.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 -- 172.21.15.133:0/304050072 >> 172.21.15.133:0/304050072 conn(0x7f821c06e140 msgr2=0x7f821c070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:40.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 -- 172.21.15.133:0/304050072 shutdown_connections 2024-09-11T23:44:40.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:40.583+0000 7f8223383640 1 -- 172.21.15.133:0/304050072 wait complete. 2024-09-11T23:44:40.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:40 smithi133 ceph-mon[31430]: pgmap v244: 97 pgs: 97 active+clean; 69 GiB data, 232 GiB used, 841 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 805 MiB/s wr, 1.93k op/s 2024-09-11T23:44:41.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:40 smithi138 ceph-mon[35142]: pgmap v244: 97 pgs: 97 active+clean; 69 GiB data, 232 GiB used, 841 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 805 MiB/s wr, 1.93k op/s 2024-09-11T23:44:41.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:40 smithi178 ceph-mon[35882]: pgmap v244: 97 pgs: 97 active+clean; 69 GiB data, 232 GiB used, 841 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 805 MiB/s wr, 1.93k op/s 2024-09-11T23:44:41.653 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:44:42.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:41 smithi138 ceph-mon[35142]: pgmap v245: 97 pgs: 97 active+clean; 69 GiB data, 234 GiB used, 839 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 658 MiB/s wr, 1.63k op/s 2024-09-11T23:44:42.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:41 smithi133 ceph-mon[31430]: pgmap v245: 97 pgs: 97 active+clean; 69 GiB data, 234 GiB used, 839 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 658 MiB/s wr, 1.63k op/s 2024-09-11T23:44:42.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:41 smithi178 ceph-mon[35882]: pgmap v245: 97 pgs: 97 active+clean; 69 GiB data, 234 GiB used, 839 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 658 MiB/s wr, 1.63k op/s 2024-09-11T23:44:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:43.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.602+0000 7fae34117640 1 -- 172.21.15.133:0/4160490503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae2c109450 msgr2=0x7fae2c110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:43.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.602+0000 7fae34117640 1 --2- 172.21.15.133:0/4160490503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae2c109450 0x7fae2c110250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae28009860 tx=0x7fae2802a380 comp rx=0 tx=0).stop 2024-09-11T23:44:43.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.602+0000 7fae34117640 1 -- 172.21.15.133:0/4160490503 shutdown_connections 2024-09-11T23:44:43.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.602+0000 7fae34117640 1 --2- 172.21.15.133:0/4160490503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c11b140 0x7fae2c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.602+0000 7fae34117640 1 --2- 172.21.15.133:0/4160490503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae2c109830 0x7fae2c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.602+0000 7fae34117640 1 --2- 172.21.15.133:0/4160490503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae2c109450 0x7fae2c110250 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae28009860 tx=0x7fae2802a380 comp rx=0 tx=0).stop 2024-09-11T23:44:43.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.602+0000 7fae34117640 1 -- 172.21.15.133:0/4160490503 >> 172.21.15.133:0/4160490503 conn(0x7fae2c06e140 msgr2=0x7fae2c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:43.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.607+0000 7fae34117640 1 -- 172.21.15.133:0/4160490503 shutdown_connections 2024-09-11T23:44:43.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.612+0000 7fae34117640 1 -- 172.21.15.133:0/4160490503 wait complete. 2024-09-11T23:44:43.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.613+0000 7fae34117640 1 Processor -- start 2024-09-11T23:44:43.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.613+0000 7fae34117640 1 -- start start 2024-09-11T23:44:43.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.614+0000 7fae34117640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae2c109830 0x7fae2c1a7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.614+0000 7fae34117640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c11b140 0x7fae2c1a79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.614+0000 7fae34117640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae2c1abd50 0x7fae2c1ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.614+0000 7fae34117640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c120df0 con 0x7fae2c11b140 2024-09-11T23:44:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.614+0000 7fae34117640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c120c70 con 0x7fae2c1abd50 2024-09-11T23:44:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.615+0000 7fae34117640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c120f70 con 0x7fae2c109830 2024-09-11T23:44:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.615+0000 7fae316b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c11b140 0x7fae2c1a79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.615+0000 7fae31eb3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae2c109830 0x7fae2c1a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.615+0000 7fae31eb3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae2c109830 0x7fae2c1a7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42546/0 (socket says 172.21.15.133:42546) 2024-09-11T23:44:43.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.615+0000 7fae31eb3640 1 -- 172.21.15.133:0/3644349141 learned_addr learned my addr 172.21.15.133:0/3644349141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:43.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.615+0000 7fae326b4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae2c1abd50 0x7fae2c1ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.615+0000 7fae316b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c11b140 0x7fae2c1a79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44128/0 (socket says 172.21.15.133:44128) 2024-09-11T23:44:43.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.618+0000 7fae31eb3640 1 -- 172.21.15.133:0/3644349141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae2c1abd50 msgr2=0x7fae2c1ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:43.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.618+0000 7fae31eb3640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae2c1abd50 0x7fae2c1ac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.618+0000 7fae31eb3640 1 -- 172.21.15.133:0/3644349141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c11b140 msgr2=0x7fae2c1a79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:43.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.618+0000 7fae31eb3640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c11b140 0x7fae2c1a79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.618+0000 7fae31eb3640 1 -- 172.21.15.133:0/3644349141 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2c1ac810 con 0x7fae2c109830 2024-09-11T23:44:43.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.618+0000 7fae31eb3640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae2c109830 0x7fae2c1a7480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fae28009460 tx=0x7fae28033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:43.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.619+0000 7fae22ffd640 1 -- 172.21.15.133:0/3644349141 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28037040 con 0x7fae2c109830 2024-09-11T23:44:43.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.619+0000 7fae22ffd640 1 -- 172.21.15.133:0/3644349141 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae28004240 con 0x7fae2c109830 2024-09-11T23:44:43.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.619+0000 7fae22ffd640 1 -- 172.21.15.133:0/3644349141 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28005520 con 0x7fae2c109830 2024-09-11T23:44:43.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.622+0000 7fae34117640 1 -- 172.21.15.133:0/3644349141 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2c1acaa0 con 0x7fae2c109830 2024-09-11T23:44:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.623+0000 7fae34117640 1 -- 172.21.15.133:0/3644349141 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2c1b41d0 con 0x7fae2c109830 2024-09-11T23:44:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.625+0000 7fae22ffd640 1 -- 172.21.15.133:0/3644349141 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fae28042060 con 0x7fae2c109830 2024-09-11T23:44:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.626+0000 7fae22ffd640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae180a2f00 0x7fae180a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.627+0000 7fae22ffd640 1 -- 172.21.15.133:0/3644349141 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fae280ee3e0 con 0x7fae2c109830 2024-09-11T23:44:43.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.632+0000 7fae316b2640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae180a2f00 0x7fae180a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.633+0000 7fae316b2640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae180a2f00 0x7fae180a53c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fae2c11d920 tx=0x7fae1c009260 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:43.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.635+0000 7fae364f0740 1 Processor -- start 2024-09-11T23:44:43.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.635+0000 7fae364f0740 1 -- start start 2024-09-11T23:44:43.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.635+0000 7fae364f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca44bb40 0x55bcca44a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.635+0000 7fae364f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca44aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.635+0000 7fae364f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca455370 0x55bcca457760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.635+0000 7fae364f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bcca3b02f0 con 0x55bcca44a5d0 2024-09-11T23:44:43.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.635+0000 7fae364f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bcca357cf0 con 0x55bcca455370 2024-09-11T23:44:43.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.635+0000 7fae364f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bcca3ac2b0 con 0x55bcca44bb40 2024-09-11T23:44:43.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.636+0000 7fae31eb3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca44bb40 0x55bcca44a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.636+0000 7fae31eb3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca44bb40 0x55bcca44a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42560/0 (socket says 172.21.15.133:42560) 2024-09-11T23:44:43.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.636+0000 7fae31eb3640 1 -- 172.21.15.133:0/1405707164 learned_addr learned my addr 172.21.15.133:0/1405707164 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:43.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.636+0000 7fae326b4640 1 --2- 172.21.15.133:0/1405707164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca455370 0x55bcca457760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.636+0000 7fae31eb3640 1 -- 172.21.15.133:0/1405707164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca455370 msgr2=0x55bcca457760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:43.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.636+0000 7fae31eb3640 1 --2- 172.21.15.133:0/1405707164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca455370 0x55bcca457760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.636+0000 7fae31eb3640 1 -- 172.21.15.133:0/1405707164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 msgr2=0x55bcca44aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:43.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.636+0000 7fae31eb3640 1 --2- 172.21.15.133:0/1405707164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca44aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.636+0000 7fae31eb3640 1 -- 172.21.15.133:0/1405707164 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcca33c3a0 con 0x55bcca44bb40 2024-09-11T23:44:43.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.637+0000 7fae31eb3640 1 --2- 172.21.15.133:0/1405707164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca44bb40 0x55bcca44a090 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fae280c3130 tx=0x7fae2802a380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=376f2d0c6a312746 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:43.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.641+0000 7fae17fff640 1 -- 172.21.15.133:0/1405707164 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae280ba350 con 0x55bcca44bb40 2024-09-11T23:44:43.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.641+0000 7fae17fff640 1 -- 172.21.15.133:0/1405707164 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae280043f0 con 0x55bcca44bb40 2024-09-11T23:44:43.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.641+0000 7fae17fff640 1 -- 172.21.15.133:0/1405707164 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae280b7060 con 0x55bcca44bb40 2024-09-11T23:44:43.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.642+0000 7fae364f0740 1 -- 172.21.15.133:0/1405707164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca44bb40 msgr2=0x55bcca44a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:43.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.642+0000 7fae364f0740 1 --2- 172.21.15.133:0/1405707164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca44bb40 0x55bcca44a090 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fae280c3130 tx=0x7fae2802a380 comp rx=0 tx=0).stop 2024-09-11T23:44:43.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.643+0000 7fae364f0740 1 -- 172.21.15.133:0/1405707164 shutdown_connections 2024-09-11T23:44:43.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.643+0000 7fae364f0740 1 --2- 172.21.15.133:0/1405707164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca455370 0x55bcca457760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.643+0000 7fae364f0740 1 --2- 172.21.15.133:0/1405707164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca44aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.643+0000 7fae364f0740 1 --2- 172.21.15.133:0/1405707164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca44bb40 0x55bcca44a090 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.643+0000 7fae364f0740 1 -- 172.21.15.133:0/1405707164 >> 172.21.15.133:0/1405707164 conn(0x55bcca413310 msgr2=0x55bcca446c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:43.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.647+0000 7fae364f0740 1 -- 172.21.15.133:0/1405707164 shutdown_connections 2024-09-11T23:44:43.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.647+0000 7fae364f0740 1 -- 172.21.15.133:0/1405707164 wait complete. 2024-09-11T23:44:43.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.648+0000 7fae364f0740 1 -- start start 2024-09-11T23:44:43.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.648+0000 7fae364f0740 1 Processor -- start 2024-09-11T23:44:43.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.649+0000 7fae364f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca452140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.649+0000 7fae364f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca452680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.649+0000 7fae364f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca452bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.649+0000 7fae364f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bcca457e30 con 0x55bcca44a5d0 2024-09-11T23:44:43.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.649+0000 7fae364f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bcca36b060 con 0x55bcca44bb40 2024-09-11T23:44:43.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.649+0000 7fae364f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bcca457fb0 con 0x55bcca455370 2024-09-11T23:44:43.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.649+0000 7fae364f0740 10 client.0 _ll_register_callbacks cb 0x55bcca0be8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:43.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.650+0000 7fae326b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca452bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.650+0000 7fae326b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca452bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42562/0 (socket says 172.21.15.133:42562) 2024-09-11T23:44:43.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.650+0000 7fae326b4640 1 -- 172.21.15.133:0/2530104404 learned_addr learned my addr 172.21.15.133:0/2530104404 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:43.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.650+0000 7fae316b2640 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca452680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.650+0000 7fae326b4640 1 -- 172.21.15.133:0/2530104404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 msgr2=0x55bcca452680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.651+0000 7fae326b4640 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca452680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.651+0000 7fae326b4640 1 -- 172.21.15.133:0/2530104404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 msgr2=0x55bcca452140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.651+0000 7fae326b4640 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca452140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.651+0000 7fae326b4640 1 -- 172.21.15.133:0/2530104404 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcca1213c0 con 0x55bcca455370 2024-09-11T23:44:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.651+0000 7fae326b4640 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca452bc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae240129f0 tx=0x7fae24012eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db51aa4a2fe3df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.651+0000 7fae157fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:44:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.651+0000 7fae157fa640 1 -- 172.21.15.133:0/2530104404 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2400f9c0 con 0x55bcca455370 2024-09-11T23:44:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.652+0000 7fae364f0740 1 -- 172.21.15.133:0/2530104404 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bcca3bd860 con 0x55bcca455370 2024-09-11T23:44:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.652+0000 7fae157fa640 1 -- 172.21.15.133:0/2530104404 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fae2400fb60 con 0x55bcca455370 2024-09-11T23:44:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.652+0000 7fae157fa640 1 -- 172.21.15.133:0/2530104404 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae2400fd60 con 0x55bcca455370 2024-09-11T23:44:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.652+0000 7fae364f0740 10 client.26839 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.652+0000 7fae364f0740 1 -- 172.21.15.133:0/2530104404 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bcca463950 con 0x55bcca455370 2024-09-11T23:44:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.653+0000 7fae157fa640 1 -- 172.21.15.133:0/2530104404 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bcca3bd860 con 0x55bcca455370 2024-09-11T23:44:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.655+0000 7fae157fa640 1 -- 172.21.15.133:0/2530104404 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fae24004650 con 0x55bcca455370 2024-09-11T23:44:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.655+0000 7fae364f0740 10 client.26839 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.655+0000 7fae364f0740 10 client.26839 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:44:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.655+0000 7fae364f0740 20 client.26839 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.655+0000 7fae364f0740 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55bcca465250 0x55bcca4676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.655+0000 7fae364f0740 4 client.26839 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.655+0000 7fae364f0740 1 -- 172.21.15.133:0/2530104404 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcca3b7d70 con 0x55bcca465250 2024-09-11T23:44:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.656+0000 7fae31eb3640 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55bcca465250 0x55bcca4676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.657+0000 7fae31eb3640 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55bcca465250 0x55bcca4676d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de0bb002aa75de1f server_cookie=781e9102dd0c3753 in_seq=0 out_seq=0 2024-09-11T23:44:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.657+0000 7fae157fa640 10 client.26839 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:44:43.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.658+0000 7fae157fa640 1 -- 172.21.15.133:0/2530104404 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcca3b7d70 con 0x55bcca465250 2024-09-11T23:44:43.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.658+0000 7fae157fa640 10 client.26839 handle_command_reply: tid=0 2024-09-11T23:44:43.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.658+0000 7fae364f0740 1 client.26839 shutdown 2024-09-11T23:44:43.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.658+0000 7fae364f0740 1 -- 172.21.15.133:0/2530104404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 msgr2=0x55bcca452bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:43.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.658+0000 7fae364f0740 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca452bc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae240129f0 tx=0x7fae24012eb0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.665+0000 7fae364f0740 1 -- 172.21.15.133:0/2530104404 shutdown_connections 2024-09-11T23:44:43.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.665+0000 7fae364f0740 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55bcca465250 0x55bcca4676d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.665+0000 7fae364f0740 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca452bc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.665+0000 7fae364f0740 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca452680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.665+0000 7fae364f0740 1 --2- 172.21.15.133:0/2530104404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca452140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.665+0000 7fae364f0740 1 -- 172.21.15.133:0/2530104404 >> 172.21.15.133:0/2530104404 conn(0x55bcca413310 msgr2=0x55bcca44c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:43.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.665+0000 7fae157fa640 0 client.26839 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:44:43.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.667+0000 7fae364f0740 1 -- 172.21.15.133:0/2530104404 shutdown_connections 2024-09-11T23:44:43.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.668+0000 7fae364f0740 1 -- 172.21.15.133:0/2530104404 wait complete. 2024-09-11T23:44:43.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.668+0000 7fae364f0740 20 client.26839 trim_cache size 0 max 16384 2024-09-11T23:44:43.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.698+0000 7fae364f0740 1 Processor -- start 2024-09-11T23:44:43.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.700+0000 7fae364f0740 1 -- start start 2024-09-11T23:44:43.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.701+0000 7fae364f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca4540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.701+0000 7fae364f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca44d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.701+0000 7fae364f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca44d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.701+0000 7fae364f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1bcb10 con 0x55bcca44a5d0 2024-09-11T23:44:43.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.701+0000 7fae364f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1be650 con 0x55bcca44bb40 2024-09-11T23:44:43.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.701+0000 7fae364f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1bd1e0 con 0x55bcca455370 2024-09-11T23:44:43.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.701+0000 7fae316b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca44d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.701+0000 7fae316b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca44d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38554/0 (socket says 172.21.15.133:38554) 2024-09-11T23:44:43.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.701+0000 7fae316b2640 1 -- 172.21.15.133:0/764011406 learned_addr learned my addr 172.21.15.133:0/764011406 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:43.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.702+0000 7fae326b4640 1 --2- 172.21.15.133:0/764011406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca44d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.702+0000 7fae31eb3640 1 --2- 172.21.15.133:0/764011406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca4540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.702+0000 7fae326b4640 1 -- 172.21.15.133:0/764011406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 msgr2=0x55bcca44d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:43.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.702+0000 7fae326b4640 1 --2- 172.21.15.133:0/764011406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca44d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.702+0000 7fae326b4640 1 -- 172.21.15.133:0/764011406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 msgr2=0x55bcca4540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:43.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.702+0000 7fae326b4640 1 --2- 172.21.15.133:0/764011406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca4540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.702+0000 7fae326b4640 1 -- 172.21.15.133:0/764011406 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcca44b740 con 0x55bcca455370 2024-09-11T23:44:43.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.703+0000 7fae326b4640 1 --2- 172.21.15.133:0/764011406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca44d660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae24010430 tx=0x7fae2400ff00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd2717c14e78455c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:43.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.706+0000 7fae177fe640 1 -- 172.21.15.133:0/764011406 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2400f9c0 con 0x55bcca455370 2024-09-11T23:44:43.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.706+0000 7fae177fe640 1 -- 172.21.15.133:0/764011406 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae2404b070 con 0x55bcca455370 2024-09-11T23:44:43.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.708+0000 7fae364f0740 1 -- 172.21.15.133:0/764011406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 msgr2=0x55bcca44d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:43.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.708+0000 7fae364f0740 1 --2- 172.21.15.133:0/764011406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca44d660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae24010430 tx=0x7fae2400ff00 comp rx=0 tx=0).stop 2024-09-11T23:44:43.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.710+0000 7fae364f0740 1 -- 172.21.15.133:0/764011406 shutdown_connections 2024-09-11T23:44:43.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.710+0000 7fae364f0740 1 --2- 172.21.15.133:0/764011406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca44d660 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.710+0000 7fae364f0740 1 --2- 172.21.15.133:0/764011406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca44d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.710+0000 7fae364f0740 1 --2- 172.21.15.133:0/764011406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca4540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.710+0000 7fae364f0740 1 -- 172.21.15.133:0/764011406 >> 172.21.15.133:0/764011406 conn(0x55bcca413310 msgr2=0x55bcca4539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:43.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.714+0000 7fae364f0740 1 -- 172.21.15.133:0/764011406 shutdown_connections 2024-09-11T23:44:43.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.718+0000 7fae364f0740 1 -- 172.21.15.133:0/764011406 wait complete. 2024-09-11T23:44:43.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.718+0000 7fae364f0740 1 -- start start 2024-09-11T23:44:43.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.719+0000 7fae364f0740 1 Processor -- start 2024-09-11T23:44:43.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.720+0000 7fae364f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca45cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.720+0000 7fae364f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca45d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.720+0000 7fae364f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca44e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.720+0000 7fae364f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1bdf80 con 0x55bcca44a5d0 2024-09-11T23:44:43.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.720+0000 7fae364f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1bd8b0 con 0x55bcca44bb40 2024-09-11T23:44:43.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.720+0000 7fae364f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c1baba0 con 0x55bcca455370 2024-09-11T23:44:43.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.720+0000 7fae364f0740 10 client.0 _ll_register_callbacks cb 0x7fae24009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:43.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.723+0000 7fae326b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca44e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.723+0000 7fae326b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca44e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42580/0 (socket says 172.21.15.133:42580) 2024-09-11T23:44:43.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.723+0000 7fae326b4640 1 -- 172.21.15.133:0/172012800 learned_addr learned my addr 172.21.15.133:0/172012800 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:43.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.727+0000 7fae326b4640 1 -- 172.21.15.133:0/172012800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 msgr2=0x55bcca45d530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:43.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.727+0000 7fae326b4640 1 --2- 172.21.15.133:0/172012800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca45d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.727+0000 7fae326b4640 1 -- 172.21.15.133:0/172012800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 msgr2=0x55bcca45cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.727+0000 7fae326b4640 1 --2- 172.21.15.133:0/172012800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca45cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.727+0000 7fae326b4640 1 -- 172.21.15.133:0/172012800 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcca44f030 con 0x55bcca455370 2024-09-11T23:44:43.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.728+0000 7fae326b4640 1 --2- 172.21.15.133:0/172012800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca44e990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fae240129c0 tx=0x7fae24010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=425c25c1d5f63919 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:43.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.731+0000 7fae157fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:44:43.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.731+0000 7fae157fa640 1 -- 172.21.15.133:0/172012800 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae240107a0 con 0x55bcca455370 2024-09-11T23:44:43.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.731+0000 7fae157fa640 1 -- 172.21.15.133:0/172012800 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fae24021020 con 0x55bcca455370 2024-09-11T23:44:43.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.731+0000 7fae364f0740 1 -- 172.21.15.133:0/172012800 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae2404b070 con 0x55bcca455370 2024-09-11T23:44:43.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.736+0000 7fae157fa640 1 -- 172.21.15.133:0/172012800 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae24050470 con 0x55bcca455370 2024-09-11T23:44:43.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.736+0000 7fae364f0740 10 client.26845 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:43.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.737+0000 7fae364f0740 1 -- 172.21.15.133:0/172012800 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bcca4e0960 con 0x55bcca455370 2024-09-11T23:44:43.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.740+0000 7fae157fa640 1 -- 172.21.15.133:0/172012800 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae2404b070 con 0x55bcca455370 2024-09-11T23:44:43.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.740+0000 7fae157fa640 1 -- 172.21.15.133:0/172012800 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fae24059020 con 0x55bcca455370 2024-09-11T23:44:43.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.741+0000 7fae364f0740 10 client.26845 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:43.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.741+0000 7fae364f0740 10 client.26845 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:44:43.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.741+0000 7fae364f0740 20 client.26845 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:43.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.741+0000 7fae364f0740 1 --2- 172.21.15.133:0/172012800 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55bcca4e0f20 0x55bcca4e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:43.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.742+0000 7fae364f0740 4 client.26845 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:44:43.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.742+0000 7fae364f0740 1 -- 172.21.15.133:0/172012800 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bcca334170 con 0x55bcca4e0f20 2024-09-11T23:44:43.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.742+0000 7fae31eb3640 1 --2- 172.21.15.133:0/172012800 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55bcca4e0f20 0x55bcca4e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:43.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.748+0000 7fae31eb3640 1 --2- 172.21.15.133:0/172012800 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55bcca4e0f20 0x55bcca4e3380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cfacfb568ca15004 server_cookie=b10a6a51016cf134 in_seq=0 out_seq=0 2024-09-11T23:44:43.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.748+0000 7fae157fa640 10 client.26845 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:44:43.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.748+0000 7fae157fa640 1 -- 172.21.15.133:0/172012800 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bcca334170 con 0x55bcca4e0f20 2024-09-11T23:44:43.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.748+0000 7fae157fa640 10 client.26845 handle_command_reply: tid=0 2024-09-11T23:44:43.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.748+0000 7fae364f0740 1 client.26845 shutdown 2024-09-11T23:44:43.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.749+0000 7fae364f0740 1 -- 172.21.15.133:0/172012800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 msgr2=0x55bcca44e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:43.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.749+0000 7fae364f0740 1 --2- 172.21.15.133:0/172012800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca44e990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fae240129c0 tx=0x7fae24010c10 comp rx=0 tx=0).stop 2024-09-11T23:44:43.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.754+0000 7fae364f0740 1 -- 172.21.15.133:0/172012800 shutdown_connections 2024-09-11T23:44:43.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.754+0000 7fae364f0740 1 --2- 172.21.15.133:0/172012800 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55bcca4e0f20 0x55bcca4e3380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.754+0000 7fae364f0740 1 --2- 172.21.15.133:0/172012800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bcca455370 0x55bcca44e990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.754+0000 7fae364f0740 1 --2- 172.21.15.133:0/172012800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcca44bb40 0x55bcca45d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.754+0000 7fae364f0740 1 --2- 172.21.15.133:0/172012800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcca44a5d0 0x55bcca45cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.754+0000 7fae364f0740 1 -- 172.21.15.133:0/172012800 >> 172.21.15.133:0/172012800 conn(0x55bcca413310 msgr2=0x55bcca4541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:43.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.754+0000 7fae157fa640 0 client.26845 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:44:43.758 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:44:43.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.756+0000 7fae364f0740 1 -- 172.21.15.133:0/172012800 shutdown_connections 2024-09-11T23:44:43.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.756+0000 7fae364f0740 1 -- 172.21.15.133:0/172012800 wait complete. 2024-09-11T23:44:43.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.756+0000 7fae364f0740 20 client.26845 trim_cache size 0 max 16384 2024-09-11T23:44:43.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.758+0000 7fae34117640 1 -- 172.21.15.133:0/3644349141 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae180a2f00 msgr2=0x7fae180a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:43.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.758+0000 7fae34117640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae180a2f00 0x7fae180a53c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fae2c11d920 tx=0x7fae1c009260 comp rx=0 tx=0).stop 2024-09-11T23:44:43.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.758+0000 7fae34117640 1 -- 172.21.15.133:0/3644349141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae2c109830 msgr2=0x7fae2c1a7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:43.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.758+0000 7fae34117640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae2c109830 0x7fae2c1a7480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fae28009460 tx=0x7fae28033d10 comp rx=0 tx=0).stop 2024-09-11T23:44:43.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.759+0000 7fae34117640 1 -- 172.21.15.133:0/3644349141 shutdown_connections 2024-09-11T23:44:43.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.759+0000 7fae34117640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae180a2f00 0x7fae180a53c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.759+0000 7fae34117640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae2c1abd50 0x7fae2c1ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.759+0000 7fae34117640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c11b140 0x7fae2c1a79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.759+0000 7fae34117640 1 --2- 172.21.15.133:0/3644349141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae2c109830 0x7fae2c1a7480 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:43.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.759+0000 7fae34117640 1 -- 172.21.15.133:0/3644349141 >> 172.21.15.133:0/3644349141 conn(0x7fae2c06e140 msgr2=0x7fae2c11c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:43.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.760+0000 7fae34117640 1 -- 172.21.15.133:0/3644349141 shutdown_connections 2024-09-11T23:44:43.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:43.760+0000 7fae34117640 1 -- 172.21.15.133:0/3644349141 wait complete. 2024-09-11T23:44:44.619 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:44 smithi133 ceph-mon[31430]: pgmap v246: 97 pgs: 97 active+clean; 69 GiB data, 236 GiB used, 837 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 762 MiB/s wr, 1.90k op/s 2024-09-11T23:44:44.724 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:44:44.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:44 smithi138 ceph-mon[35142]: pgmap v246: 97 pgs: 97 active+clean; 69 GiB data, 236 GiB used, 837 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 762 MiB/s wr, 1.90k op/s 2024-09-11T23:44:45.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:44 smithi178 ceph-mon[35882]: pgmap v246: 97 pgs: 97 active+clean; 69 GiB data, 236 GiB used, 837 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 762 MiB/s wr, 1.90k op/s 2024-09-11T23:44:45.427 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:44:45.427 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-11T23:44:45.428 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:44:45.498 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.304+0000 7f830d3d2640 1 -- 172.21.15.133:0/526359938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830806e4f0 msgr2=0x7f830806e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.304+0000 7f830d3d2640 1 --2- 172.21.15.133:0/526359938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830806e4f0 0x7f830806e930 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f83000074e0 tx=0x7f830002b0a0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.305+0000 7f830d3d2640 1 -- 172.21.15.133:0/526359938 shutdown_connections 2024-09-11T23:44:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.305+0000 7f830d3d2640 1 --2- 172.21.15.133:0/526359938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f830806ee70 0x7f830807a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.305+0000 7f830d3d2640 1 --2- 172.21.15.133:0/526359938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830806e4f0 0x7f830806e930 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.305+0000 7f830d3d2640 1 --2- 172.21.15.133:0/526359938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f830810e090 0x7f830810e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.305+0000 7f830d3d2640 1 -- 172.21.15.133:0/526359938 >> 172.21.15.133:0/526359938 conn(0x7f8308073180 msgr2=0x7f8308073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.306+0000 7f830d3d2640 1 -- 172.21.15.133:0/526359938 shutdown_connections 2024-09-11T23:44:46.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.306+0000 7f830d3d2640 1 -- 172.21.15.133:0/526359938 wait complete. 2024-09-11T23:44:46.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.308+0000 7f830d3d2640 1 Processor -- start 2024-09-11T23:44:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.312+0000 7f830d3d2640 1 -- start start 2024-09-11T23:44:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.313+0000 7f830d3d2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830806ee70 0x7f8308084c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.313+0000 7f830d3d2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f830810e090 0x7f830807ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.313+0000 7f830d3d2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f830807f230 0x7f830807f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.313+0000 7f830d3d2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f830807d8a0 con 0x7f830810e090 2024-09-11T23:44:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.313+0000 7f830d3d2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f830807d720 con 0x7f830806ee70 2024-09-11T23:44:46.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.313+0000 7f830d3d2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f830807da20 con 0x7f830807f230 2024-09-11T23:44:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.314+0000 7f830cbd1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f830807f230 0x7f830807f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.315+0000 7f830cbd1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f830807f230 0x7f830807f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42610/0 (socket says 172.21.15.133:42610) 2024-09-11T23:44:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.315+0000 7f830cbd1640 1 -- 172.21.15.133:0/2215853235 learned_addr learned my addr 172.21.15.133:0/2215853235 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.316+0000 7f830cbd1640 1 -- 172.21.15.133:0/2215853235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830806ee70 msgr2=0x7f8308084c70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:44:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.316+0000 7f830cbd1640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830806ee70 0x7f8308084c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.316+0000 7f830cbd1640 1 -- 172.21.15.133:0/2215853235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f830810e090 msgr2=0x7f830807ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.316+0000 7f830cbd1640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f830810e090 0x7f830807ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.316+0000 7f830cbd1640 1 -- 172.21.15.133:0/2215853235 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8308130f40 con 0x7f830807f230 2024-09-11T23:44:46.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.317+0000 7f830cbd1640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f830807f230 0x7f830807f6e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00e9a0 tx=0x7f82fc00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.318+0000 7f83057fa640 1 -- 172.21.15.133:0/2215853235 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00ccc0 con 0x7f830807f230 2024-09-11T23:44:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.318+0000 7f830d3d2640 1 -- 172.21.15.133:0/2215853235 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8308131230 con 0x7f830807f230 2024-09-11T23:44:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.318+0000 7f830d3d2640 1 -- 172.21.15.133:0/2215853235 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8308131770 con 0x7f830807f230 2024-09-11T23:44:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.318+0000 7f83057fa640 1 -- 172.21.15.133:0/2215853235 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82fc00ce60 con 0x7f830807f230 2024-09-11T23:44:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.319+0000 7f83057fa640 1 -- 172.21.15.133:0/2215853235 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc010690 con 0x7f830807f230 2024-09-11T23:44:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.337+0000 7f8310a0d740 1 Processor -- start 2024-09-11T23:44:46.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.348+0000 7f8310a0d740 1 -- start start 2024-09-11T23:44:46.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.350+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 0x55a156f80270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.350+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f807b0 0x55a156f80bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.350+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f8b550 0x55a156f8d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.350+0000 7f8310a0d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a156ee62f0 con 0x55a156f807b0 2024-09-11T23:44:46.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.350+0000 7f8310a0d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a156e8dcf0 con 0x55a156f81d20 2024-09-11T23:44:46.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.350+0000 7f8310a0d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a156ee22b0 con 0x55a156f8b550 2024-09-11T23:44:46.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.351+0000 7f83057fa640 1 -- 172.21.15.133:0/2215853235 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f82fc010830 con 0x7f830807f230 2024-09-11T23:44:46.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.351+0000 7f83077fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 0x55a156f80270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.351+0000 7f83077fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 0x55a156f80270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38570/0 (socket says 172.21.15.133:38570) 2024-09-11T23:44:46.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.351+0000 7f83077fe640 1 -- 172.21.15.133:0/2738953507 learned_addr learned my addr 172.21.15.133:0/2738953507 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:46.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.351+0000 7f83077fe640 1 --2- 172.21.15.133:0/2738953507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f8b550 0x55a156f8d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.352+0000 7f83077fe640 1 -- 172.21.15.133:0/2738953507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f8b550 msgr2=0x55a156f8d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:46.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.352+0000 7f83077fe640 1 --2- 172.21.15.133:0/2738953507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f8b550 0x55a156f8d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.352+0000 7f83077fe640 1 -- 172.21.15.133:0/2738953507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f807b0 msgr2=0x55a156f80bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:46.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.352+0000 7f83077fe640 1 --2- 172.21.15.133:0/2738953507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f807b0 0x55a156f80bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.352+0000 7f83077fe640 1 -- 172.21.15.133:0/2738953507 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a156e723a0 con 0x55a156f81d20 2024-09-11T23:44:46.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.353+0000 7f83077fe640 1 --2- 172.21.15.133:0/2738953507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 0x55a156f80270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8300004620 tx=0x7f830000d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65d01b6c6c32238b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:46.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.354+0000 7f83057fa640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f00a2f30 0x7f82f00a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.355+0000 7f83057fa640 1 -- 172.21.15.133:0/2215853235 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f82fc0d0740 con 0x7f830807f230 2024-09-11T23:44:46.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.359+0000 7f8307fff640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f00a2f30 0x7f82f00a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.356+0000 7f82ee7fc640 1 -- 172.21.15.133:0/2738953507 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830002b9e0 con 0x55a156f81d20 2024-09-11T23:44:46.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.359+0000 7f82ee7fc640 1 -- 172.21.15.133:0/2738953507 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830002bb80 con 0x55a156f81d20 2024-09-11T23:44:46.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.359+0000 7f82ee7fc640 1 -- 172.21.15.133:0/2738953507 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300033440 con 0x55a156f81d20 2024-09-11T23:44:46.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.363+0000 7f8310a0d740 1 -- 172.21.15.133:0/2738953507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 msgr2=0x55a156f80270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:46.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.363+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2738953507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 0x55a156f80270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8300004620 tx=0x7f830000d000 comp rx=0 tx=0).stop 2024-09-11T23:44:46.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.365+0000 7f8310a0d740 1 -- 172.21.15.133:0/2738953507 shutdown_connections 2024-09-11T23:44:46.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.365+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2738953507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f8b550 0x55a156f8d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.365+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2738953507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f807b0 0x55a156f80bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.365+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2738953507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 0x55a156f80270 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.365+0000 7f8310a0d740 1 -- 172.21.15.133:0/2738953507 >> 172.21.15.133:0/2738953507 conn(0x55a156f49310 msgr2=0x55a156f7ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:46.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.367+0000 7f8310a0d740 1 -- 172.21.15.133:0/2738953507 shutdown_connections 2024-09-11T23:44:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.367+0000 7f8310a0d740 1 -- 172.21.15.133:0/2738953507 wait complete. 2024-09-11T23:44:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.368+0000 7f8310a0d740 1 -- start start 2024-09-11T23:44:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.369+0000 7f8310a0d740 1 Processor -- start 2024-09-11T23:44:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.370+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f807b0 0x55a156f8afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.370+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f81d20 0x55a156f88290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.370+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f8b550 0x55a156f88a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.370+0000 7f8310a0d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a156f8e1f0 con 0x55a156f81d20 2024-09-11T23:44:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.370+0000 7f8310a0d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a156ea1060 con 0x55a156f8b550 2024-09-11T23:44:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.370+0000 7f8310a0d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a156f8e370 con 0x55a156f807b0 2024-09-11T23:44:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.370+0000 7f8310a0d740 10 client.0 _ll_register_callbacks cb 0x55a156bf48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.375+0000 7f83077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f81d20 0x55a156f88290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.375+0000 7f83077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f81d20 0x55a156f88290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44208/0 (socket says 172.21.15.133:44208) 2024-09-11T23:44:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.375+0000 7f83077fe640 1 -- 172.21.15.133:0/95060599 learned_addr learned my addr 172.21.15.133:0/95060599 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:46.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.377+0000 7f830cbd1640 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f8b550 0x55a156f88a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.377+0000 7f8307fff640 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f807b0 0x55a156f8afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.378+0000 7f8307fff640 1 -- 172.21.15.133:0/95060599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f8b550 msgr2=0x55a156f88a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:46.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.378+0000 7f8307fff640 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f8b550 0x55a156f88a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.378+0000 7f8307fff640 1 -- 172.21.15.133:0/95060599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f81d20 msgr2=0x55a156f88290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:46.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.378+0000 7f8307fff640 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f81d20 0x55a156f88290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.378+0000 7f8307fff640 1 -- 172.21.15.133:0/95060599 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a156c573c0 con 0x55a156f807b0 2024-09-11T23:44:46.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.378+0000 7f8307fff640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f00a2f30 0x7f82f00a53f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55a156f7dca0 tx=0x7f82f8009250 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:46.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.378+0000 7f8307fff640 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f807b0 0x55a156f8afe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f82f800a440 tx=0x7f82f8004bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d18d5f9feaf248a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:46.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.379+0000 7f82e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:44:46.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.379+0000 7f82e77fe640 1 -- 172.21.15.133:0/95060599 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8018ce0 con 0x55a156f807b0 2024-09-11T23:44:46.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.379+0000 7f8310a0d740 1 -- 172.21.15.133:0/95060599 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f830002bb80 con 0x55a156f807b0 2024-09-11T23:44:46.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.379+0000 7f82e77fe640 1 -- 172.21.15.133:0/95060599 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f82f8023480 con 0x55a156f807b0 2024-09-11T23:44:46.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.382+0000 7f82e77fe640 1 -- 172.21.15.133:0/95060599 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f82f8023680 con 0x55a156f807b0 2024-09-11T23:44:46.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.382+0000 7f8310a0d740 10 client.26857 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:46.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.382+0000 7f8310a0d740 1 -- 172.21.15.133:0/95060599 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a156f99e00 con 0x55a156f807b0 2024-09-11T23:44:46.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.389+0000 7f82e77fe640 1 -- 172.21.15.133:0/95060599 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830002bb80 con 0x55a156f807b0 2024-09-11T23:44:46.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.389+0000 7f82e77fe640 1 -- 172.21.15.133:0/95060599 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f82f801a900 con 0x55a156f807b0 2024-09-11T23:44:46.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.389+0000 7f8310a0d740 10 client.26857 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:46.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.389+0000 7f8310a0d740 10 client.26857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:46.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.389+0000 7f8310a0d740 20 client.26857 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:46.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.389+0000 7f8310a0d740 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a156f9b760 0x55a156f9dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.389+0000 7f8310a0d740 4 client.26857 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:46.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.389+0000 7f8310a0d740 1 -- 172.21.15.133:0/95060599 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a156ef3860 con 0x55a156f9b760 2024-09-11T23:44:46.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.407+0000 7f83077fe640 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a156f9b760 0x55a156f9dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.417+0000 7f82e77fe640 10 client.26857 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:46.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.417+0000 7f83077fe640 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a156f9b760 0x55a156f9dbe0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e91d02a73ac1fa86 server_cookie=eb9836927df081a5 in_seq=0 out_seq=0 2024-09-11T23:44:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.418+0000 7f82e77fe640 1 -- 172.21.15.133:0/95060599 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a156ef3860 con 0x55a156f9b760 2024-09-11T23:44:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.418+0000 7f82e77fe640 10 client.26857 handle_command_reply: tid=0 2024-09-11T23:44:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.418+0000 7f8310a0d740 1 client.26857 shutdown 2024-09-11T23:44:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.418+0000 7f8310a0d740 1 -- 172.21.15.133:0/95060599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f807b0 msgr2=0x55a156f8afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.418+0000 7f8310a0d740 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f807b0 0x55a156f8afe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f82f800a440 tx=0x7f82f8004bc0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.418+0000 7f8310a0d740 1 -- 172.21.15.133:0/95060599 shutdown_connections 2024-09-11T23:44:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.418+0000 7f8310a0d740 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a156f9b760 0x55a156f9dbe0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.418+0000 7f8310a0d740 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f8b550 0x55a156f88a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.418+0000 7f8310a0d740 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f81d20 0x55a156f88290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.418+0000 7f8310a0d740 1 --2- 172.21.15.133:0/95060599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f807b0 0x55a156f8afe0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.418+0000 7f8310a0d740 1 -- 172.21.15.133:0/95060599 >> 172.21.15.133:0/95060599 conn(0x55a156f49310 msgr2=0x55a156f82ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:46.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.419+0000 7f82e77fe640 0 client.26857 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:46.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.419+0000 7f8310a0d740 1 -- 172.21.15.133:0/95060599 shutdown_connections 2024-09-11T23:44:46.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.419+0000 7f8310a0d740 1 -- 172.21.15.133:0/95060599 wait complete. 2024-09-11T23:44:46.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.419+0000 7f8310a0d740 20 client.26857 trim_cache size 0 max 16384 2024-09-11T23:44:46.450 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:46.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.457+0000 7f8310a0d740 1 Processor -- start 2024-09-11T23:44:46.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.457+0000 7f8310a0d740 1 -- start start 2024-09-11T23:44:46.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.458+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f807b0 0x55a156f87290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.458+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f81d20 0x55a156fa34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.458+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f8b550 0x55a156fa3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.459+0000 7f83077fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f81d20 0x55a156fa34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.459+0000 7f83077fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f81d20 0x55a156fa34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42652/0 (socket says 172.21.15.133:42652) 2024-09-11T23:44:46.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.459+0000 7f83077fe640 1 -- 172.21.15.133:0/2540921515 learned_addr learned my addr 172.21.15.133:0/2540921515 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:46.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.459+0000 7f8310a0d740 1 -- 172.21.15.133:0/2540921515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82f400e770 con 0x55a156f8b550 2024-09-11T23:44:46.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.459+0000 7f8310a0d740 1 -- 172.21.15.133:0/2540921515 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82f40102b0 con 0x55a156f807b0 2024-09-11T23:44:46.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.459+0000 7f8310a0d740 1 -- 172.21.15.133:0/2540921515 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82f400ee40 con 0x55a156f81d20 2024-09-11T23:44:46.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.459+0000 7f830cbd1640 1 --2- 172.21.15.133:0/2540921515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f8b550 0x55a156fa3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.462+0000 7f8307fff640 1 --2- 172.21.15.133:0/2540921515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f807b0 0x55a156f87290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.463+0000 7f8307fff640 1 -- 172.21.15.133:0/2540921515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f81d20 msgr2=0x55a156fa34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:46.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.463+0000 7f8307fff640 1 --2- 172.21.15.133:0/2540921515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f81d20 0x55a156fa34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.463+0000 7f8307fff640 1 -- 172.21.15.133:0/2540921515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f8b550 msgr2=0x55a156fa3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:46.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.463+0000 7f8307fff640 1 --2- 172.21.15.133:0/2540921515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f8b550 0x55a156fa3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.463+0000 7f8307fff640 1 -- 172.21.15.133:0/2540921515 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a156f8e360 con 0x55a156f807b0 2024-09-11T23:44:46.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.464+0000 7f8307fff640 1 --2- 172.21.15.133:0/2540921515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f807b0 0x55a156f87290 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c820 tx=0x7f82f8023610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=849836bdb8051ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:46.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.465+0000 7f82edffb640 1 -- 172.21.15.133:0/2540921515 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8038020 con 0x55a156f807b0 2024-09-11T23:44:46.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.466+0000 7f82edffb640 1 -- 172.21.15.133:0/2540921515 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f80193b0 con 0x55a156f807b0 2024-09-11T23:44:46.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.466+0000 7f82edffb640 1 -- 172.21.15.133:0/2540921515 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8030040 con 0x55a156f807b0 2024-09-11T23:44:46.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.469+0000 7f8310a0d740 1 -- 172.21.15.133:0/2540921515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f807b0 msgr2=0x55a156f87290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:46.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.469+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2540921515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f807b0 0x55a156f87290 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c820 tx=0x7f82f8023610 comp rx=0 tx=0).stop 2024-09-11T23:44:46.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.469+0000 7f8310a0d740 1 -- 172.21.15.133:0/2540921515 shutdown_connections 2024-09-11T23:44:46.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.469+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2540921515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f8b550 0x55a156fa3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.469+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2540921515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f81d20 0x55a156fa34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.469+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2540921515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f807b0 0x55a156f87290 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c820 tx=0x7f82f8023610 comp rx=0 tx=0).stop 2024-09-11T23:44:46.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.469+0000 7f8310a0d740 1 -- 172.21.15.133:0/2540921515 >> 172.21.15.133:0/2540921515 conn(0x55a156f49310 msgr2=0x55a156f86b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:46.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.479+0000 7f8310a0d740 1 -- 172.21.15.133:0/2540921515 shutdown_connections 2024-09-11T23:44:46.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.480+0000 7f8310a0d740 1 -- 172.21.15.133:0/2540921515 wait complete. 2024-09-11T23:44:46.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.480+0000 7f8310a0d740 1 -- start start 2024-09-11T23:44:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.481+0000 7f8310a0d740 1 Processor -- start 2024-09-11T23:44:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.482+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 0x55a156f93d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.482+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f8b550 0x55a156f84ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.482+0000 7f8310a0d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f853e0 0x55a156f85860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.482+0000 7f8310a0d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82f400fbe0 con 0x55a156f8b550 2024-09-11T23:44:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.482+0000 7f8310a0d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82f400f510 con 0x55a156f81d20 2024-09-11T23:44:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.482+0000 7f8310a0d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82f400c770 con 0x55a156f853e0 2024-09-11T23:44:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.482+0000 7f8310a0d740 10 client.0 _ll_register_callbacks cb 0x7f82f800a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.483+0000 7f830cbd1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f853e0 0x55a156f85860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.483+0000 7f830cbd1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f853e0 0x55a156f85860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42658/0 (socket says 172.21.15.133:42658) 2024-09-11T23:44:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.483+0000 7f830cbd1640 1 -- 172.21.15.133:0/2032814659 learned_addr learned my addr 172.21.15.133:0/2032814659 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:46.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.483+0000 7f8307fff640 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 0x55a156f93d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.483+0000 7f830cbd1640 1 -- 172.21.15.133:0/2032814659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 msgr2=0x55a156f93d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:46.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.483+0000 7f830cbd1640 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 0x55a156f93d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.483+0000 7f830cbd1640 1 -- 172.21.15.133:0/2032814659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f8b550 msgr2=0x55a156f84ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:46.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.483+0000 7f830cbd1640 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f8b550 0x55a156f84ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.483+0000 7f830cbd1640 1 -- 172.21.15.133:0/2032814659 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1570055c0 con 0x55a156f853e0 2024-09-11T23:44:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.489+0000 7f830cbd1640 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f853e0 0x55a156f85860 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f82fc004c80 tx=0x7f82fc09d400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb530f1446bcf8d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.489+0000 7f82e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:44:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.489+0000 7f82e77fe640 1 -- 172.21.15.133:0/2032814659 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0b9020 con 0x55a156f853e0 2024-09-11T23:44:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.489+0000 7f82e77fe640 1 -- 172.21.15.133:0/2032814659 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f82fc0ce0f0 con 0x55a156f853e0 2024-09-11T23:44:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.489+0000 7f8310a0d740 1 -- 172.21.15.133:0/2032814659 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82f80193b0 con 0x55a156f853e0 2024-09-11T23:44:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.489+0000 7f82e77fe640 1 -- 172.21.15.133:0/2032814659 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f82fc0a58b0 con 0x55a156f853e0 2024-09-11T23:44:46.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.490+0000 7f8310a0d740 10 client.26863 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:46.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.490+0000 7f8310a0d740 1 -- 172.21.15.133:0/2032814659 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1570068e0 con 0x55a156f853e0 2024-09-11T23:44:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.491+0000 7f82e77fe640 1 -- 172.21.15.133:0/2032814659 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82fc0a5420 con 0x55a156f853e0 2024-09-11T23:44:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.491+0000 7f82e77fe640 1 -- 172.21.15.133:0/2032814659 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f82fc0a5b00 con 0x55a156f853e0 2024-09-11T23:44:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.492+0000 7f8310a0d740 10 client.26863 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.492+0000 7f8310a0d740 10 client.26863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.492+0000 7f8310a0d740 20 client.26863 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.492+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a157006f60 0x55a157009380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.492+0000 7f8310a0d740 4 client.26863 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:44:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.492+0000 7f8310a0d740 1 -- 172.21.15.133:0/2032814659 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f82f80193b0 con 0x55a157006f60 2024-09-11T23:44:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.498+0000 7f8307fff640 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a157006f60 0x55a157009380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.511+0000 7f82e77fe640 10 client.26863 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.511+0000 7f8307fff640 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a157006f60 0x55a157009380 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a052273461e833c server_cookie=3160a18c389a0c6a in_seq=0 out_seq=0 2024-09-11T23:44:46.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f82e77fe640 1 -- 172.21.15.133:0/2032814659 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f82f80193b0 con 0x55a157006f60 2024-09-11T23:44:46.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f82e77fe640 10 client.26863 handle_command_reply: tid=0 2024-09-11T23:44:46.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f8310a0d740 1 client.26863 shutdown 2024-09-11T23:44:46.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f8310a0d740 1 -- 172.21.15.133:0/2032814659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f853e0 msgr2=0x55a156f85860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:46.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f853e0 0x55a156f85860 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f82fc004c80 tx=0x7f82fc09d400 comp rx=0 tx=0).stop 2024-09-11T23:44:46.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f8310a0d740 1 -- 172.21.15.133:0/2032814659 shutdown_connections 2024-09-11T23:44:46.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a157006f60 0x55a157009380 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a156f853e0 0x55a156f85860 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a156f8b550 0x55a156f84ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f8310a0d740 1 --2- 172.21.15.133:0/2032814659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a156f81d20 0x55a156f93d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f8310a0d740 1 -- 172.21.15.133:0/2032814659 >> 172.21.15.133:0/2032814659 conn(0x55a156f49310 msgr2=0x55a156f873a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:46.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.513+0000 7f82e77fe640 0 client.26863 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:46.517 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:44:46.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.515+0000 7f8310a0d740 1 -- 172.21.15.133:0/2032814659 shutdown_connections 2024-09-11T23:44:46.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.515+0000 7f8310a0d740 1 -- 172.21.15.133:0/2032814659 wait complete. 2024-09-11T23:44:46.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.515+0000 7f8310a0d740 20 client.26863 trim_cache size 0 max 16384 2024-09-11T23:44:46.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.516+0000 7f830d3d2640 1 -- 172.21.15.133:0/2215853235 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f00a2f30 msgr2=0x7f82f00a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:46.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.516+0000 7f830d3d2640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f00a2f30 0x7f82f00a53f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55a156f7dca0 tx=0x7f82f8009250 comp rx=0 tx=0).stop 2024-09-11T23:44:46.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.516+0000 7f830d3d2640 1 -- 172.21.15.133:0/2215853235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f830807f230 msgr2=0x7f830807f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:46.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.517+0000 7f830d3d2640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f830807f230 0x7f830807f6e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00e9a0 tx=0x7f82fc00ee60 comp rx=0 tx=0).stop 2024-09-11T23:44:46.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.517+0000 7f830d3d2640 1 -- 172.21.15.133:0/2215853235 shutdown_connections 2024-09-11T23:44:46.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.517+0000 7f830d3d2640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82f00a2f30 0x7f82f00a53f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.517+0000 7f830d3d2640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f830807f230 0x7f830807f6e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.517+0000 7f830d3d2640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f830810e090 0x7f830807ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.517+0000 7f830d3d2640 1 --2- 172.21.15.133:0/2215853235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830806ee70 0x7f8308084c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:46.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.517+0000 7f830d3d2640 1 -- 172.21.15.133:0/2215853235 >> 172.21.15.133:0/2215853235 conn(0x7f8308073180 msgr2=0x7f830806cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:46.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.517+0000 7f830d3d2640 1 -- 172.21.15.133:0/2215853235 shutdown_connections 2024-09-11T23:44:46.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:46.518+0000 7f830d3d2640 1 -- 172.21.15.133:0/2215853235 wait complete. 2024-09-11T23:44:46.715 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:46 smithi133 ceph-mon[31430]: pgmap v247: 97 pgs: 97 active+clean; 70 GiB data, 235 GiB used, 837 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 742 MiB/s wr, 1.93k op/s 2024-09-11T23:44:46.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:46 smithi138 ceph-mon[35142]: pgmap v247: 97 pgs: 97 active+clean; 70 GiB data, 235 GiB used, 837 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 742 MiB/s wr, 1.93k op/s 2024-09-11T23:44:47.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:46 smithi178 ceph-mon[35882]: pgmap v247: 97 pgs: 97 active+clean; 70 GiB data, 235 GiB used, 837 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 742 MiB/s wr, 1.93k op/s 2024-09-11T23:44:48.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:47 smithi178 ceph-mon[35882]: pgmap v248: 97 pgs: 97 active+clean; 70 GiB data, 236 GiB used, 837 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 606 MiB/s wr, 1.72k op/s 2024-09-11T23:44:48.105 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:47 smithi133 ceph-mon[31430]: pgmap v248: 97 pgs: 97 active+clean; 70 GiB data, 236 GiB used, 837 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 606 MiB/s wr, 1.72k op/s 2024-09-11T23:44:48.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:47 smithi138 ceph-mon[35142]: pgmap v248: 97 pgs: 97 active+clean; 70 GiB data, 236 GiB used, 837 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 606 MiB/s wr, 1.72k op/s 2024-09-11T23:44:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.339+0000 7f349a448640 1 -- 172.21.15.133:0/2666088482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3494105560 msgr2=0x7f349410d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.339+0000 7f349a448640 1 --2- 172.21.15.133:0/2666088482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3494105560 0x7f349410d0e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f348800da00 tx=0x7f348802c4b0 comp rx=0 tx=0).stop 2024-09-11T23:44:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.359+0000 7f349a448640 1 -- 172.21.15.133:0/2666088482 shutdown_connections 2024-09-11T23:44:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.359+0000 7f349a448640 1 --2- 172.21.15.133:0/2666088482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3494105560 0x7f349410d0e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.359+0000 7f349a448640 1 --2- 172.21.15.133:0/2666088482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3494100de0 0x7f3494104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.359+0000 7f349a448640 1 --2- 172.21.15.133:0/2666088482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3494100430 0x7f3494100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.359+0000 7f349a448640 1 -- 172.21.15.133:0/2666088482 >> 172.21.15.133:0/2666088482 conn(0x7f3494077230 msgr2=0x7f34940fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.361+0000 7f349a448640 1 -- 172.21.15.133:0/2666088482 shutdown_connections 2024-09-11T23:44:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.361+0000 7f349a448640 1 -- 172.21.15.133:0/2666088482 wait complete. 2024-09-11T23:44:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.362+0000 7f349a448640 1 Processor -- start 2024-09-11T23:44:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.363+0000 7f349a448640 1 -- start start 2024-09-11T23:44:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.363+0000 7f349a448640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3494100430 0x7f3494073950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.363+0000 7f349a448640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3494100de0 0x7f3494073e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.363+0000 7f349a448640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3494105560 0x7f349406cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.363+0000 7f349a448640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3494118550 con 0x7f3494100430 2024-09-11T23:44:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.363+0000 7f349a448640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34941183d0 con 0x7f3494105560 2024-09-11T23:44:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.363+0000 7f349a448640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f34941186d0 con 0x7f3494100de0 2024-09-11T23:44:48.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.369+0000 7f34937fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3494100de0 0x7f3494073e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:48.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.369+0000 7f34989e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3494105560 0x7f349406cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:48.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.369+0000 7f34989e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3494105560 0x7f349406cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38620/0 (socket says 172.21.15.133:38620) 2024-09-11T23:44:48.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.369+0000 7f34989e5640 1 -- 172.21.15.133:0/231922260 learned_addr learned my addr 172.21.15.133:0/231922260 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:48.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.369+0000 7f3493fff640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3494100430 0x7f3494073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:48.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.376+0000 7f34989e5640 1 -- 172.21.15.133:0/231922260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3494100de0 msgr2=0x7f3494073e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:48.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.376+0000 7f34989e5640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3494100de0 0x7f3494073e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:48.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.376+0000 7f34989e5640 1 -- 172.21.15.133:0/231922260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3494100430 msgr2=0x7f3494073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:48.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.376+0000 7f34989e5640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3494100430 0x7f3494073950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:48.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.376+0000 7f34989e5640 1 -- 172.21.15.133:0/231922260 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349406d430 con 0x7f3494105560 2024-09-11T23:44:48.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.381+0000 7f34989e5640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3494105560 0x7f349406cb70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3488005ab0 tx=0x7f3488033630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:48.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.385+0000 7f34917fa640 1 -- 172.21.15.133:0/231922260 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488040070 con 0x7f3494105560 2024-09-11T23:44:48.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.385+0000 7f349a448640 1 -- 172.21.15.133:0/231922260 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349406d6c0 con 0x7f3494105560 2024-09-11T23:44:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.385+0000 7f349a448640 1 -- 172.21.15.133:0/231922260 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3494071650 con 0x7f3494105560 2024-09-11T23:44:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.387+0000 7f34917fa640 1 -- 172.21.15.133:0/231922260 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f348803c3f0 con 0x7f3494105560 2024-09-11T23:44:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.387+0000 7f34917fa640 1 -- 172.21.15.133:0/231922260 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488037030 con 0x7f3494105560 2024-09-11T23:44:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.391+0000 7f34917fa640 1 -- 172.21.15.133:0/231922260 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f348800c640 con 0x7f3494105560 2024-09-11T23:44:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.392+0000 7f34917fa640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f347c0a2f20 0x7f347c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.393+0000 7f34917fa640 1 -- 172.21.15.133:0/231922260 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f34880f11a0 con 0x7f3494105560 2024-09-11T23:44:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.393+0000 7f3493fff640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f347c0a2f20 0x7f347c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.395+0000 7f349a448640 1 -- 172.21.15.133:0/231922260 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3494101910 con 0x7f3494105560 2024-09-11T23:44:48.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.406+0000 7f34917fa640 1 -- 172.21.15.133:0/231922260 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3488049030 con 0x7f3494105560 2024-09-11T23:44:48.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.425+0000 7f3493fff640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f347c0a2f20 0x7f347c0a53e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f348c007ec0 tx=0x7f348c0075b0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:48.715 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:44:48.715 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:44:48.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.909+0000 7f349a448640 1 -- 172.21.15.133:0/231922260 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f349406df10 con 0x7f3494105560 2024-09-11T23:44:48.915 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:44:48.916 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:44:48.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.915+0000 7f34917fa640 1 -- 172.21.15.133:0/231922260 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7f34880bdda0 con 0x7f3494105560 2024-09-11T23:44:48.916 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:44:48.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.926+0000 7f349a448640 1 -- 172.21.15.133:0/231922260 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f347c0a2f20 msgr2=0x7f347c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:48.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.926+0000 7f349a448640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f347c0a2f20 0x7f347c0a53e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f348c007ec0 tx=0x7f348c0075b0 comp rx=0 tx=0).stop 2024-09-11T23:44:48.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.926+0000 7f349a448640 1 -- 172.21.15.133:0/231922260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3494105560 msgr2=0x7f349406cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:48.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.926+0000 7f349a448640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3494105560 0x7f349406cb70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3488005ab0 tx=0x7f3488033630 comp rx=0 tx=0).stop 2024-09-11T23:44:48.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.928+0000 7f349a448640 1 -- 172.21.15.133:0/231922260 shutdown_connections 2024-09-11T23:44:48.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.928+0000 7f349a448640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f347c0a2f20 0x7f347c0a53e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:48.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.928+0000 7f349a448640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3494105560 0x7f349406cb70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:48.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.928+0000 7f349a448640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3494100de0 0x7f3494073e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:48.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.928+0000 7f349a448640 1 --2- 172.21.15.133:0/231922260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3494100430 0x7f3494073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:48.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.928+0000 7f349a448640 1 -- 172.21.15.133:0/231922260 >> 172.21.15.133:0/231922260 conn(0x7f3494077230 msgr2=0x7f3494103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:48.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.933+0000 7f349a448640 1 -- 172.21.15.133:0/231922260 shutdown_connections 2024-09-11T23:44:48.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:48.935+0000 7f349a448640 1 -- 172.21.15.133:0/231922260 wait complete. 2024-09-11T23:44:49.085 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:49 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/231922260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:44:49.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:49 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/231922260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:44:49.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:49 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/231922260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:44:49.892 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-11T23:44:49.893 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-11T23:44:49.959 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:50.409 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:50 smithi133 ceph-mon[31430]: pgmap v249: 97 pgs: 97 active+clean; 70 GiB data, 240 GiB used, 833 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 760 MiB/s wr, 2.40k op/s 2024-09-11T23:44:50.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:50 smithi178 ceph-mon[35882]: pgmap v249: 97 pgs: 97 active+clean; 70 GiB data, 240 GiB used, 833 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 760 MiB/s wr, 2.40k op/s 2024-09-11T23:44:50.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:50 smithi138 ceph-mon[35142]: pgmap v249: 97 pgs: 97 active+clean; 70 GiB data, 240 GiB used, 833 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 760 MiB/s wr, 2.40k op/s 2024-09-11T23:44:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.942+0000 7f6549d01640 1 -- 172.21.15.133:0/286375430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654410e060 msgr2=0x7f654410e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.942+0000 7f6549d01640 1 --2- 172.21.15.133:0/286375430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654410e060 0x7f654410e440 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6534009860 tx=0x7f653402a360 comp rx=0 tx=0).stop 2024-09-11T23:44:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.942+0000 7f6549d01640 1 -- 172.21.15.133:0/286375430 shutdown_connections 2024-09-11T23:44:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.942+0000 7f6549d01640 1 --2- 172.21.15.133:0/286375430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6544108980 0x7f6544114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.942+0000 7f6549d01640 1 --2- 172.21.15.133:0/286375430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6544108020 0x7f6544108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.942+0000 7f6549d01640 1 --2- 172.21.15.133:0/286375430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654410e060 0x7f654410e440 secure :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6534009860 tx=0x7f653402a360 comp rx=0 tx=0).stop 2024-09-11T23:44:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.942+0000 7f6549d01640 1 -- 172.21.15.133:0/286375430 >> 172.21.15.133:0/286375430 conn(0x7f6544077980 msgr2=0x7f6544079da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.943+0000 7f6549d01640 1 -- 172.21.15.133:0/286375430 shutdown_connections 2024-09-11T23:44:50.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.948+0000 7f6549d01640 1 -- 172.21.15.133:0/286375430 wait complete. 2024-09-11T23:44:50.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.950+0000 7f6549d01640 1 Processor -- start 2024-09-11T23:44:50.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.951+0000 7f6549d01640 1 -- start start 2024-09-11T23:44:50.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.951+0000 7f6549d01640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6544108020 0x7f65441a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:50.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.952+0000 7f6549d01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6544108980 0x7f65441a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:50.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.962+0000 7f6549d01640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f654419e960 0x7f654419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:50.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.962+0000 7f6548cff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6544108020 0x7f65441a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:50.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.962+0000 7f6549d01640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6544118450 con 0x7f6544108020 2024-09-11T23:44:50.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.962+0000 7f6549d01640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65441182d0 con 0x7f6544108980 2024-09-11T23:44:50.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.962+0000 7f6549d01640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65441185d0 con 0x7f654419e960 2024-09-11T23:44:50.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.963+0000 7f6548cff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6544108020 0x7f65441a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44258/0 (socket says 172.21.15.133:44258) 2024-09-11T23:44:50.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.963+0000 7f6548cff640 1 -- 172.21.15.133:0/1670415788 learned_addr learned my addr 172.21.15.133:0/1670415788 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:50.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.967+0000 7f6549500640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f654419e960 0x7f654419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:50.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.967+0000 7f6543fff640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6544108980 0x7f65441a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:50.973 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:50.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.973+0000 7f6549500640 1 -- 172.21.15.133:0/1670415788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6544108980 msgr2=0x7f65441a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:50.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.973+0000 7f6549500640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6544108980 0x7f65441a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:50.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.973+0000 7f6549500640 1 -- 172.21.15.133:0/1670415788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6544108020 msgr2=0x7f65441a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:50.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.973+0000 7f6549500640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6544108020 0x7f65441a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:50.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.974+0000 7f6549500640 1 -- 172.21.15.133:0/1670415788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654419f480 con 0x7f654419e960 2024-09-11T23:44:50.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.978+0000 7f6549500640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f654419e960 0x7f654419ee10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f653c00be20 tx=0x7f653c00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:50.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.983+0000 7f6541ffb640 1 -- 172.21.15.133:0/1670415788 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c019070 con 0x7f654419e960 2024-09-11T23:44:50.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.983+0000 7f6541ffb640 1 -- 172.21.15.133:0/1670415788 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f653c00cd70 con 0x7f654419e960 2024-09-11T23:44:50.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.983+0000 7f6541ffb640 1 -- 172.21.15.133:0/1670415788 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c0127e0 con 0x7f654419e960 2024-09-11T23:44:50.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.983+0000 7f6549d01640 1 -- 172.21.15.133:0/1670415788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654419f770 con 0x7f654419e960 2024-09-11T23:44:50.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.984+0000 7f6549d01640 1 -- 172.21.15.133:0/1670415788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65441ab900 con 0x7f654419e960 2024-09-11T23:44:50.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.994+0000 7f6541ffb640 1 -- 172.21.15.133:0/1670415788 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f653c01c070 con 0x7f654419e960 2024-09-11T23:44:50.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.996+0000 7f6541ffb640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f652c0a2f30 0x7f652c0a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:50.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:50.997+0000 7f6541ffb640 1 -- 172.21.15.133:0/1670415788 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f653c0cfa30 con 0x7f654419e960 2024-09-11T23:44:51.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.000+0000 7f654d33c740 1 Processor -- start 2024-09-11T23:44:51.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.000+0000 7f654d33c740 1 -- start start 2024-09-11T23:44:51.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.001+0000 7f654d33c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba1d70 0x55a0feba2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.001+0000 7f654d33c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 0x55a0febaca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.002+0000 7f654d33c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 0x55a0febaf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.002+0000 7f654d33c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a0feb07280 con 0x55a0feba1d70 2024-09-11T23:44:51.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.002+0000 7f654d33c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a0feaaeb10 con 0x55a0febacf50 2024-09-11T23:44:51.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.002+0000 7f654d33c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0feb03240 con 0x55a0feba2690 2024-09-11T23:44:51.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.002+0000 7f6548cff640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f652c0a2f30 0x7f652c0a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.002+0000 7f6543fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 0x55a0febaca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.002+0000 7f6543fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 0x55a0febaca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42710/0 (socket says 172.21.15.133:42710) 2024-09-11T23:44:51.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.002+0000 7f6543fff640 1 -- 172.21.15.133:0/2857701862 learned_addr learned my addr 172.21.15.133:0/2857701862 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:51.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.003+0000 7f6549500640 1 --2- 172.21.15.133:0/2857701862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 0x55a0febaf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.004+0000 7f6543fff640 1 -- 172.21.15.133:0/2857701862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 msgr2=0x55a0febaf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:51.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.004+0000 7f6543fff640 1 --2- 172.21.15.133:0/2857701862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 0x55a0febaf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.004+0000 7f6543fff640 1 -- 172.21.15.133:0/2857701862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba1d70 msgr2=0x55a0feba2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:51.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.004+0000 7f6543fff640 1 --2- 172.21.15.133:0/2857701862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba1d70 0x55a0feba2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.004+0000 7f6543fff640 1 -- 172.21.15.133:0/2857701862 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0feabb1f0 con 0x55a0feba2690 2024-09-11T23:44:51.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.005+0000 7f6543fff640 1 --2- 172.21.15.133:0/2857701862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 0x55a0febaca10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6538009c60 tx=0x7f6538011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2484f74699248e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:51.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.005+0000 7f652affd640 1 -- 172.21.15.133:0/2857701862 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538004230 con 0x55a0feba2690 2024-09-11T23:44:51.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.005+0000 7f652affd640 1 -- 172.21.15.133:0/2857701862 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65380043d0 con 0x55a0feba2690 2024-09-11T23:44:51.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.007+0000 7f654d33c740 1 -- 172.21.15.133:0/2857701862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 msgr2=0x55a0febaca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:51.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.007+0000 7f654d33c740 1 --2- 172.21.15.133:0/2857701862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 0x55a0febaca10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6538009c60 tx=0x7f6538011810 comp rx=0 tx=0).stop 2024-09-11T23:44:51.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.009+0000 7f654d33c740 1 -- 172.21.15.133:0/2857701862 shutdown_connections 2024-09-11T23:44:51.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.009+0000 7f654d33c740 1 --2- 172.21.15.133:0/2857701862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 0x55a0febaf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.009+0000 7f654d33c740 1 --2- 172.21.15.133:0/2857701862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 0x55a0febaca10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.009+0000 7f654d33c740 1 --2- 172.21.15.133:0/2857701862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba1d70 0x55a0feba2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.009+0000 7f654d33c740 1 -- 172.21.15.133:0/2857701862 >> 172.21.15.133:0/2857701862 conn(0x55a0feb9e300 msgr2=0x55a0feb9e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:51.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.014+0000 7f6548cff640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f652c0a2f30 0x7f652c0a53f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6534009830 tx=0x7f65340094d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:51.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.014+0000 7f654d33c740 1 -- 172.21.15.133:0/2857701862 shutdown_connections 2024-09-11T23:44:51.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.017+0000 7f654d33c740 1 -- 172.21.15.133:0/2857701862 wait complete. 2024-09-11T23:44:51.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.017+0000 7f654d33c740 1 -- start start 2024-09-11T23:44:51.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.018+0000 7f654d33c740 1 Processor -- start 2024-09-11T23:44:51.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.019+0000 7f654d33c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0feba1d70 0x55a0febac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.019+0000 7f654d33c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba2690 0x55a0feba9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.019+0000 7f654d33c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0febacf50 0x55a0febaa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.019+0000 7f654d33c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a0febafbc0 con 0x55a0feba2690 2024-09-11T23:44:51.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.019+0000 7f654d33c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a0feac1f10 con 0x55a0feba1d70 2024-09-11T23:44:51.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.019+0000 7f654d33c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0febafd40 con 0x55a0febacf50 2024-09-11T23:44:51.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.020+0000 7f654d33c740 10 client.0 _ll_register_callbacks cb 0x55a0fea8aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:51.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.021+0000 7f6543fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba2690 0x55a0feba9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.021+0000 7f6543fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba2690 0x55a0feba9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44272/0 (socket says 172.21.15.133:44272) 2024-09-11T23:44:51.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.021+0000 7f6543fff640 1 -- 172.21.15.133:0/3950890053 learned_addr learned my addr 172.21.15.133:0/3950890053 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:51.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.022+0000 7f6548cff640 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0feba1d70 0x55a0febac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.024+0000 7f6543fff640 1 -- 172.21.15.133:0/3950890053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0febacf50 msgr2=0x55a0febaa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:51.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.024+0000 7f6543fff640 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0febacf50 0x55a0febaa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.024+0000 7f6543fff640 1 -- 172.21.15.133:0/3950890053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0feba1d70 msgr2=0x55a0febac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:51.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.024+0000 7f6543fff640 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0feba1d70 0x55a0febac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.024+0000 7f6543fff640 1 -- 172.21.15.133:0/3950890053 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0fe8786a0 con 0x55a0feba2690 2024-09-11T23:44:51.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.027+0000 7f6543fff640 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba2690 0x55a0feba9d20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6538005c80 tx=0x7f65380094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e7f8d4576466fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:51.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.027+0000 7f6523fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:51.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.029+0000 7f6523fff640 1 -- 172.21.15.133:0/3950890053 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538028020 con 0x55a0feba2690 2024-09-11T23:44:51.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.030+0000 7f654d33c740 1 -- 172.21.15.133:0/3950890053 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65380043d0 con 0x55a0feba2690 2024-09-11T23:44:51.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.030+0000 7f6523fff640 1 -- 172.21.15.133:0/3950890053 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f65380037f0 con 0x55a0feba2690 2024-09-11T23:44:51.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.031+0000 7f6523fff640 1 -- 172.21.15.133:0/3950890053 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6538033050 con 0x55a0feba2690 2024-09-11T23:44:51.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.031+0000 7f654d33c740 10 client.17496 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:51.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.031+0000 7f654d33c740 1 -- 172.21.15.133:0/3950890053 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0febbb700 con 0x55a0feba2690 2024-09-11T23:44:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.033+0000 7f6523fff640 1 -- 172.21.15.133:0/3950890053 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65380043d0 con 0x55a0feba2690 2024-09-11T23:44:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.034+0000 7f6523fff640 1 -- 172.21.15.133:0/3950890053 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6538018070 con 0x55a0feba2690 2024-09-11T23:44:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.034+0000 7f654d33c740 10 client.17496 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.034+0000 7f654d33c740 10 client.17496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.034+0000 7f654d33c740 20 client.17496 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.034+0000 7f654d33c740 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a0febbcfa0 0x55a0febbf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.034+0000 7f654d33c740 4 client.17496 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:51.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.034+0000 7f654d33c740 1 -- 172.21.15.133:0/3950890053 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0feb147f0 con 0x55a0febbcfa0 2024-09-11T23:44:51.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.040+0000 7f6548cff640 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a0febbcfa0 0x55a0febbf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.045+0000 7f6548cff640 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a0febbcfa0 0x55a0febbf440 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c90aa9fd4f7 server_cookie=88e0083234f794b in_seq=0 out_seq=0 2024-09-11T23:44:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.045+0000 7f6523fff640 10 client.17496 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.047+0000 7f6523fff640 1 -- 172.21.15.133:0/3950890053 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0feb147f0 con 0x55a0febbcfa0 2024-09-11T23:44:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.047+0000 7f6523fff640 10 client.17496 handle_command_reply: tid=0 2024-09-11T23:44:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.047+0000 7f654d33c740 1 client.17496 shutdown 2024-09-11T23:44:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.048+0000 7f654d33c740 1 -- 172.21.15.133:0/3950890053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba2690 msgr2=0x55a0feba9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.048+0000 7f654d33c740 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba2690 0x55a0feba9d20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6538005c80 tx=0x7f65380094c0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.052+0000 7f654d33c740 1 -- 172.21.15.133:0/3950890053 shutdown_connections 2024-09-11T23:44:51.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.052+0000 7f654d33c740 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a0febbcfa0 0x55a0febbf440 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.052+0000 7f654d33c740 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0febacf50 0x55a0febaa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.052+0000 7f654d33c740 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba2690 0x55a0feba9d20 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.052+0000 7f654d33c740 1 --2- 172.21.15.133:0/3950890053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0feba1d70 0x55a0febac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.052+0000 7f654d33c740 1 -- 172.21.15.133:0/3950890053 >> 172.21.15.133:0/3950890053 conn(0x55a0feb9e300 msgr2=0x55a0feb9d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.052+0000 7f6523fff640 0 client.17496 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.052+0000 7f654d33c740 1 -- 172.21.15.133:0/3950890053 shutdown_connections 2024-09-11T23:44:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.052+0000 7f654d33c740 1 -- 172.21.15.133:0/3950890053 wait complete. 2024-09-11T23:44:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.053+0000 7f654d33c740 20 client.17496 trim_cache size 0 max 16384 2024-09-11T23:44:51.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.099+0000 7f654d33c740 1 Processor -- start 2024-09-11T23:44:51.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.101+0000 7f654d33c740 1 -- start start 2024-09-11T23:44:51.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.102+0000 7f654d33c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0feba1d70 0x55a0febb8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.102+0000 7f654d33c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 0x55a0feb9d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.102+0000 7f654d33c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0febacf50 0x55a0febc4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.102+0000 7f654d33c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f65441b4250 con 0x55a0febacf50 2024-09-11T23:44:51.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.102+0000 7f654d33c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65441b5d90 con 0x55a0feba1d70 2024-09-11T23:44:51.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.102+0000 7f654d33c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65441b4920 con 0x55a0feba2690 2024-09-11T23:44:51.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.106+0000 7f6549500640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0febacf50 0x55a0febc4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.106+0000 7f6549500640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0febacf50 0x55a0febc4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44286/0 (socket says 172.21.15.133:44286) 2024-09-11T23:44:51.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.106+0000 7f6549500640 1 -- 172.21.15.133:0/402440449 learned_addr learned my addr 172.21.15.133:0/402440449 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:51.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.106+0000 7f6548cff640 1 --2- 172.21.15.133:0/402440449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0feba1d70 0x55a0febb8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.106+0000 7f6543fff640 1 --2- 172.21.15.133:0/402440449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 0x55a0feb9d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.110+0000 7f6548cff640 1 -- 172.21.15.133:0/402440449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 msgr2=0x55a0feb9d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:51.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.110+0000 7f6548cff640 1 --2- 172.21.15.133:0/402440449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 0x55a0feb9d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.110+0000 7f6548cff640 1 -- 172.21.15.133:0/402440449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0febacf50 msgr2=0x55a0febc4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:51.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.110+0000 7f6548cff640 1 --2- 172.21.15.133:0/402440449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0febacf50 0x55a0febc4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.110+0000 7f6548cff640 1 -- 172.21.15.133:0/402440449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0feb9e130 con 0x55a0feba1d70 2024-09-11T23:44:51.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.111+0000 7f6548cff640 1 --2- 172.21.15.133:0/402440449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0feba1d70 0x55a0febb8e70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f653402a950 tx=0x7f653403d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58bdb69c3b4a405d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.112+0000 7f652a7fc640 1 -- 172.21.15.133:0/402440449 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534059070 con 0x55a0feba1d70 2024-09-11T23:44:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.112+0000 7f652a7fc640 1 -- 172.21.15.133:0/402440449 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6534044040 con 0x55a0feba1d70 2024-09-11T23:44:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.113+0000 7f654d33c740 1 -- 172.21.15.133:0/402440449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0feba1d70 msgr2=0x55a0febb8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.114+0000 7f654d33c740 1 --2- 172.21.15.133:0/402440449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0feba1d70 0x55a0febb8e70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f653402a950 tx=0x7f653403d000 comp rx=0 tx=0).stop 2024-09-11T23:44:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.114+0000 7f654d33c740 1 -- 172.21.15.133:0/402440449 shutdown_connections 2024-09-11T23:44:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.114+0000 7f654d33c740 1 --2- 172.21.15.133:0/402440449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0febacf50 0x55a0febc4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.114+0000 7f654d33c740 1 --2- 172.21.15.133:0/402440449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0feba2690 0x55a0feb9d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.114+0000 7f654d33c740 1 --2- 172.21.15.133:0/402440449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0feba1d70 0x55a0febb8e70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.114+0000 7f654d33c740 1 -- 172.21.15.133:0/402440449 >> 172.21.15.133:0/402440449 conn(0x55a0feb9e300 msgr2=0x55a0feb9ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:51.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.120+0000 7f654d33c740 1 -- 172.21.15.133:0/402440449 shutdown_connections 2024-09-11T23:44:51.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.120+0000 7f654d33c740 1 -- 172.21.15.133:0/402440449 wait complete. 2024-09-11T23:44:51.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.121+0000 7f654d33c740 1 -- start start 2024-09-11T23:44:51.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.121+0000 7f654d33c740 1 Processor -- start 2024-09-11T23:44:51.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.122+0000 7f654d33c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba2690 0x55a0febadad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.122+0000 7f654d33c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 0x55a0febae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.123+0000 7f654d33c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0febb3fe0 0x55a0febb4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.123+0000 7f6543fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 0x55a0febae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.123+0000 7f6543fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 0x55a0febae010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:38678/0 (socket says 172.21.15.133:38678) 2024-09-11T23:44:51.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.123+0000 7f6543fff640 1 -- 172.21.15.133:0/3478382773 learned_addr learned my addr 172.21.15.133:0/3478382773 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:51.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.123+0000 7f6549500640 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0febb3fe0 0x55a0febb4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.123+0000 7f654d33c740 1 -- 172.21.15.133:0/3478382773 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f65441b56c0 con 0x55a0feba2690 2024-09-11T23:44:51.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.123+0000 7f654d33c740 1 -- 172.21.15.133:0/3478382773 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65441b4ff0 con 0x55a0febacf50 2024-09-11T23:44:51.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.124+0000 7f654d33c740 1 -- 172.21.15.133:0/3478382773 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65441b22e0 con 0x55a0febb3fe0 2024-09-11T23:44:51.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.124+0000 7f654d33c740 10 client.0 _ll_register_callbacks cb 0x7f6538009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:51.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.124+0000 7f6543fff640 1 -- 172.21.15.133:0/3478382773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0febb3fe0 msgr2=0x55a0febb4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:51.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.124+0000 7f6543fff640 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0febb3fe0 0x55a0febb4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.124+0000 7f6543fff640 1 -- 172.21.15.133:0/3478382773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba2690 msgr2=0x55a0febadad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:51.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.124+0000 7f6543fff640 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba2690 0x55a0febadad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.124+0000 7f6543fff640 1 -- 172.21.15.133:0/3478382773 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0febb4a20 con 0x55a0febacf50 2024-09-11T23:44:51.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.124+0000 7f6543fff640 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 0x55a0febae010 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f65380092a0 tx=0x7f6538019c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2cf685bb273bef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:51.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.125+0000 7f654d33c740 1 -- 172.21.15.133:0/3478382773 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a0feb0ed00 con 0x55a0febacf50 2024-09-11T23:44:51.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.125+0000 7f6523fff640 10 client.27020 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:44:51.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.125+0000 7f6523fff640 1 -- 172.21.15.133:0/3478382773 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653803c060 con 0x55a0febacf50 2024-09-11T23:44:51.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.125+0000 7f6523fff640 1 -- 172.21.15.133:0/3478382773 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f65380037f0 con 0x55a0febacf50 2024-09-11T23:44:51.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.126+0000 7f6523fff640 1 -- 172.21.15.133:0/3478382773 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6538028040 con 0x55a0febacf50 2024-09-11T23:44:51.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.126+0000 7f654d33c740 10 client.27020 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:51.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.126+0000 7f654d33c740 1 -- 172.21.15.133:0/3478382773 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55a0febb5c20 con 0x55a0febacf50 2024-09-11T23:44:51.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.128+0000 7f6523fff640 1 -- 172.21.15.133:0/3478382773 <== mon.2 v2:172.21.15.138:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6538018070 con 0x55a0febacf50 2024-09-11T23:44:51.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.128+0000 7f654d33c740 10 client.27020 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:51.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.128+0000 7f654d33c740 10 client.27020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:51.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.128+0000 7f654d33c740 20 client.27020 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:51.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.129+0000 7f654d33c740 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a0fec281b0 0x55a0fec2a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:51.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.129+0000 7f654d33c740 4 client.27020 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:44:51.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.129+0000 7f654d33c740 1 -- 172.21.15.133:0/3478382773 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a0feb0ed00 con 0x55a0fec281b0 2024-09-11T23:44:51.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.130+0000 7f6548cff640 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a0fec281b0 0x55a0fec2a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:51.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.137+0000 7f6548cff640 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a0fec281b0 0x55a0fec2a5f0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=addd6a5d0f71156a server_cookie=9d2e106781b82130 in_seq=0 out_seq=0 2024-09-11T23:44:51.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.138+0000 7f6523fff640 10 client.27020 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:51.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.142+0000 7f6523fff640 1 -- 172.21.15.133:0/3478382773 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a0feb0ed00 con 0x55a0fec281b0 2024-09-11T23:44:51.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.142+0000 7f6523fff640 10 client.27020 handle_command_reply: tid=0 2024-09-11T23:44:51.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.142+0000 7f654d33c740 1 client.27020 shutdown 2024-09-11T23:44:51.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.144+0000 7f654d33c740 1 -- 172.21.15.133:0/3478382773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 msgr2=0x55a0febae010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:51.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.144+0000 7f654d33c740 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 0x55a0febae010 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f65380092a0 tx=0x7f6538019c90 comp rx=0 tx=0).stop 2024-09-11T23:44:51.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.144+0000 7f654d33c740 1 -- 172.21.15.133:0/3478382773 shutdown_connections 2024-09-11T23:44:51.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.144+0000 7f654d33c740 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a0fec281b0 0x55a0fec2a5f0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.144+0000 7f654d33c740 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0febb3fe0 0x55a0febb4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.144+0000 7f654d33c740 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0febacf50 0x55a0febae010 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.145+0000 7f654d33c740 1 --2- 172.21.15.133:0/3478382773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0feba2690 0x55a0febadad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.145+0000 7f654d33c740 1 -- 172.21.15.133:0/3478382773 >> 172.21.15.133:0/3478382773 conn(0x55a0feb9e300 msgr2=0x55a0feb9d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.145+0000 7f6523fff640 0 client.27020 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.145+0000 7f654d33c740 1 -- 172.21.15.133:0/3478382773 shutdown_connections 2024-09-11T23:44:51.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.146+0000 7f654d33c740 1 -- 172.21.15.133:0/3478382773 wait complete. 2024-09-11T23:44:51.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.146+0000 7f654d33c740 20 client.27020 trim_cache size 0 max 16384 2024-09-11T23:44:51.154 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:44:51.154 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:44:51.154 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "bc2d33af-99bc-4ba9-80aa-f573d40a624e", 2024-09-11T23:44:51.154 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:44:51.154 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:44:51.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.153+0000 7f6549d01640 1 -- 172.21.15.133:0/1670415788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f652c0a2f30 msgr2=0x7f652c0a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:51.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.153+0000 7f6549d01640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f652c0a2f30 0x7f652c0a53f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6534009830 tx=0x7f65340094d0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.153+0000 7f6549d01640 1 -- 172.21.15.133:0/1670415788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f654419e960 msgr2=0x7f654419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.153+0000 7f6549d01640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f654419e960 0x7f654419ee10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f653c00be20 tx=0x7f653c00be50 comp rx=0 tx=0).stop 2024-09-11T23:44:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.153+0000 7f6549d01640 1 -- 172.21.15.133:0/1670415788 shutdown_connections 2024-09-11T23:44:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.153+0000 7f6549d01640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f652c0a2f30 0x7f652c0a53f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.153+0000 7f6549d01640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f654419e960 0x7f654419ee10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.153+0000 7f6549d01640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6544108980 0x7f65441a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.153+0000 7f6549d01640 1 --2- 172.21.15.133:0/1670415788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6544108020 0x7f65441a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.153+0000 7f6549d01640 1 -- 172.21.15.133:0/1670415788 >> 172.21.15.133:0/1670415788 conn(0x7f6544077980 msgr2=0x7f65440792a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.154+0000 7f6549d01640 1 -- 172.21.15.133:0/1670415788 shutdown_connections 2024-09-11T23:44:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:51.154+0000 7f6549d01640 1 -- 172.21.15.133:0/1670415788 wait complete. 2024-09-11T23:44:52.007 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:51 smithi133 ceph-mon[31430]: pgmap v250: 97 pgs: 97 active+clean; 70 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 635 MiB/s wr, 2.19k op/s 2024-09-11T23:44:52.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:51 smithi138 ceph-mon[35142]: pgmap v250: 97 pgs: 97 active+clean; 70 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 635 MiB/s wr, 2.19k op/s 2024-09-11T23:44:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:51 smithi178 ceph-mon[35882]: pgmap v250: 97 pgs: 97 active+clean; 70 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 635 MiB/s wr, 2.19k op/s 2024-09-11T23:44:52.648 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:44:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.498+0000 7f87daf5b640 1 -- 172.21.15.133:0/650537338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87cc0b7550 msgr2=0x7f87cc0b9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:52.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.498+0000 7f87daf5b640 1 --2- 172.21.15.133:0/650537338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87cc0b7550 0x7f87cc0b9940 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009900 tx=0x7f87c402a480 comp rx=0 tx=0).stop 2024-09-11T23:44:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.499+0000 7f87daf5b640 1 -- 172.21.15.133:0/650537338 shutdown_connections 2024-09-11T23:44:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.499+0000 7f87daf5b640 1 --2- 172.21.15.133:0/650537338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87cc0b7550 0x7f87cc0b9940 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.499+0000 7f87daf5b640 1 --2- 172.21.15.133:0/650537338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87cc0a5150 0x7f87cc0b7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.499+0000 7f87daf5b640 1 --2- 172.21.15.133:0/650537338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87cc0a4830 0x7f87cc0a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.499+0000 7f87daf5b640 1 -- 172.21.15.133:0/650537338 >> 172.21.15.133:0/650537338 conn(0x7f87cc01a0f0 msgr2=0x7f87cc01a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.501+0000 7f87daf5b640 1 -- 172.21.15.133:0/650537338 shutdown_connections 2024-09-11T23:44:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.501+0000 7f87daf5b640 1 -- 172.21.15.133:0/650537338 wait complete. 2024-09-11T23:44:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.503+0000 7f87daf5b640 1 Processor -- start 2024-09-11T23:44:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.504+0000 7f87daf5b640 1 -- start start 2024-09-11T23:44:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.504+0000 7f87daf5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87cc0a4830 0x7f87cc1435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.504+0000 7f87daf5b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87cc0a5150 0x7f87cc143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.505+0000 7f87daf5b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87cc0b7550 0x7f87cc147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.505+0000 7f87daf5b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0bcfc0 con 0x7f87cc0a4830 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.505+0000 7f87daf5b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0bce40 con 0x7f87cc0b7550 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.505+0000 7f87daf5b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0bd140 con 0x7f87cc0a5150 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.506+0000 7f87d9758640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87cc0a5150 0x7f87cc143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.506+0000 7f87d9758640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87cc0a5150 0x7f87cc143b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:41468/0 (socket says 172.21.15.133:41468) 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.506+0000 7f87d9758640 1 -- 172.21.15.133:0/121375883 learned_addr learned my addr 172.21.15.133:0/121375883 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.506+0000 7f87da75a640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87cc0b7550 0x7f87cc147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.506+0000 7f87d9f59640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87cc0a4830 0x7f87cc1435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.507+0000 7f87d9758640 1 -- 172.21.15.133:0/121375883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87cc0b7550 msgr2=0x7f87cc147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.507+0000 7f87d9758640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87cc0b7550 0x7f87cc147e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.507+0000 7f87d9758640 1 -- 172.21.15.133:0/121375883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87cc0a4830 msgr2=0x7f87cc1435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.507+0000 7f87d9758640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87cc0a4830 0x7f87cc1435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.507+0000 7f87d9758640 1 -- 172.21.15.133:0/121375883 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc148570 con 0x7f87cc0a5150 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.509+0000 7f87d9758640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87cc0a5150 0x7f87cc143b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f87d40679d0 tx=0x7f87d406eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.509+0000 7f87caffd640 1 -- 172.21.15.133:0/121375883 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d406bb90 con 0x7f87cc0a5150 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.510+0000 7f87caffd640 1 -- 172.21.15.133:0/121375883 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d406bd30 con 0x7f87cc0a5150 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.510+0000 7f87caffd640 1 -- 172.21.15.133:0/121375883 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d40786c0 con 0x7f87cc0a5150 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.511+0000 7f87daf5b640 1 -- 172.21.15.133:0/121375883 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87cc148860 con 0x7f87cc0a5150 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.511+0000 7f87daf5b640 1 -- 172.21.15.133:0/121375883 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87cc150140 con 0x7f87cc0a5150 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.517+0000 7f87de596740 1 Processor -- start 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.518+0000 7f87de596740 1 -- start start 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.519+0000 7f87de596740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a536ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.519+0000 7f87de596740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a537220 0x55ec6a5415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.519+0000 7f87de596740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a541ae0 0x55ec6a543ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.519+0000 7f87de596740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ec6a49c2a0 con 0x55ec6a536900 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.519+0000 7f87de596740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ec6a443b10 con 0x55ec6a541ae0 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.519+0000 7f87de596740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ec6a498260 con 0x55ec6a537220 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.522+0000 7f87da75a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a536ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.522+0000 7f87d9f59640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a537220 0x55ec6a5415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.522+0000 7f87da75a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a536ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47228/0 (socket says 172.21.15.133:47228) 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.522+0000 7f87da75a640 1 -- 172.21.15.133:0/3956852077 learned_addr learned my addr 172.21.15.133:0/3956852077 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.522+0000 7f87caffd640 1 -- 172.21.15.133:0/121375883 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f87d4078070 con 0x7f87cc0a5150 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.523+0000 7f87da75a640 1 -- 172.21.15.133:0/3956852077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a537220 msgr2=0x55ec6a5415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.523+0000 7f87da75a640 1 --2- 172.21.15.133:0/3956852077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a537220 0x55ec6a5415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.523+0000 7f87da75a640 1 -- 172.21.15.133:0/3956852077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a541ae0 msgr2=0x55ec6a543ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:52.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.523+0000 7f87da75a640 1 --2- 172.21.15.133:0/3956852077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a541ae0 0x55ec6a543ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.523+0000 7f87da75a640 1 -- 172.21.15.133:0/3956852077 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec6a20db90 con 0x55ec6a536900 2024-09-11T23:44:52.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.524+0000 7f87d9f59640 1 --2- 172.21.15.133:0/3956852077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a537220 0x55ec6a5415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:44:52.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.525+0000 7f87caffd640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87c00a2e50 0x7f87c00a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.526+0000 7f87caffd640 1 -- 172.21.15.133:0/121375883 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f87d41327b0 con 0x7f87cc0a5150 2024-09-11T23:44:52.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.526+0000 7f87da75a640 1 --2- 172.21.15.133:0/3956852077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a536ce0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009900 tx=0x7f87c4031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d771bafff9947b7b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.526+0000 7f87d9f59640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87c00a2e50 0x7f87c00a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.527+0000 7f87bffff640 1 -- 172.21.15.133:0/3956852077 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4040070 con 0x55ec6a536900 2024-09-11T23:44:52.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.527+0000 7f87bffff640 1 -- 172.21.15.133:0/3956852077 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c4005020 con 0x55ec6a536900 2024-09-11T23:44:52.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.527+0000 7f87bffff640 1 -- 172.21.15.133:0/3956852077 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4035040 con 0x55ec6a536900 2024-09-11T23:44:52.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.530+0000 7f87de596740 1 -- 172.21.15.133:0/3956852077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 msgr2=0x55ec6a536ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.530+0000 7f87de596740 1 --2- 172.21.15.133:0/3956852077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a536ce0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009900 tx=0x7f87c4031660 comp rx=0 tx=0).stop 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.530+0000 7f87d9f59640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87c00a2e50 0x7f87c00a5310 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55ec6a533cf0 tx=0x7f87d0008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.536+0000 7f87de596740 1 -- 172.21.15.133:0/3956852077 shutdown_connections 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.536+0000 7f87de596740 1 --2- 172.21.15.133:0/3956852077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a541ae0 0x55ec6a543ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.536+0000 7f87de596740 1 --2- 172.21.15.133:0/3956852077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a537220 0x55ec6a5415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.536+0000 7f87de596740 1 --2- 172.21.15.133:0/3956852077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a536ce0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.536+0000 7f87de596740 1 -- 172.21.15.133:0/3956852077 >> 172.21.15.133:0/3956852077 conn(0x55ec6a532e90 msgr2=0x55ec6a533270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.540+0000 7f87de596740 1 -- 172.21.15.133:0/3956852077 shutdown_connections 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.542+0000 7f87de596740 1 -- 172.21.15.133:0/3956852077 wait complete. 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.543+0000 7f87de596740 1 -- start start 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.543+0000 7f87de596740 1 Processor -- start 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.545+0000 7f87de596740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a53eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.546+0000 7f87de596740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a537220 0x55ec6a53f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.546+0000 7f87de596740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 0x55ec6a53f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.546+0000 7f87de596740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ec6a5556c0 con 0x55ec6a536900 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.546+0000 7f87de596740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ec6a4246e0 con 0x55ec6a537220 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.546+0000 7f87de596740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ec6a555840 con 0x55ec6a541ae0 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.546+0000 7f87de596740 10 client.0 _ll_register_callbacks cb 0x55ec6a448a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.547+0000 7f87d9f59640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a53eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.547+0000 7f87da75a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 0x55ec6a53f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.547+0000 7f87d9f59640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a53eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47242/0 (socket says 172.21.15.133:47242) 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.547+0000 7f87d9f59640 1 -- 172.21.15.133:0/1346995055 learned_addr learned my addr 172.21.15.133:0/1346995055 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.548+0000 7f87d9f59640 1 -- 172.21.15.133:0/1346995055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 msgr2=0x55ec6a53f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.548+0000 7f87d9f59640 1 --2- 172.21.15.133:0/1346995055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 0x55ec6a53f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.548+0000 7f87d9f59640 1 -- 172.21.15.133:0/1346995055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a537220 msgr2=0x55ec6a53f060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:52.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.548+0000 7f87d9f59640 1 --2- 172.21.15.133:0/1346995055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a537220 0x55ec6a53f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.548+0000 7f87d9f59640 1 -- 172.21.15.133:0/1346995055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec6a55ed60 con 0x55ec6a536900 2024-09-11T23:44:52.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.548+0000 7f87d9f59640 1 --2- 172.21.15.133:0/1346995055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a53eb20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f87d0015770 tx=0x7f87d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72fc6b54681ce98 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.549+0000 7f87bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:52.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.549+0000 7f87bd7fa640 1 -- 172.21.15.133:0/1346995055 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d001c050 con 0x55ec6a536900 2024-09-11T23:44:52.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.550+0000 7f87de596740 1 -- 172.21.15.133:0/1346995055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c4005020 con 0x55ec6a536900 2024-09-11T23:44:52.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.550+0000 7f87bd7fa640 1 -- 172.21.15.133:0/1346995055 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f87d0016ae0 con 0x55ec6a536900 2024-09-11T23:44:52.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.550+0000 7f87bd7fa640 1 -- 172.21.15.133:0/1346995055 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87d002f050 con 0x55ec6a536900 2024-09-11T23:44:52.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.554+0000 7f87de596740 10 client.17511 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:52.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.554+0000 7f87de596740 1 -- 172.21.15.133:0/1346995055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ec6a561680 con 0x55ec6a536900 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.555+0000 7f87bd7fa640 1 -- 172.21.15.133:0/1346995055 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c4005020 con 0x55ec6a536900 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.555+0000 7f87bd7fa640 1 -- 172.21.15.133:0/1346995055 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f87d00143d0 con 0x55ec6a536900 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.555+0000 7f87de596740 10 client.17511 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.555+0000 7f87de596740 10 client.17511 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.556+0000 7f87de596740 20 client.17511 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.556+0000 7f87de596740 1 --2- 172.21.15.133:0/1346995055 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ec6a562f60 0x55ec6a5653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.556+0000 7f87de596740 4 client.17511 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.556+0000 7f87de596740 1 -- 172.21.15.133:0/1346995055 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec6a4a99d0 con 0x55ec6a562f60 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.565+0000 7f87d9758640 1 --2- 172.21.15.133:0/1346995055 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ec6a562f60 0x55ec6a5653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.575+0000 7f87d9758640 1 --2- 172.21.15.133:0/1346995055 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ec6a562f60 0x55ec6a5653b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df562420a366ca1e server_cookie=8254e71b9ed031ae in_seq=0 out_seq=0 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.575+0000 7f87bd7fa640 10 client.17511 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:52.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.578+0000 7f87bd7fa640 1 -- 172.21.15.133:0/1346995055 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec6a4a99d0 con 0x55ec6a562f60 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.578+0000 7f87bd7fa640 10 client.17511 handle_command_reply: tid=0 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.578+0000 7f87de596740 1 client.17511 shutdown 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.579+0000 7f87de596740 1 -- 172.21.15.133:0/1346995055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 msgr2=0x55ec6a53eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.579+0000 7f87de596740 1 --2- 172.21.15.133:0/1346995055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a53eb20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f87d0015770 tx=0x7f87d00046c0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.579+0000 7f87de596740 1 -- 172.21.15.133:0/1346995055 shutdown_connections 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.579+0000 7f87de596740 1 --2- 172.21.15.133:0/1346995055 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ec6a562f60 0x55ec6a5653b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.579+0000 7f87de596740 1 --2- 172.21.15.133:0/1346995055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 0x55ec6a53f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.579+0000 7f87bd7fa640 0 client.17511 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.579+0000 7f87de596740 1 --2- 172.21.15.133:0/1346995055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a537220 0x55ec6a53f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.579+0000 7f87de596740 1 --2- 172.21.15.133:0/1346995055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a53eb20 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.579+0000 7f87de596740 1 -- 172.21.15.133:0/1346995055 >> 172.21.15.133:0/1346995055 conn(0x55ec6a532e90 msgr2=0x55ec6a538bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.580+0000 7f87de596740 1 -- 172.21.15.133:0/1346995055 shutdown_connections 2024-09-11T23:44:52.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.580+0000 7f87de596740 1 -- 172.21.15.133:0/1346995055 wait complete. 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.581+0000 7f87de596740 20 client.17511 trim_cache size 0 max 16384 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.613+0000 7f87de596740 1 Processor -- start 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.614+0000 7f87de596740 1 -- start start 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.614+0000 7f87de596740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a536900 0x55ec6a53f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.615+0000 7f87de596740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a537220 0x55ec6a56acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.615+0000 7f87d9f59640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a536900 0x55ec6a53f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.615+0000 7f87d9f59640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a536900 0x55ec6a53f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:41488/0 (socket says 172.21.15.133:41488) 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.615+0000 7f87d9f59640 1 -- 172.21.15.133:0/2865234640 learned_addr learned my addr 172.21.15.133:0/2865234640 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.615+0000 7f87de596740 1 --2- 172.21.15.133:0/2865234640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a541ae0 0x55ec6a56b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.616+0000 7f87de596740 1 -- 172.21.15.133:0/2865234640 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0b4d40 con 0x55ec6a537220 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.616+0000 7f87de596740 1 -- 172.21.15.133:0/2865234640 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc151b90 con 0x55ec6a541ae0 2024-09-11T23:44:52.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.616+0000 7f87d9758640 1 --2- 172.21.15.133:0/2865234640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a537220 0x55ec6a56acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.616+0000 7f87da75a640 1 --2- 172.21.15.133:0/2865234640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a541ae0 0x55ec6a56b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.616+0000 7f87de596740 1 -- 172.21.15.133:0/2865234640 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc150720 con 0x55ec6a536900 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.616+0000 7f87d9f59640 1 -- 172.21.15.133:0/2865234640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a541ae0 msgr2=0x55ec6a56b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.616+0000 7f87d9f59640 1 --2- 172.21.15.133:0/2865234640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a541ae0 0x55ec6a56b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.616+0000 7f87d9f59640 1 -- 172.21.15.133:0/2865234640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a537220 msgr2=0x55ec6a56acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.616+0000 7f87d9f59640 1 --2- 172.21.15.133:0/2865234640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a537220 0x55ec6a56acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.616+0000 7f87d9f59640 1 -- 172.21.15.133:0/2865234640 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec6a533b20 con 0x55ec6a536900 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.616+0000 7f87d9758640 1 --2- 172.21.15.133:0/2865234640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a537220 0x55ec6a56acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.617+0000 7f87d9f59640 1 --2- 172.21.15.133:0/2865234640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a536900 0x55ec6a53f3f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f87d0010660 tx=0x7f87d0006da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8be78c81f030d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.617+0000 7f87bf7fe640 1 -- 172.21.15.133:0/2865234640 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0037070 con 0x55ec6a536900 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.617+0000 7f87bf7fe640 1 -- 172.21.15.133:0/2865234640 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d0026070 con 0x55ec6a536900 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.617+0000 7f87bf7fe640 1 -- 172.21.15.133:0/2865234640 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d001c040 con 0x55ec6a536900 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.618+0000 7f87de596740 1 -- 172.21.15.133:0/2865234640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a536900 msgr2=0x55ec6a53f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:52.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.618+0000 7f87de596740 1 --2- 172.21.15.133:0/2865234640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a536900 0x55ec6a53f3f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f87d0010660 tx=0x7f87d0006da0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.618+0000 7f87de596740 1 -- 172.21.15.133:0/2865234640 shutdown_connections 2024-09-11T23:44:52.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.618+0000 7f87de596740 1 --2- 172.21.15.133:0/2865234640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a541ae0 0x55ec6a56b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.618+0000 7f87de596740 1 --2- 172.21.15.133:0/2865234640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a537220 0x55ec6a56acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.618+0000 7f87de596740 1 --2- 172.21.15.133:0/2865234640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a536900 0x55ec6a53f3f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.618+0000 7f87de596740 1 -- 172.21.15.133:0/2865234640 >> 172.21.15.133:0/2865234640 conn(0x55ec6a532e90 msgr2=0x55ec6a540ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:52.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.618+0000 7f87de596740 1 -- 172.21.15.133:0/2865234640 shutdown_connections 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.618+0000 7f87de596740 1 -- 172.21.15.133:0/2865234640 wait complete. 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.619+0000 7f87de596740 1 -- start start 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.619+0000 7f87de596740 1 Processor -- start 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.619+0000 7f87de596740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a556b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.619+0000 7f87de596740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a537220 0x55ec6a557060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.619+0000 7f87de596740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 0x55ec6a5575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.619+0000 7f87de596740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1514c0 con 0x55ec6a536900 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.619+0000 7f87de596740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc150df0 con 0x55ec6a537220 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.619+0000 7f87de596740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0a4c10 con 0x55ec6a541ae0 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.619+0000 7f87de596740 10 client.0 _ll_register_callbacks cb 0x7f87d0015470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.619+0000 7f87da75a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 0x55ec6a5575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87da75a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 0x55ec6a5575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:41490/0 (socket says 172.21.15.133:41490) 2024-09-11T23:44:52.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87da75a640 1 -- 172.21.15.133:0/3887717201 learned_addr learned my addr 172.21.15.133:0/3887717201 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:52.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87d9f59640 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a556b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87d9758640 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a537220 0x55ec6a557060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87d9f59640 1 -- 172.21.15.133:0/3887717201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 msgr2=0x55ec6a5575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:52.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87d9f59640 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 0x55ec6a5575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87d9f59640 1 -- 172.21.15.133:0/3887717201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a537220 msgr2=0x55ec6a557060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:52.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87d9f59640 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a537220 0x55ec6a557060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87d9f59640 1 -- 172.21.15.133:0/3887717201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec6a5ccf30 con 0x55ec6a536900 2024-09-11T23:44:52.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87da75a640 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 0x55ec6a5575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:52.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87d9f59640 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a556b20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f87d0008010 tx=0x7f87d0014890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbb7a5d67d0cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.620+0000 7f87bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:52.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.621+0000 7f87bd7fa640 1 -- 172.21.15.133:0/3887717201 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0040070 con 0x55ec6a536900 2024-09-11T23:44:52.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.621+0000 7f87de596740 1 -- 172.21.15.133:0/3887717201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d0026070 con 0x55ec6a536900 2024-09-11T23:44:52.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.621+0000 7f87bd7fa640 1 -- 172.21.15.133:0/3887717201 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f87d0014b90 con 0x55ec6a536900 2024-09-11T23:44:52.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.621+0000 7f87bd7fa640 1 -- 172.21.15.133:0/3887717201 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87d004b050 con 0x55ec6a536900 2024-09-11T23:44:52.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.621+0000 7f87de596740 10 client.17517 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:52.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.622+0000 7f87de596740 1 -- 172.21.15.133:0/3887717201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ec6a5ce210 con 0x55ec6a536900 2024-09-11T23:44:52.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.622+0000 7f87bd7fa640 1 -- 172.21.15.133:0/3887717201 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d0026070 con 0x55ec6a536900 2024-09-11T23:44:52.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.622+0000 7f87bd7fa640 1 -- 172.21.15.133:0/3887717201 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f87d003c9f0 con 0x55ec6a536900 2024-09-11T23:44:52.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.622+0000 7f87de596740 10 client.17517 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:52.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.622+0000 7f87de596740 10 client.17517 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:44:52.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.622+0000 7f87de596740 20 client.17517 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:52.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.622+0000 7f87de596740 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ec6a5ce830 0x55ec6a5d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.622+0000 7f87de596740 4 client.17517 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-11T23:44:52.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.622+0000 7f87de596740 1 -- 172.21.15.133:0/3887717201 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x55ec6a4a3d20 con 0x55ec6a5ce830 2024-09-11T23:44:52.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.622+0000 7f87d9758640 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ec6a5ce830 0x55ec6a5d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.623+0000 7f87bd7fa640 10 client.17517 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:52.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:52.623+0000 7f87d9758640 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ec6a5ce830 0x55ec6a5d0c20 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bfb505ab0c429a9 server_cookie=bee46ffa9358a08e in_seq=0 out_seq=0 2024-09-11T23:44:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.860+0000 7fd924f11640 1 -- 172.21.15.133:0/3203002060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108cf0 msgr2=0x7fd92010cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.860+0000 7fd924f11640 1 --2- 172.21.15.133:0/3203002060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108cf0 0x7fd92010cd30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd918009ca0 tx=0x7fd91802a8d0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.861+0000 7fd924f11640 1 -- 172.21.15.133:0/3203002060 shutdown_connections 2024-09-11T23:44:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.861+0000 7fd924f11640 1 --2- 172.21.15.133:0/3203002060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd92010d490 0x7fd920114fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.861+0000 7fd924f11640 1 --2- 172.21.15.133:0/3203002060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108cf0 0x7fd92010cd30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.861+0000 7fd924f11640 1 --2- 172.21.15.133:0/3203002060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd920108340 0x7fd920108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.861+0000 7fd924f11640 1 -- 172.21.15.133:0/3203002060 >> 172.21.15.133:0/3203002060 conn(0x7fd920077a40 msgr2=0x7fd920079e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.864+0000 7fd924f11640 1 -- 172.21.15.133:0/3203002060 shutdown_connections 2024-09-11T23:44:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.864+0000 7fd924f11640 1 -- 172.21.15.133:0/3203002060 wait complete. 2024-09-11T23:44:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.866+0000 7fd924f11640 1 Processor -- start 2024-09-11T23:44:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.870+0000 7fd924f11640 1 -- start start 2024-09-11T23:44:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.872+0000 7fd924f11640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108340 0x7fd9201a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.872+0000 7fd924f11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd92010d490 0x7fd9201a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.872+0000 7fd924f11640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd92019e770 0x7fd92019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.872+0000 7fd924f11640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201184e0 con 0x7fd92010d490 2024-09-11T23:44:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.872+0000 7fd924f11640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd920118360 con 0x7fd920108340 2024-09-11T23:44:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.872+0000 7fd924f11640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd920118660 con 0x7fd92019e770 2024-09-11T23:44:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.873+0000 7fd91dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd92010d490 0x7fd9201a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.873+0000 7fd91dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd92010d490 0x7fd9201a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47278/0 (socket says 172.21.15.133:47278) 2024-09-11T23:44:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.873+0000 7fd91dd9b640 1 -- 172.21.15.133:0/962008237 learned_addr learned my addr 172.21.15.133:0/962008237 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.875+0000 7fd91e59c640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108340 0x7fd9201a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.878+0000 7fd91dd9b640 1 -- 172.21.15.133:0/962008237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd92019e770 msgr2=0x7fd92019ebf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:44:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.878+0000 7fd91dd9b640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd92019e770 0x7fd92019ebf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.878+0000 7fd91dd9b640 1 -- 172.21.15.133:0/962008237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108340 msgr2=0x7fd9201a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.878+0000 7fd91dd9b640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108340 0x7fd9201a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.878+0000 7fd91dd9b640 1 -- 172.21.15.133:0/962008237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92019f370 con 0x7fd92010d490 2024-09-11T23:44:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.878+0000 7fd91e59c640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108340 0x7fd9201a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:44:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.879+0000 7fd91dd9b640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd92010d490 0x7fd9201a4bf0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd9180098d0 tx=0x7fd918031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.879+0000 7fd90f7fe640 1 -- 172.21.15.133:0/962008237 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91802faf0 con 0x7fd92010d490 2024-09-11T23:44:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.880+0000 7fd90f7fe640 1 -- 172.21.15.133:0/962008237 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd91802fc90 con 0x7fd92010d490 2024-09-11T23:44:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.880+0000 7fd924f11640 1 -- 172.21.15.133:0/962008237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92019f600 con 0x7fd92010d490 2024-09-11T23:44:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.880+0000 7fd924f11640 1 -- 172.21.15.133:0/962008237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9201ab6e0 con 0x7fd92010d490 2024-09-11T23:44:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.881+0000 7fd90f7fe640 1 -- 172.21.15.133:0/962008237 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9180467e0 con 0x7fd92010d490 2024-09-11T23:44:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.885+0000 7fd927cac740 1 Processor -- start 2024-09-11T23:44:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.888+0000 7fd90f7fe640 1 -- 172.21.15.133:0/962008237 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd9180305d0 con 0x7fd92010d490 2024-09-11T23:44:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.890+0000 7fd90f7fe640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd9080a2e20 0x7fd9080a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.891+0000 7fd90f7fe640 1 -- 172.21.15.133:0/962008237 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fd9180f06b0 con 0x7fd92010d490 2024-09-11T23:44:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.896+0000 7fd927cac740 1 -- start start 2024-09-11T23:44:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.896+0000 7fd927cac740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dcabed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.897+0000 7fd927cac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 0x559a5dcb6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.897+0000 7fd927cac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.897+0000 7fd927cac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559a5dc102f0 con 0x559a5dcb6cb0 2024-09-11T23:44:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.897+0000 7fd927cac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a5dbb8270 con 0x559a5dcac410 2024-09-11T23:44:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.897+0000 7fd927cac740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559a5dc0c2b0 con 0x559a5dcabaf0 2024-09-11T23:44:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.900+0000 7fd91e59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dcabed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.902+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd9080a2e20 0x7fd9080a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.902+0000 7fd91ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.902+0000 7fd91ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47282/0 (socket says 172.21.15.133:47282) 2024-09-11T23:44:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.902+0000 7fd91ed9d640 1 -- 172.21.15.133:0/2805297868 learned_addr learned my addr 172.21.15.133:0/2805297868 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.902+0000 7fd91dd9b640 1 --2- 172.21.15.133:0/2805297868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 0x559a5dcb6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.905+0000 7fd91ed9d640 1 -- 172.21.15.133:0/2805297868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 msgr2=0x559a5dcabed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.905+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/2805297868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dcabed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.905+0000 7fd91ed9d640 1 -- 172.21.15.133:0/2805297868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 msgr2=0x559a5dcb6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.905+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/2805297868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 0x559a5dcb6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.905+0000 7fd91ed9d640 1 -- 172.21.15.133:0/2805297868 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a5dbb7340 con 0x559a5dcb6cb0 2024-09-11T23:44:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.907+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/2805297868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb90a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd910013f80 tx=0x7fd910018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a2671929b7614 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.907+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd9080a2e20 0x7fd9080a52e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd9100136f0 tx=0x7fd91000e940 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.909+0000 7fd90cff9640 1 -- 172.21.15.133:0/2805297868 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910020070 con 0x559a5dcb6cb0 2024-09-11T23:44:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.909+0000 7fd90cff9640 1 -- 172.21.15.133:0/2805297868 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd910005500 con 0x559a5dcb6cb0 2024-09-11T23:44:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.909+0000 7fd90cff9640 1 -- 172.21.15.133:0/2805297868 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910004a60 con 0x559a5dcb6cb0 2024-09-11T23:44:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.911+0000 7fd927cac740 1 -- 172.21.15.133:0/2805297868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 msgr2=0x559a5dcb90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.911+0000 7fd927cac740 1 --2- 172.21.15.133:0/2805297868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb90a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd910013f80 tx=0x7fd910018000 comp rx=0 tx=0).stop 2024-09-11T23:44:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.914+0000 7fd927cac740 1 -- 172.21.15.133:0/2805297868 shutdown_connections 2024-09-11T23:44:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.914+0000 7fd927cac740 1 --2- 172.21.15.133:0/2805297868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb90a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.914+0000 7fd927cac740 1 --2- 172.21.15.133:0/2805297868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 0x559a5dcb6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.914+0000 7fd927cac740 1 --2- 172.21.15.133:0/2805297868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dcabed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.914+0000 7fd927cac740 1 -- 172.21.15.133:0/2805297868 >> 172.21.15.133:0/2805297868 conn(0x559a5dca8060 msgr2=0x559a5dca8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.919+0000 7fd927cac740 1 -- 172.21.15.133:0/2805297868 shutdown_connections 2024-09-11T23:44:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.919+0000 7fd927cac740 1 -- 172.21.15.133:0/2805297868 wait complete. 2024-09-11T23:44:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.919+0000 7fd927cac740 1 -- start start 2024-09-11T23:44:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.920+0000 7fd927cac740 1 Processor -- start 2024-09-11T23:44:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.920+0000 7fd927cac740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dcb3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.921+0000 7fd927cac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 0x559a5dcb4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.921+0000 7fd927cac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.921+0000 7fd927cac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559a5dcb9880 con 0x559a5dcb6cb0 2024-09-11T23:44:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.921+0000 7fd927cac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a5db99290 con 0x559a5dcac410 2024-09-11T23:44:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.921+0000 7fd927cac740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559a5dcb9a00 con 0x559a5dcabaf0 2024-09-11T23:44:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.921+0000 7fd927cac740 10 client.0 _ll_register_callbacks cb 0x559a5dbca140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.921+0000 7fd91ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.922+0000 7fd91ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47286/0 (socket says 172.21.15.133:47286) 2024-09-11T23:44:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.922+0000 7fd91ed9d640 1 -- 172.21.15.133:0/148391046 learned_addr learned my addr 172.21.15.133:0/148391046 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.922+0000 7fd91ed9d640 1 -- 172.21.15.133:0/148391046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 msgr2=0x559a5dcb3c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.922+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/148391046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dcb3c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.922+0000 7fd91ed9d640 1 -- 172.21.15.133:0/148391046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 msgr2=0x559a5dcb4140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.922+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/148391046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 0x559a5dcb4140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.922+0000 7fd91ed9d640 1 -- 172.21.15.133:0/148391046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a5d923210 con 0x559a5dcb6cb0 2024-09-11T23:44:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.923+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/148391046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb4680 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd910005300 tx=0x7fd910005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4017f1dac7e930a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.923+0000 7fd905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.923+0000 7fd905ffb640 1 -- 172.21.15.133:0/148391046 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910033060 con 0x559a5dcb6cb0 2024-09-11T23:44:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.923+0000 7fd905ffb640 1 -- 172.21.15.133:0/148391046 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fd910004cf0 con 0x559a5dcb6cb0 2024-09-11T23:44:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.923+0000 7fd927cac740 1 -- 172.21.15.133:0/148391046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd910005500 con 0x559a5dcb6cb0 2024-09-11T23:44:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.923+0000 7fd905ffb640 1 -- 172.21.15.133:0/148391046 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd910020040 con 0x559a5dcb6cb0 2024-09-11T23:44:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.923+0000 7fd927cac740 10 client.17529 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.923+0000 7fd927cac740 1 -- 172.21.15.133:0/148391046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a5d9ef620 con 0x559a5dcb6cb0 2024-09-11T23:44:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.924+0000 7fd905ffb640 1 -- 172.21.15.133:0/148391046 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd910005500 con 0x559a5dcb6cb0 2024-09-11T23:44:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.924+0000 7fd905ffb640 1 -- 172.21.15.133:0/148391046 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd91000f040 con 0x559a5dcb6cb0 2024-09-11T23:44:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.925+0000 7fd927cac740 10 client.17529 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.925+0000 7fd927cac740 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.925+0000 7fd927cac740 20 client.17529 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.925+0000 7fd927cac740 1 --2- 172.21.15.133:0/148391046 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559a5dcc6c10 0x559a5dcc9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.925+0000 7fd927cac740 4 client.17529 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.925+0000 7fd927cac740 1 -- 172.21.15.133:0/148391046 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a5dc1da20 con 0x559a5dcc6c10 2024-09-11T23:44:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.926+0000 7fd91e59c640 1 --2- 172.21.15.133:0/148391046 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559a5dcc6c10 0x559a5dcc9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.929+0000 7fd91e59c640 1 --2- 172.21.15.133:0/148391046 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559a5dcc6c10 0x559a5dcc9000 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8f88a1be692746 server_cookie=46b1330a9c586fa5 in_seq=0 out_seq=0 2024-09-11T23:44:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.929+0000 7fd905ffb640 10 client.17529 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.938+0000 7fd905ffb640 1 -- 172.21.15.133:0/148391046 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a5dc1da20 con 0x559a5dcc6c10 2024-09-11T23:44:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.938+0000 7fd905ffb640 10 client.17529 handle_command_reply: tid=0 2024-09-11T23:44:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.938+0000 7fd927cac740 1 client.17529 shutdown 2024-09-11T23:44:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.938+0000 7fd927cac740 1 -- 172.21.15.133:0/148391046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 msgr2=0x559a5dcb4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.938+0000 7fd927cac740 1 --2- 172.21.15.133:0/148391046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb4680 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd910005300 tx=0x7fd910005330 comp rx=0 tx=0).stop 2024-09-11T23:44:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.938+0000 7fd927cac740 1 -- 172.21.15.133:0/148391046 shutdown_connections 2024-09-11T23:44:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.938+0000 7fd927cac740 1 --2- 172.21.15.133:0/148391046 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559a5dcc6c10 0x559a5dcc9000 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.939+0000 7fd905ffb640 0 client.17529 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.939+0000 7fd927cac740 1 --2- 172.21.15.133:0/148391046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dcb4680 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.939+0000 7fd927cac740 1 --2- 172.21.15.133:0/148391046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 0x559a5dcb4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.939+0000 7fd927cac740 1 --2- 172.21.15.133:0/148391046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dcb3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.939+0000 7fd927cac740 1 -- 172.21.15.133:0/148391046 >> 172.21.15.133:0/148391046 conn(0x559a5dca8060 msgr2=0x559a5dcaddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.939+0000 7fd927cac740 1 -- 172.21.15.133:0/148391046 shutdown_connections 2024-09-11T23:44:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.939+0000 7fd927cac740 1 -- 172.21.15.133:0/148391046 wait complete. 2024-09-11T23:44:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.940+0000 7fd927cac740 20 client.17529 trim_cache size 0 max 16384 2024-09-11T23:44:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.975+0000 7fd927cac740 1 Processor -- start 2024-09-11T23:44:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.975+0000 7fd927cac740 1 -- start start 2024-09-11T23:44:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.976+0000 7fd927cac740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dcae560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.976+0000 7fd927cac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 0x559a5dcb44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.976+0000 7fd927cac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dccc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.976+0000 7fd927cac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201b3ab0 con 0x559a5dcb6cb0 2024-09-11T23:44:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.976+0000 7fd927cac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201b55f0 con 0x559a5dcac410 2024-09-11T23:44:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.976+0000 7fd927cac740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201b4180 con 0x559a5dcabaf0 2024-09-11T23:44:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.976+0000 7fd91ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dccc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.976+0000 7fd91ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dccc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47298/0 (socket says 172.21.15.133:47298) 2024-09-11T23:44:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.976+0000 7fd91ed9d640 1 -- 172.21.15.133:0/2793487128 learned_addr learned my addr 172.21.15.133:0/2793487128 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.976+0000 7fd91dd9b640 1 --2- 172.21.15.133:0/2793487128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 0x559a5dcb44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.977+0000 7fd91ed9d640 1 -- 172.21.15.133:0/2793487128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 msgr2=0x559a5dcae560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:44:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.977+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/2793487128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dcae560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.977+0000 7fd91ed9d640 1 -- 172.21.15.133:0/2793487128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 msgr2=0x559a5dcb44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.977+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/2793487128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 0x559a5dcb44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.977+0000 7fd91ed9d640 1 -- 172.21.15.133:0/2793487128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a5dcab8a0 con 0x559a5dcb6cb0 2024-09-11T23:44:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.977+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/2793487128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dccc020 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd9100052d0 tx=0x7fd910013f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0c5d9e037c50bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.977+0000 7fd907fff640 1 -- 172.21.15.133:0/2793487128 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910033040 con 0x559a5dcb6cb0 2024-09-11T23:44:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.977+0000 7fd907fff640 1 -- 172.21.15.133:0/2793487128 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd910011650 con 0x559a5dcb6cb0 2024-09-11T23:44:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.977+0000 7fd907fff640 1 -- 172.21.15.133:0/2793487128 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91003d080 con 0x559a5dcb6cb0 2024-09-11T23:44:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.978+0000 7fd927cac740 1 -- 172.21.15.133:0/2793487128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 msgr2=0x559a5dccc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.978+0000 7fd927cac740 1 --2- 172.21.15.133:0/2793487128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dccc020 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd9100052d0 tx=0x7fd910013f80 comp rx=0 tx=0).stop 2024-09-11T23:44:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.978+0000 7fd927cac740 1 -- 172.21.15.133:0/2793487128 shutdown_connections 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.978+0000 7fd927cac740 1 --2- 172.21.15.133:0/2793487128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcb6cb0 0x559a5dccc020 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.978+0000 7fd927cac740 1 --2- 172.21.15.133:0/2793487128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcac410 0x559a5dcb44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.978+0000 7fd927cac740 1 --2- 172.21.15.133:0/2793487128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dcae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.978+0000 7fd927cac740 1 -- 172.21.15.133:0/2793487128 >> 172.21.15.133:0/2793487128 conn(0x559a5dca8060 msgr2=0x559a5dcb5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.978+0000 7fd927cac740 1 -- 172.21.15.133:0/2793487128 shutdown_connections 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.978+0000 7fd927cac740 1 -- 172.21.15.133:0/2793487128 wait complete. 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.979+0000 7fd927cac740 1 -- start start 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.979+0000 7fd927cac740 1 Processor -- start 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd927cac740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dccc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd927cac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcac410 0x559a5dccc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd927cac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcb6cb0 0x559a5dcc9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd927cac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201b4f20 con 0x559a5dcac410 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd927cac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201b4850 con 0x559a5dcb6cb0 2024-09-11T23:44:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd927cac740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201b1ab0 con 0x559a5dcabaf0 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd927cac740 10 client.0 _ll_register_callbacks cb 0x7fd918005f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd91dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcac410 0x559a5dccc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd91dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcac410 0x559a5dccc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47312/0 (socket says 172.21.15.133:47312) 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd91dd9b640 1 -- 172.21.15.133:0/1295667892 learned_addr learned my addr 172.21.15.133:0/1295667892 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd91e59c640 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dccc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.980+0000 7fd91ed9d640 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcb6cb0 0x559a5dcc9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.981+0000 7fd91dd9b640 1 -- 172.21.15.133:0/1295667892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 msgr2=0x559a5dccc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.981+0000 7fd91dd9b640 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dccc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.981+0000 7fd91dd9b640 1 -- 172.21.15.133:0/1295667892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcb6cb0 msgr2=0x559a5dcc9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.981+0000 7fd91dd9b640 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcb6cb0 0x559a5dcc9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.981+0000 7fd91dd9b640 1 -- 172.21.15.133:0/1295667892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a5dca32d0 con 0x559a5dcac410 2024-09-11T23:44:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.981+0000 7fd91dd9b640 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcac410 0x559a5dccc950 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd9180edf50 tx=0x7fd91802add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3052d338da07ee58 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.981+0000 7fd905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.981+0000 7fd905ffb640 1 -- 172.21.15.133:0/1295667892 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9180d9020 con 0x559a5dcac410 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.981+0000 7fd905ffb640 1 -- 172.21.15.133:0/1295667892 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fd9180c54a0 con 0x559a5dcac410 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.981+0000 7fd927cac740 1 -- 172.21.15.133:0/1295667892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd910011650 con 0x559a5dcac410 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.982+0000 7fd905ffb640 1 -- 172.21.15.133:0/1295667892 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd9180ee260 con 0x559a5dcac410 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.982+0000 7fd927cac740 10 client.17535 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.982+0000 7fd927cac740 1 -- 172.21.15.133:0/1295667892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a5dd31040 con 0x559a5dcac410 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.982+0000 7fd905ffb640 1 -- 172.21.15.133:0/1295667892 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd918036040 con 0x559a5dcac410 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.982+0000 7fd905ffb640 1 -- 172.21.15.133:0/1295667892 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd9180c5b80 con 0x559a5dcac410 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.982+0000 7fd927cac740 10 client.17535 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.982+0000 7fd927cac740 10 client.17535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.982+0000 7fd927cac740 20 client.17535 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.982+0000 7fd927cac740 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559a5dcc6c10 0x559a5dd337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.983+0000 7fd927cac740 4 client.17535 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:44:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.983+0000 7fd927cac740 1 -- 172.21.15.133:0/1295667892 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd910011650 con 0x559a5dcc6c10 2024-09-11T23:44:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.985+0000 7fd91e59c640 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559a5dcc6c10 0x559a5dd337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.990+0000 7fd905ffb640 10 client.17535 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:52.990+0000 7fd91e59c640 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559a5dcc6c10 0x559a5dd337e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d98152f58cd90b0f server_cookie=ddda82a872097243 in_seq=0 out_seq=0 2024-09-11T23:44:53.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.036+0000 7f87bd7fa640 1 -- 172.21.15.133:0/3887717201 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ec6a4a3d20 con 0x55ec6a5ce830 2024-09-11T23:44:53.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.036+0000 7f87bd7fa640 10 client.17517 handle_command_reply: tid=0 2024-09-11T23:44:53.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.036+0000 7f87de596740 1 client.17517 shutdown 2024-09-11T23:44:53.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.036+0000 7f87de596740 1 -- 172.21.15.133:0/3887717201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 msgr2=0x55ec6a556b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:53.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.036+0000 7f87de596740 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a556b20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f87d0008010 tx=0x7f87d0014890 comp rx=0 tx=0).stop 2024-09-11T23:44:53.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:52 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:44:53.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:52 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:44:53.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:52 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:44:53.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:52 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:44:53.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.037+0000 7f87de596740 1 -- 172.21.15.133:0/3887717201 shutdown_connections 2024-09-11T23:44:53.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.037+0000 7f87de596740 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55ec6a5ce830 0x55ec6a5d0c20 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.037+0000 7f87de596740 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec6a541ae0 0x55ec6a5575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.037+0000 7f87de596740 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec6a537220 0x55ec6a557060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.037+0000 7f87de596740 1 --2- 172.21.15.133:0/3887717201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec6a536900 0x55ec6a556b20 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.037+0000 7f87bd7fa640 0 client.17517 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:53.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.037+0000 7f87de596740 1 -- 172.21.15.133:0/3887717201 >> 172.21.15.133:0/3887717201 conn(0x55ec6a532e90 msgr2=0x55ec6a5317e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:53.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.037+0000 7f87de596740 1 -- 172.21.15.133:0/3887717201 shutdown_connections 2024-09-11T23:44:53.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.037+0000 7f87de596740 1 -- 172.21.15.133:0/3887717201 wait complete. 2024-09-11T23:44:53.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.037+0000 7f87de596740 20 client.17517 trim_cache size 0 max 16384 2024-09-11T23:44:53.039 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:44:53.039 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:44:53.039 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:44:53.039 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 30, 2024-09-11T23:44:53.039 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:44:53.039 INFO:teuthology.orchestra.run.smithi133.stdout: "b2f50c87": { 2024-09-11T23:44:53.040 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 30, 2024-09-11T23:44:53.040 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:44:53.040 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:44:53.040 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:44:53.040 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:44:53.040 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:44:53.040 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:44:53.040 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 139.4, 2024-09-11T23:44:53.040 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:44:53.040 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:44:53.040 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:44:53.041 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:44:53.041 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:44:53.041 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:44:53.041 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:53.041 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:53.041 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:53.041 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:53.041 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:53.042 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:44:53.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.041+0000 7f87daf5b640 1 -- 172.21.15.133:0/121375883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87c00a2e50 msgr2=0x7f87c00a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.041+0000 7f87daf5b640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87c00a2e50 0x7f87c00a5310 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55ec6a533cf0 tx=0x7f87d0008040 comp rx=0 tx=0).stop 2024-09-11T23:44:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.041+0000 7f87daf5b640 1 -- 172.21.15.133:0/121375883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87cc0a5150 msgr2=0x7f87cc143b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.041+0000 7f87daf5b640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87cc0a5150 0x7f87cc143b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f87d40679d0 tx=0x7f87d406eeb0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.041+0000 7f87daf5b640 1 -- 172.21.15.133:0/121375883 shutdown_connections 2024-09-11T23:44:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.041+0000 7f87daf5b640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f87c00a2e50 0x7f87c00a5310 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.041+0000 7f87daf5b640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87cc0b7550 0x7f87cc147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.041+0000 7f87daf5b640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87cc0a5150 0x7f87cc143b00 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f87d40679d0 tx=0x7f87d406eeb0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.041+0000 7f87daf5b640 1 --2- 172.21.15.133:0/121375883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87cc0a4830 0x7f87cc1435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.041+0000 7f87daf5b640 1 -- 172.21.15.133:0/121375883 >> 172.21.15.133:0/121375883 conn(0x7f87cc01a0f0 msgr2=0x7f87cc0b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.042+0000 7f87daf5b640 1 -- 172.21.15.133:0/121375883 shutdown_connections 2024-09-11T23:44:53.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:53.042+0000 7f87daf5b640 1 -- 172.21.15.133:0/121375883 wait complete. 2024-09-11T23:44:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.079+0000 7fd905ffb640 1 -- 172.21.15.133:0/1295667892 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fd910011650 con 0x559a5dcc6c10 2024-09-11T23:44:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.079+0000 7fd905ffb640 10 client.17535 handle_command_reply: tid=0 2024-09-11T23:44:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.080+0000 7fd927cac740 1 client.17535 shutdown 2024-09-11T23:44:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd927cac740 1 -- 172.21.15.133:0/1295667892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcac410 msgr2=0x559a5dccc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd927cac740 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcac410 0x559a5dccc950 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd9180edf50 tx=0x7fd91802add0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd927cac740 1 -- 172.21.15.133:0/1295667892 shutdown_connections 2024-09-11T23:44:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd927cac740 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559a5dcc6c10 0x559a5dd337e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd927cac740 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a5dcb6cb0 0x559a5dcc9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd927cac740 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559a5dcac410 0x559a5dccc950 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd927cac740 1 --2- 172.21.15.133:0/1295667892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a5dcabaf0 0x559a5dccc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd927cac740 1 -- 172.21.15.133:0/1295667892 >> 172.21.15.133:0/1295667892 conn(0x559a5dca8060 msgr2=0x559a5dca69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd905ffb640 0 client.17535 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd927cac740 1 -- 172.21.15.133:0/1295667892 shutdown_connections 2024-09-11T23:44:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd927cac740 1 -- 172.21.15.133:0/1295667892 wait complete. 2024-09-11T23:44:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.081+0000 7fd927cac740 20 client.17535 trim_cache size 0 max 16384 2024-09-11T23:44:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:44:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:44:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:44:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:44:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.082+0000 7fd924f11640 1 -- 172.21.15.133:0/962008237 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd9080a2e20 msgr2=0x7fd9080a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.082+0000 7fd924f11640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd9080a2e20 0x7fd9080a52e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd9100136f0 tx=0x7fd91000e940 comp rx=0 tx=0).stop 2024-09-11T23:44:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.082+0000 7fd924f11640 1 -- 172.21.15.133:0/962008237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd92010d490 msgr2=0x7fd9201a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.082+0000 7fd924f11640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd92010d490 0x7fd9201a4bf0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd9180098d0 tx=0x7fd918031a00 comp rx=0 tx=0).stop 2024-09-11T23:44:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.083+0000 7fd924f11640 1 -- 172.21.15.133:0/962008237 shutdown_connections 2024-09-11T23:44:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.083+0000 7fd924f11640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd9080a2e20 0x7fd9080a52e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.083+0000 7fd924f11640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd92019e770 0x7fd92019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.083+0000 7fd924f11640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd92010d490 0x7fd9201a4bf0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.083+0000 7fd924f11640 1 --2- 172.21.15.133:0/962008237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108340 0x7fd9201a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.083+0000 7fd924f11640 1 -- 172.21.15.133:0/962008237 >> 172.21.15.133:0/962008237 conn(0x7fd920077a40 msgr2=0x7fd920078560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.083+0000 7fd924f11640 1 -- 172.21.15.133:0/962008237 shutdown_connections 2024-09-11T23:44:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:44:53.083+0000 7fd924f11640 1 -- 172.21.15.133:0/962008237 wait complete. 2024-09-11T23:44:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:52 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:44:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:52 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:44:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:52 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:44:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:52 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:44:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:52 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:44:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:52 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:44:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:52 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:44:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:52 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:44:53.315 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:44:53.796 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b2f50c87', quiesce took 8.4 seconds. Will release after: 10.999999999999998 2024-09-11T23:44:54.016 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:53 smithi133 ceph-mon[31430]: pgmap v251: 97 pgs: 97 active+clean; 70 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 712 MiB/s wr, 2.17k op/s 2024-09-11T23:44:54.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:53 smithi138 ceph-mon[35142]: pgmap v251: 97 pgs: 97 active+clean; 70 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 712 MiB/s wr, 2.17k op/s 2024-09-11T23:44:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.202+0000 7f29b1a25640 1 -- 172.21.15.133:0/1636934102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ac102570 msgr2=0x7f29ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.202+0000 7f29b1a25640 1 --2- 172.21.15.133:0/1636934102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ac102570 0x7f29ac1029b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009900 tx=0x7f29a002a480 comp rx=0 tx=0).stop 2024-09-11T23:44:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.203+0000 7f29b1a25640 1 -- 172.21.15.133:0/1636934102 shutdown_connections 2024-09-11T23:44:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.203+0000 7f29b1a25640 1 --2- 172.21.15.133:0/1636934102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac102ef0 0x7f29ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.203+0000 7f29b1a25640 1 --2- 172.21.15.133:0/1636934102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ac102570 0x7f29ac1029b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.203+0000 7f29b1a25640 1 --2- 172.21.15.133:0/1636934102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac100e20 0x7f29ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.203+0000 7f29b1a25640 1 -- 172.21.15.133:0/1636934102 >> 172.21.15.133:0/1636934102 conn(0x7f29ac0fc7d0 msgr2=0x7f29ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:54.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.203+0000 7f29b1a25640 1 -- 172.21.15.133:0/1636934102 shutdown_connections 2024-09-11T23:44:54.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.203+0000 7f29b1a25640 1 -- 172.21.15.133:0/1636934102 wait complete. 2024-09-11T23:44:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.203+0000 7f29b1a25640 1 Processor -- start 2024-09-11T23:44:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.203+0000 7f29b1a25640 1 -- start start 2024-09-11T23:44:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29b1a25640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac100e20 0x7f29ac1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29b1a25640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac102570 0x7f29ac1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29b1a25640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ac102ef0 0x7f29ac19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29b1a25640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac110f80 con 0x7f29ac102ef0 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29b1a25640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac110e00 con 0x7f29ac102570 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29b1a25640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac111100 con 0x7f29ac100e20 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ac102ef0 0x7f29ac19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29aa7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac102570 0x7f29ac1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ac102ef0 0x7f29ac19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47326/0 (socket says 172.21.15.133:47326) 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29aa7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac102570 0x7f29ac1977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58272/0 (socket says 172.21.15.133:58272) 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29ab7fe640 1 -- 172.21.15.133:0/2763664191 learned_addr learned my addr 172.21.15.133:0/2763664191 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29aaffd640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac100e20 0x7f29ac1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29aa7fc640 1 -- 172.21.15.133:0/2763664191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac100e20 msgr2=0x7f29ac1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac100e20 0x7f29ac1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29aa7fc640 1 -- 172.21.15.133:0/2763664191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ac102ef0 msgr2=0x7f29ac19bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:54.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ac102ef0 0x7f29ac19bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29aa7fc640 1 -- 172.21.15.133:0/2763664191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac19c260 con 0x7f29ac102570 2024-09-11T23:44:54.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29ab7fe640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ac102ef0 0x7f29ac19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:54.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac102570 0x7f29ac1977f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f29a0006010 tx=0x7f29a0003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:54.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.204+0000 7f29b0a23640 1 -- 172.21.15.133:0/2763664191 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a003f070 con 0x7f29ac102570 2024-09-11T23:44:54.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.205+0000 7f29b0a23640 1 -- 172.21.15.133:0/2763664191 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29a0031c20 con 0x7f29ac102570 2024-09-11T23:44:54.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.205+0000 7f29b1a25640 1 -- 172.21.15.133:0/2763664191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ac19c4f0 con 0x7f29ac102570 2024-09-11T23:44:54.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.205+0000 7f29b0a23640 1 -- 172.21.15.133:0/2763664191 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0035030 con 0x7f29ac102570 2024-09-11T23:44:54.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.205+0000 7f29b1a25640 1 -- 172.21.15.133:0/2763664191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ac069f70 con 0x7f29ac102570 2024-09-11T23:44:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29b0a23640 1 -- 172.21.15.133:0/2763664191 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f29a0042070 con 0x7f29ac102570 2024-09-11T23:44:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29b3dfe740 1 Processor -- start 2024-09-11T23:44:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29b3dfe740 1 -- start start 2024-09-11T23:44:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d44a0 0x55da727d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d4dc0 0x55da727df140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55da7273a2e0 con 0x55da727df680 2024-09-11T23:44:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da726e1ce0 con 0x55da727d4dc0 2024-09-11T23:44:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da727362a0 con 0x55da727d44a0 2024-09-11T23:44:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29aa7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d44a0 0x55da727d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d4dc0 0x55da727df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29aa7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727e1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47330/0 (socket says 172.21.15.133:47330) 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.207+0000 7f29aa7fc640 1 -- 172.21.15.133:0/4026233563 learned_addr learned my addr 172.21.15.133:0/4026233563 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f29b0a23640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f29940a2cf0 0x7f29940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f29ab7fe640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f29940a2cf0 0x7f29940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f29b0a23640 1 -- 172.21.15.133:0/2763664191 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f29a00efc60 con 0x7f29ac102570 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f29aa7fc640 1 -- 172.21.15.133:0/4026233563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d44a0 msgr2=0x55da727d4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/4026233563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d44a0 0x55da727d4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f29aa7fc640 1 -- 172.21.15.133:0/4026233563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d4dc0 msgr2=0x55da727df140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/4026233563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d4dc0 0x55da727df140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f29aa7fc640 1 -- 172.21.15.133:0/4026233563 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da726c6440 con 0x55da727df680 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f29aaffd640 1 --2- 172.21.15.133:0/4026233563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d4dc0 0x55da727df140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/4026233563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727e1a70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f29a00b8620 tx=0x7f29a002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c595bee4be1b84d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:54.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f2991ffb640 1 -- 172.21.15.133:0/4026233563 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0049070 con 0x55da727df680 2024-09-11T23:44:54.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f29ab7fe640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f29940a2cf0 0x7f29940a51b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c510 tx=0x7f29a4004ab0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:54.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f2991ffb640 1 -- 172.21.15.133:0/4026233563 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29a0031dd0 con 0x55da727df680 2024-09-11T23:44:54.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.208+0000 7f2991ffb640 1 -- 172.21.15.133:0/4026233563 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00b88d0 con 0x55da727df680 2024-09-11T23:44:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.209+0000 7f29b3dfe740 1 -- 172.21.15.133:0/4026233563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 msgr2=0x55da727e1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.209+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/4026233563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727e1a70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f29a00b8620 tx=0x7f29a002a480 comp rx=0 tx=0).stop 2024-09-11T23:44:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.209+0000 7f29b3dfe740 1 -- 172.21.15.133:0/4026233563 shutdown_connections 2024-09-11T23:44:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.209+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/4026233563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727e1a70 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.209+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/4026233563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d4dc0 0x55da727df140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.209+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/4026233563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d44a0 0x55da727d4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.209+0000 7f29b3dfe740 1 -- 172.21.15.133:0/4026233563 >> 172.21.15.133:0/4026233563 conn(0x55da727d0a30 msgr2=0x55da727d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.209+0000 7f29b3dfe740 1 -- 172.21.15.133:0/4026233563 shutdown_connections 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.209+0000 7f29b3dfe740 1 -- 172.21.15.133:0/4026233563 wait complete. 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.209+0000 7f29b3dfe740 1 -- start start 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29b3dfe740 1 Processor -- start 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727d44a0 0x55da727df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727dc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727df680 0x55da727dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55da727e2010 con 0x55da727d44a0 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da726f5050 con 0x55da727df680 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da727e2190 con 0x55da727d4dc0 2024-09-11T23:44:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29b3dfe740 10 client.0 _ll_register_callbacks cb 0x55da72735160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29aaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727d44a0 0x55da727df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29aaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727d44a0 0x55da727df0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47338/0 (socket says 172.21.15.133:47338) 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29aaffd640 1 -- 172.21.15.133:0/2248454883 learned_addr learned my addr 172.21.15.133:0/2248454883 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29ab7fe640 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727df680 0x55da727dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727dc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29aaffd640 1 -- 172.21.15.133:0/2248454883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 msgr2=0x55da727dc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29aaffd640 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727dc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29aaffd640 1 -- 172.21.15.133:0/2248454883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727df680 msgr2=0x55da727dcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29aaffd640 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727df680 0x55da727dcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.210+0000 7f29aaffd640 1 -- 172.21.15.133:0/2248454883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da724ab3c0 con 0x55da727d44a0 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f29aaffd640 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727d44a0 0x55da727df0d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f299c010a50 tx=0x7f299c010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d43ad02f719e1b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f298affd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f298affd640 1 -- 172.21.15.133:0/2248454883 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c01e060 con 0x55da727d44a0 2024-09-11T23:44:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f298affd640 1 -- 172.21.15.133:0/2248454883 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f299c00fbd0 con 0x55da727d44a0 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f29b3dfe740 1 -- 172.21.15.133:0/2248454883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da72747850 con 0x55da727d44a0 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f298affd640 1 -- 172.21.15.133:0/2248454883 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f299c00fdd0 con 0x55da727d44a0 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f29b3dfe740 10 client.17547 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f29b3dfe740 1 -- 172.21.15.133:0/2248454883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55da727edab0 con 0x55da727d44a0 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f298affd640 1 -- 172.21.15.133:0/2248454883 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55da72747850 con 0x55da727d44a0 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f298affd640 1 -- 172.21.15.133:0/2248454883 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f299c021520 con 0x55da727d44a0 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f29b3dfe740 10 client.17547 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f29b3dfe740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f29b3dfe740 20 client.17547 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55da727eef70 0x55da727f1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f29b3dfe740 4 client.17547 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.211+0000 7f29b3dfe740 1 -- 172.21.15.133:0/2248454883 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da72741d60 con 0x55da727eef70 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.212+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55da727eef70 0x55da727f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.212+0000 7f298affd640 10 client.17547 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.212+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55da727eef70 0x55da727f1360 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7f8bd5b57072fd server_cookie=8a6883817d83a173 in_seq=0 out_seq=0 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f298affd640 1 -- 172.21.15.133:0/2248454883 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da72741d60 con 0x55da727eef70 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f298affd640 10 client.17547 handle_command_reply: tid=0 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 1 client.17547 shutdown 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 1 -- 172.21.15.133:0/2248454883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727d44a0 msgr2=0x55da727df0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727d44a0 0x55da727df0d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f299c010a50 tx=0x7f299c010f10 comp rx=0 tx=0).stop 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 1 -- 172.21.15.133:0/2248454883 shutdown_connections 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55da727eef70 0x55da727f1360 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727df680 0x55da727dcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727dc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/2248454883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727d44a0 0x55da727df0d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 1 -- 172.21.15.133:0/2248454883 >> 172.21.15.133:0/2248454883 conn(0x55da727d0a30 msgr2=0x55da727d6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f298affd640 0 client.17547 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 1 -- 172.21.15.133:0/2248454883 shutdown_connections 2024-09-11T23:44:54.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 1 -- 172.21.15.133:0/2248454883 wait complete. 2024-09-11T23:44:54.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.213+0000 7f29b3dfe740 20 client.17547 trim_cache size 0 max 16384 2024-09-11T23:44:54.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29b3dfe740 1 Processor -- start 2024-09-11T23:44:54.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29b3dfe740 1 -- start start 2024-09-11T23:44:54.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 0x55da727d77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727f7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac1ab3e0 con 0x55da727df680 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac1acf20 con 0x55da727d44a0 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac1abab0 con 0x55da727d4dc0 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 0x55da727d77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47344/0 (socket says 172.21.15.133:47344) 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29ab7fe640 1 -- 172.21.15.133:0/3718875232 learned_addr learned my addr 172.21.15.133:0/3718875232 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/3718875232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.230+0000 7f29aaffd640 1 -- 172.21.15.133:0/3718875232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 msgr2=0x55da727f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29aaffd640 1 --2- 172.21.15.133:0/3718875232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727f6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29aaffd640 1 -- 172.21.15.133:0/3718875232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 msgr2=0x55da727f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29aaffd640 1 --2- 172.21.15.133:0/3718875232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29aaffd640 1 -- 172.21.15.133:0/3718875232 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da727db330 con 0x55da727d44a0 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/3718875232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29ab7fe640 1 --2- 172.21.15.133:0/3718875232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29aaffd640 1 --2- 172.21.15.133:0/3718875232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 0x55da727d77a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f299c021970 tx=0x7f299c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e7b0aa90606533d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29917fa640 1 -- 172.21.15.133:0/3718875232 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c00a340 con 0x55da727d44a0 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29917fa640 1 -- 172.21.15.133:0/3718875232 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f299c021b60 con 0x55da727d44a0 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29917fa640 1 -- 172.21.15.133:0/3718875232 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c004660 con 0x55da727d44a0 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29b3dfe740 1 -- 172.21.15.133:0/3718875232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 msgr2=0x55da727d77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.231+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/3718875232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 0x55da727d77a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f299c021970 tx=0x7f299c0058d0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 -- 172.21.15.133:0/3718875232 shutdown_connections 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/3718875232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da727f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/3718875232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/3718875232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 0x55da727d77a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 -- 172.21.15.133:0/3718875232 >> 172.21.15.133:0/3718875232 conn(0x55da727d0a30 msgr2=0x55da727d7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:54.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 -- 172.21.15.133:0/3718875232 shutdown_connections 2024-09-11T23:44:54.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 -- 172.21.15.133:0/3718875232 wait complete. 2024-09-11T23:44:54.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 -- start start 2024-09-11T23:44:54.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 Processor -- start 2024-09-11T23:44:54.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 0x55da727e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da72851c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac1ac850 con 0x55da727df680 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac1ac180 con 0x55da727d44a0 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac1a93e0 con 0x55da727d4dc0 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.232+0000 7f29b3dfe740 10 client.0 _ll_register_callbacks cb 0x7f299c00a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 0x55da727e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 0x55da727e29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58306/0 (socket says 172.21.15.133:58306) 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29aaffd640 1 -- 172.21.15.133:0/74900318 learned_addr learned my addr 172.21.15.133:0/74900318 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29ab7fe640 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da72851c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29aaffd640 1 -- 172.21.15.133:0/74900318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 msgr2=0x55da727e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29aaffd640 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29aaffd640 1 -- 172.21.15.133:0/74900318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 msgr2=0x55da72851c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29aaffd640 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da72851c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29aaffd640 1 -- 172.21.15.133:0/74900318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da727dce80 con 0x55da727d44a0 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29aaffd640 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 0x55da727e29e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f299c0218f0 tx=0x7f299c00fc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cdf5bb3647e1e20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f298affd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f298affd640 1 -- 172.21.15.133:0/74900318 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c01e070 con 0x55da727d44a0 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f298affd640 1 -- 172.21.15.133:0/74900318 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f299c021370 con 0x55da727d44a0 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29b3dfe740 1 -- 172.21.15.133:0/74900318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f299c021b60 con 0x55da727d44a0 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f298affd640 1 -- 172.21.15.133:0/74900318 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f299c03c050 con 0x55da727d44a0 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29b3dfe740 10 client.27062 fetch_fsmap learned FSMap version 30 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.233+0000 7f29b3dfe740 1 -- 172.21.15.133:0/74900318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55da72859450 con 0x55da727d44a0 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.234+0000 7f298affd640 1 -- 172.21.15.133:0/74900318 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f299c021b60 con 0x55da727d44a0 2024-09-11T23:44:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.234+0000 7f298affd640 1 -- 172.21.15.133:0/74900318 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f299c02b030 con 0x55da727d44a0 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.234+0000 7f29b3dfe740 10 client.27062 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.234+0000 7f29b3dfe740 10 client.27062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.234+0000 7f29b3dfe740 20 client.27062 populate_metadata read hostname 'smithi133' 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.234+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55da72859a60 0x55da7285be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.234+0000 7f29b3dfe740 4 client.27062 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.234+0000 7f29b3dfe740 1 -- 172.21.15.133:0/74900318 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da726be210 con 0x55da72859a60 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.234+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55da72859a60 0x55da7285be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f298affd640 10 client.27062 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29aa7fc640 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55da72859a60 0x55da7285be50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d795ecea0c58ea server_cookie=579cd6b2153485eb in_seq=0 out_seq=0 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f298affd640 1 -- 172.21.15.133:0/74900318 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55da726be210 con 0x55da72859a60 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f298affd640 10 client.27062 handle_command_reply: tid=0 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 1 client.27062 shutdown 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 1 -- 172.21.15.133:0/74900318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 msgr2=0x55da727e29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 0x55da727e29e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f299c0218f0 tx=0x7f299c00fc40 comp rx=0 tx=0).stop 2024-09-11T23:44:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 1 -- 172.21.15.133:0/74900318 shutdown_connections 2024-09-11T23:44:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55da72859a60 0x55da7285be50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da727df680 0x55da72851c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da727d4dc0 0x55da727e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f298affd640 0 client.27062 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:44:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 1 --2- 172.21.15.133:0/74900318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da727d44a0 0x55da727e29e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 1 -- 172.21.15.133:0/74900318 >> 172.21.15.133:0/74900318 conn(0x55da727d0a30 msgr2=0x55da727d78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 1 -- 172.21.15.133:0/74900318 shutdown_connections 2024-09-11T23:44:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 1 -- 172.21.15.133:0/74900318 wait complete. 2024-09-11T23:44:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.235+0000 7f29b3dfe740 20 client.27062 trim_cache size 0 max 16384 2024-09-11T23:44:54.243 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:44:54.244 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:44:54.244 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-09-11T23:44:54.244 INFO:teuthology.orchestra.run.smithi133.stdout: "bc2d33af-99bc-4ba9-80aa-f573d40a624e": { 2024-09-11T23:44:54.244 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-09-11T23:44:54.244 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "bc2d33af-99bc-4ba9-80aa-f573d40a624e", 2024-09-11T23:44:54.244 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2024-09-11T23:44:54.244 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:54.244 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:44:54.244 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:44:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 -- 172.21.15.133:0/2763664191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f29940a2cf0 msgr2=0x7f29940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f29940a2cf0 0x7f29940a51b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c510 tx=0x7f29a4004ab0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 -- 172.21.15.133:0/2763664191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac102570 msgr2=0x7f29ac1977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:44:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac102570 0x7f29ac1977f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f29a0006010 tx=0x7f29a0003ba0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 -- 172.21.15.133:0/2763664191 shutdown_connections 2024-09-11T23:44:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f29940a2cf0 0x7f29940a51b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ac102ef0 0x7f29ac19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac102570 0x7f29ac1977f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 --2- 172.21.15.133:0/2763664191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac100e20 0x7f29ac1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:44:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 -- 172.21.15.133:0/2763664191 >> 172.21.15.133:0/2763664191 conn(0x7f29ac0fc7d0 msgr2=0x7f29ac106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:44:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 -- 172.21.15.133:0/2763664191 shutdown_connections 2024-09-11T23:44:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:44:54.236+0000 7f29b1a25640 1 -- 172.21.15.133:0/2763664191 wait complete. 2024-09-11T23:44:54.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:53 smithi178 ceph-mon[35882]: pgmap v251: 97 pgs: 97 active+clean; 70 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 712 MiB/s wr, 2.17k op/s 2024-09-11T23:44:54.917 INFO:tasks.cephfs.filesystem:scrub status for tag:bc2d33af-99bc-4ba9-80aa-f573d40a624e - {'path': '/', 'tag': 'bc2d33af-99bc-4ba9-80aa-f573d40a624e', 'options': 'recursive,force'} 2024-09-11T23:44:55.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:54 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:44:55.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:54 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:44:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:54 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:44:56.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:55 smithi138 ceph-mon[35142]: pgmap v252: 97 pgs: 97 active+clean; 70 GiB data, 246 GiB used, 826 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 669 MiB/s wr, 1.80k op/s 2024-09-11T23:44:56.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:55 smithi133 ceph-mon[31430]: pgmap v252: 97 pgs: 97 active+clean; 70 GiB data, 246 GiB used, 826 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 669 MiB/s wr, 1.80k op/s 2024-09-11T23:44:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:55 smithi178 ceph-mon[35882]: pgmap v252: 97 pgs: 97 active+clean; 70 GiB data, 246 GiB used, 826 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 669 MiB/s wr, 1.80k op/s 2024-09-11T23:44:58.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:44:58 smithi138 ceph-mon[35142]: pgmap v253: 97 pgs: 97 active+clean; 70 GiB data, 246 GiB used, 827 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 551 MiB/s wr, 1.35k op/s 2024-09-11T23:44:58.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:44:58 smithi133 ceph-mon[31430]: pgmap v253: 97 pgs: 97 active+clean; 70 GiB data, 246 GiB used, 827 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 551 MiB/s wr, 1.35k op/s 2024-09-11T23:44:59.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:44:58 smithi178 ceph-mon[35882]: pgmap v253: 97 pgs: 97 active+clean; 70 GiB data, 246 GiB used, 827 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 551 MiB/s wr, 1.35k op/s 2024-09-11T23:45:00.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:00 smithi138 ceph-mon[35142]: pgmap v254: 97 pgs: 97 active+clean; 70 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 546 MiB/s wr, 1.33k op/s 2024-09-11T23:45:00.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:00 smithi133 ceph-mon[31430]: pgmap v254: 97 pgs: 97 active+clean; 70 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 546 MiB/s wr, 1.33k op/s 2024-09-11T23:45:01.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:00 smithi178 ceph-mon[35882]: pgmap v254: 97 pgs: 97 active+clean; 70 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 546 MiB/s wr, 1.33k op/s 2024-09-11T23:45:02.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:02 smithi138 ceph-mon[35142]: pgmap v255: 97 pgs: 97 active+clean; 70 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 369 MiB/s wr, 592 op/s 2024-09-11T23:45:02.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:02 smithi133 ceph-mon[31430]: pgmap v255: 97 pgs: 97 active+clean; 70 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 369 MiB/s wr, 592 op/s 2024-09-11T23:45:03.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:02 smithi178 ceph-mon[35882]: pgmap v255: 97 pgs: 97 active+clean; 70 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 369 MiB/s wr, 592 op/s 2024-09-11T23:45:04.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:45:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:04 smithi138 ceph-mon[35142]: pgmap v256: 97 pgs: 97 active+clean; 70 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 304 MiB/s wr, 339 op/s 2024-09-11T23:45:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:04 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:04 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:04 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:04.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:04 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:45:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:04 smithi133 ceph-mon[31430]: pgmap v256: 97 pgs: 97 active+clean; 70 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 304 MiB/s wr, 339 op/s 2024-09-11T23:45:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:04 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:04 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:04 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:04 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:45:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:04 smithi178 ceph-mon[35882]: pgmap v256: 97 pgs: 97 active+clean; 70 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 304 MiB/s wr, 339 op/s 2024-09-11T23:45:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:04 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:45:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:04 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:45:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:04 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:45:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:04 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:45:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:05.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.546+0000 7fd7affd8640 1 -- 172.21.15.133:0/4032926840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a8101df0 msgr2=0x7fd7a81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:05.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.546+0000 7fd7affd8640 1 --2- 172.21.15.133:0/4032926840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a8101df0 0x7fd7a81021d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009900 tx=0x7fd7a402a410 comp rx=0 tx=0).stop 2024-09-11T23:45:05.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.546+0000 7fd7affd8640 1 -- 172.21.15.133:0/4032926840 shutdown_connections 2024-09-11T23:45:05.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.546+0000 7fd7affd8640 1 --2- 172.21.15.133:0/4032926840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a8106f20 0x7fd7a810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:05.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.546+0000 7fd7affd8640 1 --2- 172.21.15.133:0/4032926840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7a81027a0 0x7fd7a81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:05.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.546+0000 7fd7affd8640 1 --2- 172.21.15.133:0/4032926840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a8101df0 0x7fd7a81021d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:05.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.546+0000 7fd7affd8640 1 -- 172.21.15.133:0/4032926840 >> 172.21.15.133:0/4032926840 conn(0x7fd7a80fd8f0 msgr2=0x7fd7a80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:05.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.546+0000 7fd7affd8640 1 -- 172.21.15.133:0/4032926840 shutdown_connections 2024-09-11T23:45:05.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.546+0000 7fd7affd8640 1 -- 172.21.15.133:0/4032926840 wait complete. 2024-09-11T23:45:05.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7affd8640 1 Processor -- start 2024-09-11T23:45:05.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7affd8640 1 -- start start 2024-09-11T23:45:05.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7affd8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a8101df0 0x7fd7a81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:05.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7affd8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7a81027a0 0x7fd7a81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:05.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7affd8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a8106f20 0x7fd7a819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:05.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7affd8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8111f80 con 0x7fd7a8106f20 2024-09-11T23:45:05.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7affd8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8111e00 con 0x7fd7a81027a0 2024-09-11T23:45:05.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7affd8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8112100 con 0x7fd7a8101df0 2024-09-11T23:45:05.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7ae575640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a8106f20 0x7fd7a819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:05.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7ae575640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a8106f20 0x7fd7a819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38970/0 (socket says 172.21.15.133:38970) 2024-09-11T23:45:05.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7ae575640 1 -- 172.21.15.133:0/2753835624 learned_addr learned my addr 172.21.15.133:0/2753835624 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:05.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7ad573640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7a81027a0 0x7fd7a81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:05.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7add74640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a8101df0 0x7fd7a81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:05.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7ae575640 1 -- 172.21.15.133:0/2753835624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a8101df0 msgr2=0x7fd7a81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7ae575640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a8101df0 0x7fd7a81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7ae575640 1 -- 172.21.15.133:0/2753835624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7a81027a0 msgr2=0x7fd7a81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7ae575640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7a81027a0 0x7fd7a81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7ae575640 1 -- 172.21.15.133:0/2753835624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a819d450 con 0x7fd7a8106f20 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.547+0000 7fd7add74640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a8101df0 0x7fd7a81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.548+0000 7fd7ae575640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a8106f20 0x7fd7a819cd70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009fd0 tx=0x7fd7a000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.548+0000 7fd79effd640 1 -- 172.21.15.133:0/2753835624 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0017070 con 0x7fd7a8106f20 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.548+0000 7fd79effd640 1 -- 172.21.15.133:0/2753835624 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7a0003e50 con 0x7fd7a8106f20 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.548+0000 7fd7affd8640 1 -- 172.21.15.133:0/2753835624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a819d740 con 0x7fd7a8106f20 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.548+0000 7fd79effd640 1 -- 172.21.15.133:0/2753835624 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a00106c0 con 0x7fd7a8106f20 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.548+0000 7fd7affd8640 1 -- 172.21.15.133:0/2753835624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a81a5020 con 0x7fd7a8106f20 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.549+0000 7fd7affd8640 1 -- 172.21.15.133:0/2753835624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a81021d0 con 0x7fd7a8106f20 2024-09-11T23:45:05.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.550+0000 7fd79effd640 1 -- 172.21.15.133:0/2753835624 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd7a0003970 con 0x7fd7a8106f20 2024-09-11T23:45:05.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.551+0000 7fd79effd640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd7940a2d70 0x7fd7940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:05.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.551+0000 7fd7add74640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd7940a2d70 0x7fd7940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:05.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.551+0000 7fd79effd640 1 -- 172.21.15.133:0/2753835624 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fd7a00cf0c0 con 0x7fd7a8106f20 2024-09-11T23:45:05.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.551+0000 7fd7add74640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd7940a2d70 0x7fd7940a5230 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4005ab0 tx=0x7fd7a4005a20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:05.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.555+0000 7fd79effd640 1 -- 172.21.15.133:0/2753835624 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7a009bd20 con 0x7fd7a8106f20 2024-09-11T23:45:05.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.771+0000 7fd7affd8640 1 -- 172.21.15.133:0/2753835624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7a810a160 con 0x7fd7a8106f20 2024-09-11T23:45:05.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.772+0000 7fd79effd640 1 -- 172.21.15.133:0/2753835624 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7fd7a009b620 con 0x7fd7a8106f20 2024-09-11T23:45:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:45:05.772 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:45:05.772 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:45:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 -- 172.21.15.133:0/2753835624 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd7940a2d70 msgr2=0x7fd7940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd7940a2d70 0x7fd7940a5230 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4005ab0 tx=0x7fd7a4005a20 comp rx=0 tx=0).stop 2024-09-11T23:45:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 -- 172.21.15.133:0/2753835624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a8106f20 msgr2=0x7fd7a819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a8106f20 0x7fd7a819cd70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009fd0 tx=0x7fd7a000b260 comp rx=0 tx=0).stop 2024-09-11T23:45:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 -- 172.21.15.133:0/2753835624 shutdown_connections 2024-09-11T23:45:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd7940a2d70 0x7fd7940a5230 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7a8106f20 0x7fd7a819cd70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7a81027a0 0x7fd7a81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 --2- 172.21.15.133:0/2753835624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a8101df0 0x7fd7a81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 -- 172.21.15.133:0/2753835624 >> 172.21.15.133:0/2753835624 conn(0x7fd7a80fd8f0 msgr2=0x7fd7a80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 -- 172.21.15.133:0/2753835624 shutdown_connections 2024-09-11T23:45:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:05.774+0000 7fd7affd8640 1 -- 172.21.15.133:0/2753835624 wait complete. 2024-09-11T23:45:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.177+0000 7fe4820e6640 1 -- 172.21.15.133:0/1275229772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c101350 msgr2=0x7fe47c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.177+0000 7fe4820e6640 1 --2- 172.21.15.133:0/1275229772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c101350 0x7fe47c10d880 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe474009cd0 tx=0x7fe47402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.177+0000 7fe4820e6640 1 -- 172.21.15.133:0/1275229772 shutdown_connections 2024-09-11T23:45:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.177+0000 7fe4820e6640 1 --2- 172.21.15.133:0/1275229772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c101350 0x7fe47c10d880 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.177+0000 7fe4820e6640 1 --2- 172.21.15.133:0/1275229772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe47c1009b0 0x7fe47c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.177+0000 7fe4820e6640 1 --2- 172.21.15.133:0/1275229772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe47c1069c0 0x7fe47c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.177+0000 7fe4820e6640 1 -- 172.21.15.133:0/1275229772 >> 172.21.15.133:0/1275229772 conn(0x7fe47c077990 msgr2=0x7fe47c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.177+0000 7fe4820e6640 1 -- 172.21.15.133:0/1275229772 shutdown_connections 2024-09-11T23:45:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.177+0000 7fe4820e6640 1 -- 172.21.15.133:0/1275229772 wait complete. 2024-09-11T23:45:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe4820e6640 1 Processor -- start 2024-09-11T23:45:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe4820e6640 1 -- start start 2024-09-11T23:45:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe4820e6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe47c1009b0 0x7fe47c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe4820e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c101350 0x7fe47c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe4820e6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe47c1069c0 0x7fe47c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe4820e6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c110e00 con 0x7fe47c101350 2024-09-11T23:45:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe4820e6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c110c80 con 0x7fe47c1009b0 2024-09-11T23:45:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe4820e6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c110f80 con 0x7fe47c1069c0 2024-09-11T23:45:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c101350 0x7fe47c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c101350 0x7fe47c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38982/0 (socket says 172.21.15.133:38982) 2024-09-11T23:45:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe47affd640 1 -- 172.21.15.133:0/405242066 learned_addr learned my addr 172.21.15.133:0/405242066 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe47b7fe640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe47c1009b0 0x7fe47c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.178+0000 7fe47bfff640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe47c1069c0 0x7fe47c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe47affd640 1 -- 172.21.15.133:0/405242066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe47c1069c0 msgr2=0x7fe47c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe47affd640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe47c1069c0 0x7fe47c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe47affd640 1 -- 172.21.15.133:0/405242066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe47c1009b0 msgr2=0x7fe47c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe47affd640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe47c1009b0 0x7fe47c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe47affd640 1 -- 172.21.15.133:0/405242066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c19d2e0 con 0x7fe47c101350 2024-09-11T23:45:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe47b7fe640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe47c1009b0 0x7fe47c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe47affd640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c101350 0x7fe47c198830 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe47000ea60 tx=0x7fe47000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe478ff9640 1 -- 172.21.15.133:0/405242066 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47000cde0 con 0x7fe47c101350 2024-09-11T23:45:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe4820e6640 1 -- 172.21.15.133:0/405242066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe47c19d5d0 con 0x7fe47c101350 2024-09-11T23:45:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe478ff9640 1 -- 172.21.15.133:0/405242066 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4700035f0 con 0x7fe47c101350 2024-09-11T23:45:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe478ff9640 1 -- 172.21.15.133:0/405242066 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470010670 con 0x7fe47c101350 2024-09-11T23:45:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.179+0000 7fe4820e6640 1 -- 172.21.15.133:0/405242066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe47c1a4e60 con 0x7fe47c101350 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe478ff9640 1 -- 172.21.15.133:0/405242066 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe4700037a0 con 0x7fe47c101350 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe484e81740 1 Processor -- start 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe484e81740 1 -- start start 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe484e81740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 0x55edb38dbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe484e81740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38eab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe484e81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb38ed4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe484e81740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55edb38402f0 con 0x55edb38dba10 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe484e81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55edb37e8270 con 0x55edb38eb0c0 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe484e81740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55edb383c2b0 con 0x55edb38dc330 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe478ff9640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe4640a2da0 0x7fe4640a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe47affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb38ed4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe47bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 0x55edb38dbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe47affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb38ed4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:36786/0 (socket says 172.21.15.133:36786) 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.182+0000 7fe47affd640 1 -- 172.21.15.133:0/4211348116 learned_addr learned my addr 172.21.15.133:0/4211348116 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe47b7fe640 1 --2- 172.21.15.133:0/4211348116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe47bfff640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe4640a2da0 0x7fe4640a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe478ff9640 1 -- 172.21.15.133:0/405242066 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fe4700cf6c0 con 0x7fe47c101350 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe47affd640 1 -- 172.21.15.133:0/4211348116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 msgr2=0x55edb38eab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe47affd640 1 --2- 172.21.15.133:0/4211348116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38eab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe47affd640 1 -- 172.21.15.133:0/4211348116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 msgr2=0x55edb38dbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe47affd640 1 --2- 172.21.15.133:0/4211348116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 0x55edb38dbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe47affd640 1 -- 172.21.15.133:0/4211348116 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edb37e7340 con 0x55edb38eb0c0 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe47affd640 1 --2- 172.21.15.133:0/4211348116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb38ed4b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe470098080 tx=0x7fe47006ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c480de8b3f24cf3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe47bfff640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe4640a2da0 0x7fe4640a5260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe474004ab0 tx=0x7fe474004a40 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe461ffb640 1 -- 172.21.15.133:0/4211348116 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47009bc20 con 0x55edb38eb0c0 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe461ffb640 1 -- 172.21.15.133:0/4211348116 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe47009ffd0 con 0x55edb38eb0c0 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.183+0000 7fe461ffb640 1 -- 172.21.15.133:0/4211348116 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4700a4400 con 0x55edb38eb0c0 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.184+0000 7fe484e81740 1 -- 172.21.15.133:0/4211348116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 msgr2=0x55edb38ed4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.184+0000 7fe484e81740 1 --2- 172.21.15.133:0/4211348116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb38ed4b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe470098080 tx=0x7fe47006ec90 comp rx=0 tx=0).stop 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.184+0000 7fe484e81740 1 -- 172.21.15.133:0/4211348116 shutdown_connections 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.184+0000 7fe484e81740 1 --2- 172.21.15.133:0/4211348116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb38ed4b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.184+0000 7fe484e81740 1 --2- 172.21.15.133:0/4211348116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.184+0000 7fe484e81740 1 --2- 172.21.15.133:0/4211348116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 0x55edb38dbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.184+0000 7fe484e81740 1 -- 172.21.15.133:0/4211348116 >> 172.21.15.133:0/4211348116 conn(0x55edb38d8060 msgr2=0x55edb38d8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.184+0000 7fe484e81740 1 -- 172.21.15.133:0/4211348116 shutdown_connections 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.184+0000 7fe484e81740 1 -- 172.21.15.133:0/4211348116 wait complete. 2024-09-11T23:45:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.184+0000 7fe484e81740 1 -- start start 2024-09-11T23:45:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe484e81740 1 Processor -- start 2024-09-11T23:45:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe484e81740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 0x55edb38e8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe484e81740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38e8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe484e81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb38e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe484e81740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55edb38edc90 con 0x55edb38dba10 2024-09-11T23:45:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe484e81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55edb37c9290 con 0x55edb38eb0c0 2024-09-11T23:45:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe484e81740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55edb38ede10 con 0x55edb38dc330 2024-09-11T23:45:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe484e81740 10 client.0 _ll_register_callbacks cb 0x55edb37fa140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 0x55edb38e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 0x55edb38e8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38996/0 (socket says 172.21.15.133:38996) 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47b7fe640 1 -- 172.21.15.133:0/1464205787 learned_addr learned my addr 172.21.15.133:0/1464205787 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47bfff640 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb38e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47affd640 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47bfff640 1 -- 172.21.15.133:0/1464205787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 msgr2=0x55edb38e8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47bfff640 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38e8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47bfff640 1 -- 172.21.15.133:0/1464205787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 msgr2=0x55edb38e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47bfff640 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 0x55edb38e8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47bfff640 1 -- 172.21.15.133:0/1464205787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edb3553210 con 0x55edb38eb0c0 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47b7fe640 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 0x55edb38e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.185+0000 7fe47bfff640 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb38e8a80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe474005e30 tx=0x7fe4740095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=958b1fc96b7f3ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe45affd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:45:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe45affd640 1 -- 172.21.15.133:0/1464205787 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4740305a0 con 0x55edb38eb0c0 2024-09-11T23:45:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe45affd640 1 -- 172.21.15.133:0/1464205787 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fe474030740 con 0x55edb38eb0c0 2024-09-11T23:45:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe484e81740 1 -- 172.21.15.133:0/1464205787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe47009ffd0 con 0x55edb38eb0c0 2024-09-11T23:45:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe45affd640 1 -- 172.21.15.133:0/1464205787 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe474030940 con 0x55edb38eb0c0 2024-09-11T23:45:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe484e81740 10 client.27080 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe484e81740 1 -- 172.21.15.133:0/1464205787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55edb361f620 con 0x55edb38eb0c0 2024-09-11T23:45:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe45affd640 1 -- 172.21.15.133:0/1464205787 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe47009ffd0 con 0x55edb38eb0c0 2024-09-11T23:45:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe45affd640 1 -- 172.21.15.133:0/1464205787 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe474048040 con 0x55edb38eb0c0 2024-09-11T23:45:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe484e81740 10 client.27080 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe484e81740 10 client.27080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe484e81740 20 client.27080 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.186+0000 7fe484e81740 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55edb38fb060 0x55edb38fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.187+0000 7fe484e81740 4 client.27080 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.187+0000 7fe484e81740 1 -- 172.21.15.133:0/1464205787 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edb384da20 con 0x55edb38fb060 2024-09-11T23:45:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.187+0000 7fe47b7fe640 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55edb38fb060 0x55edb38fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.187+0000 7fe45affd640 10 client.27080 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.187+0000 7fe47b7fe640 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55edb38fb060 0x55edb38fd450 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=187f3afe2ff7a3d9 server_cookie=21124824ea342d45 in_seq=0 out_seq=0 2024-09-11T23:45:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe45affd640 1 -- 172.21.15.133:0/1464205787 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edb384da20 con 0x55edb38fb060 2024-09-11T23:45:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe45affd640 10 client.27080 handle_command_reply: tid=0 2024-09-11T23:45:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 1 client.27080 shutdown 2024-09-11T23:45:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 1 -- 172.21.15.133:0/1464205787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 msgr2=0x55edb38e8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb38e8a80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe474005e30 tx=0x7fe4740095c0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 1 -- 172.21.15.133:0/1464205787 shutdown_connections 2024-09-11T23:45:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55edb38fb060 0x55edb38fd450 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb38e8a80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38e8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 1 --2- 172.21.15.133:0/1464205787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dba10 0x55edb38e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe45affd640 0 client.27080 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 1 -- 172.21.15.133:0/1464205787 >> 172.21.15.133:0/1464205787 conn(0x55edb38d8060 msgr2=0x55edb38d71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 1 -- 172.21.15.133:0/1464205787 shutdown_connections 2024-09-11T23:45:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 1 -- 172.21.15.133:0/1464205787 wait complete. 2024-09-11T23:45:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.188+0000 7fe484e81740 20 client.27080 trim_cache size 0 max 16384 2024-09-11T23:45:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.202+0000 7fe484e81740 1 Processor -- start 2024-09-11T23:45:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.202+0000 7fe484e81740 1 -- start start 2024-09-11T23:45:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.202+0000 7fe484e81740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dba10 0x55edb38e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.202+0000 7fe484e81740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dc330 0x55edb3902d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.202+0000 7fe484e81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb3903260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.202+0000 7fe484e81740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c1ab500 con 0x55edb38dc330 2024-09-11T23:45:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.202+0000 7fe484e81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c1ad040 con 0x55edb38eb0c0 2024-09-11T23:45:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.202+0000 7fe484e81740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c1abbd0 con 0x55edb38dba10 2024-09-11T23:45:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dc330 0x55edb3902d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dc330 0x55edb3902d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39008/0 (socket says 172.21.15.133:39008) 2024-09-11T23:45:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47affd640 1 -- 172.21.15.133:0/4154858067 learned_addr learned my addr 172.21.15.133:0/4154858067 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47bfff640 1 --2- 172.21.15.133:0/4154858067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb3903260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47b7fe640 1 --2- 172.21.15.133:0/4154858067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dba10 0x55edb38e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47affd640 1 -- 172.21.15.133:0/4154858067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dba10 msgr2=0x55edb38e9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47affd640 1 --2- 172.21.15.133:0/4154858067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dba10 0x55edb38e9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47affd640 1 -- 172.21.15.133:0/4154858067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 msgr2=0x55edb3903260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47affd640 1 --2- 172.21.15.133:0/4154858067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb3903260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47affd640 1 -- 172.21.15.133:0/4154858067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edb38fd7c0 con 0x55edb38dc330 2024-09-11T23:45:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47bfff640 1 --2- 172.21.15.133:0/4154858067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb3903260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe47affd640 1 --2- 172.21.15.133:0/4154858067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dc330 0x55edb3902d20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe470099ed0 tx=0x7fe470003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac65ab16dbf2f7ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe4617fa640 1 -- 172.21.15.133:0/4154858067 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47009ffd0 con 0x55edb38dc330 2024-09-11T23:45:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe4617fa640 1 -- 172.21.15.133:0/4154858067 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe47009bc20 con 0x55edb38dc330 2024-09-11T23:45:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.203+0000 7fe4617fa640 1 -- 172.21.15.133:0/4154858067 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4700a4400 con 0x55edb38dc330 2024-09-11T23:45:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.204+0000 7fe484e81740 1 -- 172.21.15.133:0/4154858067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dc330 msgr2=0x55edb3902d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.204+0000 7fe484e81740 1 --2- 172.21.15.133:0/4154858067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dc330 0x55edb3902d20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe470099ed0 tx=0x7fe470003f30 comp rx=0 tx=0).stop 2024-09-11T23:45:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.204+0000 7fe484e81740 1 -- 172.21.15.133:0/4154858067 shutdown_connections 2024-09-11T23:45:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.204+0000 7fe484e81740 1 --2- 172.21.15.133:0/4154858067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38eb0c0 0x55edb3903260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.204+0000 7fe484e81740 1 --2- 172.21.15.133:0/4154858067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38dc330 0x55edb3902d20 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.204+0000 7fe484e81740 1 --2- 172.21.15.133:0/4154858067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dba10 0x55edb38e9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.204+0000 7fe484e81740 1 -- 172.21.15.133:0/4154858067 >> 172.21.15.133:0/4154858067 conn(0x55edb38d8060 msgr2=0x55edb38e9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.204+0000 7fe484e81740 1 -- 172.21.15.133:0/4154858067 shutdown_connections 2024-09-11T23:45:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.204+0000 7fe484e81740 1 -- 172.21.15.133:0/4154858067 wait complete. 2024-09-11T23:45:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.204+0000 7fe484e81740 1 -- start start 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe484e81740 1 Processor -- start 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe484e81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38dba10 0x55edb38e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe484e81740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe484e81740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38eb0c0 0x55edb38e5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe484e81740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c1ac970 con 0x55edb38eb0c0 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe484e81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c1ac2a0 con 0x55edb38dba10 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe484e81740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c1a9520 con 0x55edb38dc330 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe484e81740 10 client.0 _ll_register_callbacks cb 0x7fe474031e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe47bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38eb0c0 0x55edb38e5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe47bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38eb0c0 0x55edb38e5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39022/0 (socket says 172.21.15.133:39022) 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe47bfff640 1 -- 172.21.15.133:0/645281902 learned_addr learned my addr 172.21.15.133:0/645281902 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe47b7fe640 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38dba10 0x55edb38e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe47affd640 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe47bfff640 1 -- 172.21.15.133:0/645281902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 msgr2=0x55edb38e55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe47bfff640 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38e55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe47bfff640 1 -- 172.21.15.133:0/645281902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38dba10 msgr2=0x55edb38e5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe47bfff640 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38dba10 0x55edb38e5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe47bfff640 1 -- 172.21.15.133:0/645281902 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edb38e2e90 con 0x55edb38eb0c0 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe47bfff640 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38eb0c0 0x55edb38e5b00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe47404aab0 tx=0x7fe474004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb25f8b2875136c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.205+0000 7fe45affd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe45affd640 1 -- 172.21.15.133:0/645281902 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474030cf0 con 0x55edb38eb0c0 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe45affd640 1 -- 172.21.15.133:0/645281902 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fe47404a440 con 0x55edb38eb0c0 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe484e81740 1 -- 172.21.15.133:0/645281902 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe47009bc20 con 0x55edb38eb0c0 2024-09-11T23:45:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe45affd640 1 -- 172.21.15.133:0/645281902 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe474030e90 con 0x55edb38eb0c0 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe484e81740 10 client.17577 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe484e81740 1 -- 172.21.15.133:0/645281902 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55edb3965ea0 con 0x55edb38eb0c0 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe45affd640 1 -- 172.21.15.133:0/645281902 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe47009bc20 con 0x55edb38eb0c0 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe45affd640 1 -- 172.21.15.133:0/645281902 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe474048040 con 0x55edb38eb0c0 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe484e81740 10 client.17577 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe484e81740 10 client.17577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe484e81740 20 client.17577 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe484e81740 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55edb3966620 0x55edb3968a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe484e81740 4 client.17577 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe484e81740 1 -- 172.21.15.133:0/645281902 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55edb3847d70 con 0x55edb3966620 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.206+0000 7fe47b7fe640 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55edb3966620 0x55edb3968a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.207+0000 7fe45affd640 10 client.17577 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.207+0000 7fe47b7fe640 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55edb3966620 0x55edb3968a40 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1770fa1199b89 server_cookie=ccc404aaff103cf5 in_seq=0 out_seq=0 2024-09-11T23:45:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe45affd640 1 -- 172.21.15.133:0/645281902 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55edb3847d70 con 0x55edb3966620 2024-09-11T23:45:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe45affd640 10 client.17577 handle_command_reply: tid=0 2024-09-11T23:45:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 1 client.17577 shutdown 2024-09-11T23:45:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 1 -- 172.21.15.133:0/645281902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38eb0c0 msgr2=0x55edb38e5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38eb0c0 0x55edb38e5b00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe47404aab0 tx=0x7fe474004950 comp rx=0 tx=0).stop 2024-09-11T23:45:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 1 -- 172.21.15.133:0/645281902 shutdown_connections 2024-09-11T23:45:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55edb3966620 0x55edb3968a40 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edb38eb0c0 0x55edb38e5b00 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55edb38dc330 0x55edb38e55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe45affd640 0 client.17577 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 1 --2- 172.21.15.133:0/645281902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edb38dba10 0x55edb38e5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 1 -- 172.21.15.133:0/645281902 >> 172.21.15.133:0/645281902 conn(0x55edb38d8060 msgr2=0x55edb38ddd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 1 -- 172.21.15.133:0/645281902 shutdown_connections 2024-09-11T23:45:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 1 -- 172.21.15.133:0/645281902 wait complete. 2024-09-11T23:45:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.220+0000 7fe484e81740 20 client.17577 trim_cache size 0 max 16384 2024-09-11T23:45:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:45:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:45:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:45:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:45:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 -- 172.21.15.133:0/405242066 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe4640a2da0 msgr2=0x7fe4640a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe4640a2da0 0x7fe4640a5260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe474004ab0 tx=0x7fe474004a40 comp rx=0 tx=0).stop 2024-09-11T23:45:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 -- 172.21.15.133:0/405242066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c101350 msgr2=0x7fe47c198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c101350 0x7fe47c198830 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe47000ea60 tx=0x7fe47000ef20 comp rx=0 tx=0).stop 2024-09-11T23:45:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 -- 172.21.15.133:0/405242066 shutdown_connections 2024-09-11T23:45:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe4640a2da0 0x7fe4640a5260 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe47c1069c0 0x7fe47c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c101350 0x7fe47c198830 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 --2- 172.21.15.133:0/405242066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe47c1009b0 0x7fe47c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 -- 172.21.15.133:0/405242066 >> 172.21.15.133:0/405242066 conn(0x7fe47c077990 msgr2=0x7fe47c069a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 -- 172.21.15.133:0/405242066 shutdown_connections 2024-09-11T23:45:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:06.221+0000 7fe4820e6640 1 -- 172.21.15.133:0/405242066 wait complete. 2024-09-11T23:45:06.601 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id b2f50c87 --release --await' 2024-09-11T23:45:06.602 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id b2f50c87 --release --await 2024-09-11T23:45:06.901 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:06.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:06 smithi138 ceph-mon[35142]: pgmap v257: 97 pgs: 97 active+clean; 70 GiB data, 239 GiB used, 833 GiB / 1.0 TiB avail; 19 KiB/s rd, 150 MiB/s wr, 134 op/s 2024-09-11T23:45:06.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:06 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2753835624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:06.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:06 smithi133 ceph-mon[31430]: pgmap v257: 97 pgs: 97 active+clean; 70 GiB data, 239 GiB used, 833 GiB / 1.0 TiB avail; 19 KiB/s rd, 150 MiB/s wr, 134 op/s 2024-09-11T23:45:06.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:06 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2753835624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:06 smithi178 ceph-mon[35882]: pgmap v257: 97 pgs: 97 active+clean; 70 GiB data, 239 GiB used, 833 GiB / 1.0 TiB avail; 19 KiB/s rd, 150 MiB/s wr, 134 op/s 2024-09-11T23:45:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:06 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2753835624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:07.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.391+0000 7f9567eca640 1 -- 172.21.15.133:0/3178972614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9560100e20 msgr2=0x7f9560101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:07.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.391+0000 7f9567eca640 1 --2- 172.21.15.133:0/3178972614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9560100e20 0x7f9560101200 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f955c009900 tx=0x7f955c02a480 comp rx=0 tx=0).stop 2024-09-11T23:45:07.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.391+0000 7f9567eca640 1 -- 172.21.15.133:0/3178972614 shutdown_connections 2024-09-11T23:45:07.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.391+0000 7f9567eca640 1 --2- 172.21.15.133:0/3178972614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9560102ef0 0x7f956010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.391+0000 7f9567eca640 1 --2- 172.21.15.133:0/3178972614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560102570 0x7f95601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.391+0000 7f9567eca640 1 --2- 172.21.15.133:0/3178972614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9560100e20 0x7f9560101200 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.391+0000 7f9567eca640 1 -- 172.21.15.133:0/3178972614 >> 172.21.15.133:0/3178972614 conn(0x7f95600fc7d0 msgr2=0x7f95600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:07.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.391+0000 7f9567eca640 1 -- 172.21.15.133:0/3178972614 shutdown_connections 2024-09-11T23:45:07.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.391+0000 7f9567eca640 1 -- 172.21.15.133:0/3178972614 wait complete. 2024-09-11T23:45:07.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.391+0000 7f9567eca640 1 Processor -- start 2024-09-11T23:45:07.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.391+0000 7f9567eca640 1 -- start start 2024-09-11T23:45:07.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9567eca640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560100e20 0x7f95600fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9567eca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9560102570 0x7f95600fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9567eca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9560102ef0 0x7f95600fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9567eca640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9560110fa0 con 0x7f9560102ef0 2024-09-11T23:45:07.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9567eca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9560110e20 con 0x7f9560102570 2024-09-11T23:45:07.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9567eca640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9560111120 con 0x7f9560100e20 2024-09-11T23:45:07.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9566467640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9560102ef0 0x7f95600fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9566467640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9560102ef0 0x7f95600fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39040/0 (socket says 172.21.15.133:39040) 2024-09-11T23:45:07.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9566467640 1 -- 172.21.15.133:0/2064633186 learned_addr learned my addr 172.21.15.133:0/2064633186 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:07.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9565c66640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560100e20 0x7f95600fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9565465640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9560102570 0x7f95600fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9566467640 1 -- 172.21.15.133:0/2064633186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560100e20 msgr2=0x7f95600fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:07.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9566467640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560100e20 0x7f95600fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9566467640 1 -- 172.21.15.133:0/2064633186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9560102570 msgr2=0x7f95600fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:07.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9566467640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9560102570 0x7f95600fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9566467640 1 -- 172.21.15.133:0/2064633186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95601a69f0 con 0x7f9560102ef0 2024-09-11T23:45:07.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9565465640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9560102570 0x7f95600fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:07.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9566467640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9560102ef0 0x7f95600fd700 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9558009fd0 tx=0x7f955800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:07.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9556ffd640 1 -- 172.21.15.133:0/2064633186 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558017070 con 0x7f9560102ef0 2024-09-11T23:45:07.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9556ffd640 1 -- 172.21.15.133:0/2064633186 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9558003e50 con 0x7f9560102ef0 2024-09-11T23:45:07.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9567eca640 1 -- 172.21.15.133:0/2064633186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95601a6ce0 con 0x7f9560102ef0 2024-09-11T23:45:07.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.392+0000 7f9556ffd640 1 -- 172.21.15.133:0/2064633186 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95580106c0 con 0x7f9560102ef0 2024-09-11T23:45:07.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.393+0000 7f9567eca640 1 -- 172.21.15.133:0/2064633186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95601a71d0 con 0x7f9560102ef0 2024-09-11T23:45:07.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.395+0000 7f9556ffd640 1 -- 172.21.15.133:0/2064633186 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9558003970 con 0x7f9560102ef0 2024-09-11T23:45:07.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.395+0000 7f956a2a3740 1 Processor -- start 2024-09-11T23:45:07.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.395+0000 7f956a2a3740 1 -- start start 2024-09-11T23:45:07.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.395+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992cf6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.395+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992cf6980 0x55d992d00d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.395+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992d01240 0x55d992d03630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.395+0000 7f956a2a3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d992c5b620 con 0x55d992cf6060 2024-09-11T23:45:07.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.395+0000 7f956a2a3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d992c02e50 con 0x55d992d01240 2024-09-11T23:45:07.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.395+0000 7f956a2a3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d992c575e0 con 0x55d992cf6980 2024-09-11T23:45:07.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.395+0000 7f9556ffd640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f954c0a2cf0 0x7f954c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9565465640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992cf6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9565465640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992cf6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39048/0 (socket says 172.21.15.133:39048) 2024-09-11T23:45:07.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9565465640 1 -- 172.21.15.133:0/411393288 learned_addr learned my addr 172.21.15.133:0/411393288 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:07.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9566467640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f954c0a2cf0 0x7f954c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9565c66640 1 --2- 172.21.15.133:0/411393288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992cf6980 0x55d992d00d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9565465640 1 --2- 172.21.15.133:0/411393288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992d01240 0x55d992d03630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9556ffd640 1 -- 172.21.15.133:0/2064633186 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f95580cf1b0 con 0x7f9560102ef0 2024-09-11T23:45:07.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9565465640 1 -- 172.21.15.133:0/411393288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992cf6980 msgr2=0x55d992d00d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:07.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9565465640 1 --2- 172.21.15.133:0/411393288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992cf6980 0x55d992d00d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9565465640 1 -- 172.21.15.133:0/411393288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992d01240 msgr2=0x55d992d03630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:07.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9565465640 1 --2- 172.21.15.133:0/411393288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992d01240 0x55d992d03630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9565465640 1 -- 172.21.15.133:0/411393288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d992bf3d40 con 0x55d992cf6060 2024-09-11T23:45:07.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9566467640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f954c0a2cf0 0x7f954c0a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9558097b70 tx=0x7f955806d6d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:07.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f9565465640 1 --2- 172.21.15.133:0/411393288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992cf6440 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f9550006fd0 tx=0x7f9550007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13809f77f4c8d658 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:07.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f954bfff640 1 -- 172.21.15.133:0/411393288 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955001c070 con 0x55d992cf6060 2024-09-11T23:45:07.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f954bfff640 1 -- 172.21.15.133:0/411393288 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9550010040 con 0x55d992cf6060 2024-09-11T23:45:07.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.396+0000 7f954bfff640 1 -- 172.21.15.133:0/411393288 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550014730 con 0x55d992cf6060 2024-09-11T23:45:07.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.397+0000 7f956a2a3740 1 -- 172.21.15.133:0/411393288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 msgr2=0x55d992cf6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:07.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.397+0000 7f956a2a3740 1 --2- 172.21.15.133:0/411393288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992cf6440 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f9550006fd0 tx=0x7f9550007420 comp rx=0 tx=0).stop 2024-09-11T23:45:07.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.397+0000 7f956a2a3740 1 -- 172.21.15.133:0/411393288 shutdown_connections 2024-09-11T23:45:07.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.397+0000 7f956a2a3740 1 --2- 172.21.15.133:0/411393288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992d01240 0x55d992d03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.397+0000 7f956a2a3740 1 --2- 172.21.15.133:0/411393288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992cf6980 0x55d992d00d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.397+0000 7f956a2a3740 1 --2- 172.21.15.133:0/411393288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992cf6440 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.397+0000 7f956a2a3740 1 -- 172.21.15.133:0/411393288 >> 172.21.15.133:0/411393288 conn(0x55d992cf25f0 msgr2=0x55d992cf29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:07.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.397+0000 7f956a2a3740 1 -- 172.21.15.133:0/411393288 shutdown_connections 2024-09-11T23:45:07.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.397+0000 7f956a2a3740 1 -- 172.21.15.133:0/411393288 wait complete. 2024-09-11T23:45:07.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f956a2a3740 1 -- start start 2024-09-11T23:45:07.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f956a2a3740 1 Processor -- start 2024-09-11T23:45:07.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6060 0x55d992cfe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6980 0x55d992cfe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992d01240 0x55d992cfeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f956a2a3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d992d03e10 con 0x55d992cf6980 2024-09-11T23:45:07.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f956a2a3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d992be3f10 con 0x55d992cf6060 2024-09-11T23:45:07.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f956a2a3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d992d03f90 con 0x55d992d01240 2024-09-11T23:45:07.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f956a2a3740 10 client.0 _ll_register_callbacks cb 0x55d992c07db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:07.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f9565c66640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6980 0x55d992cfe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f9565c66640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6980 0x55d992cfe660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39064/0 (socket says 172.21.15.133:39064) 2024-09-11T23:45:07.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f9565465640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6060 0x55d992cfe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.398+0000 7f9565c66640 1 -- 172.21.15.133:0/322310431 learned_addr learned my addr 172.21.15.133:0/322310431 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:07.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f9565465640 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992d01240 0x55d992cfeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f9565c66640 1 -- 172.21.15.133:0/322310431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992d01240 msgr2=0x55d992cfeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:07.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f9565c66640 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992d01240 0x55d992cfeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f9565c66640 1 -- 172.21.15.133:0/322310431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6060 msgr2=0x55d992cfe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:07.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f9565c66640 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6060 0x55d992cfe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f9565c66640 1 -- 172.21.15.133:0/322310431 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99296a580 con 0x55d992cf6980 2024-09-11T23:45:07.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f9565c66640 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6980 0x55d992cfe660 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f955c02a9a0 tx=0x7f955c004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd5c87ba566ec4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:07.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f95497fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:07.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f95497fa640 1 -- 172.21.15.133:0/322310431 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c035070 con 0x55d992cf6980 2024-09-11T23:45:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f95497fa640 1 -- 172.21.15.133:0/322310431 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f955c003ac0 con 0x55d992cf6980 2024-09-11T23:45:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f956a2a3740 1 -- 172.21.15.133:0/322310431 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9550010040 con 0x55d992cf6980 2024-09-11T23:45:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f95497fa640 1 -- 172.21.15.133:0/322310431 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f955c003cc0 con 0x55d992cf6980 2024-09-11T23:45:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f956a2a3740 10 client.17589 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f956a2a3740 1 -- 172.21.15.133:0/322310431 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d992a34960 con 0x55d992cf6980 2024-09-11T23:45:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.399+0000 7f95497fa640 1 -- 172.21.15.133:0/322310431 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9550010040 con 0x55d992cf6980 2024-09-11T23:45:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.400+0000 7f95497fa640 1 -- 172.21.15.133:0/322310431 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f955c04e560 con 0x55d992cf6980 2024-09-11T23:45:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.400+0000 7f956a2a3740 10 client.17589 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.400+0000 7f956a2a3740 10 client.17589 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:45:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.400+0000 7f956a2a3740 20 client.17589 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.400+0000 7f956a2a3740 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d992d10d50 0x55d992d131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.400+0000 7f956a2a3740 4 client.17589 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.400+0000 7f956a2a3740 1 -- 172.21.15.133:0/322310431 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d992c68b90 con 0x55d992d10d50 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.400+0000 7f9565465640 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d992d10d50 0x55d992d131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.401+0000 7f9565465640 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d992d10d50 0x55d992d131d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f361615c2efc0460 server_cookie=6dabdc75257f518f in_seq=0 out_seq=0 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.401+0000 7f95497fa640 10 client.17589 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.401+0000 7f95497fa640 1 -- 172.21.15.133:0/322310431 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d992c68b90 con 0x55d992d10d50 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.401+0000 7f95497fa640 10 client.17589 handle_command_reply: tid=0 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.401+0000 7f956a2a3740 1 client.17589 shutdown 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.401+0000 7f956a2a3740 1 -- 172.21.15.133:0/322310431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6980 msgr2=0x55d992cfe660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.401+0000 7f956a2a3740 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6980 0x55d992cfe660 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f955c02a9a0 tx=0x7f955c004440 comp rx=0 tx=0).stop 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.401+0000 7f956a2a3740 1 -- 172.21.15.133:0/322310431 shutdown_connections 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.401+0000 7f956a2a3740 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d992d10d50 0x55d992d131d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.402+0000 7f956a2a3740 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992d01240 0x55d992cfeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.402+0000 7f956a2a3740 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6980 0x55d992cfe660 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.402+0000 7f95497fa640 0 client.17589 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.402+0000 7f956a2a3740 1 --2- 172.21.15.133:0/322310431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6060 0x55d992cfe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.402+0000 7f956a2a3740 1 -- 172.21.15.133:0/322310431 >> 172.21.15.133:0/322310431 conn(0x55d992cf25f0 msgr2=0x55d992cf82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.402+0000 7f956a2a3740 1 -- 172.21.15.133:0/322310431 shutdown_connections 2024-09-11T23:45:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.402+0000 7f956a2a3740 1 -- 172.21.15.133:0/322310431 wait complete. 2024-09-11T23:45:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.402+0000 7f956a2a3740 20 client.17589 trim_cache size 0 max 16384 2024-09-11T23:45:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.418+0000 7f956a2a3740 1 Processor -- start 2024-09-11T23:45:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.418+0000 7f956a2a3740 1 -- start start 2024-09-11T23:45:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6060 0x55d992cffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992cf6980 0x55d992cf8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992d01240 0x55d992d18aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f956a2a3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f95601af970 con 0x55d992d01240 2024-09-11T23:45:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f956a2a3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95601b14b0 con 0x55d992cf6060 2024-09-11T23:45:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f956a2a3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95601b0040 con 0x55d992cf6980 2024-09-11T23:45:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f9565465640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992d01240 0x55d992d18aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f9565465640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992d01240 0x55d992d18aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39066/0 (socket says 172.21.15.133:39066) 2024-09-11T23:45:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f9565465640 1 -- 172.21.15.133:0/3235230944 learned_addr learned my addr 172.21.15.133:0/3235230944 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f9565465640 1 --2- 172.21.15.133:0/3235230944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6060 0x55d992cffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f9565c66640 1 --2- 172.21.15.133:0/3235230944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992cf6980 0x55d992cf8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f9565465640 1 -- 172.21.15.133:0/3235230944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992cf6980 msgr2=0x55d992cf8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f9565465640 1 --2- 172.21.15.133:0/3235230944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992cf6980 0x55d992cf8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f9565465640 1 -- 172.21.15.133:0/3235230944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6060 msgr2=0x55d992cffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f9565465640 1 --2- 172.21.15.133:0/3235230944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6060 0x55d992cffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f9565465640 1 -- 172.21.15.133:0/3235230944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d992d03990 con 0x55d992d01240 2024-09-11T23:45:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f9565465640 1 --2- 172.21.15.133:0/3235230944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992d01240 0x55d992d18aa0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9550017f20 tx=0x7f9550017f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78daf409fa2d6d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:07.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f954b7fe640 1 -- 172.21.15.133:0/3235230944 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955000fe60 con 0x55d992d01240 2024-09-11T23:45:07.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f954b7fe640 1 -- 172.21.15.133:0/3235230944 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9550017900 con 0x55d992d01240 2024-09-11T23:45:07.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.419+0000 7f954b7fe640 1 -- 172.21.15.133:0/3235230944 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955001c040 con 0x55d992d01240 2024-09-11T23:45:07.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.420+0000 7f956a2a3740 1 -- 172.21.15.133:0/3235230944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992d01240 msgr2=0x55d992d18aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:07.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.420+0000 7f956a2a3740 1 --2- 172.21.15.133:0/3235230944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992d01240 0x55d992d18aa0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9550017f20 tx=0x7f9550017f50 comp rx=0 tx=0).stop 2024-09-11T23:45:07.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.420+0000 7f956a2a3740 1 -- 172.21.15.133:0/3235230944 shutdown_connections 2024-09-11T23:45:07.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.420+0000 7f956a2a3740 1 --2- 172.21.15.133:0/3235230944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992d01240 0x55d992d18aa0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.420+0000 7f956a2a3740 1 --2- 172.21.15.133:0/3235230944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992cf6980 0x55d992cf8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.420+0000 7f956a2a3740 1 --2- 172.21.15.133:0/3235230944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6060 0x55d992cffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.420+0000 7f956a2a3740 1 -- 172.21.15.133:0/3235230944 >> 172.21.15.133:0/3235230944 conn(0x55d992cf25f0 msgr2=0x55d992cf1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.420+0000 7f956a2a3740 1 -- 172.21.15.133:0/3235230944 shutdown_connections 2024-09-11T23:45:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.420+0000 7f956a2a3740 1 -- 172.21.15.133:0/3235230944 wait complete. 2024-09-11T23:45:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.420+0000 7f956a2a3740 1 -- start start 2024-09-11T23:45:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f956a2a3740 1 Processor -- start 2024-09-11T23:45:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992d09720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6980 0x55d992d05a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f956a2a3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992d01240 0x55d992d05fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f956a2a3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f95601b0de0 con 0x55d992cf6060 2024-09-11T23:45:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f956a2a3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95601b0710 con 0x55d992cf6980 2024-09-11T23:45:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f956a2a3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95601ad970 con 0x55d992d01240 2024-09-11T23:45:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f956a2a3740 10 client.0 _ll_register_callbacks cb 0x7f955c0040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:07.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f9565465640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992d09720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f9565465640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992d09720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39070/0 (socket says 172.21.15.133:39070) 2024-09-11T23:45:07.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f9565465640 1 -- 172.21.15.133:0/3279689094 learned_addr learned my addr 172.21.15.133:0/3279689094 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:07.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f9565c66640 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6980 0x55d992d05a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f9565465640 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992d01240 0x55d992d05fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f9565465640 1 -- 172.21.15.133:0/3279689094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992d01240 msgr2=0x55d992d05fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:07.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f9565465640 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992d01240 0x55d992d05fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f9565465640 1 -- 172.21.15.133:0/3279689094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6980 msgr2=0x55d992d05a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:07.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.421+0000 7f9565465640 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6980 0x55d992d05a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f9565465640 1 -- 172.21.15.133:0/3279689094 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d992d06590 con 0x55d992cf6060 2024-09-11T23:45:07.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f9565465640 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992d09720 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9550006b50 tx=0x7f9550014c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300c720641f2fdf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:07.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f95497fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:07.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f95497fa640 1 -- 172.21.15.133:0/3279689094 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955001c070 con 0x55d992cf6060 2024-09-11T23:45:07.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f956a2a3740 1 -- 172.21.15.133:0/3279689094 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9550017900 con 0x55d992cf6060 2024-09-11T23:45:07.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f95497fa640 1 -- 172.21.15.133:0/3279689094 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f9550017ab0 con 0x55d992cf6060 2024-09-11T23:45:07.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f95497fa640 1 -- 172.21.15.133:0/3279689094 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9550049050 con 0x55d992cf6060 2024-09-11T23:45:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f956a2a3740 10 client.17595 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f956a2a3740 1 -- 172.21.15.133:0/3279689094 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d992d7c8c0 con 0x55d992cf6060 2024-09-11T23:45:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f95497fa640 1 -- 172.21.15.133:0/3279689094 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9550017900 con 0x55d992cf6060 2024-09-11T23:45:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f95497fa640 1 -- 172.21.15.133:0/3279689094 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9550022040 con 0x55d992cf6060 2024-09-11T23:45:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f956a2a3740 10 client.17595 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f956a2a3740 10 client.17595 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:45:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f956a2a3740 20 client.17595 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f956a2a3740 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d992d7cd30 0x55d992d7f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f956a2a3740 4 client.17595 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b2f50c87", "release": true, "await": true}] 2024-09-11T23:45:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.422+0000 7f956a2a3740 1 -- 172.21.15.133:0/3279689094 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b2f50c87", "release": true, "await": true}) -- 0x55d992c630a0 con 0x55d992d7cd30 2024-09-11T23:45:07.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.423+0000 7f9565c66640 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d992d7cd30 0x55d992d7f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:07.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.423+0000 7f95497fa640 10 client.17595 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:07.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.423+0000 7f9565c66640 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d992d7cd30 0x55d992d7f120 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2860d51c390b9acf server_cookie=2b18f072b840c81a in_seq=0 out_seq=0 2024-09-11T23:45:07.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.428+0000 7f95497fa640 1 -- 172.21.15.133:0/3279689094 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d992c630a0 con 0x55d992d7cd30 2024-09-11T23:45:07.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.428+0000 7f95497fa640 10 client.17595 handle_command_reply: tid=0 2024-09-11T23:45:07.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.428+0000 7f956a2a3740 1 client.17595 shutdown 2024-09-11T23:45:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f956a2a3740 1 -- 172.21.15.133:0/3279689094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 msgr2=0x55d992d09720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f956a2a3740 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992d09720 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9550006b50 tx=0x7f9550014c90 comp rx=0 tx=0).stop 2024-09-11T23:45:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f956a2a3740 1 -- 172.21.15.133:0/3279689094 shutdown_connections 2024-09-11T23:45:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f956a2a3740 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d992d7cd30 0x55d992d7f120 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f956a2a3740 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d992d01240 0x55d992d05fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f956a2a3740 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d992cf6980 0x55d992d05a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f95497fa640 0 client.17595 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f956a2a3740 1 --2- 172.21.15.133:0/3279689094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d992cf6060 0x55d992d09720 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f956a2a3740 1 -- 172.21.15.133:0/3279689094 >> 172.21.15.133:0/3279689094 conn(0x55d992cf25f0 msgr2=0x55d992cf0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f956a2a3740 1 -- 172.21.15.133:0/3279689094 shutdown_connections 2024-09-11T23:45:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f956a2a3740 1 -- 172.21.15.133:0/3279689094 wait complete. 2024-09-11T23:45:07.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.429+0000 7f956a2a3740 20 client.17595 trim_cache size 0 max 16384 2024-09-11T23:45:07.436 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:45:07.436 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:45:07.436 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:45:07.436 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 32, 2024-09-11T23:45:07.436 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:45:07.436 INFO:teuthology.orchestra.run.smithi133.stdout: "b2f50c87": { 2024-09-11T23:45:07.436 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 32, 2024-09-11T23:45:07.436 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:45:07.436 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:45:07.437 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:45:07.437 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:45:07.437 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:45:07.437 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:45:07.437 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 139.4, 2024-09-11T23:45:07.437 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:45:07.437 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:45:07.437 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:45:07.437 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:45:07.437 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:45:07.437 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:45:07.438 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:07.438 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:07.438 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:07.438 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:07.438 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:07.438 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:45:07.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.430+0000 7f9567eca640 1 -- 172.21.15.133:0/2064633186 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f954c0a2cf0 msgr2=0x7f954c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:07.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.430+0000 7f9567eca640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f954c0a2cf0 0x7f954c0a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9558097b70 tx=0x7f955806d6d0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.430+0000 7f9567eca640 1 -- 172.21.15.133:0/2064633186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9560102ef0 msgr2=0x7f95600fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:07.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.430+0000 7f9567eca640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9560102ef0 0x7f95600fd700 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9558009fd0 tx=0x7f955800b260 comp rx=0 tx=0).stop 2024-09-11T23:45:07.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.430+0000 7f9567eca640 1 -- 172.21.15.133:0/2064633186 shutdown_connections 2024-09-11T23:45:07.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.430+0000 7f9567eca640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f954c0a2cf0 0x7f954c0a51b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.430+0000 7f9567eca640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9560102ef0 0x7f95600fd700 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.430+0000 7f9567eca640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9560102570 0x7f95600fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.430+0000 7f9567eca640 1 --2- 172.21.15.133:0/2064633186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560100e20 0x7f95600fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:07.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.430+0000 7f9567eca640 1 -- 172.21.15.133:0/2064633186 >> 172.21.15.133:0/2064633186 conn(0x7f95600fc7d0 msgr2=0x7f956006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:07.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.430+0000 7f9567eca640 1 -- 172.21.15.133:0/2064633186 shutdown_connections 2024-09-11T23:45:07.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:07.431+0000 7f9567eca640 1 -- 172.21.15.133:0/2064633186 wait complete. 2024-09-11T23:45:08.666 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b2f50c87', total seconds elapsed: 23.2 2024-09-11T23:45:08.666 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-11T23:45:08.667 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:45:08.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:08 smithi133 ceph-mon[31430]: pgmap v258: 97 pgs: 97 active+clean; 70 GiB data, 239 GiB used, 833 GiB / 1.0 TiB avail; 26 KiB/s rd, 43 MiB/s wr, 66 op/s 2024-09-11T23:45:09.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:08 smithi178 ceph-mon[35882]: pgmap v258: 97 pgs: 97 active+clean; 70 GiB data, 239 GiB used, 833 GiB / 1.0 TiB avail; 26 KiB/s rd, 43 MiB/s wr, 66 op/s 2024-09-11T23:45:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:08 smithi138 ceph-mon[35142]: pgmap v258: 97 pgs: 97 active+clean; 70 GiB data, 239 GiB used, 833 GiB / 1.0 TiB avail; 26 KiB/s rd, 43 MiB/s wr, 66 op/s 2024-09-11T23:45:09.460 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:10.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.558+0000 7f74437b4640 1 -- 172.21.15.133:0/594013739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c112ba0 msgr2=0x7f743c114f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:10.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.558+0000 7f74437b4640 1 --2- 172.21.15.133:0/594013739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c112ba0 0x7f743c114f90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7434009ca0 tx=0x7f743402a980 comp rx=0 tx=0).stop 2024-09-11T23:45:10.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.559+0000 7f74437b4640 1 -- 172.21.15.133:0/594013739 shutdown_connections 2024-09-11T23:45:10.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.559+0000 7f74437b4640 1 --2- 172.21.15.133:0/594013739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c112ba0 0x7f743c114f90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:10.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.559+0000 7f74437b4640 1 --2- 172.21.15.133:0/594013739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f743c074eb0 0x7f743c112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:10.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.559+0000 7f74437b4640 1 --2- 172.21.15.133:0/594013739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f743c074590 0x7f743c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:10.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.559+0000 7f74437b4640 1 -- 172.21.15.133:0/594013739 >> 172.21.15.133:0/594013739 conn(0x7f743c0fd8b0 msgr2=0x7f743c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:10.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.566+0000 7f74437b4640 1 -- 172.21.15.133:0/594013739 shutdown_connections 2024-09-11T23:45:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.566+0000 7f74437b4640 1 -- 172.21.15.133:0/594013739 wait complete. 2024-09-11T23:45:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.567+0000 7f74437b4640 1 Processor -- start 2024-09-11T23:45:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.567+0000 7f74437b4640 1 -- start start 2024-09-11T23:45:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.567+0000 7f74437b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f743c074590 0x7f743c112150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.568+0000 7f74437b4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c074eb0 0x7f743c10b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.568+0000 7f74437b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f743c112ba0 0x7f743c10b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:10.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.568+0000 7f74437b4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f743c118660 con 0x7f743c074590 2024-09-11T23:45:10.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.568+0000 7f74437b4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1184e0 con 0x7f743c074eb0 2024-09-11T23:45:10.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.568+0000 7f74437b4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1187e0 con 0x7f743c112ba0 2024-09-11T23:45:10.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.568+0000 7f7441550640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f743c074590 0x7f743c112150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:10.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.568+0000 7f7441550640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f743c074590 0x7f743c112150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39086/0 (socket says 172.21.15.133:39086) 2024-09-11T23:45:10.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.568+0000 7f7441550640 1 -- 172.21.15.133:0/2815977148 learned_addr learned my addr 172.21.15.133:0/2815977148 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:10.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.568+0000 7f7440d4f640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c074eb0 0x7f743c10b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:10.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.569+0000 7f7441550640 1 -- 172.21.15.133:0/2815977148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f743c112ba0 msgr2=0x7f743c10b720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:45:10.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.569+0000 7f7441550640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f743c112ba0 0x7f743c10b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:10.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.569+0000 7f7441550640 1 -- 172.21.15.133:0/2815977148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c074eb0 msgr2=0x7f743c10b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:10.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.569+0000 7f7441550640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c074eb0 0x7f743c10b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:10.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.569+0000 7f7441550640 1 -- 172.21.15.133:0/2815977148 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c10bcb0 con 0x7f743c074590 2024-09-11T23:45:10.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.570+0000 7f7441550640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f743c074590 0x7f743c112150 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f742c00b450 tx=0x7f742c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:10.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.573+0000 7f74327fc640 1 -- 172.21.15.133:0/2815977148 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c015020 con 0x7f743c074590 2024-09-11T23:45:10.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.573+0000 7f74437b4640 1 -- 172.21.15.133:0/2815977148 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c10bfa0 con 0x7f743c074590 2024-09-11T23:45:10.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.573+0000 7f74327fc640 1 -- 172.21.15.133:0/2815977148 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f742c011d70 con 0x7f743c074590 2024-09-11T23:45:10.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.573+0000 7f74327fc640 1 -- 172.21.15.133:0/2815977148 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c00f340 con 0x7f743c074590 2024-09-11T23:45:10.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.575+0000 7f74437b4640 1 -- 172.21.15.133:0/2815977148 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c10ff90 con 0x7f743c074590 2024-09-11T23:45:10.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.580+0000 7f74437b4640 1 -- 172.21.15.133:0/2815977148 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743c075cd0 con 0x7f743c074590 2024-09-11T23:45:10.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.587+0000 7f74327fc640 1 -- 172.21.15.133:0/2815977148 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f742c018020 con 0x7f743c074590 2024-09-11T23:45:10.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.588+0000 7f74327fc640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f74280a2f00 0x7f74280a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:10.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.588+0000 7f7440d4f640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f74280a2f00 0x7f74280a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:10.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.588+0000 7f74327fc640 1 -- 172.21.15.133:0/2815977148 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f742c0cebd0 con 0x7f743c074590 2024-09-11T23:45:10.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.596+0000 7f7440d4f640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f74280a2f00 0x7f74280a53c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f743c10c9a0 tx=0x7f7438008040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:10.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:10.596+0000 7f74327fc640 1 -- 172.21.15.133:0/2815977148 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f742c00d040 con 0x7f743c074590 2024-09-11T23:45:11.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.028+0000 7f74437b4640 1 -- 172.21.15.133:0/2815977148 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f743c074970 con 0x7f743c074590 2024-09-11T23:45:11.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.030+0000 7f74327fc640 1 -- 172.21.15.133:0/2815977148 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7f742c0b9e20 con 0x7f743c074590 2024-09-11T23:45:11.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:10 smithi133 ceph-mon[31430]: pgmap v259: 97 pgs: 97 active+clean; 70 GiB data, 233 GiB used, 839 GiB / 1.0 TiB avail; 98 KiB/s rd, 91 MiB/s wr, 380 op/s 2024-09-11T23:45:11.035 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:45:11.035 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:45:11.036 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:45:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.038+0000 7f7427fff640 1 -- 172.21.15.133:0/2815977148 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f74280a2f00 msgr2=0x7f74280a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.038+0000 7f7427fff640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f74280a2f00 0x7f74280a53c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f743c10c9a0 tx=0x7f7438008040 comp rx=0 tx=0).stop 2024-09-11T23:45:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.038+0000 7f7427fff640 1 -- 172.21.15.133:0/2815977148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f743c074590 msgr2=0x7f743c112150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.038+0000 7f7427fff640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f743c074590 0x7f743c112150 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f742c00b450 tx=0x7f742c00b910 comp rx=0 tx=0).stop 2024-09-11T23:45:11.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.038+0000 7f7427fff640 1 -- 172.21.15.133:0/2815977148 shutdown_connections 2024-09-11T23:45:11.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.038+0000 7f7427fff640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f74280a2f00 0x7f74280a53c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:11.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.038+0000 7f7427fff640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f743c112ba0 0x7f743c10b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:11.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.038+0000 7f7427fff640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f743c074eb0 0x7f743c10b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:11.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.038+0000 7f7427fff640 1 --2- 172.21.15.133:0/2815977148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f743c074590 0x7f743c112150 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:11.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.039+0000 7f7427fff640 1 -- 172.21.15.133:0/2815977148 >> 172.21.15.133:0/2815977148 conn(0x7f743c0fd8b0 msgr2=0x7f743c0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:10 smithi178 ceph-mon[35882]: pgmap v259: 97 pgs: 97 active+clean; 70 GiB data, 233 GiB used, 839 GiB / 1.0 TiB avail; 98 KiB/s rd, 91 MiB/s wr, 380 op/s 2024-09-11T23:45:11.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.048+0000 7f7427fff640 1 -- 172.21.15.133:0/2815977148 shutdown_connections 2024-09-11T23:45:11.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:11.049+0000 7f7427fff640 1 -- 172.21.15.133:0/2815977148 wait complete. 2024-09-11T23:45:11.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:10 smithi138 ceph-mon[35142]: pgmap v259: 97 pgs: 97 active+clean; 70 GiB data, 233 GiB used, 839 GiB / 1.0 TiB avail; 98 KiB/s rd, 91 MiB/s wr, 380 op/s 2024-09-11T23:45:12.021 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-11T23:45:12.021 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-11T23:45:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:11 smithi178 ceph-mon[35882]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:45:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:11 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2815977148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:11 smithi178 ceph-mon[35882]: pgmap v261: 97 pgs: 97 active+clean; 70 GiB data, 231 GiB used, 842 GiB / 1.0 TiB avail; 106 KiB/s rd, 140 MiB/s wr, 670 op/s 2024-09-11T23:45:12.087 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:11 smithi133 ceph-mon[31430]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:45:12.088 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:11 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2815977148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:12.088 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:11 smithi133 ceph-mon[31430]: pgmap v261: 97 pgs: 97 active+clean; 70 GiB data, 231 GiB used, 842 GiB / 1.0 TiB avail; 106 KiB/s rd, 140 MiB/s wr, 670 op/s 2024-09-11T23:45:12.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:11 smithi138 ceph-mon[35142]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:45:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:11 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2815977148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:11 smithi138 ceph-mon[35142]: pgmap v261: 97 pgs: 97 active+clean; 70 GiB data, 231 GiB used, 842 GiB / 1.0 TiB avail; 106 KiB/s rd, 140 MiB/s wr, 670 op/s 2024-09-11T23:45:12.757 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:13.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.650+0000 7f5888d94640 1 -- 172.21.15.133:0/1497693866 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c030e30 con 0x7f588411b140 2024-09-11T23:45:13.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.650+0000 7f588bffa640 1 -- 172.21.15.133:0/1497693866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f588411b140 msgr2=0x7f588411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:13.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.650+0000 7f588bffa640 1 --2- 172.21.15.133:0/1497693866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f588411b140 0x7f588411d530 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f587c009be0 tx=0x7f587c02a710 comp rx=0 tx=0).stop 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.650+0000 7f588bffa640 1 -- 172.21.15.133:0/1497693866 shutdown_connections 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.650+0000 7f588bffa640 1 --2- 172.21.15.133:0/1497693866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f588411b140 0x7f588411d530 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.650+0000 7f588bffa640 1 --2- 172.21.15.133:0/1497693866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5884109830 0x7f5884109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.650+0000 7f588bffa640 1 --2- 172.21.15.133:0/1497693866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5884109450 0x7f5884110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.650+0000 7f588bffa640 1 -- 172.21.15.133:0/1497693866 >> 172.21.15.133:0/1497693866 conn(0x7f588406e140 msgr2=0x7f588406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.650+0000 7f588bffa640 1 -- 172.21.15.133:0/1497693866 shutdown_connections 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.650+0000 7f588bffa640 1 -- 172.21.15.133:0/1497693866 wait complete. 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.651+0000 7f588bffa640 1 Processor -- start 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.651+0000 7f588bffa640 1 -- start start 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.652+0000 7f588bffa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5884109450 0x7f588411a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.652+0000 7f588bffa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5884109830 0x7f588411a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.652+0000 7f588bffa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f588411b140 0x7f5884114490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.652+0000 7f588bffa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5884110f30 con 0x7f5884109450 2024-09-11T23:45:13.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.652+0000 7f588bffa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5884110db0 con 0x7f588411b140 2024-09-11T23:45:13.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.652+0000 7f588bffa640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58841110b0 con 0x7f5884109830 2024-09-11T23:45:13.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.654+0000 7f5889595640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5884109830 0x7f588411a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.654+0000 7f588a597640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f588411b140 0x7f5884114490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.657+0000 7f5889595640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5884109830 0x7f588411a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:46126/0 (socket says 172.21.15.133:46126) 2024-09-11T23:45:13.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.658+0000 7f5889595640 1 -- 172.21.15.133:0/1315122646 learned_addr learned my addr 172.21.15.133:0/1315122646 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:13.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.660+0000 7f5889d96640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5884109450 0x7f588411a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.662+0000 7f5889595640 1 -- 172.21.15.133:0/1315122646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f588411b140 msgr2=0x7f5884114490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:13.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.662+0000 7f5889595640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f588411b140 0x7f5884114490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.662+0000 7f5889595640 1 -- 172.21.15.133:0/1315122646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5884109450 msgr2=0x7f588411a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:13.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.662+0000 7f5889595640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5884109450 0x7f588411a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.662+0000 7f5889595640 1 -- 172.21.15.133:0/1315122646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5884114d50 con 0x7f5884109830 2024-09-11T23:45:13.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.663+0000 7f5889595640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5884109830 0x7f588411a950 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f588000e9c0 tx=0x7f588000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:13.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.663+0000 7f587affd640 1 -- 172.21.15.133:0/1315122646 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588000ccc0 con 0x7f5884109830 2024-09-11T23:45:13.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.663+0000 7f587affd640 1 -- 172.21.15.133:0/1315122646 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5880005500 con 0x7f5884109830 2024-09-11T23:45:13.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.663+0000 7f587affd640 1 -- 172.21.15.133:0/1315122646 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5880010430 con 0x7f5884109830 2024-09-11T23:45:13.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.665+0000 7f588bffa640 1 -- 172.21.15.133:0/1315122646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5884115040 con 0x7f5884109830 2024-09-11T23:45:13.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.665+0000 7f588bffa640 1 -- 172.21.15.133:0/1315122646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588407a620 con 0x7f5884109830 2024-09-11T23:45:13.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.677+0000 7f588e3d3740 1 Processor -- start 2024-09-11T23:45:13.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.678+0000 7f587affd640 1 -- 172.21.15.133:0/1315122646 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f58800046c0 con 0x7f5884109830 2024-09-11T23:45:13.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.679+0000 7f588e3d3740 1 -- start start 2024-09-11T23:45:13.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.680+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f316b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.680+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f317080 0x55d58f321400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.681+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f323d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.681+0000 7f588e3d3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d58f27c2a0 con 0x55d58f321940 2024-09-11T23:45:13.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.681+0000 7f588e3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d58f223b10 con 0x55d58f317080 2024-09-11T23:45:13.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.681+0000 7f588e3d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d58f278260 con 0x55d58f316760 2024-09-11T23:45:13.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.682+0000 7f587affd640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58700a2e40 0x7f58700a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.682+0000 7f587affd640 1 -- 172.21.15.133:0/1315122646 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f58800cf6b0 con 0x7f5884109830 2024-09-11T23:45:13.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.683+0000 7f588a597640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58700a2e40 0x7f58700a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.683+0000 7f588a597640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f316b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.685+0000 7f588a597640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f316b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:46132/0 (socket says 172.21.15.133:46132) 2024-09-11T23:45:13.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.685+0000 7f588a597640 1 -- 172.21.15.133:0/2601298331 learned_addr learned my addr 172.21.15.133:0/2601298331 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:13.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.686+0000 7f5889595640 1 --2- 172.21.15.133:0/2601298331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f323d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.686+0000 7f588a597640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58700a2e40 0x7f58700a5300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f587c02ad00 tx=0x7f587c031500 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.688+0000 7f588a597640 1 -- 172.21.15.133:0/2601298331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f317080 msgr2=0x55d58f321400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.688+0000 7f588a597640 1 --2- 172.21.15.133:0/2601298331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f317080 0x55d58f321400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.688+0000 7f588a597640 1 -- 172.21.15.133:0/2601298331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 msgr2=0x55d58f323d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.688+0000 7f588a597640 1 --2- 172.21.15.133:0/2601298331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f323d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.688+0000 7f588a597640 1 -- 172.21.15.133:0/2601298331 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d58efedb90 con 0x55d58f316760 2024-09-11T23:45:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.689+0000 7f5889595640 1 --2- 172.21.15.133:0/2601298331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f323d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:13.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.694+0000 7f588a597640 1 --2- 172.21.15.133:0/2601298331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f316b40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f587c009be0 tx=0x7f587c046000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96e01510478fee93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:13.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.698+0000 7f586ffff640 1 -- 172.21.15.133:0/2601298331 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c04f070 con 0x55d58f316760 2024-09-11T23:45:13.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.698+0000 7f586ffff640 1 -- 172.21.15.133:0/2601298331 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c031c60 con 0x55d58f316760 2024-09-11T23:45:13.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.698+0000 7f586ffff640 1 -- 172.21.15.133:0/2601298331 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c036040 con 0x55d58f316760 2024-09-11T23:45:13.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.699+0000 7f588e3d3740 1 -- 172.21.15.133:0/2601298331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 msgr2=0x55d58f316b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:13.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.699+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2601298331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f316b40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f587c009be0 tx=0x7f587c046000 comp rx=0 tx=0).stop 2024-09-11T23:45:13.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.700+0000 7f588e3d3740 1 -- 172.21.15.133:0/2601298331 shutdown_connections 2024-09-11T23:45:13.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.700+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2601298331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f323d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.700+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2601298331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f317080 0x55d58f321400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.700+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2601298331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f316b40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.700+0000 7f588e3d3740 1 -- 172.21.15.133:0/2601298331 >> 172.21.15.133:0/2601298331 conn(0x55d58f312cf0 msgr2=0x55d58f3130d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.704+0000 7f588e3d3740 1 -- 172.21.15.133:0/2601298331 shutdown_connections 2024-09-11T23:45:13.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.707+0000 7f588e3d3740 1 -- 172.21.15.133:0/2601298331 wait complete. 2024-09-11T23:45:13.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.708+0000 7f588e3d3740 1 -- start start 2024-09-11T23:45:13.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.708+0000 7f588e3d3740 1 Processor -- start 2024-09-11T23:45:13.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.709+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f31e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.709+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f317080 0x55d58f31ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.709+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f31f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.709+0000 7f588e3d3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d58f324510 con 0x55d58f321940 2024-09-11T23:45:13.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.709+0000 7f588e3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d58f2046e0 con 0x55d58f317080 2024-09-11T23:45:13.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.709+0000 7f588e3d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d58f324690 con 0x55d58f316760 2024-09-11T23:45:13.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.709+0000 7f588e3d3740 10 client.0 _ll_register_callbacks cb 0x55d58f228a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:13.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.711+0000 7f5889595640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f317080 0x55d58f31ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.711+0000 7f5889595640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f317080 0x55d58f31ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:54322/0 (socket says 172.21.15.133:54322) 2024-09-11T23:45:13.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.711+0000 7f5889595640 1 -- 172.21.15.133:0/2023321899 learned_addr learned my addr 172.21.15.133:0/2023321899 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.713+0000 7f5889d96640 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f31e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.713+0000 7f5889595640 1 -- 172.21.15.133:0/2023321899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 msgr2=0x55d58f31e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.713+0000 7f5889595640 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f31e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.713+0000 7f5889595640 1 -- 172.21.15.133:0/2023321899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 msgr2=0x55d58f31f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.713+0000 7f5889595640 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f31f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.713+0000 7f5889595640 1 -- 172.21.15.133:0/2023321899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d58f32dae0 con 0x55d58f317080 2024-09-11T23:45:13.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.714+0000 7f5889595640 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f317080 0x55d58f31ee30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58800cd410 tx=0x7f588009b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47e6d125e5fcbc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:13.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.714+0000 7f586d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:45:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.715+0000 7f586d7fa640 1 -- 172.21.15.133:0/2023321899 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588009ff90 con 0x55d58f317080 2024-09-11T23:45:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.715+0000 7f586d7fa640 1 -- 172.21.15.133:0/2023321899 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f58800c7d00 con 0x55d58f317080 2024-09-11T23:45:13.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.717+0000 7f588e3d3740 1 -- 172.21.15.133:0/2023321899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f587c031c60 con 0x55d58f317080 2024-09-11T23:45:13.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.725+0000 7f586d7fa640 1 -- 172.21.15.133:0/2023321899 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58800a4850 con 0x55d58f317080 2024-09-11T23:45:13.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.726+0000 7f588e3d3740 10 client.27116 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:13.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.726+0000 7f588e3d3740 1 -- 172.21.15.133:0/2023321899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d58f330450 con 0x55d58f317080 2024-09-11T23:45:13.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.727+0000 7f586d7fa640 1 -- 172.21.15.133:0/2023321899 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c031c60 con 0x55d58f317080 2024-09-11T23:45:13.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.728+0000 7f586d7fa640 1 -- 172.21.15.133:0/2023321899 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5880005000 con 0x55d58f317080 2024-09-11T23:45:13.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.731+0000 7f588e3d3740 10 client.27116 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:13.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.731+0000 7f588e3d3740 10 client.27116 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:45:13.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.731+0000 7f588e3d3740 20 client.27116 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:13.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.731+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d58f3318b0 0x55d58f333d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.731+0000 7f588e3d3740 4 client.27116 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:13.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.731+0000 7f588e3d3740 1 -- 172.21.15.133:0/2023321899 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d58f2899d0 con 0x55d58f3318b0 2024-09-11T23:45:13.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.732+0000 7f5889d96640 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d58f3318b0 0x55d58f333d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.746+0000 7f5889d96640 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d58f3318b0 0x55d58f333d50 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4cbb07d7e350a03 server_cookie=839545866bbbfd3f in_seq=0 out_seq=0 2024-09-11T23:45:13.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.746+0000 7f586d7fa640 10 client.27116 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:13.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.748+0000 7f586d7fa640 1 -- 172.21.15.133:0/2023321899 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d58f2899d0 con 0x55d58f3318b0 2024-09-11T23:45:13.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.748+0000 7f586d7fa640 10 client.27116 handle_command_reply: tid=0 2024-09-11T23:45:13.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.748+0000 7f588e3d3740 1 client.27116 shutdown 2024-09-11T23:45:13.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.751+0000 7f588e3d3740 1 -- 172.21.15.133:0/2023321899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f317080 msgr2=0x55d58f31ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:13.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.751+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f317080 0x55d58f31ee30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58800cd410 tx=0x7f588009b600 comp rx=0 tx=0).stop 2024-09-11T23:45:13.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.751+0000 7f588e3d3740 1 -- 172.21.15.133:0/2023321899 shutdown_connections 2024-09-11T23:45:13.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.751+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d58f3318b0 0x55d58f333d50 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.751+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f31f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.751+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f317080 0x55d58f31ee30 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58800cd410 tx=0x7f588009b600 comp rx=0 tx=0).stop 2024-09-11T23:45:13.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.751+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2023321899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f31e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.751+0000 7f588e3d3740 1 -- 172.21.15.133:0/2023321899 >> 172.21.15.133:0/2023321899 conn(0x55d58f312cf0 msgr2=0x55d58f3189d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:13.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.751+0000 7f586d7fa640 0 client.27116 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:13.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.752+0000 7f588e3d3740 1 -- 172.21.15.133:0/2023321899 shutdown_connections 2024-09-11T23:45:13.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.752+0000 7f588e3d3740 1 -- 172.21.15.133:0/2023321899 wait complete. 2024-09-11T23:45:13.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.752+0000 7f588e3d3740 20 client.27116 trim_cache size 0 max 16384 2024-09-11T23:45:13.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.794+0000 7f588e3d3740 1 Processor -- start 2024-09-11T23:45:13.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.794+0000 7f588e3d3740 1 -- start start 2024-09-11T23:45:13.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.795+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f316760 0x55d58f339620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.796+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f321940 0x55d58f339b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.796+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f31bb50 0x55d58f31bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.796+0000 7f588e3d3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5884082e10 con 0x55d58f316760 2024-09-11T23:45:13.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.797+0000 7f588e3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5884084950 con 0x55d58f31bb50 2024-09-11T23:45:13.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.797+0000 7f588e3d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58840834e0 con 0x55d58f321940 2024-09-11T23:45:13.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.798+0000 7f5889595640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f321940 0x55d58f339b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.799+0000 7f588a597640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f31bb50 0x55d58f31bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.799+0000 7f5889595640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f321940 0x55d58f339b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:46148/0 (socket says 172.21.15.133:46148) 2024-09-11T23:45:13.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.799+0000 7f5889595640 1 -- 172.21.15.133:0/1904712412 learned_addr learned my addr 172.21.15.133:0/1904712412 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:13.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.799+0000 7f588a597640 1 -- 172.21.15.133:0/1904712412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f321940 msgr2=0x55d58f339b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:13.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.799+0000 7f588a597640 1 --2- 172.21.15.133:0/1904712412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f321940 0x55d58f339b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.800+0000 7f588a597640 1 -- 172.21.15.133:0/1904712412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f316760 msgr2=0x55d58f339620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:13.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.800+0000 7f588a597640 1 --2- 172.21.15.133:0/1904712412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f316760 0x55d58f339620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.800+0000 7f588a597640 1 -- 172.21.15.133:0/1904712412 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d58f313980 con 0x55d58f31bb50 2024-09-11T23:45:13.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.801+0000 7f588a597640 1 --2- 172.21.15.133:0/1904712412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f31bb50 0x55d58f31bf90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f587c02ac30 tx=0x7f587c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6324ae506e344d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:13.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.801+0000 7f586f7fe640 1 -- 172.21.15.133:0/1904712412 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c058070 con 0x55d58f31bb50 2024-09-11T23:45:13.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.801+0000 7f586f7fe640 1 -- 172.21.15.133:0/1904712412 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c031c60 con 0x55d58f31bb50 2024-09-11T23:45:13.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.803+0000 7f588e3d3740 1 -- 172.21.15.133:0/1904712412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f31bb50 msgr2=0x55d58f31bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:13.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.803+0000 7f588e3d3740 1 --2- 172.21.15.133:0/1904712412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f31bb50 0x55d58f31bf90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f587c02ac30 tx=0x7f587c02a710 comp rx=0 tx=0).stop 2024-09-11T23:45:13.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.804+0000 7f588e3d3740 1 -- 172.21.15.133:0/1904712412 shutdown_connections 2024-09-11T23:45:13.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.804+0000 7f588e3d3740 1 --2- 172.21.15.133:0/1904712412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f31bb50 0x55d58f31bf90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.804+0000 7f588e3d3740 1 --2- 172.21.15.133:0/1904712412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f321940 0x55d58f339b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.804+0000 7f588e3d3740 1 --2- 172.21.15.133:0/1904712412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f316760 0x55d58f339620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.804+0000 7f588e3d3740 1 -- 172.21.15.133:0/1904712412 >> 172.21.15.133:0/1904712412 conn(0x55d58f312cf0 msgr2=0x55d58f31f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:13.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.804+0000 7f588e3d3740 1 -- 172.21.15.133:0/1904712412 shutdown_connections 2024-09-11T23:45:13.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.804+0000 7f588e3d3740 1 -- 172.21.15.133:0/1904712412 wait complete. 2024-09-11T23:45:13.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.805+0000 7f588e3d3740 1 -- start start 2024-09-11T23:45:13.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.805+0000 7f588e3d3740 1 Processor -- start 2024-09-11T23:45:13.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.806+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f329d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.806+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f31bb50 0x55d58f3268b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.806+0000 7f588e3d3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f326df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.806+0000 7f588e3d3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5884084280 con 0x55d58f321940 2024-09-11T23:45:13.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.806+0000 7f588e3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5884083bb0 con 0x55d58f31bb50 2024-09-11T23:45:13.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.806+0000 7f588e3d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5884080e10 con 0x55d58f316760 2024-09-11T23:45:13.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.808+0000 7f588e3d3740 10 client.0 _ll_register_callbacks cb 0x7f588009b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:13.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.808+0000 7f588a597640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f326df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.808+0000 7f588a597640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f326df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36962/0 (socket says 172.21.15.133:36962) 2024-09-11T23:45:13.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.808+0000 7f588a597640 1 -- 172.21.15.133:0/2677757982 learned_addr learned my addr 172.21.15.133:0/2677757982 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:13.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.809+0000 7f5889d96640 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f329d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.809+0000 7f5889595640 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f31bb50 0x55d58f3268b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.809+0000 7f588a597640 1 -- 172.21.15.133:0/2677757982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 msgr2=0x55d58f329d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:13.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.809+0000 7f588a597640 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f329d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.809+0000 7f588a597640 1 -- 172.21.15.133:0/2677757982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f31bb50 msgr2=0x55d58f3268b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:13.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.809+0000 7f588a597640 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f31bb50 0x55d58f3268b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:13.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.809+0000 7f588a597640 1 -- 172.21.15.133:0/2677757982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d58f327360 con 0x55d58f321940 2024-09-11T23:45:13.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.809+0000 7f588a597640 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f326df0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f587c009b30 tx=0x7f587c030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299c539d85fa7202 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:13.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.811+0000 7f586d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:13.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.811+0000 7f586d7fa640 1 -- 172.21.15.133:0/2677757982 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c058040 con 0x55d58f321940 2024-09-11T23:45:13.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.811+0000 7f588e3d3740 1 -- 172.21.15.133:0/2677757982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f587c031c60 con 0x55d58f321940 2024-09-11T23:45:13.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.811+0000 7f586d7fa640 1 -- 172.21.15.133:0/2677757982 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f587c031980 con 0x55d58f321940 2024-09-11T23:45:13.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.813+0000 7f586d7fa640 1 -- 172.21.15.133:0/2677757982 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f587c06b050 con 0x55d58f321940 2024-09-11T23:45:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.813+0000 7f588e3d3740 10 client.17610 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.813+0000 7f588e3d3740 1 -- 172.21.15.133:0/2677757982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d58f39d380 con 0x55d58f321940 2024-09-11T23:45:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.814+0000 7f586d7fa640 1 -- 172.21.15.133:0/2677757982 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c031c60 con 0x55d58f321940 2024-09-11T23:45:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.814+0000 7f586d7fa640 1 -- 172.21.15.133:0/2677757982 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f587c05b050 con 0x55d58f321940 2024-09-11T23:45:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.814+0000 7f588e3d3740 10 client.17610 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.814+0000 7f588e3d3740 10 client.17610 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:45:13.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.814+0000 7f588e3d3740 20 client.17610 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:13.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.814+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d58f39da00 0x55d58f39fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:13.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.814+0000 7f588e3d3740 4 client.17610 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-11T23:45:13.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.814+0000 7f588e3d3740 1 -- 172.21.15.133:0/2677757982 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x55d58f283d20 con 0x55d58f39da00 2024-09-11T23:45:13.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.815+0000 7f5889d96640 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d58f39da00 0x55d58f39fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:13.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.822+0000 7f5889d96640 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d58f39da00 0x55d58f39fe20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ab9f167405e18e server_cookie=a1ee83095c022ee4 in_seq=0 out_seq=0 2024-09-11T23:45:13.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:13.823+0000 7f586d7fa640 10 client.17610 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f586d7fa640 1 -- 172.21.15.133:0/2677757982 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d58f283d20 con 0x55d58f39da00 2024-09-11T23:45:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f586d7fa640 10 client.17610 handle_command_reply: tid=0 2024-09-11T23:45:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f588e3d3740 1 client.17610 shutdown 2024-09-11T23:45:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f588e3d3740 1 -- 172.21.15.133:0/2677757982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 msgr2=0x55d58f326df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f326df0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f587c009b30 tx=0x7f587c030cc0 comp rx=0 tx=0).stop 2024-09-11T23:45:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f588e3d3740 1 -- 172.21.15.133:0/2677757982 shutdown_connections 2024-09-11T23:45:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d58f39da00 0x55d58f39fe20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d58f321940 0x55d58f326df0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58f31bb50 0x55d58f3268b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f588e3d3740 1 --2- 172.21.15.133:0/2677757982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d58f316760 0x55d58f329d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f586d7fa640 0 client.17610 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.118+0000 7f588e3d3740 1 -- 172.21.15.133:0/2677757982 >> 172.21.15.133:0/2677757982 conn(0x55d58f312cf0 msgr2=0x55d58f320970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.119+0000 7f588e3d3740 1 -- 172.21.15.133:0/2677757982 shutdown_connections 2024-09-11T23:45:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.119+0000 7f588e3d3740 1 -- 172.21.15.133:0/2677757982 wait complete. 2024-09-11T23:45:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.119+0000 7f588e3d3740 20 client.17610 trim_cache size 0 max 16384 2024-09-11T23:45:14.120 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:45:14.120 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:45:14.120 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:45:14.120 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 34, 2024-09-11T23:45:14.120 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:45:14.120 INFO:teuthology.orchestra.run.smithi133.stdout: "ff3854b3": { 2024-09-11T23:45:14.121 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 34, 2024-09-11T23:45:14.121 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:45:14.121 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:45:14.121 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:45:14.121 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:45:14.121 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:45:14.121 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:45:14.121 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 139.4, 2024-09-11T23:45:14.121 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:45:14.122 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:45:14.122 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:45:14.122 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:45:14.122 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:45:14.122 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:45:14.122 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:14.122 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:14.122 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:14.122 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:14.122 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:14.122 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:45:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 -- 172.21.15.133:0/1315122646 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58700a2e40 msgr2=0x7f58700a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58700a2e40 0x7f58700a5300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f587c02ad00 tx=0x7f587c031500 comp rx=0 tx=0).stop 2024-09-11T23:45:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 -- 172.21.15.133:0/1315122646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5884109830 msgr2=0x7f588411a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5884109830 0x7f588411a950 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f588000e9c0 tx=0x7f588000ee80 comp rx=0 tx=0).stop 2024-09-11T23:45:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 -- 172.21.15.133:0/1315122646 shutdown_connections 2024-09-11T23:45:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58700a2e40 0x7f58700a5300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f588411b140 0x7f5884114490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5884109830 0x7f588411a950 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 --2- 172.21.15.133:0/1315122646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5884109450 0x7f588411a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 -- 172.21.15.133:0/1315122646 >> 172.21.15.133:0/1315122646 conn(0x7f588406e140 msgr2=0x7f5884072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 -- 172.21.15.133:0/1315122646 shutdown_connections 2024-09-11T23:45:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:14.120+0000 7f588bffa640 1 -- 172.21.15.133:0/1315122646 wait complete. 2024-09-11T23:45:14.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:14 smithi138 ceph-mon[35142]: pgmap v262: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 68 GiB data, 230 GiB used, 843 GiB / 1.0 TiB avail; 101 KiB/s rd, 290 MiB/s wr, 1.72k op/s 2024-09-11T23:45:14.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:14 smithi133 ceph-mon[31430]: pgmap v262: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 68 GiB data, 230 GiB used, 843 GiB / 1.0 TiB avail; 101 KiB/s rd, 290 MiB/s wr, 1.72k op/s 2024-09-11T23:45:15.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:14 smithi178 ceph-mon[35882]: pgmap v262: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 68 GiB data, 230 GiB used, 843 GiB / 1.0 TiB avail; 101 KiB/s rd, 290 MiB/s wr, 1.72k op/s 2024-09-11T23:45:15.286 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ff3854b3', quiesce took 6.6 seconds. Will release after: 12.799999999999999 2024-09-11T23:45:16.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:16 smithi138 ceph-mon[35142]: pgmap v263: 97 pgs: 18 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 52 active+clean; 65 GiB data, 234 GiB used, 838 GiB / 1.0 TiB avail; 921 KiB/s rd, 408 MiB/s wr, 2.64k op/s 2024-09-11T23:45:16.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:16 smithi133 ceph-mon[31430]: pgmap v263: 97 pgs: 18 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 52 active+clean; 65 GiB data, 234 GiB used, 838 GiB / 1.0 TiB avail; 921 KiB/s rd, 408 MiB/s wr, 2.64k op/s 2024-09-11T23:45:17.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:16 smithi178 ceph-mon[35882]: pgmap v263: 97 pgs: 18 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 52 active+clean; 65 GiB data, 234 GiB used, 838 GiB / 1.0 TiB avail; 921 KiB/s rd, 408 MiB/s wr, 2.64k op/s 2024-09-11T23:45:18.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:18 smithi138 ceph-mon[35142]: pgmap v264: 97 pgs: 18 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 52 active+clean; 65 GiB data, 235 GiB used, 838 GiB / 1.0 TiB avail; 910 KiB/s rd, 418 MiB/s wr, 2.72k op/s 2024-09-11T23:45:18.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:18 smithi133 ceph-mon[31430]: pgmap v264: 97 pgs: 18 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 52 active+clean; 65 GiB data, 235 GiB used, 838 GiB / 1.0 TiB avail; 910 KiB/s rd, 418 MiB/s wr, 2.72k op/s 2024-09-11T23:45:19.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:18 smithi178 ceph-mon[35882]: pgmap v264: 97 pgs: 18 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 52 active+clean; 65 GiB data, 235 GiB used, 838 GiB / 1.0 TiB avail; 910 KiB/s rd, 418 MiB/s wr, 2.72k op/s 2024-09-11T23:45:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.397+0000 7fa5b1a53640 1 -- 172.21.15.133:0/560515589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac101390 msgr2=0x7fa5ac105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.397+0000 7fa5b1a53640 1 --2- 172.21.15.133:0/560515589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac101390 0x7fa5ac105850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa594009860 tx=0x7fa59402a3f0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.397+0000 7fa5b1a53640 1 -- 172.21.15.133:0/560515589 shutdown_connections 2024-09-11T23:45:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.397+0000 7fa5b1a53640 1 --2- 172.21.15.133:0/560515589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac105ea0 0x7fa5ac106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.397+0000 7fa5b1a53640 1 --2- 172.21.15.133:0/560515589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac101390 0x7fa5ac105850 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.397+0000 7fa5b1a53640 1 --2- 172.21.15.133:0/560515589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac100a70 0x7fa5ac100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.397+0000 7fa5b1a53640 1 -- 172.21.15.133:0/560515589 >> 172.21.15.133:0/560515589 conn(0x7fa5ac077990 msgr2=0x7fa5ac0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.399+0000 7fa5b1a53640 1 -- 172.21.15.133:0/560515589 shutdown_connections 2024-09-11T23:45:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.399+0000 7fa5b1a53640 1 -- 172.21.15.133:0/560515589 wait complete. 2024-09-11T23:45:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.400+0000 7fa5b1a53640 1 Processor -- start 2024-09-11T23:45:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.402+0000 7fa5b1a53640 1 -- start start 2024-09-11T23:45:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.403+0000 7fa5b1a53640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac100a70 0x7fa5ac0739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.403+0000 7fa5b1a53640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac101390 0x7fa5ac06ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.403+0000 7fa5b1a53640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac105ea0 0x7fa5ac06cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.403+0000 7fa5b1a53640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac118520 con 0x7fa5ac100a70 2024-09-11T23:45:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.403+0000 7fa5b1a53640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1183a0 con 0x7fa5ac105ea0 2024-09-11T23:45:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.403+0000 7fa5b1a53640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1186a0 con 0x7fa5ac101390 2024-09-11T23:45:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.403+0000 7fa5aaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac100a70 0x7fa5ac0739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.403+0000 7fa5aaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac100a70 0x7fa5ac0739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36990/0 (socket says 172.21.15.133:36990) 2024-09-11T23:45:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.403+0000 7fa5aaffd640 1 -- 172.21.15.133:0/2325896015 learned_addr learned my addr 172.21.15.133:0/2325896015 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.404+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac101390 0x7fa5ac06ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.404+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac105ea0 0x7fa5ac06cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.405+0000 7fa5aa7fc640 1 -- 172.21.15.133:0/2325896015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac105ea0 msgr2=0x7fa5ac06cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.405+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac105ea0 0x7fa5ac06cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.405+0000 7fa5aa7fc640 1 -- 172.21.15.133:0/2325896015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac100a70 msgr2=0x7fa5ac0739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.405+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac100a70 0x7fa5ac0739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.405+0000 7fa5aa7fc640 1 -- 172.21.15.133:0/2325896015 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac06d4e0 con 0x7fa5ac101390 2024-09-11T23:45:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.410+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac101390 0x7fa5ac06ca30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa59402a910 tx=0x7fa594004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.411+0000 7fa5b0a51640 1 -- 172.21.15.133:0/2325896015 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594031600 con 0x7fa5ac101390 2024-09-11T23:45:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.411+0000 7fa5b0a51640 1 -- 172.21.15.133:0/2325896015 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa594030880 con 0x7fa5ac101390 2024-09-11T23:45:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.411+0000 7fa5b0a51640 1 -- 172.21.15.133:0/2325896015 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594004920 con 0x7fa5ac101390 2024-09-11T23:45:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.412+0000 7fa5b1a53640 1 -- 172.21.15.133:0/2325896015 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac06d770 con 0x7fa5ac101390 2024-09-11T23:45:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.412+0000 7fa5b1a53640 1 -- 172.21.15.133:0/2325896015 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac0716b0 con 0x7fa5ac101390 2024-09-11T23:45:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.416+0000 7fa5b47ee740 1 Processor -- start 2024-09-11T23:45:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.417+0000 7fa5b47ee740 1 -- start start 2024-09-11T23:45:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.417+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb63af0 0x561e8cb63ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.417+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb64410 0x561e8cb6e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.417+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 0x561e8cb710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.417+0000 7fa5b47ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561e8cac82f0 con 0x561e8cb63af0 2024-09-11T23:45:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.417+0000 7fa5b47ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e8ca70270 con 0x561e8cb64410 2024-09-11T23:45:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.417+0000 7fa5b47ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e8cac42b0 con 0x561e8cb6ecb0 2024-09-11T23:45:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.418+0000 7fa5ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb63af0 0x561e8cb63ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.418+0000 7fa5ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb63af0 0x561e8cb63ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36996/0 (socket says 172.21.15.133:36996) 2024-09-11T23:45:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.418+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/1049651969 learned_addr learned my addr 172.21.15.133:0/1049651969 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.418+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/1049651969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb64410 0x561e8cb6e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.419+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/1049651969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 msgr2=0x561e8cb710a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:45:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.419+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/1049651969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 0x561e8cb710a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.419+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/1049651969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb64410 msgr2=0x561e8cb6e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.419+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/1049651969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb64410 0x561e8cb6e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.419+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/1049651969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e8ca6f340 con 0x561e8cb63af0 2024-09-11T23:45:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.419+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/1049651969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb63af0 0x561e8cb63ed0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4004640 tx=0x7fa5a4009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793e4ba5ecb99498 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.421+0000 7fa5a1ffb640 1 -- 172.21.15.133:0/1049651969 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4003bd0 con 0x561e8cb63af0 2024-09-11T23:45:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.422+0000 7fa5b0a51640 1 -- 172.21.15.133:0/2325896015 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa594039040 con 0x7fa5ac101390 2024-09-11T23:45:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.422+0000 7fa5a1ffb640 1 -- 172.21.15.133:0/1049651969 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a4003d70 con 0x561e8cb63af0 2024-09-11T23:45:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.423+0000 7fa5b0a51640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5980a2f80 0x7fa5980a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.423+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1049651969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb63af0 msgr2=0x561e8cb63ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.423+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/1049651969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb63af0 0x561e8cb63ed0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4004640 tx=0x7fa5a4009b70 comp rx=0 tx=0).stop 2024-09-11T23:45:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.423+0000 7fa5b0a51640 1 -- 172.21.15.133:0/2325896015 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7fa5940eff90 con 0x7fa5ac101390 2024-09-11T23:45:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.426+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5980a2f80 0x7fa5980a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.426+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1049651969 shutdown_connections 2024-09-11T23:45:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.426+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/1049651969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 0x561e8cb710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.426+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/1049651969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb64410 0x561e8cb6e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.426+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/1049651969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb63af0 0x561e8cb63ed0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.426+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1049651969 >> 172.21.15.133:0/1049651969 conn(0x561e8cb60060 msgr2=0x561e8cb60440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.426+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5980a2f80 0x7fa5980a5440 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x561e8cb60ec0 tx=0x7fa59c00b040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.427+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1049651969 shutdown_connections 2024-09-11T23:45:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.427+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1049651969 wait complete. 2024-09-11T23:45:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.427+0000 7fa5b47ee740 1 -- start start 2024-09-11T23:45:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.428+0000 7fa5b47ee740 1 Processor -- start 2024-09-11T23:45:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.428+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb6bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.428+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb64410 0x561e8cb6c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.428+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 0x561e8cb6c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.428+0000 7fa5b47ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561e8cb718d0 con 0x561e8cb64410 2024-09-11T23:45:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.428+0000 7fa5b47ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e8ca51290 con 0x561e8cb63af0 2024-09-11T23:45:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.428+0000 7fa5b47ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e8cb71a50 con 0x561e8cb6ecb0 2024-09-11T23:45:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.429+0000 7fa5b47ee740 10 client.0 _ll_register_callbacks cb 0x561e8ca82140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.431+0000 7fa5aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb6bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.431+0000 7fa5aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb6bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:54364/0 (socket says 172.21.15.133:54364) 2024-09-11T23:45:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.431+0000 7fa5aaffd640 1 -- 172.21.15.133:0/1493068278 learned_addr learned my addr 172.21.15.133:0/1493068278 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.431+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb64410 0x561e8cb6c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.434+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 0x561e8cb6c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.434+0000 7fa5aaffd640 1 -- 172.21.15.133:0/1493068278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 msgr2=0x561e8cb6c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.434+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 0x561e8cb6c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.434+0000 7fa5aaffd640 1 -- 172.21.15.133:0/1493068278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb64410 msgr2=0x561e8cb6c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.434+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb64410 0x561e8cb6c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.434+0000 7fa5aaffd640 1 -- 172.21.15.133:0/1493068278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8c7db210 con 0x561e8cb63af0 2024-09-11T23:45:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.435+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb6bb50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa59c017e70 tx=0x7fa59c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8e27efb9876d628 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.435+0000 7fa58affd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:45:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.436+0000 7fa58affd640 1 -- 172.21.15.133:0/1493068278 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c01b070 con 0x561e8cb63af0 2024-09-11T23:45:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.436+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1493068278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a4003d70 con 0x561e8cb63af0 2024-09-11T23:45:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.436+0000 7fa58affd640 1 -- 172.21.15.133:0/1493068278 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7fa59c0217e0 con 0x561e8cb63af0 2024-09-11T23:45:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.437+0000 7fa58affd640 1 -- 172.21.15.133:0/1493068278 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa59c030050 con 0x561e8cb63af0 2024-09-11T23:45:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.440+0000 7fa5b47ee740 10 client.27134 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.440+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1493068278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e8c8a7620 con 0x561e8cb63af0 2024-09-11T23:45:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.445+0000 7fa58affd640 1 -- 172.21.15.133:0/1493068278 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a4003d70 con 0x561e8cb63af0 2024-09-11T23:45:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.445+0000 7fa58affd640 1 -- 172.21.15.133:0/1493068278 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa59c022a80 con 0x561e8cb63af0 2024-09-11T23:45:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.445+0000 7fa5b47ee740 10 client.27134 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.445+0000 7fa5b47ee740 10 client.27134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.445+0000 7fa5b47ee740 20 client.27134 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.445+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561e8cb7ed10 0x561e8cb81100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.445+0000 7fa5b47ee740 4 client.27134 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.445+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1493068278 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e8cad5a20 con 0x561e8cb7ed10 2024-09-11T23:45:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.447+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561e8cb7ed10 0x561e8cb81100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.456+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561e8cb7ed10 0x561e8cb81100 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1e5c22d5ec9db43 server_cookie=cbe011f397f82c80 in_seq=0 out_seq=0 2024-09-11T23:45:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.456+0000 7fa58affd640 10 client.27134 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.459+0000 7fa58affd640 1 -- 172.21.15.133:0/1493068278 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e8cad5a20 con 0x561e8cb7ed10 2024-09-11T23:45:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.459+0000 7fa58affd640 10 client.27134 handle_command_reply: tid=0 2024-09-11T23:45:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.462+0000 7fa5b47ee740 1 client.27134 shutdown 2024-09-11T23:45:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.462+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1493068278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 msgr2=0x561e8cb6bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.462+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb6bb50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa59c017e70 tx=0x7fa59c004a40 comp rx=0 tx=0).stop 2024-09-11T23:45:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.467+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1493068278 shutdown_connections 2024-09-11T23:45:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.467+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561e8cb7ed10 0x561e8cb81100 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.467+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 0x561e8cb6c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.467+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb64410 0x561e8cb6c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.467+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/1493068278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb6bb50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.467+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1493068278 >> 172.21.15.133:0/1493068278 conn(0x561e8cb60060 msgr2=0x561e8cb65dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.467+0000 7fa58affd640 0 client.27134 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.473+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1493068278 shutdown_connections 2024-09-11T23:45:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.476+0000 7fa5b47ee740 1 -- 172.21.15.133:0/1493068278 wait complete. 2024-09-11T23:45:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.477+0000 7fa5b47ee740 20 client.27134 trim_cache size 0 max 16384 2024-09-11T23:45:19.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.520+0000 7fa5b47ee740 1 Processor -- start 2024-09-11T23:45:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.524+0000 7fa5b47ee740 1 -- start start 2024-09-11T23:45:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.524+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.524+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb64410 0x561e8cb6c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.524+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb6ecb0 0x561e8cb869e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.524+0000 7fa5b47ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac10fa70 con 0x561e8cb6ecb0 2024-09-11T23:45:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.524+0000 7fa5b47ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1b5830 con 0x561e8cb63af0 2024-09-11T23:45:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.524+0000 7fa5b47ee740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac110140 con 0x561e8cb64410 2024-09-11T23:45:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.527+0000 7fa5ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb6ecb0 0x561e8cb869e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.527+0000 7fa5ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb6ecb0 0x561e8cb869e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37020/0 (socket says 172.21.15.133:37020) 2024-09-11T23:45:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.527+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/3749856054 learned_addr learned my addr 172.21.15.133:0/3749856054 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.530+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/3749856054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.530+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/3749856054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb64410 msgr2=0x561e8cb6c400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:45:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.530+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/3749856054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb64410 0x561e8cb6c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.530+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/3749856054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 msgr2=0x561e8cb665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.530+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/3749856054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.530+0000 7fa5ab7fe640 1 -- 172.21.15.133:0/3749856054 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e8cb7d8b0 con 0x561e8cb6ecb0 2024-09-11T23:45:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.534+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/3749856054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb6ecb0 0x561e8cb869e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4004120 tx=0x7fa5a4004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5397b7554cad1b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.541+0000 7fa5a17fa640 1 -- 172.21.15.133:0/3749856054 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a403b3e0 con 0x561e8cb6ecb0 2024-09-11T23:45:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.542+0000 7fa5a17fa640 1 -- 172.21.15.133:0/3749856054 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a4009dc0 con 0x561e8cb6ecb0 2024-09-11T23:45:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.542+0000 7fa5a17fa640 1 -- 172.21.15.133:0/3749856054 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40054e0 con 0x561e8cb6ecb0 2024-09-11T23:45:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.544+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3749856054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb6ecb0 msgr2=0x561e8cb869e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.544+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/3749856054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb6ecb0 0x561e8cb869e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4004120 tx=0x7fa5a4004150 comp rx=0 tx=0).stop 2024-09-11T23:45:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.545+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3749856054 shutdown_connections 2024-09-11T23:45:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.545+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/3749856054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb6ecb0 0x561e8cb869e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.545+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/3749856054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb64410 0x561e8cb6c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.545+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/3749856054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.545+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3749856054 >> 172.21.15.133:0/3749856054 conn(0x561e8cb60060 msgr2=0x561e8cb6dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.549+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3749856054 shutdown_connections 2024-09-11T23:45:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.549+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3749856054 wait complete. 2024-09-11T23:45:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.549+0000 7fa5b47ee740 1 -- start start 2024-09-11T23:45:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.550+0000 7fa5b47ee740 1 Processor -- start 2024-09-11T23:45:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.552+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb76fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.552+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb64410 0x561e8cb77510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.552+0000 7fa5b47ee740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 0x561e8cb68440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.552+0000 7fa5b47ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1b5160 con 0x561e8cb64410 2024-09-11T23:45:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.552+0000 7fa5b47ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1b4ab0 con 0x561e8cb63af0 2024-09-11T23:45:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.553+0000 7fa5aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb76fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.553+0000 7fa5aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb76fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:54370/0 (socket says 172.21.15.133:54370) 2024-09-11T23:45:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.553+0000 7fa5aaffd640 1 -- 172.21.15.133:0/3962632668 learned_addr learned my addr 172.21.15.133:0/3962632668 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.553+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3962632668 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac10da70 con 0x561e8cb6ecb0 2024-09-11T23:45:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.553+0000 7fa5b47ee740 10 client.0 _ll_register_callbacks cb 0x7fa59c009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.556+0000 7fa5ab7fe640 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 0x561e8cb68440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.556+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb64410 0x561e8cb77510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.556+0000 7fa5aaffd640 1 -- 172.21.15.133:0/3962632668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 msgr2=0x561e8cb68440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.556+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 0x561e8cb68440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.556+0000 7fa5aaffd640 1 -- 172.21.15.133:0/3962632668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb64410 msgr2=0x561e8cb77510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.556+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb64410 0x561e8cb77510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.556+0000 7fa5aaffd640 1 -- 172.21.15.133:0/3962632668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8cb5b2d0 con 0x561e8cb63af0 2024-09-11T23:45:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.560+0000 7fa5aaffd640 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb76fd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa59c013470 tx=0x7fa59c022700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c899ce78b120b696 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.563+0000 7fa58affd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:45:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.563+0000 7fa58affd640 1 -- 172.21.15.133:0/3962632668 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c01b040 con 0x561e8cb63af0 2024-09-11T23:45:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.563+0000 7fa58affd640 1 -- 172.21.15.133:0/3962632668 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7fa59c0215e0 con 0x561e8cb63af0 2024-09-11T23:45:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.564+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3962632668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a4009dc0 con 0x561e8cb63af0 2024-09-11T23:45:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.571+0000 7fa58affd640 1 -- 172.21.15.133:0/3962632668 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa59c0217e0 con 0x561e8cb63af0 2024-09-11T23:45:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.578+0000 7fa5b47ee740 10 client.27140 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.578+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3962632668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e8cbe9320 con 0x561e8cb63af0 2024-09-11T23:45:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.581+0000 7fa58affd640 1 -- 172.21.15.133:0/3962632668 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa59c027070 con 0x561e8cb63af0 2024-09-11T23:45:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.581+0000 7fa58affd640 1 -- 172.21.15.133:0/3962632668 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa59c013a10 con 0x561e8cb63af0 2024-09-11T23:45:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.581+0000 7fa5b47ee740 10 client.27140 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.581+0000 7fa5b47ee740 10 client.27140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.581+0000 7fa5b47ee740 20 client.27140 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.581+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561e8cbe9a10 0x561e8cbebe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.581+0000 7fa5b47ee740 4 client.27140 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:45:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.581+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3962632668 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa5a4009dc0 con 0x561e8cbe9a10 2024-09-11T23:45:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.582+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561e8cbe9a10 0x561e8cbebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.584+0000 7fa5aa7fc640 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561e8cbe9a10 0x561e8cbebe30 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5962a42f50bdb88 server_cookie=bfb24cc2a6d4cb31 in_seq=0 out_seq=0 2024-09-11T23:45:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.584+0000 7fa58affd640 10 client.27140 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.756+0000 7fa58affd640 1 -- 172.21.15.133:0/3962632668 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa5a4009dc0 con 0x561e8cbe9a10 2024-09-11T23:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.756+0000 7fa58affd640 10 client.27140 handle_command_reply: tid=0 2024-09-11T23:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.756+0000 7fa5b47ee740 1 client.27140 shutdown 2024-09-11T23:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3962632668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 msgr2=0x561e8cb76fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb76fd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa59c013470 tx=0x7fa59c022700 comp rx=0 tx=0).stop 2024-09-11T23:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3962632668 shutdown_connections 2024-09-11T23:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x561e8cbe9a10 0x561e8cbebe30 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e8cb6ecb0 0x561e8cb68440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8cb64410 0x561e8cb77510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b47ee740 1 --2- 172.21.15.133:0/3962632668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e8cb63af0 0x561e8cb76fd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa58affd640 0 client.27140 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3962632668 >> 172.21.15.133:0/3962632668 conn(0x561e8cb60060 msgr2=0x561e8cb5e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3962632668 shutdown_connections 2024-09-11T23:45:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b47ee740 1 -- 172.21.15.133:0/3962632668 wait complete. 2024-09-11T23:45:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b47ee740 20 client.27140 trim_cache size 0 max 16384 2024-09-11T23:45:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:45:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:45:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b1a53640 1 -- 172.21.15.133:0/2325896015 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5980a2f80 msgr2=0x7fa5980a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b1a53640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5980a2f80 0x7fa5980a5440 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x561e8cb60ec0 tx=0x7fa59c00b040 comp rx=0 tx=0).stop 2024-09-11T23:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b1a53640 1 -- 172.21.15.133:0/2325896015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac101390 msgr2=0x7fa5ac06ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b1a53640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac101390 0x7fa5ac06ca30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa59402a910 tx=0x7fa594004270 comp rx=0 tx=0).stop 2024-09-11T23:45:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b1a53640 1 -- 172.21.15.133:0/2325896015 shutdown_connections 2024-09-11T23:45:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b1a53640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa5980a2f80 0x7fa5980a5440 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b1a53640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5ac105ea0 0x7fa5ac06cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b1a53640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5ac101390 0x7fa5ac06ca30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b1a53640 1 --2- 172.21.15.133:0/2325896015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ac100a70 0x7fa5ac0739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.757+0000 7fa5b1a53640 1 -- 172.21.15.133:0/2325896015 >> 172.21.15.133:0/2325896015 conn(0x7fa5ac077990 msgr2=0x7fa5ac103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.758+0000 7fa5b1a53640 1 -- 172.21.15.133:0/2325896015 shutdown_connections 2024-09-11T23:45:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:19.758+0000 7fa5b1a53640 1 -- 172.21.15.133:0/2325896015 wait complete. 2024-09-11T23:45:20.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:19 smithi178 ceph-mon[35882]: pgmap v265: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 61 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 382 MiB/s wr, 2.59k op/s 2024-09-11T23:45:20.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:19 smithi138 ceph-mon[35142]: pgmap v265: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 61 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 382 MiB/s wr, 2.59k op/s 2024-09-11T23:45:20.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:19 smithi133 ceph-mon[31430]: pgmap v265: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 61 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 382 MiB/s wr, 2.59k op/s 2024-09-11T23:45:22.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:22 smithi138 ceph-mon[35142]: pgmap v266: 97 pgs: 97 active+clean; 61 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 330 MiB/s wr, 2.29k op/s 2024-09-11T23:45:22.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:22 smithi133 ceph-mon[31430]: pgmap v266: 97 pgs: 97 active+clean; 61 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 330 MiB/s wr, 2.29k op/s 2024-09-11T23:45:23.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:22 smithi178 ceph-mon[35882]: pgmap v266: 97 pgs: 97 active+clean; 61 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 330 MiB/s wr, 2.29k op/s 2024-09-11T23:45:24.918 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:45:24.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:24 smithi138 ceph-mon[35142]: pgmap v267: 97 pgs: 97 active+clean; 62 GiB data, 219 GiB used, 854 GiB / 1.0 TiB avail; 969 KiB/s rd, 404 MiB/s wr, 2.28k op/s 2024-09-11T23:45:24.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:24 smithi138 ceph-mon[35142]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:45:24.962 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:24 smithi133 ceph-mon[31430]: pgmap v267: 97 pgs: 97 active+clean; 62 GiB data, 219 GiB used, 854 GiB / 1.0 TiB avail; 969 KiB/s rd, 404 MiB/s wr, 2.28k op/s 2024-09-11T23:45:24.963 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:24 smithi133 ceph-mon[31430]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:45:25.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:24 smithi178 ceph-mon[35882]: pgmap v267: 97 pgs: 97 active+clean; 62 GiB data, 219 GiB used, 854 GiB / 1.0 TiB avail; 969 KiB/s rd, 404 MiB/s wr, 2.28k op/s 2024-09-11T23:45:25.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:24 smithi178 ceph-mon[35882]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:45:25.465 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:26.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.020+0000 7fbebff58640 1 -- 172.21.15.133:0/452539476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeb8101de0 msgr2=0x7fbeb81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:26.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.020+0000 7fbebff58640 1 --2- 172.21.15.133:0/452539476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeb8101de0 0x7fbeb81021c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009900 tx=0x7fbeb402a410 comp rx=0 tx=0).stop 2024-09-11T23:45:26.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.020+0000 7fbebff58640 1 -- 172.21.15.133:0/452539476 shutdown_connections 2024-09-11T23:45:26.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.020+0000 7fbebff58640 1 --2- 172.21.15.133:0/452539476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb8106f10 0x7fbeb810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.020+0000 7fbebff58640 1 --2- 172.21.15.133:0/452539476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeb8102790 0x7fbeb81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.020+0000 7fbebff58640 1 --2- 172.21.15.133:0/452539476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeb8101de0 0x7fbeb81021c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.020+0000 7fbebff58640 1 -- 172.21.15.133:0/452539476 >> 172.21.15.133:0/452539476 conn(0x7fbeb80fd8e0 msgr2=0x7fbeb80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:26.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.020+0000 7fbebff58640 1 -- 172.21.15.133:0/452539476 shutdown_connections 2024-09-11T23:45:26.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.020+0000 7fbebff58640 1 -- 172.21.15.133:0/452539476 wait complete. 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.021+0000 7fbebff58640 1 Processor -- start 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.021+0000 7fbebff58640 1 -- start start 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.021+0000 7fbebff58640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb8101de0 0x7fbeb8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.021+0000 7fbebff58640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeb8102790 0x7fbeb8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.021+0000 7fbebff58640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeb8106f10 0x7fbeb819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.021+0000 7fbebff58640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8111f30 con 0x7fbeb8102790 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.021+0000 7fbebff58640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8111db0 con 0x7fbeb8106f10 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.021+0000 7fbebff58640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81120b0 con 0x7fbeb8101de0 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebd4f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeb8102790 0x7fbeb8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebd4f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeb8102790 0x7fbeb8198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40578/0 (socket says 172.21.15.133:40578) 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebd4f3640 1 -- 172.21.15.133:0/2175182504 learned_addr learned my addr 172.21.15.133:0/2175182504 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebe4f5640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeb8106f10 0x7fbeb819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb8101de0 0x7fbeb8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebd4f3640 1 -- 172.21.15.133:0/2175182504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb8101de0 msgr2=0x7fbeb8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:26.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb8101de0 0x7fbeb8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebd4f3640 1 -- 172.21.15.133:0/2175182504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeb8106f10 msgr2=0x7fbeb819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:26.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeb8106f10 0x7fbeb819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebd4f3640 1 -- 172.21.15.133:0/2175182504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb819d3c0 con 0x7fbeb8102790 2024-09-11T23:45:26.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb8101de0 0x7fbeb8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:45:26.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeb8102790 0x7fbeb8198950 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbea800d950 tx=0x7fbea800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:26.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbeaeffd640 1 -- 172.21.15.133:0/2175182504 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8017070 con 0x7fbeb8102790 2024-09-11T23:45:26.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbeaeffd640 1 -- 172.21.15.133:0/2175182504 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea8009c30 con 0x7fbeb8102790 2024-09-11T23:45:26.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebff58640 1 -- 172.21.15.133:0/2175182504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb819d6b0 con 0x7fbeb8102790 2024-09-11T23:45:26.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbeaeffd640 1 -- 172.21.15.133:0/2175182504 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea800f6b0 con 0x7fbeb8102790 2024-09-11T23:45:26.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.022+0000 7fbebff58640 1 -- 172.21.15.133:0/2175182504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb81a4f90 con 0x7fbeb8102790 2024-09-11T23:45:26.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.024+0000 7fbeaeffd640 1 -- 172.21.15.133:0/2175182504 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbea80109d0 con 0x7fbeb8102790 2024-09-11T23:45:26.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbec2331740 1 Processor -- start 2024-09-11T23:45:26.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbec2331740 1 -- start start 2024-09-11T23:45:26.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbec2331740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d323f74a0 0x558d323f7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbec2331740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 0x558d32402140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbec2331740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d32402680 0x558d32404a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbec2331740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558d3235d2e0 con 0x558d32402680 2024-09-11T23:45:26.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbec2331740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d32304ce0 con 0x558d323f74a0 2024-09-11T23:45:26.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbec2331740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558d323592a0 con 0x558d323f7dc0 2024-09-11T23:45:26.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbeaeffd640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbea40a2cf0 0x7fbea40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbebd4f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d32402680 0x558d32404a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbebdcf4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 0x558d32402140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbebdcf4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 0x558d32402140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:51932/0 (socket says 172.21.15.133:51932) 2024-09-11T23:45:26.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.025+0000 7fbebdcf4640 1 -- 172.21.15.133:0/2044778015 learned_addr learned my addr 172.21.15.133:0/2044778015 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:26.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbebe4f5640 1 --2- 172.21.15.133:0/2044778015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d323f74a0 0x558d323f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbebe4f5640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbea40a2cf0 0x7fbea40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbeaeffd640 1 -- 172.21.15.133:0/2175182504 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fbea80cf0a0 con 0x7fbeb8102790 2024-09-11T23:45:26.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbebdcf4640 1 -- 172.21.15.133:0/2044778015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d323f74a0 msgr2=0x558d323f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:26.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/2044778015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d323f74a0 0x558d323f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbebdcf4640 1 -- 172.21.15.133:0/2044778015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d32402680 msgr2=0x558d32404a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:26.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/2044778015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d32402680 0x558d32404a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbebdcf4640 1 -- 172.21.15.133:0/2044778015 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d322e9440 con 0x558d323f7dc0 2024-09-11T23:45:26.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbebe4f5640 1 --2- 172.21.15.133:0/2044778015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d323f74a0 0x558d323f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:26.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/2044778015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d32402680 0x558d32404a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:26.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/2044778015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 0x558d32402140 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4031550 tx=0x7fbeb4032cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a2c1c8e837409e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:26.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbebe4f5640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbea40a2cf0 0x7fbea40a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009fd0 tx=0x7fbeb0009600 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:26.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbea3fff640 1 -- 172.21.15.133:0/2044778015 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb403f070 con 0x558d323f7dc0 2024-09-11T23:45:26.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbea3fff640 1 -- 172.21.15.133:0/2044778015 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb40317b0 con 0x558d323f7dc0 2024-09-11T23:45:26.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.026+0000 7fbea3fff640 1 -- 172.21.15.133:0/2044778015 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4035070 con 0x558d323f7dc0 2024-09-11T23:45:26.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.027+0000 7fbec2331740 1 -- 172.21.15.133:0/2044778015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 msgr2=0x558d32402140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:26.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.027+0000 7fbec2331740 1 --2- 172.21.15.133:0/2044778015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 0x558d32402140 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4031550 tx=0x7fbeb4032cd0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.027+0000 7fbec2331740 1 -- 172.21.15.133:0/2044778015 shutdown_connections 2024-09-11T23:45:26.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.027+0000 7fbec2331740 1 --2- 172.21.15.133:0/2044778015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d32402680 0x558d32404a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.027+0000 7fbec2331740 1 --2- 172.21.15.133:0/2044778015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 0x558d32402140 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.027+0000 7fbec2331740 1 --2- 172.21.15.133:0/2044778015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d323f74a0 0x558d323f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.027+0000 7fbec2331740 1 -- 172.21.15.133:0/2044778015 >> 172.21.15.133:0/2044778015 conn(0x558d323f3a30 msgr2=0x558d323f3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.027+0000 7fbec2331740 1 -- 172.21.15.133:0/2044778015 shutdown_connections 2024-09-11T23:45:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.027+0000 7fbec2331740 1 -- 172.21.15.133:0/2044778015 wait complete. 2024-09-11T23:45:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.027+0000 7fbec2331740 1 -- start start 2024-09-11T23:45:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbec2331740 1 Processor -- start 2024-09-11T23:45:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbec2331740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d32401f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbec2331740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d323f7dc0 0x558d323ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbec2331740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d32402680 0x558d323ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbec2331740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558d32405250 con 0x558d323f74a0 2024-09-11T23:45:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbec2331740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d32318050 con 0x558d323f7dc0 2024-09-11T23:45:26.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbec2331740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558d324053d0 con 0x558d32402680 2024-09-11T23:45:26.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbec2331740 10 client.0 _ll_register_callbacks cb 0x558d32358160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:26.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbebdcf4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d32401f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbebdcf4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d32401f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40590/0 (socket says 172.21.15.133:40590) 2024-09-11T23:45:26.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbebdcf4640 1 -- 172.21.15.133:0/2712102151 learned_addr learned my addr 172.21.15.133:0/2712102151 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:26.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbebe4f5640 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d32402680 0x558d323ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d323f7dc0 0x558d323ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbebdcf4640 1 -- 172.21.15.133:0/2712102151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d32402680 msgr2=0x558d323ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:26.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d32402680 0x558d323ff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbebdcf4640 1 -- 172.21.15.133:0/2712102151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d323f7dc0 msgr2=0x558d323ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:26.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.028+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d323f7dc0 0x558d323ff220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbebdcf4640 1 -- 172.21.15.133:0/2712102151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d320ce3c0 con 0x558d323f74a0 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbebe4f5640 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d32402680 0x558d323ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d32401f70 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40052a0 tx=0x7fbeb4032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a589e1a1686b6d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbea17fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbea17fa640 1 -- 172.21.15.133:0/2712102151 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb403f070 con 0x558d323f74a0 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbea17fa640 1 -- 172.21.15.133:0/2712102151 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fbeb4004040 con 0x558d323f74a0 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbec2331740 1 -- 172.21.15.133:0/2712102151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb40317b0 con 0x558d323f74a0 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbea17fa640 1 -- 172.21.15.133:0/2712102151 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbeb4035040 con 0x558d323f74a0 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbec2331740 10 client.17637 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbec2331740 1 -- 172.21.15.133:0/2712102151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d32410bc0 con 0x558d323f74a0 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbea17fa640 1 -- 172.21.15.133:0/2712102151 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb40317b0 con 0x558d323f74a0 2024-09-11T23:45:26.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.029+0000 7fbea17fa640 1 -- 172.21.15.133:0/2712102151 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbeb40035f0 con 0x558d323f74a0 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.030+0000 7fbec2331740 10 client.17637 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.030+0000 7fbec2331740 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.030+0000 7fbec2331740 20 client.17637 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.030+0000 7fbec2331740 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d32411f10 0x558d32414300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.030+0000 7fbec2331740 4 client.17637 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.030+0000 7fbec2331740 1 -- 172.21.15.133:0/2712102151 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d3236a850 con 0x558d32411f10 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.030+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d32411f10 0x558d32414300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.030+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d32411f10 0x558d32414300 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0fa2457a8fe0b2 server_cookie=5cd2960897a75f6a in_seq=0 out_seq=0 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.030+0000 7fbea17fa640 10 client.17637 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbea17fa640 1 -- 172.21.15.133:0/2712102151 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d3236a850 con 0x558d32411f10 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbea17fa640 10 client.17637 handle_command_reply: tid=0 2024-09-11T23:45:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 1 client.17637 shutdown 2024-09-11T23:45:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 1 -- 172.21.15.133:0/2712102151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 msgr2=0x558d32401f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d32401f70 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40052a0 tx=0x7fbeb4032fb0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 1 -- 172.21.15.133:0/2712102151 shutdown_connections 2024-09-11T23:45:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d32411f10 0x558d32414300 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d32402680 0x558d323ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d323f7dc0 0x558d323ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 1 --2- 172.21.15.133:0/2712102151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d32401f70 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbea17fa640 0 client.17637 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 1 -- 172.21.15.133:0/2712102151 >> 172.21.15.133:0/2712102151 conn(0x558d323f3a30 msgr2=0x558d323f9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 1 -- 172.21.15.133:0/2712102151 shutdown_connections 2024-09-11T23:45:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 1 -- 172.21.15.133:0/2712102151 wait complete. 2024-09-11T23:45:26.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.031+0000 7fbec2331740 20 client.17637 trim_cache size 0 max 16384 2024-09-11T23:45:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbec2331740 1 Processor -- start 2024-09-11T23:45:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbec2331740 1 -- start start 2024-09-11T23:45:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbec2331740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f74a0 0x558d323fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbec2331740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f7dc0 0x558d3240e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbec2331740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d32402680 0x558d323fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbec2331740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ad7e0 con 0x558d323f7dc0 2024-09-11T23:45:26.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbec2331740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81af320 con 0x558d32402680 2024-09-11T23:45:26.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbec2331740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81adeb0 con 0x558d323f74a0 2024-09-11T23:45:26.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbebd4f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f7dc0 0x558d3240e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbebd4f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f7dc0 0x558d3240e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40594/0 (socket says 172.21.15.133:40594) 2024-09-11T23:45:26.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbebd4f3640 1 -- 172.21.15.133:0/3666682754 learned_addr learned my addr 172.21.15.133:0/3666682754 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:26.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbebe4f5640 1 --2- 172.21.15.133:0/3666682754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d32402680 0x558d323fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.050+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/3666682754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f74a0 0x558d323fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.051+0000 7fbebd4f3640 1 -- 172.21.15.133:0/3666682754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f74a0 msgr2=0x558d323fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:26.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.051+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/3666682754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f74a0 0x558d323fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.051+0000 7fbebd4f3640 1 -- 172.21.15.133:0/3666682754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d32402680 msgr2=0x558d323fdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:26.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.051+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/3666682754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d32402680 0x558d323fdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.051+0000 7fbebd4f3640 1 -- 172.21.15.133:0/3666682754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d323f8a20 con 0x558d323f7dc0 2024-09-11T23:45:26.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.051+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/3666682754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f7dc0 0x558d3240e4b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbea809b590 tx=0x7fbea8086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6165459779e40ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:26.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.051+0000 7fbea37fe640 1 -- 172.21.15.133:0/3666682754 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8020070 con 0x558d323f7dc0 2024-09-11T23:45:26.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.051+0000 7fbea37fe640 1 -- 172.21.15.133:0/3666682754 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea80a0ec0 con 0x558d323f7dc0 2024-09-11T23:45:26.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.051+0000 7fbea37fe640 1 -- 172.21.15.133:0/3666682754 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8097040 con 0x558d323f7dc0 2024-09-11T23:45:26.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.052+0000 7fbec2331740 1 -- 172.21.15.133:0/3666682754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f7dc0 msgr2=0x558d3240e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:26.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.052+0000 7fbec2331740 1 --2- 172.21.15.133:0/3666682754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f7dc0 0x558d3240e4b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbea809b590 tx=0x7fbea8086040 comp rx=0 tx=0).stop 2024-09-11T23:45:26.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.052+0000 7fbec2331740 1 -- 172.21.15.133:0/3666682754 shutdown_connections 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.052+0000 7fbec2331740 1 --2- 172.21.15.133:0/3666682754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d32402680 0x558d323fdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.052+0000 7fbec2331740 1 --2- 172.21.15.133:0/3666682754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f7dc0 0x558d3240e4b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.052+0000 7fbec2331740 1 --2- 172.21.15.133:0/3666682754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f74a0 0x558d323fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.052+0000 7fbec2331740 1 -- 172.21.15.133:0/3666682754 >> 172.21.15.133:0/3666682754 conn(0x558d323f3a30 msgr2=0x558d323f8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.052+0000 7fbec2331740 1 -- 172.21.15.133:0/3666682754 shutdown_connections 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.052+0000 7fbec2331740 1 -- 172.21.15.133:0/3666682754 wait complete. 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.052+0000 7fbec2331740 1 -- start start 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.052+0000 7fbec2331740 1 Processor -- start 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbec2331740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d3240ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbec2331740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 0x558d324060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbec2331740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d32402680 0x558d324065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbec2331740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81aec50 con 0x558d323f74a0 2024-09-11T23:45:26.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbec2331740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ae580 con 0x558d32402680 2024-09-11T23:45:26.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbec2331740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ab7e0 con 0x558d323f7dc0 2024-09-11T23:45:26.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbec2331740 10 client.0 _ll_register_callbacks cb 0x7fbea800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:26.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebdcf4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d3240ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebdcf4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d3240ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40606/0 (socket says 172.21.15.133:40606) 2024-09-11T23:45:26.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebdcf4640 1 -- 172.21.15.133:0/835504138 learned_addr learned my addr 172.21.15.133:0/835504138 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:26.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebe4f5640 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d32402680 0x558d324065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 0x558d324060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebdcf4640 1 -- 172.21.15.133:0/835504138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 msgr2=0x558d324060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:26.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 0x558d324060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebdcf4640 1 -- 172.21.15.133:0/835504138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d32402680 msgr2=0x558d324065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:26.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d32402680 0x558d324065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebdcf4640 1 -- 172.21.15.133:0/835504138 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d32406dc0 con 0x558d323f74a0 2024-09-11T23:45:26.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 0x558d324060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:45:26.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebe4f5640 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d32402680 0x558d324065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:26.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbebdcf4640 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d3240ad70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40092a0 tx=0x7fbeb4005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca83b2aaee8286a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:26.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.053+0000 7fbea17fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:26.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbea17fa640 1 -- 172.21.15.133:0/835504138 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb403f040 con 0x558d323f74a0 2024-09-11T23:45:26.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbea17fa640 1 -- 172.21.15.133:0/835504138 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fbeb4003c30 con 0x558d323f74a0 2024-09-11T23:45:26.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbec2331740 1 -- 172.21.15.133:0/835504138 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea80a0ec0 con 0x558d323f74a0 2024-09-11T23:45:26.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbea17fa640 1 -- 172.21.15.133:0/835504138 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbeb4065050 con 0x558d323f74a0 2024-09-11T23:45:26.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbec2331740 10 client.17643 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbec2331740 1 -- 172.21.15.133:0/835504138 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d3247c8c0 con 0x558d323f74a0 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbea17fa640 1 -- 172.21.15.133:0/835504138 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea80a0ec0 con 0x558d323f74a0 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbea17fa640 1 -- 172.21.15.133:0/835504138 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbeb405e060 con 0x558d323f74a0 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbec2331740 10 client.17643 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbec2331740 10 client.17643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbec2331740 20 client.17643 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbec2331740 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d3247d030 0x558d3247f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbec2331740 4 client.17643 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.054+0000 7fbec2331740 1 -- 172.21.15.133:0/835504138 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d323ffaf0 con 0x558d3247d030 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.055+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d3247d030 0x558d3247f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.055+0000 7fbea17fa640 10 client.17643 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.055+0000 7fbebd4f3640 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d3247d030 0x558d3247f470 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67333ce10d21b79 server_cookie=1350b645e29ad7b3 in_seq=0 out_seq=0 2024-09-11T23:45:26.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.055+0000 7fbea17fa640 1 -- 172.21.15.133:0/835504138 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558d323ffaf0 con 0x558d3247d030 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.055+0000 7fbea17fa640 10 client.17643 handle_command_reply: tid=0 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.055+0000 7fbec2331740 1 client.17643 shutdown 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.055+0000 7fbec2331740 1 -- 172.21.15.133:0/835504138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 msgr2=0x558d3240ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.055+0000 7fbec2331740 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d3240ad70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40092a0 tx=0x7fbeb4005230 comp rx=0 tx=0).stop 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbec2331740 1 -- 172.21.15.133:0/835504138 shutdown_connections 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbec2331740 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558d3247d030 0x558d3247f470 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbec2331740 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d32402680 0x558d324065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbec2331740 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558d323f7dc0 0x558d324060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbec2331740 1 --2- 172.21.15.133:0/835504138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d323f74a0 0x558d3240ad70 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbea17fa640 0 client.17643 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbec2331740 1 -- 172.21.15.133:0/835504138 >> 172.21.15.133:0/835504138 conn(0x558d323f3a30 msgr2=0x558d32400a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:26.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbec2331740 1 -- 172.21.15.133:0/835504138 shutdown_connections 2024-09-11T23:45:26.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbec2331740 1 -- 172.21.15.133:0/835504138 wait complete. 2024-09-11T23:45:26.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbec2331740 20 client.17643 trim_cache size 0 max 16384 2024-09-11T23:45:26.071 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:45:26.071 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:45:26.071 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:45:26.071 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:45:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 -- 172.21.15.133:0/2175182504 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbea40a2cf0 msgr2=0x7fbea40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbea40a2cf0 0x7fbea40a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009fd0 tx=0x7fbeb0009600 comp rx=0 tx=0).stop 2024-09-11T23:45:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 -- 172.21.15.133:0/2175182504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeb8102790 msgr2=0x7fbeb8198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeb8102790 0x7fbeb8198950 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbea800d950 tx=0x7fbea800de10 comp rx=0 tx=0).stop 2024-09-11T23:45:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 -- 172.21.15.133:0/2175182504 shutdown_connections 2024-09-11T23:45:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fbea40a2cf0 0x7fbea40a51b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbeb8106f10 0x7fbeb819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeb8102790 0x7fbeb8198950 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 --2- 172.21.15.133:0/2175182504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb8101de0 0x7fbeb8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:26.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 -- 172.21.15.133:0/2175182504 >> 172.21.15.133:0/2175182504 conn(0x7fbeb80fd8e0 msgr2=0x7fbeb80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:26.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 -- 172.21.15.133:0/2175182504 shutdown_connections 2024-09-11T23:45:26.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:26.056+0000 7fbebff58640 1 -- 172.21.15.133:0/2175182504 wait complete. 2024-09-11T23:45:26.805 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:45:26.805 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:45:26.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:26 smithi138 ceph-mon[35142]: pgmap v269: 97 pgs: 97 active+clean; 63 GiB data, 221 GiB used, 851 GiB / 1.0 TiB avail; 343 KiB/s rd, 256 MiB/s wr, 860 op/s 2024-09-11T23:45:26.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:26 smithi133 ceph-mon[31430]: pgmap v269: 97 pgs: 97 active+clean; 63 GiB data, 221 GiB used, 851 GiB / 1.0 TiB avail; 343 KiB/s rd, 256 MiB/s wr, 860 op/s 2024-09-11T23:45:27.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:26 smithi178 ceph-mon[35882]: pgmap v269: 97 pgs: 97 active+clean; 63 GiB data, 221 GiB used, 851 GiB / 1.0 TiB avail; 343 KiB/s rd, 256 MiB/s wr, 860 op/s 2024-09-11T23:45:27.102 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:27.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.585+0000 7ff7bec4c640 1 -- 172.21.15.133:0/1228441399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b8102ef0 msgr2=0x7ff7b810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:27.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.585+0000 7ff7bec4c640 1 --2- 172.21.15.133:0/1228441399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b8102ef0 0x7ff7b810da40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009ca0 tx=0x7ff7b002a630 comp rx=0 tx=0).stop 2024-09-11T23:45:27.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 -- 172.21.15.133:0/1228441399 shutdown_connections 2024-09-11T23:45:27.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 --2- 172.21.15.133:0/1228441399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b8102ef0 0x7ff7b810da40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 --2- 172.21.15.133:0/1228441399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b8102570 0x7ff7b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 --2- 172.21.15.133:0/1228441399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7b8100e20 0x7ff7b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 -- 172.21.15.133:0/1228441399 >> 172.21.15.133:0/1228441399 conn(0x7ff7b80fc7d0 msgr2=0x7ff7b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:27.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 -- 172.21.15.133:0/1228441399 shutdown_connections 2024-09-11T23:45:27.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 -- 172.21.15.133:0/1228441399 wait complete. 2024-09-11T23:45:27.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 Processor -- start 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 -- start start 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b8100e20 0x7ff7b8196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b8102570 0x7ff7b8196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7b8102ef0 0x7ff7b819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b810fea0 con 0x7ff7b8102570 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b810fd20 con 0x7ff7b8100e20 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.586+0000 7ff7bec4c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b8110020 con 0x7ff7b8102ef0 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b8102570 0x7ff7b8196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b8102570 0x7ff7b8196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40628/0 (socket says 172.21.15.133:40628) 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b7fff640 1 -- 172.21.15.133:0/3913351734 learned_addr learned my addr 172.21.15.133:0/3913351734 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7bc9e8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b8100e20 0x7ff7b8196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b7fff640 1 -- 172.21.15.133:0/3913351734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7b8102ef0 msgr2=0x7ff7b819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7bd1e9640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7b8102ef0 0x7ff7b819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:27.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b7fff640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7b8102ef0 0x7ff7b819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b7fff640 1 -- 172.21.15.133:0/3913351734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b8100e20 msgr2=0x7ff7b8196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b7fff640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b8100e20 0x7ff7b8196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b7fff640 1 -- 172.21.15.133:0/3913351734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b819b2f0 con 0x7ff7b8102570 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7bd1e9640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7b8102ef0 0x7ff7b819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b7fff640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b8102570 0x7ff7b8196880 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009be0 tx=0x7ff7a40071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b5ffb640 1 -- 172.21.15.133:0/3913351734 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4007d40 con 0x7ff7b8102570 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b5ffb640 1 -- 172.21.15.133:0/3913351734 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7a401e070 con 0x7ff7b8102570 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7bec4c640 1 -- 172.21.15.133:0/3913351734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b819b5e0 con 0x7ff7b8102570 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7b5ffb640 1 -- 172.21.15.133:0/3913351734 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40186f0 con 0x7ff7b8102570 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.587+0000 7ff7bec4c640 1 -- 172.21.15.133:0/3913351734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b81a2ec0 con 0x7ff7b8102570 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.588+0000 7ff7bec4c640 1 -- 172.21.15.133:0/3913351734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b8101200 con 0x7ff7b8102570 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.589+0000 7ff7b5ffb640 1 -- 172.21.15.133:0/3913351734 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff7a40193f0 con 0x7ff7b8102570 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.590+0000 7ff7b5ffb640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7a80a2cf0 0x7ff7a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:27.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.590+0000 7ff7b5ffb640 1 -- 172.21.15.133:0/3913351734 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7ff7a40d2fb0 con 0x7ff7b8102570 2024-09-11T23:45:27.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.590+0000 7ff7bc9e8640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7a80a2cf0 0x7ff7a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:27.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.591+0000 7ff7bc9e8640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7a80a2cf0 0x7ff7a80a51b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009990 tx=0x7ff7ac009110 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:27.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.594+0000 7ff7b5ffb640 1 -- 172.21.15.133:0/3913351734 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7a4016070 con 0x7ff7b8102570 2024-09-11T23:45:27.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.807+0000 7ff7bec4c640 1 -- 172.21.15.133:0/3913351734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7b81a3090 con 0x7ff7b8102570 2024-09-11T23:45:27.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.808+0000 7ff7b5ffb640 1 -- 172.21.15.133:0/3913351734 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7ff7a409fc10 con 0x7ff7b8102570 2024-09-11T23:45:27.808 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:45:27.808 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:45:27.808 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:45:27.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.810+0000 7ff7bec4c640 1 -- 172.21.15.133:0/3913351734 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7a80a2cf0 msgr2=0x7ff7a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:27.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.810+0000 7ff7bec4c640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7a80a2cf0 0x7ff7a80a51b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009990 tx=0x7ff7ac009110 comp rx=0 tx=0).stop 2024-09-11T23:45:27.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.810+0000 7ff7bec4c640 1 -- 172.21.15.133:0/3913351734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b8102570 msgr2=0x7ff7b8196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:27.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.810+0000 7ff7bec4c640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b8102570 0x7ff7b8196880 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009be0 tx=0x7ff7a40071c0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.810+0000 7ff7bec4c640 1 -- 172.21.15.133:0/3913351734 shutdown_connections 2024-09-11T23:45:27.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.810+0000 7ff7bec4c640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7a80a2cf0 0x7ff7a80a51b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.810+0000 7ff7bec4c640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7b8102ef0 0x7ff7b819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.810+0000 7ff7bec4c640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b8102570 0x7ff7b8196880 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.810+0000 7ff7bec4c640 1 --2- 172.21.15.133:0/3913351734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b8100e20 0x7ff7b8196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:27.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.810+0000 7ff7bec4c640 1 -- 172.21.15.133:0/3913351734 >> 172.21.15.133:0/3913351734 conn(0x7ff7b80fc7d0 msgr2=0x7ff7b8106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:27.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.811+0000 7ff7bec4c640 1 -- 172.21.15.133:0/3913351734 shutdown_connections 2024-09-11T23:45:27.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:27.811+0000 7ff7bec4c640 1 -- 172.21.15.133:0/3913351734 wait complete. 2024-09-11T23:45:28.087 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:45:28.635 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:45:28.674 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:28.922 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:28 smithi133 ceph-mon[31430]: pgmap v270: 97 pgs: 97 active+clean; 63 GiB data, 220 GiB used, 852 GiB / 1.0 TiB avail; 343 KiB/s rd, 273 MiB/s wr, 853 op/s 2024-09-11T23:45:28.922 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:28 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3913351734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:28.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:28 smithi138 ceph-mon[35142]: pgmap v270: 97 pgs: 97 active+clean; 63 GiB data, 220 GiB used, 852 GiB / 1.0 TiB avail; 343 KiB/s rd, 273 MiB/s wr, 853 op/s 2024-09-11T23:45:28.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:28 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3913351734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:29.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:28 smithi178 ceph-mon[35882]: pgmap v270: 97 pgs: 97 active+clean; 63 GiB data, 220 GiB used, 852 GiB / 1.0 TiB avail; 343 KiB/s rd, 273 MiB/s wr, 853 op/s 2024-09-11T23:45:29.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:28 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3913351734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.144+0000 7efe05445640 1 -- 172.21.15.133:0/1212527683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00102ef0 msgr2=0x7efe0010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.144+0000 7efe05445640 1 --2- 172.21.15.133:0/1212527683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00102ef0 0x7efe0010da40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009ca0 tx=0x7efdf802a950 comp rx=0 tx=0).stop 2024-09-11T23:45:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.144+0000 7efe05445640 1 -- 172.21.15.133:0/1212527683 shutdown_connections 2024-09-11T23:45:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.144+0000 7efe05445640 1 --2- 172.21.15.133:0/1212527683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00102ef0 0x7efe0010da40 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.144+0000 7efe05445640 1 --2- 172.21.15.133:0/1212527683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe00102570 0x7efe001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.144+0000 7efe05445640 1 --2- 172.21.15.133:0/1212527683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe00100e20 0x7efe00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.144+0000 7efe05445640 1 -- 172.21.15.133:0/1212527683 >> 172.21.15.133:0/1212527683 conn(0x7efe000fc7d0 msgr2=0x7efe000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.144+0000 7efe05445640 1 -- 172.21.15.133:0/1212527683 shutdown_connections 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.144+0000 7efe05445640 1 -- 172.21.15.133:0/1212527683 wait complete. 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.145+0000 7efe05445640 1 Processor -- start 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.145+0000 7efe05445640 1 -- start start 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.145+0000 7efe05445640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe00100e20 0x7efe00198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.145+0000 7efe05445640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe00102570 0x7efe00198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.145+0000 7efe05445640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00102ef0 0x7efe0019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.145+0000 7efe05445640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efe00110f80 con 0x7efe00102ef0 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.145+0000 7efe05445640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe00110e00 con 0x7efe00102570 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.145+0000 7efe05445640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe00111100 con 0x7efe00100e20 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdff7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00102ef0 0x7efe0019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdff7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00102ef0 0x7efe0019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40644/0 (socket says 172.21.15.133:40644) 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdff7fe640 1 -- 172.21.15.133:0/2254815272 learned_addr learned my addr 172.21.15.133:0/2254815272 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdfe7fc640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe00102570 0x7efe00198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdfeffd640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe00100e20 0x7efe00198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdff7fe640 1 -- 172.21.15.133:0/2254815272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe00100e20 msgr2=0x7efe00198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdff7fe640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe00100e20 0x7efe00198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdff7fe640 1 -- 172.21.15.133:0/2254815272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe00102570 msgr2=0x7efe00198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdff7fe640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe00102570 0x7efe00198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdff7fe640 1 -- 172.21.15.133:0/2254815272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0019d4e0 con 0x7efe00102ef0 2024-09-11T23:45:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdfeffd640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe00100e20 0x7efe00198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdfe7fc640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe00102570 0x7efe00198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.146+0000 7efdff7fe640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00102ef0 0x7efe0019ce00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7efdf8004af0 tx=0x7efdf802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.147+0000 7efde7fff640 1 -- 172.21.15.133:0/2254815272 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8034040 con 0x7efe00102ef0 2024-09-11T23:45:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.147+0000 7efde7fff640 1 -- 172.21.15.133:0/2254815272 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf8030920 con 0x7efe00102ef0 2024-09-11T23:45:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.147+0000 7efe05445640 1 -- 172.21.15.133:0/2254815272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0019d770 con 0x7efe00102ef0 2024-09-11T23:45:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.147+0000 7efde7fff640 1 -- 172.21.15.133:0/2254815272 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8004820 con 0x7efe00102ef0 2024-09-11T23:45:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.147+0000 7efe05445640 1 -- 172.21.15.133:0/2254815272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0019dc00 con 0x7efe00102ef0 2024-09-11T23:45:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.149+0000 7efe05445640 1 -- 172.21.15.133:0/2254815272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe00101200 con 0x7efe00102ef0 2024-09-11T23:45:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.150+0000 7efde7fff640 1 -- 172.21.15.133:0/2254815272 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7efdf8037020 con 0x7efe00102ef0 2024-09-11T23:45:29.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.150+0000 7efde7fff640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efde80a2cf0 0x7efde80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.150+0000 7efde7fff640 1 -- 172.21.15.133:0/2254815272 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7efdf80f0230 con 0x7efe00102ef0 2024-09-11T23:45:29.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.150+0000 7efdfeffd640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efde80a2cf0 0x7efde80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.151+0000 7efdfeffd640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efde80a2cf0 0x7efde80a51b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009c20 tx=0x7efdf0005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:29.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.155+0000 7efde7fff640 1 -- 172.21.15.133:0/2254815272 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdf80bcfa0 con 0x7efe00102ef0 2024-09-11T23:45:29.183 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:29.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.372+0000 7efe05445640 1 -- 172.21.15.133:0/2254815272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe0010a0a0 con 0x7efe00102ef0 2024-09-11T23:45:29.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.372+0000 7efde7fff640 1 -- 172.21.15.133:0/2254815272 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7efdf80bc8a0 con 0x7efe00102ef0 2024-09-11T23:45:29.372 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:45:29.372 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:45:29.373 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:45:29.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 -- 172.21.15.133:0/2254815272 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efde80a2cf0 msgr2=0x7efde80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efde80a2cf0 0x7efde80a51b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009c20 tx=0x7efdf0005f90 comp rx=0 tx=0).stop 2024-09-11T23:45:29.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 -- 172.21.15.133:0/2254815272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00102ef0 msgr2=0x7efe0019ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00102ef0 0x7efe0019ce00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7efdf8004af0 tx=0x7efdf802f5b0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 -- 172.21.15.133:0/2254815272 shutdown_connections 2024-09-11T23:45:29.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efde80a2cf0 0x7efde80a51b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00102ef0 0x7efe0019ce00 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe00102570 0x7efe00198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 --2- 172.21.15.133:0/2254815272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe00100e20 0x7efe00198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 -- 172.21.15.133:0/2254815272 >> 172.21.15.133:0/2254815272 conn(0x7efe000fc7d0 msgr2=0x7efe0006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:29.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 -- 172.21.15.133:0/2254815272 shutdown_connections 2024-09-11T23:45:29.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.375+0000 7efe05445640 1 -- 172.21.15.133:0/2254815272 wait complete. 2024-09-11T23:45:29.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:29 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2254815272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:29.868 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:29 smithi133 ceph-mon[31430]: pgmap v271: 97 pgs: 97 active+clean; 63 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 20 KiB/s rd, 275 MiB/s wr, 878 op/s 2024-09-11T23:45:29.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.866+0000 7fb9aa520640 1 -- 172.21.15.133:0/3093655617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9a4100e20 msgr2=0x7fb9a4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.866+0000 7fb9aa520640 1 --2- 172.21.15.133:0/3093655617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9a4100e20 0x7fb9a4101200 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb994009900 tx=0x7fb99402a480 comp rx=0 tx=0).stop 2024-09-11T23:45:29.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.866+0000 7fb9aa520640 1 -- 172.21.15.133:0/3093655617 shutdown_connections 2024-09-11T23:45:29.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.866+0000 7fb9aa520640 1 --2- 172.21.15.133:0/3093655617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9a4102ef0 0x7fb9a410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.866+0000 7fb9aa520640 1 --2- 172.21.15.133:0/3093655617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a4102570 0x7fb9a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.866+0000 7fb9aa520640 1 --2- 172.21.15.133:0/3093655617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9a4100e20 0x7fb9a4101200 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.866+0000 7fb9aa520640 1 -- 172.21.15.133:0/3093655617 >> 172.21.15.133:0/3093655617 conn(0x7fb9a40fc7d0 msgr2=0x7fb9a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.866+0000 7fb9aa520640 1 -- 172.21.15.133:0/3093655617 shutdown_connections 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.866+0000 7fb9aa520640 1 -- 172.21.15.133:0/3093655617 wait complete. 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9aa520640 1 Processor -- start 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9aa520640 1 -- start start 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9aa520640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9a4100e20 0x7fb9a4197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9aa520640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a4102570 0x7fb9a4197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9aa520640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9a4102ef0 0x7fb9a419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9aa520640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4110fa0 con 0x7fb9a4100e20 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9aa520640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4110e20 con 0x7fb9a4102570 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9aa520640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4111120 con 0x7fb9a4102ef0 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9a3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9a4100e20 0x7fb9a4197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9a3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9a4100e20 0x7fb9a4197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40670/0 (socket says 172.21.15.133:40670) 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9a3fff640 1 -- 172.21.15.133:0/4101005470 learned_addr learned my addr 172.21.15.133:0/4101005470 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.867+0000 7fb9a37fe640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a4102570 0x7fb9a4197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9a8abd640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9a4102ef0 0x7fb9a419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9a3fff640 1 -- 172.21.15.133:0/4101005470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9a4102ef0 msgr2=0x7fb9a419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9a3fff640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9a4102ef0 0x7fb9a419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9a3fff640 1 -- 172.21.15.133:0/4101005470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a4102570 msgr2=0x7fb9a4197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9a3fff640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a4102570 0x7fb9a4197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9a3fff640 1 -- 172.21.15.133:0/4101005470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a419c3f0 con 0x7fb9a4100e20 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9a37fe640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a4102570 0x7fb9a4197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9a3fff640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9a4100e20 0x7fb9a4197440 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb994006010 tx=0x7fb994003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9a17fa640 1 -- 172.21.15.133:0/4101005470 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99403f070 con 0x7fb9a4100e20 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9aa520640 1 -- 172.21.15.133:0/4101005470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a419c680 con 0x7fb9a4100e20 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9a17fa640 1 -- 172.21.15.133:0/4101005470 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb994031c30 con 0x7fb9a4100e20 2024-09-11T23:45:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9a17fa640 1 -- 172.21.15.133:0/4101005470 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994035030 con 0x7fb9a4100e20 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.868+0000 7fb9aa520640 1 -- 172.21.15.133:0/4101005470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a41a3fc0 con 0x7fb9a4100e20 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9ac8f9740 1 Processor -- start 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a17fa640 1 -- 172.21.15.133:0/4101005470 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb994042070 con 0x7fb9a4100e20 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9ac8f9740 1 -- start start 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cccb40 0x55bd10ccb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 0x55bd10ccba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cd8580 0x55bd10cda970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bd10c312f0 con 0x55bd10ccb5d0 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bd10bd8cf0 con 0x55bd10cd8580 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bd10c2d2b0 con 0x55bd10cccb40 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 0x55bd10ccba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 0x55bd10ccba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40672/0 (socket says 172.21.15.133:40672) 2024-09-11T23:45:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cd8580 0x55bd10cda970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a3fff640 1 -- 172.21.15.133:0/4102202868 learned_addr learned my addr 172.21.15.133:0/4102202868 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a8abd640 1 --2- 172.21.15.133:0/4102202868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cccb40 0x55bd10ccb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a37fe640 1 -- 172.21.15.133:0/4102202868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cccb40 msgr2=0x55bd10ccb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a37fe640 1 --2- 172.21.15.133:0/4102202868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cccb40 0x55bd10ccb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a37fe640 1 -- 172.21.15.133:0/4102202868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 msgr2=0x55bd10ccba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a37fe640 1 --2- 172.21.15.133:0/4102202868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 0x55bd10ccba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a37fe640 1 -- 172.21.15.133:0/4102202868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd10bbd3a0 con 0x55bd10cd8580 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.871+0000 7fb9a17fa640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb98c0a2cf0 0x7fb98c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.872+0000 7fb9a8abd640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb98c0a2cf0 0x7fb98c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.872+0000 7fb9a17fa640 1 -- 172.21.15.133:0/4101005470 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fb9940efb50 con 0x7fb9a4100e20 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.872+0000 7fb9a37fe640 1 --2- 172.21.15.133:0/4102202868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cd8580 0x55bd10cda970 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb998009fa0 tx=0x7fb9980118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b371143cbc0dd5e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.872+0000 7fb98a7fc640 1 -- 172.21.15.133:0/4102202868 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998004030 con 0x55bd10cd8580 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.872+0000 7fb98a7fc640 1 -- 172.21.15.133:0/4102202868 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb998005500 con 0x55bd10cd8580 2024-09-11T23:45:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.872+0000 7fb98a7fc640 1 -- 172.21.15.133:0/4102202868 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998004e10 con 0x55bd10cd8580 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/4102202868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cd8580 msgr2=0x55bd10cda970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9a8abd640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb98c0a2cf0 0x7fb98c0a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb99c007d80 tx=0x7fb99c007430 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/4102202868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cd8580 0x55bd10cda970 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb998009fa0 tx=0x7fb9980118d0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/4102202868 shutdown_connections 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/4102202868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cd8580 0x55bd10cda970 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/4102202868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 0x55bd10ccba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/4102202868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cccb40 0x55bd10ccb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/4102202868 >> 172.21.15.133:0/4102202868 conn(0x55bd10c94310 msgr2=0x55bd10cc7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/4102202868 shutdown_connections 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/4102202868 wait complete. 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9ac8f9740 1 -- start start 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.873+0000 7fb9ac8f9740 1 Processor -- start 2024-09-11T23:45:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10ccb5d0 0x55bd10cd7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 0x55bd10cd5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cd5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bd10cdb280 con 0x55bd10cd8580 2024-09-11T23:45:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bd10bec060 con 0x55bd10cccb40 2024-09-11T23:45:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bd10cdb400 con 0x55bd10ccb5d0 2024-09-11T23:45:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9ac8f9740 10 client.0 _ll_register_callbacks cb 0x55bd1093f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a8abd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cd5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 0x55bd10cd5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a8abd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cd5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40678/0 (socket says 172.21.15.133:40678) 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a8abd640 1 -- 172.21.15.133:0/3259264704 learned_addr learned my addr 172.21.15.133:0/3259264704 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a3fff640 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10ccb5d0 0x55bd10cd7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a8abd640 1 -- 172.21.15.133:0/3259264704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10ccb5d0 msgr2=0x55bd10cd7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a8abd640 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10ccb5d0 0x55bd10cd7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a8abd640 1 -- 172.21.15.133:0/3259264704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 msgr2=0x55bd10cd5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a8abd640 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 0x55bd10cd5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a8abd640 1 -- 172.21.15.133:0/3259264704 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd109a23c0 con 0x55bd10cd8580 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a37fe640 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 0x55bd10cd5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9a8abd640 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cd5840 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb99c014a70 tx=0x7fb99c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fec071601f1838 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9837fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9837fe640 1 -- 172.21.15.133:0/3259264704 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c01a050 con 0x55bd10cd8580 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9837fe640 1 -- 172.21.15.133:0/3259264704 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fb99c012a00 con 0x55bd10cd8580 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.874+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/3259264704 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb998005500 con 0x55bd10cd8580 2024-09-11T23:45:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9837fe640 1 -- 172.21.15.133:0/3259264704 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb99c012c00 con 0x55bd10cd8580 2024-09-11T23:45:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9ac8f9740 10 client.17667 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/3259264704 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bd10ce6d70 con 0x55bd10cd8580 2024-09-11T23:45:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9837fe640 1 -- 172.21.15.133:0/3259264704 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb998005500 con 0x55bd10cd8580 2024-09-11T23:45:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9837fe640 1 -- 172.21.15.133:0/3259264704 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb99c011070 con 0x55bd10cd8580 2024-09-11T23:45:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9ac8f9740 10 client.17667 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9ac8f9740 10 client.17667 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:45:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9ac8f9740 20 client.17667 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55bd10ce8230 0x55bd10cea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9ac8f9740 4 client.17667 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:29.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/3259264704 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd10c3e860 con 0x55bd10ce8230 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.875+0000 7fb9a3fff640 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55bd10ce8230 0x55bd10cea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9837fe640 10 client.17667 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9a3fff640 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55bd10ce8230 0x55bd10cea6b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c68e35e02ab8406 server_cookie=b900c5ce5f74a426 in_seq=0 out_seq=0 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9837fe640 1 -- 172.21.15.133:0/3259264704 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd10c3e860 con 0x55bd10ce8230 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9837fe640 10 client.17667 handle_command_reply: tid=0 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9ac8f9740 1 client.17667 shutdown 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/3259264704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 msgr2=0x55bd10cd5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cd5840 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb99c014a70 tx=0x7fb99c004a40 comp rx=0 tx=0).stop 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/3259264704 shutdown_connections 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55bd10ce8230 0x55bd10cea6b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cd5840 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 0x55bd10cd5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/3259264704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10ccb5d0 0x55bd10cd7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.876+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/3259264704 >> 172.21.15.133:0/3259264704 conn(0x55bd10c94310 msgr2=0x55bd10ccd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.877+0000 7fb9837fe640 0 client.17667 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.877+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/3259264704 shutdown_connections 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.877+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/3259264704 wait complete. 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.877+0000 7fb9ac8f9740 20 client.17667 trim_cache size 0 max 16384 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.889+0000 7fb9ac8f9740 1 Processor -- start 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.889+0000 7fb9ac8f9740 1 -- start start 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10ccb5d0 0x55bd10ce4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cccb40 0x55bd10cc7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cc7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a41ab5d0 con 0x55bd10cd8580 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a41ad110 con 0x55bd10ccb5d0 2024-09-11T23:45:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a41abca0 con 0x55bd10cccb40 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a8abd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cc7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a8abd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cc7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40680/0 (socket says 172.21.15.133:40680) 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a8abd640 1 -- 172.21.15.133:0/1016105646 learned_addr learned my addr 172.21.15.133:0/1016105646 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a3fff640 1 --2- 172.21.15.133:0/1016105646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10ccb5d0 0x55bd10ce4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a8abd640 1 -- 172.21.15.133:0/1016105646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cccb40 msgr2=0x55bd10cc7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a37fe640 1 --2- 172.21.15.133:0/1016105646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cccb40 0x55bd10cc7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a8abd640 1 --2- 172.21.15.133:0/1016105646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cccb40 0x55bd10cc7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a8abd640 1 -- 172.21.15.133:0/1016105646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10ccb5d0 msgr2=0x55bd10ce4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a8abd640 1 --2- 172.21.15.133:0/1016105646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10ccb5d0 0x55bd10ce4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a8abd640 1 -- 172.21.15.133:0/1016105646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd10cd7b30 con 0x55bd10cd8580 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a3fff640 1 --2- 172.21.15.133:0/1016105646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10ccb5d0 0x55bd10ce4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.890+0000 7fb9a8abd640 1 --2- 172.21.15.133:0/1016105646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cc7c40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb99c008010 tx=0x7fb99c014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3752411795169356 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb989ffb640 1 -- 172.21.15.133:0/1016105646 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c036070 con 0x55bd10cd8580 2024-09-11T23:45:29.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb989ffb640 1 -- 172.21.15.133:0/1016105646 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb99c024070 con 0x55bd10cd8580 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb989ffb640 1 -- 172.21.15.133:0/1016105646 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c01a070 con 0x55bd10cd8580 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/1016105646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 msgr2=0x55bd10cc7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/1016105646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cc7c40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb99c008010 tx=0x7fb99c014780 comp rx=0 tx=0).stop 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/1016105646 shutdown_connections 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/1016105646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10cd8580 0x55bd10cc7c40 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/1016105646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cccb40 0x55bd10cc7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/1016105646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10ccb5d0 0x55bd10ce4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/1016105646 >> 172.21.15.133:0/1016105646 conn(0x55bd10c94310 msgr2=0x55bd10cd51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/1016105646 shutdown_connections 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.891+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/1016105646 wait complete. 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9ac8f9740 1 -- start start 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9ac8f9740 1 Processor -- start 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 0x55bd10cdb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 0x55bd10cdbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9ac8f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cd8580 0x55bd10d50f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a41aca40 con 0x55bd10ccb5d0 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a41ac370 con 0x55bd10cccb40 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9ac8f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a41a95d0 con 0x55bd10cd8580 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9ac8f9740 10 client.0 _ll_register_callbacks cb 0x7fb99c008040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9a3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 0x55bd10cdb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9a3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 0x55bd10cdb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40694/0 (socket says 172.21.15.133:40694) 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9a3fff640 1 -- 172.21.15.133:0/2592341742 learned_addr learned my addr 172.21.15.133:0/2592341742 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9a37fe640 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 0x55bd10cdbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9a8abd640 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cd8580 0x55bd10d50f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9a3fff640 1 -- 172.21.15.133:0/2592341742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cd8580 msgr2=0x55bd10d50f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9a3fff640 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cd8580 0x55bd10d50f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9a3fff640 1 -- 172.21.15.133:0/2592341742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 msgr2=0x55bd10cdbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9a3fff640 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 0x55bd10cdbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9a3fff640 1 -- 172.21.15.133:0/2592341742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd10cdc4f0 con 0x55bd10ccb5d0 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.892+0000 7fb9a37fe640 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 0x55bd10cdbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9a3fff640 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 0x55bd10cdb6e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb9940c47f0 tx=0x7fb9940b8350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8583dac3111159dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9837fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9837fe640 1 -- 172.21.15.133:0/2592341742 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994049040 con 0x55bd10ccb5d0 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9837fe640 1 -- 172.21.15.133:0/2592341742 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fb9940dc0c0 con 0x55bd10ccb5d0 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/2592341742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb99c024070 con 0x55bd10ccb5d0 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9837fe640 1 -- 172.21.15.133:0/2592341742 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9940bba10 con 0x55bd10ccb5d0 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9ac8f9740 10 client.17673 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/2592341742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bd10d52630 con 0x55bd10ccb5d0 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9837fe640 1 -- 172.21.15.133:0/2592341742 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb99c024070 con 0x55bd10ccb5d0 2024-09-11T23:45:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9837fe640 1 -- 172.21.15.133:0/2592341742 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb99c00ed20 con 0x55bd10ccb5d0 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9ac8f9740 10 client.17673 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9ac8f9740 10 client.17673 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9ac8f9740 20 client.17673 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55bd10d52d50 0x55bd10d55140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9ac8f9740 4 client.17673 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.893+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/2592341742 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd10c38d70 con 0x55bd10d52d50 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9a37fe640 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55bd10d52d50 0x55bd10d55140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9837fe640 10 client.17673 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9a37fe640 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55bd10d52d50 0x55bd10d55140 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=629195b0fdb08584 server_cookie=29a8b302de5ef5e6 in_seq=0 out_seq=0 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9837fe640 1 -- 172.21.15.133:0/2592341742 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd10c38d70 con 0x55bd10d52d50 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9837fe640 10 client.17673 handle_command_reply: tid=0 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9ac8f9740 1 client.17673 shutdown 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/2592341742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 msgr2=0x55bd10cdb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 0x55bd10cdb6e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb9940c47f0 tx=0x7fb9940b8350 comp rx=0 tx=0).stop 2024-09-11T23:45:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/2592341742 shutdown_connections 2024-09-11T23:45:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55bd10d52d50 0x55bd10d55140 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9837fe640 0 client.17673 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:45:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd10cd8580 0x55bd10d50f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd10cccb40 0x55bd10cdbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9ac8f9740 1 --2- 172.21.15.133:0/2592341742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd10ccb5d0 0x55bd10cdb6e0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.894+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/2592341742 >> 172.21.15.133:0/2592341742 conn(0x55bd10c94310 msgr2=0x55bd10cc76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.895+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/2592341742 shutdown_connections 2024-09-11T23:45:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.895+0000 7fb9ac8f9740 1 -- 172.21.15.133:0/2592341742 wait complete. 2024-09-11T23:45:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.895+0000 7fb9ac8f9740 20 client.17673 trim_cache size 0 max 16384 2024-09-11T23:45:29.912 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:45:29.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 -- 172.21.15.133:0/4101005470 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb98c0a2cf0 msgr2=0x7fb98c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb98c0a2cf0 0x7fb98c0a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb99c007d80 tx=0x7fb99c007430 comp rx=0 tx=0).stop 2024-09-11T23:45:29.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 -- 172.21.15.133:0/4101005470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9a4100e20 msgr2=0x7fb9a4197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:29.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9a4100e20 0x7fb9a4197440 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb994006010 tx=0x7fb994003c10 comp rx=0 tx=0).stop 2024-09-11T23:45:29.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 -- 172.21.15.133:0/4101005470 shutdown_connections 2024-09-11T23:45:29.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb98c0a2cf0 0x7fb98c0a51b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9a4102ef0 0x7fb9a419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a4102570 0x7fb9a4197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 --2- 172.21.15.133:0/4101005470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9a4100e20 0x7fb9a4197440 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:29.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 -- 172.21.15.133:0/4101005470 >> 172.21.15.133:0/4101005470 conn(0x7fb9a40fc7d0 msgr2=0x7fb9a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:29.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 -- 172.21.15.133:0/4101005470 shutdown_connections 2024-09-11T23:45:29.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:29.896+0000 7fb9aa520640 1 -- 172.21.15.133:0/4101005470 wait complete. 2024-09-11T23:45:30.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:29 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2254815272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:30.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:29 smithi178 ceph-mon[35882]: pgmap v271: 97 pgs: 97 active+clean; 63 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 20 KiB/s rd, 275 MiB/s wr, 878 op/s 2024-09-11T23:45:30.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:29 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2254815272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:45:30.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:29 smithi138 ceph-mon[35142]: pgmap v271: 97 pgs: 97 active+clean; 63 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 20 KiB/s rd, 275 MiB/s wr, 878 op/s 2024-09-11T23:45:30.643 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:45:31.269 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id ff3854b3 --release --await' 2024-09-11T23:45:31.269 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id ff3854b3 --release --await 2024-09-11T23:45:31.372 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:31.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.827+0000 7f5803b9f640 1 -- 172.21.15.133:0/1869650088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57fc102570 msgr2=0x7f57fc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:31.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.827+0000 7f5803b9f640 1 --2- 172.21.15.133:0/1869650088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57fc102570 0x7f57fc1029b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009900 tx=0x7f57ec02a480 comp rx=0 tx=0).stop 2024-09-11T23:45:31.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.827+0000 7f5803b9f640 1 -- 172.21.15.133:0/1869650088 shutdown_connections 2024-09-11T23:45:31.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.827+0000 7f5803b9f640 1 --2- 172.21.15.133:0/1869650088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57fc102ef0 0x7f57fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.827+0000 7f5803b9f640 1 --2- 172.21.15.133:0/1869650088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57fc102570 0x7f57fc1029b0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.827+0000 7f5803b9f640 1 --2- 172.21.15.133:0/1869650088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57fc100e20 0x7f57fc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.827+0000 7f5803b9f640 1 -- 172.21.15.133:0/1869650088 >> 172.21.15.133:0/1869650088 conn(0x7f57fc0fc7d0 msgr2=0x7f57fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.827+0000 7f5803b9f640 1 -- 172.21.15.133:0/1869650088 shutdown_connections 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.827+0000 7f5803b9f640 1 -- 172.21.15.133:0/1869650088 wait complete. 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.827+0000 7f5803b9f640 1 Processor -- start 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f5803b9f640 1 -- start start 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f5803b9f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57fc100e20 0x7f57fc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f5803b9f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57fc102570 0x7f57fc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f5803b9f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57fc102ef0 0x7f57fc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f5803b9f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc110fa0 con 0x7f57fc100e20 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f5803b9f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc110e20 con 0x7f57fc102ef0 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f5803b9f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc111120 con 0x7f57fc102570 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f580193b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57fc100e20 0x7f57fc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f580193b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57fc100e20 0x7f57fc0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40720/0 (socket says 172.21.15.133:40720) 2024-09-11T23:45:31.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f580193b640 1 -- 172.21.15.133:0/1816717426 learned_addr learned my addr 172.21.15.133:0/1816717426 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:31.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f580213c640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57fc102ef0 0x7f57fc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f580193b640 1 -- 172.21.15.133:0/1816717426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57fc102570 msgr2=0x7f57fc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:31.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f580113a640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57fc102570 0x7f57fc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f580193b640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57fc102570 0x7f57fc0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f580193b640 1 -- 172.21.15.133:0/1816717426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57fc102ef0 msgr2=0x7f57fc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:31.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f580193b640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57fc102ef0 0x7f57fc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f580193b640 1 -- 172.21.15.133:0/1816717426 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc1a6a20 con 0x7f57fc100e20 2024-09-11T23:45:31.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.828+0000 7f580193b640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57fc100e20 0x7f57fc0fcc80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f57f800b4a0 tx=0x7f57f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:31.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.829+0000 7f57f2ffd640 1 -- 172.21.15.133:0/1816717426 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8014070 con 0x7f57fc100e20 2024-09-11T23:45:31.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.829+0000 7f57f2ffd640 1 -- 172.21.15.133:0/1816717426 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f8010af0 con 0x7f57fc100e20 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.829+0000 7f5803b9f640 1 -- 172.21.15.133:0/1816717426 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57fc1a6d10 con 0x7f57fc100e20 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.829+0000 7f57f2ffd640 1 -- 172.21.15.133:0/1816717426 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8017d10 con 0x7f57fc100e20 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.829+0000 7f5803b9f640 1 -- 172.21.15.133:0/1816717426 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57fc1a71a0 con 0x7f57fc100e20 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.831+0000 7f57f2ffd640 1 -- 172.21.15.133:0/1816717426 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f57f8017070 con 0x7f57fc100e20 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.831+0000 7f5805f78740 1 Processor -- start 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.831+0000 7f5805f78740 1 -- start start 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f5805f78740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa40b40 0x55ed4aa3f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f5805f78740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa3fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f5805f78740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa4a370 0x55ed4aa4c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f5805f78740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ed4a9a52f0 con 0x55ed4aa40b40 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f5805f78740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ed4a94ccf0 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f5805f78740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed4a9a12b0 con 0x55ed4aa4a370 2024-09-11T23:45:31.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f57f2ffd640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57e80a2cf0 0x7f57e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580213c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa40b40 0x55ed4aa3f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580213c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa40b40 0x55ed4aa3f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40734/0 (socket says 172.21.15.133:40734) 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580213c640 1 -- 172.21.15.133:0/1924282189 learned_addr learned my addr 172.21.15.133:0/1924282189 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580193b640 1 --2- 172.21.15.133:0/1924282189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa3fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580113a640 1 --2- 172.21.15.133:0/1924282189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa4a370 0x55ed4aa4c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580213c640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57e80a2cf0 0x7f57e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f57f2ffd640 1 -- 172.21.15.133:0/1816717426 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f57f80ce700 con 0x7f57fc100e20 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580213c640 1 -- 172.21.15.133:0/1924282189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa4a370 msgr2=0x55ed4aa4c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580213c640 1 --2- 172.21.15.133:0/1924282189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa4a370 0x55ed4aa4c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580213c640 1 -- 172.21.15.133:0/1924282189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa3f5d0 msgr2=0x55ed4aa3fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580213c640 1 --2- 172.21.15.133:0/1924282189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa3fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580213c640 1 -- 172.21.15.133:0/1924282189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed4a9313a0 con 0x55ed4aa40b40 2024-09-11T23:45:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.832+0000 7f580213c640 1 --2- 172.21.15.133:0/1924282189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa40b40 0x55ed4aa3f090 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f57f4005f40 tx=0x7f57f4014f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f4b5b787bc3459 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.833+0000 7f57e7fff640 1 -- 172.21.15.133:0/1924282189 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4012d90 con 0x55ed4aa40b40 2024-09-11T23:45:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.833+0000 7f57e7fff640 1 -- 172.21.15.133:0/1924282189 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f4023070 con 0x55ed4aa40b40 2024-09-11T23:45:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.833+0000 7f57e7fff640 1 -- 172.21.15.133:0/1924282189 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f400e360 con 0x55ed4aa40b40 2024-09-11T23:45:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.833+0000 7f580213c640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57e80a2cf0 0x7f57e80a51b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f57f4022000 tx=0x7f57f4020000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.833+0000 7f5805f78740 1 -- 172.21.15.133:0/1924282189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa40b40 msgr2=0x55ed4aa3f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.833+0000 7f5805f78740 1 --2- 172.21.15.133:0/1924282189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa40b40 0x55ed4aa3f090 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f57f4005f40 tx=0x7f57f4014f70 comp rx=0 tx=0).stop 2024-09-11T23:45:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.833+0000 7f5805f78740 1 -- 172.21.15.133:0/1924282189 shutdown_connections 2024-09-11T23:45:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.833+0000 7f5805f78740 1 --2- 172.21.15.133:0/1924282189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa4a370 0x55ed4aa4c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.833+0000 7f5805f78740 1 --2- 172.21.15.133:0/1924282189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa3fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.833+0000 7f5805f78740 1 --2- 172.21.15.133:0/1924282189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa40b40 0x55ed4aa3f090 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.833+0000 7f5805f78740 1 -- 172.21.15.133:0/1924282189 >> 172.21.15.133:0/1924282189 conn(0x55ed4aa08310 msgr2=0x55ed4aa3bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:31.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.834+0000 7f5805f78740 1 -- 172.21.15.133:0/1924282189 shutdown_connections 2024-09-11T23:45:31.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.834+0000 7f5805f78740 1 -- 172.21.15.133:0/1924282189 wait complete. 2024-09-11T23:45:31.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.834+0000 7f5805f78740 1 -- start start 2024-09-11T23:45:31.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.834+0000 7f5805f78740 1 Processor -- start 2024-09-11T23:45:31.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.834+0000 7f5805f78740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa49ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.834+0000 7f5805f78740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa46f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.834+0000 7f5805f78740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa47490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.834+0000 7f5805f78740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ed4aa4d070 con 0x55ed4aa4a370 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.834+0000 7f5805f78740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ed4a960060 con 0x55ed4aa40b40 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.834+0000 7f5805f78740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed4aa4d1f0 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.834+0000 7f5805f78740 10 client.0 _ll_register_callbacks cb 0x55ed4a6b38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f580213c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa47490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f580213c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa47490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40742/0 (socket says 172.21.15.133:40742) 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f580213c640 1 -- 172.21.15.133:0/4037004896 learned_addr learned my addr 172.21.15.133:0/4037004896 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f580113a640 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa46f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f580193b640 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa49ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f580213c640 1 -- 172.21.15.133:0/4037004896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 msgr2=0x55ed4aa49ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f580213c640 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa49ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f580213c640 1 -- 172.21.15.133:0/4037004896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 msgr2=0x55ed4aa46f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f580213c640 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa46f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f580213c640 1 -- 172.21.15.133:0/4037004896 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed4a7163c0 con 0x55ed4aa4a370 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f580213c640 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa47490 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f57f4005fd0 tx=0x7f57f4012960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df3de001318de6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f57e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f57e57fa640 1 -- 172.21.15.133:0/4037004896 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4012d90 con 0x55ed4aa4a370 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f57e57fa640 1 -- 172.21.15.133:0/4037004896 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f57f401ec50 con 0x55ed4aa4a370 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f5805f78740 1 -- 172.21.15.133:0/4037004896 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f4023070 con 0x55ed4aa4a370 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f57e57fa640 1 -- 172.21.15.133:0/4037004896 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57f401dcf0 con 0x55ed4aa4a370 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f5805f78740 10 client.17685 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.835+0000 7f5805f78740 1 -- 172.21.15.133:0/4037004896 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed4aa58a90 con 0x55ed4aa4a370 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.836+0000 7f57e57fa640 1 -- 172.21.15.133:0/4037004896 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f4023070 con 0x55ed4aa4a370 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.836+0000 7f57e57fa640 1 -- 172.21.15.133:0/4037004896 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f57f400f040 con 0x55ed4aa4a370 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.836+0000 7f5805f78740 10 client.17685 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.836+0000 7f5805f78740 10 client.17685 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.836+0000 7f5805f78740 20 client.17685 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:31.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.836+0000 7f5805f78740 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55ed4aa59f50 0x55ed4aa5c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.836+0000 7f5805f78740 4 client.17685 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.836+0000 7f5805f78740 1 -- 172.21.15.133:0/4037004896 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed4a9b2860 con 0x55ed4aa59f50 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.836+0000 7f580193b640 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55ed4aa59f50 0x55ed4aa5c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.836+0000 7f57e57fa640 10 client.17685 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.836+0000 7f580193b640 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55ed4aa59f50 0x55ed4aa5c340 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f5bb38b6d627d818 server_cookie=28c231cba79e346e in_seq=0 out_seq=0 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f57e57fa640 1 -- 172.21.15.133:0/4037004896 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed4a9b2860 con 0x55ed4aa59f50 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f57e57fa640 10 client.17685 handle_command_reply: tid=0 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 1 client.17685 shutdown 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 1 -- 172.21.15.133:0/4037004896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 msgr2=0x55ed4aa47490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa47490 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f57f4005fd0 tx=0x7f57f4012960 comp rx=0 tx=0).stop 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 1 -- 172.21.15.133:0/4037004896 shutdown_connections 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55ed4aa59f50 0x55ed4aa5c340 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa47490 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa46f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 1 --2- 172.21.15.133:0/4037004896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa49ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f57e57fa640 0 client.17685 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:45:31.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 1 -- 172.21.15.133:0/4037004896 >> 172.21.15.133:0/4037004896 conn(0x55ed4aa08310 msgr2=0x55ed4aa41930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:31.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 1 -- 172.21.15.133:0/4037004896 shutdown_connections 2024-09-11T23:45:31.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 1 -- 172.21.15.133:0/4037004896 wait complete. 2024-09-11T23:45:31.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.837+0000 7f5805f78740 20 client.17685 trim_cache size 0 max 16384 2024-09-11T23:45:31.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.850+0000 7f5805f78740 1 Processor -- start 2024-09-11T23:45:31.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.850+0000 7f5805f78740 1 -- start start 2024-09-11T23:45:31.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.850+0000 7f5805f78740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa56340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f5805f78740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa42120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f5805f78740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa42660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f5805f78740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1afab0 con 0x55ed4aa4a370 2024-09-11T23:45:31.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f5805f78740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1b15f0 con 0x55ed4aa40b40 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f5805f78740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1b0180 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580213c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa42660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580113a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa42120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580213c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa42660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40752/0 (socket says 172.21.15.133:40752) 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580213c640 1 -- 172.21.15.133:0/2266285571 learned_addr learned my addr 172.21.15.133:0/2266285571 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580193b640 1 --2- 172.21.15.133:0/2266285571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa56340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580113a640 1 -- 172.21.15.133:0/2266285571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 msgr2=0x55ed4aa56340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580113a640 1 --2- 172.21.15.133:0/2266285571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa56340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580113a640 1 -- 172.21.15.133:0/2266285571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 msgr2=0x55ed4aa42660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580113a640 1 --2- 172.21.15.133:0/2266285571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa42660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580113a640 1 -- 172.21.15.133:0/2266285571 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed4aa3bed0 con 0x55ed4aa40b40 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580193b640 1 --2- 172.21.15.133:0/2266285571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa56340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:31.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.851+0000 7f580113a640 1 --2- 172.21.15.133:0/2266285571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa42120 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f57ec006010 tx=0x7f57ec004310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da613a3760e8ab09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f57e77fe640 1 -- 172.21.15.133:0/2266285571 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec035040 con 0x55ed4aa40b40 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f57e77fe640 1 -- 172.21.15.133:0/2266285571 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57ec041070 con 0x55ed4aa40b40 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f57e77fe640 1 -- 172.21.15.133:0/2266285571 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0035f0 con 0x55ed4aa40b40 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f5805f78740 1 -- 172.21.15.133:0/2266285571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 msgr2=0x55ed4aa42120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f5805f78740 1 --2- 172.21.15.133:0/2266285571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa42120 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f57ec006010 tx=0x7f57ec004310 comp rx=0 tx=0).stop 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f5805f78740 1 -- 172.21.15.133:0/2266285571 shutdown_connections 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f5805f78740 1 --2- 172.21.15.133:0/2266285571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa42660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f5805f78740 1 --2- 172.21.15.133:0/2266285571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa42120 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f5805f78740 1 --2- 172.21.15.133:0/2266285571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa56340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f5805f78740 1 -- 172.21.15.133:0/2266285571 >> 172.21.15.133:0/2266285571 conn(0x55ed4aa08310 msgr2=0x55ed4aa46040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f5805f78740 1 -- 172.21.15.133:0/2266285571 shutdown_connections 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.852+0000 7f5805f78740 1 -- 172.21.15.133:0/2266285571 wait complete. 2024-09-11T23:45:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.853+0000 7f5805f78740 1 -- start start 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.853+0000 7f5805f78740 1 Processor -- start 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.853+0000 7f5805f78740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa52910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.853+0000 7f5805f78740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa43940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.853+0000 7f5805f78740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa43e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.853+0000 7f5805f78740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1b0f20 con 0x55ed4aa4a370 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.853+0000 7f5805f78740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1b0850 con 0x55ed4aa40b40 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.853+0000 7f5805f78740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1adab0 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.853+0000 7f5805f78740 10 client.0 _ll_register_callbacks cb 0x7f57f801de10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580213c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa43e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580213c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa43e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40754/0 (socket says 172.21.15.133:40754) 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580193b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa52910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580213c640 1 -- 172.21.15.133:0/3709412982 learned_addr learned my addr 172.21.15.133:0/3709412982 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580113a640 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa43940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580193b640 1 -- 172.21.15.133:0/3709412982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 msgr2=0x55ed4aa43940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580193b640 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa43940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580193b640 1 -- 172.21.15.133:0/3709412982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 msgr2=0x55ed4aa43e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580193b640 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa43e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580193b640 1 -- 172.21.15.133:0/3709412982 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed4aa443f0 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580213c640 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa43e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f580193b640 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa52910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f57f80cbfa0 tx=0x7f57f806cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50cdf5744024eaa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f57e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f57e57fa640 1 -- 172.21.15.133:0/3709412982 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80a39e0 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f57e57fa640 1 -- 172.21.15.133:0/3709412982 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f57f8010ca0 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.854+0000 7f5805f78740 1 -- 172.21.15.133:0/3709412982 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57ec041070 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f57e57fa640 1 -- 172.21.15.133:0/3709412982 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57f80cc110 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f5805f78740 10 client.27010 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f5805f78740 1 -- 172.21.15.133:0/3709412982 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed4aac4680 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f57e57fa640 1 -- 172.21.15.133:0/3709412982 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57ec041070 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f57e57fa640 1 -- 172.21.15.133:0/3709412982 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f57f80b1070 con 0x55ed4aa3f5d0 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f5805f78740 10 client.27010 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f5805f78740 10 client.27010 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f5805f78740 20 client.27010 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f5805f78740 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55ed4aac4da0 0x55ed4aac71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f5805f78740 4 client.27010 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f5805f78740 1 -- 172.21.15.133:0/3709412982 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed4a9acd70 con 0x55ed4aac4da0 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.855+0000 7f580113a640 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55ed4aac4da0 0x55ed4aac71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f580113a640 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55ed4aac4da0 0x55ed4aac71c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52c8fad046728d49 server_cookie=2e0d3e5e16bed3b5 in_seq=0 out_seq=0 2024-09-11T23:45:31.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f57e57fa640 10 client.27010 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f57e57fa640 1 -- 172.21.15.133:0/3709412982 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed4a9acd70 con 0x55ed4aac4da0 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f57e57fa640 10 client.27010 handle_command_reply: tid=0 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f5805f78740 1 client.27010 shutdown 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f5805f78740 1 -- 172.21.15.133:0/3709412982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 msgr2=0x55ed4aa52910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f5805f78740 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa52910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f57f80cbfa0 tx=0x7f57f806cd30 comp rx=0 tx=0).stop 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f5805f78740 1 -- 172.21.15.133:0/3709412982 shutdown_connections 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f5805f78740 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55ed4aac4da0 0x55ed4aac71c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f5805f78740 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ed4aa4a370 0x55ed4aa43e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f5805f78740 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed4aa40b40 0x55ed4aa43940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f5805f78740 1 --2- 172.21.15.133:0/3709412982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed4aa3f5d0 0x55ed4aa52910 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f57e57fa640 0 client.27010 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f5805f78740 1 -- 172.21.15.133:0/3709412982 >> 172.21.15.133:0/3709412982 conn(0x55ed4aa08310 msgr2=0x55ed4aa487b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f5805f78740 1 -- 172.21.15.133:0/3709412982 shutdown_connections 2024-09-11T23:45:31.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.856+0000 7f5805f78740 1 -- 172.21.15.133:0/3709412982 wait complete. 2024-09-11T23:45:31.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5805f78740 20 client.27010 trim_cache size 0 max 16384 2024-09-11T23:45:31.871 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:45:31.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 -- 172.21.15.133:0/1816717426 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57e80a2cf0 msgr2=0x7f57e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:31.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57e80a2cf0 0x7f57e80a51b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f57f4022000 tx=0x7f57f4020000 comp rx=0 tx=0).stop 2024-09-11T23:45:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 -- 172.21.15.133:0/1816717426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57fc100e20 msgr2=0x7f57fc0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57fc100e20 0x7f57fc0fcc80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f57f800b4a0 tx=0x7f57f800b960 comp rx=0 tx=0).stop 2024-09-11T23:45:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 -- 172.21.15.133:0/1816717426 shutdown_connections 2024-09-11T23:45:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57e80a2cf0 0x7f57e80a51b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57fc102ef0 0x7f57fc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57fc102570 0x7f57fc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 --2- 172.21.15.133:0/1816717426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57fc100e20 0x7f57fc0fcc80 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 -- 172.21.15.133:0/1816717426 >> 172.21.15.133:0/1816717426 conn(0x7f57fc0fc7d0 msgr2=0x7f57fc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 -- 172.21.15.133:0/1816717426 shutdown_connections 2024-09-11T23:45:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:31.857+0000 7f5803b9f640 1 -- 172.21.15.133:0/1816717426 wait complete. 2024-09-11T23:45:32.614 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:45:32.619 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:32.742 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:32 smithi133 ceph-mon[31430]: pgmap v272: 97 pgs: 97 active+clean; 63 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 20 KiB/s rd, 259 MiB/s wr, 821 op/s 2024-09-11T23:45:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.853+0000 7f2dc2627640 1 -- 172.21.15.133:0/1336294196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc1009b0 msgr2=0x7f2dbc100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.853+0000 7f2dc2627640 1 --2- 172.21.15.133:0/1336294196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc1009b0 0x7f2dbc100e10 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009930 tx=0x7f2da402a420 comp rx=0 tx=0).stop 2024-09-11T23:45:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.853+0000 7f2dc2627640 1 -- 172.21.15.133:0/1336294196 shutdown_connections 2024-09-11T23:45:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.853+0000 7f2dc2627640 1 --2- 172.21.15.133:0/1336294196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dbc101350 0x7f2dbc10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.853+0000 7f2dc2627640 1 --2- 172.21.15.133:0/1336294196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc1009b0 0x7f2dbc100e10 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.853+0000 7f2dc2627640 1 --2- 172.21.15.133:0/1336294196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dbc1069c0 0x7f2dbc106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.853+0000 7f2dc2627640 1 -- 172.21.15.133:0/1336294196 >> 172.21.15.133:0/1336294196 conn(0x7f2dbc077990 msgr2=0x7f2dbc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.853+0000 7f2dc2627640 1 -- 172.21.15.133:0/1336294196 shutdown_connections 2024-09-11T23:45:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.853+0000 7f2dc2627640 1 -- 172.21.15.133:0/1336294196 wait complete. 2024-09-11T23:45:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.854+0000 7f2dc2627640 1 Processor -- start 2024-09-11T23:45:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.854+0000 7f2dc2627640 1 -- start start 2024-09-11T23:45:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.854+0000 7f2dc2627640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc1009b0 0x7f2dbc1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.854+0000 7f2dc2627640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dbc101350 0x7f2dbc197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.854+0000 7f2dc2627640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dbc1069c0 0x7f2dbc19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.854+0000 7f2dc2627640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc110e00 con 0x7f2dbc1009b0 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.854+0000 7f2dc2627640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc110c80 con 0x7f2dbc101350 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.854+0000 7f2dc2627640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc110f80 con 0x7f2dbc1069c0 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.854+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc1009b0 0x7f2dbc1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc1009b0 0x7f2dbc1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47990/0 (socket says 172.21.15.133:47990) 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dbbfff640 1 -- 172.21.15.133:0/10140929 learned_addr learned my addr 172.21.15.133:0/10140929 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dbc101350 0x7f2dbc197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dbc1069c0 0x7f2dbc19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dbbfff640 1 -- 172.21.15.133:0/10140929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dbc1069c0 msgr2=0x7f2dbc19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dbc1069c0 0x7f2dbc19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dbbfff640 1 -- 172.21.15.133:0/10140929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dbc101350 msgr2=0x7f2dbc197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dbc101350 0x7f2dbc197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dbbfff640 1 -- 172.21.15.133:0/10140929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc19c2e0 con 0x7f2dbc1009b0 2024-09-11T23:45:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dbc101350 0x7f2dbc197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc1009b0 0x7f2dbc1972f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009860 tx=0x7f2dac009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2db97fa640 1 -- 172.21.15.133:0/10140929 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac005020 con 0x7f2dbc1009b0 2024-09-11T23:45:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dc2627640 1 -- 172.21.15.133:0/10140929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dbc19c5d0 con 0x7f2dbc1009b0 2024-09-11T23:45:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2db97fa640 1 -- 172.21.15.133:0/10140929 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dac0047a0 con 0x7f2dbc1009b0 2024-09-11T23:45:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2db97fa640 1 -- 172.21.15.133:0/10140929 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac007ba0 con 0x7f2dbc1009b0 2024-09-11T23:45:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.855+0000 7f2dc2627640 1 -- 172.21.15.133:0/10140929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dbc0698a0 con 0x7f2dbc1009b0 2024-09-11T23:45:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.857+0000 7f2db97fa640 1 -- 172.21.15.133:0/10140929 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2dac004950 con 0x7f2dbc1009b0 2024-09-11T23:45:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc53c5740 1 Processor -- start 2024-09-11T23:45:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc53c5740 1 -- start start 2024-09-11T23:45:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a2bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a2c410 0x55cce1a36770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a36cb0 0x55cce1a390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cce19902f0 con 0x55cce1a2baf0 2024-09-11T23:45:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cce1938270 con 0x55cce1a36cb0 2024-09-11T23:45:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cce198c2b0 con 0x55cce1a2c410 2024-09-11T23:45:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2db97fa640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2da80a2da0 0x7f2da80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc0bc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a2bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc0bc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a2bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47998/0 (socket says 172.21.15.133:47998) 2024-09-11T23:45:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc0bc4640 1 -- 172.21.15.133:0/516910463 learned_addr learned my addr 172.21.15.133:0/516910463 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/516910463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a36cb0 0x55cce1a390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2da80a2da0 0x7f2da80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/516910463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a2c410 0x55cce1a36770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.858+0000 7f2db97fa640 1 -- 172.21.15.133:0/10140929 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f2dac0d0950 con 0x7f2dbc1009b0 2024-09-11T23:45:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.859+0000 7f2dc0bc4640 1 -- 172.21.15.133:0/516910463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a2c410 msgr2=0x55cce1a36770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.859+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/516910463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a2c410 0x55cce1a36770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.859+0000 7f2dc0bc4640 1 -- 172.21.15.133:0/516910463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a36cb0 msgr2=0x55cce1a390a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.859+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/516910463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a36cb0 0x55cce1a390a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.859+0000 7f2dc0bc4640 1 -- 172.21.15.133:0/516910463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cce1937340 con 0x55cce1a2baf0 2024-09-11T23:45:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.859+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/516910463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a2bed0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2db4012d00 tx=0x7f2db4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0e04519c44ff40 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.859+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2da80a2da0 0x7f2da80a5260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009fd0 tx=0x7f2db4017000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.859+0000 7f2db27fc640 1 -- 172.21.15.133:0/516910463 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4022070 con 0x55cce1a2baf0 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.859+0000 7f2db27fc640 1 -- 172.21.15.133:0/516910463 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db40124f0 con 0x55cce1a2baf0 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.859+0000 7f2db27fc640 1 -- 172.21.15.133:0/516910463 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4005020 con 0x55cce1a2baf0 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.860+0000 7f2dc53c5740 1 -- 172.21.15.133:0/516910463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 msgr2=0x55cce1a2bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.860+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/516910463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a2bed0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2db4012d00 tx=0x7f2db4010740 comp rx=0 tx=0).stop 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.860+0000 7f2dc53c5740 1 -- 172.21.15.133:0/516910463 shutdown_connections 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.860+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/516910463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a36cb0 0x55cce1a390a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.860+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/516910463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a2c410 0x55cce1a36770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.860+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/516910463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a2bed0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.860+0000 7f2dc53c5740 1 -- 172.21.15.133:0/516910463 >> 172.21.15.133:0/516910463 conn(0x55cce1a28060 msgr2=0x55cce1a28440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.860+0000 7f2dc53c5740 1 -- 172.21.15.133:0/516910463 shutdown_connections 2024-09-11T23:45:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.860+0000 7f2dc53c5740 1 -- 172.21.15.133:0/516910463 wait complete. 2024-09-11T23:45:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.860+0000 7f2dc53c5740 1 -- start start 2024-09-11T23:45:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.860+0000 7f2dc53c5740 1 Processor -- start 2024-09-11T23:45:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a33c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a2c410 0x55cce1a34170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 0x55cce1a346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cce1a39880 con 0x55cce1a2baf0 2024-09-11T23:45:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cce1919290 con 0x55cce1a2c410 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cce1a39a00 con 0x55cce1a36cb0 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dc53c5740 10 client.0 _ll_register_callbacks cb 0x55cce194a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a33c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a33c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48010/0 (socket says 172.21.15.133:48010) 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dbbfff640 1 -- 172.21.15.133:0/2449529165 learned_addr learned my addr 172.21.15.133:0/2449529165 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a2c410 0x55cce1a34170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 0x55cce1a346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dbbfff640 1 -- 172.21.15.133:0/2449529165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 msgr2=0x55cce1a346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 0x55cce1a346b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dbbfff640 1 -- 172.21.15.133:0/2449529165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a2c410 msgr2=0x55cce1a34170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a2c410 0x55cce1a34170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dbbfff640 1 -- 172.21.15.133:0/2449529165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cce16a3210 con 0x55cce1a2baf0 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 0x55cce1a346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a2c410 0x55cce1a34170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a33c30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f2dac098b90 tx=0x7f2dac09cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97ed63fc6501e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2d9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/2449529165 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac026040 con 0x55cce1a2baf0 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/2449529165 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f2dac099520 con 0x55cce1a2baf0 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.861+0000 7f2dc53c5740 1 -- 172.21.15.133:0/2449529165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db40124f0 con 0x55cce1a2baf0 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/2449529165 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2dac09d5b0 con 0x55cce1a2baf0 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2dc53c5740 10 client.17703 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2dc53c5740 1 -- 172.21.15.133:0/2449529165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cce176f620 con 0x55cce1a2baf0 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/2449529165 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db40124f0 con 0x55cce1a2baf0 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/2449529165 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2dac0a5690 con 0x55cce1a2baf0 2024-09-11T23:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2dc53c5740 10 client.17703 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2dc53c5740 10 client.17703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2dc53c5740 20 client.17703 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cce1a46d40 0x55cce1a49130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2dc53c5740 4 client.17703 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2dc53c5740 1 -- 172.21.15.133:0/2449529165 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cce199da20 con 0x55cce1a46d40 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.862+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cce1a46d40 0x55cce1a49130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.863+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cce1a46d40 0x55cce1a49130 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a405396a15e68c13 server_cookie=24df9ff0177cb47f in_seq=0 out_seq=0 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.863+0000 7f2d9b7fe640 10 client.17703 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.863+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/2449529165 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cce199da20 con 0x55cce1a46d40 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.863+0000 7f2d9b7fe640 10 client.17703 handle_command_reply: tid=0 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.863+0000 7f2dc53c5740 1 client.17703 shutdown 2024-09-11T23:45:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2dc53c5740 1 -- 172.21.15.133:0/2449529165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 msgr2=0x55cce1a33c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a33c30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f2dac098b90 tx=0x7f2dac09cd40 comp rx=0 tx=0).stop 2024-09-11T23:45:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2dc53c5740 1 -- 172.21.15.133:0/2449529165 shutdown_connections 2024-09-11T23:45:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cce1a46d40 0x55cce1a49130 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 0x55cce1a346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2d9b7fe640 0 client.17703 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a2c410 0x55cce1a34170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/2449529165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a33c30 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2dc53c5740 1 -- 172.21.15.133:0/2449529165 >> 172.21.15.133:0/2449529165 conn(0x55cce1a28060 msgr2=0x55cce1a2dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2dc53c5740 1 -- 172.21.15.133:0/2449529165 shutdown_connections 2024-09-11T23:45:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2dc53c5740 1 -- 172.21.15.133:0/2449529165 wait complete. 2024-09-11T23:45:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.864+0000 7f2dc53c5740 20 client.17703 trim_cache size 0 max 16384 2024-09-11T23:45:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.886+0000 7f2dc53c5740 1 Processor -- start 2024-09-11T23:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.886+0000 7f2dc53c5740 1 -- start start 2024-09-11T23:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.886+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a2baf0 0x55cce1a2e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.886+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2c410 0x55cce1a344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.886+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 0x55cce1a4e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.886+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1ab4e0 con 0x55cce1a2c410 2024-09-11T23:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.886+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1ad020 con 0x55cce1a2baf0 2024-09-11T23:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.886+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1abbb0 con 0x55cce1a36cb0 2024-09-11T23:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2dc0bc4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 0x55cce1a4e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2dc0bc4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 0x55cce1a4e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40106/0 (socket says 172.21.15.133:40106) 2024-09-11T23:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2dc0bc4640 1 -- 172.21.15.133:0/3032942215 learned_addr learned my addr 172.21.15.133:0/3032942215 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/3032942215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2c410 0x55cce1a344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/3032942215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a2baf0 0x55cce1a2e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2dc0bc4640 1 -- 172.21.15.133:0/3032942215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a2baf0 msgr2=0x55cce1a2e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/3032942215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a2baf0 0x55cce1a2e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2dc0bc4640 1 -- 172.21.15.133:0/3032942215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2c410 msgr2=0x55cce1a344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/3032942215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2c410 0x55cce1a344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2dc0bc4640 1 -- 172.21.15.133:0/3032942215 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cce1a232d0 con 0x55cce1a36cb0 2024-09-11T23:45:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/3032942215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 0x55cce1a4e970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2db4013010 tx=0x7f2db4012b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e382810f8bdaf28 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2db1ffb640 1 -- 172.21.15.133:0/3032942215 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4022040 con 0x55cce1a36cb0 2024-09-11T23:45:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2db1ffb640 1 -- 172.21.15.133:0/3032942215 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db402a070 con 0x55cce1a36cb0 2024-09-11T23:45:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.887+0000 7f2db1ffb640 1 -- 172.21.15.133:0/3032942215 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db400cb40 con 0x55cce1a36cb0 2024-09-11T23:45:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.888+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3032942215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 msgr2=0x55cce1a4e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.888+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/3032942215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 0x55cce1a4e970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2db4013010 tx=0x7f2db4012b80 comp rx=0 tx=0).stop 2024-09-11T23:45:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.888+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3032942215 shutdown_connections 2024-09-11T23:45:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.888+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/3032942215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a36cb0 0x55cce1a4e970 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.888+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/3032942215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2c410 0x55cce1a344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.888+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/3032942215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a2baf0 0x55cce1a2e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.888+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3032942215 >> 172.21.15.133:0/3032942215 conn(0x55cce1a28060 msgr2=0x55cce1a35cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.888+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3032942215 shutdown_connections 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.888+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3032942215 wait complete. 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dc53c5740 1 -- start start 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dc53c5740 1 Processor -- start 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a3ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a2c410 0x55cce1a3f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dc53c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a36cb0 0x55cce1a3a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1ac950 con 0x55cce1a2baf0 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1ac280 con 0x55cce1a36cb0 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dc53c5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1a94e0 con 0x55cce1a2c410 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dc53c5740 10 client.0 _ll_register_callbacks cb 0x7f2dac0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a3ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a3ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48030/0 (socket says 172.21.15.133:48030) 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dbbfff640 1 -- 172.21.15.133:0/3310499481 learned_addr learned my addr 172.21.15.133:0/3310499481 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a2c410 0x55cce1a3f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.889+0000 7f2dc0bc4640 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a36cb0 0x55cce1a3a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2dbbfff640 1 -- 172.21.15.133:0/3310499481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a2c410 msgr2=0x55cce1a3f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a2c410 0x55cce1a3f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2dbbfff640 1 -- 172.21.15.133:0/3310499481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a36cb0 msgr2=0x55cce1a3a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a36cb0 0x55cce1a3a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2dbbfff640 1 -- 172.21.15.133:0/3310499481 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cce1a3aa40 con 0x55cce1a2baf0 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a3ed60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2dac004f80 tx=0x7f2dac09cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecb8c722fb8b608 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2d9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/3310499481 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac099150 con 0x55cce1a2baf0 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/3310499481 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f2dac0bbaf0 con 0x55cce1a2baf0 2024-09-11T23:45:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3310499481 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db402a070 con 0x55cce1a2baf0 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/3310499481 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2dac067f40 con 0x55cce1a2baf0 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2dc53c5740 10 client.17709 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3310499481 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cce1ab1430 con 0x55cce1a2baf0 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.890+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/3310499481 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dac09ceb0 con 0x55cce1a2baf0 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.891+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/3310499481 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2dac0051c0 con 0x55cce1a2baf0 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.891+0000 7f2dc53c5740 10 client.17709 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.891+0000 7f2dc53c5740 10 client.17709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.891+0000 7f2dc53c5740 20 client.17709 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.891+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cce1a46d40 0x55cce1ab3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.891+0000 7f2dc53c5740 4 client.17709 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.891+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3310499481 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f2db402a070 con 0x55cce1a46d40 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.891+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cce1a46d40 0x55cce1ab3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.891+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cce1a46d40 0x55cce1ab3bb0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b7a77151c57d126 server_cookie=d22820b8c85b8889 in_seq=0 out_seq=0 2024-09-11T23:45:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.891+0000 7f2d9b7fe640 10 client.17709 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:32.900 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:32 smithi138 ceph-mon[35142]: pgmap v272: 97 pgs: 97 active+clean; 63 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 20 KiB/s rd, 259 MiB/s wr, 821 op/s 2024-09-11T23:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/3310499481 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f2db402a070 con 0x55cce1a46d40 2024-09-11T23:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2d9b7fe640 10 client.17709 handle_command_reply: tid=0 2024-09-11T23:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 1 client.17709 shutdown 2024-09-11T23:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3310499481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 msgr2=0x55cce1a3ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a3ed60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2dac004f80 tx=0x7f2dac09cdb0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3310499481 shutdown_connections 2024-09-11T23:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cce1a46d40 0x55cce1ab3bb0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cce1a36cb0 0x55cce1a3a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2d9b7fe640 0 client.17709 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cce1a2c410 0x55cce1a3f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 1 --2- 172.21.15.133:0/3310499481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cce1a2baf0 0x55cce1a3ed60 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3310499481 >> 172.21.15.133:0/3310499481 conn(0x55cce1a28060 msgr2=0x55cce1a271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3310499481 shutdown_connections 2024-09-11T23:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 1 -- 172.21.15.133:0/3310499481 wait complete. 2024-09-11T23:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.923+0000 7f2dc53c5740 20 client.17709 trim_cache size 0 max 16384 2024-09-11T23:45:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:45:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:45:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:45:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 -- 172.21.15.133:0/10140929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2da80a2da0 msgr2=0x7f2da80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2da80a2da0 0x7f2da80a5260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009fd0 tx=0x7f2db4017000 comp rx=0 tx=0).stop 2024-09-11T23:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 -- 172.21.15.133:0/10140929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc1009b0 msgr2=0x7f2dbc1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc1009b0 0x7f2dbc1972f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009860 tx=0x7f2dac009d20 comp rx=0 tx=0).stop 2024-09-11T23:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 -- 172.21.15.133:0/10140929 shutdown_connections 2024-09-11T23:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2da80a2da0 0x7f2da80a5260 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dbc1069c0 0x7f2dbc19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dbc101350 0x7f2dbc197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 --2- 172.21.15.133:0/10140929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc1009b0 0x7f2dbc1972f0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 -- 172.21.15.133:0/10140929 >> 172.21.15.133:0/10140929 conn(0x7f2dbc077990 msgr2=0x7f2dbc1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 -- 172.21.15.133:0/10140929 shutdown_connections 2024-09-11T23:45:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:32.924+0000 7f2dc2627640 1 -- 172.21.15.133:0/10140929 wait complete. 2024-09-11T23:45:33.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:32 smithi178 ceph-mon[35882]: pgmap v272: 97 pgs: 97 active+clean; 63 GiB data, 217 GiB used, 856 GiB / 1.0 TiB avail; 20 KiB/s rd, 259 MiB/s wr, 821 op/s 2024-09-11T23:45:33.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.117+0000 7f66a90a7640 1 -- 172.21.15.133:0/119065990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66a41027a0 msgr2=0x7f66a41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:33.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.117+0000 7f66a90a7640 1 --2- 172.21.15.133:0/119065990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66a41027a0 0x7f66a41067f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f668c009900 tx=0x7f668c02a410 comp rx=0 tx=0).stop 2024-09-11T23:45:33.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.117+0000 7f66a90a7640 1 -- 172.21.15.133:0/119065990 shutdown_connections 2024-09-11T23:45:33.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.117+0000 7f66a90a7640 1 --2- 172.21.15.133:0/119065990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66a4106f20 0x7f66a410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.117+0000 7f66a90a7640 1 --2- 172.21.15.133:0/119065990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66a41027a0 0x7f66a41067f0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.117+0000 7f66a90a7640 1 --2- 172.21.15.133:0/119065990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a4101df0 0x7f66a41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.117+0000 7f66a90a7640 1 -- 172.21.15.133:0/119065990 >> 172.21.15.133:0/119065990 conn(0x7f66a40fd8b0 msgr2=0x7f66a40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.117+0000 7f66a90a7640 1 -- 172.21.15.133:0/119065990 shutdown_connections 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.117+0000 7f66a90a7640 1 -- 172.21.15.133:0/119065990 wait complete. 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.118+0000 7f66a90a7640 1 Processor -- start 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.118+0000 7f66a90a7640 1 -- start start 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.118+0000 7f66a90a7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a4101df0 0x7f66a4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.118+0000 7f66a90a7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66a41027a0 0x7f66a4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.118+0000 7f66a90a7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66a4106f20 0x7f66a419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.118+0000 7f66a90a7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4111fa0 con 0x7f66a4106f20 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.118+0000 7f66a90a7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4111e20 con 0x7f66a41027a0 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.118+0000 7f66a90a7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4112120 con 0x7f66a4101df0 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66a4106f20 0x7f66a419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66a4106f20 0x7f66a419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48052/0 (socket says 172.21.15.133:48052) 2024-09-11T23:45:33.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a359e640 1 -- 172.21.15.133:0/2106284785 learned_addr learned my addr 172.21.15.133:0/2106284785 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a4101df0 0x7f66a4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f669a59c640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66a41027a0 0x7f66a4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a359e640 1 -- 172.21.15.133:0/2106284785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a4101df0 msgr2=0x7f66a4198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a359e640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a4101df0 0x7f66a4198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a359e640 1 -- 172.21.15.133:0/2106284785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66a41027a0 msgr2=0x7f66a4198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a359e640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66a41027a0 0x7f66a4198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a359e640 1 -- 172.21.15.133:0/2106284785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a419d530 con 0x7f66a4106f20 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a359e640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66a4106f20 0x7f66a419ce50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f669c00eaa0 tx=0x7f669c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a0d99640 1 -- 172.21.15.133:0/2106284785 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c004d20 con 0x7f66a4106f20 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a90a7640 1 -- 172.21.15.133:0/2106284785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a419d820 con 0x7f66a4106f20 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a0d99640 1 -- 172.21.15.133:0/2106284785 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c005020 con 0x7f66a4106f20 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a0d99640 1 -- 172.21.15.133:0/2106284785 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c0116d0 con 0x7f66a4106f20 2024-09-11T23:45:33.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.119+0000 7f66a90a7640 1 -- 172.21.15.133:0/2106284785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a41a5100 con 0x7f66a4106f20 2024-09-11T23:45:33.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.122+0000 7f66a0d99640 1 -- 172.21.15.133:0/2106284785 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f669c0051d0 con 0x7f66a4106f20 2024-09-11T23:45:33.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.122+0000 7f66ab480740 1 Processor -- start 2024-09-11T23:45:33.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.122+0000 7f66ab480740 1 -- start start 2024-09-11T23:45:33.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66ab480740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b3e060 0x5558e4b3e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66ab480740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b48d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66ab480740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b49240 0x5558e4b4b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66ab480740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5558e4aa3620 con 0x5558e4b49240 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66ab480740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5558e4a4ae50 con 0x5558e4b3e060 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66ab480740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5558e4a9f5e0 con 0x5558e4b3e980 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f669a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b49240 0x5558e4b4b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f669a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b49240 0x5558e4b4b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48062/0 (socket says 172.21.15.133:48062) 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f669a59c640 1 -- 172.21.15.133:0/129194163 learned_addr learned my addr 172.21.15.133:0/129194163 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/129194163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b48d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f669a59c640 1 --2- 172.21.15.133:0/129194163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b3e060 0x5558e4b3e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66a2d9d640 1 -- 172.21.15.133:0/129194163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b3e060 msgr2=0x5558e4b3e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/129194163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b3e060 0x5558e4b3e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66a2d9d640 1 -- 172.21.15.133:0/129194163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b49240 msgr2=0x5558e4b4b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/129194163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b49240 0x5558e4b4b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66a2d9d640 1 -- 172.21.15.133:0/129194163 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558e4a3bd40 con 0x5558e4b3e980 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f669a59c640 1 --2- 172.21.15.133:0/129194163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b49240 0x5558e4b4b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.123+0000 7f66a0d99640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66900a2cf0 0x7f66900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.124+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/129194163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b48d00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6694010ae0 tx=0x7f66940093a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d51a8bec2f6f120 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.124+0000 7f669a59c640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66900a2cf0 0x7f66900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.124+0000 7f66a0d99640 1 -- 172.21.15.133:0/2106284785 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f669c0cf5f0 con 0x7f66a4106f20 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.124+0000 7f669959a640 1 -- 172.21.15.133:0/129194163 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669401e070 con 0x5558e4b3e980 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.124+0000 7f669959a640 1 -- 172.21.15.133:0/129194163 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6694009d80 con 0x5558e4b3e980 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.124+0000 7f669959a640 1 -- 172.21.15.133:0/129194163 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694018770 con 0x5558e4b3e980 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.125+0000 7f66ab480740 1 -- 172.21.15.133:0/129194163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 msgr2=0x5558e4b48d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.125+0000 7f66ab480740 1 --2- 172.21.15.133:0/129194163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b48d00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6694010ae0 tx=0x7f66940093a0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.125+0000 7f669a59c640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66900a2cf0 0x7f66900a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f668c005ec0 tx=0x7f668c000bc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.125+0000 7f66ab480740 1 -- 172.21.15.133:0/129194163 shutdown_connections 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.125+0000 7f66ab480740 1 --2- 172.21.15.133:0/129194163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b49240 0x5558e4b4b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.125+0000 7f66ab480740 1 --2- 172.21.15.133:0/129194163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b48d00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.125+0000 7f66ab480740 1 --2- 172.21.15.133:0/129194163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b3e060 0x5558e4b3e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.125+0000 7f66ab480740 1 -- 172.21.15.133:0/129194163 >> 172.21.15.133:0/129194163 conn(0x5558e4b3a5f0 msgr2=0x5558e4b3a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.125+0000 7f66ab480740 1 -- 172.21.15.133:0/129194163 shutdown_connections 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66ab480740 1 -- 172.21.15.133:0/129194163 wait complete. 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66ab480740 1 -- start start 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66ab480740 1 Processor -- start 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66ab480740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b46190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66ab480740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66ab480740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b49240 0x5558e4b46c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66ab480740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5558e4b4be10 con 0x5558e4b3e060 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66ab480740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5558e4a2bf10 con 0x5558e4b49240 2024-09-11T23:45:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66ab480740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5558e4b4bf90 con 0x5558e4b3e980 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66ab480740 10 client.0 _ll_register_callbacks cb 0x5558e4a4fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66a2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b46190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66a2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b46190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48070/0 (socket says 172.21.15.133:48070) 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.126+0000 7f66a2d9d640 1 -- 172.21.15.133:0/1145079238 learned_addr learned my addr 172.21.15.133:0/1145079238 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f669a59c640 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f66a359e640 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b49240 0x5558e4b46c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f66a2d9d640 1 -- 172.21.15.133:0/1145079238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 msgr2=0x5558e4b466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f66a2d9d640 1 -- 172.21.15.133:0/1145079238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b49240 msgr2=0x5558e4b46c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b49240 0x5558e4b46c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f66a2d9d640 1 -- 172.21.15.133:0/1145079238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558e47b2580 con 0x5558e4b3e060 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f669a59c640 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b46190 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6694004810 tx=0x7f66940106f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e82c2951b5fd533 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f668a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f668a7fc640 1 -- 172.21.15.133:0/1145079238 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669401e040 con 0x5558e4b3e060 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f668a7fc640 1 -- 172.21.15.133:0/1145079238 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f6694019b30 con 0x5558e4b3e060 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f66ab480740 1 -- 172.21.15.133:0/1145079238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6694009d80 con 0x5558e4b3e060 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f668a7fc640 1 -- 172.21.15.133:0/1145079238 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6694019d30 con 0x5558e4b3e060 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f66ab480740 10 client.17721 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.127+0000 7f66ab480740 1 -- 172.21.15.133:0/1145079238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5558e487c960 con 0x5558e4b3e060 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.128+0000 7f668a7fc640 1 -- 172.21.15.133:0/1145079238 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6694009d80 con 0x5558e4b3e060 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.128+0000 7f668a7fc640 1 -- 172.21.15.133:0/1145079238 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6694021070 con 0x5558e4b3e060 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.128+0000 7f66ab480740 10 client.17721 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.128+0000 7f66ab480740 10 client.17721 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.128+0000 7f66ab480740 20 client.17721 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:33.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.128+0000 7f66ab480740 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558e4b58db0 0x5558e4b5b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.128+0000 7f66ab480740 4 client.17721 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:33.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.128+0000 7f66ab480740 1 -- 172.21.15.133:0/1145079238 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558e4ab0b90 con 0x5558e4b58db0 2024-09-11T23:45:33.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.128+0000 7f669a59c640 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558e4b58db0 0x5558e4b5b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.129+0000 7f668a7fc640 10 client.17721 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:33.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.129+0000 7f669a59c640 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558e4b58db0 0x5558e4b5b1a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abc5144a005c78c6 server_cookie=20a112c5db2b225b in_seq=0 out_seq=0 2024-09-11T23:45:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.129+0000 7f668a7fc640 1 -- 172.21.15.133:0/1145079238 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558e4ab0b90 con 0x5558e4b58db0 2024-09-11T23:45:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.129+0000 7f668a7fc640 10 client.17721 handle_command_reply: tid=0 2024-09-11T23:45:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.129+0000 7f66ab480740 1 client.17721 shutdown 2024-09-11T23:45:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f66ab480740 1 -- 172.21.15.133:0/1145079238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 msgr2=0x5558e4b46190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f66ab480740 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b46190 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6694004810 tx=0x7f66940106f0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f66ab480740 1 -- 172.21.15.133:0/1145079238 shutdown_connections 2024-09-11T23:45:33.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f66ab480740 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558e4b58db0 0x5558e4b5b1a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f66ab480740 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b49240 0x5558e4b46c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f66ab480740 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f66ab480740 1 --2- 172.21.15.133:0/1145079238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b46190 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f66ab480740 1 -- 172.21.15.133:0/1145079238 >> 172.21.15.133:0/1145079238 conn(0x5558e4b3a5f0 msgr2=0x5558e4b402d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:33.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f668a7fc640 0 client.17721 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:33.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f66ab480740 1 -- 172.21.15.133:0/1145079238 shutdown_connections 2024-09-11T23:45:33.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f66ab480740 1 -- 172.21.15.133:0/1145079238 wait complete. 2024-09-11T23:45:33.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.130+0000 7f66ab480740 20 client.17721 trim_cache size 0 max 16384 2024-09-11T23:45:33.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.152+0000 7f66ab480740 1 Processor -- start 2024-09-11T23:45:33.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.152+0000 7f66ab480740 1 -- start start 2024-09-11T23:45:33.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f66ab480740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b3e060 0x5558e4b39a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f66ab480740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e980 0x5558e4b607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f66ab480740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b49240 0x5558e4b60d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f66ab480740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41ab580 con 0x5558e4b3e980 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f66ab480740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41ad0c0 con 0x5558e4b3e060 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f66ab480740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41abc50 con 0x5558e4b49240 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f669a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e980 0x5558e4b607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f669a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e980 0x5558e4b607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48072/0 (socket says 172.21.15.133:48072) 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f669a59c640 1 -- 172.21.15.133:0/690024595 learned_addr learned my addr 172.21.15.133:0/690024595 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f66a359e640 1 --2- 172.21.15.133:0/690024595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b49240 0x5558e4b60d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/690024595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b3e060 0x5558e4b39a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f669a59c640 1 -- 172.21.15.133:0/690024595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b49240 msgr2=0x5558e4b60d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f669a59c640 1 --2- 172.21.15.133:0/690024595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b49240 0x5558e4b60d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f669a59c640 1 -- 172.21.15.133:0/690024595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b3e060 msgr2=0x5558e4b39a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f669a59c640 1 --2- 172.21.15.133:0/690024595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b3e060 0x5558e4b39a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f669a59c640 1 -- 172.21.15.133:0/690024595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558e4b3dde0 con 0x5558e4b3e980 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/690024595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b3e060 0x5558e4b39a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.153+0000 7f669a59c640 1 --2- 172.21.15.133:0/690024595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e980 0x5558e4b607d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f668c009900 tx=0x7f668c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb29832cf83cf7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.154+0000 7f6698d99640 1 -- 172.21.15.133:0/690024595 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c059070 con 0x5558e4b3e980 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.154+0000 7f6698d99640 1 -- 172.21.15.133:0/690024595 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f668c005520 con 0x5558e4b3e980 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.154+0000 7f6698d99640 1 -- 172.21.15.133:0/690024595 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c035040 con 0x5558e4b3e980 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.154+0000 7f66ab480740 1 -- 172.21.15.133:0/690024595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e980 msgr2=0x5558e4b607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.154+0000 7f66ab480740 1 --2- 172.21.15.133:0/690024595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e980 0x5558e4b607d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f668c009900 tx=0x7f668c009260 comp rx=0 tx=0).stop 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.154+0000 7f66ab480740 1 -- 172.21.15.133:0/690024595 shutdown_connections 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.154+0000 7f66ab480740 1 --2- 172.21.15.133:0/690024595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b49240 0x5558e4b60d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 --2- 172.21.15.133:0/690024595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e980 0x5558e4b607d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 --2- 172.21.15.133:0/690024595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b3e060 0x5558e4b39a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 -- 172.21.15.133:0/690024595 >> 172.21.15.133:0/690024595 conn(0x5558e4b3a5f0 msgr2=0x5558e4b479f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 -- 172.21.15.133:0/690024595 shutdown_connections 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 -- 172.21.15.133:0/690024595 wait complete. 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 -- start start 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 Processor -- start 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b43130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b43670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b49240 0x5558e4b43bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41ac9f0 con 0x5558e4b3e060 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41ac320 con 0x5558e4b49240 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41a9580 con 0x5558e4b3e980 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66ab480740 10 client.0 _ll_register_callbacks cb 0x7f668c032e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66a2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b43130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66a2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b43130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48084/0 (socket says 172.21.15.133:48084) 2024-09-11T23:45:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.155+0000 7f66a2d9d640 1 -- 172.21.15.133:0/4073623024 learned_addr learned my addr 172.21.15.133:0/4073623024 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f669a59c640 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b43670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f66a359e640 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b49240 0x5558e4b43bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f66a2d9d640 1 -- 172.21.15.133:0/4073623024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 msgr2=0x5558e4b43670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b43670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f66a2d9d640 1 -- 172.21.15.133:0/4073623024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b49240 msgr2=0x5558e4b43bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b49240 0x5558e4b43bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f66a2d9d640 1 -- 172.21.15.133:0/4073623024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558e4bc27a0 con 0x5558e4b3e060 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f66a359e640 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b49240 0x5558e4b43bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f66a2d9d640 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b43130 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6694022620 tx=0x7f6694022ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94976ad6ef734b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f668a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f668a7fc640 1 -- 172.21.15.133:0/4073623024 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694019c60 con 0x5558e4b3e060 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f668a7fc640 1 -- 172.21.15.133:0/4073623024 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f6694019e00 con 0x5558e4b3e060 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.156+0000 7f66ab480740 1 -- 172.21.15.133:0/4073623024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f668c005520 con 0x5558e4b3e060 2024-09-11T23:45:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f668a7fc640 1 -- 172.21.15.133:0/4073623024 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f66940343f0 con 0x5558e4b3e060 2024-09-11T23:45:33.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f66ab480740 10 client.17727 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:33.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f66ab480740 1 -- 172.21.15.133:0/4073623024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5558e4bc3990 con 0x5558e4b3e060 2024-09-11T23:45:33.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f668a7fc640 1 -- 172.21.15.133:0/4073623024 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f668c005520 con 0x5558e4b3e060 2024-09-11T23:45:33.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f668a7fc640 1 -- 172.21.15.133:0/4073623024 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6694021050 con 0x5558e4b3e060 2024-09-11T23:45:33.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f66ab480740 10 client.17727 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:33.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f66ab480740 10 client.17727 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:45:33.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f66ab480740 20 client.17727 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:33.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f66ab480740 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558e4bc4100 0x5558e4bc6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:33.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f66ab480740 4 client.17727 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ff3854b3", "release": true, "await": true}] 2024-09-11T23:45:33.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f66ab480740 1 -- 172.21.15.133:0/4073623024 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ff3854b3", "release": true, "await": true}) -- 0x5558e4aab0a0 con 0x5558e4bc4100 2024-09-11T23:45:33.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.157+0000 7f669a59c640 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558e4bc4100 0x5558e4bc6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:33.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.158+0000 7f668a7fc640 10 client.17727 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:33.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.158+0000 7f669a59c640 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558e4bc4100 0x5558e4bc6520 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4325c3a7bc3f226 server_cookie=55b368ad946e6928 in_seq=0 out_seq=0 2024-09-11T23:45:33.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f668a7fc640 1 -- 172.21.15.133:0/4073623024 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5558e4aab0a0 con 0x5558e4bc4100 2024-09-11T23:45:33.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f668a7fc640 10 client.17727 handle_command_reply: tid=0 2024-09-11T23:45:33.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f66ab480740 1 client.17727 shutdown 2024-09-11T23:45:33.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f66ab480740 1 -- 172.21.15.133:0/4073623024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 msgr2=0x5558e4b43130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:33.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f66ab480740 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b43130 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6694022620 tx=0x7f6694022ba0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f66ab480740 1 -- 172.21.15.133:0/4073623024 shutdown_connections 2024-09-11T23:45:33.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f66ab480740 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5558e4bc4100 0x5558e4bc6520 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f66ab480740 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558e4b49240 0x5558e4b43bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f66ab480740 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5558e4b3e980 0x5558e4b43670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f668a7fc640 0 client.17727 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:33.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f66ab480740 1 --2- 172.21.15.133:0/4073623024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558e4b3e060 0x5558e4b43130 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f66ab480740 1 -- 172.21.15.133:0/4073623024 >> 172.21.15.133:0/4073623024 conn(0x5558e4b3a5f0 msgr2=0x5558e4b3fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:33.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f66ab480740 1 -- 172.21.15.133:0/4073623024 shutdown_connections 2024-09-11T23:45:33.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.164+0000 7f66ab480740 1 -- 172.21.15.133:0/4073623024 wait complete. 2024-09-11T23:45:33.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.165+0000 7f66ab480740 20 client.17727 trim_cache size 0 max 16384 2024-09-11T23:45:33.165 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:45:33.165 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:45:33.165 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 36, 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "ff3854b3": { 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 36, 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 139.4, 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:45:33.166 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:45:33.167 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:45:33.167 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:45:33.167 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:45:33.167 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:33.167 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:33.167 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:33.167 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:33.167 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:33.167 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:45:33.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.165+0000 7f66a90a7640 1 -- 172.21.15.133:0/2106284785 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66900a2cf0 msgr2=0x7f66900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:33.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.165+0000 7f66a90a7640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66900a2cf0 0x7f66900a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f668c005ec0 tx=0x7f668c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.165+0000 7f66a90a7640 1 -- 172.21.15.133:0/2106284785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66a4106f20 msgr2=0x7f66a419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:33.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.166+0000 7f66a90a7640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66a4106f20 0x7f66a419ce50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f669c00eaa0 tx=0x7f669c00eea0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.166+0000 7f66a90a7640 1 -- 172.21.15.133:0/2106284785 shutdown_connections 2024-09-11T23:45:33.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.166+0000 7f66a90a7640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66900a2cf0 0x7f66900a51b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.166+0000 7f66a90a7640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66a4106f20 0x7f66a419ce50 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.166+0000 7f66a90a7640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66a41027a0 0x7f66a4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.166+0000 7f66a90a7640 1 --2- 172.21.15.133:0/2106284785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a4101df0 0x7f66a4198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:33.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.166+0000 7f66a90a7640 1 -- 172.21.15.133:0/2106284785 >> 172.21.15.133:0/2106284785 conn(0x7f66a40fd8b0 msgr2=0x7f66a40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:33.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.166+0000 7f66a90a7640 1 -- 172.21.15.133:0/2106284785 shutdown_connections 2024-09-11T23:45:33.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:33.166+0000 7f66a90a7640 1 -- 172.21.15.133:0/2106284785 wait complete. 2024-09-11T23:45:33.170 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:34.607 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:45:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.276+0000 7f54ec9da640 1 -- 172.21.15.133:0/2760605927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e811b140 msgr2=0x7f54e811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.276+0000 7f54ec9da640 1 --2- 172.21.15.133:0/2760605927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e811b140 0x7f54e811d530 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009ca0 tx=0x7f54e002a8f0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.281+0000 7f54ec9da640 1 -- 172.21.15.133:0/2760605927 shutdown_connections 2024-09-11T23:45:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.281+0000 7f54ec9da640 1 --2- 172.21.15.133:0/2760605927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e811b140 0x7f54e811d530 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.281+0000 7f54ec9da640 1 --2- 172.21.15.133:0/2760605927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54e806ea30 0x7f54e806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.281+0000 7f54ec9da640 1 --2- 172.21.15.133:0/2760605927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e8110190 0x7f54e806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.281+0000 7f54ec9da640 1 -- 172.21.15.133:0/2760605927 >> 172.21.15.133:0/2760605927 conn(0x7f54e8073180 msgr2=0x7f54e8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.286+0000 7f54ec9da640 1 -- 172.21.15.133:0/2760605927 shutdown_connections 2024-09-11T23:45:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.287+0000 7f54ec9da640 1 -- 172.21.15.133:0/2760605927 wait complete. 2024-09-11T23:45:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.288+0000 7f54ec9da640 1 Processor -- start 2024-09-11T23:45:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.293+0000 7f54ec9da640 1 -- start start 2024-09-11T23:45:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.294+0000 7f54ec9da640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54e806ea30 0x7f54e81a7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.294+0000 7f54ec9da640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e8110190 0x7f54e81a7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.294+0000 7f54ec9da640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e811b140 0x7f54e81ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.294+0000 7f54ec9da640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8120c40 con 0x7f54e8110190 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.294+0000 7f54ec9da640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8120ac0 con 0x7f54e811b140 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.294+0000 7f54ec9da640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8120dc0 con 0x7f54e806ea30 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.294+0000 7f54e759e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e811b140 0x7f54e81ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.294+0000 7f54e759e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e811b140 0x7f54e81ab910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:57940/0 (socket says 172.21.15.133:57940) 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.294+0000 7f54e759e640 1 -- 172.21.15.133:0/2890252867 learned_addr learned my addr 172.21.15.133:0/2890252867 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.295+0000 7f54e659c640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e8110190 0x7f54e81a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.297+0000 7f54e759e640 1 -- 172.21.15.133:0/2890252867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54e806ea30 msgr2=0x7f54e81a7040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.297+0000 7f54e759e640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54e806ea30 0x7f54e81a7040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.297+0000 7f54e759e640 1 -- 172.21.15.133:0/2890252867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e8110190 msgr2=0x7f54e81a7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:34.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.297+0000 7f54e759e640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e8110190 0x7f54e81a7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.297+0000 7f54e759e640 1 -- 172.21.15.133:0/2890252867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e81abff0 con 0x7f54e811b140 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.297+0000 7f54e659c640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e8110190 0x7f54e81a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.301+0000 7f54e759e640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e811b140 0x7f54e81ab910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f54e002ae60 tx=0x7f54e0031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.306+0000 7f54cffff640 1 -- 172.21.15.133:0/2890252867 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0031b50 con 0x7f54e811b140 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.306+0000 7f54cffff640 1 -- 172.21.15.133:0/2890252867 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e0040070 con 0x7f54e811b140 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.306+0000 7f54cffff640 1 -- 172.21.15.133:0/2890252867 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e00305f0 con 0x7f54e811b140 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.308+0000 7f54ec9da640 1 -- 172.21.15.133:0/2890252867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e81ac280 con 0x7f54e811b140 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.308+0000 7f54ec9da640 1 -- 172.21.15.133:0/2890252867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e81ac740 con 0x7f54e811b140 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.313+0000 7f54ef5b4740 1 Processor -- start 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.313+0000 7f54ef5b4740 1 -- start start 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.313+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c68fcf0 0x55e60c68e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.313+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68e780 0x55e60c68ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.313+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c699520 0x55e60c69b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.313+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e60c5f42f0 con 0x55e60c68e780 2024-09-11T23:45:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.313+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e60c59bcf0 con 0x55e60c68fcf0 2024-09-11T23:45:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.313+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e60c5f02b0 con 0x55e60c699520 2024-09-11T23:45:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.320+0000 7f54e659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c699520 0x55e60c69b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.320+0000 7f54e659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c68fcf0 0x55e60c68e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.320+0000 7f54e659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c68fcf0 0x55e60c68e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:57946/0 (socket says 172.21.15.133:57946) 2024-09-11T23:45:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.320+0000 7f54e659c640 1 -- 172.21.15.133:0/3735386264 learned_addr learned my addr 172.21.15.133:0/3735386264 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.320+0000 7f54cffff640 1 -- 172.21.15.133:0/2890252867 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f54e002f640 con 0x7f54e811b140 2024-09-11T23:45:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.321+0000 7f54cffff640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f54d00a2f00 0x7f54d00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.322+0000 7f54e659c640 1 -- 172.21.15.133:0/3735386264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c699520 msgr2=0x55e60c69b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.322+0000 7f54e659c640 1 --2- 172.21.15.133:0/3735386264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c699520 0x55e60c69b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.322+0000 7f54e659c640 1 -- 172.21.15.133:0/3735386264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68e780 msgr2=0x55e60c68ebc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:45:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.322+0000 7f54e659c640 1 --2- 172.21.15.133:0/3735386264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68e780 0x55e60c68ebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.322+0000 7f54e659c640 1 -- 172.21.15.133:0/3735386264 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60c5803a0 con 0x55e60c68fcf0 2024-09-11T23:45:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.322+0000 7f54cffff640 1 -- 172.21.15.133:0/2890252867 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f54e00f0990 con 0x7f54e811b140 2024-09-11T23:45:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.325+0000 7f54e759e640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f54d00a2f00 0x7f54d00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.326+0000 7f54e659c640 1 --2- 172.21.15.133:0/3735386264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c68fcf0 0x55e60c68e240 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0127f0 tx=0x7f54dc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e6ac9d4f026b76c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.326+0000 7f54e759e640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f54d00a2f00 0x7f54d00a53c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f54e00ee230 tx=0x7f54e0005f60 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.330+0000 7f54cd7fa640 1 -- 172.21.15.133:0/3735386264 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0120d0 con 0x55e60c68fcf0 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.330+0000 7f54cd7fa640 1 -- 172.21.15.133:0/3735386264 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54dc012270 con 0x55e60c68fcf0 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.331+0000 7f54ef5b4740 1 -- 172.21.15.133:0/3735386264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c68fcf0 msgr2=0x55e60c68e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.331+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/3735386264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c68fcf0 0x55e60c68e240 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0127f0 tx=0x7f54dc005f90 comp rx=0 tx=0).stop 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.333+0000 7f54ef5b4740 1 -- 172.21.15.133:0/3735386264 shutdown_connections 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.333+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/3735386264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c699520 0x55e60c69b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.333+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/3735386264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68e780 0x55e60c68ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.333+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/3735386264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c68fcf0 0x55e60c68e240 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.333+0000 7f54ef5b4740 1 -- 172.21.15.133:0/3735386264 >> 172.21.15.133:0/3735386264 conn(0x55e60c657310 msgr2=0x55e60c68ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.335+0000 7f54ef5b4740 1 -- 172.21.15.133:0/3735386264 shutdown_connections 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.342+0000 7f54ef5b4740 1 -- 172.21.15.133:0/3735386264 wait complete. 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.342+0000 7f54ef5b4740 1 -- start start 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.343+0000 7f54ef5b4740 1 Processor -- start 2024-09-11T23:45:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.344+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 0x55e60c698e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.344+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68fcf0 0x55e60c6960c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.344+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c699520 0x55e60c696600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.344+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e60c6ad270 con 0x55e60c68fcf0 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.344+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e60c5af060 con 0x55e60c699520 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.344+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e60c6ad3f0 con 0x55e60c68e780 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.344+0000 7f54ef5b4740 10 client.0 _ll_register_callbacks cb 0x55e60c3028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.346+0000 7f54e659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 0x55e60c698e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.346+0000 7f54e659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c699520 0x55e60c696600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.347+0000 7f54e659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 0x55e60c698e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40172/0 (socket says 172.21.15.133:40172) 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.347+0000 7f54e659c640 1 -- 172.21.15.133:0/426364873 learned_addr learned my addr 172.21.15.133:0/426364873 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.350+0000 7f54e659c640 1 -- 172.21.15.133:0/426364873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c699520 msgr2=0x55e60c696600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.350+0000 7f54e659c640 1 --2- 172.21.15.133:0/426364873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c699520 0x55e60c696600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.350+0000 7f54e659c640 1 -- 172.21.15.133:0/426364873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68fcf0 msgr2=0x55e60c6960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.350+0000 7f54e659c640 1 --2- 172.21.15.133:0/426364873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68fcf0 0x55e60c6960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.350+0000 7f54e659c640 1 -- 172.21.15.133:0/426364873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60c3653c0 con 0x55e60c68e780 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.351+0000 7f54e659c640 1 --2- 172.21.15.133:0/426364873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 0x55e60c698e10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00fbd0 tx=0x7f54dc00fcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34530ef120a85da2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.351+0000 7f54caff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.355+0000 7f54caff5640 1 -- 172.21.15.133:0/426364873 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc025020 con 0x55e60c68e780 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.356+0000 7f54ef5b4740 1 -- 172.21.15.133:0/426364873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54dc012270 con 0x55e60c68e780 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.356+0000 7f54caff5640 1 -- 172.21.15.133:0/426364873 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f54dc0052d0 con 0x55e60c68e780 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.359+0000 7f54caff5640 1 -- 172.21.15.133:0/426364873 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f54dc0054d0 con 0x55e60c68e780 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.368+0000 7f54ef5b4740 10 client.27049 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.368+0000 7f54ef5b4740 1 -- 172.21.15.133:0/426364873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e60c6b8ca0 con 0x55e60c68e780 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.371+0000 7f54caff5640 1 -- 172.21.15.133:0/426364873 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54dc012270 con 0x55e60c68e780 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.372+0000 7f54caff5640 1 -- 172.21.15.133:0/426364873 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f54dc021450 con 0x55e60c68e780 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.373+0000 7f54ef5b4740 10 client.27049 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.373+0000 7f54ef5b4740 10 client.27049 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:45:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.373+0000 7f54ef5b4740 20 client.27049 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.374+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/426364873 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55e60c6ba100 0x55e60c6bc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.374+0000 7f54ef5b4740 4 client.27049 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.374+0000 7f54ef5b4740 1 -- 172.21.15.133:0/426364873 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e60c601860 con 0x55e60c6ba100 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.378+0000 7f54e6d9d640 1 --2- 172.21.15.133:0/426364873 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55e60c6ba100 0x55e60c6bc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.384+0000 7f54e6d9d640 1 --2- 172.21.15.133:0/426364873 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55e60c6ba100 0x55e60c6bc4f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa0f11a0de63372b server_cookie=1b297b5294197c82 in_seq=0 out_seq=0 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.385+0000 7f54caff5640 10 client.27049 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.386+0000 7f54caff5640 1 -- 172.21.15.133:0/426364873 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e60c601860 con 0x55e60c6ba100 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.386+0000 7f54caff5640 10 client.27049 handle_command_reply: tid=0 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.387+0000 7f54ef5b4740 1 client.27049 shutdown 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.387+0000 7f54ef5b4740 1 -- 172.21.15.133:0/426364873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 msgr2=0x55e60c698e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.387+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/426364873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 0x55e60c698e10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00fbd0 tx=0x7f54dc00fcd0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.390+0000 7f54ef5b4740 1 -- 172.21.15.133:0/426364873 shutdown_connections 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.390+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/426364873 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55e60c6ba100 0x55e60c6bc4f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.390+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/426364873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c699520 0x55e60c696600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.390+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/426364873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68fcf0 0x55e60c6960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.390+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/426364873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 0x55e60c698e10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.390+0000 7f54ef5b4740 1 -- 172.21.15.133:0/426364873 >> 172.21.15.133:0/426364873 conn(0x55e60c657310 msgr2=0x55e60c690b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.390+0000 7f54caff5640 0 client.27049 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.390+0000 7f54ef5b4740 1 -- 172.21.15.133:0/426364873 shutdown_connections 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.390+0000 7f54ef5b4740 1 -- 172.21.15.133:0/426364873 wait complete. 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.390+0000 7f54ef5b4740 20 client.27049 trim_cache size 0 max 16384 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.425+0000 7f54ef5b4740 1 Processor -- start 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.425+0000 7f54ef5b4740 1 -- start start 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.426+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 0x55e60c6b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.426+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68fcf0 0x55e60c694990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.426+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c699520 0x55e60c6912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.426+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81b3c50 con 0x55e60c68fcf0 2024-09-11T23:45:34.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.426+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81b5770 con 0x55e60c699520 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.427+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81b4300 con 0x55e60c68e780 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.427+0000 7f54e6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68fcf0 0x55e60c694990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.427+0000 7f54e6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68fcf0 0x55e60c694990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48118/0 (socket says 172.21.15.133:48118) 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.427+0000 7f54e6d9d640 1 -- 172.21.15.133:0/1507914682 learned_addr learned my addr 172.21.15.133:0/1507914682 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.427+0000 7f54e659c640 1 --2- 172.21.15.133:0/1507914682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 0x55e60c6b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.428+0000 7f54e6d9d640 1 -- 172.21.15.133:0/1507914682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 msgr2=0x55e60c6b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.428+0000 7f54e6d9d640 1 --2- 172.21.15.133:0/1507914682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 0x55e60c6b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.428+0000 7f54e6d9d640 1 -- 172.21.15.133:0/1507914682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c699520 msgr2=0x55e60c6912d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.428+0000 7f54e6d9d640 1 --2- 172.21.15.133:0/1507914682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c699520 0x55e60c6912d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.428+0000 7f54e6d9d640 1 -- 172.21.15.133:0/1507914682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60c699140 con 0x55e60c68fcf0 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.428+0000 7f54e6d9d640 1 --2- 172.21.15.133:0/1507914682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68fcf0 0x55e60c694990 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f54d8000c00 tx=0x7f54d8009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe6b21dfdd83850 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.429+0000 7f54ccff9640 1 -- 172.21.15.133:0/1507914682 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8022070 con 0x55e60c68fcf0 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.429+0000 7f54ccff9640 1 -- 172.21.15.133:0/1507914682 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d8012c70 con 0x55e60c68fcf0 2024-09-11T23:45:34.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.431+0000 7f54ef5b4740 1 -- 172.21.15.133:0/1507914682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68fcf0 msgr2=0x55e60c694990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.431+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/1507914682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68fcf0 0x55e60c694990 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f54d8000c00 tx=0x7f54d8009cc0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.432+0000 7f54ef5b4740 1 -- 172.21.15.133:0/1507914682 shutdown_connections 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.432+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/1507914682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c699520 0x55e60c6912d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.432+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/1507914682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c68fcf0 0x55e60c694990 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.432+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/1507914682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c68e780 0x55e60c6b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.432+0000 7f54ef5b4740 1 -- 172.21.15.133:0/1507914682 >> 172.21.15.133:0/1507914682 conn(0x55e60c657310 msgr2=0x55e60c689ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.439+0000 7f54ef5b4740 1 -- 172.21.15.133:0/1507914682 shutdown_connections 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.439+0000 7f54ef5b4740 1 -- 172.21.15.133:0/1507914682 wait complete. 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.440+0000 7f54ef5b4740 1 -- start start 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.440+0000 7f54ef5b4740 1 Processor -- start 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.440+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c68e780 0x55e60c6ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.440+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c699520 0x55e60c6adc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.440+0000 7f54ef5b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c6ae140 0x55e60c722c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.440+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81b50a0 con 0x55e60c699520 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.440+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81b49d0 con 0x55e60c68e780 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.440+0000 7f54ef5b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81a83a0 con 0x55e60c6ae140 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.440+0000 7f54ef5b4740 10 client.0 _ll_register_callbacks cb 0x7f54dc009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.441+0000 7f54e6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c699520 0x55e60c6adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.441+0000 7f54e6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c699520 0x55e60c6adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48124/0 (socket says 172.21.15.133:48124) 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.441+0000 7f54e6d9d640 1 -- 172.21.15.133:0/767090738 learned_addr learned my addr 172.21.15.133:0/767090738 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.442+0000 7f54e6d9d640 1 -- 172.21.15.133:0/767090738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c6ae140 msgr2=0x55e60c722c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.442+0000 7f54e6d9d640 1 --2- 172.21.15.133:0/767090738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c6ae140 0x55e60c722c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.442+0000 7f54e6d9d640 1 -- 172.21.15.133:0/767090738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c68e780 msgr2=0x55e60c6ad6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.442+0000 7f54e6d9d640 1 --2- 172.21.15.133:0/767090738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c68e780 0x55e60c6ad6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.442+0000 7f54e6d9d640 1 -- 172.21.15.133:0/767090738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60c723230 con 0x55e60c699520 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.443+0000 7f54e6d9d640 1 --2- 172.21.15.133:0/767090738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c699520 0x55e60c6adc00 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f54d801d670 tx=0x7f54d801d6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d645ba279e75655 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.443+0000 7f54caff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:34.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.444+0000 7f54caff5640 1 -- 172.21.15.133:0/767090738 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8022040 con 0x55e60c699520 2024-09-11T23:45:34.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.444+0000 7f54caff5640 1 -- 172.21.15.133:0/767090738 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f54d8004040 con 0x55e60c699520 2024-09-11T23:45:34.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.444+0000 7f54ef5b4740 1 -- 172.21.15.133:0/767090738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e60c5fbd70 con 0x55e60c699520 2024-09-11T23:45:34.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.445+0000 7f54caff5640 1 -- 172.21.15.133:0/767090738 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f54d8009230 con 0x55e60c699520 2024-09-11T23:45:34.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.445+0000 7f54ef5b4740 10 client.17739 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:34.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.445+0000 7f54ef5b4740 1 -- 172.21.15.133:0/767090738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e60c724580 con 0x55e60c699520 2024-09-11T23:45:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.447+0000 7f54caff5640 1 -- 172.21.15.133:0/767090738 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e60c5fbd70 con 0x55e60c699520 2024-09-11T23:45:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.447+0000 7f54caff5640 1 -- 172.21.15.133:0/767090738 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f54d80036b0 con 0x55e60c699520 2024-09-11T23:45:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.448+0000 7f54ef5b4740 10 client.17739 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.448+0000 7f54ef5b4740 10 client.17739 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:45:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.448+0000 7f54ef5b4740 20 client.17739 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.448+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/767090738 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55e60c724bc0 0x55e60c726fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.448+0000 7f54ef5b4740 4 client.17739 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:45:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.448+0000 7f54ef5b4740 1 -- 172.21.15.133:0/767090738 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e60c578170 con 0x55e60c724bc0 2024-09-11T23:45:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.449+0000 7f54e659c640 1 --2- 172.21.15.133:0/767090738 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55e60c724bc0 0x55e60c726fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.457+0000 7f54e659c640 1 --2- 172.21.15.133:0/767090738 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55e60c724bc0 0x55e60c726fe0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90bc8e5c535cdc3f server_cookie=323c3a40d738d09e in_seq=0 out_seq=0 2024-09-11T23:45:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.457+0000 7f54caff5640 10 client.17739 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:45:34.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.458+0000 7f54caff5640 1 -- 172.21.15.133:0/767090738 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e60c578170 con 0x55e60c724bc0 2024-09-11T23:45:34.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.458+0000 7f54caff5640 10 client.17739 handle_command_reply: tid=0 2024-09-11T23:45:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.459+0000 7f54ef5b4740 1 client.17739 shutdown 2024-09-11T23:45:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.466+0000 7f54ef5b4740 1 -- 172.21.15.133:0/767090738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c699520 msgr2=0x55e60c6adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.466+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/767090738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c699520 0x55e60c6adc00 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f54d801d670 tx=0x7f54d801d6a0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.466+0000 7f54ef5b4740 1 -- 172.21.15.133:0/767090738 shutdown_connections 2024-09-11T23:45:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.466+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/767090738 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55e60c724bc0 0x55e60c726fe0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.466+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/767090738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e60c6ae140 0x55e60c722c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.466+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/767090738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e60c699520 0x55e60c6adc00 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.466+0000 7f54ef5b4740 1 --2- 172.21.15.133:0/767090738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e60c68e780 0x55e60c6ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.466+0000 7f54ef5b4740 1 -- 172.21.15.133:0/767090738 >> 172.21.15.133:0/767090738 conn(0x55e60c657310 msgr2=0x55e60c695950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.484+0000 7f54caff5640 0 client.17739 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:45:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.486+0000 7f54ef5b4740 1 -- 172.21.15.133:0/767090738 shutdown_connections 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.486+0000 7f54ef5b4740 1 -- 172.21.15.133:0/767090738 wait complete. 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.487+0000 7f54ef5b4740 20 client.17739 trim_cache size 0 max 16384 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.491+0000 7f54ec9da640 1 -- 172.21.15.133:0/2890252867 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f54d00a2f00 msgr2=0x7f54d00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.491+0000 7f54ec9da640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f54d00a2f00 0x7f54d00a53c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f54e00ee230 tx=0x7f54e0005f60 comp rx=0 tx=0).stop 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.491+0000 7f54ec9da640 1 -- 172.21.15.133:0/2890252867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e811b140 msgr2=0x7f54e81ab910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.491+0000 7f54ec9da640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e811b140 0x7f54e81ab910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f54e002ae60 tx=0x7f54e0031a20 comp rx=0 tx=0).stop 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.492+0000 7f54ec9da640 1 -- 172.21.15.133:0/2890252867 shutdown_connections 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.492+0000 7f54ec9da640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f54d00a2f00 0x7f54d00a53c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.492+0000 7f54ec9da640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e811b140 0x7f54e81ab910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.492+0000 7f54ec9da640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e8110190 0x7f54e81a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.492+0000 7f54ec9da640 1 --2- 172.21.15.133:0/2890252867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54e806ea30 0x7f54e81a7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:34.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.492+0000 7f54ec9da640 1 -- 172.21.15.133:0/2890252867 >> 172.21.15.133:0/2890252867 conn(0x7f54e8073180 msgr2=0x7f54e8119230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.493+0000 7f54ec9da640 1 -- 172.21.15.133:0/2890252867 shutdown_connections 2024-09-11T23:45:34.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:34.493+0000 7f54ec9da640 1 -- 172.21.15.133:0/2890252867 wait complete. 2024-09-11T23:45:34.627 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ff3854b3', total seconds elapsed: 26.0 2024-09-11T23:45:34.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:34 smithi138 ceph-mon[35142]: pgmap v273: 97 pgs: 97 active+clean; 63 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 22 KiB/s rd, 131 MiB/s wr, 557 op/s 2024-09-11T23:45:34.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:34 smithi133 ceph-mon[31430]: pgmap v273: 97 pgs: 97 active+clean; 63 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 22 KiB/s rd, 131 MiB/s wr, 557 op/s 2024-09-11T23:45:35.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:34 smithi178 ceph-mon[35882]: pgmap v273: 97 pgs: 97 active+clean; 63 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 22 KiB/s rd, 131 MiB/s wr, 557 op/s 2024-09-11T23:45:35.926 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:45:37.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:36 smithi178 ceph-mon[35882]: pgmap v274: 97 pgs: 97 active+clean; 63 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 19 KiB/s rd, 97 MiB/s wr, 479 op/s 2024-09-11T23:45:37.088 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:37.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:36 smithi138 ceph-mon[35142]: pgmap v274: 97 pgs: 97 active+clean; 63 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 19 KiB/s rd, 97 MiB/s wr, 479 op/s 2024-09-11T23:45:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:36 smithi133 ceph-mon[31430]: pgmap v274: 97 pgs: 97 active+clean; 63 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 19 KiB/s rd, 97 MiB/s wr, 479 op/s 2024-09-11T23:45:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.955+0000 7efdbcfde640 1 -- 172.21.15.133:0/1692444086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb810ea80 msgr2=0x7efdb81153d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.957+0000 7efdbcfde640 1 --2- 172.21.15.133:0/1692444086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb810ea80 0x7efdb81153d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009cd0 tx=0x7efdb002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:45:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.966+0000 7efdbcfde640 1 -- 172.21.15.133:0/1692444086 shutdown_connections 2024-09-11T23:45:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.966+0000 7efdbcfde640 1 --2- 172.21.15.133:0/1692444086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb810ea80 0x7efdb81153d0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.966+0000 7efdbcfde640 1 --2- 172.21.15.133:0/1692444086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdb8109e00 0x7efdb810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.966+0000 7efdbcfde640 1 --2- 172.21.15.133:0/1692444086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdb8109450 0x7efdb8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.966+0000 7efdbcfde640 1 -- 172.21.15.133:0/1692444086 >> 172.21.15.133:0/1692444086 conn(0x7efdb8073210 msgr2=0x7efdb8073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:37.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.968+0000 7efdbcfde640 1 -- 172.21.15.133:0/1692444086 shutdown_connections 2024-09-11T23:45:37.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.969+0000 7efdbcfde640 1 -- 172.21.15.133:0/1692444086 wait complete. 2024-09-11T23:45:37.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.970+0000 7efdbcfde640 1 Processor -- start 2024-09-11T23:45:37.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.971+0000 7efdbcfde640 1 -- start start 2024-09-11T23:45:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.971+0000 7efdbcfde640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdb8109450 0x7efdb819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.971+0000 7efdbcfde640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdb8109e00 0x7efdb819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.972+0000 7efdbcfde640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb810ea80 0x7efdb81a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.972+0000 7efdbcfde640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8118840 con 0x7efdb810ea80 2024-09-11T23:45:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.972+0000 7efdbcfde640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efdb81186c0 con 0x7efdb8109450 2024-09-11T23:45:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.972+0000 7efdbcfde640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efdb81189c0 con 0x7efdb8109e00 2024-09-11T23:45:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.972+0000 7efdb6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb810ea80 0x7efdb81a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.972+0000 7efdb6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb810ea80 0x7efdb81a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48142/0 (socket says 172.21.15.133:48142) 2024-09-11T23:45:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.972+0000 7efdb6d9d640 1 -- 172.21.15.133:0/1213389885 learned_addr learned my addr 172.21.15.133:0/1213389885 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.973+0000 7efdb6d9d640 1 -- 172.21.15.133:0/1213389885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdb8109e00 msgr2=0x7efdb819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.973+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdb8109e00 0x7efdb819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.973+0000 7efdb6d9d640 1 -- 172.21.15.133:0/1213389885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdb8109450 msgr2=0x7efdb819f710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:45:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.973+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdb8109450 0x7efdb819f710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.973+0000 7efdb6d9d640 1 -- 172.21.15.133:0/1213389885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb81a4ca0 con 0x7efdb810ea80 2024-09-11T23:45:37.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.974+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb810ea80 0x7efdb81a4760 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7efdb0003ba0 tx=0x7efdb0003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:37.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.976+0000 7efda77fe640 1 -- 172.21.15.133:0/1213389885 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0004240 con 0x7efdb810ea80 2024-09-11T23:45:37.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.976+0000 7efda77fe640 1 -- 172.21.15.133:0/1213389885 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdb0031d50 con 0x7efdb810ea80 2024-09-11T23:45:37.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.976+0000 7efdbcfde640 1 -- 172.21.15.133:0/1213389885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb81a4f30 con 0x7efdb810ea80 2024-09-11T23:45:37.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.977+0000 7efdbcfde640 1 -- 172.21.15.133:0/1213389885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb81a53b0 con 0x7efdb810ea80 2024-09-11T23:45:37.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.978+0000 7efda77fe640 1 -- 172.21.15.133:0/1213389885 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0030e20 con 0x7efdb810ea80 2024-09-11T23:45:37.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.982+0000 7efda77fe640 1 -- 172.21.15.133:0/1213389885 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7efdb0030730 con 0x7efdb810ea80 2024-09-11T23:45:37.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.983+0000 7efda77fe640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efda00a2e00 0x7efda00a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:37.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:37.985+0000 7efda77fe640 1 -- 172.21.15.133:0/1213389885 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7efdb00bb2c0 con 0x7efdb810ea80 2024-09-11T23:45:38.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.001+0000 7efdb659c640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efda00a2e00 0x7efda00a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:38.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.002+0000 7efdb659c640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efda00a2e00 0x7efda00a52c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efda800b990 tx=0x7efda800b370 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:38.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.003+0000 7efdbf3b7740 1 Processor -- start 2024-09-11T23:45:38.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.003+0000 7efdbf3b7740 1 -- start start 2024-09-11T23:45:38.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.004+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cd3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.004+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd3780 0x55d2d9cd3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.005+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9ce0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.005+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9c392f0 con 0x55d2d9cd3780 2024-09-11T23:45:38.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.005+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9be0cf0 con 0x55d2d9cde520 2024-09-11T23:45:38.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.005+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9c352b0 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.006+0000 7efdb659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cd3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:38.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.006+0000 7efdb6d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9ce0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:38.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.007+0000 7efdb6d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9ce0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:57984/0 (socket says 172.21.15.133:57984) 2024-09-11T23:45:38.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.007+0000 7efdb6d9d640 1 -- 172.21.15.133:0/33604160 learned_addr learned my addr 172.21.15.133:0/33604160 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:38.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.009+0000 7efdb6d9d640 1 -- 172.21.15.133:0/33604160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cd4cf0 msgr2=0x55d2d9cd3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:38.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.009+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/33604160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cd3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.009+0000 7efdb6d9d640 1 -- 172.21.15.133:0/33604160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd3780 msgr2=0x55d2d9cd3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:38.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.009+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/33604160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd3780 0x55d2d9cd3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.009+0000 7efdb6d9d640 1 -- 172.21.15.133:0/33604160 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d9bc53a0 con 0x55d2d9cde520 2024-09-11T23:45:38.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.011+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/33604160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9ce0910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efdb0030b70 tx=0x7efdb00bcb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40f38f75e8d1e1af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.012+0000 7efda4ff9640 1 -- 172.21.15.133:0/33604160 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb00d5020 con 0x55d2d9cde520 2024-09-11T23:45:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.012+0000 7efda4ff9640 1 -- 172.21.15.133:0/33604160 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdb0036040 con 0x55d2d9cde520 2024-09-11T23:45:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.012+0000 7efda4ff9640 1 -- 172.21.15.133:0/33604160 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0004da0 con 0x55d2d9cde520 2024-09-11T23:45:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.015+0000 7efdbf3b7740 1 -- 172.21.15.133:0/33604160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 msgr2=0x55d2d9ce0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.015+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/33604160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9ce0910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efdb0030b70 tx=0x7efdb00bcb00 comp rx=0 tx=0).stop 2024-09-11T23:45:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.016+0000 7efdbf3b7740 1 -- 172.21.15.133:0/33604160 shutdown_connections 2024-09-11T23:45:38.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.016+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/33604160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9ce0910 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.016+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/33604160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd3780 0x55d2d9cd3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.016+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/33604160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cd3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.016+0000 7efdbf3b7740 1 -- 172.21.15.133:0/33604160 >> 172.21.15.133:0/33604160 conn(0x55d2d9c9c310 msgr2=0x55d2d9ccfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:38.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.017+0000 7efdbf3b7740 1 -- 172.21.15.133:0/33604160 shutdown_connections 2024-09-11T23:45:38.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.018+0000 7efdbf3b7740 1 -- 172.21.15.133:0/33604160 wait complete. 2024-09-11T23:45:38.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.020+0000 7efdbf3b7740 1 -- start start 2024-09-11T23:45:38.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.020+0000 7efdbf3b7740 1 Processor -- start 2024-09-11T23:45:38.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.049+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd3780 0x55d2d9cddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.049+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cdb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.049+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cde520 0x55d2d9cdb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.049+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9ce0fe0 con 0x55d2d9cd3780 2024-09-11T23:45:38.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.049+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9bf4060 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.049+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9ce1160 con 0x55d2d9cde520 2024-09-11T23:45:38.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.049+0000 7efdbf3b7740 10 client.0 _ll_register_callbacks cb 0x55d2d99478c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:38.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.070+0000 7efdb5d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cdb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:38.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.070+0000 7efdb5d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cdb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58000/0 (socket says 172.21.15.133:58000) 2024-09-11T23:45:38.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.070+0000 7efdb5d9b640 1 -- 172.21.15.133:0/3506159632 learned_addr learned my addr 172.21.15.133:0/3506159632 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:38.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.073+0000 7efdb5d9b640 1 -- 172.21.15.133:0/3506159632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cde520 msgr2=0x55d2d9cdb990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:45:38.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.073+0000 7efdb5d9b640 1 --2- 172.21.15.133:0/3506159632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cde520 0x55d2d9cdb990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.073+0000 7efdb5d9b640 1 -- 172.21.15.133:0/3506159632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd3780 msgr2=0x55d2d9cddfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:38.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.073+0000 7efdb5d9b640 1 --2- 172.21.15.133:0/3506159632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd3780 0x55d2d9cddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.073+0000 7efdb5d9b640 1 -- 172.21.15.133:0/3506159632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d99aa3c0 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.076+0000 7efdb5d9b640 1 --2- 172.21.15.133:0/3506159632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cdb250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efdac00bb10 tx=0x7efdac00d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c162adb3659842 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:38.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.077+0000 7efd9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:45:38.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.077+0000 7efd9dffb640 1 -- 172.21.15.133:0/3506159632 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac016070 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.077+0000 7efd9dffb640 1 -- 172.21.15.133:0/3506159632 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7efdac010460 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.077+0000 7efdbf3b7740 1 -- 172.21.15.133:0/3506159632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdb0036040 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.081+0000 7efd9dffb640 1 -- 172.21.15.133:0/3506159632 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efdac020050 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.081+0000 7efdbf3b7740 10 client.27254 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:38.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.081+0000 7efdbf3b7740 1 -- 172.21.15.133:0/3506159632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d2d9cecb60 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.082+0000 7efd9dffb640 1 -- 172.21.15.133:0/3506159632 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdac010b40 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.082+0000 7efd9dffb640 1 -- 172.21.15.133:0/3506159632 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efdac01bb40 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.082+0000 7efdbf3b7740 10 client.27254 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.083+0000 7efdbf3b7740 10 client.27254 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:45:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.083+0000 7efdbf3b7740 20 client.27254 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.083+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/3506159632 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55d2d9cee070 0x55d2d9cf0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.083+0000 7efdbf3b7740 4 client.27254 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.083+0000 7efdbf3b7740 1 -- 172.21.15.133:0/3506159632 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efdb0036040 con 0x55d2d9cee070 2024-09-11T23:45:38.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.083+0000 7efdb659c640 1 --2- 172.21.15.133:0/3506159632 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55d2d9cee070 0x55d2d9cf0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.084+0000 7efd9dffb640 10 client.27254 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:45:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.085+0000 7efdb659c640 1 --2- 172.21.15.133:0/3506159632 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55d2d9cee070 0x55d2d9cf0530 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cde7aad691668933 server_cookie=79ce19c02cc2b855 in_seq=0 out_seq=0 2024-09-11T23:45:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.090+0000 7efd9dffb640 1 -- 172.21.15.133:0/3506159632 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7efdb0036040 con 0x55d2d9cee070 2024-09-11T23:45:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.090+0000 7efd9dffb640 10 client.27254 handle_command_reply: tid=0 2024-09-11T23:45:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.090+0000 7efdbf3b7740 1 client.27254 shutdown 2024-09-11T23:45:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.094+0000 7efdbf3b7740 1 -- 172.21.15.133:0/3506159632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd4cf0 msgr2=0x55d2d9cdb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.094+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/3506159632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cdb250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efdac00bb10 tx=0x7efdac00d2e0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.094+0000 7efdbf3b7740 1 -- 172.21.15.133:0/3506159632 shutdown_connections 2024-09-11T23:45:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.094+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/3506159632 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55d2d9cee070 0x55d2d9cf0530 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.094+0000 7efd9dffb640 0 client.27254 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:45:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.094+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/3506159632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cde520 0x55d2d9cdb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.094+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/3506159632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cdb250 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.094+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/3506159632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd3780 0x55d2d9cddfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.094+0000 7efdbf3b7740 1 -- 172.21.15.133:0/3506159632 >> 172.21.15.133:0/3506159632 conn(0x55d2d9c9c310 msgr2=0x55d2d9cd5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.094+0000 7efdbf3b7740 1 -- 172.21.15.133:0/3506159632 shutdown_connections 2024-09-11T23:45:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.094+0000 7efdbf3b7740 1 -- 172.21.15.133:0/3506159632 wait complete. 2024-09-11T23:45:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.095+0000 7efdbf3b7740 20 client.27254 trim_cache size 0 max 16384 2024-09-11T23:45:38.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.130+0000 7efdbf3b7740 1 Processor -- start 2024-09-11T23:45:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.130+0000 7efdbf3b7740 1 -- start start 2024-09-11T23:45:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.131+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cd3780 0x55d2d9cdab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.131+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.131+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9cdd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.131+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efd9800e760 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.132+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd980102a0 con 0x55d2d9cde520 2024-09-11T23:45:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.132+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd9800ee30 con 0x55d2d9cd3780 2024-09-11T23:45:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.133+0000 7efdb6d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9cdd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.133+0000 7efdb6d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9cdd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:58016/0 (socket says 172.21.15.133:58016) 2024-09-11T23:45:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.133+0000 7efdb6d9d640 1 -- 172.21.15.133:0/2347029105 learned_addr learned my addr 172.21.15.133:0/2347029105 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.133+0000 7efdb659c640 1 --2- 172.21.15.133:0/2347029105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cd3780 0x55d2d9cdab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:38.315 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:37 smithi178 ceph-mon[35882]: pgmap v275: 97 pgs: 97 active+clean; 63 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 19 KiB/s rd, 77 MiB/s wr, 577 op/s 2024-09-11T23:45:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.137+0000 7efdb6d9d640 1 -- 172.21.15.133:0/2347029105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cd3780 msgr2=0x55d2d9cdab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.137+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/2347029105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cd3780 0x55d2d9cdab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.137+0000 7efdb6d9d640 1 -- 172.21.15.133:0/2347029105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd4cf0 msgr2=0x55d2d9cea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:38.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.137+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/2347029105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.137+0000 7efdb6d9d640 1 -- 172.21.15.133:0/2347029105 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d9cd0b10 con 0x55d2d9cde520 2024-09-11T23:45:38.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.139+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/2347029105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9cdd1c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efdb0006010 tx=0x7efdb0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3799a25fb0bbc74 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:38.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.139+0000 7efd9ffff640 1 -- 172.21.15.133:0/2347029105 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb00d5030 con 0x55d2d9cde520 2024-09-11T23:45:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.140+0000 7efd9ffff640 1 -- 172.21.15.133:0/2347029105 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdb0036050 con 0x55d2d9cde520 2024-09-11T23:45:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.140+0000 7efd9ffff640 1 -- 172.21.15.133:0/2347029105 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb00c1370 con 0x55d2d9cde520 2024-09-11T23:45:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.141+0000 7efdbf3b7740 1 -- 172.21.15.133:0/2347029105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 msgr2=0x55d2d9cdd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.141+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/2347029105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9cdd1c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efdb0006010 tx=0x7efdb0009920 comp rx=0 tx=0).stop 2024-09-11T23:45:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.141+0000 7efdbf3b7740 1 -- 172.21.15.133:0/2347029105 shutdown_connections 2024-09-11T23:45:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.142+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/2347029105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cde520 0x55d2d9cdd1c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.142+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/2347029105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9cea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.142+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/2347029105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cd3780 0x55d2d9cdab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.142+0000 7efdbf3b7740 1 -- 172.21.15.133:0/2347029105 >> 172.21.15.133:0/2347029105 conn(0x55d2d9c9c310 msgr2=0x55d2d9cdcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:38.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.142+0000 7efdbf3b7740 1 -- 172.21.15.133:0/2347029105 shutdown_connections 2024-09-11T23:45:38.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.142+0000 7efdbf3b7740 1 -- 172.21.15.133:0/2347029105 wait complete. 2024-09-11T23:45:38.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.143+0000 7efdbf3b7740 1 -- start start 2024-09-11T23:45:38.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.143+0000 7efdbf3b7740 1 Processor -- start 2024-09-11T23:45:38.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.144+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd3780 0x55d2d9ce6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.144+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9ce1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.144+0000 7efdbf3b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cde520 0x55d2d9ce2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.144+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efd9800fbd0 con 0x55d2d9cd4cf0 2024-09-11T23:45:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.144+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd9800f500 con 0x55d2d9cd3780 2024-09-11T23:45:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.144+0000 7efdbf3b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd9800c760 con 0x55d2d9cde520 2024-09-11T23:45:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.145+0000 7efdbf3b7740 10 client.0 _ll_register_callbacks cb 0x7efdac005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.145+0000 7efdb6d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cde520 0x55d2d9ce2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.145+0000 7efdb6d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cde520 0x55d2d9ce2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40236/0 (socket says 172.21.15.133:40236) 2024-09-11T23:45:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.145+0000 7efdb6d9d640 1 -- 172.21.15.133:0/1611827443 learned_addr learned my addr 172.21.15.133:0/1611827443 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.147+0000 7efdb659c640 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd3780 0x55d2d9ce6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.150+0000 7efdb6d9d640 1 -- 172.21.15.133:0/1611827443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd3780 msgr2=0x55d2d9ce6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.150+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd3780 0x55d2d9ce6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.150+0000 7efdb6d9d640 1 -- 172.21.15.133:0/1611827443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd4cf0 msgr2=0x55d2d9ce1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.150+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9ce1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.150+0000 7efdb6d9d640 1 -- 172.21.15.133:0/1611827443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d9d57e10 con 0x55d2d9cde520 2024-09-11T23:45:38.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.154+0000 7efdb6d9d640 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cde520 0x55d2d9ce2780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efdb00c5700 tx=0x7efdb00bc640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f895d6d08bce74 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:38.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.154+0000 7efd9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:45:38.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.156+0000 7efd9dffb640 1 -- 172.21.15.133:0/1611827443 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb00d5070 con 0x55d2d9cde520 2024-09-11T23:45:38.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.156+0000 7efd9dffb640 1 -- 172.21.15.133:0/1611827443 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7efdb00b8e90 con 0x55d2d9cde520 2024-09-11T23:45:38.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.157+0000 7efdbf3b7740 1 -- 172.21.15.133:0/1611827443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdb0036050 con 0x55d2d9cde520 2024-09-11T23:45:38.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.159+0000 7efd9dffb640 1 -- 172.21.15.133:0/1611827443 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efdb00cb050 con 0x55d2d9cde520 2024-09-11T23:45:38.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.160+0000 7efdbf3b7740 10 client.27064 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:38.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.161+0000 7efdbf3b7740 1 -- 172.21.15.133:0/1611827443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d2d9d58fd0 con 0x55d2d9cde520 2024-09-11T23:45:38.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.162+0000 7efd9dffb640 1 -- 172.21.15.133:0/1611827443 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdb00bc730 con 0x55d2d9cde520 2024-09-11T23:45:38.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.163+0000 7efd9dffb640 1 -- 172.21.15.133:0/1611827443 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efdb00bf030 con 0x55d2d9cde520 2024-09-11T23:45:38.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.163+0000 7efdbf3b7740 10 client.27064 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:38.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.163+0000 7efdbf3b7740 10 client.27064 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:45:38.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.163+0000 7efdbf3b7740 20 client.27064 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:38.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.163+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55d2d9cee070 0x55d2d9d5b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:38.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.163+0000 7efdbf3b7740 4 client.27064 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:45:38.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.163+0000 7efdbf3b7740 1 -- 172.21.15.133:0/1611827443 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7efdb0036050 con 0x55d2d9cee070 2024-09-11T23:45:38.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.170+0000 7efdb659c640 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55d2d9cee070 0x55d2d9d5b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:38.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.178+0000 7efdb659c640 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55d2d9cee070 0x55d2d9d5b260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1053cf051b29b8d8 server_cookie=9f7136bd05bf3a5f in_seq=0 out_seq=0 2024-09-11T23:45:38.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.178+0000 7efd9dffb640 10 client.27064 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:45:38.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.179+0000 7efd9dffb640 1 -- 172.21.15.133:0/1611827443 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7efdb0036050 con 0x55d2d9cee070 2024-09-11T23:45:38.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.179+0000 7efd9dffb640 10 client.27064 handle_command_reply: tid=0 2024-09-11T23:45:38.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.179+0000 7efdbf3b7740 1 client.27064 shutdown 2024-09-11T23:45:38.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.179+0000 7efdbf3b7740 1 -- 172.21.15.133:0/1611827443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cde520 msgr2=0x55d2d9ce2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:38.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.179+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cde520 0x55d2d9ce2780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efdb00c5700 tx=0x7efdb00bc640 comp rx=0 tx=0).stop 2024-09-11T23:45:38.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.181+0000 7efdbf3b7740 1 -- 172.21.15.133:0/1611827443 shutdown_connections 2024-09-11T23:45:38.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.181+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55d2d9cee070 0x55d2d9d5b260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.181+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2d9cde520 0x55d2d9ce2780 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.181+0000 7efd9dffb640 0 client.27064 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:45:38.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.181+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9cd4cf0 0x55d2d9ce1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.183+0000 7efdbf3b7740 1 --2- 172.21.15.133:0/1611827443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d9cd3780 0x55d2d9ce6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.183+0000 7efdbf3b7740 1 -- 172.21.15.133:0/1611827443 >> 172.21.15.133:0/1611827443 conn(0x55d2d9c9c310 msgr2=0x55d2d9cdd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:38.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.183+0000 7efdbf3b7740 1 -- 172.21.15.133:0/1611827443 shutdown_connections 2024-09-11T23:45:38.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.183+0000 7efdbf3b7740 1 -- 172.21.15.133:0/1611827443 wait complete. 2024-09-11T23:45:38.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.183+0000 7efdbf3b7740 20 client.27064 trim_cache size 0 max 16384 2024-09-11T23:45:38.330 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:37 smithi133 ceph-mon[31430]: pgmap v275: 97 pgs: 97 active+clean; 63 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 19 KiB/s rd, 77 MiB/s wr, 577 op/s 2024-09-11T23:45:38.330 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:45:38.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.189+0000 7efdbcfde640 1 -- 172.21.15.133:0/1213389885 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efda00a2e00 msgr2=0x7efda00a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:38.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.189+0000 7efdbcfde640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efda00a2e00 0x7efda00a52c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efda800b990 tx=0x7efda800b370 comp rx=0 tx=0).stop 2024-09-11T23:45:38.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.189+0000 7efdbcfde640 1 -- 172.21.15.133:0/1213389885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb810ea80 msgr2=0x7efdb81a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:38.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.189+0000 7efdbcfde640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb810ea80 0x7efdb81a4760 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7efdb0003ba0 tx=0x7efdb0003bd0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.189+0000 7efdbcfde640 1 -- 172.21.15.133:0/1213389885 shutdown_connections 2024-09-11T23:45:38.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.189+0000 7efdbcfde640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7efda00a2e00 0x7efda00a52c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.189+0000 7efdbcfde640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb810ea80 0x7efdb81a4760 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.189+0000 7efdbcfde640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdb8109e00 0x7efdb819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.189+0000 7efdbcfde640 1 --2- 172.21.15.133:0/1213389885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdb8109450 0x7efdb819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:38.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.189+0000 7efdbcfde640 1 -- 172.21.15.133:0/1213389885 >> 172.21.15.133:0/1213389885 conn(0x7efdb8073210 msgr2=0x7efdb810d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:38.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.190+0000 7efdbcfde640 1 -- 172.21.15.133:0/1213389885 shutdown_connections 2024-09-11T23:45:38.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:38.190+0000 7efdbcfde640 1 -- 172.21.15.133:0/1213389885 wait complete. 2024-09-11T23:45:38.450 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:37 smithi138 ceph-mon[35142]: pgmap v275: 97 pgs: 97 active+clean; 63 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 19 KiB/s rd, 77 MiB/s wr, 577 op/s 2024-09-11T23:45:39.142 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:45:40.127 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:40.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:40 smithi138 ceph-mon[35142]: pgmap v276: 97 pgs: 97 active+clean; 64 GiB data, 202 GiB used, 870 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 228 MiB/s wr, 1.64k op/s 2024-09-11T23:45:40.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:40 smithi133 ceph-mon[31430]: pgmap v276: 97 pgs: 97 active+clean; 64 GiB data, 202 GiB used, 870 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 228 MiB/s wr, 1.64k op/s 2024-09-11T23:45:41.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:40 smithi178 ceph-mon[35882]: pgmap v276: 97 pgs: 97 active+clean; 64 GiB data, 202 GiB used, 870 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 228 MiB/s wr, 1.64k op/s 2024-09-11T23:45:41.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.227+0000 7f38f2ffd640 1 -- 172.21.15.133:0/22585413 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec02f340 con 0x7f38f411b140 2024-09-11T23:45:41.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.227+0000 7f38f8828640 1 -- 172.21.15.133:0/22585413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38f411b140 msgr2=0x7f38f411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:41.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.227+0000 7f38f8828640 1 --2- 172.21.15.133:0/22585413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38f411b140 0x7f38f411d530 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009ca0 tx=0x7f38ec02a8f0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.228+0000 7f38f8828640 1 -- 172.21.15.133:0/22585413 shutdown_connections 2024-09-11T23:45:41.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.228+0000 7f38f8828640 1 --2- 172.21.15.133:0/22585413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38f411b140 0x7f38f411d530 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.228+0000 7f38f8828640 1 --2- 172.21.15.133:0/22585413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38f406ea30 0x7f38f406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.228+0000 7f38f8828640 1 --2- 172.21.15.133:0/22585413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38f4110190 0x7f38f406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.228+0000 7f38f8828640 1 -- 172.21.15.133:0/22585413 >> 172.21.15.133:0/22585413 conn(0x7f38f4073180 msgr2=0x7f38f4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:41.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.235+0000 7f38f8828640 1 -- 172.21.15.133:0/22585413 shutdown_connections 2024-09-11T23:45:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.235+0000 7f38f8828640 1 -- 172.21.15.133:0/22585413 wait complete. 2024-09-11T23:45:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.236+0000 7f38f8828640 1 Processor -- start 2024-09-11T23:45:41.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.236+0000 7f38f8828640 1 -- start start 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.237+0000 7f38f8828640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38f406ea30 0x7f38f41196c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.237+0000 7f38f8828640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38f4110190 0x7f38f4119c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.237+0000 7f38f8828640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38f411b140 0x7f38f4112930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.237+0000 7f38f8828640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38f4120be0 con 0x7f38f411b140 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.237+0000 7f38f8828640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38f4120a60 con 0x7f38f406ea30 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.237+0000 7f38f8828640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f38f4120d60 con 0x7f38f4110190 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.237+0000 7f38ebfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38f4110190 0x7f38f4119c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.237+0000 7f38ebfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38f4110190 0x7f38f4119c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40264/0 (socket says 172.21.15.133:40264) 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.237+0000 7f38ebfff640 1 -- 172.21.15.133:0/1196267920 learned_addr learned my addr 172.21.15.133:0/1196267920 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.238+0000 7f38f37fe640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38f406ea30 0x7f38f41196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.238+0000 7f38ebfff640 1 -- 172.21.15.133:0/1196267920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38f406ea30 msgr2=0x7f38f41196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.238+0000 7f38ebfff640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38f406ea30 0x7f38f41196c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.238+0000 7f38ebfff640 1 -- 172.21.15.133:0/1196267920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38f411b140 msgr2=0x7f38f4112930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:41.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.238+0000 7f38ebfff640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38f411b140 0x7f38f4112930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.238+0000 7f38ebfff640 1 -- 172.21.15.133:0/1196267920 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f41131f0 con 0x7f38f4110190 2024-09-11T23:45:41.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.239+0000 7f38ebfff640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38f4110190 0x7f38f4119c00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d950 tx=0x7f38dc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:41.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.239+0000 7f38f17fa640 1 -- 172.21.15.133:0/1196267920 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc017030 con 0x7f38f4110190 2024-09-11T23:45:41.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.240+0000 7f38f8828640 1 -- 172.21.15.133:0/1196267920 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f41134e0 con 0x7f38f4110190 2024-09-11T23:45:41.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.240+0000 7f38f8828640 1 -- 172.21.15.133:0/1196267920 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f4117500 con 0x7f38f4110190 2024-09-11T23:45:41.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.245+0000 7f38fbe63740 1 Processor -- start 2024-09-11T23:45:41.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.246+0000 7f38fbe63740 1 -- start start 2024-09-11T23:45:41.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.247+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 0x55f5755d9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.247+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755d97b0 0x55f5755d9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.247+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755e4550 0x55f5755e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.247+0000 7f38fbe63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f57553f2f0 con 0x55f5755e4550 2024-09-11T23:45:41.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.248+0000 7f38f3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 0x55f5755d9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.248+0000 7f38f3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 0x55f5755d9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40270/0 (socket says 172.21.15.133:40270) 2024-09-11T23:45:41.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.248+0000 7f38f3fff640 1 -- 172.21.15.133:0/2766188688 learned_addr learned my addr 172.21.15.133:0/2766188688 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:41.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.248+0000 7f38fbe63740 1 -- 172.21.15.133:0/2766188688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f5754e6cf0 con 0x55f5755d97b0 2024-09-11T23:45:41.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.248+0000 7f38fbe63740 1 -- 172.21.15.133:0/2766188688 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f57553b2b0 con 0x55f5755dad20 2024-09-11T23:45:41.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.252+0000 7f38ebfff640 1 --2- 172.21.15.133:0/2766188688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755e4550 0x55f5755e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.253+0000 7f38f37fe640 1 --2- 172.21.15.133:0/2766188688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755d97b0 0x55f5755d9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.253+0000 7f38f17fa640 1 -- 172.21.15.133:0/1196267920 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38dc004020 con 0x7f38f4110190 2024-09-11T23:45:41.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.253+0000 7f38f17fa640 1 -- 172.21.15.133:0/1196267920 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc003d80 con 0x7f38f4110190 2024-09-11T23:45:41.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.253+0000 7f38f17fa640 1 -- 172.21.15.133:0/1196267920 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f38dc01a070 con 0x7f38f4110190 2024-09-11T23:45:41.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.253+0000 7f38f3fff640 1 -- 172.21.15.133:0/2766188688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755d97b0 msgr2=0x55f5755d9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:41.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.253+0000 7f38f3fff640 1 --2- 172.21.15.133:0/2766188688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755d97b0 0x55f5755d9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.253+0000 7f38f3fff640 1 -- 172.21.15.133:0/2766188688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755e4550 msgr2=0x55f5755e6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:41.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.253+0000 7f38f3fff640 1 --2- 172.21.15.133:0/2766188688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755e4550 0x55f5755e6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.253+0000 7f38f3fff640 1 -- 172.21.15.133:0/2766188688 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5754cb3a0 con 0x55f5755dad20 2024-09-11T23:45:41.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.256+0000 7f38f17fa640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38e00a2ec0 0x7f38e00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.256+0000 7f38f17fa640 1 -- 172.21.15.133:0/1196267920 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f38dc0d2120 con 0x7f38f4110190 2024-09-11T23:45:41.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.256+0000 7f38f3fff640 1 --2- 172.21.15.133:0/2766188688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 0x55f5755d9270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009bf0 tx=0x7f38ec02f4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b313c950e3c1f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.257+0000 7f38f37fe640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38e00a2ec0 0x7f38e00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.258+0000 7f38e9ffb640 1 -- 172.21.15.133:0/2766188688 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec031ba0 con 0x55f5755dad20 2024-09-11T23:45:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.258+0000 7f38e9ffb640 1 -- 172.21.15.133:0/2766188688 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38ec03f070 con 0x55f5755dad20 2024-09-11T23:45:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.258+0000 7f38e9ffb640 1 -- 172.21.15.133:0/2766188688 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0042a0 con 0x55f5755dad20 2024-09-11T23:45:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.259+0000 7f38fbe63740 1 -- 172.21.15.133:0/2766188688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 msgr2=0x55f5755d9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.259+0000 7f38fbe63740 1 --2- 172.21.15.133:0/2766188688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 0x55f5755d9270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009bf0 tx=0x7f38ec02f4d0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.260+0000 7f38fbe63740 1 -- 172.21.15.133:0/2766188688 shutdown_connections 2024-09-11T23:45:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.260+0000 7f38fbe63740 1 --2- 172.21.15.133:0/2766188688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755e4550 0x55f5755e6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.260+0000 7f38fbe63740 1 --2- 172.21.15.133:0/2766188688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755d97b0 0x55f5755d9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.260+0000 7f38fbe63740 1 --2- 172.21.15.133:0/2766188688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 0x55f5755d9270 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.260+0000 7f38fbe63740 1 -- 172.21.15.133:0/2766188688 >> 172.21.15.133:0/2766188688 conn(0x55f5755a2310 msgr2=0x55f5755d5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:41.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.260+0000 7f38fbe63740 1 -- 172.21.15.133:0/2766188688 shutdown_connections 2024-09-11T23:45:41.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.260+0000 7f38fbe63740 1 -- 172.21.15.133:0/2766188688 wait complete. 2024-09-11T23:45:41.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.261+0000 7f38fbe63740 1 -- start start 2024-09-11T23:45:41.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.262+0000 7f38fbe63740 1 Processor -- start 2024-09-11T23:45:41.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.262+0000 7f38f37fe640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38e00a2ec0 0x7f38e00a5380 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f38e4005e10 tx=0x7f38e4005da0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:41.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.263+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 0x55f5755e3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.263+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755dad20 0x55f5755e1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.263+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.263+0000 7f38fbe63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f5755e7240 con 0x55f5755d97b0 2024-09-11T23:45:41.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.263+0000 7f38fbe63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f5754fa060 con 0x55f5755dad20 2024-09-11T23:45:41.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.263+0000 7f38fbe63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f5755e73c0 con 0x55f5755e4550 2024-09-11T23:45:41.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.263+0000 7f38fbe63740 10 client.0 _ll_register_callbacks cb 0x55f57524d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.263+0000 7f38f3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.263+0000 7f38f3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40276/0 (socket says 172.21.15.133:40276) 2024-09-11T23:45:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.263+0000 7f38f3fff640 1 -- 172.21.15.133:0/1104700843 learned_addr learned my addr 172.21.15.133:0/1104700843 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.263+0000 7f38f37fe640 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 0x55f5755e3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.264+0000 7f38f3fff640 1 -- 172.21.15.133:0/1104700843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755dad20 msgr2=0x55f5755e1190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:45:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.264+0000 7f38f3fff640 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755dad20 0x55f5755e1190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.264+0000 7f38f3fff640 1 -- 172.21.15.133:0/1104700843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 msgr2=0x55f5755e3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.264+0000 7f38f3fff640 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 0x55f5755e3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.264+0000 7f38f3fff640 1 -- 172.21.15.133:0/1104700843 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5752b03c0 con 0x55f5755e4550 2024-09-11T23:45:41.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.264+0000 7f38f3fff640 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e1710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f38ec031dd0 tx=0x7f38ec031e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b783ddd086a29933 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:41.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.264+0000 7f38d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:45:41.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.264+0000 7f38d2ffd640 1 -- 172.21.15.133:0/1104700843 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec003d80 con 0x55f5755e4550 2024-09-11T23:45:41.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.265+0000 7f38d2ffd640 1 -- 172.21.15.133:0/1104700843 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f38ec004040 con 0x55f5755e4550 2024-09-11T23:45:41.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.265+0000 7f38fbe63740 1 -- 172.21.15.133:0/1104700843 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38ec03f070 con 0x55f5755e4550 2024-09-11T23:45:41.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.266+0000 7f38d2ffd640 1 -- 172.21.15.133:0/1104700843 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f38ec004280 con 0x55f5755e4550 2024-09-11T23:45:41.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.266+0000 7f38fbe63740 10 client.27073 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:41.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.266+0000 7f38fbe63740 1 -- 172.21.15.133:0/1104700843 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f5755e7240 con 0x55f5755e4550 2024-09-11T23:45:41.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.266+0000 7f38d2ffd640 1 -- 172.21.15.133:0/1104700843 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38ec03f070 con 0x55f5755e4550 2024-09-11T23:45:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.267+0000 7f38d2ffd640 1 -- 172.21.15.133:0/1104700843 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f38ec0305f0 con 0x55f5755e4550 2024-09-11T23:45:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.268+0000 7f38fbe63740 10 client.27073 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.268+0000 7f38fbe63740 10 client.27073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.268+0000 7f38fbe63740 20 client.27073 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.268+0000 7f38fbe63740 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f5755f3ed0 0x55f5755f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.268+0000 7f38fbe63740 4 client.27073 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.268+0000 7f38fbe63740 1 -- 172.21.15.133:0/1104700843 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f57554c860 con 0x55f5755f3ed0 2024-09-11T23:45:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.269+0000 7f38f37fe640 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f5755f3ed0 0x55f5755f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.270+0000 7f38d2ffd640 10 client.27073 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:41.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.270+0000 7f38f37fe640 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f5755f3ed0 0x55f5755f62c0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33a4f475d6c7c462 server_cookie=559faf7ef6f81623 in_seq=0 out_seq=0 2024-09-11T23:45:41.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.275+0000 7f38d2ffd640 1 -- 172.21.15.133:0/1104700843 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f57554c860 con 0x55f5755f3ed0 2024-09-11T23:45:41.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.275+0000 7f38d2ffd640 10 client.27073 handle_command_reply: tid=0 2024-09-11T23:45:41.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.275+0000 7f38fbe63740 1 client.27073 shutdown 2024-09-11T23:45:41.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.276+0000 7f38fbe63740 1 -- 172.21.15.133:0/1104700843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 msgr2=0x55f5755e1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:41.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.276+0000 7f38fbe63740 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e1710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f38ec031dd0 tx=0x7f38ec031e00 comp rx=0 tx=0).stop 2024-09-11T23:45:41.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.276+0000 7f38fbe63740 1 -- 172.21.15.133:0/1104700843 shutdown_connections 2024-09-11T23:45:41.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.276+0000 7f38fbe63740 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f5755f3ed0 0x55f5755f62c0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.276+0000 7f38fbe63740 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e1710 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.276+0000 7f38fbe63740 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755dad20 0x55f5755e1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.276+0000 7f38fbe63740 1 --2- 172.21.15.133:0/1104700843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 0x55f5755e3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.276+0000 7f38fbe63740 1 -- 172.21.15.133:0/1104700843 >> 172.21.15.133:0/1104700843 conn(0x55f5755a2310 msgr2=0x55f5755dbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:41.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.276+0000 7f38d2ffd640 0 client.27073 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:41.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.276+0000 7f38fbe63740 1 -- 172.21.15.133:0/1104700843 shutdown_connections 2024-09-11T23:45:41.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.277+0000 7f38fbe63740 1 -- 172.21.15.133:0/1104700843 wait complete. 2024-09-11T23:45:41.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.277+0000 7f38fbe63740 20 client.27073 trim_cache size 0 max 16384 2024-09-11T23:45:41.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.304+0000 7f38fbe63740 1 Processor -- start 2024-09-11T23:45:41.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.304+0000 7f38fbe63740 1 -- start start 2024-09-11T23:45:41.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.304+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 0x55f5755e3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.304+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 0x55f5755dc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.304+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755e4550 0x55f5755dc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.304+0000 7f38fbe63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38d400e6b0 con 0x55f5755d97b0 2024-09-11T23:45:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.304+0000 7f38fbe63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38d40101f0 con 0x55f5755e4550 2024-09-11T23:45:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.304+0000 7f38fbe63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f38d400ed80 con 0x55f5755dad20 2024-09-11T23:45:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.305+0000 7f38f37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 0x55f5755e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.305+0000 7f38f37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 0x55f5755e3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48224/0 (socket says 172.21.15.133:48224) 2024-09-11T23:45:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.305+0000 7f38f37fe640 1 -- 172.21.15.133:0/150452683 learned_addr learned my addr 172.21.15.133:0/150452683 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.305+0000 7f38ebfff640 1 --2- 172.21.15.133:0/150452683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 0x55f5755dc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.305+0000 7f38f3fff640 1 --2- 172.21.15.133:0/150452683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755e4550 0x55f5755dc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.305+0000 7f38f37fe640 1 -- 172.21.15.133:0/150452683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 msgr2=0x55f5755dc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.305+0000 7f38f37fe640 1 --2- 172.21.15.133:0/150452683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 0x55f5755dc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.305+0000 7f38f37fe640 1 -- 172.21.15.133:0/150452683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755e4550 msgr2=0x55f5755dc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.305+0000 7f38f37fe640 1 --2- 172.21.15.133:0/150452683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755e4550 0x55f5755dc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.305+0000 7f38f37fe640 1 -- 172.21.15.133:0/150452683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5755e0160 con 0x55f5755d97b0 2024-09-11T23:45:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.306+0000 7f38f37fe640 1 --2- 172.21.15.133:0/150452683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 0x55f5755e3100 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f38e4010eb0 tx=0x7f38e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db252c6c5e7dce24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:41.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.306+0000 7f38e97fa640 1 -- 172.21.15.133:0/150452683 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40285a0 con 0x55f5755d97b0 2024-09-11T23:45:41.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.306+0000 7f38e97fa640 1 -- 172.21.15.133:0/150452683 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e402e070 con 0x55f5755d97b0 2024-09-11T23:45:41.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.306+0000 7f38e97fa640 1 -- 172.21.15.133:0/150452683 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e401fde0 con 0x55f5755d97b0 2024-09-11T23:45:41.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.307+0000 7f38fbe63740 1 -- 172.21.15.133:0/150452683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 msgr2=0x55f5755e3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:41.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.307+0000 7f38fbe63740 1 --2- 172.21.15.133:0/150452683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 0x55f5755e3100 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f38e4010eb0 tx=0x7f38e4013000 comp rx=0 tx=0).stop 2024-09-11T23:45:41.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.307+0000 7f38fbe63740 1 -- 172.21.15.133:0/150452683 shutdown_connections 2024-09-11T23:45:41.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.307+0000 7f38fbe63740 1 --2- 172.21.15.133:0/150452683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755e4550 0x55f5755dc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.307+0000 7f38fbe63740 1 --2- 172.21.15.133:0/150452683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755dad20 0x55f5755dc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.307+0000 7f38fbe63740 1 --2- 172.21.15.133:0/150452683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755d97b0 0x55f5755e3100 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.307+0000 7f38fbe63740 1 -- 172.21.15.133:0/150452683 >> 172.21.15.133:0/150452683 conn(0x55f5755a2310 msgr2=0x55f5755e29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:41.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.307+0000 7f38fbe63740 1 -- 172.21.15.133:0/150452683 shutdown_connections 2024-09-11T23:45:41.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.307+0000 7f38fbe63740 1 -- 172.21.15.133:0/150452683 wait complete. 2024-09-11T23:45:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.308+0000 7f38fbe63740 1 -- start start 2024-09-11T23:45:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.308+0000 7f38fbe63740 1 Processor -- start 2024-09-11T23:45:41.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.308+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755d97b0 0x55f5755ecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.308+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755dad20 0x55f5755e8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.308+0000 7f38fbe63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.308+0000 7f38fbe63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38d400fb20 con 0x55f5755dad20 2024-09-11T23:45:41.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.308+0000 7f38fbe63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38d400f450 con 0x55f5755d97b0 2024-09-11T23:45:41.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.308+0000 7f38fbe63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f38d400c6b0 con 0x55f5755e4550 2024-09-11T23:45:41.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.309+0000 7f38fbe63740 10 client.0 _ll_register_callbacks cb 0x7f38ec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:41.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.309+0000 7f38ebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755dad20 0x55f5755e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.309+0000 7f38f3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.309+0000 7f38f3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:40294/0 (socket says 172.21.15.133:40294) 2024-09-11T23:45:41.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.309+0000 7f38f3fff640 1 -- 172.21.15.133:0/153652709 learned_addr learned my addr 172.21.15.133:0/153652709 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:41.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.309+0000 7f38f37fe640 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755d97b0 0x55f5755ecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.310+0000 7f38f3fff640 1 -- 172.21.15.133:0/153652709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755d97b0 msgr2=0x55f5755ecc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:41.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.310+0000 7f38f3fff640 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755d97b0 0x55f5755ecc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.310+0000 7f38f3fff640 1 -- 172.21.15.133:0/153652709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755dad20 msgr2=0x55f5755e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:41.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.310+0000 7f38f3fff640 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755dad20 0x55f5755e8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.310+0000 7f38f3fff640 1 -- 172.21.15.133:0/153652709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5755e8b00 con 0x55f5755e4550 2024-09-11T23:45:41.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.310+0000 7f38f3fff640 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e8540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009ca0 tx=0x7f38ec02f480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=464e0a9ec1ff7c07 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:41.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.310+0000 7f38d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:45:41.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.310+0000 7f38d2ffd640 1 -- 172.21.15.133:0/153652709 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec05e060 con 0x55f5755e4550 2024-09-11T23:45:41.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.311+0000 7f38d2ffd640 1 -- 172.21.15.133:0/153652709 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f38ec030be0 con 0x55f5755e4550 2024-09-11T23:45:41.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.311+0000 7f38fbe63740 1 -- 172.21.15.133:0/153652709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38e402e070 con 0x55f5755e4550 2024-09-11T23:45:41.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.311+0000 7f38d2ffd640 1 -- 172.21.15.133:0/153652709 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f38ec030e20 con 0x55f5755e4550 2024-09-11T23:45:41.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.311+0000 7f38fbe63740 10 client.27079 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:41.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.311+0000 7f38fbe63740 1 -- 172.21.15.133:0/153652709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f57565e690 con 0x55f5755e4550 2024-09-11T23:45:41.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.311+0000 7f38d2ffd640 1 -- 172.21.15.133:0/153652709 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e402e070 con 0x55f5755e4550 2024-09-11T23:45:41.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.313+0000 7f38d2ffd640 1 -- 172.21.15.133:0/153652709 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f38ec02f640 con 0x55f5755e4550 2024-09-11T23:45:41.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.313+0000 7f38fbe63740 10 client.27079 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:41.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.313+0000 7f38fbe63740 10 client.27079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:41.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.313+0000 7f38fbe63740 20 client.27079 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:41.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.313+0000 7f38fbe63740 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f57565ed00 0x55f575661120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:41.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.313+0000 7f38fbe63740 4 client.27079 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:45:41.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.313+0000 7f38fbe63740 1 -- 172.21.15.133:0/153652709 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f575546d70 con 0x55f57565ed00 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.313+0000 7f38f37fe640 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f57565ed00 0x55f575661120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.313+0000 7f38d2ffd640 10 client.27079 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.313+0000 7f38f37fe640 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f57565ed00 0x55f575661120 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74beb193d3ffd0c7 server_cookie=b43011e1af0e55b3 in_seq=0 out_seq=0 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38d2ffd640 1 -- 172.21.15.133:0/153652709 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f575546d70 con 0x55f57565ed00 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38d2ffd640 10 client.27079 handle_command_reply: tid=0 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38fbe63740 1 client.27079 shutdown 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38fbe63740 1 -- 172.21.15.133:0/153652709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 msgr2=0x55f5755e8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38fbe63740 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e8540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009ca0 tx=0x7f38ec02f480 comp rx=0 tx=0).stop 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38fbe63740 1 -- 172.21.15.133:0/153652709 shutdown_connections 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38fbe63740 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55f57565ed00 0x55f575661120 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38d2ffd640 0 client.27079 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38fbe63740 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f5755e4550 0x55f5755e8540 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38fbe63740 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5755dad20 0x55f5755e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38fbe63740 1 --2- 172.21.15.133:0/153652709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5755d97b0 0x55f5755ecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38fbe63740 1 -- 172.21.15.133:0/153652709 >> 172.21.15.133:0/153652709 conn(0x55f5755a2310 msgr2=0x55f5755e3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:41.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38fbe63740 1 -- 172.21.15.133:0/153652709 shutdown_connections 2024-09-11T23:45:41.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.314+0000 7f38fbe63740 1 -- 172.21.15.133:0/153652709 wait complete. 2024-09-11T23:45:41.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.315+0000 7f38fbe63740 20 client.27079 trim_cache size 0 max 16384 2024-09-11T23:45:41.316 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:45:41.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 -- 172.21.15.133:0/1196267920 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38e00a2ec0 msgr2=0x7f38e00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:41.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38e00a2ec0 0x7f38e00a5380 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f38e4005e10 tx=0x7f38e4005da0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 -- 172.21.15.133:0/1196267920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38f4110190 msgr2=0x7f38f4119c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:41.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38f4110190 0x7f38f4119c00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d950 tx=0x7f38dc00de10 comp rx=0 tx=0).stop 2024-09-11T23:45:41.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 -- 172.21.15.133:0/1196267920 shutdown_connections 2024-09-11T23:45:41.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f38e00a2ec0 0x7f38e00a5380 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38f411b140 0x7f38f4112930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38f4110190 0x7f38f4119c00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 --2- 172.21.15.133:0/1196267920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38f406ea30 0x7f38f41196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:41.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 -- 172.21.15.133:0/1196267920 >> 172.21.15.133:0/1196267920 conn(0x7f38f4073180 msgr2=0x7f38f406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:41.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 -- 172.21.15.133:0/1196267920 shutdown_connections 2024-09-11T23:45:41.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:41.316+0000 7f38f8828640 1 -- 172.21.15.133:0/1196267920 wait complete. 2024-09-11T23:45:43.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:42 smithi178 ceph-mon[35882]: pgmap v277: 97 pgs: 97 active+clean; 64 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 242 MiB/s wr, 1.65k op/s 2024-09-11T23:45:43.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:42 smithi138 ceph-mon[35142]: pgmap v277: 97 pgs: 97 active+clean; 64 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 242 MiB/s wr, 1.65k op/s 2024-09-11T23:45:43.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:42 smithi133 ceph-mon[31430]: pgmap v277: 97 pgs: 97 active+clean; 64 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 242 MiB/s wr, 1.65k op/s 2024-09-11T23:45:43.553 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:45:43.554 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:45:43.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:43 smithi133 ceph-mon[31430]: pgmap v278: 97 pgs: 97 active+clean; 66 GiB data, 208 GiB used, 865 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 424 MiB/s wr, 2.47k op/s 2024-09-11T23:45:44.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:43 smithi138 ceph-mon[35142]: pgmap v278: 97 pgs: 97 active+clean; 66 GiB data, 208 GiB used, 865 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 424 MiB/s wr, 2.47k op/s 2024-09-11T23:45:44.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:43 smithi178 ceph-mon[35882]: pgmap v278: 97 pgs: 97 active+clean; 66 GiB data, 208 GiB used, 865 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 424 MiB/s wr, 2.47k op/s 2024-09-11T23:45:44.357 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:45.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.059+0000 7f78df556640 1 -- 172.21.15.133:0/1719933750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d8102780 msgr2=0x7f78d81067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:45.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.059+0000 7f78df556640 1 --2- 172.21.15.133:0/1719933750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d8102780 0x7f78d81067d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009900 tx=0x7f78c802a410 comp rx=0 tx=0).stop 2024-09-11T23:45:45.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.060+0000 7f78df556640 1 -- 172.21.15.133:0/1719933750 shutdown_connections 2024-09-11T23:45:45.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.060+0000 7f78df556640 1 --2- 172.21.15.133:0/1719933750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d8106f00 0x7f78d810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.060+0000 7f78df556640 1 --2- 172.21.15.133:0/1719933750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d8102780 0x7f78d81067d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.060+0000 7f78df556640 1 --2- 172.21.15.133:0/1719933750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d8101dd0 0x7f78d81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.060+0000 7f78df556640 1 -- 172.21.15.133:0/1719933750 >> 172.21.15.133:0/1719933750 conn(0x7f78d80fd870 msgr2=0x7f78d80ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:45.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.061+0000 7f78df556640 1 -- 172.21.15.133:0/1719933750 shutdown_connections 2024-09-11T23:45:45.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.062+0000 7f78df556640 1 -- 172.21.15.133:0/1719933750 wait complete. 2024-09-11T23:45:45.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.063+0000 7f78df556640 1 Processor -- start 2024-09-11T23:45:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.063+0000 7f78df556640 1 -- start start 2024-09-11T23:45:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.063+0000 7f78df556640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d8101dd0 0x7f78d810db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.063+0000 7f78df556640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d8102780 0x7f78d810e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.063+0000 7f78df556640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d8106f00 0x7f78d81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.063+0000 7f78df556640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8111fd0 con 0x7f78d8106f00 2024-09-11T23:45:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.063+0000 7f78df556640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8111e50 con 0x7f78d8101dd0 2024-09-11T23:45:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.063+0000 7f78df556640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8112150 con 0x7f78d8102780 2024-09-11T23:45:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.064+0000 7f78dcaf1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d8102780 0x7f78d810e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.064+0000 7f78ddaf3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d8106f00 0x7f78d81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.064+0000 7f78ddaf3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d8106f00 0x7f78d81989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48970/0 (socket says 172.21.15.133:48970) 2024-09-11T23:45:45.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.064+0000 7f78ddaf3640 1 -- 172.21.15.133:0/4270876605 learned_addr learned my addr 172.21.15.133:0/4270876605 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:45.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.064+0000 7f78dd2f2640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d8101dd0 0x7f78d810db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.065+0000 7f78ddaf3640 1 -- 172.21.15.133:0/4270876605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d8102780 msgr2=0x7f78d810e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:45.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.065+0000 7f78ddaf3640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d8102780 0x7f78d810e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.065+0000 7f78ddaf3640 1 -- 172.21.15.133:0/4270876605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d8101dd0 msgr2=0x7f78d810db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:45.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.065+0000 7f78ddaf3640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d8101dd0 0x7f78d810db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.065+0000 7f78ddaf3640 1 -- 172.21.15.133:0/4270876605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d81990f0 con 0x7f78d8106f00 2024-09-11T23:45:45.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.065+0000 7f78ddaf3640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d8106f00 0x7f78d81989f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f78d000b970 tx=0x7f78d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:45.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.066+0000 7f78ce7fc640 1 -- 172.21.15.133:0/4270876605 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0019070 con 0x7f78d8106f00 2024-09-11T23:45:45.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.066+0000 7f78ce7fc640 1 -- 172.21.15.133:0/4270876605 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d000cb40 con 0x7f78d8106f00 2024-09-11T23:45:45.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.066+0000 7f78df556640 1 -- 172.21.15.133:0/4270876605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d8199380 con 0x7f78d8106f00 2024-09-11T23:45:45.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.068+0000 7f78ce7fc640 1 -- 172.21.15.133:0/4270876605 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0007650 con 0x7f78d8106f00 2024-09-11T23:45:45.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.068+0000 7f78df556640 1 -- 172.21.15.133:0/4270876605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d819bac0 con 0x7f78d8106f00 2024-09-11T23:45:45.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.071+0000 7f78ce7fc640 1 -- 172.21.15.133:0/4270876605 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f78d001c070 con 0x7f78d8106f00 2024-09-11T23:45:45.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.072+0000 7f78e192f740 1 Processor -- start 2024-09-11T23:45:45.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.072+0000 7f78e192f740 1 -- start start 2024-09-11T23:45:45.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78ce7fc640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f78c40a2c50 0x7f78c40a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78e192f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf1b60 0x562f3eaf1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78e192f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eafc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78e192f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eaff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78e192f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562f3ea57280 con 0x562f3eaf2480 2024-09-11T23:45:45.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78e192f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f3e9feb10 con 0x562f3eaf1b60 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78e192f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562f3ea53240 con 0x562f3eafcd40 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78dcaf1640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f78c40a2c50 0x7f78c40a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78ddaf3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eaff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78ddaf3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eaff130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33670/0 (socket says 172.21.15.133:33670) 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78ddaf3640 1 -- 172.21.15.133:0/2223599497 learned_addr learned my addr 172.21.15.133:0/2223599497 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78dcaf1640 1 --2- 172.21.15.133:0/2223599497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eafc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78dd2f2640 1 --2- 172.21.15.133:0/2223599497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf1b60 0x562f3eaf1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78ce7fc640 1 -- 172.21.15.133:0/4270876605 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f78d00ce990 con 0x7f78d8106f00 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78ddaf3640 1 -- 172.21.15.133:0/2223599497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf1b60 msgr2=0x562f3eaf1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78ddaf3640 1 --2- 172.21.15.133:0/2223599497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf1b60 0x562f3eaf1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78ddaf3640 1 -- 172.21.15.133:0/2223599497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 msgr2=0x562f3eafc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78ddaf3640 1 --2- 172.21.15.133:0/2223599497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eafc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.073+0000 7f78ddaf3640 1 -- 172.21.15.133:0/2223599497 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3ea0b1f0 con 0x562f3eafcd40 2024-09-11T23:45:45.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.074+0000 7f78dd2f2640 1 --2- 172.21.15.133:0/2223599497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf1b60 0x562f3eaf1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:45.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.074+0000 7f78ddaf3640 1 --2- 172.21.15.133:0/2223599497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eaff130 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f78d0016000 tx=0x7f78d006e070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2c896cdf32a1830 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:45.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.074+0000 7f78dcaf1640 1 --2- 172.21.15.133:0/2223599497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eafc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:45:45.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.074+0000 7f78c37fe640 1 -- 172.21.15.133:0/2223599497 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d009b700 con 0x562f3eafcd40 2024-09-11T23:45:45.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.074+0000 7f78c37fe640 1 -- 172.21.15.133:0/2223599497 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d00bb0c0 con 0x562f3eafcd40 2024-09-11T23:45:45.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.074+0000 7f78c37fe640 1 -- 172.21.15.133:0/2223599497 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d00cc3a0 con 0x562f3eafcd40 2024-09-11T23:45:45.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.076+0000 7f78dcaf1640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f78c40a2c50 0x7f78c40a5110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f78d810eed0 tx=0x7f78c8005a20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:45.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.076+0000 7f78e192f740 1 -- 172.21.15.133:0/2223599497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 msgr2=0x562f3eaff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:45.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.076+0000 7f78e192f740 1 --2- 172.21.15.133:0/2223599497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eaff130 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f78d0016000 tx=0x7f78d006e070 comp rx=0 tx=0).stop 2024-09-11T23:45:45.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.077+0000 7f78e192f740 1 -- 172.21.15.133:0/2223599497 shutdown_connections 2024-09-11T23:45:45.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.077+0000 7f78e192f740 1 --2- 172.21.15.133:0/2223599497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eaff130 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.077+0000 7f78e192f740 1 --2- 172.21.15.133:0/2223599497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eafc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.077+0000 7f78e192f740 1 --2- 172.21.15.133:0/2223599497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf1b60 0x562f3eaf1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.078+0000 7f78e192f740 1 -- 172.21.15.133:0/2223599497 >> 172.21.15.133:0/2223599497 conn(0x562f3eaee0f0 msgr2=0x562f3eaee4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:45.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.079+0000 7f78e192f740 1 -- 172.21.15.133:0/2223599497 shutdown_connections 2024-09-11T23:45:45.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.079+0000 7f78e192f740 1 -- 172.21.15.133:0/2223599497 wait complete. 2024-09-11T23:45:45.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.080+0000 7f78e192f740 1 -- start start 2024-09-11T23:45:45.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.080+0000 7f78e192f740 1 Processor -- start 2024-09-11T23:45:45.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.081+0000 7f78e192f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf1b60 0x562f3eaf9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.082+0000 7f78e192f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eafa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.082+0000 7f78e192f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eafa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.083+0000 7f78dcaf1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eafa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.083+0000 7f78dcaf1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eafa220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48996/0 (socket says 172.21.15.133:48996) 2024-09-11T23:45:45.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.083+0000 7f78dcaf1640 1 -- 172.21.15.133:0/207410109 learned_addr learned my addr 172.21.15.133:0/207410109 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:45.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.083+0000 7f78e192f740 1 -- 172.21.15.133:0/207410109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562f3eaff910 con 0x562f3eaf2480 2024-09-11T23:45:45.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.083+0000 7f78e192f740 1 -- 172.21.15.133:0/207410109 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f3ea11f10 con 0x562f3eaf1b60 2024-09-11T23:45:45.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.083+0000 7f78e192f740 1 -- 172.21.15.133:0/207410109 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562f3eaffa90 con 0x562f3eafcd40 2024-09-11T23:45:45.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.084+0000 7f78e192f740 10 client.0 _ll_register_callbacks cb 0x562f3e9daa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:45.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.084+0000 7f78dcaf1640 1 -- 172.21.15.133:0/207410109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 msgr2=0x562f3eafa760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:45:45.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.084+0000 7f78dcaf1640 1 --2- 172.21.15.133:0/207410109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eafa760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.084+0000 7f78dcaf1640 1 -- 172.21.15.133:0/207410109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf1b60 msgr2=0x562f3eaf9ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:45:45.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.084+0000 7f78dcaf1640 1 --2- 172.21.15.133:0/207410109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf1b60 0x562f3eaf9ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.084+0000 7f78dcaf1640 1 -- 172.21.15.133:0/207410109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3e7c86a0 con 0x562f3eaf2480 2024-09-11T23:45:45.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.084+0000 7f78dcaf1640 1 --2- 172.21.15.133:0/207410109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eafa220 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f78c8033000 tx=0x7f78c802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932aae8161a6a26 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:45.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.085+0000 7f78e192f740 1 -- 172.21.15.133:0/207410109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d00bb0c0 con 0x562f3eaf2480 2024-09-11T23:45:45.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.085+0000 7f78c0ff9640 10 client.17772 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:45.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.085+0000 7f78c0ff9640 1 -- 172.21.15.133:0/207410109 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8035040 con 0x562f3eaf2480 2024-09-11T23:45:45.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.085+0000 7f78c0ff9640 1 -- 172.21.15.133:0/207410109 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f78c803da90 con 0x562f3eaf2480 2024-09-11T23:45:45.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.085+0000 7f78c0ff9640 1 -- 172.21.15.133:0/207410109 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f78c803dc90 con 0x562f3eaf2480 2024-09-11T23:45:45.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.086+0000 7f78e192f740 10 client.17772 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:45.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.086+0000 7f78e192f740 1 -- 172.21.15.133:0/207410109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({fsmap=30}) -- 0x562f3eb0b5d0 con 0x562f3eaf2480 2024-09-11T23:45:45.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.087+0000 7f78c0ff9640 1 -- 172.21.15.133:0/207410109 <== mon.0 v2:172.21.15.133:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f78c803a040 con 0x562f3eaf2480 2024-09-11T23:45:45.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.087+0000 7f78e192f740 10 client.17772 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:45.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.087+0000 7f78e192f740 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:45.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.087+0000 7f78e192f740 20 client.17772 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:45.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.087+0000 7f78e192f740 1 --2- 172.21.15.133:0/207410109 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562f3eb0c980 0x562f3eb0ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.087+0000 7f78e192f740 4 client.17772 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:45.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.087+0000 7f78e192f740 1 -- 172.21.15.133:0/207410109 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f3ea647f0 con 0x562f3eb0c980 2024-09-11T23:45:45.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.088+0000 7f78dd2f2640 1 --2- 172.21.15.133:0/207410109 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562f3eb0c980 0x562f3eb0ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.096+0000 7f78dd2f2640 1 --2- 172.21.15.133:0/207410109 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562f3eb0c980 0x562f3eb0ed70 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f0f4415aa683c1 server_cookie=ebfa7afae359c1bf in_seq=0 out_seq=0 2024-09-11T23:45:45.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.097+0000 7f78c0ff9640 10 client.17772 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:45.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.097+0000 7f78c0ff9640 1 -- 172.21.15.133:0/207410109 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f3ea647f0 con 0x562f3eb0c980 2024-09-11T23:45:45.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.098+0000 7f78c0ff9640 10 client.17772 handle_command_reply: tid=0 2024-09-11T23:45:45.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.098+0000 7f78e192f740 1 client.17772 shutdown 2024-09-11T23:45:45.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.099+0000 7f78e192f740 1 -- 172.21.15.133:0/207410109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 msgr2=0x562f3eafa220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:45.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.099+0000 7f78e192f740 1 --2- 172.21.15.133:0/207410109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eafa220 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f78c8033000 tx=0x7f78c802a410 comp rx=0 tx=0).stop 2024-09-11T23:45:45.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.099+0000 7f78e192f740 1 -- 172.21.15.133:0/207410109 shutdown_connections 2024-09-11T23:45:45.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.099+0000 7f78e192f740 1 --2- 172.21.15.133:0/207410109 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562f3eb0c980 0x562f3eb0ed70 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.099+0000 7f78e192f740 1 --2- 172.21.15.133:0/207410109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eafa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.099+0000 7f78e192f740 1 --2- 172.21.15.133:0/207410109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eafa220 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.099+0000 7f78e192f740 1 --2- 172.21.15.133:0/207410109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf1b60 0x562f3eaf9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.099+0000 7f78e192f740 1 -- 172.21.15.133:0/207410109 >> 172.21.15.133:0/207410109 conn(0x562f3eaee0f0 msgr2=0x562f3eaf3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:45.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.099+0000 7f78c0ff9640 0 client.17772 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.101+0000 7f78e192f740 1 -- 172.21.15.133:0/207410109 shutdown_connections 2024-09-11T23:45:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.101+0000 7f78e192f740 1 -- 172.21.15.133:0/207410109 wait complete. 2024-09-11T23:45:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.102+0000 7f78e192f740 20 client.17772 trim_cache size 0 max 16384 2024-09-11T23:45:45.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.136+0000 7f78e192f740 1 Processor -- start 2024-09-11T23:45:45.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.136+0000 7f78e192f740 1 -- start start 2024-09-11T23:45:45.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.137+0000 7f78e192f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf1b60 0x562f3eaf3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.137+0000 7f78e192f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf2480 0x562f3eb14580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.138+0000 7f78dd2f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf1b60 0x562f3eaf3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.138+0000 7f78dd2f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf1b60 0x562f3eaf3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49010/0 (socket says 172.21.15.133:49010) 2024-09-11T23:45:45.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.138+0000 7f78dd2f2640 1 -- 172.21.15.133:0/1988126229 learned_addr learned my addr 172.21.15.133:0/1988126229 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:45.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.138+0000 7f78e192f740 1 --2- 172.21.15.133:0/1988126229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eb14ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.138+0000 7f78e192f740 1 -- 172.21.15.133:0/1988126229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81ab460 con 0x562f3eaf1b60 2024-09-11T23:45:45.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.139+0000 7f78e192f740 1 -- 172.21.15.133:0/1988126229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81acfa0 con 0x562f3eaf2480 2024-09-11T23:45:45.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.139+0000 7f78e192f740 1 -- 172.21.15.133:0/1988126229 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81abb30 con 0x562f3eafcd40 2024-09-11T23:45:45.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.139+0000 7f78dd2f2640 1 -- 172.21.15.133:0/1988126229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 msgr2=0x562f3eb14ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:45:45.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.139+0000 7f78dd2f2640 1 --2- 172.21.15.133:0/1988126229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eb14ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.139+0000 7f78dd2f2640 1 -- 172.21.15.133:0/1988126229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf2480 msgr2=0x562f3eb14580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:45:45.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.139+0000 7f78dd2f2640 1 --2- 172.21.15.133:0/1988126229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf2480 0x562f3eb14580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.139+0000 7f78dd2f2640 1 -- 172.21.15.133:0/1988126229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3eaf8c90 con 0x562f3eaf1b60 2024-09-11T23:45:45.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.140+0000 7f78dd2f2640 1 --2- 172.21.15.133:0/1988126229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf1b60 0x562f3eaf3d10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f78d810ef90 tx=0x7f78d4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327c4734b05d29c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:45.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.142+0000 7f78c2ffd640 1 -- 172.21.15.133:0/1988126229 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d40193b0 con 0x562f3eaf1b60 2024-09-11T23:45:45.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.142+0000 7f78c2ffd640 1 -- 172.21.15.133:0/1988126229 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d4019a30 con 0x562f3eaf1b60 2024-09-11T23:45:45.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.143+0000 7f78c2ffd640 1 -- 172.21.15.133:0/1988126229 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d400fbf0 con 0x562f3eaf1b60 2024-09-11T23:45:45.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.144+0000 7f78e192f740 1 -- 172.21.15.133:0/1988126229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf1b60 msgr2=0x562f3eaf3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:45.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.144+0000 7f78e192f740 1 --2- 172.21.15.133:0/1988126229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf1b60 0x562f3eaf3d10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f78d810ef90 tx=0x7f78d4005dc0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.147+0000 7f78e192f740 1 -- 172.21.15.133:0/1988126229 shutdown_connections 2024-09-11T23:45:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.147+0000 7f78e192f740 1 --2- 172.21.15.133:0/1988126229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eafcd40 0x562f3eb14ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.147+0000 7f78e192f740 1 --2- 172.21.15.133:0/1988126229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eaf2480 0x562f3eb14580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.147+0000 7f78e192f740 1 --2- 172.21.15.133:0/1988126229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf1b60 0x562f3eaf3d10 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.147+0000 7f78e192f740 1 -- 172.21.15.133:0/1988126229 >> 172.21.15.133:0/1988126229 conn(0x562f3eaee0f0 msgr2=0x562f3eaf3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:45.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.150+0000 7f78e192f740 1 -- 172.21.15.133:0/1988126229 shutdown_connections 2024-09-11T23:45:45.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.155+0000 7f78e192f740 1 -- 172.21.15.133:0/1988126229 wait complete. 2024-09-11T23:45:45.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.155+0000 7f78e192f740 1 -- start start 2024-09-11T23:45:45.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.156+0000 7f78e192f740 1 Processor -- start 2024-09-11T23:45:45.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.157+0000 7f78e192f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eaf1b60 0x562f3eb04e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.157+0000 7f78e192f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eb05380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.157+0000 7f78e192f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eafcd40 0x562f3eb01210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.157+0000 7f78e192f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81ac8d0 con 0x562f3eaf2480 2024-09-11T23:45:45.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.157+0000 7f78e192f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81ac200 con 0x562f3eafcd40 2024-09-11T23:45:45.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.157+0000 7f78e192f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81a9460 con 0x562f3eaf1b60 2024-09-11T23:45:45.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.157+0000 7f78e192f740 10 client.0 _ll_register_callbacks cb 0x7f78c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:45.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.158+0000 7f78dcaf1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eb05380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.161+0000 7f78dcaf1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eb05380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49020/0 (socket says 172.21.15.133:49020) 2024-09-11T23:45:45.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.161+0000 7f78dcaf1640 1 -- 172.21.15.133:0/2779508452 learned_addr learned my addr 172.21.15.133:0/2779508452 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:45.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.163+0000 7f78ddaf3640 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eafcd40 0x562f3eb01210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.163+0000 7f78dd2f2640 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eaf1b60 0x562f3eb04e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.167+0000 7f78dcaf1640 1 -- 172.21.15.133:0/2779508452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eaf1b60 msgr2=0x562f3eb04e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.167+0000 7f78dcaf1640 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eaf1b60 0x562f3eb04e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.167+0000 7f78dcaf1640 1 -- 172.21.15.133:0/2779508452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eafcd40 msgr2=0x562f3eb01210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.167+0000 7f78dcaf1640 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eafcd40 0x562f3eb01210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.167+0000 7f78dcaf1640 1 -- 172.21.15.133:0/2779508452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3eb01a40 con 0x562f3eaf2480 2024-09-11T23:45:45.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.169+0000 7f78dcaf1640 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eb05380 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f78c800b110 tx=0x7f78c8043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8696161b34b48a66 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:45.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.170+0000 7f78c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:45:45.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.171+0000 7f78c0ff9640 1 -- 172.21.15.133:0/2779508452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8035050 con 0x562f3eaf2480 2024-09-11T23:45:45.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.171+0000 7f78c0ff9640 1 -- 172.21.15.133:0/2779508452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f78c8032250 con 0x562f3eaf2480 2024-09-11T23:45:45.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.173+0000 7f78e192f740 1 -- 172.21.15.133:0/2779508452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d4019a30 con 0x562f3eaf2480 2024-09-11T23:45:45.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.176+0000 7f78c0ff9640 1 -- 172.21.15.133:0/2779508452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f78c8032450 con 0x562f3eaf2480 2024-09-11T23:45:45.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.176+0000 7f78e192f740 10 client.17778 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:45.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.176+0000 7f78e192f740 1 -- 172.21.15.133:0/2779508452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f3eb780c0 con 0x562f3eaf2480 2024-09-11T23:45:45.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.179+0000 7f78c0ff9640 1 -- 172.21.15.133:0/2779508452 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d4019a30 con 0x562f3eaf2480 2024-09-11T23:45:45.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.180+0000 7f78c0ff9640 1 -- 172.21.15.133:0/2779508452 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f78c8031830 con 0x562f3eaf2480 2024-09-11T23:45:45.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.181+0000 7f78e192f740 10 client.17778 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:45.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.181+0000 7f78e192f740 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:45.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.181+0000 7f78e192f740 20 client.17778 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:45.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.181+0000 7f78e192f740 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562f3eb787d0 0x562f3eb7abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:45.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.181+0000 7f78e192f740 4 client.17778 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:45:45.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.181+0000 7f78e192f740 1 -- 172.21.15.133:0/2779508452 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562f3ea5ed00 con 0x562f3eb787d0 2024-09-11T23:45:45.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.186+0000 7f78dd2f2640 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562f3eb787d0 0x562f3eb7abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.194+0000 7f78dd2f2640 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562f3eb787d0 0x562f3eb7abc0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689475993318fcbe server_cookie=a1a0946de461a321 in_seq=0 out_seq=0 2024-09-11T23:45:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.195+0000 7f78c0ff9640 10 client.17778 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:45.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.199+0000 7f78c0ff9640 1 -- 172.21.15.133:0/2779508452 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562f3ea5ed00 con 0x562f3eb787d0 2024-09-11T23:45:45.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.199+0000 7f78c0ff9640 10 client.17778 handle_command_reply: tid=0 2024-09-11T23:45:45.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.199+0000 7f78e192f740 1 client.17778 shutdown 2024-09-11T23:45:45.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.200+0000 7f78e192f740 1 -- 172.21.15.133:0/2779508452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 msgr2=0x562f3eb05380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:45.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.200+0000 7f78e192f740 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eb05380 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f78c800b110 tx=0x7f78c8043000 comp rx=0 tx=0).stop 2024-09-11T23:45:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.211+0000 7f78e192f740 1 -- 172.21.15.133:0/2779508452 shutdown_connections 2024-09-11T23:45:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.211+0000 7f78e192f740 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562f3eb787d0 0x562f3eb7abc0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.211+0000 7f78e192f740 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f3eafcd40 0x562f3eb01210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.211+0000 7f78e192f740 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f3eaf2480 0x562f3eb05380 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.211+0000 7f78e192f740 1 --2- 172.21.15.133:0/2779508452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f3eaf1b60 0x562f3eb04e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.211+0000 7f78e192f740 1 -- 172.21.15.133:0/2779508452 >> 172.21.15.133:0/2779508452 conn(0x562f3eaee0f0 msgr2=0x562f3eaed270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.212+0000 7f78c0ff9640 0 client.17778 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.212+0000 7f78e192f740 1 -- 172.21.15.133:0/2779508452 shutdown_connections 2024-09-11T23:45:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.212+0000 7f78e192f740 1 -- 172.21.15.133:0/2779508452 wait complete. 2024-09-11T23:45:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.212+0000 7f78e192f740 20 client.17778 trim_cache size 0 max 16384 2024-09-11T23:45:45.214 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:45:45.214 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:45:45.214 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "76ff1c82-4bb6-4d19-a948-752500fcd905", 2024-09-11T23:45:45.214 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:45:45.214 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:45:45.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.220+0000 7f78df556640 1 -- 172.21.15.133:0/4270876605 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f78c40a2c50 msgr2=0x7f78c40a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:45.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.220+0000 7f78df556640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f78c40a2c50 0x7f78c40a5110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f78d810eed0 tx=0x7f78c8005a20 comp rx=0 tx=0).stop 2024-09-11T23:45:45.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.220+0000 7f78df556640 1 -- 172.21.15.133:0/4270876605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d8106f00 msgr2=0x7f78d81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:45.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.220+0000 7f78df556640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d8106f00 0x7f78d81989f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f78d000b970 tx=0x7f78d000be30 comp rx=0 tx=0).stop 2024-09-11T23:45:45.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.220+0000 7f78df556640 1 -- 172.21.15.133:0/4270876605 shutdown_connections 2024-09-11T23:45:45.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.220+0000 7f78df556640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f78c40a2c50 0x7f78c40a5110 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.220+0000 7f78df556640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d8106f00 0x7f78d81989f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.220+0000 7f78df556640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d8102780 0x7f78d810e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.220+0000 7f78df556640 1 --2- 172.21.15.133:0/4270876605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d8101dd0 0x7f78d810db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:45.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.220+0000 7f78df556640 1 -- 172.21.15.133:0/4270876605 >> 172.21.15.133:0/4270876605 conn(0x7f78d80fd870 msgr2=0x7f78d8074730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:45.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.225+0000 7f78df556640 1 -- 172.21.15.133:0/4270876605 shutdown_connections 2024-09-11T23:45:45.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:45.225+0000 7f78df556640 1 -- 172.21.15.133:0/4270876605 wait complete. 2024-09-11T23:45:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.180+0000 7fc247a23640 1 -- 172.21.15.133:0/2902373409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2401091f0 msgr2=0x7fc240110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.181+0000 7fc247a23640 1 --2- 172.21.15.133:0/2902373409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2401091f0 0x7fc240110250 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc238009be0 tx=0x7fc23802a6e0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.181+0000 7fc247a23640 1 -- 172.21.15.133:0/2902373409 shutdown_connections 2024-09-11T23:45:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.181+0000 7fc247a23640 1 --2- 172.21.15.133:0/2902373409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc24011b140 0x7fc24011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.181+0000 7fc247a23640 1 --2- 172.21.15.133:0/2902373409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2401095d0 0x7fc240109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.181+0000 7fc247a23640 1 --2- 172.21.15.133:0/2902373409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2401091f0 0x7fc240110250 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc238009be0 tx=0x7fc23802a6e0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.181+0000 7fc247a23640 1 -- 172.21.15.133:0/2902373409 >> 172.21.15.133:0/2902373409 conn(0x7fc24006cc60 msgr2=0x7fc24006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.181+0000 7fc247a23640 1 -- 172.21.15.133:0/2902373409 shutdown_connections 2024-09-11T23:45:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.182+0000 7fc247a23640 1 -- 172.21.15.133:0/2902373409 wait complete. 2024-09-11T23:45:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.183+0000 7fc247a23640 1 Processor -- start 2024-09-11T23:45:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.184+0000 7fc247a23640 1 -- start start 2024-09-11T23:45:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.184+0000 7fc247a23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2401095d0 0x7fc24011a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.184+0000 7fc247a23640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc24011b140 0x7fc24011abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.184+0000 7fc247a23640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc240114880 0x7fc240114d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.184+0000 7fc247a23640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc240110f30 con 0x7fc2401095d0 2024-09-11T23:45:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.184+0000 7fc247a23640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc240110db0 con 0x7fc240114880 2024-09-11T23:45:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.184+0000 7fc247a23640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401110b0 con 0x7fc24011b140 2024-09-11T23:45:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.185+0000 7fc244fbe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc24011b140 0x7fc24011abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.185+0000 7fc244fbe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc24011b140 0x7fc24011abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33714/0 (socket says 172.21.15.133:33714) 2024-09-11T23:45:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.185+0000 7fc244fbe640 1 -- 172.21.15.133:0/2800554947 learned_addr learned my addr 172.21.15.133:0/2800554947 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.186+0000 7fc244fbe640 1 -- 172.21.15.133:0/2800554947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc240114880 msgr2=0x7fc240114d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.186+0000 7fc244fbe640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc240114880 0x7fc240114d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.186+0000 7fc244fbe640 1 -- 172.21.15.133:0/2800554947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2401095d0 msgr2=0x7fc24011a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.186+0000 7fc244fbe640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2401095d0 0x7fc24011a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.186+0000 7fc244fbe640 1 -- 172.21.15.133:0/2800554947 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2401bc2e0 con 0x7fc24011b140 2024-09-11T23:45:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.191+0000 7fc244fbe640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc24011b140 0x7fc24011abf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00e9c0 tx=0x7fc23c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.197+0000 7fc2367fc640 1 -- 172.21.15.133:0/2800554947 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c00cde0 con 0x7fc24011b140 2024-09-11T23:45:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.198+0000 7fc247a23640 1 -- 172.21.15.133:0/2800554947 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2401bc4b0 con 0x7fc24011b140 2024-09-11T23:45:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.198+0000 7fc247a23640 1 -- 172.21.15.133:0/2800554947 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2401bc9a0 con 0x7fc24011b140 2024-09-11T23:45:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.198+0000 7fc2367fc640 1 -- 172.21.15.133:0/2800554947 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc23c0035f0 con 0x7fc24011b140 2024-09-11T23:45:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.198+0000 7fc2367fc640 1 -- 172.21.15.133:0/2800554947 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c010690 con 0x7fc24011b140 2024-09-11T23:45:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.210+0000 7fc2367fc640 1 -- 172.21.15.133:0/2800554947 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc23c003f80 con 0x7fc24011b140 2024-09-11T23:45:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.210+0000 7fc24a7bb740 1 Processor -- start 2024-09-11T23:45:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.210+0000 7fc24a7bb740 1 -- start start 2024-09-11T23:45:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.210+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5c9cc0 0x560c8b5ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.210+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.210+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5d4e80 0x560c8b5d7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.210+0000 7fc24a7bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560c8b52e2f0 con 0x560c8b5c9cc0 2024-09-11T23:45:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.210+0000 7fc24a7bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c8b4d6270 con 0x560c8b5d4e80 2024-09-11T23:45:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.210+0000 7fc24a7bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560c8b52a2b0 con 0x560c8b5ca5e0 2024-09-11T23:45:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.211+0000 7fc245fc0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5c9cc0 0x560c8b5ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.211+0000 7fc245fc0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5c9cc0 0x560c8b5ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49046/0 (socket says 172.21.15.133:49046) 2024-09-11T23:45:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.211+0000 7fc245fc0640 1 -- 172.21.15.133:0/1740257945 learned_addr learned my addr 172.21.15.133:0/1740257945 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.214+0000 7fc244fbe640 1 --2- 172.21.15.133:0/1740257945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5d4e80 0x560c8b5d7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.214+0000 7fc2457bf640 1 --2- 172.21.15.133:0/1740257945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.225+0000 7fc2367fc640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc22c0a2f00 0x7fc22c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.225+0000 7fc2367fc640 1 -- 172.21.15.133:0/2800554947 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fc23c0cf780 con 0x7fc24011b140 2024-09-11T23:45:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.226+0000 7fc245fc0640 1 -- 172.21.15.133:0/1740257945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5ca5e0 msgr2=0x560c8b5d4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.226+0000 7fc245fc0640 1 --2- 172.21.15.133:0/1740257945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.226+0000 7fc245fc0640 1 -- 172.21.15.133:0/1740257945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5d4e80 msgr2=0x560c8b5d7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.226+0000 7fc245fc0640 1 --2- 172.21.15.133:0/1740257945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5d4e80 0x560c8b5d7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.226+0000 7fc245fc0640 1 -- 172.21.15.133:0/1740257945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c8b4d5340 con 0x560c8b5c9cc0 2024-09-11T23:45:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.228+0000 7fc245fc0640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc22c0a2f00 0x7fc22c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.229+0000 7fc245fc0640 1 --2- 172.21.15.133:0/1740257945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5c9cc0 0x560c8b5ca0a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc2300098f0 tx=0x7fc230009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b66c7e444017c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.232+0000 7fc22b7fe640 1 -- 172.21.15.133:0/1740257945 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23000f040 con 0x560c8b5c9cc0 2024-09-11T23:45:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.232+0000 7fc22b7fe640 1 -- 172.21.15.133:0/1740257945 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc230013df0 con 0x560c8b5c9cc0 2024-09-11T23:45:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.234+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1740257945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5c9cc0 msgr2=0x560c8b5ca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.234+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/1740257945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5c9cc0 0x560c8b5ca0a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc2300098f0 tx=0x7fc230009db0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.236+0000 7fc245fc0640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc22c0a2f00 0x7fc22c0a53c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc23001d000 tx=0x7fc23001a000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.239+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1740257945 shutdown_connections 2024-09-11T23:45:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.239+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/1740257945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5d4e80 0x560c8b5d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.239+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/1740257945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.239+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/1740257945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5c9cc0 0x560c8b5ca0a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.239+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1740257945 >> 172.21.15.133:0/1740257945 conn(0x560c8b5c6230 msgr2=0x560c8b5c6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.232+0000 7fc22b7fe640 1 -- 172.21.15.133:0/1740257945 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23001eb60 con 0x560c8b5c9cc0 2024-09-11T23:45:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.246+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1740257945 shutdown_connections 2024-09-11T23:45:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.247+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1740257945 wait complete. 2024-09-11T23:45:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.247+0000 7fc24a7bb740 1 -- start start 2024-09-11T23:45:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.248+0000 7fc24a7bb740 1 Processor -- start 2024-09-11T23:45:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.249+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.249+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5d4e80 0x560c8b5d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.249+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5d1fe0 0x560c8b5d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.249+0000 7fc24a7bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560c8b5d7aa0 con 0x560c8b5ca5e0 2024-09-11T23:45:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.249+0000 7fc24a7bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c8b4b7290 con 0x560c8b5d4e80 2024-09-11T23:45:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.249+0000 7fc24a7bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560c8b5d7c20 con 0x560c8b5d1fe0 2024-09-11T23:45:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.249+0000 7fc24a7bb740 10 client.0 _ll_register_callbacks cb 0x560c8b4e8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.250+0000 7fc244fbe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5d4e80 0x560c8b5d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.250+0000 7fc244fbe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5d4e80 0x560c8b5d1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37458/0 (socket says 172.21.15.133:37458) 2024-09-11T23:45:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.250+0000 7fc244fbe640 1 -- 172.21.15.133:0/1893627717 learned_addr learned my addr 172.21.15.133:0/1893627717 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.250+0000 7fc2457bf640 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.250+0000 7fc245fc0640 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5d1fe0 0x560c8b5d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.251+0000 7fc245fc0640 1 -- 172.21.15.133:0/1893627717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5d4e80 msgr2=0x560c8b5d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.251+0000 7fc245fc0640 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5d4e80 0x560c8b5d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.251+0000 7fc245fc0640 1 -- 172.21.15.133:0/1893627717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5ca5e0 msgr2=0x560c8b5d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.251+0000 7fc245fc0640 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.251+0000 7fc245fc0640 1 -- 172.21.15.133:0/1893627717 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c8b241210 con 0x560c8b5d1fe0 2024-09-11T23:45:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.252+0000 7fc245fc0640 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5d1fe0 0x560c8b5d2420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc23000b4b0 tx=0x7fc2300047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cf5f9ee7e000c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.267+0000 7fc228ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.267+0000 7fc228ff9640 1 -- 172.21.15.133:0/1893627717 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23002b060 con 0x560c8b5d1fe0 2024-09-11T23:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.267+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1893627717 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc230013df0 con 0x560c8b5d1fe0 2024-09-11T23:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.267+0000 7fc228ff9640 1 -- 172.21.15.133:0/1893627717 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fc230013910 con 0x560c8b5d1fe0 2024-09-11T23:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.268+0000 7fc228ff9640 1 -- 172.21.15.133:0/1893627717 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc23000bdc0 con 0x560c8b5d1fe0 2024-09-11T23:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.268+0000 7fc24a7bb740 10 client.27097 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.268+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1893627717 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560c8b30d620 con 0x560c8b5d1fe0 2024-09-11T23:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.268+0000 7fc228ff9640 1 -- 172.21.15.133:0/1893627717 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc230013df0 con 0x560c8b5d1fe0 2024-09-11T23:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.268+0000 7fc228ff9640 1 -- 172.21.15.133:0/1893627717 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc230030a50 con 0x560c8b5d1fe0 2024-09-11T23:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.268+0000 7fc24a7bb740 10 client.27097 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.268+0000 7fc24a7bb740 10 client.27097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.268+0000 7fc24a7bb740 20 client.27097 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.268+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560c8b5e51b0 0x560c8b5e7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.269+0000 7fc24a7bb740 4 client.27097 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.269+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1893627717 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c8b53ba20 con 0x560c8b5e51b0 2024-09-11T23:45:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.269+0000 7fc2457bf640 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560c8b5e51b0 0x560c8b5e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.270+0000 7fc2457bf640 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560c8b5e51b0 0x560c8b5e7610 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95a8bc801b3e620e server_cookie=4c1cf35ed0e94b22 in_seq=0 out_seq=0 2024-09-11T23:45:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.270+0000 7fc228ff9640 10 client.27097 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.271+0000 7fc228ff9640 1 -- 172.21.15.133:0/1893627717 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c8b53ba20 con 0x560c8b5e51b0 2024-09-11T23:45:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.271+0000 7fc228ff9640 10 client.27097 handle_command_reply: tid=0 2024-09-11T23:45:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.271+0000 7fc24a7bb740 1 client.27097 shutdown 2024-09-11T23:45:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.281+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1893627717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5d1fe0 msgr2=0x560c8b5d2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.281+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5d1fe0 0x560c8b5d2420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc23000b4b0 tx=0x7fc2300047c0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.286+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1893627717 shutdown_connections 2024-09-11T23:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.286+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560c8b5e51b0 0x560c8b5e7610 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.286+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5d1fe0 0x560c8b5d2420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.286+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5d4e80 0x560c8b5d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.286+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/1893627717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.286+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1893627717 >> 172.21.15.133:0/1893627717 conn(0x560c8b5c6230 msgr2=0x560c8b5cb770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.286+0000 7fc228ff9640 0 client.27097 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.289+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1893627717 shutdown_connections 2024-09-11T23:45:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.289+0000 7fc24a7bb740 1 -- 172.21.15.133:0/1893627717 wait complete. 2024-09-11T23:45:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.289+0000 7fc24a7bb740 20 client.27097 trim_cache size 0 max 16384 2024-09-11T23:45:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.338+0000 7fc24a7bb740 1 Processor -- start 2024-09-11T23:45:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.340+0000 7fc24a7bb740 1 -- start start 2024-09-11T23:45:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.340+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5d4960 0x560c8b5ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.341+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5c6230 0x560c8b5d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.341+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.341+0000 7fc24a7bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc22400e7d0 con 0x560c8b5d4960 2024-09-11T23:45:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.341+0000 7fc24a7bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc224010310 con 0x560c8b5ca5e0 2024-09-11T23:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.341+0000 7fc24a7bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc22400eea0 con 0x560c8b5c6230 2024-09-11T23:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.342+0000 7fc244fbe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5c6230 0x560c8b5d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.342+0000 7fc244fbe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5c6230 0x560c8b5d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33734/0 (socket says 172.21.15.133:33734) 2024-09-11T23:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.342+0000 7fc244fbe640 1 -- 172.21.15.133:0/3527790980 learned_addr learned my addr 172.21.15.133:0/3527790980 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.342+0000 7fc2457bf640 1 --2- 172.21.15.133:0/3527790980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5d4960 0x560c8b5ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.343+0000 7fc245fc0640 1 --2- 172.21.15.133:0/3527790980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.343+0000 7fc245fc0640 1 -- 172.21.15.133:0/3527790980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5c6230 msgr2=0x560c8b5d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.343+0000 7fc245fc0640 1 --2- 172.21.15.133:0/3527790980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5c6230 0x560c8b5d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.343+0000 7fc245fc0640 1 -- 172.21.15.133:0/3527790980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5d4960 msgr2=0x560c8b5ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.343+0000 7fc245fc0640 1 --2- 172.21.15.133:0/3527790980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5d4960 0x560c8b5ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.343+0000 7fc245fc0640 1 -- 172.21.15.133:0/3527790980 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c8b5cc830 con 0x560c8b5ca5e0 2024-09-11T23:45:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.344+0000 7fc245fc0640 1 --2- 172.21.15.133:0/3527790980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d1880 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc230009df0 tx=0x7fc230006f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1f7487cfb95aedf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.345+0000 7fc22affd640 1 -- 172.21.15.133:0/3527790980 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23002b070 con 0x560c8b5ca5e0 2024-09-11T23:45:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.346+0000 7fc22affd640 1 -- 172.21.15.133:0/3527790980 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc23000b9c0 con 0x560c8b5ca5e0 2024-09-11T23:45:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.346+0000 7fc22affd640 1 -- 172.21.15.133:0/3527790980 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230025770 con 0x560c8b5ca5e0 2024-09-11T23:45:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.348+0000 7fc24a7bb740 1 -- 172.21.15.133:0/3527790980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5ca5e0 msgr2=0x560c8b5d1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.348+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/3527790980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d1880 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc230009df0 tx=0x7fc230006f30 comp rx=0 tx=0).stop 2024-09-11T23:45:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.348+0000 7fc24a7bb740 1 -- 172.21.15.133:0/3527790980 shutdown_connections 2024-09-11T23:45:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.348+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/3527790980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5ca5e0 0x560c8b5d1880 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.348+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/3527790980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5c6230 0x560c8b5d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.348+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/3527790980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5d4960 0x560c8b5ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.348+0000 7fc24a7bb740 1 -- 172.21.15.133:0/3527790980 >> 172.21.15.133:0/3527790980 conn(0x560c8b5ca200 msgr2=0x560c8b5d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.348+0000 7fc24a7bb740 1 -- 172.21.15.133:0/3527790980 shutdown_connections 2024-09-11T23:45:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.349+0000 7fc24a7bb740 1 -- 172.21.15.133:0/3527790980 wait complete. 2024-09-11T23:45:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.349+0000 7fc24a7bb740 1 -- start start 2024-09-11T23:45:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.349+0000 7fc24a7bb740 1 Processor -- start 2024-09-11T23:45:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.350+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5c6230 0x560c8b5dcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.350+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5ca5e0 0x560c8b5dd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.351+0000 7fc24a7bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5d4960 0x560c8b5ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.351+0000 7fc24a7bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc22400fc40 con 0x560c8b5ca5e0 2024-09-11T23:45:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.352+0000 7fc245fc0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5d4960 0x560c8b5ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.353+0000 7fc2457bf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5c6230 0x560c8b5dcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.353+0000 7fc2457bf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5c6230 0x560c8b5dcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37462/0 (socket says 172.21.15.133:37462) 2024-09-11T23:45:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.353+0000 7fc2457bf640 1 -- 172.21.15.133:0/122859118 learned_addr learned my addr 172.21.15.133:0/122859118 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.353+0000 7fc244fbe640 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5ca5e0 0x560c8b5dd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.353+0000 7fc24a7bb740 1 -- 172.21.15.133:0/122859118 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc22400f570 con 0x560c8b5c6230 2024-09-11T23:45:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.353+0000 7fc24a7bb740 1 -- 172.21.15.133:0/122859118 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc22400c7d0 con 0x560c8b5d4960 2024-09-11T23:45:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.353+0000 7fc24a7bb740 10 client.0 _ll_register_callbacks cb 0x7fc238034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.354+0000 7fc2457bf640 1 -- 172.21.15.133:0/122859118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5d4960 msgr2=0x560c8b5ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.354+0000 7fc2457bf640 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5d4960 0x560c8b5ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.354+0000 7fc2457bf640 1 -- 172.21.15.133:0/122859118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5ca5e0 msgr2=0x560c8b5dd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.354+0000 7fc2457bf640 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5ca5e0 0x560c8b5dd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.354+0000 7fc2457bf640 1 -- 172.21.15.133:0/122859118 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c8b5cf240 con 0x560c8b5c6230 2024-09-11T23:45:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.363+0000 7fc2457bf640 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5c6230 0x560c8b5dcf30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc2380099a0 tx=0x7fc238030b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a941af2689a00f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.371+0000 7fc228ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:45:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.371+0000 7fc228ff9640 1 -- 172.21.15.133:0/122859118 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238036040 con 0x560c8b5c6230 2024-09-11T23:45:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.371+0000 7fc228ff9640 1 -- 172.21.15.133:0/122859118 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fc238030c30 con 0x560c8b5c6230 2024-09-11T23:45:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.384+0000 7fc24a7bb740 1 -- 172.21.15.133:0/122859118 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560c8b535d70 con 0x560c8b5c6230 2024-09-11T23:45:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.391+0000 7fc228ff9640 1 -- 172.21.15.133:0/122859118 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc238030e30 con 0x560c8b5c6230 2024-09-11T23:45:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.391+0000 7fc24a7bb740 10 client.27290 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.391+0000 7fc24a7bb740 1 -- 172.21.15.133:0/122859118 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560c8b5ec7c0 con 0x560c8b5c6230 2024-09-11T23:45:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.394+0000 7fc228ff9640 1 -- 172.21.15.133:0/122859118 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc238003970 con 0x560c8b5c6230 2024-09-11T23:45:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.394+0000 7fc228ff9640 1 -- 172.21.15.133:0/122859118 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc2380319d0 con 0x560c8b5c6230 2024-09-11T23:45:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.394+0000 7fc24a7bb740 10 client.27290 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.394+0000 7fc24a7bb740 10 client.27290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.394+0000 7fc24a7bb740 20 client.27290 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.394+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560c8b5d1fe0 0x560c8b651f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.394+0000 7fc24a7bb740 4 client.27290 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:45:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.394+0000 7fc24a7bb740 1 -- 172.21.15.133:0/122859118 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c8b535d70 con 0x560c8b5d1fe0 2024-09-11T23:45:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.396+0000 7fc244fbe640 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560c8b5d1fe0 0x560c8b651f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.403+0000 7fc244fbe640 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560c8b5d1fe0 0x560c8b651f00 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abb2e0acaaaff4ac server_cookie=be967ed61a6e106 in_seq=0 out_seq=0 2024-09-11T23:45:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:46.403+0000 7fc228ff9640 10 client.27290 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:46.504 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:45:46.972 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:46 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:45:46.972 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:46 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:45:46.972 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:46 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:45:46.972 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:46 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:45:46.972 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:46 smithi133 ceph-mon[31430]: pgmap v279: 97 pgs: 97 active+clean; 66 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 516 MiB/s wr, 2.81k op/s 2024-09-11T23:45:47.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:46 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:45:47.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:46 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:45:47.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:46 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:45:47.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:46 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:45:47.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:46 smithi178 ceph-mon[35882]: pgmap v279: 97 pgs: 97 active+clean; 66 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 516 MiB/s wr, 2.81k op/s 2024-09-11T23:45:47.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:46 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:45:47.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:46 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:45:47.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:46 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:45:47.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:46 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:45:47.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:46 smithi138 ceph-mon[35142]: pgmap v279: 97 pgs: 97 active+clean; 66 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 516 MiB/s wr, 2.81k op/s 2024-09-11T23:45:47.410 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:45:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.603+0000 7fc228ff9640 1 -- 172.21.15.133:0/122859118 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c8b535d70 con 0x560c8b5d1fe0 2024-09-11T23:45:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.603+0000 7fc228ff9640 10 client.27290 handle_command_reply: tid=0 2024-09-11T23:45:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.604+0000 7fc24a7bb740 1 client.27290 shutdown 2024-09-11T23:45:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.610+0000 7fc24a7bb740 1 -- 172.21.15.133:0/122859118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5c6230 msgr2=0x560c8b5dcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.610+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5c6230 0x560c8b5dcf30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc2380099a0 tx=0x7fc238030b40 comp rx=0 tx=0).stop 2024-09-11T23:45:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.620+0000 7fc24a7bb740 1 -- 172.21.15.133:0/122859118 shutdown_connections 2024-09-11T23:45:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.620+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560c8b5d1fe0 0x560c8b651f00 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.620+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c8b5d4960 0x560c8b5ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.620+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560c8b5ca5e0 0x560c8b5dd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.620+0000 7fc24a7bb740 1 --2- 172.21.15.133:0/122859118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c8b5c6230 0x560c8b5dcf30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.620+0000 7fc24a7bb740 1 -- 172.21.15.133:0/122859118 >> 172.21.15.133:0/122859118 conn(0x560c8b5ca200 msgr2=0x560c8b5d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.620+0000 7fc228ff9640 0 client.27290 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.620+0000 7fc24a7bb740 1 -- 172.21.15.133:0/122859118 shutdown_connections 2024-09-11T23:45:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.621+0000 7fc24a7bb740 1 -- 172.21.15.133:0/122859118 wait complete. 2024-09-11T23:45:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.621+0000 7fc24a7bb740 20 client.27290 trim_cache size 0 max 16384 2024-09-11T23:45:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:45:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:45:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:45:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:45:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.626+0000 7fc247a23640 1 -- 172.21.15.133:0/2800554947 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc22c0a2f00 msgr2=0x7fc22c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.626+0000 7fc247a23640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc22c0a2f00 0x7fc22c0a53c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc23001d000 tx=0x7fc23001a000 comp rx=0 tx=0).stop 2024-09-11T23:45:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.626+0000 7fc247a23640 1 -- 172.21.15.133:0/2800554947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc24011b140 msgr2=0x7fc24011abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.627+0000 7fc247a23640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc24011b140 0x7fc24011abf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00e9c0 tx=0x7fc23c00ee80 comp rx=0 tx=0).stop 2024-09-11T23:45:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.633+0000 7fc247a23640 1 -- 172.21.15.133:0/2800554947 shutdown_connections 2024-09-11T23:45:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.633+0000 7fc247a23640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc22c0a2f00 0x7fc22c0a53c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.633+0000 7fc247a23640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc240114880 0x7fc240114d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.633+0000 7fc247a23640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc24011b140 0x7fc24011abf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.633+0000 7fc247a23640 1 --2- 172.21.15.133:0/2800554947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2401095d0 0x7fc24011a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.633+0000 7fc247a23640 1 -- 172.21.15.133:0/2800554947 >> 172.21.15.133:0/2800554947 conn(0x7fc24006cc60 msgr2=0x7fc240071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.634+0000 7fc247a23640 1 -- 172.21.15.133:0/2800554947 shutdown_connections 2024-09-11T23:45:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:45:47.634+0000 7fc247a23640 1 -- 172.21.15.133:0/2800554947 wait complete. 2024-09-11T23:45:48.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.453+0000 7f4cb8233640 1 -- 172.21.15.133:0/793509796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb0109450 msgr2=0x7f4cb0110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:48.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.453+0000 7f4cb4fcd640 1 -- 172.21.15.133:0/793509796 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac035020 con 0x7f4cb0109450 2024-09-11T23:45:48.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.453+0000 7f4cb8233640 1 --2- 172.21.15.133:0/793509796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb0109450 0x7f4cb0110250 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009900 tx=0x7f4cac02a440 comp rx=0 tx=0).stop 2024-09-11T23:45:48.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.458+0000 7f4cb8233640 1 -- 172.21.15.133:0/793509796 shutdown_connections 2024-09-11T23:45:48.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.458+0000 7f4cb8233640 1 --2- 172.21.15.133:0/793509796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cb011b140 0x7f4cb011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.458+0000 7f4cb8233640 1 --2- 172.21.15.133:0/793509796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb0109830 0x7f4cb0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.458+0000 7f4cb8233640 1 --2- 172.21.15.133:0/793509796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb0109450 0x7f4cb0110250 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.458+0000 7f4cb8233640 1 -- 172.21.15.133:0/793509796 >> 172.21.15.133:0/793509796 conn(0x7f4cb006e140 msgr2=0x7f4cb006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:48.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.458+0000 7f4cb8233640 1 -- 172.21.15.133:0/793509796 shutdown_connections 2024-09-11T23:45:48.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.458+0000 7f4cb8233640 1 -- 172.21.15.133:0/793509796 wait complete. 2024-09-11T23:45:48.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.460+0000 7f4cb8233640 1 Processor -- start 2024-09-11T23:45:48.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.460+0000 7f4cb8233640 1 -- start start 2024-09-11T23:45:48.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.460+0000 7f4cb8233640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb0109450 0x7f4cb011a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.460+0000 7f4cb8233640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cb0109830 0x7f4cb011a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.460+0000 7f4cb8233640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb011b140 0x7f4cb01144b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.460+0000 7f4cb8233640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb0110f30 con 0x7f4cb011b140 2024-09-11T23:45:48.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.460+0000 7f4cb8233640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb0110db0 con 0x7f4cb0109450 2024-09-11T23:45:48.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.460+0000 7f4cb8233640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb01110b0 con 0x7f4cb0109830 2024-09-11T23:45:48.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.462+0000 7f4cb67d0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb011b140 0x7f4cb01144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.462+0000 7f4cb67d0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb011b140 0x7f4cb01144b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49098/0 (socket says 172.21.15.133:49098) 2024-09-11T23:45:48.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.462+0000 7f4cb67d0640 1 -- 172.21.15.133:0/170743916 learned_addr learned my addr 172.21.15.133:0/170743916 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:48.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.465+0000 7f4cb67d0640 1 -- 172.21.15.133:0/170743916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cb0109830 msgr2=0x7f4cb011a970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:45:48.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.465+0000 7f4cb67d0640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cb0109830 0x7f4cb011a970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.465+0000 7f4cb5fcf640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb0109450 0x7f4cb011a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.465+0000 7f4cb67d0640 1 -- 172.21.15.133:0/170743916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb0109450 msgr2=0x7f4cb011a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:48.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.465+0000 7f4cb67d0640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb0109450 0x7f4cb011a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.465+0000 7f4cb67d0640 1 -- 172.21.15.133:0/170743916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb0114d70 con 0x7f4cb011b140 2024-09-11T23:45:48.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.471+0000 7f4cb67d0640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb011b140 0x7f4cb01144b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800cab0 tx=0x7f4ca800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:48.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.484+0000 7f4ca6ffd640 1 -- 172.21.15.133:0/170743916 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca800ae20 con 0x7f4cb011b140 2024-09-11T23:45:48.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.485+0000 7f4cb8233640 1 -- 172.21.15.133:0/170743916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb0115060 con 0x7f4cb011b140 2024-09-11T23:45:48.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.485+0000 7f4cb8233640 1 -- 172.21.15.133:0/170743916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb007a620 con 0x7f4cb011b140 2024-09-11T23:45:48.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.485+0000 7f4ca6ffd640 1 -- 172.21.15.133:0/170743916 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ca800f930 con 0x7f4cb011b140 2024-09-11T23:45:48.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.485+0000 7f4ca6ffd640 1 -- 172.21.15.133:0/170743916 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8018b80 con 0x7f4cb011b140 2024-09-11T23:45:48.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.494+0000 7f4cba60c740 1 Processor -- start 2024-09-11T23:45:48.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.499+0000 7f4cba60c740 1 -- start start 2024-09-11T23:45:48.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.499+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 0x558c52dab880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.499+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.499+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52db6680 0x558c52db8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.499+0000 7f4cba60c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558c52d112e0 con 0x558c52db6680 2024-09-11T23:45:48.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.499+0000 7f4cba60c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558c52cb8ce0 con 0x558c52dab4a0 2024-09-11T23:45:48.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.499+0000 7f4cba60c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558c52d0d2a0 con 0x558c52dabdc0 2024-09-11T23:45:48.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.500+0000 7f4ca6ffd640 1 -- 172.21.15.133:0/170743916 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4ca800f450 con 0x7f4cb011b140 2024-09-11T23:45:48.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.501+0000 7f4cb57ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 0x558c52dab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.502+0000 7f4ca6ffd640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4c9c0a2e80 0x7f4c9c0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.502+0000 7f4cb5fcf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.502+0000 7f4cb57ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 0x558c52dab880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37504/0 (socket says 172.21.15.133:37504) 2024-09-11T23:45:48.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.502+0000 7f4cb57ce640 1 -- 172.21.15.133:0/634848256 learned_addr learned my addr 172.21.15.133:0/634848256 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:48.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.502+0000 7f4cb67d0640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4c9c0a2e80 0x7f4c9c0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.503+0000 7f4cb57ce640 1 -- 172.21.15.133:0/634848256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 msgr2=0x558c52db6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:48.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.503+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/634848256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.503+0000 7f4cb57ce640 1 -- 172.21.15.133:0/634848256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52db6680 msgr2=0x558c52db8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:48.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.503+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/634848256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52db6680 0x558c52db8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.503+0000 7f4cb57ce640 1 -- 172.21.15.133:0/634848256 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c52c9d440 con 0x558c52dab4a0 2024-09-11T23:45:48.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.503+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/634848256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 0x558c52dab880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000c9e0 tx=0x7f4ca000cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d94cf98733f3e70e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:48.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.503+0000 7f4cb67d0640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4c9c0a2e80 0x7f4c9c0a5340 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8011000 tx=0x7f4ca806be80 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:48.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.506+0000 7f4ca6ffd640 1 -- 172.21.15.133:0/170743916 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f4ca809a040 con 0x7f4cb011b140 2024-09-11T23:45:48.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.506+0000 7f4c9bfff640 1 -- 172.21.15.133:0/634848256 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca000e9c0 con 0x558c52dab4a0 2024-09-11T23:45:48.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.506+0000 7f4c9bfff640 1 -- 172.21.15.133:0/634848256 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ca000eb60 con 0x558c52dab4a0 2024-09-11T23:45:48.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.507+0000 7f4cba60c740 1 -- 172.21.15.133:0/634848256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 msgr2=0x558c52dab880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:48.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.507+0000 7f4cba60c740 1 --2- 172.21.15.133:0/634848256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 0x558c52dab880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000c9e0 tx=0x7f4ca000cea0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.507+0000 7f4cba60c740 1 -- 172.21.15.133:0/634848256 shutdown_connections 2024-09-11T23:45:48.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.507+0000 7f4cba60c740 1 --2- 172.21.15.133:0/634848256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52db6680 0x558c52db8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.507+0000 7f4cba60c740 1 --2- 172.21.15.133:0/634848256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.507+0000 7f4cba60c740 1 --2- 172.21.15.133:0/634848256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 0x558c52dab880 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.507+0000 7f4cba60c740 1 -- 172.21.15.133:0/634848256 >> 172.21.15.133:0/634848256 conn(0x558c52da7a30 msgr2=0x558c52da7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:48.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.508+0000 7f4cba60c740 1 -- 172.21.15.133:0/634848256 shutdown_connections 2024-09-11T23:45:48.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.508+0000 7f4cba60c740 1 -- 172.21.15.133:0/634848256 wait complete. 2024-09-11T23:45:48.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.509+0000 7f4cba60c740 1 -- start start 2024-09-11T23:45:48.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.509+0000 7f4cba60c740 1 Processor -- start 2024-09-11T23:45:48.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.509+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 0x558c52db5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.510+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.510+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52db6680 0x558c52db3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.510+0000 7f4cba60c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558c52db92a0 con 0x558c52db6680 2024-09-11T23:45:48.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.510+0000 7f4cba60c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558c52ccc050 con 0x558c52dab4a0 2024-09-11T23:45:48.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.510+0000 7f4cba60c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558c52db9420 con 0x558c52dabdc0 2024-09-11T23:45:48.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.510+0000 7f4cba60c740 10 client.0 _ll_register_callbacks cb 0x558c52d0c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:48.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.511+0000 7f4cb5fcf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.511+0000 7f4cb5fcf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33762/0 (socket says 172.21.15.133:33762) 2024-09-11T23:45:48.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.511+0000 7f4cb5fcf640 1 -- 172.21.15.133:0/2618613727 learned_addr learned my addr 172.21.15.133:0/2618613727 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:48.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.511+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 0x558c52db5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.513+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52db6680 0x558c52db3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.514+0000 7f4cb57ce640 1 -- 172.21.15.133:0/2618613727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 msgr2=0x558c52db32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:48.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.514+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.514+0000 7f4cb57ce640 1 -- 172.21.15.133:0/2618613727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52db6680 msgr2=0x558c52db3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:48.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.514+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52db6680 0x558c52db3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.514+0000 7f4cb57ce640 1 -- 172.21.15.133:0/2618613727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c52a823c0 con 0x558c52dab4a0 2024-09-11T23:45:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.519+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 0x558c52db5ff0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0005aa0 tx=0x7f4ca0005c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67f02b4b7195d92f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.519+0000 7f4c997fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:45:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.523+0000 7f4c997fa640 1 -- 172.21.15.133:0/2618613727 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0004020 con 0x558c52dab4a0 2024-09-11T23:45:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.523+0000 7f4c997fa640 1 -- 172.21.15.133:0/2618613727 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f4ca00041c0 con 0x558c52dab4a0 2024-09-11T23:45:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.523+0000 7f4cba60c740 1 -- 172.21.15.133:0/2618613727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558c52d1e850 con 0x558c52dab4a0 2024-09-11T23:45:48.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.527+0000 7f4c997fa640 1 -- 172.21.15.133:0/2618613727 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4ca00043c0 con 0x558c52dab4a0 2024-09-11T23:45:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.527+0000 7f4cba60c740 10 client.27296 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.527+0000 7f4cba60c740 1 -- 172.21.15.133:0/2618613727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c52dc4de0 con 0x558c52dab4a0 2024-09-11T23:45:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.530+0000 7f4c997fa640 1 -- 172.21.15.133:0/2618613727 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558c52d1e850 con 0x558c52dab4a0 2024-09-11T23:45:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.530+0000 7f4c997fa640 1 -- 172.21.15.133:0/2618613727 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4ca000f040 con 0x558c52dab4a0 2024-09-11T23:45:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.530+0000 7f4cba60c740 10 client.27296 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.530+0000 7f4cba60c740 10 client.27296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.530+0000 7f4cba60c740 20 client.27296 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.530+0000 7f4cba60c740 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558c52dc6130 0x558c52dc8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.530+0000 7f4cba60c740 4 client.27296 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:45:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.530+0000 7f4cba60c740 1 -- 172.21.15.133:0/2618613727 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c52d18d60 con 0x558c52dc6130 2024-09-11T23:45:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.531+0000 7f4cb5fcf640 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558c52dc6130 0x558c52dc8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.532+0000 7f4c997fa640 10 client.27296 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.532+0000 7f4cb5fcf640 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558c52dc6130 0x558c52dc8520 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1adcf68ad6320bd server_cookie=becf2673289108cd in_seq=0 out_seq=0 2024-09-11T23:45:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.537+0000 7f4c997fa640 1 -- 172.21.15.133:0/2618613727 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c52d18d60 con 0x558c52dc6130 2024-09-11T23:45:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.537+0000 7f4c997fa640 10 client.27296 handle_command_reply: tid=0 2024-09-11T23:45:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.538+0000 7f4cba60c740 1 client.27296 shutdown 2024-09-11T23:45:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.538+0000 7f4cba60c740 1 -- 172.21.15.133:0/2618613727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 msgr2=0x558c52db5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.538+0000 7f4cba60c740 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 0x558c52db5ff0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0005aa0 tx=0x7f4ca0005c10 comp rx=0 tx=0).stop 2024-09-11T23:45:48.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.541+0000 7f4cba60c740 1 -- 172.21.15.133:0/2618613727 shutdown_connections 2024-09-11T23:45:48.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.541+0000 7f4cba60c740 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558c52dc6130 0x558c52dc8520 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.541+0000 7f4cba60c740 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52db6680 0x558c52db3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.541+0000 7f4cba60c740 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.541+0000 7f4cba60c740 1 --2- 172.21.15.133:0/2618613727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52dab4a0 0x558c52db5ff0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.541+0000 7f4cba60c740 1 -- 172.21.15.133:0/2618613727 >> 172.21.15.133:0/2618613727 conn(0x558c52da7a30 msgr2=0x558c52da6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:48.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.541+0000 7f4c997fa640 0 client.27296 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:48.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.541+0000 7f4cba60c740 1 -- 172.21.15.133:0/2618613727 shutdown_connections 2024-09-11T23:45:48.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.541+0000 7f4cba60c740 1 -- 172.21.15.133:0/2618613727 wait complete. 2024-09-11T23:45:48.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.542+0000 7f4cba60c740 20 client.27296 trim_cache size 0 max 16384 2024-09-11T23:45:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.630+0000 7f4cba60c740 1 Processor -- start 2024-09-11T23:45:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.631+0000 7f4cba60c740 1 -- start start 2024-09-11T23:45:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.632+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52dab4a0 0x558c52dc25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.632+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.632+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52db6680 0x558c52dae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.632+0000 7f4cba60c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9400e780 con 0x558c52dab4a0 2024-09-11T23:45:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.632+0000 7f4cba60c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c940102c0 con 0x558c52db6680 2024-09-11T23:45:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.632+0000 7f4cba60c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9400ee50 con 0x558c52dabdc0 2024-09-11T23:45:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.641+0000 7f4cb5fcf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.641+0000 7f4cb57ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52dab4a0 0x558c52dc25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.641+0000 7f4cb57ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52dab4a0 0x558c52dc25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49134/0 (socket says 172.21.15.133:49134) 2024-09-11T23:45:48.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.641+0000 7f4cb57ce640 1 -- 172.21.15.133:0/1299087010 learned_addr learned my addr 172.21.15.133:0/1299087010 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:48.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.644+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/1299087010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52db6680 0x558c52dae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.645+0000 7f4cb5fcf640 1 -- 172.21.15.133:0/1299087010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52db6680 msgr2=0x558c52dae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:48.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.645+0000 7f4cb5fcf640 1 --2- 172.21.15.133:0/1299087010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52db6680 0x558c52dae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.645+0000 7f4cb5fcf640 1 -- 172.21.15.133:0/1299087010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52dab4a0 msgr2=0x558c52dc25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:48.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.645+0000 7f4cb5fcf640 1 --2- 172.21.15.133:0/1299087010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52dab4a0 0x558c52dc25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.645+0000 7f4cb5fcf640 1 -- 172.21.15.133:0/1299087010 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c52db2250 con 0x558c52dabdc0 2024-09-11T23:45:48.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.646+0000 7f4cb5fcf640 1 --2- 172.21.15.133:0/1299087010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db1b70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x558c52db48e0 tx=0x7f4cac031480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30dbd88f9f5ebd65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:48.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.652+0000 7f4c9b7fe640 1 -- 172.21.15.133:0/1299087010 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac04b070 con 0x558c52dabdc0 2024-09-11T23:45:48.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.652+0000 7f4c9b7fe640 1 -- 172.21.15.133:0/1299087010 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cac005520 con 0x558c52dabdc0 2024-09-11T23:45:48.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.652+0000 7f4c9b7fe640 1 -- 172.21.15.133:0/1299087010 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac035040 con 0x558c52dabdc0 2024-09-11T23:45:48.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.654+0000 7f4cba60c740 1 -- 172.21.15.133:0/1299087010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 msgr2=0x558c52db1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:48.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.654+0000 7f4cba60c740 1 --2- 172.21.15.133:0/1299087010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db1b70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x558c52db48e0 tx=0x7f4cac031480 comp rx=0 tx=0).stop 2024-09-11T23:45:48.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.654+0000 7f4cba60c740 1 -- 172.21.15.133:0/1299087010 shutdown_connections 2024-09-11T23:45:48.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.654+0000 7f4cba60c740 1 --2- 172.21.15.133:0/1299087010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52db6680 0x558c52dae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.654+0000 7f4cba60c740 1 --2- 172.21.15.133:0/1299087010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dabdc0 0x558c52db1b70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.654+0000 7f4cba60c740 1 --2- 172.21.15.133:0/1299087010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52dab4a0 0x558c52dc25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.654+0000 7f4cba60c740 1 -- 172.21.15.133:0/1299087010 >> 172.21.15.133:0/1299087010 conn(0x558c52da7a30 msgr2=0x558c52dae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:48.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.655+0000 7f4cba60c740 1 -- 172.21.15.133:0/1299087010 shutdown_connections 2024-09-11T23:45:48.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.655+0000 7f4cba60c740 1 -- 172.21.15.133:0/1299087010 wait complete. 2024-09-11T23:45:48.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.656+0000 7f4cba60c740 1 -- start start 2024-09-11T23:45:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.659+0000 7f4cba60c740 1 Processor -- start 2024-09-11T23:45:48.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.661+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dab4a0 0x558c52dbed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.661+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52db6680 0x558c52db9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.661+0000 7f4cba60c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52dba4f0 0x558c52dba970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.662+0000 7f4cba60c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9400fbf0 con 0x558c52dba4f0 2024-09-11T23:45:48.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.662+0000 7f4cb5fcf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52db6680 0x558c52db9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.662+0000 7f4cb5fcf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52db6680 0x558c52db9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37522/0 (socket says 172.21.15.133:37522) 2024-09-11T23:45:48.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.662+0000 7f4cb5fcf640 1 -- 172.21.15.133:0/3089743230 learned_addr learned my addr 172.21.15.133:0/3089743230 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:45:48.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.662+0000 7f4cba60c740 1 -- 172.21.15.133:0/3089743230 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9400f520 con 0x558c52db6680 2024-09-11T23:45:48.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.662+0000 7f4cba60c740 1 -- 172.21.15.133:0/3089743230 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9400c780 con 0x558c52dab4a0 2024-09-11T23:45:48.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.662+0000 7f4cba60c740 10 client.0 _ll_register_callbacks cb 0x7f4ca000c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:45:48.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.663+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dab4a0 0x558c52dbed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.667+0000 7f4cb57ce640 1 -- 172.21.15.133:0/3089743230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52db6680 msgr2=0x558c52db9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:48.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.667+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52db6680 0x558c52db9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.667+0000 7f4cb57ce640 1 -- 172.21.15.133:0/3089743230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52dba4f0 msgr2=0x558c52dba970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:45:48.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.667+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52dba4f0 0x558c52dba970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.667+0000 7f4cb57ce640 1 -- 172.21.15.133:0/3089743230 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c52db3b70 con 0x558c52dab4a0 2024-09-11T23:45:48.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.667+0000 7f4cb57ce640 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dab4a0 0x558c52dbed20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4ca001ac60 tx=0x7f4ca000e7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db185b4bcbc5b932 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:45:48.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.668+0000 7f4c997fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:45:48.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.668+0000 7f4c997fa640 1 -- 172.21.15.133:0/3089743230 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0036070 con 0x558c52dab4a0 2024-09-11T23:45:48.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.669+0000 7f4c997fa640 1 -- 172.21.15.133:0/3089743230 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f4ca0009050 con 0x558c52dab4a0 2024-09-11T23:45:48.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.669+0000 7f4cba60c740 1 -- 172.21.15.133:0/3089743230 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cac005520 con 0x558c52dab4a0 2024-09-11T23:45:48.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.672+0000 7f4c997fa640 1 -- 172.21.15.133:0/3089743230 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4ca0040050 con 0x558c52dab4a0 2024-09-11T23:45:48.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.676+0000 7f4cba60c740 10 client.27112 fetch_fsmap learned FSMap version 30 2024-09-11T23:45:48.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.676+0000 7f4cba60c740 1 -- 172.21.15.133:0/3089743230 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c52e30d10 con 0x558c52dab4a0 2024-09-11T23:45:48.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.679+0000 7f4c997fa640 1 -- 172.21.15.133:0/3089743230 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cac005520 con 0x558c52dab4a0 2024-09-11T23:45:48.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.679+0000 7f4c997fa640 1 -- 172.21.15.133:0/3089743230 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4ca0004870 con 0x558c52dab4a0 2024-09-11T23:45:48.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.680+0000 7f4cba60c740 10 client.27112 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:45:48.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.680+0000 7f4cba60c740 10 client.27112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:45:48.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.680+0000 7f4cba60c740 20 client.27112 populate_metadata read hostname 'smithi133' 2024-09-11T23:45:48.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.680+0000 7f4cba60c740 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558c52e31310 0x558c52e33700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:45:48.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.680+0000 7f4cba60c740 4 client.27112 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:45:48.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.680+0000 7f4cba60c740 1 -- 172.21.15.133:0/3089743230 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c52c95210 con 0x558c52e31310 2024-09-11T23:45:48.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.681+0000 7f4cb5fcf640 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558c52e31310 0x558c52e33700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:45:48.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.688+0000 7f4cb5fcf640 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558c52e31310 0x558c52e33700 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce41f30dde73d01b server_cookie=b098ff89570144e in_seq=0 out_seq=0 2024-09-11T23:45:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.688+0000 7f4c997fa640 10 client.27112 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.691+0000 7f4c997fa640 1 -- 172.21.15.133:0/3089743230 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558c52c95210 con 0x558c52e31310 2024-09-11T23:45:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.691+0000 7f4c997fa640 10 client.27112 handle_command_reply: tid=0 2024-09-11T23:45:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.695+0000 7f4cba60c740 1 client.27112 shutdown 2024-09-11T23:45:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.696+0000 7f4cba60c740 1 -- 172.21.15.133:0/3089743230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dab4a0 msgr2=0x558c52dbed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.696+0000 7f4cba60c740 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dab4a0 0x558c52dbed20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4ca001ac60 tx=0x7f4ca000e7b0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.697+0000 7f4cba60c740 1 -- 172.21.15.133:0/3089743230 shutdown_connections 2024-09-11T23:45:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.697+0000 7f4cba60c740 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558c52e31310 0x558c52e33700 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.697+0000 7f4cba60c740 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c52dba4f0 0x558c52dba970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.697+0000 7f4cba60c740 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558c52db6680 0x558c52db9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.697+0000 7f4cba60c740 1 --2- 172.21.15.133:0/3089743230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c52dab4a0 0x558c52dbed20 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4ca001ac60 tx=0x7f4ca000e7b0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.697+0000 7f4cba60c740 1 -- 172.21.15.133:0/3089743230 >> 172.21.15.133:0/3089743230 conn(0x558c52da7a30 msgr2=0x558c52da6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.697+0000 7f4c997fa640 0 client.27112 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:45:48.701 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:45:48.701 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:45:48.701 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-09-11T23:45:48.701 INFO:teuthology.orchestra.run.smithi133.stdout: "76ff1c82-4bb6-4d19-a948-752500fcd905": { 2024-09-11T23:45:48.701 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-09-11T23:45:48.701 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "76ff1c82-4bb6-4d19-a948-752500fcd905", 2024-09-11T23:45:48.701 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2024-09-11T23:45:48.701 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:48.702 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:45:48.702 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:45:48.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.697+0000 7f4cba60c740 1 -- 172.21.15.133:0/3089743230 shutdown_connections 2024-09-11T23:45:48.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.698+0000 7f4cba60c740 1 -- 172.21.15.133:0/3089743230 wait complete. 2024-09-11T23:45:48.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.698+0000 7f4cba60c740 20 client.27112 trim_cache size 0 max 16384 2024-09-11T23:45:48.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.705+0000 7f4cb8233640 1 -- 172.21.15.133:0/170743916 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4c9c0a2e80 msgr2=0x7f4c9c0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:48.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.705+0000 7f4cb8233640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4c9c0a2e80 0x7f4c9c0a5340 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8011000 tx=0x7f4ca806be80 comp rx=0 tx=0).stop 2024-09-11T23:45:48.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.705+0000 7f4cb8233640 1 -- 172.21.15.133:0/170743916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb011b140 msgr2=0x7f4cb01144b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:45:48.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.705+0000 7f4cb8233640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb011b140 0x7f4cb01144b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800cab0 tx=0x7f4ca800cf70 comp rx=0 tx=0).stop 2024-09-11T23:45:48.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.705+0000 7f4cb8233640 1 -- 172.21.15.133:0/170743916 shutdown_connections 2024-09-11T23:45:48.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.705+0000 7f4cb8233640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4c9c0a2e80 0x7f4c9c0a5340 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.705+0000 7f4cb8233640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cb011b140 0x7f4cb01144b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.705+0000 7f4cb8233640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cb0109830 0x7f4cb011a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.705+0000 7f4cb8233640 1 --2- 172.21.15.133:0/170743916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cb0109450 0x7f4cb011a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:45:48.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.705+0000 7f4cb8233640 1 -- 172.21.15.133:0/170743916 >> 172.21.15.133:0/170743916 conn(0x7f4cb006e140 msgr2=0x7f4cb0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:45:48.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.708+0000 7f4cb8233640 1 -- 172.21.15.133:0/170743916 shutdown_connections 2024-09-11T23:45:48.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:45:48.709+0000 7f4cb8233640 1 -- 172.21.15.133:0/170743916 wait complete. 2024-09-11T23:45:49.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:48 smithi178 ceph-mon[35882]: pgmap v280: 97 pgs: 97 active+clean; 67 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 561 MiB/s wr, 2.86k op/s 2024-09-11T23:45:49.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:48 smithi138 ceph-mon[35142]: pgmap v280: 97 pgs: 97 active+clean; 67 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 561 MiB/s wr, 2.86k op/s 2024-09-11T23:45:49.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:48 smithi133 ceph-mon[31430]: pgmap v280: 97 pgs: 97 active+clean; 67 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 561 MiB/s wr, 2.86k op/s 2024-09-11T23:45:49.924 INFO:tasks.cephfs.filesystem:scrub status for tag:76ff1c82-4bb6-4d19-a948-752500fcd905 - {'path': '/', 'tag': '76ff1c82-4bb6-4d19-a948-752500fcd905', 'options': 'recursive,force'} 2024-09-11T23:45:50.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:49 smithi138 ceph-mon[35142]: pgmap v281: 97 pgs: 97 active+clean; 69 GiB data, 219 GiB used, 853 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 749 MiB/s wr, 2.89k op/s 2024-09-11T23:45:50.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:49 smithi133 ceph-mon[31430]: pgmap v281: 97 pgs: 97 active+clean; 69 GiB data, 219 GiB used, 853 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 749 MiB/s wr, 2.89k op/s 2024-09-11T23:45:50.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:49 smithi178 ceph-mon[35882]: pgmap v281: 97 pgs: 97 active+clean; 69 GiB data, 219 GiB used, 853 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 749 MiB/s wr, 2.89k op/s 2024-09-11T23:45:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:50 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:45:51.381 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:50 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:45:51.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:50 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:45:52.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:52 smithi178 ceph-mon[35882]: pgmap v282: 97 pgs: 97 active+clean; 69 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 614 MiB/s wr, 1.85k op/s 2024-09-11T23:45:52.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:52 smithi138 ceph-mon[35142]: pgmap v282: 97 pgs: 97 active+clean; 69 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 614 MiB/s wr, 1.85k op/s 2024-09-11T23:45:52.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:52 smithi133 ceph-mon[31430]: pgmap v282: 97 pgs: 97 active+clean; 69 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 614 MiB/s wr, 1.85k op/s 2024-09-11T23:45:54.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:54 smithi138 ceph-mon[35142]: pgmap v283: 97 pgs: 97 active+clean; 70 GiB data, 224 GiB used, 848 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 782 MiB/s wr, 2.18k op/s 2024-09-11T23:45:54.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:54 smithi133 ceph-mon[31430]: pgmap v283: 97 pgs: 97 active+clean; 70 GiB data, 224 GiB used, 848 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 782 MiB/s wr, 2.18k op/s 2024-09-11T23:45:55.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:54 smithi178 ceph-mon[35882]: pgmap v283: 97 pgs: 97 active+clean; 70 GiB data, 224 GiB used, 848 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 782 MiB/s wr, 2.18k op/s 2024-09-11T23:45:56.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:56 smithi133 ceph-mon[31430]: pgmap v284: 97 pgs: 97 active+clean; 70 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 693 MiB/s wr, 1.53k op/s 2024-09-11T23:45:57.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:56 smithi178 ceph-mon[35882]: pgmap v284: 97 pgs: 97 active+clean; 70 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 693 MiB/s wr, 1.53k op/s 2024-09-11T23:45:57.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:56 smithi138 ceph-mon[35142]: pgmap v284: 97 pgs: 97 active+clean; 70 GiB data, 228 GiB used, 845 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 693 MiB/s wr, 1.53k op/s 2024-09-11T23:45:59.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:58 smithi178 ceph-mon[35882]: pgmap v285: 97 pgs: 97 active+clean; 70 GiB data, 229 GiB used, 843 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 653 MiB/s wr, 1.26k op/s 2024-09-11T23:45:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:58 smithi138 ceph-mon[35142]: pgmap v285: 97 pgs: 97 active+clean; 70 GiB data, 229 GiB used, 843 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 653 MiB/s wr, 1.26k op/s 2024-09-11T23:45:59.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:58 smithi133 ceph-mon[31430]: pgmap v285: 97 pgs: 97 active+clean; 70 GiB data, 229 GiB used, 843 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 653 MiB/s wr, 1.26k op/s 2024-09-11T23:46:00.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:45:59 smithi138 ceph-mon[35142]: pgmap v286: 97 pgs: 97 active+clean; 70 GiB data, 234 GiB used, 839 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 841 MiB/s wr, 1.49k op/s 2024-09-11T23:46:00.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:45:59 smithi133 ceph-mon[31430]: pgmap v286: 97 pgs: 97 active+clean; 70 GiB data, 234 GiB used, 839 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 841 MiB/s wr, 1.49k op/s 2024-09-11T23:46:00.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:45:59 smithi178 ceph-mon[35882]: pgmap v286: 97 pgs: 97 active+clean; 70 GiB data, 234 GiB used, 839 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 841 MiB/s wr, 1.49k op/s 2024-09-11T23:46:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.034+0000 7fa386220640 1 -- 172.21.15.133:0/2577988735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa38011b140 msgr2=0x7fa38011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.034+0000 7fa386220640 1 --2- 172.21.15.133:0/2577988735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa38011b140 0x7fa38011d530 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa37001a910 tx=0x7fa37003b470 comp rx=0 tx=0).stop 2024-09-11T23:46:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.035+0000 7fa386220640 1 -- 172.21.15.133:0/2577988735 shutdown_connections 2024-09-11T23:46:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.035+0000 7fa386220640 1 --2- 172.21.15.133:0/2577988735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa38011b140 0x7fa38011d530 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.035+0000 7fa386220640 1 --2- 172.21.15.133:0/2577988735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3801095d0 0x7fa380109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.035+0000 7fa386220640 1 --2- 172.21.15.133:0/2577988735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3801091f0 0x7fa380110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.035+0000 7fa386220640 1 -- 172.21.15.133:0/2577988735 >> 172.21.15.133:0/2577988735 conn(0x7fa38006cbf0 msgr2=0x7fa38006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.037+0000 7fa386220640 1 -- 172.21.15.133:0/2577988735 shutdown_connections 2024-09-11T23:46:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.040+0000 7fa386220640 1 -- 172.21.15.133:0/2577988735 wait complete. 2024-09-11T23:46:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.045+0000 7fa386220640 1 Processor -- start 2024-09-11T23:46:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.045+0000 7fa386220640 1 -- start start 2024-09-11T23:46:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.045+0000 7fa386220640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3801091f0 0x7fa38011a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.045+0000 7fa386220640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3801095d0 0x7fa380116960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.045+0000 7fa386220640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38011b140 0x7fa380116ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.046+0000 7fa386220640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa380113400 con 0x7fa3801095d0 2024-09-11T23:46:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.046+0000 7fa386220640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa380113280 con 0x7fa3801091f0 2024-09-11T23:46:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.046+0000 7fa386220640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa380113580 con 0x7fa38011b140 2024-09-11T23:46:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.046+0000 7fa385a1f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38011b140 0x7fa380116ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.046+0000 7fa385a1f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38011b140 0x7fa380116ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:43568/0 (socket says 172.21.15.133:43568) 2024-09-11T23:46:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.046+0000 7fa385a1f640 1 -- 172.21.15.133:0/2357873551 learned_addr learned my addr 172.21.15.133:0/2357873551 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.046+0000 7fa38521e640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3801091f0 0x7fa38011a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.047+0000 7fa385a1f640 1 -- 172.21.15.133:0/2357873551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3801091f0 msgr2=0x7fa38011a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.047+0000 7fa385a1f640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3801091f0 0x7fa38011a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.047+0000 7fa385a1f640 1 -- 172.21.15.133:0/2357873551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3801095d0 msgr2=0x7fa380116960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.047+0000 7fa385a1f640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3801095d0 0x7fa380116960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.047+0000 7fa385a1f640 1 -- 172.21.15.133:0/2357873551 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3801175d0 con 0x7fa38011b140 2024-09-11T23:46:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.048+0000 7fa385a1f640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38011b140 0x7fa380116ea0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa370005ab0 tx=0x7fa370005370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.048+0000 7fa3767fc640 1 -- 172.21.15.133:0/2357873551 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37004f020 con 0x7fa38011b140 2024-09-11T23:46:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.048+0000 7fa3767fc640 1 -- 172.21.15.133:0/2357873551 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa370042400 con 0x7fa38011b140 2024-09-11T23:46:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.048+0000 7fa386220640 1 -- 172.21.15.133:0/2357873551 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38007a450 con 0x7fa38011b140 2024-09-11T23:46:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.049+0000 7fa3767fc640 1 -- 172.21.15.133:0/2357873551 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370046070 con 0x7fa38011b140 2024-09-11T23:46:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.049+0000 7fa386220640 1 -- 172.21.15.133:0/2357873551 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa38007a990 con 0x7fa38011b140 2024-09-11T23:46:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.053+0000 7fa38a220740 1 Processor -- start 2024-09-11T23:46:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.053+0000 7fa38a220740 1 -- start start 2024-09-11T23:46:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.054+0000 7fa38a220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc38ccc0 0x559cbc38d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.054+0000 7fa38a220740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 0x559cbc397940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.054+0000 7fa38a220740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc397e80 0x559cbc39a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.054+0000 7fa38a220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559cbc2f12f0 con 0x559cbc38ccc0 2024-09-11T23:46:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.054+0000 7fa38a220740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559cbc299270 con 0x559cbc38d5e0 2024-09-11T23:46:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.054+0000 7fa38a220740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559cbc2ed2b0 con 0x559cbc397e80 2024-09-11T23:46:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.055+0000 7fa38521e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 0x559cbc397940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.055+0000 7fa38521e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 0x559cbc397940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33616/0 (socket says 172.21.15.133:33616) 2024-09-11T23:46:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.055+0000 7fa38521e640 1 -- 172.21.15.133:0/800928460 learned_addr learned my addr 172.21.15.133:0/800928460 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.056+0000 7fa384a1d640 1 --2- 172.21.15.133:0/800928460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc397e80 0x559cbc39a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.057+0000 7fa384a1d640 1 --2- 172.21.15.133:0/800928460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc38ccc0 0x559cbc38d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.057+0000 7fa384a1d640 1 -- 172.21.15.133:0/800928460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc397e80 msgr2=0x559cbc39a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.057+0000 7fa384a1d640 1 --2- 172.21.15.133:0/800928460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc397e80 0x559cbc39a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.057+0000 7fa384a1d640 1 -- 172.21.15.133:0/800928460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 msgr2=0x559cbc397940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.057+0000 7fa384a1d640 1 --2- 172.21.15.133:0/800928460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 0x559cbc397940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.057+0000 7fa384a1d640 1 -- 172.21.15.133:0/800928460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cbc298340 con 0x559cbc38ccc0 2024-09-11T23:46:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.062+0000 7fa3767fc640 1 -- 172.21.15.133:0/2357873551 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa370052020 con 0x7fa38011b140 2024-09-11T23:46:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.063+0000 7fa3767fc640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa36c0a2e30 0x7fa36c0a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.064+0000 7fa3767fc640 1 -- 172.21.15.133:0/2357873551 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fa370100450 con 0x7fa38011b140 2024-09-11T23:46:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.064+0000 7fa384a1d640 1 --2- 172.21.15.133:0/800928460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc38ccc0 0x559cbc38d0a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa3801181b0 tx=0x7fa378006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfaccd2c151fe0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.065+0000 7fa38521e640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa36c0a2e30 0x7fa36c0a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.065+0000 7fa36b7fe640 1 -- 172.21.15.133:0/800928460 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378003a10 con 0x559cbc38ccc0 2024-09-11T23:46:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.066+0000 7fa36b7fe640 1 -- 172.21.15.133:0/800928460 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa378003bb0 con 0x559cbc38ccc0 2024-09-11T23:46:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.068+0000 7fa38a220740 1 -- 172.21.15.133:0/800928460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc38ccc0 msgr2=0x559cbc38d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.068+0000 7fa38a220740 1 --2- 172.21.15.133:0/800928460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc38ccc0 0x559cbc38d0a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa3801181b0 tx=0x7fa378006ca0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.069+0000 7fa38521e640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa36c0a2e30 0x7fa36c0a52f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x559cbc38a090 tx=0x7fa37c009110 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.071+0000 7fa38a220740 1 -- 172.21.15.133:0/800928460 shutdown_connections 2024-09-11T23:46:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.071+0000 7fa38a220740 1 --2- 172.21.15.133:0/800928460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc397e80 0x559cbc39a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.071+0000 7fa38a220740 1 --2- 172.21.15.133:0/800928460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 0x559cbc397940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.071+0000 7fa38a220740 1 --2- 172.21.15.133:0/800928460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc38ccc0 0x559cbc38d0a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.071+0000 7fa38a220740 1 -- 172.21.15.133:0/800928460 >> 172.21.15.133:0/800928460 conn(0x559cbc389230 msgr2=0x559cbc389610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.072+0000 7fa38a220740 1 -- 172.21.15.133:0/800928460 shutdown_connections 2024-09-11T23:46:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.072+0000 7fa38a220740 1 -- 172.21.15.133:0/800928460 wait complete. 2024-09-11T23:46:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.072+0000 7fa38a220740 1 -- start start 2024-09-11T23:46:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.072+0000 7fa38a220740 1 Processor -- start 2024-09-11T23:46:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.073+0000 7fa38a220740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38ccc0 0x559cbc394ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.073+0000 7fa38a220740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 0x559cbc3951e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.073+0000 7fa38a220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc395720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.073+0000 7fa38a220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559cbc39aaa0 con 0x559cbc397e80 2024-09-11T23:46:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.073+0000 7fa38a220740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559cbc27a290 con 0x559cbc38d5e0 2024-09-11T23:46:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.073+0000 7fa38a220740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559cbc39ac20 con 0x559cbc38ccc0 2024-09-11T23:46:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.073+0000 7fa38a220740 10 client.0 _ll_register_callbacks cb 0x559cbc2ab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.074+0000 7fa384a1d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 0x559cbc3951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.074+0000 7fa384a1d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 0x559cbc3951e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:33630/0 (socket says 172.21.15.133:33630) 2024-09-11T23:46:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.074+0000 7fa384a1d640 1 -- 172.21.15.133:0/3306780901 learned_addr learned my addr 172.21.15.133:0/3306780901 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.075+0000 7fa384a1d640 1 -- 172.21.15.133:0/3306780901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38ccc0 msgr2=0x559cbc394ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:46:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.075+0000 7fa384a1d640 1 --2- 172.21.15.133:0/3306780901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38ccc0 0x559cbc394ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.075+0000 7fa384a1d640 1 -- 172.21.15.133:0/3306780901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 msgr2=0x559cbc395720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.075+0000 7fa384a1d640 1 --2- 172.21.15.133:0/3306780901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc395720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.075+0000 7fa384a1d640 1 -- 172.21.15.133:0/3306780901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cbc004210 con 0x559cbc38d5e0 2024-09-11T23:46:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.076+0000 7fa384a1d640 1 --2- 172.21.15.133:0/3306780901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 0x559cbc3951e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa378004730 tx=0x7fa378003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebf192cd51f75c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.092+0000 7fa368ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:46:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.092+0000 7fa368ff9640 1 -- 172.21.15.133:0/3306780901 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378003a10 con 0x559cbc38d5e0 2024-09-11T23:46:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.092+0000 7fa368ff9640 1 -- 172.21.15.133:0/3306780901 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fa378022420 con 0x559cbc38d5e0 2024-09-11T23:46:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.092+0000 7fa38a220740 1 -- 172.21.15.133:0/3306780901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa378003bb0 con 0x559cbc38d5e0 2024-09-11T23:46:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.094+0000 7fa368ff9640 1 -- 172.21.15.133:0/3306780901 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa378022620 con 0x559cbc38d5e0 2024-09-11T23:46:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.095+0000 7fa38a220740 10 client.27314 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.095+0000 7fa38a220740 1 -- 172.21.15.133:0/3306780901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559cbc0d0620 con 0x559cbc38d5e0 2024-09-11T23:46:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.095+0000 7fa368ff9640 1 -- 172.21.15.133:0/3306780901 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa378003bb0 con 0x559cbc38d5e0 2024-09-11T23:46:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.096+0000 7fa368ff9640 1 -- 172.21.15.133:0/3306780901 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa37801a440 con 0x559cbc38d5e0 2024-09-11T23:46:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.096+0000 7fa38a220740 10 client.27314 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.096+0000 7fa38a220740 10 client.27314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.096+0000 7fa38a220740 20 client.27314 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.096+0000 7fa38a220740 1 --2- 172.21.15.133:0/3306780901 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559cbc3a7d30 0x559cbc3aa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.096+0000 7fa38a220740 4 client.27314 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.096+0000 7fa38a220740 1 -- 172.21.15.133:0/3306780901 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cbc2fea20 con 0x559cbc3a7d30 2024-09-11T23:46:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.097+0000 7fa38521e640 1 --2- 172.21.15.133:0/3306780901 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559cbc3a7d30 0x559cbc3aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.100+0000 7fa38521e640 1 --2- 172.21.15.133:0/3306780901 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559cbc3a7d30 0x559cbc3aa120 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccbb629691a0e57f server_cookie=844f8583f051776b in_seq=0 out_seq=0 2024-09-11T23:46:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.100+0000 7fa368ff9640 10 client.27314 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.102+0000 7fa368ff9640 1 -- 172.21.15.133:0/3306780901 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559cbc2fea20 con 0x559cbc3a7d30 2024-09-11T23:46:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.102+0000 7fa368ff9640 10 client.27314 handle_command_reply: tid=0 2024-09-11T23:46:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.102+0000 7fa38a220740 1 client.27314 shutdown 2024-09-11T23:46:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.107+0000 7fa38a220740 1 -- 172.21.15.133:0/3306780901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 msgr2=0x559cbc3951e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.107+0000 7fa38a220740 1 --2- 172.21.15.133:0/3306780901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 0x559cbc3951e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa378004730 tx=0x7fa378003f90 comp rx=0 tx=0).stop 2024-09-11T23:46:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.107+0000 7fa38a220740 1 -- 172.21.15.133:0/3306780901 shutdown_connections 2024-09-11T23:46:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.107+0000 7fa38a220740 1 --2- 172.21.15.133:0/3306780901 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559cbc3a7d30 0x559cbc3aa120 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.107+0000 7fa38a220740 1 --2- 172.21.15.133:0/3306780901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc395720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.107+0000 7fa38a220740 1 --2- 172.21.15.133:0/3306780901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38d5e0 0x559cbc3951e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.107+0000 7fa38a220740 1 --2- 172.21.15.133:0/3306780901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38ccc0 0x559cbc394ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.107+0000 7fa368ff9640 0 client.27314 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.107+0000 7fa38a220740 1 -- 172.21.15.133:0/3306780901 >> 172.21.15.133:0/3306780901 conn(0x559cbc389230 msgr2=0x559cbc38ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.111+0000 7fa38a220740 1 -- 172.21.15.133:0/3306780901 shutdown_connections 2024-09-11T23:46:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.111+0000 7fa38a220740 1 -- 172.21.15.133:0/3306780901 wait complete. 2024-09-11T23:46:01.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.113+0000 7fa38a220740 20 client.27314 trim_cache size 0 max 16384 2024-09-11T23:46:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.148+0000 7fa38a220740 1 Processor -- start 2024-09-11T23:46:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.148+0000 7fa38a220740 1 -- start start 2024-09-11T23:46:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.149+0000 7fa38a220740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38ccc0 0x559cbc395550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.149+0000 7fa38a220740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38d5e0 0x559cbc3af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.149+0000 7fa38a220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc3afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.149+0000 7fa38a220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa36400e7d0 con 0x559cbc397e80 2024-09-11T23:46:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.149+0000 7fa38a220740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa364010310 con 0x559cbc38ccc0 2024-09-11T23:46:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.150+0000 7fa384a1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38d5e0 0x559cbc3af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.150+0000 7fa384a1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38d5e0 0x559cbc3af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:43604/0 (socket says 172.21.15.133:43604) 2024-09-11T23:46:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.150+0000 7fa384a1d640 1 -- 172.21.15.133:0/1277442120 learned_addr learned my addr 172.21.15.133:0/1277442120 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.150+0000 7fa38a220740 1 -- 172.21.15.133:0/1277442120 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa36400eea0 con 0x559cbc38d5e0 2024-09-11T23:46:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.154+0000 7fa384a1d640 1 -- 172.21.15.133:0/1277442120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38ccc0 msgr2=0x559cbc395550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.154+0000 7fa384a1d640 1 --2- 172.21.15.133:0/1277442120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38ccc0 0x559cbc395550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.154+0000 7fa384a1d640 1 -- 172.21.15.133:0/1277442120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 msgr2=0x559cbc3afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.154+0000 7fa384a1d640 1 --2- 172.21.15.133:0/1277442120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc3afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.154+0000 7fa384a1d640 1 -- 172.21.15.133:0/1277442120 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cbc389ec0 con 0x559cbc38d5e0 2024-09-11T23:46:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.156+0000 7fa384a1d640 1 --2- 172.21.15.133:0/1277442120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38d5e0 0x559cbc3af8d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa378000f80 tx=0x7fa37800d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=845ddd8d4b838513 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.156+0000 7fa36affd640 1 -- 172.21.15.133:0/1277442120 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378039070 con 0x559cbc38d5e0 2024-09-11T23:46:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.159+0000 7fa36affd640 1 -- 172.21.15.133:0/1277442120 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa378011040 con 0x559cbc38d5e0 2024-09-11T23:46:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.159+0000 7fa36affd640 1 -- 172.21.15.133:0/1277442120 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378004410 con 0x559cbc38d5e0 2024-09-11T23:46:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.161+0000 7fa38a220740 1 -- 172.21.15.133:0/1277442120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38d5e0 msgr2=0x559cbc3af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.161+0000 7fa38a220740 1 --2- 172.21.15.133:0/1277442120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38d5e0 0x559cbc3af8d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa378000f80 tx=0x7fa37800d4a0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.162+0000 7fa38a220740 1 -- 172.21.15.133:0/1277442120 shutdown_connections 2024-09-11T23:46:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.162+0000 7fa38a220740 1 --2- 172.21.15.133:0/1277442120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc3afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.162+0000 7fa38a220740 1 --2- 172.21.15.133:0/1277442120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38d5e0 0x559cbc3af8d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.162+0000 7fa38a220740 1 --2- 172.21.15.133:0/1277442120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38ccc0 0x559cbc395550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.162+0000 7fa38a220740 1 -- 172.21.15.133:0/1277442120 >> 172.21.15.133:0/1277442120 conn(0x559cbc389230 msgr2=0x559cbc393d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.164+0000 7fa38a220740 1 -- 172.21.15.133:0/1277442120 shutdown_connections 2024-09-11T23:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.164+0000 7fa38a220740 1 -- 172.21.15.133:0/1277442120 wait complete. 2024-09-11T23:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.165+0000 7fa38a220740 1 -- start start 2024-09-11T23:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.165+0000 7fa38a220740 1 Processor -- start 2024-09-11T23:46:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.166+0000 7fa38a220740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38ccc0 0x559cbc3a0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.166+0000 7fa38a220740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38d5e0 0x559cbc3a0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.166+0000 7fa38a220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc391840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.166+0000 7fa38a220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa36400fc40 con 0x559cbc397e80 2024-09-11T23:46:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.166+0000 7fa38a220740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa36400f570 con 0x559cbc38ccc0 2024-09-11T23:46:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.166+0000 7fa38a220740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa36400c7d0 con 0x559cbc38d5e0 2024-09-11T23:46:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.166+0000 7fa38a220740 10 client.0 _ll_register_callbacks cb 0x7fa37c0094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.167+0000 7fa385a1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc391840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.167+0000 7fa38521e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38ccc0 0x559cbc3a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.168+0000 7fa385a1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc391840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46526/0 (socket says 172.21.15.133:46526) 2024-09-11T23:46:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.168+0000 7fa385a1f640 1 -- 172.21.15.133:0/63869048 learned_addr learned my addr 172.21.15.133:0/63869048 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.171+0000 7fa385a1f640 1 -- 172.21.15.133:0/63869048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38d5e0 msgr2=0x559cbc3a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.171+0000 7fa385a1f640 1 --2- 172.21.15.133:0/63869048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38d5e0 0x559cbc3a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.171+0000 7fa385a1f640 1 -- 172.21.15.133:0/63869048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38ccc0 msgr2=0x559cbc3a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.171+0000 7fa385a1f640 1 --2- 172.21.15.133:0/63869048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38ccc0 0x559cbc3a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.171+0000 7fa385a1f640 1 -- 172.21.15.133:0/63869048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cbc392070 con 0x559cbc397e80 2024-09-11T23:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.172+0000 7fa385a1f640 1 --2- 172.21.15.133:0/63869048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc391840 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa3700fdbe0 tx=0x7fa3700cc7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af58a4750bca29df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.172+0000 7fa368ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:46:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.174+0000 7fa368ff9640 1 -- 172.21.15.133:0/63869048 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3700cd050 con 0x559cbc397e80 2024-09-11T23:46:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.174+0000 7fa368ff9640 1 -- 172.21.15.133:0/63869048 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fa3700fde00 con 0x559cbc397e80 2024-09-11T23:46:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.175+0000 7fa38a220740 1 -- 172.21.15.133:0/63869048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa378011040 con 0x559cbc397e80 2024-09-11T23:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.176+0000 7fa368ff9640 1 -- 172.21.15.133:0/63869048 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa3700fe000 con 0x559cbc397e80 2024-09-11T23:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.179+0000 7fa38a220740 10 client.17811 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.179+0000 7fa38a220740 1 -- 172.21.15.133:0/63869048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559cbc39fd40 con 0x559cbc397e80 2024-09-11T23:46:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.181+0000 7fa368ff9640 1 -- 172.21.15.133:0/63869048 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa378011040 con 0x559cbc397e80 2024-09-11T23:46:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.181+0000 7fa368ff9640 1 -- 172.21.15.133:0/63869048 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa3700b70d0 con 0x559cbc397e80 2024-09-11T23:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.182+0000 7fa38a220740 10 client.17811 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.182+0000 7fa38a220740 10 client.17811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.182+0000 7fa38a220740 20 client.17811 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.182+0000 7fa38a220740 1 --2- 172.21.15.133:0/63869048 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559cbc412c30 0x559cbc415050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.182+0000 7fa38a220740 4 client.17811 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.182+0000 7fa38a220740 1 -- 172.21.15.133:0/63869048 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559cbc2f8d70 con 0x559cbc412c30 2024-09-11T23:46:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.188+0000 7fa38521e640 1 --2- 172.21.15.133:0/63869048 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559cbc412c30 0x559cbc415050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.192+0000 7fa368ff9640 10 client.17811 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:01.192+0000 7fa38521e640 1 --2- 172.21.15.133:0/63869048 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559cbc412c30 0x559cbc415050 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8f1e21c7ca360c2 server_cookie=1c42ce5fdaeb2664 in_seq=0 out_seq=0 2024-09-11T23:46:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.030+0000 7fa368ff9640 1 -- 172.21.15.133:0/63869048 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559cbc2f8d70 con 0x559cbc412c30 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.030+0000 7fa368ff9640 10 client.17811 handle_command_reply: tid=0 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.030+0000 7fa38a220740 1 client.17811 shutdown 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.030+0000 7fa38a220740 1 -- 172.21.15.133:0/63869048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 msgr2=0x559cbc391840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.030+0000 7fa38a220740 1 --2- 172.21.15.133:0/63869048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc391840 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa3700fdbe0 tx=0x7fa3700cc7e0 comp rx=0 tx=0).stop 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.030+0000 7fa38a220740 1 -- 172.21.15.133:0/63869048 shutdown_connections 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.030+0000 7fa38a220740 1 --2- 172.21.15.133:0/63869048 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559cbc412c30 0x559cbc415050 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.030+0000 7fa38a220740 1 --2- 172.21.15.133:0/63869048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cbc397e80 0x559cbc391840 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.030+0000 7fa38a220740 1 --2- 172.21.15.133:0/63869048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559cbc38d5e0 0x559cbc3a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.030+0000 7fa38a220740 1 --2- 172.21.15.133:0/63869048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbc38ccc0 0x559cbc3a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.030+0000 7fa38a220740 1 -- 172.21.15.133:0/63869048 >> 172.21.15.133:0/63869048 conn(0x559cbc389230 msgr2=0x559cbc395560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.031+0000 7fa368ff9640 0 client.17811 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.031+0000 7fa38a220740 1 -- 172.21.15.133:0/63869048 shutdown_connections 2024-09-11T23:46:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.031+0000 7fa38a220740 1 -- 172.21.15.133:0/63869048 wait complete. 2024-09-11T23:46:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.031+0000 7fa38a220740 20 client.17811 trim_cache size 0 max 16384 2024-09-11T23:46:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:46:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:46:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:46:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:46:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 -- 172.21.15.133:0/2357873551 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa36c0a2e30 msgr2=0x7fa36c0a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa36c0a2e30 0x7fa36c0a52f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x559cbc38a090 tx=0x7fa37c009110 comp rx=0 tx=0).stop 2024-09-11T23:46:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 -- 172.21.15.133:0/2357873551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38011b140 msgr2=0x7fa380116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38011b140 0x7fa380116ea0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa370005ab0 tx=0x7fa370005370 comp rx=0 tx=0).stop 2024-09-11T23:46:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 -- 172.21.15.133:0/2357873551 shutdown_connections 2024-09-11T23:46:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa36c0a2e30 0x7fa36c0a52f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38011b140 0x7fa380116ea0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3801095d0 0x7fa380116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 --2- 172.21.15.133:0/2357873551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3801091f0 0x7fa38011a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 -- 172.21.15.133:0/2357873551 >> 172.21.15.133:0/2357873551 conn(0x7fa38006cbf0 msgr2=0x7fa380070600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 -- 172.21.15.133:0/2357873551 shutdown_connections 2024-09-11T23:46:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:02.032+0000 7fa386220640 1 -- 172.21.15.133:0/2357873551 wait complete. 2024-09-11T23:46:02.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:02 smithi138 ceph-mon[35142]: pgmap v287: 97 pgs: 97 active+clean; 70 GiB data, 235 GiB used, 838 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 669 MiB/s wr, 1.35k op/s 2024-09-11T23:46:02.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:02 smithi133 ceph-mon[31430]: pgmap v287: 97 pgs: 97 active+clean; 70 GiB data, 235 GiB used, 838 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 669 MiB/s wr, 1.35k op/s 2024-09-11T23:46:02.991 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:02 smithi178 ceph-mon[35882]: pgmap v287: 97 pgs: 97 active+clean; 70 GiB data, 235 GiB used, 838 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 669 MiB/s wr, 1.35k op/s 2024-09-11T23:46:04.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:03 smithi138 ceph-mon[35142]: pgmap v288: 97 pgs: 97 active+clean; 70 GiB data, 237 GiB used, 835 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 839 MiB/s wr, 1.70k op/s 2024-09-11T23:46:04.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:03 smithi133 ceph-mon[31430]: pgmap v288: 97 pgs: 97 active+clean; 70 GiB data, 237 GiB used, 835 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 839 MiB/s wr, 1.70k op/s 2024-09-11T23:46:04.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:03 smithi178 ceph-mon[35882]: pgmap v288: 97 pgs: 97 active+clean; 70 GiB data, 237 GiB used, 835 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 839 MiB/s wr, 1.70k op/s 2024-09-11T23:46:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:04 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:05.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:04 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:04 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:46:06.201 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:05 smithi138 ceph-mon[35142]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:46:06.201 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:05 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:06.201 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:05 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:06.202 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:05 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:46:06.202 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:05 smithi138 ceph-mon[35142]: pgmap v290: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 70 GiB data, 240 GiB used, 832 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 749 MiB/s wr, 1.33k op/s 2024-09-11T23:46:06.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:05 smithi133 ceph-mon[31430]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:46:06.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:05 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:06.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:05 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:06.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:05 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:46:06.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:05 smithi133 ceph-mon[31430]: pgmap v290: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 70 GiB data, 240 GiB used, 832 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 749 MiB/s wr, 1.33k op/s 2024-09-11T23:46:06.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:05 smithi178 ceph-mon[35882]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:46:06.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:05 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:46:06.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:05 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:46:06.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:05 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:46:06.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:05 smithi178 ceph-mon[35882]: pgmap v290: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 70 GiB data, 240 GiB used, 832 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 749 MiB/s wr, 1.33k op/s 2024-09-11T23:46:08.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:08 smithi138 ceph-mon[35142]: pgmap v291: 97 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 78 active+clean; 70 GiB data, 242 GiB used, 831 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 745 MiB/s wr, 1.55k op/s 2024-09-11T23:46:08.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:08 smithi133 ceph-mon[31430]: pgmap v291: 97 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 78 active+clean; 70 GiB data, 242 GiB used, 831 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 745 MiB/s wr, 1.55k op/s 2024-09-11T23:46:09.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:08 smithi178 ceph-mon[35882]: pgmap v291: 97 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 78 active+clean; 70 GiB data, 242 GiB used, 831 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 745 MiB/s wr, 1.55k op/s 2024-09-11T23:46:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:10 smithi178 ceph-mon[35882]: pgmap v292: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 66 GiB data, 240 GiB used, 832 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 699 MiB/s wr, 2.65k op/s 2024-09-11T23:46:11.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:10 smithi138 ceph-mon[35142]: pgmap v292: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 66 GiB data, 240 GiB used, 832 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 699 MiB/s wr, 2.65k op/s 2024-09-11T23:46:11.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:10 smithi133 ceph-mon[31430]: pgmap v292: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 66 GiB data, 240 GiB used, 832 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 699 MiB/s wr, 2.65k op/s 2024-09-11T23:46:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:11 smithi178 ceph-mon[35882]: pgmap v293: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 65 GiB data, 241 GiB used, 832 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 688 MiB/s wr, 2.83k op/s 2024-09-11T23:46:12.117 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:11 smithi133 ceph-mon[31430]: pgmap v293: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 65 GiB data, 241 GiB used, 832 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 688 MiB/s wr, 2.83k op/s 2024-09-11T23:46:12.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:11 smithi138 ceph-mon[35142]: pgmap v293: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 65 GiB data, 241 GiB used, 832 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 688 MiB/s wr, 2.83k op/s 2024-09-11T23:46:14.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:13 smithi138 ceph-mon[35142]: pgmap v294: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 64 GiB data, 239 GiB used, 834 GiB / 1.0 TiB avail; 469 KiB/s rd, 639 MiB/s wr, 4.04k op/s 2024-09-11T23:46:14.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:13 smithi133 ceph-mon[31430]: pgmap v294: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 64 GiB data, 239 GiB used, 834 GiB / 1.0 TiB avail; 469 KiB/s rd, 639 MiB/s wr, 4.04k op/s 2024-09-11T23:46:14.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:13 smithi178 ceph-mon[35882]: pgmap v294: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 64 GiB data, 239 GiB used, 834 GiB / 1.0 TiB avail; 469 KiB/s rd, 639 MiB/s wr, 4.04k op/s 2024-09-11T23:46:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.252+0000 7f0986d2c640 1 -- 172.21.15.133:0/2208281402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09780a4850 msgr2=0x7f09780a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.252+0000 7f0986d2c640 1 --2- 172.21.15.133:0/2208281402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09780a4850 0x7f09780a4c30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f0974009860 tx=0x7f097402a410 comp rx=0 tx=0).stop 2024-09-11T23:46:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.253+0000 7f0986d2c640 1 -- 172.21.15.133:0/2208281402 shutdown_connections 2024-09-11T23:46:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.253+0000 7f0986d2c640 1 --2- 172.21.15.133:0/2208281402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09780b7570 0x7f09780b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.253+0000 7f0986d2c640 1 --2- 172.21.15.133:0/2208281402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09780a5170 0x7f09780b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.253+0000 7f0986d2c640 1 --2- 172.21.15.133:0/2208281402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09780a4850 0x7f09780a4c30 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.253+0000 7f0986d2c640 1 -- 172.21.15.133:0/2208281402 >> 172.21.15.133:0/2208281402 conn(0x7f097801a140 msgr2=0x7f097801a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.253+0000 7f0986d2c640 1 -- 172.21.15.133:0/2208281402 shutdown_connections 2024-09-11T23:46:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.253+0000 7f0986d2c640 1 -- 172.21.15.133:0/2208281402 wait complete. 2024-09-11T23:46:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.254+0000 7f0986d2c640 1 Processor -- start 2024-09-11T23:46:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.255+0000 7f0986d2c640 1 -- start start 2024-09-11T23:46:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.255+0000 7f0986d2c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09780a5170 0x7f09781432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.255+0000 7f0986d2c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09780b7570 0x7f0978143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.255+0000 7f0986d2c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0978147bb0 0x7f0978148060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.255+0000 7f0986d2c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09780bd000 con 0x7f09780b7570 2024-09-11T23:46:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.255+0000 7f0986d2c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09780bce80 con 0x7f0978147bb0 2024-09-11T23:46:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.255+0000 7f0986d2c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09780bd180 con 0x7f09780a5170 2024-09-11T23:46:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.257+0000 7f09852c9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0978147bb0 0x7f0978148060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.257+0000 7f09852c9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0978147bb0 0x7f0978148060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59078/0 (socket says 172.21.15.133:59078) 2024-09-11T23:46:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.257+0000 7f09852c9640 1 -- 172.21.15.133:0/4174388560 learned_addr learned my addr 172.21.15.133:0/4174388560 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.257+0000 7f097ffff640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09780b7570 0x7f0978143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.257+0000 7f0984ac8640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09780a5170 0x7f09781432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.258+0000 7f09852c9640 1 -- 172.21.15.133:0/4174388560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09780a5170 msgr2=0x7f09781432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.258+0000 7f09852c9640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09780a5170 0x7f09781432f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.258+0000 7f09852c9640 1 -- 172.21.15.133:0/4174388560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09780b7570 msgr2=0x7f0978143830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.258+0000 7f09852c9640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09780b7570 0x7f0978143830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.258+0000 7f09852c9640 1 -- 172.21.15.133:0/4174388560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0978148790 con 0x7f0978147bb0 2024-09-11T23:46:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.259+0000 7f09852c9640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0978147bb0 0x7f0978148060 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0980067b40 tx=0x7f0980073d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.260+0000 7f097dffb640 1 -- 172.21.15.133:0/4174388560 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980071c90 con 0x7f0978147bb0 2024-09-11T23:46:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.260+0000 7f0986d2c640 1 -- 172.21.15.133:0/4174388560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097800b810 con 0x7f0978147bb0 2024-09-11T23:46:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.261+0000 7f0986d2c640 1 -- 172.21.15.133:0/4174388560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097800bce0 con 0x7f0978147bb0 2024-09-11T23:46:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.261+0000 7f097dffb640 1 -- 172.21.15.133:0/4174388560 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0980071e30 con 0x7f0978147bb0 2024-09-11T23:46:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.261+0000 7f097dffb640 1 -- 172.21.15.133:0/4174388560 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980074820 con 0x7f0978147bb0 2024-09-11T23:46:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.271+0000 7f097dffb640 1 -- 172.21.15.133:0/4174388560 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f09800749c0 con 0x7f0978147bb0 2024-09-11T23:46:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.273+0000 7f097dffb640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09700a2f00 0x7f09700a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.274+0000 7f0984ac8640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09700a2f00 0x7f09700a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.274+0000 7f097dffb640 1 -- 172.21.15.133:0/4174388560 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f0980134790 con 0x7f0978147bb0 2024-09-11T23:46:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.279+0000 7f0989ac0740 1 Processor -- start 2024-09-11T23:46:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.279+0000 7f0984ac8640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09700a2f00 0x7f09700a53c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f09740092e0 tx=0x7f0974005d60 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.279+0000 7f0989ac0740 1 -- start start 2024-09-11T23:46:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.280+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587683ffcc0 0x5587684000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.280+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587684005e0 0x55876840a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.280+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 0x55876840d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.281+0000 7f0989ac0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5587683642f0 con 0x55876840ae80 2024-09-11T23:46:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.281+0000 7f0989ac0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55876830c270 con 0x5587684005e0 2024-09-11T23:46:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.281+0000 7f0989ac0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5587683602b0 con 0x5587683ffcc0 2024-09-11T23:46:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.281+0000 7f0984ac8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587683ffcc0 0x5587684000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.281+0000 7f0984ac8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587683ffcc0 0x5587684000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:43930/0 (socket says 172.21.15.133:43930) 2024-09-11T23:46:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.281+0000 7f0984ac8640 1 -- 172.21.15.133:0/1183923038 learned_addr learned my addr 172.21.15.133:0/1183923038 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.281+0000 7f097ffff640 1 --2- 172.21.15.133:0/1183923038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587684005e0 0x55876840a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.282+0000 7f097ffff640 1 -- 172.21.15.133:0/1183923038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587683ffcc0 msgr2=0x5587684000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:15.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.282+0000 7f097ffff640 1 --2- 172.21.15.133:0/1183923038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587683ffcc0 0x5587684000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.282+0000 7f097ffff640 1 -- 172.21.15.133:0/1183923038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 msgr2=0x55876840d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.282+0000 7f097ffff640 1 --2- 172.21.15.133:0/1183923038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 0x55876840d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.282+0000 7f097ffff640 1 -- 172.21.15.133:0/1183923038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876830b340 con 0x5587684005e0 2024-09-11T23:46:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.283+0000 7f097ffff640 1 --2- 172.21.15.133:0/1183923038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587684005e0 0x55876840a940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f096c00bbc0 tx=0x7f096c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c985f041cc3700 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.292+0000 7f096affd640 1 -- 172.21.15.133:0/1183923038 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c004030 con 0x5587684005e0 2024-09-11T23:46:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.292+0000 7f096affd640 1 -- 172.21.15.133:0/1183923038 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096c0041d0 con 0x5587684005e0 2024-09-11T23:46:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.292+0000 7f096affd640 1 -- 172.21.15.133:0/1183923038 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c0116a0 con 0x5587684005e0 2024-09-11T23:46:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.293+0000 7f0989ac0740 1 -- 172.21.15.133:0/1183923038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587684005e0 msgr2=0x55876840a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.293+0000 7f0989ac0740 1 --2- 172.21.15.133:0/1183923038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587684005e0 0x55876840a940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f096c00bbc0 tx=0x7f096c0107a0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.302+0000 7f0989ac0740 1 -- 172.21.15.133:0/1183923038 shutdown_connections 2024-09-11T23:46:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.302+0000 7f0989ac0740 1 --2- 172.21.15.133:0/1183923038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 0x55876840d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.302+0000 7f0989ac0740 1 --2- 172.21.15.133:0/1183923038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587684005e0 0x55876840a940 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.302+0000 7f0989ac0740 1 --2- 172.21.15.133:0/1183923038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587683ffcc0 0x5587684000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.302+0000 7f0989ac0740 1 -- 172.21.15.133:0/1183923038 >> 172.21.15.133:0/1183923038 conn(0x5587683fc230 msgr2=0x5587683fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.302+0000 7f0989ac0740 1 -- 172.21.15.133:0/1183923038 shutdown_connections 2024-09-11T23:46:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.302+0000 7f0989ac0740 1 -- 172.21.15.133:0/1183923038 wait complete. 2024-09-11T23:46:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.303+0000 7f0989ac0740 1 -- start start 2024-09-11T23:46:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.303+0000 7f0989ac0740 1 Processor -- start 2024-09-11T23:46:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.304+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x558768407e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.304+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587684005e0 0x558768408340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.304+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 0x558768408880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.304+0000 7f0989ac0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55876840da50 con 0x55876840ae80 2024-09-11T23:46:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.304+0000 7f0989ac0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5587682ed290 con 0x5587683ffcc0 2024-09-11T23:46:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.304+0000 7f0989ac0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55876840dbd0 con 0x5587684005e0 2024-09-11T23:46:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.304+0000 7f0989ac0740 10 client.0 _ll_register_callbacks cb 0x55876831e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.306+0000 7f097ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587684005e0 0x558768408340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.306+0000 7f097ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587684005e0 0x558768408340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:43932/0 (socket says 172.21.15.133:43932) 2024-09-11T23:46:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.306+0000 7f097ffff640 1 -- 172.21.15.133:0/3922394446 learned_addr learned my addr 172.21.15.133:0/3922394446 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.306+0000 7f0984ac8640 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x558768407e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.308+0000 7f097ffff640 1 -- 172.21.15.133:0/3922394446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 msgr2=0x558768407e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.308+0000 7f097ffff640 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x558768407e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.308+0000 7f097ffff640 1 -- 172.21.15.133:0/3922394446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 msgr2=0x558768408880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.308+0000 7f097ffff640 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 0x558768408880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.308+0000 7f097ffff640 1 -- 172.21.15.133:0/3922394446 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558768077210 con 0x5587684005e0 2024-09-11T23:46:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.310+0000 7f097ffff640 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587684005e0 0x558768408340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f096c011010 tx=0x7f096c003bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a300ab7cf5f53d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.310+0000 7f095ffff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.311+0000 7f095ffff640 1 -- 172.21.15.133:0/3922394446 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c004380 con 0x5587684005e0 2024-09-11T23:46:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.311+0000 7f0989ac0740 1 -- 172.21.15.133:0/3922394446 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f096c0041d0 con 0x5587684005e0 2024-09-11T23:46:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.311+0000 7f095ffff640 1 -- 172.21.15.133:0/3922394446 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f096c0035f0 con 0x5587684005e0 2024-09-11T23:46:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.316+0000 7f095ffff640 1 -- 172.21.15.133:0/3922394446 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f096c0037f0 con 0x5587684005e0 2024-09-11T23:46:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.316+0000 7f0989ac0740 10 client.27130 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.316+0000 7f0989ac0740 1 -- 172.21.15.133:0/3922394446 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558768143620 con 0x5587684005e0 2024-09-11T23:46:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.317+0000 7f095ffff640 1 -- 172.21.15.133:0/3922394446 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096c0041d0 con 0x5587684005e0 2024-09-11T23:46:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.318+0000 7f095ffff640 1 -- 172.21.15.133:0/3922394446 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f096c004a30 con 0x5587684005e0 2024-09-11T23:46:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.319+0000 7f0989ac0740 10 client.27130 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.319+0000 7f0989ac0740 10 client.27130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.319+0000 7f0989ac0740 20 client.27130 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.319+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55876841b300 0x55876841d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.319+0000 7f0989ac0740 4 client.27130 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.319+0000 7f0989ac0740 1 -- 172.21.15.133:0/3922394446 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558768371a20 con 0x55876841b300 2024-09-11T23:46:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.320+0000 7f0984ac8640 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55876841b300 0x55876841d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.323+0000 7f0984ac8640 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55876841b300 0x55876841d760 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af1b6d2fa6f3c78 server_cookie=639fee3ea3268800 in_seq=0 out_seq=0 2024-09-11T23:46:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.324+0000 7f095ffff640 10 client.27130 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.325+0000 7f095ffff640 1 -- 172.21.15.133:0/3922394446 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558768371a20 con 0x55876841b300 2024-09-11T23:46:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.325+0000 7f095ffff640 10 client.27130 handle_command_reply: tid=0 2024-09-11T23:46:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.326+0000 7f0989ac0740 1 client.27130 shutdown 2024-09-11T23:46:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.327+0000 7f0989ac0740 1 -- 172.21.15.133:0/3922394446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587684005e0 msgr2=0x558768408340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.327+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587684005e0 0x558768408340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f096c011010 tx=0x7f096c003bf0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.327+0000 7f0989ac0740 1 -- 172.21.15.133:0/3922394446 shutdown_connections 2024-09-11T23:46:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.327+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55876841b300 0x55876841d760 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.327+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 0x558768408880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.327+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587684005e0 0x558768408340 secure :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f096c011010 tx=0x7f096c003bf0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.327+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3922394446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x558768407e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.327+0000 7f0989ac0740 1 -- 172.21.15.133:0/3922394446 >> 172.21.15.133:0/3922394446 conn(0x5587683fc230 msgr2=0x558768401f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.328+0000 7f095ffff640 0 client.27130 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.328+0000 7f0989ac0740 1 -- 172.21.15.133:0/3922394446 shutdown_connections 2024-09-11T23:46:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.328+0000 7f0989ac0740 1 -- 172.21.15.133:0/3922394446 wait complete. 2024-09-11T23:46:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.328+0000 7f0989ac0740 20 client.27130 trim_cache size 0 max 16384 2024-09-11T23:46:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.376+0000 7f0989ac0740 1 Processor -- start 2024-09-11T23:46:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.382+0000 7f0989ac0740 1 -- start start 2024-09-11T23:46:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.383+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x5587683f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.383+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 0x5587684086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.383+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558768409d70 0x558768423070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.383+0000 7f0989ac0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f096400e580 con 0x55876840ae80 2024-09-11T23:46:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.383+0000 7f0989ac0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09640100c0 con 0x5587683ffcc0 2024-09-11T23:46:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.384+0000 7f0989ac0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f096400ec50 con 0x558768409d70 2024-09-11T23:46:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.384+0000 7f0984ac8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x5587683f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.384+0000 7f0984ac8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x5587683f6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59104/0 (socket says 172.21.15.133:59104) 2024-09-11T23:46:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.384+0000 7f0984ac8640 1 -- 172.21.15.133:0/1839537946 learned_addr learned my addr 172.21.15.133:0/1839537946 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.385+0000 7f0984ac8640 1 -- 172.21.15.133:0/1839537946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558768409d70 msgr2=0x558768423070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.385+0000 7f0984ac8640 1 --2- 172.21.15.133:0/1839537946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558768409d70 0x558768423070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.385+0000 7f0984ac8640 1 -- 172.21.15.133:0/1839537946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 msgr2=0x5587684086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.385+0000 7f0984ac8640 1 --2- 172.21.15.133:0/1839537946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 0x5587684086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.385+0000 7f0984ac8640 1 -- 172.21.15.133:0/1839537946 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558768406d90 con 0x5587683ffcc0 2024-09-11T23:46:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.385+0000 7f0984ac8640 1 --2- 172.21.15.133:0/1839537946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x5587683f6ed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0974009860 tx=0x7f097403c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1511aaf621c9a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.400+0000 7f096a7fc640 1 -- 172.21.15.133:0/1839537946 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09740048c0 con 0x5587683ffcc0 2024-09-11T23:46:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.400+0000 7f096a7fc640 1 -- 172.21.15.133:0/1839537946 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09740533b0 con 0x5587683ffcc0 2024-09-11T23:46:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.400+0000 7f096a7fc640 1 -- 172.21.15.133:0/1839537946 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09740526c0 con 0x5587683ffcc0 2024-09-11T23:46:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.404+0000 7f0989ac0740 1 -- 172.21.15.133:0/1839537946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 msgr2=0x5587683f6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.404+0000 7f0989ac0740 1 --2- 172.21.15.133:0/1839537946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x5587683f6ed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0974009860 tx=0x7f097403c000 comp rx=0 tx=0).stop 2024-09-11T23:46:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.404+0000 7f0989ac0740 1 -- 172.21.15.133:0/1839537946 shutdown_connections 2024-09-11T23:46:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.404+0000 7f0989ac0740 1 --2- 172.21.15.133:0/1839537946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558768409d70 0x558768423070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.404+0000 7f0989ac0740 1 --2- 172.21.15.133:0/1839537946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55876840ae80 0x5587684086b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.404+0000 7f0989ac0740 1 --2- 172.21.15.133:0/1839537946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x5587683f6ed0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.404+0000 7f0989ac0740 1 -- 172.21.15.133:0/1839537946 >> 172.21.15.133:0/1839537946 conn(0x5587683fc230 msgr2=0x558768409660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.404+0000 7f0989ac0740 1 -- 172.21.15.133:0/1839537946 shutdown_connections 2024-09-11T23:46:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.404+0000 7f0989ac0740 1 -- 172.21.15.133:0/1839537946 wait complete. 2024-09-11T23:46:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.405+0000 7f0989ac0740 1 -- start start 2024-09-11T23:46:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.405+0000 7f0989ac0740 1 Processor -- start 2024-09-11T23:46:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.406+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x558768413470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.406+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558768409d70 0x55876840e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.406+0000 7f0989ac0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55876840ae80 0x55876840ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.406+0000 7f0989ac0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f096400f9f0 con 0x558768409d70 2024-09-11T23:46:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.406+0000 7f0989ac0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f096400f320 con 0x5587683ffcc0 2024-09-11T23:46:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.406+0000 7f0989ac0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f096400c610 con 0x55876840ae80 2024-09-11T23:46:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.406+0000 7f0989ac0740 10 client.0 _ll_register_callbacks cb 0x7f097403f050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.414+0000 7f0984ac8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x558768413470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.414+0000 7f0984ac8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x558768413470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59114/0 (socket says 172.21.15.133:59114) 2024-09-11T23:46:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.414+0000 7f0984ac8640 1 -- 172.21.15.133:0/3911139853 learned_addr learned my addr 172.21.15.133:0/3911139853 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.414+0000 7f09852c9640 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55876840ae80 0x55876840ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.414+0000 7f097ffff640 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558768409d70 0x55876840e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.418+0000 7f09852c9640 1 -- 172.21.15.133:0/3911139853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 msgr2=0x558768413470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.418+0000 7f09852c9640 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x558768413470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.418+0000 7f09852c9640 1 -- 172.21.15.133:0/3911139853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558768409d70 msgr2=0x55876840e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.418+0000 7f09852c9640 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558768409d70 0x55876840e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.418+0000 7f09852c9640 1 -- 172.21.15.133:0/3911139853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55876840f340 con 0x55876840ae80 2024-09-11T23:46:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.426+0000 7f09852c9640 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55876840ae80 0x55876840ecd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0980131e90 tx=0x7f09801005c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c048cbac55473971 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.426+0000 7f095ffff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.429+0000 7f095ffff640 1 -- 172.21.15.133:0/3911139853 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980132320 con 0x55876840ae80 2024-09-11T23:46:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.429+0000 7f0989ac0740 1 -- 172.21.15.133:0/3911139853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09740533b0 con 0x55876840ae80 2024-09-11T23:46:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.430+0000 7f095ffff640 1 -- 172.21.15.133:0/3911139853 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f0980109640 con 0x55876840ae80 2024-09-11T23:46:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.434+0000 7f095ffff640 1 -- 172.21.15.133:0/3911139853 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0980100c60 con 0x55876840ae80 2024-09-11T23:46:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.434+0000 7f0989ac0740 10 client.27133 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.434+0000 7f0989ac0740 1 -- 172.21.15.133:0/3911139853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558768496f30 con 0x55876840ae80 2024-09-11T23:46:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.439+0000 7f095ffff640 1 -- 172.21.15.133:0/3911139853 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09740533b0 con 0x55876840ae80 2024-09-11T23:46:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.440+0000 7f095ffff640 1 -- 172.21.15.133:0/3911139853 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f09800fcb80 con 0x55876840ae80 2024-09-11T23:46:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.440+0000 7f0989ac0740 10 client.27133 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.440+0000 7f0989ac0740 10 client.27133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.440+0000 7f0989ac0740 20 client.27133 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.440+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587684976f0 0x558768499ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.440+0000 7f0989ac0740 4 client.27133 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:46:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.440+0000 7f0989ac0740 1 -- 172.21.15.133:0/3911139853 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55876836bd70 con 0x5587684976f0 2024-09-11T23:46:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.445+0000 7f0984ac8640 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587684976f0 0x558768499ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.463+0000 7f0984ac8640 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587684976f0 0x558768499ae0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ecd9c7b3ecaf0 server_cookie=626288f5dc92e4a0 in_seq=0 out_seq=0 2024-09-11T23:46:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:15.463+0000 7f095ffff640 10 client.27133 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.774+0000 7f095ffff640 1 -- 172.21.15.133:0/3911139853 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55876836bd70 con 0x5587684976f0 2024-09-11T23:46:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.774+0000 7f095ffff640 10 client.27133 handle_command_reply: tid=0 2024-09-11T23:46:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f0989ac0740 1 client.27133 shutdown 2024-09-11T23:46:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f0989ac0740 1 -- 172.21.15.133:0/3911139853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55876840ae80 msgr2=0x55876840ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55876840ae80 0x55876840ecd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0980131e90 tx=0x7f09801005c0 comp rx=0 tx=0).stop 2024-09-11T23:46:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f0989ac0740 1 -- 172.21.15.133:0/3911139853 shutdown_connections 2024-09-11T23:46:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5587684976f0 0x558768499ae0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55876840ae80 0x55876840ecd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558768409d70 0x55876840e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f0989ac0740 1 --2- 172.21.15.133:0/3911139853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587683ffcc0 0x558768413470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f0989ac0740 1 -- 172.21.15.133:0/3911139853 >> 172.21.15.133:0/3911139853 conn(0x5587683fc230 msgr2=0x558768401720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f095ffff640 0 client.27133 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f0989ac0740 1 -- 172.21.15.133:0/3911139853 shutdown_connections 2024-09-11T23:46:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.775+0000 7f0989ac0740 1 -- 172.21.15.133:0/3911139853 wait complete. 2024-09-11T23:46:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.776+0000 7f0989ac0740 20 client.27133 trim_cache size 0 max 16384 2024-09-11T23:46:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:46:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:46:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:46:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:46:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 -- 172.21.15.133:0/4174388560 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09700a2f00 msgr2=0x7f09700a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09700a2f00 0x7f09700a53c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f09740092e0 tx=0x7f0974005d60 comp rx=0 tx=0).stop 2024-09-11T23:46:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 -- 172.21.15.133:0/4174388560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0978147bb0 msgr2=0x7f0978148060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0978147bb0 0x7f0978148060 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0980067b40 tx=0x7f0980073d80 comp rx=0 tx=0).stop 2024-09-11T23:46:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 -- 172.21.15.133:0/4174388560 shutdown_connections 2024-09-11T23:46:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09700a2f00 0x7f09700a53c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0978147bb0 0x7f0978148060 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09780b7570 0x7f0978143830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 --2- 172.21.15.133:0/4174388560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09780a5170 0x7f09781432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 -- 172.21.15.133:0/4174388560 >> 172.21.15.133:0/4174388560 conn(0x7f097801a140 msgr2=0x7f09780b6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 -- 172.21.15.133:0/4174388560 shutdown_connections 2024-09-11T23:46:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:16.778+0000 7f0986d2c640 1 -- 172.21.15.133:0/4174388560 wait complete. 2024-09-11T23:46:17.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:16 smithi178 ceph-mon[35882]: pgmap v295: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 64 GiB data, 241 GiB used, 832 GiB / 1.0 TiB avail; 439 KiB/s rd, 613 MiB/s wr, 4.27k op/s 2024-09-11T23:46:17.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:16 smithi138 ceph-mon[35142]: pgmap v295: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 64 GiB data, 241 GiB used, 832 GiB / 1.0 TiB avail; 439 KiB/s rd, 613 MiB/s wr, 4.27k op/s 2024-09-11T23:46:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:16 smithi133 ceph-mon[31430]: pgmap v295: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 64 GiB data, 241 GiB used, 832 GiB / 1.0 TiB avail; 439 KiB/s rd, 613 MiB/s wr, 4.27k op/s 2024-09-11T23:46:18.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:17 smithi178 ceph-mon[35882]: pgmap v296: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 63 GiB data, 241 GiB used, 831 GiB / 1.0 TiB avail; 695 KiB/s rd, 556 MiB/s wr, 4.08k op/s 2024-09-11T23:46:18.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:17 smithi138 ceph-mon[35142]: pgmap v296: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 63 GiB data, 241 GiB used, 831 GiB / 1.0 TiB avail; 695 KiB/s rd, 556 MiB/s wr, 4.08k op/s 2024-09-11T23:46:18.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:17 smithi133 ceph-mon[31430]: pgmap v296: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 63 GiB data, 241 GiB used, 831 GiB / 1.0 TiB avail; 695 KiB/s rd, 556 MiB/s wr, 4.08k op/s 2024-09-11T23:46:19.925 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:46:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:21.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:20 smithi133 ceph-mon[31430]: pgmap v297: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 64 GiB data, 246 GiB used, 827 GiB / 1.0 TiB avail; 893 KiB/s rd, 739 MiB/s wr, 5.70k op/s 2024-09-11T23:46:21.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:20 smithi178 ceph-mon[35882]: pgmap v297: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 64 GiB data, 246 GiB used, 827 GiB / 1.0 TiB avail; 893 KiB/s rd, 739 MiB/s wr, 5.70k op/s 2024-09-11T23:46:21.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:20 smithi138 ceph-mon[35142]: pgmap v297: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 64 GiB data, 246 GiB used, 827 GiB / 1.0 TiB avail; 893 KiB/s rd, 739 MiB/s wr, 5.70k op/s 2024-09-11T23:46:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.825+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/1314793547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc411b140 msgr2=0x7f5cc411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.825+0000 7f5cc9bcf640 1 --2- 172.21.15.133:0/1314793547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc411b140 0x7f5cc411d530 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc01acb0 tx=0x7f5cbc03b920 comp rx=0 tx=0).stop 2024-09-11T23:46:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.826+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/1314793547 shutdown_connections 2024-09-11T23:46:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.826+0000 7f5cc9bcf640 1 --2- 172.21.15.133:0/1314793547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc411b140 0x7f5cc411d530 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.826+0000 7f5cc9bcf640 1 --2- 172.21.15.133:0/1314793547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc406ea30 0x7f5cc406ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.826+0000 7f5cc9bcf640 1 --2- 172.21.15.133:0/1314793547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cc4110220 0x7f5cc406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.826+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/1314793547 >> 172.21.15.133:0/1314793547 conn(0x7f5cc4073210 msgr2=0x7f5cc4073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.826+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/1314793547 shutdown_connections 2024-09-11T23:46:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.826+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/1314793547 wait complete. 2024-09-11T23:46:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.827+0000 7f5cc9bcf640 1 Processor -- start 2024-09-11T23:46:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.827+0000 7f5cc9bcf640 1 -- start start 2024-09-11T23:46:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.828+0000 7f5cc9bcf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc406ea30 0x7f5cc411a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.828+0000 7f5cc9bcf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cc4110220 0x7f5cc4116890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.828+0000 7f5cc9bcf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc411b140 0x7f5cc4116dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.828+0000 7f5cc9bcf640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41132d0 con 0x7f5cc411b140 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.828+0000 7f5cc9bcf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4113150 con 0x7f5cc4110220 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.828+0000 7f5cc9bcf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4113450 con 0x7f5cc406ea30 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.828+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc411b140 0x7f5cc4116dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.828+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc411b140 0x7f5cc4116dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50650/0 (socket says 172.21.15.133:50650) 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.828+0000 7f5cc3fff640 1 -- 172.21.15.133:0/884008017 learned_addr learned my addr 172.21.15.133:0/884008017 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.828+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc406ea30 0x7f5cc411a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.829+0000 7f5cc37fe640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cc4110220 0x7f5cc4116890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.829+0000 7f5cbb7fe640 1 -- 172.21.15.133:0/884008017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cc4110220 msgr2=0x7f5cc4116890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.829+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cc4110220 0x7f5cc4116890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.829+0000 7f5cbb7fe640 1 -- 172.21.15.133:0/884008017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc411b140 msgr2=0x7f5cc4116dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:21.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.829+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc411b140 0x7f5cc4116dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.829+0000 7f5cbb7fe640 1 -- 172.21.15.133:0/884008017 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc41174d0 con 0x7f5cc406ea30 2024-09-11T23:46:21.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.830+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc406ea30 0x7f5cc411a810 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00d6d0 tx=0x7f5cac00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:21.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.834+0000 7f5cc17fa640 1 -- 172.21.15.133:0/884008017 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac017070 con 0x7f5cc406ea30 2024-09-11T23:46:21.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.834+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/884008017 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc407c470 con 0x7f5cc406ea30 2024-09-11T23:46:21.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.834+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/884008017 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc407c9b0 con 0x7f5cc406ea30 2024-09-11T23:46:21.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.839+0000 7f5ccbfa8740 1 Processor -- start 2024-09-11T23:46:21.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.847+0000 7f5ccbfa8740 1 -- start start 2024-09-11T23:46:21.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.847+0000 7f5ccbfa8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72d670 0x55c24a72da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.848+0000 7f5ccbfa8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72df90 0x55c24a738310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.848+0000 7f5ccbfa8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a738850 0x55c24a73ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.848+0000 7f5ccbfa8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c24a6932e0 con 0x55c24a738850 2024-09-11T23:46:21.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.848+0000 7f5ccbfa8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c24a63ace0 con 0x55c24a72df90 2024-09-11T23:46:21.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.848+0000 7f5ccbfa8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c24a68f2a0 con 0x55c24a72d670 2024-09-11T23:46:21.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.848+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72df90 0x55c24a738310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.849+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72df90 0x55c24a738310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59142/0 (socket says 172.21.15.133:59142) 2024-09-11T23:46:21.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.849+0000 7f5cbb7fe640 1 -- 172.21.15.133:0/1957887070 learned_addr learned my addr 172.21.15.133:0/1957887070 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:21.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.849+0000 7f5cc3fff640 1 --2- 172.21.15.133:0/1957887070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72d670 0x55c24a72da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.850+0000 7f5cbb7fe640 1 -- 172.21.15.133:0/1957887070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72d670 msgr2=0x55c24a72da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.850+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/1957887070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72d670 0x55c24a72da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.850+0000 7f5cbb7fe640 1 -- 172.21.15.133:0/1957887070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a738850 msgr2=0x55c24a73ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.850+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/1957887070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a738850 0x55c24a73ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.850+0000 7f5cbb7fe640 1 -- 172.21.15.133:0/1957887070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c24a61f440 con 0x55c24a72df90 2024-09-11T23:46:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.851+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/1957887070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72df90 0x55c24a738310 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5cac010b50 tx=0x7f5cac010f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9de582d21828f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.852+0000 7f5cc17fa640 1 -- 172.21.15.133:0/884008017 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cac009820 con 0x7f5cc406ea30 2024-09-11T23:46:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.852+0000 7f5cc17fa640 1 -- 172.21.15.133:0/884008017 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac004ca0 con 0x7f5cc406ea30 2024-09-11T23:46:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.854+0000 7f5cc17fa640 1 -- 172.21.15.133:0/884008017 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5cac0248a0 con 0x7f5cc406ea30 2024-09-11T23:46:21.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.856+0000 7f5cc17fa640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5cb00a2f00 0x7f5cb00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.857+0000 7f5cc17fa640 1 -- 172.21.15.133:0/884008017 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f5cac0d3650 con 0x7f5cc406ea30 2024-09-11T23:46:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.857+0000 7f5cb9ffb640 1 -- 172.21.15.133:0/1957887070 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac056070 con 0x55c24a72df90 2024-09-11T23:46:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.858+0000 7f5cb9ffb640 1 -- 172.21.15.133:0/1957887070 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cac01bd00 con 0x55c24a72df90 2024-09-11T23:46:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.858+0000 7f5cb9ffb640 1 -- 172.21.15.133:0/1957887070 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac020060 con 0x55c24a72df90 2024-09-11T23:46:21.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.863+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1957887070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72df90 msgr2=0x55c24a738310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:21.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.863+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1957887070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72df90 0x55c24a738310 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5cac010b50 tx=0x7f5cac010f50 comp rx=0 tx=0).stop 2024-09-11T23:46:21.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.863+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1957887070 shutdown_connections 2024-09-11T23:46:21.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.863+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1957887070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a738850 0x55c24a73ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.863+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1957887070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72df90 0x55c24a738310 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.863+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1957887070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72d670 0x55c24a72da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.863+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1957887070 >> 172.21.15.133:0/1957887070 conn(0x55c24a729c00 msgr2=0x55c24a729fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:21.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.866+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1957887070 shutdown_connections 2024-09-11T23:46:21.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.866+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1957887070 wait complete. 2024-09-11T23:46:21.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.867+0000 7f5ccbfa8740 1 -- start start 2024-09-11T23:46:21.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.868+0000 7f5ccbfa8740 1 Processor -- start 2024-09-11T23:46:21.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.868+0000 7f5ccbfa8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a7381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.868+0000 7f5ccbfa8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 0x55c24a735470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.869+0000 7f5ccbfa8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 0x55c24a7359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.869+0000 7f5cc37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 0x55c24a735470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.869+0000 7f5cc37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 0x55c24a735470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50670/0 (socket says 172.21.15.133:50670) 2024-09-11T23:46:21.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.869+0000 7f5cc37fe640 1 -- 172.21.15.133:0/1702495736 learned_addr learned my addr 172.21.15.133:0/1702495736 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:21.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.869+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1702495736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c24a73b3c0 con 0x55c24a72df90 2024-09-11T23:46:21.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.869+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1702495736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c24a64e050 con 0x55c24a72d670 2024-09-11T23:46:21.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.869+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1702495736 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c24a73b540 con 0x55c24a738850 2024-09-11T23:46:21.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.869+0000 7f5ccbfa8740 10 client.0 _ll_register_callbacks cb 0x55c24a68e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:21.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.869+0000 7f5cc3fff640 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 0x55c24a7359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.873+0000 7f5cc3fff640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5cb00a2f00 0x7f5cb00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.873+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a7381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.876+0000 7f5cc3fff640 1 -- 172.21.15.133:0/1702495736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 msgr2=0x55c24a7381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:21.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.876+0000 7f5cc3fff640 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a7381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.876+0000 7f5cc3fff640 1 -- 172.21.15.133:0/1702495736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 msgr2=0x55c24a735470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:21.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.876+0000 7f5cc3fff640 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 0x55c24a735470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.876+0000 7f5cc3fff640 1 -- 172.21.15.133:0/1702495736 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c24a4043c0 con 0x55c24a738850 2024-09-11T23:46:21.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.876+0000 7f5cc3fff640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5cb00a2f00 0x7f5cb00a53c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc006010 tx=0x7f5cbc01a970 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:21.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.879+0000 7f5cc3fff640 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 0x55c24a7359d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc040a40 tx=0x7f5cbc040f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd06684f0b3df6a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:21.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.879+0000 7f5ca2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:21.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.880+0000 7f5ca2ffd640 1 -- 172.21.15.133:0/1702495736 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc054040 con 0x55c24a738850 2024-09-11T23:46:21.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.880+0000 7f5ca2ffd640 1 -- 172.21.15.133:0/1702495736 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f5cbc04b8f0 con 0x55c24a738850 2024-09-11T23:46:21.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.880+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1702495736 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cac01bd00 con 0x55c24a738850 2024-09-11T23:46:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.896+0000 7f5ca2ffd640 1 -- 172.21.15.133:0/1702495736 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5cbc045040 con 0x55c24a738850 2024-09-11T23:46:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.896+0000 7f5ccbfa8740 10 client.27145 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.896+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1702495736 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c24a746e30 con 0x55c24a738850 2024-09-11T23:46:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.897+0000 7f5ca2ffd640 1 -- 172.21.15.133:0/1702495736 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cbc06d470 con 0x55c24a738850 2024-09-11T23:46:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.897+0000 7f5ca2ffd640 1 -- 172.21.15.133:0/1702495736 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5cbc063430 con 0x55c24a738850 2024-09-11T23:46:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.897+0000 7f5ccbfa8740 10 client.27145 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.897+0000 7f5ccbfa8740 10 client.27145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.897+0000 7f5ccbfa8740 20 client.27145 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.897+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c24a7483a0 0x55c24a74a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.897+0000 7f5ccbfa8740 4 client.27145 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.897+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1702495736 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5cac01bd00 con 0x55c24a7483a0 2024-09-11T23:46:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.898+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c24a7483a0 0x55c24a74a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.899+0000 7f5ca2ffd640 10 client.27145 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:21.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.900+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c24a7483a0 0x55c24a74a800 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a4fe4e4dd678c8 server_cookie=ec9605492815a6ea in_seq=0 out_seq=0 2024-09-11T23:46:21.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.903+0000 7f5ca2ffd640 1 -- 172.21.15.133:0/1702495736 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5cac01bd00 con 0x55c24a7483a0 2024-09-11T23:46:21.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.903+0000 7f5ca2ffd640 10 client.27145 handle_command_reply: tid=0 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.903+0000 7f5ccbfa8740 1 client.27145 shutdown 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.903+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1702495736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 msgr2=0x55c24a7359d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.903+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 0x55c24a7359d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc040a40 tx=0x7f5cbc040f00 comp rx=0 tx=0).stop 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.904+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1702495736 shutdown_connections 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.904+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c24a7483a0 0x55c24a74a800 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.904+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 0x55c24a7359d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.904+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 0x55c24a735470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.904+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1702495736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a7381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.904+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1702495736 >> 172.21.15.133:0/1702495736 conn(0x55c24a729c00 msgr2=0x55c24a72f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.904+0000 7f5ca2ffd640 0 client.27145 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.904+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1702495736 shutdown_connections 2024-09-11T23:46:21.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.904+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1702495736 wait complete. 2024-09-11T23:46:21.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.906+0000 7f5ccbfa8740 20 client.27145 trim_cache size 0 max 16384 2024-09-11T23:46:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.948+0000 7f5ccbfa8740 1 Processor -- start 2024-09-11T23:46:21.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.948+0000 7f5ccbfa8740 1 -- start start 2024-09-11T23:46:21.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.950+0000 7f5ccbfa8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a736cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.950+0000 7f5ccbfa8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 0x55c24a750110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.951+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a736cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.951+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a736cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59162/0 (socket says 172.21.15.133:59162) 2024-09-11T23:46:21.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.952+0000 7f5cbb7fe640 1 -- 172.21.15.133:0/3392803003 learned_addr learned my addr 172.21.15.133:0/3392803003 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:21.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.952+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/3392803003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 0x55c24a750650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.953+0000 7f5cc37fe640 1 --2- 172.21.15.133:0/3392803003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 0x55c24a750110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.953+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/3392803003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca400e800 con 0x55c24a72df90 2024-09-11T23:46:21.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.953+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/3392803003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca4010340 con 0x55c24a72d670 2024-09-11T23:46:21.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.954+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/3392803003 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca400eed0 con 0x55c24a738850 2024-09-11T23:46:21.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.955+0000 7f5cc3fff640 1 --2- 172.21.15.133:0/3392803003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 0x55c24a750650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.955+0000 7f5cc37fe640 1 -- 172.21.15.133:0/3392803003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 msgr2=0x55c24a750650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:21.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.955+0000 7f5cc37fe640 1 --2- 172.21.15.133:0/3392803003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 0x55c24a750650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.955+0000 7f5cc37fe640 1 -- 172.21.15.133:0/3392803003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 msgr2=0x55c24a736cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:21.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.955+0000 7f5cc37fe640 1 --2- 172.21.15.133:0/3392803003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a736cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.955+0000 7f5cc37fe640 1 -- 172.21.15.133:0/3392803003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c24a737b80 con 0x55c24a72df90 2024-09-11T23:46:21.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.955+0000 7f5cc3fff640 1 --2- 172.21.15.133:0/3392803003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 0x55c24a750650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:46:21.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.956+0000 7f5cc37fe640 1 --2- 172.21.15.133:0/3392803003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 0x55c24a750110 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x55c24a736ab0 tx=0x7f5cb400bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de07861a07cf85c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:21.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.956+0000 7f5cb97fa640 1 -- 172.21.15.133:0/3392803003 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4018070 con 0x55c24a72df90 2024-09-11T23:46:21.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.957+0000 7f5cb97fa640 1 -- 172.21.15.133:0/3392803003 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb4014430 con 0x55c24a72df90 2024-09-11T23:46:21.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.957+0000 7f5cb97fa640 1 -- 172.21.15.133:0/3392803003 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4013630 con 0x55c24a72df90 2024-09-11T23:46:21.961 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:21 smithi133 ceph-mon[31430]: pgmap v298: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 64 GiB data, 246 GiB used, 827 GiB / 1.0 TiB avail; 891 KiB/s rd, 541 MiB/s wr, 4.45k op/s 2024-09-11T23:46:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.959+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/3392803003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 msgr2=0x55c24a750110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.960+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/3392803003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 0x55c24a750110 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x55c24a736ab0 tx=0x7f5cb400bc30 comp rx=0 tx=0).stop 2024-09-11T23:46:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.970+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/3392803003 shutdown_connections 2024-09-11T23:46:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.970+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/3392803003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a738850 0x55c24a750650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.970+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/3392803003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a72df90 0x55c24a750110 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.970+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/3392803003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a736cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.970+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/3392803003 >> 172.21.15.133:0/3392803003 conn(0x55c24a729c00 msgr2=0x55c24a728d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.970+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/3392803003 shutdown_connections 2024-09-11T23:46:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.970+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/3392803003 wait complete. 2024-09-11T23:46:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.971+0000 7f5ccbfa8740 1 -- start start 2024-09-11T23:46:21.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.972+0000 7f5ccbfa8740 1 Processor -- start 2024-09-11T23:46:21.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.979+0000 7f5ccbfa8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a73ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.979+0000 7f5ccbfa8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72df90 0x55c24a73f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.979+0000 7f5ccbfa8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a738850 0x55c24a7b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.980+0000 7f5cc37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72df90 0x55c24a73f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.980+0000 7f5cc37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72df90 0x55c24a73f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:43988/0 (socket says 172.21.15.133:43988) 2024-09-11T23:46:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.980+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a73ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.980+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a73ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59166/0 (socket says 172.21.15.133:59166) 2024-09-11T23:46:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.980+0000 7f5cc37fe640 1 -- 172.21.15.133:0/1915580020 learned_addr learned my addr 172.21.15.133:0/1915580020 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.981+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1915580020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca400fc70 con 0x55c24a738850 2024-09-11T23:46:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.981+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1915580020 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca400f5a0 con 0x55c24a72d670 2024-09-11T23:46:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.981+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1915580020 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca400c800 con 0x55c24a72df90 2024-09-11T23:46:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.981+0000 7f5ccbfa8740 10 client.0 _ll_register_callbacks cb 0x7f5cbc01ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.981+0000 7f5cc37fe640 1 -- 172.21.15.133:0/1915580020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 msgr2=0x55c24a73ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.981+0000 7f5cc37fe640 1 --2- 172.21.15.133:0/1915580020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a73ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.981+0000 7f5cc37fe640 1 -- 172.21.15.133:0/1915580020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a738850 msgr2=0x55c24a7b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.981+0000 7f5cc37fe640 1 --2- 172.21.15.133:0/1915580020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a738850 0x55c24a7b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.981+0000 7f5cc37fe640 1 -- 172.21.15.133:0/1915580020 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c24a7b2630 con 0x55c24a72df90 2024-09-11T23:46:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.982+0000 7f5cc37fe640 1 --2- 172.21.15.133:0/1915580020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72df90 0x55c24a73f1b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4014dd0 tx=0x7f5cb4006d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89a382a67521505b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:21.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.982+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1915580020 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb4014430 con 0x55c24a72df90 2024-09-11T23:46:21.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.982+0000 7f5ca2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.983+0000 7f5ca2ffd640 1 -- 172.21.15.133:0/1915580020 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4018040 con 0x55c24a72df90 2024-09-11T23:46:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.988+0000 7f5ca2ffd640 1 -- 172.21.15.133:0/1915580020 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f5cb40147f0 con 0x55c24a72df90 2024-09-11T23:46:21.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.989+0000 7f5ca2ffd640 1 -- 172.21.15.133:0/1915580020 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5cb40149f0 con 0x55c24a72df90 2024-09-11T23:46:21.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.989+0000 7f5ccbfa8740 10 client.27151 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:21.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.989+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1915580020 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55c24a7b37b0 con 0x55c24a72df90 2024-09-11T23:46:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.991+0000 7f5ca2ffd640 1 -- 172.21.15.133:0/1915580020 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5cb4028480 con 0x55c24a72df90 2024-09-11T23:46:21.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.995+0000 7f5ccbfa8740 10 client.27151 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:21.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.995+0000 7f5ccbfa8740 10 client.27151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:21.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.995+0000 7f5ccbfa8740 20 client.27151 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:21.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.995+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1915580020 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c24a7b3c80 0x55c24a7b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:21.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.995+0000 7f5ccbfa8740 4 client.27151 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:46:21.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.995+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1915580020 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5cb4014430 con 0x55c24a7b3c80 2024-09-11T23:46:21.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:21.997+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/1915580020 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c24a7b3c80 0x55c24a7b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.004+0000 7f5ca2ffd640 10 client.27151 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.004+0000 7f5cbb7fe640 1 --2- 172.21.15.133:0/1915580020 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c24a7b3c80 0x55c24a7b60a0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673935e49d1f33ad server_cookie=c49cbf768b4c55eb in_seq=0 out_seq=0 2024-09-11T23:46:22.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.011+0000 7f5ca2ffd640 1 -- 172.21.15.133:0/1915580020 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f5cb4014430 con 0x55c24a7b3c80 2024-09-11T23:46:22.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.011+0000 7f5ca2ffd640 10 client.27151 handle_command_reply: tid=0 2024-09-11T23:46:22.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.011+0000 7f5ccbfa8740 1 client.27151 shutdown 2024-09-11T23:46:22.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.012+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1915580020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72df90 msgr2=0x55c24a73f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:22.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.012+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1915580020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72df90 0x55c24a73f1b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4014dd0 tx=0x7f5cb4006d60 comp rx=0 tx=0).stop 2024-09-11T23:46:22.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.018+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1915580020 shutdown_connections 2024-09-11T23:46:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.019+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1915580020 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c24a7b3c80 0x55c24a7b60a0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.019+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1915580020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24a738850 0x55c24a7b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.019+0000 7f5ca2ffd640 0 client.27151 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.019+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1915580020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c24a72df90 0x55c24a73f1b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.019+0000 7f5ccbfa8740 1 --2- 172.21.15.133:0/1915580020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c24a72d670 0x55c24a73ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.019+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1915580020 >> 172.21.15.133:0/1915580020 conn(0x55c24a729c00 msgr2=0x55c24a734c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.019+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1915580020 shutdown_connections 2024-09-11T23:46:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.019+0000 7f5ccbfa8740 1 -- 172.21.15.133:0/1915580020 wait complete. 2024-09-11T23:46:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.019+0000 7f5ccbfa8740 20 client.27151 trim_cache size 0 max 16384 2024-09-11T23:46:22.023 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:46:22.023 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:46:22.023 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:46:22.023 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:46:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.023+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/884008017 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5cb00a2f00 msgr2=0x7f5cb00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.023+0000 7f5cc9bcf640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5cb00a2f00 0x7f5cb00a53c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc006010 tx=0x7f5cbc01a970 comp rx=0 tx=0).stop 2024-09-11T23:46:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.024+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/884008017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc406ea30 msgr2=0x7f5cc411a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.024+0000 7f5cc9bcf640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc406ea30 0x7f5cc411a810 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00d6d0 tx=0x7f5cac00db90 comp rx=0 tx=0).stop 2024-09-11T23:46:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.024+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/884008017 shutdown_connections 2024-09-11T23:46:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.024+0000 7f5cc9bcf640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f5cb00a2f00 0x7f5cb00a53c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.024+0000 7f5cc9bcf640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc411b140 0x7f5cc4116dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.024+0000 7f5cc9bcf640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cc4110220 0x7f5cc4116890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.024+0000 7f5cc9bcf640 1 --2- 172.21.15.133:0/884008017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cc406ea30 0x7f5cc411a810 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:22.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.024+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/884008017 >> 172.21.15.133:0/884008017 conn(0x7f5cc4073210 msgr2=0x7f5cc411b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:22.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.024+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/884008017 shutdown_connections 2024-09-11T23:46:22.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:22.024+0000 7f5cc9bcf640 1 -- 172.21.15.133:0/884008017 wait complete. 2024-09-11T23:46:22.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:21 smithi138 ceph-mon[35142]: pgmap v298: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 64 GiB data, 246 GiB used, 827 GiB / 1.0 TiB avail; 891 KiB/s rd, 541 MiB/s wr, 4.45k op/s 2024-09-11T23:46:22.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:21 smithi178 ceph-mon[35882]: pgmap v298: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 64 GiB data, 246 GiB used, 827 GiB / 1.0 TiB avail; 891 KiB/s rd, 541 MiB/s wr, 4.45k op/s 2024-09-11T23:46:23.034 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:46:23.034 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:46:23.784 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:24.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.570+0000 7f3dacd50640 1 -- 172.21.15.133:0/2638559021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8106f20 msgr2=0x7f3da810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:24.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.570+0000 7f3dacd50640 1 --2- 172.21.15.133:0/2638559021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8106f20 0x7f3da810eaa0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009900 tx=0x7f3d9c02a410 comp rx=0 tx=0).stop 2024-09-11T23:46:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.572+0000 7f3dacd50640 1 -- 172.21.15.133:0/2638559021 shutdown_connections 2024-09-11T23:46:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.572+0000 7f3dacd50640 1 --2- 172.21.15.133:0/2638559021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8106f20 0x7f3da810eaa0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.572+0000 7f3dacd50640 1 --2- 172.21.15.133:0/2638559021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da81027a0 0x7f3da81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.572+0000 7f3dacd50640 1 --2- 172.21.15.133:0/2638559021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da8101df0 0x7f3da81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.572+0000 7f3dacd50640 1 -- 172.21.15.133:0/2638559021 >> 172.21.15.133:0/2638559021 conn(0x7f3da80fd8b0 msgr2=0x7f3da80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:24.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.573+0000 7f3dacd50640 1 -- 172.21.15.133:0/2638559021 shutdown_connections 2024-09-11T23:46:24.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.573+0000 7f3dacd50640 1 -- 172.21.15.133:0/2638559021 wait complete. 2024-09-11T23:46:24.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.574+0000 7f3dacd50640 1 Processor -- start 2024-09-11T23:46:24.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.574+0000 7f3dacd50640 1 -- start start 2024-09-11T23:46:24.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.575+0000 7f3dacd50640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8101df0 0x7f3da8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:24.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.575+0000 7f3dacd50640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da81027a0 0x7f3da8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:24.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.575+0000 7f3dacd50640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da8106f20 0x7f3da819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:24.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.575+0000 7f3dacd50640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3da8112020 con 0x7f3da8101df0 2024-09-11T23:46:24.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.575+0000 7f3dacd50640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3da8111ea0 con 0x7f3da81027a0 2024-09-11T23:46:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.575+0000 7f3dacd50640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3da81121a0 con 0x7f3da8106f20 2024-09-11T23:46:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.575+0000 7f3da6d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da8106f20 0x7f3da819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.575+0000 7f3da6d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da8106f20 0x7f3da819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42202/0 (socket says 172.21.15.133:42202) 2024-09-11T23:46:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.575+0000 7f3da6d9d640 1 -- 172.21.15.133:0/1416661703 learned_addr learned my addr 172.21.15.133:0/1416661703 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.576+0000 7f3da5d9b640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da81027a0 0x7f3da8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:24.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.577+0000 7f3da6d9d640 1 -- 172.21.15.133:0/1416661703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da81027a0 msgr2=0x7f3da8198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:24.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.577+0000 7f3da6d9d640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da81027a0 0x7f3da8198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:24.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.577+0000 7f3da6d9d640 1 -- 172.21.15.133:0/1416661703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8101df0 msgr2=0x7f3da8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:24.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.577+0000 7f3da6d9d640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8101df0 0x7f3da8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:24.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.577+0000 7f3da6d9d640 1 -- 172.21.15.133:0/1416661703 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da819d5f0 con 0x7f3da8106f20 2024-09-11T23:46:24.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.578+0000 7f3da6d9d640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da8106f20 0x7f3da819cf10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c02a980 tx=0x7f3d9c03cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:24.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.579+0000 7f3d977fe640 1 -- 172.21.15.133:0/1416661703 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c035030 con 0x7f3da8106f20 2024-09-11T23:46:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.579+0000 7f3dacd50640 1 -- 172.21.15.133:0/1416661703 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da819d880 con 0x7f3da8106f20 2024-09-11T23:46:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.579+0000 7f3dacd50640 1 -- 172.21.15.133:0/1416661703 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da8074420 con 0x7f3da8106f20 2024-09-11T23:46:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.580+0000 7f3d977fe640 1 -- 172.21.15.133:0/1416661703 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d9c003ce0 con 0x7f3da8106f20 2024-09-11T23:46:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.581+0000 7f3d977fe640 1 -- 172.21.15.133:0/1416661703 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c044640 con 0x7f3da8106f20 2024-09-11T23:46:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.583+0000 7f3d977fe640 1 -- 172.21.15.133:0/1416661703 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3d9c0448a0 con 0x7f3da8106f20 2024-09-11T23:46:24.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.584+0000 7f3d977fe640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3d900a2e40 0x7f3d900a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:24.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.584+0000 7f3dacd50640 1 -- 172.21.15.133:0/1416661703 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d88005290 con 0x7f3da8106f20 2024-09-11T23:46:24.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.585+0000 7f3d977fe640 1 -- 172.21.15.133:0/1416661703 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f3d9c0f3c30 con 0x7f3da8106f20 2024-09-11T23:46:24.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.587+0000 7f3da659c640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3d900a2e40 0x7f3d900a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:24.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.590+0000 7f3d977fe640 1 -- 172.21.15.133:0/1416661703 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d9c0c18a0 con 0x7f3da8106f20 2024-09-11T23:46:24.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:24.594+0000 7f3da659c640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3d900a2e40 0x7f3d900a5300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009710 tx=0x7f3d98006c90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:24.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:24 smithi133 ceph-mon[31430]: pgmap v299: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 65 GiB data, 251 GiB used, 822 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 699 MiB/s wr, 4.78k op/s 2024-09-11T23:46:25.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:24 smithi178 ceph-mon[35882]: pgmap v299: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 65 GiB data, 251 GiB used, 822 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 699 MiB/s wr, 4.78k op/s 2024-09-11T23:46:25.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.081+0000 7f3dacd50640 1 -- 172.21.15.133:0/1416661703 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d88005850 con 0x7f3da8106f20 2024-09-11T23:46:25.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.082+0000 7f3d977fe640 1 -- 172.21.15.133:0/1416661703 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7f3d9c0c11a0 con 0x7f3da8106f20 2024-09-11T23:46:25.087 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:46:25.087 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:46:25.087 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:46:25.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.093+0000 7f3d957fa640 1 -- 172.21.15.133:0/1416661703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3d900a2e40 msgr2=0x7f3d900a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:25.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.093+0000 7f3d957fa640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3d900a2e40 0x7f3d900a5300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009710 tx=0x7f3d98006c90 comp rx=0 tx=0).stop 2024-09-11T23:46:25.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.093+0000 7f3d957fa640 1 -- 172.21.15.133:0/1416661703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da8106f20 msgr2=0x7f3da819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:25.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.093+0000 7f3d957fa640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da8106f20 0x7f3da819cf10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c02a980 tx=0x7f3d9c03cc90 comp rx=0 tx=0).stop 2024-09-11T23:46:25.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.094+0000 7f3d957fa640 1 -- 172.21.15.133:0/1416661703 shutdown_connections 2024-09-11T23:46:25.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.094+0000 7f3d957fa640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3d900a2e40 0x7f3d900a5300 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009710 tx=0x7f3d98006c90 comp rx=0 tx=0).stop 2024-09-11T23:46:25.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.094+0000 7f3d957fa640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da8106f20 0x7f3da819cf10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:25.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.094+0000 7f3d957fa640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da81027a0 0x7f3da8198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:25.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.094+0000 7f3d957fa640 1 --2- 172.21.15.133:0/1416661703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8101df0 0x7f3da8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:25.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.094+0000 7f3d957fa640 1 -- 172.21.15.133:0/1416661703 >> 172.21.15.133:0/1416661703 conn(0x7f3da80fd8b0 msgr2=0x7f3da80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:25.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.095+0000 7f3d957fa640 1 -- 172.21.15.133:0/1416661703 shutdown_connections 2024-09-11T23:46:25.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:25.096+0000 7f3d957fa640 1 -- 172.21.15.133:0/1416661703 wait complete. 2024-09-11T23:46:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:24 smithi138 ceph-mon[35142]: pgmap v299: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 65 GiB data, 251 GiB used, 822 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 699 MiB/s wr, 4.78k op/s 2024-09-11T23:46:26.000 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:46:26.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:25 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1416661703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:46:26.045 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:25 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1416661703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:46:26.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:25 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1416661703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:46:26.812 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:27.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:26 smithi133 ceph-mon[31430]: pgmap v300: 97 pgs: 97 active+clean; 65 GiB data, 255 GiB used, 818 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 654 MiB/s wr, 3.64k op/s 2024-09-11T23:46:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:26 smithi138 ceph-mon[35142]: pgmap v300: 97 pgs: 97 active+clean; 65 GiB data, 255 GiB used, 818 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 654 MiB/s wr, 3.64k op/s 2024-09-11T23:46:27.292 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:26 smithi178 ceph-mon[35882]: pgmap v300: 97 pgs: 97 active+clean; 65 GiB data, 255 GiB used, 818 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 654 MiB/s wr, 3.64k op/s 2024-09-11T23:46:27.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.820+0000 7f082df35640 1 -- 172.21.15.133:0/776054961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0828110190 msgr2=0x7f082806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:27.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.820+0000 7f082df35640 1 --2- 172.21.15.133:0/776054961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0828110190 0x7f082806e4f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0820009be0 tx=0x7f082002a710 comp rx=0 tx=0).stop 2024-09-11T23:46:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.824+0000 7f082df35640 1 -- 172.21.15.133:0/776054961 shutdown_connections 2024-09-11T23:46:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.824+0000 7f082df35640 1 --2- 172.21.15.133:0/776054961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f082811b140 0x7f082811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.824+0000 7f082df35640 1 --2- 172.21.15.133:0/776054961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f082806ea30 0x7f082806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.824+0000 7f082df35640 1 --2- 172.21.15.133:0/776054961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0828110190 0x7f082806e4f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.824+0000 7f082df35640 1 -- 172.21.15.133:0/776054961 >> 172.21.15.133:0/776054961 conn(0x7f0828073180 msgr2=0x7f0828073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:27.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.827+0000 7f082df35640 1 -- 172.21.15.133:0/776054961 shutdown_connections 2024-09-11T23:46:27.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.831+0000 7f082df35640 1 -- 172.21.15.133:0/776054961 wait complete. 2024-09-11T23:46:27.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.832+0000 7f082df35640 1 Processor -- start 2024-09-11T23:46:27.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.833+0000 7f082df35640 1 -- start start 2024-09-11T23:46:27.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.833+0000 7f082df35640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f082806ea30 0x7f08281186d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:27.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.833+0000 7f082df35640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0828110190 0x7f0828111760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:27.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.833+0000 7f082df35640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f082811b140 0x7f0828111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:27.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.834+0000 7f082df35640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0828120be0 con 0x7f0828110190 2024-09-11T23:46:27.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.834+0000 7f082df35640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0828120a60 con 0x7f082811b140 2024-09-11T23:46:27.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.834+0000 7f082df35640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0828120d60 con 0x7f082806ea30 2024-09-11T23:46:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.835+0000 7f0827fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0828110190 0x7f0828111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.835+0000 7f0827fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0828110190 0x7f0828111760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51886/0 (socket says 172.21.15.133:51886) 2024-09-11T23:46:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.835+0000 7f0827fff640 1 -- 172.21.15.133:0/2562457809 learned_addr learned my addr 172.21.15.133:0/2562457809 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.836+0000 7f0827fff640 1 -- 172.21.15.133:0/2562457809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f082806ea30 msgr2=0x7f08281186d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:46:27.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.836+0000 7f0827fff640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f082806ea30 0x7f08281186d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.836+0000 7f0827fff640 1 -- 172.21.15.133:0/2562457809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f082811b140 msgr2=0x7f0828111ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:46:27.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.836+0000 7f0827fff640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f082811b140 0x7f0828111ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.836+0000 7f0827fff640 1 -- 172.21.15.133:0/2562457809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0828112260 con 0x7f0828110190 2024-09-11T23:46:27.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.837+0000 7f0827fff640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0828110190 0x7f0828111760 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f081800d950 tx=0x7f081800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:27.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.837+0000 7f0825ffb640 1 -- 172.21.15.133:0/2562457809 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818017070 con 0x7f0828110190 2024-09-11T23:46:27.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.838+0000 7f0825ffb640 1 -- 172.21.15.133:0/2562457809 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0818009c30 con 0x7f0828110190 2024-09-11T23:46:27.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.838+0000 7f0825ffb640 1 -- 172.21.15.133:0/2562457809 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081800f690 con 0x7f0828110190 2024-09-11T23:46:27.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.841+0000 7f082df35640 1 -- 172.21.15.133:0/2562457809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0828112550 con 0x7f0828110190 2024-09-11T23:46:27.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.842+0000 7f082df35640 1 -- 172.21.15.133:0/2562457809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0828116430 con 0x7f0828110190 2024-09-11T23:46:27.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.851+0000 7f0831570740 1 Processor -- start 2024-09-11T23:46:27.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.856+0000 7f0825ffb640 1 -- 172.21.15.133:0/2562457809 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f08180109d0 con 0x7f0828110190 2024-09-11T23:46:27.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.860+0000 7f0825ffb640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08100a2f00 0x7f08100a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:27.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.860+0000 7f0825ffb640 1 -- 172.21.15.133:0/2562457809 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f08180cedf0 con 0x7f0828110190 2024-09-11T23:46:27.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.861+0000 7f0831570740 1 -- start start 2024-09-11T23:46:27.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.862+0000 7f0831570740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1ecf0 0x556719a1d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:27.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.862+0000 7f0831570740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 0x556719a1dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:27.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.862+0000 7f0831570740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a28520 0x556719a2a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:27.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.862+0000 7f0831570740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5567199832f0 con 0x556719a1ecf0 2024-09-11T23:46:27.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.862+0000 7f0831570740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55671992acf0 con 0x556719a1d780 2024-09-11T23:46:27.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.862+0000 7f0831570740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55671997f2b0 con 0x556719a28520 2024-09-11T23:46:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.865+0000 7f0827fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 0x556719a1dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.865+0000 7f0827fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 0x556719a1dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37278/0 (socket says 172.21.15.133:37278) 2024-09-11T23:46:27.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.865+0000 7f0827fff640 1 -- 172.21.15.133:0/1637073263 learned_addr learned my addr 172.21.15.133:0/1637073263 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:27.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.866+0000 7f082d734640 1 --2- 172.21.15.133:0/1637073263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a28520 0x556719a2a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:27.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.870+0000 7f082d734640 1 -- 172.21.15.133:0/1637073263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 msgr2=0x556719a1dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:27.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.870+0000 7f082d734640 1 --2- 172.21.15.133:0/1637073263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 0x556719a1dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.870+0000 7f082d734640 1 -- 172.21.15.133:0/1637073263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1ecf0 msgr2=0x556719a1d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:27.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.870+0000 7f082d734640 1 --2- 172.21.15.133:0/1637073263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1ecf0 0x556719a1d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.870+0000 7f082d734640 1 -- 172.21.15.133:0/1637073263 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55671990f3a0 con 0x556719a28520 2024-09-11T23:46:27.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.873+0000 7f082d734640 1 --2- 172.21.15.133:0/1637073263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a28520 0x556719a2a910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f081c00da30 tx=0x7f081c00de30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c48936a506c8823 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:27.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.873+0000 7f080effd640 1 -- 172.21.15.133:0/1637073263 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c00fc00 con 0x556719a28520 2024-09-11T23:46:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.873+0000 7f080effd640 1 -- 172.21.15.133:0/1637073263 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f081c00fda0 con 0x556719a28520 2024-09-11T23:46:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.875+0000 7f0831570740 1 -- 172.21.15.133:0/1637073263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a28520 msgr2=0x556719a2a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.875+0000 7f0831570740 1 --2- 172.21.15.133:0/1637073263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a28520 0x556719a2a910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f081c00da30 tx=0x7f081c00de30 comp rx=0 tx=0).stop 2024-09-11T23:46:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.880+0000 7f082d734640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08100a2f00 0x7f08100a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.882+0000 7f0831570740 1 -- 172.21.15.133:0/1637073263 shutdown_connections 2024-09-11T23:46:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.882+0000 7f0831570740 1 --2- 172.21.15.133:0/1637073263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a28520 0x556719a2a910 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.882+0000 7f0831570740 1 --2- 172.21.15.133:0/1637073263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 0x556719a1dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.882+0000 7f0831570740 1 --2- 172.21.15.133:0/1637073263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1ecf0 0x556719a1d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.882+0000 7f0831570740 1 -- 172.21.15.133:0/1637073263 >> 172.21.15.133:0/1637073263 conn(0x5567199e6310 msgr2=0x556719a19e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:27.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.887+0000 7f0831570740 1 -- 172.21.15.133:0/1637073263 shutdown_connections 2024-09-11T23:46:27.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.887+0000 7f082d734640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08100a2f00 0x7f08100a53c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f081c00d7f0 tx=0x7f081c00d370 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.898+0000 7f0831570740 1 -- 172.21.15.133:0/1637073263 wait complete. 2024-09-11T23:46:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.898+0000 7f0831570740 1 -- start start 2024-09-11T23:46:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.899+0000 7f0831570740 1 Processor -- start 2024-09-11T23:46:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.900+0000 7f0831570740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 0x556719a27e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.900+0000 7f0831570740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a1ecf0 0x556719a250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.900+0000 7f0831570740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a28520 0x556719a25630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.900+0000 7f0831570740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556719a2b270 con 0x556719a28520 2024-09-11T23:46:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.900+0000 7f0831570740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55671993e060 con 0x556719a1d780 2024-09-11T23:46:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.900+0000 7f0831570740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556719a2b3f0 con 0x556719a1ecf0 2024-09-11T23:46:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.900+0000 7f0831570740 10 client.0 _ll_register_callbacks cb 0x5567196918c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:27.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.901+0000 7f082d734640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a28520 0x556719a25630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:27.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.901+0000 7f082d734640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a28520 0x556719a25630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51910/0 (socket says 172.21.15.133:51910) 2024-09-11T23:46:27.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.901+0000 7f082d734640 1 -- 172.21.15.133:0/3892007961 learned_addr learned my addr 172.21.15.133:0/3892007961 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:27.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.904+0000 7f0827fff640 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a1ecf0 0x556719a250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:27.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.904+0000 7f082cf33640 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 0x556719a27e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:27.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.906+0000 7f082cf33640 1 -- 172.21.15.133:0/3892007961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a1ecf0 msgr2=0x556719a250f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:27.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.906+0000 7f082cf33640 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a1ecf0 0x556719a250f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.906+0000 7f082cf33640 1 -- 172.21.15.133:0/3892007961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a28520 msgr2=0x556719a25630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:27.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.906+0000 7f082cf33640 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a28520 0x556719a25630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:27.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.906+0000 7f082cf33640 1 -- 172.21.15.133:0/3892007961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567196f43c0 con 0x556719a1d780 2024-09-11T23:46:27.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.908+0000 7f082cf33640 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 0x556719a27e40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f082002ad00 tx=0x7f08200305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ec84ad9e4359f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:27.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:27.908+0000 7f0807fff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:46:27.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:27 smithi133 ceph-mon[31430]: pgmap v301: 97 pgs: 97 active+clean; 66 GiB data, 256 GiB used, 816 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 632 MiB/s wr, 3.16k op/s 2024-09-11T23:46:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.166+0000 7f0807fff640 1 -- 172.21.15.133:0/3892007961 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820041020 con 0x556719a1d780 2024-09-11T23:46:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.166+0000 7f0831570740 1 -- 172.21.15.133:0/3892007961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f081c00fda0 con 0x556719a1d780 2024-09-11T23:46:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.166+0000 7f0807fff640 1 -- 172.21.15.133:0/3892007961 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f082003c470 con 0x556719a1d780 2024-09-11T23:46:28.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:27 smithi178 ceph-mon[35882]: pgmap v301: 97 pgs: 97 active+clean; 66 GiB data, 256 GiB used, 816 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 632 MiB/s wr, 3.16k op/s 2024-09-11T23:46:28.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.420+0000 7f0807fff640 1 -- 172.21.15.133:0/3892007961 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f082004c050 con 0x556719a1d780 2024-09-11T23:46:28.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.421+0000 7f0831570740 10 client.27365 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:28.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.421+0000 7f0831570740 1 -- 172.21.15.133:0/3892007961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556719a36cd0 con 0x556719a1d780 2024-09-11T23:46:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.423+0000 7f0807fff640 1 -- 172.21.15.133:0/3892007961 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f081c00fda0 con 0x556719a1d780 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.423+0000 7f0807fff640 1 -- 172.21.15.133:0/3892007961 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f08200316f0 con 0x556719a1d780 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.424+0000 7f0831570740 10 client.27365 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.424+0000 7f0831570740 10 client.27365 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.424+0000 7f0831570740 20 client.27365 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.425+0000 7f0831570740 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x556719a38570 0x556719a3a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.425+0000 7f0831570740 4 client.27365 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.425+0000 7f0831570740 1 -- 172.21.15.133:0/3892007961 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556719990860 con 0x556719a38570 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.426+0000 7f0827fff640 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x556719a38570 0x556719a3a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.427+0000 7f0827fff640 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x556719a38570 0x556719a3a9a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=53fb9818058f9bae server_cookie=bf9cbc12978ee83b in_seq=0 out_seq=0 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.427+0000 7f0807fff640 10 client.27365 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.428+0000 7f0807fff640 1 -- 172.21.15.133:0/3892007961 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556719990860 con 0x556719a38570 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.428+0000 7f0807fff640 10 client.27365 handle_command_reply: tid=0 2024-09-11T23:46:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.428+0000 7f0831570740 1 client.27365 shutdown 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.428+0000 7f0831570740 1 -- 172.21.15.133:0/3892007961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 msgr2=0x556719a27e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.428+0000 7f0831570740 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 0x556719a27e40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f082002ad00 tx=0x7f08200305d0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.429+0000 7f0831570740 1 -- 172.21.15.133:0/3892007961 shutdown_connections 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.429+0000 7f0831570740 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x556719a38570 0x556719a3a9a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.429+0000 7f0831570740 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a28520 0x556719a25630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.429+0000 7f0831570740 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a1ecf0 0x556719a250f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.429+0000 7f0831570740 1 --2- 172.21.15.133:0/3892007961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a1d780 0x556719a27e40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.429+0000 7f0831570740 1 -- 172.21.15.133:0/3892007961 >> 172.21.15.133:0/3892007961 conn(0x5567199e6310 msgr2=0x556719a1fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.429+0000 7f0807fff640 0 client.27365 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.429+0000 7f0831570740 1 -- 172.21.15.133:0/3892007961 shutdown_connections 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.430+0000 7f0831570740 1 -- 172.21.15.133:0/3892007961 wait complete. 2024-09-11T23:46:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.430+0000 7f0831570740 20 client.27365 trim_cache size 0 max 16384 2024-09-11T23:46:28.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:27 smithi138 ceph-mon[35142]: pgmap v301: 97 pgs: 97 active+clean; 66 GiB data, 256 GiB used, 816 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 632 MiB/s wr, 3.16k op/s 2024-09-11T23:46:28.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.461+0000 7f0831570740 1 Processor -- start 2024-09-11T23:46:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.461+0000 7f0831570740 1 -- start start 2024-09-11T23:46:28.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.467+0000 7f0831570740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1d780 0x556719a34560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:28.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.467+0000 7f0831570740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a1ecf0 0x556719a202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:28.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.467+0000 7f0831570740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 0x556719a20810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:28.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.468+0000 7f0831570740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08281c4dc0 con 0x556719a1d780 2024-09-11T23:46:28.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.468+0000 7f0831570740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08281c6900 con 0x556719a28520 2024-09-11T23:46:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.468+0000 7f0831570740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08281c5490 con 0x556719a1ecf0 2024-09-11T23:46:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.469+0000 7f082d734640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 0x556719a20810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.469+0000 7f082d734640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 0x556719a20810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37300/0 (socket says 172.21.15.133:37300) 2024-09-11T23:46:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.469+0000 7f082d734640 1 -- 172.21.15.133:0/2817921049 learned_addr learned my addr 172.21.15.133:0/2817921049 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.469+0000 7f0827fff640 1 --2- 172.21.15.133:0/2817921049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a1ecf0 0x556719a202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:28.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.473+0000 7f082cf33640 1 --2- 172.21.15.133:0/2817921049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1d780 0x556719a34560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:28.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.473+0000 7f0827fff640 1 -- 172.21.15.133:0/2817921049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 msgr2=0x556719a20810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:28.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.473+0000 7f0827fff640 1 --2- 172.21.15.133:0/2817921049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 0x556719a20810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.473+0000 7f0827fff640 1 -- 172.21.15.133:0/2817921049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1d780 msgr2=0x556719a34560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:28.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.473+0000 7f0827fff640 1 --2- 172.21.15.133:0/2817921049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1d780 0x556719a34560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.473+0000 7f0827fff640 1 -- 172.21.15.133:0/2817921049 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556719a281b0 con 0x556719a1ecf0 2024-09-11T23:46:28.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.477+0000 7f0827fff640 1 --2- 172.21.15.133:0/2817921049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a1ecf0 0x556719a202d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f08180a3a80 tx=0x7f0818000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d71480e5a9e66198 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:28.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.481+0000 7f080e7fc640 1 -- 172.21.15.133:0/2817921049 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08180b8070 con 0x556719a1ecf0 2024-09-11T23:46:28.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.481+0000 7f080e7fc640 1 -- 172.21.15.133:0/2817921049 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08180a3fb0 con 0x556719a1ecf0 2024-09-11T23:46:28.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.483+0000 7f0831570740 1 -- 172.21.15.133:0/2817921049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a1ecf0 msgr2=0x556719a202d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:28.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.483+0000 7f0831570740 1 --2- 172.21.15.133:0/2817921049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a1ecf0 0x556719a202d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f08180a3a80 tx=0x7f0818000bc0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.483+0000 7f0831570740 1 -- 172.21.15.133:0/2817921049 shutdown_connections 2024-09-11T23:46:28.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.483+0000 7f0831570740 1 --2- 172.21.15.133:0/2817921049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 0x556719a20810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.483+0000 7f0831570740 1 --2- 172.21.15.133:0/2817921049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a1ecf0 0x556719a202d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.483+0000 7f0831570740 1 --2- 172.21.15.133:0/2817921049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1d780 0x556719a34560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.483+0000 7f0831570740 1 -- 172.21.15.133:0/2817921049 >> 172.21.15.133:0/2817921049 conn(0x5567199e6310 msgr2=0x556719a3a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:28.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.491+0000 7f0831570740 1 -- 172.21.15.133:0/2817921049 shutdown_connections 2024-09-11T23:46:28.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.491+0000 7f0831570740 1 -- 172.21.15.133:0/2817921049 wait complete. 2024-09-11T23:46:28.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.491+0000 7f0831570740 1 -- start start 2024-09-11T23:46:28.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.492+0000 7f0831570740 1 Processor -- start 2024-09-11T23:46:28.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.493+0000 7f0831570740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1d780 0x556719a2b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:28.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.501+0000 7f0831570740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 0x556719a2bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:28.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.502+0000 7f0831570740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a2c110 0x556719aa1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:28.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.502+0000 7f0831570740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08281c6230 con 0x556719a1d780 2024-09-11T23:46:28.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.503+0000 7f0827fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 0x556719a2bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:28.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.503+0000 7f0827fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 0x556719a2bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37312/0 (socket says 172.21.15.133:37312) 2024-09-11T23:46:28.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.503+0000 7f0827fff640 1 -- 172.21.15.133:0/3374419452 learned_addr learned my addr 172.21.15.133:0/3374419452 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:28.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.503+0000 7f082d734640 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a2c110 0x556719aa1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:28.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.503+0000 7f0831570740 1 -- 172.21.15.133:0/3374419452 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08281c5b60 con 0x556719a28520 2024-09-11T23:46:28.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.503+0000 7f0831570740 1 -- 172.21.15.133:0/3374419452 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08281c2dc0 con 0x556719a2c110 2024-09-11T23:46:28.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.505+0000 7f082cf33640 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1d780 0x556719a2b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:28.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.505+0000 7f0831570740 10 client.0 _ll_register_callbacks cb 0x7f0820034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:28.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.512+0000 7f082cf33640 1 -- 172.21.15.133:0/3374419452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a2c110 msgr2=0x556719aa1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:28.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.512+0000 7f082cf33640 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a2c110 0x556719aa1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.512+0000 7f082cf33640 1 -- 172.21.15.133:0/3374419452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 msgr2=0x556719a2bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:28.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.512+0000 7f082cf33640 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 0x556719a2bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.512+0000 7f082cf33640 1 -- 172.21.15.133:0/3374419452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556719aa1780 con 0x556719a1d780 2024-09-11T23:46:28.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.513+0000 7f082cf33640 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1d780 0x556719a2b690 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0820009be0 tx=0x7f08200092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a306d3199e1ec09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:28.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.513+0000 7f0807fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:46:28.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.515+0000 7f0807fff640 1 -- 172.21.15.133:0/3374419452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820041070 con 0x556719a1d780 2024-09-11T23:46:28.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.515+0000 7f0807fff640 1 -- 172.21.15.133:0/3374419452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f0820031440 con 0x556719a1d780 2024-09-11T23:46:28.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.515+0000 7f0831570740 1 -- 172.21.15.133:0/3374419452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55671998ad70 con 0x556719a1d780 2024-09-11T23:46:28.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.516+0000 7f0807fff640 1 -- 172.21.15.133:0/3374419452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0820036040 con 0x556719a1d780 2024-09-11T23:46:28.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.516+0000 7f0831570740 10 client.17847 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:28.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.516+0000 7f0831570740 1 -- 172.21.15.133:0/3374419452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556719aa29b0 con 0x556719a1d780 2024-09-11T23:46:28.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.518+0000 7f0807fff640 1 -- 172.21.15.133:0/3374419452 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0820030d60 con 0x556719a1d780 2024-09-11T23:46:28.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.518+0000 7f0807fff640 1 -- 172.21.15.133:0/3374419452 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0820031ce0 con 0x556719a1d780 2024-09-11T23:46:28.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.520+0000 7f0831570740 10 client.17847 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:28.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.520+0000 7f0831570740 10 client.17847 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:46:28.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.520+0000 7f0831570740 20 client.17847 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:28.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.520+0000 7f0831570740 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x556719aa3070 0x556719aa5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:28.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.521+0000 7f0831570740 4 client.17847 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:46:28.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.521+0000 7f0831570740 1 -- 172.21.15.133:0/3374419452 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55671998ad70 con 0x556719aa3070 2024-09-11T23:46:28.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.521+0000 7f0827fff640 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x556719aa3070 0x556719aa5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:28.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.523+0000 7f0827fff640 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x556719aa3070 0x556719aa5460 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1802d9e8668a24a server_cookie=df74655ea7886029 in_seq=0 out_seq=0 2024-09-11T23:46:28.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.525+0000 7f0807fff640 10 client.17847 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:46:28.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.525+0000 7f0807fff640 1 -- 172.21.15.133:0/3374419452 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55671998ad70 con 0x556719aa3070 2024-09-11T23:46:28.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.525+0000 7f0807fff640 10 client.17847 handle_command_reply: tid=0 2024-09-11T23:46:28.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.526+0000 7f0831570740 1 client.17847 shutdown 2024-09-11T23:46:28.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.526+0000 7f0831570740 1 -- 172.21.15.133:0/3374419452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1d780 msgr2=0x556719a2b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:28.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.526+0000 7f0831570740 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1d780 0x556719a2b690 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0820009be0 tx=0x7f08200092f0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.528+0000 7f0831570740 1 -- 172.21.15.133:0/3374419452 shutdown_connections 2024-09-11T23:46:28.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.528+0000 7f0831570740 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x556719aa3070 0x556719aa5460 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.529+0000 7f0831570740 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556719a2c110 0x556719aa1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.529+0000 7f0831570740 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556719a28520 0x556719a2bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.529+0000 7f0831570740 1 --2- 172.21.15.133:0/3374419452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556719a1d780 0x556719a2b690 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.529+0000 7f0831570740 1 -- 172.21.15.133:0/3374419452 >> 172.21.15.133:0/3374419452 conn(0x5567199e6310 msgr2=0x556719a249b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:28.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.529+0000 7f0807fff640 0 client.17847 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:46:28.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.529+0000 7f0831570740 1 -- 172.21.15.133:0/3374419452 shutdown_connections 2024-09-11T23:46:28.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.530+0000 7f0831570740 1 -- 172.21.15.133:0/3374419452 wait complete. 2024-09-11T23:46:28.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.531+0000 7f0831570740 20 client.17847 trim_cache size 0 max 16384 2024-09-11T23:46:28.537 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:46:28.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.537+0000 7f082df35640 1 -- 172.21.15.133:0/2562457809 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08100a2f00 msgr2=0x7f08100a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:28.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.537+0000 7f082df35640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08100a2f00 0x7f08100a53c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f081c00d7f0 tx=0x7f081c00d370 comp rx=0 tx=0).stop 2024-09-11T23:46:28.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.537+0000 7f082df35640 1 -- 172.21.15.133:0/2562457809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0828110190 msgr2=0x7f0828111760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:28.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.537+0000 7f082df35640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0828110190 0x7f0828111760 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f081800d950 tx=0x7f081800de10 comp rx=0 tx=0).stop 2024-09-11T23:46:28.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.537+0000 7f082df35640 1 -- 172.21.15.133:0/2562457809 shutdown_connections 2024-09-11T23:46:28.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.537+0000 7f082df35640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08100a2f00 0x7f08100a53c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.537+0000 7f082df35640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f082811b140 0x7f0828111ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.537+0000 7f082df35640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0828110190 0x7f0828111760 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.537+0000 7f082df35640 1 --2- 172.21.15.133:0/2562457809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f082806ea30 0x7f08281186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:28.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.537+0000 7f082df35640 1 -- 172.21.15.133:0/2562457809 >> 172.21.15.133:0/2562457809 conn(0x7f0828073180 msgr2=0x7f082810e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:28.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.539+0000 7f082df35640 1 -- 172.21.15.133:0/2562457809 shutdown_connections 2024-09-11T23:46:28.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:28.541+0000 7f082df35640 1 -- 172.21.15.133:0/2562457809 wait complete. 2024-09-11T23:46:29.722 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:46:29.942 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:29 smithi133 ceph-mon[31430]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:46:29.942 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:29 smithi133 ceph-mon[31430]: pgmap v303: 97 pgs: 97 active+clean; 68 GiB data, 265 GiB used, 807 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 728 MiB/s wr, 1.57k op/s 2024-09-11T23:46:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.157+0000 7f0bd5043640 1 -- 172.21.15.133:0/2424228444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd011b140 msgr2=0x7f0bd011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.157+0000 7f0bd5043640 1 --2- 172.21.15.133:0/2424228444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd011b140 0x7f0bd011d530 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009900 tx=0x7f0bc402a410 comp rx=0 tx=0).stop 2024-09-11T23:46:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.159+0000 7f0bd5043640 1 -- 172.21.15.133:0/2424228444 shutdown_connections 2024-09-11T23:46:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.159+0000 7f0bd5043640 1 --2- 172.21.15.133:0/2424228444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd011b140 0x7f0bd011d530 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.159+0000 7f0bd5043640 1 --2- 172.21.15.133:0/2424228444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd01095d0 0x7f0bd0109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.159+0000 7f0bd5043640 1 --2- 172.21.15.133:0/2424228444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd01091f0 0x7f0bd0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.159+0000 7f0bd5043640 1 -- 172.21.15.133:0/2424228444 >> 172.21.15.133:0/2424228444 conn(0x7f0bd006cbf0 msgr2=0x7f0bd006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.161+0000 7f0bd5043640 1 -- 172.21.15.133:0/2424228444 shutdown_connections 2024-09-11T23:46:30.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.161+0000 7f0bd5043640 1 -- 172.21.15.133:0/2424228444 wait complete. 2024-09-11T23:46:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.162+0000 7f0bd5043640 1 Processor -- start 2024-09-11T23:46:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.163+0000 7f0bd5043640 1 -- start start 2024-09-11T23:46:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.163+0000 7f0bd5043640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd01091f0 0x7f0bd011a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.163+0000 7f0bd5043640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd01095d0 0x7f0bd011a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.164+0000 7f0bd5043640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd011b140 0x7f0bd0114450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.164+0000 7f0bd5043640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0110ef0 con 0x7f0bd01095d0 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.164+0000 7f0bd5043640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0110d70 con 0x7f0bd011b140 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.164+0000 7f0bd5043640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0111070 con 0x7f0bd01091f0 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.164+0000 7f0bcf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd01095d0 0x7f0bd011a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.164+0000 7f0bcf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd01095d0 0x7f0bd011a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51948/0 (socket says 172.21.15.133:51948) 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.164+0000 7f0bcf7fe640 1 -- 172.21.15.133:0/4181002162 learned_addr learned my addr 172.21.15.133:0/4181002162 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.164+0000 7f0bcffff640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd01091f0 0x7f0bd011a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.165+0000 7f0bcf7fe640 1 -- 172.21.15.133:0/4181002162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd01091f0 msgr2=0x7f0bd011a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.165+0000 7f0bcf7fe640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd01091f0 0x7f0bd011a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.165+0000 7f0bcf7fe640 1 -- 172.21.15.133:0/4181002162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd011b140 msgr2=0x7f0bd0114450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.165+0000 7f0bcf7fe640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd011b140 0x7f0bd0114450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.165+0000 7f0bcf7fe640 1 -- 172.21.15.133:0/4181002162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd0114d10 con 0x7f0bd01095d0 2024-09-11T23:46:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.165+0000 7f0bcf7fe640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd01095d0 0x7f0bd011a920 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800ebc0 tx=0x7f0bc800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.166+0000 7f0bcd7fa640 1 -- 172.21.15.133:0/4181002162 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8010630 con 0x7f0bd01095d0 2024-09-11T23:46:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.166+0000 7f0bcd7fa640 1 -- 172.21.15.133:0/4181002162 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc8010cb0 con 0x7f0bd01095d0 2024-09-11T23:46:30.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.167+0000 7f0bcd7fa640 1 -- 172.21.15.133:0/4181002162 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8018c00 con 0x7f0bd01095d0 2024-09-11T23:46:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.169+0000 7f0bd5043640 1 -- 172.21.15.133:0/4181002162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd0115000 con 0x7f0bd01095d0 2024-09-11T23:46:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.169+0000 7f0bd5043640 1 -- 172.21.15.133:0/4181002162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd01bc470 con 0x7f0bd01095d0 2024-09-11T23:46:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.179+0000 7f0bd9046740 1 Processor -- start 2024-09-11T23:46:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.180+0000 7f0bd9046740 1 -- start start 2024-09-11T23:46:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.180+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a1fcc0 0x556839a200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.181+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a205e0 0x556839a2a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.181+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a2ae80 0x556839a2d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.181+0000 7f0bd9046740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5568399842f0 con 0x556839a2ae80 2024-09-11T23:46:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.181+0000 7f0bd9046740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55683992c270 con 0x556839a1fcc0 2024-09-11T23:46:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.182+0000 7f0bcf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a2ae80 0x556839a2d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.181+0000 7f0bd4842640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a1fcc0 0x556839a200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.182+0000 7f0bd4842640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a1fcc0 0x556839a200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37340/0 (socket says 172.21.15.133:37340) 2024-09-11T23:46:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.182+0000 7f0bd4842640 1 -- 172.21.15.133:0/669025422 learned_addr learned my addr 172.21.15.133:0/669025422 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.182+0000 7f0bd9046740 1 -- 172.21.15.133:0/669025422 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5568399802b0 con 0x556839a205e0 2024-09-11T23:46:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.182+0000 7f0bcd7fa640 1 -- 172.21.15.133:0/4181002162 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0bc80107d0 con 0x7f0bd01095d0 2024-09-11T23:46:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.183+0000 7f0bd4842640 1 -- 172.21.15.133:0/669025422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a205e0 msgr2=0x556839a2a940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:46:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.183+0000 7f0bd4842640 1 --2- 172.21.15.133:0/669025422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a205e0 0x556839a2a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.183+0000 7f0bd4842640 1 -- 172.21.15.133:0/669025422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a2ae80 msgr2=0x556839a2d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.183+0000 7f0bd4842640 1 --2- 172.21.15.133:0/669025422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a2ae80 0x556839a2d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.183+0000 7f0bd4842640 1 -- 172.21.15.133:0/669025422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683992b340 con 0x556839a1fcc0 2024-09-11T23:46:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.184+0000 7f0bd4842640 1 --2- 172.21.15.133:0/669025422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a1fcc0 0x556839a200a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009230 tx=0x7f0bc4009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f3435103c3f29ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.186+0000 7f0bcd7fa640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bb80a2ce0 0x7f0bb80a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.189 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.188+0000 7f0bb67fc640 1 -- 172.21.15.133:0/669025422 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc403e030 con 0x556839a1fcc0 2024-09-11T23:46:30.189 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.188+0000 7f0bcd7fa640 1 -- 172.21.15.133:0/4181002162 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f0bc8099e60 con 0x7f0bd01095d0 2024-09-11T23:46:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.191+0000 7f0bcffff640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bb80a2ce0 0x7f0bb80a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.193+0000 7f0bb67fc640 1 -- 172.21.15.133:0/669025422 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc4031ba0 con 0x556839a1fcc0 2024-09-11T23:46:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.193+0000 7f0bb67fc640 1 -- 172.21.15.133:0/669025422 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4035040 con 0x556839a1fcc0 2024-09-11T23:46:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.194+0000 7f0bd9046740 1 -- 172.21.15.133:0/669025422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a1fcc0 msgr2=0x556839a200a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.194+0000 7f0bd9046740 1 --2- 172.21.15.133:0/669025422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a1fcc0 0x556839a200a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009230 tx=0x7f0bc4009680 comp rx=0 tx=0).stop 2024-09-11T23:46:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.195+0000 7f0bcffff640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bb80a2ce0 0x7f0bb80a51a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x556839a1d090 tx=0x7f0bc000b410 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.197+0000 7f0bd9046740 1 -- 172.21.15.133:0/669025422 shutdown_connections 2024-09-11T23:46:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.197+0000 7f0bd9046740 1 --2- 172.21.15.133:0/669025422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a2ae80 0x556839a2d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.197+0000 7f0bd9046740 1 --2- 172.21.15.133:0/669025422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a205e0 0x556839a2a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.197+0000 7f0bd9046740 1 --2- 172.21.15.133:0/669025422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a1fcc0 0x556839a200a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.197+0000 7f0bd9046740 1 -- 172.21.15.133:0/669025422 >> 172.21.15.133:0/669025422 conn(0x556839a1c230 msgr2=0x556839a1c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.197+0000 7f0bd9046740 1 -- 172.21.15.133:0/669025422 shutdown_connections 2024-09-11T23:46:30.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:29 smithi138 ceph-mon[35142]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:46:30.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:29 smithi138 ceph-mon[35142]: pgmap v303: 97 pgs: 97 active+clean; 68 GiB data, 265 GiB used, 807 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 728 MiB/s wr, 1.57k op/s 2024-09-11T23:46:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.197+0000 7f0bd9046740 1 -- 172.21.15.133:0/669025422 wait complete. 2024-09-11T23:46:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.198+0000 7f0bd9046740 1 -- start start 2024-09-11T23:46:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.198+0000 7f0bd9046740 1 Processor -- start 2024-09-11T23:46:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.199+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a1fcc0 0x556839a27dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.199+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a205e0 0x556839a28310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.199+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a2ae80 0x556839a28850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.199+0000 7f0bd9046740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556839a2da50 con 0x556839a205e0 2024-09-11T23:46:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.199+0000 7f0bd9046740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55683990d290 con 0x556839a2ae80 2024-09-11T23:46:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.199+0000 7f0bd9046740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556839a2dbd0 con 0x556839a1fcc0 2024-09-11T23:46:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.199+0000 7f0bd9046740 10 client.0 _ll_register_callbacks cb 0x55683993e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.200+0000 7f0bcffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a1fcc0 0x556839a27dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.200+0000 7f0bcffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a1fcc0 0x556839a27dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42282/0 (socket says 172.21.15.133:42282) 2024-09-11T23:46:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.200+0000 7f0bcffff640 1 -- 172.21.15.133:0/1651310697 learned_addr learned my addr 172.21.15.133:0/1651310697 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.200+0000 7f0bd4842640 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a2ae80 0x556839a28850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.201+0000 7f0bcf7fe640 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a205e0 0x556839a28310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.202+0000 7f0bd4842640 1 -- 172.21.15.133:0/1651310697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a1fcc0 msgr2=0x556839a27dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.202+0000 7f0bd4842640 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a1fcc0 0x556839a27dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.202+0000 7f0bd4842640 1 -- 172.21.15.133:0/1651310697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a205e0 msgr2=0x556839a28310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.202+0000 7f0bd4842640 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a205e0 0x556839a28310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.202+0000 7f0bd4842640 1 -- 172.21.15.133:0/1651310697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556839697210 con 0x556839a2ae80 2024-09-11T23:46:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.203+0000 7f0bd4842640 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a2ae80 0x556839a28850 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4005ab0 tx=0x7f0bc4031f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73506a497bde8c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.203+0000 7f0baf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:46:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.204+0000 7f0baf7fe640 1 -- 172.21.15.133:0/1651310697 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc403e040 con 0x556839a2ae80 2024-09-11T23:46:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.205+0000 7f0bd9046740 1 -- 172.21.15.133:0/1651310697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc4031ba0 con 0x556839a2ae80 2024-09-11T23:46:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.205+0000 7f0baf7fe640 1 -- 172.21.15.133:0/1651310697 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f0bc404c680 con 0x556839a2ae80 2024-09-11T23:46:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.206+0000 7f0baf7fe640 1 -- 172.21.15.133:0/1651310697 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0bc4035040 con 0x556839a2ae80 2024-09-11T23:46:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.208+0000 7f0bd9046740 10 client.27380 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.208+0000 7f0bd9046740 1 -- 172.21.15.133:0/1651310697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556839763620 con 0x556839a2ae80 2024-09-11T23:46:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.211+0000 7f0baf7fe640 1 -- 172.21.15.133:0/1651310697 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc4031ba0 con 0x556839a2ae80 2024-09-11T23:46:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.213+0000 7f0baf7fe640 1 -- 172.21.15.133:0/1651310697 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0bc403a3f0 con 0x556839a2ae80 2024-09-11T23:46:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.214+0000 7f0bd9046740 10 client.27380 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.214+0000 7f0bd9046740 10 client.27380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.214+0000 7f0bd9046740 20 client.27380 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.214+0000 7f0bd9046740 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556839a3acd0 0x556839a3d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.214+0000 7f0bd9046740 4 client.27380 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.214+0000 7f0bd9046740 1 -- 172.21.15.133:0/1651310697 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556839991a20 con 0x556839a3acd0 2024-09-11T23:46:30.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.219+0000 7f0bcffff640 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556839a3acd0 0x556839a3d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.232+0000 7f0bcffff640 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556839a3acd0 0x556839a3d0c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=270e4c479b2f275 server_cookie=41dda8d077768648 in_seq=0 out_seq=0 2024-09-11T23:46:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.232+0000 7f0baf7fe640 10 client.27380 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.237+0000 7f0baf7fe640 1 -- 172.21.15.133:0/1651310697 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556839991a20 con 0x556839a3acd0 2024-09-11T23:46:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.237+0000 7f0baf7fe640 10 client.27380 handle_command_reply: tid=0 2024-09-11T23:46:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.237+0000 7f0bd9046740 1 client.27380 shutdown 2024-09-11T23:46:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.237+0000 7f0bd9046740 1 -- 172.21.15.133:0/1651310697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a2ae80 msgr2=0x556839a28850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.237+0000 7f0bd9046740 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a2ae80 0x556839a28850 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4005ab0 tx=0x7f0bc4031f80 comp rx=0 tx=0).stop 2024-09-11T23:46:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.240+0000 7f0bd9046740 1 -- 172.21.15.133:0/1651310697 shutdown_connections 2024-09-11T23:46:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.241+0000 7f0bd9046740 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556839a3acd0 0x556839a3d0c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.241+0000 7f0bd9046740 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a2ae80 0x556839a28850 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.241+0000 7f0bd9046740 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a205e0 0x556839a28310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.241+0000 7f0bd9046740 1 --2- 172.21.15.133:0/1651310697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a1fcc0 0x556839a27dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.241+0000 7f0bd9046740 1 -- 172.21.15.133:0/1651310697 >> 172.21.15.133:0/1651310697 conn(0x556839a1c230 msgr2=0x556839a21f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.241+0000 7f0baf7fe640 0 client.27380 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.243+0000 7f0bd9046740 1 -- 172.21.15.133:0/1651310697 shutdown_connections 2024-09-11T23:46:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.244+0000 7f0bd9046740 1 -- 172.21.15.133:0/1651310697 wait complete. 2024-09-11T23:46:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.245+0000 7f0bd9046740 20 client.27380 trim_cache size 0 max 16384 2024-09-11T23:46:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.287+0000 7f0bd9046740 1 Processor -- start 2024-09-11T23:46:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.287+0000 7f0bd9046740 1 -- start start 2024-09-11T23:46:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.287+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a1fcc0 0x556839a28680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.287+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a205e0 0x556839a25010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.287+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 0x556839a25550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.288+0000 7f0bd9046740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01c4890 con 0x556839a1fcc0 2024-09-11T23:46:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.288+0000 7f0bd9046740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01c63d0 con 0x556839a205e0 2024-09-11T23:46:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.288+0000 7f0bd9046740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01c4f60 con 0x556839a2ae80 2024-09-11T23:46:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.288+0000 7f0bd4842640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 0x556839a25550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.288+0000 7f0bcf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a205e0 0x556839a25010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.288+0000 7f0bd4842640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 0x556839a25550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42296/0 (socket says 172.21.15.133:42296) 2024-09-11T23:46:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.288+0000 7f0bcf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a205e0 0x556839a25010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:37364/0 (socket says 172.21.15.133:37364) 2024-09-11T23:46:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.288+0000 7f0bd4842640 1 -- 172.21.15.133:0/4164252854 learned_addr learned my addr 172.21.15.133:0/4164252854 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.290+0000 7f0bcf7fe640 1 -- 172.21.15.133:0/4164252854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 msgr2=0x556839a25550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.290+0000 7f0bcf7fe640 1 --2- 172.21.15.133:0/4164252854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 0x556839a25550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.290+0000 7f0bcf7fe640 1 -- 172.21.15.133:0/4164252854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a1fcc0 msgr2=0x556839a28680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.290+0000 7f0bcf7fe640 1 --2- 172.21.15.133:0/4164252854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a1fcc0 0x556839a28680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.290+0000 7f0bcf7fe640 1 -- 172.21.15.133:0/4164252854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556839a2a920 con 0x556839a205e0 2024-09-11T23:46:30.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:29 smithi178 ceph-mon[35882]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:46:30.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:29 smithi178 ceph-mon[35882]: pgmap v303: 97 pgs: 97 active+clean; 68 GiB data, 265 GiB used, 807 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 728 MiB/s wr, 1.57k op/s 2024-09-11T23:46:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.290+0000 7f0bcf7fe640 1 --2- 172.21.15.133:0/4164252854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a205e0 0x556839a25010 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80ccff0 tx=0x7f0bc8009a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bcf7d92b73dea44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.291+0000 7f0bb5ffb640 1 -- 172.21.15.133:0/4164252854 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8010e60 con 0x556839a205e0 2024-09-11T23:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.291+0000 7f0bb5ffb640 1 -- 172.21.15.133:0/4164252854 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc80a0170 con 0x556839a205e0 2024-09-11T23:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.291+0000 7f0bb5ffb640 1 -- 172.21.15.133:0/4164252854 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80a45a0 con 0x556839a205e0 2024-09-11T23:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.293+0000 7f0bd9046740 1 -- 172.21.15.133:0/4164252854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a205e0 msgr2=0x556839a25010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.293+0000 7f0bd9046740 1 --2- 172.21.15.133:0/4164252854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a205e0 0x556839a25010 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80ccff0 tx=0x7f0bc8009a60 comp rx=0 tx=0).stop 2024-09-11T23:46:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.308+0000 7f0bd9046740 1 -- 172.21.15.133:0/4164252854 shutdown_connections 2024-09-11T23:46:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.308+0000 7f0bd9046740 1 --2- 172.21.15.133:0/4164252854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 0x556839a25550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.308+0000 7f0bd9046740 1 --2- 172.21.15.133:0/4164252854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a205e0 0x556839a25010 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.308+0000 7f0bd9046740 1 --2- 172.21.15.133:0/4164252854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a1fcc0 0x556839a28680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.308+0000 7f0bd9046740 1 -- 172.21.15.133:0/4164252854 >> 172.21.15.133:0/4164252854 conn(0x556839a1c230 msgr2=0x556839a21f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.309+0000 7f0bd9046740 1 -- 172.21.15.133:0/4164252854 shutdown_connections 2024-09-11T23:46:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.309+0000 7f0bd9046740 1 -- 172.21.15.133:0/4164252854 wait complete. 2024-09-11T23:46:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.309+0000 7f0bd9046740 1 -- start start 2024-09-11T23:46:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.310+0000 7f0bd9046740 1 Processor -- start 2024-09-11T23:46:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.310+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a1fcc0 0x556839a40450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.310+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a205e0 0x556839a40990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.311+0000 7f0bd9046740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 0x556839a3d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.311+0000 7f0bd9046740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01c5d00 con 0x556839a205e0 2024-09-11T23:46:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.311+0000 7f0bd9046740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01c5630 con 0x556839a1fcc0 2024-09-11T23:46:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.311+0000 7f0bd9046740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01c2920 con 0x556839a2ae80 2024-09-11T23:46:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.311+0000 7f0bd9046740 10 client.0 _ll_register_callbacks cb 0x7f0bc4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.311+0000 7f0bd4842640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 0x556839a3d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.311+0000 7f0bd4842640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 0x556839a3d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42308/0 (socket says 172.21.15.133:42308) 2024-09-11T23:46:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.311+0000 7f0bd4842640 1 -- 172.21.15.133:0/389640930 learned_addr learned my addr 172.21.15.133:0/389640930 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.312+0000 7f0bd4842640 1 -- 172.21.15.133:0/389640930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a1fcc0 msgr2=0x556839a40450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:46:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.312+0000 7f0bd4842640 1 --2- 172.21.15.133:0/389640930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a1fcc0 0x556839a40450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.312+0000 7f0bd4842640 1 -- 172.21.15.133:0/389640930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a205e0 msgr2=0x556839a40990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.312+0000 7f0bd4842640 1 --2- 172.21.15.133:0/389640930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a205e0 0x556839a40990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.312+0000 7f0bd4842640 1 -- 172.21.15.133:0/389640930 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556839a3e210 con 0x556839a2ae80 2024-09-11T23:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.328+0000 7f0bd4842640 1 --2- 172.21.15.133:0/389640930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 0x556839a3d9e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4005ab0 tx=0x7f0bc4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf36cf44d1eb0a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.329+0000 7f0baf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.329+0000 7f0baf7fe640 1 -- 172.21.15.133:0/389640930 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4059070 con 0x556839a2ae80 2024-09-11T23:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.329+0000 7f0baf7fe640 1 -- 172.21.15.133:0/389640930 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f0bc403aa30 con 0x556839a2ae80 2024-09-11T23:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.330+0000 7f0bd9046740 1 -- 172.21.15.133:0/389640930 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc80a0170 con 0x556839a2ae80 2024-09-11T23:46:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.348+0000 7f0baf7fe640 1 -- 172.21.15.133:0/389640930 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0bc403e040 con 0x556839a2ae80 2024-09-11T23:46:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.348+0000 7f0bd9046740 10 client.27181 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.348+0000 7f0bd9046740 1 -- 172.21.15.133:0/389640930 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556839aa5240 con 0x556839a2ae80 2024-09-11T23:46:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.351+0000 7f0baf7fe640 1 -- 172.21.15.133:0/389640930 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc80a0170 con 0x556839a2ae80 2024-09-11T23:46:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.351+0000 7f0baf7fe640 1 -- 172.21.15.133:0/389640930 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0bc403ac50 con 0x556839a2ae80 2024-09-11T23:46:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.351+0000 7f0bd9046740 10 client.27181 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.351+0000 7f0bd9046740 10 client.27181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.351+0000 7f0bd9046740 20 client.27181 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.351+0000 7f0bd9046740 1 --2- 172.21.15.133:0/389640930 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556839aa5910 0x556839aa7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.351+0000 7f0bd9046740 4 client.27181 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:46:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.351+0000 7f0bd9046740 1 -- 172.21.15.133:0/389640930 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55683998bd70 con 0x556839aa5910 2024-09-11T23:46:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.352+0000 7f0bcffff640 1 --2- 172.21.15.133:0/389640930 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556839aa5910 0x556839aa7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.356+0000 7f0bcffff640 1 --2- 172.21.15.133:0/389640930 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556839aa5910 0x556839aa7d50 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d611733ccf06f2b4 server_cookie=2233f477791e776d in_seq=0 out_seq=0 2024-09-11T23:46:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.356+0000 7f0baf7fe640 10 client.27181 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.834+0000 7f0baf7fe640 1 -- 172.21.15.133:0/389640930 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55683998bd70 con 0x556839aa5910 2024-09-11T23:46:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.834+0000 7f0baf7fe640 10 client.27181 handle_command_reply: tid=0 2024-09-11T23:46:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.834+0000 7f0bd9046740 1 client.27181 shutdown 2024-09-11T23:46:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0bd9046740 1 -- 172.21.15.133:0/389640930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 msgr2=0x556839a3d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0bd9046740 1 --2- 172.21.15.133:0/389640930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 0x556839a3d9e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4005ab0 tx=0x7f0bc4004270 comp rx=0 tx=0).stop 2024-09-11T23:46:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0bd9046740 1 -- 172.21.15.133:0/389640930 shutdown_connections 2024-09-11T23:46:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0bd9046740 1 --2- 172.21.15.133:0/389640930 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x556839aa5910 0x556839aa7d50 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0bd9046740 1 --2- 172.21.15.133:0/389640930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556839a2ae80 0x556839a3d9e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0bd9046740 1 --2- 172.21.15.133:0/389640930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556839a205e0 0x556839a40990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0bd9046740 1 --2- 172.21.15.133:0/389640930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556839a1fcc0 0x556839a40450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0bd9046740 1 -- 172.21.15.133:0/389640930 >> 172.21.15.133:0/389640930 conn(0x556839a1c230 msgr2=0x556839a29630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0baf7fe640 0 client.27181 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0bd9046740 1 -- 172.21.15.133:0/389640930 shutdown_connections 2024-09-11T23:46:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0bd9046740 1 -- 172.21.15.133:0/389640930 wait complete. 2024-09-11T23:46:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.835+0000 7f0bd9046740 20 client.27181 trim_cache size 0 max 16384 2024-09-11T23:46:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:46:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:46:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:46:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:46:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.837+0000 7f0bd5043640 1 -- 172.21.15.133:0/4181002162 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bb80a2ce0 msgr2=0x7f0bb80a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.837+0000 7f0bd5043640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bb80a2ce0 0x7f0bb80a51a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x556839a1d090 tx=0x7f0bc000b410 comp rx=0 tx=0).stop 2024-09-11T23:46:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.837+0000 7f0bd5043640 1 -- 172.21.15.133:0/4181002162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd01095d0 msgr2=0x7f0bd011a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.837+0000 7f0bd5043640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd01095d0 0x7f0bd011a920 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800ebc0 tx=0x7f0bc800c6a0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.840+0000 7f0bd5043640 1 -- 172.21.15.133:0/4181002162 shutdown_connections 2024-09-11T23:46:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.840+0000 7f0bd5043640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bb80a2ce0 0x7f0bb80a51a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.840+0000 7f0bd5043640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd011b140 0x7f0bd0114450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.840+0000 7f0bd5043640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd01095d0 0x7f0bd011a920 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.840+0000 7f0bd5043640 1 --2- 172.21.15.133:0/4181002162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd01091f0 0x7f0bd011a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.840+0000 7f0bd5043640 1 -- 172.21.15.133:0/4181002162 >> 172.21.15.133:0/4181002162 conn(0x7f0bd006cbf0 msgr2=0x7f0bd0071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.841+0000 7f0bd5043640 1 -- 172.21.15.133:0/4181002162 shutdown_connections 2024-09-11T23:46:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:30.841+0000 7f0bd5043640 1 -- 172.21.15.133:0/4181002162 wait complete. 2024-09-11T23:46:30.906 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:32.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.775+0000 7ff476641640 1 -- 172.21.15.133:0/1284995336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47011b140 msgr2=0x7ff47011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:32.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.775+0000 7ff476641640 1 --2- 172.21.15.133:0/1284995336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47011b140 0x7ff47011d530 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff46001a910 tx=0x7ff46003b470 comp rx=0 tx=0).stop 2024-09-11T23:46:32.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.781+0000 7ff476641640 1 -- 172.21.15.133:0/1284995336 shutdown_connections 2024-09-11T23:46:32.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.781+0000 7ff476641640 1 --2- 172.21.15.133:0/1284995336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47011b140 0x7ff47011d530 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.781+0000 7ff476641640 1 --2- 172.21.15.133:0/1284995336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff470109830 0x7ff470109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.781+0000 7ff476641640 1 --2- 172.21.15.133:0/1284995336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff470109450 0x7ff470110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.781+0000 7ff476641640 1 -- 172.21.15.133:0/1284995336 >> 172.21.15.133:0/1284995336 conn(0x7ff47006e140 msgr2=0x7ff47006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.783+0000 7ff476641640 1 -- 172.21.15.133:0/1284995336 shutdown_connections 2024-09-11T23:46:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.783+0000 7ff476641640 1 -- 172.21.15.133:0/1284995336 wait complete. 2024-09-11T23:46:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.784+0000 7ff476641640 1 Processor -- start 2024-09-11T23:46:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.784+0000 7ff476641640 1 -- start start 2024-09-11T23:46:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.784+0000 7ff476641640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff470109450 0x7ff47011a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.784+0000 7ff476641640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff470109830 0x7ff4701168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.784+0000 7ff476641640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47011b140 0x7ff470116e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.784+0000 7ff476641640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff4701132d0 con 0x7ff470109830 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.784+0000 7ff476641640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff470113150 con 0x7ff47011b140 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.785+0000 7ff476641640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff470113450 con 0x7ff470109450 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.787+0000 7ff46ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff470109450 0x7ff47011a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.787+0000 7ff46ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff470109450 0x7ff47011a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42328/0 (socket says 172.21.15.133:42328) 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.787+0000 7ff46ffff640 1 -- 172.21.15.133:0/3680476469 learned_addr learned my addr 172.21.15.133:0/3680476469 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.787+0000 7ff474bde640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47011b140 0x7ff470116e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.789+0000 7ff46ffff640 1 -- 172.21.15.133:0/3680476469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47011b140 msgr2=0x7ff470116e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.789+0000 7ff46ffff640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47011b140 0x7ff470116e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.789+0000 7ff46ffff640 1 -- 172.21.15.133:0/3680476469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff470109830 msgr2=0x7ff4701168c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.789+0000 7ff46ffff640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff470109830 0x7ff4701168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.789+0000 7ff46ffff640 1 -- 172.21.15.133:0/3680476469 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff470117440 con 0x7ff470109450 2024-09-11T23:46:32.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.790+0000 7ff46ffff640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff470109450 0x7ff47011a7d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff46400ea20 tx=0x7ff46400eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.790+0000 7ff46d7fa640 1 -- 172.21.15.133:0/3680476469 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46400cd80 con 0x7ff470109450 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.790+0000 7ff46d7fa640 1 -- 172.21.15.133:0/3680476469 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff464010910 con 0x7ff470109450 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.790+0000 7ff46d7fa640 1 -- 172.21.15.133:0/3680476469 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464018920 con 0x7ff470109450 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.794+0000 7ff476641640 1 -- 172.21.15.133:0/3680476469 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4701ab170 con 0x7ff470109450 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.794+0000 7ff476641640 1 -- 172.21.15.133:0/3680476469 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4701ab510 con 0x7ff470109450 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.799+0000 7ff478a1a740 1 Processor -- start 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.804+0000 7ff478a1a740 1 -- start start 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff46d7fa640 1 -- 172.21.15.133:0/3680476469 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff464010ac0 con 0x7ff470109450 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb14ab40 0x55c5bb14af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb1493e0 0x55c5bb149840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb149e60 0x55c5bb156640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff478a1a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c5bb0af2f0 con 0x55c5bb149e60 2024-09-11T23:46:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff478a1a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c5bb056cf0 con 0x55c5bb14ab40 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff478a1a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c5bb0ab2b0 con 0x55c5bb1493e0 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff46ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb1493e0 0x55c5bb149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff46ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb1493e0 0x55c5bb149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42334/0 (socket says 172.21.15.133:42334) 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff46ffff640 1 -- 172.21.15.133:0/594250812 learned_addr learned my addr 172.21.15.133:0/594250812 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff474bde640 1 --2- 172.21.15.133:0/594250812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb14ab40 0x55c5bb14af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff46ffff640 1 -- 172.21.15.133:0/594250812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb14ab40 msgr2=0x55c5bb14af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff46ffff640 1 --2- 172.21.15.133:0/594250812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb14ab40 0x55c5bb14af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff46ffff640 1 -- 172.21.15.133:0/594250812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb149e60 msgr2=0x55c5bb156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff46ffff640 1 --2- 172.21.15.133:0/594250812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb149e60 0x55c5bb156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.805+0000 7ff46ffff640 1 -- 172.21.15.133:0/594250812 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5bb03b3a0 con 0x55c5bb1493e0 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.806+0000 7ff46ffff640 1 --2- 172.21.15.133:0/594250812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb1493e0 0x55c5bb149840 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff4640cd1b0 tx=0x7ff4640d3cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd2ef727b823362 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:32.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.807+0000 7ff476641640 1 -- 172.21.15.133:0/594250812 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4640d8ba0 con 0x55c5bb1493e0 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.807+0000 7ff476641640 1 -- 172.21.15.133:0/594250812 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4640d8d40 con 0x55c5bb1493e0 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.808+0000 7ff478a1a740 1 -- 172.21.15.133:0/594250812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb1493e0 msgr2=0x55c5bb149840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.808+0000 7ff478a1a740 1 --2- 172.21.15.133:0/594250812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb1493e0 0x55c5bb149840 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff4640cd1b0 tx=0x7ff4640d3cf0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.808+0000 7ff476641640 1 -- 172.21.15.133:0/594250812 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4640dc6d0 con 0x55c5bb1493e0 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.808+0000 7ff46d7fa640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4580a2e40 0x7ff4580a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.809+0000 7ff46d7fa640 1 -- 172.21.15.133:0/3680476469 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7ff4640cfb40 con 0x7ff470109450 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.811+0000 7ff474bde640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4580a2e40 0x7ff4580a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.812+0000 7ff478a1a740 1 -- 172.21.15.133:0/594250812 shutdown_connections 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.812+0000 7ff478a1a740 1 --2- 172.21.15.133:0/594250812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb149e60 0x55c5bb156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.812+0000 7ff478a1a740 1 --2- 172.21.15.133:0/594250812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb1493e0 0x55c5bb149840 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.812+0000 7ff478a1a740 1 --2- 172.21.15.133:0/594250812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb14ab40 0x55c5bb14af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.812+0000 7ff478a1a740 1 -- 172.21.15.133:0/594250812 >> 172.21.15.133:0/594250812 conn(0x55c5bb112310 msgr2=0x55c5bb145f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.813+0000 7ff474bde640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4580a2e40 0x7ff4580a5300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff460006010 tx=0x7ff460005b50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.815+0000 7ff478a1a740 1 -- 172.21.15.133:0/594250812 shutdown_connections 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.815+0000 7ff478a1a740 1 -- 172.21.15.133:0/594250812 wait complete. 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.816+0000 7ff478a1a740 1 -- start start 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.816+0000 7ff478a1a740 1 Processor -- start 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.817+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb1493e0 0x55c5bb1564e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.817+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb149e60 0x55c5bb153790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.817+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb14ab40 0x55c5bb153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.817+0000 7ff478a1a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c5bb1570c0 con 0x55c5bb1493e0 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.817+0000 7ff478a1a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c5bb06a060 con 0x55c5bb14ab40 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.817+0000 7ff478a1a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c5bb157240 con 0x55c5bb149e60 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.817+0000 7ff478a1a740 10 client.0 _ll_register_callbacks cb 0x55c5badbd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.818+0000 7ff46f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb149e60 0x55c5bb153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.818+0000 7ff46f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb149e60 0x55c5bb153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42344/0 (socket says 172.21.15.133:42344) 2024-09-11T23:46:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.818+0000 7ff46f7fe640 1 -- 172.21.15.133:0/3671784032 learned_addr learned my addr 172.21.15.133:0/3671784032 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.818+0000 7ff474bde640 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb14ab40 0x55c5bb153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.819+0000 7ff46f7fe640 1 -- 172.21.15.133:0/3671784032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb14ab40 msgr2=0x55c5bb153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.819+0000 7ff46f7fe640 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb14ab40 0x55c5bb153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.819+0000 7ff46f7fe640 1 -- 172.21.15.133:0/3671784032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb1493e0 msgr2=0x55c5bb1564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.819+0000 7ff46f7fe640 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb1493e0 0x55c5bb1564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.819+0000 7ff46f7fe640 1 -- 172.21.15.133:0/3671784032 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5bae203c0 con 0x55c5bb149e60 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.819+0000 7ff46f7fe640 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb149e60 0x55c5bb153790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff46800b410 tx=0x7ff46800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6450e465e443a4fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.822+0000 7ff44f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.822+0000 7ff44f7fe640 1 -- 172.21.15.133:0/3671784032 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4680115a0 con 0x55c5bb149e60 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.822+0000 7ff478a1a740 1 -- 172.21.15.133:0/3671784032 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4640d8d40 con 0x55c5bb149e60 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.822+0000 7ff44f7fe640 1 -- 172.21.15.133:0/3671784032 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7ff468011740 con 0x55c5bb149e60 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.822+0000 7ff44f7fe640 1 -- 172.21.15.133:0/3671784032 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff468020050 con 0x55c5bb149e60 2024-09-11T23:46:32.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.822+0000 7ff478a1a740 10 client.27193 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.823+0000 7ff478a1a740 1 -- 172.21.15.133:0/3671784032 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c5bb162c00 con 0x55c5bb149e60 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.823+0000 7ff44f7fe640 1 -- 172.21.15.133:0/3671784032 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4640d8d40 con 0x55c5bb149e60 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.825+0000 7ff44f7fe640 1 -- 172.21.15.133:0/3671784032 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff468011070 con 0x55c5bb149e60 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.825+0000 7ff478a1a740 10 client.27193 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.825+0000 7ff478a1a740 10 client.27193 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.825+0000 7ff478a1a740 20 client.27193 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.825+0000 7ff478a1a740 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c5bb164550 0x55c5bb1669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.825+0000 7ff478a1a740 4 client.27193 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.825+0000 7ff478a1a740 1 -- 172.21.15.133:0/3671784032 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5bb0bc860 con 0x55c5bb164550 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.826+0000 7ff46ffff640 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c5bb164550 0x55c5bb1669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.828+0000 7ff46ffff640 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c5bb164550 0x55c5bb1669d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62821a408d0c456f server_cookie=9f0781734f6bf48e in_seq=0 out_seq=0 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.828+0000 7ff44f7fe640 10 client.27193 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.835+0000 7ff44f7fe640 1 -- 172.21.15.133:0/3671784032 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5bb0bc860 con 0x55c5bb164550 2024-09-11T23:46:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.835+0000 7ff44f7fe640 10 client.27193 handle_command_reply: tid=0 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.835+0000 7ff478a1a740 1 client.27193 shutdown 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.839+0000 7ff478a1a740 1 -- 172.21.15.133:0/3671784032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb149e60 msgr2=0x55c5bb153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.839+0000 7ff478a1a740 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb149e60 0x55c5bb153790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff46800b410 tx=0x7ff46800b8d0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.840+0000 7ff478a1a740 1 -- 172.21.15.133:0/3671784032 shutdown_connections 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.840+0000 7ff478a1a740 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c5bb164550 0x55c5bb1669d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.840+0000 7ff478a1a740 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb14ab40 0x55c5bb153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.840+0000 7ff478a1a740 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb149e60 0x55c5bb153790 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.840+0000 7ff478a1a740 1 --2- 172.21.15.133:0/3671784032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb1493e0 0x55c5bb1564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.840+0000 7ff478a1a740 1 -- 172.21.15.133:0/3671784032 >> 172.21.15.133:0/3671784032 conn(0x55c5bb112310 msgr2=0x55c5bb1451a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.840+0000 7ff44f7fe640 0 client.27193 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.840+0000 7ff478a1a740 1 -- 172.21.15.133:0/3671784032 shutdown_connections 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.840+0000 7ff478a1a740 1 -- 172.21.15.133:0/3671784032 wait complete. 2024-09-11T23:46:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.841+0000 7ff478a1a740 20 client.27193 trim_cache size 0 max 16384 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.875+0000 7ff478a1a740 1 Processor -- start 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.875+0000 7ff478a1a740 1 -- start start 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.875+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb1493e0 0x55c5bb14bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.875+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb149e60 0x55c5bb1603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.875+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb14ab40 0x55c5bb155700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.876+0000 7ff478a1a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff4701b3dd0 con 0x55c5bb14ab40 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.876+0000 7ff478a1a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4701b5910 con 0x55c5bb149e60 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.876+0000 7ff478a1a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4701b44a0 con 0x55c5bb1493e0 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.876+0000 7ff474bde640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb14ab40 0x55c5bb155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.876+0000 7ff474bde640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb14ab40 0x55c5bb155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52048/0 (socket says 172.21.15.133:52048) 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.876+0000 7ff474bde640 1 -- 172.21.15.133:0/1041734916 learned_addr learned my addr 172.21.15.133:0/1041734916 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.877+0000 7ff474bde640 1 -- 172.21.15.133:0/1041734916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb1493e0 msgr2=0x55c5bb14bf30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:46:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.877+0000 7ff474bde640 1 --2- 172.21.15.133:0/1041734916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb1493e0 0x55c5bb14bf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.877+0000 7ff474bde640 1 -- 172.21.15.133:0/1041734916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb149e60 msgr2=0x55c5bb1603e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.877+0000 7ff474bde640 1 --2- 172.21.15.133:0/1041734916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb149e60 0x55c5bb1603e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.877+0000 7ff474bde640 1 -- 172.21.15.133:0/1041734916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5bb154060 con 0x55c5bb14ab40 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.877+0000 7ff474bde640 1 --2- 172.21.15.133:0/1041734916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb14ab40 0x55c5bb155700 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff46001a910 tx=0x7ff460051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc85df2f1feaf2dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.878+0000 7ff455ffb640 1 -- 172.21.15.133:0/1041734916 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460004c40 con 0x55c5bb14ab40 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.878+0000 7ff455ffb640 1 -- 172.21.15.133:0/1041734916 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff460061070 con 0x55c5bb14ab40 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.883+0000 7ff455ffb640 1 -- 172.21.15.133:0/1041734916 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460066470 con 0x55c5bb14ab40 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.884+0000 7ff478a1a740 1 -- 172.21.15.133:0/1041734916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb14ab40 msgr2=0x55c5bb155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.884+0000 7ff478a1a740 1 --2- 172.21.15.133:0/1041734916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb14ab40 0x55c5bb155700 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff46001a910 tx=0x7ff460051000 comp rx=0 tx=0).stop 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.885+0000 7ff478a1a740 1 -- 172.21.15.133:0/1041734916 shutdown_connections 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.885+0000 7ff478a1a740 1 --2- 172.21.15.133:0/1041734916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb14ab40 0x55c5bb155700 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.885+0000 7ff478a1a740 1 --2- 172.21.15.133:0/1041734916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb149e60 0x55c5bb1603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.885+0000 7ff478a1a740 1 --2- 172.21.15.133:0/1041734916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb1493e0 0x55c5bb14bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.885+0000 7ff478a1a740 1 -- 172.21.15.133:0/1041734916 >> 172.21.15.133:0/1041734916 conn(0x55c5bb112310 msgr2=0x55c5bb154ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.886+0000 7ff478a1a740 1 -- 172.21.15.133:0/1041734916 shutdown_connections 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.886+0000 7ff478a1a740 1 -- 172.21.15.133:0/1041734916 wait complete. 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.886+0000 7ff478a1a740 1 -- start start 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.887+0000 7ff478a1a740 1 Processor -- start 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.887+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb1493e0 0x55c5bb14d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.887+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb149e60 0x55c5bb14dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.887+0000 7ff478a1a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb14ab40 0x55c5bb15b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.887+0000 7ff478a1a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff4701b5240 con 0x55c5bb149e60 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.887+0000 7ff478a1a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4701b4b70 con 0x55c5bb1493e0 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.887+0000 7ff478a1a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4701b1e60 con 0x55c5bb14ab40 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.887+0000 7ff478a1a740 10 client.0 _ll_register_callbacks cb 0x7ff4680094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:32.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.888+0000 7ff474bde640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb14ab40 0x55c5bb15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.888+0000 7ff474bde640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb14ab40 0x55c5bb15b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:42356/0 (socket says 172.21.15.133:42356) 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.888+0000 7ff474bde640 1 -- 172.21.15.133:0/2721981408 learned_addr learned my addr 172.21.15.133:0/2721981408 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.888+0000 7ff46ffff640 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb1493e0 0x55c5bb14d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.888+0000 7ff474bde640 1 -- 172.21.15.133:0/2721981408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb1493e0 msgr2=0x55c5bb14d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.888+0000 7ff474bde640 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb1493e0 0x55c5bb14d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.888+0000 7ff474bde640 1 -- 172.21.15.133:0/2721981408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb149e60 msgr2=0x55c5bb14dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.888+0000 7ff474bde640 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb149e60 0x55c5bb14dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.888+0000 7ff474bde640 1 -- 172.21.15.133:0/2721981408 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5bb15ba90 con 0x55c5bb14ab40 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.889+0000 7ff474bde640 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb14ab40 0x55c5bb15b550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff46003ba60 tx=0x7ff460019110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a8e54cde291269 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.889+0000 7ff44f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.890+0000 7ff44f7fe640 1 -- 172.21.15.133:0/2721981408 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460040720 con 0x55c5bb14ab40 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.890+0000 7ff478a1a740 1 -- 172.21.15.133:0/2721981408 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff460061070 con 0x55c5bb14ab40 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.890+0000 7ff44f7fe640 1 -- 172.21.15.133:0/2721981408 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7ff4600408c0 con 0x55c5bb14ab40 2024-09-11T23:46:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.890+0000 7ff44f7fe640 1 -- 172.21.15.133:0/2721981408 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff460040ac0 con 0x55c5bb14ab40 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.892+0000 7ff478a1a740 10 client.27196 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.892+0000 7ff478a1a740 1 -- 172.21.15.133:0/2721981408 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c5bb15cd90 con 0x55c5bb14ab40 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.894+0000 7ff44f7fe640 1 -- 172.21.15.133:0/2721981408 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff460061070 con 0x55c5bb14ab40 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.894+0000 7ff44f7fe640 1 -- 172.21.15.133:0/2721981408 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff4600425e0 con 0x55c5bb14ab40 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.894+0000 7ff478a1a740 10 client.27196 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.894+0000 7ff478a1a740 10 client.27196 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.894+0000 7ff478a1a740 20 client.27196 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.894+0000 7ff478a1a740 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c5bb1cf2b0 0x55c5bb1d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.894+0000 7ff478a1a740 4 client.27196 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.894+0000 7ff478a1a740 1 -- 172.21.15.133:0/2721981408 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5bb0b6d70 con 0x55c5bb1cf2b0 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.895+0000 7ff46ffff640 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c5bb1cf2b0 0x55c5bb1d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.902+0000 7ff46ffff640 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c5bb1cf2b0 0x55c5bb1d16a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da41357e753ca26e server_cookie=fc93866c80723614 in_seq=0 out_seq=0 2024-09-11T23:46:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.902+0000 7ff44f7fe640 10 client.27196 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:46:32.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.904+0000 7ff44f7fe640 1 -- 172.21.15.133:0/2721981408 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5bb0b6d70 con 0x55c5bb1cf2b0 2024-09-11T23:46:32.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.904+0000 7ff44f7fe640 10 client.27196 handle_command_reply: tid=0 2024-09-11T23:46:32.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.904+0000 7ff478a1a740 1 client.27196 shutdown 2024-09-11T23:46:32.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.904+0000 7ff478a1a740 1 -- 172.21.15.133:0/2721981408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb14ab40 msgr2=0x55c5bb15b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:32.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.904+0000 7ff478a1a740 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb14ab40 0x55c5bb15b550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff46003ba60 tx=0x7ff460019110 comp rx=0 tx=0).stop 2024-09-11T23:46:32.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.904+0000 7ff478a1a740 1 -- 172.21.15.133:0/2721981408 shutdown_connections 2024-09-11T23:46:32.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.904+0000 7ff478a1a740 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55c5bb1cf2b0 0x55c5bb1d16a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.904+0000 7ff478a1a740 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5bb14ab40 0x55c5bb15b550 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.063 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:46:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.904+0000 7ff478a1a740 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5bb149e60 0x55c5bb14dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.904+0000 7ff478a1a740 1 --2- 172.21.15.133:0/2721981408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5bb1493e0 0x55c5bb14d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.904+0000 7ff478a1a740 1 -- 172.21.15.133:0/2721981408 >> 172.21.15.133:0/2721981408 conn(0x55c5bb112310 msgr2=0x55c5bb1451a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.905+0000 7ff44f7fe640 0 client.27196 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:46:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.905+0000 7ff478a1a740 1 -- 172.21.15.133:0/2721981408 shutdown_connections 2024-09-11T23:46:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.905+0000 7ff478a1a740 1 -- 172.21.15.133:0/2721981408 wait complete. 2024-09-11T23:46:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.910+0000 7ff478a1a740 20 client.27196 trim_cache size 0 max 16384 2024-09-11T23:46:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.911+0000 7ff456ffd640 1 -- 172.21.15.133:0/3680476469 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4580a2e40 msgr2=0x7ff4580a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.911+0000 7ff456ffd640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4580a2e40 0x7ff4580a5300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff460006010 tx=0x7ff460005b50 comp rx=0 tx=0).stop 2024-09-11T23:46:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.911+0000 7ff456ffd640 1 -- 172.21.15.133:0/3680476469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff470109450 msgr2=0x7ff47011a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:32.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.911+0000 7ff456ffd640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff470109450 0x7ff47011a7d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff46400ea20 tx=0x7ff46400eee0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.911+0000 7ff456ffd640 1 -- 172.21.15.133:0/3680476469 shutdown_connections 2024-09-11T23:46:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.912+0000 7ff456ffd640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff4580a2e40 0x7ff4580a5300 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.912+0000 7ff456ffd640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff47011b140 0x7ff470116e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.912+0000 7ff456ffd640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff470109830 0x7ff4701168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.912+0000 7ff456ffd640 1 --2- 172.21.15.133:0/3680476469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff470109450 0x7ff47011a7d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.912+0000 7ff456ffd640 1 -- 172.21.15.133:0/3680476469 >> 172.21.15.133:0/3680476469 conn(0x7ff47006e140 msgr2=0x7ff470072330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.913+0000 7ff456ffd640 1 -- 172.21.15.133:0/3680476469 shutdown_connections 2024-09-11T23:46:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:31.913+0000 7ff456ffd640 1 -- 172.21.15.133:0/3680476469 wait complete. 2024-09-11T23:46:32.974 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:32 smithi133 ceph-mon[31430]: pgmap v304: 97 pgs: 97 active+clean; 68 GiB data, 265 GiB used, 807 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 727 MiB/s wr, 1.57k op/s 2024-09-11T23:46:33.181 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:46:33.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:32 smithi138 ceph-mon[35142]: pgmap v304: 97 pgs: 97 active+clean; 68 GiB data, 265 GiB used, 807 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 727 MiB/s wr, 1.57k op/s 2024-09-11T23:46:33.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:32 smithi178 ceph-mon[35882]: pgmap v304: 97 pgs: 97 active+clean; 68 GiB data, 265 GiB used, 807 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 727 MiB/s wr, 1.57k op/s 2024-09-11T23:46:33.965 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:34.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:34 smithi178 ceph-mon[35882]: pgmap v305: 97 pgs: 97 active+clean; 69 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 729 MiB/s wr, 1.65k op/s 2024-09-11T23:46:34.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:34 smithi138 ceph-mon[35142]: pgmap v305: 97 pgs: 97 active+clean; 69 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 729 MiB/s wr, 1.65k op/s 2024-09-11T23:46:34.460 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:34 smithi133 ceph-mon[31430]: pgmap v305: 97 pgs: 97 active+clean; 69 GiB data, 269 GiB used, 804 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 729 MiB/s wr, 1.65k op/s 2024-09-11T23:46:34.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.939+0000 7f1f5ea48640 1 -- 172.21.15.133:0/643116687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f58074590 msgr2=0x7f1f58074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:34.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.939+0000 7f1f5ea48640 1 --2- 172.21.15.133:0/643116687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f58074590 0x7f1f58074970 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009900 tx=0x7f1f5002a4d0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.939+0000 7f1f5ea48640 1 -- 172.21.15.133:0/643116687 shutdown_connections 2024-09-11T23:46:34.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.939+0000 7f1f5ea48640 1 --2- 172.21.15.133:0/643116687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f581129d0 0x7f1f58114dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.939+0000 7f1f5ea48640 1 --2- 172.21.15.133:0/643116687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f58074eb0 0x7f1f58112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.939+0000 7f1f5ea48640 1 --2- 172.21.15.133:0/643116687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f58074590 0x7f1f58074970 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.939+0000 7f1f5ea48640 1 -- 172.21.15.133:0/643116687 >> 172.21.15.133:0/643116687 conn(0x7f1f580fd740 msgr2=0x7f1f580ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:34.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.943+0000 7f1f5ea48640 1 -- 172.21.15.133:0/643116687 shutdown_connections 2024-09-11T23:46:34.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.944+0000 7f1f5ea48640 1 -- 172.21.15.133:0/643116687 wait complete. 2024-09-11T23:46:34.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.945+0000 7f1f5ea48640 1 Processor -- start 2024-09-11T23:46:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.952+0000 7f1f5ea48640 1 -- start start 2024-09-11T23:46:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.952+0000 7f1f5ea48640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f58074590 0x7f1f58073840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.952+0000 7f1f5ea48640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f58074eb0 0x7f1f58073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.952+0000 7f1f5ea48640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f581129d0 0x7f1f5806c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.952+0000 7f1f5ea48640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58118450 con 0x7f1f58074eb0 2024-09-11T23:46:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.952+0000 7f1f5ea48640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581182d0 con 0x7f1f581129d0 2024-09-11T23:46:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.952+0000 7f1f5ea48640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581185d0 con 0x7f1f58074590 2024-09-11T23:46:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.956+0000 7f1f5da46640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f58074590 0x7f1f58073840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.956+0000 7f1f5e247640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f581129d0 0x7f1f5806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.956+0000 7f1f5e247640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f581129d0 0x7f1f5806c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52114/0 (socket says 172.21.15.133:52114) 2024-09-11T23:46:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.956+0000 7f1f5e247640 1 -- 172.21.15.133:0/1956707275 learned_addr learned my addr 172.21.15.133:0/1956707275 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.956+0000 7f1f5d245640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f58074eb0 0x7f1f58073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.957+0000 7f1f5e247640 1 -- 172.21.15.133:0/1956707275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f58074590 msgr2=0x7f1f58073840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.957+0000 7f1f5e247640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f58074590 0x7f1f58073840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.957+0000 7f1f5e247640 1 -- 172.21.15.133:0/1956707275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f58074eb0 msgr2=0x7f1f58073d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.957+0000 7f1f5e247640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f58074eb0 0x7f1f58073d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.957+0000 7f1f5e247640 1 -- 172.21.15.133:0/1956707275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5806d210 con 0x7f1f581129d0 2024-09-11T23:46:34.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.959+0000 7f1f5e247640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f581129d0 0x7f1f5806c950 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800d720 tx=0x7f1f4800dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:34.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.962+0000 7f1f4effd640 1 -- 172.21.15.133:0/1956707275 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48017070 con 0x7f1f581129d0 2024-09-11T23:46:34.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.962+0000 7f1f5ea48640 1 -- 172.21.15.133:0/1956707275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5806d500 con 0x7f1f581129d0 2024-09-11T23:46:34.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.963+0000 7f1f5ea48640 1 -- 172.21.15.133:0/1956707275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f58071540 con 0x7f1f581129d0 2024-09-11T23:46:34.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.963+0000 7f1f4effd640 1 -- 172.21.15.133:0/1956707275 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f4800de30 con 0x7f1f581129d0 2024-09-11T23:46:34.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.963+0000 7f1f4effd640 1 -- 172.21.15.133:0/1956707275 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48004d90 con 0x7f1f581129d0 2024-09-11T23:46:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.979+0000 7f1f4effd640 1 -- 172.21.15.133:0/1956707275 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1f48010460 con 0x7f1f581129d0 2024-09-11T23:46:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.984+0000 7f1f4effd640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1f400a2f00 0x7f1f400a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.985+0000 7f1f62083740 1 Processor -- start 2024-09-11T23:46:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.985+0000 7f1f4effd640 1 -- 172.21.15.133:0/1956707275 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f1f480cec60 con 0x7f1f581129d0 2024-09-11T23:46:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.986+0000 7f1f62083740 1 -- start start 2024-09-11T23:46:34.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.986+0000 7f1f62083740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691981cf0 0x558691980240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:34.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.986+0000 7f1f62083740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 0x558691980bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:34.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.986+0000 7f1f62083740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55869198b520 0x55869198d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:34.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.986+0000 7f1f62083740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5586918e62f0 con 0x558691981cf0 2024-09-11T23:46:34.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.986+0000 7f1f62083740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55869188dcf0 con 0x558691980780 2024-09-11T23:46:34.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.986+0000 7f1f62083740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5586918e22b0 con 0x55869198b520 2024-09-11T23:46:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.990+0000 7f1f5e247640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55869198b520 0x55869198d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.990+0000 7f1f5d245640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 0x558691980bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.990+0000 7f1f5da46640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1f400a2f00 0x7f1f400a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.990+0000 7f1f5da46640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691981cf0 0x558691980240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.990+0000 7f1f5da46640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691981cf0 0x558691980240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52084/0 (socket says 172.21.15.133:52084) 2024-09-11T23:46:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.990+0000 7f1f5da46640 1 -- 172.21.15.133:0/2421219053 learned_addr learned my addr 172.21.15.133:0/2421219053 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.992+0000 7f1f5d245640 1 -- 172.21.15.133:0/2421219053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55869198b520 msgr2=0x55869198d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.992+0000 7f1f5d245640 1 --2- 172.21.15.133:0/2421219053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55869198b520 0x55869198d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.992+0000 7f1f5d245640 1 -- 172.21.15.133:0/2421219053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691981cf0 msgr2=0x558691980240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:34.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.992+0000 7f1f5d245640 1 --2- 172.21.15.133:0/2421219053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691981cf0 0x558691980240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.992+0000 7f1f5d245640 1 -- 172.21.15.133:0/2421219053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586918723a0 con 0x558691980780 2024-09-11T23:46:34.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.992+0000 7f1f5d245640 1 --2- 172.21.15.133:0/2421219053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 0x558691980bc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1f5806e120 tx=0x7f1f5400c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f93afdfa3b086bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:34.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.993+0000 7f1f3ffff640 1 -- 172.21.15.133:0/2421219053 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5401a070 con 0x558691980780 2024-09-11T23:46:34.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.993+0000 7f1f5da46640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1f400a2f00 0x7f1f400a53c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b230 tx=0x7f1f50009580 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:34.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.993+0000 7f1f3ffff640 1 -- 172.21.15.133:0/2421219053 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f540158f0 con 0x558691980780 2024-09-11T23:46:34.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.993+0000 7f1f3ffff640 1 -- 172.21.15.133:0/2421219053 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400a9c0 con 0x558691980780 2024-09-11T23:46:34.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.995+0000 7f1f62083740 1 -- 172.21.15.133:0/2421219053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 msgr2=0x558691980bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:34.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.995+0000 7f1f62083740 1 --2- 172.21.15.133:0/2421219053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 0x558691980bc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1f5806e120 tx=0x7f1f5400c7f0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.996+0000 7f1f62083740 1 -- 172.21.15.133:0/2421219053 shutdown_connections 2024-09-11T23:46:34.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.996+0000 7f1f62083740 1 --2- 172.21.15.133:0/2421219053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55869198b520 0x55869198d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.996+0000 7f1f62083740 1 --2- 172.21.15.133:0/2421219053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 0x558691980bc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.996+0000 7f1f62083740 1 --2- 172.21.15.133:0/2421219053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691981cf0 0x558691980240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:34.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.996+0000 7f1f62083740 1 -- 172.21.15.133:0/2421219053 >> 172.21.15.133:0/2421219053 conn(0x558691949310 msgr2=0x55869197ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:35.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:34.999+0000 7f1f62083740 1 -- 172.21.15.133:0/2421219053 shutdown_connections 2024-09-11T23:46:35.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.004+0000 7f1f62083740 1 -- 172.21.15.133:0/2421219053 wait complete. 2024-09-11T23:46:35.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.005+0000 7f1f62083740 1 -- start start 2024-09-11T23:46:35.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.005+0000 7f1f62083740 1 Processor -- start 2024-09-11T23:46:35.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.008+0000 7f1f62083740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 0x55869198aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:35.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.008+0000 7f1f62083740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691981cf0 0x558691988190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:35.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.008+0000 7f1f62083740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55869198b520 0x558691988780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:35.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.008+0000 7f1f62083740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55869198e220 con 0x558691981cf0 2024-09-11T23:46:35.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.008+0000 7f1f62083740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5586918a1060 con 0x558691980780 2024-09-11T23:46:35.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.008+0000 7f1f62083740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55869198e3a0 con 0x55869198b520 2024-09-11T23:46:35.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.008+0000 7f1f62083740 10 client.0 _ll_register_callbacks cb 0x5586915f48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:35.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.011+0000 7f1f5da46640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 0x55869198aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:35.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.011+0000 7f1f5da46640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 0x55869198aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52144/0 (socket says 172.21.15.133:52144) 2024-09-11T23:46:35.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.011+0000 7f1f5da46640 1 -- 172.21.15.133:0/194916376 learned_addr learned my addr 172.21.15.133:0/194916376 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:35.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.012+0000 7f1f5da46640 1 -- 172.21.15.133:0/194916376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55869198b520 msgr2=0x558691988780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:46:35.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.012+0000 7f1f5da46640 1 --2- 172.21.15.133:0/194916376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55869198b520 0x558691988780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.012+0000 7f1f5da46640 1 -- 172.21.15.133:0/194916376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691981cf0 msgr2=0x558691988190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:46:35.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.012+0000 7f1f5da46640 1 --2- 172.21.15.133:0/194916376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691981cf0 0x558691988190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.012+0000 7f1f5da46640 1 -- 172.21.15.133:0/194916376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586916573c0 con 0x558691980780 2024-09-11T23:46:35.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.013+0000 7f1f5da46640 1 --2- 172.21.15.133:0/194916376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 0x55869198aee0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1f50005ab0 tx=0x7f1f5002a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408010a92d8c949c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:35.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.013+0000 7f1f3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:46:35.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.014+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/194916376 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50050030 con 0x558691980780 2024-09-11T23:46:35.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.014+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/194916376 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f1f50031500 con 0x558691980780 2024-09-11T23:46:35.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.014+0000 7f1f62083740 1 -- 172.21.15.133:0/194916376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f540158f0 con 0x558691980780 2024-09-11T23:46:35.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.014+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/194916376 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f50031700 con 0x558691980780 2024-09-11T23:46:35.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.015+0000 7f1f62083740 10 client.27410 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:35.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.015+0000 7f1f62083740 1 -- 172.21.15.133:0/194916376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558691999d50 con 0x558691980780 2024-09-11T23:46:35.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.015+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/194916376 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f540158f0 con 0x558691980780 2024-09-11T23:46:35.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.016+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/194916376 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1f5005e9b0 con 0x558691980780 2024-09-11T23:46:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.016+0000 7f1f62083740 10 client.27410 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.016+0000 7f1f62083740 10 client.27410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:46:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.016+0000 7f1f62083740 20 client.27410 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.016+0000 7f1f62083740 1 --2- 172.21.15.133:0/194916376 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55869199b2d0 0x55869199d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.016+0000 7f1f62083740 4 client.27410 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.016+0000 7f1f62083740 1 -- 172.21.15.133:0/194916376 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586918f3860 con 0x55869199b2d0 2024-09-11T23:46:35.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.018+0000 7f1f5d245640 1 --2- 172.21.15.133:0/194916376 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55869199b2d0 0x55869199d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:35.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.020+0000 7f1f5d245640 1 --2- 172.21.15.133:0/194916376 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55869199b2d0 0x55869199d790 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3fb2f5f0fdd1adc server_cookie=1754fb943b4e1003 in_seq=0 out_seq=0 2024-09-11T23:46:35.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.020+0000 7f1f3d7fa640 10 client.27410 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:46:35.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.023+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/194916376 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586918f3860 con 0x55869199b2d0 2024-09-11T23:46:35.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.023+0000 7f1f3d7fa640 10 client.27410 handle_command_reply: tid=0 2024-09-11T23:46:35.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.024+0000 7f1f62083740 1 client.27410 shutdown 2024-09-11T23:46:35.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.024+0000 7f1f62083740 1 -- 172.21.15.133:0/194916376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 msgr2=0x55869198aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:35.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.024+0000 7f1f62083740 1 --2- 172.21.15.133:0/194916376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 0x55869198aee0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1f50005ab0 tx=0x7f1f5002a4d0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.024+0000 7f1f62083740 1 -- 172.21.15.133:0/194916376 shutdown_connections 2024-09-11T23:46:35.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.024+0000 7f1f62083740 1 --2- 172.21.15.133:0/194916376 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55869199b2d0 0x55869199d790 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.024+0000 7f1f62083740 1 --2- 172.21.15.133:0/194916376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55869198b520 0x558691988780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.024+0000 7f1f62083740 1 --2- 172.21.15.133:0/194916376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691981cf0 0x558691988190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.024+0000 7f1f62083740 1 --2- 172.21.15.133:0/194916376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691980780 0x55869198aee0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.024+0000 7f1f62083740 1 -- 172.21.15.133:0/194916376 >> 172.21.15.133:0/194916376 conn(0x558691949310 msgr2=0x558691982ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:35.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.025+0000 7f1f3d7fa640 0 client.27410 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:46:35.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.027+0000 7f1f62083740 1 -- 172.21.15.133:0/194916376 shutdown_connections 2024-09-11T23:46:35.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.027+0000 7f1f62083740 1 -- 172.21.15.133:0/194916376 wait complete. 2024-09-11T23:46:35.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.032+0000 7f1f62083740 20 client.27410 trim_cache size 0 max 16384 2024-09-11T23:46:35.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.060+0000 7f1f62083740 1 Processor -- start 2024-09-11T23:46:35.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.061+0000 7f1f62083740 1 -- start start 2024-09-11T23:46:35.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.061+0000 7f1f62083740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691980780 0x558691997520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:35.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.061+0000 7f1f62083740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691981cf0 0x5586919814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:35.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.062+0000 7f1f5da46640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691980780 0x558691997520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:35.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.062+0000 7f1f5da46640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691980780 0x558691997520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:44770/0 (socket says 172.21.15.133:44770) 2024-09-11T23:46:35.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.062+0000 7f1f62083740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55869198b520 0x5586919872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:35.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.062+0000 7f1f5da46640 1 -- 172.21.15.133:0/3387349650 learned_addr learned my addr 172.21.15.133:0/3387349650 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:35.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.062+0000 7f1f62083740 1 -- 172.21.15.133:0/3387349650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4400e720 con 0x55869198b520 2024-09-11T23:46:35.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.062+0000 7f1f5e247640 1 --2- 172.21.15.133:0/3387349650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55869198b520 0x5586919872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:35.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.062+0000 7f1f62083740 1 -- 172.21.15.133:0/3387349650 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44010260 con 0x558691981cf0 2024-09-11T23:46:35.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.062+0000 7f1f62083740 1 -- 172.21.15.133:0/3387349650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4400edf0 con 0x558691980780 2024-09-11T23:46:35.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.063+0000 7f1f5e247640 1 -- 172.21.15.133:0/3387349650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691980780 msgr2=0x558691997520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:35.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.063+0000 7f1f5e247640 1 --2- 172.21.15.133:0/3387349650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691980780 0x558691997520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.063+0000 7f1f5e247640 1 -- 172.21.15.133:0/3387349650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691981cf0 msgr2=0x5586919814b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:35.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.063+0000 7f1f5e247640 1 --2- 172.21.15.133:0/3387349650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691981cf0 0x5586919814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.063+0000 7f1f5e247640 1 -- 172.21.15.133:0/3387349650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558691980480 con 0x55869198b520 2024-09-11T23:46:35.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.064+0000 7f1f5e247640 1 --2- 172.21.15.133:0/3387349650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55869198b520 0x5586919872c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1f480cc390 tx=0x7f1f4809b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc08b8acf1094ac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:35.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.067+0000 7f1f3f7fe640 1 -- 172.21.15.133:0/3387349650 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48020040 con 0x55869198b520 2024-09-11T23:46:35.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.067+0000 7f1f3f7fe640 1 -- 172.21.15.133:0/3387349650 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f480a3dd0 con 0x55869198b520 2024-09-11T23:46:35.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.067+0000 7f1f3f7fe640 1 -- 172.21.15.133:0/3387349650 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f480bb240 con 0x55869198b520 2024-09-11T23:46:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.070+0000 7f1f62083740 1 -- 172.21.15.133:0/3387349650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55869198b520 msgr2=0x5586919872c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.070+0000 7f1f62083740 1 --2- 172.21.15.133:0/3387349650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55869198b520 0x5586919872c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1f480cc390 tx=0x7f1f4809b060 comp rx=0 tx=0).stop 2024-09-11T23:46:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.071+0000 7f1f62083740 1 -- 172.21.15.133:0/3387349650 shutdown_connections 2024-09-11T23:46:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.071+0000 7f1f62083740 1 --2- 172.21.15.133:0/3387349650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55869198b520 0x5586919872c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.071+0000 7f1f62083740 1 --2- 172.21.15.133:0/3387349650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558691981cf0 0x5586919814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.071+0000 7f1f62083740 1 --2- 172.21.15.133:0/3387349650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691980780 0x558691997520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.071+0000 7f1f62083740 1 -- 172.21.15.133:0/3387349650 >> 172.21.15.133:0/3387349650 conn(0x558691949310 msgr2=0x558691977c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:35.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.077+0000 7f1f62083740 1 -- 172.21.15.133:0/3387349650 shutdown_connections 2024-09-11T23:46:35.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.077+0000 7f1f62083740 1 -- 172.21.15.133:0/3387349650 wait complete. 2024-09-11T23:46:35.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.077+0000 7f1f62083740 1 -- start start 2024-09-11T23:46:35.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.077+0000 7f1f62083740 1 Processor -- start 2024-09-11T23:46:35.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.078+0000 7f1f62083740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691980780 0x55869198ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:35.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.078+0000 7f1f62083740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691981cf0 0x55869198f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:35.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.078+0000 7f1f62083740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869198b520 0x558691a04060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:35.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.078+0000 7f1f62083740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4400fb90 con 0x558691980780 2024-09-11T23:46:35.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.078+0000 7f1f62083740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4400f4c0 con 0x55869198b520 2024-09-11T23:46:35.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.078+0000 7f1f62083740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4400c720 con 0x558691981cf0 2024-09-11T23:46:35.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.078+0000 7f1f62083740 10 client.0 _ll_register_callbacks cb 0x7f1f50004d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:35.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.078+0000 7f1f5d245640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691981cf0 0x55869198f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:35.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.078+0000 7f1f5d245640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691981cf0 0x55869198f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:44776/0 (socket says 172.21.15.133:44776) 2024-09-11T23:46:35.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.078+0000 7f1f5d245640 1 -- 172.21.15.133:0/104173082 learned_addr learned my addr 172.21.15.133:0/104173082 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:35.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.079+0000 7f1f5e247640 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869198b520 0x558691a04060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:35.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.080+0000 7f1f5da46640 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691980780 0x55869198ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:35.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.082+0000 7f1f5d245640 1 -- 172.21.15.133:0/104173082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869198b520 msgr2=0x558691a04060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:35.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.082+0000 7f1f5d245640 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869198b520 0x558691a04060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.082+0000 7f1f5d245640 1 -- 172.21.15.133:0/104173082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691980780 msgr2=0x55869198ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:35.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.082+0000 7f1f5d245640 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691980780 0x55869198ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.082+0000 7f1f5d245640 1 -- 172.21.15.133:0/104173082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558691a04620 con 0x558691981cf0 2024-09-11T23:46:35.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.086+0000 7f1f5d245640 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691981cf0 0x55869198f130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c3f0 tx=0x7f1f54004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=356b647ce577bd3a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:35.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.087+0000 7f1f5da46640 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691980780 0x55869198ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:46:35.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.087+0000 7f1f3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:35.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.089+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/104173082 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5401a040 con 0x558691981cf0 2024-09-11T23:46:35.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.089+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/104173082 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f1f54015650 con 0x558691981cf0 2024-09-11T23:46:35.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.092+0000 7f1f62083740 1 -- 172.21.15.133:0/104173082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f480a3dd0 con 0x558691981cf0 2024-09-11T23:46:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.099+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/104173082 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f540095d0 con 0x558691981cf0 2024-09-11T23:46:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.099+0000 7f1f62083740 10 client.27211 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.100+0000 7f1f62083740 1 -- 172.21.15.133:0/104173082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558691a05940 con 0x558691981cf0 2024-09-11T23:46:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.101+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/104173082 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f480a3dd0 con 0x558691981cf0 2024-09-11T23:46:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.101+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/104173082 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1f54029020 con 0x558691981cf0 2024-09-11T23:46:35.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.103+0000 7f1f62083740 10 client.27211 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:35.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.103+0000 7f1f62083740 10 client.27211 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:46:35.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.103+0000 7f1f62083740 20 client.27211 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:35.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.103+0000 7f1f62083740 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558691a05ff0 0x558691a08410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:35.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.103+0000 7f1f62083740 4 client.27211 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:46:35.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.103+0000 7f1f62083740 1 -- 172.21.15.133:0/104173082 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586918edd70 con 0x558691a05ff0 2024-09-11T23:46:35.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.104+0000 7f1f5da46640 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558691a05ff0 0x558691a08410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:35.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.106+0000 7f1f5da46640 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558691a05ff0 0x558691a08410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2ba9f9fa9a38720 server_cookie=a93594f954bb35bb in_seq=0 out_seq=0 2024-09-11T23:46:35.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.106+0000 7f1f3d7fa640 10 client.27211 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:46:35.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.106+0000 7f1f3d7fa640 1 -- 172.21.15.133:0/104173082 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586918edd70 con 0x558691a05ff0 2024-09-11T23:46:35.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.106+0000 7f1f3d7fa640 10 client.27211 handle_command_reply: tid=0 2024-09-11T23:46:35.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.106+0000 7f1f62083740 1 client.27211 shutdown 2024-09-11T23:46:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.108+0000 7f1f62083740 1 -- 172.21.15.133:0/104173082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691981cf0 msgr2=0x55869198f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.108+0000 7f1f62083740 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691981cf0 0x55869198f130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c3f0 tx=0x7f1f54004b50 comp rx=0 tx=0).stop 2024-09-11T23:46:35.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.108+0000 7f1f62083740 1 -- 172.21.15.133:0/104173082 shutdown_connections 2024-09-11T23:46:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.108+0000 7f1f62083740 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x558691a05ff0 0x558691a08410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.108+0000 7f1f62083740 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869198b520 0x558691a04060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.108+0000 7f1f62083740 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558691981cf0 0x55869198f130 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.108+0000 7f1f62083740 1 --2- 172.21.15.133:0/104173082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558691980780 0x55869198ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.108+0000 7f1f62083740 1 -- 172.21.15.133:0/104173082 >> 172.21.15.133:0/104173082 conn(0x558691949310 msgr2=0x5586919872c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:35.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.108+0000 7f1f3d7fa640 0 client.27211 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:46:35.114 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:46:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.109+0000 7f1f62083740 1 -- 172.21.15.133:0/104173082 shutdown_connections 2024-09-11T23:46:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.109+0000 7f1f62083740 1 -- 172.21.15.133:0/104173082 wait complete. 2024-09-11T23:46:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.110+0000 7f1f62083740 20 client.27211 trim_cache size 0 max 16384 2024-09-11T23:46:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.112+0000 7f1f5ea48640 1 -- 172.21.15.133:0/1956707275 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1f400a2f00 msgr2=0x7f1f400a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.112+0000 7f1f5ea48640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1f400a2f00 0x7f1f400a53c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b230 tx=0x7f1f50009580 comp rx=0 tx=0).stop 2024-09-11T23:46:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.112+0000 7f1f5ea48640 1 -- 172.21.15.133:0/1956707275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f581129d0 msgr2=0x7f1f5806c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.112+0000 7f1f5ea48640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f581129d0 0x7f1f5806c950 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800d720 tx=0x7f1f4800dbe0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.113+0000 7f1f5ea48640 1 -- 172.21.15.133:0/1956707275 shutdown_connections 2024-09-11T23:46:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.113+0000 7f1f5ea48640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1f400a2f00 0x7f1f400a53c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.113+0000 7f1f5ea48640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f581129d0 0x7f1f5806c950 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.113+0000 7f1f5ea48640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f58074eb0 0x7f1f58073d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.113+0000 7f1f5ea48640 1 --2- 172.21.15.133:0/1956707275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f58074590 0x7f1f58073840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.113+0000 7f1f5ea48640 1 -- 172.21.15.133:0/1956707275 >> 172.21.15.133:0/1956707275 conn(0x7f1f580fd740 msgr2=0x7f1f580fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.113+0000 7f1f5ea48640 1 -- 172.21.15.133:0/1956707275 shutdown_connections 2024-09-11T23:46:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:35.113+0000 7f1f5ea48640 1 -- 172.21.15.133:0/1956707275 wait complete. 2024-09-11T23:46:36.318 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:46:36.961 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:36 smithi133 ceph-mon[31430]: pgmap v306: 97 pgs: 97 active+clean; 69 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 733 MiB/s wr, 1.72k op/s 2024-09-11T23:46:37.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:36 smithi178 ceph-mon[35882]: pgmap v306: 97 pgs: 97 active+clean; 69 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 733 MiB/s wr, 1.72k op/s 2024-09-11T23:46:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:36 smithi138 ceph-mon[35142]: pgmap v306: 97 pgs: 97 active+clean; 69 GiB data, 272 GiB used, 801 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 733 MiB/s wr, 1.72k op/s 2024-09-11T23:46:37.283 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:38.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.256+0000 7f6503fff640 1 -- 172.21.15.133:0/3229356378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650411b140 msgr2=0x7f650411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:38.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.256+0000 7f6503fff640 1 --2- 172.21.15.133:0/3229356378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650411b140 0x7f650411d530 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009ca0 tx=0x7f64fc02a8f0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.257+0000 7f6503fff640 1 -- 172.21.15.133:0/3229356378 shutdown_connections 2024-09-11T23:46:38.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.257+0000 7f6503fff640 1 --2- 172.21.15.133:0/3229356378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650411b140 0x7f650411d530 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.257+0000 7f6503fff640 1 --2- 172.21.15.133:0/3229356378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f650406ea30 0x7f650406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.257+0000 7f6503fff640 1 --2- 172.21.15.133:0/3229356378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6504110190 0x7f650406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.257+0000 7f6503fff640 1 -- 172.21.15.133:0/3229356378 >> 172.21.15.133:0/3229356378 conn(0x7f6504073180 msgr2=0x7f6504073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:38.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.274+0000 7f6503fff640 1 -- 172.21.15.133:0/3229356378 shutdown_connections 2024-09-11T23:46:38.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.275+0000 7f6503fff640 1 -- 172.21.15.133:0/3229356378 wait complete. 2024-09-11T23:46:38.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.276+0000 7f6503fff640 1 Processor -- start 2024-09-11T23:46:38.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.276+0000 7f6503fff640 1 -- start start 2024-09-11T23:46:38.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.277+0000 7f6503fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650406ea30 0x7f650411ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.277+0000 7f6503fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6504110190 0x7f650411d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.277+0000 7f6503fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6504115d80 0x7f6504116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.277+0000 7f6503fff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6504120ce0 con 0x7f650406ea30 2024-09-11T23:46:38.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.278+0000 7f6503fff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6504120b60 con 0x7f6504110190 2024-09-11T23:46:38.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.278+0000 7f6503fff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6504120e60 con 0x7f6504115d80 2024-09-11T23:46:38.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.279+0000 7f65027fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6504110190 0x7f650411d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.279+0000 7f65027fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6504110190 0x7f650411d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52172/0 (socket says 172.21.15.133:52172) 2024-09-11T23:46:38.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.279+0000 7f65027fc640 1 -- 172.21.15.133:0/852676056 learned_addr learned my addr 172.21.15.133:0/852676056 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:38.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.280+0000 7f65037fe640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6504115d80 0x7f6504116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.280+0000 7f65027fc640 1 -- 172.21.15.133:0/852676056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6504115d80 msgr2=0x7f6504116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:38.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.280+0000 7f65027fc640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6504115d80 0x7f6504116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.280+0000 7f65027fc640 1 -- 172.21.15.133:0/852676056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650406ea30 msgr2=0x7f650411ccf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:46:38.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.280+0000 7f6502ffd640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650406ea30 0x7f650411ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.280+0000 7f65027fc640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650406ea30 0x7f650411ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.280+0000 7f65027fc640 1 -- 172.21.15.133:0/852676056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65041169b0 con 0x7f6504110190 2024-09-11T23:46:38.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.286+0000 7f65027fc640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6504110190 0x7f650411d230 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f64f800b450 tx=0x7f64f800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:38.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.289+0000 7f65088af640 1 -- 172.21.15.133:0/852676056 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8015020 con 0x7f6504110190 2024-09-11T23:46:38.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.289+0000 7f6503fff640 1 -- 172.21.15.133:0/852676056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650411a820 con 0x7f6504110190 2024-09-11T23:46:38.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.290+0000 7f6503fff640 1 -- 172.21.15.133:0/852676056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650411ad00 con 0x7f6504110190 2024-09-11T23:46:38.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.296+0000 7f65088af640 1 -- 172.21.15.133:0/852676056 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f8011d70 con 0x7f6504110190 2024-09-11T23:46:38.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.296+0000 7f65088af640 1 -- 172.21.15.133:0/852676056 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f800f340 con 0x7f6504110190 2024-09-11T23:46:38.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.302+0000 7f650bc8a740 1 Processor -- start 2024-09-11T23:46:38.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.303+0000 7f650bc8a740 1 -- start start 2024-09-11T23:46:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.303+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a79374290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.303+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 0x558a79374c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.303+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a79381960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.303+0000 7f650bc8a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a792da2f0 con 0x558a79375d40 2024-09-11T23:46:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.303+0000 7f650bc8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a79281cf0 con 0x558a7937f570 2024-09-11T23:46:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.303+0000 7f650bc8a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558a792d62b0 con 0x558a793747d0 2024-09-11T23:46:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.304+0000 7f65037fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a79374290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.304+0000 7f65037fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a79374290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52148/0 (socket says 172.21.15.133:52148) 2024-09-11T23:46:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.304+0000 7f65037fe640 1 -- 172.21.15.133:0/2824322610 learned_addr learned my addr 172.21.15.133:0/2824322610 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.305+0000 7f65027fc640 1 --2- 172.21.15.133:0/2824322610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a79381960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.305+0000 7f6502ffd640 1 --2- 172.21.15.133:0/2824322610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 0x558a79374c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.307+0000 7f65027fc640 1 -- 172.21.15.133:0/2824322610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 msgr2=0x558a79374c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:38.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.307+0000 7f65027fc640 1 --2- 172.21.15.133:0/2824322610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 0x558a79374c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.307+0000 7f65027fc640 1 -- 172.21.15.133:0/2824322610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 msgr2=0x558a79374290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:38.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.307+0000 7f65027fc640 1 --2- 172.21.15.133:0/2824322610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a79374290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.307+0000 7f65027fc640 1 -- 172.21.15.133:0/2824322610 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a792663a0 con 0x558a7937f570 2024-09-11T23:46:38.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.308+0000 7f65027fc640 1 --2- 172.21.15.133:0/2824322610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a79381960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f64f80cca40 tx=0x7f64f80d74b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32806f4b5214856d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.310+0000 7f65088af640 1 -- 172.21.15.133:0/852676056 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f64f8018020 con 0x7f6504110190 2024-09-11T23:46:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.311+0000 7f65088af640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f64ec0a2f00 0x7f64ec0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.312+0000 7f65088af640 1 -- 172.21.15.133:0/852676056 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f64f80cf200 con 0x7f6504110190 2024-09-11T23:46:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.314+0000 7f64e9ffb640 1 -- 172.21.15.133:0/2824322610 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8097af0 con 0x558a7937f570 2024-09-11T23:46:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.314+0000 7f64e9ffb640 1 -- 172.21.15.133:0/2824322610 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f809bf10 con 0x558a7937f570 2024-09-11T23:46:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.316+0000 7f650bc8a740 1 -- 172.21.15.133:0/2824322610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 msgr2=0x558a79381960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.316+0000 7f650bc8a740 1 --2- 172.21.15.133:0/2824322610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a79381960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f64f80cca40 tx=0x7f64f80d74b0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.318+0000 7f650bc8a740 1 -- 172.21.15.133:0/2824322610 shutdown_connections 2024-09-11T23:46:38.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.324+0000 7f650bc8a740 1 --2- 172.21.15.133:0/2824322610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a79381960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.325+0000 7f650bc8a740 1 --2- 172.21.15.133:0/2824322610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 0x558a79374c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.325+0000 7f650bc8a740 1 --2- 172.21.15.133:0/2824322610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a79374290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.325+0000 7f650bc8a740 1 -- 172.21.15.133:0/2824322610 >> 172.21.15.133:0/2824322610 conn(0x558a7933d310 msgr2=0x558a79370e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:38.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.326+0000 7f650bc8a740 1 -- 172.21.15.133:0/2824322610 shutdown_connections 2024-09-11T23:46:38.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.327+0000 7f650bc8a740 1 -- 172.21.15.133:0/2824322610 wait complete. 2024-09-11T23:46:38.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.329+0000 7f650bc8a740 1 -- start start 2024-09-11T23:46:38.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.330+0000 7f650bc8a740 1 Processor -- start 2024-09-11T23:46:38.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.331+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 0x558a7937ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.332+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a7937c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.332+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a7937c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.332+0000 7f6502ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 0x558a7937ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.332+0000 7f6502ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 0x558a7937ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:44804/0 (socket says 172.21.15.133:44804) 2024-09-11T23:46:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.333+0000 7f6502ffd640 1 -- 172.21.15.133:0/1480359028 learned_addr learned my addr 172.21.15.133:0/1480359028 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.333+0000 7f650bc8a740 1 -- 172.21.15.133:0/1480359028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a793822c0 con 0x558a79375d40 2024-09-11T23:46:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.333+0000 7f650bc8a740 1 -- 172.21.15.133:0/1480359028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a79295060 con 0x558a7937f570 2024-09-11T23:46:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.333+0000 7f650bc8a740 1 -- 172.21.15.133:0/1480359028 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558a79382440 con 0x558a793747d0 2024-09-11T23:46:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.333+0000 7f65037fe640 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a7937c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.334+0000 7f65027fc640 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a7937c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.338+0000 7f65037fe640 1 -- 172.21.15.133:0/1480359028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 msgr2=0x558a7937ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:38.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.338+0000 7f65037fe640 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 0x558a7937ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.338+0000 7f65037fe640 1 -- 172.21.15.133:0/1480359028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 msgr2=0x558a7937c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:38.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.338+0000 7f65037fe640 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a7937c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.338+0000 7f65037fe640 1 -- 172.21.15.133:0/1480359028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7904b3c0 con 0x558a7937f570 2024-09-11T23:46:38.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.339+0000 7f65027fc640 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a7937c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:46:38.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.339+0000 7f6502ffd640 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 0x558a7937ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:46:38.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.334+0000 7f650bc8a740 10 client.0 _ll_register_callbacks cb 0x558a78fe88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:38.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.339+0000 7f650bc8a740 1 -- 172.21.15.133:0/1480359028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f809bf10 con 0x558a7937f570 2024-09-11T23:46:38.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.343+0000 7f65037fe640 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a7937c670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f64fc006010 tx=0x7f64fc003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83e5876894a5f08b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:38.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.343+0000 7f65037fe640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f64ec0a2f00 0x7f64ec0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.344+0000 7f64e77f6640 10 client.27422 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:46:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.350+0000 7f64e77f6640 1 -- 172.21.15.133:0/1480359028 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc050060 con 0x558a7937f570 2024-09-11T23:46:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.351+0000 7f64e77f6640 1 -- 172.21.15.133:0/1480359028 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f64fc004350 con 0x558a7937f570 2024-09-11T23:46:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.351+0000 7f64e77f6640 1 -- 172.21.15.133:0/1480359028 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64fc004590 con 0x558a7937f570 2024-09-11T23:46:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.351+0000 7f650bc8a740 10 client.27422 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.352+0000 7f650bc8a740 1 -- 172.21.15.133:0/1480359028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=30}) -- 0x558a7938dc20 con 0x558a7937f570 2024-09-11T23:46:38.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.354+0000 7f64e77f6640 1 -- 172.21.15.133:0/1480359028 <== mon.2 v2:172.21.15.138:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f64fc047040 con 0x558a7937f570 2024-09-11T23:46:38.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.354+0000 7f650bc8a740 10 client.27422 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:38.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.354+0000 7f650bc8a740 10 client.27422 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:46:38.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.354+0000 7f650bc8a740 20 client.27422 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:38.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.354+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x558a7938f570 0x558a79391a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.354+0000 7f650bc8a740 4 client.27422 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:38.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.354+0000 7f650bc8a740 1 -- 172.21.15.133:0/1480359028 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a792e7860 con 0x558a7938f570 2024-09-11T23:46:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.358+0000 7f6502ffd640 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x558a7938f570 0x558a79391a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.360+0000 7f6502ffd640 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x558a7938f570 0x558a79391a30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a2f97e22c7b8df5 server_cookie=f431c9ce10ac8c2a in_seq=0 out_seq=0 2024-09-11T23:46:38.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.360+0000 7f64e77f6640 10 client.27422 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:46:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.362+0000 7f64e77f6640 1 -- 172.21.15.133:0/1480359028 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f64f400a820 con 0x558a7938f570 2024-09-11T23:46:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.362+0000 7f64e77f6640 10 client.27422 handle_command_reply: tid=0 2024-09-11T23:46:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.362+0000 7f650bc8a740 1 client.27422 shutdown 2024-09-11T23:46:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.363+0000 7f650bc8a740 1 -- 172.21.15.133:0/1480359028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 msgr2=0x558a7937c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.363+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a7937c670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f64fc006010 tx=0x7f64fc003e90 comp rx=0 tx=0).stop 2024-09-11T23:46:38.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.364+0000 7f65037fe640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f64ec0a2f00 0x7f64ec0a53c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009ca0 tx=0x7f64fc02f340 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:38.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.384+0000 7f650bc8a740 1 -- 172.21.15.133:0/1480359028 shutdown_connections 2024-09-11T23:46:38.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.384+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x558a7938f570 0x558a79391a30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.384+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a7937c670 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.384+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a7937c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.384+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1480359028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a793747d0 0x558a7937ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.384+0000 7f650bc8a740 1 -- 172.21.15.133:0/1480359028 >> 172.21.15.133:0/1480359028 conn(0x558a7933d310 msgr2=0x558a79376b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:38.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.385+0000 7f64e77f6640 0 client.27422 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:46:38.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.385+0000 7f650bc8a740 1 -- 172.21.15.133:0/1480359028 shutdown_connections 2024-09-11T23:46:38.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.385+0000 7f650bc8a740 1 -- 172.21.15.133:0/1480359028 wait complete. 2024-09-11T23:46:38.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.390+0000 7f650bc8a740 20 client.27422 trim_cache size 0 max 16384 2024-09-11T23:46:38.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.436+0000 7f650bc8a740 1 Processor -- start 2024-09-11T23:46:38.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.437+0000 7f650bc8a740 1 -- start start 2024-09-11T23:46:38.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.438+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a793747d0 0x558a7938b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.438+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a79375d40 0x558a79377320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.438+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a79377860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.438+0000 7f650bc8a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f64f000e770 con 0x558a793747d0 2024-09-11T23:46:38.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.438+0000 7f650bc8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64f00102b0 con 0x558a7937f570 2024-09-11T23:46:38.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.438+0000 7f650bc8a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64f000ee40 con 0x558a79375d40 2024-09-11T23:46:38.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.438+0000 7f6502ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a793747d0 0x558a7938b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.438+0000 7f6502ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a793747d0 0x558a7938b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52168/0 (socket says 172.21.15.133:52168) 2024-09-11T23:46:38.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.438+0000 7f6502ffd640 1 -- 172.21.15.133:0/573341067 learned_addr learned my addr 172.21.15.133:0/573341067 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:38.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.439+0000 7f65037fe640 1 --2- 172.21.15.133:0/573341067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a79377860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.439+0000 7f65027fc640 1 --2- 172.21.15.133:0/573341067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a79375d40 0x558a79377320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.448+0000 7f65027fc640 1 -- 172.21.15.133:0/573341067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 msgr2=0x558a79377860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:38.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.448+0000 7f65027fc640 1 --2- 172.21.15.133:0/573341067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a79377860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.448+0000 7f65027fc640 1 -- 172.21.15.133:0/573341067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a793747d0 msgr2=0x558a7938b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:38.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.448+0000 7f65027fc640 1 --2- 172.21.15.133:0/573341067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a793747d0 0x558a7938b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.448+0000 7f65027fc640 1 -- 172.21.15.133:0/573341067 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a79374f50 con 0x558a79375d40 2024-09-11T23:46:38.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.457+0000 7f65027fc640 1 --2- 172.21.15.133:0/573341067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a79375d40 0x558a79377320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f64f8011040 tx=0x7f64f80d26c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abd1d39e89f404c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:38.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.463+0000 7f64e97fa640 1 -- 172.21.15.133:0/573341067 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f809b350 con 0x558a79375d40 2024-09-11T23:46:38.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.463+0000 7f64e97fa640 1 -- 172.21.15.133:0/573341067 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f80d2a40 con 0x558a79375d40 2024-09-11T23:46:38.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.463+0000 7f64e97fa640 1 -- 172.21.15.133:0/573341067 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f809b5e0 con 0x558a79375d40 2024-09-11T23:46:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.465+0000 7f650bc8a740 1 -- 172.21.15.133:0/573341067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a79375d40 msgr2=0x558a79377320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.465+0000 7f650bc8a740 1 --2- 172.21.15.133:0/573341067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a79375d40 0x558a79377320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f64f8011040 tx=0x7f64f80d26c0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.465+0000 7f650bc8a740 1 -- 172.21.15.133:0/573341067 shutdown_connections 2024-09-11T23:46:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.465+0000 7f650bc8a740 1 --2- 172.21.15.133:0/573341067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a7937f570 0x558a79377860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.465+0000 7f650bc8a740 1 --2- 172.21.15.133:0/573341067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a79375d40 0x558a79377320 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.465+0000 7f650bc8a740 1 --2- 172.21.15.133:0/573341067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a793747d0 0x558a7938b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.465+0000 7f650bc8a740 1 -- 172.21.15.133:0/573341067 >> 172.21.15.133:0/573341067 conn(0x558a7933d310 msgr2=0x558a7937b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:38.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.469+0000 7f650bc8a740 1 -- 172.21.15.133:0/573341067 shutdown_connections 2024-09-11T23:46:38.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.469+0000 7f650bc8a740 1 -- 172.21.15.133:0/573341067 wait complete. 2024-09-11T23:46:38.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.470+0000 7f650bc8a740 1 -- start start 2024-09-11T23:46:38.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.471+0000 7f650bc8a740 1 Processor -- start 2024-09-11T23:46:38.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.476+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a793747d0 0x558a79382d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.476+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a79383250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.476+0000 7f650bc8a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7937f570 0x558a793f8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.476+0000 7f650bc8a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f64f000fbe0 con 0x558a79375d40 2024-09-11T23:46:38.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.476+0000 7f650bc8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64f000f510 con 0x558a793747d0 2024-09-11T23:46:38.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.476+0000 7f650bc8a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64f000c770 con 0x558a7937f570 2024-09-11T23:46:38.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.476+0000 7f650bc8a740 10 client.0 _ll_register_callbacks cb 0x7f64fc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:38.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.479+0000 7f6502ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a793747d0 0x558a79382d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.479+0000 7f6502ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a793747d0 0x558a79382d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52212/0 (socket says 172.21.15.133:52212) 2024-09-11T23:46:38.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.479+0000 7f6502ffd640 1 -- 172.21.15.133:0/1777618638 learned_addr learned my addr 172.21.15.133:0/1777618638 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:38.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.479+0000 7f65027fc640 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a79383250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.479+0000 7f65037fe640 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7937f570 0x558a793f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.480+0000 7f65027fc640 1 -- 172.21.15.133:0/1777618638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7937f570 msgr2=0x558a793f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:38.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.480+0000 7f65027fc640 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7937f570 0x558a793f8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.480+0000 7f65027fc640 1 -- 172.21.15.133:0/1777618638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a793747d0 msgr2=0x558a79382d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:38.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.480+0000 7f65027fc640 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a793747d0 0x558a79382d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.480+0000 7f65027fc640 1 -- 172.21.15.133:0/1777618638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a793f8840 con 0x558a79375d40 2024-09-11T23:46:38.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.481+0000 7f65027fc640 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a79383250 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x558a7938c190 tx=0x7f64f8099d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93d1c2fd11b8e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:38.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.481+0000 7f64e77f6640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:46:38.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.481+0000 7f64e77f6640 1 -- 172.21.15.133:0/1777618638 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f809b5e0 con 0x558a79375d40 2024-09-11T23:46:38.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.482+0000 7f64e77f6640 1 -- 172.21.15.133:0/1777618638 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f64f809bc70 con 0x558a79375d40 2024-09-11T23:46:38.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.484+0000 7f650bc8a740 1 -- 172.21.15.133:0/1777618638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f80d2a40 con 0x558a79375d40 2024-09-11T23:46:38.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.488+0000 7f64e77f6640 1 -- 172.21.15.133:0/1777618638 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64f809b350 con 0x558a79375d40 2024-09-11T23:46:38.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.488+0000 7f650bc8a740 10 client.17886 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:38.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.488+0000 7f650bc8a740 1 -- 172.21.15.133:0/1777618638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a793f9ac0 con 0x558a79375d40 2024-09-11T23:46:38.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.490+0000 7f64e77f6640 1 -- 172.21.15.133:0/1777618638 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f80d2a40 con 0x558a79375d40 2024-09-11T23:46:38.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.490+0000 7f64e77f6640 1 -- 172.21.15.133:0/1777618638 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f64f809cb40 con 0x558a79375d40 2024-09-11T23:46:38.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.491+0000 7f650bc8a740 10 client.17886 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:38.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.491+0000 7f650bc8a740 10 client.17886 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:46:38.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.491+0000 7f650bc8a740 20 client.17886 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:38.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.491+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x558a793f9c90 0x558a793fc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:38.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.491+0000 7f650bc8a740 4 client.17886 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:46:38.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.491+0000 7f650bc8a740 1 -- 172.21.15.133:0/1777618638 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a792e1d70 con 0x558a793f9c90 2024-09-11T23:46:38.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.500+0000 7f6502ffd640 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x558a793f9c90 0x558a793fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:38.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.505+0000 7f6502ffd640 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x558a793f9c90 0x558a793fc080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8d08ae8235feb79 server_cookie=ea237a3fd5d39728 in_seq=0 out_seq=0 2024-09-11T23:46:38.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.505+0000 7f64e77f6640 10 client.17886 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:46:38.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.505+0000 7f64e77f6640 1 -- 172.21.15.133:0/1777618638 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a792e1d70 con 0x558a793f9c90 2024-09-11T23:46:38.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.505+0000 7f64e77f6640 10 client.17886 handle_command_reply: tid=0 2024-09-11T23:46:38.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.505+0000 7f650bc8a740 1 client.17886 shutdown 2024-09-11T23:46:38.510 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:46:38.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.506+0000 7f650bc8a740 1 -- 172.21.15.133:0/1777618638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 msgr2=0x558a79383250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:38.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.506+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a79383250 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x558a7938c190 tx=0x7f64f8099d20 comp rx=0 tx=0).stop 2024-09-11T23:46:38.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.506+0000 7f650bc8a740 1 -- 172.21.15.133:0/1777618638 shutdown_connections 2024-09-11T23:46:38.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.506+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x558a793f9c90 0x558a793fc080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.506+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7937f570 0x558a793f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.506+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a79375d40 0x558a79383250 secure :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0x558a7938c190 tx=0x7f64f8099d20 comp rx=0 tx=0).stop 2024-09-11T23:46:38.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.506+0000 7f650bc8a740 1 --2- 172.21.15.133:0/1777618638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a793747d0 0x558a79382d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.506+0000 7f650bc8a740 1 -- 172.21.15.133:0/1777618638 >> 172.21.15.133:0/1777618638 conn(0x558a7933d310 msgr2=0x558a7937d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:38.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.507+0000 7f64e77f6640 0 client.17886 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:46:38.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.507+0000 7f650bc8a740 1 -- 172.21.15.133:0/1777618638 shutdown_connections 2024-09-11T23:46:38.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.507+0000 7f650bc8a740 1 -- 172.21.15.133:0/1777618638 wait complete. 2024-09-11T23:46:38.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.507+0000 7f650bc8a740 20 client.17886 trim_cache size 0 max 16384 2024-09-11T23:46:38.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.509+0000 7f6503fff640 1 -- 172.21.15.133:0/852676056 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f64ec0a2f00 msgr2=0x7f64ec0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:38.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.509+0000 7f6503fff640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f64ec0a2f00 0x7f64ec0a53c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009ca0 tx=0x7f64fc02f340 comp rx=0 tx=0).stop 2024-09-11T23:46:38.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.509+0000 7f6503fff640 1 -- 172.21.15.133:0/852676056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6504110190 msgr2=0x7f650411d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:38.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.509+0000 7f6503fff640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6504110190 0x7f650411d230 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f64f800b450 tx=0x7f64f800b910 comp rx=0 tx=0).stop 2024-09-11T23:46:38.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.509+0000 7f6503fff640 1 -- 172.21.15.133:0/852676056 shutdown_connections 2024-09-11T23:46:38.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.509+0000 7f6503fff640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f64ec0a2f00 0x7f64ec0a53c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.509+0000 7f6503fff640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6504115d80 0x7f6504116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.509+0000 7f6503fff640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6504110190 0x7f650411d230 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.509+0000 7f6503fff640 1 --2- 172.21.15.133:0/852676056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650406ea30 0x7f650411ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:38.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.509+0000 7f6503fff640 1 -- 172.21.15.133:0/852676056 >> 172.21.15.133:0/852676056 conn(0x7f6504073180 msgr2=0x7f650406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:38.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.510+0000 7f6503fff640 1 -- 172.21.15.133:0/852676056 shutdown_connections 2024-09-11T23:46:38.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:38.510+0000 7f6503fff640 1 -- 172.21.15.133:0/852676056 wait complete. 2024-09-11T23:46:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:38 smithi138 ceph-mon[35142]: pgmap v307: 97 pgs: 97 active+clean; 69 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 688 MiB/s wr, 1.66k op/s 2024-09-11T23:46:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:38 smithi133 ceph-mon[31430]: pgmap v307: 97 pgs: 97 active+clean; 69 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 688 MiB/s wr, 1.66k op/s 2024-09-11T23:46:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:38 smithi178 ceph-mon[35882]: pgmap v307: 97 pgs: 97 active+clean; 69 GiB data, 273 GiB used, 800 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 688 MiB/s wr, 1.66k op/s 2024-09-11T23:46:39.561 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:46:40.327 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:41.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:40 smithi133 ceph-mon[31430]: pgmap v308: 97 pgs: 97 active+clean; 70 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 740 MiB/s wr, 2.06k op/s 2024-09-11T23:46:41.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:40 smithi138 ceph-mon[35142]: pgmap v308: 97 pgs: 97 active+clean; 70 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 740 MiB/s wr, 2.06k op/s 2024-09-11T23:46:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.216+0000 7f42a2ccc640 1 -- 172.21.15.133:0/2725868998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f429c101020 msgr2=0x7f429c10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.216+0000 7f42a2ccc640 1 --2- 172.21.15.133:0/2725868998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f429c101020 0x7f429c10d0e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f429400ade0 tx=0x7f429402b810 comp rx=0 tx=0).stop 2024-09-11T23:46:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.216+0000 7f42a2ccc640 1 -- 172.21.15.133:0/2725868998 shutdown_connections 2024-09-11T23:46:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.216+0000 7f42a2ccc640 1 --2- 172.21.15.133:0/2725868998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f429c101020 0x7f429c10d0e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.216+0000 7f42a2ccc640 1 --2- 172.21.15.133:0/2725868998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429c100680 0x7f429c100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.217+0000 7f42a2ccc640 1 --2- 172.21.15.133:0/2725868998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f429c105220 0x7f429c105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.217+0000 7f42a2ccc640 1 -- 172.21.15.133:0/2725868998 >> 172.21.15.133:0/2725868998 conn(0x7f429c077210 msgr2=0x7f429c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.217+0000 7f42a2ccc640 1 -- 172.21.15.133:0/2725868998 shutdown_connections 2024-09-11T23:46:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.217+0000 7f42a2ccc640 1 -- 172.21.15.133:0/2725868998 wait complete. 2024-09-11T23:46:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.219+0000 7f42a2ccc640 1 Processor -- start 2024-09-11T23:46:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.219+0000 7f42a2ccc640 1 -- start start 2024-09-11T23:46:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.219+0000 7f42a2ccc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429c105220 0x7f429c10c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.219+0000 7f42a2ccc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f429c10c6a0 0x7f429c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.219+0000 7f42a2ccc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f429c19c130 0x7f429c19c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.219+0000 7f42a2ccc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f429c10ff20 con 0x7f429c19c130 2024-09-11T23:46:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.219+0000 7f42a2ccc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f429c10fda0 con 0x7f429c105220 2024-09-11T23:46:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.219+0000 7f42a2ccc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f429c1100a0 con 0x7f429c10c6a0 2024-09-11T23:46:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.220+0000 7f429bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f429c10c6a0 0x7f429c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.220+0000 7f429bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f429c10c6a0 0x7f429c19bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:44844/0 (socket says 172.21.15.133:44844) 2024-09-11T23:46:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.220+0000 7f429bfff640 1 -- 172.21.15.133:0/29283341 learned_addr learned my addr 172.21.15.133:0/29283341 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.220+0000 7f42a0a68640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429c105220 0x7f429c10c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.220+0000 7f42a0a68640 1 -- 172.21.15.133:0/29283341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f429c10c6a0 msgr2=0x7f429c19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.220+0000 7f42a0a68640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f429c10c6a0 0x7f429c19bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.220+0000 7f42a0a68640 1 -- 172.21.15.133:0/29283341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f429c19c130 msgr2=0x7f429c19c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.220+0000 7f42a0a68640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f429c19c130 0x7f429c19c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.220+0000 7f42a0a68640 1 -- 172.21.15.133:0/29283341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c19eca0 con 0x7f429c105220 2024-09-11T23:46:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.225+0000 7f42a0a68640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429c105220 0x7f429c10c160 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f429000d700 tx=0x7f429000dbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.225+0000 7f4299ffb640 1 -- 172.21.15.133:0/29283341 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290017070 con 0x7f429c105220 2024-09-11T23:46:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.225+0000 7f4299ffb640 1 -- 172.21.15.133:0/29283341 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4290009820 con 0x7f429c105220 2024-09-11T23:46:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.225+0000 7f4299ffb640 1 -- 172.21.15.133:0/29283341 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290004ce0 con 0x7f429c105220 2024-09-11T23:46:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.225+0000 7f42a2ccc640 1 -- 172.21.15.133:0/29283341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429c19ef90 con 0x7f429c105220 2024-09-11T23:46:41.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.226+0000 7f42a2ccc640 1 -- 172.21.15.133:0/29283341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f429c19f480 con 0x7f429c105220 2024-09-11T23:46:41.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.234+0000 7f42a50a5740 1 Processor -- start 2024-09-11T23:46:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.234+0000 7f42a50a5740 1 -- start start 2024-09-11T23:46:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.235+0000 7f42a50a5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa528b40 0x55a9fa527090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.235+0000 7f42a50a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 0x55a9fa527a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.235+0000 7f42a50a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa532370 0x55a9fa534760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.235+0000 7f42a50a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a9fa48d2f0 con 0x55a9fa532370 2024-09-11T23:46:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.235+0000 7f42a50a5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a9fa434cf0 con 0x55a9fa528b40 2024-09-11T23:46:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.235+0000 7f42a50a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a9fa4892b0 con 0x55a9fa5275d0 2024-09-11T23:46:41.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.236+0000 7f4299ffb640 1 -- 172.21.15.133:0/29283341 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4290005220 con 0x7f429c105220 2024-09-11T23:46:41.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.238+0000 7f4299ffb640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f428c0a2db0 0x7f428c0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.238+0000 7f4299ffb640 1 -- 172.21.15.133:0/29283341 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f42900ce680 con 0x7f429c105220 2024-09-11T23:46:41.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.239+0000 7f429bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa532370 0x55a9fa534760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.239+0000 7f429bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa532370 0x55a9fa534760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52200/0 (socket says 172.21.15.133:52200) 2024-09-11T23:46:41.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.239+0000 7f429bfff640 1 -- 172.21.15.133:0/3745285194 learned_addr learned my addr 172.21.15.133:0/3745285194 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:41.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.240+0000 7f42a1269640 1 --2- 172.21.15.133:0/3745285194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa528b40 0x55a9fa527090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.241+0000 7f42a0a68640 1 --2- 172.21.15.133:0/3745285194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 0x55a9fa527a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.244+0000 7f42a0a68640 1 -- 172.21.15.133:0/3745285194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa528b40 msgr2=0x55a9fa527090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:41.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.244+0000 7f42a0a68640 1 --2- 172.21.15.133:0/3745285194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa528b40 0x55a9fa527090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.244+0000 7f42a0a68640 1 -- 172.21.15.133:0/3745285194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa532370 msgr2=0x55a9fa534760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:41.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.244+0000 7f42a0a68640 1 --2- 172.21.15.133:0/3745285194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa532370 0x55a9fa534760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.244+0000 7f42a0a68640 1 -- 172.21.15.133:0/3745285194 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9fa4193a0 con 0x55a9fa5275d0 2024-09-11T23:46:41.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.245+0000 7f42a1269640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f428c0a2db0 0x7f428c0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.246+0000 7f42a0a68640 1 --2- 172.21.15.133:0/3745285194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 0x55a9fa527a10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f42900cbe10 tx=0x7f429009b2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c165d62b041167e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:41.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.248+0000 7f4286ffd640 1 -- 172.21.15.133:0/3745285194 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42900a3850 con 0x55a9fa5275d0 2024-09-11T23:46:41.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.248+0000 7f4286ffd640 1 -- 172.21.15.133:0/3745285194 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4290009d60 con 0x55a9fa5275d0 2024-09-11T23:46:41.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.249+0000 7f42a50a5740 1 -- 172.21.15.133:0/3745285194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 msgr2=0x55a9fa527a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:41.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.249+0000 7f42a50a5740 1 --2- 172.21.15.133:0/3745285194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 0x55a9fa527a10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f42900cbe10 tx=0x7f429009b2e0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.252+0000 7f42a50a5740 1 -- 172.21.15.133:0/3745285194 shutdown_connections 2024-09-11T23:46:41.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.252+0000 7f42a50a5740 1 --2- 172.21.15.133:0/3745285194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa532370 0x55a9fa534760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.252+0000 7f42a50a5740 1 --2- 172.21.15.133:0/3745285194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 0x55a9fa527a10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.252+0000 7f42a50a5740 1 --2- 172.21.15.133:0/3745285194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa528b40 0x55a9fa527090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.252+0000 7f42a50a5740 1 -- 172.21.15.133:0/3745285194 >> 172.21.15.133:0/3745285194 conn(0x55a9fa4f0310 msgr2=0x55a9fa523c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:41.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.259+0000 7f42a50a5740 1 -- 172.21.15.133:0/3745285194 shutdown_connections 2024-09-11T23:46:41.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.263+0000 7f42a50a5740 1 -- 172.21.15.133:0/3745285194 wait complete. 2024-09-11T23:46:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.264+0000 7f42a50a5740 1 -- start start 2024-09-11T23:46:41.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.264+0000 7f42a50a5740 1 Processor -- start 2024-09-11T23:46:41.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.268+0000 7f42a50a5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa5275d0 0x55a9fa531c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.268+0000 7f42a50a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa52ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.268+0000 7f42a50a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa532370 0x55a9fa52f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.268+0000 7f42a50a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a9fa5350b0 con 0x55a9fa528b40 2024-09-11T23:46:41.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.268+0000 7f42a50a5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a9fa448060 con 0x55a9fa5275d0 2024-09-11T23:46:41.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.268+0000 7f42a50a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a9fa535230 con 0x55a9fa532370 2024-09-11T23:46:41.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.268+0000 7f42a50a5740 10 client.0 _ll_register_callbacks cb 0x55a9fa19b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:41.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.271+0000 7f429bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa52ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.271+0000 7f429bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa52ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52202/0 (socket says 172.21.15.133:52202) 2024-09-11T23:46:41.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.271+0000 7f429bfff640 1 -- 172.21.15.133:0/3082541566 learned_addr learned my addr 172.21.15.133:0/3082541566 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:41.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.271+0000 7f42a0a68640 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa5275d0 0x55a9fa531c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.272+0000 7f429bfff640 1 -- 172.21.15.133:0/3082541566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa532370 msgr2=0x55a9fa52f460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:46:41.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.272+0000 7f429bfff640 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa532370 0x55a9fa52f460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.272+0000 7f429bfff640 1 -- 172.21.15.133:0/3082541566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa5275d0 msgr2=0x55a9fa531c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:41.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.272+0000 7f429bfff640 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa5275d0 0x55a9fa531c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.272+0000 7f429bfff640 1 -- 172.21.15.133:0/3082541566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9fa1fe3c0 con 0x55a9fa528b40 2024-09-11T23:46:41.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.273+0000 7f42a1269640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f428c0a2db0 0x7f428c0a5270 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f429402be00 tx=0x7f429400a6d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:41.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.274+0000 7f429bfff640 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa52ef20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4288009f10 tx=0x7f4288011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bad1905f3d9a88d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:41.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.274+0000 7f427bfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:46:41.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.276+0000 7f427bfff640 1 -- 172.21.15.133:0/3082541566 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288019070 con 0x55a9fa528b40 2024-09-11T23:46:41.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.276+0000 7f42a50a5740 1 -- 172.21.15.133:0/3082541566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4290009d60 con 0x55a9fa528b40 2024-09-11T23:46:41.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.277+0000 7f427bfff640 1 -- 172.21.15.133:0/3082541566 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f4288015870 con 0x55a9fa528b40 2024-09-11T23:46:41.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.277+0000 7f427bfff640 1 -- 172.21.15.133:0/3082541566 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4288025050 con 0x55a9fa528b40 2024-09-11T23:46:41.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.279+0000 7f42a50a5740 10 client.17895 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:41.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.279+0000 7f42a50a5740 1 -- 172.21.15.133:0/3082541566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a9fa540b60 con 0x55a9fa528b40 2024-09-11T23:46:41.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.283+0000 7f427bfff640 1 -- 172.21.15.133:0/3082541566 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4290009d60 con 0x55a9fa528b40 2024-09-11T23:46:41.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.283+0000 7f427bfff640 1 -- 172.21.15.133:0/3082541566 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4288014c00 con 0x55a9fa528b40 2024-09-11T23:46:41.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.283+0000 7f42a50a5740 10 client.17895 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:41.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.283+0000 7f42a50a5740 10 client.17895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:41.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.283+0000 7f42a50a5740 20 client.17895 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:41.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.283+0000 7f42a50a5740 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a9fa542430 0x55a9fa544890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.283+0000 7f42a50a5740 4 client.17895 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:41.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.283+0000 7f42a50a5740 1 -- 172.21.15.133:0/3082541566 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9fa49a860 con 0x55a9fa542430 2024-09-11T23:46:41.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.286+0000 7f42a0a68640 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a9fa542430 0x55a9fa544890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.289+0000 7f42a0a68640 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a9fa542430 0x55a9fa544890 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99248cdff8a9c161 server_cookie=b246bbf0baa16ab9 in_seq=0 out_seq=0 2024-09-11T23:46:41.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.289+0000 7f427bfff640 10 client.17895 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:40 smithi178 ceph-mon[35882]: pgmap v308: 97 pgs: 97 active+clean; 70 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 740 MiB/s wr, 2.06k op/s 2024-09-11T23:46:41.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.290+0000 7f427bfff640 1 -- 172.21.15.133:0/3082541566 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9fa49a860 con 0x55a9fa542430 2024-09-11T23:46:41.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.290+0000 7f427bfff640 10 client.17895 handle_command_reply: tid=0 2024-09-11T23:46:41.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.290+0000 7f42a50a5740 1 client.17895 shutdown 2024-09-11T23:46:41.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.291+0000 7f42a50a5740 1 -- 172.21.15.133:0/3082541566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 msgr2=0x55a9fa52ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:41.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.291+0000 7f42a50a5740 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa52ef20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4288009f10 tx=0x7f4288011630 comp rx=0 tx=0).stop 2024-09-11T23:46:41.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.295+0000 7f42a50a5740 1 -- 172.21.15.133:0/3082541566 shutdown_connections 2024-09-11T23:46:41.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.295+0000 7f42a50a5740 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a9fa542430 0x55a9fa544890 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.295+0000 7f42a50a5740 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa532370 0x55a9fa52f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.295+0000 7f42a50a5740 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa52ef20 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.295+0000 7f42a50a5740 1 --2- 172.21.15.133:0/3082541566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa5275d0 0x55a9fa531c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.295+0000 7f42a50a5740 1 -- 172.21.15.133:0/3082541566 >> 172.21.15.133:0/3082541566 conn(0x55a9fa4f0310 msgr2=0x55a9fa529960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:41.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.295+0000 7f427bfff640 0 client.17895 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:41.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.295+0000 7f42a50a5740 1 -- 172.21.15.133:0/3082541566 shutdown_connections 2024-09-11T23:46:41.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.295+0000 7f42a50a5740 1 -- 172.21.15.133:0/3082541566 wait complete. 2024-09-11T23:46:41.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.295+0000 7f42a50a5740 20 client.17895 trim_cache size 0 max 16384 2024-09-11T23:46:41.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.326+0000 7f42a50a5740 1 Processor -- start 2024-09-11T23:46:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.327+0000 7f42a50a5740 1 -- start start 2024-09-11T23:46:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.328+0000 7f42a50a5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa5275d0 0x55a9fa53e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.328+0000 7f42a50a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa52a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.328+0000 7f42a50a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa532370 0x55a9fa52a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.328+0000 7f42a50a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f429c1c77c0 con 0x55a9fa528b40 2024-09-11T23:46:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.328+0000 7f42a50a5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f429c073b70 con 0x55a9fa5275d0 2024-09-11T23:46:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.329+0000 7f42a50a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f429c1c70f0 con 0x55a9fa532370 2024-09-11T23:46:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.329+0000 7f42a1269640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa532370 0x55a9fa52a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.329+0000 7f42a1269640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa532370 0x55a9fa52a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:44872/0 (socket says 172.21.15.133:44872) 2024-09-11T23:46:41.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.329+0000 7f42a1269640 1 -- 172.21.15.133:0/240882614 learned_addr learned my addr 172.21.15.133:0/240882614 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:41.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.329+0000 7f42a0a68640 1 --2- 172.21.15.133:0/240882614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa5275d0 0x55a9fa53e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.329+0000 7f42a1269640 1 -- 172.21.15.133:0/240882614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa5275d0 msgr2=0x55a9fa53e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:41.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.329+0000 7f42a1269640 1 --2- 172.21.15.133:0/240882614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa5275d0 0x55a9fa53e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.329+0000 7f42a1269640 1 -- 172.21.15.133:0/240882614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 msgr2=0x55a9fa52a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:41.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.329+0000 7f42a1269640 1 --2- 172.21.15.133:0/240882614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa52a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.329+0000 7f42a1269640 1 -- 172.21.15.133:0/240882614 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9fa531890 con 0x55a9fa532370 2024-09-11T23:46:41.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.330+0000 7f42a1269640 1 --2- 172.21.15.133:0/240882614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa532370 0x55a9fa52a660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f429400ad00 tx=0x7f4294040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24870ac4a98c5311 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:41.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.331+0000 7f42867fc640 1 -- 172.21.15.133:0/240882614 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294044040 con 0x55a9fa532370 2024-09-11T23:46:41.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.331+0000 7f42867fc640 1 -- 172.21.15.133:0/240882614 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4294030040 con 0x55a9fa532370 2024-09-11T23:46:41.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.332+0000 7f42a50a5740 1 -- 172.21.15.133:0/240882614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa532370 msgr2=0x55a9fa52a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:41.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.332+0000 7f42a50a5740 1 --2- 172.21.15.133:0/240882614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa532370 0x55a9fa52a660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f429400ad00 tx=0x7f4294040000 comp rx=0 tx=0).stop 2024-09-11T23:46:41.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.333+0000 7f42a50a5740 1 -- 172.21.15.133:0/240882614 shutdown_connections 2024-09-11T23:46:41.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.333+0000 7f42a50a5740 1 --2- 172.21.15.133:0/240882614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa532370 0x55a9fa52a660 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.333+0000 7f42a50a5740 1 --2- 172.21.15.133:0/240882614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa52a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.333+0000 7f42a50a5740 1 --2- 172.21.15.133:0/240882614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa5275d0 0x55a9fa53e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.333+0000 7f42a50a5740 1 -- 172.21.15.133:0/240882614 >> 172.21.15.133:0/240882614 conn(0x55a9fa4f0310 msgr2=0x55a9fa530fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:41.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.334+0000 7f42a50a5740 1 -- 172.21.15.133:0/240882614 shutdown_connections 2024-09-11T23:46:41.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.334+0000 7f42a50a5740 1 -- 172.21.15.133:0/240882614 wait complete. 2024-09-11T23:46:41.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.334+0000 7f42a50a5740 1 -- start start 2024-09-11T23:46:41.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.335+0000 7f42a50a5740 1 Processor -- start 2024-09-11T23:46:41.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.336+0000 7f42a50a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 0x55a9fa53a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.336+0000 7f42a50a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa53aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.336+0000 7f42a0a68640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 0x55a9fa53a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.336+0000 7f42a0a68640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 0x55a9fa53a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:44888/0 (socket says 172.21.15.133:44888) 2024-09-11T23:46:41.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.336+0000 7f42a0a68640 1 -- 172.21.15.133:0/1513912967 learned_addr learned my addr 172.21.15.133:0/1513912967 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:41.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.336+0000 7f42a50a5740 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa532370 0x55a9fa52b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.337+0000 7f42a50a5740 1 -- 172.21.15.133:0/1513912967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f429c1c6350 con 0x55a9fa528b40 2024-09-11T23:46:41.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.338+0000 7f42a50a5740 1 -- 172.21.15.133:0/1513912967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f429c074240 con 0x55a9fa532370 2024-09-11T23:46:41.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.338+0000 7f42a50a5740 1 -- 172.21.15.133:0/1513912967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f429c1c6a20 con 0x55a9fa5275d0 2024-09-11T23:46:41.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.338+0000 7f42a50a5740 10 client.0 _ll_register_callbacks cb 0x7f4288009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:41.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.339+0000 7f429bfff640 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa53aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.339+0000 7f42a1269640 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa532370 0x55a9fa52b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.339+0000 7f42a0a68640 1 -- 172.21.15.133:0/1513912967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa532370 msgr2=0x55a9fa52b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:41.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.339+0000 7f42a0a68640 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa532370 0x55a9fa52b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.339+0000 7f42a0a68640 1 -- 172.21.15.133:0/1513912967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 msgr2=0x55a9fa53aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:41.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.339+0000 7f42a0a68640 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa53aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.339+0000 7f42a0a68640 1 -- 172.21.15.133:0/1513912967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9fa52bb70 con 0x55a9fa5275d0 2024-09-11T23:46:41.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.344+0000 7f42a0a68640 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 0x55a9fa53a510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4290005890 tx=0x7f4290005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b68a6a07985b041 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:41.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.344+0000 7f427bfff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:41.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.345+0000 7f427bfff640 1 -- 172.21.15.133:0/1513912967 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290096e80 con 0x55a9fa5275d0 2024-09-11T23:46:41.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.345+0000 7f427bfff640 1 -- 172.21.15.133:0/1513912967 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f4290009d60 con 0x55a9fa5275d0 2024-09-11T23:46:41.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.345+0000 7f42a50a5740 1 -- 172.21.15.133:0/1513912967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4294030040 con 0x55a9fa5275d0 2024-09-11T23:46:41.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.346+0000 7f427bfff640 1 -- 172.21.15.133:0/1513912967 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4290098ee0 con 0x55a9fa5275d0 2024-09-11T23:46:41.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.346+0000 7f42a50a5740 10 client.27241 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:41.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.346+0000 7f42a50a5740 1 -- 172.21.15.133:0/1513912967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a9fa5aca70 con 0x55a9fa5275d0 2024-09-11T23:46:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.346+0000 7f427bfff640 1 -- 172.21.15.133:0/1513912967 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42900cc090 con 0x55a9fa5275d0 2024-09-11T23:46:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.346+0000 7f427bfff640 1 -- 172.21.15.133:0/1513912967 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4290080070 con 0x55a9fa5275d0 2024-09-11T23:46:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.346+0000 7f42a50a5740 10 client.27241 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.346+0000 7f42a50a5740 10 client.27241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.346+0000 7f42a50a5740 20 client.27241 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.346+0000 7f42a50a5740 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a9fa5acca0 0x55a9fa5af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.346+0000 7f42a50a5740 4 client.27241 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:46:41.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.346+0000 7f42a50a5740 1 -- 172.21.15.133:0/1513912967 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4294030040 con 0x55a9fa5acca0 2024-09-11T23:46:41.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.347+0000 7f429bfff640 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a9fa5acca0 0x55a9fa5af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.348+0000 7f429bfff640 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a9fa5acca0 0x55a9fa5af090 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e15be9fd2eaa3a server_cookie=a715cd7e7477daa5 in_seq=0 out_seq=0 2024-09-11T23:46:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.348+0000 7f427bfff640 10 client.27241 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.349+0000 7f427bfff640 1 -- 172.21.15.133:0/1513912967 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4294030040 con 0x55a9fa5acca0 2024-09-11T23:46:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.349+0000 7f427bfff640 10 client.27241 handle_command_reply: tid=0 2024-09-11T23:46:41.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.349+0000 7f42a50a5740 1 client.27241 shutdown 2024-09-11T23:46:41.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.358+0000 7f42a50a5740 1 -- 172.21.15.133:0/1513912967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 msgr2=0x55a9fa53a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:41.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.358+0000 7f42a50a5740 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 0x55a9fa53a510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4290005890 tx=0x7f4290005bf0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.363+0000 7f42a50a5740 1 -- 172.21.15.133:0/1513912967 shutdown_connections 2024-09-11T23:46:41.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.363+0000 7f42a50a5740 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a9fa5acca0 0x55a9fa5af090 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.363+0000 7f42a50a5740 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9fa532370 0x55a9fa52b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.363+0000 7f42a50a5740 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9fa528b40 0x55a9fa53aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.363+0000 7f42a50a5740 1 --2- 172.21.15.133:0/1513912967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9fa5275d0 0x55a9fa53a510 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.363+0000 7f42a50a5740 1 -- 172.21.15.133:0/1513912967 >> 172.21.15.133:0/1513912967 conn(0x55a9fa4f0310 msgr2=0x55a9fa52e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:41.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.364+0000 7f427bfff640 0 client.27241 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.370+0000 7f42a50a5740 1 -- 172.21.15.133:0/1513912967 shutdown_connections 2024-09-11T23:46:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.370+0000 7f42a50a5740 1 -- 172.21.15.133:0/1513912967 wait complete. 2024-09-11T23:46:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.370+0000 7f42a50a5740 20 client.27241 trim_cache size 0 max 16384 2024-09-11T23:46:41.374 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:46:41.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.376+0000 7f42a2ccc640 1 -- 172.21.15.133:0/29283341 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f428c0a2db0 msgr2=0x7f428c0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:41.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.376+0000 7f42a2ccc640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f428c0a2db0 0x7f428c0a5270 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f429402be00 tx=0x7f429400a6d0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.376+0000 7f42a2ccc640 1 -- 172.21.15.133:0/29283341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429c105220 msgr2=0x7f429c10c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:41.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.376+0000 7f42a2ccc640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429c105220 0x7f429c10c160 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f429000d700 tx=0x7f429000dbc0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.377+0000 7f42a2ccc640 1 -- 172.21.15.133:0/29283341 shutdown_connections 2024-09-11T23:46:41.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.377+0000 7f42a2ccc640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f428c0a2db0 0x7f428c0a5270 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.377+0000 7f42a2ccc640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f429c19c130 0x7f429c19c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.377+0000 7f42a2ccc640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f429c10c6a0 0x7f429c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.377+0000 7f42a2ccc640 1 --2- 172.21.15.133:0/29283341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429c105220 0x7f429c10c160 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:41.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.377+0000 7f42a2ccc640 1 -- 172.21.15.133:0/29283341 >> 172.21.15.133:0/29283341 conn(0x7f429c077210 msgr2=0x7f429c0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:41.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.378+0000 7f42a2ccc640 1 -- 172.21.15.133:0/29283341 shutdown_connections 2024-09-11T23:46:41.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:41.378+0000 7f42a2ccc640 1 -- 172.21.15.133:0/29283341 wait complete. 2024-09-11T23:46:42.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:42 smithi138 ceph-mon[35142]: pgmap v309: 97 pgs: 97 active+clean; 70 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 610 MiB/s wr, 1.77k op/s 2024-09-11T23:46:42.462 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:42 smithi133 ceph-mon[31430]: pgmap v309: 97 pgs: 97 active+clean; 70 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 610 MiB/s wr, 1.77k op/s 2024-09-11T23:46:42.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:42 smithi178 ceph-mon[35882]: pgmap v309: 97 pgs: 97 active+clean; 70 GiB data, 280 GiB used, 793 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 610 MiB/s wr, 1.77k op/s 2024-09-11T23:46:43.617 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:46:43.617 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:46:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.269+0000 7f9ab6677640 1 -- 172.21.15.133:0/2016261846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab01091f0 msgr2=0x7f9ab0110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.269+0000 7f9ab6677640 1 --2- 172.21.15.133:0/2016261846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab01091f0 0x7f9ab0110250 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009860 tx=0x7f9aa802a380 comp rx=0 tx=0).stop 2024-09-11T23:46:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.276+0000 7f9ab6677640 1 -- 172.21.15.133:0/2016261846 shutdown_connections 2024-09-11T23:46:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.276+0000 7f9ab6677640 1 --2- 172.21.15.133:0/2016261846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ab011b140 0x7f9ab011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.276+0000 7f9ab6677640 1 --2- 172.21.15.133:0/2016261846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab01095d0 0x7f9ab0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.276+0000 7f9ab6677640 1 --2- 172.21.15.133:0/2016261846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab01091f0 0x7f9ab0110250 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.276+0000 7f9ab6677640 1 -- 172.21.15.133:0/2016261846 >> 172.21.15.133:0/2016261846 conn(0x7f9ab006ce00 msgr2=0x7f9ab006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.280+0000 7f9ab6677640 1 -- 172.21.15.133:0/2016261846 shutdown_connections 2024-09-11T23:46:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.282+0000 7f9ab6677640 1 -- 172.21.15.133:0/2016261846 wait complete. 2024-09-11T23:46:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.283+0000 7f9ab6677640 1 Processor -- start 2024-09-11T23:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.285+0000 7f9ab6677640 1 -- start start 2024-09-11T23:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.286+0000 7f9ab6677640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab01091f0 0x7f9ab01a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.286+0000 7f9ab6677640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab01095d0 0x7f9ab01a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.286+0000 7f9ab6677640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ab011b140 0x7f9ab01ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.286+0000 7f9ab6677640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0120cf0 con 0x7f9ab01091f0 2024-09-11T23:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.286+0000 7f9ab6677640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0120b70 con 0x7f9ab01095d0 2024-09-11T23:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.286+0000 7f9ab6677640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0120e70 con 0x7f9ab011b140 2024-09-11T23:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.287+0000 7f9aaffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab01091f0 0x7f9ab01a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.287+0000 7f9aaffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab01091f0 0x7f9ab01a70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46720/0 (socket says 172.21.15.133:46720) 2024-09-11T23:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.288+0000 7f9aaffff640 1 -- 172.21.15.133:0/3145470678 learned_addr learned my addr 172.21.15.133:0/3145470678 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.289+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab01095d0 0x7f9ab01a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.290+0000 7f9aaffff640 1 -- 172.21.15.133:0/3145470678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ab011b140 msgr2=0x7f9ab01ab960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:46:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.290+0000 7f9aaffff640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ab011b140 0x7f9ab01ab960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.291+0000 7f9aaffff640 1 -- 172.21.15.133:0/3145470678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab01095d0 msgr2=0x7f9ab01a75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.291+0000 7f9aaffff640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab01095d0 0x7f9ab01a75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.291+0000 7f9aaffff640 1 -- 172.21.15.133:0/3145470678 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab01ac090 con 0x7f9ab01091f0 2024-09-11T23:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.291+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab01095d0 0x7f9ab01a75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:46:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.292+0000 7f9aaffff640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab01091f0 0x7f9ab01a70a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8035040 tx=0x7f9aa8030460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.293+0000 7f9aad7fa640 1 -- 172.21.15.133:0/3145470678 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8037040 con 0x7f9ab01091f0 2024-09-11T23:46:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.293+0000 7f9ab6677640 1 -- 172.21.15.133:0/3145470678 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab01ac320 con 0x7f9ab01091f0 2024-09-11T23:46:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.293+0000 7f9ab6677640 1 -- 172.21.15.133:0/3145470678 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab01b3c10 con 0x7f9ab01091f0 2024-09-11T23:46:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.295+0000 7f9aad7fa640 1 -- 172.21.15.133:0/3145470678 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aa8033e20 con 0x7f9ab01091f0 2024-09-11T23:46:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.295+0000 7f9aad7fa640 1 -- 172.21.15.133:0/3145470678 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8037040 con 0x7f9ab01091f0 2024-09-11T23:46:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.297+0000 7f9ab940a740 1 Processor -- start 2024-09-11T23:46:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.297+0000 7f9ab940a740 1 -- start start 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab940a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff17af0 0x55fa3ff17ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab940a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff18410 0x55fa3ff22770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab940a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff250a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab940a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fa3fe7c2f0 con 0x55fa3ff17af0 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab940a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fa3fe24270 con 0x55fa3ff18410 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab940a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa3fe782b0 con 0x55fa3ff22cb0 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab4c14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff17af0 0x55fa3ff17ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab4c14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff17af0 0x55fa3ff17ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46736/0 (socket says 172.21.15.133:46736) 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab4c14640 1 -- 172.21.15.133:0/2536066912 learned_addr learned my addr 172.21.15.133:0/2536066912 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab4c14640 1 -- 172.21.15.133:0/2536066912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 msgr2=0x55fa3ff250a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab4c14640 1 --2- 172.21.15.133:0/2536066912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff250a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab4c14640 1 -- 172.21.15.133:0/2536066912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff18410 msgr2=0x55fa3ff22770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:46:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab4c14640 1 --2- 172.21.15.133:0/2536066912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff18410 0x55fa3ff22770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.299+0000 7f9ab4c14640 1 -- 172.21.15.133:0/2536066912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa3fe23340 con 0x55fa3ff17af0 2024-09-11T23:46:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.300+0000 7f9ab4c14640 1 --2- 172.21.15.133:0/2536066912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff17af0 0x55fa3ff17ed0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009bd0 tx=0x7f9a9800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53a4739f54eaff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.301+0000 7f9aad7fa640 1 -- 172.21.15.133:0/3145470678 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9aa8005210 con 0x7f9ab01091f0 2024-09-11T23:46:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.301+0000 7f9aa67fc640 1 -- 172.21.15.133:0/2536066912 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98005020 con 0x55fa3ff17af0 2024-09-11T23:46:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.301+0000 7f9aa67fc640 1 -- 172.21.15.133:0/2536066912 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a980047a0 con 0x55fa3ff17af0 2024-09-11T23:46:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.302+0000 7f9aad7fa640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a9c0a3200 0x7f9a9c0a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.302+0000 7f9aad7fa640 1 -- 172.21.15.133:0/3145470678 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f9aa8033c20 con 0x7f9ab01091f0 2024-09-11T23:46:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.303+0000 7f9aaffff640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a9c0a3200 0x7f9a9c0a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.305+0000 7f9ab940a740 1 -- 172.21.15.133:0/2536066912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff17af0 msgr2=0x55fa3ff17ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.305+0000 7f9ab940a740 1 --2- 172.21.15.133:0/2536066912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff17af0 0x55fa3ff17ed0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009bd0 tx=0x7f9a9800e720 comp rx=0 tx=0).stop 2024-09-11T23:46:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.305+0000 7f9aaffff640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a9c0a3200 0x7f9a9c0a56c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55fa3ff14ec0 tx=0x7f9aa80b7020 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.305+0000 7f9ab940a740 1 -- 172.21.15.133:0/2536066912 shutdown_connections 2024-09-11T23:46:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.305+0000 7f9ab940a740 1 --2- 172.21.15.133:0/2536066912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff250a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.305+0000 7f9ab940a740 1 --2- 172.21.15.133:0/2536066912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff18410 0x55fa3ff22770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.305+0000 7f9ab940a740 1 --2- 172.21.15.133:0/2536066912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff17af0 0x55fa3ff17ed0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.305+0000 7f9ab940a740 1 -- 172.21.15.133:0/2536066912 >> 172.21.15.133:0/2536066912 conn(0x55fa3ff14060 msgr2=0x55fa3ff14440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.305+0000 7f9ab940a740 1 -- 172.21.15.133:0/2536066912 shutdown_connections 2024-09-11T23:46:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.306+0000 7f9ab940a740 1 -- 172.21.15.133:0/2536066912 wait complete. 2024-09-11T23:46:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.306+0000 7f9ab940a740 1 -- start start 2024-09-11T23:46:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.307+0000 7f9ab940a740 1 Processor -- start 2024-09-11T23:46:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.307+0000 7f9ab940a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff18410 0x55fa3ff225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.307+0000 7f9ab940a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff1f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.307+0000 7f9ab940a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff20210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.307+0000 7f9ab940a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fa3ff258d0 con 0x55fa3ff18410 2024-09-11T23:46:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.307+0000 7f9ab940a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fa3fe05290 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.307+0000 7f9ab940a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa3ff25a50 con 0x55fa3ff22cb0 2024-09-11T23:46:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.307+0000 7f9ab940a740 10 client.0 _ll_register_callbacks cb 0x55fa3fe36140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.308+0000 7f9ab4c14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff20210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.308+0000 7f9aaf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff1f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.308+0000 7f9ab4c14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff20210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:50582/0 (socket says 172.21.15.133:50582) 2024-09-11T23:46:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.308+0000 7f9ab4c14640 1 -- 172.21.15.133:0/2447937105 learned_addr learned my addr 172.21.15.133:0/2447937105 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.309+0000 7f9aaf7fe640 1 -- 172.21.15.133:0/2447937105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 msgr2=0x55fa3ff20210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.309+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/2447937105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff20210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.309+0000 7f9aaf7fe640 1 -- 172.21.15.133:0/2447937105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff18410 msgr2=0x55fa3ff225f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.309+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/2447937105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff18410 0x55fa3ff225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.309+0000 7f9aaf7fe640 1 -- 172.21.15.133:0/2447937105 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa3fb8f210 con 0x55fa3ff22cb0 2024-09-11T23:46:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.309+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/2447937105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff1f890 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55fa3ff1a690 tx=0x7f9aa000d870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ae614cb232692fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:44.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.309+0000 7f9a8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.310+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/2447937105 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa001c060 con 0x55fa3ff22cb0 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.310+0000 7f9ab940a740 1 -- 172.21.15.133:0/2447937105 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fa3fe89a20 con 0x55fa3ff22cb0 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.310+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/2447937105 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f9aa0004030 con 0x55fa3ff22cb0 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.310+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/2447937105 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9aa0003e40 con 0x55fa3ff22cb0 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.310+0000 7f9ab940a740 10 client.27247 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.310+0000 7f9ab940a740 1 -- 172.21.15.133:0/2447937105 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa3fc5b620 con 0x55fa3ff22cb0 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.311+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/2447937105 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fa3fe89a20 con 0x55fa3ff22cb0 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.311+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/2447937105 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9aa000b040 con 0x55fa3ff22cb0 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.311+0000 7f9ab940a740 10 client.27247 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.311+0000 7f9ab940a740 10 client.27247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.311+0000 7f9ab940a740 20 client.27247 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.311+0000 7f9ab940a740 1 --2- 172.21.15.133:0/2447937105 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fa3ff28e60 0x55fa3ff2b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.311+0000 7f9ab940a740 4 client.27247 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.311+0000 7f9ab940a740 1 -- 172.21.15.133:0/2447937105 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa3fe83d70 con 0x55fa3ff28e60 2024-09-11T23:46:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.311+0000 7f9ab4c14640 1 --2- 172.21.15.133:0/2447937105 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fa3ff28e60 0x55fa3ff2b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.313+0000 7f9a8f7fe640 10 client.27247 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.313+0000 7f9ab4c14640 1 --2- 172.21.15.133:0/2447937105 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fa3ff28e60 0x55fa3ff2b2a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=345d79fc78723180 server_cookie=1e187e8cbbd71c48 in_seq=0 out_seq=0 2024-09-11T23:46:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.316+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/2447937105 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa3fe83d70 con 0x55fa3ff28e60 2024-09-11T23:46:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.316+0000 7f9a8f7fe640 10 client.27247 handle_command_reply: tid=0 2024-09-11T23:46:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.318+0000 7f9ab940a740 1 client.27247 shutdown 2024-09-11T23:46:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.318+0000 7f9ab940a740 1 -- 172.21.15.133:0/2447937105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 msgr2=0x55fa3ff1f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.318+0000 7f9ab940a740 1 --2- 172.21.15.133:0/2447937105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff1f890 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55fa3ff1a690 tx=0x7f9aa000d870 comp rx=0 tx=0).stop 2024-09-11T23:46:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.318+0000 7f9ab940a740 1 -- 172.21.15.133:0/2447937105 shutdown_connections 2024-09-11T23:46:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.318+0000 7f9ab940a740 1 --2- 172.21.15.133:0/2447937105 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fa3ff28e60 0x55fa3ff2b2a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.318+0000 7f9ab940a740 1 --2- 172.21.15.133:0/2447937105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff20210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.318+0000 7f9ab940a740 1 --2- 172.21.15.133:0/2447937105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff1f890 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.319+0000 7f9ab940a740 1 --2- 172.21.15.133:0/2447937105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff18410 0x55fa3ff225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.319+0000 7f9ab940a740 1 -- 172.21.15.133:0/2447937105 >> 172.21.15.133:0/2447937105 conn(0x55fa3ff14060 msgr2=0x55fa3ff195a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.319+0000 7f9a8f7fe640 0 client.27247 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.319+0000 7f9ab940a740 1 -- 172.21.15.133:0/2447937105 shutdown_connections 2024-09-11T23:46:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.319+0000 7f9ab940a740 1 -- 172.21.15.133:0/2447937105 wait complete. 2024-09-11T23:46:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.319+0000 7f9ab940a740 20 client.27247 trim_cache size 0 max 16384 2024-09-11T23:46:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.363+0000 7f9ab940a740 1 Processor -- start 2024-09-11T23:46:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.364+0000 7f9ab940a740 1 -- start start 2024-09-11T23:46:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.368+0000 7f9ab940a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff18410 0x55fa3ff1f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.368+0000 7f9ab940a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff1f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.368+0000 7f9ab940a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff19550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.368+0000 7f9ab940a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9000e780 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.368+0000 7f9ab940a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a900102c0 con 0x55fa3ff18410 2024-09-11T23:46:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.368+0000 7f9ab940a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9000ee50 con 0x55fa3ff22cb0 2024-09-11T23:46:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.369+0000 7f9aaf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff1f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.369+0000 7f9ab4c14640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff19550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.369+0000 7f9aaf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff1f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46768/0 (socket says 172.21.15.133:46768) 2024-09-11T23:46:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.369+0000 7f9aaf7fe640 1 -- 172.21.15.133:0/1438569055 learned_addr learned my addr 172.21.15.133:0/1438569055 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.372+0000 7f9aaf7fe640 1 -- 172.21.15.133:0/1438569055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 msgr2=0x55fa3ff19550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.373+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/1438569055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff19550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.373+0000 7f9aaf7fe640 1 -- 172.21.15.133:0/1438569055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff18410 msgr2=0x55fa3ff1f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.373+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/1438569055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff18410 0x55fa3ff1f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.373+0000 7f9aaf7fe640 1 -- 172.21.15.133:0/1438569055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa3ff20580 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.378+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/1438569055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff1f670 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0021cf0 tx=0x7f9aa000e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a29807e8cc53cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.394+0000 7f9aa5ffb640 1 -- 172.21.15.133:0/1438569055 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa002f070 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.395+0000 7f9aa5ffb640 1 -- 172.21.15.133:0/1438569055 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aa0016380 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.395+0000 7f9aa5ffb640 1 -- 172.21.15.133:0/1438569055 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00057c0 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.396+0000 7f9ab940a740 1 -- 172.21.15.133:0/1438569055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff1fdd0 msgr2=0x55fa3ff1f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.396+0000 7f9ab940a740 1 --2- 172.21.15.133:0/1438569055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff1f670 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0021cf0 tx=0x7f9aa000e8a0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.399+0000 7f9ab940a740 1 -- 172.21.15.133:0/1438569055 shutdown_connections 2024-09-11T23:46:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.399+0000 7f9ab940a740 1 --2- 172.21.15.133:0/1438569055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff19550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.399+0000 7f9ab940a740 1 --2- 172.21.15.133:0/1438569055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff1f670 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.399+0000 7f9ab940a740 1 --2- 172.21.15.133:0/1438569055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff18410 0x55fa3ff1f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.399+0000 7f9ab940a740 1 -- 172.21.15.133:0/1438569055 >> 172.21.15.133:0/1438569055 conn(0x55fa3ff14060 msgr2=0x55fa3ff1e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.407+0000 7f9ab940a740 1 -- 172.21.15.133:0/1438569055 shutdown_connections 2024-09-11T23:46:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.407+0000 7f9ab940a740 1 -- 172.21.15.133:0/1438569055 wait complete. 2024-09-11T23:46:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.408+0000 7f9ab940a740 1 -- start start 2024-09-11T23:46:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.409+0000 7f9ab940a740 1 Processor -- start 2024-09-11T23:46:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.410+0000 7f9ab940a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff18410 0x55fa3ff2eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.410+0000 7f9ab940a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff2f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.410+0000 7f9ab4c14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff18410 0x55fa3ff2eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.410+0000 7f9ab4c14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff18410 0x55fa3ff2eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46776/0 (socket says 172.21.15.133:46776) 2024-09-11T23:46:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.410+0000 7f9ab4c14640 1 -- 172.21.15.133:0/77875293 learned_addr learned my addr 172.21.15.133:0/77875293 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.410+0000 7f9ab940a740 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff2f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.411+0000 7f9ab940a740 1 -- 172.21.15.133:0/77875293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9000fbf0 con 0x55fa3ff18410 2024-09-11T23:46:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.411+0000 7f9ab940a740 1 -- 172.21.15.133:0/77875293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9000f520 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.411+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff2f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.411+0000 7f9ab940a740 1 -- 172.21.15.133:0/77875293 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9000c780 con 0x55fa3ff22cb0 2024-09-11T23:46:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.411+0000 7f9ab940a740 10 client.0 _ll_register_callbacks cb 0x7f9aa000e450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.411+0000 7f9ab4c14640 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff2f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.412+0000 7f9aaf7fe640 1 -- 172.21.15.133:0/77875293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 msgr2=0x55fa3ff2f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.412+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff2f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.412+0000 7f9aaf7fe640 1 -- 172.21.15.133:0/77875293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff18410 msgr2=0x55fa3ff2eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.412+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff18410 0x55fa3ff2eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.412+0000 7f9aaf7fe640 1 -- 172.21.15.133:0/77875293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa3ffbe610 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.412+0000 7f9aaf7fe640 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff2f060 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55fa3ff22c30 tx=0x7f9aa0005220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a748c8de99f007d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.412+0000 7f9a8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:46:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.418+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/77875293 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa002f070 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.418+0000 7f9ab940a740 1 -- 172.21.15.133:0/77875293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fa3fe00900 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.418+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/77875293 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f9aa0021880 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.423+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/77875293 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9aa001c050 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.423+0000 7f9ab940a740 10 client.27455 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.423+0000 7f9ab940a740 1 -- 172.21.15.133:0/77875293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa3ffbf8c0 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.427+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/77875293 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fa3fe00900 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.429+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/77875293 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9aa000b070 con 0x55fa3ff1fdd0 2024-09-11T23:46:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.429+0000 7f9ab940a740 10 client.27455 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.429+0000 7f9ab940a740 10 client.27455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.429+0000 7f9ab940a740 20 client.27455 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.429+0000 7f9ab940a740 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fa3ffbfed0 0x55fa3ffc22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.429+0000 7f9ab940a740 4 client.27455 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:46:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.429+0000 7f9ab940a740 1 -- 172.21.15.133:0/77875293 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa3fe4d820 con 0x55fa3ffbfed0 2024-09-11T23:46:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.437+0000 7f9ab4c14640 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fa3ffbfed0 0x55fa3ffc22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.438+0000 7f9ab4c14640 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fa3ffbfed0 0x55fa3ffc22c0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cfa79b78cb1bd29 server_cookie=15ae668402df860c in_seq=0 out_seq=0 2024-09-11T23:46:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:44.438+0000 7f9a8f7fe640 10 client.27455 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:44.787 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:45.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:44 smithi133 ceph-mon[31430]: pgmap v310: 97 pgs: 97 active+clean; 70 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 802 MiB/s wr, 2.54k op/s 2024-09-11T23:46:45.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:44 smithi178 ceph-mon[35882]: pgmap v310: 97 pgs: 97 active+clean; 70 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 802 MiB/s wr, 2.54k op/s 2024-09-11T23:46:45.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:44 smithi138 ceph-mon[35142]: pgmap v310: 97 pgs: 97 active+clean; 70 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 802 MiB/s wr, 2.54k op/s 2024-09-11T23:46:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.749+0000 7fb596104640 1 -- 172.21.15.133:0/1592735066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb59011b140 msgr2=0x7fb59011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.749+0000 7fb596104640 1 --2- 172.21.15.133:0/1592735066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb59011b140 0x7fb59011d530 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb584009860 tx=0x7fb58402a380 comp rx=0 tx=0).stop 2024-09-11T23:46:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.749+0000 7fb596104640 1 -- 172.21.15.133:0/1592735066 shutdown_connections 2024-09-11T23:46:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.749+0000 7fb596104640 1 --2- 172.21.15.133:0/1592735066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb59011b140 0x7fb59011d530 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.749+0000 7fb596104640 1 --2- 172.21.15.133:0/1592735066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb590109830 0x7fb590109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.749+0000 7fb596104640 1 --2- 172.21.15.133:0/1592735066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb590109450 0x7fb590110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.749+0000 7fb596104640 1 -- 172.21.15.133:0/1592735066 >> 172.21.15.133:0/1592735066 conn(0x7fb59006e140 msgr2=0x7fb59006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.750+0000 7fb596104640 1 -- 172.21.15.133:0/1592735066 shutdown_connections 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.750+0000 7fb596104640 1 -- 172.21.15.133:0/1592735066 wait complete. 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.751+0000 7fb596104640 1 Processor -- start 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.751+0000 7fb596104640 1 -- start start 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.751+0000 7fb596104640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb590109450 0x7fb5901a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.751+0000 7fb596104640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb590109830 0x7fb5901a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.751+0000 7fb596104640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5901ab840 0x7fb5901abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.751+0000 7fb596104640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb590120e10 con 0x7fb590109830 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.751+0000 7fb596104640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb590120c90 con 0x7fb590109450 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.751+0000 7fb596104640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb590120f90 con 0x7fb5901ab840 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.752+0000 7fb58effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb590109830 0x7fb5901a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.752+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb590109450 0x7fb5901a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.753+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb590109450 0x7fb5901a6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:50616/0 (socket says 172.21.15.133:50616) 2024-09-11T23:46:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.753+0000 7fb58f7fe640 1 -- 172.21.15.133:0/965168384 learned_addr learned my addr 172.21.15.133:0/965168384 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.753+0000 7fb58ffff640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5901ab840 0x7fb5901abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.753+0000 7fb58f7fe640 1 -- 172.21.15.133:0/965168384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5901ab840 msgr2=0x7fb5901abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.753+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5901ab840 0x7fb5901abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.753+0000 7fb58f7fe640 1 -- 172.21.15.133:0/965168384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb590109830 msgr2=0x7fb5901a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.753+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb590109830 0x7fb5901a74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.753+0000 7fb58f7fe640 1 -- 172.21.15.133:0/965168384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5901ac390 con 0x7fb590109450 2024-09-11T23:46:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.754+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb590109450 0x7fb5901a6f70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb58000b4a0 tx=0x7fb58000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:45.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.794+0000 7fb58cff9640 1 -- 172.21.15.133:0/965168384 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580014070 con 0x7fb590109450 2024-09-11T23:46:45.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.795+0000 7fb596104640 1 -- 172.21.15.133:0/965168384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5901ac680 con 0x7fb590109450 2024-09-11T23:46:45.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.795+0000 7fb596104640 1 -- 172.21.15.133:0/965168384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59007a7c0 con 0x7fb590109450 2024-09-11T23:46:45.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.800+0000 7fb58cff9640 1 -- 172.21.15.133:0/965168384 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb580010af0 con 0x7fb590109450 2024-09-11T23:46:45.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.800+0000 7fb58cff9640 1 -- 172.21.15.133:0/965168384 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580017d10 con 0x7fb590109450 2024-09-11T23:46:45.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.801+0000 7fb5984dd740 1 Processor -- start 2024-09-11T23:46:45.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.801+0000 7fb5984dd740 1 -- start start 2024-09-11T23:46:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.803+0000 7fb5984dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 0x5593f6988f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.803+0000 7fb5984dd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6989480 0x5593f6993800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.803+0000 7fb5984dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6993d40 0x5593f6996130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.809+0000 7fb5984dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5593f68ee280 con 0x5593f6988b60 2024-09-11T23:46:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.809+0000 7fb5984dd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5593f6895b10 con 0x5593f6989480 2024-09-11T23:46:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.809+0000 7fb5984dd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5593f68ea240 con 0x5593f6993d40 2024-09-11T23:46:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.810+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6989480 0x5593f6993800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.810+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6989480 0x5593f6993800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:50624/0 (socket says 172.21.15.133:50624) 2024-09-11T23:46:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.810+0000 7fb58f7fe640 1 -- 172.21.15.133:0/1478449337 learned_addr learned my addr 172.21.15.133:0/1478449337 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.810+0000 7fb58effd640 1 --2- 172.21.15.133:0/1478449337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6993d40 0x5593f6996130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.811+0000 7fb58ffff640 1 --2- 172.21.15.133:0/1478449337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 0x5593f6988f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.811+0000 7fb58f7fe640 1 -- 172.21.15.133:0/1478449337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6993d40 msgr2=0x5593f6996130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.811+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/1478449337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6993d40 0x5593f6996130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.811+0000 7fb58f7fe640 1 -- 172.21.15.133:0/1478449337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 msgr2=0x5593f6988f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.811+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/1478449337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 0x5593f6988f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.811+0000 7fb58f7fe640 1 -- 172.21.15.133:0/1478449337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593f68a21f0 con 0x5593f6989480 2024-09-11T23:46:45.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.814+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/1478449337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6989480 0x5593f6993800 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb580017fd0 tx=0x7fb580019c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=428e9495b340be95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:45.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.825+0000 7fb575ffb640 1 -- 172.21.15.133:0/1478449337 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58001e060 con 0x5593f6989480 2024-09-11T23:46:45.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.825+0000 7fb575ffb640 1 -- 172.21.15.133:0/1478449337 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb580019e10 con 0x5593f6989480 2024-09-11T23:46:45.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.825+0000 7fb575ffb640 1 -- 172.21.15.133:0/1478449337 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580023ba0 con 0x5593f6989480 2024-09-11T23:46:45.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.826+0000 7fb5984dd740 1 -- 172.21.15.133:0/1478449337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6989480 msgr2=0x5593f6993800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:45.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.826+0000 7fb5984dd740 1 --2- 172.21.15.133:0/1478449337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6989480 0x5593f6993800 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb580017fd0 tx=0x7fb580019c20 comp rx=0 tx=0).stop 2024-09-11T23:46:45.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.827+0000 7fb5984dd740 1 -- 172.21.15.133:0/1478449337 shutdown_connections 2024-09-11T23:46:45.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.827+0000 7fb5984dd740 1 --2- 172.21.15.133:0/1478449337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6993d40 0x5593f6996130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.827+0000 7fb5984dd740 1 --2- 172.21.15.133:0/1478449337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6989480 0x5593f6993800 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.827+0000 7fb5984dd740 1 --2- 172.21.15.133:0/1478449337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 0x5593f6988f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.827+0000 7fb5984dd740 1 -- 172.21.15.133:0/1478449337 >> 172.21.15.133:0/1478449337 conn(0x5593f69850f0 msgr2=0x5593f69854d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:45.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.834+0000 7fb58cff9640 1 -- 172.21.15.133:0/965168384 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb580017070 con 0x7fb590109450 2024-09-11T23:46:45.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.835+0000 7fb5984dd740 1 -- 172.21.15.133:0/1478449337 shutdown_connections 2024-09-11T23:46:45.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.835+0000 7fb5984dd740 1 -- 172.21.15.133:0/1478449337 wait complete. 2024-09-11T23:46:45.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.835+0000 7fb5984dd740 1 -- start start 2024-09-11T23:46:45.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.836+0000 7fb58cff9640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5780a2e10 0x7fb5780a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.836+0000 7fb5984dd740 1 Processor -- start 2024-09-11T23:46:45.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.837+0000 7fb5984dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 0x5593f6990c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.837+0000 7fb5984dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6989480 0x5593f6991140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.837+0000 7fb5984dd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6993d40 0x5593f6991680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.837+0000 7fb5984dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5593f6996910 con 0x5593f6988b60 2024-09-11T23:46:45.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.837+0000 7fb5984dd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5593f68a8f10 con 0x5593f6993d40 2024-09-11T23:46:45.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.837+0000 7fb5984dd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5593f6996a90 con 0x5593f6989480 2024-09-11T23:46:45.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.837+0000 7fb5984dd740 10 client.0 _ll_register_callbacks cb 0x5593f6871a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:45.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.838+0000 7fb58cff9640 1 -- 172.21.15.133:0/965168384 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fb5800d1230 con 0x7fb590109450 2024-09-11T23:46:45.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 0x5593f6990c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 0x5593f6990c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46804/0 (socket says 172.21.15.133:46804) 2024-09-11T23:46:45.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58f7fe640 1 -- 172.21.15.133:0/689899987 learned_addr learned my addr 172.21.15.133:0/689899987 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:45.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58effd640 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6989480 0x5593f6991140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58ffff640 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6993d40 0x5593f6991680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58ffff640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5780a2e10 0x7fb5780a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58f7fe640 1 -- 172.21.15.133:0/689899987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6989480 msgr2=0x5593f6991140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:45.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6989480 0x5593f6991140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58f7fe640 1 -- 172.21.15.133:0/689899987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6993d40 msgr2=0x5593f6991680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:45.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6993d40 0x5593f6991680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58f7fe640 1 -- 172.21.15.133:0/689899987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593f665f6a0 con 0x5593f6988b60 2024-09-11T23:46:45.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 0x5593f6990c00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fb580099e70 tx=0x7fb580019450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cda4ed36d8e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:45.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.842+0000 7fb56effd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:46:45.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.851+0000 7fb56effd640 1 -- 172.21.15.133:0/689899987 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800a1dc0 con 0x5593f6988b60 2024-09-11T23:46:45.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.855+0000 7fb5984dd740 1 -- 172.21.15.133:0/689899987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb580019e10 con 0x5593f6988b60 2024-09-11T23:46:45.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.855+0000 7fb56effd640 1 -- 172.21.15.133:0/689899987 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fb58009bd10 con 0x5593f6988b60 2024-09-11T23:46:45.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.857+0000 7fb56effd640 1 -- 172.21.15.133:0/689899987 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb5800a65f0 con 0x5593f6988b60 2024-09-11T23:46:45.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.857+0000 7fb5984dd740 10 client.17922 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:45.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.857+0000 7fb5984dd740 1 -- 172.21.15.133:0/689899987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5593f69b3590 con 0x5593f6988b60 2024-09-11T23:46:45.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.859+0000 7fb56effd640 1 -- 172.21.15.133:0/689899987 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb580019e10 con 0x5593f6988b60 2024-09-11T23:46:45.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.859+0000 7fb56effd640 1 -- 172.21.15.133:0/689899987 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb5800a40e0 con 0x5593f6988b60 2024-09-11T23:46:45.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.859+0000 7fb5984dd740 10 client.17922 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:45.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.859+0000 7fb5984dd740 10 client.17922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:45.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.859+0000 7fb5984dd740 20 client.17922 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:45.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.859+0000 7fb5984dd740 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5593f69b4a40 0x5593f69b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.859+0000 7fb5984dd740 4 client.17922 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:45.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.859+0000 7fb5984dd740 1 -- 172.21.15.133:0/689899987 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593f68fb7f0 con 0x5593f69b4a40 2024-09-11T23:46:45.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.861+0000 7fb58effd640 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5593f69b4a40 0x5593f69b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.865+0000 7fb58ffff640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5780a2e10 0x7fb5780a52d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb584005e00 tx=0x7fb584009280 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:45.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.870+0000 7fb58effd640 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5593f69b4a40 0x5593f69b6ee0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b45ddae82a4ba58 server_cookie=7fae130dfc137742 in_seq=0 out_seq=0 2024-09-11T23:46:45.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.870+0000 7fb56effd640 10 client.17922 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:45.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.876+0000 7fb56effd640 1 -- 172.21.15.133:0/689899987 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593f68fb7f0 con 0x5593f69b4a40 2024-09-11T23:46:45.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.876+0000 7fb56effd640 10 client.17922 handle_command_reply: tid=0 2024-09-11T23:46:45.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.876+0000 7fb5984dd740 1 client.17922 shutdown 2024-09-11T23:46:45.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.880+0000 7fb5984dd740 1 -- 172.21.15.133:0/689899987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 msgr2=0x5593f6990c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:45.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.880+0000 7fb5984dd740 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 0x5593f6990c00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fb580099e70 tx=0x7fb580019450 comp rx=0 tx=0).stop 2024-09-11T23:46:45.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.883+0000 7fb5984dd740 1 -- 172.21.15.133:0/689899987 shutdown_connections 2024-09-11T23:46:45.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.883+0000 7fb5984dd740 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5593f69b4a40 0x5593f69b6ee0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.883+0000 7fb5984dd740 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6993d40 0x5593f6991680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.883+0000 7fb5984dd740 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6989480 0x5593f6991140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.883+0000 7fb5984dd740 1 --2- 172.21.15.133:0/689899987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6988b60 0x5593f6990c00 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.883+0000 7fb5984dd740 1 -- 172.21.15.133:0/689899987 >> 172.21.15.133:0/689899987 conn(0x5593f69850f0 msgr2=0x5593f6984270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:45.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.883+0000 7fb56effd640 0 client.17922 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:45.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.883+0000 7fb5984dd740 1 -- 172.21.15.133:0/689899987 shutdown_connections 2024-09-11T23:46:45.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.883+0000 7fb5984dd740 1 -- 172.21.15.133:0/689899987 wait complete. 2024-09-11T23:46:45.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.883+0000 7fb5984dd740 20 client.17922 trim_cache size 0 max 16384 2024-09-11T23:46:45.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.934+0000 7fb5984dd740 1 Processor -- start 2024-09-11T23:46:45.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.938+0000 7fb5984dd740 1 -- start start 2024-09-11T23:46:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.938+0000 7fb5984dd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6988b60 0x5593f6992a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.938+0000 7fb5984dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6989480 0x5593f69914d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.939+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6988b60 0x5593f6992a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.939+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6988b60 0x5593f6992a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:50634/0 (socket says 172.21.15.133:50634) 2024-09-11T23:46:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.939+0000 7fb58f7fe640 1 -- 172.21.15.133:0/3110143862 learned_addr learned my addr 172.21.15.133:0/3110143862 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.939+0000 7fb5984dd740 1 --2- 172.21.15.133:0/3110143862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6993d40 0x5593f69bc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.939+0000 7fb58effd640 1 --2- 172.21.15.133:0/3110143862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6989480 0x5593f69914d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.939+0000 7fb5984dd740 1 -- 172.21.15.133:0/3110143862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c00e6d0 con 0x5593f6993d40 2024-09-11T23:46:45.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.939+0000 7fb5984dd740 1 -- 172.21.15.133:0/3110143862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c010210 con 0x5593f6988b60 2024-09-11T23:46:45.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.939+0000 7fb5984dd740 1 -- 172.21.15.133:0/3110143862 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c00eda0 con 0x5593f6989480 2024-09-11T23:46:45.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.944+0000 7fb58f7fe640 1 -- 172.21.15.133:0/3110143862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6989480 msgr2=0x5593f69914d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:45.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.944+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/3110143862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6989480 0x5593f69914d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.944+0000 7fb58f7fe640 1 -- 172.21.15.133:0/3110143862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6993d40 msgr2=0x5593f69bc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:45.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.944+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/3110143862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6993d40 0x5593f69bc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.944+0000 7fb58f7fe640 1 -- 172.21.15.133:0/3110143862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593f6986280 con 0x5593f6988b60 2024-09-11T23:46:45.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.944+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/3110143862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6988b60 0x5593f6992a60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb58009a3e0 tx=0x7fb58009db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f1ef669770beaa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:45.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.944+0000 7fb5757fa640 1 -- 172.21.15.133:0/3110143862 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800cebe0 con 0x5593f6988b60 2024-09-11T23:46:45.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.944+0000 7fb5757fa640 1 -- 172.21.15.133:0/3110143862 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb58009bd10 con 0x5593f6988b60 2024-09-11T23:46:45.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.944+0000 7fb5757fa640 1 -- 172.21.15.133:0/3110143862 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800a4240 con 0x5593f6988b60 2024-09-11T23:46:45.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.946+0000 7fb5984dd740 1 -- 172.21.15.133:0/3110143862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6988b60 msgr2=0x5593f6992a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:45.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.946+0000 7fb5984dd740 1 --2- 172.21.15.133:0/3110143862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6988b60 0x5593f6992a60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb58009a3e0 tx=0x7fb58009db90 comp rx=0 tx=0).stop 2024-09-11T23:46:45.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.947+0000 7fb5984dd740 1 -- 172.21.15.133:0/3110143862 shutdown_connections 2024-09-11T23:46:45.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.947+0000 7fb5984dd740 1 --2- 172.21.15.133:0/3110143862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6993d40 0x5593f69bc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.947+0000 7fb5984dd740 1 --2- 172.21.15.133:0/3110143862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6989480 0x5593f69914d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.947+0000 7fb5984dd740 1 --2- 172.21.15.133:0/3110143862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6988b60 0x5593f6992a60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.947+0000 7fb5984dd740 1 -- 172.21.15.133:0/3110143862 >> 172.21.15.133:0/3110143862 conn(0x5593f69850f0 msgr2=0x5593f6992350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:45.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.949+0000 7fb5984dd740 1 -- 172.21.15.133:0/3110143862 shutdown_connections 2024-09-11T23:46:45.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.949+0000 7fb5984dd740 1 -- 172.21.15.133:0/3110143862 wait complete. 2024-09-11T23:46:45.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.950+0000 7fb5984dd740 1 -- start start 2024-09-11T23:46:45.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.951+0000 7fb5984dd740 1 Processor -- start 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.951+0000 7fb5984dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6988b60 0x5593f69ace00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.952+0000 7fb5984dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6989480 0x5593f69ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.952+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6988b60 0x5593f69ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.952+0000 7fb58f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6988b60 0x5593f69ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53332/0 (socket says 172.21.15.133:53332) 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.952+0000 7fb58f7fe640 1 -- 172.21.15.133:0/780313394 learned_addr learned my addr 172.21.15.133:0/780313394 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.952+0000 7fb5984dd740 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6993d40 0x5593f698d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.952+0000 7fb5984dd740 1 -- 172.21.15.133:0/780313394 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c00fb40 con 0x5593f6989480 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.952+0000 7fb5984dd740 1 -- 172.21.15.133:0/780313394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c00f470 con 0x5593f6993d40 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.953+0000 7fb5984dd740 1 -- 172.21.15.133:0/780313394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c00c6d0 con 0x5593f6988b60 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.953+0000 7fb5984dd740 10 client.0 _ll_register_callbacks cb 0x7fb580099cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.953+0000 7fb58ffff640 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6993d40 0x5593f698d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.953+0000 7fb58f7fe640 1 -- 172.21.15.133:0/780313394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6993d40 msgr2=0x5593f698d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:45.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.953+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6993d40 0x5593f698d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.953+0000 7fb58f7fe640 1 -- 172.21.15.133:0/780313394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6989480 msgr2=0x5593f69ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:45.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.953+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6989480 0x5593f69ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.953+0000 7fb58f7fe640 1 -- 172.21.15.133:0/780313394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593f698d820 con 0x5593f6988b60 2024-09-11T23:46:45.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.953+0000 7fb58ffff640 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6993d40 0x5593f698d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:46:45.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.953+0000 7fb58f7fe640 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6988b60 0x5593f69ace00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb58009d710 tx=0x7fb580019c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4d78013ecd7f80f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:45.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.954+0000 7fb56effd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:45.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.954+0000 7fb56effd640 1 -- 172.21.15.133:0/780313394 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800a40f0 con 0x5593f6988b60 2024-09-11T23:46:45.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.954+0000 7fb56effd640 1 -- 172.21.15.133:0/780313394 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fb580019450 con 0x5593f6988b60 2024-09-11T23:46:45.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.954+0000 7fb5984dd740 1 -- 172.21.15.133:0/780313394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb58009bd10 con 0x5593f6988b60 2024-09-11T23:46:45.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.955+0000 7fb56effd640 1 -- 172.21.15.133:0/780313394 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb5800a4290 con 0x5593f6988b60 2024-09-11T23:46:45.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.955+0000 7fb5984dd740 10 client.27265 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:45.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.955+0000 7fb5984dd740 1 -- 172.21.15.133:0/780313394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5593f698d820 con 0x5593f6988b60 2024-09-11T23:46:45.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.960+0000 7fb56effd640 1 -- 172.21.15.133:0/780313394 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5800aa070 con 0x5593f6988b60 2024-09-11T23:46:45.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.960+0000 7fb56effd640 1 -- 172.21.15.133:0/780313394 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb58009ce60 con 0x5593f6988b60 2024-09-11T23:46:45.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.961+0000 7fb5984dd740 10 client.27265 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:45.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.961+0000 7fb5984dd740 10 client.27265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:45.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.961+0000 7fb5984dd740 20 client.27265 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:45.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.961+0000 7fb5984dd740 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5593f6a1f2d0 0x5593f6a216c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:45.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.961+0000 7fb5984dd740 4 client.27265 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:46:45.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.961+0000 7fb5984dd740 1 -- 172.21.15.133:0/780313394 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb58009bd10 con 0x5593f6a1f2d0 2024-09-11T23:46:45.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.961+0000 7fb58effd640 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5593f6a1f2d0 0x5593f6a216c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:45.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.962+0000 7fb58effd640 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5593f6a1f2d0 0x5593f6a216c0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee35f6d478eff7d server_cookie=e0a2848c18d1ad06 in_seq=0 out_seq=0 2024-09-11T23:46:45.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.962+0000 7fb56effd640 10 client.27265 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:45.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.964+0000 7fb56effd640 1 -- 172.21.15.133:0/780313394 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb58009bd10 con 0x5593f6a1f2d0 2024-09-11T23:46:45.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.964+0000 7fb56effd640 10 client.27265 handle_command_reply: tid=0 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.964+0000 7fb5984dd740 1 client.27265 shutdown 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.964+0000 7fb5984dd740 1 -- 172.21.15.133:0/780313394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6988b60 msgr2=0x5593f69ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.965+0000 7fb5984dd740 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6988b60 0x5593f69ace00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb58009d710 tx=0x7fb580019c20 comp rx=0 tx=0).stop 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.965+0000 7fb5984dd740 1 -- 172.21.15.133:0/780313394 shutdown_connections 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.965+0000 7fb5984dd740 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5593f6a1f2d0 0x5593f6a216c0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.965+0000 7fb5984dd740 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593f6993d40 0x5593f698d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.965+0000 7fb56effd640 0 client.27265 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.965+0000 7fb5984dd740 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5593f6989480 0x5593f69ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.965+0000 7fb5984dd740 1 --2- 172.21.15.133:0/780313394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5593f6988b60 0x5593f69ace00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.965+0000 7fb5984dd740 1 -- 172.21.15.133:0/780313394 >> 172.21.15.133:0/780313394 conn(0x5593f69850f0 msgr2=0x5593f6984270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.966+0000 7fb5984dd740 1 -- 172.21.15.133:0/780313394 shutdown_connections 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.966+0000 7fb5984dd740 1 -- 172.21.15.133:0/780313394 wait complete. 2024-09-11T23:46:45.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.966+0000 7fb5984dd740 20 client.27265 trim_cache size 0 max 16384 2024-09-11T23:46:45.968 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:46:45.968 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:46:45.968 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "6fa4e143-ffa3-4877-842b-c09731cc8ae8", 2024-09-11T23:46:45.968 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:46:45.968 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:46:45.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.975+0000 7fb596104640 1 -- 172.21.15.133:0/965168384 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5780a2e10 msgr2=0x7fb5780a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:45.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.975+0000 7fb596104640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5780a2e10 0x7fb5780a52d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb584005e00 tx=0x7fb584009280 comp rx=0 tx=0).stop 2024-09-11T23:46:45.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.975+0000 7fb596104640 1 -- 172.21.15.133:0/965168384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb590109450 msgr2=0x7fb5901a6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:45.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.975+0000 7fb596104640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb590109450 0x7fb5901a6f70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb58000b4a0 tx=0x7fb58000b960 comp rx=0 tx=0).stop 2024-09-11T23:46:45.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.975+0000 7fb596104640 1 -- 172.21.15.133:0/965168384 shutdown_connections 2024-09-11T23:46:45.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.975+0000 7fb596104640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5780a2e10 0x7fb5780a52d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.975+0000 7fb596104640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5901ab840 0x7fb5901abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.975+0000 7fb596104640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb590109830 0x7fb5901a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.975+0000 7fb596104640 1 --2- 172.21.15.133:0/965168384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb590109450 0x7fb5901a6f70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:45.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.975+0000 7fb596104640 1 -- 172.21.15.133:0/965168384 >> 172.21.15.133:0/965168384 conn(0x7fb59006e140 msgr2=0x7fb590119190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:45.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.980+0000 7fb596104640 1 -- 172.21.15.133:0/965168384 shutdown_connections 2024-09-11T23:46:45.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:45.980+0000 7fb596104640 1 -- 172.21.15.133:0/965168384 wait complete. 2024-09-11T23:46:46.685 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:46:46.686 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-11T23:46:46.686 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:46:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.031+0000 7f9a8f7fe640 1 -- 172.21.15.133:0/77875293 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa3fe4d820 con 0x55fa3ffbfed0 2024-09-11T23:46:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.031+0000 7f9a8f7fe640 10 client.27455 handle_command_reply: tid=0 2024-09-11T23:46:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.031+0000 7f9ab940a740 1 client.27455 shutdown 2024-09-11T23:46:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.032+0000 7f9ab940a740 1 -- 172.21.15.133:0/77875293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 msgr2=0x55fa3ff2f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.032+0000 7f9ab940a740 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff2f060 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55fa3ff22c30 tx=0x7f9aa0005220 comp rx=0 tx=0).stop 2024-09-11T23:46:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.035+0000 7f9ab940a740 1 -- 172.21.15.133:0/77875293 shutdown_connections 2024-09-11T23:46:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.035+0000 7f9ab940a740 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55fa3ffbfed0 0x55fa3ffc22c0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.035+0000 7f9ab940a740 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa3ff22cb0 0x55fa3ff2f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.035+0000 7f9ab940a740 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa3ff1fdd0 0x55fa3ff2f060 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.035+0000 7f9ab940a740 1 --2- 172.21.15.133:0/77875293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa3ff18410 0x55fa3ff2eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.035+0000 7f9ab940a740 1 -- 172.21.15.133:0/77875293 >> 172.21.15.133:0/77875293 conn(0x55fa3ff14060 msgr2=0x55fa3ff1f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.035+0000 7f9a8f7fe640 0 client.27455 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.040+0000 7f9ab940a740 1 -- 172.21.15.133:0/77875293 shutdown_connections 2024-09-11T23:46:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.040+0000 7f9ab940a740 1 -- 172.21.15.133:0/77875293 wait complete. 2024-09-11T23:46:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.040+0000 7f9ab940a740 20 client.27455 trim_cache size 0 max 16384 2024-09-11T23:46:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:46:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:46:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:46:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:46:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.044+0000 7f9ab6677640 1 -- 172.21.15.133:0/3145470678 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a9c0a3200 msgr2=0x7f9a9c0a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.044+0000 7f9ab6677640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a9c0a3200 0x7f9a9c0a56c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55fa3ff14ec0 tx=0x7f9aa80b7020 comp rx=0 tx=0).stop 2024-09-11T23:46:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.044+0000 7f9ab6677640 1 -- 172.21.15.133:0/3145470678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab01091f0 msgr2=0x7f9ab01a70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.044+0000 7f9ab6677640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab01091f0 0x7f9ab01a70a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8035040 tx=0x7f9aa8030460 comp rx=0 tx=0).stop 2024-09-11T23:46:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.050+0000 7f9ab6677640 1 -- 172.21.15.133:0/3145470678 shutdown_connections 2024-09-11T23:46:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.050+0000 7f9ab6677640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f9a9c0a3200 0x7f9a9c0a56c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.050+0000 7f9ab6677640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ab011b140 0x7f9ab01ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.050+0000 7f9ab6677640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab01095d0 0x7f9ab01a75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.050+0000 7f9ab6677640 1 --2- 172.21.15.133:0/3145470678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab01091f0 0x7f9ab01a70a0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.050+0000 7f9ab6677640 1 -- 172.21.15.133:0/3145470678 >> 172.21.15.133:0/3145470678 conn(0x7f9ab006ce00 msgr2=0x7f9ab011b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.052+0000 7f9ab6677640 1 -- 172.21.15.133:0/3145470678 shutdown_connections 2024-09-11T23:46:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:46:47.053+0000 7f9ab6677640 1 -- 172.21.15.133:0/3145470678 wait complete. 2024-09-11T23:46:47.061 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:46 smithi133 ceph-mon[31430]: pgmap v311: 97 pgs: 97 active+clean; 70 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 725 MiB/s wr, 2.47k op/s 2024-09-11T23:46:47.063 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:46:47.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:46 smithi138 ceph-mon[35142]: pgmap v311: 97 pgs: 97 active+clean; 70 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 725 MiB/s wr, 2.47k op/s 2024-09-11T23:46:47.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:46 smithi178 ceph-mon[35882]: pgmap v311: 97 pgs: 97 active+clean; 70 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 725 MiB/s wr, 2.47k op/s 2024-09-11T23:46:47.663 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:48.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:47 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:46:48.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:47 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:46:48.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:47 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:46:48.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:47 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:46:48.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:47 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:46:48.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:47 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:46:48.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:47 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:46:48.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:47 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:46:48.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:47 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:46:48.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:47 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:46:48.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:47 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:46:48.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:47 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:46:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.572+0000 7f2b35565640 1 -- 172.21.15.133:0/3395599268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b30100a90 msgr2=0x7f2b30100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.572+0000 7f2b35565640 1 --2- 172.21.15.133:0/3395599268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b30100a90 0x7f2b30100ef0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009890 tx=0x7f2b2402a3b0 comp rx=0 tx=0).stop 2024-09-11T23:46:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.574+0000 7f2b35565640 1 -- 172.21.15.133:0/3395599268 shutdown_connections 2024-09-11T23:46:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.574+0000 7f2b35565640 1 --2- 172.21.15.133:0/3395599268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b30101430 0x7f2b3010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.574+0000 7f2b35565640 1 --2- 172.21.15.133:0/3395599268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b30100a90 0x7f2b30100ef0 secure :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009890 tx=0x7f2b2402a3b0 comp rx=0 tx=0).stop 2024-09-11T23:46:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.574+0000 7f2b35565640 1 --2- 172.21.15.133:0/3395599268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b30106a90 0x7f2b30106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:48.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.574+0000 7f2b35565640 1 -- 172.21.15.133:0/3395599268 >> 172.21.15.133:0/3395599268 conn(0x7f2b300fc7d0 msgr2=0x7f2b300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:48.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.575+0000 7f2b35565640 1 -- 172.21.15.133:0/3395599268 shutdown_connections 2024-09-11T23:46:48.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.575+0000 7f2b35565640 1 -- 172.21.15.133:0/3395599268 wait complete. 2024-09-11T23:46:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.576+0000 7f2b35565640 1 Processor -- start 2024-09-11T23:46:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.577+0000 7f2b35565640 1 -- start start 2024-09-11T23:46:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.577+0000 7f2b35565640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b30101430 0x7f2b30073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.577+0000 7f2b35565640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b30106a90 0x7f2b3006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.577+0000 7f2b35565640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b3006d240 0x7f2b3006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.577+0000 7f2b35565640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3010fd80 con 0x7f2b3006d240 2024-09-11T23:46:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.577+0000 7f2b35565640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3010fc00 con 0x7f2b30101430 2024-09-11T23:46:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.577+0000 7f2b35565640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3010ff00 con 0x7f2b30106a90 2024-09-11T23:46:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.580+0000 7f2b2effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b30101430 0x7f2b30073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.580+0000 7f2b2f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b3006d240 0x7f2b3006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.580+0000 7f2b2f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b3006d240 0x7f2b3006d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46844/0 (socket says 172.21.15.133:46844) 2024-09-11T23:46:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.580+0000 7f2b2f7fe640 1 -- 172.21.15.133:0/1354418886 learned_addr learned my addr 172.21.15.133:0/1354418886 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.581+0000 7f2b2e7fc640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b30106a90 0x7f2b3006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.584+0000 7f2b2f7fe640 1 -- 172.21.15.133:0/1354418886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b30106a90 msgr2=0x7f2b3006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.584+0000 7f2b2f7fe640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b30106a90 0x7f2b3006cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.584+0000 7f2b2f7fe640 1 -- 172.21.15.133:0/1354418886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b30101430 msgr2=0x7f2b30073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.584+0000 7f2b2f7fe640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b30101430 0x7f2b30073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.584+0000 7f2b2f7fe640 1 -- 172.21.15.133:0/1354418886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b30071880 con 0x7f2b3006d240 2024-09-11T23:46:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.584+0000 7f2b2f7fe640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b3006d240 0x7f2b3006d6f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009e20 tx=0x7f2b2800e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:48.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.588+0000 7f2b17fff640 1 -- 172.21.15.133:0/1354418886 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28004b20 con 0x7f2b3006d240 2024-09-11T23:46:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.588+0000 7f2b17fff640 1 -- 172.21.15.133:0/1354418886 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b28004cc0 con 0x7f2b3006d240 2024-09-11T23:46:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.588+0000 7f2b17fff640 1 -- 172.21.15.133:0/1354418886 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28005830 con 0x7f2b3006d240 2024-09-11T23:46:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.589+0000 7f2b35565640 1 -- 172.21.15.133:0/1354418886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b30071b70 con 0x7f2b3006d240 2024-09-11T23:46:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.589+0000 7f2b35565640 1 -- 172.21.15.133:0/1354418886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b30072030 con 0x7f2b3006d240 2024-09-11T23:46:48.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.590+0000 7f2b35565640 1 -- 172.21.15.133:0/1354418886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b301021d0 con 0x7f2b3006d240 2024-09-11T23:46:48.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.596+0000 7f2b17fff640 1 -- 172.21.15.133:0/1354418886 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2b2801b020 con 0x7f2b3006d240 2024-09-11T23:46:48.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.598+0000 7f2b17fff640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2b180a2ec0 0x7f2b180a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.600+0000 7f2b17fff640 1 -- 172.21.15.133:0/1354418886 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f2b280cf930 con 0x7f2b3006d240 2024-09-11T23:46:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.602+0000 7f2b2effd640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2b180a2ec0 0x7f2b180a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:48.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.603+0000 7f2b2effd640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2b180a2ec0 0x7f2b180a5380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2b20005fd0 tx=0x7f2b20009260 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:48.612+0000 7f2b17fff640 1 -- 172.21.15.133:0/1354418886 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b28022020 con 0x7f2b3006d240 2024-09-11T23:46:48.740 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:49.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.129+0000 7f2b35565640 1 -- 172.21.15.133:0/1354418886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b30100ef0 con 0x7f2b3006d240 2024-09-11T23:46:49.133 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:46:49.133 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:46:49.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.132+0000 7f2b17fff640 1 -- 172.21.15.133:0/1354418886 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7f2b2809c640 con 0x7f2b3006d240 2024-09-11T23:46:49.133 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:46:49.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.137+0000 7f2b35565640 1 -- 172.21.15.133:0/1354418886 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2b180a2ec0 msgr2=0x7f2b180a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:49.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.137+0000 7f2b35565640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2b180a2ec0 0x7f2b180a5380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2b20005fd0 tx=0x7f2b20009260 comp rx=0 tx=0).stop 2024-09-11T23:46:49.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.137+0000 7f2b35565640 1 -- 172.21.15.133:0/1354418886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b3006d240 msgr2=0x7f2b3006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:49.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.137+0000 7f2b35565640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b3006d240 0x7f2b3006d6f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009e20 tx=0x7f2b2800e490 comp rx=0 tx=0).stop 2024-09-11T23:46:49.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.138+0000 7f2b35565640 1 -- 172.21.15.133:0/1354418886 shutdown_connections 2024-09-11T23:46:49.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.138+0000 7f2b35565640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2b180a2ec0 0x7f2b180a5380 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.138+0000 7f2b35565640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b3006d240 0x7f2b3006d6f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.138+0000 7f2b35565640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b30106a90 0x7f2b3006cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.138+0000 7f2b35565640 1 --2- 172.21.15.133:0/1354418886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b30101430 0x7f2b30073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.138+0000 7f2b35565640 1 -- 172.21.15.133:0/1354418886 >> 172.21.15.133:0/1354418886 conn(0x7f2b300fc7d0 msgr2=0x7f2b300fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:49.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.140+0000 7f2b35565640 1 -- 172.21.15.133:0/1354418886 shutdown_connections 2024-09-11T23:46:49.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.140+0000 7f2b35565640 1 -- 172.21.15.133:0/1354418886 wait complete. 2024-09-11T23:46:49.166 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:48 smithi133 ceph-mon[31430]: pgmap v312: 97 pgs: 97 active+clean; 70 GiB data, 284 GiB used, 789 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 614 MiB/s wr, 2.15k op/s 2024-09-11T23:46:49.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:48 smithi138 ceph-mon[35142]: pgmap v312: 97 pgs: 97 active+clean; 70 GiB data, 284 GiB used, 789 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 614 MiB/s wr, 2.15k op/s 2024-09-11T23:46:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:48 smithi178 ceph-mon[35882]: pgmap v312: 97 pgs: 97 active+clean; 70 GiB data, 284 GiB used, 789 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 614 MiB/s wr, 2.15k op/s 2024-09-11T23:46:49.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.748+0000 7f94ede5a640 1 -- 172.21.15.133:0/2705462124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94e8100de0 msgr2=0x7f94e8104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.748+0000 7f94ede5a640 1 --2- 172.21.15.133:0/2705462124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94e8100de0 0x7f94e8104e30 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009900 tx=0x7f94d002a410 comp rx=0 tx=0).stop 2024-09-11T23:46:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.748+0000 7f94ede5a640 1 -- 172.21.15.133:0/2705462124 shutdown_connections 2024-09-11T23:46:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.748+0000 7f94ede5a640 1 --2- 172.21.15.133:0/2705462124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e8105560 0x7f94e810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.748+0000 7f94ede5a640 1 --2- 172.21.15.133:0/2705462124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94e8100de0 0x7f94e8104e30 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.748+0000 7f94ede5a640 1 --2- 172.21.15.133:0/2705462124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e8100430 0x7f94e8100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.748+0000 7f94ede5a640 1 -- 172.21.15.133:0/2705462124 >> 172.21.15.133:0/2705462124 conn(0x7f94e8077230 msgr2=0x7f94e80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.748+0000 7f94ede5a640 1 -- 172.21.15.133:0/2705462124 shutdown_connections 2024-09-11T23:46:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.748+0000 7f94ede5a640 1 -- 172.21.15.133:0/2705462124 wait complete. 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.749+0000 7f94ede5a640 1 Processor -- start 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.749+0000 7f94ede5a640 1 -- start start 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.750+0000 7f94ede5a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94e8100430 0x7f94e80fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.750+0000 7f94ede5a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e8100de0 0x7f94e80fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.750+0000 7f94ede5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e8105560 0x7f94e80fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.750+0000 7f94ede5a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94e810fd60 con 0x7f94e8100430 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.750+0000 7f94ede5a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94e810fbe0 con 0x7f94e8100de0 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.750+0000 7f94ede5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94e810fee0 con 0x7f94e8105560 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.750+0000 7f94dffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e8100de0 0x7f94e80fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.750+0000 7f94dffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e8100de0 0x7f94e80fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:50674/0 (socket says 172.21.15.133:50674) 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.750+0000 7f94dffff640 1 -- 172.21.15.133:0/856990358 learned_addr learned my addr 172.21.15.133:0/856990358 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:49.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.750+0000 7f94e77fe640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94e8100430 0x7f94e80fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.751+0000 7f94dffff640 1 -- 172.21.15.133:0/856990358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e8105560 msgr2=0x7f94e80fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:49.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.751+0000 7f94dffff640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e8105560 0x7f94e80fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.751+0000 7f94dffff640 1 -- 172.21.15.133:0/856990358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94e8100430 msgr2=0x7f94e80fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:49.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.751+0000 7f94dffff640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94e8100430 0x7f94e80fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.751+0000 7f94dffff640 1 -- 172.21.15.133:0/856990358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e81aae80 con 0x7f94e8100de0 2024-09-11T23:46:49.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.751+0000 7f94dffff640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e8100de0 0x7f94e80fcc40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f94d0006010 tx=0x7f94d00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:49.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.752+0000 7f94e57fa640 1 -- 172.21.15.133:0/856990358 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d003f030 con 0x7f94e8100de0 2024-09-11T23:46:49.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.752+0000 7f94e57fa640 1 -- 172.21.15.133:0/856990358 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d0003ce0 con 0x7f94e8100de0 2024-09-11T23:46:49.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.752+0000 7f94e57fa640 1 -- 172.21.15.133:0/856990358 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0035070 con 0x7f94e8100de0 2024-09-11T23:46:49.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.754+0000 7f94ede5a640 1 -- 172.21.15.133:0/856990358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e81ab050 con 0x7f94e8100de0 2024-09-11T23:46:49.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.754+0000 7f94ede5a640 1 -- 172.21.15.133:0/856990358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e81ab450 con 0x7f94e8100de0 2024-09-11T23:46:49.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.758+0000 7f94e57fa640 1 -- 172.21.15.133:0/856990358 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f94d0042030 con 0x7f94e8100de0 2024-09-11T23:46:49.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.760+0000 7f94e57fa640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94d40a2df0 0x7f94d40a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.760+0000 7f94e57fa640 1 -- 172.21.15.133:0/856990358 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f94d00efb00 con 0x7f94e8100de0 2024-09-11T23:46:49.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.761+0000 7f94e77fe640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94d40a2df0 0x7f94d40a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.763+0000 7f94f0233740 1 Processor -- start 2024-09-11T23:46:49.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.763+0000 7f94f0233740 1 -- start start 2024-09-11T23:46:49.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.764+0000 7f94f0233740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a66447a4a0 0x55a66447a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.764+0000 7f94f0233740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 0x55a664485140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.764+0000 7f94f0233740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a664485680 0x55a664487a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.764+0000 7f94f0233740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a6643e02e0 con 0x55a66447a4a0 2024-09-11T23:46:49.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.764+0000 7f94f0233740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a664387ce0 con 0x55a66447adc0 2024-09-11T23:46:49.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.764+0000 7f94f0233740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a6643dc2a0 con 0x55a664485680 2024-09-11T23:46:49.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.765+0000 7f94e7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a664485680 0x55a664487a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.765+0000 7f94e7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a664485680 0x55a664487a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53392/0 (socket says 172.21.15.133:53392) 2024-09-11T23:46:49.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.765+0000 7f94e7fff640 1 -- 172.21.15.133:0/1402201331 learned_addr learned my addr 172.21.15.133:0/1402201331 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:49.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.765+0000 7f94dffff640 1 --2- 172.21.15.133:0/1402201331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 0x55a664485140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.766+0000 7f94e7fff640 1 -- 172.21.15.133:0/1402201331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 msgr2=0x55a664485140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:49.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.766+0000 7f94e7fff640 1 --2- 172.21.15.133:0/1402201331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 0x55a664485140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.766+0000 7f94e7fff640 1 -- 172.21.15.133:0/1402201331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a66447a4a0 msgr2=0x55a66447a880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:46:49.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.766+0000 7f94e7fff640 1 --2- 172.21.15.133:0/1402201331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a66447a4a0 0x55a66447a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.766+0000 7f94e7fff640 1 -- 172.21.15.133:0/1402201331 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a66436c440 con 0x55a664485680 2024-09-11T23:46:49.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.767+0000 7f94e7fff640 1 --2- 172.21.15.133:0/1402201331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a664485680 0x55a664487a70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f94e000af90 tx=0x7f94e000e880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=744aa91fd22202c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:49.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.768+0000 7f94ddffb640 1 -- 172.21.15.133:0/1402201331 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0004b20 con 0x55a664485680 2024-09-11T23:46:49.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.768+0000 7f94ddffb640 1 -- 172.21.15.133:0/1402201331 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e0004cc0 con 0x55a664485680 2024-09-11T23:46:49.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.770+0000 7f94e77fe640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94d40a2df0 0x7f94d40a52b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009990 tx=0x7f94d8009110 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:49.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.770+0000 7f94f0233740 1 -- 172.21.15.133:0/1402201331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a664485680 msgr2=0x55a664487a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:49.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.771+0000 7f94f0233740 1 --2- 172.21.15.133:0/1402201331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a664485680 0x55a664487a70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f94e000af90 tx=0x7f94e000e880 comp rx=0 tx=0).stop 2024-09-11T23:46:49.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.771+0000 7f94f0233740 1 -- 172.21.15.133:0/1402201331 shutdown_connections 2024-09-11T23:46:49.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.771+0000 7f94f0233740 1 --2- 172.21.15.133:0/1402201331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a664485680 0x55a664487a70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.771+0000 7f94f0233740 1 --2- 172.21.15.133:0/1402201331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 0x55a664485140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.771+0000 7f94f0233740 1 --2- 172.21.15.133:0/1402201331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a66447a4a0 0x55a66447a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.771+0000 7f94f0233740 1 -- 172.21.15.133:0/1402201331 >> 172.21.15.133:0/1402201331 conn(0x55a664476a30 msgr2=0x55a664476e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:49.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.771+0000 7f94f0233740 1 -- 172.21.15.133:0/1402201331 shutdown_connections 2024-09-11T23:46:49.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.772+0000 7f94f0233740 1 -- 172.21.15.133:0/1402201331 wait complete. 2024-09-11T23:46:49.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.772+0000 7f94f0233740 1 -- start start 2024-09-11T23:46:49.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.773+0000 7f94f0233740 1 Processor -- start 2024-09-11T23:46:49.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.774+0000 7f94f0233740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447a4a0 0x55a664484f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.774+0000 7f94f0233740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 0x55a664482200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.775+0000 7f94f0233740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 0x55a664482740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.775+0000 7f94e77fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447a4a0 0x55a664484f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.775+0000 7f94e77fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447a4a0 0x55a664484f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:53398/0 (socket says 172.21.15.133:53398) 2024-09-11T23:46:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.775+0000 7f94dffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 0x55a664482200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.775+0000 7f94dffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 0x55a664482200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:50684/0 (socket says 172.21.15.133:50684) 2024-09-11T23:46:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.775+0000 7f94e77fe640 1 -- 172.21.15.133:0/3098041620 learned_addr learned my addr 172.21.15.133:0/3098041620 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.775+0000 7f94f0233740 1 -- 172.21.15.133:0/3098041620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a6644992b0 con 0x55a664485680 2024-09-11T23:46:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.775+0000 7f94e7fff640 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 0x55a664482740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.775+0000 7f94f0233740 1 -- 172.21.15.133:0/3098041620 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a66439b050 con 0x55a66447adc0 2024-09-11T23:46:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.775+0000 7f94f0233740 1 -- 172.21.15.133:0/3098041620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a664499430 con 0x55a66447a4a0 2024-09-11T23:46:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.775+0000 7f94f0233740 10 client.0 _ll_register_callbacks cb 0x55a6643db160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.779+0000 7f94dffff640 1 -- 172.21.15.133:0/3098041620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447a4a0 msgr2=0x55a664484f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.779+0000 7f94dffff640 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447a4a0 0x55a664484f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.779+0000 7f94dffff640 1 -- 172.21.15.133:0/3098041620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 msgr2=0x55a664482740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.779+0000 7f94dffff640 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 0x55a664482740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.779+0000 7f94dffff640 1 -- 172.21.15.133:0/3098041620 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6641513c0 con 0x55a66447adc0 2024-09-11T23:46:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.779+0000 7f94e7fff640 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 0x55a664482740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:46:49.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.779+0000 7f94e77fe640 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447a4a0 0x55a664484f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:46:49.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.780+0000 7f94dffff640 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 0x55a664482200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f94d00ed230 tx=0x7f94d00bbf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1550f07a99233 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:49.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.780+0000 7f94c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:46:49.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.781+0000 7f94c6ffd640 1 -- 172.21.15.133:0/3098041620 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d00bcb00 con 0x55a66447adc0 2024-09-11T23:46:49.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.781+0000 7f94c6ffd640 1 -- 172.21.15.133:0/3098041620 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f94d00d5260 con 0x55a66447adc0 2024-09-11T23:46:49.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.781+0000 7f94f0233740 1 -- 172.21.15.133:0/3098041620 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94e0004cc0 con 0x55a66447adc0 2024-09-11T23:46:49.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.781+0000 7f94c6ffd640 1 -- 172.21.15.133:0/3098041620 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94d00bc400 con 0x55a66447adc0 2024-09-11T23:46:49.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.781+0000 7f94f0233740 10 client.27482 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:49.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.781+0000 7f94f0233740 1 -- 172.21.15.133:0/3098041620 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6644992b0 con 0x55a66447adc0 2024-09-11T23:46:49.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.782+0000 7f94c6ffd640 1 -- 172.21.15.133:0/3098041620 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d00c07b0 con 0x55a66447adc0 2024-09-11T23:46:49.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.782+0000 7f94c6ffd640 1 -- 172.21.15.133:0/3098041620 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f94d009e070 con 0x55a66447adc0 2024-09-11T23:46:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.783+0000 7f94f0233740 10 client.27482 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.783+0000 7f94f0233740 10 client.27482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.783+0000 7f94f0233740 20 client.27482 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.783+0000 7f94f0233740 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6644a6160 0x55a6644a8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.783+0000 7f94f0233740 4 client.27482 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.783+0000 7f94f0233740 1 -- 172.21.15.133:0/3098041620 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f94e0004cc0 con 0x55a6644a6160 2024-09-11T23:46:49.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.784+0000 7f94e77fe640 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6644a6160 0x55a6644a8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.785+0000 7f94c6ffd640 10 client.27482 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:49.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.785+0000 7f94e77fe640 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6644a6160 0x55a6644a8590 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ccb5c27831ca02 server_cookie=d6cf55cab0d91a83 in_seq=0 out_seq=0 2024-09-11T23:46:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.787+0000 7f94c6ffd640 1 -- 172.21.15.133:0/3098041620 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f94e0004cc0 con 0x55a6644a6160 2024-09-11T23:46:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.787+0000 7f94c6ffd640 10 client.27482 handle_command_reply: tid=0 2024-09-11T23:46:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.787+0000 7f94f0233740 1 client.27482 shutdown 2024-09-11T23:46:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.788+0000 7f94f0233740 1 -- 172.21.15.133:0/3098041620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 msgr2=0x55a664482200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.788+0000 7f94f0233740 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 0x55a664482200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f94d00ed230 tx=0x7f94d00bbf40 comp rx=0 tx=0).stop 2024-09-11T23:46:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.788+0000 7f94f0233740 1 -- 172.21.15.133:0/3098041620 shutdown_connections 2024-09-11T23:46:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.788+0000 7f94f0233740 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6644a6160 0x55a6644a8590 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.788+0000 7f94f0233740 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 0x55a664482740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.788+0000 7f94c6ffd640 0 client.27482 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.788+0000 7f94f0233740 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447adc0 0x55a664482200 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.788+0000 7f94f0233740 1 --2- 172.21.15.133:0/3098041620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447a4a0 0x55a664484f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.788+0000 7f94f0233740 1 -- 172.21.15.133:0/3098041620 >> 172.21.15.133:0/3098041620 conn(0x55a664476a30 msgr2=0x55a66447c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.788+0000 7f94f0233740 1 -- 172.21.15.133:0/3098041620 shutdown_connections 2024-09-11T23:46:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.788+0000 7f94f0233740 1 -- 172.21.15.133:0/3098041620 wait complete. 2024-09-11T23:46:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.789+0000 7f94f0233740 20 client.27482 trim_cache size 0 max 16384 2024-09-11T23:46:49.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.812+0000 7f94f0233740 1 Processor -- start 2024-09-11T23:46:49.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.812+0000 7f94f0233740 1 -- start start 2024-09-11T23:46:49.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94f0233740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447a4a0 0x55a6644a2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94f0233740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a66447adc0 0x55a6644adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94f0233740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a664485680 0x55a6644ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94f0233740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81a4cd0 con 0x55a66447adc0 2024-09-11T23:46:49.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94f0233740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81ad0d0 con 0x55a664485680 2024-09-11T23:46:49.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94f0233740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81abc60 con 0x55a66447a4a0 2024-09-11T23:46:49.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94dffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a66447adc0 0x55a6644adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94dffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a66447adc0 0x55a6644adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46880/0 (socket says 172.21.15.133:46880) 2024-09-11T23:46:49.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94dffff640 1 -- 172.21.15.133:0/1927730525 learned_addr learned my addr 172.21.15.133:0/1927730525 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:49.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94e7fff640 1 --2- 172.21.15.133:0/1927730525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a664485680 0x55a6644ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94dffff640 1 -- 172.21.15.133:0/1927730525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447a4a0 msgr2=0x55a6644a2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:49.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94dffff640 1 --2- 172.21.15.133:0/1927730525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447a4a0 0x55a6644a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94dffff640 1 -- 172.21.15.133:0/1927730525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a664485680 msgr2=0x55a6644ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:49.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.813+0000 7f94dffff640 1 --2- 172.21.15.133:0/1927730525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a664485680 0x55a6644ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.814+0000 7f94dffff640 1 -- 172.21.15.133:0/1927730525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6644764d0 con 0x55a66447adc0 2024-09-11T23:46:49.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.814+0000 7f94dffff640 1 --2- 172.21.15.133:0/1927730525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a66447adc0 0x55a6644adde0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f94d00bc100 tx=0x7f94d0031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a8a24a2dcb7151 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:49.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.814+0000 7f94dd7fa640 1 -- 172.21.15.133:0/1927730525 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d00bcb00 con 0x55a66447adc0 2024-09-11T23:46:49.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.814+0000 7f94dd7fa640 1 -- 172.21.15.133:0/1927730525 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d00c07b0 con 0x55a66447adc0 2024-09-11T23:46:49.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.814+0000 7f94dd7fa640 1 -- 172.21.15.133:0/1927730525 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d00d5240 con 0x55a66447adc0 2024-09-11T23:46:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.815+0000 7f94f0233740 1 -- 172.21.15.133:0/1927730525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a66447adc0 msgr2=0x55a6644adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.815+0000 7f94f0233740 1 --2- 172.21.15.133:0/1927730525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a66447adc0 0x55a6644adde0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f94d00bc100 tx=0x7f94d0031a60 comp rx=0 tx=0).stop 2024-09-11T23:46:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.816+0000 7f94f0233740 1 -- 172.21.15.133:0/1927730525 shutdown_connections 2024-09-11T23:46:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.816+0000 7f94f0233740 1 --2- 172.21.15.133:0/1927730525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a664485680 0x55a6644ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.816+0000 7f94f0233740 1 --2- 172.21.15.133:0/1927730525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a66447adc0 0x55a6644adde0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.816+0000 7f94f0233740 1 --2- 172.21.15.133:0/1927730525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447a4a0 0x55a6644a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.816+0000 7f94f0233740 1 -- 172.21.15.133:0/1927730525 >> 172.21.15.133:0/1927730525 conn(0x55a664476a30 msgr2=0x55a6644812f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.816+0000 7f94f0233740 1 -- 172.21.15.133:0/1927730525 shutdown_connections 2024-09-11T23:46:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.816+0000 7f94f0233740 1 -- 172.21.15.133:0/1927730525 wait complete. 2024-09-11T23:46:49.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.816+0000 7f94f0233740 1 -- start start 2024-09-11T23:46:49.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.817+0000 7f94f0233740 1 Processor -- start 2024-09-11T23:46:49.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.817+0000 7f94f0233740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447a4a0 0x55a664499c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.817+0000 7f94f0233740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447adc0 0x55a66449a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.818+0000 7f94f0233740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 0x55a66450eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.818+0000 7f94e7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 0x55a66450eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.819+0000 7f94e77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447a4a0 0x55a664499c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.819+0000 7f94e77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447a4a0 0x55a664499c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:50706/0 (socket says 172.21.15.133:50706) 2024-09-11T23:46:49.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.820+0000 7f94e7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 0x55a66450eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46884/0 (socket says 172.21.15.133:46884) 2024-09-11T23:46:49.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.821+0000 7f94f0233740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81aca00 con 0x55a664485680 2024-09-11T23:46:49.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.821+0000 7f94f0233740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81ac330 con 0x55a66447a4a0 2024-09-11T23:46:49.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.821+0000 7f94f0233740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81a2cd0 con 0x55a66447adc0 2024-09-11T23:46:49.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.821+0000 7f94f0233740 10 client.0 _ll_register_callbacks cb 0x7f94d0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:49.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.822+0000 7f94e77fe640 1 -- 172.21.15.133:0/421087347 learned_addr learned my addr 172.21.15.133:0/421087347 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:49.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.822+0000 7f94e7fff640 1 -- 172.21.15.133:0/421087347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447adc0 msgr2=0x55a66449a170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:46:49.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.822+0000 7f94e7fff640 1 --2- 172.21.15.133:0/421087347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447adc0 0x55a66449a170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.822+0000 7f94e7fff640 1 -- 172.21.15.133:0/421087347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447a4a0 msgr2=0x55a664499c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:49.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.822+0000 7f94e7fff640 1 --2- 172.21.15.133:0/421087347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447a4a0 0x55a664499c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.822+0000 7f94e7fff640 1 -- 172.21.15.133:0/421087347 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a66450f390 con 0x55a664485680 2024-09-11T23:46:49.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.823+0000 7f94e7fff640 1 --2- 172.21.15.133:0/421087347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 0x55a66450eda0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f94e0018040 tx=0x7f94e0003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e3cf0092cae3f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:49.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.824+0000 7f94c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:46:49.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.824+0000 7f94c6ffd640 1 -- 172.21.15.133:0/421087347 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e001a040 con 0x55a664485680 2024-09-11T23:46:49.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.824+0000 7f94c6ffd640 1 -- 172.21.15.133:0/421087347 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f94e000ab80 con 0x55a664485680 2024-09-11T23:46:49.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.825+0000 7f94f0233740 1 -- 172.21.15.133:0/421087347 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94d00c07b0 con 0x55a664485680 2024-09-11T23:46:49.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.826+0000 7f94c6ffd640 1 -- 172.21.15.133:0/421087347 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94e0016410 con 0x55a664485680 2024-09-11T23:46:49.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.826+0000 7f94f0233740 10 client.17943 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:49.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.826+0000 7f94f0233740 1 -- 172.21.15.133:0/421087347 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a664510660 con 0x55a664485680 2024-09-11T23:46:49.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.826+0000 7f94c6ffd640 1 -- 172.21.15.133:0/421087347 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e0004520 con 0x55a664485680 2024-09-11T23:46:49.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.827+0000 7f94c6ffd640 1 -- 172.21.15.133:0/421087347 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f94e000b040 con 0x55a664485680 2024-09-11T23:46:49.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.827+0000 7f94f0233740 10 client.17943 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:49.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.827+0000 7f94f0233740 10 client.17943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:46:49.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.827+0000 7f94f0233740 20 client.17943 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:49.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.827+0000 7f94f0233740 1 --2- 172.21.15.133:0/421087347 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6644a6160 0x55a664512b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:49.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.827+0000 7f94f0233740 4 client.17943 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:46:49.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.827+0000 7f94f0233740 1 -- 172.21.15.133:0/421087347 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f94d00c07b0 con 0x55a6644a6160 2024-09-11T23:46:49.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.828+0000 7f94e77fe640 1 --2- 172.21.15.133:0/421087347 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6644a6160 0x55a664512b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:49.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.829+0000 7f94e77fe640 1 --2- 172.21.15.133:0/421087347 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6644a6160 0x55a664512b60 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c95d57d4f914a87 server_cookie=1d673c2bd3a24888 in_seq=0 out_seq=0 2024-09-11T23:46:49.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.829+0000 7f94c6ffd640 10 client.17943 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:49.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94c6ffd640 1 -- 172.21.15.133:0/421087347 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f94d00c07b0 con 0x55a6644a6160 2024-09-11T23:46:49.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94c6ffd640 10 client.17943 handle_command_reply: tid=0 2024-09-11T23:46:49.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94f0233740 1 client.17943 shutdown 2024-09-11T23:46:49.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94f0233740 1 -- 172.21.15.133:0/421087347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 msgr2=0x55a66450eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:49.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94f0233740 1 --2- 172.21.15.133:0/421087347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 0x55a66450eda0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f94e0018040 tx=0x7f94e0003e20 comp rx=0 tx=0).stop 2024-09-11T23:46:49.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94f0233740 1 -- 172.21.15.133:0/421087347 shutdown_connections 2024-09-11T23:46:49.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94f0233740 1 --2- 172.21.15.133:0/421087347 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55a6644a6160 0x55a664512b60 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94f0233740 1 --2- 172.21.15.133:0/421087347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a664485680 0x55a66450eda0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94f0233740 1 --2- 172.21.15.133:0/421087347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66447adc0 0x55a66449a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94f0233740 1 --2- 172.21.15.133:0/421087347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a66447a4a0 0x55a664499c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94f0233740 1 -- 172.21.15.133:0/421087347 >> 172.21.15.133:0/421087347 conn(0x55a664476a30 msgr2=0x55a66447bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:49.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.830+0000 7f94c6ffd640 0 client.17943 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:49.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.831+0000 7f94f0233740 1 -- 172.21.15.133:0/421087347 shutdown_connections 2024-09-11T23:46:49.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.831+0000 7f94f0233740 1 -- 172.21.15.133:0/421087347 wait complete. 2024-09-11T23:46:49.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.831+0000 7f94f0233740 20 client.17943 trim_cache size 0 max 16384 2024-09-11T23:46:49.832 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:46:49.832 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:46:49.832 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:46:49.832 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:46:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.833+0000 7f94ede5a640 1 -- 172.21.15.133:0/856990358 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94d40a2df0 msgr2=0x7f94d40a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.833+0000 7f94ede5a640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94d40a2df0 0x7f94d40a52b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009990 tx=0x7f94d8009110 comp rx=0 tx=0).stop 2024-09-11T23:46:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.833+0000 7f94ede5a640 1 -- 172.21.15.133:0/856990358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e8100de0 msgr2=0x7f94e80fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.833+0000 7f94ede5a640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e8100de0 0x7f94e80fcc40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f94d0006010 tx=0x7f94d00042f0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.833+0000 7f94ede5a640 1 -- 172.21.15.133:0/856990358 shutdown_connections 2024-09-11T23:46:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.833+0000 7f94ede5a640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f94d40a2df0 0x7f94d40a52b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.833+0000 7f94ede5a640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e8105560 0x7f94e80fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.833+0000 7f94ede5a640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e8100de0 0x7f94e80fcc40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.833+0000 7f94ede5a640 1 --2- 172.21.15.133:0/856990358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94e8100430 0x7f94e80fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:49.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.833+0000 7f94ede5a640 1 -- 172.21.15.133:0/856990358 >> 172.21.15.133:0/856990358 conn(0x7f94e8077230 msgr2=0x7f94e8103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:49.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.834+0000 7f94ede5a640 1 -- 172.21.15.133:0/856990358 shutdown_connections 2024-09-11T23:46:49.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:49.834+0000 7f94ede5a640 1 -- 172.21.15.133:0/856990358 wait complete. 2024-09-11T23:46:50.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:50 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1354418886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:46:50.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:50 smithi178 ceph-mon[35882]: pgmap v313: 97 pgs: 97 active+clean; 70 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 893 MiB/s wr, 2.61k op/s 2024-09-11T23:46:50.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:50 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:46:50.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:50 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1354418886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:46:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:50 smithi138 ceph-mon[35142]: pgmap v313: 97 pgs: 97 active+clean; 70 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 893 MiB/s wr, 2.61k op/s 2024-09-11T23:46:50.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:50 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:46:50.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:50 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1354418886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:46:50.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:50 smithi133 ceph-mon[31430]: pgmap v313: 97 pgs: 97 active+clean; 70 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 893 MiB/s wr, 2.61k op/s 2024-09-11T23:46:50.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:50 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:46:50.631 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-11T23:46:50.631 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-11T23:46:51.432 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:46:51.432 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:46:51.714 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:52.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.542+0000 7fed60e31640 1 -- 172.21.15.133:0/3858447673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed5c106f10 msgr2=0x7fed5c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:52.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.542+0000 7fed60e31640 1 --2- 172.21.15.133:0/3858447673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed5c106f10 0x7fed5c10ea90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fed54009ca0 tx=0x7fed5402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.542+0000 7fed60e31640 1 -- 172.21.15.133:0/3858447673 shutdown_connections 2024-09-11T23:46:52.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.542+0000 7fed60e31640 1 --2- 172.21.15.133:0/3858447673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed5c106f10 0x7fed5c10ea90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.542+0000 7fed60e31640 1 --2- 172.21.15.133:0/3858447673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c102790 0x7fed5c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.542+0000 7fed60e31640 1 --2- 172.21.15.133:0/3858447673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed5c101de0 0x7fed5c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.542+0000 7fed60e31640 1 -- 172.21.15.133:0/3858447673 >> 172.21.15.133:0/3858447673 conn(0x7fed5c0fd880 msgr2=0x7fed5c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:52.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.542+0000 7fed60e31640 1 -- 172.21.15.133:0/3858447673 shutdown_connections 2024-09-11T23:46:52.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.542+0000 7fed60e31640 1 -- 172.21.15.133:0/3858447673 wait complete. 2024-09-11T23:46:52.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.543+0000 7fed60e31640 1 Processor -- start 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.543+0000 7fed60e31640 1 -- start start 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.543+0000 7fed60e31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c101de0 0x7fed5c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.543+0000 7fed60e31640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed5c102790 0x7fed5c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.543+0000 7fed60e31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed5c106f10 0x7fed5c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed60e31640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c111f70 con 0x7fed5c101de0 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed60e31640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c111df0 con 0x7fed5c106f10 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed60e31640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1120f0 con 0x7fed5c102790 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed5a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c101de0 0x7fed5c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed59d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed5c102790 0x7fed5c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed5a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c101de0 0x7fed5c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37968/0 (socket says 172.21.15.133:37968) 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed5a59c640 1 -- 172.21.15.133:0/3118391750 learned_addr learned my addr 172.21.15.133:0/3118391750 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:52.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed59d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed5c102790 0x7fed5c198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33788/0 (socket says 172.21.15.133:33788) 2024-09-11T23:46:52.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed5ad9d640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed5c106f10 0x7fed5c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed5a59c640 1 -- 172.21.15.133:0/3118391750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed5c102790 msgr2=0x7fed5c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:52.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed5a59c640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed5c102790 0x7fed5c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed5a59c640 1 -- 172.21.15.133:0/3118391750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed5c106f10 msgr2=0x7fed5c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:52.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed5a59c640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed5c106f10 0x7fed5c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed5a59c640 1 -- 172.21.15.133:0/3118391750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c19d370 con 0x7fed5c101de0 2024-09-11T23:46:52.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.544+0000 7fed5ad9d640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed5c106f10 0x7fed5c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:46:52.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.545+0000 7fed5a59c640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c101de0 0x7fed5c1983c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00c890 tx=0x7fed4c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:52.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.545+0000 7fed4b7fe640 1 -- 172.21.15.133:0/3118391750 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c004220 con 0x7fed5c101de0 2024-09-11T23:46:52.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.545+0000 7fed4b7fe640 1 -- 172.21.15.133:0/3118391750 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed4c0043c0 con 0x7fed5c101de0 2024-09-11T23:46:52.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.545+0000 7fed60e31640 1 -- 172.21.15.133:0/3118391750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c19d660 con 0x7fed5c101de0 2024-09-11T23:46:52.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.545+0000 7fed4b7fe640 1 -- 172.21.15.133:0/3118391750 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0049e0 con 0x7fed5c101de0 2024-09-11T23:46:52.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.545+0000 7fed60e31640 1 -- 172.21.15.133:0/3118391750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c1a4f40 con 0x7fed5c101de0 2024-09-11T23:46:52.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.549+0000 7fed6320a740 1 Processor -- start 2024-09-11T23:46:52.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.550+0000 7fed6320a740 1 -- start start 2024-09-11T23:46:52.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.550+0000 7fed6320a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e646710 0x55cf5e646af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.550+0000 7fed6320a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e647030 0x55cf5e65a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.550+0000 7fed6320a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e6514f0 0x55cf5e65c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.550+0000 7fed4b7fe640 1 -- 172.21.15.133:0/3118391750 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fed4c0079b0 con 0x7fed5c101de0 2024-09-11T23:46:52.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed59d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e6514f0 0x55cf5e65c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed59d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e6514f0 0x55cf5e65c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33794/0 (socket says 172.21.15.133:33794) 2024-09-11T23:46:52.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed59d9b640 1 -- 172.21.15.133:0/1345332805 learned_addr learned my addr 172.21.15.133:0/1345332805 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:52.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed6320a740 1 -- 172.21.15.133:0/1345332805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cf5e5ab160 con 0x55cf5e646710 2024-09-11T23:46:52.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed6320a740 1 -- 172.21.15.133:0/1345332805 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf5e552720 con 0x55cf5e647030 2024-09-11T23:46:52.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed6320a740 1 -- 172.21.15.133:0/1345332805 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cf5e5a7120 con 0x55cf5e6514f0 2024-09-11T23:46:52.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed5a59c640 1 --2- 172.21.15.133:0/1345332805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e647030 0x55cf5e65a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed59d9b640 1 -- 172.21.15.133:0/1345332805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e647030 msgr2=0x55cf5e65a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:52.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed59d9b640 1 --2- 172.21.15.133:0/1345332805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e647030 0x55cf5e65a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed59d9b640 1 -- 172.21.15.133:0/1345332805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e646710 msgr2=0x55cf5e646af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:52.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed5ad9d640 1 --2- 172.21.15.133:0/1345332805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e646710 0x55cf5e646af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed59d9b640 1 --2- 172.21.15.133:0/1345332805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e646710 0x55cf5e646af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed59d9b640 1 -- 172.21.15.133:0/1345332805 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5e31cba0 con 0x55cf5e6514f0 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.551+0000 7fed5ad9d640 1 --2- 172.21.15.133:0/1345332805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e646710 0x55cf5e646af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.552+0000 7fed59d9b640 1 --2- 172.21.15.133:0/1345332805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e6514f0 0x55cf5e65c640 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fed50009c10 tx=0x7fed50011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f46196c714613755 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.552+0000 7fed4b7fe640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed440a2d70 0x7fed440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.552+0000 7fed48ff9640 1 -- 172.21.15.133:0/1345332805 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50004230 con 0x55cf5e6514f0 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.552+0000 7fed48ff9640 1 -- 172.21.15.133:0/1345332805 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed500043d0 con 0x55cf5e6514f0 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.552+0000 7fed4b7fe640 1 -- 172.21.15.133:0/3118391750 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fed4c0cf3f0 con 0x7fed5c101de0 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.553+0000 7fed48ff9640 1 -- 172.21.15.133:0/1345332805 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50019760 con 0x55cf5e6514f0 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.553+0000 7fed5ad9d640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed440a2d70 0x7fed440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.553+0000 7fed6320a740 1 -- 172.21.15.133:0/1345332805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e6514f0 msgr2=0x55cf5e65c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.553+0000 7fed6320a740 1 --2- 172.21.15.133:0/1345332805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e6514f0 0x55cf5e65c640 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fed50009c10 tx=0x7fed50011810 comp rx=0 tx=0).stop 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.553+0000 7fed6320a740 1 -- 172.21.15.133:0/1345332805 shutdown_connections 2024-09-11T23:46:52.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.553+0000 7fed6320a740 1 --2- 172.21.15.133:0/1345332805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e6514f0 0x55cf5e65c640 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.553+0000 7fed6320a740 1 --2- 172.21.15.133:0/1345332805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e647030 0x55cf5e65a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.553+0000 7fed6320a740 1 --2- 172.21.15.133:0/1345332805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e646710 0x55cf5e646af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.553+0000 7fed6320a740 1 -- 172.21.15.133:0/1345332805 >> 172.21.15.133:0/1345332805 conn(0x55cf5e642de0 msgr2=0x55cf5e6431c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:52.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.553+0000 7fed6320a740 1 -- 172.21.15.133:0/1345332805 shutdown_connections 2024-09-11T23:46:52.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.553+0000 7fed6320a740 1 -- 172.21.15.133:0/1345332805 wait complete. 2024-09-11T23:46:52.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.554+0000 7fed6320a740 1 -- start start 2024-09-11T23:46:52.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.554+0000 7fed5ad9d640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed440a2d70 0x7fed440a5230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fed54005cc0 tx=0x7fed54005f80 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:52.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.554+0000 7fed6320a740 1 Processor -- start 2024-09-11T23:46:52.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.555+0000 7fed6320a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e646710 0x55cf5e64e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.555+0000 7fed6320a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e647030 0x55cf5e64e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.555+0000 7fed5a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e646710 0x55cf5e64e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.555+0000 7fed5a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e646710 0x55cf5e64e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:33810/0 (socket says 172.21.15.133:33810) 2024-09-11T23:46:52.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.555+0000 7fed59d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e647030 0x55cf5e64e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.555+0000 7fed59d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e647030 0x55cf5e64e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37994/0 (socket says 172.21.15.133:37994) 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.555+0000 7fed5a59c640 1 -- 172.21.15.133:0/2727120131 learned_addr learned my addr 172.21.15.133:0/2727120131 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.555+0000 7fed6320a740 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e64edc0 0x55cf5e6663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.555+0000 7fed6320a740 1 -- 172.21.15.133:0/2727120131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cf5e65cec0 con 0x55cf5e647030 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.555+0000 7fed6320a740 1 -- 172.21.15.133:0/2727120131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf5e533900 con 0x55cf5e64edc0 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.555+0000 7fed6320a740 1 -- 172.21.15.133:0/2727120131 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cf5e65d040 con 0x55cf5e646710 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.556+0000 7fed5ad9d640 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e64edc0 0x55cf5e6663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.556+0000 7fed6320a740 10 client.0 _ll_register_callbacks cb 0x55cf5e531780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.556+0000 7fed5a59c640 1 -- 172.21.15.133:0/2727120131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e64edc0 msgr2=0x55cf5e6663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.556+0000 7fed5a59c640 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e64edc0 0x55cf5e6663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.556+0000 7fed5a59c640 1 -- 172.21.15.133:0/2727120131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e647030 msgr2=0x55cf5e64e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:52.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.556+0000 7fed5a59c640 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e647030 0x55cf5e64e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.556+0000 7fed5a59c640 1 -- 172.21.15.133:0/2727120131 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5e573c50 con 0x55cf5e646710 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.556+0000 7fed6320a740 1 -- 172.21.15.133:0/2727120131 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed500043d0 con 0x55cf5e646710 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.556+0000 7fed5a59c640 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e646710 0x55cf5e64e340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0058c0 tx=0x7fed4c0d2b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=555c4557e4297198 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.556+0000 7fed41ffb640 10 client.27286 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.557+0000 7fed41ffb640 1 -- 172.21.15.133:0/2727120131 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c01d040 con 0x55cf5e646710 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.557+0000 7fed41ffb640 1 -- 172.21.15.133:0/2727120131 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fed4c0860c0 con 0x55cf5e646710 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.557+0000 7fed41ffb640 1 -- 172.21.15.133:0/2727120131 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed4c09b490 con 0x55cf5e646710 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.557+0000 7fed6320a740 10 client.27286 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.557+0000 7fed6320a740 1 -- 172.21.15.133:0/2727120131 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55cf5e668e40 con 0x55cf5e646710 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.557+0000 7fed59d9b640 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e647030 0x55cf5e64e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.557+0000 7fed5ad9d640 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e64edc0 0x55cf5e6663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:46:52.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.557+0000 7fed41ffb640 1 -- 172.21.15.133:0/2727120131 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed4c0a4130 con 0x55cf5e646710 2024-09-11T23:46:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.557+0000 7fed6320a740 10 client.27286 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.557+0000 7fed6320a740 10 client.27286 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:46:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.558+0000 7fed6320a740 20 client.27286 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.558+0000 7fed6320a740 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf5e66a260 0x55cf5e66c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.558+0000 7fed6320a740 4 client.27286 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.558+0000 7fed6320a740 1 -- 172.21.15.133:0/2727120131 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf5e5b8890 con 0x55cf5e66a260 2024-09-11T23:46:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.558+0000 7fed59d9b640 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf5e66a260 0x55cf5e66c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.558+0000 7fed59d9b640 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf5e66a260 0x55cf5e66c650 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f564052d1f2632 server_cookie=d2f7061fe38d8e23 in_seq=0 out_seq=0 2024-09-11T23:46:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.558+0000 7fed41ffb640 10 client.27286 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.559+0000 7fed41ffb640 1 -- 172.21.15.133:0/2727120131 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf5e5b8890 con 0x55cf5e66a260 2024-09-11T23:46:52.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.559+0000 7fed41ffb640 10 client.27286 handle_command_reply: tid=0 2024-09-11T23:46:52.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.559+0000 7fed6320a740 1 client.27286 shutdown 2024-09-11T23:46:52.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed6320a740 1 -- 172.21.15.133:0/2727120131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e646710 msgr2=0x55cf5e64e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed6320a740 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e646710 0x55cf5e64e340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0058c0 tx=0x7fed4c0d2b20 comp rx=0 tx=0).stop 2024-09-11T23:46:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed6320a740 1 -- 172.21.15.133:0/2727120131 shutdown_connections 2024-09-11T23:46:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed6320a740 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf5e66a260 0x55cf5e66c650 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed6320a740 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e64edc0 0x55cf5e6663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed6320a740 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e647030 0x55cf5e64e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed6320a740 1 --2- 172.21.15.133:0/2727120131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e646710 0x55cf5e64e340 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed41ffb640 0 client.27286 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed6320a740 1 -- 172.21.15.133:0/2727120131 >> 172.21.15.133:0/2727120131 conn(0x55cf5e642de0 msgr2=0x55cf5e648b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed6320a740 1 -- 172.21.15.133:0/2727120131 shutdown_connections 2024-09-11T23:46:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed6320a740 1 -- 172.21.15.133:0/2727120131 wait complete. 2024-09-11T23:46:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.560+0000 7fed6320a740 20 client.27286 trim_cache size 0 max 16384 2024-09-11T23:46:52.586 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:52.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.595+0000 7fed6320a740 1 Processor -- start 2024-09-11T23:46:52.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.595+0000 7fed6320a740 1 -- start start 2024-09-11T23:46:52.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed6320a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 0x55cf5e6488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed6320a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e647030 0x55cf5e648e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed6320a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e64edc0 0x55cf5e6661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed6320a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1ab590 con 0x55cf5e64edc0 2024-09-11T23:46:52.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed6320a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1ad0d0 con 0x55cf5e646710 2024-09-11T23:46:52.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed6320a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1abc60 con 0x55cf5e647030 2024-09-11T23:46:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed5ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e64edc0 0x55cf5e6661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed5ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e64edc0 0x55cf5e6661b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37996/0 (socket says 172.21.15.133:37996) 2024-09-11T23:46:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed5ad9d640 1 -- 172.21.15.133:0/1964381439 learned_addr learned my addr 172.21.15.133:0/1964381439 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed5a59c640 1 --2- 172.21.15.133:0/1964381439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 0x55cf5e6488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed59d9b640 1 --2- 172.21.15.133:0/1964381439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e647030 0x55cf5e648e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed5a59c640 1 -- 172.21.15.133:0/1964381439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e647030 msgr2=0x55cf5e648e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:52.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed5a59c640 1 --2- 172.21.15.133:0/1964381439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e647030 0x55cf5e648e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed5a59c640 1 -- 172.21.15.133:0/1964381439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e64edc0 msgr2=0x55cf5e6661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:52.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed5a59c640 1 --2- 172.21.15.133:0/1964381439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e64edc0 0x55cf5e6661b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed5a59c640 1 -- 172.21.15.133:0/1964381439 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5e66c9c0 con 0x55cf5e646710 2024-09-11T23:46:52.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.596+0000 7fed59d9b640 1 --2- 172.21.15.133:0/1964381439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e647030 0x55cf5e648e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:46:52.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.597+0000 7fed5a59c640 1 --2- 172.21.15.133:0/1964381439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 0x55cf5e6488c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fed4c098320 tx=0x7fed4c086050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdf193af266aa152 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:52.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.597+0000 7fed43fff640 1 -- 172.21.15.133:0/1964381439 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c01d040 con 0x55cf5e646710 2024-09-11T23:46:52.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.597+0000 7fed43fff640 1 -- 172.21.15.133:0/1964381439 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed500043d0 con 0x55cf5e646710 2024-09-11T23:46:52.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.597+0000 7fed43fff640 1 -- 172.21.15.133:0/1964381439 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c09b2b0 con 0x55cf5e646710 2024-09-11T23:46:52.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.598+0000 7fed6320a740 1 -- 172.21.15.133:0/1964381439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 msgr2=0x55cf5e6488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:52.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.598+0000 7fed6320a740 1 --2- 172.21.15.133:0/1964381439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 0x55cf5e6488c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fed4c098320 tx=0x7fed4c086050 comp rx=0 tx=0).stop 2024-09-11T23:46:52.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.598+0000 7fed6320a740 1 -- 172.21.15.133:0/1964381439 shutdown_connections 2024-09-11T23:46:52.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.598+0000 7fed6320a740 1 --2- 172.21.15.133:0/1964381439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e64edc0 0x55cf5e6661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.598+0000 7fed6320a740 1 --2- 172.21.15.133:0/1964381439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e647030 0x55cf5e648e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.598+0000 7fed6320a740 1 --2- 172.21.15.133:0/1964381439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 0x55cf5e6488c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.598+0000 7fed6320a740 1 -- 172.21.15.133:0/1964381439 >> 172.21.15.133:0/1964381439 conn(0x55cf5e642de0 msgr2=0x55cf5e6481b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:52.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.598+0000 7fed6320a740 1 -- 172.21.15.133:0/1964381439 shutdown_connections 2024-09-11T23:46:52.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.598+0000 7fed6320a740 1 -- 172.21.15.133:0/1964381439 wait complete. 2024-09-11T23:46:52.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.599+0000 7fed6320a740 1 -- start start 2024-09-11T23:46:52.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.599+0000 7fed6320a740 1 Processor -- start 2024-09-11T23:46:52.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.599+0000 7fed6320a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 0x55cf5e65e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.599+0000 7fed6320a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e647030 0x55cf5e65e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed6320a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e64edc0 0x55cf5e65ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed5a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 0x55cf5e65e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed5a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 0x55cf5e65e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52770/0 (socket says 172.21.15.133:52770) 2024-09-11T23:46:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed5a59c640 1 -- 172.21.15.133:0/1327847907 learned_addr learned my addr 172.21.15.133:0/1327847907 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed6320a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1aca00 con 0x55cf5e64edc0 2024-09-11T23:46:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed6320a740 1 -- 172.21.15.133:0/1327847907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1ac330 con 0x55cf5e646710 2024-09-11T23:46:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed59d9b640 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e647030 0x55cf5e65e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed6320a740 1 -- 172.21.15.133:0/1327847907 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1a95b0 con 0x55cf5e647030 2024-09-11T23:46:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed6320a740 10 client.0 _ll_register_callbacks cb 0x7fed4c004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed5a59c640 1 -- 172.21.15.133:0/1327847907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e647030 msgr2=0x55cf5e65e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed5a59c640 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e647030 0x55cf5e65e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed5a59c640 1 -- 172.21.15.133:0/1327847907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e64edc0 msgr2=0x55cf5e65ec60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:46:52.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed5a59c640 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e64edc0 0x55cf5e65ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.600+0000 7fed5a59c640 1 -- 172.21.15.133:0/1327847907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5e646e60 con 0x55cf5e646710 2024-09-11T23:46:52.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.601+0000 7fed5a59c640 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 0x55cf5e65e1e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0058c0 tx=0x7fed4c0cd0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fd28c4290a34713 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:52.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.601+0000 7fed41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:46:52.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.601+0000 7fed41ffb640 1 -- 172.21.15.133:0/1327847907 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0b8070 con 0x55cf5e646710 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.601+0000 7fed41ffb640 1 -- 172.21.15.133:0/1327847907 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fed4c0ba2f0 con 0x55cf5e646710 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.601+0000 7fed6320a740 1 -- 172.21.15.133:0/1327847907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed500043d0 con 0x55cf5e646710 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.601+0000 7fed41ffb640 1 -- 172.21.15.133:0/1327847907 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed4c01d070 con 0x55cf5e646710 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.601+0000 7fed6320a740 10 client.27506 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.601+0000 7fed6320a740 1 -- 172.21.15.133:0/1327847907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cf5e646e60 con 0x55cf5e646710 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.602+0000 7fed41ffb640 1 -- 172.21.15.133:0/1327847907 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed500043d0 con 0x55cf5e646710 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.602+0000 7fed41ffb640 1 -- 172.21.15.133:0/1327847907 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed4c0b4bc0 con 0x55cf5e646710 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.602+0000 7fed6320a740 10 client.27506 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.602+0000 7fed6320a740 10 client.27506 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.602+0000 7fed6320a740 20 client.27506 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.602+0000 7fed6320a740 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf5e6d53e0 0x55cf5e6d77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.602+0000 7fed6320a740 4 client.27506 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.602+0000 7fed6320a740 1 -- 172.21.15.133:0/1327847907 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x55cf5e5b2be0 con 0x55cf5e6d53e0 2024-09-11T23:46:52.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.602+0000 7fed59d9b640 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf5e6d53e0 0x55cf5e6d77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.603+0000 7fed41ffb640 10 client.27506 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.603+0000 7fed59d9b640 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf5e6d53e0 0x55cf5e6d77d0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc836269b9aa89 server_cookie=af601000d9f7e45f in_seq=0 out_seq=0 2024-09-11T23:46:52.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.802+0000 7fed41ffb640 1 -- 172.21.15.133:0/1327847907 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cf5e5b2be0 con 0x55cf5e6d53e0 2024-09-11T23:46:52.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.802+0000 7fed41ffb640 10 client.27506 handle_command_reply: tid=0 2024-09-11T23:46:52.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.802+0000 7fed6320a740 1 client.27506 shutdown 2024-09-11T23:46:52.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed6320a740 1 -- 172.21.15.133:0/1327847907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 msgr2=0x55cf5e65e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:52.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed6320a740 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 0x55cf5e65e1e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0058c0 tx=0x7fed4c0cd0b0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed6320a740 1 -- 172.21.15.133:0/1327847907 shutdown_connections 2024-09-11T23:46:52.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed6320a740 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55cf5e6d53e0 0x55cf5e6d77d0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed6320a740 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf5e64edc0 0x55cf5e65ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed41ffb640 0 client.27506 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:46:52.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed6320a740 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf5e647030 0x55cf5e65e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed6320a740 1 --2- 172.21.15.133:0/1327847907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5e646710 0x55cf5e65e1e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed6320a740 1 -- 172.21.15.133:0/1327847907 >> 172.21.15.133:0/1327847907 conn(0x55cf5e642de0 msgr2=0x55cf5e6489d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:52.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed6320a740 1 -- 172.21.15.133:0/1327847907 shutdown_connections 2024-09-11T23:46:52.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed6320a740 1 -- 172.21.15.133:0/1327847907 wait complete. 2024-09-11T23:46:52.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed6320a740 20 client.27506 trim_cache size 0 max 16384 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 38, 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: "d0886d4a": { 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 38, 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:46:52.805 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 142.6, 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:46:52.806 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:46:52.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed60e31640 1 -- 172.21.15.133:0/3118391750 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed440a2d70 msgr2=0x7fed440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:52.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed60e31640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed440a2d70 0x7fed440a5230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fed54005cc0 tx=0x7fed54005f80 comp rx=0 tx=0).stop 2024-09-11T23:46:52.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed60e31640 1 -- 172.21.15.133:0/3118391750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c101de0 msgr2=0x7fed5c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:52.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed60e31640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c101de0 0x7fed5c1983c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00c890 tx=0x7fed4c00cd50 comp rx=0 tx=0).stop 2024-09-11T23:46:52.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed60e31640 1 -- 172.21.15.133:0/3118391750 shutdown_connections 2024-09-11T23:46:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed60e31640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed440a2d70 0x7fed440a5230 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed60e31640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed5c106f10 0x7fed5c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed60e31640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed5c102790 0x7fed5c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed60e31640 1 --2- 172.21.15.133:0/3118391750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c101de0 0x7fed5c1983c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.803+0000 7fed60e31640 1 -- 172.21.15.133:0/3118391750 >> 172.21.15.133:0/3118391750 conn(0x7fed5c0fd880 msgr2=0x7fed5c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.804+0000 7fed60e31640 1 -- 172.21.15.133:0/3118391750 shutdown_connections 2024-09-11T23:46:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:52.804+0000 7fed60e31640 1 -- 172.21.15.133:0/3118391750 wait complete. 2024-09-11T23:46:53.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:52 smithi178 ceph-mon[35882]: pgmap v314: 97 pgs: 97 active+clean; 70 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 604 MiB/s wr, 1.85k op/s 2024-09-11T23:46:53.182 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:52 smithi133 ceph-mon[31430]: pgmap v314: 97 pgs: 97 active+clean; 70 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 604 MiB/s wr, 1.85k op/s 2024-09-11T23:46:53.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:52 smithi138 ceph-mon[35142]: pgmap v314: 97 pgs: 97 active+clean; 70 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 604 MiB/s wr, 1.85k op/s 2024-09-11T23:46:53.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 -- 172.21.15.133:0/83576868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b4106f20 msgr2=0x7f62b410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:53.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 --2- 172.21.15.133:0/83576868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b4106f20 0x7f62b410eaa0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009ca0 tx=0x7f62ac02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:46:53.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 -- 172.21.15.133:0/83576868 shutdown_connections 2024-09-11T23:46:53.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 --2- 172.21.15.133:0/83576868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b4106f20 0x7f62b410eaa0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:53.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 --2- 172.21.15.133:0/83576868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b41027a0 0x7f62b41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:53.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 --2- 172.21.15.133:0/83576868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4101df0 0x7f62b41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:53.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 -- 172.21.15.133:0/83576868 >> 172.21.15.133:0/83576868 conn(0x7f62b40fd8f0 msgr2=0x7f62b40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:53.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 -- 172.21.15.133:0/83576868 shutdown_connections 2024-09-11T23:46:53.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 -- 172.21.15.133:0/83576868 wait complete. 2024-09-11T23:46:53.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 Processor -- start 2024-09-11T23:46:53.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 -- start start 2024-09-11T23:46:53.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.264+0000 7f62bc343640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4101df0 0x7f62b41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:53.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62bc343640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b41027a0 0x7f62b4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62bc343640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b4106f20 0x7f62b419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62bc343640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4111f80 con 0x7f62b4106f20 2024-09-11T23:46:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62bc343640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4111e00 con 0x7f62b4101df0 2024-09-11T23:46:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62bc343640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4112100 con 0x7f62b41027a0 2024-09-11T23:46:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ba8e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b4106f20 0x7f62b419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ba8e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b4106f20 0x7f62b419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38026/0 (socket says 172.21.15.133:38026) 2024-09-11T23:46:53.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ba8e0640 1 -- 172.21.15.133:0/1588192642 learned_addr learned my addr 172.21.15.133:0/1588192642 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:53.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ba0df640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4101df0 0x7f62b41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:53.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62b98de640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b41027a0 0x7f62b4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:53.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ba8e0640 1 -- 172.21.15.133:0/1588192642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b41027a0 msgr2=0x7f62b4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:53.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ba8e0640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b41027a0 0x7f62b4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:53.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ba8e0640 1 -- 172.21.15.133:0/1588192642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4101df0 msgr2=0x7f62b41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:53.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ba8e0640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4101df0 0x7f62b41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:53.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ba8e0640 1 -- 172.21.15.133:0/1588192642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b419d370 con 0x7f62b4106f20 2024-09-11T23:46:53.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ba0df640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4101df0 0x7f62b41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ba8e0640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b4106f20 0x7f62b419cc90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f62ac031a20 tx=0x7f62ac031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ab7fe640 1 -- 172.21.15.133:0/1588192642 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac003c30 con 0x7f62b4106f20 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ab7fe640 1 -- 172.21.15.133:0/1588192642 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62ac003dd0 con 0x7f62b4106f20 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62ab7fe640 1 -- 172.21.15.133:0/1588192642 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac02fc40 con 0x7f62b4106f20 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62bc343640 1 -- 172.21.15.133:0/1588192642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b419d600 con 0x7f62b4106f20 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.265+0000 7f62bc343640 1 -- 172.21.15.133:0/1588192642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b41a4f40 con 0x7f62b4106f20 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.266+0000 7f62bc343640 1 -- 172.21.15.133:0/1588192642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b41021d0 con 0x7f62b4106f20 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.268+0000 7f62ab7fe640 1 -- 172.21.15.133:0/1588192642 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f62ac004050 con 0x7f62b4106f20 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.268+0000 7f62ab7fe640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f62a00a2cf0 0x7f62a00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.268+0000 7f62ba0df640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f62a00a2cf0 0x7f62a00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.268+0000 7f62ab7fe640 1 -- 172.21.15.133:0/1588192642 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f62ac0f0540 con 0x7f62b4106f20 2024-09-11T23:46:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.269+0000 7f62ba0df640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f62a00a2cf0 0x7f62a00a51b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f62b000bfd0 tx=0x7f62b000b390 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.272+0000 7f62ab7fe640 1 -- 172.21.15.133:0/1588192642 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62ac0bd460 con 0x7f62b4106f20 2024-09-11T23:46:53.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.482+0000 7f62bc343640 1 -- 172.21.15.133:0/1588192642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62b410a160 con 0x7f62b4106f20 2024-09-11T23:46:53.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.482+0000 7f62ab7fe640 1 -- 172.21.15.133:0/1588192642 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7f62ac0bcd60 con 0x7f62b4106f20 2024-09-11T23:46:53.482 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:46:53.482 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:46:53.482 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:46:53.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 -- 172.21.15.133:0/1588192642 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f62a00a2cf0 msgr2=0x7f62a00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:53.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f62a00a2cf0 0x7f62a00a51b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f62b000bfd0 tx=0x7f62b000b390 comp rx=0 tx=0).stop 2024-09-11T23:46:53.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 -- 172.21.15.133:0/1588192642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b4106f20 msgr2=0x7f62b419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:53.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b4106f20 0x7f62b419cc90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f62ac031a20 tx=0x7f62ac031a50 comp rx=0 tx=0).stop 2024-09-11T23:46:53.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 -- 172.21.15.133:0/1588192642 shutdown_connections 2024-09-11T23:46:53.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f62a00a2cf0 0x7f62a00a51b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:53.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62b4106f20 0x7f62b419cc90 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:53.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62b41027a0 0x7f62b4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:53.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 --2- 172.21.15.133:0/1588192642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4101df0 0x7f62b41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:53.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 -- 172.21.15.133:0/1588192642 >> 172.21.15.133:0/1588192642 conn(0x7f62b40fd8f0 msgr2=0x7f62b40fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:53.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 -- 172.21.15.133:0/1588192642 shutdown_connections 2024-09-11T23:46:53.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:53.485+0000 7f62bc343640 1 -- 172.21.15.133:0/1588192642 wait complete. 2024-09-11T23:46:54.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:53 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1588192642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:46:54.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:53 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1588192642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:46:54.182 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd0886d4a', quiesce took 7.5 seconds. Will release after: 15.05 2024-09-11T23:46:54.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:53 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1588192642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:46:54.934 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:54 smithi133 ceph-mon[31430]: pgmap v315: 97 pgs: 97 active+clean; 70 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 766 MiB/s wr, 2.11k op/s 2024-09-11T23:46:54.961 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:46:55.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:54 smithi178 ceph-mon[35882]: pgmap v315: 97 pgs: 97 active+clean; 70 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 766 MiB/s wr, 2.11k op/s 2024-09-11T23:46:55.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:54 smithi138 ceph-mon[35142]: pgmap v315: 97 pgs: 97 active+clean; 70 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 766 MiB/s wr, 2.11k op/s 2024-09-11T23:46:55.212 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:55.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.686+0000 7f739ca30640 1 -- 172.21.15.133:0/648009963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73981027a0 msgr2=0x7f73981067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:55.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.686+0000 7f739ca30640 1 --2- 172.21.15.133:0/648009963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73981027a0 0x7f73981067f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7380009900 tx=0x7f738002a410 comp rx=0 tx=0).stop 2024-09-11T23:46:55.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.686+0000 7f739ca30640 1 -- 172.21.15.133:0/648009963 shutdown_connections 2024-09-11T23:46:55.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.686+0000 7f739ca30640 1 --2- 172.21.15.133:0/648009963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7398106f20 0x7f739810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.686+0000 7f739ca30640 1 --2- 172.21.15.133:0/648009963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73981027a0 0x7f73981067f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.686+0000 7f739ca30640 1 --2- 172.21.15.133:0/648009963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7398101df0 0x7f73981021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.686+0000 7f739ca30640 1 -- 172.21.15.133:0/648009963 >> 172.21.15.133:0/648009963 conn(0x7f73980fd8f0 msgr2=0x7f73980ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.686+0000 7f739ca30640 1 -- 172.21.15.133:0/648009963 shutdown_connections 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.686+0000 7f739ca30640 1 -- 172.21.15.133:0/648009963 wait complete. 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.687+0000 7f739ca30640 1 Processor -- start 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.687+0000 7f739ca30640 1 -- start start 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.687+0000 7f739ca30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7398101df0 0x7f7398198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.687+0000 7f739ca30640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73981027a0 0x7f73981989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.687+0000 7f739ca30640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7398106f20 0x7f739819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.687+0000 7f739ca30640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7398111fd0 con 0x7f7398101df0 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.687+0000 7f739ca30640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7398111e50 con 0x7f7398106f20 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.687+0000 7f739ca30640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7398112150 con 0x7f73981027a0 2024-09-11T23:46:55.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.687+0000 7f739659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7398101df0 0x7f7398198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.687+0000 7f739659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7398101df0 0x7f7398198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38046/0 (socket says 172.21.15.133:38046) 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.687+0000 7f739659c640 1 -- 172.21.15.133:0/691819027 learned_addr learned my addr 172.21.15.133:0/691819027 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f7396d9d640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7398106f20 0x7f739819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f738dd9b640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73981027a0 0x7f73981989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f739659c640 1 -- 172.21.15.133:0/691819027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73981027a0 msgr2=0x7f73981989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f739659c640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73981027a0 0x7f73981989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f739659c640 1 -- 172.21.15.133:0/691819027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7398106f20 msgr2=0x7f739819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f739659c640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7398106f20 0x7f739819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f739659c640 1 -- 172.21.15.133:0/691819027 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739819d440 con 0x7f7398101df0 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f738dd9b640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73981027a0 0x7f73981989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f739659c640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7398101df0 0x7f7398198490 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f738800d770 tx=0x7f738800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f738ffff640 1 -- 172.21.15.133:0/691819027 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388017070 con 0x7f7398101df0 2024-09-11T23:46:55.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f739ca30640 1 -- 172.21.15.133:0/691819027 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739819d730 con 0x7f7398101df0 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f738ffff640 1 -- 172.21.15.133:0/691819027 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7388005500 con 0x7f7398101df0 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f738ffff640 1 -- 172.21.15.133:0/691819027 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388004e00 con 0x7f7398101df0 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.688+0000 7f739ca30640 1 -- 172.21.15.133:0/691819027 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73981a5010 con 0x7f7398101df0 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.690+0000 7f738ffff640 1 -- 172.21.15.133:0/691819027 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7388010460 con 0x7f7398101df0 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f739ee09740 1 Processor -- start 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f739ee09740 1 -- start start 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f739ee09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320dda9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f739ee09740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320dda95d0 0x56320dda9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f739ee09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320ddb4370 0x56320ddb6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f739ee09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56320dd0f2f0 con 0x56320ddaab40 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f739ee09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56320dcb6cf0 con 0x56320ddb4370 2024-09-11T23:46:55.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f739ee09740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56320dd0b2b0 con 0x56320dda95d0 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f7396d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320dda9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f7396d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320dda9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38048/0 (socket says 172.21.15.133:38048) 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f7396d9d640 1 -- 172.21.15.133:0/4061737410 learned_addr learned my addr 172.21.15.133:0/4061737410 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f738dd9b640 1 --2- 172.21.15.133:0/4061737410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320ddb4370 0x56320ddb6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f739659c640 1 --2- 172.21.15.133:0/4061737410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320dda95d0 0x56320dda9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.691+0000 7f738ffff640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73840a2cf0 0x7f73840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.692+0000 7f7396d9d640 1 -- 172.21.15.133:0/4061737410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320dda95d0 msgr2=0x56320dda9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.692+0000 7f7396d9d640 1 --2- 172.21.15.133:0/4061737410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320dda95d0 0x56320dda9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.692+0000 7f7396d9d640 1 -- 172.21.15.133:0/4061737410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320ddb4370 msgr2=0x56320ddb6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.692+0000 7f7396d9d640 1 --2- 172.21.15.133:0/4061737410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320ddb4370 0x56320ddb6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.692+0000 7f7396d9d640 1 -- 172.21.15.133:0/4061737410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320dc9b3a0 con 0x56320ddaab40 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.692+0000 7f7396d9d640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73840a2cf0 0x7f73840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.692+0000 7f738ffff640 1 -- 172.21.15.133:0/691819027 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f73880cef00 con 0x7f7398101df0 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.692+0000 7f7396d9d640 1 --2- 172.21.15.133:0/4061737410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320dda9090 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f73900104a0 tx=0x7f73900108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a381426e68676f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.692+0000 7f738cd99640 1 -- 172.21.15.133:0/4061737410 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739001d910 con 0x56320ddaab40 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.692+0000 7f738cd99640 1 -- 172.21.15.133:0/4061737410 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7390011440 con 0x56320ddaab40 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.692+0000 7f738cd99640 1 -- 172.21.15.133:0/4061737410 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739001c640 con 0x56320ddaab40 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.693+0000 7f739ee09740 1 -- 172.21.15.133:0/4061737410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 msgr2=0x56320dda9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.693+0000 7f7396d9d640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73840a2cf0 0x7f73840a51b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7390021000 tx=0x7f739001e000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.693+0000 7f739ee09740 1 --2- 172.21.15.133:0/4061737410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320dda9090 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f73900104a0 tx=0x7f73900108a0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.693+0000 7f739ee09740 1 -- 172.21.15.133:0/4061737410 shutdown_connections 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.693+0000 7f739ee09740 1 --2- 172.21.15.133:0/4061737410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320ddb4370 0x56320ddb6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.693+0000 7f739ee09740 1 --2- 172.21.15.133:0/4061737410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320dda95d0 0x56320dda9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.693+0000 7f739ee09740 1 --2- 172.21.15.133:0/4061737410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320dda9090 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.693+0000 7f739ee09740 1 -- 172.21.15.133:0/4061737410 >> 172.21.15.133:0/4061737410 conn(0x56320dd72310 msgr2=0x56320dda5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.693+0000 7f739ee09740 1 -- 172.21.15.133:0/4061737410 shutdown_connections 2024-09-11T23:46:55.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.693+0000 7f739ee09740 1 -- 172.21.15.133:0/4061737410 wait complete. 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.693+0000 7f739ee09740 1 -- start start 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f739ee09740 1 Processor -- start 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f739ee09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320ddb3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f739ee09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddb1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f739ee09740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddb4370 0x56320ddb1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f739ee09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56320ddb7070 con 0x56320ddaab40 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f739ee09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56320dcca060 con 0x56320dda95d0 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f739ee09740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56320ddb71f0 con 0x56320ddb4370 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f739ee09740 10 client.0 _ll_register_callbacks cb 0x56320da1d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f738dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddb1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f738dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddb1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38054/0 (socket says 172.21.15.133:38054) 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f738dd9b640 1 -- 172.21.15.133:0/44216570 learned_addr learned my addr 172.21.15.133:0/44216570 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:55.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f739659c640 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320ddb3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f7396d9d640 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddb4370 0x56320ddb1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f738dd9b640 1 -- 172.21.15.133:0/44216570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddb4370 msgr2=0x56320ddb1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f738dd9b640 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddb4370 0x56320ddb1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f738dd9b640 1 -- 172.21.15.133:0/44216570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 msgr2=0x56320ddb3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f738dd9b640 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320ddb3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f738dd9b640 1 -- 172.21.15.133:0/44216570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320da803c0 con 0x56320ddaab40 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f7396d9d640 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddb4370 0x56320ddb1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f738dd9b640 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddb1010 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7380009900 tx=0x7f7380004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41046c240ec6f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f737dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f737dffb640 1 -- 172.21.15.133:0/44216570 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380035040 con 0x56320ddaab40 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f737dffb640 1 -- 172.21.15.133:0/44216570 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f7380005450 con 0x56320ddaab40 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.694+0000 7f739ee09740 1 -- 172.21.15.133:0/44216570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7390011440 con 0x56320ddaab40 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f737dffb640 1 -- 172.21.15.133:0/44216570 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7380005650 con 0x56320ddaab40 2024-09-11T23:46:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f739ee09740 10 client.17973 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f739ee09740 1 -- 172.21.15.133:0/44216570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56320ddc2b60 con 0x56320ddaab40 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f737dffb640 1 -- 172.21.15.133:0/44216570 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7390011440 con 0x56320ddaab40 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f737dffb640 1 -- 172.21.15.133:0/44216570 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7380003b50 con 0x56320ddaab40 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f739ee09740 10 client.17973 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f739ee09740 10 client.17973 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f739ee09740 20 client.17973 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f739ee09740 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x56320ddc4020 0x56320ddc64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f739ee09740 4 client.17973 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f739ee09740 1 -- 172.21.15.133:0/44216570 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56320dd1c860 con 0x56320ddc4020 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.695+0000 7f739659c640 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x56320ddc4020 0x56320ddc64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f737dffb640 10 client.17973 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f739659c640 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x56320ddc4020 0x56320ddc64a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=554ae9dbfd548bae server_cookie=811d13f4339ff083 in_seq=0 out_seq=0 2024-09-11T23:46:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f737dffb640 1 -- 172.21.15.133:0/44216570 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56320dd1c860 con 0x56320ddc4020 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f737dffb640 10 client.17973 handle_command_reply: tid=0 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f739ee09740 1 client.17973 shutdown 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f739ee09740 1 -- 172.21.15.133:0/44216570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 msgr2=0x56320ddb1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f739ee09740 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddb1010 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7380009900 tx=0x7f7380004440 comp rx=0 tx=0).stop 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f739ee09740 1 -- 172.21.15.133:0/44216570 shutdown_connections 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f739ee09740 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x56320ddc4020 0x56320ddc64a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f739ee09740 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddb4370 0x56320ddb1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f737dffb640 0 client.17973 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f739ee09740 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddb1010 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f739ee09740 1 --2- 172.21.15.133:0/44216570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320ddb3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.696+0000 7f739ee09740 1 -- 172.21.15.133:0/44216570 >> 172.21.15.133:0/44216570 conn(0x56320dd72310 msgr2=0x56320ddab960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.697+0000 7f739ee09740 1 -- 172.21.15.133:0/44216570 shutdown_connections 2024-09-11T23:46:55.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.697+0000 7f739ee09740 1 -- 172.21.15.133:0/44216570 wait complete. 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.697+0000 7f739ee09740 20 client.17973 trim_cache size 0 max 16384 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.709+0000 7f739ee09740 1 Processor -- start 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.709+0000 7f739ee09740 1 -- start start 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.709+0000 7f739ee09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320dda5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f739ee09740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddaab40 0x56320ddc0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f739ee09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddb4370 0x56320ddcbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f739ee09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73981ad790 con 0x56320ddb4370 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f739ee09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73981af2d0 con 0x56320dda95d0 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f739ee09740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73981ade60 con 0x56320ddaab40 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f7396d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddb4370 0x56320ddcbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f7396d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddb4370 0x56320ddcbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38064/0 (socket says 172.21.15.133:38064) 2024-09-11T23:46:55.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f7396d9d640 1 -- 172.21.15.133:0/2335554489 learned_addr learned my addr 172.21.15.133:0/2335554489 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f739659c640 1 --2- 172.21.15.133:0/2335554489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320dda5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f738dd9b640 1 --2- 172.21.15.133:0/2335554489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddaab40 0x56320ddc0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f7396d9d640 1 -- 172.21.15.133:0/2335554489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddaab40 msgr2=0x56320ddc0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f7396d9d640 1 --2- 172.21.15.133:0/2335554489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddaab40 0x56320ddc0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f7396d9d640 1 -- 172.21.15.133:0/2335554489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 msgr2=0x56320dda5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f7396d9d640 1 --2- 172.21.15.133:0/2335554489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320dda5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f7396d9d640 1 -- 172.21.15.133:0/2335554489 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320ddb40d0 con 0x56320ddb4370 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f739659c640 1 --2- 172.21.15.133:0/2335554489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320dda5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f7396d9d640 1 --2- 172.21.15.133:0/2335554489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddb4370 0x56320ddcbd70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7390016010 tx=0x7f7390004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170caa0ca4ae280e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f737ffff640 1 -- 172.21.15.133:0/2335554489 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390009a00 con 0x56320ddb4370 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f737ffff640 1 -- 172.21.15.133:0/2335554489 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739001cdb0 con 0x56320ddb4370 2024-09-11T23:46:55.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.710+0000 7f737ffff640 1 -- 172.21.15.133:0/2335554489 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739001d890 con 0x56320ddb4370 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 -- 172.21.15.133:0/2335554489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddb4370 msgr2=0x56320ddcbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 --2- 172.21.15.133:0/2335554489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddb4370 0x56320ddcbd70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7390016010 tx=0x7f7390004a00 comp rx=0 tx=0).stop 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 -- 172.21.15.133:0/2335554489 shutdown_connections 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 --2- 172.21.15.133:0/2335554489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddb4370 0x56320ddcbd70 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 --2- 172.21.15.133:0/2335554489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddaab40 0x56320ddc0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 --2- 172.21.15.133:0/2335554489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320dda5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 -- 172.21.15.133:0/2335554489 >> 172.21.15.133:0/2335554489 conn(0x56320dd72310 msgr2=0x56320dda5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 -- 172.21.15.133:0/2335554489 shutdown_connections 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 -- 172.21.15.133:0/2335554489 wait complete. 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 -- start start 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 Processor -- start 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320ddbc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddbc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddb4370 0x56320ddb9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73981aec00 con 0x56320ddaab40 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73981ae530 con 0x56320dda95d0 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.711+0000 7f739ee09740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73981ab790 con 0x56320ddb4370 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f739ee09740 10 client.0 _ll_register_callbacks cb 0x7f7380009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f738dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddbc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f738dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddbc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38078/0 (socket says 172.21.15.133:38078) 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f738dd9b640 1 -- 172.21.15.133:0/1880473988 learned_addr learned my addr 172.21.15.133:0/1880473988 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f739659c640 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320ddbc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f7396d9d640 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddb4370 0x56320ddb9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f739659c640 1 -- 172.21.15.133:0/1880473988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddb4370 msgr2=0x56320ddb9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f739659c640 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddb4370 0x56320ddb9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f739659c640 1 -- 172.21.15.133:0/1880473988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 msgr2=0x56320ddbc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f739659c640 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddbc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f739659c640 1 -- 172.21.15.133:0/1880473988 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320ddb9990 con 0x56320dda95d0 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f738dd9b640 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddbc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f739659c640 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320ddbc480 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f73880cc630 tx=0x7f7388096ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f3557f0d16c2054 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f737dffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f737dffb640 1 -- 172.21.15.133:0/1880473988 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388020040 con 0x56320dda95d0 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f737dffb640 1 -- 172.21.15.133:0/1880473988 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f7388097a70 con 0x56320dda95d0 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f739ee09740 1 -- 172.21.15.133:0/1880473988 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f739001cdb0 con 0x56320dda95d0 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f737dffb640 1 -- 172.21.15.133:0/1880473988 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f738809b400 con 0x56320dda95d0 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f739ee09740 10 client.27530 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.712+0000 7f739ee09740 1 -- 172.21.15.133:0/1880473988 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56320de2f4f0 con 0x56320dda95d0 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.713+0000 7f737dffb640 1 -- 172.21.15.133:0/1880473988 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739001cdb0 con 0x56320dda95d0 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.713+0000 7f737dffb640 1 -- 172.21.15.133:0/1880473988 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f73880d0040 con 0x56320dda95d0 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.713+0000 7f739ee09740 10 client.27530 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:55.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.713+0000 7f739ee09740 10 client.27530 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.713+0000 7f739ee09740 20 client.27530 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.713+0000 7f739ee09740 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x56320ddc4020 0x56320de31ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.713+0000 7f739ee09740 4 client.27530 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.713+0000 7f739ee09740 1 -- 172.21.15.133:0/1880473988 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56320dd16d70 con 0x56320ddc4020 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.713+0000 7f738dd9b640 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x56320ddc4020 0x56320de31ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.713+0000 7f737dffb640 10 client.27530 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.713+0000 7f738dd9b640 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x56320ddc4020 0x56320de31ca0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd8c3f530ef5deda server_cookie=b28bcbad25a52b57 in_seq=0 out_seq=0 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f737dffb640 1 -- 172.21.15.133:0/1880473988 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56320dd16d70 con 0x56320ddc4020 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f737dffb640 10 client.27530 handle_command_reply: tid=0 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 1 client.27530 shutdown 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 1 -- 172.21.15.133:0/1880473988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 msgr2=0x56320ddbc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320ddbc480 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f73880cc630 tx=0x7f7388096ff0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 1 -- 172.21.15.133:0/1880473988 shutdown_connections 2024-09-11T23:46:55.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x56320ddc4020 0x56320de31ca0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320ddb4370 0x56320ddb9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56320ddaab40 0x56320ddbc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 1 --2- 172.21.15.133:0/1880473988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56320dda95d0 0x56320ddbc480 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f737dffb640 0 client.27530 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:46:55.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 1 -- 172.21.15.133:0/1880473988 >> 172.21.15.133:0/1880473988 conn(0x56320dd72310 msgr2=0x56320ddb2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:55.738 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:46:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 1 -- 172.21.15.133:0/1880473988 shutdown_connections 2024-09-11T23:46:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 1 -- 172.21.15.133:0/1880473988 wait complete. 2024-09-11T23:46:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ee09740 20 client.27530 trim_cache size 0 max 16384 2024-09-11T23:46:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ca30640 1 -- 172.21.15.133:0/691819027 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73840a2cf0 msgr2=0x7f73840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ca30640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73840a2cf0 0x7f73840a51b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7390021000 tx=0x7f739001e000 comp rx=0 tx=0).stop 2024-09-11T23:46:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ca30640 1 -- 172.21.15.133:0/691819027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7398101df0 msgr2=0x7f7398198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ca30640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7398101df0 0x7f7398198490 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f738800d770 tx=0x7f738800dc30 comp rx=0 tx=0).stop 2024-09-11T23:46:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ca30640 1 -- 172.21.15.133:0/691819027 shutdown_connections 2024-09-11T23:46:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ca30640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f73840a2cf0 0x7f73840a51b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ca30640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7398106f20 0x7f739819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ca30640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73981027a0 0x7f73981989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ca30640 1 --2- 172.21.15.133:0/691819027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7398101df0 0x7f7398198490 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:55.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.714+0000 7f739ca30640 1 -- 172.21.15.133:0/691819027 >> 172.21.15.133:0/691819027 conn(0x7f73980fd8f0 msgr2=0x7f73980fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:55.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.715+0000 7f739ca30640 1 -- 172.21.15.133:0/691819027 shutdown_connections 2024-09-11T23:46:55.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:55.715+0000 7f739ca30640 1 -- 172.21.15.133:0/691819027 wait complete. 2024-09-11T23:46:56.705 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:46:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:56.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:56 smithi133 ceph-mon[31430]: pgmap v316: 97 pgs: 97 active+clean; 70 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 641 MiB/s wr, 1.45k op/s 2024-09-11T23:46:57.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:56 smithi178 ceph-mon[35882]: pgmap v316: 97 pgs: 97 active+clean; 70 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 641 MiB/s wr, 1.45k op/s 2024-09-11T23:46:57.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:56 smithi138 ceph-mon[35142]: pgmap v316: 97 pgs: 97 active+clean; 70 GiB data, 296 GiB used, 777 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 641 MiB/s wr, 1.45k op/s 2024-09-11T23:46:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.463+0000 7f179d2f0640 1 -- 172.21.15.133:0/2843087936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1798102dd0 msgr2=0x7f17981031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.463+0000 7f179d2f0640 1 --2- 172.21.15.133:0/2843087936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1798102dd0 0x7f17981031b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1788009930 tx=0x7f178802a420 comp rx=0 tx=0).stop 2024-09-11T23:46:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.463+0000 7f179d2f0640 1 -- 172.21.15.133:0/2843087936 shutdown_connections 2024-09-11T23:46:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.463+0000 7f179d2f0640 1 --2- 172.21.15.133:0/2843087936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1798108300 0x7f179810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.463+0000 7f179d2f0640 1 --2- 172.21.15.133:0/2843087936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1798103780 0x7f1798107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.463+0000 7f179d2f0640 1 --2- 172.21.15.133:0/2843087936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1798102dd0 0x7f17981031b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.463+0000 7f179d2f0640 1 -- 172.21.15.133:0/2843087936 >> 172.21.15.133:0/2843087936 conn(0x7f17980fc7d0 msgr2=0x7f17980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.463+0000 7f179d2f0640 1 -- 172.21.15.133:0/2843087936 shutdown_connections 2024-09-11T23:46:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.463+0000 7f179d2f0640 1 -- 172.21.15.133:0/2843087936 wait complete. 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.464+0000 7f179d2f0640 1 Processor -- start 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.464+0000 7f179d2f0640 1 -- start start 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.464+0000 7f179d2f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1798102dd0 0x7f1798100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.464+0000 7f179d2f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1798103780 0x7f1798100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.464+0000 7f179d2f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1798108300 0x7f17981014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.464+0000 7f179d2f0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f17981120b0 con 0x7f1798108300 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.464+0000 7f179d2f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1798111f30 con 0x7f1798102dd0 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.464+0000 7f179d2f0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1798112230 con 0x7f1798103780 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f17977fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1798108300 0x7f17981014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f17977fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1798108300 0x7f17981014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38102/0 (socket says 172.21.15.133:38102) 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f1796ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1798102dd0 0x7f1798100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f17977fe640 1 -- 172.21.15.133:0/808441863 learned_addr learned my addr 172.21.15.133:0/808441863 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f178e7fc640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1798103780 0x7f1798100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f17977fe640 1 -- 172.21.15.133:0/808441863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1798103780 msgr2=0x7f1798100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f17977fe640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1798103780 0x7f1798100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f17977fe640 1 -- 172.21.15.133:0/808441863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1798102dd0 msgr2=0x7f1798100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f17977fe640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1798102dd0 0x7f1798100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f17977fe640 1 -- 172.21.15.133:0/808441863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17981a6ab0 con 0x7f1798108300 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f1796ffd640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1798102dd0 0x7f1798100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f17977fe640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1798108300 0x7f17981014b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1790009fd0 tx=0x7f179000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f1794ff9640 1 -- 172.21.15.133:0/808441863 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790017070 con 0x7f1798108300 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f179d2f0640 1 -- 172.21.15.133:0/808441863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17981a6da0 con 0x7f1798108300 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f1794ff9640 1 -- 172.21.15.133:0/808441863 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1790003e50 con 0x7f1798108300 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f1794ff9640 1 -- 172.21.15.133:0/808441863 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17900106c0 con 0x7f1798108300 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.465+0000 7f179d2f0640 1 -- 172.21.15.133:0/808441863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17981a7260 con 0x7f1798108300 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.468+0000 7f179f6c9740 1 Processor -- start 2024-09-11T23:46:57.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.468+0000 7f1794ff9640 1 -- 172.21.15.133:0/808441863 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1790003970 con 0x7f1798108300 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.468+0000 7f179f6c9740 1 -- start start 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f11b40 0x55f947f10090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 0x55f947f10a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f179f6c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f947e762f0 con 0x55f947f1b370 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f179f6c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f947e1dcf0 con 0x55f947f105d0 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f179f6c9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f947e722b0 con 0x55f947f11b40 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f178e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f178e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f1d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38112/0 (socket says 172.21.15.133:38112) 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f178e7fc640 1 -- 172.21.15.133:0/1868248579 learned_addr learned my addr 172.21.15.133:0/1868248579 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f1796ffd640 1 --2- 172.21.15.133:0/1868248579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 0x55f947f10a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f178e7fc640 1 --2- 172.21.15.133:0/1868248579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f11b40 0x55f947f10090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.469+0000 7f1794ff9640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f17840a2cf0 0x7f17840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.470+0000 7f178e7fc640 1 -- 172.21.15.133:0/1868248579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f11b40 msgr2=0x55f947f10090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.470+0000 7f178e7fc640 1 --2- 172.21.15.133:0/1868248579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f11b40 0x55f947f10090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.470+0000 7f178e7fc640 1 -- 172.21.15.133:0/1868248579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 msgr2=0x55f947f10a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.470+0000 7f17977fe640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f17840a2cf0 0x7f17840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.470+0000 7f178e7fc640 1 --2- 172.21.15.133:0/1868248579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 0x55f947f10a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.470+0000 7f178e7fc640 1 -- 172.21.15.133:0/1868248579 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f947e023a0 con 0x55f947f1b370 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.470+0000 7f1794ff9640 1 -- 172.21.15.133:0/808441863 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f17900cf0c0 con 0x7f1798108300 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.470+0000 7f178e7fc640 1 --2- 172.21.15.133:0/1868248579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f1d760 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f17800116b0 tx=0x7f178000a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cc604ea25fd074 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.470+0000 7f178d7fa640 1 -- 172.21.15.133:0/1868248579 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780004020 con 0x55f947f1b370 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.470+0000 7f178d7fa640 1 -- 172.21.15.133:0/1868248579 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1780005500 con 0x55f947f1b370 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.470+0000 7f178d7fa640 1 -- 172.21.15.133:0/1868248579 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780004ce0 con 0x55f947f1b370 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f179f6c9740 1 -- 172.21.15.133:0/1868248579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 msgr2=0x55f947f1d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f17977fe640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f17840a2cf0 0x7f17840a51b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1790061660 tx=0x7f179006a600 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f179f6c9740 1 --2- 172.21.15.133:0/1868248579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f1d760 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f17800116b0 tx=0x7f178000a640 comp rx=0 tx=0).stop 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f179f6c9740 1 -- 172.21.15.133:0/1868248579 shutdown_connections 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f179f6c9740 1 --2- 172.21.15.133:0/1868248579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f1d760 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f179f6c9740 1 --2- 172.21.15.133:0/1868248579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 0x55f947f10a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f179f6c9740 1 --2- 172.21.15.133:0/1868248579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f11b40 0x55f947f10090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f179f6c9740 1 -- 172.21.15.133:0/1868248579 >> 172.21.15.133:0/1868248579 conn(0x55f947ed9310 msgr2=0x55f947f0cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f179f6c9740 1 -- 172.21.15.133:0/1868248579 shutdown_connections 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f179f6c9740 1 -- 172.21.15.133:0/1868248579 wait complete. 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f179f6c9740 1 -- start start 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.471+0000 7f179f6c9740 1 Processor -- start 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 0x55f947f1ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f11b40 0x55f947f17ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f1b370 0x55f947f185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f179f6c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f947f1e070 con 0x55f947f11b40 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f179f6c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f947e31060 con 0x55f947f105d0 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f179f6c9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f947f1e1f0 con 0x55f947f1b370 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f178e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 0x55f947f1ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f179f6c9740 10 client.0 _ll_register_callbacks cb 0x55f947b848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f178e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 0x55f947f1ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52894/0 (socket says 172.21.15.133:52894) 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f178e7fc640 1 -- 172.21.15.133:0/2596075849 learned_addr learned my addr 172.21.15.133:0/2596075849 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f1796ffd640 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f11b40 0x55f947f17ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f178e7fc640 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f1b370 0x55f947f185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f178e7fc640 1 -- 172.21.15.133:0/2596075849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f1b370 msgr2=0x55f947f185e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f178e7fc640 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f1b370 0x55f947f185e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f178e7fc640 1 -- 172.21.15.133:0/2596075849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f11b40 msgr2=0x55f947f17ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:57.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f178e7fc640 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f11b40 0x55f947f17ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f178e7fc640 1 -- 172.21.15.133:0/2596075849 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f947be73c0 con 0x55f947f105d0 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.472+0000 7f178e7fc640 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 0x55f947f1ad40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f17800054d0 tx=0x7f178001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e0ef70eb8ec415 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.473+0000 7f177e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.473+0000 7f177e7fc640 1 -- 172.21.15.133:0/2596075849 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780004c70 con 0x55f947f105d0 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.473+0000 7f177e7fc640 1 -- 172.21.15.133:0/2596075849 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f1780004e10 con 0x55f947f105d0 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.473+0000 7f179f6c9740 1 -- 172.21.15.133:0/2596075849 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1780005500 con 0x55f947f105d0 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.473+0000 7f177e7fc640 1 -- 172.21.15.133:0/2596075849 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1780034050 con 0x55f947f105d0 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.473+0000 7f179f6c9740 10 client.27542 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.473+0000 7f179f6c9740 1 -- 172.21.15.133:0/2596075849 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f947f29ac0 con 0x55f947f105d0 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.474+0000 7f177e7fc640 1 -- 172.21.15.133:0/2596075849 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1780005500 con 0x55f947f105d0 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.474+0000 7f177e7fc640 1 -- 172.21.15.133:0/2596075849 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f178001e5f0 con 0x55f947f105d0 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.474+0000 7f179f6c9740 10 client.27542 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.474+0000 7f179f6c9740 10 client.27542 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:46:57.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.474+0000 7f179f6c9740 20 client.27542 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:57.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.474+0000 7f179f6c9740 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55f947f2ae10 0x55f947f2d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.474+0000 7f179f6c9740 4 client.27542 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:57.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.474+0000 7f179f6c9740 1 -- 172.21.15.133:0/2596075849 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f947e83860 con 0x55f947f2ae10 2024-09-11T23:46:57.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.474+0000 7f1796ffd640 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55f947f2ae10 0x55f947f2d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.474+0000 7f177e7fc640 10 client.27542 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:46:57.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.474+0000 7f1796ffd640 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55f947f2ae10 0x55f947f2d200 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=685257234a65cc5e server_cookie=51b8a3731b0472aa in_seq=0 out_seq=0 2024-09-11T23:46:57.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f177e7fc640 1 -- 172.21.15.133:0/2596075849 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f947e83860 con 0x55f947f2ae10 2024-09-11T23:46:57.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f177e7fc640 10 client.27542 handle_command_reply: tid=0 2024-09-11T23:46:57.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f179f6c9740 1 client.27542 shutdown 2024-09-11T23:46:57.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f179f6c9740 1 -- 172.21.15.133:0/2596075849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 msgr2=0x55f947f1ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:57.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f179f6c9740 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 0x55f947f1ad40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f17800054d0 tx=0x7f178001a040 comp rx=0 tx=0).stop 2024-09-11T23:46:57.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f179f6c9740 1 -- 172.21.15.133:0/2596075849 shutdown_connections 2024-09-11T23:46:57.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f179f6c9740 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55f947f2ae10 0x55f947f2d200 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f179f6c9740 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f1b370 0x55f947f185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f179f6c9740 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f11b40 0x55f947f17ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f177e7fc640 0 client.27542 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:46:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f179f6c9740 1 --2- 172.21.15.133:0/2596075849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f105d0 0x55f947f1ad40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f179f6c9740 1 -- 172.21.15.133:0/2596075849 >> 172.21.15.133:0/2596075849 conn(0x55f947ed9310 msgr2=0x55f947f12960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.475+0000 7f179f6c9740 1 -- 172.21.15.133:0/2596075849 shutdown_connections 2024-09-11T23:46:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.476+0000 7f179f6c9740 1 -- 172.21.15.133:0/2596075849 wait complete. 2024-09-11T23:46:57.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.476+0000 7f179f6c9740 20 client.27542 trim_cache size 0 max 16384 2024-09-11T23:46:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f179f6c9740 1 Processor -- start 2024-09-11T23:46:57.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f179f6c9740 1 -- start start 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f105d0 0x55f947f0cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f11b40 0x55f947f27310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f1b370 0x55f947f16fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f179f6c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f17981afa30 con 0x55f947f105d0 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f179f6c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17981b1570 con 0x55f947f1b370 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f179f6c9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17981b0100 con 0x55f947f11b40 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f178e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f105d0 0x55f947f0cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f178e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f105d0 0x55f947f0cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38118/0 (socket says 172.21.15.133:38118) 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f178e7fc640 1 -- 172.21.15.133:0/127933045 learned_addr learned my addr 172.21.15.133:0/127933045 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f1796ffd640 1 --2- 172.21.15.133:0/127933045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f11b40 0x55f947f27310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f178e7fc640 1 --2- 172.21.15.133:0/127933045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f1b370 0x55f947f16fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f178e7fc640 1 -- 172.21.15.133:0/127933045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f11b40 msgr2=0x55f947f27310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f178e7fc640 1 --2- 172.21.15.133:0/127933045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f11b40 0x55f947f27310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f178e7fc640 1 -- 172.21.15.133:0/127933045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f1b370 msgr2=0x55f947f16fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.489+0000 7f178e7fc640 1 --2- 172.21.15.133:0/127933045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f1b370 0x55f947f16fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.490+0000 7f178e7fc640 1 -- 172.21.15.133:0/127933045 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f947f11300 con 0x55f947f105d0 2024-09-11T23:46:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.490+0000 7f1796ffd640 1 --2- 172.21.15.133:0/127933045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f11b40 0x55f947f27310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:46:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.490+0000 7f178e7fc640 1 --2- 172.21.15.133:0/127933045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f105d0 0x55f947f0cd70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1780001eb0 tx=0x7f17800049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495cf4f9c63bce78 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.490+0000 7f178cff9640 1 -- 172.21.15.133:0/127933045 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780005330 con 0x55f947f105d0 2024-09-11T23:46:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.490+0000 7f178cff9640 1 -- 172.21.15.133:0/127933045 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1780022070 con 0x55f947f105d0 2024-09-11T23:46:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.490+0000 7f178cff9640 1 -- 172.21.15.133:0/127933045 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178000f640 con 0x55f947f105d0 2024-09-11T23:46:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 -- 172.21.15.133:0/127933045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f105d0 msgr2=0x55f947f0cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 --2- 172.21.15.133:0/127933045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f105d0 0x55f947f0cd70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1780001eb0 tx=0x7f17800049f0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 -- 172.21.15.133:0/127933045 shutdown_connections 2024-09-11T23:46:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 --2- 172.21.15.133:0/127933045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f1b370 0x55f947f16fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 --2- 172.21.15.133:0/127933045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f11b40 0x55f947f27310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 --2- 172.21.15.133:0/127933045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f105d0 0x55f947f0cd70 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 -- 172.21.15.133:0/127933045 >> 172.21.15.133:0/127933045 conn(0x55f947ed9310 msgr2=0x55f947f168c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 -- 172.21.15.133:0/127933045 shutdown_connections 2024-09-11T23:46:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 -- 172.21.15.133:0/127933045 wait complete. 2024-09-11T23:46:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 -- start start 2024-09-11T23:46:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 Processor -- start 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f105d0 0x55f947f23480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f11b40 0x55f947f239c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f145e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f17981b0ea0 con 0x55f947f1b370 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17981b07d0 con 0x55f947f11b40 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17981ada30 con 0x55f947f105d0 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.491+0000 7f179f6c9740 10 client.0 _ll_register_callbacks cb 0x7f178000ad50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f178e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f145e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f178e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f145e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38132/0 (socket says 172.21.15.133:38132) 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f178e7fc640 1 -- 172.21.15.133:0/4050516625 learned_addr learned my addr 172.21.15.133:0/4050516625 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f1796ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f11b40 0x55f947f239c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f178e7fc640 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f105d0 0x55f947f23480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f1796ffd640 1 -- 172.21.15.133:0/4050516625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f105d0 msgr2=0x55f947f23480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f1796ffd640 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f105d0 0x55f947f23480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f1796ffd640 1 -- 172.21.15.133:0/4050516625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 msgr2=0x55f947f145e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f1796ffd640 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f145e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f1796ffd640 1 -- 172.21.15.133:0/4050516625 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f947f14c20 con 0x55f947f11b40 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f178e7fc640 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f1796ffd640 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f11b40 0x55f947f239c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1788005940 tx=0x7f17880041f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da429d80adddfaa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f177e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f177e7fc640 1 -- 172.21.15.133:0/4050516625 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788035040 con 0x55f947f11b40 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f179f6c9740 1 -- 172.21.15.133:0/4050516625 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1780022070 con 0x55f947f11b40 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f177e7fc640 1 -- 172.21.15.133:0/4050516625 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f1788031af0 con 0x55f947f11b40 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f177e7fc640 1 -- 172.21.15.133:0/4050516625 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1788059050 con 0x55f947f11b40 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.492+0000 7f179f6c9740 10 client.27548 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.493+0000 7f179f6c9740 1 -- 172.21.15.133:0/4050516625 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f947f95330 con 0x55f947f11b40 2024-09-11T23:46:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.493+0000 7f177e7fc640 1 -- 172.21.15.133:0/4050516625 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1780022070 con 0x55f947f11b40 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.493+0000 7f177e7fc640 1 -- 172.21.15.133:0/4050516625 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f178804a430 con 0x55f947f11b40 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.493+0000 7f179f6c9740 10 client.27548 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.493+0000 7f179f6c9740 10 client.27548 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.493+0000 7f179f6c9740 20 client.27548 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.493+0000 7f179f6c9740 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55f947f2ae10 0x55f947f97a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.493+0000 7f179f6c9740 4 client.27548 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.493+0000 7f179f6c9740 1 -- 172.21.15.133:0/4050516625 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f947e7dd70 con 0x55f947f2ae10 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.493+0000 7f178e7fc640 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55f947f2ae10 0x55f947f97a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f177e7fc640 10 client.27548 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f178e7fc640 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55f947f2ae10 0x55f947f97a40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bddea075ce79940f server_cookie=9b7e918eb7177d9 in_seq=0 out_seq=0 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f177e7fc640 1 -- 172.21.15.133:0/4050516625 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f947e7dd70 con 0x55f947f2ae10 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f177e7fc640 10 client.27548 handle_command_reply: tid=0 2024-09-11T23:46:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 1 client.27548 shutdown 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 1 -- 172.21.15.133:0/4050516625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f11b40 msgr2=0x55f947f239c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f11b40 0x55f947f239c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1788005940 tx=0x7f17880041f0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 1 -- 172.21.15.133:0/4050516625 shutdown_connections 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55f947f2ae10 0x55f947f97a40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f947f1b370 0x55f947f145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f947f11b40 0x55f947f239c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f177e7fc640 0 client.27548 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 1 --2- 172.21.15.133:0/4050516625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f947f105d0 0x55f947f23480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 1 -- 172.21.15.133:0/4050516625 >> 172.21.15.133:0/4050516625 conn(0x55f947ed9310 msgr2=0x55f947f12080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 1 -- 172.21.15.133:0/4050516625 shutdown_connections 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 1 -- 172.21.15.133:0/4050516625 wait complete. 2024-09-11T23:46:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179f6c9740 20 client.27548 trim_cache size 0 max 16384 2024-09-11T23:46:57.500 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:46:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.494+0000 7f179d2f0640 1 -- 172.21.15.133:0/808441863 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f17840a2cf0 msgr2=0x7f17840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.495+0000 7f179d2f0640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f17840a2cf0 0x7f17840a51b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1790061660 tx=0x7f179006a600 comp rx=0 tx=0).stop 2024-09-11T23:46:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.495+0000 7f179d2f0640 1 -- 172.21.15.133:0/808441863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1798108300 msgr2=0x7f17981014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.495+0000 7f179d2f0640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1798108300 0x7f17981014b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1790009fd0 tx=0x7f179000b260 comp rx=0 tx=0).stop 2024-09-11T23:46:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.495+0000 7f179d2f0640 1 -- 172.21.15.133:0/808441863 shutdown_connections 2024-09-11T23:46:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.495+0000 7f179d2f0640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f17840a2cf0 0x7f17840a51b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.495+0000 7f179d2f0640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1798108300 0x7f17981014b0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.495+0000 7f179d2f0640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1798103780 0x7f1798100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.495+0000 7f179d2f0640 1 --2- 172.21.15.133:0/808441863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1798102dd0 0x7f1798100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.495+0000 7f179d2f0640 1 -- 172.21.15.133:0/808441863 >> 172.21.15.133:0/808441863 conn(0x7f17980fc7d0 msgr2=0x7f1798105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.495+0000 7f179d2f0640 1 -- 172.21.15.133:0/808441863 shutdown_connections 2024-09-11T23:46:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:57.495+0000 7f179d2f0640 1 -- 172.21.15.133:0/808441863 wait complete. 2024-09-11T23:46:58.529 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:46:58.779 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:46:58.901 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:46:58 smithi133 ceph-mon[31430]: pgmap v317: 97 pgs: 97 active+clean; 70 GiB data, 296 GiB used, 776 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 536 MiB/s wr, 915 op/s 2024-09-11T23:46:59.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:46:58 smithi178 ceph-mon[35882]: pgmap v317: 97 pgs: 97 active+clean; 70 GiB data, 296 GiB used, 776 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 536 MiB/s wr, 915 op/s 2024-09-11T23:46:59.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:46:58 smithi138 ceph-mon[35142]: pgmap v317: 97 pgs: 97 active+clean; 70 GiB data, 296 GiB used, 776 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 536 MiB/s wr, 915 op/s 2024-09-11T23:46:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.250+0000 7fed7632f640 1 -- 172.21.15.133:0/548917532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed701036e0 msgr2=0x7fed70107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.250+0000 7fed7632f640 1 --2- 172.21.15.133:0/548917532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed701036e0 0x7fed70107b90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fed58009930 tx=0x7fed5802a420 comp rx=0 tx=0).stop 2024-09-11T23:46:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.250+0000 7fed7632f640 1 -- 172.21.15.133:0/548917532 shutdown_connections 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.250+0000 7fed7632f640 1 --2- 172.21.15.133:0/548917532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed701082a0 0x7fed7010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.250+0000 7fed7632f640 1 --2- 172.21.15.133:0/548917532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed701036e0 0x7fed70107b90 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.250+0000 7fed7632f640 1 --2- 172.21.15.133:0/548917532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed70102d30 0x7fed70103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.250+0000 7fed7632f640 1 -- 172.21.15.133:0/548917532 >> 172.21.15.133:0/548917532 conn(0x7fed700fc7d0 msgr2=0x7fed700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.250+0000 7fed7632f640 1 -- 172.21.15.133:0/548917532 shutdown_connections 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.250+0000 7fed7632f640 1 -- 172.21.15.133:0/548917532 wait complete. 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed7632f640 1 Processor -- start 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed7632f640 1 -- start start 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed7632f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed70102d30 0x7fed70198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed7632f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed701036e0 0x7fed70198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed7632f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed701082a0 0x7fed7019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed7632f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed701120c0 con 0x7fed701082a0 2024-09-11T23:46:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed7632f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed70111f40 con 0x7fed70102d30 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed7632f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed70112240 con 0x7fed701036e0 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed748cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed701082a0 0x7fed7019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed748cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed701082a0 0x7fed7019cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38140/0 (socket says 172.21.15.133:38140) 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed748cc640 1 -- 172.21.15.133:0/2392622810 learned_addr learned my addr 172.21.15.133:0/2392622810 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed6f7fe640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed701036e0 0x7fed70198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.251+0000 7fed6ffff640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed70102d30 0x7fed70198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.252+0000 7fed748cc640 1 -- 172.21.15.133:0/2392622810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed701036e0 msgr2=0x7fed70198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.252+0000 7fed748cc640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed701036e0 0x7fed70198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.252+0000 7fed748cc640 1 -- 172.21.15.133:0/2392622810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed70102d30 msgr2=0x7fed70198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.252+0000 7fed748cc640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed70102d30 0x7fed70198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.252+0000 7fed748cc640 1 -- 172.21.15.133:0/2392622810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7019d5f0 con 0x7fed701082a0 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.252+0000 7fed748cc640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed701082a0 0x7fed7019cf10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fed6800eaa0 tx=0x7fed6800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.252+0000 7fed6d7fa640 1 -- 172.21.15.133:0/2392622810 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68004d20 con 0x7fed701082a0 2024-09-11T23:46:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.252+0000 7fed7632f640 1 -- 172.21.15.133:0/2392622810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7019d8e0 con 0x7fed701082a0 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.252+0000 7fed6d7fa640 1 -- 172.21.15.133:0/2392622810 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed68005020 con 0x7fed701082a0 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.252+0000 7fed6d7fa640 1 -- 172.21.15.133:0/2392622810 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed680116d0 con 0x7fed701082a0 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.252+0000 7fed7632f640 1 -- 172.21.15.133:0/2392622810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed701a51c0 con 0x7fed701082a0 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.254+0000 7fed6d7fa640 1 -- 172.21.15.133:0/2392622810 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fed680051d0 con 0x7fed701082a0 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed78708740 1 Processor -- start 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed78708740 1 -- start start 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed6d7fa640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed5c0a2cf0 0x7fed5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed78708740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x558576020090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed78708740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585760205d0 0x558576020a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed78708740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 0x55857602d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed78708740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558575f862f0 con 0x55857602b370 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed78708740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558575f2dcf0 con 0x5585760205d0 2024-09-11T23:46:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed78708740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558575f822b0 con 0x558576021b40 2024-09-11T23:46:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed6f7fe640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed5c0a2cf0 0x7fed5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed6d7fa640 1 -- 172.21.15.133:0/2392622810 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fed680cf5f0 con 0x7fed701082a0 2024-09-11T23:46:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed748cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 0x55857602d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed748cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 0x55857602d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38154/0 (socket says 172.21.15.133:38154) 2024-09-11T23:46:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed748cc640 1 -- 172.21.15.133:0/2205946243 learned_addr learned my addr 172.21.15.133:0/2205946243 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed6f7fe640 1 --2- 172.21.15.133:0/2205946243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585760205d0 0x558576020a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.255+0000 7fed6ffff640 1 --2- 172.21.15.133:0/2205946243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x558576020090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.256+0000 7fed6ffff640 1 -- 172.21.15.133:0/2205946243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585760205d0 msgr2=0x558576020a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.256+0000 7fed6ffff640 1 --2- 172.21.15.133:0/2205946243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585760205d0 0x558576020a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.256+0000 7fed6ffff640 1 -- 172.21.15.133:0/2205946243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 msgr2=0x55857602d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.256+0000 7fed6ffff640 1 --2- 172.21.15.133:0/2205946243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 0x55857602d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.256+0000 7fed6ffff640 1 -- 172.21.15.133:0/2205946243 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558575f123a0 con 0x558576021b40 2024-09-11T23:46:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.256+0000 7fed6f7fe640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed5c0a2cf0 0x7fed5c0a51b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fed58005940 tx=0x7fed58032f80 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.256+0000 7fed6ffff640 1 --2- 172.21.15.133:0/2205946243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x558576020090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fed600099a0 tx=0x7fed60009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3ab8c4718629d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.256+0000 7fed667fc640 1 -- 172.21.15.133:0/2205946243 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60017070 con 0x558576021b40 2024-09-11T23:46:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.256+0000 7fed667fc640 1 -- 172.21.15.133:0/2205946243 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed60013470 con 0x558576021b40 2024-09-11T23:46:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.256+0000 7fed667fc640 1 -- 172.21.15.133:0/2205946243 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60012630 con 0x558576021b40 2024-09-11T23:46:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.257+0000 7fed78708740 1 -- 172.21.15.133:0/2205946243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 msgr2=0x558576020090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.257+0000 7fed78708740 1 --2- 172.21.15.133:0/2205946243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x558576020090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fed600099a0 tx=0x7fed60009da0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.257+0000 7fed78708740 1 -- 172.21.15.133:0/2205946243 shutdown_connections 2024-09-11T23:46:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.257+0000 7fed78708740 1 --2- 172.21.15.133:0/2205946243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 0x55857602d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.387 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:46:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.257+0000 7fed78708740 1 --2- 172.21.15.133:0/2205946243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585760205d0 0x558576020a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.257+0000 7fed78708740 1 --2- 172.21.15.133:0/2205946243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x558576020090 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.257+0000 7fed78708740 1 -- 172.21.15.133:0/2205946243 >> 172.21.15.133:0/2205946243 conn(0x558575fe9310 msgr2=0x55857601cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.257+0000 7fed78708740 1 -- 172.21.15.133:0/2205946243 shutdown_connections 2024-09-11T23:46:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.257+0000 7fed78708740 1 -- 172.21.15.133:0/2205946243 wait complete. 2024-09-11T23:46:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.257+0000 7fed78708740 1 -- start start 2024-09-11T23:46:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.257+0000 7fed78708740 1 Processor -- start 2024-09-11T23:46:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed78708740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585760205d0 0x55857602ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed78708740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x558576027f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed78708740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 0x5585760284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed78708740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55857602e070 con 0x55857602b370 2024-09-11T23:46:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed78708740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558575f41060 con 0x5585760205d0 2024-09-11T23:46:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed78708740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55857602e1f0 con 0x558576021b40 2024-09-11T23:46:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed78708740 10 client.0 _ll_register_callbacks cb 0x558575c948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed748cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 0x5585760284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed748cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 0x5585760284e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38162/0 (socket says 172.21.15.133:38162) 2024-09-11T23:46:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed748cc640 1 -- 172.21.15.133:0/902205338 learned_addr learned my addr 172.21.15.133:0/902205338 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed6ffff640 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585760205d0 0x55857602ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed6f7fe640 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x558576027f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed748cc640 1 -- 172.21.15.133:0/902205338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 msgr2=0x558576027f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed748cc640 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x558576027f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed748cc640 1 -- 172.21.15.133:0/902205338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585760205d0 msgr2=0x55857602ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed748cc640 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585760205d0 0x55857602ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed748cc640 1 -- 172.21.15.133:0/902205338 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558575cf73c0 con 0x55857602b370 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed748cc640 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 0x5585760284e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fed680cce30 tx=0x7fed6809b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3278ae9bd83d042a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.258+0000 7fed4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed4f7fe640 1 -- 172.21.15.133:0/902205338 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6801f040 con 0x55857602b370 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed4f7fe640 1 -- 172.21.15.133:0/902205338 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fed6809afa0 con 0x55857602b370 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed78708740 1 -- 172.21.15.133:0/902205338 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed60013470 con 0x55857602b370 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed4f7fe640 1 -- 172.21.15.133:0/902205338 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed680a47c0 con 0x55857602b370 2024-09-11T23:46:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed78708740 10 client.18009 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed78708740 1 -- 172.21.15.133:0/902205338 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558576039ae0 con 0x55857602b370 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed4f7fe640 1 -- 172.21.15.133:0/902205338 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed60013470 con 0x55857602b370 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed4f7fe640 1 -- 172.21.15.133:0/902205338 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed6800f040 con 0x55857602b370 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed78708740 10 client.18009 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed78708740 10 client.18009 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed78708740 20 client.18009 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed78708740 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55857603aef0 0x55857603d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed78708740 4 client.18009 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed78708740 1 -- 172.21.15.133:0/902205338 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558575f93860 con 0x55857603aef0 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.259+0000 7fed6ffff640 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55857603aef0 0x55857603d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.260+0000 7fed4f7fe640 10 client.18009 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.260+0000 7fed6ffff640 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55857603aef0 0x55857603d2e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d23789ef2fc9747 server_cookie=8f3b45493b98c090 in_seq=0 out_seq=0 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed4f7fe640 1 -- 172.21.15.133:0/902205338 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558575f93860 con 0x55857603aef0 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed4f7fe640 10 client.18009 handle_command_reply: tid=0 2024-09-11T23:46:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 1 client.18009 shutdown 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 1 -- 172.21.15.133:0/902205338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 msgr2=0x5585760284e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 0x5585760284e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fed680cce30 tx=0x7fed6809b5d0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 1 -- 172.21.15.133:0/902205338 shutdown_connections 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55857603aef0 0x55857603d2e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55857602b370 0x5585760284e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x558576027f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 1 --2- 172.21.15.133:0/902205338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585760205d0 0x55857602ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed4f7fe640 0 client.18009 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 1 -- 172.21.15.133:0/902205338 >> 172.21.15.133:0/902205338 conn(0x558575fe9310 msgr2=0x558576022960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 1 -- 172.21.15.133:0/902205338 shutdown_connections 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 1 -- 172.21.15.133:0/902205338 wait complete. 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.261+0000 7fed78708740 20 client.18009 trim_cache size 0 max 16384 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.274+0000 7fed78708740 1 Processor -- start 2024-09-11T23:46:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed78708740 1 -- start start 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed78708740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576037310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed78708740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558576021b40 0x558576023120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed78708740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55857602b370 0x558576023660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed78708740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed701ad800 con 0x5585760205d0 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed78708740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed701af340 con 0x558576021b40 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed78708740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed701aded0 con 0x55857602b370 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed6ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576037310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed6ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576037310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38176/0 (socket says 172.21.15.133:38176) 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed6ffff640 1 -- 172.21.15.133:0/1704909424 learned_addr learned my addr 172.21.15.133:0/1704909424 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed6f7fe640 1 --2- 172.21.15.133:0/1704909424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558576021b40 0x558576023120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed6ffff640 1 -- 172.21.15.133:0/1704909424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55857602b370 msgr2=0x558576023660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed6ffff640 1 --2- 172.21.15.133:0/1704909424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55857602b370 0x558576023660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed6ffff640 1 -- 172.21.15.133:0/1704909424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558576021b40 msgr2=0x558576023120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:59.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed6ffff640 1 --2- 172.21.15.133:0/1704909424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558576021b40 0x558576023120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed6ffff640 1 -- 172.21.15.133:0/1704909424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55857603d960 con 0x5585760205d0 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed6f7fe640 1 --2- 172.21.15.133:0/1704909424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558576021b40 0x558576023120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed6ffff640 1 --2- 172.21.15.133:0/1704909424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576037310 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fed6000ec20 tx=0x7fed60009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d713a7323a409934 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed65ffb640 1 -- 172.21.15.133:0/1704909424 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60017040 con 0x5585760205d0 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed65ffb640 1 -- 172.21.15.133:0/1704909424 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed6002b070 con 0x5585760205d0 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.275+0000 7fed65ffb640 1 -- 172.21.15.133:0/1704909424 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60013890 con 0x5585760205d0 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.276+0000 7fed78708740 1 -- 172.21.15.133:0/1704909424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 msgr2=0x558576037310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.276+0000 7fed78708740 1 --2- 172.21.15.133:0/1704909424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576037310 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fed6000ec20 tx=0x7fed60009110 comp rx=0 tx=0).stop 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.276+0000 7fed78708740 1 -- 172.21.15.133:0/1704909424 shutdown_connections 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.276+0000 7fed78708740 1 --2- 172.21.15.133:0/1704909424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55857602b370 0x558576023660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.276+0000 7fed78708740 1 --2- 172.21.15.133:0/1704909424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558576021b40 0x558576023120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.276+0000 7fed78708740 1 --2- 172.21.15.133:0/1704909424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576037310 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.276+0000 7fed78708740 1 -- 172.21.15.133:0/1704909424 >> 172.21.15.133:0/1704909424 conn(0x558575fe9310 msgr2=0x558576027f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.276+0000 7fed78708740 1 -- 172.21.15.133:0/1704909424 shutdown_connections 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.276+0000 7fed78708740 1 -- 172.21.15.133:0/1704909424 wait complete. 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.276+0000 7fed78708740 1 -- start start 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed78708740 1 Processor -- start 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed78708740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576033c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed78708740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x55857602efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed78708740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55857602b370 0x55857602f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed78708740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed701aec70 con 0x5585760205d0 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed78708740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed701ae5a0 con 0x55857602b370 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed78708740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed701ab800 con 0x558576021b40 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed78708740 10 client.0 _ll_register_callbacks cb 0x7fed6809c250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed6ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576033c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed6ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576033c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38178/0 (socket says 172.21.15.133:38178) 2024-09-11T23:46:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed6ffff640 1 -- 172.21.15.133:0/645938786 learned_addr learned my addr 172.21.15.133:0/645938786 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed748cc640 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55857602b370 0x55857602f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed6f7fe640 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x55857602efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed6ffff640 1 -- 172.21.15.133:0/645938786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 msgr2=0x55857602efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed6ffff640 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x55857602efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed6ffff640 1 -- 172.21.15.133:0/645938786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55857602b370 msgr2=0x55857602f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed6ffff640 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55857602b370 0x55857602f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.277+0000 7fed6ffff640 1 -- 172.21.15.133:0/645938786 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55857602fd60 con 0x5585760205d0 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed6ffff640 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576033c80 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fed60026fd0 tx=0x7fed6000ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749c078d8cf95742 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed4f7fe640 1 -- 172.21.15.133:0/645938786 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60020070 con 0x5585760205d0 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed4f7fe640 1 -- 172.21.15.133:0/645938786 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fed60005bd0 con 0x5585760205d0 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed78708740 1 -- 172.21.15.133:0/645938786 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed6002b070 con 0x5585760205d0 2024-09-11T23:46:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed4f7fe640 1 -- 172.21.15.133:0/645938786 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed60017040 con 0x5585760205d0 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed78708740 10 client.18015 fetch_fsmap learned FSMap version 30 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed78708740 1 -- 172.21.15.133:0/645938786 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5585760a5b90 con 0x5585760205d0 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed4f7fe640 1 -- 172.21.15.133:0/645938786 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed6002b070 con 0x5585760205d0 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed4f7fe640 1 -- 172.21.15.133:0/645938786 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed60006200 con 0x5585760205d0 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed78708740 10 client.18015 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed78708740 10 client.18015 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed78708740 20 client.18015 populate_metadata read hostname 'smithi133' 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed78708740 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5585760a5ec0 0x5585760a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed78708740 4 client.18015 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.278+0000 7fed78708740 1 -- 172.21.15.133:0/645938786 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558575f8dd70 con 0x5585760a5ec0 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed6f7fe640 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5585760a5ec0 0x5585760a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed6f7fe640 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5585760a5ec0 0x5585760a82b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1191b2d2a2a24c9 server_cookie=61b69ff732a74edf in_seq=0 out_seq=0 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed4f7fe640 10 client.18015 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:46:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed4f7fe640 1 -- 172.21.15.133:0/645938786 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558575f8dd70 con 0x5585760a5ec0 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed4f7fe640 10 client.18015 handle_command_reply: tid=0 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed78708740 1 client.18015 shutdown 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed78708740 1 -- 172.21.15.133:0/645938786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 msgr2=0x558576033c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed78708740 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576033c80 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fed60026fd0 tx=0x7fed6000ede0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed78708740 1 -- 172.21.15.133:0/645938786 shutdown_connections 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed78708740 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5585760a5ec0 0x5585760a82b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed78708740 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55857602b370 0x55857602f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed78708740 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558576021b40 0x55857602efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed4f7fe640 0 client.18015 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed78708740 1 --2- 172.21.15.133:0/645938786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585760205d0 0x558576033c80 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.279+0000 7fed78708740 1 -- 172.21.15.133:0/645938786 >> 172.21.15.133:0/645938786 conn(0x558575fe9310 msgr2=0x55857601b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed78708740 1 -- 172.21.15.133:0/645938786 shutdown_connections 2024-09-11T23:46:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed78708740 1 -- 172.21.15.133:0/645938786 wait complete. 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed78708740 20 client.18015 trim_cache size 0 max 16384 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 -- 172.21.15.133:0/2392622810 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed5c0a2cf0 msgr2=0x7fed5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed5c0a2cf0 0x7fed5c0a51b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fed58005940 tx=0x7fed58032f80 comp rx=0 tx=0).stop 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 -- 172.21.15.133:0/2392622810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed701082a0 msgr2=0x7fed7019cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed701082a0 0x7fed7019cf10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fed6800eaa0 tx=0x7fed6800eea0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 -- 172.21.15.133:0/2392622810 shutdown_connections 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fed5c0a2cf0 0x7fed5c0a51b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed701082a0 0x7fed7019cf10 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed701036e0 0x7fed70198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 --2- 172.21.15.133:0/2392622810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed70102d30 0x7fed70198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 -- 172.21.15.133:0/2392622810 >> 172.21.15.133:0/2392622810 conn(0x7fed700fc7d0 msgr2=0x7fed70105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 -- 172.21.15.133:0/2392622810 shutdown_connections 2024-09-11T23:46:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:46:59.280+0000 7fed7632f640 1 -- 172.21.15.133:0/2392622810 wait complete. 2024-09-11T23:47:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.204+0000 7faeadad7640 1 -- 172.21.15.133:0/3746301644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8101350 msgr2=0x7faea810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.204+0000 7faeadad7640 1 --2- 172.21.15.133:0/3746301644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8101350 0x7faea810d880 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faea0009ca0 tx=0x7faea002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.204+0000 7faeadad7640 1 -- 172.21.15.133:0/3746301644 shutdown_connections 2024-09-11T23:47:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.204+0000 7faeadad7640 1 --2- 172.21.15.133:0/3746301644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8101350 0x7faea810d880 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.204+0000 7faeadad7640 1 --2- 172.21.15.133:0/3746301644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea81009b0 0x7faea8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.204+0000 7faeadad7640 1 --2- 172.21.15.133:0/3746301644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea81069c0 0x7faea8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.204+0000 7faeadad7640 1 -- 172.21.15.133:0/3746301644 >> 172.21.15.133:0/3746301644 conn(0x7faea8077990 msgr2=0x7faea80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.204+0000 7faeadad7640 1 -- 172.21.15.133:0/3746301644 shutdown_connections 2024-09-11T23:47:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.204+0000 7faeadad7640 1 -- 172.21.15.133:0/3746301644 wait complete. 2024-09-11T23:47:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.204+0000 7faeadad7640 1 Processor -- start 2024-09-11T23:47:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faeadad7640 1 -- start start 2024-09-11T23:47:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faeadad7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea81009b0 0x7faea8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faeadad7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8101350 0x7faea8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faeadad7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea81069c0 0x7faea81a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faeadad7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faea8110e50 con 0x7faea81069c0 2024-09-11T23:47:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faeadad7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faea8110cd0 con 0x7faea8101350 2024-09-11T23:47:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faeadad7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faea8110fd0 con 0x7faea81009b0 2024-09-11T23:47:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faea7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea81069c0 0x7faea81a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faea6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8101350 0x7faea8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faea7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea81069c0 0x7faea81a41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38188/0 (socket says 172.21.15.133:38188) 2024-09-11T23:47:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faea77fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea81009b0 0x7faea8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faea7fff640 1 -- 172.21.15.133:0/3719401694 learned_addr learned my addr 172.21.15.133:0/3719401694 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faea6ffd640 1 -- 172.21.15.133:0/3719401694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea81009b0 msgr2=0x7faea8199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faea6ffd640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea81009b0 0x7faea8199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faea6ffd640 1 -- 172.21.15.133:0/3719401694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea81069c0 msgr2=0x7faea81a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faea6ffd640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea81069c0 0x7faea81a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faea6ffd640 1 -- 172.21.15.133:0/3719401694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea81a46e0 con 0x7faea8101350 2024-09-11T23:47:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.205+0000 7faea77fe640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea81009b0 0x7faea8199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.206+0000 7faea6ffd640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8101350 0x7faea8199660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0099c0 tx=0x7fae9c009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.206+0000 7faea4ff9640 1 -- 172.21.15.133:0/3719401694 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0078c0 con 0x7faea8101350 2024-09-11T23:47:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.206+0000 7faea4ff9640 1 -- 172.21.15.133:0/3719401694 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae9c005020 con 0x7faea8101350 2024-09-11T23:47:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.206+0000 7faeadad7640 1 -- 172.21.15.133:0/3719401694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea8196f10 con 0x7faea8101350 2024-09-11T23:47:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.206+0000 7faea4ff9640 1 -- 172.21.15.133:0/3719401694 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c004d90 con 0x7faea8101350 2024-09-11T23:47:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.206+0000 7faeadad7640 1 -- 172.21.15.133:0/3719401694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea81974b0 con 0x7faea8101350 2024-09-11T23:47:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.208+0000 7faea4ff9640 1 -- 172.21.15.133:0/3719401694 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fae9c01a420 con 0x7faea8101350 2024-09-11T23:47:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faeb0872740 1 Processor -- start 2024-09-11T23:47:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faeb0872740 1 -- start start 2024-09-11T23:47:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faeb0872740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 0x55eec4916ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faeb0872740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec4921770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faeb0872740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec49240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faeb0872740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55eec487b2f0 con 0x55eec4917410 2024-09-11T23:47:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faeb0872740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55eec4823270 con 0x55eec4916af0 2024-09-11T23:47:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faeb0872740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eec48772b0 con 0x55eec4921cb0 2024-09-11T23:47:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faea77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec4921770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faea7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 0x55eec4916ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faea6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec49240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faea7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 0x55eec4916ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53012/0 (socket says 172.21.15.133:53012) 2024-09-11T23:47:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faea7fff640 1 -- 172.21.15.133:0/1678126618 learned_addr learned my addr 172.21.15.133:0/1678126618 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.209+0000 7faea4ff9640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae900a2e20 0x7fae900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7faea7fff640 1 -- 172.21.15.133:0/1678126618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 msgr2=0x55eec49240a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7faea7fff640 1 --2- 172.21.15.133:0/1678126618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec49240a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7faea7fff640 1 -- 172.21.15.133:0/1678126618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 msgr2=0x55eec4921770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7faea7fff640 1 --2- 172.21.15.133:0/1678126618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec4921770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7faea7fff640 1 -- 172.21.15.133:0/1678126618 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec4822340 con 0x55eec4916af0 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7faea6ffd640 1 --2- 172.21.15.133:0/1678126618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec49240a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7faea4ff9640 1 -- 172.21.15.133:0/3719401694 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fae9c0d2810 con 0x7faea8101350 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7faea7fff640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae900a2e20 0x7fae900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7faea7fff640 1 --2- 172.21.15.133:0/1678126618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 0x55eec4916ed0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faea0005fb0 tx=0x7faea002f3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa9ac9c6a602fbb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7fae8dffb640 1 -- 172.21.15.133:0/1678126618 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0031510 con 0x55eec4916af0 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7fae8dffb640 1 -- 172.21.15.133:0/1678126618 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea0036070 con 0x55eec4916af0 2024-09-11T23:47:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.210+0000 7fae8dffb640 1 -- 172.21.15.133:0/1678126618 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0004360 con 0x55eec4916af0 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.211+0000 7faeb0872740 1 -- 172.21.15.133:0/1678126618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 msgr2=0x55eec4916ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.211+0000 7faeb0872740 1 --2- 172.21.15.133:0/1678126618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 0x55eec4916ed0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faea0005fb0 tx=0x7faea002f3e0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.211+0000 7faea7fff640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae900a2e20 0x7fae900a52e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faea0055000 tx=0x7faea0052000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.211+0000 7faeb0872740 1 -- 172.21.15.133:0/1678126618 shutdown_connections 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.211+0000 7faeb0872740 1 --2- 172.21.15.133:0/1678126618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec49240a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.211+0000 7faeb0872740 1 --2- 172.21.15.133:0/1678126618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec4921770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.211+0000 7faeb0872740 1 --2- 172.21.15.133:0/1678126618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 0x55eec4916ed0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.211+0000 7faeb0872740 1 -- 172.21.15.133:0/1678126618 >> 172.21.15.133:0/1678126618 conn(0x55eec4913060 msgr2=0x55eec4913440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.211+0000 7faeb0872740 1 -- 172.21.15.133:0/1678126618 shutdown_connections 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.211+0000 7faeb0872740 1 -- 172.21.15.133:0/1678126618 wait complete. 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.211+0000 7faeb0872740 1 -- start start 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faeb0872740 1 Processor -- start 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faeb0872740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4916af0 0x55eec491ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faeb0872740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec491f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faeb0872740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4921cb0 0x55eec491f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faeb0872740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55eec4935650 con 0x55eec4917410 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faeb0872740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55eec4804290 con 0x55eec4921cb0 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faeb0872740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eec49357d0 con 0x55eec4916af0 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faea6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec491f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faeb0872740 10 client.0 _ll_register_callbacks cb 0x55eec4835140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faea6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec491f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38202/0 (socket says 172.21.15.133:38202) 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faea6ffd640 1 -- 172.21.15.133:0/3363588396 learned_addr learned my addr 172.21.15.133:0/3363588396 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faea77fe640 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4916af0 0x55eec491ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faea7fff640 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4921cb0 0x55eec491f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faea6ffd640 1 -- 172.21.15.133:0/3363588396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4916af0 msgr2=0x55eec491ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faea6ffd640 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4916af0 0x55eec491ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faea6ffd640 1 -- 172.21.15.133:0/3363588396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4921cb0 msgr2=0x55eec491f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faea6ffd640 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4921cb0 0x55eec491f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.212+0000 7faea6ffd640 1 -- 172.21.15.133:0/3363588396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec458e210 con 0x55eec4917410 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.213+0000 7faea6ffd640 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec491f1a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fae9c09b280 tx=0x7fae9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd50d067140652b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.213+0000 7fae86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.213+0000 7fae86ffd640 1 -- 172.21.15.133:0/3363588396 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c09ee20 con 0x55eec4917410 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.213+0000 7fae86ffd640 1 -- 172.21.15.133:0/3363588396 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fae9c005540 con 0x55eec4917410 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.213+0000 7faeb0872740 1 -- 172.21.15.133:0/3363588396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea0036070 con 0x55eec4917410 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.213+0000 7fae86ffd640 1 -- 172.21.15.133:0/3363588396 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae9c0a3120 con 0x55eec4917410 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.213+0000 7faeb0872740 10 client.18024 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.213+0000 7faeb0872740 1 -- 172.21.15.133:0/3363588396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eec465a620 con 0x55eec4917410 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.213+0000 7fae86ffd640 1 -- 172.21.15.133:0/3363588396 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea0036070 con 0x55eec4917410 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.213+0000 7fae86ffd640 1 -- 172.21.15.133:0/3363588396 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fae9c09e300 con 0x55eec4917410 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.214+0000 7faeb0872740 10 client.18024 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.214+0000 7faeb0872740 10 client.18024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.214+0000 7faeb0872740 20 client.18024 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.214+0000 7faeb0872740 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55eec4942a10 0x55eec4944e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.214+0000 7faeb0872740 4 client.18024 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.214+0000 7faeb0872740 1 -- 172.21.15.133:0/3363588396 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eec4888a20 con 0x55eec4942a10 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.214+0000 7faea77fe640 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55eec4942a10 0x55eec4944e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.214+0000 7faea77fe640 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55eec4942a10 0x55eec4944e00 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65f6ba1abac1d27 server_cookie=cb47a4646bd8538e in_seq=0 out_seq=0 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.214+0000 7fae86ffd640 10 client.18024 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7fae86ffd640 1 -- 172.21.15.133:0/3363588396 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eec4888a20 con 0x55eec4942a10 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7fae86ffd640 10 client.18024 handle_command_reply: tid=0 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 1 client.18024 shutdown 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 1 -- 172.21.15.133:0/3363588396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 msgr2=0x55eec491f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec491f1a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fae9c09b280 tx=0x7fae9c0046c0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 1 -- 172.21.15.133:0/3363588396 shutdown_connections 2024-09-11T23:47:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55eec4942a10 0x55eec4944e00 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4921cb0 0x55eec491f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7fae86ffd640 0 client.18024 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec491f1a0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 1 --2- 172.21.15.133:0/3363588396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4916af0 0x55eec491ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 1 -- 172.21.15.133:0/3363588396 >> 172.21.15.133:0/3363588396 conn(0x55eec4913060 msgr2=0x55eec4918dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 1 -- 172.21.15.133:0/3363588396 shutdown_connections 2024-09-11T23:47:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 1 -- 172.21.15.133:0/3363588396 wait complete. 2024-09-11T23:47:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.215+0000 7faeb0872740 20 client.18024 trim_cache size 0 max 16384 2024-09-11T23:47:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faeb0872740 1 Processor -- start 2024-09-11T23:47:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faeb0872740 1 -- start start 2024-09-11T23:47:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faeb0872740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 0x55eec491f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faeb0872740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec494a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faeb0872740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec494abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faeb0872740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ab490 con 0x55eec4917410 2024-09-11T23:47:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faeb0872740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faea81acfd0 con 0x55eec4916af0 2024-09-11T23:47:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faeb0872740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faea81abb60 con 0x55eec4921cb0 2024-09-11T23:47:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faea6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec494a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faea77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 0x55eec491f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faea6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec494a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38210/0 (socket says 172.21.15.133:38210) 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faea6ffd640 1 -- 172.21.15.133:0/4145175476 learned_addr learned my addr 172.21.15.133:0/4145175476 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.233+0000 7faea7fff640 1 --2- 172.21.15.133:0/4145175476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec494abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.234+0000 7faea77fe640 1 -- 172.21.15.133:0/4145175476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 msgr2=0x55eec494abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.234+0000 7faea77fe640 1 --2- 172.21.15.133:0/4145175476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec494abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.234+0000 7faea77fe640 1 -- 172.21.15.133:0/4145175476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 msgr2=0x55eec494a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.234+0000 7faea77fe640 1 --2- 172.21.15.133:0/4145175476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec494a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.234+0000 7faea77fe640 1 -- 172.21.15.133:0/4145175476 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec493e8b0 con 0x55eec4916af0 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.234+0000 7faea77fe640 1 --2- 172.21.15.133:0/4145175476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 0x55eec491f330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0d82f0 tx=0x7fae980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38e6223236402ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.234+0000 7fae8d7fa640 1 -- 172.21.15.133:0/4145175476 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98024030 con 0x55eec4916af0 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.234+0000 7fae8d7fa640 1 -- 172.21.15.133:0/4145175476 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae98005020 con 0x55eec4916af0 2024-09-11T23:47:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.234+0000 7fae8d7fa640 1 -- 172.21.15.133:0/4145175476 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98004a00 con 0x55eec4916af0 2024-09-11T23:47:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.235+0000 7faeb0872740 1 -- 172.21.15.133:0/4145175476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 msgr2=0x55eec491f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.235+0000 7faeb0872740 1 --2- 172.21.15.133:0/4145175476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 0x55eec491f330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0d82f0 tx=0x7fae980046c0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.235+0000 7faeb0872740 1 -- 172.21.15.133:0/4145175476 shutdown_connections 2024-09-11T23:47:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.235+0000 7faeb0872740 1 --2- 172.21.15.133:0/4145175476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec494abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.235+0000 7faeb0872740 1 --2- 172.21.15.133:0/4145175476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4917410 0x55eec494a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.235+0000 7faeb0872740 1 --2- 172.21.15.133:0/4145175476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4916af0 0x55eec491f330 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.235+0000 7faeb0872740 1 -- 172.21.15.133:0/4145175476 >> 172.21.15.133:0/4145175476 conn(0x55eec4913060 msgr2=0x55eec491ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.235+0000 7faeb0872740 1 -- 172.21.15.133:0/4145175476 shutdown_connections 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.235+0000 7faeb0872740 1 -- 172.21.15.133:0/4145175476 wait complete. 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.235+0000 7faeb0872740 1 -- start start 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faeb0872740 1 Processor -- start 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faeb0872740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4916af0 0x55eec493b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faeb0872740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4917410 0x55eec4936360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faeb0872740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec49368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faeb0872740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ac900 con 0x55eec4916af0 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faeb0872740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ac230 con 0x55eec4917410 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faeb0872740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faea81a9490 con 0x55eec4921cb0 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faeb0872740 10 client.0 _ll_register_callbacks cb 0x7fae9c004f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faea6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4917410 0x55eec4936360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faea77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4916af0 0x55eec493b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faea6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4917410 0x55eec4936360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53044/0 (socket says 172.21.15.133:53044) 2024-09-11T23:47:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faea77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4916af0 0x55eec493b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38218/0 (socket says 172.21.15.133:38218) 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.236+0000 7faea6ffd640 1 -- 172.21.15.133:0/2309637861 learned_addr learned my addr 172.21.15.133:0/2309637861 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7faea7fff640 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec49368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7faea6ffd640 1 -- 172.21.15.133:0/2309637861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 msgr2=0x55eec49368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7faea6ffd640 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec49368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7faea6ffd640 1 -- 172.21.15.133:0/2309637861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4916af0 msgr2=0x55eec493b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7faea6ffd640 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4916af0 0x55eec493b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7faea6ffd640 1 -- 172.21.15.133:0/2309637861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec4936f90 con 0x55eec4917410 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7faea6ffd640 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4917410 0x55eec4936360 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae9c007590 tx=0x7fae9c005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5044d803ee71d617 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7fae86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7fae86ffd640 1 -- 172.21.15.133:0/2309637861 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0bc060 con 0x55eec4917410 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7fae86ffd640 1 -- 172.21.15.133:0/2309637861 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fae9c09e480 con 0x55eec4917410 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7faeb0872740 1 -- 172.21.15.133:0/2309637861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae98005020 con 0x55eec4917410 2024-09-11T23:47:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7fae86ffd640 1 -- 172.21.15.133:0/2309637861 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae9c09ee20 con 0x55eec4917410 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7faeb0872740 10 client.27581 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.237+0000 7faeb0872740 1 -- 172.21.15.133:0/2309637861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eec49ad540 con 0x55eec4917410 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.238+0000 7fae86ffd640 1 -- 172.21.15.133:0/2309637861 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae98005020 con 0x55eec4917410 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.238+0000 7fae86ffd640 1 -- 172.21.15.133:0/2309637861 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fae9c0a42b0 con 0x55eec4917410 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.238+0000 7faeb0872740 10 client.27581 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.238+0000 7faeb0872740 10 client.27581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.238+0000 7faeb0872740 20 client.27581 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.238+0000 7faeb0872740 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55eec4942a10 0x55eec49afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.238+0000 7faeb0872740 4 client.27581 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.238+0000 7faeb0872740 1 -- 172.21.15.133:0/2309637861 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eec4882d70 con 0x55eec4942a10 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.238+0000 7faea77fe640 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55eec4942a10 0x55eec49afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.239+0000 7fae86ffd640 10 client.27581 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.239+0000 7faea77fe640 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55eec4942a10 0x55eec49afcc0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b39981b677c2a7ab server_cookie=ba44ed5a1d64b7eb in_seq=0 out_seq=0 2024-09-11T23:47:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.252+0000 7fae86ffd640 1 -- 172.21.15.133:0/2309637861 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eec4882d70 con 0x55eec4942a10 2024-09-11T23:47:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.252+0000 7fae86ffd640 10 client.27581 handle_command_reply: tid=0 2024-09-11T23:47:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.253+0000 7faeb0872740 1 client.27581 shutdown 2024-09-11T23:47:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.253+0000 7faeb0872740 1 -- 172.21.15.133:0/2309637861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4917410 msgr2=0x55eec4936360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.253+0000 7faeb0872740 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4917410 0x55eec4936360 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae9c007590 tx=0x7fae9c005770 comp rx=0 tx=0).stop 2024-09-11T23:47:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.253+0000 7faeb0872740 1 -- 172.21.15.133:0/2309637861 shutdown_connections 2024-09-11T23:47:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.253+0000 7faeb0872740 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55eec4942a10 0x55eec49afcc0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.253+0000 7faeb0872740 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eec4921cb0 0x55eec49368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.253+0000 7faeb0872740 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eec4917410 0x55eec4936360 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.253+0000 7faeb0872740 1 --2- 172.21.15.133:0/2309637861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eec4916af0 0x55eec493b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.253+0000 7faeb0872740 1 -- 172.21.15.133:0/2309637861 >> 172.21.15.133:0/2309637861 conn(0x55eec4913060 msgr2=0x55eec493e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.253+0000 7fae86ffd640 0 client.27581 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.253+0000 7faeb0872740 1 -- 172.21.15.133:0/2309637861 shutdown_connections 2024-09-11T23:47:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.254+0000 7faeb0872740 1 -- 172.21.15.133:0/2309637861 wait complete. 2024-09-11T23:47:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.254+0000 7faeb0872740 20 client.27581 trim_cache size 0 max 16384 2024-09-11T23:47:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:47:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:47:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:47:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:47:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 -- 172.21.15.133:0/3719401694 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae900a2e20 msgr2=0x7fae900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae900a2e20 0x7fae900a52e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faea0055000 tx=0x7faea0052000 comp rx=0 tx=0).stop 2024-09-11T23:47:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 -- 172.21.15.133:0/3719401694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8101350 msgr2=0x7faea8199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8101350 0x7faea8199660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0099c0 tx=0x7fae9c009e80 comp rx=0 tx=0).stop 2024-09-11T23:47:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 -- 172.21.15.133:0/3719401694 shutdown_connections 2024-09-11T23:47:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fae900a2e20 0x7fae900a52e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea81069c0 0x7faea81a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8101350 0x7faea8199660 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 --2- 172.21.15.133:0/3719401694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea81009b0 0x7faea8199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 -- 172.21.15.133:0/3719401694 >> 172.21.15.133:0/3719401694 conn(0x7faea8077990 msgr2=0x7faea80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 -- 172.21.15.133:0/3719401694 shutdown_connections 2024-09-11T23:47:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:00.255+0000 7faeadad7640 1 -- 172.21.15.133:0/3719401694 wait complete. 2024-09-11T23:47:00.273 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:47:00.525 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:00.935 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:00 smithi133 ceph-mon[31430]: pgmap v318: 97 pgs: 97 active+clean; 70 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 545 MiB/s wr, 951 op/s 2024-09-11T23:47:01.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:00 smithi178 ceph-mon[35882]: pgmap v318: 97 pgs: 97 active+clean; 70 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 545 MiB/s wr, 951 op/s 2024-09-11T23:47:01.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.160+0000 7f0859e63640 1 -- 172.21.15.133:0/3993704266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08541027a0 msgr2=0x7f08541067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:01.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.160+0000 7f0859e63640 1 --2- 172.21.15.133:0/3993704266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08541027a0 0x7f08541067f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0844009900 tx=0x7f084402a410 comp rx=0 tx=0).stop 2024-09-11T23:47:01.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.160+0000 7f0859e63640 1 -- 172.21.15.133:0/3993704266 shutdown_connections 2024-09-11T23:47:01.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.160+0000 7f0859e63640 1 --2- 172.21.15.133:0/3993704266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0854106f20 0x7f085410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.160+0000 7f0859e63640 1 --2- 172.21.15.133:0/3993704266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08541027a0 0x7f08541067f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.160+0000 7f0859e63640 1 --2- 172.21.15.133:0/3993704266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854101df0 0x7f08541021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.160+0000 7f0859e63640 1 -- 172.21.15.133:0/3993704266 >> 172.21.15.133:0/3993704266 conn(0x7f08540fd8f0 msgr2=0x7f08540ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:01.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.160+0000 7f0859e63640 1 -- 172.21.15.133:0/3993704266 shutdown_connections 2024-09-11T23:47:01.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.160+0000 7f0859e63640 1 -- 172.21.15.133:0/3993704266 wait complete. 2024-09-11T23:47:01.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f0859e63640 1 Processor -- start 2024-09-11T23:47:01.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f0859e63640 1 -- start start 2024-09-11T23:47:01.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f0859e63640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0854101df0 0x7f0854198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f0859e63640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08541027a0 0x7f08541989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f0859e63640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0854106f20 0x7f085419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f0859e63640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0854111f80 con 0x7f0854106f20 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f0859e63640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0854111e00 con 0x7f0854101df0 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f0859e63640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0854112100 con 0x7f08541027a0 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f08537fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0854101df0 0x7f0854198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f08537fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0854101df0 0x7f0854198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53066/0 (socket says 172.21.15.133:53066) 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f0853fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0854106f20 0x7f085419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f08537fe640 1 -- 172.21.15.133:0/2997442049 learned_addr learned my addr 172.21.15.133:0/2997442049 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.161+0000 7f0852ffd640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08541027a0 0x7f08541989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.162+0000 7f08537fe640 1 -- 172.21.15.133:0/2997442049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08541027a0 msgr2=0x7f08541989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.162+0000 7f08537fe640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08541027a0 0x7f08541989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.162+0000 7f08537fe640 1 -- 172.21.15.133:0/2997442049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0854106f20 msgr2=0x7f085419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.162+0000 7f08537fe640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0854106f20 0x7f085419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.162+0000 7f08537fe640 1 -- 172.21.15.133:0/2997442049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085419d420 con 0x7f0854101df0 2024-09-11T23:47:01.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.162+0000 7f08537fe640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0854101df0 0x7f0854198470 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f083c00d770 tx=0x7f083c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:01.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.162+0000 7f0850ff9640 1 -- 172.21.15.133:0/2997442049 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c017070 con 0x7f0854101df0 2024-09-11T23:47:01.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.162+0000 7f0859e63640 1 -- 172.21.15.133:0/2997442049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085419d710 con 0x7f0854101df0 2024-09-11T23:47:01.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.162+0000 7f0850ff9640 1 -- 172.21.15.133:0/2997442049 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f083c005500 con 0x7f0854101df0 2024-09-11T23:47:01.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.162+0000 7f0850ff9640 1 -- 172.21.15.133:0/2997442049 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c004e00 con 0x7f0854101df0 2024-09-11T23:47:01.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.162+0000 7f0859e63640 1 -- 172.21.15.133:0/2997442049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0854074420 con 0x7f0854101df0 2024-09-11T23:47:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.164+0000 7f0850ff9640 1 -- 172.21.15.133:0/2997442049 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f083c004fa0 con 0x7f0854101df0 2024-09-11T23:47:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f085c23c740 1 Processor -- start 2024-09-11T23:47:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f085c23c740 1 -- start start 2024-09-11T23:47:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f085c23c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556548466b40 0x556548465090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f085c23c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565484655d0 0x556548465a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f085c23c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x556548472760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f085c23c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5565483cb2f0 con 0x5565484655d0 2024-09-11T23:47:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f085c23c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556548372cf0 con 0x556548470370 2024-09-11T23:47:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f085c23c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5565483c72b0 con 0x556548466b40 2024-09-11T23:47:01.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f0850ff9640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08400a2cf0 0x7f08400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f08537fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565484655d0 0x556548465a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f0852ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x556548472760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f08537fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565484655d0 0x556548465a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38224/0 (socket says 172.21.15.133:38224) 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f08537fe640 1 -- 172.21.15.133:0/414955186 learned_addr learned my addr 172.21.15.133:0/414955186 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f0853fff640 1 --2- 172.21.15.133:0/414955186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556548466b40 0x556548465090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.165+0000 7f0853fff640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08400a2cf0 0x7f08400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.166+0000 7f0850ff9640 1 -- 172.21.15.133:0/2997442049 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f083c0ced90 con 0x7f0854101df0 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.166+0000 7f0852ffd640 1 -- 172.21.15.133:0/414955186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556548466b40 msgr2=0x556548465090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.166+0000 7f0852ffd640 1 --2- 172.21.15.133:0/414955186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556548466b40 0x556548465090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.166+0000 7f0852ffd640 1 -- 172.21.15.133:0/414955186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565484655d0 msgr2=0x556548465a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.166+0000 7f0852ffd640 1 --2- 172.21.15.133:0/414955186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565484655d0 0x556548465a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.166+0000 7f0852ffd640 1 -- 172.21.15.133:0/414955186 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565483573a0 con 0x556548470370 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.166+0000 7f0853fff640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08400a2cf0 0x7f08400a51b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f084c005fd0 tx=0x7f084c009500 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.166+0000 7f0852ffd640 1 --2- 172.21.15.133:0/414955186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x556548472760 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0844005940 tx=0x7f0844005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c195801b765df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:01.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.166+0000 7f0849ffb640 1 -- 172.21.15.133:0/414955186 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844040070 con 0x556548470370 2024-09-11T23:47:01.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.166+0000 7f0849ffb640 1 -- 172.21.15.133:0/414955186 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08440318f0 con 0x556548470370 2024-09-11T23:47:01.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.166+0000 7f0849ffb640 1 -- 172.21.15.133:0/414955186 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844035040 con 0x556548470370 2024-09-11T23:47:01.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.167+0000 7f085c23c740 1 -- 172.21.15.133:0/414955186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 msgr2=0x556548472760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:01.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.167+0000 7f085c23c740 1 --2- 172.21.15.133:0/414955186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x556548472760 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0844005940 tx=0x7f0844005e00 comp rx=0 tx=0).stop 2024-09-11T23:47:01.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.167+0000 7f085c23c740 1 -- 172.21.15.133:0/414955186 shutdown_connections 2024-09-11T23:47:01.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.167+0000 7f085c23c740 1 --2- 172.21.15.133:0/414955186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x556548472760 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.167+0000 7f085c23c740 1 --2- 172.21.15.133:0/414955186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565484655d0 0x556548465a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.167+0000 7f085c23c740 1 --2- 172.21.15.133:0/414955186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556548466b40 0x556548465090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.167+0000 7f085c23c740 1 -- 172.21.15.133:0/414955186 >> 172.21.15.133:0/414955186 conn(0x55654842e310 msgr2=0x556548461c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.167+0000 7f085c23c740 1 -- 172.21.15.133:0/414955186 shutdown_connections 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.167+0000 7f085c23c740 1 -- 172.21.15.133:0/414955186 wait complete. 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.167+0000 7f085c23c740 1 -- start start 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f085c23c740 1 Processor -- start 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f085c23c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x55654846fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f085c23c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x55654846cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f085c23c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x55654846d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f085c23c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556548473010 con 0x556548466b40 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f085c23c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556548386060 con 0x556548470370 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f085c23c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556548473190 con 0x5565484655d0 2024-09-11T23:47:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f085c23c740 10 client.0 _ll_register_callbacks cb 0x5565480d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f0852ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x55654846cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f0852ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x55654846cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38240/0 (socket says 172.21.15.133:38240) 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f0852ffd640 1 -- 172.21.15.133:0/410514605 learned_addr learned my addr 172.21.15.133:0/410514605 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f08537fe640 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x55654846fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f0853fff640 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x55654846d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f0852ffd640 1 -- 172.21.15.133:0/410514605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 msgr2=0x55654846fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f0852ffd640 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x55654846fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f0852ffd640 1 -- 172.21.15.133:0/410514605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 msgr2=0x55654846d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f0852ffd640 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x55654846d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f0852ffd640 1 -- 172.21.15.133:0/410514605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654813c3c0 con 0x556548466b40 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.168+0000 7f08537fe640 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x55654846fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f0852ffd640 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x55654846cfd0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0844009900 tx=0x7f0844005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4512075a3134f35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f0832ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f0832ffd640 1 -- 172.21.15.133:0/410514605 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844040030 con 0x556548466b40 2024-09-11T23:47:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f0832ffd640 1 -- 172.21.15.133:0/410514605 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f08440316f0 con 0x556548466b40 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f085c23c740 1 -- 172.21.15.133:0/410514605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08440318f0 con 0x556548466b40 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f0832ffd640 1 -- 172.21.15.133:0/410514605 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0844035070 con 0x556548466b40 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f085c23c740 10 client.18042 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f085c23c740 1 -- 172.21.15.133:0/410514605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55654847ea60 con 0x556548466b40 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f0832ffd640 1 -- 172.21.15.133:0/410514605 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08440318f0 con 0x556548466b40 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f0832ffd640 1 -- 172.21.15.133:0/410514605 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0844004c30 con 0x556548466b40 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f085c23c740 10 client.18042 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f085c23c740 10 client.18042 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f085c23c740 20 client.18042 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f085c23c740 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55654847fdb0 0x5565484821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f085c23c740 4 client.18042 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.169+0000 7f085c23c740 1 -- 172.21.15.133:0/410514605 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565483d8860 con 0x55654847fdb0 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.170+0000 7f08537fe640 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55654847fdb0 0x5565484821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.170+0000 7f08537fe640 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55654847fdb0 0x5565484821a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ed131b0b4e6b3e6 server_cookie=467697150d98b9f2 in_seq=0 out_seq=0 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.170+0000 7f0832ffd640 10 client.18042 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:47:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f0832ffd640 1 -- 172.21.15.133:0/410514605 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565483d8860 con 0x55654847fdb0 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f0832ffd640 10 client.18042 handle_command_reply: tid=0 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 1 client.18042 shutdown 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 1 -- 172.21.15.133:0/410514605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 msgr2=0x55654846cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x55654846cfd0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0844009900 tx=0x7f0844005250 comp rx=0 tx=0).stop 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 1 -- 172.21.15.133:0/410514605 shutdown_connections 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55654847fdb0 0x5565484821a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x55654846d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x55654846cfd0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 1 --2- 172.21.15.133:0/410514605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x55654846fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 1 -- 172.21.15.133:0/410514605 >> 172.21.15.133:0/410514605 conn(0x55654842e310 msgr2=0x556548467960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f0832ffd640 0 client.18042 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 1 -- 172.21.15.133:0/410514605 shutdown_connections 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 1 -- 172.21.15.133:0/410514605 wait complete. 2024-09-11T23:47:01.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.171+0000 7f085c23c740 20 client.18042 trim_cache size 0 max 16384 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.184+0000 7f085c23c740 1 Processor -- start 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.184+0000 7f085c23c740 1 -- start start 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.184+0000 7f085c23c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x55654847c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.184+0000 7f085c23c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x5565484877d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.184+0000 7f085c23c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x556548487d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.184+0000 7f085c23c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08541af940 con 0x556548466b40 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.184+0000 7f085c23c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08541b1480 con 0x556548470370 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.184+0000 7f085c23c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08541b0010 con 0x5565484655d0 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f0852ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x5565484877d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f0852ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x5565484877d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38242/0 (socket says 172.21.15.133:38242) 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f0852ffd640 1 -- 172.21.15.133:0/91723248 learned_addr learned my addr 172.21.15.133:0/91723248 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f0853fff640 1 --2- 172.21.15.133:0/91723248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x556548487d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f08537fe640 1 --2- 172.21.15.133:0/91723248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x55654847c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f0852ffd640 1 -- 172.21.15.133:0/91723248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 msgr2=0x55654847c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f0852ffd640 1 --2- 172.21.15.133:0/91723248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x55654847c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f0852ffd640 1 -- 172.21.15.133:0/91723248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 msgr2=0x556548487d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f0852ffd640 1 --2- 172.21.15.133:0/91723248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x556548487d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f0852ffd640 1 -- 172.21.15.133:0/91723248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654846d8e0 con 0x556548466b40 2024-09-11T23:47:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f08537fe640 1 --2- 172.21.15.133:0/91723248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x55654847c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:47:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f0852ffd640 1 --2- 172.21.15.133:0/91723248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x5565484877d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0844004ff0 tx=0x7f0844005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1112d55592c4bf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f08497fa640 1 -- 172.21.15.133:0/91723248 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084405b020 con 0x556548466b40 2024-09-11T23:47:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f08497fa640 1 -- 172.21.15.133:0/91723248 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08440052c0 con 0x556548466b40 2024-09-11T23:47:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.185+0000 7f08497fa640 1 -- 172.21.15.133:0/91723248 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844040070 con 0x556548466b40 2024-09-11T23:47:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.186+0000 7f085c23c740 1 -- 172.21.15.133:0/91723248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 msgr2=0x5565484877d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.186+0000 7f085c23c740 1 --2- 172.21.15.133:0/91723248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x5565484877d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0844004ff0 tx=0x7f0844005900 comp rx=0 tx=0).stop 2024-09-11T23:47:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.186+0000 7f085c23c740 1 -- 172.21.15.133:0/91723248 shutdown_connections 2024-09-11T23:47:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.186+0000 7f085c23c740 1 --2- 172.21.15.133:0/91723248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548470370 0x556548487d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.186+0000 7f085c23c740 1 --2- 172.21.15.133:0/91723248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548466b40 0x5565484877d0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.186+0000 7f085c23c740 1 --2- 172.21.15.133:0/91723248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x55654847c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.186+0000 7f085c23c740 1 -- 172.21.15.133:0/91723248 >> 172.21.15.133:0/91723248 conn(0x55654842e310 msgr2=0x55654846b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.186+0000 7f085c23c740 1 -- 172.21.15.133:0/91723248 shutdown_connections 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.186+0000 7f085c23c740 1 -- 172.21.15.133:0/91723248 wait complete. 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.186+0000 7f085c23c740 1 -- start start 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f085c23c740 1 Processor -- start 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f085c23c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x5565484789b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f085c23c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548466b40 0x556548473ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f085c23c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548470370 0x556548474220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f085c23c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08541b0db0 con 0x556548470370 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f085c23c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08541b06e0 con 0x556548466b40 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f085c23c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08541ad940 con 0x5565484655d0 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f085c23c740 10 client.0 _ll_register_callbacks cb 0x7f083c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f0853fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548470370 0x556548474220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f0853fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548470370 0x556548474220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38252/0 (socket says 172.21.15.133:38252) 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f0853fff640 1 -- 172.21.15.133:0/3440718962 learned_addr learned my addr 172.21.15.133:0/3440718962 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f08537fe640 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x5565484789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f0852ffd640 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548466b40 0x556548473ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f0853fff640 1 -- 172.21.15.133:0/3440718962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 msgr2=0x5565484789b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f0853fff640 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x5565484789b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f0853fff640 1 -- 172.21.15.133:0/3440718962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548466b40 msgr2=0x556548473ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f0853fff640 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548466b40 0x556548473ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f0853fff640 1 -- 172.21.15.133:0/3440718962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556548474860 con 0x556548470370 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.187+0000 7f0852ffd640 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548466b40 0x556548473ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.188+0000 7f0853fff640 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548470370 0x556548474220 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f084c01f960 tx=0x7f084c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e6f144023af58e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.188+0000 7f0832ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.188+0000 7f0832ffd640 1 -- 172.21.15.133:0/3440718962 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c004d20 con 0x556548470370 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.188+0000 7f0832ffd640 1 -- 172.21.15.133:0/3440718962 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f084c005020 con 0x556548470370 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.188+0000 7f085c23c740 1 -- 172.21.15.133:0/3440718962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08440052c0 con 0x556548470370 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.188+0000 7f0832ffd640 1 -- 172.21.15.133:0/3440718962 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f084c005220 con 0x556548470370 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.188+0000 7f085c23c740 10 client.18048 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.188+0000 7f085c23c740 1 -- 172.21.15.133:0/3440718962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565484ea560 con 0x556548470370 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.188+0000 7f0832ffd640 1 -- 172.21.15.133:0/3440718962 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08440052c0 con 0x556548470370 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.188+0000 7f0832ffd640 1 -- 172.21.15.133:0/3440718962 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f084c01d070 con 0x556548470370 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f085c23c740 10 client.18048 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f085c23c740 10 client.18048 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f085c23c740 20 client.18048 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f085c23c740 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5565484ead90 0x5565484ed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f085c23c740 4 client.18048 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f085c23c740 1 -- 172.21.15.133:0/3440718962 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565483d2d70 con 0x5565484ead90 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f08537fe640 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5565484ead90 0x5565484ed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f0832ffd640 10 client.18048 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f08537fe640 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5565484ead90 0x5565484ed1d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd4b02e7da2f028b server_cookie=3b2b895be1e08896 in_seq=0 out_seq=0 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f0832ffd640 1 -- 172.21.15.133:0/3440718962 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565483d2d70 con 0x5565484ead90 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f0832ffd640 10 client.18048 handle_command_reply: tid=0 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.189+0000 7f085c23c740 1 client.18048 shutdown 2024-09-11T23:47:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f085c23c740 1 -- 172.21.15.133:0/3440718962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548470370 msgr2=0x556548474220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f085c23c740 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548470370 0x556548474220 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f084c01f960 tx=0x7f084c004ab0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f085c23c740 1 -- 172.21.15.133:0/3440718962 shutdown_connections 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f085c23c740 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5565484ead90 0x5565484ed1d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f0832ffd640 0 client.18048 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f085c23c740 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556548470370 0x556548474220 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f085c23c740 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556548466b40 0x556548473ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f085c23c740 1 --2- 172.21.15.133:0/3440718962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565484655d0 0x5565484789b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f085c23c740 1 -- 172.21.15.133:0/3440718962 >> 172.21.15.133:0/3440718962 conn(0x55654842e310 msgr2=0x55654846f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f085c23c740 1 -- 172.21.15.133:0/3440718962 shutdown_connections 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f085c23c740 1 -- 172.21.15.133:0/3440718962 wait complete. 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f085c23c740 20 client.18048 trim_cache size 0 max 16384 2024-09-11T23:47:01.194 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:47:01.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f0859e63640 1 -- 172.21.15.133:0/2997442049 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08400a2cf0 msgr2=0x7f08400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:01.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.190+0000 7f0859e63640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08400a2cf0 0x7f08400a51b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f084c005fd0 tx=0x7f084c009500 comp rx=0 tx=0).stop 2024-09-11T23:47:01.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.191+0000 7f0859e63640 1 -- 172.21.15.133:0/2997442049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0854101df0 msgr2=0x7f0854198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:01.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.191+0000 7f0859e63640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0854101df0 0x7f0854198470 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f083c00d770 tx=0x7f083c00dc30 comp rx=0 tx=0).stop 2024-09-11T23:47:01.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.191+0000 7f0859e63640 1 -- 172.21.15.133:0/2997442049 shutdown_connections 2024-09-11T23:47:01.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.191+0000 7f0859e63640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f08400a2cf0 0x7f08400a51b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.191+0000 7f0859e63640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0854106f20 0x7f085419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.191+0000 7f0859e63640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08541027a0 0x7f08541989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.191+0000 7f0859e63640 1 --2- 172.21.15.133:0/2997442049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0854101df0 0x7f0854198470 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:01.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.191+0000 7f0859e63640 1 -- 172.21.15.133:0/2997442049 >> 172.21.15.133:0/2997442049 conn(0x7f08540fd8f0 msgr2=0x7f08540fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:01.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.191+0000 7f0859e63640 1 -- 172.21.15.133:0/2997442049 shutdown_connections 2024-09-11T23:47:01.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:01.191+0000 7f0859e63640 1 -- 172.21.15.133:0/2997442049 wait complete. 2024-09-11T23:47:01.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:00 smithi138 ceph-mon[35142]: pgmap v318: 97 pgs: 97 active+clean; 70 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 545 MiB/s wr, 951 op/s 2024-09-11T23:47:01.948 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:47:02.203 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:02.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.699+0000 7fc58a423640 1 -- 172.21.15.133:0/3693824049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584101420 msgr2=0x7fc58410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:02.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.699+0000 7fc58a423640 1 --2- 172.21.15.133:0/3693824049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584101420 0x7fc58410d950 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009cd0 tx=0x7fc57c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.699+0000 7fc58a423640 1 -- 172.21.15.133:0/3693824049 shutdown_connections 2024-09-11T23:47:02.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.699+0000 7fc58a423640 1 --2- 172.21.15.133:0/3693824049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584101420 0x7fc58410d950 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.699+0000 7fc58a423640 1 --2- 172.21.15.133:0/3693824049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584100a80 0x7fc584100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.699+0000 7fc58a423640 1 --2- 172.21.15.133:0/3693824049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc584106a80 0x7fc584106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.699+0000 7fc58a423640 1 -- 172.21.15.133:0/3693824049 >> 172.21.15.133:0/3693824049 conn(0x7fc5840fc7a0 msgr2=0x7fc5840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:02.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.700+0000 7fc58a423640 1 -- 172.21.15.133:0/3693824049 shutdown_connections 2024-09-11T23:47:02.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.700+0000 7fc58a423640 1 -- 172.21.15.133:0/3693824049 wait complete. 2024-09-11T23:47:02.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.700+0000 7fc58a423640 1 Processor -- start 2024-09-11T23:47:02.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.700+0000 7fc58a423640 1 -- start start 2024-09-11T23:47:02.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc58a423640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584100a80 0x7fc584199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc58a423640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584101420 0x7fc584199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc58a423640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc584106a80 0x7fc5841a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc58a423640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc584110ea0 con 0x7fc584101420 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc58a423640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc584110d20 con 0x7fc584106a80 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc58a423640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc584111020 con 0x7fc584100a80 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584101420 0x7fc584199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584101420 0x7fc584199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42594/0 (socket says 172.21.15.133:42594) 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc57bfff640 1 -- 172.21.15.133:0/3503758362 learned_addr learned my addr 172.21.15.133:0/3503758362 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc5889c0640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc584106a80 0x7fc5841a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc583fff640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584100a80 0x7fc584199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc57bfff640 1 -- 172.21.15.133:0/3503758362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584100a80 msgr2=0x7fc584199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc57bfff640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584100a80 0x7fc584199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc57bfff640 1 -- 172.21.15.133:0/3503758362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc584106a80 msgr2=0x7fc5841a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:02.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc57bfff640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc584106a80 0x7fc5841a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc57bfff640 1 -- 172.21.15.133:0/3503758362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5841a46f0 con 0x7fc584101420 2024-09-11T23:47:02.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.701+0000 7fc57bfff640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584101420 0x7fc584199670 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00b4a0 tx=0x7fc56c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:02.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.702+0000 7fc581ffb640 1 -- 172.21.15.133:0/3503758362 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c014070 con 0x7fc584101420 2024-09-11T23:47:02.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.702+0000 7fc581ffb640 1 -- 172.21.15.133:0/3503758362 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc56c010af0 con 0x7fc584101420 2024-09-11T23:47:02.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.702+0000 7fc581ffb640 1 -- 172.21.15.133:0/3503758362 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c017d10 con 0x7fc584101420 2024-09-11T23:47:02.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.702+0000 7fc58a423640 1 -- 172.21.15.133:0/3503758362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5841a48c0 con 0x7fc584101420 2024-09-11T23:47:02.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.702+0000 7fc58a423640 1 -- 172.21.15.133:0/3503758362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5841a4dd0 con 0x7fc584101420 2024-09-11T23:47:02.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.704+0000 7fc581ffb640 1 -- 172.21.15.133:0/3503758362 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc56c017070 con 0x7fc584101420 2024-09-11T23:47:02.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.704+0000 7fc58c7fc740 1 Processor -- start 2024-09-11T23:47:02.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.704+0000 7fc58c7fc740 1 -- start start 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69da090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69da5d0 0x5611e69daa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69e7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5611e69402f0 con 0x5611e69e5370 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5611e68e7cf0 con 0x5611e69dbb40 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5611e693c2b0 con 0x5611e69da5d0 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc5889c0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69da090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc5889c0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69da090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59118/0 (socket says 172.21.15.133:59118) 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc5889c0640 1 -- 172.21.15.133:0/3986794127 learned_addr learned my addr 172.21.15.133:0/3986794127 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc581ffb640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc5700a2cf0 0x7fc5700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc583fff640 1 --2- 172.21.15.133:0/3986794127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69da5d0 0x5611e69daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc5889c0640 1 -- 172.21.15.133:0/3986794127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69da5d0 msgr2=0x5611e69daa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:02.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc5889c0640 1 --2- 172.21.15.133:0/3986794127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69da5d0 0x5611e69daa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc5889c0640 1 -- 172.21.15.133:0/3986794127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 msgr2=0x5611e69e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:02.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc5889c0640 1 --2- 172.21.15.133:0/3986794127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69e7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc5889c0640 1 -- 172.21.15.133:0/3986794127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e68cc3a0 con 0x5611e69dbb40 2024-09-11T23:47:02.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc5889c0640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc5700a2cf0 0x7fc5700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc581ffb640 1 -- 172.21.15.133:0/3503758362 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fc56c0ce6d0 con 0x7fc584101420 2024-09-11T23:47:02.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.705+0000 7fc5889c0640 1 --2- 172.21.15.133:0/3986794127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69da090 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02ae00 tx=0x7fc57c001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91b8b5ab245d6572 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:02.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc57a7fc640 1 -- 172.21.15.133:0/3986794127 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c041040 con 0x5611e69dbb40 2024-09-11T23:47:02.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc57a7fc640 1 -- 172.21.15.133:0/3986794127 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc57c043070 con 0x5611e69dbb40 2024-09-11T23:47:02.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc57a7fc640 1 -- 172.21.15.133:0/3986794127 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c030a40 con 0x5611e69dbb40 2024-09-11T23:47:02.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3986794127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 msgr2=0x5611e69da090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:02.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc5889c0640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc5700a2cf0 0x7fc5700a51b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc57c04c000 tx=0x7fc57c04d000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:02.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/3986794127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69da090 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02ae00 tx=0x7fc57c001150 comp rx=0 tx=0).stop 2024-09-11T23:47:02.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3986794127 shutdown_connections 2024-09-11T23:47:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/3986794127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/3986794127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69da5d0 0x5611e69daa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/3986794127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69da090 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3986794127 >> 172.21.15.133:0/3986794127 conn(0x5611e69a3310 msgr2=0x5611e69d6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3986794127 shutdown_connections 2024-09-11T23:47:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.706+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3986794127 wait complete. 2024-09-11T23:47:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.707+0000 7fc58c7fc740 1 -- start start 2024-09-11T23:47:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.707+0000 7fc58c7fc740 1 Processor -- start 2024-09-11T23:47:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.707+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69da5d0 0x5611e69e4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.707+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69dbb40 0x5611e69e2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.707+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69e27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.707+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5611e69ec480 con 0x5611e69e5370 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.707+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5611e68fb060 con 0x5611e69da5d0 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.707+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5611e69ec600 con 0x5611e69dbb40 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.707+0000 7fc58c7fc740 10 client.0 _ll_register_callbacks cb 0x5611e664e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.707+0000 7fc5889c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69e27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.707+0000 7fc583fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69da5d0 0x5611e69e4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5889c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69e27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42600/0 (socket says 172.21.15.133:42600) 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5889c0640 1 -- 172.21.15.133:0/3795497105 learned_addr learned my addr 172.21.15.133:0/3795497105 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc583fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69da5d0 0x5611e69e4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59130/0 (socket says 172.21.15.133:59130) 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc57bfff640 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69dbb40 0x5611e69e2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5889c0640 1 -- 172.21.15.133:0/3795497105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69dbb40 msgr2=0x5611e69e2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5889c0640 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69dbb40 0x5611e69e2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5889c0640 1 -- 172.21.15.133:0/3795497105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69da5d0 msgr2=0x5611e69e4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:02.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5889c0640 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69da5d0 0x5611e69e4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5889c0640 1 -- 172.21.15.133:0/3795497105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e66b13c0 con 0x5611e69e5370 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc583fff640 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69da5d0 0x5611e69e4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5889c0640 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69e27f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009c70 tx=0x7fc57c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aacbf871bf31747 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5637fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5637fe640 1 -- 172.21.15.133:0/3795497105 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c059070 con 0x5611e69e5370 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5637fe640 1 -- 172.21.15.133:0/3795497105 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fc57c004040 con 0x5611e69e5370 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3795497105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc57c043070 con 0x5611e69e5370 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc5637fe640 1 -- 172.21.15.133:0/3795497105 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc57c065050 con 0x5611e69e5370 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc58c7fc740 10 client.18060 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.708+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3795497105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611e69f7f70 con 0x5611e69e5370 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.709+0000 7fc5637fe640 1 -- 172.21.15.133:0/3795497105 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc57c043070 con 0x5611e69e5370 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.709+0000 7fc5637fe640 1 -- 172.21.15.133:0/3795497105 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc57c030070 con 0x5611e69e5370 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.709+0000 7fc58c7fc740 10 client.18060 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:02.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.709+0000 7fc58c7fc740 10 client.18060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:02.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.709+0000 7fc58c7fc740 20 client.18060 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:02.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.709+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611e69f92c0 0x5611e69fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.709+0000 7fc58c7fc740 4 client.18060 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:02.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.709+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3795497105 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611e694d860 con 0x5611e69f92c0 2024-09-11T23:47:02.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.709+0000 7fc583fff640 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611e69f92c0 0x5611e69fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.710+0000 7fc583fff640 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611e69f92c0 0x5611e69fb6b0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4139b4e56240c2b2 server_cookie=9f352f313c8867a9 in_seq=0 out_seq=0 2024-09-11T23:47:02.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.710+0000 7fc5637fe640 10 client.18060 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:02.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.710+0000 7fc5637fe640 1 -- 172.21.15.133:0/3795497105 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611e694d860 con 0x5611e69f92c0 2024-09-11T23:47:02.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.710+0000 7fc5637fe640 10 client.18060 handle_command_reply: tid=0 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.710+0000 7fc58c7fc740 1 client.18060 shutdown 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3795497105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 msgr2=0x5611e69e27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69e27f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009c70 tx=0x7fc57c0098e0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3795497105 shutdown_connections 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611e69f92c0 0x5611e69fb6b0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69e27f0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69dbb40 0x5611e69e2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/3795497105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69da5d0 0x5611e69e4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc5637fe640 0 client.18060 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3795497105 >> 172.21.15.133:0/3795497105 conn(0x5611e69a3310 msgr2=0x5611e69dc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3795497105 shutdown_connections 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc58c7fc740 1 -- 172.21.15.133:0/3795497105 wait complete. 2024-09-11T23:47:02.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.711+0000 7fc58c7fc740 20 client.18060 trim_cache size 0 max 16384 2024-09-11T23:47:02.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc58c7fc740 1 Processor -- start 2024-09-11T23:47:02.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc58c7fc740 1 -- start start 2024-09-11T23:47:02.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69da5d0 0x5611e69dd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69dbb40 0x5611e69dd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69e5370 0x5611e69fe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841ab620 con 0x5611e69da5d0 2024-09-11T23:47:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841ad160 con 0x5611e69e5370 2024-09-11T23:47:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841abcf0 con 0x5611e69dbb40 2024-09-11T23:47:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc583fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69da5d0 0x5611e69dd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc583fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69da5d0 0x5611e69dd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42608/0 (socket says 172.21.15.133:42608) 2024-09-11T23:47:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc583fff640 1 -- 172.21.15.133:0/1293176688 learned_addr learned my addr 172.21.15.133:0/1293176688 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc57bfff640 1 --2- 172.21.15.133:0/1293176688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69dbb40 0x5611e69dd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc5889c0640 1 --2- 172.21.15.133:0/1293176688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69e5370 0x5611e69fe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc583fff640 1 -- 172.21.15.133:0/1293176688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69dbb40 msgr2=0x5611e69dd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc583fff640 1 --2- 172.21.15.133:0/1293176688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69dbb40 0x5611e69dd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc583fff640 1 -- 172.21.15.133:0/1293176688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69e5370 msgr2=0x5611e69fe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc583fff640 1 --2- 172.21.15.133:0/1293176688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69e5370 0x5611e69fe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.724+0000 7fc583fff640 1 -- 172.21.15.133:0/1293176688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e69d64f0 con 0x5611e69da5d0 2024-09-11T23:47:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc583fff640 1 --2- 172.21.15.133:0/1293176688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69da5d0 0x5611e69dd120 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc574000c00 tx=0x7fc574004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17881811f1a884fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc579ffb640 1 -- 172.21.15.133:0/1293176688 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574003d00 con 0x5611e69da5d0 2024-09-11T23:47:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc579ffb640 1 -- 172.21.15.133:0/1293176688 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc574020960 con 0x5611e69da5d0 2024-09-11T23:47:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc579ffb640 1 -- 172.21.15.133:0/1293176688 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574013b20 con 0x5611e69da5d0 2024-09-11T23:47:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc58c7fc740 1 -- 172.21.15.133:0/1293176688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69da5d0 msgr2=0x5611e69dd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/1293176688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69da5d0 0x5611e69dd120 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc574000c00 tx=0x7fc574004580 comp rx=0 tx=0).stop 2024-09-11T23:47:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc58c7fc740 1 -- 172.21.15.133:0/1293176688 shutdown_connections 2024-09-11T23:47:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/1293176688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69e5370 0x5611e69fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/1293176688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69dbb40 0x5611e69dd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/1293176688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69da5d0 0x5611e69dd120 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc58c7fc740 1 -- 172.21.15.133:0/1293176688 >> 172.21.15.133:0/1293176688 conn(0x5611e69a3310 msgr2=0x5611e69e1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc58c7fc740 1 -- 172.21.15.133:0/1293176688 shutdown_connections 2024-09-11T23:47:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.725+0000 7fc58c7fc740 1 -- 172.21.15.133:0/1293176688 wait complete. 2024-09-11T23:47:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc58c7fc740 1 -- start start 2024-09-11T23:47:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc58c7fc740 1 Processor -- start 2024-09-11T23:47:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69da5d0 0x5611e69fef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69fc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc58c7fc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69fc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841aca90 con 0x5611e69e5370 2024-09-11T23:47:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841ac3c0 con 0x5611e69dbb40 2024-09-11T23:47:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc58c7fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841a9620 con 0x5611e69da5d0 2024-09-11T23:47:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc58c7fc740 10 client.0 _ll_register_callbacks cb 0x7fc56c097060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc5889c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69fc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc583fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69da5d0 0x5611e69fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69fc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59144/0 (socket says 172.21.15.133:59144) 2024-09-11T23:47:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.726+0000 7fc57bfff640 1 -- 172.21.15.133:0/699769658 learned_addr learned my addr 172.21.15.133:0/699769658 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc57bfff640 1 -- 172.21.15.133:0/699769658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69da5d0 msgr2=0x5611e69fef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc57bfff640 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69da5d0 0x5611e69fef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc57bfff640 1 -- 172.21.15.133:0/699769658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 msgr2=0x5611e69fc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc57bfff640 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69fc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc57bfff640 1 -- 172.21.15.133:0/699769658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e69fd020 con 0x5611e69dbb40 2024-09-11T23:47:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc5889c0640 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69fc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc57bfff640 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69fc250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc56c09acd0 tx=0x7fc56c09a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a80c7482f35284d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc5637fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:47:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc5637fe640 1 -- 172.21.15.133:0/699769658 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c01e040 con 0x5611e69dbb40 2024-09-11T23:47:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc5637fe640 1 -- 172.21.15.133:0/699769658 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fc56c0973b0 con 0x5611e69dbb40 2024-09-11T23:47:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc58c7fc740 1 -- 172.21.15.133:0/699769658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc574020960 con 0x5611e69dbb40 2024-09-11T23:47:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc5637fe640 1 -- 172.21.15.133:0/699769658 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc56c09f0f0 con 0x5611e69dbb40 2024-09-11T23:47:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc58c7fc740 10 client.27614 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc58c7fc740 1 -- 172.21.15.133:0/699769658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611e6a63d30 con 0x5611e69dbb40 2024-09-11T23:47:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.727+0000 7fc5637fe640 1 -- 172.21.15.133:0/699769658 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc574020960 con 0x5611e69dbb40 2024-09-11T23:47:02.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc5637fe640 1 -- 172.21.15.133:0/699769658 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc56c079b60 con 0x5611e69dbb40 2024-09-11T23:47:02.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc58c7fc740 10 client.27614 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:02.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc58c7fc740 10 client.27614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:02.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc58c7fc740 20 client.27614 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:02.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611e6a64450 0x5611e6a66840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:02.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc58c7fc740 4 client.27614 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:47:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc58c7fc740 1 -- 172.21.15.133:0/699769658 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611e6947d70 con 0x5611e6a64450 2024-09-11T23:47:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc583fff640 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611e6a64450 0x5611e6a66840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc583fff640 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611e6a64450 0x5611e6a66840 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6268974e77aa9561 server_cookie=6fbf85bf4ed2769 in_seq=0 out_seq=0 2024-09-11T23:47:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc5637fe640 10 client.27614 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc5637fe640 1 -- 172.21.15.133:0/699769658 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611e6947d70 con 0x5611e6a64450 2024-09-11T23:47:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc5637fe640 10 client.27614 handle_command_reply: tid=0 2024-09-11T23:47:02.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.728+0000 7fc58c7fc740 1 client.27614 shutdown 2024-09-11T23:47:02.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58c7fc740 1 -- 172.21.15.133:0/699769658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 msgr2=0x5611e69fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:02.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69fc250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc56c09acd0 tx=0x7fc56c09a760 comp rx=0 tx=0).stop 2024-09-11T23:47:02.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58c7fc740 1 -- 172.21.15.133:0/699769658 shutdown_connections 2024-09-11T23:47:02.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5611e6a64450 0x5611e6a66840 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5611e69e5370 0x5611e69fc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611e69dbb40 0x5611e69fc250 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc5637fe640 0 client.27614 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58c7fc740 1 --2- 172.21.15.133:0/699769658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611e69da5d0 0x5611e69fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58c7fc740 1 -- 172.21.15.133:0/699769658 >> 172.21.15.133:0/699769658 conn(0x5611e69a3310 msgr2=0x5611e69e38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58c7fc740 1 -- 172.21.15.133:0/699769658 shutdown_connections 2024-09-11T23:47:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58c7fc740 1 -- 172.21.15.133:0/699769658 wait complete. 2024-09-11T23:47:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58c7fc740 20 client.27614 trim_cache size 0 max 16384 2024-09-11T23:47:02.744 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:47:02.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58a423640 1 -- 172.21.15.133:0/3503758362 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc5700a2cf0 msgr2=0x7fc5700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:02.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58a423640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc5700a2cf0 0x7fc5700a51b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc57c04c000 tx=0x7fc57c04d000 comp rx=0 tx=0).stop 2024-09-11T23:47:02.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58a423640 1 -- 172.21.15.133:0/3503758362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584101420 msgr2=0x7fc584199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:02.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58a423640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584101420 0x7fc584199670 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00b4a0 tx=0x7fc56c00b960 comp rx=0 tx=0).stop 2024-09-11T23:47:02.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58a423640 1 -- 172.21.15.133:0/3503758362 shutdown_connections 2024-09-11T23:47:02.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58a423640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc5700a2cf0 0x7fc5700a51b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58a423640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc584106a80 0x7fc5841a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58a423640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc584101420 0x7fc584199670 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58a423640 1 --2- 172.21.15.133:0/3503758362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584100a80 0x7fc584199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:02.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.729+0000 7fc58a423640 1 -- 172.21.15.133:0/3503758362 >> 172.21.15.133:0/3503758362 conn(0x7fc5840fc7a0 msgr2=0x7fc5840fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:02.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.730+0000 7fc58a423640 1 -- 172.21.15.133:0/3503758362 shutdown_connections 2024-09-11T23:47:02.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:02.730+0000 7fc58a423640 1 -- 172.21.15.133:0/3503758362 wait complete. 2024-09-11T23:47:02.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:02 smithi133 ceph-mon[31430]: pgmap v319: 97 pgs: 97 active+clean; 70 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 255 MiB/s wr, 459 op/s 2024-09-11T23:47:03.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:02 smithi178 ceph-mon[35882]: pgmap v319: 97 pgs: 97 active+clean; 70 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 255 MiB/s wr, 459 op/s 2024-09-11T23:47:03.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:02 smithi138 ceph-mon[35142]: pgmap v319: 97 pgs: 97 active+clean; 70 GiB data, 294 GiB used, 779 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 255 MiB/s wr, 459 op/s 2024-09-11T23:47:04.565 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:47:04.565 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:47:04.920 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:04.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:04 smithi133 ceph-mon[31430]: pgmap v320: 97 pgs: 97 active+clean; 70 GiB data, 291 GiB used, 782 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 276 MiB/s wr, 552 op/s 2024-09-11T23:47:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:04 smithi178 ceph-mon[35882]: pgmap v320: 97 pgs: 97 active+clean; 70 GiB data, 291 GiB used, 782 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 276 MiB/s wr, 552 op/s 2024-09-11T23:47:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:04 smithi138 ceph-mon[35142]: pgmap v320: 97 pgs: 97 active+clean; 70 GiB data, 291 GiB used, 782 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 276 MiB/s wr, 552 op/s 2024-09-11T23:47:05.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.419+0000 7ff7abe16640 1 -- 172.21.15.133:0/50252809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a4102790 msgr2=0x7ff7a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:05.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.419+0000 7ff7abe16640 1 --2- 172.21.15.133:0/50252809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a4102790 0x7ff7a41067e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff794009900 tx=0x7ff79402a410 comp rx=0 tx=0).stop 2024-09-11T23:47:05.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.419+0000 7ff7abe16640 1 -- 172.21.15.133:0/50252809 shutdown_connections 2024-09-11T23:47:05.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.419+0000 7ff7abe16640 1 --2- 172.21.15.133:0/50252809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a4106f10 0x7ff7a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.419+0000 7ff7abe16640 1 --2- 172.21.15.133:0/50252809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a4102790 0x7ff7a41067e0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.419+0000 7ff7abe16640 1 --2- 172.21.15.133:0/50252809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a4101de0 0x7ff7a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.419+0000 7ff7abe16640 1 -- 172.21.15.133:0/50252809 >> 172.21.15.133:0/50252809 conn(0x7ff7a40fd8e0 msgr2=0x7ff7a40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.419+0000 7ff7abe16640 1 -- 172.21.15.133:0/50252809 shutdown_connections 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.419+0000 7ff7abe16640 1 -- 172.21.15.133:0/50252809 wait complete. 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7abe16640 1 Processor -- start 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7abe16640 1 -- start start 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7abe16640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a4101de0 0x7ff7a41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7abe16640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a4102790 0x7ff7a41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7abe16640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a4106f10 0x7ff7a419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7abe16640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111f70 con 0x7ff7a4101de0 2024-09-11T23:47:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7abe16640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111df0 con 0x7ff7a4102790 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7abe16640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41120f0 con 0x7ff7a4106f10 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7a9bb2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a4101de0 0x7ff7a41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7a9bb2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a4101de0 0x7ff7a41984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42636/0 (socket says 172.21.15.133:42636) 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7a9bb2640 1 -- 172.21.15.133:0/2785624109 learned_addr learned my addr 172.21.15.133:0/2785624109 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a4102790 0x7ff7a41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.420+0000 7ff7aa3b3640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a4106f10 0x7ff7a419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff7a9bb2640 1 -- 172.21.15.133:0/2785624109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a4106f10 msgr2=0x7ff7a419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a4106f10 0x7ff7a419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff7a9bb2640 1 -- 172.21.15.133:0/2785624109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a4102790 msgr2=0x7ff7a41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a4102790 0x7ff7a41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff7a9bb2640 1 -- 172.21.15.133:0/2785624109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a419d460 con 0x7ff7a4101de0 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a4102790 0x7ff7a41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a4101de0 0x7ff7a41984b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000e9f0 tx=0x7ff7a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff79affd640 1 -- 172.21.15.133:0/2785624109 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000cd80 con 0x7ff7a4101de0 2024-09-11T23:47:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff79affd640 1 -- 172.21.15.133:0/2785624109 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7a0010910 con 0x7ff7a4101de0 2024-09-11T23:47:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff7abe16640 1 -- 172.21.15.133:0/2785624109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a419d750 con 0x7ff7a4101de0 2024-09-11T23:47:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff79affd640 1 -- 172.21.15.133:0/2785624109 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0018990 con 0x7ff7a4101de0 2024-09-11T23:47:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.421+0000 7ff7abe16640 1 -- 172.21.15.133:0/2785624109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a41a5030 con 0x7ff7a4101de0 2024-09-11T23:47:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff79affd640 1 -- 172.21.15.133:0/2785624109 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff7a0010ac0 con 0x7ff7a4101de0 2024-09-11T23:47:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7ae1ef740 1 Processor -- start 2024-09-11T23:47:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7ae1ef740 1 -- start start 2024-09-11T23:47:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf560e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf561340 0x55d7cf56fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf570150 0x55d7cf572540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d7cf4c6280 con 0x55d7cf560a20 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d7cf46db10 con 0x55d7cf561340 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d7cf4c2240 con 0x55d7cf570150 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7aa3b3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf560e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7aa3b3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf560e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42650/0 (socket says 172.21.15.133:42650) 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7aa3b3640 1 -- 172.21.15.133:0/1671738474 learned_addr learned my addr 172.21.15.133:0/1671738474 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff79affd640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7900a2cf0 0x7ff7900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.424+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/1671738474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf561340 0x55d7cf56fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/1671738474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf570150 0x55d7cf572540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff7aa3b3640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7900a2cf0 0x7ff7900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff7aa3b3640 1 -- 172.21.15.133:0/1671738474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf570150 msgr2=0x55d7cf572540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff7aa3b3640 1 --2- 172.21.15.133:0/1671738474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf570150 0x55d7cf572540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff7aa3b3640 1 -- 172.21.15.133:0/1671738474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf561340 msgr2=0x55d7cf56fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff7aa3b3640 1 --2- 172.21.15.133:0/1671738474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf561340 0x55d7cf56fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff7aa3b3640 1 -- 172.21.15.133:0/1671738474 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7cf47a1f0 con 0x55d7cf560a20 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/1671738474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf561340 0x55d7cf56fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff79affd640 1 -- 172.21.15.133:0/2785624109 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7ff7a00cf720 con 0x7ff7a4101de0 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff7aa3b3640 1 --2- 172.21.15.133:0/1671738474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf560e00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff79c014a60 tx=0x7ff79c014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46ac3cf49139554 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff78ffff640 1 -- 172.21.15.133:0/1671738474 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c01ec50 con 0x55d7cf560a20 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff78ffff640 1 -- 172.21.15.133:0/1671738474 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79c023070 con 0x55d7cf560a20 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.425+0000 7ff78ffff640 1 -- 172.21.15.133:0/1671738474 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c005670 con 0x55d7cf560a20 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1671738474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 msgr2=0x55d7cf560e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7aa3b3640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7900a2cf0 0x7ff7900a51b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff79c022000 tx=0x7ff79c01e000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1671738474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf560e00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff79c014a60 tx=0x7ff79c014f20 comp rx=0 tx=0).stop 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1671738474 shutdown_connections 2024-09-11T23:47:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1671738474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf570150 0x55d7cf572540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1671738474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf561340 0x55d7cf56fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1671738474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf560e00 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1671738474 >> 172.21.15.133:0/1671738474 conn(0x55d7cf55d0f0 msgr2=0x55d7cf55d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1671738474 shutdown_connections 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1671738474 wait complete. 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7ae1ef740 1 -- start start 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.426+0000 7ff7ae1ef740 1 Processor -- start 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf56d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf561340 0x55d7cf56d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf570150 0x55d7cf56db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d7cf572d20 con 0x55d7cf560a20 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d7cf480f10 con 0x55d7cf570150 2024-09-11T23:47:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d7cf572ea0 con 0x55d7cf561340 2024-09-11T23:47:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7ae1ef740 10 client.0 _ll_register_callbacks cb 0x55d7cf449a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7a9bb2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf56d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7a9bb2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf56d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42656/0 (socket says 172.21.15.133:42656) 2024-09-11T23:47:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7a9bb2640 1 -- 172.21.15.133:0/1673291013 learned_addr learned my addr 172.21.15.133:0/1673291013 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf561340 0x55d7cf56d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7aa3b3640 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf570150 0x55d7cf56db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7a9bb2640 1 -- 172.21.15.133:0/1673291013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf561340 msgr2=0x55d7cf56d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf561340 0x55d7cf56d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7a9bb2640 1 -- 172.21.15.133:0/1673291013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf570150 msgr2=0x55d7cf56db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf570150 0x55d7cf56db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.427+0000 7ff7a9bb2640 1 -- 172.21.15.133:0/1673291013 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7cf2376a0 con 0x55d7cf560a20 2024-09-11T23:47:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf56d0d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00ccf60 tx=0x7ff7a0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5069aa8750cffac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff78d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff78d7fa640 1 -- 172.21.15.133:0/1673291013 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00cfdd0 con 0x55d7cf560a20 2024-09-11T23:47:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff78d7fa640 1 -- 172.21.15.133:0/1673291013 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7ff7a0018b30 con 0x55d7cf560a20 2024-09-11T23:47:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1673291013 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79c023070 con 0x55d7cf560a20 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff78d7fa640 1 -- 172.21.15.133:0/1673291013 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7a0018d50 con 0x55d7cf560a20 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff7ae1ef740 10 client.18078 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1673291013 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d7cf57ea10 con 0x55d7cf560a20 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff78d7fa640 1 -- 172.21.15.133:0/1673291013 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79c023070 con 0x55d7cf560a20 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff78d7fa640 1 -- 172.21.15.133:0/1673291013 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff7a009b180 con 0x55d7cf560a20 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff7ae1ef740 10 client.18078 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff7ae1ef740 10 client.18078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff7ae1ef740 20 client.18078 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d7cf57fe20 0x55d7cf582210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff7ae1ef740 4 client.18078 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1673291013 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7cf4d37f0 con 0x55d7cf57fe20 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.428+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d7cf57fe20 0x55d7cf582210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.429+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d7cf57fe20 0x55d7cf582210 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8ecfde17a6f36af server_cookie=75da2c55b5c293fe in_seq=0 out_seq=0 2024-09-11T23:47:05.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.429+0000 7ff78d7fa640 10 client.18078 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.429+0000 7ff78d7fa640 1 -- 172.21.15.133:0/1673291013 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7cf4d37f0 con 0x55d7cf57fe20 2024-09-11T23:47:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.429+0000 7ff78d7fa640 10 client.18078 handle_command_reply: tid=0 2024-09-11T23:47:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.429+0000 7ff7ae1ef740 1 client.18078 shutdown 2024-09-11T23:47:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1673291013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 msgr2=0x55d7cf56d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf56d0d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00ccf60 tx=0x7ff7a0009310 comp rx=0 tx=0).stop 2024-09-11T23:47:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1673291013 shutdown_connections 2024-09-11T23:47:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d7cf57fe20 0x55d7cf582210 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf570150 0x55d7cf56db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf561340 0x55d7cf56d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff78d7fa640 0 client.18078 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1673291013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf560a20 0x55d7cf56d0d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1673291013 >> 172.21.15.133:0/1673291013 conn(0x55d7cf55d0f0 msgr2=0x55d7cf562e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1673291013 shutdown_connections 2024-09-11T23:47:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1673291013 wait complete. 2024-09-11T23:47:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.430+0000 7ff7ae1ef740 20 client.18078 trim_cache size 0 max 16384 2024-09-11T23:47:05.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7ae1ef740 1 Processor -- start 2024-09-11T23:47:05.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7ae1ef740 1 -- start start 2024-09-11T23:47:05.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf560a20 0x55d7cf587a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf587f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf570150 0x55d7cf56a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41ab670 con 0x55d7cf561340 2024-09-11T23:47:05.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41ad1b0 con 0x55d7cf560a20 2024-09-11T23:47:05.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41abd40 con 0x55d7cf570150 2024-09-11T23:47:05.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7a93b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf587f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7a93b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf587f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42664/0 (socket says 172.21.15.133:42664) 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7a93b1640 1 -- 172.21.15.133:0/1402842559 learned_addr learned my addr 172.21.15.133:0/1402842559 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/1402842559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf560a20 0x55d7cf587a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7aa3b3640 1 --2- 172.21.15.133:0/1402842559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf570150 0x55d7cf56a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7a93b1640 1 -- 172.21.15.133:0/1402842559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf570150 msgr2=0x55d7cf56a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/1402842559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf570150 0x55d7cf56a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7a93b1640 1 -- 172.21.15.133:0/1402842559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf560a20 msgr2=0x55d7cf587a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/1402842559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf560a20 0x55d7cf587a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7a93b1640 1 -- 172.21.15.133:0/1402842559 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7cf56f9a0 con 0x55d7cf561340 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.447+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/1402842559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf560a20 0x55d7cf587a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.448+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/1402842559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf587f80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff794009900 tx=0x7ff794032930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62431a7e3df5cf75 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.448+0000 7ff78f7fe640 1 -- 172.21.15.133:0/1402842559 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794035040 con 0x55d7cf561340 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.448+0000 7ff78f7fe640 1 -- 172.21.15.133:0/1402842559 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff794005020 con 0x55d7cf561340 2024-09-11T23:47:05.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.448+0000 7ff78f7fe640 1 -- 172.21.15.133:0/1402842559 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794004ba0 con 0x55d7cf561340 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.448+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1402842559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 msgr2=0x55d7cf587f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.448+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1402842559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf587f80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff794009900 tx=0x7ff794032930 comp rx=0 tx=0).stop 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.448+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1402842559 shutdown_connections 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.448+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1402842559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf570150 0x55d7cf56a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.448+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1402842559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf587f80 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.448+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/1402842559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf560a20 0x55d7cf587a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.448+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1402842559 >> 172.21.15.133:0/1402842559 conn(0x55d7cf55d0f0 msgr2=0x55d7cf56f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1402842559 shutdown_connections 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/1402842559 wait complete. 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7ae1ef740 1 -- start start 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7ae1ef740 1 Processor -- start 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf560a20 0x55d7cf574a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf574fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7ae1ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf570150 0x55d7cf5e29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41acae0 con 0x55d7cf561340 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41ac410 con 0x55d7cf570150 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7ae1ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41a9670 con 0x55d7cf560a20 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7ae1ef740 10 client.0 _ll_register_callbacks cb 0x7ff7a000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7a93b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf574fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.449+0000 7ff7a93b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf574fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42672/0 (socket says 172.21.15.133:42672) 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7a93b1640 1 -- 172.21.15.133:0/2204380354 learned_addr learned my addr 172.21.15.133:0/2204380354 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7aa3b3640 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf570150 0x55d7cf5e29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf560a20 0x55d7cf574a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7a93b1640 1 -- 172.21.15.133:0/2204380354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf560a20 msgr2=0x55d7cf574a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf560a20 0x55d7cf574a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7a93b1640 1 -- 172.21.15.133:0/2204380354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf570150 msgr2=0x55d7cf5e29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf570150 0x55d7cf5e29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7a93b1640 1 -- 172.21.15.133:0/2204380354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7cf5e3200 con 0x55d7cf561340 2024-09-11T23:47:05.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7aa3b3640 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf570150 0x55d7cf5e29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7a93b1640 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf574fa0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff794033000 tx=0x7ff794000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8512482cd2d0c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff78d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff78d7fa640 1 -- 172.21.15.133:0/2204380354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794057070 con 0x55d7cf561340 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff78d7fa640 1 -- 172.21.15.133:0/2204380354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7ff7940051d0 con 0x55d7cf561340 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/2204380354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff794005020 con 0x55d7cf561340 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff78d7fa640 1 -- 172.21.15.133:0/2204380354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff794063050 con 0x55d7cf561340 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7ae1ef740 10 client.18084 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.450+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/2204380354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d7cf5eb550 con 0x55d7cf561340 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.451+0000 7ff78d7fa640 1 -- 172.21.15.133:0/2204380354 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff794005020 con 0x55d7cf561340 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.451+0000 7ff78d7fa640 1 -- 172.21.15.133:0/2204380354 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff7940035f0 con 0x55d7cf561340 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.451+0000 7ff7ae1ef740 10 client.18084 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.451+0000 7ff7ae1ef740 10 client.18084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.451+0000 7ff7ae1ef740 20 client.18084 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.451+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d7cf5ebca0 0x55d7cf5ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.451+0000 7ff7ae1ef740 4 client.18084 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.451+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/2204380354 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d7cf4cdd00 con 0x55d7cf5ebca0 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.451+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d7cf5ebca0 0x55d7cf5ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.451+0000 7ff7a9bb2640 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d7cf5ebca0 0x55d7cf5ee0c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14268af28c3e97eb server_cookie=380b4b567ad1c74a in_seq=0 out_seq=0 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.451+0000 7ff78d7fa640 10 client.18084 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.452+0000 7ff78d7fa640 1 -- 172.21.15.133:0/2204380354 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d7cf4cdd00 con 0x55d7cf5ebca0 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.452+0000 7ff78d7fa640 10 client.18084 handle_command_reply: tid=0 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.452+0000 7ff7ae1ef740 1 client.18084 shutdown 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.452+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/2204380354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 msgr2=0x55d7cf574fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.452+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf574fa0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff794033000 tx=0x7ff794000bc0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/2204380354 shutdown_connections 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55d7cf5ebca0 0x55d7cf5ee0c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7cf570150 0x55d7cf5e29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7cf561340 0x55d7cf574fa0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7ae1ef740 1 --2- 172.21.15.133:0/2204380354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d7cf560a20 0x55d7cf574a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff78d7fa640 0 client.18084 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:05.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/2204380354 >> 172.21.15.133:0/2204380354 conn(0x55d7cf55d0f0 msgr2=0x55d7cf562510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:05.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/2204380354 shutdown_connections 2024-09-11T23:47:05.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7ae1ef740 1 -- 172.21.15.133:0/2204380354 wait complete. 2024-09-11T23:47:05.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7ae1ef740 20 client.18084 trim_cache size 0 max 16384 2024-09-11T23:47:05.464 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:47:05.464 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:47:05.464 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "aabde9bd-76b6-4b6e-8ce1-2b390ce6547d", 2024-09-11T23:47:05.464 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:47:05.464 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:47:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7abe16640 1 -- 172.21.15.133:0/2785624109 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7900a2cf0 msgr2=0x7ff7900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7abe16640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7900a2cf0 0x7ff7900a51b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff79c022000 tx=0x7ff79c01e000 comp rx=0 tx=0).stop 2024-09-11T23:47:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7abe16640 1 -- 172.21.15.133:0/2785624109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a4101de0 msgr2=0x7ff7a41984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7abe16640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a4101de0 0x7ff7a41984b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000e9f0 tx=0x7ff7a000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7abe16640 1 -- 172.21.15.133:0/2785624109 shutdown_connections 2024-09-11T23:47:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7abe16640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff7900a2cf0 0x7ff7900a51b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7abe16640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a4106f10 0x7ff7a419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7abe16640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a4102790 0x7ff7a41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7abe16640 1 --2- 172.21.15.133:0/2785624109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a4101de0 0x7ff7a41984b0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.453+0000 7ff7abe16640 1 -- 172.21.15.133:0/2785624109 >> 172.21.15.133:0/2785624109 conn(0x7ff7a40fd8e0 msgr2=0x7ff7a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.454+0000 7ff7abe16640 1 -- 172.21.15.133:0/2785624109 shutdown_connections 2024-09-11T23:47:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:05.454+0000 7ff7abe16640 1 -- 172.21.15.133:0/2785624109 wait complete. 2024-09-11T23:47:06.093 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:05 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:06.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:05 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:06.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:05 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:47:06.848 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:47:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:06 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:47:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:06 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:47:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:06 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:47:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:06 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:47:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:06 smithi138 ceph-mon[35142]: pgmap v321: 97 pgs: 97 active+clean; 70 GiB data, 287 GiB used, 786 GiB / 1.0 TiB avail; 378 KiB/s rd, 122 MiB/s wr, 316 op/s 2024-09-11T23:47:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:06 smithi138 ceph-mon[35142]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:47:07.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:06 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:47:07.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:06 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:47:07.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:06 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:47:07.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:06 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:47:07.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:06 smithi133 ceph-mon[31430]: pgmap v321: 97 pgs: 97 active+clean; 70 GiB data, 287 GiB used, 786 GiB / 1.0 TiB avail; 378 KiB/s rd, 122 MiB/s wr, 316 op/s 2024-09-11T23:47:07.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:06 smithi133 ceph-mon[31430]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:47:07.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:06 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:47:07.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:06 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:47:07.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:06 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:47:07.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:06 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:47:07.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:06 smithi178 ceph-mon[35882]: pgmap v321: 97 pgs: 97 active+clean; 70 GiB data, 287 GiB used, 786 GiB / 1.0 TiB avail; 378 KiB/s rd, 122 MiB/s wr, 316 op/s 2024-09-11T23:47:07.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:06 smithi178 ceph-mon[35882]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:47:08.066 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:08.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.552+0000 7f171a431640 1 -- 172.21.15.133:0/1729032230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1714102570 msgr2=0x7f17141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.552+0000 7f171a431640 1 --2- 172.21.15.133:0/1729032230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1714102570 0x7f17141029b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1708009900 tx=0x7f170802a480 comp rx=0 tx=0).stop 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.552+0000 7f171a431640 1 -- 172.21.15.133:0/1729032230 shutdown_connections 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.552+0000 7f171a431640 1 --2- 172.21.15.133:0/1729032230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1714102ef0 0x7f171410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.552+0000 7f171a431640 1 --2- 172.21.15.133:0/1729032230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1714102570 0x7f17141029b0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.552+0000 7f171a431640 1 --2- 172.21.15.133:0/1729032230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1714100e20 0x7f1714101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.552+0000 7f171a431640 1 -- 172.21.15.133:0/1729032230 >> 172.21.15.133:0/1729032230 conn(0x7f17140fc7d0 msgr2=0x7f17140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.552+0000 7f171a431640 1 -- 172.21.15.133:0/1729032230 shutdown_connections 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.552+0000 7f171a431640 1 -- 172.21.15.133:0/1729032230 wait complete. 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f171a431640 1 Processor -- start 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f171a431640 1 -- start start 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f171a431640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1714100e20 0x7f17140fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f171a431640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1714102570 0x7f17140fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f171a431640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1714102ef0 0x7f17140fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f171a431640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1714110f80 con 0x7f1714102570 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f171a431640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1714110e00 con 0x7f1714100e20 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f171a431640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1714111100 con 0x7f1714102ef0 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f1713fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1714100e20 0x7f17140fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f1713fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1714100e20 0x7f17140fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59228/0 (socket says 172.21.15.133:59228) 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f17137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1714102570 0x7f17140fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f1713fff640 1 -- 172.21.15.133:0/3604967010 learned_addr learned my addr 172.21.15.133:0/3604967010 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.553+0000 7f17189ce640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1714102ef0 0x7f17140fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.554+0000 7f1713fff640 1 -- 172.21.15.133:0/3604967010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1714102ef0 msgr2=0x7f17140fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.554+0000 7f1713fff640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1714102ef0 0x7f17140fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.554+0000 7f1713fff640 1 -- 172.21.15.133:0/3604967010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1714102570 msgr2=0x7f17140fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:08.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.554+0000 7f1713fff640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1714102570 0x7f17140fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.554+0000 7f1713fff640 1 -- 172.21.15.133:0/3604967010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17141a69c0 con 0x7f1714100e20 2024-09-11T23:47:08.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.554+0000 7f1713fff640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1714100e20 0x7f17140fcc80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f170400e9f0 tx=0x7f170400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:08.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.554+0000 7f17117fa640 1 -- 172.21.15.133:0/3604967010 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170400cd80 con 0x7f1714100e20 2024-09-11T23:47:08.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.554+0000 7f17117fa640 1 -- 172.21.15.133:0/3604967010 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1704010910 con 0x7f1714100e20 2024-09-11T23:47:08.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.554+0000 7f171a431640 1 -- 172.21.15.133:0/3604967010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17141a6cb0 con 0x7f1714100e20 2024-09-11T23:47:08.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.554+0000 7f17117fa640 1 -- 172.21.15.133:0/3604967010 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704018990 con 0x7f1714100e20 2024-09-11T23:47:08.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.554+0000 7f171a431640 1 -- 172.21.15.133:0/3604967010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17141a71c0 con 0x7f1714100e20 2024-09-11T23:47:08.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.556+0000 7f17117fa640 1 -- 172.21.15.133:0/3604967010 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1704010ac0 con 0x7f1714100e20 2024-09-11T23:47:08.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f171c80a740 1 Processor -- start 2024-09-11T23:47:08.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f171c80a740 1 -- start start 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17117fa640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16fc0a2cf0 0x7f16fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f171c80a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7de7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f171c80a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de7dc0 0x560aa7df2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f171c80a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7df2680 0x560aa7df4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f171c80a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560aa7d4d2e0 con 0x560aa7de7dc0 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f171c80a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560aa7cf4ce0 con 0x560aa7de74a0 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17117fa640 1 -- 172.21.15.133:0/3604967010 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f17040cf730 con 0x7f1714100e20 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f171c80a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560aa7d492a0 con 0x560aa7df2680 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17189ce640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16fc0a2cf0 0x7f16fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de7dc0 0x560aa7df2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de7dc0 0x560aa7df2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42692/0 (socket says 172.21.15.133:42692) 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17137fe640 1 -- 172.21.15.133:0/3457055119 learned_addr learned my addr 172.21.15.133:0/3457055119 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:08.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f1713fff640 1 --2- 172.21.15.133:0/3457055119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7de7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17189ce640 1 --2- 172.21.15.133:0/3457055119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7df2680 0x560aa7df4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17137fe640 1 -- 172.21.15.133:0/3457055119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7df2680 msgr2=0x560aa7df4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17137fe640 1 --2- 172.21.15.133:0/3457055119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7df2680 0x560aa7df4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17137fe640 1 -- 172.21.15.133:0/3457055119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 msgr2=0x560aa7de7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17137fe640 1 --2- 172.21.15.133:0/3457055119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7de7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.557+0000 7f17137fe640 1 -- 172.21.15.133:0/3457055119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aa7cd9440 con 0x560aa7de7dc0 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.558+0000 7f17189ce640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16fc0a2cf0 0x7f16fc0a51b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f170c0136d0 tx=0x7f170c013ad0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.558+0000 7f17137fe640 1 --2- 172.21.15.133:0/3457055119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de7dc0 0x560aa7df2140 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1708009900 tx=0x7f17080040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd10ab516e9a18 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.558+0000 7f16fa7fc640 1 -- 172.21.15.133:0/3457055119 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708035030 con 0x560aa7de7dc0 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.558+0000 7f16fa7fc640 1 -- 172.21.15.133:0/3457055119 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1708040070 con 0x560aa7de7dc0 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.558+0000 7f16fa7fc640 1 -- 172.21.15.133:0/3457055119 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708031930 con 0x560aa7de7dc0 2024-09-11T23:47:08.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.558+0000 7f171c80a740 1 -- 172.21.15.133:0/3457055119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de7dc0 msgr2=0x560aa7df2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 --2- 172.21.15.133:0/3457055119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de7dc0 0x560aa7df2140 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1708009900 tx=0x7f17080040c0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 -- 172.21.15.133:0/3457055119 shutdown_connections 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 --2- 172.21.15.133:0/3457055119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7df2680 0x560aa7df4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 --2- 172.21.15.133:0/3457055119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de7dc0 0x560aa7df2140 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 --2- 172.21.15.133:0/3457055119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7de7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 -- 172.21.15.133:0/3457055119 >> 172.21.15.133:0/3457055119 conn(0x560aa7de3a30 msgr2=0x560aa7de3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 -- 172.21.15.133:0/3457055119 shutdown_connections 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 -- 172.21.15.133:0/3457055119 wait complete. 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 -- start start 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 Processor -- start 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7df1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 0x560aa7def1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7df2680 0x560aa7def710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560aa7df5250 con 0x560aa7df2680 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560aa7d08050 con 0x560aa7de74a0 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.559+0000 7f171c80a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560aa7df53d0 con 0x560aa7de7dc0 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f171c80a740 10 client.0 _ll_register_callbacks cb 0x560aa7d48160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f17189ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7df2680 0x560aa7def710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f17189ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7df2680 0x560aa7def710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42704/0 (socket says 172.21.15.133:42704) 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f17189ce640 1 -- 172.21.15.133:0/4232089122 learned_addr learned my addr 172.21.15.133:0/4232089122 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f17137fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 0x560aa7def1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f1713fff640 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7df1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f17189ce640 1 -- 172.21.15.133:0/4232089122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 msgr2=0x560aa7def1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f17189ce640 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 0x560aa7def1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f17189ce640 1 -- 172.21.15.133:0/4232089122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 msgr2=0x560aa7df1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f17189ce640 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7df1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f17189ce640 1 -- 172.21.15.133:0/4232089122 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aa7abe3c0 con 0x560aa7df2680 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f17137fe640 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 0x560aa7def1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f17189ce640 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7df2680 0x560aa7def710 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f170c00fa90 tx=0x7f170c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbef8af8675f29e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f16f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f16f37fe640 1 -- 172.21.15.133:0/4232089122 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c010e00 con 0x560aa7df2680 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f16f37fe640 1 -- 172.21.15.133:0/4232089122 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f170c01f440 con 0x560aa7df2680 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f171c80a740 1 -- 172.21.15.133:0/4232089122 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1708040070 con 0x560aa7df2680 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f16f37fe640 1 -- 172.21.15.133:0/4232089122 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f170c02d050 con 0x560aa7df2680 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f171c80a740 10 client.18096 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.560+0000 7f171c80a740 1 -- 172.21.15.133:0/4232089122 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560aa7e00bd0 con 0x560aa7df2680 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.561+0000 7f16f37fe640 1 -- 172.21.15.133:0/4232089122 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1708040070 con 0x560aa7df2680 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.561+0000 7f16f37fe640 1 -- 172.21.15.133:0/4232089122 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f170c0286b0 con 0x560aa7df2680 2024-09-11T23:47:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.561+0000 7f171c80a740 10 client.18096 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:08.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.561+0000 7f171c80a740 10 client.18096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:08.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.561+0000 7f171c80a740 20 client.18096 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:08.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.561+0000 7f171c80a740 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560aa7e02090 0x560aa7e04530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.561+0000 7f171c80a740 4 client.18096 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:08.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.561+0000 7f171c80a740 1 -- 172.21.15.133:0/4232089122 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aa7d5a850 con 0x560aa7e02090 2024-09-11T23:47:08.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.561+0000 7f1713fff640 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560aa7e02090 0x560aa7e04530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.561+0000 7f16f37fe640 10 client.18096 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:08.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.561+0000 7f1713fff640 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560aa7e02090 0x560aa7e04530 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8cba7e221ca39b8 server_cookie=2e170b801d8c8593 in_seq=0 out_seq=0 2024-09-11T23:47:08.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f16f37fe640 1 -- 172.21.15.133:0/4232089122 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560aa7d5a850 con 0x560aa7e02090 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f16f37fe640 10 client.18096 handle_command_reply: tid=0 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 1 client.18096 shutdown 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 1 -- 172.21.15.133:0/4232089122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7df2680 msgr2=0x560aa7def710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7df2680 0x560aa7def710 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f170c00fa90 tx=0x7f170c004ab0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 1 -- 172.21.15.133:0/4232089122 shutdown_connections 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560aa7e02090 0x560aa7e04530 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7df2680 0x560aa7def710 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 0x560aa7def1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 1 --2- 172.21.15.133:0/4232089122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7df1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f16f37fe640 0 client.18096 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 1 -- 172.21.15.133:0/4232089122 >> 172.21.15.133:0/4232089122 conn(0x560aa7de3a30 msgr2=0x560aa7de9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 1 -- 172.21.15.133:0/4232089122 shutdown_connections 2024-09-11T23:47:08.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 1 -- 172.21.15.133:0/4232089122 wait complete. 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.562+0000 7f171c80a740 20 client.18096 trim_cache size 0 max 16384 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.579+0000 7f171c80a740 1 Processor -- start 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.579+0000 7f171c80a740 1 -- start start 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.579+0000 7f171c80a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de3a30 0x560aa7df1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.579+0000 7f171c80a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7e09e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.579+0000 7f171c80a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 0x560aa7e0a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.579+0000 7f171c80a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f17141af990 con 0x560aa7de3a30 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.579+0000 7f171c80a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17141b14d0 con 0x560aa7de74a0 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.579+0000 7f171c80a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17141b0060 con 0x560aa7de7dc0 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f1713fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de3a30 0x560aa7df1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f17137fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7e09e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f1713fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de3a30 0x560aa7df1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42710/0 (socket says 172.21.15.133:42710) 2024-09-11T23:47:08.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f1713fff640 1 -- 172.21.15.133:0/3031669415 learned_addr learned my addr 172.21.15.133:0/3031669415 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f17189ce640 1 --2- 172.21.15.133:0/3031669415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 0x560aa7e0a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f1713fff640 1 -- 172.21.15.133:0/3031669415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 msgr2=0x560aa7e0a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f1713fff640 1 --2- 172.21.15.133:0/3031669415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 0x560aa7e0a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f1713fff640 1 -- 172.21.15.133:0/3031669415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 msgr2=0x560aa7e09e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f1713fff640 1 --2- 172.21.15.133:0/3031669415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7e09e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f1713fff640 1 -- 172.21.15.133:0/3031669415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aa7de3860 con 0x560aa7de3a30 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f17189ce640 1 --2- 172.21.15.133:0/3031669415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 0x560aa7e0a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f17137fe640 1 --2- 172.21.15.133:0/3031669415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7e09e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f1713fff640 1 --2- 172.21.15.133:0/3031669415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de3a30 0x560aa7df1140 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1704009380 tx=0x7f1704097fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09bb0c5187899e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f16f9ffb640 1 -- 172.21.15.133:0/3031669415 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17040cd110 con 0x560aa7de3a30 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f16f9ffb640 1 -- 172.21.15.133:0/3031669415 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17040b1030 con 0x560aa7de3a30 2024-09-11T23:47:08.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.580+0000 7f16f9ffb640 1 -- 172.21.15.133:0/3031669415 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170409b3c0 con 0x560aa7de3a30 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.581+0000 7f171c80a740 1 -- 172.21.15.133:0/3031669415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de3a30 msgr2=0x560aa7df1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.581+0000 7f171c80a740 1 --2- 172.21.15.133:0/3031669415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de3a30 0x560aa7df1140 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1704009380 tx=0x7f1704097fb0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.581+0000 7f171c80a740 1 -- 172.21.15.133:0/3031669415 shutdown_connections 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.581+0000 7f171c80a740 1 --2- 172.21.15.133:0/3031669415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de7dc0 0x560aa7e0a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.581+0000 7f171c80a740 1 --2- 172.21.15.133:0/3031669415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de74a0 0x560aa7e09e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.581+0000 7f171c80a740 1 --2- 172.21.15.133:0/3031669415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de3a30 0x560aa7df1140 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.581+0000 7f171c80a740 1 -- 172.21.15.133:0/3031669415 >> 172.21.15.133:0/3031669415 conn(0x560aa7def870 msgr2=0x560aa7df0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.581+0000 7f171c80a740 1 -- 172.21.15.133:0/3031669415 shutdown_connections 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.581+0000 7f171c80a740 1 -- 172.21.15.133:0/3031669415 wait complete. 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.581+0000 7f171c80a740 1 -- start start 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f171c80a740 1 Processor -- start 2024-09-11T23:47:08.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f171c80a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de3a30 0x560aa7dfae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f171c80a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de74a0 0x560aa7df8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f171c80a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de7dc0 0x560aa7df8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f171c80a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f17141b0e00 con 0x560aa7de74a0 2024-09-11T23:47:08.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f171c80a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17141b0730 con 0x560aa7de7dc0 2024-09-11T23:47:08.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f171c80a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17141ad990 con 0x560aa7de3a30 2024-09-11T23:47:08.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f171c80a740 10 client.0 _ll_register_callbacks cb 0x7f170c00f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:08.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f17137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de74a0 0x560aa7df8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f17137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de74a0 0x560aa7df8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42724/0 (socket says 172.21.15.133:42724) 2024-09-11T23:47:08.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f17137fe640 1 -- 172.21.15.133:0/1111915184 learned_addr learned my addr 172.21.15.133:0/1111915184 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:08.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f17189ce640 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de7dc0 0x560aa7df8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f1713fff640 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de3a30 0x560aa7dfae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f17137fe640 1 -- 172.21.15.133:0/1111915184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de3a30 msgr2=0x560aa7dfae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:08.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f17137fe640 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de3a30 0x560aa7dfae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f17137fe640 1 -- 172.21.15.133:0/1111915184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de7dc0 msgr2=0x560aa7df8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:08.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f17137fe640 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de7dc0 0x560aa7df8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f17137fe640 1 -- 172.21.15.133:0/1111915184 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aa7df8e90 con 0x560aa7de74a0 2024-09-11T23:47:08.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.582+0000 7f1713fff640 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de3a30 0x560aa7dfae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:47:08.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.583+0000 7f17137fe640 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de74a0 0x560aa7df8150 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x560aa7de4890 tx=0x7f1708031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c6874eeca624b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:08.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.583+0000 7f16f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.583+0000 7f16f37fe640 1 -- 172.21.15.133:0/1111915184 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708035070 con 0x560aa7de74a0 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.583+0000 7f16f37fe640 1 -- 172.21.15.133:0/1111915184 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f1708031cf0 con 0x560aa7de74a0 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.583+0000 7f171c80a740 1 -- 172.21.15.133:0/1111915184 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17040b1030 con 0x560aa7de74a0 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.583+0000 7f16f37fe640 1 -- 172.21.15.133:0/1111915184 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1708005020 con 0x560aa7de74a0 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.583+0000 7f171c80a740 10 client.18102 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.583+0000 7f171c80a740 1 -- 172.21.15.133:0/1111915184 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560aa7e6cde0 con 0x560aa7de74a0 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.583+0000 7f16f37fe640 1 -- 172.21.15.133:0/1111915184 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17040b1030 con 0x560aa7de74a0 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f16f37fe640 1 -- 172.21.15.133:0/1111915184 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f17080035f0 con 0x560aa7de74a0 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f171c80a740 10 client.18102 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f171c80a740 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f171c80a740 20 client.18102 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f171c80a740 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560aa7df2680 0x560aa7e6f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f171c80a740 4 client.18102 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:47:08.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f171c80a740 1 -- 172.21.15.133:0/1111915184 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560aa7d54d60 con 0x560aa7df2680 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f1713fff640 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560aa7df2680 0x560aa7e6f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f1713fff640 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560aa7df2680 0x560aa7e6f120 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7ca226b900dbf49 server_cookie=494dd827061dc210 in_seq=0 out_seq=0 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f16f37fe640 10 client.18102 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f16f37fe640 1 -- 172.21.15.133:0/1111915184 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560aa7d54d60 con 0x560aa7df2680 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f16f37fe640 10 client.18102 handle_command_reply: tid=0 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.584+0000 7f171c80a740 1 client.18102 shutdown 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f171c80a740 1 -- 172.21.15.133:0/1111915184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de74a0 msgr2=0x560aa7df8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f171c80a740 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de74a0 0x560aa7df8150 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x560aa7de4890 tx=0x7f1708031770 comp rx=0 tx=0).stop 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f171c80a740 1 -- 172.21.15.133:0/1111915184 shutdown_connections 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f171c80a740 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560aa7df2680 0x560aa7e6f120 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f171c80a740 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560aa7de7dc0 0x560aa7df8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f171c80a740 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560aa7de74a0 0x560aa7df8150 secure :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x560aa7de4890 tx=0x7f1708031770 comp rx=0 tx=0).stop 2024-09-11T23:47:08.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f171c80a740 1 --2- 172.21.15.133:0/1111915184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aa7de3a30 0x560aa7dfae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f171c80a740 1 -- 172.21.15.133:0/1111915184 >> 172.21.15.133:0/1111915184 conn(0x560aa7def870 msgr2=0x560aa7de2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:08.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f16f37fe640 0 client.18102 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:08.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f171c80a740 1 -- 172.21.15.133:0/1111915184 shutdown_connections 2024-09-11T23:47:08.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f171c80a740 1 -- 172.21.15.133:0/1111915184 wait complete. 2024-09-11T23:47:08.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.585+0000 7f171c80a740 20 client.18102 trim_cache size 0 max 16384 2024-09-11T23:47:08.603 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:47:08.603 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:47:08.603 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-09-11T23:47:08.603 INFO:teuthology.orchestra.run.smithi133.stdout: "aabde9bd-76b6-4b6e-8ce1-2b390ce6547d": { 2024-09-11T23:47:08.603 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-09-11T23:47:08.603 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "aabde9bd-76b6-4b6e-8ce1-2b390ce6547d", 2024-09-11T23:47:08.604 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2024-09-11T23:47:08.604 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:08.604 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:08.604 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:47:08.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 -- 172.21.15.133:0/3604967010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16fc0a2cf0 msgr2=0x7f16fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:08.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16fc0a2cf0 0x7f16fc0a51b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f170c0136d0 tx=0x7f170c013ad0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 -- 172.21.15.133:0/3604967010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1714100e20 msgr2=0x7f17140fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:08.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1714100e20 0x7f17140fcc80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f170400e9f0 tx=0x7f170400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 -- 172.21.15.133:0/3604967010 shutdown_connections 2024-09-11T23:47:08.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f16fc0a2cf0 0x7f16fc0a51b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1714102ef0 0x7f17140fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1714102570 0x7f17140fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 --2- 172.21.15.133:0/3604967010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1714100e20 0x7f17140fcc80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:08.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 -- 172.21.15.133:0/3604967010 >> 172.21.15.133:0/3604967010 conn(0x7f17140fc7d0 msgr2=0x7f171406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:08.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 -- 172.21.15.133:0/3604967010 shutdown_connections 2024-09-11T23:47:08.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:08.586+0000 7f171a431640 1 -- 172.21.15.133:0/3604967010 wait complete. 2024-09-11T23:47:08.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: pgmap v323: 97 pgs: 97 active+clean; 70 GiB data, 287 GiB used, 786 GiB / 1.0 TiB avail; 29 KiB/s rd, 65 MiB/s wr, 254 op/s 2024-09-11T23:47:08.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.791 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:08 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: pgmap v323: 97 pgs: 97 active+clean; 70 GiB data, 287 GiB used, 786 GiB / 1.0 TiB avail; 29 KiB/s rd, 65 MiB/s wr, 254 op/s 2024-09-11T23:47:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:08 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:08.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: pgmap v323: 97 pgs: 97 active+clean; 70 GiB data, 287 GiB used, 786 GiB / 1.0 TiB avail; 29 KiB/s rd, 65 MiB/s wr, 254 op/s 2024-09-11T23:47:08.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:08.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:08 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:09.233 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:47:09.476 INFO:tasks.cephfs.filesystem:scrub status for tag:aabde9bd-76b6-4b6e-8ce1-2b390ce6547d - {'path': '/', 'tag': 'aabde9bd-76b6-4b6e-8ce1-2b390ce6547d', 'options': 'recursive,force'} 2024-09-11T23:47:09.546 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:09.938 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:09 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi138 to 176.0M 2024-09-11T23:47:09.938 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:09 smithi133 ceph-mon[31430]: Unable to set osd_memory_target on smithi138 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:47:09.938 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:09 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T23:47:09.938 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:09 smithi133 ceph-mon[31430]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T23:47:09.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:09 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi138 to 176.0M 2024-09-11T23:47:09.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:09 smithi138 ceph-mon[35142]: Unable to set osd_memory_target on smithi138 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:47:09.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:09 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T23:47:09.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:09 smithi138 ceph-mon[35142]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T23:47:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:09 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi138 to 176.0M 2024-09-11T23:47:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:09 smithi178 ceph-mon[35882]: Unable to set osd_memory_target on smithi138 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-11T23:47:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:09 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T23:47:10.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:09 smithi178 ceph-mon[35882]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T23:47:10.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.131+0000 7fec60f70640 1 -- 172.21.15.133:0/3729832609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec5c11b140 msgr2=0x7fec5c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:10.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.131+0000 7fec60f70640 1 --2- 172.21.15.133:0/3729832609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec5c11b140 0x7fec5c11d530 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fec54009900 tx=0x7fec5402a440 comp rx=0 tx=0).stop 2024-09-11T23:47:10.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.132+0000 7fec60f70640 1 -- 172.21.15.133:0/3729832609 shutdown_connections 2024-09-11T23:47:10.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.132+0000 7fec60f70640 1 --2- 172.21.15.133:0/3729832609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec5c11b140 0x7fec5c11d530 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.132+0000 7fec60f70640 1 --2- 172.21.15.133:0/3729832609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c06ea30 0x7fec5c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.132+0000 7fec60f70640 1 --2- 172.21.15.133:0/3729832609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec5c110190 0x7fec5c06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.132+0000 7fec60f70640 1 -- 172.21.15.133:0/3729832609 >> 172.21.15.133:0/3729832609 conn(0x7fec5c073180 msgr2=0x7fec5c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:10.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.132+0000 7fec60f70640 1 -- 172.21.15.133:0/3729832609 shutdown_connections 2024-09-11T23:47:10.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.132+0000 7fec60f70640 1 -- 172.21.15.133:0/3729832609 wait complete. 2024-09-11T23:47:10.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.134+0000 7fec60f70640 1 Processor -- start 2024-09-11T23:47:10.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.135+0000 7fec60f70640 1 -- start start 2024-09-11T23:47:10.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.136+0000 7fec60f70640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec5c06ea30 0x7fec5c1a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.136+0000 7fec60f70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec5c110190 0x7fec5c1a7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.136+0000 7fec60f70640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c11b140 0x7fec5c1abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.137+0000 7fec52ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec5c110190 0x7fec5c1a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:10.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.137+0000 7fec52ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec5c110190 0x7fec5c1a7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42738/0 (socket says 172.21.15.133:42738) 2024-09-11T23:47:10.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.137+0000 7fec60f70640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c120ce0 con 0x7fec5c110190 2024-09-11T23:47:10.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.137+0000 7fec60f70640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c120b60 con 0x7fec5c11b140 2024-09-11T23:47:10.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.137+0000 7fec60f70640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c120e60 con 0x7fec5c06ea30 2024-09-11T23:47:10.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.137+0000 7fec52ffd640 1 -- 172.21.15.133:0/2078225452 learned_addr learned my addr 172.21.15.133:0/2078225452 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:10.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.138+0000 7fec5bfff640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c11b140 0x7fec5c1abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:10.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.138+0000 7fec52ffd640 1 -- 172.21.15.133:0/2078225452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec5c06ea30 msgr2=0x7fec5c1a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:10.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.138+0000 7fec52ffd640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec5c06ea30 0x7fec5c1a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.138+0000 7fec52ffd640 1 -- 172.21.15.133:0/2078225452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c11b140 msgr2=0x7fec5c1abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:10.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.138+0000 7fec52ffd640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c11b140 0x7fec5c1abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.138+0000 7fec52ffd640 1 -- 172.21.15.133:0/2078225452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c1ac1d0 con 0x7fec5c110190 2024-09-11T23:47:10.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.139+0000 7fec52ffd640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec5c110190 0x7fec5c1a7760 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fec4400b450 tx=0x7fec4400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:10.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.139+0000 7fec597fa640 1 -- 172.21.15.133:0/2078225452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44015020 con 0x7fec5c110190 2024-09-11T23:47:10.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.139+0000 7fec60f70640 1 -- 172.21.15.133:0/2078225452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5c1ac4c0 con 0x7fec5c110190 2024-09-11T23:47:10.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.139+0000 7fec60f70640 1 -- 172.21.15.133:0/2078225452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5c1b3da0 con 0x7fec5c110190 2024-09-11T23:47:10.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.139+0000 7fec597fa640 1 -- 172.21.15.133:0/2078225452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec44011d70 con 0x7fec5c110190 2024-09-11T23:47:10.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.140+0000 7fec597fa640 1 -- 172.21.15.133:0/2078225452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4400f340 con 0x7fec5c110190 2024-09-11T23:47:10.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.141+0000 7fec60f70640 1 -- 172.21.15.133:0/2078225452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c005290 con 0x7fec5c110190 2024-09-11T23:47:10.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.146+0000 7fec597fa640 1 -- 172.21.15.133:0/2078225452 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fec44018020 con 0x7fec5c110190 2024-09-11T23:47:10.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.148+0000 7fec597fa640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec480a2df0 0x7fec480a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:10.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.148+0000 7fec597fa640 1 -- 172.21.15.133:0/2078225452 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fec440cf220 con 0x7fec5c110190 2024-09-11T23:47:10.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.151+0000 7fec5b7fe640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec480a2df0 0x7fec480a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:10.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.153+0000 7fec597fa640 1 -- 172.21.15.133:0/2078225452 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec4401f020 con 0x7fec5c110190 2024-09-11T23:47:10.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.155+0000 7fec5b7fe640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec480a2df0 0x7fec480a52b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00abb0 tx=0x7fec4c009250 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.411+0000 7fec60f70640 1 -- 172.21.15.133:0/2078225452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec3c005580 con 0x7fec5c110190 2024-09-11T23:47:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.412+0000 7fec597fa640 1 -- 172.21.15.133:0/2078225452 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7fec4400d040 con 0x7fec5c110190 2024-09-11T23:47:10.412 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:47:10.412 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:47:10.412 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:47:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.414+0000 7fec60f70640 1 -- 172.21.15.133:0/2078225452 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec480a2df0 msgr2=0x7fec480a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.414+0000 7fec60f70640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec480a2df0 0x7fec480a52b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00abb0 tx=0x7fec4c009250 comp rx=0 tx=0).stop 2024-09-11T23:47:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.414+0000 7fec60f70640 1 -- 172.21.15.133:0/2078225452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec5c110190 msgr2=0x7fec5c1a7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.414+0000 7fec60f70640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec5c110190 0x7fec5c1a7760 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fec4400b450 tx=0x7fec4400b910 comp rx=0 tx=0).stop 2024-09-11T23:47:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.414+0000 7fec60f70640 1 -- 172.21.15.133:0/2078225452 shutdown_connections 2024-09-11T23:47:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.414+0000 7fec60f70640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fec480a2df0 0x7fec480a52b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.414+0000 7fec60f70640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec5c11b140 0x7fec5c1abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.414+0000 7fec60f70640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec5c110190 0x7fec5c1a7760 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.414+0000 7fec60f70640 1 --2- 172.21.15.133:0/2078225452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec5c06ea30 0x7fec5c1a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.414+0000 7fec60f70640 1 -- 172.21.15.133:0/2078225452 >> 172.21.15.133:0/2078225452 conn(0x7fec5c073180 msgr2=0x7fec5c119310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.414+0000 7fec60f70640 1 -- 172.21.15.133:0/2078225452 shutdown_connections 2024-09-11T23:47:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:10.415+0000 7fec60f70640 1 -- 172.21.15.133:0/2078225452 wait complete. 2024-09-11T23:47:10.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:10 smithi133 ceph-mon[31430]: pgmap v324: 97 pgs: 97 active+clean; 60 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 32 KiB/s rd, 59 MiB/s wr, 235 op/s 2024-09-11T23:47:10.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:10 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:47:10.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:10 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:10.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:10 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:10.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:10 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2078225452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:10 smithi138 ceph-mon[35142]: pgmap v324: 97 pgs: 97 active+clean; 60 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 32 KiB/s rd, 59 MiB/s wr, 235 op/s 2024-09-11T23:47:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:10 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:47:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:10 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:10 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:10.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:10 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2078225452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:10 smithi178 ceph-mon[35882]: pgmap v324: 97 pgs: 97 active+clean; 60 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 32 KiB/s rd, 59 MiB/s wr, 235 op/s 2024-09-11T23:47:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:10 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:47:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:10 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:10 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:11.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:10 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2078225452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:11.127 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id d0886d4a --release --await' 2024-09-11T23:47:11.127 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id d0886d4a --release --await 2024-09-11T23:47:11.376 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:11.817 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:11 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:11.817 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:11 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:11.817 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:11 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:11.817 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:11 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:11.817 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:11 smithi133 ceph-mon[31430]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-11T23:47:11.817 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:11 smithi133 ceph-mon[31430]: Unable to set osd_memory_target on smithi133 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-11T23:47:11.817 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:11 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:11.817 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:11 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:11.817 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:11 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:11.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.817+0000 7f7623258640 1 -- 172.21.15.133:0/1129575398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f761c102ef0 msgr2=0x7f761c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:11.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.817+0000 7f7623258640 1 --2- 172.21.15.133:0/1129575398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f761c102ef0 0x7f761c10da40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7614009ca0 tx=0x7f761402a950 comp rx=0 tx=0).stop 2024-09-11T23:47:11.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.817+0000 7f7623258640 1 -- 172.21.15.133:0/1129575398 shutdown_connections 2024-09-11T23:47:11.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.817+0000 7f7623258640 1 --2- 172.21.15.133:0/1129575398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f761c102ef0 0x7f761c10da40 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.817+0000 7f7623258640 1 --2- 172.21.15.133:0/1129575398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761c102570 0x7f761c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.817+0000 7f7623258640 1 --2- 172.21.15.133:0/1129575398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f761c100e20 0x7f761c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.817+0000 7f7623258640 1 -- 172.21.15.133:0/1129575398 >> 172.21.15.133:0/1129575398 conn(0x7f761c0fc7d0 msgr2=0x7f761c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.817+0000 7f7623258640 1 -- 172.21.15.133:0/1129575398 shutdown_connections 2024-09-11T23:47:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.817+0000 7f7623258640 1 -- 172.21.15.133:0/1129575398 wait complete. 2024-09-11T23:47:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.817+0000 7f7623258640 1 Processor -- start 2024-09-11T23:47:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.817+0000 7f7623258640 1 -- start start 2024-09-11T23:47:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7623258640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f761c100e20 0x7f761c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7623258640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761c102570 0x7f761c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7623258640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f761c102ef0 0x7f761c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7623258640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f761c110f80 con 0x7f761c100e20 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7623258640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f761c110e00 con 0x7f761c102ef0 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7623258640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f761c111100 con 0x7f761c102570 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7620ff4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f761c100e20 0x7f761c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7620ff4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f761c100e20 0x7f761c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42764/0 (socket says 172.21.15.133:42764) 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7620ff4640 1 -- 172.21.15.133:0/3827825639 learned_addr learned my addr 172.21.15.133:0/3827825639 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f76217f5640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f761c102ef0 0x7f761c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f761bfff640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761c102570 0x7f761c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7620ff4640 1 -- 172.21.15.133:0/3827825639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761c102570 msgr2=0x7f761c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7620ff4640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761c102570 0x7f761c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7620ff4640 1 -- 172.21.15.133:0/3827825639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f761c102ef0 msgr2=0x7f761c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7620ff4640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f761c102ef0 0x7f761c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7620ff4640 1 -- 172.21.15.133:0/3827825639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c1a6a20 con 0x7f761c100e20 2024-09-11T23:47:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f76217f5640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f761c102ef0 0x7f761c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:47:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f761bfff640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761c102570 0x7f761c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:47:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7620ff4640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f761c100e20 0x7f761c0fcc80 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f761000e9f0 tx=0x7f761000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7619ffb640 1 -- 172.21.15.133:0/3827825639 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761000cd80 con 0x7f761c100e20 2024-09-11T23:47:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7619ffb640 1 -- 172.21.15.133:0/3827825639 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7610010910 con 0x7f761c100e20 2024-09-11T23:47:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7623258640 1 -- 172.21.15.133:0/3827825639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c1a6d10 con 0x7f761c100e20 2024-09-11T23:47:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.818+0000 7f7619ffb640 1 -- 172.21.15.133:0/3827825639 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610018990 con 0x7f761c100e20 2024-09-11T23:47:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.819+0000 7f7623258640 1 -- 172.21.15.133:0/3827825639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c1a71f0 con 0x7f761c100e20 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.821+0000 7f7625631740 1 Processor -- start 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.821+0000 7f7625631740 1 -- start start 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f7625631740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0870440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f7625631740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf087ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f7625631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf087d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f7625631740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bbf07d5620 con 0x55bbf0870980 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f7625631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbf077ce50 con 0x55bbf087b240 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f7625631740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbf07d15e0 con 0x55bbf0870060 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f761bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf087d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f761bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf087d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:59320/0 (socket says 172.21.15.133:59320) 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f761bfff640 1 -- 172.21.15.133:0/1170180798 learned_addr learned my addr 172.21.15.133:0/1170180798 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f7619ffb640 1 -- 172.21.15.133:0/3827825639 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7610010ac0 con 0x7f761c100e20 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f76217f5640 1 --2- 172.21.15.133:0/1170180798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0870440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f761bfff640 1 -- 172.21.15.133:0/1170180798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 msgr2=0x55bbf0870440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f761bfff640 1 --2- 172.21.15.133:0/1170180798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0870440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f761bfff640 1 -- 172.21.15.133:0/1170180798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 msgr2=0x55bbf087ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f7620ff4640 1 --2- 172.21.15.133:0/1170180798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf087ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f761bfff640 1 --2- 172.21.15.133:0/1170180798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf087ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f761bfff640 1 -- 172.21.15.133:0/1170180798 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbf076dd40 con 0x55bbf087b240 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.822+0000 7f7620ff4640 1 --2- 172.21.15.133:0/1170180798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf087ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.823+0000 7f761bfff640 1 --2- 172.21.15.133:0/1170180798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf087d630 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f760800bac0 tx=0x7f760800bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7ea3c248682f45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.823+0000 7f7606ffd640 1 -- 172.21.15.133:0/1170180798 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608017420 con 0x55bbf087b240 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.823+0000 7f7606ffd640 1 -- 172.21.15.133:0/1170180798 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7608017aa0 con 0x55bbf087b240 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.823+0000 7f7606ffd640 1 -- 172.21.15.133:0/1170180798 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760800fb60 con 0x55bbf087b240 2024-09-11T23:47:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.823+0000 7f7619ffb640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f760c0a2cf0 0x7f760c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.823+0000 7f76217f5640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f760c0a2cf0 0x7f760c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.823+0000 7f7619ffb640 1 -- 172.21.15.133:0/3827825639 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f76100cf890 con 0x7f761c100e20 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f7625631740 1 -- 172.21.15.133:0/1170180798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 msgr2=0x55bbf087d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f7625631740 1 --2- 172.21.15.133:0/1170180798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf087d630 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f760800bac0 tx=0x7f760800bf80 comp rx=0 tx=0).stop 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f7625631740 1 -- 172.21.15.133:0/1170180798 shutdown_connections 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f7625631740 1 --2- 172.21.15.133:0/1170180798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf087d630 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f7625631740 1 --2- 172.21.15.133:0/1170180798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf087ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f7625631740 1 --2- 172.21.15.133:0/1170180798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0870440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f7625631740 1 -- 172.21.15.133:0/1170180798 >> 172.21.15.133:0/1170180798 conn(0x55bbf086c5f0 msgr2=0x55bbf086c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f76217f5640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f760c0a2cf0 0x7f760c0a51b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7614009430 tx=0x7f7614009360 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f7625631740 1 -- 172.21.15.133:0/1170180798 shutdown_connections 2024-09-11T23:47:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f7625631740 1 -- 172.21.15.133:0/1170180798 wait complete. 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f7625631740 1 -- start start 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.824+0000 7f7625631740 1 Processor -- start 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7625631740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870060 0x55bbf0878060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7625631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf0870980 0x55bbf08785a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7625631740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf087b240 0x55bbf0878ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7625631740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bbf087de10 con 0x55bbf0870060 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7625631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbf075df10 con 0x55bbf0870980 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7625631740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbf087df90 con 0x55bbf087b240 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7625631740 10 client.0 _ll_register_callbacks cb 0x55bbf0781db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7620ff4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870060 0x55bbf0878060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7620ff4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870060 0x55bbf0878060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42794/0 (socket says 172.21.15.133:42794) 2024-09-11T23:47:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f761bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf0870980 0x55bbf08785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7620ff4640 1 -- 172.21.15.133:0/308434000 learned_addr learned my addr 172.21.15.133:0/308434000 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f76217f5640 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf087b240 0x55bbf0878ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7620ff4640 1 -- 172.21.15.133:0/308434000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf087b240 msgr2=0x55bbf0878ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7620ff4640 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf087b240 0x55bbf0878ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7620ff4640 1 -- 172.21.15.133:0/308434000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf0870980 msgr2=0x55bbf08785a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7620ff4640 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf0870980 0x55bbf08785a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.825+0000 7f7620ff4640 1 -- 172.21.15.133:0/308434000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbf04e4580 con 0x55bbf0870060 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.826+0000 7f761bfff640 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf0870980 0x55bbf08785a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.826+0000 7f7620ff4640 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870060 0x55bbf0878060 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f761000e9c0 tx=0x7f761009c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb045f3f2c27270 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.826+0000 7f75fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.826+0000 7f75fbfff640 1 -- 172.21.15.133:0/308434000 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610018d30 con 0x55bbf0870060 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.826+0000 7f75fbfff640 1 -- 172.21.15.133:0/308434000 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f761009b210 con 0x55bbf0870060 2024-09-11T23:47:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.826+0000 7f7625631740 1 -- 172.21.15.133:0/308434000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7608017aa0 con 0x55bbf0870060 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.826+0000 7f75fbfff640 1 -- 172.21.15.133:0/308434000 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f76100a4a30 con 0x55bbf0870060 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.826+0000 7f7625631740 10 client.18117 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.826+0000 7f7625631740 1 -- 172.21.15.133:0/308434000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bbf05ae960 con 0x55bbf0870060 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.826+0000 7f75fbfff640 1 -- 172.21.15.133:0/308434000 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7608017aa0 con 0x55bbf0870060 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.826+0000 7f75fbfff640 1 -- 172.21.15.133:0/308434000 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f761009ac80 con 0x55bbf0870060 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.827+0000 7f7625631740 10 client.18117 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.827+0000 7f7625631740 10 client.18117 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.827+0000 7f7625631740 20 client.18117 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.827+0000 7f7625631740 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bbf088ac50 0x55bbf088d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.827+0000 7f7625631740 4 client.18117 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.827+0000 7f7625631740 1 -- 172.21.15.133:0/308434000 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbf07e2b90 con 0x55bbf088ac50 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.827+0000 7f761bfff640 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bbf088ac50 0x55bbf088d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.827+0000 7f75fbfff640 10 client.18117 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.827+0000 7f761bfff640 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bbf088ac50 0x55bbf088d0c0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=605a86932d09ca92 server_cookie=ccea5921a75d4add in_seq=0 out_seq=0 2024-09-11T23:47:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f75fbfff640 1 -- 172.21.15.133:0/308434000 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbf07e2b90 con 0x55bbf088ac50 2024-09-11T23:47:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f75fbfff640 10 client.18117 handle_command_reply: tid=0 2024-09-11T23:47:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 1 client.18117 shutdown 2024-09-11T23:47:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 1 -- 172.21.15.133:0/308434000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870060 msgr2=0x55bbf0878060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870060 0x55bbf0878060 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f761000e9c0 tx=0x7f761009c580 comp rx=0 tx=0).stop 2024-09-11T23:47:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 1 -- 172.21.15.133:0/308434000 shutdown_connections 2024-09-11T23:47:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bbf088ac50 0x55bbf088d0c0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf087b240 0x55bbf0878ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf0870980 0x55bbf08785a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 1 --2- 172.21.15.133:0/308434000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870060 0x55bbf0878060 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 1 -- 172.21.15.133:0/308434000 >> 172.21.15.133:0/308434000 conn(0x55bbf086c5f0 msgr2=0x55bbf0872320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f75fbfff640 0 client.18117 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 1 -- 172.21.15.133:0/308434000 shutdown_connections 2024-09-11T23:47:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 1 -- 172.21.15.133:0/308434000 wait complete. 2024-09-11T23:47:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.828+0000 7f7625631740 20 client.18117 trim_cache size 0 max 16384 2024-09-11T23:47:11.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f7625631740 1 Processor -- start 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f7625631740 1 -- start start 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f7625631740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0879fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f7625631740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0892980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f7625631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf0892ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f7625631740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1afa20 con 0x55bbf0870980 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f7625631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1b1560 con 0x55bbf087b240 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f7625631740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1b00f0 con 0x55bbf0870060 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f761bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0892980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f761bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0892980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42808/0 (socket says 172.21.15.133:42808) 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f761bfff640 1 -- 172.21.15.133:0/360511541 learned_addr learned my addr 172.21.15.133:0/360511541 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.846+0000 7f76217f5640 1 --2- 172.21.15.133:0/360511541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf0892ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f7620ff4640 1 --2- 172.21.15.133:0/360511541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0879fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f761bfff640 1 -- 172.21.15.133:0/360511541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 msgr2=0x55bbf0879fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f761bfff640 1 --2- 172.21.15.133:0/360511541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0879fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f761bfff640 1 -- 172.21.15.133:0/360511541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 msgr2=0x55bbf0892ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f761bfff640 1 --2- 172.21.15.133:0/360511541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf0892ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f761bfff640 1 -- 172.21.15.133:0/360511541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbf086abb0 con 0x55bbf0870980 2024-09-11T23:47:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f7620ff4640 1 --2- 172.21.15.133:0/360511541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0879fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:47:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f76217f5640 1 --2- 172.21.15.133:0/360511541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf0892ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f761bfff640 1 --2- 172.21.15.133:0/360511541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0892980 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7608005940 tx=0x7f760800b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ee6a28833f285e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f76067fc640 1 -- 172.21.15.133:0/360511541 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608017420 con 0x55bbf0870980 2024-09-11T23:47:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f76067fc640 1 -- 172.21.15.133:0/360511541 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f760802e070 con 0x55bbf0870980 2024-09-11T23:47:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.847+0000 7f76067fc640 1 -- 172.21.15.133:0/360511541 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608029340 con 0x55bbf0870980 2024-09-11T23:47:11.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.848+0000 7f7625631740 1 -- 172.21.15.133:0/360511541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 msgr2=0x55bbf0892980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:11.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.848+0000 7f7625631740 1 --2- 172.21.15.133:0/360511541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0892980 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7608005940 tx=0x7f760800b3f0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.848+0000 7f7625631740 1 -- 172.21.15.133:0/360511541 shutdown_connections 2024-09-11T23:47:11.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.848+0000 7f7625631740 1 --2- 172.21.15.133:0/360511541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf0892ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.848+0000 7f7625631740 1 --2- 172.21.15.133:0/360511541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0892980 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.848+0000 7f7625631740 1 --2- 172.21.15.133:0/360511541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0879fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.848+0000 7f7625631740 1 -- 172.21.15.133:0/360511541 >> 172.21.15.133:0/360511541 conn(0x55bbf086c5f0 msgr2=0x55bbf08798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.848+0000 7f7625631740 1 -- 172.21.15.133:0/360511541 shutdown_connections 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.848+0000 7f7625631740 1 -- 172.21.15.133:0/360511541 wait complete. 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.848+0000 7f7625631740 1 -- start start 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.848+0000 7f7625631740 1 Processor -- start 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f7625631740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0883800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f7625631740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0874cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f7625631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf0875210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f7625631740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1b0e90 con 0x55bbf0870980 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f7625631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1b07c0 con 0x55bbf087b240 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f7625631740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1ada20 con 0x55bbf0870060 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f761bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0874cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f7625631740 10 client.0 _ll_register_callbacks cb 0x7f761009b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f761bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0874cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42822/0 (socket says 172.21.15.133:42822) 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f761bfff640 1 -- 172.21.15.133:0/3625466732 learned_addr learned my addr 172.21.15.133:0/3625466732 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f7620ff4640 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0883800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f76217f5640 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf0875210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f761bfff640 1 -- 172.21.15.133:0/3625466732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 msgr2=0x55bbf0883800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f761bfff640 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0883800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f761bfff640 1 -- 172.21.15.133:0/3625466732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 msgr2=0x55bbf0875210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f761bfff640 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf0875210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f761bfff640 1 -- 172.21.15.133:0/3625466732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbf0875950 con 0x55bbf0870980 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f76217f5640 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf0875210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f761bfff640 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0874cd0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f760802b040 tx=0x7f760800ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230fec24c8719750 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.849+0000 7f75fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f75fbfff640 1 -- 172.21.15.133:0/3625466732 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608039070 con 0x55bbf0870980 2024-09-11T23:47:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f75fbfff640 1 -- 172.21.15.133:0/3625466732 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f760800fde0 con 0x55bbf0870980 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f7625631740 1 -- 172.21.15.133:0/3625466732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f760802e070 con 0x55bbf0870980 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f75fbfff640 1 -- 172.21.15.133:0/3625466732 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7608045050 con 0x55bbf0870980 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f7625631740 10 client.18123 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f7625631740 1 -- 172.21.15.133:0/3625466732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bbf08f5950 con 0x55bbf0870980 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f75fbfff640 1 -- 172.21.15.133:0/3625466732 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f760802e070 con 0x55bbf0870980 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f75fbfff640 1 -- 172.21.15.133:0/3625466732 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7608029380 con 0x55bbf0870980 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f7625631740 10 client.18123 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f7625631740 10 client.18123 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f7625631740 20 client.18123 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f7625631740 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bbf08f5dd0 0x55bbf08f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f7625631740 4 client.18123 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d0886d4a", "release": true, "await": true}] 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f7625631740 1 -- 172.21.15.133:0/3625466732 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d0886d4a", "release": true, "await": true}) -- 0x55bbf07dd0a0 con 0x55bbf08f5dd0 2024-09-11T23:47:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.850+0000 7f7620ff4640 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bbf08f5dd0 0x55bbf08f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.851+0000 7f7620ff4640 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bbf08f5dd0 0x55bbf08f81f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa569fcb90d20f3d server_cookie=2be6be41440ef21e in_seq=0 out_seq=0 2024-09-11T23:47:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.851+0000 7f75fbfff640 10 client.18123 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.854+0000 7f75fbfff640 1 -- 172.21.15.133:0/3625466732 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bbf07dd0a0 con 0x55bbf08f5dd0 2024-09-11T23:47:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.854+0000 7f75fbfff640 10 client.18123 handle_command_reply: tid=0 2024-09-11T23:47:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.854+0000 7f7625631740 1 client.18123 shutdown 2024-09-11T23:47:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.854+0000 7f7625631740 1 -- 172.21.15.133:0/3625466732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 msgr2=0x55bbf0874cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.854+0000 7f7625631740 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0874cd0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f760802b040 tx=0x7f760800ba60 comp rx=0 tx=0).stop 2024-09-11T23:47:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.854+0000 7f7625631740 1 -- 172.21.15.133:0/3625466732 shutdown_connections 2024-09-11T23:47:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.854+0000 7f7625631740 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bbf08f5dd0 0x55bbf08f81f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.855+0000 7f7625631740 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf087b240 0x55bbf0875210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.855+0000 7f7625631740 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbf0870980 0x55bbf0874cd0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.855+0000 7f7625631740 1 --2- 172.21.15.133:0/3625466732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbf0870060 0x55bbf0883800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.855+0000 7f7625631740 1 -- 172.21.15.133:0/3625466732 >> 172.21.15.133:0/3625466732 conn(0x55bbf086c5f0 msgr2=0x55bbf086b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.855+0000 7f75fbfff640 0 client.18123 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.855+0000 7f7625631740 1 -- 172.21.15.133:0/3625466732 shutdown_connections 2024-09-11T23:47:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.855+0000 7f7625631740 1 -- 172.21.15.133:0/3625466732 wait complete. 2024-09-11T23:47:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.855+0000 7f7625631740 20 client.18123 trim_cache size 0 max 16384 2024-09-11T23:47:11.857 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:47:11.857 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:47:11.857 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:47:11.857 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 40, 2024-09-11T23:47:11.857 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "d0886d4a": { 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 40, 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 142.6, 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:47:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:47:11.859 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:47:11.859 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:11.859 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:11.859 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:11.859 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:11.859 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:11.859 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:47:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.856+0000 7f7623258640 1 -- 172.21.15.133:0/3827825639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f760c0a2cf0 msgr2=0x7f760c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.856+0000 7f7623258640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f760c0a2cf0 0x7f760c0a51b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7614009430 tx=0x7f7614009360 comp rx=0 tx=0).stop 2024-09-11T23:47:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.856+0000 7f7623258640 1 -- 172.21.15.133:0/3827825639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f761c100e20 msgr2=0x7f761c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.856+0000 7f7623258640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f761c100e20 0x7f761c0fcc80 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f761000e9f0 tx=0x7f761000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.857+0000 7f7623258640 1 -- 172.21.15.133:0/3827825639 shutdown_connections 2024-09-11T23:47:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.857+0000 7f7623258640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f760c0a2cf0 0x7f760c0a51b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.857+0000 7f7623258640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f761c102ef0 0x7f761c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.857+0000 7f7623258640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761c102570 0x7f761c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.857+0000 7f7623258640 1 --2- 172.21.15.133:0/3827825639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f761c100e20 0x7f761c0fcc80 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:11.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.857+0000 7f7623258640 1 -- 172.21.15.133:0/3827825639 >> 172.21.15.133:0/3827825639 conn(0x7f761c0fc7d0 msgr2=0x7f761c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:11.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.857+0000 7f7623258640 1 -- 172.21.15.133:0/3827825639 shutdown_connections 2024-09-11T23:47:11.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:11.857+0000 7f7623258640 1 -- 172.21.15.133:0/3827825639 wait complete. 2024-09-11T23:47:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:11 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:11 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:11 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:11 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:11 smithi138 ceph-mon[35142]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-11T23:47:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:11 smithi138 ceph-mon[35142]: Unable to set osd_memory_target on smithi133 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-11T23:47:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:11 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:11 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:11.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:11 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:11 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:12.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:11 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:12.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:11 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:12.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:11 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:47:12.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:11 smithi178 ceph-mon[35882]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-11T23:47:12.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:11 smithi178 ceph-mon[35882]: Unable to set osd_memory_target on smithi133 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-11T23:47:12.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:11 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:47:12.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:11 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:47:12.041 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:11 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:47:13.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:12 smithi178 ceph-mon[35882]: pgmap v325: 97 pgs: 97 active+clean; 60 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 32 KiB/s rd, 59 MiB/s wr, 235 op/s 2024-09-11T23:47:13.042 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:12 smithi133 ceph-mon[31430]: pgmap v325: 97 pgs: 97 active+clean; 60 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 32 KiB/s rd, 59 MiB/s wr, 235 op/s 2024-09-11T23:47:13.044 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd0886d4a', total seconds elapsed: 26.4 2024-09-11T23:47:13.045 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-11T23:47:13.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:47:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:12 smithi138 ceph-mon[35142]: pgmap v325: 97 pgs: 97 active+clean; 60 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 32 KiB/s rd, 59 MiB/s wr, 235 op/s 2024-09-11T23:47:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.511+0000 7fba1359e640 1 -- 172.21.15.133:0/2850785436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba141018b0 msgr2=0x7fba14101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.511+0000 7fba1359e640 1 --2- 172.21.15.133:0/2850785436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba141018b0 0x7fba14101d10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fba04009930 tx=0x7fba0402a420 comp rx=0 tx=0).stop 2024-09-11T23:47:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.512+0000 7fba1359e640 1 -- 172.21.15.133:0/2850785436 shutdown_connections 2024-09-11T23:47:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.512+0000 7fba1359e640 1 --2- 172.21.15.133:0/2850785436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba14102250 0x7fba1410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.512+0000 7fba1359e640 1 --2- 172.21.15.133:0/2850785436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba141018b0 0x7fba14101d10 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.512+0000 7fba1359e640 1 --2- 172.21.15.133:0/2850785436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba141078c0 0x7fba14107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.512+0000 7fba1359e640 1 -- 172.21.15.133:0/2850785436 >> 172.21.15.133:0/2850785436 conn(0x7fba140fd5d0 msgr2=0x7fba140ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.512+0000 7fba1359e640 1 -- 172.21.15.133:0/2850785436 shutdown_connections 2024-09-11T23:47:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.512+0000 7fba1359e640 1 -- 172.21.15.133:0/2850785436 wait complete. 2024-09-11T23:47:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.514+0000 7fba1359e640 1 Processor -- start 2024-09-11T23:47:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.514+0000 7fba1359e640 1 -- start start 2024-09-11T23:47:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.514+0000 7fba1359e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba141018b0 0x7fba14198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.514+0000 7fba1359e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba14102250 0x7fba141987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.514+0000 7fba1359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba141078c0 0x7fba1419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.514+0000 7fba1359e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fba14111d50 con 0x7fba141078c0 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.515+0000 7fba1359e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba14111bd0 con 0x7fba141018b0 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.515+0000 7fba1359e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fba14111ed0 con 0x7fba14102250 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.515+0000 7fba12d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba141078c0 0x7fba1419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.515+0000 7fba12d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba141078c0 0x7fba1419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56778/0 (socket says 172.21.15.133:56778) 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.515+0000 7fba12d9d640 1 -- 172.21.15.133:0/1692223829 learned_addr learned my addr 172.21.15.133:0/1692223829 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.515+0000 7fba12d9d640 1 -- 172.21.15.133:0/1692223829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba14102250 msgr2=0x7fba141987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.515+0000 7fba12d9d640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba14102250 0x7fba141987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.515+0000 7fba12d9d640 1 -- 172.21.15.133:0/1692223829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba141018b0 msgr2=0x7fba14198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.515+0000 7fba12d9d640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba141018b0 0x7fba14198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.515+0000 7fba12d9d640 1 -- 172.21.15.133:0/1692223829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1419d270 con 0x7fba141078c0 2024-09-11T23:47:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.516+0000 7fba12d9d640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba141078c0 0x7fba1419cb40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00b970 tx=0x7fba0c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.516+0000 7fba0b7fe640 1 -- 172.21.15.133:0/1692223829 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c019070 con 0x7fba141078c0 2024-09-11T23:47:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.516+0000 7fba0b7fe640 1 -- 172.21.15.133:0/1692223829 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba0c00cb40 con 0x7fba141078c0 2024-09-11T23:47:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.516+0000 7fba0b7fe640 1 -- 172.21.15.133:0/1692223829 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c007650 con 0x7fba141078c0 2024-09-11T23:47:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.516+0000 7fba1359e640 1 -- 172.21.15.133:0/1692223829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1419d560 con 0x7fba141078c0 2024-09-11T23:47:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.516+0000 7fba1359e640 1 -- 172.21.15.133:0/1692223829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba141a4df0 con 0x7fba141078c0 2024-09-11T23:47:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.521+0000 7fba1bc2c740 1 Processor -- start 2024-09-11T23:47:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.523+0000 7fba0b7fe640 1 -- 172.21.15.133:0/1692223829 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fba0c01c070 con 0x7fba141078c0 2024-09-11T23:47:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.524+0000 7fba1bc2c740 1 -- start start 2024-09-11T23:47:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.525+0000 7fba0b7fe640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba000a2d70 0x7fba000a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.525+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 0x559fc287a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.525+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc287a5e0 0x559fc2884940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.525+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc2884e80 0x559fc2887270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.525+0000 7fba1bc2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559fc27de2f0 con 0x559fc2884e80 2024-09-11T23:47:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.526+0000 7fba1bc2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559fc2786270 con 0x559fc287a5e0 2024-09-11T23:47:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.526+0000 7fba1bc2c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559fc27da2b0 con 0x559fc2879cc0 2024-09-11T23:47:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.526+0000 7fba0b7fe640 1 -- 172.21.15.133:0/1692223829 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fba0c0ce930 con 0x7fba141078c0 2024-09-11T23:47:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.526+0000 7fba0bfff640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba000a2d70 0x7fba000a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.528+0000 7fba1259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 0x559fc287a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.528+0000 7fba1259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 0x559fc287a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48552/0 (socket says 172.21.15.133:48552) 2024-09-11T23:47:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.528+0000 7fba1259c640 1 -- 172.21.15.133:0/3798062311 learned_addr learned my addr 172.21.15.133:0/3798062311 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.529+0000 7fba0bfff640 1 --2- 172.21.15.133:0/3798062311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc287a5e0 0x559fc2884940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.529+0000 7fba1259c640 1 -- 172.21.15.133:0/3798062311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc287a5e0 msgr2=0x559fc2884940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.529+0000 7fba1259c640 1 --2- 172.21.15.133:0/3798062311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc287a5e0 0x559fc2884940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.529+0000 7fba1259c640 1 -- 172.21.15.133:0/3798062311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc2884e80 msgr2=0x559fc2887270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.529+0000 7fba1259c640 1 --2- 172.21.15.133:0/3798062311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc2884e80 0x559fc2887270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.529+0000 7fba1259c640 1 -- 172.21.15.133:0/3798062311 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc2785340 con 0x559fc2879cc0 2024-09-11T23:47:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.530+0000 7fba1259c640 1 --2- 172.21.15.133:0/3798062311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 0x559fc287a0a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00bb10 tx=0x7fb9fc00d2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b3e7ea2fadeb3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.531+0000 7fba08ff9640 1 -- 172.21.15.133:0/3798062311 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc016070 con 0x559fc2879cc0 2024-09-11T23:47:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.532+0000 7fba08ff9640 1 -- 172.21.15.133:0/3798062311 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9fc005500 con 0x559fc2879cc0 2024-09-11T23:47:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.532+0000 7fba08ff9640 1 -- 172.21.15.133:0/3798062311 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc010460 con 0x559fc2879cc0 2024-09-11T23:47:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.536+0000 7fba1bc2c740 1 -- 172.21.15.133:0/3798062311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 msgr2=0x559fc287a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.536+0000 7fba0bfff640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba000a2d70 0x7fba000a5230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fba040096c0 tx=0x7fba04004260 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.537+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/3798062311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 0x559fc287a0a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00bb10 tx=0x7fb9fc00d2e0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.541+0000 7fba1bc2c740 1 -- 172.21.15.133:0/3798062311 shutdown_connections 2024-09-11T23:47:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.541+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/3798062311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc2884e80 0x559fc2887270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.541+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/3798062311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc287a5e0 0x559fc2884940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.541+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/3798062311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 0x559fc287a0a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.541+0000 7fba1bc2c740 1 -- 172.21.15.133:0/3798062311 >> 172.21.15.133:0/3798062311 conn(0x559fc2876230 msgr2=0x559fc2876610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.541+0000 7fba1bc2c740 1 -- 172.21.15.133:0/3798062311 shutdown_connections 2024-09-11T23:47:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.541+0000 7fba1bc2c740 1 -- 172.21.15.133:0/3798062311 wait complete. 2024-09-11T23:47:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.542+0000 7fba1bc2c740 1 -- start start 2024-09-11T23:47:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.542+0000 7fba1bc2c740 1 Processor -- start 2024-09-11T23:47:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.543+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2879cc0 0x559fc2881e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.543+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc287a5e0 0x559fc2882370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.543+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2884e80 0x559fc28828b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.543+0000 7fba1bc2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559fc2887a50 con 0x559fc287a5e0 2024-09-11T23:47:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.543+0000 7fba1bc2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559fc2767290 con 0x559fc2879cc0 2024-09-11T23:47:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.543+0000 7fba1bc2c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559fc2887bd0 con 0x559fc2884e80 2024-09-11T23:47:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.543+0000 7fba1bc2c740 10 client.0 _ll_register_callbacks cb 0x559fc2798140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.544+0000 7fba1259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2879cc0 0x559fc2881e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.544+0000 7fba1259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2879cc0 0x559fc2881e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:36582/0 (socket says 172.21.15.133:36582) 2024-09-11T23:47:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.544+0000 7fba1259c640 1 -- 172.21.15.133:0/1338047060 learned_addr learned my addr 172.21.15.133:0/1338047060 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.545+0000 7fba12d9d640 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2884e80 0x559fc28828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.545+0000 7fba1259c640 1 -- 172.21.15.133:0/1338047060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2884e80 msgr2=0x559fc28828b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.545+0000 7fba1259c640 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2884e80 0x559fc28828b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.545+0000 7fba1259c640 1 -- 172.21.15.133:0/1338047060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc287a5e0 msgr2=0x559fc2882370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.545+0000 7fba1259c640 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc287a5e0 0x559fc2882370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.545+0000 7fba1259c640 1 -- 172.21.15.133:0/1338047060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc24f1210 con 0x559fc2879cc0 2024-09-11T23:47:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.546+0000 7fba1259c640 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2879cc0 0x559fc2881e30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc004a00 tx=0x7fb9fc005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31fa94aaac82ef59 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.549+0000 7fb9f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:47:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.549+0000 7fb9f9ffb640 1 -- 172.21.15.133:0/1338047060 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc016070 con 0x559fc2879cc0 2024-09-11T23:47:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.549+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1338047060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9fc005500 con 0x559fc2879cc0 2024-09-11T23:47:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.550+0000 7fb9f9ffb640 1 -- 172.21.15.133:0/1338047060 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fb9fc005220 con 0x559fc2879cc0 2024-09-11T23:47:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.550+0000 7fb9f9ffb640 1 -- 172.21.15.133:0/1338047060 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9fc00dae0 con 0x559fc2879cc0 2024-09-11T23:47:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.550+0000 7fba1bc2c740 10 client.27674 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.550+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1338047060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559fc25bd620 con 0x559fc2879cc0 2024-09-11T23:47:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.551+0000 7fb9f9ffb640 1 -- 172.21.15.133:0/1338047060 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9fc005500 con 0x559fc2879cc0 2024-09-11T23:47:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.552+0000 7fb9f9ffb640 1 -- 172.21.15.133:0/1338047060 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb9fc01b430 con 0x559fc2879cc0 2024-09-11T23:47:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.552+0000 7fba1bc2c740 10 client.27674 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.552+0000 7fba1bc2c740 10 client.27674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.552+0000 7fba1bc2c740 20 client.27674 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.552+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559fc28ae5f0 0x559fc28b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.552+0000 7fba1bc2c740 4 client.27674 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.552+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1338047060 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fc27eba20 con 0x559fc28ae5f0 2024-09-11T23:47:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.556+0000 7fba0bfff640 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559fc28ae5f0 0x559fc28b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.559+0000 7fba0bfff640 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559fc28ae5f0 0x559fc28b09e0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da844318d8b67a8 server_cookie=20c5abcf200f3bc0 in_seq=0 out_seq=0 2024-09-11T23:47:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.563+0000 7fb9f9ffb640 10 client.27674 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.563+0000 7fb9f9ffb640 1 -- 172.21.15.133:0/1338047060 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fc27eba20 con 0x559fc28ae5f0 2024-09-11T23:47:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.563+0000 7fb9f9ffb640 10 client.27674 handle_command_reply: tid=0 2024-09-11T23:47:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.568+0000 7fba1bc2c740 1 client.27674 shutdown 2024-09-11T23:47:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1338047060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2879cc0 msgr2=0x559fc2881e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2879cc0 0x559fc2881e30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc004a00 tx=0x7fb9fc005c90 comp rx=0 tx=0).stop 2024-09-11T23:47:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1338047060 shutdown_connections 2024-09-11T23:47:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559fc28ae5f0 0x559fc28b09e0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2884e80 0x559fc28828b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc287a5e0 0x559fc2882370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1338047060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2879cc0 0x559fc2881e30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fb9f9ffb640 0 client.27674 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1338047060 >> 172.21.15.133:0/1338047060 conn(0x559fc2876230 msgr2=0x559fc287bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1338047060 shutdown_connections 2024-09-11T23:47:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1338047060 wait complete. 2024-09-11T23:47:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.569+0000 7fba1bc2c740 20 client.27674 trim_cache size 0 max 16384 2024-09-11T23:47:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.611+0000 7fba1bc2c740 1 Processor -- start 2024-09-11T23:47:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.612+0000 7fba1bc2c740 1 -- start start 2024-09-11T23:47:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.613+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 0x559fc2883da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.613+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc287a5e0 0x559fc2882380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.613+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2884e80 0x559fc28b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.614+0000 7fba1bc2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fba140744c0 con 0x559fc287a5e0 2024-09-11T23:47:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.614+0000 7fba1bc2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba14076000 con 0x559fc2884e80 2024-09-11T23:47:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.614+0000 7fba1bc2c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fba14074b90 con 0x559fc2879cc0 2024-09-11T23:47:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.617+0000 7fba1259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 0x559fc2883da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.617+0000 7fba12d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2884e80 0x559fc28b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.617+0000 7fba12d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2884e80 0x559fc28b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:36588/0 (socket says 172.21.15.133:36588) 2024-09-11T23:47:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.617+0000 7fba12d9d640 1 -- 172.21.15.133:0/3471624192 learned_addr learned my addr 172.21.15.133:0/3471624192 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.617+0000 7fba0bfff640 1 --2- 172.21.15.133:0/3471624192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc287a5e0 0x559fc2882380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.623+0000 7fba12d9d640 1 -- 172.21.15.133:0/3471624192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 msgr2=0x559fc2883da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.623+0000 7fba12d9d640 1 --2- 172.21.15.133:0/3471624192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 0x559fc2883da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.623+0000 7fba12d9d640 1 -- 172.21.15.133:0/3471624192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc287a5e0 msgr2=0x559fc2882380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.623+0000 7fba12d9d640 1 --2- 172.21.15.133:0/3471624192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc287a5e0 0x559fc2882380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.623+0000 7fba12d9d640 1 -- 172.21.15.133:0/3471624192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc2880380 con 0x559fc2884e80 2024-09-11T23:47:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.627+0000 7fba12d9d640 1 --2- 172.21.15.133:0/3471624192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2884e80 0x559fc28b6100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0cc200 tx=0x7fba0c0bb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92985cfad7a1b378 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.631+0000 7fb9fbfff640 1 -- 172.21.15.133:0/3471624192 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c0b5070 con 0x559fc2884e80 2024-09-11T23:47:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.631+0000 7fb9fbfff640 1 -- 172.21.15.133:0/3471624192 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba0c09f320 con 0x559fc2884e80 2024-09-11T23:47:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.631+0000 7fb9fbfff640 1 -- 172.21.15.133:0/3471624192 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c0a3750 con 0x559fc2884e80 2024-09-11T23:47:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.637+0000 7fba1bc2c740 1 -- 172.21.15.133:0/3471624192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2884e80 msgr2=0x559fc28b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.637+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/3471624192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2884e80 0x559fc28b6100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0cc200 tx=0x7fba0c0bb040 comp rx=0 tx=0).stop 2024-09-11T23:47:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.639+0000 7fba1bc2c740 1 -- 172.21.15.133:0/3471624192 shutdown_connections 2024-09-11T23:47:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.639+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/3471624192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2884e80 0x559fc28b6100 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.639+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/3471624192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc287a5e0 0x559fc2882380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.639+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/3471624192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc2879cc0 0x559fc2883da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.639+0000 7fba1bc2c740 1 -- 172.21.15.133:0/3471624192 >> 172.21.15.133:0/3471624192 conn(0x559fc2876230 msgr2=0x559fc2883690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.642+0000 7fba1bc2c740 1 -- 172.21.15.133:0/3471624192 shutdown_connections 2024-09-11T23:47:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.642+0000 7fba1bc2c740 1 -- 172.21.15.133:0/3471624192 wait complete. 2024-09-11T23:47:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.643+0000 7fba1bc2c740 1 -- start start 2024-09-11T23:47:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.643+0000 7fba1bc2c740 1 Processor -- start 2024-09-11T23:47:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.644+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc2879cc0 0x559fc287eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.644+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc287a5e0 0x559fc287f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.644+0000 7fba1bc2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2884e80 0x559fc287f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.645+0000 7fba0bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc287a5e0 0x559fc287f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.645+0000 7fba0bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc287a5e0 0x559fc287f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48574/0 (socket says 172.21.15.133:48574) 2024-09-11T23:47:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.645+0000 7fba0bfff640 1 -- 172.21.15.133:0/1867056488 learned_addr learned my addr 172.21.15.133:0/1867056488 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.645+0000 7fba1bc2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fba14075930 con 0x559fc2879cc0 2024-09-11T23:47:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.645+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1867056488 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba14075260 con 0x559fc2884e80 2024-09-11T23:47:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.645+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1867056488 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fba140724c0 con 0x559fc287a5e0 2024-09-11T23:47:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.645+0000 7fba1bc2c740 10 client.0 _ll_register_callbacks cb 0x7fb9fc00b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.645+0000 7fba0bfff640 1 -- 172.21.15.133:0/1867056488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2884e80 msgr2=0x559fc287f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.645+0000 7fba0bfff640 1 --2- 172.21.15.133:0/1867056488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2884e80 0x559fc287f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.646+0000 7fba0bfff640 1 -- 172.21.15.133:0/1867056488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc2879cc0 msgr2=0x559fc287eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.646+0000 7fba0bfff640 1 --2- 172.21.15.133:0/1867056488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc2879cc0 0x559fc287eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.646+0000 7fba0bfff640 1 -- 172.21.15.133:0/1867056488 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc2918180 con 0x559fc287a5e0 2024-09-11T23:47:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.646+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1867056488 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba0c09f320 con 0x559fc287a5e0 2024-09-11T23:47:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.652+0000 7fba0bfff640 1 --2- 172.21.15.133:0/1867056488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc287a5e0 0x559fc287f3f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fba04000c00 tx=0x7fba04009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e418c46b9baf85ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.652+0000 7fb9f9ffb640 10 client.27439 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:47:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.654+0000 7fb9f9ffb640 1 -- 172.21.15.133:0/1867056488 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04035040 con 0x559fc287a5e0 2024-09-11T23:47:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.654+0000 7fb9f9ffb640 1 -- 172.21.15.133:0/1867056488 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fba04003cc0 con 0x559fc287a5e0 2024-09-11T23:47:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.654+0000 7fb9f9ffb640 1 -- 172.21.15.133:0/1867056488 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fba0405e450 con 0x559fc287a5e0 2024-09-11T23:47:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.655+0000 7fba1bc2c740 10 client.27439 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.655+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1867056488 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=30}) -- 0x559fc29193a0 con 0x559fc287a5e0 2024-09-11T23:47:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.657+0000 7fb9f9ffb640 1 -- 172.21.15.133:0/1867056488 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fba04053070 con 0x559fc287a5e0 2024-09-11T23:47:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.657+0000 7fba1bc2c740 10 client.27439 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.657+0000 7fba1bc2c740 10 client.27439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.657+0000 7fba1bc2c740 20 client.27439 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.657+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1867056488 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559fc2919a20 0x559fc291be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.658+0000 7fba1bc2c740 4 client.27439 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:47:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.658+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1867056488 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fc27e5d70 con 0x559fc2919a20 2024-09-11T23:47:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.662+0000 7fba1259c640 1 --2- 172.21.15.133:0/1867056488 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559fc2919a20 0x559fc291be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.669+0000 7fba1259c640 1 --2- 172.21.15.133:0/1867056488 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559fc2919a20 0x559fc291be80 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a7669aabe6babc server_cookie=171ab0895610d17 in_seq=0 out_seq=0 2024-09-11T23:47:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:13.669+0000 7fb9f9ffb640 10 client.27439 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:13.876 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.628+0000 7fb9f9ffb640 1 -- 172.21.15.133:0/1867056488 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559fc27e5d70 con 0x559fc2919a20 2024-09-11T23:47:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.628+0000 7fb9f9ffb640 10 client.27439 handle_command_reply: tid=0 2024-09-11T23:47:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.628+0000 7fba1bc2c740 1 client.27439 shutdown 2024-09-11T23:47:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1867056488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc287a5e0 msgr2=0x559fc287f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1867056488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc287a5e0 0x559fc287f3f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fba04000c00 tx=0x7fba04009260 comp rx=0 tx=0).stop 2024-09-11T23:47:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1867056488 shutdown_connections 2024-09-11T23:47:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1867056488 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559fc2919a20 0x559fc291be80 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1867056488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fc2884e80 0x559fc287f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1867056488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559fc287a5e0 0x559fc287f3f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fba1bc2c740 1 --2- 172.21.15.133:0/1867056488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fc2879cc0 0x559fc287eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1867056488 >> 172.21.15.133:0/1867056488 conn(0x559fc2876230 msgr2=0x559fc2898d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fb9f9ffb640 0 client.27439 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1867056488 shutdown_connections 2024-09-11T23:47:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fba1bc2c740 1 -- 172.21.15.133:0/1867056488 wait complete. 2024-09-11T23:47:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.629+0000 7fba1bc2c740 20 client.27439 trim_cache size 0 max 16384 2024-09-11T23:47:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:47:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:47:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:47:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 -- 172.21.15.133:0/1692223829 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba000a2d70 msgr2=0x7fba000a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba000a2d70 0x7fba000a5230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fba040096c0 tx=0x7fba04004260 comp rx=0 tx=0).stop 2024-09-11T23:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 -- 172.21.15.133:0/1692223829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba141078c0 msgr2=0x7fba1419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba141078c0 0x7fba1419cb40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00b970 tx=0x7fba0c00be30 comp rx=0 tx=0).stop 2024-09-11T23:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 -- 172.21.15.133:0/1692223829 shutdown_connections 2024-09-11T23:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fba000a2d70 0x7fba000a5230 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba141078c0 0x7fba1419cb40 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba14102250 0x7fba141987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 --2- 172.21.15.133:0/1692223829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba141018b0 0x7fba14198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 -- 172.21.15.133:0/1692223829 >> 172.21.15.133:0/1692223829 conn(0x7fba140fd5d0 msgr2=0x7fba140fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 -- 172.21.15.133:0/1692223829 shutdown_connections 2024-09-11T23:47:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:14.634+0000 7fba1359e640 1 -- 172.21.15.133:0/1692223829 wait complete. 2024-09-11T23:47:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.656+0000 7fb5c9073640 1 -- 172.21.15.133:0/3984911514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c406ea30 msgr2=0x7fb5c406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.656+0000 7fb5c9073640 1 --2- 172.21.15.133:0/3984911514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c406ea30 0x7fb5c406ee70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009900 tx=0x7fb5b802a440 comp rx=0 tx=0).stop 2024-09-11T23:47:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.658+0000 7fb5c9073640 1 -- 172.21.15.133:0/3984911514 shutdown_connections 2024-09-11T23:47:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.658+0000 7fb5c9073640 1 --2- 172.21.15.133:0/3984911514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5c411b140 0x7fb5c411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.658+0000 7fb5c9073640 1 --2- 172.21.15.133:0/3984911514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c406ea30 0x7fb5c406ee70 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.658+0000 7fb5c9073640 1 --2- 172.21.15.133:0/3984911514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c4110190 0x7fb5c406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.658+0000 7fb5c9073640 1 -- 172.21.15.133:0/3984911514 >> 172.21.15.133:0/3984911514 conn(0x7fb5c4073180 msgr2=0x7fb5c4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:14.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.659+0000 7fb5c9073640 1 -- 172.21.15.133:0/3984911514 shutdown_connections 2024-09-11T23:47:14.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.660+0000 7fb5c9073640 1 -- 172.21.15.133:0/3984911514 wait complete. 2024-09-11T23:47:14.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.661+0000 7fb5c9073640 1 Processor -- start 2024-09-11T23:47:14.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.661+0000 7fb5c9073640 1 -- start start 2024-09-11T23:47:14.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.661+0000 7fb5c9073640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c406ea30 0x7fb5c41a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:14.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.661+0000 7fb5c9073640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c4110190 0x7fb5c41a7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:14.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.661+0000 7fb5c9073640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5c411b140 0x7fb5c41ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:14.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.661+0000 7fb5c9073640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4120c40 con 0x7fb5c4110190 2024-09-11T23:47:14.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.661+0000 7fb5c9073640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4120ac0 con 0x7fb5c411b140 2024-09-11T23:47:14.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.661+0000 7fb5c9073640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4120dc0 con 0x7fb5c406ea30 2024-09-11T23:47:14.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.662+0000 7fb5c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c406ea30 0x7fb5c41a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:14.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.662+0000 7fb5c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c406ea30 0x7fb5c41a70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48596/0 (socket says 172.21.15.133:48596) 2024-09-11T23:47:14.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.662+0000 7fb5c3fff640 1 -- 172.21.15.133:0/783793967 learned_addr learned my addr 172.21.15.133:0/783793967 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:14.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.662+0000 7fb5c37fe640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c4110190 0x7fb5c41a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:14.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.662+0000 7fb5c8872640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5c411b140 0x7fb5c41ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:14.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.663+0000 7fb5c3fff640 1 -- 172.21.15.133:0/783793967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5c411b140 msgr2=0x7fb5c41ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:14.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.663+0000 7fb5c3fff640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5c411b140 0x7fb5c41ab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.663+0000 7fb5c3fff640 1 -- 172.21.15.133:0/783793967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c4110190 msgr2=0x7fb5c41a7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:14.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.663+0000 7fb5c3fff640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c4110190 0x7fb5c41a7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:14.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.663+0000 7fb5c3fff640 1 -- 172.21.15.133:0/783793967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c41ac090 con 0x7fb5c406ea30 2024-09-11T23:47:14.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.663+0000 7fb5c3fff640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c406ea30 0x7fb5c41a70e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400c920 tx=0x7fb5b400cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:14.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.663+0000 7fb5c17fa640 1 -- 172.21.15.133:0/783793967 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4007c70 con 0x7fb5c406ea30 2024-09-11T23:47:14.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.664+0000 7fb5c17fa640 1 -- 172.21.15.133:0/783793967 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b4007e10 con 0x7fb5c406ea30 2024-09-11T23:47:14.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.664+0000 7fb5c9073640 1 -- 172.21.15.133:0/783793967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c41ac380 con 0x7fb5c406ea30 2024-09-11T23:47:14.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.664+0000 7fb5c17fa640 1 -- 172.21.15.133:0/783793967 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40189a0 con 0x7fb5c406ea30 2024-09-11T23:47:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.664+0000 7fb5c9073640 1 -- 172.21.15.133:0/783793967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c41b3c60 con 0x7fb5c406ea30 2024-09-11T23:47:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.666+0000 7fb5c9073640 1 -- 172.21.15.133:0/783793967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c406ee70 con 0x7fb5c406ea30 2024-09-11T23:47:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.668+0000 7fb5c17fa640 1 -- 172.21.15.133:0/783793967 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb5b4018b40 con 0x7fb5c406ea30 2024-09-11T23:47:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.669+0000 7fb5c17fa640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5ac0a2dc0 0x7fb5ac0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.670+0000 7fb5c17fa640 1 -- 172.21.15.133:0/783793967 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fb5b4099b00 con 0x7fb5c406ea30 2024-09-11T23:47:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.671+0000 7fb5c37fe640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5ac0a2dc0 0x7fb5ac0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:14.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.674+0000 7fb5c37fe640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5ac0a2dc0 0x7fb5ac0a5280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b8005b50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:14.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:14.680+0000 7fb5c17fa640 1 -- 172.21.15.133:0/783793967 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5b4014030 con 0x7fb5c406ea30 2024-09-11T23:47:14.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:14 smithi133 ceph-mon[31430]: pgmap v326: 97 pgs: 97 active+clean; 60 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 10 KiB/s rd, 117 MiB/s wr, 323 op/s 2024-09-11T23:47:14.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:14 smithi133 ceph-mon[31430]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:47:15.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:14 smithi178 ceph-mon[35882]: pgmap v326: 97 pgs: 97 active+clean; 60 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 10 KiB/s rd, 117 MiB/s wr, 323 op/s 2024-09-11T23:47:15.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:14 smithi178 ceph-mon[35882]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:47:15.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.119+0000 7fb5c9073640 1 -- 172.21.15.133:0/783793967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5c41b3e30 con 0x7fb5c406ea30 2024-09-11T23:47:15.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.124+0000 7fb5c17fa640 1 -- 172.21.15.133:0/783793967 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7fb5b409c160 con 0x7fb5c406ea30 2024-09-11T23:47:15.124 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:47:15.126 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:47:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:47:15.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.130+0000 7fb5c9073640 1 -- 172.21.15.133:0/783793967 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5ac0a2dc0 msgr2=0x7fb5ac0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:15.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.130+0000 7fb5c9073640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5ac0a2dc0 0x7fb5ac0a5280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b8005b50 comp rx=0 tx=0).stop 2024-09-11T23:47:15.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.131+0000 7fb5c9073640 1 -- 172.21.15.133:0/783793967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c406ea30 msgr2=0x7fb5c41a70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:15.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.131+0000 7fb5c9073640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c406ea30 0x7fb5c41a70e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400c920 tx=0x7fb5b400cde0 comp rx=0 tx=0).stop 2024-09-11T23:47:15.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.131+0000 7fb5c9073640 1 -- 172.21.15.133:0/783793967 shutdown_connections 2024-09-11T23:47:15.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.131+0000 7fb5c9073640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb5ac0a2dc0 0x7fb5ac0a5280 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:15.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.131+0000 7fb5c9073640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5c411b140 0x7fb5c41ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:15.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.131+0000 7fb5c9073640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c4110190 0x7fb5c41a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:15.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.131+0000 7fb5c9073640 1 --2- 172.21.15.133:0/783793967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5c406ea30 0x7fb5c41a70e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:15.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.131+0000 7fb5c9073640 1 -- 172.21.15.133:0/783793967 >> 172.21.15.133:0/783793967 conn(0x7fb5c4073180 msgr2=0x7fb5c4119270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:15.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.131+0000 7fb5c9073640 1 -- 172.21.15.133:0/783793967 shutdown_connections 2024-09-11T23:47:15.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:15.131+0000 7fb5c9073640 1 -- 172.21.15.133:0/783793967 wait complete. 2024-09-11T23:47:15.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:14 smithi138 ceph-mon[35142]: pgmap v326: 97 pgs: 97 active+clean; 60 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 10 KiB/s rd, 117 MiB/s wr, 323 op/s 2024-09-11T23:47:15.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:14 smithi138 ceph-mon[35142]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:47:16.089 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:15 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/783793967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:16.167 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-11T23:47:16.168 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-11T23:47:16.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:15 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/783793967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:16.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:15 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/783793967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:17.111 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:17.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:16 smithi138 ceph-mon[35142]: pgmap v328: 97 pgs: 97 active+clean; 60 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 12 KiB/s rd, 202 MiB/s wr, 629 op/s 2024-09-11T23:47:17.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:16 smithi133 ceph-mon[31430]: pgmap v328: 97 pgs: 97 active+clean; 60 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 12 KiB/s rd, 202 MiB/s wr, 629 op/s 2024-09-11T23:47:17.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:16 smithi178 ceph-mon[35882]: pgmap v328: 97 pgs: 97 active+clean; 60 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 12 KiB/s rd, 202 MiB/s wr, 629 op/s 2024-09-11T23:47:18.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.277+0000 7f4df3d65640 1 -- 172.21.15.133:0/2187373835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dec106f20 msgr2=0x7f4dec10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:18.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.277+0000 7f4df3d65640 1 --2- 172.21.15.133:0/2187373835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dec106f20 0x7f4dec10eaa0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009ca0 tx=0x7f4de402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.277+0000 7f4df3d65640 1 -- 172.21.15.133:0/2187373835 shutdown_connections 2024-09-11T23:47:18.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.277+0000 7f4df3d65640 1 --2- 172.21.15.133:0/2187373835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dec106f20 0x7f4dec10eaa0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.277+0000 7f4df3d65640 1 --2- 172.21.15.133:0/2187373835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dec1027a0 0x7f4dec1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.277+0000 7f4df3d65640 1 --2- 172.21.15.133:0/2187373835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dec101df0 0x7f4dec1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.277+0000 7f4df3d65640 1 -- 172.21.15.133:0/2187373835 >> 172.21.15.133:0/2187373835 conn(0x7f4dec0fd8b0 msgr2=0x7f4dec0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:18.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.293+0000 7f4df3d65640 1 -- 172.21.15.133:0/2187373835 shutdown_connections 2024-09-11T23:47:18.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.293+0000 7f4df3d65640 1 -- 172.21.15.133:0/2187373835 wait complete. 2024-09-11T23:47:18.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.294+0000 7f4df3d65640 1 Processor -- start 2024-09-11T23:47:18.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.294+0000 7f4df3d65640 1 -- start start 2024-09-11T23:47:18.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.294+0000 7f4df3d65640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dec101df0 0x7f4dec06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.294+0000 7f4df3d65640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dec1027a0 0x7f4dec06d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.294+0000 7f4df3d65640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dec106f20 0x7f4dec070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.294+0000 7f4df3d65640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec06c1e0 con 0x7f4dec106f20 2024-09-11T23:47:18.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.294+0000 7f4df3d65640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec06c060 con 0x7f4dec1027a0 2024-09-11T23:47:18.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.294+0000 7f4df3d65640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec06c360 con 0x7f4dec101df0 2024-09-11T23:47:18.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.295+0000 7f4df2302640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dec106f20 0x7f4dec070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.295+0000 7f4df2302640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dec106f20 0x7f4dec070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56848/0 (socket says 172.21.15.133:56848) 2024-09-11T23:47:18.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.295+0000 7f4df2302640 1 -- 172.21.15.133:0/3569784307 learned_addr learned my addr 172.21.15.133:0/3569784307 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:18.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.296+0000 7f4df2302640 1 -- 172.21.15.133:0/3569784307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dec101df0 msgr2=0x7f4dec06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.296+0000 7f4df2302640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dec101df0 0x7f4dec06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.296+0000 7f4df2302640 1 -- 172.21.15.133:0/3569784307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dec1027a0 msgr2=0x7f4dec06d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.296+0000 7f4df2302640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dec1027a0 0x7f4dec06d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.296+0000 7f4df2302640 1 -- 172.21.15.133:0/3569784307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dec071200 con 0x7f4dec106f20 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.296+0000 7f4df2302640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dec106f20 0x7f4dec070cc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4de402ae30 tx=0x7f4de40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.297+0000 7f4ddeffd640 1 -- 172.21.15.133:0/3569784307 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4031bb0 con 0x7f4dec106f20 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.297+0000 7f4ddeffd640 1 -- 172.21.15.133:0/3569784307 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de4040070 con 0x7f4dec106f20 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.297+0000 7f4ddeffd640 1 -- 172.21.15.133:0/3569784307 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de40305c0 con 0x7f4dec106f20 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.297+0000 7f4df3d65640 1 -- 172.21.15.133:0/3569784307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dec071490 con 0x7f4dec106f20 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.298+0000 7f4df3d65640 1 -- 172.21.15.133:0/3569784307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dec0719d0 con 0x7f4dec106f20 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.303+0000 7f4df613e740 1 Processor -- start 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.303+0000 7f4df613e740 1 -- start start 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.303+0000 7f4df613e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f7a850 0x558952f7ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.303+0000 7f4df613e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.304+0000 7f4df613e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f85a30 0x558952f87e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.304+0000 7f4df613e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558952edf160 con 0x558952f85a30 2024-09-11T23:47:18.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.304+0000 7f4df613e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558952e86720 con 0x558952f7b170 2024-09-11T23:47:18.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.304+0000 7f4df613e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558952edb120 con 0x558952f7a850 2024-09-11T23:47:18.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.312+0000 7f4df1b01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.312+0000 7f4df1300640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f7a850 0x558952f7ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.312+0000 7f4df1300640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f85a30 0x558952f87e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.312+0000 7f4df1300640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f85a30 0x558952f87e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56864/0 (socket says 172.21.15.133:56864) 2024-09-11T23:47:18.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.312+0000 7f4df1300640 1 -- 172.21.15.133:0/1432950428 learned_addr learned my addr 172.21.15.133:0/1432950428 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.324+0000 7f4df1300640 1 -- 172.21.15.133:0/1432950428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 msgr2=0x558952f854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.324+0000 7f4df1300640 1 --2- 172.21.15.133:0/1432950428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f854f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.324+0000 7f4df1300640 1 -- 172.21.15.133:0/1432950428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f85a30 msgr2=0x558952f87e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.324+0000 7f4df1300640 1 --2- 172.21.15.133:0/1432950428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f85a30 0x558952f87e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.324+0000 7f4df1300640 1 -- 172.21.15.133:0/1432950428 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558952c50ba0 con 0x558952f7a850 2024-09-11T23:47:18.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.326+0000 7f4df1300640 1 --2- 172.21.15.133:0/1432950428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f7a850 0x558952f7ac30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4de0005e00 tx=0x7f4de0005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efe0699f2ceead26 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.327+0000 7f4ddeffd640 1 -- 172.21.15.133:0/3569784307 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4de402f610 con 0x7f4dec106f20 2024-09-11T23:47:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.328+0000 7f4dd7fff640 1 -- 172.21.15.133:0/1432950428 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0004020 con 0x558952f7a850 2024-09-11T23:47:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.328+0000 7f4ddeffd640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4dd80a2f00 0x7f4dd80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.328+0000 7f4dd7fff640 1 -- 172.21.15.133:0/1432950428 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de0005500 con 0x558952f7a850 2024-09-11T23:47:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.328+0000 7f4ddeffd640 1 -- 172.21.15.133:0/3569784307 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f4de40f0860 con 0x7f4dec106f20 2024-09-11T23:47:18.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.329+0000 7f4df613e740 1 -- 172.21.15.133:0/1432950428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f7a850 msgr2=0x558952f7ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.329+0000 7f4df613e740 1 --2- 172.21.15.133:0/1432950428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f7a850 0x558952f7ac30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4de0005e00 tx=0x7f4de0005c70 comp rx=0 tx=0).stop 2024-09-11T23:47:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.329+0000 7f4df613e740 1 -- 172.21.15.133:0/1432950428 shutdown_connections 2024-09-11T23:47:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.329+0000 7f4df613e740 1 --2- 172.21.15.133:0/1432950428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f85a30 0x558952f87e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.330+0000 7f4df613e740 1 --2- 172.21.15.133:0/1432950428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.330+0000 7f4df613e740 1 --2- 172.21.15.133:0/1432950428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f7a850 0x558952f7ac30 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4de0005e00 tx=0x7f4de0005c70 comp rx=0 tx=0).stop 2024-09-11T23:47:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.330+0000 7f4df613e740 1 -- 172.21.15.133:0/1432950428 >> 172.21.15.133:0/1432950428 conn(0x558952f76de0 msgr2=0x558952f771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.331+0000 7f4df1b01640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4dd80a2f00 0x7f4dd80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.331+0000 7f4df613e740 1 -- 172.21.15.133:0/1432950428 shutdown_connections 2024-09-11T23:47:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.331+0000 7f4df613e740 1 -- 172.21.15.133:0/1432950428 wait complete. 2024-09-11T23:47:18.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.332+0000 7f4df613e740 1 -- start start 2024-09-11T23:47:18.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.332+0000 7f4df613e740 1 Processor -- start 2024-09-11T23:47:18.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.333+0000 7f4df613e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f82b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.333+0000 7f4df613e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f85a30 0x558952f83090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.333+0000 7f4df613e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f835d0 0x558952f91b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.333+0000 7f4df613e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558952f88650 con 0x558952f835d0 2024-09-11T23:47:18.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.333+0000 7f4df613e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558952e67900 con 0x558952f7b170 2024-09-11T23:47:18.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.333+0000 7f4df613e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558952f887d0 con 0x558952f85a30 2024-09-11T23:47:18.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.333+0000 7f4df613e740 10 client.0 _ll_register_callbacks cb 0x558952e65780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:18.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.335+0000 7f4df1300640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f85a30 0x558952f83090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.335+0000 7f4df2302640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f835d0 0x558952f91b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.335+0000 7f4df1b01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f82b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.335+0000 7f4df1b01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f82b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:36664/0 (socket says 172.21.15.133:36664) 2024-09-11T23:47:18.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.335+0000 7f4df1b01640 1 -- 172.21.15.133:0/3230611977 learned_addr learned my addr 172.21.15.133:0/3230611977 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:18.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.335+0000 7f4df1b01640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4dd80a2f00 0x7f4dd80a53c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x558952f77c40 tx=0x7f4de8011440 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:18.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.337+0000 7f4df2302640 1 -- 172.21.15.133:0/3230611977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f85a30 msgr2=0x558952f83090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:18.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.337+0000 7f4df2302640 1 --2- 172.21.15.133:0/3230611977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f85a30 0x558952f83090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.337+0000 7f4df2302640 1 -- 172.21.15.133:0/3230611977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 msgr2=0x558952f82b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:18.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.337+0000 7f4df2302640 1 --2- 172.21.15.133:0/3230611977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f82b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.337+0000 7f4df2302640 1 -- 172.21.15.133:0/3230611977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558952ea7c50 con 0x558952f835d0 2024-09-11T23:47:18.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.338+0000 7f4df2302640 1 --2- 172.21.15.133:0/3230611977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f835d0 0x558952f91b40 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4de40ee0a0 tx=0x7f4de40bc9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf92fc37979061e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:18.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.340+0000 7f4dd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:18.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.340+0000 7f4dd57fa640 1 -- 172.21.15.133:0/3230611977 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de40ee420 con 0x558952f835d0 2024-09-11T23:47:18.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.340+0000 7f4df613e740 1 -- 172.21.15.133:0/3230611977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558952eec890 con 0x558952f835d0 2024-09-11T23:47:18.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.340+0000 7f4dd57fa640 1 -- 172.21.15.133:0/3230611977 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f4de40c5650 con 0x558952f835d0 2024-09-11T23:47:18.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.341+0000 7f4dd57fa640 1 -- 172.21.15.133:0/3230611977 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4de40bd570 con 0x558952f835d0 2024-09-11T23:47:18.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.341+0000 7f4df613e740 10 client.18150 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:18.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.341+0000 7f4df613e740 1 -- 172.21.15.133:0/3230611977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558952f88650 con 0x558952f835d0 2024-09-11T23:47:18.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.342+0000 7f4dd57fa640 1 -- 172.21.15.133:0/3230611977 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558952eec890 con 0x558952f835d0 2024-09-11T23:47:18.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.342+0000 7f4dd57fa640 1 -- 172.21.15.133:0/3230611977 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4de404d020 con 0x558952f835d0 2024-09-11T23:47:18.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.342+0000 7f4df613e740 10 client.18150 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:18.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.342+0000 7f4df613e740 10 client.18150 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:47:18.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.342+0000 7f4df613e740 20 client.18150 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:18.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.343+0000 7f4df613e740 1 --2- 172.21.15.133:0/3230611977 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558952f95c30 0x558952f98070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.343+0000 7f4df613e740 4 client.18150 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:18.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.343+0000 7f4df613e740 1 -- 172.21.15.133:0/3230611977 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558952ee6be0 con 0x558952f95c30 2024-09-11T23:47:18.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.345+0000 7f4df1b01640 1 --2- 172.21.15.133:0/3230611977 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558952f95c30 0x558952f98070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.349+0000 7f4df1b01640 1 --2- 172.21.15.133:0/3230611977 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558952f95c30 0x558952f98070 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=127d68360494163a server_cookie=fcca59559e38aa4c in_seq=0 out_seq=0 2024-09-11T23:47:18.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.350+0000 7f4dd57fa640 10 client.18150 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:18.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.357+0000 7f4dd57fa640 1 -- 172.21.15.133:0/3230611977 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558952ee6be0 con 0x558952f95c30 2024-09-11T23:47:18.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.357+0000 7f4dd57fa640 10 client.18150 handle_command_reply: tid=0 2024-09-11T23:47:18.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.357+0000 7f4df613e740 1 client.18150 shutdown 2024-09-11T23:47:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.359+0000 7f4df613e740 1 -- 172.21.15.133:0/3230611977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f835d0 msgr2=0x558952f91b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.359+0000 7f4df613e740 1 --2- 172.21.15.133:0/3230611977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f835d0 0x558952f91b40 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4de40ee0a0 tx=0x7f4de40bc9b0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.359+0000 7f4df613e740 1 -- 172.21.15.133:0/3230611977 shutdown_connections 2024-09-11T23:47:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.359+0000 7f4df613e740 1 --2- 172.21.15.133:0/3230611977 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558952f95c30 0x558952f98070 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.359+0000 7f4df613e740 1 --2- 172.21.15.133:0/3230611977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f835d0 0x558952f91b40 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.359+0000 7f4df613e740 1 --2- 172.21.15.133:0/3230611977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f85a30 0x558952f83090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.359+0000 7f4df613e740 1 --2- 172.21.15.133:0/3230611977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f82b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.359+0000 7f4df613e740 1 -- 172.21.15.133:0/3230611977 >> 172.21.15.133:0/3230611977 conn(0x558952f76de0 msgr2=0x558952f7cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:18.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.359+0000 7f4dd57fa640 0 client.18150 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:18.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.359+0000 7f4df613e740 1 -- 172.21.15.133:0/3230611977 shutdown_connections 2024-09-11T23:47:18.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.359+0000 7f4df613e740 1 -- 172.21.15.133:0/3230611977 wait complete. 2024-09-11T23:47:18.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.360+0000 7f4df613e740 20 client.18150 trim_cache size 0 max 16384 2024-09-11T23:47:18.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.425+0000 7f4df613e740 1 Processor -- start 2024-09-11T23:47:18.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.427+0000 7f4df613e740 1 -- start start 2024-09-11T23:47:18.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.428+0000 7f4df613e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f7dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.428+0000 7f4df613e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f835d0 0x558952f7d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.428+0000 7f4df613e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f85a30 0x558952f75db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.428+0000 7f4df613e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec1b3c10 con 0x558952f85a30 2024-09-11T23:47:18.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.428+0000 7f4df613e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec1b5750 con 0x558952f7b170 2024-09-11T23:47:18.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.428+0000 7f4df613e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec1b42e0 con 0x558952f835d0 2024-09-11T23:47:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.428+0000 7f4df1b01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f7dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.428+0000 7f4df1b01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f7dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:36666/0 (socket says 172.21.15.133:36666) 2024-09-11T23:47:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.428+0000 7f4df1b01640 1 -- 172.21.15.133:0/1028603905 learned_addr learned my addr 172.21.15.133:0/1028603905 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.428+0000 7f4df1300640 1 --2- 172.21.15.133:0/1028603905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f835d0 0x558952f7d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.429+0000 7f4df1b01640 1 -- 172.21.15.133:0/1028603905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f835d0 msgr2=0x558952f7d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.429+0000 7f4df1b01640 1 --2- 172.21.15.133:0/1028603905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f835d0 0x558952f7d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.429+0000 7f4df1b01640 1 -- 172.21.15.133:0/1028603905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f85a30 msgr2=0x558952f75db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.429+0000 7f4df1b01640 1 --2- 172.21.15.133:0/1028603905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f85a30 0x558952f75db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.429+0000 7f4df1b01640 1 -- 172.21.15.133:0/1028603905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558952f88190 con 0x558952f7b170 2024-09-11T23:47:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.431+0000 7f4df1b01640 1 --2- 172.21.15.133:0/1028603905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f7dfe0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4de8011940 tx=0x7f4de801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af0b7a8e790b196f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.432+0000 7f4dd77fe640 1 -- 172.21.15.133:0/1028603905 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8017040 con 0x558952f7b170 2024-09-11T23:47:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.432+0000 7f4dd77fe640 1 -- 172.21.15.133:0/1028603905 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de8019050 con 0x558952f7b170 2024-09-11T23:47:18.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.432+0000 7f4dd77fe640 1 -- 172.21.15.133:0/1028603905 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de801d440 con 0x558952f7b170 2024-09-11T23:47:18.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.435+0000 7f4df613e740 1 -- 172.21.15.133:0/1028603905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 msgr2=0x558952f7dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:18.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.435+0000 7f4df613e740 1 --2- 172.21.15.133:0/1028603905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f7dfe0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4de8011940 tx=0x7f4de801c000 comp rx=0 tx=0).stop 2024-09-11T23:47:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.435+0000 7f4df613e740 1 -- 172.21.15.133:0/1028603905 shutdown_connections 2024-09-11T23:47:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.435+0000 7f4df613e740 1 --2- 172.21.15.133:0/1028603905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f85a30 0x558952f75db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.435+0000 7f4df613e740 1 --2- 172.21.15.133:0/1028603905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f835d0 0x558952f7d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.435+0000 7f4df613e740 1 --2- 172.21.15.133:0/1028603905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f7dfe0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.436+0000 7f4df613e740 1 -- 172.21.15.133:0/1028603905 >> 172.21.15.133:0/1028603905 conn(0x558952f76de0 msgr2=0x558952f756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:18.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.440+0000 7f4df613e740 1 -- 172.21.15.133:0/1028603905 shutdown_connections 2024-09-11T23:47:18.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.440+0000 7f4df613e740 1 -- 172.21.15.133:0/1028603905 wait complete. 2024-09-11T23:47:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.441+0000 7f4df613e740 1 -- start start 2024-09-11T23:47:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.441+0000 7f4df613e740 1 Processor -- start 2024-09-11T23:47:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.442+0000 7f4df613e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f89890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.443+0000 7f4df1b01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f89890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.443+0000 7f4df1b01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f89890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:36668/0 (socket says 172.21.15.133:36668) 2024-09-11T23:47:18.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.453+0000 7f4df1b01640 1 -- 172.21.15.133:0/2846386762 learned_addr learned my addr 172.21.15.133:0/2846386762 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:18.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.453+0000 7f4df613e740 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f835d0 0x558952f89dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.453+0000 7f4df613e740 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f85a30 0x558952f8a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.454+0000 7f4df613e740 1 -- 172.21.15.133:0/2846386762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec1b5080 con 0x558952f835d0 2024-09-11T23:47:18.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.454+0000 7f4df613e740 1 -- 172.21.15.133:0/2846386762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec1b49b0 con 0x558952f7b170 2024-09-11T23:47:18.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.454+0000 7f4df613e740 1 -- 172.21.15.133:0/2846386762 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec1021d0 con 0x558952f85a30 2024-09-11T23:47:18.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.454+0000 7f4df613e740 10 client.0 _ll_register_callbacks cb 0x7f4de402f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.454+0000 7f4df2302640 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f85a30 0x558952f8a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.454+0000 7f4df1b01640 1 -- 172.21.15.133:0/2846386762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f85a30 msgr2=0x558952f8a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.454+0000 7f4df1b01640 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f85a30 0x558952f8a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.454+0000 7f4df1b01640 1 -- 172.21.15.133:0/2846386762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f835d0 msgr2=0x558952f89dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.454+0000 7f4df1b01640 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f835d0 0x558952f89dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.454+0000 7f4df1b01640 1 -- 172.21.15.133:0/2846386762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558952fff9c0 con 0x558952f7b170 2024-09-11T23:47:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.455+0000 7f4df1b01640 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f89890 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4de8000c00 tx=0x7f4de8003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=531b65fc8b1d0c55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.455+0000 7f4dd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:47:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.455+0000 7f4dd57fa640 1 -- 172.21.15.133:0/2846386762 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8017050 con 0x558952f7b170 2024-09-11T23:47:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.456+0000 7f4dd57fa640 1 -- 172.21.15.133:0/2846386762 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f4de8003b30 con 0x558952f7b170 2024-09-11T23:47:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.456+0000 7f4df613e740 1 -- 172.21.15.133:0/2846386762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4de8019050 con 0x558952f7b170 2024-09-11T23:47:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.456+0000 7f4dd57fa640 1 -- 172.21.15.133:0/2846386762 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4de800f480 con 0x558952f7b170 2024-09-11T23:47:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.458+0000 7f4df613e740 10 client.27695 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.458+0000 7f4df613e740 1 -- 172.21.15.133:0/2846386762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558953000c40 con 0x558952f7b170 2024-09-11T23:47:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.466+0000 7f4dd57fa640 1 -- 172.21.15.133:0/2846386762 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de800fd40 con 0x558952f7b170 2024-09-11T23:47:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.466+0000 7f4dd57fa640 1 -- 172.21.15.133:0/2846386762 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4de801d9b0 con 0x558952f7b170 2024-09-11T23:47:18.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.469+0000 7f4df613e740 10 client.27695 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.469+0000 7f4df613e740 10 client.27695 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:47:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.469+0000 7f4df613e740 20 client.27695 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.469+0000 7f4df613e740 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558953001220 0x558953003610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.469+0000 7f4df613e740 4 client.27695 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-11T23:47:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.469+0000 7f4df613e740 1 -- 172.21.15.133:0/2846386762 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x7f4de8019050 con 0x558953001220 2024-09-11T23:47:18.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.473+0000 7f4df1300640 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558953001220 0x558953003610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:18.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.486+0000 7f4dd57fa640 10 client.27695 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:18.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.486+0000 7f4df1300640 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558953001220 0x558953003610 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada98d20f63d6ddd server_cookie=aafa3a3878227a67 in_seq=0 out_seq=0 2024-09-11T23:47:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.774+0000 7f4dd57fa640 1 -- 172.21.15.133:0/2846386762 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7f4de8019050 con 0x558953001220 2024-09-11T23:47:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.774+0000 7f4dd57fa640 10 client.27695 handle_command_reply: tid=0 2024-09-11T23:47:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.774+0000 7f4df613e740 1 client.27695 shutdown 2024-09-11T23:47:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df613e740 1 -- 172.21.15.133:0/2846386762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 msgr2=0x558952f89890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df613e740 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f89890 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4de8000c00 tx=0x7f4de8003680 comp rx=0 tx=0).stop 2024-09-11T23:47:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df613e740 1 -- 172.21.15.133:0/2846386762 shutdown_connections 2024-09-11T23:47:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df613e740 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x558953001220 0x558953003610 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df613e740 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558952f85a30 0x558952f8a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df613e740 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558952f835d0 0x558952f89dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4dd57fa640 0 client.27695 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:18.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df613e740 1 --2- 172.21.15.133:0/2846386762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558952f7b170 0x558952f89890 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df613e740 1 -- 172.21.15.133:0/2846386762 >> 172.21.15.133:0/2846386762 conn(0x558952f76de0 msgr2=0x558952f82c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:18.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df613e740 1 -- 172.21.15.133:0/2846386762 shutdown_connections 2024-09-11T23:47:18.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df613e740 1 -- 172.21.15.133:0/2846386762 wait complete. 2024-09-11T23:47:18.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df613e740 20 client.27695 trim_cache size 0 max 16384 2024-09-11T23:47:18.776 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:47:18.776 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:47:18.776 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:47:18.776 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 42, 2024-09-11T23:47:18.776 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "3c36fe3": { 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 42, 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 142.6, 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "QUIESCED", 2024-09-11T23:47:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:47:18.778 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:18.778 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:18.778 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:18.778 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:18.778 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:18.778 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df3d65640 1 -- 172.21.15.133:0/3569784307 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4dd80a2f00 msgr2=0x7f4dd80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df3d65640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4dd80a2f00 0x7f4dd80a53c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x558952f77c40 tx=0x7f4de8011440 comp rx=0 tx=0).stop 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df3d65640 1 -- 172.21.15.133:0/3569784307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dec106f20 msgr2=0x7f4dec070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.775+0000 7f4df3d65640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dec106f20 0x7f4dec070cc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4de402ae30 tx=0x7f4de40319f0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.776+0000 7f4df3d65640 1 -- 172.21.15.133:0/3569784307 shutdown_connections 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.776+0000 7f4df3d65640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4dd80a2f00 0x7f4dd80a53c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.776+0000 7f4df3d65640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dec106f20 0x7f4dec070cc0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.776+0000 7f4df3d65640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dec1027a0 0x7f4dec06d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.776+0000 7f4df3d65640 1 --2- 172.21.15.133:0/3569784307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dec101df0 0x7f4dec06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.776+0000 7f4df3d65640 1 -- 172.21.15.133:0/3569784307 >> 172.21.15.133:0/3569784307 conn(0x7f4dec0fd8b0 msgr2=0x7f4dec1057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.776+0000 7f4df3d65640 1 -- 172.21.15.133:0/3569784307 shutdown_connections 2024-09-11T23:47:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:18.776+0000 7f4df3d65640 1 -- 172.21.15.133:0/3569784307 wait complete. 2024-09-11T23:47:19.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:18 smithi133 ceph-mon[31430]: pgmap v329: 97 pgs: 97 active+clean; 60 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 392 KiB/s rd, 214 MiB/s wr, 706 op/s 2024-09-11T23:47:19.200 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:18 smithi138 ceph-mon[35142]: pgmap v329: 97 pgs: 97 active+clean; 60 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 392 KiB/s rd, 214 MiB/s wr, 706 op/s 2024-09-11T23:47:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:18 smithi178 ceph-mon[35882]: pgmap v329: 97 pgs: 97 active+clean; 60 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 392 KiB/s rd, 214 MiB/s wr, 706 op/s 2024-09-11T23:47:19.584 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3c36fe3', quiesce took 6.5 seconds. Will release after: 16.05 2024-09-11T23:47:21.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:20 smithi138 ceph-mon[35142]: pgmap v330: 97 pgs: 97 active+clean; 61 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 407 MiB/s wr, 1.76k op/s 2024-09-11T23:47:21.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:20 smithi133 ceph-mon[31430]: pgmap v330: 97 pgs: 97 active+clean; 61 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 407 MiB/s wr, 1.76k op/s 2024-09-11T23:47:21.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:20 smithi178 ceph-mon[35882]: pgmap v330: 97 pgs: 97 active+clean; 61 GiB data, 264 GiB used, 809 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 407 MiB/s wr, 1.76k op/s 2024-09-11T23:47:23.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:22 smithi138 ceph-mon[35142]: pgmap v331: 97 pgs: 97 active+clean; 61 GiB data, 260 GiB used, 812 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 484 MiB/s wr, 2.15k op/s 2024-09-11T23:47:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:22 smithi133 ceph-mon[31430]: pgmap v331: 97 pgs: 97 active+clean; 61 GiB data, 260 GiB used, 812 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 484 MiB/s wr, 2.15k op/s 2024-09-11T23:47:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:22 smithi178 ceph-mon[35882]: pgmap v331: 97 pgs: 97 active+clean; 61 GiB data, 260 GiB used, 812 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 484 MiB/s wr, 2.15k op/s 2024-09-11T23:47:24.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:24 smithi138 ceph-mon[35142]: pgmap v332: 97 pgs: 97 active+clean; 62 GiB data, 262 GiB used, 811 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 419 MiB/s wr, 1.99k op/s 2024-09-11T23:47:24.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:24 smithi133 ceph-mon[31430]: pgmap v332: 97 pgs: 97 active+clean; 62 GiB data, 262 GiB used, 811 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 419 MiB/s wr, 1.99k op/s 2024-09-11T23:47:24.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:24 smithi178 ceph-mon[35882]: pgmap v332: 97 pgs: 97 active+clean; 62 GiB data, 262 GiB used, 811 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 419 MiB/s wr, 1.99k op/s 2024-09-11T23:47:27.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:26 smithi178 ceph-mon[35882]: pgmap v333: 97 pgs: 97 active+clean; 62 GiB data, 262 GiB used, 811 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 350 MiB/s wr, 1.68k op/s 2024-09-11T23:47:27.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:26 smithi138 ceph-mon[35142]: pgmap v333: 97 pgs: 97 active+clean; 62 GiB data, 262 GiB used, 811 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 350 MiB/s wr, 1.68k op/s 2024-09-11T23:47:27.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:26 smithi133 ceph-mon[31430]: pgmap v333: 97 pgs: 97 active+clean; 62 GiB data, 262 GiB used, 811 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 350 MiB/s wr, 1.68k op/s 2024-09-11T23:47:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.832+0000 7f24dc05e640 1 -- 172.21.15.133:0/2248180408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d4113420 msgr2=0x7f24d4115810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.832+0000 7f24dc05e640 1 --2- 172.21.15.133:0/2248180408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d4113420 0x7f24d4115810 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009c10 tx=0x7f24cc02a6e0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.834+0000 7f24dc05e640 1 -- 172.21.15.133:0/2248180408 shutdown_connections 2024-09-11T23:47:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.834+0000 7f24dc05e640 1 --2- 172.21.15.133:0/2248180408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d4113420 0x7f24d4115810 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.834+0000 7f24dc05e640 1 --2- 172.21.15.133:0/2248180408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24d4109d80 0x7f24d410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.834+0000 7f24dc05e640 1 --2- 172.21.15.133:0/2248180408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24d4069820 0x7f24d410e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.834+0000 7f24dc05e640 1 -- 172.21.15.133:0/2248180408 >> 172.21.15.133:0/2248180408 conn(0x7f24d406d210 msgr2=0x7f24d406d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.835+0000 7f24dc05e640 1 -- 172.21.15.133:0/2248180408 shutdown_connections 2024-09-11T23:47:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.836+0000 7f24dc05e640 1 -- 172.21.15.133:0/2248180408 wait complete. 2024-09-11T23:47:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.837+0000 7f24dc05e640 1 Processor -- start 2024-09-11T23:47:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.837+0000 7f24dc05e640 1 -- start start 2024-09-11T23:47:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.837+0000 7f24dc05e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24d4069820 0x7f24d41a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.837+0000 7f24dc05e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d4109d80 0x7f24d41a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.837+0000 7f24dc05e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24d4113420 0x7f24d41a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.837+0000 7f24dc05e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24d411a700 con 0x7f24d4113420 2024-09-11T23:47:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.837+0000 7f24dc05e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24d411a580 con 0x7f24d4069820 2024-09-11T23:47:27.839 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.837+0000 7f24dc05e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24d411a880 con 0x7f24d4109d80 2024-09-11T23:47:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.838+0000 7f24da5fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24d4113420 0x7f24d41a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.838+0000 7f24da5fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24d4113420 0x7f24d41a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59948/0 (socket says 172.21.15.133:59948) 2024-09-11T23:47:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.838+0000 7f24da5fb640 1 -- 172.21.15.133:0/2684706756 learned_addr learned my addr 172.21.15.133:0/2684706756 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.838+0000 7f24d9dfa640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24d4069820 0x7f24d41a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.838+0000 7f24d95f9640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d4109d80 0x7f24d41a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.838+0000 7f24da5fb640 1 -- 172.21.15.133:0/2684706756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d4109d80 msgr2=0x7f24d41a6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.838+0000 7f24da5fb640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d4109d80 0x7f24d41a6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.838+0000 7f24da5fb640 1 -- 172.21.15.133:0/2684706756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24d4069820 msgr2=0x7f24d41a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.838+0000 7f24da5fb640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24d4069820 0x7f24d41a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.839+0000 7f24da5fb640 1 -- 172.21.15.133:0/2684706756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d41a1130 con 0x7f24d4113420 2024-09-11T23:47:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.839+0000 7f24da5fb640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24d4113420 0x7f24d41a09b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f24cc034040 tx=0x7f24cc0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.839+0000 7f24caffd640 1 -- 172.21.15.133:0/2684706756 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc041030 con 0x7f24d4113420 2024-09-11T23:47:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.840+0000 7f24caffd640 1 -- 172.21.15.133:0/2684706756 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24cc003e50 con 0x7f24d4113420 2024-09-11T23:47:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.841+0000 7f24dc05e640 1 -- 172.21.15.133:0/2684706756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d41a13c0 con 0x7f24d4113420 2024-09-11T23:47:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.841+0000 7f24dc05e640 1 -- 172.21.15.133:0/2684706756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d4116e60 con 0x7f24d4113420 2024-09-11T23:47:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.842+0000 7f24caffd640 1 -- 172.21.15.133:0/2684706756 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc036040 con 0x7f24d4113420 2024-09-11T23:47:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.846+0000 7f24caffd640 1 -- 172.21.15.133:0/2684706756 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f24cc044070 con 0x7f24d4113420 2024-09-11T23:47:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.846+0000 7f24dedf6740 1 Processor -- start 2024-09-11T23:47:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.846+0000 7f24dedf6740 1 -- start start 2024-09-11T23:47:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca855af0 0x5556ca855ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca856410 0x5556ca860770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca8630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24dedf6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5556ca7ba2f0 con 0x5556ca860cb0 2024-09-11T23:47:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24dedf6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5556ca762270 con 0x5556ca856410 2024-09-11T23:47:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24dedf6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5556ca7b62b0 con 0x5556ca855af0 2024-09-11T23:47:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24d95f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca8630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24d95f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca8630a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59956/0 (socket says 172.21.15.133:59956) 2024-09-11T23:47:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24d95f9640 1 -- 172.21.15.133:0/408964223 learned_addr learned my addr 172.21.15.133:0/408964223 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24d95f9640 1 -- 172.21.15.133:0/408964223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca855af0 msgr2=0x5556ca855ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24d95f9640 1 --2- 172.21.15.133:0/408964223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca855af0 0x5556ca855ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24d95f9640 1 -- 172.21.15.133:0/408964223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca856410 msgr2=0x5556ca860770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24d95f9640 1 --2- 172.21.15.133:0/408964223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca856410 0x5556ca860770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.847+0000 7f24d95f9640 1 -- 172.21.15.133:0/408964223 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ca761340 con 0x5556ca860cb0 2024-09-11T23:47:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.850+0000 7f24caffd640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f24c00a2ce0 0x7f24c00a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.850+0000 7f24caffd640 1 -- 172.21.15.133:0/2684706756 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f24cc0f0060 con 0x7f24d4113420 2024-09-11T23:47:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.851+0000 7f24d95f9640 1 --2- 172.21.15.133:0/408964223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca8630a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f24d00096a0 tx=0x7f24d0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13bfd714e0ac83d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.851+0000 7f24d9dfa640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f24c00a2ce0 0x7f24c00a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.851+0000 7f24bffff640 1 -- 172.21.15.133:0/408964223 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0046070 con 0x5556ca860cb0 2024-09-11T23:47:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.852+0000 7f24bffff640 1 -- 172.21.15.133:0/408964223 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d0009db0 con 0x5556ca860cb0 2024-09-11T23:47:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.853+0000 7f24bffff640 1 -- 172.21.15.133:0/408964223 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0003850 con 0x5556ca860cb0 2024-09-11T23:47:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.853+0000 7f24dedf6740 1 -- 172.21.15.133:0/408964223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 msgr2=0x5556ca8630a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.853+0000 7f24dedf6740 1 --2- 172.21.15.133:0/408964223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca8630a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f24d00096a0 tx=0x7f24d0009b60 comp rx=0 tx=0).stop 2024-09-11T23:47:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.853+0000 7f24d9dfa640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f24c00a2ce0 0x7f24c00a51a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009510 tx=0x7f24c4009460 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.854+0000 7f24dedf6740 1 -- 172.21.15.133:0/408964223 shutdown_connections 2024-09-11T23:47:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.854+0000 7f24dedf6740 1 --2- 172.21.15.133:0/408964223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca8630a0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.854+0000 7f24dedf6740 1 --2- 172.21.15.133:0/408964223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca856410 0x5556ca860770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.854+0000 7f24dedf6740 1 --2- 172.21.15.133:0/408964223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca855af0 0x5556ca855ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.854+0000 7f24dedf6740 1 -- 172.21.15.133:0/408964223 >> 172.21.15.133:0/408964223 conn(0x5556ca852060 msgr2=0x5556ca852440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.855+0000 7f24dedf6740 1 -- 172.21.15.133:0/408964223 shutdown_connections 2024-09-11T23:47:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.855+0000 7f24dedf6740 1 -- 172.21.15.133:0/408964223 wait complete. 2024-09-11T23:47:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.856+0000 7f24dedf6740 1 -- start start 2024-09-11T23:47:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.856+0000 7f24dedf6740 1 Processor -- start 2024-09-11T23:47:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.857+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca855af0 0x5556ca85dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.857+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca856410 0x5556ca85e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.857+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca85e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.857+0000 7f24dedf6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5556ca8638d0 con 0x5556ca860cb0 2024-09-11T23:47:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.857+0000 7f24dedf6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5556ca743290 con 0x5556ca855af0 2024-09-11T23:47:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.857+0000 7f24dedf6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5556ca863a50 con 0x5556ca856410 2024-09-11T23:47:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.857+0000 7f24dedf6740 10 client.0 _ll_register_callbacks cb 0x5556ca774140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.858+0000 7f24da5fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca85e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.858+0000 7f24da5fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca85e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59962/0 (socket says 172.21.15.133:59962) 2024-09-11T23:47:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.858+0000 7f24da5fb640 1 -- 172.21.15.133:0/3408439258 learned_addr learned my addr 172.21.15.133:0/3408439258 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.858+0000 7f24d9dfa640 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca855af0 0x5556ca85dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.859+0000 7f24da5fb640 1 -- 172.21.15.133:0/3408439258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca856410 msgr2=0x5556ca85e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.859+0000 7f24da5fb640 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca856410 0x5556ca85e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.859+0000 7f24da5fb640 1 -- 172.21.15.133:0/3408439258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca855af0 msgr2=0x5556ca85dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.859+0000 7f24da5fb640 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca855af0 0x5556ca85dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.859+0000 7f24da5fb640 1 -- 172.21.15.133:0/3408439258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ca4cd210 con 0x5556ca860cb0 2024-09-11T23:47:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.860+0000 7f24da5fb640 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca85e650 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f24cc004af0 tx=0x7f24cc0bc100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75c366eb08658f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.860+0000 7f24bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.861+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3408439258 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc03ae10 con 0x5556ca860cb0 2024-09-11T23:47:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.861+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3408439258 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f24cc0b8c30 con 0x5556ca860cb0 2024-09-11T23:47:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.861+0000 7f24dedf6740 1 -- 172.21.15.133:0/3408439258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d0009db0 con 0x5556ca860cb0 2024-09-11T23:47:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.862+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3408439258 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24cc0bc5c0 con 0x5556ca860cb0 2024-09-11T23:47:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.862+0000 7f24dedf6740 10 client.18162 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.862+0000 7f24dedf6740 1 -- 172.21.15.133:0/3408439258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556ca599620 con 0x5556ca860cb0 2024-09-11T23:47:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.866+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3408439258 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d0009db0 con 0x5556ca860cb0 2024-09-11T23:47:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.868+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3408439258 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f24cc04e060 con 0x5556ca860cb0 2024-09-11T23:47:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.868+0000 7f24dedf6740 10 client.18162 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.868+0000 7f24dedf6740 10 client.18162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.868+0000 7f24dedf6740 20 client.18162 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.868+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ca870c60 0x5556ca873120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.868+0000 7f24dedf6740 4 client.18162 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.868+0000 7f24dedf6740 1 -- 172.21.15.133:0/3408439258 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556ca7c7a20 con 0x5556ca870c60 2024-09-11T23:47:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.872+0000 7f24d9dfa640 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ca870c60 0x5556ca873120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.879+0000 7f24d9dfa640 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ca870c60 0x5556ca873120 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9becd506c37a9482 server_cookie=25a8595a08638bf1 in_seq=0 out_seq=0 2024-09-11T23:47:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.879+0000 7f24bd7fa640 10 client.18162 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.880+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3408439258 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556ca7c7a20 con 0x5556ca870c60 2024-09-11T23:47:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.880+0000 7f24bd7fa640 10 client.18162 handle_command_reply: tid=0 2024-09-11T23:47:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.881+0000 7f24dedf6740 1 client.18162 shutdown 2024-09-11T23:47:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.882+0000 7f24dedf6740 1 -- 172.21.15.133:0/3408439258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 msgr2=0x5556ca85e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.882+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca85e650 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f24cc004af0 tx=0x7f24cc0bc100 comp rx=0 tx=0).stop 2024-09-11T23:47:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.887+0000 7f24dedf6740 1 -- 172.21.15.133:0/3408439258 shutdown_connections 2024-09-11T23:47:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.887+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ca870c60 0x5556ca873120 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.887+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca85e650 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.887+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca856410 0x5556ca85e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.887+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3408439258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca855af0 0x5556ca85dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.887+0000 7f24dedf6740 1 -- 172.21.15.133:0/3408439258 >> 172.21.15.133:0/3408439258 conn(0x5556ca852060 msgr2=0x5556ca857dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.887+0000 7f24bd7fa640 0 client.18162 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.892+0000 7f24dedf6740 1 -- 172.21.15.133:0/3408439258 shutdown_connections 2024-09-11T23:47:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.896+0000 7f24dedf6740 1 -- 172.21.15.133:0/3408439258 wait complete. 2024-09-11T23:47:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.897+0000 7f24dedf6740 20 client.18162 trim_cache size 0 max 16384 2024-09-11T23:47:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.921+0000 7f24dedf6740 1 Processor -- start 2024-09-11T23:47:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.921+0000 7f24dedf6740 1 -- start start 2024-09-11T23:47:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.921+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca855af0 0x5556ca85c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca856410 0x5556ca85e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca860cb0 0x5556ca878a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24dedf6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24d407a760 con 0x5556ca855af0 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24dedf6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24d407c2a0 con 0x5556ca860cb0 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24dedf6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24d407ae30 con 0x5556ca856410 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24d9dfa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca855af0 0x5556ca85c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24d9dfa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca855af0 0x5556ca85c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59968/0 (socket says 172.21.15.133:59968) 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24d9dfa640 1 -- 172.21.15.133:0/2983306097 learned_addr learned my addr 172.21.15.133:0/2983306097 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24d95f9640 1 --2- 172.21.15.133:0/2983306097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca856410 0x5556ca85e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24d9dfa640 1 -- 172.21.15.133:0/2983306097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca856410 msgr2=0x5556ca85e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24d9dfa640 1 --2- 172.21.15.133:0/2983306097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca856410 0x5556ca85e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24d9dfa640 1 -- 172.21.15.133:0/2983306097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca860cb0 msgr2=0x5556ca878a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24d9dfa640 1 --2- 172.21.15.133:0/2983306097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca860cb0 0x5556ca878a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24d9dfa640 1 -- 172.21.15.133:0/2983306097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ca85cb60 con 0x5556ca855af0 2024-09-11T23:47:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24d9dfa640 1 --2- 172.21.15.133:0/2983306097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca855af0 0x5556ca85c480 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x5556ca852ec0 tx=0x7f24c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d4896727c1c091 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.922+0000 7f24bf7fe640 1 -- 172.21.15.133:0/2983306097 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c40295a0 con 0x5556ca855af0 2024-09-11T23:47:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.923+0000 7f24bf7fe640 1 -- 172.21.15.133:0/2983306097 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c402f070 con 0x5556ca855af0 2024-09-11T23:47:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.923+0000 7f24bf7fe640 1 -- 172.21.15.133:0/2983306097 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4020c20 con 0x5556ca855af0 2024-09-11T23:47:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.923+0000 7f24dedf6740 1 -- 172.21.15.133:0/2983306097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca855af0 msgr2=0x5556ca85c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.923+0000 7f24dedf6740 1 --2- 172.21.15.133:0/2983306097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca855af0 0x5556ca85c480 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x5556ca852ec0 tx=0x7f24c4014000 comp rx=0 tx=0).stop 2024-09-11T23:47:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.923+0000 7f24dedf6740 1 -- 172.21.15.133:0/2983306097 shutdown_connections 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.923+0000 7f24dedf6740 1 --2- 172.21.15.133:0/2983306097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca860cb0 0x5556ca878a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.923+0000 7f24dedf6740 1 --2- 172.21.15.133:0/2983306097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca856410 0x5556ca85e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.923+0000 7f24dedf6740 1 --2- 172.21.15.133:0/2983306097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca855af0 0x5556ca85c480 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.923+0000 7f24dedf6740 1 -- 172.21.15.133:0/2983306097 >> 172.21.15.133:0/2983306097 conn(0x5556ca852060 msgr2=0x5556ca85fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.923+0000 7f24dedf6740 1 -- 172.21.15.133:0/2983306097 shutdown_connections 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.923+0000 7f24dedf6740 1 -- 172.21.15.133:0/2983306097 wait complete. 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.924+0000 7f24dedf6740 1 -- start start 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.924+0000 7f24dedf6740 1 Processor -- start 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.924+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca855af0 0x5556ca8690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.924+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca856410 0x5556ca8695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.924+0000 7f24dedf6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca864540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.924+0000 7f24dedf6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24d407bbd0 con 0x5556ca860cb0 2024-09-11T23:47:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.924+0000 7f24dedf6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24d407b500 con 0x5556ca856410 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.924+0000 7f24dedf6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24d40787f0 con 0x5556ca855af0 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.924+0000 7f24dedf6740 10 client.0 _ll_register_callbacks cb 0x7f24cc005f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24d9dfa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca855af0 0x5556ca8690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24d9dfa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca855af0 0x5556ca8690b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:35250/0 (socket says 172.21.15.133:35250) 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24da5fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca864540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24d9dfa640 1 -- 172.21.15.133:0/3603546883 learned_addr learned my addr 172.21.15.133:0/3603546883 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24da5fb640 1 -- 172.21.15.133:0/3603546883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca855af0 msgr2=0x5556ca8690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24da5fb640 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca855af0 0x5556ca8690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24d95f9640 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca856410 0x5556ca8695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24da5fb640 1 -- 172.21.15.133:0/3603546883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca856410 msgr2=0x5556ca8695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24da5fb640 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca856410 0x5556ca8695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24da5fb640 1 -- 172.21.15.133:0/3603546883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ca864d40 con 0x5556ca860cb0 2024-09-11T23:47:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24da5fb640 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca864540 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0ba870 tx=0x7f24cc02a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14e2b05e09f96dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.925+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3603546883 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc004da0 con 0x5556ca860cb0 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3603546883 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f24cc0bba60 con 0x5556ca860cb0 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24dedf6740 1 -- 172.21.15.133:0/3603546883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c402f070 con 0x5556ca860cb0 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3603546883 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24cc0b8860 con 0x5556ca860cb0 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24dedf6740 10 client.18168 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24dedf6740 1 -- 172.21.15.133:0/3603546883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556ca864d40 con 0x5556ca860cb0 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3603546883 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c402f070 con 0x5556ca860cb0 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3603546883 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f24cc0a5b90 con 0x5556ca860cb0 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24dedf6740 10 client.18168 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24dedf6740 10 client.18168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24dedf6740 20 client.18168 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ca8db9f0 0x5556ca8ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24dedf6740 4 client.18168 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:47:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.926+0000 7f24dedf6740 1 -- 172.21.15.133:0/3603546883 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556ca7c1d70 con 0x5556ca8db9f0 2024-09-11T23:47:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.927+0000 7f24d9dfa640 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ca8db9f0 0x5556ca8ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.927+0000 7f24bd7fa640 10 client.18168 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:27.927+0000 7f24d9dfa640 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ca8db9f0 0x5556ca8ddde0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ea4a741dd2fe2a2 server_cookie=d1d87d16f09a5563 in_seq=0 out_seq=0 2024-09-11T23:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.344+0000 7f24bd7fa640 1 -- 172.21.15.133:0/3603546883 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5556ca7c1d70 con 0x5556ca8db9f0 2024-09-11T23:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.344+0000 7f24bd7fa640 10 client.18168 handle_command_reply: tid=0 2024-09-11T23:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.344+0000 7f24dedf6740 1 client.18168 shutdown 2024-09-11T23:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.344+0000 7f24dedf6740 1 -- 172.21.15.133:0/3603546883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 msgr2=0x5556ca864540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.344+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca864540 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0ba870 tx=0x7f24cc02a6e0 comp rx=0 tx=0).stop 2024-09-11T23:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.345+0000 7f24dedf6740 1 -- 172.21.15.133:0/3603546883 shutdown_connections 2024-09-11T23:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.345+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5556ca8db9f0 0x5556ca8ddde0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.345+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556ca860cb0 0x5556ca864540 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.345+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556ca856410 0x5556ca8695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.345+0000 7f24dedf6740 1 --2- 172.21.15.133:0/3603546883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5556ca855af0 0x5556ca8690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.345+0000 7f24dedf6740 1 -- 172.21.15.133:0/3603546883 >> 172.21.15.133:0/3603546883 conn(0x5556ca852060 msgr2=0x5556ca857d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.345+0000 7f24bd7fa640 0 client.18168 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.345+0000 7f24dedf6740 1 -- 172.21.15.133:0/3603546883 shutdown_connections 2024-09-11T23:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.346+0000 7f24dedf6740 1 -- 172.21.15.133:0/3603546883 wait complete. 2024-09-11T23:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.346+0000 7f24dedf6740 20 client.18168 trim_cache size 0 max 16384 2024-09-11T23:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.347+0000 7f24dc05e640 1 -- 172.21.15.133:0/2684706756 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f24c00a2ce0 msgr2=0x7f24c00a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.347+0000 7f24dc05e640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f24c00a2ce0 0x7f24c00a51a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009510 tx=0x7f24c4009460 comp rx=0 tx=0).stop 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.347+0000 7f24dc05e640 1 -- 172.21.15.133:0/2684706756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24d4113420 msgr2=0x7f24d41a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.347+0000 7f24dc05e640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24d4113420 0x7f24d41a09b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f24cc034040 tx=0x7f24cc0305d0 comp rx=0 tx=0).stop 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.347+0000 7f24dc05e640 1 -- 172.21.15.133:0/2684706756 shutdown_connections 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.347+0000 7f24dc05e640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f24c00a2ce0 0x7f24c00a51a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.347+0000 7f24dc05e640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24d4113420 0x7f24d41a09b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.347+0000 7f24dc05e640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d4109d80 0x7f24d41a6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.347+0000 7f24dc05e640 1 --2- 172.21.15.133:0/2684706756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24d4069820 0x7f24d41a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.347+0000 7f24dc05e640 1 -- 172.21.15.133:0/2684706756 >> 172.21.15.133:0/2684706756 conn(0x7f24d406d210 msgr2=0x7f24d4072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.348+0000 7f24dc05e640 1 -- 172.21.15.133:0/2684706756 shutdown_connections 2024-09-11T23:47:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:28.348+0000 7f24dc05e640 1 -- 172.21.15.133:0/2684706756 wait complete. 2024-09-11T23:47:29.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:28 smithi138 ceph-mon[35142]: pgmap v334: 97 pgs: 97 active+clean; 62 GiB data, 260 GiB used, 812 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 296 MiB/s wr, 1.49k op/s 2024-09-11T23:47:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:28 smithi133 ceph-mon[31430]: pgmap v334: 97 pgs: 97 active+clean; 62 GiB data, 260 GiB used, 812 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 296 MiB/s wr, 1.49k op/s 2024-09-11T23:47:29.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:28 smithi178 ceph-mon[35882]: pgmap v334: 97 pgs: 97 active+clean; 62 GiB data, 260 GiB used, 812 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 296 MiB/s wr, 1.49k op/s 2024-09-11T23:47:31.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:30 smithi138 ceph-mon[35142]: pgmap v335: 97 pgs: 97 active+clean; 64 GiB data, 266 GiB used, 806 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 463 MiB/s wr, 2.54k op/s 2024-09-11T23:47:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:30 smithi133 ceph-mon[31430]: pgmap v335: 97 pgs: 97 active+clean; 64 GiB data, 266 GiB used, 806 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 463 MiB/s wr, 2.54k op/s 2024-09-11T23:47:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:30 smithi178 ceph-mon[35882]: pgmap v335: 97 pgs: 97 active+clean; 64 GiB data, 266 GiB used, 806 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 463 MiB/s wr, 2.54k op/s 2024-09-11T23:47:33.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:32 smithi178 ceph-mon[35882]: pgmap v336: 97 pgs: 97 active+clean; 64 GiB data, 268 GiB used, 805 GiB / 1.0 TiB avail; 409 KiB/s rd, 336 MiB/s wr, 1.96k op/s 2024-09-11T23:47:33.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:32 smithi138 ceph-mon[35142]: pgmap v336: 97 pgs: 97 active+clean; 64 GiB data, 268 GiB used, 805 GiB / 1.0 TiB avail; 409 KiB/s rd, 336 MiB/s wr, 1.96k op/s 2024-09-11T23:47:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:32 smithi133 ceph-mon[31430]: pgmap v336: 97 pgs: 97 active+clean; 64 GiB data, 268 GiB used, 805 GiB / 1.0 TiB avail; 409 KiB/s rd, 336 MiB/s wr, 1.96k op/s 2024-09-11T23:47:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:34 smithi138 ceph-mon[35142]: pgmap v337: 97 pgs: 97 active+clean; 65 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 414 KiB/s rd, 341 MiB/s wr, 2.20k op/s 2024-09-11T23:47:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:34 smithi133 ceph-mon[31430]: pgmap v337: 97 pgs: 97 active+clean; 65 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 414 KiB/s rd, 341 MiB/s wr, 2.20k op/s 2024-09-11T23:47:35.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:34 smithi178 ceph-mon[35882]: pgmap v337: 97 pgs: 97 active+clean; 65 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 414 KiB/s rd, 341 MiB/s wr, 2.20k op/s 2024-09-11T23:47:35.635 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:47:35.906 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:36.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.350+0000 7f35ecfd7640 1 -- 172.21.15.133:0/1282909818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e8100e20 msgr2=0x7f35e8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:36.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.350+0000 7f35ecfd7640 1 --2- 172.21.15.133:0/1282909818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e8100e20 0x7f35e8101200 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009900 tx=0x7f35d802a480 comp rx=0 tx=0).stop 2024-09-11T23:47:36.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.350+0000 7f35ecfd7640 1 -- 172.21.15.133:0/1282909818 shutdown_connections 2024-09-11T23:47:36.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.350+0000 7f35ecfd7640 1 --2- 172.21.15.133:0/1282909818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e8102ef0 0x7f35e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:36.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.350+0000 7f35ecfd7640 1 --2- 172.21.15.133:0/1282909818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e8102570 0x7f35e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:36.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.350+0000 7f35ecfd7640 1 --2- 172.21.15.133:0/1282909818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e8100e20 0x7f35e8101200 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:36.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.350+0000 7f35ecfd7640 1 -- 172.21.15.133:0/1282909818 >> 172.21.15.133:0/1282909818 conn(0x7f35e80fc7d0 msgr2=0x7f35e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:36.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.350+0000 7f35ecfd7640 1 -- 172.21.15.133:0/1282909818 shutdown_connections 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.350+0000 7f35ecfd7640 1 -- 172.21.15.133:0/1282909818 wait complete. 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35ecfd7640 1 Processor -- start 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35ecfd7640 1 -- start start 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35ecfd7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e8100e20 0x7f35e8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35ecfd7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e8102570 0x7f35e8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35ecfd7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e8102ef0 0x7f35e819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35ecfd7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8110fa0 con 0x7f35e8102570 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35ecfd7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8110e20 con 0x7f35e8100e20 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35ecfd7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111120 con 0x7f35e8102ef0 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35e659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e8100e20 0x7f35e8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35dffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e8102570 0x7f35e8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35dffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e8102570 0x7f35e8198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50602/0 (socket says 172.21.15.133:50602) 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35e659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e8100e20 0x7f35e8198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52108/0 (socket says 172.21.15.133:52108) 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35dffff640 1 -- 172.21.15.133:0/2101445098 learned_addr learned my addr 172.21.15.133:0/2101445098 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.351+0000 7f35e6d9d640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e8102ef0 0x7f35e819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35dffff640 1 -- 172.21.15.133:0/2101445098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e8102ef0 msgr2=0x7f35e819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35dffff640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e8102ef0 0x7f35e819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35dffff640 1 -- 172.21.15.133:0/2101445098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e8100e20 msgr2=0x7f35e8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35dffff640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e8100e20 0x7f35e8198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35dffff640 1 -- 172.21.15.133:0/2101445098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e819d540 con 0x7f35e8102570 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35e659c640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e8100e20 0x7f35e8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35dffff640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e8102570 0x7f35e8198ad0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f35d000c920 tx=0x7f35d000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35df7fe640 1 -- 172.21.15.133:0/2101445098 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0007c70 con 0x7f35e8102570 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35df7fe640 1 -- 172.21.15.133:0/2101445098 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d0007e10 con 0x7f35e8102570 2024-09-11T23:47:36.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35ecfd7640 1 -- 172.21.15.133:0/2101445098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e819d830 con 0x7f35e8102570 2024-09-11T23:47:36.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35df7fe640 1 -- 172.21.15.133:0/2101445098 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0018890 con 0x7f35e8102570 2024-09-11T23:47:36.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.352+0000 7f35ecfd7640 1 -- 172.21.15.133:0/2101445098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e80fcbb0 con 0x7f35e8102570 2024-09-11T23:47:36.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.353+0000 7f35ecfd7640 1 -- 172.21.15.133:0/2101445098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e8101200 con 0x7f35e8102570 2024-09-11T23:47:36.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.354+0000 7f35df7fe640 1 -- 172.21.15.133:0/2101445098 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f35d0018a50 con 0x7f35e8102570 2024-09-11T23:47:36.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.355+0000 7f35df7fe640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35d40a2d70 0x7f35d40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:36.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.355+0000 7f35e659c640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35d40a2d70 0x7f35d40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:36.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.355+0000 7f35df7fe640 1 -- 172.21.15.133:0/2101445098 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f35d00cf440 con 0x7f35e8102570 2024-09-11T23:47:36.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.355+0000 7f35e659c640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35d40a2d70 0x7f35d40a5230 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f35d8000c00 tx=0x7f35d80058d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:36.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.359+0000 7f35df7fe640 1 -- 172.21.15.133:0/2101445098 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35d0014030 con 0x7f35e8102570 2024-09-11T23:47:36.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.568+0000 7f35ecfd7640 1 -- 172.21.15.133:0/2101445098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35e810a0a0 con 0x7f35e8102570 2024-09-11T23:47:36.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.569+0000 7f35df7fe640 1 -- 172.21.15.133:0/2101445098 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7f35d009c0a0 con 0x7f35e8102570 2024-09-11T23:47:36.569 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:47:36.569 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:47:36.569 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:47:36.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.570+0000 7f35ecfd7640 1 -- 172.21.15.133:0/2101445098 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35d40a2d70 msgr2=0x7f35d40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:36.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.570+0000 7f35ecfd7640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35d40a2d70 0x7f35d40a5230 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f35d8000c00 tx=0x7f35d80058d0 comp rx=0 tx=0).stop 2024-09-11T23:47:36.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.571+0000 7f35ecfd7640 1 -- 172.21.15.133:0/2101445098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e8102570 msgr2=0x7f35e8198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:36.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.571+0000 7f35ecfd7640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e8102570 0x7f35e8198ad0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f35d000c920 tx=0x7f35d000cde0 comp rx=0 tx=0).stop 2024-09-11T23:47:36.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.571+0000 7f35ecfd7640 1 -- 172.21.15.133:0/2101445098 shutdown_connections 2024-09-11T23:47:36.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.571+0000 7f35ecfd7640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f35d40a2d70 0x7f35d40a5230 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:36.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.571+0000 7f35ecfd7640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e8102ef0 0x7f35e819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:36.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.571+0000 7f35ecfd7640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e8102570 0x7f35e8198ad0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:36.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.571+0000 7f35ecfd7640 1 --2- 172.21.15.133:0/2101445098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e8100e20 0x7f35e8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:36.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.571+0000 7f35ecfd7640 1 -- 172.21.15.133:0/2101445098 >> 172.21.15.133:0/2101445098 conn(0x7f35e80fc7d0 msgr2=0x7f35e806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:36.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.571+0000 7f35ecfd7640 1 -- 172.21.15.133:0/2101445098 shutdown_connections 2024-09-11T23:47:36.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:36.571+0000 7f35ecfd7640 1 -- 172.21.15.133:0/2101445098 wait complete. 2024-09-11T23:47:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:36 smithi138 ceph-mon[35142]: pgmap v338: 97 pgs: 97 active+clean; 65 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 12 KiB/s rd, 334 MiB/s wr, 2.25k op/s 2024-09-11T23:47:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:36 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2101445098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:37.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:36 smithi133 ceph-mon[31430]: pgmap v338: 97 pgs: 97 active+clean; 65 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 12 KiB/s rd, 334 MiB/s wr, 2.25k op/s 2024-09-11T23:47:37.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:36 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2101445098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:36 smithi178 ceph-mon[35882]: pgmap v338: 97 pgs: 97 active+clean; 65 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 12 KiB/s rd, 334 MiB/s wr, 2.25k op/s 2024-09-11T23:47:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:36 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2101445098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:37.394 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --set-id 3c36fe3 --release --await' 2024-09-11T23:47:37.395 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --set-id 3c36fe3 --release --await 2024-09-11T23:47:37.646 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:38.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.082+0000 7f58fca7b640 1 -- 172.21.15.133:0/389422220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58f8102ef0 msgr2=0x7f58f810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:38.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.082+0000 7f58fca7b640 1 --2- 172.21.15.133:0/389422220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58f8102ef0 0x7f58f810da40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009ca0 tx=0x7f58f002a950 comp rx=0 tx=0).stop 2024-09-11T23:47:38.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.082+0000 7f58fca7b640 1 -- 172.21.15.133:0/389422220 shutdown_connections 2024-09-11T23:47:38.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.082+0000 7f58fca7b640 1 --2- 172.21.15.133:0/389422220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58f8102ef0 0x7f58f810da40 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.082+0000 7f58fca7b640 1 --2- 172.21.15.133:0/389422220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8102570 0x7f58f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.082+0000 7f58fca7b640 1 --2- 172.21.15.133:0/389422220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58f8100e20 0x7f58f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.082+0000 7f58fca7b640 1 -- 172.21.15.133:0/389422220 >> 172.21.15.133:0/389422220 conn(0x7f58f80fc7d0 msgr2=0x7f58f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.082+0000 7f58fca7b640 1 -- 172.21.15.133:0/389422220 shutdown_connections 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.082+0000 7f58fca7b640 1 -- 172.21.15.133:0/389422220 wait complete. 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.083+0000 7f58fca7b640 1 Processor -- start 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.083+0000 7f58fca7b640 1 -- start start 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.083+0000 7f58fca7b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58f8100e20 0x7f58f8107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.083+0000 7f58fca7b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8102570 0x7f58f8108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.083+0000 7f58fca7b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58f8102ef0 0x7f58f8108560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.083+0000 7f58fca7b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f58f8110fa0 con 0x7f58f8100e20 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.083+0000 7f58fca7b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58f8110e20 con 0x7f58f8102ef0 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.083+0000 7f58fca7b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58f8111120 con 0x7f58f8102570 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58f659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58f8100e20 0x7f58f8107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58f659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58f8100e20 0x7f58f8107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50614/0 (socket says 172.21.15.133:50614) 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58f659c640 1 -- 172.21.15.133:0/1077595428 learned_addr learned my addr 172.21.15.133:0/1077595428 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58f6d9d640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58f8102ef0 0x7f58f8108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58effff640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8102570 0x7f58f8108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58f659c640 1 -- 172.21.15.133:0/1077595428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8102570 msgr2=0x7f58f8108020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58f659c640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8102570 0x7f58f8108020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58f659c640 1 -- 172.21.15.133:0/1077595428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58f8102ef0 msgr2=0x7f58f8108560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58f659c640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58f8102ef0 0x7f58f8108560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58f659c640 1 -- 172.21.15.133:0/1077595428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f81a69b0 con 0x7f58f8100e20 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58f659c640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58f8100e20 0x7f58f8107ae0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f58e800d950 tx=0x7f58e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58ef7fe640 1 -- 172.21.15.133:0/1077595428 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8017030 con 0x7f58f8100e20 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58fca7b640 1 -- 172.21.15.133:0/1077595428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f81a6ca0 con 0x7f58f8100e20 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58ef7fe640 1 -- 172.21.15.133:0/1077595428 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e8004020 con 0x7f58f8100e20 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58ef7fe640 1 -- 172.21.15.133:0/1077595428 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e80047a0 con 0x7f58f8100e20 2024-09-11T23:47:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.084+0000 7f58fca7b640 1 -- 172.21.15.133:0/1077595428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f81a71d0 con 0x7f58f8100e20 2024-09-11T23:47:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.087+0000 7f58ef7fe640 1 -- 172.21.15.133:0/1077595428 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f58e80041d0 con 0x7f58f8100e20 2024-09-11T23:47:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.087+0000 7f58fee54740 1 Processor -- start 2024-09-11T23:47:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.087+0000 7f58fee54740 1 -- start start 2024-09-11T23:47:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.087+0000 7f58fee54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b060 0x555a64c4b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.087+0000 7f58fee54740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b980 0x555a64c55d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.087+0000 7f58fee54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c58630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.087+0000 7f58fee54740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555a64bb0620 con 0x555a64c4b980 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.087+0000 7f58fee54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a64b57e50 con 0x555a64c56240 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.087+0000 7f58fee54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555a64bac5e0 con 0x555a64c4b060 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.087+0000 7f58ef7fe640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58e40a2cf0 0x7f58e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b980 0x555a64c55d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f6d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b060 0x555a64c4b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b980 0x555a64c55d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50628/0 (socket says 172.21.15.133:50628) 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f659c640 1 -- 172.21.15.133:0/1394099817 learned_addr learned my addr 172.21.15.133:0/1394099817 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f6d9d640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58e40a2cf0 0x7f58e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58effff640 1 --2- 172.21.15.133:0/1394099817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c58630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58ef7fe640 1 -- 172.21.15.133:0/1077595428 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f58e80cedf0 con 0x7f58f8100e20 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f659c640 1 -- 172.21.15.133:0/1394099817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b060 msgr2=0x555a64c4b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:38.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f659c640 1 --2- 172.21.15.133:0/1394099817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b060 0x555a64c4b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f659c640 1 -- 172.21.15.133:0/1394099817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 msgr2=0x555a64c58630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f659c640 1 --2- 172.21.15.133:0/1394099817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c58630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f659c640 1 -- 172.21.15.133:0/1394099817 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a64b48d40 con 0x555a64c4b980 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f6d9d640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58e40a2cf0 0x7f58e40a51b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009a60 tx=0x7f58f0009910 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58f659c640 1 --2- 172.21.15.133:0/1394099817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b980 0x555a64c55d00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f58e80cc4f0 tx=0x7f58e8096fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f7e9dbee573cce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58ecff9640 1 -- 172.21.15.133:0/1394099817 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8084890 con 0x555a64c4b980 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58ecff9640 1 -- 172.21.15.133:0/1394099817 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e80bb050 con 0x555a64c4b980 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.088+0000 7f58ecff9640 1 -- 172.21.15.133:0/1394099817 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e80979c0 con 0x555a64c4b980 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.089+0000 7f58fee54740 1 -- 172.21.15.133:0/1394099817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b980 msgr2=0x555a64c55d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.089+0000 7f58fee54740 1 --2- 172.21.15.133:0/1394099817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b980 0x555a64c55d00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f58e80cc4f0 tx=0x7f58e8096fd0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.089+0000 7f58fee54740 1 -- 172.21.15.133:0/1394099817 shutdown_connections 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.089+0000 7f58fee54740 1 --2- 172.21.15.133:0/1394099817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c58630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.089+0000 7f58fee54740 1 --2- 172.21.15.133:0/1394099817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b980 0x555a64c55d00 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.089+0000 7f58fee54740 1 --2- 172.21.15.133:0/1394099817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b060 0x555a64c4b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.089+0000 7f58fee54740 1 -- 172.21.15.133:0/1394099817 >> 172.21.15.133:0/1394099817 conn(0x555a64c475f0 msgr2=0x555a64c479d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.089+0000 7f58fee54740 1 -- 172.21.15.133:0/1394099817 shutdown_connections 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.089+0000 7f58fee54740 1 -- 172.21.15.133:0/1394099817 wait complete. 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58fee54740 1 -- start start 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58fee54740 1 Processor -- start 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58fee54740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c53150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58fee54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b980 0x555a64c53690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58fee54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c53bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58fee54740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555a64c58e10 con 0x555a64c4b060 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58fee54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a64b38f10 con 0x555a64c56240 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58fee54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555a64c58f90 con 0x555a64c4b980 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58fee54740 10 client.0 _ll_register_callbacks cb 0x555a64b5cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58f659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c53150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58f659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c53150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50644/0 (socket says 172.21.15.133:50644) 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58f659c640 1 -- 172.21.15.133:0/3696161551 learned_addr learned my addr 172.21.15.133:0/3696161551 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58f6d9d640 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c53bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.090+0000 7f58effff640 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b980 0x555a64c53690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58f659c640 1 -- 172.21.15.133:0/3696161551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b980 msgr2=0x555a64c53690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58f659c640 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b980 0x555a64c53690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58f659c640 1 -- 172.21.15.133:0/3696161551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 msgr2=0x555a64c53bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58f659c640 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c53bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58f659c640 1 -- 172.21.15.133:0/3696161551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a648bf580 con 0x555a64c4b060 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58f659c640 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c53150 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f58e809b2e0 tx=0x7f58e80ccab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8fd9817de2d4f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58ddffb640 1 -- 172.21.15.133:0/3696161551 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e809b350 con 0x555a64c4b060 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58ddffb640 1 -- 172.21.15.133:0/3696161551 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f58e80bb200 con 0x555a64c4b060 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58fee54740 1 -- 172.21.15.133:0/3696161551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e80bb050 con 0x555a64c4b060 2024-09-11T23:47:38.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58ddffb640 1 -- 172.21.15.133:0/3696161551 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58e809f700 con 0x555a64c4b060 2024-09-11T23:47:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58fee54740 10 client.18186 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58fee54740 1 -- 172.21.15.133:0/3696161551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a64989960 con 0x555a64c4b060 2024-09-11T23:47:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58ddffb640 1 -- 172.21.15.133:0/3696161551 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e80bb050 con 0x555a64c4b060 2024-09-11T23:47:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.091+0000 7f58ddffb640 1 -- 172.21.15.133:0/3696161551 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f58e809f170 con 0x555a64c4b060 2024-09-11T23:47:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.092+0000 7f58fee54740 10 client.18186 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.092+0000 7f58fee54740 10 client.18186 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:47:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.092+0000 7f58fee54740 20 client.18186 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.092+0000 7f58fee54740 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x555a64c65ca0 0x555a64c68090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.092+0000 7f58fee54740 4 client.18186 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.092+0000 7f58fee54740 1 -- 172.21.15.133:0/3696161551 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a64bbdb90 con 0x555a64c65ca0 2024-09-11T23:47:38.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.092+0000 7f58effff640 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x555a64c65ca0 0x555a64c68090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.092+0000 7f58effff640 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x555a64c65ca0 0x555a64c68090 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb91b51347ca1fba server_cookie=22245972784818eb in_seq=0 out_seq=0 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.092+0000 7f58ddffb640 10 client.18186 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58ddffb640 1 -- 172.21.15.133:0/3696161551 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a64bbdb90 con 0x555a64c65ca0 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58ddffb640 10 client.18186 handle_command_reply: tid=0 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 1 client.18186 shutdown 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 1 -- 172.21.15.133:0/3696161551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 msgr2=0x555a64c53150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c53150 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f58e809b2e0 tx=0x7f58e80ccab0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 1 -- 172.21.15.133:0/3696161551 shutdown_connections 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x555a64c65ca0 0x555a64c68090 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58ddffb640 0 client.18186 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c53bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b980 0x555a64c53690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 1 --2- 172.21.15.133:0/3696161551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c53150 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 1 -- 172.21.15.133:0/3696161551 >> 172.21.15.133:0/3696161551 conn(0x555a64c475f0 msgr2=0x555a64c4d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 1 -- 172.21.15.133:0/3696161551 shutdown_connections 2024-09-11T23:47:38.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 1 -- 172.21.15.133:0/3696161551 wait complete. 2024-09-11T23:47:38.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.093+0000 7f58fee54740 20 client.18186 trim_cache size 0 max 16384 2024-09-11T23:47:38.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58fee54740 1 Processor -- start 2024-09-11T23:47:38.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58fee54740 1 -- start start 2024-09-11T23:47:38.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58fee54740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c4dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58fee54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b980 0x555a64c503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58fee54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c508f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58fee54740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81afa10 con 0x555a64c4b060 2024-09-11T23:47:38.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58fee54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81b1550 con 0x555a64c56240 2024-09-11T23:47:38.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58fee54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81b00e0 con 0x555a64c4b980 2024-09-11T23:47:38.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58f659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c4dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58f659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c4dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50656/0 (socket says 172.21.15.133:50656) 2024-09-11T23:47:38.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58f659c640 1 -- 172.21.15.133:0/2254372987 learned_addr learned my addr 172.21.15.133:0/2254372987 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:38.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58effff640 1 --2- 172.21.15.133:0/2254372987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b980 0x555a64c503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58f6d9d640 1 --2- 172.21.15.133:0/2254372987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58f659c640 1 -- 172.21.15.133:0/2254372987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b980 msgr2=0x555a64c503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:38.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58f659c640 1 --2- 172.21.15.133:0/2254372987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b980 0x555a64c503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58f659c640 1 -- 172.21.15.133:0/2254372987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 msgr2=0x555a64c508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:38.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58f659c640 1 --2- 172.21.15.133:0/2254372987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58f659c640 1 -- 172.21.15.133:0/2254372987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a64c52100 con 0x555a64c4b060 2024-09-11T23:47:38.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.111+0000 7f58f659c640 1 --2- 172.21.15.133:0/2254372987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c4dac0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f58e809b050 tx=0x7f58e80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786be96e07a68790 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:38.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58dffff640 1 -- 172.21.15.133:0/2254372987 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e80975f0 con 0x555a64c4b060 2024-09-11T23:47:38.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58dffff640 1 -- 172.21.15.133:0/2254372987 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e80a4390 con 0x555a64c4b060 2024-09-11T23:47:38.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58dffff640 1 -- 172.21.15.133:0/2254372987 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e809a7a0 con 0x555a64c4b060 2024-09-11T23:47:38.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58fee54740 1 -- 172.21.15.133:0/2254372987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 msgr2=0x555a64c4dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:38.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58fee54740 1 --2- 172.21.15.133:0/2254372987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c4dac0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f58e809b050 tx=0x7f58e80057b0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58fee54740 1 -- 172.21.15.133:0/2254372987 shutdown_connections 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58fee54740 1 --2- 172.21.15.133:0/2254372987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c56240 0x555a64c508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58fee54740 1 --2- 172.21.15.133:0/2254372987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b980 0x555a64c503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58fee54740 1 --2- 172.21.15.133:0/2254372987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c4b060 0x555a64c4dac0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58fee54740 1 -- 172.21.15.133:0/2254372987 >> 172.21.15.133:0/2254372987 conn(0x555a64c475f0 msgr2=0x555a64c549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58fee54740 1 -- 172.21.15.133:0/2254372987 shutdown_connections 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.112+0000 7f58fee54740 1 -- 172.21.15.133:0/2254372987 wait complete. 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58fee54740 1 -- start start 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58fee54740 1 Processor -- start 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58fee54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b060 0x555a64c6b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58fee54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c4b980 0x555a64c6b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58fee54740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c56240 0x555a64c696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58fee54740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81b0e80 con 0x555a64c56240 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58fee54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81b07b0 con 0x555a64c4b980 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58fee54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81ada10 con 0x555a64c4b060 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58fee54740 10 client.0 _ll_register_callbacks cb 0x7f58e809b1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58f6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c56240 0x555a64c696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58f6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c56240 0x555a64c696c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50662/0 (socket says 172.21.15.133:50662) 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58f6d9d640 1 -- 172.21.15.133:0/1457505340 learned_addr learned my addr 172.21.15.133:0/1457505340 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58f659c640 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b060 0x555a64c6b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.113+0000 7f58effff640 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c4b980 0x555a64c6b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58f6d9d640 1 -- 172.21.15.133:0/1457505340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b060 msgr2=0x555a64c6b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58f6d9d640 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b060 0x555a64c6b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58f6d9d640 1 -- 172.21.15.133:0/1457505340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c4b980 msgr2=0x555a64c6b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58f6d9d640 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c4b980 0x555a64c6b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58f6d9d640 1 -- 172.21.15.133:0/1457505340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a64c4b7b0 con 0x555a64c56240 2024-09-11T23:47:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58f6d9d640 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c56240 0x555a64c696c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f58f0006010 tx=0x7f58f0048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db789e2429e8c2c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58ddffb640 1 -- 172.21.15.133:0/1457505340 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0034040 con 0x555a64c56240 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58ddffb640 1 -- 172.21.15.133:0/1457505340 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f58f004b450 con 0x555a64c56240 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58fee54740 1 -- 172.21.15.133:0/1457505340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e80a4390 con 0x555a64c56240 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58ddffb640 1 -- 172.21.15.133:0/1457505340 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58f0003e40 con 0x555a64c56240 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58fee54740 10 client.18192 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58fee54740 1 -- 172.21.15.133:0/1457505340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a64cd0f60 con 0x555a64c56240 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.114+0000 7f58ddffb640 1 -- 172.21.15.133:0/1457505340 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e80a4390 con 0x555a64c56240 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.115+0000 7f58ddffb640 1 -- 172.21.15.133:0/1457505340 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f58f005e4d0 con 0x555a64c56240 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.115+0000 7f58fee54740 10 client.18192 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.115+0000 7f58fee54740 10 client.18192 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.115+0000 7f58fee54740 20 client.18192 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.115+0000 7f58fee54740 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x555a64cd16b0 0x555a64cd3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:38.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.115+0000 7f58fee54740 4 client.18192 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3c36fe3", "release": true, "await": true}] 2024-09-11T23:47:38.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.115+0000 7f58fee54740 1 -- 172.21.15.133:0/1457505340 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3c36fe3", "release": true, "await": true}) -- 0x555a64bb80a0 con 0x555a64cd16b0 2024-09-11T23:47:38.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.115+0000 7f58f659c640 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x555a64cd16b0 0x555a64cd3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:38.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.115+0000 7f58ddffb640 10 client.18192 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:38.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.115+0000 7f58f659c640 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x555a64cd16b0 0x555a64cd3ad0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a193ad2f467639a server_cookie=b8462e0b2ddbfc09 in_seq=0 out_seq=0 2024-09-11T23:47:38.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.119+0000 7f58ddffb640 1 -- 172.21.15.133:0/1457505340 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x555a64bb80a0 con 0x555a64cd16b0 2024-09-11T23:47:38.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.119+0000 7f58ddffb640 10 client.18192 handle_command_reply: tid=0 2024-09-11T23:47:38.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.119+0000 7f58fee54740 1 client.18192 shutdown 2024-09-11T23:47:38.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.119+0000 7f58fee54740 1 -- 172.21.15.133:0/1457505340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c56240 msgr2=0x555a64c696c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:38.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.119+0000 7f58fee54740 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c56240 0x555a64c696c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f58f0006010 tx=0x7f58f0048000 comp rx=0 tx=0).stop 2024-09-11T23:47:38.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.120+0000 7f58fee54740 1 -- 172.21.15.133:0/1457505340 shutdown_connections 2024-09-11T23:47:38.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.120+0000 7f58fee54740 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x555a64cd16b0 0x555a64cd3ad0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.120+0000 7f58ddffb640 0 client.18192 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:38.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.120+0000 7f58fee54740 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a64c56240 0x555a64c696c0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.120+0000 7f58fee54740 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a64c4b980 0x555a64c6b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.120+0000 7f58fee54740 1 --2- 172.21.15.133:0/1457505340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a64c4b060 0x555a64c6b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.120+0000 7f58fee54740 1 -- 172.21.15.133:0/1457505340 >> 172.21.15.133:0/1457505340 conn(0x555a64c475f0 msgr2=0x555a64c61210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:38.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.120+0000 7f58fee54740 1 -- 172.21.15.133:0/1457505340 shutdown_connections 2024-09-11T23:47:38.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.120+0000 7f58fee54740 1 -- 172.21.15.133:0/1457505340 wait complete. 2024-09-11T23:47:38.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.120+0000 7f58fee54740 20 client.18192 trim_cache size 0 max 16384 2024-09-11T23:47:38.122 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:47:38.122 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 30, 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 44, 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": { 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "3c36fe3": { 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 44, 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "age_ref": 0.0, 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "timeout": 90.0, 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "expiration": 142.6, 2024-09-11T23:47:38.123 INFO:teuthology.orchestra.run.smithi133.stdout: "members": { 2024-09-11T23:47:38.124 INFO:teuthology.orchestra.run.smithi133.stdout: "file:/": { 2024-09-11T23:47:38.124 INFO:teuthology.orchestra.run.smithi133.stdout: "excluded": false, 2024-09-11T23:47:38.124 INFO:teuthology.orchestra.run.smithi133.stdout: "state": { 2024-09-11T23:47:38.124 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RELEASED", 2024-09-11T23:47:38.124 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0 2024-09-11T23:47:38.124 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:38.124 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:38.124 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:38.124 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:38.124 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:47:38.124 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:47:38.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.122+0000 7f58fca7b640 1 -- 172.21.15.133:0/1077595428 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58e40a2cf0 msgr2=0x7f58e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:38.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.122+0000 7f58fca7b640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58e40a2cf0 0x7f58e40a51b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009a60 tx=0x7f58f0009910 comp rx=0 tx=0).stop 2024-09-11T23:47:38.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.122+0000 7f58fca7b640 1 -- 172.21.15.133:0/1077595428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58f8100e20 msgr2=0x7f58f8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:38.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.122+0000 7f58fca7b640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58f8100e20 0x7f58f8107ae0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f58e800d950 tx=0x7f58e800de10 comp rx=0 tx=0).stop 2024-09-11T23:47:38.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.123+0000 7f58fca7b640 1 -- 172.21.15.133:0/1077595428 shutdown_connections 2024-09-11T23:47:38.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.123+0000 7f58fca7b640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f58e40a2cf0 0x7f58e40a51b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.123+0000 7f58fca7b640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58f8102ef0 0x7f58f8108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.123+0000 7f58fca7b640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8102570 0x7f58f8108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.123+0000 7f58fca7b640 1 --2- 172.21.15.133:0/1077595428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58f8100e20 0x7f58f8107ae0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:38.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.123+0000 7f58fca7b640 1 -- 172.21.15.133:0/1077595428 >> 172.21.15.133:0/1077595428 conn(0x7f58f80fc7d0 msgr2=0x7f58f81037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:38.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.124+0000 7f58fca7b640 1 -- 172.21.15.133:0/1077595428 shutdown_connections 2024-09-11T23:47:38.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:38.124+0000 7f58fca7b640 1 -- 172.21.15.133:0/1077595428 wait complete. 2024-09-11T23:47:39.088 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3c36fe3', total seconds elapsed: 26.0 2024-09-11T23:47:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:38 smithi138 ceph-mon[35142]: pgmap v339: 97 pgs: 97 active+clean; 65 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 12 KiB/s rd, 330 MiB/s wr, 2.25k op/s 2024-09-11T23:47:39.212 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:38 smithi133 ceph-mon[31430]: pgmap v339: 97 pgs: 97 active+clean; 65 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 12 KiB/s rd, 330 MiB/s wr, 2.25k op/s 2024-09-11T23:47:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:38 smithi178 ceph-mon[35882]: pgmap v339: 97 pgs: 97 active+clean; 65 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 12 KiB/s rd, 330 MiB/s wr, 2.25k op/s 2024-09-11T23:47:39.477 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:47:40.117 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:40.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:40 smithi138 ceph-mon[35142]: pgmap v340: 97 pgs: 97 active+clean; 65 GiB data, 254 GiB used, 819 GiB / 1.0 TiB avail; 18 KiB/s rd, 310 MiB/s wr, 2.15k op/s 2024-09-11T23:47:40.473 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:40 smithi133 ceph-mon[31430]: pgmap v340: 97 pgs: 97 active+clean; 65 GiB data, 254 GiB used, 819 GiB / 1.0 TiB avail; 18 KiB/s rd, 310 MiB/s wr, 2.15k op/s 2024-09-11T23:47:40.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:40 smithi178 ceph-mon[35882]: pgmap v340: 97 pgs: 97 active+clean; 65 GiB data, 254 GiB used, 819 GiB / 1.0 TiB avail; 18 KiB/s rd, 310 MiB/s wr, 2.15k op/s 2024-09-11T23:47:41.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.467+0000 7f763591a640 1 -- 172.21.15.133:0/914578774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76301083f0 msgr2=0x7f76301087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.467+0000 7f763591a640 1 --2- 172.21.15.133:0/914578774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76301083f0 0x7f76301087d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7620009900 tx=0x7f762002a420 comp rx=0 tx=0).stop 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.467+0000 7f763591a640 1 -- 172.21.15.133:0/914578774 shutdown_connections 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.467+0000 7f763591a640 1 --2- 172.21.15.133:0/914578774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f763010d520 0x7f76301150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.467+0000 7f763591a640 1 --2- 172.21.15.133:0/914578774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7630108da0 0x7f763010cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.467+0000 7f763591a640 1 --2- 172.21.15.133:0/914578774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76301083f0 0x7f76301087d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.467+0000 7f763591a640 1 -- 172.21.15.133:0/914578774 >> 172.21.15.133:0/914578774 conn(0x7f7630077af0 msgr2=0x7f7630079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.468+0000 7f763591a640 1 -- 172.21.15.133:0/914578774 shutdown_connections 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.468+0000 7f763591a640 1 -- 172.21.15.133:0/914578774 wait complete. 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.470+0000 7f763591a640 1 Processor -- start 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.470+0000 7f763591a640 1 -- start start 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.470+0000 7f763591a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7630108da0 0x7f763007c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.470+0000 7f763591a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f763010d520 0x7f763007ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.470+0000 7f763591a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7630080df0 0x7f76300812a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.470+0000 7f763591a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f763007b050 con 0x7f7630080df0 2024-09-11T23:47:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.470+0000 7f763591a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f763007aed0 con 0x7f763010d520 2024-09-11T23:47:41.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.470+0000 7f763591a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f763007b1d0 con 0x7f7630108da0 2024-09-11T23:47:41.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.472+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7630080df0 0x7f76300812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.473+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7630080df0 0x7f76300812a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50678/0 (socket says 172.21.15.133:50678) 2024-09-11T23:47:41.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.473+0000 7f762f7fe640 1 -- 172.21.15.133:0/1385030875 learned_addr learned my addr 172.21.15.133:0/1385030875 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:41.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.473+0000 7f762e7fc640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f763010d520 0x7f763007ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.475+0000 7f762effd640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7630108da0 0x7f763007c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.476+0000 7f762f7fe640 1 -- 172.21.15.133:0/1385030875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7630108da0 msgr2=0x7f763007c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.476+0000 7f762f7fe640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7630108da0 0x7f763007c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.476+0000 7f762f7fe640 1 -- 172.21.15.133:0/1385030875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f763010d520 msgr2=0x7f763007ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.476+0000 7f762f7fe640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f763010d520 0x7f763007ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.476+0000 7f762f7fe640 1 -- 172.21.15.133:0/1385030875 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7630081810 con 0x7f7630080df0 2024-09-11T23:47:41.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.490+0000 7f762f7fe640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7630080df0 0x7f76300812a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7628052b20 tx=0x7f76280502e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.494+0000 7f7634918640 1 -- 172.21.15.133:0/1385030875 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628004b20 con 0x7f7630080df0 2024-09-11T23:47:41.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.494+0000 7f763591a640 1 -- 172.21.15.133:0/1385030875 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7630081b00 con 0x7f7630080df0 2024-09-11T23:47:41.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.494+0000 7f763591a640 1 -- 172.21.15.133:0/1385030875 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763011e2b0 con 0x7f7630080df0 2024-09-11T23:47:41.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.494+0000 7f7634918640 1 -- 172.21.15.133:0/1385030875 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7628004cc0 con 0x7f7630080df0 2024-09-11T23:47:41.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.494+0000 7f7634918640 1 -- 172.21.15.133:0/1385030875 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628054840 con 0x7f7630080df0 2024-09-11T23:47:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.499+0000 7f7637cf3740 1 Processor -- start 2024-09-11T23:47:41.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.513+0000 7f7637cf3740 1 -- start start 2024-09-11T23:47:41.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.513+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b39880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.513+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b39dc0 0x560777b44140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.513+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b44680 0x560777b46a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.513+0000 7f7637cf3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560777a9f2e0 con 0x560777b39dc0 2024-09-11T23:47:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.513+0000 7f7637cf3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560777a46ce0 con 0x560777b394a0 2024-09-11T23:47:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.513+0000 7f7637cf3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560777a9b2a0 con 0x560777b44680 2024-09-11T23:47:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.514+0000 7f762e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b39880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.514+0000 7f762e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b39880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52182/0 (socket says 172.21.15.133:52182) 2024-09-11T23:47:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.514+0000 7f762e7fc640 1 -- 172.21.15.133:0/4166650038 learned_addr learned my addr 172.21.15.133:0/4166650038 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.516+0000 7f762e7fc640 1 --2- 172.21.15.133:0/4166650038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b44680 0x560777b46a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.516+0000 7f7634918640 1 -- 172.21.15.133:0/1385030875 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7628005020 con 0x7f7630080df0 2024-09-11T23:47:41.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.517+0000 7f762e7fc640 1 -- 172.21.15.133:0/4166650038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b44680 msgr2=0x560777b46a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.517+0000 7f762e7fc640 1 --2- 172.21.15.133:0/4166650038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b44680 0x560777b46a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.517+0000 7f762e7fc640 1 -- 172.21.15.133:0/4166650038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b39dc0 msgr2=0x560777b44140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.517+0000 7f762e7fc640 1 --2- 172.21.15.133:0/4166650038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b39dc0 0x560777b44140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.517+0000 7f762e7fc640 1 -- 172.21.15.133:0/4166650038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560777a2b440 con 0x560777b394a0 2024-09-11T23:47:41.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.518+0000 7f7634918640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76180a2e30 0x7f76180a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.518+0000 7f762effd640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76180a2e30 0x7f76180a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.519+0000 7f7634918640 1 -- 172.21.15.133:0/1385030875 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f7628113ca0 con 0x7f7630080df0 2024-09-11T23:47:41.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.522+0000 7f762e7fc640 1 --2- 172.21.15.133:0/4166650038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b39880 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f762400ad90 tx=0x7f762400a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=443b1515f4dfa70 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.524+0000 7f7615ffb640 1 -- 172.21.15.133:0/4166650038 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762401c070 con 0x560777b394a0 2024-09-11T23:47:41.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.524+0000 7f7615ffb640 1 -- 172.21.15.133:0/4166650038 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7624014c80 con 0x560777b394a0 2024-09-11T23:47:41.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.524+0000 7f7615ffb640 1 -- 172.21.15.133:0/4166650038 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76240125d0 con 0x560777b394a0 2024-09-11T23:47:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.526+0000 7f7637cf3740 1 -- 172.21.15.133:0/4166650038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 msgr2=0x560777b39880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.526+0000 7f7637cf3740 1 --2- 172.21.15.133:0/4166650038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b39880 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f762400ad90 tx=0x7f762400a250 comp rx=0 tx=0).stop 2024-09-11T23:47:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.527+0000 7f7637cf3740 1 -- 172.21.15.133:0/4166650038 shutdown_connections 2024-09-11T23:47:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.527+0000 7f7637cf3740 1 --2- 172.21.15.133:0/4166650038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b44680 0x560777b46a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.527+0000 7f7637cf3740 1 --2- 172.21.15.133:0/4166650038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b39dc0 0x560777b44140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.527+0000 7f7637cf3740 1 --2- 172.21.15.133:0/4166650038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b39880 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.527+0000 7f7637cf3740 1 -- 172.21.15.133:0/4166650038 >> 172.21.15.133:0/4166650038 conn(0x560777b35a30 msgr2=0x560777b35e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.527+0000 7f7637cf3740 1 -- 172.21.15.133:0/4166650038 shutdown_connections 2024-09-11T23:47:41.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.529+0000 7f7637cf3740 1 -- 172.21.15.133:0/4166650038 wait complete. 2024-09-11T23:47:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.529+0000 7f7637cf3740 1 -- start start 2024-09-11T23:47:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.530+0000 7f7637cf3740 1 Processor -- start 2024-09-11T23:47:41.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.531+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.531+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 0x560777b41250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.531+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b41790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.532+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b41790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.532+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b41790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50682/0 (socket says 172.21.15.133:50682) 2024-09-11T23:47:41.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.532+0000 7f762f7fe640 1 -- 172.21.15.133:0/2512816742 learned_addr learned my addr 172.21.15.133:0/2512816742 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:41.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.532+0000 7f7637cf3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560777b582b0 con 0x560777b44680 2024-09-11T23:47:41.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.532+0000 7f7637cf3740 1 -- 172.21.15.133:0/2512816742 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560777a5a050 con 0x560777b394a0 2024-09-11T23:47:41.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.532+0000 7f7637cf3740 1 -- 172.21.15.133:0/2512816742 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560777b58430 con 0x560777b39dc0 2024-09-11T23:47:41.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.532+0000 7f7637cf3740 10 client.0 _ll_register_callbacks cb 0x560777a9a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:41.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.533+0000 7f762f7fe640 1 -- 172.21.15.133:0/2512816742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 msgr2=0x560777b41250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:47:41.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.533+0000 7f762f7fe640 1 --2- 172.21.15.133:0/2512816742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 0x560777b41250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.533+0000 7f762f7fe640 1 -- 172.21.15.133:0/2512816742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 msgr2=0x560777b43fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:47:41.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.533+0000 7f762f7fe640 1 --2- 172.21.15.133:0/2512816742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b43fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.533+0000 7f762f7fe640 1 -- 172.21.15.133:0/2512816742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607778103c0 con 0x560777b44680 2024-09-11T23:47:41.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.535+0000 7f762f7fe640 1 --2- 172.21.15.133:0/2512816742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b41790 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f76280dc570 tx=0x7f76280dfb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c02f395bb81a0c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.535+0000 7f760effd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:41.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.538+0000 7f762effd640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76180a2e30 0x7f76180a52f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x560777b36890 tx=0x7f762002a920 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.542+0000 7f760effd640 1 -- 172.21.15.133:0/2512816742 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76280e01a0 con 0x560777b44680 2024-09-11T23:47:41.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.542+0000 7f760effd640 1 -- 172.21.15.133:0/2512816742 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f7628063340 con 0x560777b44680 2024-09-11T23:47:41.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.542+0000 7f7637cf3740 1 -- 172.21.15.133:0/2512816742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7624014c80 con 0x560777b44680 2024-09-11T23:47:41.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.542+0000 7f760effd640 1 -- 172.21.15.133:0/2512816742 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f76280e4550 con 0x560777b44680 2024-09-11T23:47:41.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.542+0000 7f7637cf3740 10 client.18198 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:41.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.542+0000 7f7637cf3740 1 -- 172.21.15.133:0/2512816742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560777b63c90 con 0x560777b44680 2024-09-11T23:47:41.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.543+0000 7f760effd640 1 -- 172.21.15.133:0/2512816742 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7624014c80 con 0x560777b44680 2024-09-11T23:47:41.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.543+0000 7f760effd640 1 -- 172.21.15.133:0/2512816742 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f76280e8980 con 0x560777b44680 2024-09-11T23:47:41.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.543+0000 7f7637cf3740 10 client.18198 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:41.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.543+0000 7f7637cf3740 10 client.18198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:41.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.543+0000 7f7637cf3740 20 client.18198 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:41.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.543+0000 7f7637cf3740 1 --2- 172.21.15.133:0/2512816742 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560777b655e0 0x560777b67a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.543+0000 7f7637cf3740 4 client.18198 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:41.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.543+0000 7f7637cf3740 1 -- 172.21.15.133:0/2512816742 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560777aac850 con 0x560777b655e0 2024-09-11T23:47:41.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.545+0000 7f762effd640 1 --2- 172.21.15.133:0/2512816742 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560777b655e0 0x560777b67a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.549+0000 7f762effd640 1 --2- 172.21.15.133:0/2512816742 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560777b655e0 0x560777b67a80 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=323df88c3c1a74f3 server_cookie=e2c104d5d14860f1 in_seq=0 out_seq=0 2024-09-11T23:47:41.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.549+0000 7f760effd640 10 client.18198 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.552+0000 7f760effd640 1 -- 172.21.15.133:0/2512816742 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560777aac850 con 0x560777b655e0 2024-09-11T23:47:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.552+0000 7f760effd640 10 client.18198 handle_command_reply: tid=0 2024-09-11T23:47:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.552+0000 7f7637cf3740 1 client.18198 shutdown 2024-09-11T23:47:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.553+0000 7f7637cf3740 1 -- 172.21.15.133:0/2512816742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 msgr2=0x560777b41790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.553+0000 7f7637cf3740 1 --2- 172.21.15.133:0/2512816742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b41790 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f76280dc570 tx=0x7f76280dfb00 comp rx=0 tx=0).stop 2024-09-11T23:47:41.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.553+0000 7f7637cf3740 1 -- 172.21.15.133:0/2512816742 shutdown_connections 2024-09-11T23:47:41.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.553+0000 7f7637cf3740 1 --2- 172.21.15.133:0/2512816742 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560777b655e0 0x560777b67a80 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.553+0000 7f7637cf3740 1 --2- 172.21.15.133:0/2512816742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b41790 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.553+0000 7f7637cf3740 1 --2- 172.21.15.133:0/2512816742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 0x560777b41250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.553+0000 7f7637cf3740 1 --2- 172.21.15.133:0/2512816742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.553+0000 7f7637cf3740 1 -- 172.21.15.133:0/2512816742 >> 172.21.15.133:0/2512816742 conn(0x560777b35a30 msgr2=0x560777b3b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:41.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.554+0000 7f760effd640 0 client.18198 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:41.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.554+0000 7f7637cf3740 1 -- 172.21.15.133:0/2512816742 shutdown_connections 2024-09-11T23:47:41.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.554+0000 7f7637cf3740 1 -- 172.21.15.133:0/2512816742 wait complete. 2024-09-11T23:47:41.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.555+0000 7f7637cf3740 20 client.18198 trim_cache size 0 max 16384 2024-09-11T23:47:41.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.603+0000 7f7637cf3740 1 Processor -- start 2024-09-11T23:47:41.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.603+0000 7f7637cf3740 1 -- start start 2024-09-11T23:47:41.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.604+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b615a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.604+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 0x560777b6d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.604+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b6d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.605+0000 7f762effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.605+0000 7f762effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b615a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52198/0 (socket says 172.21.15.133:52198) 2024-09-11T23:47:41.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.605+0000 7f762effd640 1 -- 172.21.15.133:0/2958178435 learned_addr learned my addr 172.21.15.133:0/2958178435 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:41.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.605+0000 7f7637cf3740 1 -- 172.21.15.133:0/2958178435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f761c00e670 con 0x560777b44680 2024-09-11T23:47:41.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.605+0000 7f7637cf3740 1 -- 172.21.15.133:0/2958178435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f761c0101b0 con 0x560777b394a0 2024-09-11T23:47:41.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.605+0000 7f7637cf3740 1 -- 172.21.15.133:0/2958178435 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f761c00ed40 con 0x560777b39dc0 2024-09-11T23:47:41.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.609+0000 7f762e7fc640 1 --2- 172.21.15.133:0/2958178435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 0x560777b6d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.609+0000 7f762f7fe640 1 --2- 172.21.15.133:0/2958178435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b6d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.609+0000 7f762effd640 1 -- 172.21.15.133:0/2958178435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 msgr2=0x560777b6d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.609+0000 7f762effd640 1 --2- 172.21.15.133:0/2958178435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 0x560777b6d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.609+0000 7f762effd640 1 -- 172.21.15.133:0/2958178435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 msgr2=0x560777b6d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.609+0000 7f762effd640 1 --2- 172.21.15.133:0/2958178435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b6d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.609+0000 7f762effd640 1 -- 172.21.15.133:0/2958178435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560777b40200 con 0x560777b394a0 2024-09-11T23:47:41.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.610+0000 7f762effd640 1 --2- 172.21.15.133:0/2958178435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b615a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f762002a990 tx=0x7f7620041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8e89fa89cf4bec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.611+0000 7f76157fa640 1 -- 172.21.15.133:0/2958178435 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7620057070 con 0x560777b394a0 2024-09-11T23:47:41.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.611+0000 7f76157fa640 1 -- 172.21.15.133:0/2958178435 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7620004d20 con 0x560777b394a0 2024-09-11T23:47:41.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.611+0000 7f76157fa640 1 -- 172.21.15.133:0/2958178435 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7620031d10 con 0x560777b394a0 2024-09-11T23:47:41.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.614+0000 7f7637cf3740 1 -- 172.21.15.133:0/2958178435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 msgr2=0x560777b615a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.614+0000 7f7637cf3740 1 --2- 172.21.15.133:0/2958178435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b615a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f762002a990 tx=0x7f7620041000 comp rx=0 tx=0).stop 2024-09-11T23:47:41.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.614+0000 7f7637cf3740 1 -- 172.21.15.133:0/2958178435 shutdown_connections 2024-09-11T23:47:41.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.614+0000 7f7637cf3740 1 --2- 172.21.15.133:0/2958178435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b6d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.614+0000 7f7637cf3740 1 --2- 172.21.15.133:0/2958178435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 0x560777b6d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.615+0000 7f7637cf3740 1 --2- 172.21.15.133:0/2958178435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b615a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.615+0000 7f7637cf3740 1 -- 172.21.15.133:0/2958178435 >> 172.21.15.133:0/2958178435 conn(0x560777b35a30 msgr2=0x560777b3c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:41.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.615+0000 7f7637cf3740 1 -- 172.21.15.133:0/2958178435 shutdown_connections 2024-09-11T23:47:41.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.615+0000 7f7637cf3740 1 -- 172.21.15.133:0/2958178435 wait complete. 2024-09-11T23:47:41.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.616+0000 7f7637cf3740 1 -- start start 2024-09-11T23:47:41.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.617+0000 7f7637cf3740 1 Processor -- start 2024-09-11T23:47:41.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.617+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b5d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.617+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 0x560777b5de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.618+0000 7f7637cf3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b58cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.618+0000 7f7637cf3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f761c00fae0 con 0x560777b44680 2024-09-11T23:47:41.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.618+0000 7f7637cf3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f761c00f410 con 0x560777b394a0 2024-09-11T23:47:41.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.618+0000 7f7637cf3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f761c00c670 con 0x560777b39dc0 2024-09-11T23:47:41.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.618+0000 7f7637cf3740 10 client.0 _ll_register_callbacks cb 0x7f7628004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:41.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.621+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b58cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.621+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b58cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50696/0 (socket says 172.21.15.133:50696) 2024-09-11T23:47:41.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.621+0000 7f762f7fe640 1 -- 172.21.15.133:0/331774040 learned_addr learned my addr 172.21.15.133:0/331774040 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:41.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.621+0000 7f762effd640 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b5d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.622+0000 7f762f7fe640 1 -- 172.21.15.133:0/331774040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 msgr2=0x560777b5de20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:47:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.622+0000 7f762f7fe640 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 0x560777b5de20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.622+0000 7f762f7fe640 1 -- 172.21.15.133:0/331774040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 msgr2=0x560777b5d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.622+0000 7f762f7fe640 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b5d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.622+0000 7f762f7fe640 1 -- 172.21.15.133:0/331774040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560777b44310 con 0x560777b44680 2024-09-11T23:47:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.622+0000 7f762f7fe640 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b58cd0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7628050fd0 tx=0x7f76280f9240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f42cc3ed80a588 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.622+0000 7f760effd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.623+0000 7f760effd640 1 -- 172.21.15.133:0/331774040 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76280dff70 con 0x560777b44680 2024-09-11T23:47:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.623+0000 7f760effd640 1 -- 172.21.15.133:0/331774040 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f7628111650 con 0x560777b44680 2024-09-11T23:47:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.623+0000 7f7637cf3740 1 -- 172.21.15.133:0/331774040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7620004d20 con 0x560777b44680 2024-09-11T23:47:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.623+0000 7f760effd640 1 -- 172.21.15.133:0/331774040 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f76280e01a0 con 0x560777b44680 2024-09-11T23:47:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.623+0000 7f7637cf3740 10 client.18204 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.623+0000 7f7637cf3740 1 -- 172.21.15.133:0/331774040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560777bcfca0 con 0x560777b44680 2024-09-11T23:47:41.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.624+0000 7f760effd640 1 -- 172.21.15.133:0/331774040 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7620004d20 con 0x560777b44680 2024-09-11T23:47:41.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.624+0000 7f760effd640 1 -- 172.21.15.133:0/331774040 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f76280dbe60 con 0x560777b44680 2024-09-11T23:47:41.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.624+0000 7f7637cf3740 10 client.18204 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:41.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.624+0000 7f7637cf3740 10 client.18204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:41.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.624+0000 7f7637cf3740 20 client.18204 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:41.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.624+0000 7f7637cf3740 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560777bd0230 0x560777bd2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.624+0000 7f7637cf3740 4 client.18204 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:47:41.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.624+0000 7f7637cf3740 1 -- 172.21.15.133:0/331774040 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560777aa6d60 con 0x560777bd0230 2024-09-11T23:47:41.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.626+0000 7f762effd640 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560777bd0230 0x560777bd2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.628+0000 7f762effd640 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560777bd0230 0x560777bd2640 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254d2fb547ba12e1 server_cookie=68c3676e466ae380 in_seq=0 out_seq=0 2024-09-11T23:47:41.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.628+0000 7f760effd640 10 client.18204 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.629+0000 7f760effd640 1 -- 172.21.15.133:0/331774040 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560777aa6d60 con 0x560777bd0230 2024-09-11T23:47:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.629+0000 7f760effd640 10 client.18204 handle_command_reply: tid=0 2024-09-11T23:47:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.629+0000 7f7637cf3740 1 client.18204 shutdown 2024-09-11T23:47:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.630+0000 7f7637cf3740 1 -- 172.21.15.133:0/331774040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 msgr2=0x560777b58cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.630+0000 7f7637cf3740 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b58cd0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7628050fd0 tx=0x7f76280f9240 comp rx=0 tx=0).stop 2024-09-11T23:47:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.630+0000 7f7637cf3740 1 -- 172.21.15.133:0/331774040 shutdown_connections 2024-09-11T23:47:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.630+0000 7f7637cf3740 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x560777bd0230 0x560777bd2640 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.630+0000 7f7637cf3740 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560777b44680 0x560777b58cd0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.630+0000 7f7637cf3740 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560777b39dc0 0x560777b5de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.630+0000 7f7637cf3740 1 --2- 172.21.15.133:0/331774040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560777b394a0 0x560777b5d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.630+0000 7f7637cf3740 1 -- 172.21.15.133:0/331774040 >> 172.21.15.133:0/331774040 conn(0x560777b35a30 msgr2=0x560777b40b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:41.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.631+0000 7f760effd640 0 client.18204 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:41.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.631+0000 7f7637cf3740 1 -- 172.21.15.133:0/331774040 shutdown_connections 2024-09-11T23:47:41.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.631+0000 7f7637cf3740 1 -- 172.21.15.133:0/331774040 wait complete. 2024-09-11T23:47:41.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.631+0000 7f7637cf3740 20 client.18204 trim_cache size 0 max 16384 2024-09-11T23:47:41.635 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:47:41.635 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:47:41.635 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:47:41.635 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:47:41.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.636+0000 7f763591a640 1 -- 172.21.15.133:0/1385030875 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76180a2e30 msgr2=0x7f76180a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.637+0000 7f763591a640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76180a2e30 0x7f76180a52f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x560777b36890 tx=0x7f762002a920 comp rx=0 tx=0).stop 2024-09-11T23:47:41.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.637+0000 7f763591a640 1 -- 172.21.15.133:0/1385030875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7630080df0 msgr2=0x7f76300812a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.637+0000 7f763591a640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7630080df0 0x7f76300812a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7628052b20 tx=0x7f76280502e0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.637+0000 7f763591a640 1 -- 172.21.15.133:0/1385030875 shutdown_connections 2024-09-11T23:47:41.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.637+0000 7f763591a640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76180a2e30 0x7f76180a52f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.637+0000 7f763591a640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7630080df0 0x7f76300812a0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.637+0000 7f763591a640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f763010d520 0x7f763007ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.637+0000 7f763591a640 1 --2- 172.21.15.133:0/1385030875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7630108da0 0x7f763007c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.637+0000 7f763591a640 1 -- 172.21.15.133:0/1385030875 >> 172.21.15.133:0/1385030875 conn(0x7f7630077af0 msgr2=0x7f7630077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:41.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.638+0000 7f763591a640 1 -- 172.21.15.133:0/1385030875 shutdown_connections 2024-09-11T23:47:41.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:41.638+0000 7f763591a640 1 -- 172.21.15.133:0/1385030875 wait complete. 2024-09-11T23:47:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.671+0000 7feeda998640 1 -- 172.21.15.133:0/2424600151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feed4109b10 msgr2=0x7feed410e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.671+0000 7feeda998640 1 --2- 172.21.15.133:0/2424600151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feed4109b10 0x7feed410e2b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feec8009900 tx=0x7feec802a440 comp rx=0 tx=0).stop 2024-09-11T23:47:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.674+0000 7feeda998640 1 -- 172.21.15.133:0/2424600151 shutdown_connections 2024-09-11T23:47:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.674+0000 7feeda998640 1 --2- 172.21.15.133:0/2424600151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed410e7f0 0x7feed410eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.674+0000 7feeda998640 1 --2- 172.21.15.133:0/2424600151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feed4109b10 0x7feed410e2b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.674+0000 7feeda998640 1 --2- 172.21.15.133:0/2424600151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed41091f0 0x7feed41095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.674+0000 7feeda998640 1 -- 172.21.15.133:0/2424600151 >> 172.21.15.133:0/2424600151 conn(0x7feed406d210 msgr2=0x7feed406d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.674+0000 7feeda998640 1 -- 172.21.15.133:0/2424600151 shutdown_connections 2024-09-11T23:47:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.674+0000 7feeda998640 1 -- 172.21.15.133:0/2424600151 wait complete. 2024-09-11T23:47:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.675+0000 7feeda998640 1 Processor -- start 2024-09-11T23:47:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.677+0000 7feeda998640 1 -- start start 2024-09-11T23:47:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.677+0000 7feeda998640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41091f0 0x7feed4118820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.677+0000 7feeda998640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feed4109b10 0x7feed4118d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.678+0000 7feeda998640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed410e7f0 0x7feed41192a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.678+0000 7feeda998640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feed411cb60 con 0x7feed410e7f0 2024-09-11T23:47:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.678+0000 7feeda998640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feed4112520 con 0x7feed41091f0 2024-09-11T23:47:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.678+0000 7feeda998640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feed411cce0 con 0x7feed4109b10 2024-09-11T23:47:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.678+0000 7feed3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41091f0 0x7feed4118820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.679+0000 7feed3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41091f0 0x7feed4118820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52236/0 (socket says 172.21.15.133:52236) 2024-09-11T23:47:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.679+0000 7feed3fff640 1 -- 172.21.15.133:0/3676506410 learned_addr learned my addr 172.21.15.133:0/3676506410 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.679+0000 7feed3fff640 1 -- 172.21.15.133:0/3676506410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feed4109b10 msgr2=0x7feed4118d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.680+0000 7feed3fff640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feed4109b10 0x7feed4118d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.680+0000 7feed3fff640 1 -- 172.21.15.133:0/3676506410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed410e7f0 msgr2=0x7feed41192a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.680+0000 7feed3fff640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed410e7f0 0x7feed41192a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.680+0000 7feed3fff640 1 -- 172.21.15.133:0/3676506410 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed41a2d10 con 0x7feed41091f0 2024-09-11T23:47:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.681+0000 7feed3fff640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41091f0 0x7feed4118820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ba50 tx=0x7feecc00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.681+0000 7feed17fa640 1 -- 172.21.15.133:0/3676506410 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc019070 con 0x7feed41091f0 2024-09-11T23:47:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.681+0000 7feeda998640 1 -- 172.21.15.133:0/3676506410 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed41a3000 con 0x7feed41091f0 2024-09-11T23:47:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.681+0000 7feeda998640 1 -- 172.21.15.133:0/3676506410 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed41a35a0 con 0x7feed41091f0 2024-09-11T23:47:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.682+0000 7feed17fa640 1 -- 172.21.15.133:0/3676506410 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feecc00cbe0 con 0x7feed41091f0 2024-09-11T23:47:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.683+0000 7feed17fa640 1 -- 172.21.15.133:0/3676506410 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc0076e0 con 0x7feed41091f0 2024-09-11T23:47:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.689+0000 7feed17fa640 1 -- 172.21.15.133:0/3676506410 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7feecc01c070 con 0x7feed41091f0 2024-09-11T23:47:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.691+0000 7feedd811740 1 Processor -- start 2024-09-11T23:47:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.692+0000 7feedd811740 1 -- start start 2024-09-11T23:47:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.692+0000 7feed17fa640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feebc0a2de0 0x7feebc0a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.694+0000 7feedd811740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e494ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.694+0000 7feedd811740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e495410 0x55638e49f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.694+0000 7feedd811740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 0x55638e4a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.693+0000 7feed17fa640 1 -- 172.21.15.133:0/3676506410 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7feecc0ce8d0 con 0x7feed41091f0 2024-09-11T23:47:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.694+0000 7feed3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e494ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.694+0000 7feed3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e494ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52246/0 (socket says 172.21.15.133:52246) 2024-09-11T23:47:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.694+0000 7feed3fff640 1 -- 172.21.15.133:0/2856124463 learned_addr learned my addr 172.21.15.133:0/2856124463 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.695+0000 7feedd811740 1 -- 172.21.15.133:0/2856124463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55638e3f92f0 con 0x55638e49fcb0 2024-09-11T23:47:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.695+0000 7feed37fe640 1 --2- 172.21.15.133:0/2856124463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e495410 0x55638e49f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.695+0000 7feedd811740 1 -- 172.21.15.133:0/2856124463 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55638e3a1270 con 0x55638e494af0 2024-09-11T23:47:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.695+0000 7feedd811740 1 -- 172.21.15.133:0/2856124463 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55638e3f52b0 con 0x55638e495410 2024-09-11T23:47:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.695+0000 7feed8f35640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feebc0a2de0 0x7feebc0a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.696+0000 7feed8f35640 1 --2- 172.21.15.133:0/2856124463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 0x55638e4a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.696+0000 7feed8f35640 1 -- 172.21.15.133:0/2856124463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e495410 msgr2=0x55638e49f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.697+0000 7feed8f35640 1 --2- 172.21.15.133:0/2856124463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e495410 0x55638e49f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.697+0000 7feed8f35640 1 -- 172.21.15.133:0/2856124463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 msgr2=0x55638e494ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.697+0000 7feed8f35640 1 --2- 172.21.15.133:0/2856124463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e494ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.697+0000 7feed8f35640 1 -- 172.21.15.133:0/2856124463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55638e3a0340 con 0x55638e49fcb0 2024-09-11T23:47:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.697+0000 7feed8f35640 1 --2- 172.21.15.133:0/2856124463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 0x55638e4a20a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7feec4011a00 tx=0x7feec4011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2764b877f052f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.698+0000 7feed8f35640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feebc0a2de0 0x7feebc0a52a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feec400e780 tx=0x7feec4005000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.701+0000 7feec27fc640 1 -- 172.21.15.133:0/2856124463 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec401f070 con 0x55638e49fcb0 2024-09-11T23:47:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.701+0000 7feec27fc640 1 -- 172.21.15.133:0/2856124463 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec40065c0 con 0x55638e49fcb0 2024-09-11T23:47:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.702+0000 7feedd811740 1 -- 172.21.15.133:0/2856124463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 msgr2=0x55638e4a20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.702+0000 7feedd811740 1 --2- 172.21.15.133:0/2856124463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 0x55638e4a20a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7feec4011a00 tx=0x7feec4011ec0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.703+0000 7feedd811740 1 -- 172.21.15.133:0/2856124463 shutdown_connections 2024-09-11T23:47:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.703+0000 7feedd811740 1 --2- 172.21.15.133:0/2856124463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 0x55638e4a20a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.703+0000 7feedd811740 1 --2- 172.21.15.133:0/2856124463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e495410 0x55638e49f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.703+0000 7feedd811740 1 --2- 172.21.15.133:0/2856124463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e494ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.703+0000 7feedd811740 1 -- 172.21.15.133:0/2856124463 >> 172.21.15.133:0/2856124463 conn(0x55638e491060 msgr2=0x55638e491440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.708+0000 7feedd811740 1 -- 172.21.15.133:0/2856124463 shutdown_connections 2024-09-11T23:47:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.708+0000 7feedd811740 1 -- 172.21.15.133:0/2856124463 wait complete. 2024-09-11T23:47:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.709+0000 7feedd811740 1 -- start start 2024-09-11T23:47:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.709+0000 7feedd811740 1 Processor -- start 2024-09-11T23:47:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.710+0000 7feedd811740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e494af0 0x55638e49cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.711+0000 7feedd811740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e495410 0x55638e49d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.711+0000 7feedd811740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 0x55638e49d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.711+0000 7feedd811740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55638e4a28d0 con 0x55638e494af0 2024-09-11T23:47:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.711+0000 7feedd811740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55638e382290 con 0x55638e495410 2024-09-11T23:47:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.711+0000 7feedd811740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55638e4a2a50 con 0x55638e49fcb0 2024-09-11T23:47:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.711+0000 7feed8f35640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 0x55638e49d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.712+0000 7feedd811740 10 client.0 _ll_register_callbacks cb 0x55638e3b3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.714+0000 7feed8f35640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 0x55638e49d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:49398/0 (socket says 172.21.15.133:49398) 2024-09-11T23:47:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.714+0000 7feed8f35640 1 -- 172.21.15.133:0/3303212192 learned_addr learned my addr 172.21.15.133:0/3303212192 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.715+0000 7feed8f35640 1 -- 172.21.15.133:0/3303212192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e495410 msgr2=0x55638e49d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.715+0000 7feed8f35640 1 --2- 172.21.15.133:0/3303212192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e495410 0x55638e49d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.715+0000 7feed8f35640 1 -- 172.21.15.133:0/3303212192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e494af0 msgr2=0x55638e49cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.715+0000 7feed8f35640 1 --2- 172.21.15.133:0/3303212192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e494af0 0x55638e49cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.715+0000 7feed8f35640 1 -- 172.21.15.133:0/3303212192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55638e10c210 con 0x55638e49fcb0 2024-09-11T23:47:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.722+0000 7feed8f35640 1 --2- 172.21.15.133:0/3303212192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 0x55638e49d620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feec400c700 tx=0x7feec400c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddbcce4d3628bed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.722+0000 7feeb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:47:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.722+0000 7feeb77fe640 1 -- 172.21.15.133:0/3303212192 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4028070 con 0x55638e49fcb0 2024-09-11T23:47:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.723+0000 7feedd811740 1 -- 172.21.15.133:0/3303212192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec40065c0 con 0x55638e49fcb0 2024-09-11T23:47:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.723+0000 7feeb77fe640 1 -- 172.21.15.133:0/3303212192 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7feec400fd00 con 0x55638e49fcb0 2024-09-11T23:47:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.723+0000 7feeb77fe640 1 -- 172.21.15.133:0/3303212192 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feec4033050 con 0x55638e49fcb0 2024-09-11T23:47:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.723+0000 7feedd811740 10 client.27508 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.723+0000 7feedd811740 1 -- 172.21.15.133:0/3303212192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55638e1d8620 con 0x55638e49fcb0 2024-09-11T23:47:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.724+0000 7feeb77fe640 1 -- 172.21.15.133:0/3303212192 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec40065c0 con 0x55638e49fcb0 2024-09-11T23:47:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.725+0000 7feeb77fe640 1 -- 172.21.15.133:0/3303212192 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feec401b3f0 con 0x55638e49fcb0 2024-09-11T23:47:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.726+0000 7feedd811740 10 client.27508 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.726+0000 7feedd811740 10 client.27508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.726+0000 7feedd811740 20 client.27508 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.726+0000 7feedd811740 1 --2- 172.21.15.133:0/3303212192 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55638e4b0160 0x55638e4b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.726+0000 7feedd811740 4 client.27508 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.726+0000 7feedd811740 1 -- 172.21.15.133:0/3303212192 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55638e406a20 con 0x55638e4b0160 2024-09-11T23:47:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.726+0000 7feed3fff640 1 --2- 172.21.15.133:0/3303212192 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55638e4b0160 0x55638e4b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.728+0000 7feed3fff640 1 --2- 172.21.15.133:0/3303212192 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55638e4b0160 0x55638e4b2620 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=165e98e55f95fb70 server_cookie=c884f28dfe902538 in_seq=0 out_seq=0 2024-09-11T23:47:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.728+0000 7feeb77fe640 10 client.27508 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.729+0000 7feeb77fe640 1 -- 172.21.15.133:0/3303212192 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55638e406a20 con 0x55638e4b0160 2024-09-11T23:47:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.729+0000 7feeb77fe640 10 client.27508 handle_command_reply: tid=0 2024-09-11T23:47:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.729+0000 7feedd811740 1 client.27508 shutdown 2024-09-11T23:47:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.730+0000 7feedd811740 1 -- 172.21.15.133:0/3303212192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 msgr2=0x55638e49d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.730+0000 7feedd811740 1 --2- 172.21.15.133:0/3303212192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 0x55638e49d620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feec400c700 tx=0x7feec400c730 comp rx=0 tx=0).stop 2024-09-11T23:47:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.730+0000 7feedd811740 1 -- 172.21.15.133:0/3303212192 shutdown_connections 2024-09-11T23:47:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.730+0000 7feedd811740 1 --2- 172.21.15.133:0/3303212192 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55638e4b0160 0x55638e4b2620 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.730+0000 7feedd811740 1 --2- 172.21.15.133:0/3303212192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 0x55638e49d620 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.730+0000 7feedd811740 1 --2- 172.21.15.133:0/3303212192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e495410 0x55638e49d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.730+0000 7feedd811740 1 --2- 172.21.15.133:0/3303212192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e494af0 0x55638e49cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.730+0000 7feedd811740 1 -- 172.21.15.133:0/3303212192 >> 172.21.15.133:0/3303212192 conn(0x55638e491060 msgr2=0x55638e496dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.730+0000 7feeb77fe640 0 client.27508 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.730+0000 7feedd811740 1 -- 172.21.15.133:0/3303212192 shutdown_connections 2024-09-11T23:47:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.730+0000 7feedd811740 1 -- 172.21.15.133:0/3303212192 wait complete. 2024-09-11T23:47:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.731+0000 7feedd811740 20 client.27508 trim_cache size 0 max 16384 2024-09-11T23:47:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.781+0000 7feedd811740 1 Processor -- start 2024-09-11T23:47:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.781+0000 7feedd811740 1 -- start start 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.781+0000 7feedd811740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e4975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.781+0000 7feedd811740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e495410 0x55638e49eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.781+0000 7feedd811740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 0x55638e49d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.781+0000 7feedd811740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feeb800e540 con 0x55638e49fcb0 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.781+0000 7feedd811740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8010080 con 0x55638e494af0 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.781+0000 7feedd811740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feeb800ec10 con 0x55638e495410 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.782+0000 7feed37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e495410 0x55638e49eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.782+0000 7feed37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e495410 0x55638e49eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:49402/0 (socket says 172.21.15.133:49402) 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.782+0000 7feed37fe640 1 -- 172.21.15.133:0/602674376 learned_addr learned my addr 172.21.15.133:0/602674376 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.782+0000 7feed3fff640 1 --2- 172.21.15.133:0/602674376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e4975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.782+0000 7feed8f35640 1 --2- 172.21.15.133:0/602674376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 0x55638e49d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.782+0000 7feed3fff640 1 -- 172.21.15.133:0/602674376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e495410 msgr2=0x55638e49eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.782+0000 7feed3fff640 1 --2- 172.21.15.133:0/602674376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e495410 0x55638e49eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.782+0000 7feed3fff640 1 -- 172.21.15.133:0/602674376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 msgr2=0x55638e49d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.782+0000 7feed3fff640 1 --2- 172.21.15.133:0/602674376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 0x55638e49d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.782+0000 7feed3fff640 1 -- 172.21.15.133:0/602674376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55638e490e90 con 0x55638e494af0 2024-09-11T23:47:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.782+0000 7feed8f35640 1 --2- 172.21.15.133:0/602674376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 0x55638e49d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:47:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.783+0000 7feed3fff640 1 --2- 172.21.15.133:0/602674376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e4975b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feecc0978b0 tx=0x7feecc0ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6533c94ac32c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.783+0000 7feec1ffb640 1 -- 172.21.15.133:0/602674376 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc09f290 con 0x55638e494af0 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.783+0000 7feec1ffb640 1 -- 172.21.15.133:0/602674376 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feecc0b6020 con 0x55638e494af0 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.783+0000 7feec1ffb640 1 -- 172.21.15.133:0/602674376 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc0cc280 con 0x55638e494af0 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.784+0000 7feedd811740 1 -- 172.21.15.133:0/602674376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 msgr2=0x55638e4975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.784+0000 7feedd811740 1 --2- 172.21.15.133:0/602674376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e4975b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feecc0978b0 tx=0x7feecc0ce040 comp rx=0 tx=0).stop 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.784+0000 7feedd811740 1 -- 172.21.15.133:0/602674376 shutdown_connections 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.784+0000 7feedd811740 1 --2- 172.21.15.133:0/602674376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e49fcb0 0x55638e49d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.784+0000 7feedd811740 1 --2- 172.21.15.133:0/602674376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e495410 0x55638e49eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.784+0000 7feedd811740 1 --2- 172.21.15.133:0/602674376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e4975b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.784+0000 7feedd811740 1 -- 172.21.15.133:0/602674376 >> 172.21.15.133:0/602674376 conn(0x55638e491060 msgr2=0x55638e49e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.784+0000 7feedd811740 1 -- 172.21.15.133:0/602674376 shutdown_connections 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.784+0000 7feedd811740 1 -- 172.21.15.133:0/602674376 wait complete. 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.785+0000 7feedd811740 1 -- start start 2024-09-11T23:47:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.785+0000 7feedd811740 1 Processor -- start 2024-09-11T23:47:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.786+0000 7feedd811740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e4a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.786+0000 7feedd811740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e495410 0x55638e4a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.786+0000 7feed3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e4a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.786+0000 7feed3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e4a2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52284/0 (socket says 172.21.15.133:52284) 2024-09-11T23:47:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.786+0000 7feed3fff640 1 -- 172.21.15.133:0/3317776252 learned_addr learned my addr 172.21.15.133:0/3317776252 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.786+0000 7feed37fe640 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e495410 0x55638e4a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.786+0000 7feedd811740 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 0x55638e4a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feedd811740 1 -- 172.21.15.133:0/3317776252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feeb800f9b0 con 0x55638e495410 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feedd811740 1 -- 172.21.15.133:0/3317776252 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feeb800f2e0 con 0x55638e494af0 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feedd811740 1 -- 172.21.15.133:0/3317776252 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feeb800c5d0 con 0x55638e49fcb0 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feed8f35640 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 0x55638e4a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feedd811740 10 client.0 _ll_register_callbacks cb 0x7feec4011800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feed3fff640 1 -- 172.21.15.133:0/3317776252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 msgr2=0x55638e4a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feed3fff640 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 0x55638e4a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feed3fff640 1 -- 172.21.15.133:0/3317776252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e495410 msgr2=0x55638e4a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feed3fff640 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e495410 0x55638e4a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feed3fff640 1 -- 172.21.15.133:0/3317776252 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55638e519160 con 0x55638e494af0 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feedd811740 1 -- 172.21.15.133:0/3317776252 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feecc0b6020 con 0x55638e494af0 2024-09-11T23:47:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feed3fff640 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e4a2de0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ba20 tx=0x7feecc09ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8045295ff82c2338 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.787+0000 7feeb77fe640 10 client.27749 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feeb77fe640 1 -- 172.21.15.133:0/3317776252 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc07d070 con 0x55638e494af0 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feeb77fe640 1 -- 172.21.15.133:0/3317776252 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7feecc098f80 con 0x55638e494af0 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feeb77fe640 1 -- 172.21.15.133:0/3317776252 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feecc09f290 con 0x55638e494af0 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feedd811740 10 client.27749 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feedd811740 1 -- 172.21.15.133:0/3317776252 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55638e51a2e0 con 0x55638e494af0 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feeb77fe640 1 -- 172.21.15.133:0/3317776252 <== mon.2 v2:172.21.15.138:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feecc080030 con 0x55638e494af0 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feedd811740 10 client.27749 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feedd811740 10 client.27749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feedd811740 20 client.27749 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feedd811740 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55638e51a870 0x55638e51cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feedd811740 4 client.27749 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:47:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.788+0000 7feedd811740 1 -- 172.21.15.133:0/3317776252 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55638e400d70 con 0x55638e51a870 2024-09-11T23:47:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.789+0000 7feed37fe640 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55638e51a870 0x55638e51cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.789+0000 7feeb77fe640 10 client.27749 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:41.789+0000 7feed37fe640 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55638e51a870 0x55638e51cc90 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370e9159de97cdca server_cookie=41e1c6e11409ea37 in_seq=0 out_seq=0 2024-09-11T23:47:43.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:42 smithi178 ceph-mon[35882]: pgmap v341: 97 pgs: 97 active+clean; 65 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 13 KiB/s rd, 140 MiB/s wr, 1.13k op/s 2024-09-11T23:47:43.071 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:47:43.071 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:47:43.075 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:42 smithi133 ceph-mon[31430]: pgmap v341: 97 pgs: 97 active+clean; 65 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 13 KiB/s rd, 140 MiB/s wr, 1.13k op/s 2024-09-11T23:47:43.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:42 smithi138 ceph-mon[35142]: pgmap v341: 97 pgs: 97 active+clean; 65 GiB data, 243 GiB used, 830 GiB / 1.0 TiB avail; 13 KiB/s rd, 140 MiB/s wr, 1.13k op/s 2024-09-11T23:47:44.098 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:44.900 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:44 smithi133 ceph-mon[31430]: pgmap v342: 97 pgs: 97 active+clean; 66 GiB data, 247 GiB used, 826 GiB / 1.0 TiB avail; 966 KiB/s rd, 253 MiB/s wr, 2.04k op/s 2024-09-11T23:47:45.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.021+0000 7f39c3a01640 1 -- 172.21.15.133:0/4271590973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39bc109830 msgr2=0x7f39bc109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:45.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.021+0000 7f39c3a01640 1 --2- 172.21.15.133:0/4271590973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39bc109830 0x7f39bc109c90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f39ac01a910 tx=0x7f39ac03b470 comp rx=0 tx=0).stop 2024-09-11T23:47:45.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.022+0000 7f39c3a01640 1 -- 172.21.15.133:0/4271590973 shutdown_connections 2024-09-11T23:47:45.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.022+0000 7f39c3a01640 1 --2- 172.21.15.133:0/4271590973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc11b140 0x7f39bc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:45.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.022+0000 7f39c3a01640 1 --2- 172.21.15.133:0/4271590973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39bc109830 0x7f39bc109c90 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f39ac01a910 tx=0x7f39ac03b470 comp rx=0 tx=0).stop 2024-09-11T23:47:45.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.022+0000 7f39c3a01640 1 --2- 172.21.15.133:0/4271590973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39bc109450 0x7f39bc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:45.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.022+0000 7f39c3a01640 1 -- 172.21.15.133:0/4271590973 >> 172.21.15.133:0/4271590973 conn(0x7f39bc06e140 msgr2=0x7f39bc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:45.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.022+0000 7f39c3a01640 1 -- 172.21.15.133:0/4271590973 shutdown_connections 2024-09-11T23:47:45.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.023+0000 7f39c3a01640 1 -- 172.21.15.133:0/4271590973 wait complete. 2024-09-11T23:47:45.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.024+0000 7f39c3a01640 1 Processor -- start 2024-09-11T23:47:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.024+0000 7f39c3a01640 1 -- start start 2024-09-11T23:47:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.024+0000 7f39c3a01640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc109450 0x7f39bc11aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.024+0000 7f39c3a01640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39bc11b140 0x7f39bc116b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.025+0000 7f39c3a01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39bc117050 0x7f39bc1174d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.025+0000 7f39c3a01640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc113320 con 0x7f39bc109450 2024-09-11T23:47:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.025+0000 7f39c3a01640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1131a0 con 0x7f39bc117050 2024-09-11T23:47:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.025+0000 7f39c3a01640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1134a0 con 0x7f39bc11b140 2024-09-11T23:47:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.025+0000 7f39c1f9e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39bc117050 0x7f39bc1174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.025+0000 7f39c1f9e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39bc117050 0x7f39bc1174d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:39840/0 (socket says 172.21.15.133:39840) 2024-09-11T23:47:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.025+0000 7f39c1f9e640 1 -- 172.21.15.133:0/2945910794 learned_addr learned my addr 172.21.15.133:0/2945910794 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.025+0000 7f39c0f9c640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39bc11b140 0x7f39bc116b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:45.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.026+0000 7f39c1f9e640 1 -- 172.21.15.133:0/2945910794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39bc11b140 msgr2=0x7f39bc116b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:45.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.026+0000 7f39c1f9e640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39bc11b140 0x7f39bc116b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:45.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.026+0000 7f39c1f9e640 1 -- 172.21.15.133:0/2945910794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc109450 msgr2=0x7f39bc11aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:45.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.026+0000 7f39c1f9e640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc109450 0x7f39bc11aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:45.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.026+0000 7f39c1f9e640 1 -- 172.21.15.133:0/2945910794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc1ab360 con 0x7f39bc117050 2024-09-11T23:47:45.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.026+0000 7f39c1f9e640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39bc117050 0x7f39bc1174d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f39b400caf0 tx=0x7f39b400cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:45.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.027+0000 7f39b27fc640 1 -- 172.21.15.133:0/2945910794 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4016030 con 0x7f39bc117050 2024-09-11T23:47:45.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.028+0000 7f39b27fc640 1 -- 172.21.15.133:0/2945910794 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b4003970 con 0x7f39bc117050 2024-09-11T23:47:45.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.028+0000 7f39c3a01640 1 -- 172.21.15.133:0/2945910794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc1ab650 con 0x7f39bc117050 2024-09-11T23:47:45.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.028+0000 7f39c3a01640 1 -- 172.21.15.133:0/2945910794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc1abb90 con 0x7f39bc117050 2024-09-11T23:47:45.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.028+0000 7f39b27fc640 1 -- 172.21.15.133:0/2945910794 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4004650 con 0x7f39bc117050 2024-09-11T23:47:45.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.032+0000 7f39b27fc640 1 -- 172.21.15.133:0/2945910794 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f39b4019070 con 0x7f39bc117050 2024-09-11T23:47:45.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.035+0000 7f39c3a01640 1 -- 172.21.15.133:0/2945910794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a00052c0 con 0x7f39bc117050 2024-09-11T23:47:45.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.036+0000 7f39b27fc640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f39a80a2f00 0x7f39a80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:45.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.037+0000 7f39b27fc640 1 -- 172.21.15.133:0/2945910794 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f39b40cfec0 con 0x7f39bc117050 2024-09-11T23:47:45.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.042+0000 7f39b27fc640 1 -- 172.21.15.133:0/2945910794 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39b409cbd0 con 0x7f39bc117050 2024-09-11T23:47:45.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.044+0000 7f39c179d640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f39a80a2f00 0x7f39a80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:45.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.055+0000 7f39c179d640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f39a80a2f00 0x7f39a80a53c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f39b80058c0 tx=0x7f39b8005850 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:45.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:44 smithi138 ceph-mon[35142]: pgmap v342: 97 pgs: 97 active+clean; 66 GiB data, 247 GiB used, 826 GiB / 1.0 TiB avail; 966 KiB/s rd, 253 MiB/s wr, 2.04k op/s 2024-09-11T23:47:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:44 smithi178 ceph-mon[35882]: pgmap v342: 97 pgs: 97 active+clean; 66 GiB data, 247 GiB used, 826 GiB / 1.0 TiB avail; 966 KiB/s rd, 253 MiB/s wr, 2.04k op/s 2024-09-11T23:47:45.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.508+0000 7f39c3a01640 1 -- 172.21.15.133:0/2945910794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39a0005880 con 0x7f39bc117050 2024-09-11T23:47:45.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.509+0000 7f39b27fc640 1 -- 172.21.15.133:0/2945910794 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11994 (secure 0 0 0) 0x7f39b409c4d0 con 0x7f39bc117050 2024-09-11T23:47:45.509 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:47:45.511 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"btime":"2024-09-11T23:42:10:734602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:42:10.734407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26357":{"gid":26357,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/3004728174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3004728174},{"type":"v1","addr":"172.21.15.178:6835","nonce":3004728174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:47:45.513 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2024-09-11T23:47:45.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.516+0000 7f39c3a01640 1 -- 172.21.15.133:0/2945910794 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f39a80a2f00 msgr2=0x7f39a80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:45.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.516+0000 7f39c3a01640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f39a80a2f00 0x7f39a80a53c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f39b80058c0 tx=0x7f39b8005850 comp rx=0 tx=0).stop 2024-09-11T23:47:45.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.516+0000 7f39c3a01640 1 -- 172.21.15.133:0/2945910794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39bc117050 msgr2=0x7f39bc1174d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:45.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.516+0000 7f39c3a01640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39bc117050 0x7f39bc1174d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f39b400caf0 tx=0x7f39b400cef0 comp rx=0 tx=0).stop 2024-09-11T23:47:45.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.516+0000 7f39c3a01640 1 -- 172.21.15.133:0/2945910794 shutdown_connections 2024-09-11T23:47:45.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.516+0000 7f39c3a01640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f39a80a2f00 0x7f39a80a53c0 secure :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f39b80058c0 tx=0x7f39b8005850 comp rx=0 tx=0).stop 2024-09-11T23:47:45.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.516+0000 7f39c3a01640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39bc117050 0x7f39bc1174d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:45.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.516+0000 7f39c3a01640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39bc11b140 0x7f39bc116b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:45.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.516+0000 7f39c3a01640 1 --2- 172.21.15.133:0/2945910794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc109450 0x7f39bc11aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:45.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.516+0000 7f39c3a01640 1 -- 172.21.15.133:0/2945910794 >> 172.21.15.133:0/2945910794 conn(0x7f39bc06e140 msgr2=0x7f39bc070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:45.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.517+0000 7f39c3a01640 1 -- 172.21.15.133:0/2945910794 shutdown_connections 2024-09-11T23:47:45.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:45.517+0000 7f39c3a01640 1 -- 172.21.15.133:0/2945910794 wait complete. 2024-09-11T23:47:46.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:45 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2945910794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:46.213 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:45 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2945910794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:46.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:45 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2945910794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:47:46.389 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:47:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.771+0000 7feeb77fe640 1 -- 172.21.15.133:0/3317776252 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55638e400d70 con 0x55638e51a870 2024-09-11T23:47:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.771+0000 7feeb77fe640 10 client.27749 handle_command_reply: tid=0 2024-09-11T23:47:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.771+0000 7feedd811740 1 client.27749 shutdown 2024-09-11T23:47:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.773+0000 7feedd811740 1 -- 172.21.15.133:0/3317776252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 msgr2=0x55638e4a2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.773+0000 7feedd811740 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e4a2de0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ba20 tx=0x7feecc09ad20 comp rx=0 tx=0).stop 2024-09-11T23:47:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.773+0000 7feedd811740 1 -- 172.21.15.133:0/3317776252 shutdown_connections 2024-09-11T23:47:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.773+0000 7feedd811740 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55638e51a870 0x55638e51cc90 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.773+0000 7feedd811740 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55638e49fcb0 0x55638e4a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.773+0000 7feedd811740 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638e495410 0x55638e4a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.773+0000 7feedd811740 1 --2- 172.21.15.133:0/3317776252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55638e494af0 0x55638e4a2de0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.773+0000 7feeb77fe640 0 client.27749 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:47:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.773+0000 7feedd811740 1 -- 172.21.15.133:0/3317776252 >> 172.21.15.133:0/3317776252 conn(0x55638e491060 msgr2=0x55638e49b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.774+0000 7feedd811740 1 -- 172.21.15.133:0/3317776252 shutdown_connections 2024-09-11T23:47:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.774+0000 7feedd811740 1 -- 172.21.15.133:0/3317776252 wait complete. 2024-09-11T23:47:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.774+0000 7feedd811740 20 client.27749 trim_cache size 0 max 16384 2024-09-11T23:47:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:47:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:47:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:47:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:47:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.775+0000 7feeda998640 1 -- 172.21.15.133:0/3676506410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feebc0a2de0 msgr2=0x7feebc0a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.775+0000 7feeda998640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feebc0a2de0 0x7feebc0a52a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feec400e780 tx=0x7feec4005000 comp rx=0 tx=0).stop 2024-09-11T23:47:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.775+0000 7feeda998640 1 -- 172.21.15.133:0/3676506410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41091f0 msgr2=0x7feed4118820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.776+0000 7feeda998640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41091f0 0x7feed4118820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ba50 tx=0x7feecc00bf10 comp rx=0 tx=0).stop 2024-09-11T23:47:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.776+0000 7feeda998640 1 -- 172.21.15.133:0/3676506410 shutdown_connections 2024-09-11T23:47:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.776+0000 7feeda998640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feebc0a2de0 0x7feebc0a52a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.776+0000 7feeda998640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feed410e7f0 0x7feed41192a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.776+0000 7feeda998640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feed4109b10 0x7feed4118d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.776+0000 7feeda998640 1 --2- 172.21.15.133:0/3676506410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed41091f0 0x7feed4118820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.776+0000 7feeda998640 1 -- 172.21.15.133:0/3676506410 >> 172.21.15.133:0/3676506410 conn(0x7feed406d210 msgr2=0x7feed410c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.777+0000 7feeda998640 1 -- 172.21.15.133:0/3676506410 shutdown_connections 2024-09-11T23:47:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:47:46.777+0000 7feeda998640 1 -- 172.21.15.133:0/3676506410 wait complete. 2024-09-11T23:47:47.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:46 smithi178 ceph-mon[35882]: pgmap v343: 97 pgs: 97 active+clean; 66 GiB data, 249 GiB used, 824 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 260 MiB/s wr, 1.95k op/s 2024-09-11T23:47:47.329 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:47.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:47 smithi138 ceph-mon[35142]: pgmap v343: 97 pgs: 97 active+clean; 66 GiB data, 249 GiB used, 824 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 260 MiB/s wr, 1.95k op/s 2024-09-11T23:47:47.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:46 smithi133 ceph-mon[31430]: pgmap v343: 97 pgs: 97 active+clean; 66 GiB data, 249 GiB used, 824 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 260 MiB/s wr, 1.95k op/s 2024-09-11T23:47:48.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.205+0000 7fb69521d640 1 -- 172.21.15.133:0/2375467375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb69011b140 msgr2=0x7fb69011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:48.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.205+0000 7fb69521d640 1 --2- 172.21.15.133:0/2375467375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb69011b140 0x7fb69011d530 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb680009900 tx=0x7fb68002a410 comp rx=0 tx=0).stop 2024-09-11T23:47:48.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.207+0000 7fb69521d640 1 -- 172.21.15.133:0/2375467375 shutdown_connections 2024-09-11T23:47:48.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.207+0000 7fb69521d640 1 --2- 172.21.15.133:0/2375467375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb69011b140 0x7fb69011d530 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.207+0000 7fb69521d640 1 --2- 172.21.15.133:0/2375467375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb69006ea30 0x7fb69006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.207+0000 7fb69521d640 1 --2- 172.21.15.133:0/2375467375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb690110190 0x7fb69006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.207+0000 7fb69521d640 1 -- 172.21.15.133:0/2375467375 >> 172.21.15.133:0/2375467375 conn(0x7fb690073180 msgr2=0x7fb690073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.208+0000 7fb69521d640 1 -- 172.21.15.133:0/2375467375 shutdown_connections 2024-09-11T23:47:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.208+0000 7fb69521d640 1 -- 172.21.15.133:0/2375467375 wait complete. 2024-09-11T23:47:48.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.210+0000 7fb69521d640 1 Processor -- start 2024-09-11T23:47:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.210+0000 7fb69521d640 1 -- start start 2024-09-11T23:47:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.211+0000 7fb69521d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb69006ea30 0x7fb690114420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.211+0000 7fb69521d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690110190 0x7fb690114960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.211+0000 7fb69521d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb69011b140 0x7fb690118d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.211+0000 7fb69521d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb690110cb0 con 0x7fb690110190 2024-09-11T23:47:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.211+0000 7fb69521d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb690110b30 con 0x7fb69006ea30 2024-09-11T23:47:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.211+0000 7fb69521d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb690110e30 con 0x7fb69011b140 2024-09-11T23:47:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.212+0000 7fb694a1c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb69011b140 0x7fb690118d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.212+0000 7fb694a1c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb69011b140 0x7fb690118d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:56180/0 (socket says 172.21.15.133:56180) 2024-09-11T23:47:48.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.212+0000 7fb694a1c640 1 -- 172.21.15.133:0/2016500632 learned_addr learned my addr 172.21.15.133:0/2016500632 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.212+0000 7fb694a1c640 1 -- 172.21.15.133:0/2016500632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb69006ea30 msgr2=0x7fb690114420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.212+0000 7fb694a1c640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb69006ea30 0x7fb690114420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.212+0000 7fb694a1c640 1 -- 172.21.15.133:0/2016500632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690110190 msgr2=0x7fb690114960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.212+0000 7fb694a1c640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690110190 0x7fb690114960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.213+0000 7fb694a1c640 1 -- 172.21.15.133:0/2016500632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6901192d0 con 0x7fb69011b140 2024-09-11T23:47:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.213+0000 7fb694a1c640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb69011b140 0x7fb690118d10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb68002a930 tx=0x7fb680003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:48.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.213+0000 7fb68d7fa640 1 -- 172.21.15.133:0/2016500632 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68003f070 con 0x7fb69011b140 2024-09-11T23:47:48.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.214+0000 7fb69521d640 1 -- 172.21.15.133:0/2016500632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb690119560 con 0x7fb69011b140 2024-09-11T23:47:48.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.214+0000 7fb69521d640 1 -- 172.21.15.133:0/2016500632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb690119a40 con 0x7fb69011b140 2024-09-11T23:47:48.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.219+0000 7fb698858740 1 Processor -- start 2024-09-11T23:47:48.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.219+0000 7fb68d7fa640 1 -- 172.21.15.133:0/2016500632 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb680031c90 con 0x7fb69011b140 2024-09-11T23:47:48.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.219+0000 7fb68d7fa640 1 -- 172.21.15.133:0/2016500632 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680035030 con 0x7fb69011b140 2024-09-11T23:47:48.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.219+0000 7fb698858740 1 -- start start 2024-09-11T23:47:48.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.220+0000 7fb698858740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f33cf0 0x55f156f32240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.220+0000 7fb698858740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f32780 0x55f156f32bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.220+0000 7fb698858740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 0x55f156f3f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.220+0000 7fb698858740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f156e982f0 con 0x55f156f3d520 2024-09-11T23:47:48.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.220+0000 7fb698858740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f156e3fcf0 con 0x55f156f32780 2024-09-11T23:47:48.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.220+0000 7fb698858740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f156e942b0 con 0x55f156f33cf0 2024-09-11T23:47:48.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.221+0000 7fb68ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f32780 0x55f156f32bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.221+0000 7fb68ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f32780 0x55f156f32bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:39862/0 (socket says 172.21.15.133:39862) 2024-09-11T23:47:48.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.221+0000 7fb68ffff640 1 -- 172.21.15.133:0/2816165859 learned_addr learned my addr 172.21.15.133:0/2816165859 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:48.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.221+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/2816165859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f33cf0 0x55f156f32240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.223+0000 7fb68d7fa640 1 -- 172.21.15.133:0/2016500632 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb680042070 con 0x7fb69011b140 2024-09-11T23:47:48.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.224+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/2816165859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 0x55f156f3f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.224+0000 7fb68ffff640 1 -- 172.21.15.133:0/2816165859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f33cf0 msgr2=0x55f156f32240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:48.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.224+0000 7fb68ffff640 1 --2- 172.21.15.133:0/2816165859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f33cf0 0x55f156f32240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.224+0000 7fb68ffff640 1 -- 172.21.15.133:0/2816165859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 msgr2=0x55f156f3f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:48.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.224+0000 7fb68ffff640 1 --2- 172.21.15.133:0/2816165859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 0x55f156f3f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.224+0000 7fb68ffff640 1 -- 172.21.15.133:0/2816165859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f156e243a0 con 0x55f156f32780 2024-09-11T23:47:48.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.224+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/2816165859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 0x55f156f3f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:47:48.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.224+0000 7fb68d7fa640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6780a2ec0 0x7fb6780a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.225+0000 7fb68d7fa640 1 -- 172.21.15.133:0/2016500632 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fb6800f0a40 con 0x7fb69011b140 2024-09-11T23:47:48.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.227+0000 7fb68ffff640 1 --2- 172.21.15.133:0/2816165859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f32780 0x55f156f32bc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb688009e20 tx=0x7fb68800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f45bb5ff90b0df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:48.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.229+0000 7fb6767fc640 1 -- 172.21.15.133:0/2816165859 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688004d20 con 0x55f156f32780 2024-09-11T23:47:48.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.229+0000 7fb6767fc640 1 -- 172.21.15.133:0/2816165859 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb688005500 con 0x55f156f32780 2024-09-11T23:47:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.231+0000 7fb698858740 1 -- 172.21.15.133:0/2816165859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f32780 msgr2=0x55f156f32bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.231+0000 7fb698858740 1 --2- 172.21.15.133:0/2816165859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f32780 0x55f156f32bc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb688009e20 tx=0x7fb68800e720 comp rx=0 tx=0).stop 2024-09-11T23:47:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.231+0000 7fb698858740 1 -- 172.21.15.133:0/2816165859 shutdown_connections 2024-09-11T23:47:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.231+0000 7fb698858740 1 --2- 172.21.15.133:0/2816165859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 0x55f156f3f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.231+0000 7fb698858740 1 --2- 172.21.15.133:0/2816165859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f32780 0x55f156f32bc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.231+0000 7fb698858740 1 --2- 172.21.15.133:0/2816165859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f33cf0 0x55f156f32240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.231+0000 7fb698858740 1 -- 172.21.15.133:0/2816165859 >> 172.21.15.133:0/2816165859 conn(0x55f156efb310 msgr2=0x55f156f2ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.236+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6780a2ec0 0x7fb6780a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.237+0000 7fb698858740 1 -- 172.21.15.133:0/2816165859 shutdown_connections 2024-09-11T23:47:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.238+0000 7fb698858740 1 -- 172.21.15.133:0/2816165859 wait complete. 2024-09-11T23:47:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.238+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6780a2ec0 0x7fb6780a5380 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55f156f33220 tx=0x7fb684005850 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.239+0000 7fb698858740 1 -- start start 2024-09-11T23:47:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.239+0000 7fb698858740 1 Processor -- start 2024-09-11T23:47:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.240+0000 7fb698858740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f33cf0 0x55f156f3cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.240+0000 7fb698858740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f3d520 0x55f156f39ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.240+0000 7fb698858740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f3a430 0x55f156f3a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.240+0000 7fb698858740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f156f402c0 con 0x55f156f33cf0 2024-09-11T23:47:48.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.240+0000 7fb698858740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f156e53060 con 0x55f156f3d520 2024-09-11T23:47:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.240+0000 7fb698858740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f156f40440 con 0x55f156f3a430 2024-09-11T23:47:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.240+0000 7fb698858740 10 client.0 _ll_register_callbacks cb 0x55f156ba68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.242+0000 7fb68ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f33cf0 0x55f156f3cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.242+0000 7fb68ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f33cf0 0x55f156f3cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58592/0 (socket says 172.21.15.133:58592) 2024-09-11T23:47:48.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.242+0000 7fb68ffff640 1 -- 172.21.15.133:0/3430165005 learned_addr learned my addr 172.21.15.133:0/3430165005 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:48.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.244+0000 7fb694a1c640 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f3a430 0x55f156f3a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.244+0000 7fb68ffff640 1 -- 172.21.15.133:0/3430165005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f3a430 msgr2=0x55f156f3a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:48.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.244+0000 7fb68ffff640 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f3a430 0x55f156f3a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.244+0000 7fb68ffff640 1 -- 172.21.15.133:0/3430165005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f3d520 msgr2=0x55f156f39ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:47:48.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.244+0000 7fb68ffff640 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f3d520 0x55f156f39ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.244+0000 7fb68ffff640 1 -- 172.21.15.133:0/3430165005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f156c093c0 con 0x55f156f33cf0 2024-09-11T23:47:48.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.246+0000 7fb68ffff640 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f33cf0 0x55f156f3cc40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb688009c30 tx=0x7fb68800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197f64d48f6c3ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:48.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.246+0000 7fb66f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:48.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.248+0000 7fb66f7fe640 1 -- 172.21.15.133:0/3430165005 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688027060 con 0x55f156f33cf0 2024-09-11T23:47:48.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.248+0000 7fb66f7fe640 1 -- 172.21.15.133:0/3430165005 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fb688003b80 con 0x55f156f33cf0 2024-09-11T23:47:48.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.250+0000 7fb698858740 1 -- 172.21.15.133:0/3430165005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f156ea5860 con 0x55f156f33cf0 2024-09-11T23:47:48.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.254+0000 7fb66f7fe640 1 -- 172.21.15.133:0/3430165005 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb6880185b0 con 0x55f156f33cf0 2024-09-11T23:47:48.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.256+0000 7fb698858740 10 client.18216 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:48.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.256+0000 7fb698858740 1 -- 172.21.15.133:0/3430165005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f156f4bf40 con 0x55f156f33cf0 2024-09-11T23:47:48.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.258+0000 7fb66f7fe640 1 -- 172.21.15.133:0/3430165005 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f156ea5860 con 0x55f156f33cf0 2024-09-11T23:47:48.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.258+0000 7fb66f7fe640 1 -- 172.21.15.133:0/3430165005 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb688005200 con 0x55f156f33cf0 2024-09-11T23:47:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.259+0000 7fb698858740 10 client.18216 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.259+0000 7fb698858740 10 client.18216 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:47:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.259+0000 7fb698858740 20 client.18216 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.259+0000 7fb698858740 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f156f4d340 0x55f156f4f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.259+0000 7fb698858740 4 client.18216 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.259+0000 7fb698858740 1 -- 172.21.15.133:0/3430165005 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f156e9fd70 con 0x55f156f4d340 2024-09-11T23:47:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.266+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f156f4d340 0x55f156f4f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.271+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f156f4d340 0x55f156f4f730 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ab3f4a04e343b88 server_cookie=c7caa2bff1dea8ed in_seq=0 out_seq=0 2024-09-11T23:47:48.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.271+0000 7fb66f7fe640 10 client.18216 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:47:48.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.277+0000 7fb66f7fe640 1 -- 172.21.15.133:0/3430165005 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f156e9fd70 con 0x55f156f4d340 2024-09-11T23:47:48.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.277+0000 7fb66f7fe640 10 client.18216 handle_command_reply: tid=0 2024-09-11T23:47:48.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.277+0000 7fb698858740 1 client.18216 shutdown 2024-09-11T23:47:48.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.282+0000 7fb698858740 1 -- 172.21.15.133:0/3430165005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f33cf0 msgr2=0x55f156f3cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:48.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.282+0000 7fb698858740 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f33cf0 0x55f156f3cc40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb688009c30 tx=0x7fb68800eef0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.283+0000 7fb698858740 1 -- 172.21.15.133:0/3430165005 shutdown_connections 2024-09-11T23:47:48.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.283+0000 7fb698858740 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f156f4d340 0x55f156f4f730 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.283+0000 7fb698858740 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f3a430 0x55f156f3a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.283+0000 7fb698858740 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f3d520 0x55f156f39ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.283+0000 7fb698858740 1 --2- 172.21.15.133:0/3430165005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f33cf0 0x55f156f3cc40 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.283+0000 7fb698858740 1 -- 172.21.15.133:0/3430165005 >> 172.21.15.133:0/3430165005 conn(0x55f156efb310 msgr2=0x55f156f34b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:48.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.284+0000 7fb66f7fe640 0 client.18216 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:47:48.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.288+0000 7fb698858740 1 -- 172.21.15.133:0/3430165005 shutdown_connections 2024-09-11T23:47:48.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.288+0000 7fb698858740 1 -- 172.21.15.133:0/3430165005 wait complete. 2024-09-11T23:47:48.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.291+0000 7fb698858740 20 client.18216 trim_cache size 0 max 16384 2024-09-11T23:47:48.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.335+0000 7fb698858740 1 Processor -- start 2024-09-11T23:47:48.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.339+0000 7fb698858740 1 -- start start 2024-09-11T23:47:48.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.339+0000 7fb698858740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f33cf0 0x55f156f2efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.339+0000 7fb698858740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3a430 0x55f156f35170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.339+0000 7fb698858740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f3d520 0x55f156f356b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.339+0000 7fb698858740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c00e7d0 con 0x55f156f3a430 2024-09-11T23:47:48.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.339+0000 7fb698858740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c010310 con 0x55f156f33cf0 2024-09-11T23:47:48.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.339+0000 7fb698858740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c00eea0 con 0x55f156f3d520 2024-09-11T23:47:48.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.342+0000 7fb68f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3a430 0x55f156f35170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.342+0000 7fb68f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3a430 0x55f156f35170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58604/0 (socket says 172.21.15.133:58604) 2024-09-11T23:47:48.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.342+0000 7fb68f7fe640 1 -- 172.21.15.133:0/968671746 learned_addr learned my addr 172.21.15.133:0/968671746 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:48.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.343+0000 7fb694a1c640 1 --2- 172.21.15.133:0/968671746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f3d520 0x55f156f356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.345+0000 7fb68f7fe640 1 -- 172.21.15.133:0/968671746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f3d520 msgr2=0x55f156f356b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:48.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.345+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/968671746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f3d520 0x55f156f356b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.345+0000 7fb68f7fe640 1 -- 172.21.15.133:0/968671746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f33cf0 msgr2=0x55f156f2efc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:47:48.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.345+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/968671746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f33cf0 0x55f156f2efc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.345+0000 7fb68f7fe640 1 -- 172.21.15.133:0/968671746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f156f2fb10 con 0x55f156f3a430 2024-09-11T23:47:48.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.347+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/968671746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3a430 0x55f156f35170 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb684005fd0 tx=0x7fb684013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b094b8ea259ff3f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:48.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.348+0000 7fb675ffb640 1 -- 172.21.15.133:0/968671746 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68402b030 con 0x55f156f3a430 2024-09-11T23:47:48.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.348+0000 7fb675ffb640 1 -- 172.21.15.133:0/968671746 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb684026330 con 0x55f156f3a430 2024-09-11T23:47:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.348+0000 7fb675ffb640 1 -- 172.21.15.133:0/968671746 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684017070 con 0x55f156f3a430 2024-09-11T23:47:48.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.350+0000 7fb698858740 1 -- 172.21.15.133:0/968671746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3a430 msgr2=0x55f156f35170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:48.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.350+0000 7fb698858740 1 --2- 172.21.15.133:0/968671746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3a430 0x55f156f35170 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb684005fd0 tx=0x7fb684013000 comp rx=0 tx=0).stop 2024-09-11T23:47:48.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.350+0000 7fb698858740 1 -- 172.21.15.133:0/968671746 shutdown_connections 2024-09-11T23:47:48.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.350+0000 7fb698858740 1 --2- 172.21.15.133:0/968671746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f3d520 0x55f156f356b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.350+0000 7fb698858740 1 --2- 172.21.15.133:0/968671746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3a430 0x55f156f35170 secure :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb684005fd0 tx=0x7fb684013000 comp rx=0 tx=0).stop 2024-09-11T23:47:48.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.350+0000 7fb698858740 1 --2- 172.21.15.133:0/968671746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f33cf0 0x55f156f2efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.350+0000 7fb698858740 1 -- 172.21.15.133:0/968671746 >> 172.21.15.133:0/968671746 conn(0x55f156efb310 msgr2=0x55f156f2e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:48.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.351+0000 7fb698858740 1 -- 172.21.15.133:0/968671746 shutdown_connections 2024-09-11T23:47:48.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.353+0000 7fb698858740 1 -- 172.21.15.133:0/968671746 wait complete. 2024-09-11T23:47:48.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.357+0000 7fb698858740 1 -- start start 2024-09-11T23:47:48.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.359+0000 7fb698858740 1 Processor -- start 2024-09-11T23:47:48.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.360+0000 7fb698858740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f33cf0 0x55f156f40770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.360+0000 7fb698858740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 0x55f156f40cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.360+0000 7fb698858740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f411f0 0x55f156fb5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.360+0000 7fb698858740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c00fc40 con 0x55f156f3d520 2024-09-11T23:47:48.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.360+0000 7fb698858740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c00f570 con 0x55f156f411f0 2024-09-11T23:47:48.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.360+0000 7fb698858740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c00c7d0 con 0x55f156f33cf0 2024-09-11T23:47:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.360+0000 7fb698858740 10 client.0 _ll_register_callbacks cb 0x7fb688009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.361+0000 7fb68ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f33cf0 0x55f156f40770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.361+0000 7fb694a1c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f411f0 0x55f156fb5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.362+0000 7fb68ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f33cf0 0x55f156f40770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:56210/0 (socket says 172.21.15.133:56210) 2024-09-11T23:47:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.362+0000 7fb68ffff640 1 -- 172.21.15.133:0/2537256588 learned_addr learned my addr 172.21.15.133:0/2537256588 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:48.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.366+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 0x55f156f40cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.367+0000 7fb68f7fe640 1 -- 172.21.15.133:0/2537256588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f33cf0 msgr2=0x55f156f40770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:48.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.367+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f33cf0 0x55f156f40770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.367+0000 7fb68f7fe640 1 -- 172.21.15.133:0/2537256588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f411f0 msgr2=0x55f156fb5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:48.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.367+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f411f0 0x55f156fb5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.367+0000 7fb68f7fe640 1 -- 172.21.15.133:0/2537256588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f156fb63f0 con 0x55f156f3d520 2024-09-11T23:47:48.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.368+0000 7fb68f7fe640 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 0x55f156f40cb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55f156f3b570 tx=0x7fb684011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b941ab0829d331 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:48.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.368+0000 7fb66f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:48.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.370+0000 7fb66f7fe640 1 -- 172.21.15.133:0/2537256588 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684036020 con 0x55f156f3d520 2024-09-11T23:47:48.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.370+0000 7fb698858740 1 -- 172.21.15.133:0/2537256588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb684026330 con 0x55f156f3d520 2024-09-11T23:47:48.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.370+0000 7fb66f7fe640 1 -- 172.21.15.133:0/2537256588 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fb6840264e0 con 0x55f156f3d520 2024-09-11T23:47:48.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.372+0000 7fb66f7fe640 1 -- 172.21.15.133:0/2537256588 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb684041050 con 0x55f156f3d520 2024-09-11T23:47:48.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.378+0000 7fb698858740 10 client.18222 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:48.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.378+0000 7fb698858740 1 -- 172.21.15.133:0/2537256588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f156fb74f0 con 0x55f156f3d520 2024-09-11T23:47:48.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.382+0000 7fb66f7fe640 1 -- 172.21.15.133:0/2537256588 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb684026330 con 0x55f156f3d520 2024-09-11T23:47:48.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.382+0000 7fb66f7fe640 1 -- 172.21.15.133:0/2537256588 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb68401dcc0 con 0x55f156f3d520 2024-09-11T23:47:48.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:48 smithi133 ceph-mon[31430]: pgmap v344: 97 pgs: 97 active+clean; 67 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 301 MiB/s wr, 2.02k op/s 2024-09-11T23:47:48.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.385+0000 7fb698858740 10 client.18222 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:48.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.385+0000 7fb698858740 10 client.18222 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:47:48.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.385+0000 7fb698858740 20 client.18222 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:48.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.385+0000 7fb698858740 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f156fb7b40 0x55f156fb9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:48.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.385+0000 7fb698858740 4 client.18222 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:47:48.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.385+0000 7fb698858740 1 -- 172.21.15.133:0/2537256588 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f156e1c170 con 0x55f156fb7b40 2024-09-11T23:47:48.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.386+0000 7fb68ffff640 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f156fb7b40 0x55f156fb9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.395+0000 7fb68ffff640 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f156fb7b40 0x55f156fb9f60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=64c74ddd1e764880 server_cookie=b567bb7f04f0f6fa in_seq=0 out_seq=0 2024-09-11T23:47:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.396+0000 7fb66f7fe640 10 client.18222 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:47:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.396+0000 7fb66f7fe640 1 -- 172.21.15.133:0/2537256588 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f156e1c170 con 0x55f156fb7b40 2024-09-11T23:47:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.396+0000 7fb66f7fe640 10 client.18222 handle_command_reply: tid=0 2024-09-11T23:47:48.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.396+0000 7fb698858740 1 client.18222 shutdown 2024-09-11T23:47:48.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.397+0000 7fb698858740 1 -- 172.21.15.133:0/2537256588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 msgr2=0x55f156f40cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:48.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.398+0000 7fb698858740 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 0x55f156f40cb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55f156f3b570 tx=0x7fb684011a50 comp rx=0 tx=0).stop 2024-09-11T23:47:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.398+0000 7fb698858740 1 -- 172.21.15.133:0/2537256588 shutdown_connections 2024-09-11T23:47:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.398+0000 7fb698858740 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55f156fb7b40 0x55f156fb9f60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.398+0000 7fb698858740 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f156f411f0 0x55f156fb5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.398+0000 7fb698858740 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f156f3d520 0x55f156f40cb0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.398+0000 7fb698858740 1 --2- 172.21.15.133:0/2537256588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f156f33cf0 0x55f156f40770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.399+0000 7fb698858740 1 -- 172.21.15.133:0/2537256588 >> 172.21.15.133:0/2537256588 conn(0x55f156efb310 msgr2=0x55f156f3b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.399+0000 7fb66f7fe640 0 client.18222 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:47:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.399+0000 7fb698858740 1 -- 172.21.15.133:0/2537256588 shutdown_connections 2024-09-11T23:47:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.399+0000 7fb698858740 1 -- 172.21.15.133:0/2537256588 wait complete. 2024-09-11T23:47:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.400+0000 7fb698858740 20 client.18222 trim_cache size 0 max 16384 2024-09-11T23:47:48.404 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.404+0000 7fb69521d640 1 -- 172.21.15.133:0/2016500632 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6780a2ec0 msgr2=0x7fb6780a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.404+0000 7fb69521d640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6780a2ec0 0x7fb6780a5380 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55f156f33220 tx=0x7fb684005850 comp rx=0 tx=0).stop 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.404+0000 7fb69521d640 1 -- 172.21.15.133:0/2016500632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb69011b140 msgr2=0x7fb690118d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.404+0000 7fb69521d640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb69011b140 0x7fb690118d10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb68002a930 tx=0x7fb680003b90 comp rx=0 tx=0).stop 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.405+0000 7fb69521d640 1 -- 172.21.15.133:0/2016500632 shutdown_connections 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.405+0000 7fb69521d640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb6780a2ec0 0x7fb6780a5380 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.405+0000 7fb69521d640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb69011b140 0x7fb690118d10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.405+0000 7fb69521d640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690110190 0x7fb690114960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.405+0000 7fb69521d640 1 --2- 172.21.15.133:0/2016500632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb69006ea30 0x7fb690114420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.405+0000 7fb69521d640 1 -- 172.21.15.133:0/2016500632 >> 172.21.15.133:0/2016500632 conn(0x7fb690073180 msgr2=0x7fb69011b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.405+0000 7fb69521d640 1 -- 172.21.15.133:0/2016500632 shutdown_connections 2024-09-11T23:47:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:48.406+0000 7fb69521d640 1 -- 172.21.15.133:0/2016500632 wait complete. 2024-09-11T23:47:48.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:48 smithi138 ceph-mon[35142]: pgmap v344: 97 pgs: 97 active+clean; 67 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 301 MiB/s wr, 2.02k op/s 2024-09-11T23:47:48.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:48 smithi178 ceph-mon[35882]: pgmap v344: 97 pgs: 97 active+clean; 67 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 301 MiB/s wr, 2.02k op/s 2024-09-11T23:47:49.512 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:47:50.281 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:51.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:50 smithi133 ceph-mon[31430]: pgmap v345: 97 pgs: 97 active+clean; 68 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 487 MiB/s wr, 2.33k op/s 2024-09-11T23:47:51.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.172+0000 7f57f8596640 1 -- 172.21.15.133:0/4044030805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57f0103670 msgr2=0x7f57f010bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:51.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.172+0000 7f57f8596640 1 --2- 172.21.15.133:0/4044030805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57f0103670 0x7f57f010bb60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009900 tx=0x7f57e002a480 comp rx=0 tx=0).stop 2024-09-11T23:47:51.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.172+0000 7f57f8596640 1 -- 172.21.15.133:0/4044030805 shutdown_connections 2024-09-11T23:47:51.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.172+0000 7f57f8596640 1 --2- 172.21.15.133:0/4044030805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57f010c180 0x7f57f010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.172+0000 7f57f8596640 1 --2- 172.21.15.133:0/4044030805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57f0103670 0x7f57f010bb60 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.172+0000 7f57f8596640 1 --2- 172.21.15.133:0/4044030805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57f0102d50 0x7f57f0103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.172+0000 7f57f8596640 1 -- 172.21.15.133:0/4044030805 >> 172.21.15.133:0/4044030805 conn(0x7f57f00fc7d0 msgr2=0x7f57f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:51.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.172+0000 7f57f8596640 1 -- 172.21.15.133:0/4044030805 shutdown_connections 2024-09-11T23:47:51.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.172+0000 7f57f8596640 1 -- 172.21.15.133:0/4044030805 wait complete. 2024-09-11T23:47:51.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f8596640 1 Processor -- start 2024-09-11T23:47:51.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f8596640 1 -- start start 2024-09-11T23:47:51.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f8596640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57f0102d50 0x7f57f0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f8596640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57f0103670 0x7f57f0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f8596640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57f010c180 0x7f57f019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f8596640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57f0111f80 con 0x7f57f0103670 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f8596640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57f0111e00 con 0x7f57f010c180 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f8596640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57f0112100 con 0x7f57f0102d50 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f5b31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57f0103670 0x7f57f0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f5b31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57f0103670 0x7f57f0198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58616/0 (socket says 172.21.15.133:58616) 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f5b31640 1 -- 172.21.15.133:0/2424769316 learned_addr learned my addr 172.21.15.133:0/2424769316 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f6332640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57f0102d50 0x7f57f0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f6b33640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57f010c180 0x7f57f019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f5b31640 1 -- 172.21.15.133:0/2424769316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57f0102d50 msgr2=0x7f57f0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f5b31640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57f0102d50 0x7f57f0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f5b31640 1 -- 172.21.15.133:0/2424769316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57f010c180 msgr2=0x7f57f019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f5b31640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57f010c180 0x7f57f019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f5b31640 1 -- 172.21.15.133:0/2424769316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f019d4b0 con 0x7f57f0103670 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.173+0000 7f57f6b33640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57f010c180 0x7f57f019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.174+0000 7f57f5b31640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57f0103670 0x7f57f0198a40 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f57e0006010 tx=0x7f57e0031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.174+0000 7f57e77fe640 1 -- 172.21.15.133:0/2424769316 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e003f030 con 0x7f57f0103670 2024-09-11T23:47:51.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.174+0000 7f57e77fe640 1 -- 172.21.15.133:0/2424769316 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e00043d0 con 0x7f57f0103670 2024-09-11T23:47:51.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.174+0000 7f57f8596640 1 -- 172.21.15.133:0/2424769316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f019d740 con 0x7f57f0103670 2024-09-11T23:47:51.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.174+0000 7f57e77fe640 1 -- 172.21.15.133:0/2424769316 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0035070 con 0x7f57f0103670 2024-09-11T23:47:51.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.174+0000 7f57f8596640 1 -- 172.21.15.133:0/2424769316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f01a5080 con 0x7f57f0103670 2024-09-11T23:47:51.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.177+0000 7f57e77fe640 1 -- 172.21.15.133:0/2424769316 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f57e0042030 con 0x7f57f0103670 2024-09-11T23:47:51.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.177+0000 7f57e77fe640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57dc0a2d70 0x7f57dc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.178+0000 7f57e77fe640 1 -- 172.21.15.133:0/2424769316 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f57e00ef6e0 con 0x7f57f0103670 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.178+0000 7f57f6332640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57dc0a2d70 0x7f57dc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.178+0000 7f57fa96f740 1 Processor -- start 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.178+0000 7f57fa96f740 1 -- start start 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.178+0000 7f57f6332640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57dc0a2d70 0x7f57dc0a5230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009bd0 tx=0x7f57ec005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.178+0000 7f57fa96f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bf5d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.178+0000 7f57fa96f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bf5da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.178+0000 7f57fa96f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf6a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57fa96f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559b5bec32f0 con 0x559b5bf5eb40 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57fa96f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559b5be6acf0 con 0x559b5bf5d5d0 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57fa96f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559b5bebf2b0 con 0x559b5bf68370 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f6332640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bf5d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f6332640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bf5d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58630/0 (socket says 172.21.15.133:58630) 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f6332640 1 -- 172.21.15.133:0/2280107419 learned_addr learned my addr 172.21.15.133:0/2280107419 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:51.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f5b31640 1 --2- 172.21.15.133:0/2280107419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bf5da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f6b33640 1 --2- 172.21.15.133:0/2280107419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf6a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f6332640 1 -- 172.21.15.133:0/2280107419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 msgr2=0x559b5bf6a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:51.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f6332640 1 --2- 172.21.15.133:0/2280107419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf6a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f6332640 1 -- 172.21.15.133:0/2280107419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 msgr2=0x559b5bf5da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:51.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f6332640 1 --2- 172.21.15.133:0/2280107419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bf5da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f6332640 1 -- 172.21.15.133:0/2280107419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b5be4f3a0 con 0x559b5bf5eb40 2024-09-11T23:47:51.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f5b31640 1 --2- 172.21.15.133:0/2280107419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bf5da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.179+0000 7f57f6332640 1 --2- 172.21.15.133:0/2280107419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bf5d090 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00e630 tx=0x7f57ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2990f89d35d12b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.180+0000 7f57e4ff9640 1 -- 172.21.15.133:0/2280107419 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec004020 con 0x559b5bf5eb40 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.180+0000 7f57e4ff9640 1 -- 172.21.15.133:0/2280107419 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57ec0041c0 con 0x559b5bf5eb40 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.180+0000 7f57e4ff9640 1 -- 172.21.15.133:0/2280107419 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec01c7a0 con 0x559b5bf5eb40 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.181+0000 7f57fa96f740 1 -- 172.21.15.133:0/2280107419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 msgr2=0x559b5bf5d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.181+0000 7f57fa96f740 1 --2- 172.21.15.133:0/2280107419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bf5d090 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00e630 tx=0x7f57ec014000 comp rx=0 tx=0).stop 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.181+0000 7f57fa96f740 1 -- 172.21.15.133:0/2280107419 shutdown_connections 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.181+0000 7f57fa96f740 1 --2- 172.21.15.133:0/2280107419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf6a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.181+0000 7f57fa96f740 1 --2- 172.21.15.133:0/2280107419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bf5da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.181+0000 7f57fa96f740 1 --2- 172.21.15.133:0/2280107419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bf5d090 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.181+0000 7f57fa96f740 1 -- 172.21.15.133:0/2280107419 >> 172.21.15.133:0/2280107419 conn(0x559b5bf26310 msgr2=0x559b5bf59c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.181+0000 7f57fa96f740 1 -- 172.21.15.133:0/2280107419 shutdown_connections 2024-09-11T23:47:51.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.181+0000 7f57fa96f740 1 -- 172.21.15.133:0/2280107419 wait complete. 2024-09-11T23:47:51.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.181+0000 7f57fa96f740 1 -- start start 2024-09-11T23:47:51.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57fa96f740 1 Processor -- start 2024-09-11T23:47:51.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57fa96f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bf67dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57fa96f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bf65080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57fa96f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf65810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57fa96f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559b5bf6b070 con 0x559b5bf5eb40 2024-09-11T23:47:51.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57fa96f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559b5be7e060 con 0x559b5bf5d5d0 2024-09-11T23:47:51.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57fa96f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559b5bf6b1f0 con 0x559b5bf68370 2024-09-11T23:47:51.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57fa96f740 10 client.0 _ll_register_callbacks cb 0x559b5bbd18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57f5b31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bf65080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57f6332640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bf67dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57f5b31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bf65080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58636/0 (socket says 172.21.15.133:58636) 2024-09-11T23:47:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57f5b31640 1 -- 172.21.15.133:0/991213998 learned_addr learned my addr 172.21.15.133:0/991213998 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57f6b33640 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf65810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57f6b33640 1 -- 172.21.15.133:0/991213998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 msgr2=0x559b5bf67dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57f6b33640 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bf67dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57f6b33640 1 -- 172.21.15.133:0/991213998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 msgr2=0x559b5bf65080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57f6b33640 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bf65080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.182+0000 7f57f6b33640 1 -- 172.21.15.133:0/991213998 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b5bc343c0 con 0x559b5bf68370 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.183+0000 7f57f6b33640 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf65810 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f57e8005fd0 tx=0x7f57e8012d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17ba85bace59b30c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.183+0000 7f57d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.183+0000 7f57d9ffb640 1 -- 172.21.15.133:0/991213998 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8029050 con 0x559b5bf68370 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.183+0000 7f57fa96f740 1 -- 172.21.15.133:0/991213998 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57ec0041c0 con 0x559b5bf68370 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.183+0000 7f57d9ffb640 1 -- 172.21.15.133:0/991213998 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f57e802d420 con 0x559b5bf68370 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.183+0000 7f57d9ffb640 1 -- 172.21.15.133:0/991213998 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57e8010e80 con 0x559b5bf68370 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.183+0000 7f57fa96f740 10 client.27547 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.183+0000 7f57fa96f740 1 -- 172.21.15.133:0/991213998 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b5bf6b570 con 0x559b5bf68370 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.183+0000 7f57d9ffb640 1 -- 172.21.15.133:0/991213998 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57ec0041c0 con 0x559b5bf68370 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.183+0000 7f57d9ffb640 1 -- 172.21.15.133:0/991213998 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f57e802c730 con 0x559b5bf68370 2024-09-11T23:47:51.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.183+0000 7f57fa96f740 10 client.27547 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.184+0000 7f57fa96f740 10 client.27547 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.184+0000 7f57fa96f740 20 client.27547 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.184+0000 7f57fa96f740 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x559b5bf6c980 0x559b5bf6edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.184+0000 7f57fa96f740 4 client.27547 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.184+0000 7f57fa96f740 1 -- 172.21.15.133:0/991213998 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b5bed0860 con 0x559b5bf6c980 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.184+0000 7f57f6332640 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x559b5bf6c980 0x559b5bf6edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.184+0000 7f57d9ffb640 10 client.27547 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.184+0000 7f57f6332640 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x559b5bf6c980 0x559b5bf6edc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ffdb59a1345f739 server_cookie=614197f0adba399 in_seq=0 out_seq=0 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57d9ffb640 1 -- 172.21.15.133:0/991213998 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b5bed0860 con 0x559b5bf6c980 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57d9ffb640 10 client.27547 handle_command_reply: tid=0 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57fa96f740 1 client.27547 shutdown 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57fa96f740 1 -- 172.21.15.133:0/991213998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 msgr2=0x559b5bf65810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:51.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57fa96f740 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf65810 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f57e8005fd0 tx=0x7f57e8012d70 comp rx=0 tx=0).stop 2024-09-11T23:47:51.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57fa96f740 1 -- 172.21.15.133:0/991213998 shutdown_connections 2024-09-11T23:47:51.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57fa96f740 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x559b5bf6c980 0x559b5bf6edc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57d9ffb640 0 client.27547 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:47:51.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57fa96f740 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf65810 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57fa96f740 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bf65080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57fa96f740 1 --2- 172.21.15.133:0/991213998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bf67dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.185+0000 7f57fa96f740 1 -- 172.21.15.133:0/991213998 >> 172.21.15.133:0/991213998 conn(0x559b5bf26310 msgr2=0x559b5bf5f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:51.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.186+0000 7f57fa96f740 1 -- 172.21.15.133:0/991213998 shutdown_connections 2024-09-11T23:47:51.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.186+0000 7f57fa96f740 1 -- 172.21.15.133:0/991213998 wait complete. 2024-09-11T23:47:51.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.186+0000 7f57fa96f740 20 client.27547 trim_cache size 0 max 16384 2024-09-11T23:47:51.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:50 smithi138 ceph-mon[35142]: pgmap v345: 97 pgs: 97 active+clean; 68 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 487 MiB/s wr, 2.33k op/s 2024-09-11T23:47:51.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57fa96f740 1 Processor -- start 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57fa96f740 1 -- start start 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57fa96f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf5d5d0 0x559b5bf7c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57fa96f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5eb40 0x559b5bf5cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57fa96f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf68370 0x559b5bf54ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57fa96f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01ab5c0 con 0x559b5bf68370 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57fa96f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01ad100 con 0x559b5bf5eb40 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57fa96f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01abc90 con 0x559b5bf5d5d0 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57f6b33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf68370 0x559b5bf54ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57f5b31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5eb40 0x559b5bf5cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57f6332640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf5d5d0 0x559b5bf7c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57f6b33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf68370 0x559b5bf54ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58652/0 (socket says 172.21.15.133:58652) 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57f5b31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5eb40 0x559b5bf5cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:39932/0 (socket says 172.21.15.133:39932) 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57f6332640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf5d5d0 0x559b5bf7c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:56262/0 (socket says 172.21.15.133:56262) 2024-09-11T23:47:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.200+0000 7f57f6b33640 1 -- 172.21.15.133:0/1647042219 learned_addr learned my addr 172.21.15.133:0/1647042219 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.201+0000 7f57f6332640 1 -- 172.21.15.133:0/1647042219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5eb40 msgr2=0x559b5bf5cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.201+0000 7f57f6332640 1 --2- 172.21.15.133:0/1647042219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5eb40 0x559b5bf5cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.201+0000 7f57f6332640 1 -- 172.21.15.133:0/1647042219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf68370 msgr2=0x559b5bf54ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.201+0000 7f57f6332640 1 --2- 172.21.15.133:0/1647042219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf68370 0x559b5bf54ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.201+0000 7f57f6332640 1 -- 172.21.15.133:0/1647042219 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b5bf5a960 con 0x559b5bf5d5d0 2024-09-11T23:47:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.201+0000 7f57f5b31640 1 --2- 172.21.15.133:0/1647042219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5eb40 0x559b5bf5cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.201+0000 7f57f6b33640 1 --2- 172.21.15.133:0/1647042219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf68370 0x559b5bf54ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.201+0000 7f57f6332640 1 --2- 172.21.15.133:0/1647042219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf5d5d0 0x559b5bf7c720 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x559b5bf6f680 tx=0x7f57ec003e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7a0f12e8ff3e033 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.201+0000 7f57dbfff640 1 -- 172.21.15.133:0/1647042219 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec005220 con 0x559b5bf5d5d0 2024-09-11T23:47:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.201+0000 7f57dbfff640 1 -- 172.21.15.133:0/1647042219 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57ec003ad0 con 0x559b5bf5d5d0 2024-09-11T23:47:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.201+0000 7f57dbfff640 1 -- 172.21.15.133:0/1647042219 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0347c0 con 0x559b5bf5d5d0 2024-09-11T23:47:51.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.202+0000 7f57fa96f740 1 -- 172.21.15.133:0/1647042219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf5d5d0 msgr2=0x559b5bf7c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:51.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.202+0000 7f57fa96f740 1 --2- 172.21.15.133:0/1647042219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf5d5d0 0x559b5bf7c720 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x559b5bf6f680 tx=0x7f57ec003e40 comp rx=0 tx=0).stop 2024-09-11T23:47:51.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.202+0000 7f57fa96f740 1 -- 172.21.15.133:0/1647042219 shutdown_connections 2024-09-11T23:47:51.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.202+0000 7f57fa96f740 1 --2- 172.21.15.133:0/1647042219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf68370 0x559b5bf54ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.202+0000 7f57fa96f740 1 --2- 172.21.15.133:0/1647042219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5eb40 0x559b5bf5cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.202+0000 7f57fa96f740 1 --2- 172.21.15.133:0/1647042219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf5d5d0 0x559b5bf7c720 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.202+0000 7f57fa96f740 1 -- 172.21.15.133:0/1647042219 >> 172.21.15.133:0/1647042219 conn(0x559b5bf26310 msgr2=0x559b5bf63950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:51.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.202+0000 7f57fa96f740 1 -- 172.21.15.133:0/1647042219 shutdown_connections 2024-09-11T23:47:51.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.202+0000 7f57fa96f740 1 -- 172.21.15.133:0/1647042219 wait complete. 2024-09-11T23:47:51.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.202+0000 7f57fa96f740 1 -- start start 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.203+0000 7f57fa96f740 1 Processor -- start 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.203+0000 7f57fa96f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bfdb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.203+0000 7f57f6332640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bfdb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.203+0000 7f57f6332640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bfdb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:39940/0 (socket says 172.21.15.133:39940) 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.203+0000 7f57fa96f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bfdbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.203+0000 7f57f6332640 1 -- 172.21.15.133:0/2769183422 learned_addr learned my addr 172.21.15.133:0/2769183422 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.203+0000 7f57fa96f740 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf819f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.203+0000 7f57fa96f740 1 -- 172.21.15.133:0/2769183422 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01aca30 con 0x559b5bf5eb40 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.203+0000 7f57f5b31640 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bfdbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.203+0000 7f57f6b33640 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.204+0000 7f57fa96f740 1 -- 172.21.15.133:0/2769183422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01ac360 con 0x559b5bf5d5d0 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.204+0000 7f57fa96f740 1 -- 172.21.15.133:0/2769183422 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01a95c0 con 0x559b5bf68370 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.204+0000 7f57f6332640 1 -- 172.21.15.133:0/2769183422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 msgr2=0x559b5bf819f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.204+0000 7f57f6332640 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf819f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.204+0000 7f57f6332640 1 -- 172.21.15.133:0/2769183422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 msgr2=0x559b5bfdbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.204+0000 7f57f6332640 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bfdbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.204+0000 7f57f6332640 1 -- 172.21.15.133:0/2769183422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b5bf60120 con 0x559b5bf5d5d0 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.204+0000 7f57f6b33640 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.205+0000 7f57f6332640 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bfdb770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009240 tx=0x7f57ec011a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80d30b973c2d2b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.205+0000 7f57d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.205+0000 7f57fa96f740 10 client.0 _ll_register_callbacks cb 0x7f57e8009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.205+0000 7f57d9ffb640 1 -- 172.21.15.133:0/2769183422 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec040070 con 0x559b5bf5d5d0 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.205+0000 7f57d9ffb640 1 -- 172.21.15.133:0/2769183422 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f57ec009040 con 0x559b5bf5d5d0 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.205+0000 7f57fa96f740 1 -- 172.21.15.133:0/2769183422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57ec003ad0 con 0x559b5bf5d5d0 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.205+0000 7f57d9ffb640 1 -- 172.21.15.133:0/2769183422 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57ec04c050 con 0x559b5bf5d5d0 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.205+0000 7f57fa96f740 10 client.27782 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:51.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.205+0000 7f57fa96f740 1 -- 172.21.15.133:0/2769183422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=30}) -- 0x559b5bfe2ad0 con 0x559b5bf5d5d0 2024-09-11T23:47:51.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.206+0000 7f57d9ffb640 1 -- 172.21.15.133:0/2769183422 <== mon.2 v2:172.21.15.138:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f57ec0035f0 con 0x559b5bf5d5d0 2024-09-11T23:47:51.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.206+0000 7f57fa96f740 10 client.27782 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:51.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.206+0000 7f57fa96f740 10 client.27782 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:47:51.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.206+0000 7f57fa96f740 20 client.27782 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:51.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.206+0000 7f57fa96f740 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x559b5bf6c980 0x559b5bfe4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:51.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.206+0000 7f57fa96f740 4 client.27782 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:47:51.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.206+0000 7f57fa96f740 1 -- 172.21.15.133:0/2769183422 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b5becad70 con 0x559b5bf6c980 2024-09-11T23:47:51.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.206+0000 7f57f5b31640 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x559b5bf6c980 0x559b5bfe4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:51.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57f5b31640 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x559b5bf6c980 0x559b5bfe4e80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=34e0c125cbcf0086 server_cookie=76e793215f3c6e65 in_seq=0 out_seq=0 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57d9ffb640 10 client.27782 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57d9ffb640 1 -- 172.21.15.133:0/2769183422 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b5becad70 con 0x559b5bf6c980 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57d9ffb640 10 client.27782 handle_command_reply: tid=0 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57fa96f740 1 client.27782 shutdown 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57fa96f740 1 -- 172.21.15.133:0/2769183422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 msgr2=0x559b5bfdb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57fa96f740 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bfdb770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009240 tx=0x7f57ec011a20 comp rx=0 tx=0).stop 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57fa96f740 1 -- 172.21.15.133:0/2769183422 shutdown_connections 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57fa96f740 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x559b5bf6c980 0x559b5bfe4e80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57fa96f740 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b5bf68370 0x559b5bf819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57fa96f740 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5bf5eb40 0x559b5bfdbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57fa96f740 1 --2- 172.21.15.133:0/2769183422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b5bf5d5d0 0x559b5bfdb770 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009240 tx=0x7f57ec011a20 comp rx=0 tx=0).stop 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57fa96f740 1 -- 172.21.15.133:0/2769183422 >> 172.21.15.133:0/2769183422 conn(0x559b5bf26310 msgr2=0x559b5bf668e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.207+0000 7f57d9ffb640 0 client.27782 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:47:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.208+0000 7f57fa96f740 1 -- 172.21.15.133:0/2769183422 shutdown_connections 2024-09-11T23:47:51.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.208+0000 7f57fa96f740 1 -- 172.21.15.133:0/2769183422 wait complete. 2024-09-11T23:47:51.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.208+0000 7f57fa96f740 20 client.27782 trim_cache size 0 max 16384 2024-09-11T23:47:51.212 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:47:51.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.209+0000 7f57f8596640 1 -- 172.21.15.133:0/2424769316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57dc0a2d70 msgr2=0x7f57dc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:51.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.209+0000 7f57f8596640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57dc0a2d70 0x7f57dc0a5230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009bd0 tx=0x7f57ec005f90 comp rx=0 tx=0).stop 2024-09-11T23:47:51.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.209+0000 7f57f8596640 1 -- 172.21.15.133:0/2424769316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57f0103670 msgr2=0x7f57f0198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:51.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.209+0000 7f57f8596640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57f0103670 0x7f57f0198a40 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f57e0006010 tx=0x7f57e0031b20 comp rx=0 tx=0).stop 2024-09-11T23:47:51.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.209+0000 7f57f8596640 1 -- 172.21.15.133:0/2424769316 shutdown_connections 2024-09-11T23:47:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.209+0000 7f57f8596640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f57dc0a2d70 0x7f57dc0a5230 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.209+0000 7f57f8596640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57f010c180 0x7f57f019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.209+0000 7f57f8596640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57f0103670 0x7f57f0198a40 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.209+0000 7f57f8596640 1 --2- 172.21.15.133:0/2424769316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57f0102d50 0x7f57f0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.209+0000 7f57f8596640 1 -- 172.21.15.133:0/2424769316 >> 172.21.15.133:0/2424769316 conn(0x7f57f00fc7d0 msgr2=0x7f57f00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.210+0000 7f57f8596640 1 -- 172.21.15.133:0/2424769316 shutdown_connections 2024-09-11T23:47:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:51.210+0000 7f57f8596640 1 -- 172.21.15.133:0/2424769316 wait complete. 2024-09-11T23:47:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:50 smithi178 ceph-mon[35882]: pgmap v345: 97 pgs: 97 active+clean; 68 GiB data, 250 GiB used, 823 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 487 MiB/s wr, 2.33k op/s 2024-09-11T23:47:52.278 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:47:53.216 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:52 smithi133 ceph-mon[31430]: pgmap v346: 97 pgs: 97 active+clean; 69 GiB data, 252 GiB used, 821 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 553 MiB/s wr, 2.49k op/s 2024-09-11T23:47:53.252 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:52 smithi178 ceph-mon[35882]: pgmap v346: 97 pgs: 97 active+clean; 69 GiB data, 252 GiB used, 821 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 553 MiB/s wr, 2.49k op/s 2024-09-11T23:47:53.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:52 smithi138 ceph-mon[35142]: pgmap v346: 97 pgs: 97 active+clean; 69 GiB data, 252 GiB used, 821 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 553 MiB/s wr, 2.49k op/s 2024-09-11T23:47:54.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.132+0000 7f4d2304f640 1 -- 172.21.15.133:0/3092049214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1c110190 msgr2=0x7f4d1c06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:54.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.132+0000 7f4d2304f640 1 --2- 172.21.15.133:0/3092049214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1c110190 0x7f4d1c06e4f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009900 tx=0x7f4d1802a400 comp rx=0 tx=0).stop 2024-09-11T23:47:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.134+0000 7f4d2304f640 1 -- 172.21.15.133:0/3092049214 shutdown_connections 2024-09-11T23:47:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.134+0000 7f4d2304f640 1 --2- 172.21.15.133:0/3092049214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c11b140 0x7f4d1c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.134+0000 7f4d2304f640 1 --2- 172.21.15.133:0/3092049214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1c06ea30 0x7f4d1c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.134+0000 7f4d2304f640 1 --2- 172.21.15.133:0/3092049214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1c110190 0x7f4d1c06e4f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.134+0000 7f4d2304f640 1 -- 172.21.15.133:0/3092049214 >> 172.21.15.133:0/3092049214 conn(0x7f4d1c073180 msgr2=0x7f4d1c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.134+0000 7f4d2304f640 1 -- 172.21.15.133:0/3092049214 shutdown_connections 2024-09-11T23:47:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.134+0000 7f4d2304f640 1 -- 172.21.15.133:0/3092049214 wait complete. 2024-09-11T23:47:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.135+0000 7f4d2304f640 1 Processor -- start 2024-09-11T23:47:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.137+0000 7f4d2304f640 1 -- start start 2024-09-11T23:47:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.137+0000 7f4d2304f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1c06ea30 0x7f4d1c1a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.137+0000 7f4d2304f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1c110190 0x7f4d1c1a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.138+0000 7f4d2304f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c11b140 0x7f4d1c1ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.138+0000 7f4d2304f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c120be0 con 0x7f4d1c06ea30 2024-09-11T23:47:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.138+0000 7f4d2304f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c120a60 con 0x7f4d1c110190 2024-09-11T23:47:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.138+0000 7f4d2304f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c120d60 con 0x7f4d1c11b140 2024-09-11T23:47:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.138+0000 7f4d2284e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c11b140 0x7f4d1c1ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.138+0000 7f4d2184c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1c110190 0x7f4d1c1a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.138+0000 7f4d2184c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1c110190 0x7f4d1c1a7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:43988/0 (socket says 172.21.15.133:43988) 2024-09-11T23:47:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.138+0000 7f4d2184c640 1 -- 172.21.15.133:0/2262883651 learned_addr learned my addr 172.21.15.133:0/2262883651 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.139+0000 7f4d2184c640 1 -- 172.21.15.133:0/2262883651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c11b140 msgr2=0x7f4d1c1ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.139+0000 7f4d2184c640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c11b140 0x7f4d1c1ab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.139+0000 7f4d2184c640 1 -- 172.21.15.133:0/2262883651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1c06ea30 msgr2=0x7f4d1c1a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.139+0000 7f4d2184c640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1c06ea30 0x7f4d1c1a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.139+0000 7f4d2184c640 1 -- 172.21.15.133:0/2262883651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c1ac090 con 0x7f4d1c110190 2024-09-11T23:47:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.141+0000 7f4d2184c640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1c110190 0x7f4d1c1a7600 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00d950 tx=0x7f4d0c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:54.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.143+0000 7f4d137fe640 1 -- 172.21.15.133:0/2262883651 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c017070 con 0x7f4d1c110190 2024-09-11T23:47:54.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.143+0000 7f4d137fe640 1 -- 172.21.15.133:0/2262883651 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0c009c30 con 0x7f4d1c110190 2024-09-11T23:47:54.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.143+0000 7f4d137fe640 1 -- 172.21.15.133:0/2262883651 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00f6b0 con 0x7f4d1c110190 2024-09-11T23:47:54.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.143+0000 7f4d2304f640 1 -- 172.21.15.133:0/2262883651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c1ac380 con 0x7f4d1c110190 2024-09-11T23:47:54.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.143+0000 7f4d2304f640 1 -- 172.21.15.133:0/2262883651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c110920 con 0x7f4d1c110190 2024-09-11T23:47:54.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.152+0000 7f4d137fe640 1 -- 172.21.15.133:0/2262883651 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4d0c00f850 con 0x7f4d1c110190 2024-09-11T23:47:54.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.154+0000 7f4d137fe640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4d080a2e40 0x7f4d080a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.155+0000 7f4d137fe640 1 -- 172.21.15.133:0/2262883651 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f4d0c0ceee0 con 0x7f4d1c110190 2024-09-11T23:47:54.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.155+0000 7f4d2668a740 1 Processor -- start 2024-09-11T23:47:54.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.156+0000 7f4d2204d640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4d080a2e40 0x7f4d080a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.157+0000 7f4d2204d640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4d080a2e40 0x7f4d080a5300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005c20 tx=0x7f4d18000bc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.158+0000 7f4d2668a740 1 -- start start 2024-09-11T23:47:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.158+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559090425d20 0x559090424270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.158+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590904247b0 0x559090424bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.158+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 0x559090431940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.158+0000 7f4d2668a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55909038a2f0 con 0x559090425d20 2024-09-11T23:47:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.158+0000 7f4d2668a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559090331cf0 con 0x5590904247b0 2024-09-11T23:47:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.158+0000 7f4d2668a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5590903862b0 con 0x55909042f550 2024-09-11T23:47:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.159+0000 7f4d2284e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 0x559090431940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.159+0000 7f4d2284e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 0x559090431940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:59598/0 (socket says 172.21.15.133:59598) 2024-09-11T23:47:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.159+0000 7f4d2284e640 1 -- 172.21.15.133:0/3091343692 learned_addr learned my addr 172.21.15.133:0/3091343692 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.159+0000 7f4d2204d640 1 --2- 172.21.15.133:0/3091343692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559090425d20 0x559090424270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.160+0000 7f4d2284e640 1 -- 172.21.15.133:0/3091343692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590904247b0 msgr2=0x559090424bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:54.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.160+0000 7f4d2284e640 1 --2- 172.21.15.133:0/3091343692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590904247b0 0x559090424bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.160+0000 7f4d2284e640 1 -- 172.21.15.133:0/3091343692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559090425d20 msgr2=0x559090424270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:54.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.160+0000 7f4d2284e640 1 --2- 172.21.15.133:0/3091343692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559090425d20 0x559090424270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.160+0000 7f4d2284e640 1 -- 172.21.15.133:0/3091343692 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590903163a0 con 0x55909042f550 2024-09-11T23:47:54.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.161+0000 7f4d2284e640 1 --2- 172.21.15.133:0/3091343692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 0x559090431940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009f20 tx=0x7f4d140118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ec42f45e8fd3d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:54.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.162+0000 7f4d10dea640 1 -- 172.21.15.133:0/3091343692 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14004d20 con 0x55909042f550 2024-09-11T23:47:54.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.162+0000 7f4d10dea640 1 -- 172.21.15.133:0/3091343692 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14004050 con 0x55909042f550 2024-09-11T23:47:54.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.163+0000 7f4d2668a740 1 -- 172.21.15.133:0/3091343692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 msgr2=0x559090431940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:54.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.163+0000 7f4d2668a740 1 --2- 172.21.15.133:0/3091343692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 0x559090431940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009f20 tx=0x7f4d140118d0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.164+0000 7f4d2668a740 1 -- 172.21.15.133:0/3091343692 shutdown_connections 2024-09-11T23:47:54.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.164+0000 7f4d2668a740 1 --2- 172.21.15.133:0/3091343692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 0x559090431940 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.164+0000 7f4d2668a740 1 --2- 172.21.15.133:0/3091343692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590904247b0 0x559090424bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.164+0000 7f4d2668a740 1 --2- 172.21.15.133:0/3091343692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559090425d20 0x559090424270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.164+0000 7f4d2668a740 1 -- 172.21.15.133:0/3091343692 >> 172.21.15.133:0/3091343692 conn(0x5590903ed310 msgr2=0x559090420e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:54.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.165+0000 7f4d2668a740 1 -- 172.21.15.133:0/3091343692 shutdown_connections 2024-09-11T23:47:54.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.165+0000 7f4d2668a740 1 -- 172.21.15.133:0/3091343692 wait complete. 2024-09-11T23:47:54.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.165+0000 7f4d2668a740 1 -- start start 2024-09-11T23:47:54.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.166+0000 7f4d2668a740 1 Processor -- start 2024-09-11T23:47:54.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.166+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590904247b0 0x55909042ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.167+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559090425d20 0x55909042c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.167+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 0x55909042c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.167+0000 7f4d2668a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5590904322a0 con 0x55909042f550 2024-09-11T23:47:54.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.167+0000 7f4d2668a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559090345060 con 0x5590904247b0 2024-09-11T23:47:54.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.167+0000 7f4d2668a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559090432420 con 0x559090425d20 2024-09-11T23:47:54.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.167+0000 7f4d2284e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 0x55909042c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.167+0000 7f4d2284e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 0x55909042c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34948/0 (socket says 172.21.15.133:34948) 2024-09-11T23:47:54.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.167+0000 7f4d2284e640 1 -- 172.21.15.133:0/2453344591 learned_addr learned my addr 172.21.15.133:0/2453344591 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:54.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.168+0000 7f4d2184c640 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559090425d20 0x55909042c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.168+0000 7f4d2668a740 10 client.0 _ll_register_callbacks cb 0x5590900988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:54.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.169+0000 7f4d2284e640 1 -- 172.21.15.133:0/2453344591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559090425d20 msgr2=0x55909042c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:54.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.169+0000 7f4d2284e640 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559090425d20 0x55909042c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.169+0000 7f4d2284e640 1 -- 172.21.15.133:0/2453344591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590904247b0 msgr2=0x55909042ee80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:47:54.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.169+0000 7f4d2284e640 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590904247b0 0x55909042ee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.169+0000 7f4d2284e640 1 -- 172.21.15.133:0/2453344591 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590900fb3c0 con 0x55909042f550 2024-09-11T23:47:54.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.170+0000 7f4d2284e640 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 0x55909042c670 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f4d14003c40 tx=0x7f4d14003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8290b935cc9e3b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:54.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.170+0000 7f4d05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:54.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.170+0000 7f4d05ffb640 1 -- 172.21.15.133:0/2453344591 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14029020 con 0x55909042f550 2024-09-11T23:47:54.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.170+0000 7f4d05ffb640 1 -- 172.21.15.133:0/2453344591 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f4d140058d0 con 0x55909042f550 2024-09-11T23:47:54.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.170+0000 7f4d2668a740 1 -- 172.21.15.133:0/2453344591 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d14004050 con 0x55909042f550 2024-09-11T23:47:54.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.171+0000 7f4d05ffb640 1 -- 172.21.15.133:0/2453344591 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d14004850 con 0x55909042f550 2024-09-11T23:47:54.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.171+0000 7f4d2668a740 10 client.18249 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:54.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.172+0000 7f4d2668a740 1 -- 172.21.15.133:0/2453344591 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55909043dd50 con 0x55909042f550 2024-09-11T23:47:54.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.173+0000 7f4d05ffb640 1 -- 172.21.15.133:0/2453344591 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14004050 con 0x55909042f550 2024-09-11T23:47:54.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.173+0000 7f4d05ffb640 1 -- 172.21.15.133:0/2453344591 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4d14025b30 con 0x55909042f550 2024-09-11T23:47:54.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.173+0000 7f4d2668a740 10 client.18249 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:54.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.173+0000 7f4d2668a740 10 client.18249 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:47:54.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.173+0000 7f4d2668a740 20 client.18249 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:54.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.173+0000 7f4d2668a740 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55909043f4e0 0x559090441960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.173+0000 7f4d2668a740 4 client.18249 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:54.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.173+0000 7f4d2668a740 1 -- 172.21.15.133:0/2453344591 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559090397860 con 0x55909043f4e0 2024-09-11T23:47:54.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.173+0000 7f4d2204d640 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55909043f4e0 0x559090441960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.181+0000 7f4d2204d640 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55909043f4e0 0x559090441960 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b07772d82260f36 server_cookie=c3e9102964a7004b in_seq=0 out_seq=0 2024-09-11T23:47:54.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.181+0000 7f4d05ffb640 10 client.18249 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:47:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.183+0000 7f4d05ffb640 1 -- 172.21.15.133:0/2453344591 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559090397860 con 0x55909043f4e0 2024-09-11T23:47:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.183+0000 7f4d05ffb640 10 client.18249 handle_command_reply: tid=0 2024-09-11T23:47:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.183+0000 7f4d2668a740 1 client.18249 shutdown 2024-09-11T23:47:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.184+0000 7f4d2668a740 1 -- 172.21.15.133:0/2453344591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 msgr2=0x55909042c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.184+0000 7f4d2668a740 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 0x55909042c670 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f4d14003c40 tx=0x7f4d14003c70 comp rx=0 tx=0).stop 2024-09-11T23:47:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.185+0000 7f4d2668a740 1 -- 172.21.15.133:0/2453344591 shutdown_connections 2024-09-11T23:47:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.185+0000 7f4d2668a740 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55909043f4e0 0x559090441960 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.185+0000 7f4d2668a740 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 0x55909042c670 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.185+0000 7f4d2668a740 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559090425d20 0x55909042c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.186+0000 7f4d2668a740 1 --2- 172.21.15.133:0/2453344591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590904247b0 0x55909042ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.186+0000 7f4d2668a740 1 -- 172.21.15.133:0/2453344591 >> 172.21.15.133:0/2453344591 conn(0x5590903ed310 msgr2=0x559090426b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.186+0000 7f4d05ffb640 0 client.18249 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:47:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.186+0000 7f4d2668a740 1 -- 172.21.15.133:0/2453344591 shutdown_connections 2024-09-11T23:47:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.186+0000 7f4d2668a740 1 -- 172.21.15.133:0/2453344591 wait complete. 2024-09-11T23:47:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.189+0000 7f4d2668a740 20 client.18249 trim_cache size 0 max 16384 2024-09-11T23:47:54.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.234+0000 7f4d2668a740 1 Processor -- start 2024-09-11T23:47:54.242 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:53 smithi133 ceph-mon[31430]: pgmap v347: 97 pgs: 97 active+clean; 69 GiB data, 257 GiB used, 816 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 750 MiB/s wr, 3.14k op/s 2024-09-11T23:47:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.241+0000 7f4d2668a740 1 -- start start 2024-09-11T23:47:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.241+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590904247b0 0x559090441950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.241+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 0x55909043b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.241+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 0x559090427300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.241+0000 7f4d2668a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0000e6b0 con 0x5590904247b0 2024-09-11T23:47:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.241+0000 7f4d2668a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d000101f0 con 0x559090425d20 2024-09-11T23:47:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.241+0000 7f4d2668a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0000ed80 con 0x55909042f550 2024-09-11T23:47:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.247+0000 7f4d2204d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590904247b0 0x559090441950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.247+0000 7f4d2204d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590904247b0 0x559090441950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34954/0 (socket says 172.21.15.133:34954) 2024-09-11T23:47:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.247+0000 7f4d2204d640 1 -- 172.21.15.133:0/1726399009 learned_addr learned my addr 172.21.15.133:0/1726399009 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.247+0000 7f4d2284e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 0x559090427300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.247+0000 7f4d2184c640 1 --2- 172.21.15.133:0/1726399009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 0x55909043b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.249+0000 7f4d2184c640 1 -- 172.21.15.133:0/1726399009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 msgr2=0x559090427300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:54.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.249+0000 7f4d2184c640 1 --2- 172.21.15.133:0/1726399009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 0x559090427300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.249+0000 7f4d2184c640 1 -- 172.21.15.133:0/1726399009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590904247b0 msgr2=0x559090441950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:54.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.249+0000 7f4d2184c640 1 --2- 172.21.15.133:0/1726399009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590904247b0 0x559090441950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.249+0000 7f4d2184c640 1 -- 172.21.15.133:0/1726399009 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559090424f30 con 0x559090425d20 2024-09-11T23:47:54.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.249+0000 7f4d2204d640 1 --2- 172.21.15.133:0/1726399009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590904247b0 0x559090441950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.254+0000 7f4d2184c640 1 --2- 172.21.15.133:0/1726399009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 0x55909043b620 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0cc610 tx=0x7f4d0c09bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd326825fbe162d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.258+0000 7f4d07fff640 1 -- 172.21.15.133:0/1726399009 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0b4070 con 0x559090425d20 2024-09-11T23:47:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.258+0000 7f4d07fff640 1 -- 172.21.15.133:0/1726399009 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0c09f790 con 0x559090425d20 2024-09-11T23:47:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.258+0000 7f4d07fff640 1 -- 172.21.15.133:0/1726399009 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c020040 con 0x559090425d20 2024-09-11T23:47:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.259+0000 7f4d2668a740 1 -- 172.21.15.133:0/1726399009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 msgr2=0x55909043b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.259+0000 7f4d2668a740 1 --2- 172.21.15.133:0/1726399009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 0x55909043b620 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0cc610 tx=0x7f4d0c09bae0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.260+0000 7f4d2668a740 1 -- 172.21.15.133:0/1726399009 shutdown_connections 2024-09-11T23:47:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.260+0000 7f4d2668a740 1 --2- 172.21.15.133:0/1726399009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55909042f550 0x559090427300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.260+0000 7f4d2668a740 1 --2- 172.21.15.133:0/1726399009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 0x55909043b620 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.260+0000 7f4d2668a740 1 --2- 172.21.15.133:0/1726399009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590904247b0 0x559090441950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.260+0000 7f4d2668a740 1 -- 172.21.15.133:0/1726399009 >> 172.21.15.133:0/1726399009 conn(0x5590903ed310 msgr2=0x55909042b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.261+0000 7f4d2668a740 1 -- 172.21.15.133:0/1726399009 shutdown_connections 2024-09-11T23:47:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.262+0000 7f4d2668a740 1 -- 172.21.15.133:0/1726399009 wait complete. 2024-09-11T23:47:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.262+0000 7f4d2668a740 1 -- start start 2024-09-11T23:47:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.263+0000 7f4d2668a740 1 Processor -- start 2024-09-11T23:47:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.264+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590904247b0 0x559090437790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.265+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 0x559090437cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.265+0000 7f4d2668a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 0x5590904283e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.266+0000 7f4d2668a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0000fb20 con 0x55909042f550 2024-09-11T23:47:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.266+0000 7f4d2668a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0000f450 con 0x559090425d20 2024-09-11T23:47:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.266+0000 7f4d2668a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0000c6b0 con 0x5590904247b0 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.266+0000 7f4d2668a740 10 client.0 _ll_register_callbacks cb 0x7f4d140093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.268+0000 7f4d2184c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 0x559090437cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.268+0000 7f4d2204d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590904247b0 0x559090437790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.268+0000 7f4d2184c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 0x559090437cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:44050/0 (socket says 172.21.15.133:44050) 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.268+0000 7f4d2184c640 1 -- 172.21.15.133:0/3182918383 learned_addr learned my addr 172.21.15.133:0/3182918383 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.269+0000 7f4d2284e640 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 0x5590904283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.270+0000 7f4d2284e640 1 -- 172.21.15.133:0/3182918383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590904247b0 msgr2=0x559090437790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.270+0000 7f4d2284e640 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590904247b0 0x559090437790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.270+0000 7f4d2284e640 1 -- 172.21.15.133:0/3182918383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 msgr2=0x559090437cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.270+0000 7f4d2284e640 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 0x559090437cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.270+0000 7f4d2284e640 1 -- 172.21.15.133:0/3182918383 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909042a090 con 0x55909042f550 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.271+0000 7f4d2284e640 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 0x5590904283e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4d14000f80 tx=0x7f4d14004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dae3c509d3bf70c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.271+0000 7f4d05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.272+0000 7f4d05ffb640 1 -- 172.21.15.133:0/3182918383 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1403a020 con 0x55909042f550 2024-09-11T23:47:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.273+0000 7f4d05ffb640 1 -- 172.21.15.133:0/3182918383 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f4d1400f720 con 0x55909042f550 2024-09-11T23:47:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.273+0000 7f4d2668a740 1 -- 172.21.15.133:0/3182918383 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d0c09f790 con 0x55909042f550 2024-09-11T23:47:54.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.275+0000 7f4d05ffb640 1 -- 172.21.15.133:0/3182918383 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d14045050 con 0x55909042f550 2024-09-11T23:47:54.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.277+0000 7f4d2668a740 10 client.18255 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:54.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.277+0000 7f4d2668a740 1 -- 172.21.15.133:0/3182918383 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5590904a9720 con 0x55909042f550 2024-09-11T23:47:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.278+0000 7f4d05ffb640 1 -- 172.21.15.133:0/3182918383 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d140056d0 con 0x55909042f550 2024-09-11T23:47:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.278+0000 7f4d05ffb640 1 -- 172.21.15.133:0/3182918383 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4d14025470 con 0x55909042f550 2024-09-11T23:47:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.278+0000 7f4d2668a740 10 client.18255 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.279+0000 7f4d2668a740 10 client.18255 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:47:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.279+0000 7f4d2668a740 20 client.18255 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.279+0000 7f4d2668a740 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5590904a9de0 0x5590904ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.279+0000 7f4d2668a740 4 client.18255 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:47:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.279+0000 7f4d2668a740 1 -- 172.21.15.133:0/3182918383 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4d0c09f790 con 0x5590904a9de0 2024-09-11T23:47:54.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:53 smithi178 ceph-mon[35882]: pgmap v347: 97 pgs: 97 active+clean; 69 GiB data, 257 GiB used, 816 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 750 MiB/s wr, 3.14k op/s 2024-09-11T23:47:54.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.287+0000 7f4d2204d640 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5590904a9de0 0x5590904ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:54.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.302+0000 7f4d05ffb640 10 client.18255 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:47:54.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.302+0000 7f4d2204d640 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5590904a9de0 0x5590904ac200 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa05b1e42fb12c26 server_cookie=1019020af3468a58 in_seq=0 out_seq=0 2024-09-11T23:47:54.313 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:47:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.306+0000 7f4d05ffb640 1 -- 172.21.15.133:0/3182918383 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4d0c09f790 con 0x5590904a9de0 2024-09-11T23:47:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.306+0000 7f4d05ffb640 10 client.18255 handle_command_reply: tid=0 2024-09-11T23:47:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.306+0000 7f4d2668a740 1 client.18255 shutdown 2024-09-11T23:47:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.309+0000 7f4d2668a740 1 -- 172.21.15.133:0/3182918383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 msgr2=0x5590904283e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.309+0000 7f4d2668a740 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 0x5590904283e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4d14000f80 tx=0x7f4d14004190 comp rx=0 tx=0).stop 2024-09-11T23:47:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.309+0000 7f4d2668a740 1 -- 172.21.15.133:0/3182918383 shutdown_connections 2024-09-11T23:47:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.309+0000 7f4d2668a740 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5590904a9de0 0x5590904ac200 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.309+0000 7f4d2668a740 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55909042f550 0x5590904283e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.309+0000 7f4d05ffb640 0 client.18255 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:47:54.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.309+0000 7f4d2668a740 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559090425d20 0x559090437cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.309+0000 7f4d2668a740 1 --2- 172.21.15.133:0/3182918383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590904247b0 0x559090437790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.309+0000 7f4d2668a740 1 -- 172.21.15.133:0/3182918383 >> 172.21.15.133:0/3182918383 conn(0x5590903ed310 msgr2=0x55909041f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:54.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.310+0000 7f4d2668a740 1 -- 172.21.15.133:0/3182918383 shutdown_connections 2024-09-11T23:47:54.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.311+0000 7f4d2668a740 1 -- 172.21.15.133:0/3182918383 wait complete. 2024-09-11T23:47:54.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.311+0000 7f4d2668a740 20 client.18255 trim_cache size 0 max 16384 2024-09-11T23:47:54.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.313+0000 7f4d2304f640 1 -- 172.21.15.133:0/2262883651 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4d080a2e40 msgr2=0x7f4d080a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:54.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.313+0000 7f4d2304f640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4d080a2e40 0x7f4d080a5300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005c20 tx=0x7f4d18000bc0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.313+0000 7f4d2304f640 1 -- 172.21.15.133:0/2262883651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1c110190 msgr2=0x7f4d1c1a7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:54.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.313+0000 7f4d2304f640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1c110190 0x7f4d1c1a7600 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00d950 tx=0x7f4d0c00de10 comp rx=0 tx=0).stop 2024-09-11T23:47:54.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.313+0000 7f4d2304f640 1 -- 172.21.15.133:0/2262883651 shutdown_connections 2024-09-11T23:47:54.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.313+0000 7f4d2304f640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f4d080a2e40 0x7f4d080a5300 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005c20 tx=0x7f4d18000bc0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.313+0000 7f4d2304f640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c11b140 0x7f4d1c1ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.313+0000 7f4d2304f640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d1c110190 0x7f4d1c1a7600 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.313+0000 7f4d2304f640 1 --2- 172.21.15.133:0/2262883651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1c06ea30 0x7f4d1c1a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:54.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.313+0000 7f4d2304f640 1 -- 172.21.15.133:0/2262883651 >> 172.21.15.133:0/2262883651 conn(0x7f4d1c073180 msgr2=0x7f4d1c1192d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:54.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.317+0000 7f4d2304f640 1 -- 172.21.15.133:0/2262883651 shutdown_connections 2024-09-11T23:47:54.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:54.317+0000 7f4d2304f640 1 -- 172.21.15.133:0/2262883651 wait complete. 2024-09-11T23:47:54.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:53 smithi138 ceph-mon[35142]: pgmap v347: 97 pgs: 97 active+clean; 69 GiB data, 257 GiB used, 816 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 750 MiB/s wr, 3.14k op/s 2024-09-11T23:47:55.453 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:47:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:57.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:56 smithi178 ceph-mon[35882]: pgmap v348: 97 pgs: 97 active+clean; 69 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 669 MiB/s wr, 2.24k op/s 2024-09-11T23:47:57.073 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:56 smithi133 ceph-mon[31430]: pgmap v348: 97 pgs: 97 active+clean; 69 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 669 MiB/s wr, 2.24k op/s 2024-09-11T23:47:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.062+0000 7ff60e50c640 1 -- 172.21.15.133:0/1068977484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff608109120 msgr2=0x7ff608110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.062+0000 7ff60e50c640 1 --2- 172.21.15.133:0/1068977484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff608109120 0x7ff608110ca0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff600013490 tx=0x7ff600032b10 comp rx=0 tx=0).stop 2024-09-11T23:47:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.062+0000 7ff60e50c640 1 -- 172.21.15.133:0/1068977484 shutdown_connections 2024-09-11T23:47:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.062+0000 7ff60e50c640 1 --2- 172.21.15.133:0/1068977484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff608109120 0x7ff608110ca0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.062+0000 7ff60e50c640 1 --2- 172.21.15.133:0/1068977484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6081049a0 0x7ff6081089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.062+0000 7ff60e50c640 1 --2- 172.21.15.133:0/1068977484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff608103ff0 0x7ff6081043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.062+0000 7ff60e50c640 1 -- 172.21.15.133:0/1068977484 >> 172.21.15.133:0/1068977484 conn(0x7ff6080ffab0 msgr2=0x7ff608101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:57.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.074+0000 7ff60e50c640 1 -- 172.21.15.133:0/1068977484 shutdown_connections 2024-09-11T23:47:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.075+0000 7ff60e50c640 1 -- 172.21.15.133:0/1068977484 wait complete. 2024-09-11T23:47:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.076+0000 7ff60e50c640 1 Processor -- start 2024-09-11T23:47:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.076+0000 7ff60e50c640 1 -- start start 2024-09-11T23:47:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.077+0000 7ff60e50c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff608103ff0 0x7ff6081a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.077+0000 7ff60e50c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6081049a0 0x7ff6081a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.077+0000 7ff60e50c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff608109120 0x7ff6081abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.077+0000 7ff60e50c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff608079030 con 0x7ff608109120 2024-09-11T23:47:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.077+0000 7ff60e50c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff608078eb0 con 0x7ff608103ff0 2024-09-11T23:47:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.077+0000 7ff60e50c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6080791b0 con 0x7ff6081049a0 2024-09-11T23:47:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.077+0000 7ff60caa9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff608109120 0x7ff6081abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.077+0000 7ff60caa9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff608109120 0x7ff6081abb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34968/0 (socket says 172.21.15.133:34968) 2024-09-11T23:47:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.077+0000 7ff60caa9640 1 -- 172.21.15.133:0/1419492608 learned_addr learned my addr 172.21.15.133:0/1419492608 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.078+0000 7ff607fff640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff608103ff0 0x7ff6081a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.078+0000 7ff60caa9640 1 -- 172.21.15.133:0/1419492608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6081049a0 msgr2=0x7ff6081a7c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:47:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.078+0000 7ff60caa9640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6081049a0 0x7ff6081a7c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.078+0000 7ff60caa9640 1 -- 172.21.15.133:0/1419492608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff608103ff0 msgr2=0x7ff6081a76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.078+0000 7ff60caa9640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff608103ff0 0x7ff6081a76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.078+0000 7ff60caa9640 1 -- 172.21.15.133:0/1419492608 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6081ac1e0 con 0x7ff608109120 2024-09-11T23:47:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.078+0000 7ff607fff640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff608103ff0 0x7ff6081a76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:47:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.079+0000 7ff60caa9640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff608109120 0x7ff6081abb00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff6000135b0 tx=0x7ff600034e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.079+0000 7ff6057fa640 1 -- 172.21.15.133:0/1419492608 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60003a040 con 0x7ff608109120 2024-09-11T23:47:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.079+0000 7ff6057fa640 1 -- 172.21.15.133:0/1419492608 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff60000f170 con 0x7ff608109120 2024-09-11T23:47:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.079+0000 7ff6057fa640 1 -- 172.21.15.133:0/1419492608 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60000c880 con 0x7ff608109120 2024-09-11T23:47:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.079+0000 7ff60e50c640 1 -- 172.21.15.133:0/1419492608 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6081ac470 con 0x7ff608109120 2024-09-11T23:47:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.080+0000 7ff60e50c640 1 -- 172.21.15.133:0/1419492608 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6081ac900 con 0x7ff608109120 2024-09-11T23:47:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.084+0000 7ff6108e5740 1 Processor -- start 2024-09-11T23:47:57.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.085+0000 7ff6057fa640 1 -- 172.21.15.133:0/1419492608 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff60003d070 con 0x7ff608109120 2024-09-11T23:47:57.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.086+0000 7ff6057fa640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5f00a2df0 0x7ff5f00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.087+0000 7ff6057fa640 1 -- 172.21.15.133:0/1419492608 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7ff6000ef6c0 con 0x7ff608109120 2024-09-11T23:47:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.087+0000 7ff6108e5740 1 -- start start 2024-09-11T23:47:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.087+0000 7ff6077fe640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5f00a2df0 0x7ff5f00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.088+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b38b40 0x55a874b37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.088+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b375d0 0x55a874b37a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.088+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b42370 0x55a874b44760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.088+0000 7ff6108e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a874a9d2f0 con 0x55a874b42370 2024-09-11T23:47:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.088+0000 7ff6108e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a874a44cf0 con 0x55a874b375d0 2024-09-11T23:47:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.088+0000 7ff6108e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a874a992b0 con 0x55a874b38b40 2024-09-11T23:47:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.088+0000 7ff60caa9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b42370 0x55a874b44760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.088+0000 7ff60caa9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b42370 0x55a874b44760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34980/0 (socket says 172.21.15.133:34980) 2024-09-11T23:47:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.088+0000 7ff60caa9640 1 -- 172.21.15.133:0/1241928647 learned_addr learned my addr 172.21.15.133:0/1241928647 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.089+0000 7ff607fff640 1 --2- 172.21.15.133:0/1241928647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b38b40 0x55a874b37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.089+0000 7ff60caa9640 1 -- 172.21.15.133:0/1241928647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b38b40 msgr2=0x55a874b37090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.089+0000 7ff60caa9640 1 --2- 172.21.15.133:0/1241928647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b38b40 0x55a874b37090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.089+0000 7ff60caa9640 1 -- 172.21.15.133:0/1241928647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b375d0 msgr2=0x55a874b37a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.089+0000 7ff6077fe640 1 --2- 172.21.15.133:0/1241928647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b375d0 0x55a874b37a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.089+0000 7ff60caa9640 1 --2- 172.21.15.133:0/1241928647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b375d0 0x55a874b37a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.089+0000 7ff60caa9640 1 -- 172.21.15.133:0/1241928647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a874a293a0 con 0x55a874b42370 2024-09-11T23:47:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.090+0000 7ff60caa9640 1 --2- 172.21.15.133:0/1241928647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b42370 0x55a874b44760 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff6000b86b0 tx=0x7ff600013140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaf4f8c166503e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.090+0000 7ff6077fe640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5f00a2df0 0x7ff5f00a52b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff608079ac0 tx=0x7ff5f800b370 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:57.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.093+0000 7ff5ee7fc640 1 -- 172.21.15.133:0/1241928647 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6000d1030 con 0x55a874b42370 2024-09-11T23:47:57.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.094+0000 7ff5ee7fc640 1 -- 172.21.15.133:0/1241928647 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6000c0090 con 0x55a874b42370 2024-09-11T23:47:57.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.094+0000 7ff5ee7fc640 1 -- 172.21.15.133:0/1241928647 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6000bbf40 con 0x55a874b42370 2024-09-11T23:47:57.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.096+0000 7ff6108e5740 1 -- 172.21.15.133:0/1241928647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b42370 msgr2=0x55a874b44760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:57.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.096+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1241928647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b42370 0x55a874b44760 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff6000b86b0 tx=0x7ff600013140 comp rx=0 tx=0).stop 2024-09-11T23:47:57.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.097+0000 7ff6108e5740 1 -- 172.21.15.133:0/1241928647 shutdown_connections 2024-09-11T23:47:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.097+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1241928647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b42370 0x55a874b44760 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.097+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1241928647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b375d0 0x55a874b37a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.097+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1241928647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b38b40 0x55a874b37090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.097+0000 7ff6108e5740 1 -- 172.21.15.133:0/1241928647 >> 172.21.15.133:0/1241928647 conn(0x55a874b00310 msgr2=0x55a874b33c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:57.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.097+0000 7ff6108e5740 1 -- 172.21.15.133:0/1241928647 shutdown_connections 2024-09-11T23:47:57.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.098+0000 7ff6108e5740 1 -- 172.21.15.133:0/1241928647 wait complete. 2024-09-11T23:47:57.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.099+0000 7ff6108e5740 1 -- start start 2024-09-11T23:47:57.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.100+0000 7ff6108e5740 1 Processor -- start 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.100+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b375d0 0x55a874b41df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.101+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b38b40 0x55a874b3f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.101+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b42370 0x55a874b3f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.101+0000 7ff6108e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a874b55e40 con 0x55a874b375d0 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.101+0000 7ff6108e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a874a58060 con 0x55a874b38b40 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.101+0000 7ff6108e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a874b55fc0 con 0x55a874b42370 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.101+0000 7ff6108e5740 10 client.0 _ll_register_callbacks cb 0x55a8747ab8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.103+0000 7ff6077fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b38b40 0x55a874b3f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.103+0000 7ff607fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b375d0 0x55a874b41df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.103+0000 7ff60caa9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b42370 0x55a874b3f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.103+0000 7ff6077fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b38b40 0x55a874b3f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:44082/0 (socket says 172.21.15.133:44082) 2024-09-11T23:47:57.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.103+0000 7ff6077fe640 1 -- 172.21.15.133:0/1138849170 learned_addr learned my addr 172.21.15.133:0/1138849170 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.105+0000 7ff6077fe640 1 -- 172.21.15.133:0/1138849170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b42370 msgr2=0x55a874b3f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.105+0000 7ff6077fe640 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b42370 0x55a874b3f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.105+0000 7ff6077fe640 1 -- 172.21.15.133:0/1138849170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b375d0 msgr2=0x55a874b41df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:57.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.105+0000 7ff6077fe640 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b375d0 0x55a874b41df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.105+0000 7ff6077fe640 1 -- 172.21.15.133:0/1138849170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87480e3c0 con 0x55a874b38b40 2024-09-11T23:47:57.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.106+0000 7ff607fff640 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b375d0 0x55a874b41df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:47:57.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.106+0000 7ff6077fe640 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b38b40 0x55a874b3f0a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800dec0 tx=0x7ff5f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2699eb6a6bf6372e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:57.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.106+0000 7ff5e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:47:57.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.108+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1138849170 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8021070 con 0x55a874b38b40 2024-09-11T23:47:57.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.108+0000 7ff6108e5740 1 -- 172.21.15.133:0/1138849170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6000c0090 con 0x55a874b38b40 2024-09-11T23:47:57.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.108+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1138849170 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7ff5f801d850 con 0x55a874b38b40 2024-09-11T23:47:57.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.109+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1138849170 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff5f802b050 con 0x55a874b38b40 2024-09-11T23:47:57.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.109+0000 7ff6108e5740 10 client.27800 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:57.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.109+0000 7ff6108e5740 1 -- 172.21.15.133:0/1138849170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a874b83920 con 0x55a874b38b40 2024-09-11T23:47:57.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.109+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1138849170 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6000c0090 con 0x55a874b38b40 2024-09-11T23:47:57.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.111+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1138849170 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff5f8026560 con 0x55a874b38b40 2024-09-11T23:47:57.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.111+0000 7ff6108e5740 10 client.27800 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:57.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.111+0000 7ff6108e5740 10 client.27800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:47:57.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.111+0000 7ff6108e5740 20 client.27800 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:57.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.111+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55a874b851c0 0x55a874b87680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.111+0000 7ff6108e5740 4 client.27800 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:47:57.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.111+0000 7ff6108e5740 1 -- 172.21.15.133:0/1138849170 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a874aaa860 con 0x55a874b851c0 2024-09-11T23:47:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.115+0000 7ff607fff640 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55a874b851c0 0x55a874b87680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.121+0000 7ff607fff640 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55a874b851c0 0x55a874b87680 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51eb1d40fdf0126a server_cookie=9a058b255bd04fa in_seq=0 out_seq=0 2024-09-11T23:47:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.121+0000 7ff5e77fe640 10 client.27800 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:47:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.122+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1138849170 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a874aaa860 con 0x55a874b851c0 2024-09-11T23:47:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.122+0000 7ff5e77fe640 10 client.27800 handle_command_reply: tid=0 2024-09-11T23:47:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.123+0000 7ff6108e5740 1 client.27800 shutdown 2024-09-11T23:47:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.123+0000 7ff6108e5740 1 -- 172.21.15.133:0/1138849170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b38b40 msgr2=0x55a874b3f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.123+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b38b40 0x55a874b3f0a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800dec0 tx=0x7ff5f80046c0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.123+0000 7ff6108e5740 1 -- 172.21.15.133:0/1138849170 shutdown_connections 2024-09-11T23:47:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.123+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55a874b851c0 0x55a874b87680 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.123+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b42370 0x55a874b3f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.123+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b38b40 0x55a874b3f0a0 secure :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800dec0 tx=0x7ff5f80046c0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.123+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1138849170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b375d0 0x55a874b41df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.123+0000 7ff6108e5740 1 -- 172.21.15.133:0/1138849170 >> 172.21.15.133:0/1138849170 conn(0x55a874b00310 msgr2=0x55a874b39960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.124+0000 7ff5e77fe640 0 client.27800 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:47:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.124+0000 7ff6108e5740 1 -- 172.21.15.133:0/1138849170 shutdown_connections 2024-09-11T23:47:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.124+0000 7ff6108e5740 1 -- 172.21.15.133:0/1138849170 wait complete. 2024-09-11T23:47:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.125+0000 7ff6108e5740 20 client.27800 trim_cache size 0 max 16384 2024-09-11T23:47:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.155+0000 7ff6108e5740 1 Processor -- start 2024-09-11T23:47:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.163+0000 7ff6108e5740 1 -- start start 2024-09-11T23:47:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.164+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b375d0 0x55a874b33d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.164+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b42370 0x55a874b5f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.164+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b3f180 0x55a874b3e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.164+0000 7ff6108e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff608196550 con 0x55a874b42370 2024-09-11T23:47:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.164+0000 7ff6108e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff608198090 con 0x55a874b3f180 2024-09-11T23:47:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.164+0000 7ff6108e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff608196c20 con 0x55a874b375d0 2024-09-11T23:47:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.168+0000 7ff60caa9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b3f180 0x55a874b3e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.168+0000 7ff60caa9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b3f180 0x55a874b3e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:44084/0 (socket says 172.21.15.133:44084) 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.168+0000 7ff60caa9640 1 -- 172.21.15.133:0/2590996975 learned_addr learned my addr 172.21.15.133:0/2590996975 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.172+0000 7ff60caa9640 1 -- 172.21.15.133:0/2590996975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b375d0 msgr2=0x55a874b33d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.172+0000 7ff60caa9640 1 --2- 172.21.15.133:0/2590996975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b375d0 0x55a874b33d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.172+0000 7ff60caa9640 1 -- 172.21.15.133:0/2590996975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b42370 msgr2=0x55a874b5f0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.172+0000 7ff60caa9640 1 --2- 172.21.15.133:0/2590996975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b42370 0x55a874b5f0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.172+0000 7ff60caa9640 1 -- 172.21.15.133:0/2590996975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a874b3fb50 con 0x55a874b3f180 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.173+0000 7ff60caa9640 1 --2- 172.21.15.133:0/2590996975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b3f180 0x55a874b3e080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6000b8140 tx=0x7ff600032b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c57d35692889866a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.173+0000 7ff5edffb640 1 -- 172.21.15.133:0/2590996975 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6000d1070 con 0x55a874b3f180 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.173+0000 7ff5edffb640 1 -- 172.21.15.133:0/2590996975 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6000d4e50 con 0x55a874b3f180 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.175+0000 7ff6108e5740 1 -- 172.21.15.133:0/2590996975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b3f180 msgr2=0x55a874b3e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.175+0000 7ff6108e5740 1 --2- 172.21.15.133:0/2590996975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b3f180 0x55a874b3e080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6000b8140 tx=0x7ff600032b10 comp rx=0 tx=0).stop 2024-09-11T23:47:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.175+0000 7ff6108e5740 1 -- 172.21.15.133:0/2590996975 shutdown_connections 2024-09-11T23:47:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.175+0000 7ff6108e5740 1 --2- 172.21.15.133:0/2590996975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874b3f180 0x55a874b3e080 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.175+0000 7ff6108e5740 1 --2- 172.21.15.133:0/2590996975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b42370 0x55a874b5f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.175+0000 7ff6108e5740 1 --2- 172.21.15.133:0/2590996975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b375d0 0x55a874b33d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.175+0000 7ff6108e5740 1 -- 172.21.15.133:0/2590996975 >> 172.21.15.133:0/2590996975 conn(0x55a874b00310 msgr2=0x55a874b3d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.177+0000 7ff6108e5740 1 -- 172.21.15.133:0/2590996975 shutdown_connections 2024-09-11T23:47:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.178+0000 7ff6108e5740 1 -- 172.21.15.133:0/2590996975 wait complete. 2024-09-11T23:47:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.178+0000 7ff6108e5740 1 -- start start 2024-09-11T23:47:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.178+0000 7ff6108e5740 1 Processor -- start 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.179+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b5b910 0x55a874b3bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.179+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b3c1d0 0x55a874b3c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.179+0000 7ff6108e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874becf50 0x55a874bef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.179+0000 7ff6108e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081979c0 con 0x55a874b5b910 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.179+0000 7ff6108e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081972f0 con 0x55a874becf50 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.179+0000 7ff6108e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081a8590 con 0x55a874b3c1d0 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.179+0000 7ff6108e5740 10 client.0 _ll_register_callbacks cb 0x7ff5f800b6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.182+0000 7ff60caa9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874becf50 0x55a874bef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.182+0000 7ff6077fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b3c1d0 0x55a874b3c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.183+0000 7ff607fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b5b910 0x55a874b3bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.183+0000 7ff607fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b5b910 0x55a874b3bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34998/0 (socket says 172.21.15.133:34998) 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.183+0000 7ff607fff640 1 -- 172.21.15.133:0/1122121886 learned_addr learned my addr 172.21.15.133:0/1122121886 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.184+0000 7ff607fff640 1 -- 172.21.15.133:0/1122121886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b3c1d0 msgr2=0x55a874b3c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.184+0000 7ff607fff640 1 --2- 172.21.15.133:0/1122121886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b3c1d0 0x55a874b3c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.184+0000 7ff607fff640 1 -- 172.21.15.133:0/1122121886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874becf50 msgr2=0x55a874bef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.184+0000 7ff607fff640 1 --2- 172.21.15.133:0/1122121886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874becf50 0x55a874bef340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.184+0000 7ff607fff640 1 -- 172.21.15.133:0/1122121886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a874bef880 con 0x55a874b5b910 2024-09-11T23:47:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.185+0000 7ff607fff640 1 --2- 172.21.15.133:0/1122121886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b5b910 0x55a874b3bc90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x55a874b87f30 tx=0x7ff5fc0144d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d9f0236ee55efa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.185+0000 7ff5e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:47:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.185+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1122121886 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc020020 con 0x55a874b5b910 2024-09-11T23:47:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.186+0000 7ff6108e5740 1 -- 172.21.15.133:0/1122121886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a874aa4d70 con 0x55a874b5b910 2024-09-11T23:47:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.186+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1122121886 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7ff5fc01c540 con 0x55a874b5b910 2024-09-11T23:47:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.186+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1122121886 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff5fc02b050 con 0x55a874b5b910 2024-09-11T23:47:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.186+0000 7ff6108e5740 10 client.18267 fetch_fsmap learned FSMap version 30 2024-09-11T23:47:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.186+0000 7ff6108e5740 1 -- 172.21.15.133:0/1122121886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a874bf0b60 con 0x55a874b5b910 2024-09-11T23:47:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.189+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1122121886 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a874aa4d70 con 0x55a874b5b910 2024-09-11T23:47:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.189+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1122121886 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff5fc024a60 con 0x55a874b5b910 2024-09-11T23:47:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.190+0000 7ff6108e5740 10 client.18267 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:47:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.190+0000 7ff6108e5740 10 client.18267 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:47:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.190+0000 7ff6108e5740 20 client.18267 populate_metadata read hostname 'smithi133' 2024-09-11T23:47:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.190+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1122121886 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55a874bf12f0 0x55a874bf3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.190+0000 7ff6108e5740 4 client.18267 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:47:57.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.190+0000 7ff6108e5740 1 -- 172.21.15.133:0/1122121886 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a874a21170 con 0x55a874bf12f0 2024-09-11T23:47:57.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:56 smithi138 ceph-mon[35142]: pgmap v348: 97 pgs: 97 active+clean; 69 GiB data, 259 GiB used, 814 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 669 MiB/s wr, 2.24k op/s 2024-09-11T23:47:57.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.196+0000 7ff6077fe640 1 --2- 172.21.15.133:0/1122121886 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55a874bf12f0 0x55a874bf3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.204+0000 7ff6077fe640 1 --2- 172.21.15.133:0/1122121886 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55a874bf12f0 0x55a874bf3710 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d397a864c505c5ba server_cookie=c362b00c7aa0a205 in_seq=0 out_seq=0 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.204+0000 7ff5e77fe640 10 client.18267 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.207+0000 7ff5e77fe640 1 -- 172.21.15.133:0/1122121886 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a874a21170 con 0x55a874bf12f0 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.207+0000 7ff5e77fe640 10 client.18267 handle_command_reply: tid=0 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.207+0000 7ff6108e5740 1 client.18267 shutdown 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.207+0000 7ff6108e5740 1 -- 172.21.15.133:0/1122121886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b5b910 msgr2=0x55a874b3bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.207+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1122121886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b5b910 0x55a874b3bc90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x55a874b87f30 tx=0x7ff5fc0144d0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.209+0000 7ff6108e5740 1 -- 172.21.15.133:0/1122121886 shutdown_connections 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.209+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1122121886 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55a874bf12f0 0x55a874bf3710 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.209+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1122121886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a874becf50 0x55a874bef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.209+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1122121886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a874b3c1d0 0x55a874b3c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.210+0000 7ff6108e5740 1 --2- 172.21.15.133:0/1122121886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a874b5b910 0x55a874b3bc90 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.210+0000 7ff6108e5740 1 -- 172.21.15.133:0/1122121886 >> 172.21.15.133:0/1122121886 conn(0x55a874b00310 msgr2=0x55a874b3e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:57.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.211+0000 7ff5e77fe640 0 client.18267 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:47:57.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.215+0000 7ff6108e5740 1 -- 172.21.15.133:0/1122121886 shutdown_connections 2024-09-11T23:47:57.224 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:47:57.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.220+0000 7ff6108e5740 1 -- 172.21.15.133:0/1122121886 wait complete. 2024-09-11T23:47:57.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.220+0000 7ff6108e5740 20 client.18267 trim_cache size 0 max 16384 2024-09-11T23:47:57.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.225+0000 7ff60e50c640 1 -- 172.21.15.133:0/1419492608 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5f00a2df0 msgr2=0x7ff5f00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:57.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.225+0000 7ff60e50c640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5f00a2df0 0x7ff5f00a52b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff608079ac0 tx=0x7ff5f800b370 comp rx=0 tx=0).stop 2024-09-11T23:47:57.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.225+0000 7ff60e50c640 1 -- 172.21.15.133:0/1419492608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff608109120 msgr2=0x7ff6081abb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:57.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.225+0000 7ff60e50c640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff608109120 0x7ff6081abb00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff6000135b0 tx=0x7ff600034e70 comp rx=0 tx=0).stop 2024-09-11T23:47:57.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.226+0000 7ff60e50c640 1 -- 172.21.15.133:0/1419492608 shutdown_connections 2024-09-11T23:47:57.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.226+0000 7ff60e50c640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff5f00a2df0 0x7ff5f00a52b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.226+0000 7ff60e50c640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff608109120 0x7ff6081abb00 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.226+0000 7ff60e50c640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6081049a0 0x7ff6081a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.226+0000 7ff60e50c640 1 --2- 172.21.15.133:0/1419492608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff608103ff0 0x7ff6081a76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:57.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.226+0000 7ff60e50c640 1 -- 172.21.15.133:0/1419492608 >> 172.21.15.133:0/1419492608 conn(0x7ff6080ffab0 msgr2=0x7ff6081079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:57.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.228+0000 7ff60e50c640 1 -- 172.21.15.133:0/1419492608 shutdown_connections 2024-09-11T23:47:57.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:57.231+0000 7ff60e50c640 1 -- 172.21.15.133:0/1419492608 wait complete. 2024-09-11T23:47:58.459 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:47:59.053 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:47:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:47:58 smithi138 ceph-mon[35142]: pgmap v349: 97 pgs: 97 active+clean; 69 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 605 MiB/s wr, 1.78k op/s 2024-09-11T23:47:59.219 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:47:58 smithi133 ceph-mon[31430]: pgmap v349: 97 pgs: 97 active+clean; 69 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 605 MiB/s wr, 1.78k op/s 2024-09-11T23:47:59.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:47:58 smithi178 ceph-mon[35882]: pgmap v349: 97 pgs: 97 active+clean; 69 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 605 MiB/s wr, 1.78k op/s 2024-09-11T23:47:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.956+0000 7fd1eb428640 1 -- 172.21.15.133:0/4165971107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1e411b140 msgr2=0x7fd1e411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.956+0000 7fd1e9424640 1 -- 172.21.15.133:0/4165971107 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc045020 con 0x7fd1e411b140 2024-09-11T23:47:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.956+0000 7fd1eb428640 1 --2- 172.21.15.133:0/4165971107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1e411b140 0x7fd1e411d530 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc01acb0 tx=0x7fd1dc03b920 comp rx=0 tx=0).stop 2024-09-11T23:47:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.957+0000 7fd1eb428640 1 -- 172.21.15.133:0/4165971107 shutdown_connections 2024-09-11T23:47:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.957+0000 7fd1eb428640 1 --2- 172.21.15.133:0/4165971107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1e411b140 0x7fd1e411d530 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.957+0000 7fd1eb428640 1 --2- 172.21.15.133:0/4165971107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e406ea30 0x7fd1e406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.957+0000 7fd1eb428640 1 --2- 172.21.15.133:0/4165971107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1e4110190 0x7fd1e406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.957+0000 7fd1eb428640 1 -- 172.21.15.133:0/4165971107 >> 172.21.15.133:0/4165971107 conn(0x7fd1e4073180 msgr2=0x7fd1e4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:47:59.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.959+0000 7fd1eb428640 1 -- 172.21.15.133:0/4165971107 shutdown_connections 2024-09-11T23:47:59.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.961+0000 7fd1eb428640 1 -- 172.21.15.133:0/4165971107 wait complete. 2024-09-11T23:47:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.962+0000 7fd1eb428640 1 Processor -- start 2024-09-11T23:47:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.967+0000 7fd1eb428640 1 -- start start 2024-09-11T23:47:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.968+0000 7fd1eb428640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1e406ea30 0x7fd1e411a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.968+0000 7fd1eb428640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e4110190 0x7fd1e4116950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.968+0000 7fd1eb428640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1e411b140 0x7fd1e4116e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.968+0000 7fd1eb428640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e4113360 con 0x7fd1e411b140 2024-09-11T23:47:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.968+0000 7fd1eb428640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e41131e0 con 0x7fd1e406ea30 2024-09-11T23:47:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.968+0000 7fd1eb428640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e41134e0 con 0x7fd1e4110190 2024-09-11T23:47:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.969+0000 7fd1e9c25640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e4110190 0x7fd1e4116950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.969+0000 7fd1e9c25640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e4110190 0x7fd1e4116950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:59664/0 (socket says 172.21.15.133:59664) 2024-09-11T23:47:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.969+0000 7fd1e9c25640 1 -- 172.21.15.133:0/3301783943 learned_addr learned my addr 172.21.15.133:0/3301783943 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.970+0000 7fd1ea426640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1e406ea30 0x7fd1e411a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.972+0000 7fd1e9c25640 1 -- 172.21.15.133:0/3301783943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1e406ea30 msgr2=0x7fd1e411a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.972+0000 7fd1e9c25640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1e406ea30 0x7fd1e411a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.972+0000 7fd1e9c25640 1 -- 172.21.15.133:0/3301783943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1e411b140 msgr2=0x7fd1e4116e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:47:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.972+0000 7fd1e9c25640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1e411b140 0x7fd1e4116e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.972+0000 7fd1e9c25640 1 -- 172.21.15.133:0/3301783943 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e4117590 con 0x7fd1e4110190 2024-09-11T23:47:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.977+0000 7fd1e9c25640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e4110190 0x7fd1e4116950 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0030a00 tx=0x7fd1e0030ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.977+0000 7fd1db7fe640 1 -- 172.21.15.133:0/3301783943 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e002ed90 con 0x7fd1e4110190 2024-09-11T23:47:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.977+0000 7fd1eb428640 1 -- 172.21.15.133:0/3301783943 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e4074420 con 0x7fd1e4110190 2024-09-11T23:47:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.977+0000 7fd1eb428640 1 -- 172.21.15.133:0/3301783943 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e40749b0 con 0x7fd1e4110190 2024-09-11T23:47:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.978+0000 7fd1db7fe640 1 -- 172.21.15.133:0/3301783943 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e0032920 con 0x7fd1e4110190 2024-09-11T23:47:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.978+0000 7fd1db7fe640 1 -- 172.21.15.133:0/3301783943 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e003a990 con 0x7fd1e4110190 2024-09-11T23:47:59.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.985+0000 7fd1db7fe640 1 -- 172.21.15.133:0/3301783943 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd1e003ab30 con 0x7fd1e4110190 2024-09-11T23:47:59.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.990+0000 7fd1eea63740 1 Processor -- start 2024-09-11T23:47:59.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.990+0000 7fd1eea63740 1 -- start start 2024-09-11T23:47:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.990+0000 7fd1eea63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 0x559ab5205270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.990+0000 7fd1eea63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab52057b0 0x559ab5205bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.990+0000 7fd1eea63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5210550 0x559ab5212940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.990+0000 7fd1eea63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559ab516b2f0 con 0x559ab5206d20 2024-09-11T23:47:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.991+0000 7fd1eea63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559ab5112cf0 con 0x559ab5210550 2024-09-11T23:47:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.991+0000 7fd1eea63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559ab51672b0 con 0x559ab52057b0 2024-09-11T23:47:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.992+0000 7fd1eac27640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 0x559ab5205270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.992+0000 7fd1db7fe640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1d00a2e30 0x7fd1d00a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:47:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.992+0000 7fd1eac27640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 0x559ab5205270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35020/0 (socket says 172.21.15.133:35020) 2024-09-11T23:47:59.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.992+0000 7fd1eac27640 1 -- 172.21.15.133:0/273102253 learned_addr learned my addr 172.21.15.133:0/273102253 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:47:59.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.993+0000 7fd1db7fe640 1 -- 172.21.15.133:0/3301783943 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fd1e00f1700 con 0x7fd1e4110190 2024-09-11T23:47:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.994+0000 7fd1ea426640 1 --2- 172.21.15.133:0/273102253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab52057b0 0x559ab5205bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.994+0000 7fd1eac27640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1d00a2e30 0x7fd1d00a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.994+0000 7fd1eac27640 1 -- 172.21.15.133:0/273102253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab52057b0 msgr2=0x559ab5205bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.994+0000 7fd1e9c25640 1 --2- 172.21.15.133:0/273102253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5210550 0x559ab5212940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:47:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.994+0000 7fd1eac27640 1 --2- 172.21.15.133:0/273102253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab52057b0 0x559ab5205bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:59.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.994+0000 7fd1eac27640 1 -- 172.21.15.133:0/273102253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5210550 msgr2=0x559ab5212940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:47:59.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.994+0000 7fd1eac27640 1 --2- 172.21.15.133:0/273102253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5210550 0x559ab5212940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:47:59.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.994+0000 7fd1eac27640 1 -- 172.21.15.133:0/273102253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ab50f73a0 con 0x559ab5206d20 2024-09-11T23:47:59.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.995+0000 7fd1eac27640 1 --2- 172.21.15.133:0/273102253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 0x559ab5205270 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc03bf10 tx=0x7fd1dc0407f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1655dcb0e4c0b6ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:47:59.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.997+0000 7fd1d8ff9640 1 -- 172.21.15.133:0/273102253 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc045030 con 0x559ab5206d20 2024-09-11T23:47:59.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.997+0000 7fd1d8ff9640 1 -- 172.21.15.133:0/273102253 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc003e10 con 0x559ab5206d20 2024-09-11T23:47:59.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.998+0000 7fd1d8ff9640 1 -- 172.21.15.133:0/273102253 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc041ac0 con 0x559ab5206d20 2024-09-11T23:48:00.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.999+0000 7fd1eea63740 1 -- 172.21.15.133:0/273102253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 msgr2=0x559ab5205270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.999+0000 7fd1eea63740 1 --2- 172.21.15.133:0/273102253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 0x559ab5205270 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc03bf10 tx=0x7fd1dc0407f0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:47:59.999+0000 7fd1eac27640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1d00a2e30 0x7fd1d00a52f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc05b000 tx=0x7fd1dc05c000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.003+0000 7fd1eea63740 1 -- 172.21.15.133:0/273102253 shutdown_connections 2024-09-11T23:48:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.003+0000 7fd1eea63740 1 --2- 172.21.15.133:0/273102253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5210550 0x559ab5212940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.003+0000 7fd1eea63740 1 --2- 172.21.15.133:0/273102253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab52057b0 0x559ab5205bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.003+0000 7fd1eea63740 1 --2- 172.21.15.133:0/273102253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 0x559ab5205270 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.003+0000 7fd1eea63740 1 -- 172.21.15.133:0/273102253 >> 172.21.15.133:0/273102253 conn(0x559ab51ce310 msgr2=0x559ab5201e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.003+0000 7fd1eea63740 1 -- 172.21.15.133:0/273102253 shutdown_connections 2024-09-11T23:48:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.003+0000 7fd1eea63740 1 -- 172.21.15.133:0/273102253 wait complete. 2024-09-11T23:48:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.004+0000 7fd1eea63740 1 -- start start 2024-09-11T23:48:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.004+0000 7fd1eea63740 1 Processor -- start 2024-09-11T23:48:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.005+0000 7fd1eea63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab52057b0 0x559ab520fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.005+0000 7fd1eea63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 0x559ab520d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.005+0000 7fd1eea63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 0x559ab520d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.005+0000 7fd1eea63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559ab52132a0 con 0x559ab5206d20 2024-09-11T23:48:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.005+0000 7fd1eea63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559ab5126060 con 0x559ab52057b0 2024-09-11T23:48:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.005+0000 7fd1eea63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559ab5213420 con 0x559ab5210550 2024-09-11T23:48:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.005+0000 7fd1eea63740 10 client.0 _ll_register_callbacks cb 0x559ab4e798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.006+0000 7fd1eac27640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 0x559ab520d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.006+0000 7fd1eac27640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 0x559ab520d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:59684/0 (socket says 172.21.15.133:59684) 2024-09-11T23:48:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.006+0000 7fd1eac27640 1 -- 172.21.15.133:0/2345541490 learned_addr learned my addr 172.21.15.133:0/2345541490 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.006+0000 7fd1ea426640 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab52057b0 0x559ab520fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.006+0000 7fd1e9c25640 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 0x559ab520d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.007+0000 7fd1ea426640 1 -- 172.21.15.133:0/2345541490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 msgr2=0x559ab520d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.007+0000 7fd1ea426640 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 0x559ab520d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.007+0000 7fd1ea426640 1 -- 172.21.15.133:0/2345541490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 msgr2=0x559ab520d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.007+0000 7fd1ea426640 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 0x559ab520d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.007+0000 7fd1ea426640 1 -- 172.21.15.133:0/2345541490 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ab4edc3c0 con 0x559ab52057b0 2024-09-11T23:48:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.007+0000 7fd1ea426640 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab52057b0 0x559ab520fe10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4006fd0 tx=0x7fd1d4011630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb947083ceb6544 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.007+0000 7fd1cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:48:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.007+0000 7fd1cdffb640 1 -- 172.21.15.133:0/2345541490 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4019070 con 0x559ab52057b0 2024-09-11T23:48:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.007+0000 7fd1eea63740 1 -- 172.21.15.133:0/2345541490 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1dc003e10 con 0x559ab52057b0 2024-09-11T23:48:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.007+0000 7fd1cdffb640 1 -- 172.21.15.133:0/2345541490 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fd1d4015660 con 0x559ab52057b0 2024-09-11T23:48:00.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.030+0000 7fd1cdffb640 1 -- 172.21.15.133:0/2345541490 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd1d4025050 con 0x559ab52057b0 2024-09-11T23:48:00.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.031+0000 7fd1eea63740 10 client.27812 fetch_fsmap learned FSMap version 30 2024-09-11T23:48:00.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.031+0000 7fd1eea63740 1 -- 172.21.15.133:0/2345541490 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559ab521ed70 con 0x559ab52057b0 2024-09-11T23:48:00.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.032+0000 7fd1cdffb640 1 -- 172.21.15.133:0/2345541490 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc003e10 con 0x559ab52057b0 2024-09-11T23:48:00.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.032+0000 7fd1cdffb640 1 -- 172.21.15.133:0/2345541490 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd1d4014b80 con 0x559ab52057b0 2024-09-11T23:48:00.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.032+0000 7fd1eea63740 10 client.27812 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:48:00.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.032+0000 7fd1eea63740 10 client.27812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:00.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.032+0000 7fd1eea63740 20 client.27812 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:00.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.032+0000 7fd1eea63740 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559ab5220690 0x559ab5222b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.032+0000 7fd1eea63740 4 client.27812 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:00.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.032+0000 7fd1eea63740 1 -- 172.21.15.133:0/2345541490 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ab5178860 con 0x559ab5220690 2024-09-11T23:48:00.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.035+0000 7fd1e9c25640 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559ab5220690 0x559ab5222b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.039+0000 7fd1e9c25640 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559ab5220690 0x559ab5222b50 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=987c150192a03ee3 server_cookie=c80ef9507deba078 in_seq=0 out_seq=0 2024-09-11T23:48:00.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.039+0000 7fd1cdffb640 10 client.27812 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:00.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.041+0000 7fd1cdffb640 1 -- 172.21.15.133:0/2345541490 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ab5178860 con 0x559ab5220690 2024-09-11T23:48:00.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.041+0000 7fd1cdffb640 10 client.27812 handle_command_reply: tid=0 2024-09-11T23:48:00.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.042+0000 7fd1eea63740 1 client.27812 shutdown 2024-09-11T23:48:00.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.042+0000 7fd1eea63740 1 -- 172.21.15.133:0/2345541490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab52057b0 msgr2=0x559ab520fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.042+0000 7fd1eea63740 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab52057b0 0x559ab520fe10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4006fd0 tx=0x7fd1d4011630 comp rx=0 tx=0).stop 2024-09-11T23:48:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.044+0000 7fd1eea63740 1 -- 172.21.15.133:0/2345541490 shutdown_connections 2024-09-11T23:48:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.044+0000 7fd1eea63740 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559ab5220690 0x559ab5222b50 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.044+0000 7fd1eea63740 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 0x559ab520d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.044+0000 7fd1eea63740 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab5206d20 0x559ab520d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.044+0000 7fd1eea63740 1 --2- 172.21.15.133:0/2345541490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab52057b0 0x559ab520fe10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.044+0000 7fd1eea63740 1 -- 172.21.15.133:0/2345541490 >> 172.21.15.133:0/2345541490 conn(0x559ab51ce310 msgr2=0x559ab5207b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.044+0000 7fd1cdffb640 0 client.27812 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.044+0000 7fd1eea63740 1 -- 172.21.15.133:0/2345541490 shutdown_connections 2024-09-11T23:48:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.044+0000 7fd1eea63740 1 -- 172.21.15.133:0/2345541490 wait complete. 2024-09-11T23:48:00.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.045+0000 7fd1eea63740 20 client.27812 trim_cache size 0 max 16384 2024-09-11T23:48:00.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.073+0000 7fd1eea63740 1 Processor -- start 2024-09-11T23:48:00.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.073+0000 7fd1eea63740 1 -- start start 2024-09-11T23:48:00.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.074+0000 7fd1eea63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 0x559ab521c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.075+0000 7fd1eea63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5206d20 0x559ab520c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.075+0000 7fd1eea63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 0x559ab520b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.075+0000 7fd1eea63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c800e770 con 0x559ab52057b0 2024-09-11T23:48:00.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.075+0000 7fd1eea63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c80102b0 con 0x559ab5206d20 2024-09-11T23:48:00.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.075+0000 7fd1eea63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c800ee40 con 0x559ab5210550 2024-09-11T23:48:00.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.076+0000 7fd1e9c25640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5206d20 0x559ab520c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.076+0000 7fd1e9c25640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5206d20 0x559ab520c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:44154/0 (socket says 172.21.15.133:44154) 2024-09-11T23:48:00.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.076+0000 7fd1e9c25640 1 -- 172.21.15.133:0/568270186 learned_addr learned my addr 172.21.15.133:0/568270186 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:00.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.076+0000 7fd1eac27640 1 --2- 172.21.15.133:0/568270186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 0x559ab520b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.077+0000 7fd1ea426640 1 --2- 172.21.15.133:0/568270186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 0x559ab521c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.078+0000 7fd1eac27640 1 -- 172.21.15.133:0/568270186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5206d20 msgr2=0x559ab520c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.078+0000 7fd1eac27640 1 --2- 172.21.15.133:0/568270186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5206d20 0x559ab520c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.078+0000 7fd1eac27640 1 -- 172.21.15.133:0/568270186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 msgr2=0x559ab521c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.078+0000 7fd1eac27640 1 --2- 172.21.15.133:0/568270186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 0x559ab521c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.078+0000 7fd1eac27640 1 -- 172.21.15.133:0/568270186 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ab5208300 con 0x559ab5210550 2024-09-11T23:48:00.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.079+0000 7fd1eac27640 1 --2- 172.21.15.133:0/568270186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 0x559ab520b990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc041660 tx=0x7fd1dc051000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a5f87b6e2150b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:00.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.079+0000 7fd1cffff640 1 -- 172.21.15.133:0/568270186 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc06a070 con 0x559ab5210550 2024-09-11T23:48:00.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.079+0000 7fd1cffff640 1 -- 172.21.15.133:0/568270186 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc041d20 con 0x559ab5210550 2024-09-11T23:48:00.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.079+0000 7fd1cffff640 1 -- 172.21.15.133:0/568270186 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc045040 con 0x559ab5210550 2024-09-11T23:48:00.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.082+0000 7fd1eea63740 1 -- 172.21.15.133:0/568270186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 msgr2=0x559ab520b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.082+0000 7fd1eea63740 1 --2- 172.21.15.133:0/568270186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 0x559ab520b990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc041660 tx=0x7fd1dc051000 comp rx=0 tx=0).stop 2024-09-11T23:48:00.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.082+0000 7fd1eea63740 1 -- 172.21.15.133:0/568270186 shutdown_connections 2024-09-11T23:48:00.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.083+0000 7fd1eea63740 1 --2- 172.21.15.133:0/568270186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5210550 0x559ab520b990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.083+0000 7fd1eea63740 1 --2- 172.21.15.133:0/568270186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5206d20 0x559ab520c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.083+0000 7fd1eea63740 1 --2- 172.21.15.133:0/568270186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 0x559ab521c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.083+0000 7fd1eea63740 1 -- 172.21.15.133:0/568270186 >> 172.21.15.133:0/568270186 conn(0x559ab51ce310 msgr2=0x559ab520d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:00.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.083+0000 7fd1eea63740 1 -- 172.21.15.133:0/568270186 shutdown_connections 2024-09-11T23:48:00.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.083+0000 7fd1eea63740 1 -- 172.21.15.133:0/568270186 wait complete. 2024-09-11T23:48:00.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.084+0000 7fd1eea63740 1 -- start start 2024-09-11T23:48:00.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.084+0000 7fd1eea63740 1 Processor -- start 2024-09-11T23:48:00.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.085+0000 7fd1eea63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 0x559ab52187c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.085+0000 7fd1ea426640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 0x559ab52187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.085+0000 7fd1ea426640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 0x559ab52187c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35056/0 (socket says 172.21.15.133:35056) 2024-09-11T23:48:00.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.086+0000 7fd1ea426640 1 -- 172.21.15.133:0/3167862261 learned_addr learned my addr 172.21.15.133:0/3167862261 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:00.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.086+0000 7fd1eea63740 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5206d20 0x559ab5218d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.086+0000 7fd1eea63740 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5210550 0x559ab5213d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.086+0000 7fd1eea63740 1 -- 172.21.15.133:0/3167862261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c800fbe0 con 0x559ab52057b0 2024-09-11T23:48:00.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.086+0000 7fd1eea63740 1 -- 172.21.15.133:0/3167862261 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c800f510 con 0x559ab5210550 2024-09-11T23:48:00.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.086+0000 7fd1eea63740 1 -- 172.21.15.133:0/3167862261 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c800c770 con 0x559ab5206d20 2024-09-11T23:48:00.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.086+0000 7fd1e9c25640 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5206d20 0x559ab5218d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.087+0000 7fd1eac27640 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5210550 0x559ab5213d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.087+0000 7fd1eea63740 10 client.0 _ll_register_callbacks cb 0x7fd1d4009430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:00.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.087+0000 7fd1ea426640 1 -- 172.21.15.133:0/3167862261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5206d20 msgr2=0x559ab5218d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.087+0000 7fd1ea426640 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5206d20 0x559ab5218d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.087+0000 7fd1ea426640 1 -- 172.21.15.133:0/3167862261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5210550 msgr2=0x559ab5213d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.087+0000 7fd1ea426640 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5210550 0x559ab5213d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.087+0000 7fd1ea426640 1 -- 172.21.15.133:0/3167862261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ab5214510 con 0x559ab52057b0 2024-09-11T23:48:00.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.089+0000 7fd1ea426640 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 0x559ab52187c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4008010 tx=0x7fd1d401d770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9427cde6b1e11d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:00.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.089+0000 7fd1cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.103+0000 7faf5d5c5640 1 -- 172.21.15.133:0/2736696499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf58100af0 msgr2=0x7faf5810cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.103+0000 7faf5d5c5640 1 --2- 172.21.15.133:0/2736696499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf58100af0 0x7faf5810cdb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7faf48009ca0 tx=0x7faf4802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.104+0000 7faf5d5c5640 1 -- 172.21.15.133:0/2736696499 shutdown_connections 2024-09-11T23:48:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.104+0000 7faf5d5c5640 1 --2- 172.21.15.133:0/2736696499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf58100af0 0x7faf5810cdb0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.104+0000 7faf5d5c5640 1 --2- 172.21.15.133:0/2736696499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf58100150 0x7faf581005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.103+0000 7fd1cdffb640 1 -- 172.21.15.133:0/3167862261 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d402d030 con 0x559ab52057b0 2024-09-11T23:48:00.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.103+0000 7fd1eea63740 1 -- 172.21.15.133:0/3167862261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1dc041d20 con 0x559ab52057b0 2024-09-11T23:48:00.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.103+0000 7fd1cdffb640 1 -- 172.21.15.133:0/3167862261 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fd1d4005990 con 0x559ab52057b0 2024-09-11T23:48:00.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.104+0000 7fd1cdffb640 1 -- 172.21.15.133:0/3167862261 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd1d4019050 con 0x559ab52057b0 2024-09-11T23:48:00.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.104+0000 7fd1eea63740 10 client.18279 fetch_fsmap learned FSMap version 30 2024-09-11T23:48:00.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.104+0000 7fd1eea63740 1 -- 172.21.15.133:0/3167862261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559ab528adb0 con 0x559ab52057b0 2024-09-11T23:48:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.104+0000 7faf5d5c5640 1 --2- 172.21.15.133:0/2736696499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf581060a0 0x7faf58106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.104+0000 7faf5d5c5640 1 -- 172.21.15.133:0/2736696499 >> 172.21.15.133:0/2736696499 conn(0x7faf58077200 msgr2=0x7faf58077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.104+0000 7faf5d5c5640 1 -- 172.21.15.133:0/2736696499 shutdown_connections 2024-09-11T23:48:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.104+0000 7faf5d5c5640 1 -- 172.21.15.133:0/2736696499 wait complete. 2024-09-11T23:48:00.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.105+0000 7fd1cdffb640 1 -- 172.21.15.133:0/3167862261 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc041d20 con 0x559ab52057b0 2024-09-11T23:48:00.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.105+0000 7fd1cdffb640 1 -- 172.21.15.133:0/3167862261 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd1d4004310 con 0x559ab52057b0 2024-09-11T23:48:00.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.105+0000 7fd1eea63740 10 client.18279 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:48:00.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.105+0000 7fd1eea63740 10 client.18279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:00.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.105+0000 7fd1eea63740 20 client.18279 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:00.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.105+0000 7fd1eea63740 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559ab528b500 0x559ab528d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.105+0000 7fd1eea63740 4 client.18279 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:48:00.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.105+0000 7fd1eea63740 1 -- 172.21.15.133:0/3167862261 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ab5172d70 con 0x559ab528b500 2024-09-11T23:48:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.106+0000 7faf5d5c5640 1 Processor -- start 2024-09-11T23:48:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.107+0000 7faf5d5c5640 1 -- start start 2024-09-11T23:48:00.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.106+0000 7fd1e9c25640 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559ab528b500 0x559ab528d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.107+0000 7faf5d5c5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf58100150 0x7faf58071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.107+0000 7faf5d5c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf58100af0 0x7faf580723e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.107+0000 7faf5d5c5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf581060a0 0x7faf58072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.107+0000 7faf5d5c5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf58110320 con 0x7faf581060a0 2024-09-11T23:48:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.107+0000 7faf5d5c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf581101a0 con 0x7faf58100af0 2024-09-11T23:48:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.108+0000 7faf5d5c5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf581104a0 con 0x7faf58100150 2024-09-11T23:48:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.108+0000 7faf57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf58100150 0x7faf58071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.108+0000 7faf57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf58100150 0x7faf58071ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:59716/0 (socket says 172.21.15.133:59716) 2024-09-11T23:48:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.108+0000 7faf57fff640 1 -- 172.21.15.133:0/1213877781 learned_addr learned my addr 172.21.15.133:0/1213877781 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:00.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.108+0000 7fd1e9c25640 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559ab528b500 0x559ab528d920 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a97d4ae61c5720 server_cookie=877f9a05f6b5ee44 in_seq=0 out_seq=0 2024-09-11T23:48:00.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.108+0000 7fd1cdffb640 10 client.18279 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:00.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.109+0000 7fd1cdffb640 1 -- 172.21.15.133:0/3167862261 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ab5172d70 con 0x559ab528b500 2024-09-11T23:48:00.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.109+0000 7fd1cdffb640 10 client.18279 handle_command_reply: tid=0 2024-09-11T23:48:00.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.109+0000 7fd1eea63740 1 client.18279 shutdown 2024-09-11T23:48:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.109+0000 7faf577fe640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf58100af0 0x7faf580723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.109+0000 7fd1eea63740 1 -- 172.21.15.133:0/3167862261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 msgr2=0x559ab52187c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.109+0000 7fd1eea63740 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 0x559ab52187c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4008010 tx=0x7fd1d401d770 comp rx=0 tx=0).stop 2024-09-11T23:48:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.109+0000 7faf57fff640 1 -- 172.21.15.133:0/1213877781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf58100af0 msgr2=0x7faf580723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.109+0000 7faf57fff640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf58100af0 0x7faf580723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.109+0000 7faf57fff640 1 -- 172.21.15.133:0/1213877781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf581060a0 msgr2=0x7faf58072920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.109+0000 7faf57fff640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf581060a0 0x7faf58072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.109+0000 7faf57fff640 1 -- 172.21.15.133:0/1213877781 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf581acd60 con 0x7faf58100150 2024-09-11T23:48:00.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.109+0000 7fd1eea63740 1 -- 172.21.15.133:0/3167862261 shutdown_connections 2024-09-11T23:48:00.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.109+0000 7fd1eea63740 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x559ab528b500 0x559ab528d920 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.110+0000 7faf57fff640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf58100150 0x7faf58071ea0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faf5000d770 tx=0x7faf5000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:00.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.110+0000 7fd1cdffb640 0 client.18279 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:00.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.110+0000 7fd1eea63740 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ab5210550 0x559ab5213d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.110+0000 7fd1eea63740 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559ab5206d20 0x559ab5218d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.110+0000 7fd1eea63740 1 --2- 172.21.15.133:0/3167862261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ab52057b0 0x559ab52187c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.110+0000 7fd1eea63740 1 -- 172.21.15.133:0/3167862261 >> 172.21.15.133:0/3167862261 conn(0x559ab51ce310 msgr2=0x559ab520d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:00.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.110+0000 7fd1eea63740 1 -- 172.21.15.133:0/3167862261 shutdown_connections 2024-09-11T23:48:00.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.110+0000 7fd1eea63740 1 -- 172.21.15.133:0/3167862261 wait complete. 2024-09-11T23:48:00.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.110+0000 7fd1eea63740 20 client.18279 trim_cache size 0 max 16384 2024-09-11T23:48:00.113 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:48:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.111+0000 7faf557fa640 1 -- 172.21.15.133:0/1213877781 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50017070 con 0x7faf58100150 2024-09-11T23:48:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.112+0000 7faf557fa640 1 -- 172.21.15.133:0/1213877781 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf50005500 con 0x7faf58100150 2024-09-11T23:48:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.112+0000 7faf5d5c5640 1 -- 172.21.15.133:0/1213877781 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf581acff0 con 0x7faf58100150 2024-09-11T23:48:00.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.112+0000 7fd1eb428640 1 -- 172.21.15.133:0/3301783943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1d00a2e30 msgr2=0x7fd1d00a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.112+0000 7fd1eb428640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1d00a2e30 0x7fd1d00a52f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc05b000 tx=0x7fd1dc05c000 comp rx=0 tx=0).stop 2024-09-11T23:48:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.112+0000 7faf557fa640 1 -- 172.21.15.133:0/1213877781 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50004d90 con 0x7faf58100150 2024-09-11T23:48:00.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.112+0000 7fd1eb428640 1 -- 172.21.15.133:0/3301783943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e4110190 msgr2=0x7fd1e4116950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.112+0000 7faf5d5c5640 1 -- 172.21.15.133:0/1213877781 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf581ad450 con 0x7faf58100150 2024-09-11T23:48:00.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.112+0000 7fd1eb428640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e4110190 0x7fd1e4116950 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0030a00 tx=0x7fd1e0030ec0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.113+0000 7fd1eb428640 1 -- 172.21.15.133:0/3301783943 shutdown_connections 2024-09-11T23:48:00.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.113+0000 7fd1eb428640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fd1d00a2e30 0x7fd1d00a52f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.113+0000 7fd1eb428640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1e411b140 0x7fd1e4116e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.113+0000 7fd1eb428640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1e4110190 0x7fd1e4116950 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.113+0000 7fd1eb428640 1 --2- 172.21.15.133:0/3301783943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1e406ea30 0x7fd1e411a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.113+0000 7fd1eb428640 1 -- 172.21.15.133:0/3301783943 >> 172.21.15.133:0/3301783943 conn(0x7fd1e4073180 msgr2=0x7fd1e411b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:00.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.113+0000 7fd1eb428640 1 -- 172.21.15.133:0/3301783943 shutdown_connections 2024-09-11T23:48:00.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:00.113+0000 7fd1eb428640 1 -- 172.21.15.133:0/3301783943 wait complete. 2024-09-11T23:48:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.125+0000 7faf557fa640 1 -- 172.21.15.133:0/1213877781 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7faf50010460 con 0x7faf58100150 2024-09-11T23:48:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.127+0000 7faf615c0740 1 Processor -- start 2024-09-11T23:48:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.127+0000 7faf615c0740 1 -- start start 2024-09-11T23:48:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.128+0000 7faf615c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99ecc0 0x55bccd99f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.128+0000 7faf615c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99f5e0 0x55bccd9a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.128+0000 7faf615c0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd9a9e80 0x55bccd9ac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.128+0000 7faf615c0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bccd9032f0 con 0x55bccd99ecc0 2024-09-11T23:48:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.128+0000 7faf615c0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bccd8ab270 con 0x55bccd9a9e80 2024-09-11T23:48:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.128+0000 7faf615c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bccd8ff2b0 con 0x55bccd99f5e0 2024-09-11T23:48:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.129+0000 7faf557fa640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faf440a2f00 0x7faf440a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.129+0000 7faf57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99f5e0 0x55bccd9a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.129+0000 7faf57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99f5e0 0x55bccd9a9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:59720/0 (socket says 172.21.15.133:59720) 2024-09-11T23:48:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.129+0000 7faf57fff640 1 -- 172.21.15.133:0/1706722139 learned_addr learned my addr 172.21.15.133:0/1706722139 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.130+0000 7faf557fa640 1 -- 172.21.15.133:0/1213877781 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7faf500cee50 con 0x7faf58100150 2024-09-11T23:48:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.130+0000 7faf577fe640 1 --2- 172.21.15.133:0/1706722139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd9a9e80 0x55bccd9ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.131+0000 7faf57fff640 1 -- 172.21.15.133:0/1706722139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd9a9e80 msgr2=0x55bccd9ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.131+0000 7faf57fff640 1 --2- 172.21.15.133:0/1706722139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd9a9e80 0x55bccd9ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.131+0000 7faf57fff640 1 -- 172.21.15.133:0/1706722139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99ecc0 msgr2=0x55bccd99f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.131+0000 7faf57fff640 1 --2- 172.21.15.133:0/1706722139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99ecc0 0x55bccd99f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.131+0000 7faf57fff640 1 -- 172.21.15.133:0/1706722139 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bccd8aa340 con 0x55bccd99f5e0 2024-09-11T23:48:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.132+0000 7faf5cdc4640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faf440a2f00 0x7faf440a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.134+0000 7faf5cdc4640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faf440a2f00 0x7faf440a53c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faf48006010 tx=0x7faf48005f80 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.139+0000 7faf57fff640 1 --2- 172.21.15.133:0/1706722139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99f5e0 0x55bccd9a9940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faf50009610 tx=0x7faf5000df50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eadde6c771408f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.140+0000 7faf4e7fc640 1 -- 172.21.15.133:0/1706722139 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50020050 con 0x55bccd99f5e0 2024-09-11T23:48:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.140+0000 7faf4e7fc640 1 -- 172.21.15.133:0/1706722139 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf5009ba50 con 0x55bccd99f5e0 2024-09-11T23:48:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.142+0000 7faf615c0740 1 -- 172.21.15.133:0/1706722139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99f5e0 msgr2=0x55bccd9a9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.142+0000 7faf615c0740 1 --2- 172.21.15.133:0/1706722139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99f5e0 0x55bccd9a9940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faf50009610 tx=0x7faf5000df50 comp rx=0 tx=0).stop 2024-09-11T23:48:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.157+0000 7faf615c0740 1 -- 172.21.15.133:0/1706722139 shutdown_connections 2024-09-11T23:48:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.157+0000 7faf615c0740 1 --2- 172.21.15.133:0/1706722139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd9a9e80 0x55bccd9ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.157+0000 7faf615c0740 1 --2- 172.21.15.133:0/1706722139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99f5e0 0x55bccd9a9940 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.157+0000 7faf615c0740 1 --2- 172.21.15.133:0/1706722139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99ecc0 0x55bccd99f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.157+0000 7faf615c0740 1 -- 172.21.15.133:0/1706722139 >> 172.21.15.133:0/1706722139 conn(0x55bccd99b230 msgr2=0x55bccd99b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.159+0000 7faf615c0740 1 -- 172.21.15.133:0/1706722139 shutdown_connections 2024-09-11T23:48:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.160+0000 7faf615c0740 1 -- 172.21.15.133:0/1706722139 wait complete. 2024-09-11T23:48:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.161+0000 7faf615c0740 1 -- start start 2024-09-11T23:48:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.162+0000 7faf615c0740 1 Processor -- start 2024-09-11T23:48:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.162+0000 7faf615c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99ecc0 0x55bccd9a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.162+0000 7faf615c0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 0x55bccd9a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.163+0000 7faf615c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd9a9e80 0x55bccd9a77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.163+0000 7faf615c0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bccd9aca40 con 0x55bccd99ecc0 2024-09-11T23:48:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.163+0000 7faf615c0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bccd88c290 con 0x55bccd99f5e0 2024-09-11T23:48:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.163+0000 7faf615c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bccd9acbc0 con 0x55bccd9a9e80 2024-09-11T23:48:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.163+0000 7faf615c0740 10 client.0 _ll_register_callbacks cb 0x55bccd8bd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.164+0000 7faf577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 0x55bccd9a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.164+0000 7faf577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 0x55bccd9a7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:44190/0 (socket says 172.21.15.133:44190) 2024-09-11T23:48:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.164+0000 7faf577fe640 1 -- 172.21.15.133:0/2583120793 learned_addr learned my addr 172.21.15.133:0/2583120793 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.164+0000 7faf5cdc4640 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd9a9e80 0x55bccd9a77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.164+0000 7faf577fe640 1 -- 172.21.15.133:0/2583120793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd9a9e80 msgr2=0x55bccd9a77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.164+0000 7faf577fe640 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd9a9e80 0x55bccd9a77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.164+0000 7faf577fe640 1 -- 172.21.15.133:0/2583120793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99ecc0 msgr2=0x55bccd9a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.164+0000 7faf577fe640 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99ecc0 0x55bccd9a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.164+0000 7faf577fe640 1 -- 172.21.15.133:0/2583120793 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bccd616210 con 0x55bccd99f5e0 2024-09-11T23:48:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.165+0000 7faf577fe640 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 0x55bccd9a7260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faf40013740 tx=0x7faf40013c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2040df1c04ec767a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.166+0000 7faf377fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:48:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.171+0000 7faf377fe640 1 -- 172.21.15.133:0/2583120793 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40004440 con 0x55bccd99f5e0 2024-09-11T23:48:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.171+0000 7faf615c0740 1 -- 172.21.15.133:0/2583120793 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf5009ba50 con 0x55bccd99f5e0 2024-09-11T23:48:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.171+0000 7faf377fe640 1 -- 172.21.15.133:0/2583120793 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7faf400119c0 con 0x55bccd99f5e0 2024-09-11T23:48:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.172+0000 7faf377fe640 1 -- 172.21.15.133:0/2583120793 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf40011bc0 con 0x55bccd99f5e0 2024-09-11T23:48:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.176+0000 7faf615c0740 10 client.27827 fetch_fsmap learned FSMap version 30 2024-09-11T23:48:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.176+0000 7faf615c0740 1 -- 172.21.15.133:0/2583120793 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bccd6e2620 con 0x55bccd99f5e0 2024-09-11T23:48:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.178+0000 7faf377fe640 1 -- 172.21.15.133:0/2583120793 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf5009ba50 con 0x55bccd99f5e0 2024-09-11T23:48:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.180+0000 7faf377fe640 1 -- 172.21.15.133:0/2583120793 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7faf4001a020 con 0x55bccd99f5e0 2024-09-11T23:48:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.180+0000 7faf615c0740 10 client.27827 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:48:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.180+0000 7faf615c0740 10 client.27827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.180+0000 7faf615c0740 20 client.27827 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.180+0000 7faf615c0740 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bccd9c26a0 0x55bccd9c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.180+0000 7faf615c0740 4 client.27827 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.180+0000 7faf615c0740 1 -- 172.21.15.133:0/2583120793 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bccd910a20 con 0x55bccd9c26a0 2024-09-11T23:48:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.184+0000 7faf57fff640 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bccd9c26a0 0x55bccd9c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.403+0000 7faf57fff640 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bccd9c26a0 0x55bccd9c4ab0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79559b61e145a0a server_cookie=a9d496f760b70570 in_seq=0 out_seq=0 2024-09-11T23:48:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.403+0000 7faf377fe640 10 client.27827 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.408+0000 7faf377fe640 1 -- 172.21.15.133:0/2583120793 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bccd910a20 con 0x55bccd9c26a0 2024-09-11T23:48:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.408+0000 7faf377fe640 10 client.27827 handle_command_reply: tid=0 2024-09-11T23:48:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.408+0000 7faf615c0740 1 client.27827 shutdown 2024-09-11T23:48:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.409+0000 7faf615c0740 1 -- 172.21.15.133:0/2583120793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 msgr2=0x55bccd9a7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.409+0000 7faf615c0740 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 0x55bccd9a7260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faf40013740 tx=0x7faf40013c00 comp rx=0 tx=0).stop 2024-09-11T23:48:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.409+0000 7faf615c0740 1 -- 172.21.15.133:0/2583120793 shutdown_connections 2024-09-11T23:48:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.409+0000 7faf615c0740 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bccd9c26a0 0x55bccd9c4ab0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.410+0000 7faf615c0740 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd9a9e80 0x55bccd9a77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.410+0000 7faf615c0740 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 0x55bccd9a7260 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.410+0000 7faf615c0740 1 --2- 172.21.15.133:0/2583120793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99ecc0 0x55bccd9a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.410+0000 7faf377fe640 0 client.27827 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.410+0000 7faf615c0740 1 -- 172.21.15.133:0/2583120793 >> 172.21.15.133:0/2583120793 conn(0x55bccd99b230 msgr2=0x55bccd9a0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.410+0000 7faf615c0740 1 -- 172.21.15.133:0/2583120793 shutdown_connections 2024-09-11T23:48:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.410+0000 7faf615c0740 1 -- 172.21.15.133:0/2583120793 wait complete. 2024-09-11T23:48:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.414+0000 7faf615c0740 20 client.27827 trim_cache size 0 max 16384 2024-09-11T23:48:00.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:00 smithi138 ceph-mon[35142]: pgmap v350: 97 pgs: 97 active+clean; 69 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 787 MiB/s wr, 2.05k op/s 2024-09-11T23:48:00.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:00 smithi133 ceph-mon[31430]: pgmap v350: 97 pgs: 97 active+clean; 69 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 787 MiB/s wr, 2.05k op/s 2024-09-11T23:48:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.484+0000 7faf615c0740 1 Processor -- start 2024-09-11T23:48:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.484+0000 7faf615c0740 1 -- start start 2024-09-11T23:48:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.485+0000 7faf615c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 0x55bccd9a1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.485+0000 7faf615c0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 0x55bccd9a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.485+0000 7faf577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 0x55bccd9a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.485+0000 7faf577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 0x55bccd9a75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:44206/0 (socket says 172.21.15.133:44206) 2024-09-11T23:48:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.485+0000 7faf615c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd9a9e80 0x55bccd99a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.485+0000 7faf57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 0x55bccd9a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.485+0000 7faf57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 0x55bccd9a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:59740/0 (socket says 172.21.15.133:59740) 2024-09-11T23:48:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.485+0000 7faf577fe640 1 -- 172.21.15.133:0/3285269446 learned_addr learned my addr 172.21.15.133:0/3285269446 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.485+0000 7faf615c0740 1 -- 172.21.15.133:0/3285269446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf581b3ab0 con 0x55bccd9a9e80 2024-09-11T23:48:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.485+0000 7faf615c0740 1 -- 172.21.15.133:0/3285269446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf581b55f0 con 0x55bccd99f5e0 2024-09-11T23:48:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.486+0000 7faf615c0740 1 -- 172.21.15.133:0/3285269446 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf581b4180 con 0x55bccd99ecc0 2024-09-11T23:48:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.488+0000 7faf57fff640 1 -- 172.21.15.133:0/3285269446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 msgr2=0x55bccd9a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.488+0000 7faf57fff640 1 --2- 172.21.15.133:0/3285269446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 0x55bccd9a75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.488+0000 7faf57fff640 1 -- 172.21.15.133:0/3285269446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd9a9e80 msgr2=0x55bccd99a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.488+0000 7faf57fff640 1 --2- 172.21.15.133:0/3285269446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd9a9e80 0x55bccd99a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.488+0000 7faf57fff640 1 -- 172.21.15.133:0/3285269446 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bccd99ea40 con 0x55bccd99ecc0 2024-09-11T23:48:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.489+0000 7faf57fff640 1 --2- 172.21.15.133:0/3285269446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 0x55bccd9a1730 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55bccd99c090 tx=0x7faf500bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d039328d7573653 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.504+0000 7faf4dffb640 1 -- 172.21.15.133:0/3285269446 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50020040 con 0x55bccd99ecc0 2024-09-11T23:48:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.504+0000 7faf4dffb640 1 -- 172.21.15.133:0/3285269446 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf500056b0 con 0x55bccd99ecc0 2024-09-11T23:48:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.506+0000 7faf615c0740 1 -- 172.21.15.133:0/3285269446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 msgr2=0x55bccd9a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.506+0000 7faf615c0740 1 --2- 172.21.15.133:0/3285269446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 0x55bccd9a1730 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55bccd99c090 tx=0x7faf500bb040 comp rx=0 tx=0).stop 2024-09-11T23:48:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.508+0000 7faf615c0740 1 -- 172.21.15.133:0/3285269446 shutdown_connections 2024-09-11T23:48:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.508+0000 7faf615c0740 1 --2- 172.21.15.133:0/3285269446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd9a9e80 0x55bccd99a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.508+0000 7faf615c0740 1 --2- 172.21.15.133:0/3285269446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd99f5e0 0x55bccd9a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.508+0000 7faf615c0740 1 --2- 172.21.15.133:0/3285269446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 0x55bccd9a1730 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.508+0000 7faf615c0740 1 -- 172.21.15.133:0/3285269446 >> 172.21.15.133:0/3285269446 conn(0x55bccd99b230 msgr2=0x55bccd999a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.509+0000 7faf615c0740 1 -- 172.21.15.133:0/3285269446 shutdown_connections 2024-09-11T23:48:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.509+0000 7faf615c0740 1 -- 172.21.15.133:0/3285269446 wait complete. 2024-09-11T23:48:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.509+0000 7faf615c0740 1 -- start start 2024-09-11T23:48:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.510+0000 7faf615c0740 1 Processor -- start 2024-09-11T23:48:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.511+0000 7faf615c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 0x55bccd9baef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.511+0000 7faf615c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99f5e0 0x55bccd9a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.511+0000 7faf615c0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd9a9e80 0x55bccd9a4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.511+0000 7faf615c0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf581b4f20 con 0x55bccd99f5e0 2024-09-11T23:48:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.511+0000 7faf615c0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf581b4850 con 0x55bccd9a9e80 2024-09-11T23:48:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.511+0000 7faf615c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faf581b1b60 con 0x55bccd99ecc0 2024-09-11T23:48:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.511+0000 7faf615c0740 10 client.0 _ll_register_callbacks cb 0x7faf40005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.511+0000 7faf57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 0x55bccd9baef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.511+0000 7faf57fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 0x55bccd9baef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:59754/0 (socket says 172.21.15.133:59754) 2024-09-11T23:48:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.511+0000 7faf57fff640 1 -- 172.21.15.133:0/1521913902 learned_addr learned my addr 172.21.15.133:0/1521913902 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.511+0000 7faf5cdc4640 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd9a9e80 0x55bccd9a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.513+0000 7faf5cdc4640 1 -- 172.21.15.133:0/1521913902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 msgr2=0x55bccd9baef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.513+0000 7faf5cdc4640 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 0x55bccd9baef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.513+0000 7faf5cdc4640 1 -- 172.21.15.133:0/1521913902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99f5e0 msgr2=0x55bccd9a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.513+0000 7faf5cdc4640 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99f5e0 0x55bccd9a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.513+0000 7faf5cdc4640 1 -- 172.21.15.133:0/1521913902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bccd9a4850 con 0x55bccd9a9e80 2024-09-11T23:48:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.514+0000 7faf5cdc4640 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd9a9e80 0x55bccd9a4020 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faf4802aeb0 tx=0x7faf48031aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9d1e7ac204d311 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.514+0000 7faf377fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:48:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.515+0000 7faf377fe640 1 -- 172.21.15.133:0/1521913902 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4802fce0 con 0x55bccd9a9e80 2024-09-11T23:48:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.515+0000 7faf615c0740 1 -- 172.21.15.133:0/1521913902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf500056b0 con 0x55bccd9a9e80 2024-09-11T23:48:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.515+0000 7faf377fe640 1 -- 172.21.15.133:0/1521913902 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7faf480305c0 con 0x55bccd9a9e80 2024-09-11T23:48:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.516+0000 7faf377fe640 1 -- 172.21.15.133:0/1521913902 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf4802fe80 con 0x55bccd9a9e80 2024-09-11T23:48:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.516+0000 7faf615c0740 10 client.27833 fetch_fsmap learned FSMap version 30 2024-09-11T23:48:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.516+0000 7faf615c0740 1 -- 172.21.15.133:0/1521913902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bccda2d400 con 0x55bccd9a9e80 2024-09-11T23:48:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.519+0000 7faf377fe640 1 -- 172.21.15.133:0/1521913902 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf500056b0 con 0x55bccd9a9e80 2024-09-11T23:48:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.520+0000 7faf377fe640 1 -- 172.21.15.133:0/1521913902 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7faf48050a90 con 0x55bccd9a9e80 2024-09-11T23:48:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.520+0000 7faf615c0740 10 client.27833 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:48:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.521+0000 7faf615c0740 10 client.27833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.521+0000 7faf615c0740 20 client.27833 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.521+0000 7faf615c0740 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bccda2db20 0x55bccda2ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.521+0000 7faf615c0740 4 client.27833 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:48:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.521+0000 7faf615c0740 1 -- 172.21.15.133:0/1521913902 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bccd90ad70 con 0x55bccda2db20 2024-09-11T23:48:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.521+0000 7faf57fff640 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bccda2db20 0x55bccda2ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.537+0000 7faf57fff640 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bccda2db20 0x55bccda2ff40 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f13cda55e1da5e server_cookie=48c099b01652ce19 in_seq=0 out_seq=0 2024-09-11T23:48:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:00.537+0000 7faf377fe640 10 client.27833 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:00.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:00 smithi178 ceph-mon[35882]: pgmap v350: 97 pgs: 97 active+clean; 69 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 787 MiB/s wr, 2.05k op/s 2024-09-11T23:48:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf377fe640 1 -- 172.21.15.133:0/1521913902 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bccd90ad70 con 0x55bccda2db20 2024-09-11T23:48:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf377fe640 10 client.27833 handle_command_reply: tid=0 2024-09-11T23:48:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf615c0740 1 client.27833 shutdown 2024-09-11T23:48:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf615c0740 1 -- 172.21.15.133:0/1521913902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd9a9e80 msgr2=0x55bccd9a4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf615c0740 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd9a9e80 0x55bccd9a4020 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faf4802aeb0 tx=0x7faf48031aa0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf615c0740 1 -- 172.21.15.133:0/1521913902 shutdown_connections 2024-09-11T23:48:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf615c0740 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55bccda2db20 0x55bccda2ff40 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf615c0740 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bccd9a9e80 0x55bccd9a4020 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf615c0740 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bccd99f5e0 0x55bccd9a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf615c0740 1 --2- 172.21.15.133:0/1521913902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bccd99ecc0 0x55bccd9baef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf615c0740 1 -- 172.21.15.133:0/1521913902 >> 172.21.15.133:0/1521913902 conn(0x55bccd99b230 msgr2=0x55bccd99a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf377fe640 0 client.27833 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf615c0740 1 -- 172.21.15.133:0/1521913902 shutdown_connections 2024-09-11T23:48:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.142+0000 7faf615c0740 1 -- 172.21.15.133:0/1521913902 wait complete. 2024-09-11T23:48:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf615c0740 20 client.27833 trim_cache size 0 max 16384 2024-09-11T23:48:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:48:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:48:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:48:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:48:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 -- 172.21.15.133:0/1213877781 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faf440a2f00 msgr2=0x7faf440a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faf440a2f00 0x7faf440a53c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faf48006010 tx=0x7faf48005f80 comp rx=0 tx=0).stop 2024-09-11T23:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 -- 172.21.15.133:0/1213877781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf58100150 msgr2=0x7faf58071ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf58100150 0x7faf58071ea0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faf5000d770 tx=0x7faf5000dc30 comp rx=0 tx=0).stop 2024-09-11T23:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 -- 172.21.15.133:0/1213877781 shutdown_connections 2024-09-11T23:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7faf440a2f00 0x7faf440a53c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf581060a0 0x7faf58072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf58100af0 0x7faf580723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 --2- 172.21.15.133:0/1213877781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf58100150 0x7faf58071ea0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 -- 172.21.15.133:0/1213877781 >> 172.21.15.133:0/1213877781 conn(0x7faf58077200 msgr2=0x7faf580feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 -- 172.21.15.133:0/1213877781 shutdown_connections 2024-09-11T23:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:01.143+0000 7faf5d5c5640 1 -- 172.21.15.133:0/1213877781 wait complete. 2024-09-11T23:48:02.025 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:01 smithi133 ceph-mon[31430]: daemon mds.f restarted 2024-09-11T23:48:02.040 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:48:01 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: ignoring --setuser ceph since I am not root 2024-09-11T23:48:02.040 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:48:01 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: ignoring --setgroup ceph since I am not root 2024-09-11T23:48:02.040 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:48:01 smithi178 ceph-mds[65475]: -- 172.21.15.178:0/386737229 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b951cc1860 con 0x55b951c9e800 2024-09-11T23:48:02.040 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:48:01 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: starting mds.f at 2024-09-11T23:48:02.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:01 smithi178 ceph-mon[35882]: daemon mds.f restarted 2024-09-11T23:48:02.062 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:48:02.062 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:48:02.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:01 smithi138 ceph-mon[35142]: daemon mds.f restarted 2024-09-11T23:48:02.477 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:02.887 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:02 smithi133 ceph-mon[31430]: pgmap v351: 97 pgs: 97 active+clean; 70 GiB data, 267 GiB used, 806 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 662 MiB/s wr, 1.81k op/s 2024-09-11T23:48:02.888 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:02 smithi133 ceph-mon[31430]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:48:02.888 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:02 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6834/2029124747,v1:172.21.15.178:6835/2029124747] up:boot 2024-09-11T23:48:02.888 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:02 smithi133 ceph-mon[31430]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T23:48:02.888 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:02 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:48:02.888 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:02 smithi133 ceph-mon[31430]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:48:03.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:02 smithi178 ceph-mon[35882]: pgmap v351: 97 pgs: 97 active+clean; 70 GiB data, 267 GiB used, 806 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 662 MiB/s wr, 1.81k op/s 2024-09-11T23:48:03.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:02 smithi178 ceph-mon[35882]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:48:03.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:02 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6834/2029124747,v1:172.21.15.178:6835/2029124747] up:boot 2024-09-11T23:48:03.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:02 smithi178 ceph-mon[35882]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T23:48:03.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:02 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:48:03.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:02 smithi178 ceph-mon[35882]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:48:03.098 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:02 smithi138 ceph-mon[35142]: pgmap v351: 97 pgs: 97 active+clean; 70 GiB data, 267 GiB used, 806 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 662 MiB/s wr, 1.81k op/s 2024-09-11T23:48:03.098 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:02 smithi138 ceph-mon[35142]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:48:03.098 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:02 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6834/2029124747,v1:172.21.15.178:6835/2029124747] up:boot 2024-09-11T23:48:03.098 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:02 smithi138 ceph-mon[35142]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T23:48:03.099 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:02 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:48:03.099 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:02 smithi138 ceph-mon[35142]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T23:48:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.188+0000 7fcf9e51b640 1 -- 172.21.15.133:0/4155641699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98106f40 msgr2=0x7fcf9810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.188+0000 7fcf9e51b640 1 --2- 172.21.15.133:0/4155641699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98106f40 0x7fcf9810eac0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009ca0 tx=0x7fcf9002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.188+0000 7fcf9e51b640 1 -- 172.21.15.133:0/4155641699 shutdown_connections 2024-09-11T23:48:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.188+0000 7fcf9e51b640 1 --2- 172.21.15.133:0/4155641699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98106f40 0x7fcf9810eac0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.188+0000 7fcf9e51b640 1 --2- 172.21.15.133:0/4155641699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf981027c0 0x7fcf98106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.188+0000 7fcf9e51b640 1 --2- 172.21.15.133:0/4155641699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf98101e10 0x7fcf981021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.188+0000 7fcf9e51b640 1 -- 172.21.15.133:0/4155641699 >> 172.21.15.133:0/4155641699 conn(0x7fcf980fd8d0 msgr2=0x7fcf980ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.188+0000 7fcf9e51b640 1 -- 172.21.15.133:0/4155641699 shutdown_connections 2024-09-11T23:48:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.188+0000 7fcf9e51b640 1 -- 172.21.15.133:0/4155641699 wait complete. 2024-09-11T23:48:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.189+0000 7fcf9e51b640 1 Processor -- start 2024-09-11T23:48:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.189+0000 7fcf9e51b640 1 -- start start 2024-09-11T23:48:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.189+0000 7fcf9e51b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98101e10 0x7fcf981a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.189+0000 7fcf9e51b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf981027c0 0x7fcf981a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.189+0000 7fcf9e51b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf98106f40 0x7fcf9810dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.189+0000 7fcf9e51b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98111fc0 con 0x7fcf98101e10 2024-09-11T23:48:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.189+0000 7fcf9e51b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98111e40 con 0x7fcf981027c0 2024-09-11T23:48:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.189+0000 7fcf9e51b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98112140 con 0x7fcf98106f40 2024-09-11T23:48:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.189+0000 7fcf97fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98101e10 0x7fcf981a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.189+0000 7fcf97fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98101e10 0x7fcf981a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39428/0 (socket says 172.21.15.133:39428) 2024-09-11T23:48:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.189+0000 7fcf97fff640 1 -- 172.21.15.133:0/3507258104 learned_addr learned my addr 172.21.15.133:0/3507258104 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf98106f40 0x7fcf9810dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf977fe640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf981027c0 0x7fcf981a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf97fff640 1 -- 172.21.15.133:0/3507258104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf98106f40 msgr2=0x7fcf9810dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf97fff640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf98106f40 0x7fcf9810dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf97fff640 1 -- 172.21.15.133:0/3507258104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf981027c0 msgr2=0x7fcf981a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf97fff640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf981027c0 0x7fcf981a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf97fff640 1 -- 172.21.15.133:0/3507258104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9810e310 con 0x7fcf98101e10 2024-09-11T23:48:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf977fe640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf981027c0 0x7fcf981a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:48:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf97fff640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98101e10 0x7fcf981a46c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800b4a0 tx=0x7fcf8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf957fa640 1 -- 172.21.15.133:0/3507258104 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88014070 con 0x7fcf98101e10 2024-09-11T23:48:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf9e51b640 1 -- 172.21.15.133:0/3507258104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9810e600 con 0x7fcf98101e10 2024-09-11T23:48:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf957fa640 1 -- 172.21.15.133:0/3507258104 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf88010af0 con 0x7fcf98101e10 2024-09-11T23:48:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf9e51b640 1 -- 172.21.15.133:0/3507258104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf981ab590 con 0x7fcf98101e10 2024-09-11T23:48:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.190+0000 7fcf957fa640 1 -- 172.21.15.133:0/3507258104 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8801e060 con 0x7fcf98101e10 2024-09-11T23:48:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.193+0000 7fcfa08f4740 1 Processor -- start 2024-09-11T23:48:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.193+0000 7fcfa08f4740 1 -- start start 2024-09-11T23:48:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.193+0000 7fcf957fa640 1 -- 172.21.15.133:0/3507258104 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcf88017070 con 0x7fcf98101e10 2024-09-11T23:48:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.193+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc17b60 0x562a1bc17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.193+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc18480 0x562a1bc22800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.193+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc22d40 0x562a1bc25130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.193+0000 7fcfa08f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562a1bb7d280 con 0x562a1bc18480 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.193+0000 7fcfa08f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562a1bb24b10 con 0x562a1bc17b60 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.193+0000 7fcfa08f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562a1bb79240 con 0x562a1bc22d40 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.193+0000 7fcf9cab8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc17b60 0x562a1bc17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.193+0000 7fcf97fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc18480 0x562a1bc22800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf9cab8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc17b60 0x562a1bc17f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53838/0 (socket says 172.21.15.133:53838) 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf9cab8640 1 -- 172.21.15.133:0/3364499383 learned_addr learned my addr 172.21.15.133:0/3364499383 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf977fe640 1 --2- 172.21.15.133:0/3364499383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc22d40 0x562a1bc25130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf9cab8640 1 -- 172.21.15.133:0/3364499383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc22d40 msgr2=0x562a1bc25130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/3364499383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc22d40 0x562a1bc25130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf9cab8640 1 -- 172.21.15.133:0/3364499383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc18480 msgr2=0x562a1bc22800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/3364499383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc18480 0x562a1bc22800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf9cab8640 1 -- 172.21.15.133:0/3364499383 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1bb311f0 con 0x562a1bc17b60 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/3364499383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc17b60 0x562a1bc17f40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcf90035000 tx=0x7fcf9002f500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=357d23347f30c4d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf957fa640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf800a2cf0 0x7fcf800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf7e7fc640 1 -- 172.21.15.133:0/3364499383 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90003970 con 0x562a1bc17b60 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf7e7fc640 1 -- 172.21.15.133:0/3364499383 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf90036040 con 0x562a1bc17b60 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf7e7fc640 1 -- 172.21.15.133:0/3364499383 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90004460 con 0x562a1bc17b60 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.194+0000 7fcf97fff640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf800a2cf0 0x7fcf800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.195+0000 7fcf957fa640 1 -- 172.21.15.133:0/3507258104 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fcf880ce680 con 0x7fcf98101e10 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.195+0000 7fcfa08f4740 1 -- 172.21.15.133:0/3364499383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc17b60 msgr2=0x562a1bc17f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.195+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/3364499383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc17b60 0x562a1bc17f40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcf90035000 tx=0x7fcf9002f500 comp rx=0 tx=0).stop 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.195+0000 7fcf97fff640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf800a2cf0 0x7fcf800a51b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcf880cc000 tx=0x7fcf880d4550 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.195+0000 7fcfa08f4740 1 -- 172.21.15.133:0/3364499383 shutdown_connections 2024-09-11T23:48:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.195+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/3364499383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc22d40 0x562a1bc25130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.195+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/3364499383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc18480 0x562a1bc22800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.195+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/3364499383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc17b60 0x562a1bc17f40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.195+0000 7fcfa08f4740 1 -- 172.21.15.133:0/3364499383 >> 172.21.15.133:0/3364499383 conn(0x562a1bc140f0 msgr2=0x562a1bc144d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.195+0000 7fcfa08f4740 1 -- 172.21.15.133:0/3364499383 shutdown_connections 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.195+0000 7fcfa08f4740 1 -- 172.21.15.133:0/3364499383 wait complete. 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.196+0000 7fcfa08f4740 1 -- start start 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.196+0000 7fcfa08f4740 1 Processor -- start 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.196+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.196+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc20140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.196+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc22d40 0x562a1bc20680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.196+0000 7fcfa08f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562a1bc25790 con 0x562a1bc17b60 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.196+0000 7fcfa08f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562a1bb37f10 con 0x562a1bc18480 2024-09-11T23:48:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.196+0000 7fcfa08f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562a1bc25910 con 0x562a1bc22d40 2024-09-11T23:48:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcfa08f4740 10 client.0 _ll_register_callbacks cb 0x562a1bb00a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf9cab8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf977fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc20140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf9cab8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39442/0 (socket says 172.21.15.133:39442) 2024-09-11T23:48:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf977fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc20140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53840/0 (socket says 172.21.15.133:53840) 2024-09-11T23:48:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf9cab8640 1 -- 172.21.15.133:0/1585338976 learned_addr learned my addr 172.21.15.133:0/1585338976 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc22d40 0x562a1bc20680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf977fe640 1 -- 172.21.15.133:0/1585338976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc22d40 msgr2=0x562a1bc20680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf977fe640 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc22d40 0x562a1bc20680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf977fe640 1 -- 172.21.15.133:0/1585338976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 msgr2=0x562a1bc1fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf977fe640 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf977fe640 1 -- 172.21.15.133:0/1585338976 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a1b8ee6a0 con 0x562a1bc18480 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf977fe640 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc20140 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c005d00 tx=0x7fcf8c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e92e104b79ab19a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf777fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf777fe640 1 -- 172.21.15.133:0/1585338976 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c004220 con 0x562a1bc18480 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcf777fe640 1 -- 172.21.15.133:0/1585338976 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fcf8c0043c0 con 0x562a1bc18480 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.197+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1585338976 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf90036040 con 0x562a1bc18480 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.198+0000 7fcf777fe640 1 -- 172.21.15.133:0/1585338976 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf8c011e90 con 0x562a1bc18480 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.198+0000 7fcfa08f4740 10 client.27845 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.198+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1585338976 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a1bc31420 con 0x562a1bc18480 2024-09-11T23:48:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.198+0000 7fcf777fe640 1 -- 172.21.15.133:0/1585338976 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf90036040 con 0x562a1bc18480 2024-09-11T23:48:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.198+0000 7fcf777fe640 1 -- 172.21.15.133:0/1585338976 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcf8c00f040 con 0x562a1bc18480 2024-09-11T23:48:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.198+0000 7fcfa08f4740 10 client.27845 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.198+0000 7fcfa08f4740 10 client.27845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.198+0000 7fcfa08f4740 20 client.27845 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.198+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562a1bc328e0 0x562a1bc34d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.198+0000 7fcfa08f4740 4 client.27845 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.198+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1585338976 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a1bb8a7f0 con 0x562a1bc328e0 2024-09-11T23:48:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.199+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562a1bc328e0 0x562a1bc34d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.199+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562a1bc328e0 0x562a1bc34d10 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42553c774a95aa09 server_cookie=27bcb8c08f189534 in_seq=0 out_seq=0 2024-09-11T23:48:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.199+0000 7fcf777fe640 10 client.27845 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcf777fe640 1 -- 172.21.15.133:0/1585338976 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a1bb8a7f0 con 0x562a1bc328e0 2024-09-11T23:48:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcf777fe640 10 client.27845 handle_command_reply: tid=0 2024-09-11T23:48:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 1 client.27845 shutdown 2024-09-11T23:48:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1585338976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 msgr2=0x562a1bc20140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc20140 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c005d00 tx=0x7fcf8c0117b0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1585338976 shutdown_connections 2024-09-11T23:48:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562a1bc328e0 0x562a1bc34d10 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc22d40 0x562a1bc20680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcf777fe640 0 client.27845 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc20140 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1585338976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1585338976 >> 172.21.15.133:0/1585338976 conn(0x562a1bc140f0 msgr2=0x562a1bc13270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1585338976 shutdown_connections 2024-09-11T23:48:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1585338976 wait complete. 2024-09-11T23:48:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.200+0000 7fcfa08f4740 20 client.27845 trim_cache size 0 max 16384 2024-09-11T23:48:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcfa08f4740 1 Processor -- start 2024-09-11T23:48:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcfa08f4740 1 -- start start 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc18480 0x562a1bc21a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc22d40 0x562a1bc3a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcfa08f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b3ce0 con 0x562a1bc17b60 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcfa08f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b5820 con 0x562a1bc22d40 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcfa08f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b43b0 con 0x562a1bc18480 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcf9cab8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcf9cab8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39444/0 (socket says 172.21.15.133:39444) 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcf9cab8640 1 -- 172.21.15.133:0/510270906 learned_addr learned my addr 172.21.15.133:0/510270906 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.225+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/510270906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc22d40 0x562a1bc3a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.226+0000 7fcf9cab8640 1 -- 172.21.15.133:0/510270906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc18480 msgr2=0x562a1bc21a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.226+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/510270906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc18480 0x562a1bc21a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.226+0000 7fcf9cab8640 1 -- 172.21.15.133:0/510270906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc22d40 msgr2=0x562a1bc3a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.226+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/510270906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc22d40 0x562a1bc3a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.226+0000 7fcf9cab8640 1 -- 172.21.15.133:0/510270906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1bc151a0 con 0x562a1bc17b60 2024-09-11T23:48:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.226+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/510270906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1a610 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009940 tx=0x7fcf90009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335a16b771ab2f84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.226+0000 7fcf7dffb640 1 -- 172.21.15.133:0/510270906 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9004a040 con 0x562a1bc17b60 2024-09-11T23:48:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.227+0000 7fcf7dffb640 1 -- 172.21.15.133:0/510270906 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf90036040 con 0x562a1bc17b60 2024-09-11T23:48:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.227+0000 7fcf7dffb640 1 -- 172.21.15.133:0/510270906 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9002f990 con 0x562a1bc17b60 2024-09-11T23:48:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.228+0000 7fcfa08f4740 1 -- 172.21.15.133:0/510270906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 msgr2=0x562a1bc1a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.228+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/510270906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1a610 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009940 tx=0x7fcf90009560 comp rx=0 tx=0).stop 2024-09-11T23:48:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.228+0000 7fcfa08f4740 1 -- 172.21.15.133:0/510270906 shutdown_connections 2024-09-11T23:48:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.228+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/510270906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc22d40 0x562a1bc3a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.228+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/510270906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc18480 0x562a1bc21a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.228+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/510270906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc17b60 0x562a1bc1a610 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.228+0000 7fcfa08f4740 1 -- 172.21.15.133:0/510270906 >> 172.21.15.133:0/510270906 conn(0x562a1bc140f0 msgr2=0x562a1bc21350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.228+0000 7fcfa08f4740 1 -- 172.21.15.133:0/510270906 shutdown_connections 2024-09-11T23:48:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.228+0000 7fcfa08f4740 1 -- 172.21.15.133:0/510270906 wait complete. 2024-09-11T23:48:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.228+0000 7fcfa08f4740 1 -- start start 2024-09-11T23:48:03.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.229+0000 7fcfa08f4740 1 Processor -- start 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.229+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc17b60 0x562a1bc2afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc1c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcfa08f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc22d40 0x562a1bc1cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcf9cab8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc17b60 0x562a1bc2afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcf9cab8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc17b60 0x562a1bc2afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:44438/0 (socket says 172.21.15.133:44438) 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcf977fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc1c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcf977fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc1c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53858/0 (socket says 172.21.15.133:53858) 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcf977fe640 1 -- 172.21.15.133:0/1720670378 learned_addr learned my addr 172.21.15.133:0/1720670378 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1720670378 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b5150 con 0x562a1bc22d40 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1720670378 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b4a80 con 0x562a1bc18480 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1720670378 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b1cc0 con 0x562a1bc17b60 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcfa08f4740 10 client.0 _ll_register_callbacks cb 0x7fcf8c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:03.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcf977fe640 1 -- 172.21.15.133:0/1720670378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc17b60 msgr2=0x562a1bc2afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcf977fe640 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc17b60 0x562a1bc2afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcf977fe640 1 -- 172.21.15.133:0/1720670378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc22d40 msgr2=0x562a1bc1cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc22d40 0x562a1bc1cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcf977fe640 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc22d40 0x562a1bc1cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.230+0000 7fcf977fe640 1 -- 172.21.15.133:0/1720670378 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a1bc1d160 con 0x562a1bc18480 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.231+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc22d40 0x562a1bc1cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.231+0000 7fcf977fe640 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc1c660 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c000c00 tx=0x7fcf8c005690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67083aef7e8b49a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.231+0000 7fcf777fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.231+0000 7fcf777fe640 1 -- 172.21.15.133:0/1720670378 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c000e60 con 0x562a1bc18480 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.231+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1720670378 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf90036040 con 0x562a1bc18480 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.231+0000 7fcf777fe640 1 -- 172.21.15.133:0/1720670378 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fcf8c0037f0 con 0x562a1bc18480 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.231+0000 7fcf777fe640 1 -- 172.21.15.133:0/1720670378 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf8c027070 con 0x562a1bc18480 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.231+0000 7fcfa08f4740 10 client.27851 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:03.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.231+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1720670378 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a1bc9d360 con 0x562a1bc18480 2024-09-11T23:48:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.232+0000 7fcf777fe640 1 -- 172.21.15.133:0/1720670378 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf90036040 con 0x562a1bc18480 2024-09-11T23:48:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.232+0000 7fcf777fe640 1 -- 172.21.15.133:0/1720670378 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcf8c01a600 con 0x562a1bc18480 2024-09-11T23:48:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.233+0000 7fcfa08f4740 10 client.27851 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.233+0000 7fcfa08f4740 10 client.27851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.233+0000 7fcfa08f4740 20 client.27851 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.233+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562a1bc328e0 0x562a1bc9fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.233+0000 7fcfa08f4740 4 client.27851 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:48:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.233+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1720670378 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562a1bb84d00 con 0x562a1bc328e0 2024-09-11T23:48:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.233+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562a1bc328e0 0x562a1bc9fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.233+0000 7fcf777fe640 10 client.27851 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.234+0000 7fcf9cab8640 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562a1bc328e0 0x562a1bc9fba0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74c52144a13a4c2 server_cookie=ee1d5619578db641 in_seq=0 out_seq=0 2024-09-11T23:48:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.235+0000 7fcf777fe640 1 -- 172.21.15.133:0/1720670378 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562a1bb84d00 con 0x562a1bc328e0 2024-09-11T23:48:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.235+0000 7fcf777fe640 10 client.27851 handle_command_reply: tid=0 2024-09-11T23:48:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.235+0000 7fcfa08f4740 1 client.27851 shutdown 2024-09-11T23:48:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.235+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1720670378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 msgr2=0x562a1bc1c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.235+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc1c660 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c000c00 tx=0x7fcf8c005690 comp rx=0 tx=0).stop 2024-09-11T23:48:03.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.236+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1720670378 shutdown_connections 2024-09-11T23:48:03.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.236+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x562a1bc328e0 0x562a1bc9fba0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.236+0000 7fcf777fe640 0 client.27851 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:03.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.236+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1bc22d40 0x562a1bc1cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.236+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a1bc18480 0x562a1bc1c660 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.236+0000 7fcfa08f4740 1 --2- 172.21.15.133:0/1720670378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562a1bc17b60 0x562a1bc2afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.236+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1720670378 >> 172.21.15.133:0/1720670378 conn(0x562a1bc140f0 msgr2=0x562a1bc21a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:03.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.236+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1720670378 shutdown_connections 2024-09-11T23:48:03.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.236+0000 7fcfa08f4740 1 -- 172.21.15.133:0/1720670378 wait complete. 2024-09-11T23:48:03.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.236+0000 7fcfa08f4740 20 client.27851 trim_cache size 0 max 16384 2024-09-11T23:48:03.238 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:48:03.238 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:48:03.238 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "aa550cb3-abec-42ac-ae65-f65fe7b54789", 2024-09-11T23:48:03.238 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:48:03.239 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:48:03.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.237+0000 7fcf9e51b640 1 -- 172.21.15.133:0/3507258104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf800a2cf0 msgr2=0x7fcf800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:03.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.237+0000 7fcf9e51b640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf800a2cf0 0x7fcf800a51b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcf880cc000 tx=0x7fcf880d4550 comp rx=0 tx=0).stop 2024-09-11T23:48:03.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.237+0000 7fcf9e51b640 1 -- 172.21.15.133:0/3507258104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98101e10 msgr2=0x7fcf981a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:03.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.237+0000 7fcf9e51b640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98101e10 0x7fcf981a46c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800b4a0 tx=0x7fcf8800b960 comp rx=0 tx=0).stop 2024-09-11T23:48:03.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.237+0000 7fcf9e51b640 1 -- 172.21.15.133:0/3507258104 shutdown_connections 2024-09-11T23:48:03.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.237+0000 7fcf9e51b640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf800a2cf0 0x7fcf800a51b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.238+0000 7fcf9e51b640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf98106f40 0x7fcf9810dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.238+0000 7fcf9e51b640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf981027c0 0x7fcf981a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.238+0000 7fcf9e51b640 1 --2- 172.21.15.133:0/3507258104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98101e10 0x7fcf981a46c0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:03.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.238+0000 7fcf9e51b640 1 -- 172.21.15.133:0/3507258104 >> 172.21.15.133:0/3507258104 conn(0x7fcf980fd8d0 msgr2=0x7fcf980fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:03.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.238+0000 7fcf9e51b640 1 -- 172.21.15.133:0/3507258104 shutdown_connections 2024-09-11T23:48:03.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:03.238+0000 7fcf9e51b640 1 -- 172.21.15.133:0/3507258104 wait complete. 2024-09-11T23:48:04.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:48:04.616 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:05.222 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:05 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:48:05.222 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:05 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:05.223 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:05 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:48:05.223 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:05 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:05.223 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:05 smithi133 ceph-mon[31430]: pgmap v353: 97 pgs: 97 active+clean; 70 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 685 MiB/s wr, 1.73k op/s 2024-09-11T23:48:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:05 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:48:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:05 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:05 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:48:05.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:05 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:05.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:05 smithi178 ceph-mon[35882]: pgmap v353: 97 pgs: 97 active+clean; 70 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 685 MiB/s wr, 1.73k op/s 2024-09-11T23:48:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.220+0000 7f484c618640 1 -- 172.21.15.133:0/2708995442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4844109830 msgr2=0x7f4844109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.220+0000 7f484c618640 1 --2- 172.21.15.133:0/2708995442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4844109830 0x7f4844109c90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4834009860 tx=0x7f483402a360 comp rx=0 tx=0).stop 2024-09-11T23:48:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.220+0000 7f484c618640 1 -- 172.21.15.133:0/2708995442 shutdown_connections 2024-09-11T23:48:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.220+0000 7f484c618640 1 --2- 172.21.15.133:0/2708995442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484411b140 0x7f484411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.220+0000 7f484c618640 1 --2- 172.21.15.133:0/2708995442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4844109830 0x7f4844109c90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.220+0000 7f484c618640 1 --2- 172.21.15.133:0/2708995442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4844109450 0x7f4844110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.220+0000 7f484c618640 1 -- 172.21.15.133:0/2708995442 >> 172.21.15.133:0/2708995442 conn(0x7f484406e140 msgr2=0x7f484406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.220+0000 7f484c618640 1 -- 172.21.15.133:0/2708995442 shutdown_connections 2024-09-11T23:48:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.220+0000 7f484c618640 1 -- 172.21.15.133:0/2708995442 wait complete. 2024-09-11T23:48:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.221+0000 7f484c618640 1 Processor -- start 2024-09-11T23:48:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.221+0000 7f484c618640 1 -- start start 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.221+0000 7f484c618640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4844109450 0x7f48441a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.221+0000 7f484c618640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4844109830 0x7f48441a7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.221+0000 7f484c618640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f484411b140 0x7f48441abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.221+0000 7f484c618640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4844120da0 con 0x7f4844109450 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.221+0000 7f484c618640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4844120c20 con 0x7f484411b140 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.221+0000 7f484c618640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4844120f20 con 0x7f4844109830 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.222+0000 7f484a3b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4844109450 0x7f48441a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.222+0000 7f484abb5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f484411b140 0x7f48441abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.222+0000 7f484a3b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4844109450 0x7f48441a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39474/0 (socket says 172.21.15.133:39474) 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.222+0000 7f484a3b4640 1 -- 172.21.15.133:0/426945388 learned_addr learned my addr 172.21.15.133:0/426945388 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.222+0000 7f484abb5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f484411b140 0x7f48441abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53876/0 (socket says 172.21.15.133:53876) 2024-09-11T23:48:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.223+0000 7f4849bb3640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4844109830 0x7f48441a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.224+0000 7f484abb5640 1 -- 172.21.15.133:0/426945388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4844109830 msgr2=0x7f48441a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.224+0000 7f484abb5640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4844109830 0x7f48441a7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.224+0000 7f484abb5640 1 -- 172.21.15.133:0/426945388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4844109450 msgr2=0x7f48441a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.224+0000 7f484abb5640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4844109450 0x7f48441a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.224+0000 7f484abb5640 1 -- 172.21.15.133:0/426945388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48441ac1f0 con 0x7f484411b140 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.226+0000 7f484abb5640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f484411b140 0x7f48441abaf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f483c00eeb0 tx=0x7f483c00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.226+0000 7f483b7fe640 1 -- 172.21.15.133:0/426945388 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c009a40 con 0x7f484411b140 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.226+0000 7f484c618640 1 -- 172.21.15.133:0/426945388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48441ac4e0 con 0x7f484411b140 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.226+0000 7f484c618640 1 -- 172.21.15.133:0/426945388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4844110920 con 0x7f484411b140 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.226+0000 7f483b7fe640 1 -- 172.21.15.133:0/426945388 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483c003e50 con 0x7f484411b140 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.226+0000 7f483b7fe640 1 -- 172.21.15.133:0/426945388 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c009a40 con 0x7f484411b140 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.232+0000 7f484e9f1740 1 Processor -- start 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.234+0000 7f483b7fe640 1 -- 172.21.15.133:0/426945388 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f483c01d060 con 0x7f484411b140 2024-09-11T23:48:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.235+0000 7f483b7fe640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f48300a2ef0 0x7f48300a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.235+0000 7f483b7fe640 1 -- 172.21.15.133:0/426945388 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7f483c01b070 con 0x7f484411b140 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.238+0000 7f484e9f1740 1 -- start start 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.238+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18b4a0 0x5610be18b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.238+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be18bdc0 0x5610be196140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.238+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be196680 0x5610be198a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.238+0000 7f484e9f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5610be0f12e0 con 0x5610be18bdc0 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.238+0000 7f484e9f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610be098ce0 con 0x5610be18b4a0 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.238+0000 7f484e9f1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5610be0ed2a0 con 0x5610be196680 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.240+0000 7f484abb5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be196680 0x5610be198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.240+0000 7f484abb5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be196680 0x5610be198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:44480/0 (socket says 172.21.15.133:44480) 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.240+0000 7f484abb5640 1 -- 172.21.15.133:0/635746467 learned_addr learned my addr 172.21.15.133:0/635746467 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.241+0000 7f484abb5640 1 -- 172.21.15.133:0/635746467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18b4a0 msgr2=0x5610be18b880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:48:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.241+0000 7f484abb5640 1 --2- 172.21.15.133:0/635746467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18b4a0 0x5610be18b880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.241+0000 7f484abb5640 1 -- 172.21.15.133:0/635746467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be18bdc0 msgr2=0x5610be196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.241+0000 7f484abb5640 1 --2- 172.21.15.133:0/635746467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be18bdc0 0x5610be196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.241+0000 7f484abb5640 1 -- 172.21.15.133:0/635746467 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610be07d440 con 0x5610be196680 2024-09-11T23:48:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.241+0000 7f484abb5640 1 --2- 172.21.15.133:0/635746467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be196680 0x5610be198a70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f483c0cc8b0 tx=0x7f483c09f4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb9efe58782092e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.241+0000 7f4838ff9640 1 -- 172.21.15.133:0/635746467 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c01a050 con 0x5610be196680 2024-09-11T23:48:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.241+0000 7f4838ff9640 1 -- 172.21.15.133:0/635746467 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483c09f110 con 0x5610be196680 2024-09-11T23:48:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.241+0000 7f4838ff9640 1 -- 172.21.15.133:0/635746467 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c01a050 con 0x5610be196680 2024-09-11T23:48:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.242+0000 7f4849bb3640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f48300a2ef0 0x7f48300a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.242+0000 7f484e9f1740 1 -- 172.21.15.133:0/635746467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be196680 msgr2=0x5610be198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.242+0000 7f484e9f1740 1 --2- 172.21.15.133:0/635746467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be196680 0x5610be198a70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f483c0cc8b0 tx=0x7f483c09f4f0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.243+0000 7f484e9f1740 1 -- 172.21.15.133:0/635746467 shutdown_connections 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.243+0000 7f484e9f1740 1 --2- 172.21.15.133:0/635746467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be196680 0x5610be198a70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.243+0000 7f484e9f1740 1 --2- 172.21.15.133:0/635746467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be18bdc0 0x5610be196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.243+0000 7f484e9f1740 1 --2- 172.21.15.133:0/635746467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18b4a0 0x5610be18b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.243+0000 7f484e9f1740 1 -- 172.21.15.133:0/635746467 >> 172.21.15.133:0/635746467 conn(0x5610be187a30 msgr2=0x5610be187e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.244+0000 7f484e9f1740 1 -- 172.21.15.133:0/635746467 shutdown_connections 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.244+0000 7f484e9f1740 1 -- 172.21.15.133:0/635746467 wait complete. 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.245+0000 7f484e9f1740 1 -- start start 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.245+0000 7f484e9f1740 1 Processor -- start 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.245+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18b4a0 0x5610be195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.245+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18bdc0 0x5610be193180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.245+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be196680 0x5610be1936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.245+0000 7f484e9f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5610be199250 con 0x5610be196680 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.245+0000 7f484e9f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610be0ac050 con 0x5610be18b4a0 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.245+0000 7f484e9f1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5610be1993d0 con 0x5610be18bdc0 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.245+0000 7f484e9f1740 10 client.0 _ll_register_callbacks cb 0x5610be0ec160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.246+0000 7f484a3b4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18b4a0 0x5610be195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.246+0000 7f484a3b4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18b4a0 0x5610be195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53902/0 (socket says 172.21.15.133:53902) 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.246+0000 7f484a3b4640 1 -- 172.21.15.133:0/448866763 learned_addr learned my addr 172.21.15.133:0/448866763 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.246+0000 7f4849bb3640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f48300a2ef0 0x7f48300a53b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f483402a950 tx=0x7f4834000bc0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.246+0000 7f4849bb3640 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18bdc0 0x5610be193180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.246+0000 7f484abb5640 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be196680 0x5610be1936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.247+0000 7f484a3b4640 1 -- 172.21.15.133:0/448866763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18bdc0 msgr2=0x5610be193180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.247+0000 7f484a3b4640 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18bdc0 0x5610be193180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.247+0000 7f484a3b4640 1 -- 172.21.15.133:0/448866763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be196680 msgr2=0x5610be1936c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.247+0000 7f484a3b4640 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be196680 0x5610be1936c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.247+0000 7f484a3b4640 1 -- 172.21.15.133:0/448866763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610bde623c0 con 0x5610be18b4a0 2024-09-11T23:48:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.248+0000 7f482dffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.248+0000 7f484a3b4640 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18b4a0 0x5610be195ed0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f484000bf70 tx=0x7f4840010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f13c9adfbedae02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.250+0000 7f482dffb640 1 -- 172.21.15.133:0/448866763 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840011910 con 0x5610be18b4a0 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.250+0000 7f484e9f1740 1 -- 172.21.15.133:0/448866763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f483c09f110 con 0x5610be18b4a0 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.250+0000 7f482dffb640 1 -- 172.21.15.133:0/448866763 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7f4840011710 con 0x5610be18b4a0 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.250+0000 7f482dffb640 1 -- 172.21.15.133:0/448866763 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4840011910 con 0x5610be18b4a0 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.250+0000 7f484e9f1740 10 client.27860 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.250+0000 7f484e9f1740 1 -- 172.21.15.133:0/448866763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5610be1a4c40 con 0x5610be18b4a0 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.253+0000 7f482dffb640 1 -- 172.21.15.133:0/448866763 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483c09f110 con 0x5610be18b4a0 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.253+0000 7f482dffb640 1 -- 172.21.15.133:0/448866763 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f484001bb80 con 0x5610be18b4a0 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.253+0000 7f484e9f1740 10 client.27860 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.253+0000 7f484e9f1740 10 client.27860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:05.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.253+0000 7f484e9f1740 20 client.27860 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.253+0000 7f484e9f1740 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5610be1a6470 0x5610be1a88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.253+0000 7f484e9f1740 4 client.27860 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.253+0000 7f484e9f1740 1 -- 172.21.15.133:0/448866763 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610be0fe850 con 0x5610be1a6470 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.254+0000 7f4849bb3640 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5610be1a6470 0x5610be1a88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.260+0000 7f4849bb3640 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5610be1a6470 0x5610be1a88d0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a467a5098f6c4713 server_cookie=b0cf84564cb7470f in_seq=0 out_seq=0 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.260+0000 7f482dffb640 10 client.27860 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.261+0000 7f482dffb640 1 -- 172.21.15.133:0/448866763 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610be0fe850 con 0x5610be1a6470 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.261+0000 7f482dffb640 10 client.27860 handle_command_reply: tid=0 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.261+0000 7f484e9f1740 1 client.27860 shutdown 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.261+0000 7f484e9f1740 1 -- 172.21.15.133:0/448866763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18b4a0 msgr2=0x5610be195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.261+0000 7f484e9f1740 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18b4a0 0x5610be195ed0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f484000bf70 tx=0x7f4840010860 comp rx=0 tx=0).stop 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.264+0000 7f484e9f1740 1 -- 172.21.15.133:0/448866763 shutdown_connections 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.264+0000 7f484e9f1740 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5610be1a6470 0x5610be1a88d0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.264+0000 7f484e9f1740 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be196680 0x5610be1936c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.264+0000 7f484e9f1740 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18bdc0 0x5610be193180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.264+0000 7f484e9f1740 1 --2- 172.21.15.133:0/448866763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18b4a0 0x5610be195ed0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.264+0000 7f484e9f1740 1 -- 172.21.15.133:0/448866763 >> 172.21.15.133:0/448866763 conn(0x5610be187a30 msgr2=0x5610be18d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.264+0000 7f482dffb640 0 client.27860 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.265+0000 7f484e9f1740 1 -- 172.21.15.133:0/448866763 shutdown_connections 2024-09-11T23:48:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.266+0000 7f484e9f1740 1 -- 172.21.15.133:0/448866763 wait complete. 2024-09-11T23:48:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.266+0000 7f484e9f1740 20 client.27860 trim_cache size 0 max 16384 2024-09-11T23:48:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.299+0000 7f484e9f1740 1 Processor -- start 2024-09-11T23:48:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.299+0000 7f484e9f1740 1 -- start start 2024-09-11T23:48:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.299+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18b4a0 0x5610be1a2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.299+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18bdc0 0x5610be18e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.299+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be196680 0x5610be18d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.299+0000 7f484e9f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f482800e9f0 con 0x5610be196680 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.299+0000 7f484e9f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4828010530 con 0x5610be18bdc0 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.299+0000 7f484e9f1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f482800f0c0 con 0x5610be18b4a0 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.300+0000 7f4849bb3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18bdc0 0x5610be18e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.300+0000 7f484abb5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be196680 0x5610be18d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.300+0000 7f4849bb3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18bdc0 0x5610be18e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53906/0 (socket says 172.21.15.133:53906) 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.300+0000 7f4849bb3640 1 -- 172.21.15.133:0/2746220825 learned_addr learned my addr 172.21.15.133:0/2746220825 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.300+0000 7f484a3b4640 1 --2- 172.21.15.133:0/2746220825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18b4a0 0x5610be1a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.300+0000 7f484abb5640 1 -- 172.21.15.133:0/2746220825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18b4a0 msgr2=0x5610be1a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.300+0000 7f484abb5640 1 --2- 172.21.15.133:0/2746220825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18b4a0 0x5610be1a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.300+0000 7f484abb5640 1 -- 172.21.15.133:0/2746220825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18bdc0 msgr2=0x5610be18e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.300+0000 7f484abb5640 1 --2- 172.21.15.133:0/2746220825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18bdc0 0x5610be18e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.300+0000 7f484abb5640 1 -- 172.21.15.133:0/2746220825 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610be196240 con 0x5610be196680 2024-09-11T23:48:05.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.300+0000 7f484abb5640 1 --2- 172.21.15.133:0/2746220825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be196680 0x5610be18d650 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f483c09f3a0 tx=0x7f483c097320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3fb6628ebfc146 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:05.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.301+0000 7f482ffff640 1 -- 172.21.15.133:0/2746220825 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c01a050 con 0x5610be196680 2024-09-11T23:48:05.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.301+0000 7f482ffff640 1 -- 172.21.15.133:0/2746220825 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483c09f0e0 con 0x5610be196680 2024-09-11T23:48:05.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.302+0000 7f484e9f1740 1 -- 172.21.15.133:0/2746220825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be196680 msgr2=0x5610be18d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:05.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.302+0000 7f482ffff640 1 -- 172.21.15.133:0/2746220825 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c01a050 con 0x5610be196680 2024-09-11T23:48:05.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.302+0000 7f484e9f1740 1 --2- 172.21.15.133:0/2746220825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be196680 0x5610be18d650 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f483c09f3a0 tx=0x7f483c097320 comp rx=0 tx=0).stop 2024-09-11T23:48:05.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.302+0000 7f484e9f1740 1 -- 172.21.15.133:0/2746220825 shutdown_connections 2024-09-11T23:48:05.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.302+0000 7f484e9f1740 1 --2- 172.21.15.133:0/2746220825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be196680 0x5610be18d650 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.302+0000 7f484e9f1740 1 --2- 172.21.15.133:0/2746220825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be18bdc0 0x5610be18e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.302+0000 7f484e9f1740 1 --2- 172.21.15.133:0/2746220825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18b4a0 0x5610be1a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.302+0000 7f484e9f1740 1 -- 172.21.15.133:0/2746220825 >> 172.21.15.133:0/2746220825 conn(0x5610be187a30 msgr2=0x5610be18cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.303+0000 7f484e9f1740 1 -- 172.21.15.133:0/2746220825 shutdown_connections 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.303+0000 7f484e9f1740 1 -- 172.21.15.133:0/2746220825 wait complete. 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.303+0000 7f484e9f1740 1 -- start start 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.303+0000 7f484e9f1740 1 Processor -- start 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.304+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18b4a0 0x5610be199e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.304+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be18bdc0 0x5610be19a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.304+0000 7f484e9f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be196680 0x5610be209170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.304+0000 7f484e9f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f482800fe60 con 0x5610be18bdc0 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.304+0000 7f484e9f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f482800f790 con 0x5610be196680 2024-09-11T23:48:05.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.304+0000 7f484e9f1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f482800c970 con 0x5610be18b4a0 2024-09-11T23:48:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.304+0000 7f484abb5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be196680 0x5610be209170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.304+0000 7f484abb5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be196680 0x5610be209170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53912/0 (socket says 172.21.15.133:53912) 2024-09-11T23:48:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.304+0000 7f484abb5640 1 -- 172.21.15.133:0/4006799420 learned_addr learned my addr 172.21.15.133:0/4006799420 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.304+0000 7f484e9f1740 10 client.0 _ll_register_callbacks cb 0x7f484000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.304+0000 7f484a3b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18b4a0 0x5610be199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.305+0000 7f4849bb3640 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be18bdc0 0x5610be19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.305+0000 7f484abb5640 1 -- 172.21.15.133:0/4006799420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18b4a0 msgr2=0x5610be199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.305+0000 7f484abb5640 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18b4a0 0x5610be199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.305+0000 7f484abb5640 1 -- 172.21.15.133:0/4006799420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be18bdc0 msgr2=0x5610be19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:05.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.305+0000 7f484abb5640 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be18bdc0 0x5610be19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.305+0000 7f484abb5640 1 -- 172.21.15.133:0/4006799420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610be2097b0 con 0x5610be196680 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.305+0000 7f484a3b4640 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18b4a0 0x5610be199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.305+0000 7f484abb5640 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be196680 0x5610be209170 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f483c0974e0 tx=0x7f483c0a3e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d3b99c2f2ec6d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.305+0000 7f482dffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.306+0000 7f482dffb640 1 -- 172.21.15.133:0/4006799420 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c01a070 con 0x5610be196680 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.306+0000 7f482dffb640 1 -- 172.21.15.133:0/4006799420 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7f483c003b80 con 0x5610be196680 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.306+0000 7f484e9f1740 1 -- 172.21.15.133:0/4006799420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f483c09f0e0 con 0x5610be196680 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.306+0000 7f482dffb640 1 -- 172.21.15.133:0/4006799420 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f483c09b780 con 0x5610be196680 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.306+0000 7f484e9f1740 10 client.27866 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.306+0000 7f484e9f1740 1 -- 172.21.15.133:0/4006799420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5610be210b50 con 0x5610be196680 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.311+0000 7f482dffb640 1 -- 172.21.15.133:0/4006799420 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483c09f0e0 con 0x5610be196680 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.311+0000 7f482dffb640 1 -- 172.21.15.133:0/4006799420 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f483c00b040 con 0x5610be196680 2024-09-11T23:48:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.311+0000 7f484e9f1740 10 client.27866 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.311+0000 7f484e9f1740 10 client.27866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.311+0000 7f484e9f1740 20 client.27866 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.312+0000 7f484e9f1740 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5610be210d20 0x5610be213110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.312+0000 7f484e9f1740 4 client.27866 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.312+0000 7f484e9f1740 1 -- 172.21.15.133:0/4006799420 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610be0f8d60 con 0x5610be210d20 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.312+0000 7f484a3b4640 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5610be210d20 0x5610be213110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.315+0000 7f482dffb640 10 client.27866 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.315+0000 7f484a3b4640 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5610be210d20 0x5610be213110 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca601b1dadc4b25 server_cookie=9f1ed968482dc078 in_seq=0 out_seq=0 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.316+0000 7f482dffb640 1 -- 172.21.15.133:0/4006799420 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5610be0f8d60 con 0x5610be210d20 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.316+0000 7f482dffb640 10 client.27866 handle_command_reply: tid=0 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.316+0000 7f484e9f1740 1 client.27866 shutdown 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.316+0000 7f484e9f1740 1 -- 172.21.15.133:0/4006799420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be196680 msgr2=0x5610be209170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.316+0000 7f484e9f1740 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be196680 0x5610be209170 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f483c0974e0 tx=0x7f483c0a3e70 comp rx=0 tx=0).stop 2024-09-11T23:48:05.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.316+0000 7f484e9f1740 1 -- 172.21.15.133:0/4006799420 shutdown_connections 2024-09-11T23:48:05.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.316+0000 7f484e9f1740 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5610be210d20 0x5610be213110 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.316+0000 7f484e9f1740 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610be196680 0x5610be209170 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.316+0000 7f484e9f1740 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610be18bdc0 0x5610be19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.316+0000 7f484e9f1740 1 --2- 172.21.15.133:0/4006799420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610be18b4a0 0x5610be199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.316+0000 7f484e9f1740 1 -- 172.21.15.133:0/4006799420 >> 172.21.15.133:0/4006799420 conn(0x5610be187a30 msgr2=0x5610be186b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:05.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.317+0000 7f482dffb640 0 client.27866 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:05.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.317+0000 7f484e9f1740 1 -- 172.21.15.133:0/4006799420 shutdown_connections 2024-09-11T23:48:05.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.317+0000 7f484e9f1740 1 -- 172.21.15.133:0/4006799420 wait complete. 2024-09-11T23:48:05.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.317+0000 7f484e9f1740 20 client.27866 trim_cache size 0 max 16384 2024-09-11T23:48:05.322 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:48:05.323 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:48:05.323 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:48:05.323 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:48:05.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 -- 172.21.15.133:0/426945388 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f48300a2ef0 msgr2=0x7f48300a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f48300a2ef0 0x7f48300a53b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f483402a950 tx=0x7f4834000bc0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 -- 172.21.15.133:0/426945388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f484411b140 msgr2=0x7f48441abaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f484411b140 0x7f48441abaf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f483c00eeb0 tx=0x7f483c00c740 comp rx=0 tx=0).stop 2024-09-11T23:48:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 -- 172.21.15.133:0/426945388 shutdown_connections 2024-09-11T23:48:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f48300a2ef0 0x7f48300a53b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f484411b140 0x7f48441abaf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4844109830 0x7f48441a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 --2- 172.21.15.133:0/426945388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4844109450 0x7f48441a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 -- 172.21.15.133:0/426945388 >> 172.21.15.133:0/426945388 conn(0x7f484406e140 msgr2=0x7f484411c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 -- 172.21.15.133:0/426945388 shutdown_connections 2024-09-11T23:48:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:05.318+0000 7f484c618640 1 -- 172.21.15.133:0/426945388 wait complete. 2024-09-11T23:48:05.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:05 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:48:05.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:05 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:05.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:05 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:48:05.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:05 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:05.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:05 smithi138 ceph-mon[35142]: pgmap v353: 97 pgs: 97 active+clean; 70 GiB data, 263 GiB used, 810 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 685 MiB/s wr, 1.73k op/s 2024-09-11T23:48:06.023 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:48:06.024 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:48:06.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:06 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:48:06.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:06 smithi178 ceph-mon[35882]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:48:06.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:06 smithi178 ceph-mon[35882]: pgmap v355: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 70 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 827 MiB/s wr, 2k op/s 2024-09-11T23:48:06.400 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:06.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:06 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:48:06.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:06 smithi138 ceph-mon[35142]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:48:06.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:06 smithi138 ceph-mon[35142]: pgmap v355: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 70 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 827 MiB/s wr, 2k op/s 2024-09-11T23:48:06.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:06 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:48:06.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:06 smithi133 ceph-mon[31430]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:48:06.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:06 smithi133 ceph-mon[31430]: pgmap v355: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 70 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 827 MiB/s wr, 2k op/s 2024-09-11T23:48:06.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.925+0000 7f7abbdd2640 1 -- 172.21.15.133:0/2365693500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab4102da0 msgr2=0x7f7ab4103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:06.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.925+0000 7f7abbdd2640 1 --2- 172.21.15.133:0/2365693500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab4102da0 0x7f7ab4103180 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009900 tx=0x7f7ab002a480 comp rx=0 tx=0).stop 2024-09-11T23:48:06.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.925+0000 7f7abbdd2640 1 -- 172.21.15.133:0/2365693500 shutdown_connections 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.925+0000 7f7abbdd2640 1 --2- 172.21.15.133:0/2365693500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ab410c180 0x7f7ab410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.925+0000 7f7abbdd2640 1 --2- 172.21.15.133:0/2365693500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ab41036c0 0x7f7ab410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.925+0000 7f7abbdd2640 1 --2- 172.21.15.133:0/2365693500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab4102da0 0x7f7ab4103180 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.925+0000 7f7abbdd2640 1 -- 172.21.15.133:0/2365693500 >> 172.21.15.133:0/2365693500 conn(0x7f7ab40fc7a0 msgr2=0x7f7ab40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.925+0000 7f7abbdd2640 1 -- 172.21.15.133:0/2365693500 shutdown_connections 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.925+0000 7f7abbdd2640 1 -- 172.21.15.133:0/2365693500 wait complete. 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7abbdd2640 1 Processor -- start 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7abbdd2640 1 -- start start 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7abbdd2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab4102da0 0x7f7ab4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7abbdd2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ab41036c0 0x7f7ab41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7abbdd2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ab410c180 0x7f7ab419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7abbdd2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4111f60 con 0x7f7ab4102da0 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7abbdd2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4111de0 con 0x7f7ab410c180 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7abbdd2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41120e0 con 0x7f7ab41036c0 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7ab9b6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab4102da0 0x7f7ab4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7ab9b6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab4102da0 0x7f7ab4198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39536/0 (socket says 172.21.15.133:39536) 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7ab9b6e640 1 -- 172.21.15.133:0/4279845969 learned_addr learned my addr 172.21.15.133:0/4279845969 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7aba36f640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ab410c180 0x7f7ab419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.926+0000 7f7ab936d640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ab41036c0 0x7f7ab41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7aba36f640 1 -- 172.21.15.133:0/4279845969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ab41036c0 msgr2=0x7f7ab41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7aba36f640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ab41036c0 0x7f7ab41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7aba36f640 1 -- 172.21.15.133:0/4279845969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab4102da0 msgr2=0x7f7ab4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7aba36f640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab4102da0 0x7f7ab4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7aba36f640 1 -- 172.21.15.133:0/4279845969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab419d410 con 0x7f7ab410c180 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7ab9b6e640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab4102da0 0x7f7ab4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:06.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7aba36f640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ab410c180 0x7f7ab419cd30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00da10 tx=0x7f7aac00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:06.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7aaaffd640 1 -- 172.21.15.133:0/4279845969 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00bd70 con 0x7f7ab410c180 2024-09-11T23:48:06.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7aaaffd640 1 -- 172.21.15.133:0/4279845969 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aac003e50 con 0x7f7ab410c180 2024-09-11T23:48:06.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7abbdd2640 1 -- 172.21.15.133:0/4279845969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab419d700 con 0x7f7ab410c180 2024-09-11T23:48:06.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7aaaffd640 1 -- 172.21.15.133:0/4279845969 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0106a0 con 0x7f7ab410c180 2024-09-11T23:48:06.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.927+0000 7f7abbdd2640 1 -- 172.21.15.133:0/4279845969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab41a4fe0 con 0x7f7ab410c180 2024-09-11T23:48:06.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.928+0000 7f7abbdd2640 1 -- 172.21.15.133:0/4279845969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab4103180 con 0x7f7ab410c180 2024-09-11T23:48:06.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.929+0000 7f7aaaffd640 1 -- 172.21.15.133:0/4279845969 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7aac004b40 con 0x7f7ab410c180 2024-09-11T23:48:06.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.930+0000 7f7aaaffd640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7aa00a2cf0 0x7f7aa00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:06.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.930+0000 7f7ab9b6e640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7aa00a2cf0 0x7f7aa00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:06.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.930+0000 7f7aaaffd640 1 -- 172.21.15.133:0/4279845969 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7f7aac0cf8c0 con 0x7f7ab410c180 2024-09-11T23:48:06.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.930+0000 7f7ab9b6e640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7aa00a2cf0 0x7f7aa00a51b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00096c0 tx=0x7f7ab0005a20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:06.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:06.933+0000 7f7aaaffd640 1 -- 172.21.15.133:0/4279845969 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7aac09c4f0 con 0x7f7ab410c180 2024-09-11T23:48:07.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.148+0000 7f7abbdd2640 1 -- 172.21.15.133:0/4279845969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ab41997c0 con 0x7f7ab410c180 2024-09-11T23:48:07.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.149+0000 7f7aaaffd640 1 -- 172.21.15.133:0/4279845969 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11994 (secure 0 0 0) 0x7f7ab41997c0 con 0x7f7ab410c180 2024-09-11T23:48:07.149 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:48:07.149 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":32,"btime":"2024-09-11T23:48:01:740980+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:48:01.740959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27616":{"gid":27616,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2029124747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2029124747},{"type":"v1","addr":"172.21.15.178:6835","nonce":2029124747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:48:07.149 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 32 2024-09-11T23:48:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 -- 172.21.15.133:0/4279845969 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7aa00a2cf0 msgr2=0x7f7aa00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7aa00a2cf0 0x7f7aa00a51b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00096c0 tx=0x7f7ab0005a20 comp rx=0 tx=0).stop 2024-09-11T23:48:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 -- 172.21.15.133:0/4279845969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ab410c180 msgr2=0x7f7ab419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ab410c180 0x7f7ab419cd30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00da10 tx=0x7f7aac00ded0 comp rx=0 tx=0).stop 2024-09-11T23:48:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 -- 172.21.15.133:0/4279845969 shutdown_connections 2024-09-11T23:48:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7aa00a2cf0 0x7f7aa00a51b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ab410c180 0x7f7ab419cd30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ab41036c0 0x7f7ab41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 --2- 172.21.15.133:0/4279845969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab4102da0 0x7f7ab4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 -- 172.21.15.133:0/4279845969 >> 172.21.15.133:0/4279845969 conn(0x7f7ab40fc7a0 msgr2=0x7f7ab40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 -- 172.21.15.133:0/4279845969 shutdown_connections 2024-09-11T23:48:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:07.151+0000 7f7abbdd2640 1 -- 172.21.15.133:0/4279845969 wait complete. 2024-09-11T23:48:07.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:07 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/4279845969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:48:07.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:07 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/4279845969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:48:07.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:07 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/4279845969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:48:07.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:48:08.179 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:08.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:08 smithi178 ceph-mon[35882]: pgmap v356: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 68 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 509 MiB/s wr, 1.53k op/s 2024-09-11T23:48:08.552 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:08 smithi133 ceph-mon[31430]: pgmap v356: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 68 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 509 MiB/s wr, 1.53k op/s 2024-09-11T23:48:08.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.635+0000 7f895bed5640 1 -- 172.21.15.133:0/32463121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8954106a60 msgr2=0x7f8954106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:08.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.635+0000 7f895bed5640 1 --2- 172.21.15.133:0/32463121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8954106a60 0x7f8954106e40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8950009930 tx=0x7f895002a420 comp rx=0 tx=0).stop 2024-09-11T23:48:08.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.635+0000 7f895bed5640 1 -- 172.21.15.133:0/32463121 shutdown_connections 2024-09-11T23:48:08.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.635+0000 7f895bed5640 1 --2- 172.21.15.133:0/32463121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8954101400 0x7f895410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.635+0000 7f895bed5640 1 --2- 172.21.15.133:0/32463121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8954100a60 0x7f8954100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.635+0000 7f895bed5640 1 --2- 172.21.15.133:0/32463121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8954106a60 0x7f8954106e40 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.635+0000 7f895bed5640 1 -- 172.21.15.133:0/32463121 >> 172.21.15.133:0/32463121 conn(0x7f89540fc780 msgr2=0x7f89540feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:08.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.635+0000 7f895bed5640 1 -- 172.21.15.133:0/32463121 shutdown_connections 2024-09-11T23:48:08.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.635+0000 7f895bed5640 1 -- 172.21.15.133:0/32463121 wait complete. 2024-09-11T23:48:08.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f895bed5640 1 Processor -- start 2024-09-11T23:48:08.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f895bed5640 1 -- start start 2024-09-11T23:48:08.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f895bed5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8954100a60 0x7f8954197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f895bed5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8954101400 0x7f89541978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f895bed5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8954106a60 0x7f895419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f895bed5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8954110e60 con 0x7f8954106a60 2024-09-11T23:48:08.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f895bed5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8954110ce0 con 0x7f8954101400 2024-09-11T23:48:08.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f895bed5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8954110fe0 con 0x7f8954100a60 2024-09-11T23:48:08.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f895a472640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8954106a60 0x7f895419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f895a472640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8954106a60 0x7f895419bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39552/0 (socket says 172.21.15.133:39552) 2024-09-11T23:48:08.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f8959c71640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8954100a60 0x7f8954197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f895a472640 1 -- 172.21.15.133:0/2629883085 learned_addr learned my addr 172.21.15.133:0/2629883085 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.636+0000 7f8959470640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8954101400 0x7f89541978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.637+0000 7f895a472640 1 -- 172.21.15.133:0/2629883085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8954100a60 msgr2=0x7f8954197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.637+0000 7f895a472640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8954100a60 0x7f8954197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.637+0000 7f895a472640 1 -- 172.21.15.133:0/2629883085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8954101400 msgr2=0x7f89541978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.637+0000 7f895a472640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8954101400 0x7f89541978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.637+0000 7f895a472640 1 -- 172.21.15.133:0/2629883085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895419c320 con 0x7f8954106a60 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.637+0000 7f895a472640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8954106a60 0x7f895419bc40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f894c00eaa0 tx=0x7f894c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.637+0000 7f894affd640 1 -- 172.21.15.133:0/2629883085 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c00cdb0 con 0x7f8954106a60 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.637+0000 7f895bed5640 1 -- 172.21.15.133:0/2629883085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895419c610 con 0x7f8954106a60 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.637+0000 7f894affd640 1 -- 172.21.15.133:0/2629883085 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f894c005020 con 0x7f8954106a60 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.637+0000 7f894affd640 1 -- 172.21.15.133:0/2629883085 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c0116b0 con 0x7f8954106a60 2024-09-11T23:48:08.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.637+0000 7f895bed5640 1 -- 172.21.15.133:0/2629883085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895406a790 con 0x7f8954106a60 2024-09-11T23:48:08.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.639+0000 7f895e2ae740 1 Processor -- start 2024-09-11T23:48:08.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f894affd640 1 -- 172.21.15.133:0/2629883085 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f894c005560 con 0x7f8954106a60 2024-09-11T23:48:08.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f895e2ae740 1 -- start start 2024-09-11T23:48:08.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9cfb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 0x555dd9cfba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d08760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f895e2ae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555dd9c612f0 con 0x555dd9d06370 2024-09-11T23:48:08.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f895e2ae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555dd9c08cf0 con 0x555dd9cfb5d0 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f895e2ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555dd9c5d2b0 con 0x555dd9cfcb40 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f8959470640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d08760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f8959470640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d08760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39554/0 (socket says 172.21.15.133:39554) 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f8959470640 1 -- 172.21.15.133:0/1504958878 learned_addr learned my addr 172.21.15.133:0/1504958878 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.640+0000 7f8959470640 1 --2- 172.21.15.133:0/1504958878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9cfb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f894affd640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f89400a2cf0 0x7f89400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f8959470640 1 -- 172.21.15.133:0/1504958878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 msgr2=0x555dd9cfb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f8959470640 1 --2- 172.21.15.133:0/1504958878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9cfb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f8959470640 1 -- 172.21.15.133:0/1504958878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 msgr2=0x555dd9cfba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f8959470640 1 --2- 172.21.15.133:0/1504958878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 0x555dd9cfba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f8959470640 1 -- 172.21.15.133:0/1504958878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dd9bed3a0 con 0x555dd9d06370 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f895a472640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f89400a2cf0 0x7f89400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f894affd640 1 -- 172.21.15.133:0/2629883085 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7f894c0cf570 con 0x7f8954106a60 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f8959470640 1 --2- 172.21.15.133:0/1504958878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d08760 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8944009f90 tx=0x7f8944005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2d24d31ce647dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f893ffff640 1 -- 172.21.15.133:0/1504958878 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944004030 con 0x555dd9d06370 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f893ffff640 1 -- 172.21.15.133:0/1504958878 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89440041d0 con 0x555dd9d06370 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.641+0000 7f893ffff640 1 -- 172.21.15.133:0/1504958878 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894400f740 con 0x555dd9d06370 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.642+0000 7f895e2ae740 1 -- 172.21.15.133:0/1504958878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 msgr2=0x555dd9d08760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.642+0000 7f895a472640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f89400a2cf0 0x7f89400a51b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f894c09b830 tx=0x7f894c0054d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.642+0000 7f895e2ae740 1 --2- 172.21.15.133:0/1504958878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d08760 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8944009f90 tx=0x7f8944005850 comp rx=0 tx=0).stop 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.642+0000 7f895e2ae740 1 -- 172.21.15.133:0/1504958878 shutdown_connections 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.642+0000 7f895e2ae740 1 --2- 172.21.15.133:0/1504958878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d08760 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.642+0000 7f895e2ae740 1 --2- 172.21.15.133:0/1504958878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 0x555dd9cfba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.642+0000 7f895e2ae740 1 --2- 172.21.15.133:0/1504958878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9cfb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.642+0000 7f895e2ae740 1 -- 172.21.15.133:0/1504958878 >> 172.21.15.133:0/1504958878 conn(0x555dd9cc4310 msgr2=0x555dd9cf7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:08.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.642+0000 7f895e2ae740 1 -- 172.21.15.133:0/1504958878 shutdown_connections 2024-09-11T23:48:08.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.642+0000 7f895e2ae740 1 -- 172.21.15.133:0/1504958878 wait complete. 2024-09-11T23:48:08.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.642+0000 7f895e2ae740 1 -- start start 2024-09-11T23:48:08.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f895e2ae740 1 Processor -- start 2024-09-11T23:48:08.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfb5d0 0x555dd9d03100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9cfcb40 0x555dd9d03640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9d06370 0x555dd9d03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f895e2ae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555dd9d09070 con 0x555dd9cfcb40 2024-09-11T23:48:08.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f895e2ae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555dd9c1c060 con 0x555dd9d06370 2024-09-11T23:48:08.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f895e2ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555dd9d091f0 con 0x555dd9cfb5d0 2024-09-11T23:48:08.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f895e2ae740 10 client.0 _ll_register_callbacks cb 0x555dd996f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:08.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f8959c71640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9cfcb40 0x555dd9d03640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f8959c71640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9cfcb40 0x555dd9d03640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39568/0 (socket says 172.21.15.133:39568) 2024-09-11T23:48:08.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f8959c71640 1 -- 172.21.15.133:0/3365935527 learned_addr learned my addr 172.21.15.133:0/3365935527 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:08.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f8959470640 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9d06370 0x555dd9d03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f8959470640 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfb5d0 0x555dd9d03100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f8959c71640 1 -- 172.21.15.133:0/3365935527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfb5d0 msgr2=0x555dd9d03100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:08.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f8959c71640 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfb5d0 0x555dd9d03100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f8959c71640 1 -- 172.21.15.133:0/3365935527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9d06370 msgr2=0x555dd9d03b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f8959c71640 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9d06370 0x555dd9d03b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f8959c71640 1 -- 172.21.15.133:0/3365935527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dd99d23c0 con 0x555dd9cfcb40 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.643+0000 7f8959470640 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9d06370 0x555dd9d03b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f8959c71640 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9cfcb40 0x555dd9d03640 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8950009930 tx=0x7f89500317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7655a4c80ba353 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f893d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f893d7fa640 1 -- 172.21.15.133:0/3365935527 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950040070 con 0x555dd9cfcb40 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f893d7fa640 1 -- 172.21.15.133:0/3365935527 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7f8950004240 con 0x555dd9cfcb40 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f895e2ae740 1 -- 172.21.15.133:0/3365935527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89440041d0 con 0x555dd9cfcb40 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f893d7fa640 1 -- 172.21.15.133:0/3365935527 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8950035040 con 0x555dd9cfcb40 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f895e2ae740 10 client.18336 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f895e2ae740 1 -- 172.21.15.133:0/3365935527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555dd9d14c00 con 0x555dd9cfcb40 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f893d7fa640 1 -- 172.21.15.133:0/3365935527 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89440041d0 con 0x555dd9cfcb40 2024-09-11T23:48:08.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f893d7fa640 1 -- 172.21.15.133:0/3365935527 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f895004d590 con 0x555dd9cfcb40 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f895e2ae740 10 client.18336 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f895e2ae740 10 client.18336 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f895e2ae740 20 client.18336 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x555dd9d16060 0x555dd9d18480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f895e2ae740 4 client.18336 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.644+0000 7f895e2ae740 1 -- 172.21.15.133:0/3365935527 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dd9c6e860 con 0x555dd9d16060 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.645+0000 7f8959470640 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x555dd9d16060 0x555dd9d18480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.645+0000 7f893d7fa640 10 client.18336 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.645+0000 7f8959470640 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x555dd9d16060 0x555dd9d18480 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e8c99a96746da8b server_cookie=6aa111f03ebeb4fe in_seq=0 out_seq=0 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f893d7fa640 1 -- 172.21.15.133:0/3365935527 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555dd9c6e860 con 0x555dd9d16060 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f893d7fa640 10 client.18336 handle_command_reply: tid=0 2024-09-11T23:48:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 1 client.18336 shutdown 2024-09-11T23:48:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 1 -- 172.21.15.133:0/3365935527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9cfcb40 msgr2=0x555dd9d03640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9cfcb40 0x555dd9d03640 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8950009930 tx=0x7f89500317e0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 1 -- 172.21.15.133:0/3365935527 shutdown_connections 2024-09-11T23:48:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x555dd9d16060 0x555dd9d18480 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9d06370 0x555dd9d03b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9cfcb40 0x555dd9d03640 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f893d7fa640 0 client.18336 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:48:08.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3365935527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfb5d0 0x555dd9d03100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 1 -- 172.21.15.133:0/3365935527 >> 172.21.15.133:0/3365935527 conn(0x555dd9cc4310 msgr2=0x555dd9cfd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:08.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 1 -- 172.21.15.133:0/3365935527 shutdown_connections 2024-09-11T23:48:08.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 1 -- 172.21.15.133:0/3365935527 wait complete. 2024-09-11T23:48:08.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.646+0000 7f895e2ae740 20 client.18336 trim_cache size 0 max 16384 2024-09-11T23:48:08.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.659+0000 7f895e2ae740 1 Processor -- start 2024-09-11T23:48:08.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.659+0000 7f895e2ae740 1 -- start start 2024-09-11T23:48:08.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.659+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 0x555dd9d039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.659+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9d1dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.659+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d1e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.659+0000 7f895e2ae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f89541afbb0 con 0x555dd9d06370 2024-09-11T23:48:08.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.659+0000 7f895e2ae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89541b16f0 con 0x555dd9cfb5d0 2024-09-11T23:48:08.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.659+0000 7f895e2ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f89541b0280 con 0x555dd9cfcb40 2024-09-11T23:48:08.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f8959470640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d1e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f8959470640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d1e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39580/0 (socket says 172.21.15.133:39580) 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f8959470640 1 -- 172.21.15.133:0/214164246 learned_addr learned my addr 172.21.15.133:0/214164246 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f8959c71640 1 --2- 172.21.15.133:0/214164246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9d1dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f8959470640 1 --2- 172.21.15.133:0/214164246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 0x555dd9d039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f8959470640 1 -- 172.21.15.133:0/214164246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 msgr2=0x555dd9d1dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f8959470640 1 --2- 172.21.15.133:0/214164246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9d1dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f8959470640 1 -- 172.21.15.133:0/214164246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 msgr2=0x555dd9d039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f8959470640 1 --2- 172.21.15.133:0/214164246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 0x555dd9d039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f8959470640 1 -- 172.21.15.133:0/214164246 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dd9cfe120 con 0x555dd9d06370 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f8959470640 1 --2- 172.21.15.133:0/214164246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d1e1f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8944003ed0 tx=0x7f894401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b983b5aa074322 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f893f7fe640 1 -- 172.21.15.133:0/214164246 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944005240 con 0x555dd9d06370 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f893f7fe640 1 -- 172.21.15.133:0/214164246 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8944003b80 con 0x555dd9d06370 2024-09-11T23:48:08.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.660+0000 7f893f7fe640 1 -- 172.21.15.133:0/214164246 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944004bd0 con 0x555dd9d06370 2024-09-11T23:48:08.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.661+0000 7f895e2ae740 1 -- 172.21.15.133:0/214164246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 msgr2=0x555dd9d1e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:08.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.661+0000 7f895e2ae740 1 --2- 172.21.15.133:0/214164246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d1e1f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8944003ed0 tx=0x7f894401a040 comp rx=0 tx=0).stop 2024-09-11T23:48:08.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.661+0000 7f895e2ae740 1 -- 172.21.15.133:0/214164246 shutdown_connections 2024-09-11T23:48:08.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.661+0000 7f895e2ae740 1 --2- 172.21.15.133:0/214164246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d1e1f0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.661+0000 7f895e2ae740 1 --2- 172.21.15.133:0/214164246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9d1dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.661+0000 7f895e2ae740 1 --2- 172.21.15.133:0/214164246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 0x555dd9d039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.661+0000 7f895e2ae740 1 -- 172.21.15.133:0/214164246 >> 172.21.15.133:0/214164246 conn(0x555dd9cc4310 msgr2=0x555dd9cf6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:08.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.661+0000 7f895e2ae740 1 -- 172.21.15.133:0/214164246 shutdown_connections 2024-09-11T23:48:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.661+0000 7f895e2ae740 1 -- 172.21.15.133:0/214164246 wait complete. 2024-09-11T23:48:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.661+0000 7f895e2ae740 1 -- start start 2024-09-11T23:48:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.661+0000 7f895e2ae740 1 Processor -- start 2024-09-11T23:48:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 0x555dd9d0ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9d09d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f895e2ae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d0a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f895e2ae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f89541b1020 con 0x555dd9d06370 2024-09-11T23:48:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f895e2ae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89541b0950 con 0x555dd9cfb5d0 2024-09-11T23:48:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f895e2ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f89541adbb0 con 0x555dd9cfcb40 2024-09-11T23:48:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f895e2ae740 10 client.0 _ll_register_callbacks cb 0x7f8950009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:08.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f8959470640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d0a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f8959470640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d0a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39586/0 (socket says 172.21.15.133:39586) 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f8959470640 1 -- 172.21.15.133:0/3366941742 learned_addr learned my addr 172.21.15.133:0/3366941742 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f8959470640 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 0x555dd9d0ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f8959c71640 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9d09d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f8959470640 1 -- 172.21.15.133:0/3366941742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 msgr2=0x555dd9d09d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f8959470640 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9d09d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f8959470640 1 -- 172.21.15.133:0/3366941742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 msgr2=0x555dd9d0ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f8959470640 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 0x555dd9d0ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f8959470640 1 -- 172.21.15.133:0/3366941742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dd9d0a910 con 0x555dd9d06370 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f8959470640 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d0a2d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8944004320 tx=0x7f8944021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a392ae8d596ec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.662+0000 7f893d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f893d7fa640 1 -- 172.21.15.133:0/3366941742 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944005030 con 0x555dd9d06370 2024-09-11T23:48:08.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f895e2ae740 1 -- 172.21.15.133:0/3366941742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8944003b80 con 0x555dd9d06370 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f893d7fa640 1 -- 172.21.15.133:0/3366941742 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7f89440051d0 con 0x555dd9d06370 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f893d7fa640 1 -- 172.21.15.133:0/3366941742 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f89440053d0 con 0x555dd9d06370 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f895e2ae740 10 client.18342 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f895e2ae740 1 -- 172.21.15.133:0/3366941742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555dd9d80ad0 con 0x555dd9d06370 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f893d7fa640 1 -- 172.21.15.133:0/3366941742 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8944003b80 con 0x555dd9d06370 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f893d7fa640 1 -- 172.21.15.133:0/3366941742 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f894401e420 con 0x555dd9d06370 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f895e2ae740 10 client.18342 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f895e2ae740 10 client.18342 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f895e2ae740 20 client.18342 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x555dd9d80f60 0x555dd9d83350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f895e2ae740 4 client.18342 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.663+0000 7f895e2ae740 1 -- 172.21.15.133:0/3366941742 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555dd9c68d70 con 0x555dd9d80f60 2024-09-11T23:48:08.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f8959c71640 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x555dd9d80f60 0x555dd9d83350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f8959c71640 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x555dd9d80f60 0x555dd9d83350 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a44804dfd7527cce server_cookie=6f8274467d376a81 in_seq=0 out_seq=0 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f893d7fa640 10 client.18342 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f893d7fa640 1 -- 172.21.15.133:0/3366941742 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555dd9c68d70 con 0x555dd9d80f60 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f893d7fa640 10 client.18342 handle_command_reply: tid=0 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 1 client.18342 shutdown 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 1 -- 172.21.15.133:0/3366941742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 msgr2=0x555dd9d0a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d0a2d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8944004320 tx=0x7f8944021040 comp rx=0 tx=0).stop 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 1 -- 172.21.15.133:0/3366941742 shutdown_connections 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x555dd9d80f60 0x555dd9d83350 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dd9d06370 0x555dd9d0a2d0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555dd9cfcb40 0x555dd9d09d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 1 --2- 172.21.15.133:0/3366941742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd9cfb5d0 0x555dd9d0ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f893d7fa640 0 client.18342 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:48:08.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 1 -- 172.21.15.133:0/3366941742 >> 172.21.15.133:0/3366941742 conn(0x555dd9cc4310 msgr2=0x555dd9cf64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:08.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 1 -- 172.21.15.133:0/3366941742 shutdown_connections 2024-09-11T23:48:08.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 1 -- 172.21.15.133:0/3366941742 wait complete. 2024-09-11T23:48:08.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.664+0000 7f895e2ae740 20 client.18342 trim_cache size 0 max 16384 2024-09-11T23:48:08.670 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:48:08.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 -- 172.21.15.133:0/2629883085 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f89400a2cf0 msgr2=0x7f89400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:08.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f89400a2cf0 0x7f89400a51b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f894c09b830 tx=0x7f894c0054d0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 -- 172.21.15.133:0/2629883085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8954106a60 msgr2=0x7f895419bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:08.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8954106a60 0x7f895419bc40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f894c00eaa0 tx=0x7f894c00ef60 comp rx=0 tx=0).stop 2024-09-11T23:48:08.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 -- 172.21.15.133:0/2629883085 shutdown_connections 2024-09-11T23:48:08.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f89400a2cf0 0x7f89400a51b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8954106a60 0x7f895419bc40 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8954101400 0x7f89541978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 --2- 172.21.15.133:0/2629883085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8954100a60 0x7f8954197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:08.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 -- 172.21.15.133:0/2629883085 >> 172.21.15.133:0/2629883085 conn(0x7f89540fc780 msgr2=0x7f89540fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:08.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 -- 172.21.15.133:0/2629883085 shutdown_connections 2024-09-11T23:48:08.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:08.665+0000 7f895bed5640 1 -- 172.21.15.133:0/2629883085 wait complete. 2024-09-11T23:48:08.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:08 smithi138 ceph-mon[35142]: pgmap v356: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 68 GiB data, 265 GiB used, 808 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 509 MiB/s wr, 1.53k op/s 2024-09-11T23:48:09.719 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:48:09.971 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:10.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.611+0000 7f214e5dd640 1 -- 172.21.15.133:0/3256129437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2140095510 msgr2=0x7f2140099120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:10.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.611+0000 7f214e5dd640 1 --2- 172.21.15.133:0/3256129437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2140095510 0x7f2140099120 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2138009930 tx=0x7f213802a480 comp rx=0 tx=0).stop 2024-09-11T23:48:10.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.612+0000 7f214e5dd640 1 -- 172.21.15.133:0/3256129437 shutdown_connections 2024-09-11T23:48:10.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.612+0000 7f214e5dd640 1 --2- 172.21.15.133:0/3256129437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2140099880 0x7f21400a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.612+0000 7f214e5dd640 1 --2- 172.21.15.133:0/3256129437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2140095510 0x7f2140099120 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.612+0000 7f214e5dd640 1 --2- 172.21.15.133:0/3256129437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2140094b60 0x7f2140094f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.612+0000 7f214e5dd640 1 -- 172.21.15.133:0/3256129437 >> 172.21.15.133:0/3256129437 conn(0x7f214000a1b0 msgr2=0x7f214000a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:10.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.614+0000 7f214e5dd640 1 -- 172.21.15.133:0/3256129437 shutdown_connections 2024-09-11T23:48:10.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.617+0000 7f214e5dd640 1 -- 172.21.15.133:0/3256129437 wait complete. 2024-09-11T23:48:10.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.618+0000 7f214e5dd640 1 Processor -- start 2024-09-11T23:48:10.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.618+0000 7f214e5dd640 1 -- start start 2024-09-11T23:48:10.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.618+0000 7f214e5dd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2140094b60 0x7f21400a9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.618+0000 7f214e5dd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2140095510 0x7f21400aa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.618+0000 7f214e5dd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2140099880 0x7f21400a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.618+0000 7f214e5dd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f21400a37c0 con 0x7f2140094b60 2024-09-11T23:48:10.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.618+0000 7f214e5dd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21400a3640 con 0x7f2140099880 2024-09-11T23:48:10.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.618+0000 7f214e5dd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f21400a3940 con 0x7f2140095510 2024-09-11T23:48:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.623+0000 7f214dddc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2140099880 0x7f21400a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.623+0000 7f214dddc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2140099880 0x7f21400a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:53998/0 (socket says 172.21.15.133:53998) 2024-09-11T23:48:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.623+0000 7f214dddc640 1 -- 172.21.15.133:0/2761348897 learned_addr learned my addr 172.21.15.133:0/2761348897 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.623+0000 7f214cdda640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2140095510 0x7f21400aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.625+0000 7f214cdda640 1 -- 172.21.15.133:0/2761348897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2140099880 msgr2=0x7f21400a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:10.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.625+0000 7f214cdda640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2140099880 0x7f21400a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.625+0000 7f214cdda640 1 -- 172.21.15.133:0/2761348897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2140094b60 msgr2=0x7f21400a9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:10.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.625+0000 7f214cdda640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2140094b60 0x7f21400a9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.625+0000 7f214cdda640 1 -- 172.21.15.133:0/2761348897 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21400a5710 con 0x7f2140095510 2024-09-11T23:48:10.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.625+0000 7f214dddc640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2140099880 0x7f21400a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:48:10.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.632+0000 7f214cdda640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2140095510 0x7f21400aa220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2138034040 tx=0x7f2138005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:10.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.634+0000 7f213e7fc640 1 -- 172.21.15.133:0/2761348897 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138041020 con 0x7f2140095510 2024-09-11T23:48:10.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.634+0000 7f214e5dd640 1 -- 172.21.15.133:0/2761348897 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21400a59a0 con 0x7f2140095510 2024-09-11T23:48:10.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.635+0000 7f214e5dd640 1 -- 172.21.15.133:0/2761348897 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21400afbd0 con 0x7f2140095510 2024-09-11T23:48:10.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.635+0000 7f213e7fc640 1 -- 172.21.15.133:0/2761348897 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2138005430 con 0x7f2140095510 2024-09-11T23:48:10.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.635+0000 7f213e7fc640 1 -- 172.21.15.133:0/2761348897 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138036040 con 0x7f2140095510 2024-09-11T23:48:10.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.639+0000 7f2151c18740 1 Processor -- start 2024-09-11T23:48:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.643+0000 7f2151c18740 1 -- start start 2024-09-11T23:48:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.644+0000 7f2151c18740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d7d20 0x5591055d6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.644+0000 7f2151c18740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d67b0 0x5591055d6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.644+0000 7f2151c18740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055e3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.645+0000 7f2151c18740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55910553c2f0 con 0x5591055d7d20 2024-09-11T23:48:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.645+0000 7f2151c18740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5591054e3cf0 con 0x5591055e1550 2024-09-11T23:48:10.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.645+0000 7f2151c18740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5591055382b0 con 0x5591055d67b0 2024-09-11T23:48:10.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.647+0000 7f213e7fc640 1 -- 172.21.15.133:0/2761348897 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2138044020 con 0x7f2140095510 2024-09-11T23:48:10.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.649+0000 7f213e7fc640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f21340a2e40 0x7f21340a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.649+0000 7f213e7fc640 1 -- 172.21.15.133:0/2761348897 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7f21380f2150 con 0x7f2140095510 2024-09-11T23:48:10.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.649+0000 7f214d5db640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d67b0 0x5591055d6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.649+0000 7f214cdda640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.649+0000 7f214cdda640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055e3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:54012/0 (socket says 172.21.15.133:54012) 2024-09-11T23:48:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.649+0000 7f214cdda640 1 -- 172.21.15.133:0/3627607022 learned_addr learned my addr 172.21.15.133:0/3627607022 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.650+0000 7f214d5db640 1 -- 172.21.15.133:0/3627607022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 msgr2=0x5591055e3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:10.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.650+0000 7f214d5db640 1 --2- 172.21.15.133:0/3627607022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055e3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.650+0000 7f214d5db640 1 -- 172.21.15.133:0/3627607022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d7d20 msgr2=0x5591055d6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:10.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.650+0000 7f214d5db640 1 --2- 172.21.15.133:0/3627607022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d7d20 0x5591055d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.650+0000 7f214d5db640 1 -- 172.21.15.133:0/3627607022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591054c83a0 con 0x5591055d67b0 2024-09-11T23:48:10.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.651+0000 7f214dddc640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f21340a2e40 0x7f21340a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.653+0000 7f214d5db640 1 --2- 172.21.15.133:0/3627607022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d67b0 0x5591055d6bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f21480679d0 tx=0x7f2148050990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb64510b807c23d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:10.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.653+0000 7f214dddc640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f21340a2e40 0x7f21340a5300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2144005fd0 tx=0x7f2144005ec0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:10.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.653+0000 7f21337fe640 1 -- 172.21.15.133:0/3627607022 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214806cb80 con 0x5591055d67b0 2024-09-11T23:48:10.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.653+0000 7f21337fe640 1 -- 172.21.15.133:0/3627607022 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2148078030 con 0x5591055d67b0 2024-09-11T23:48:10.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.655+0000 7f2151c18740 1 -- 172.21.15.133:0/3627607022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d67b0 msgr2=0x5591055d6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:10.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.655+0000 7f2151c18740 1 --2- 172.21.15.133:0/3627607022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d67b0 0x5591055d6bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f21480679d0 tx=0x7f2148050990 comp rx=0 tx=0).stop 2024-09-11T23:48:10.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.656+0000 7f2151c18740 1 -- 172.21.15.133:0/3627607022 shutdown_connections 2024-09-11T23:48:10.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.656+0000 7f2151c18740 1 --2- 172.21.15.133:0/3627607022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055e3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.656+0000 7f2151c18740 1 --2- 172.21.15.133:0/3627607022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d67b0 0x5591055d6bf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.656+0000 7f2151c18740 1 --2- 172.21.15.133:0/3627607022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d7d20 0x5591055d6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.656+0000 7f2151c18740 1 -- 172.21.15.133:0/3627607022 >> 172.21.15.133:0/3627607022 conn(0x55910559f310 msgr2=0x5591055d2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:10.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.657+0000 7f2151c18740 1 -- 172.21.15.133:0/3627607022 shutdown_connections 2024-09-11T23:48:10.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.657+0000 7f2151c18740 1 -- 172.21.15.133:0/3627607022 wait complete. 2024-09-11T23:48:10.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.657+0000 7f2151c18740 1 -- start start 2024-09-11T23:48:10.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.658+0000 7f2151c18740 1 Processor -- start 2024-09-11T23:48:10.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.658+0000 7f2151c18740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d67b0 0x5591055e0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.658+0000 7f2151c18740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d7d20 0x5591055de0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.658+0000 7f2151c18740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.658+0000 7f2151c18740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5591055e4250 con 0x5591055d7d20 2024-09-11T23:48:10.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.658+0000 7f2151c18740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5591054f7060 con 0x5591055e1550 2024-09-11T23:48:10.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.658+0000 7f2151c18740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5591055e43d0 con 0x5591055d67b0 2024-09-11T23:48:10.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.658+0000 7f2151c18740 10 client.0 _ll_register_callbacks cb 0x55910524a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:10.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.660+0000 7f214cdda640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d7d20 0x5591055de0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.660+0000 7f214cdda640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d7d20 0x5591055de0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39634/0 (socket says 172.21.15.133:39634) 2024-09-11T23:48:10.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.660+0000 7f214cdda640 1 -- 172.21.15.133:0/786060052 learned_addr learned my addr 172.21.15.133:0/786060052 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:10.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.660+0000 7f214dddc640 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.660+0000 7f214d5db640 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d67b0 0x5591055e0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.660+0000 7f214cdda640 1 -- 172.21.15.133:0/786060052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d67b0 msgr2=0x5591055e0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.660+0000 7f214cdda640 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d67b0 0x5591055e0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.660+0000 7f214cdda640 1 -- 172.21.15.133:0/786060052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 msgr2=0x5591055de5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.660+0000 7f214cdda640 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055de5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.660+0000 7f214cdda640 1 -- 172.21.15.133:0/786060052 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591052ad3c0 con 0x5591055d7d20 2024-09-11T23:48:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.661+0000 7f214cdda640 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d7d20 0x5591055de0a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f21380c6d60 tx=0x7f2138009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fddcef97712876 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.661+0000 7f2130ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:10.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.662+0000 7f2130ff9640 1 -- 172.21.15.133:0/786060052 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21380d7060 con 0x5591055d7d20 2024-09-11T23:48:10.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.662+0000 7f2130ff9640 1 -- 172.21.15.133:0/786060052 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7f21380efad0 con 0x5591055d7d20 2024-09-11T23:48:10.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.662+0000 7f2151c18740 1 -- 172.21.15.133:0/786060052 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2148078030 con 0x5591055d7d20 2024-09-11T23:48:10.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.664+0000 7f2130ff9640 1 -- 172.21.15.133:0/786060052 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f21380efcd0 con 0x5591055d7d20 2024-09-11T23:48:10.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.666+0000 7f2151c18740 10 client.18348 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:10.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.666+0000 7f2151c18740 1 -- 172.21.15.133:0/786060052 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5591055efc30 con 0x5591055d7d20 2024-09-11T23:48:10.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.670+0000 7f2130ff9640 1 -- 172.21.15.133:0/786060052 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2148078030 con 0x5591055d7d20 2024-09-11T23:48:10.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.672+0000 7f2130ff9640 1 -- 172.21.15.133:0/786060052 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f21380a7b60 con 0x5591055d7d20 2024-09-11T23:48:10.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.672+0000 7f2151c18740 10 client.18348 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:10.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.672+0000 7f2151c18740 10 client.18348 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:48:10.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.673+0000 7f2151c18740 20 client.18348 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:10.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.673+0000 7f2151c18740 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591055f1310 0x5591055f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.673+0000 7f2151c18740 4 client.18348 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:10.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.673+0000 7f2151c18740 1 -- 172.21.15.133:0/786060052 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559105549860 con 0x5591055f1310 2024-09-11T23:48:10.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.674+0000 7f214d5db640 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591055f1310 0x5591055f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.676+0000 7f214d5db640 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591055f1310 0x5591055f37b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4f9b1d1962bbce20 server_cookie=2504a990961e9e8a in_seq=0 out_seq=0 2024-09-11T23:48:10.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.676+0000 7f2130ff9640 10 client.18348 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:48:10.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.678+0000 7f2130ff9640 1 -- 172.21.15.133:0/786060052 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559105549860 con 0x5591055f1310 2024-09-11T23:48:10.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.678+0000 7f2130ff9640 10 client.18348 handle_command_reply: tid=0 2024-09-11T23:48:10.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.678+0000 7f2151c18740 1 client.18348 shutdown 2024-09-11T23:48:10.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.678+0000 7f2151c18740 1 -- 172.21.15.133:0/786060052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d7d20 msgr2=0x5591055de0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:10.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.678+0000 7f2151c18740 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d7d20 0x5591055de0a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f21380c6d60 tx=0x7f2138009260 comp rx=0 tx=0).stop 2024-09-11T23:48:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.678+0000 7f2151c18740 1 -- 172.21.15.133:0/786060052 shutdown_connections 2024-09-11T23:48:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.678+0000 7f2151c18740 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5591055f1310 0x5591055f37b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.678+0000 7f2151c18740 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055de5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.678+0000 7f2151c18740 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d7d20 0x5591055de0a0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.678+0000 7f2151c18740 1 --2- 172.21.15.133:0/786060052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d67b0 0x5591055e0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.678+0000 7f2151c18740 1 -- 172.21.15.133:0/786060052 >> 172.21.15.133:0/786060052 conn(0x55910559f310 msgr2=0x5591055d8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.679+0000 7f2130ff9640 0 client.18348 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:48:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.679+0000 7f2151c18740 1 -- 172.21.15.133:0/786060052 shutdown_connections 2024-09-11T23:48:10.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.679+0000 7f2151c18740 1 -- 172.21.15.133:0/786060052 wait complete. 2024-09-11T23:48:10.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.679+0000 7f2151c18740 20 client.18348 trim_cache size 0 max 16384 2024-09-11T23:48:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.730+0000 7f2151c18740 1 Processor -- start 2024-09-11T23:48:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.731+0000 7f2151c18740 1 -- start start 2024-09-11T23:48:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.731+0000 7f2151c18740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 0x5591055ed570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.731+0000 7f2151c18740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d7d20 0x5591055de060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.732+0000 7f2151c18740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.732+0000 7f2151c18740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f212c00e620 con 0x5591055d67b0 2024-09-11T23:48:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.732+0000 7f2151c18740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f212c010160 con 0x5591055e1550 2024-09-11T23:48:10.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.732+0000 7f2151c18740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f212c00ecf0 con 0x5591055d7d20 2024-09-11T23:48:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.732+0000 7f214cdda640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d7d20 0x5591055de060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.732+0000 7f214dddc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.733+0000 7f214cdda640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d7d20 0x5591055de060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:44636/0 (socket says 172.21.15.133:44636) 2024-09-11T23:48:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.733+0000 7f214cdda640 1 -- 172.21.15.133:0/2839988379 learned_addr learned my addr 172.21.15.133:0/2839988379 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:10.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.733+0000 7f214d5db640 1 --2- 172.21.15.133:0/2839988379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 0x5591055ed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.734+0000 7f214d5db640 1 -- 172.21.15.133:0/2839988379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d7d20 msgr2=0x5591055de060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.734+0000 7f214d5db640 1 --2- 172.21.15.133:0/2839988379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d7d20 0x5591055de060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.734+0000 7f214d5db640 1 -- 172.21.15.133:0/2839988379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 msgr2=0x5591055de5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.734+0000 7f214d5db640 1 --2- 172.21.15.133:0/2839988379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055de5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.734+0000 7f214d5db640 1 -- 172.21.15.133:0/2839988379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591055d9cc0 con 0x5591055d67b0 2024-09-11T23:48:10.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.735+0000 7f214d5db640 1 --2- 172.21.15.133:0/2839988379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 0x5591055ed570 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2148068d20 tx=0x7f21480681f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0b5a2db66eac33 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:10.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.740+0000 7f2132ffd640 1 -- 172.21.15.133:0/2839988379 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214806cda0 con 0x5591055d67b0 2024-09-11T23:48:10.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.740+0000 7f2132ffd640 1 -- 172.21.15.133:0/2839988379 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2148078040 con 0x5591055d67b0 2024-09-11T23:48:10.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.740+0000 7f2132ffd640 1 -- 172.21.15.133:0/2839988379 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214807c9a0 con 0x5591055d67b0 2024-09-11T23:48:10.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.741+0000 7f2151c18740 1 -- 172.21.15.133:0/2839988379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 msgr2=0x5591055ed570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:10.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.742+0000 7f2151c18740 1 --2- 172.21.15.133:0/2839988379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 0x5591055ed570 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2148068d20 tx=0x7f21480681f0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.744+0000 7f2151c18740 1 -- 172.21.15.133:0/2839988379 shutdown_connections 2024-09-11T23:48:10.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.744+0000 7f2151c18740 1 --2- 172.21.15.133:0/2839988379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.744+0000 7f2151c18740 1 --2- 172.21.15.133:0/2839988379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d7d20 0x5591055de060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.744+0000 7f2151c18740 1 --2- 172.21.15.133:0/2839988379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 0x5591055ed570 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.744+0000 7f2151c18740 1 -- 172.21.15.133:0/2839988379 >> 172.21.15.133:0/2839988379 conn(0x55910559f310 msgr2=0x5591055e3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:10.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.748+0000 7f2151c18740 1 -- 172.21.15.133:0/2839988379 shutdown_connections 2024-09-11T23:48:10.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.749+0000 7f2151c18740 1 -- 172.21.15.133:0/2839988379 wait complete. 2024-09-11T23:48:10.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.749+0000 7f2151c18740 1 -- start start 2024-09-11T23:48:10.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.749+0000 7f2151c18740 1 Processor -- start 2024-09-11T23:48:10.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.750+0000 7f2151c18740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 0x5591055da8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.751+0000 7f2151c18740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d7d20 0x5591055dadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.751+0000 7f214d5db640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 0x5591055da8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.751+0000 7f214d5db640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 0x5591055da8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39648/0 (socket says 172.21.15.133:39648) 2024-09-11T23:48:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.751+0000 7f214d5db640 1 -- 172.21.15.133:0/165745517 learned_addr learned my addr 172.21.15.133:0/165745517 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.751+0000 7f2151c18740 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.752+0000 7f2151c18740 1 -- 172.21.15.133:0/165745517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f212c00fa90 con 0x5591055d67b0 2024-09-11T23:48:10.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.752+0000 7f2151c18740 1 -- 172.21.15.133:0/165745517 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f212c00f3c0 con 0x5591055e1550 2024-09-11T23:48:10.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.752+0000 7f214dddc640 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.752+0000 7f2151c18740 1 -- 172.21.15.133:0/165745517 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f212c00c6b0 con 0x5591055d7d20 2024-09-11T23:48:10.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.752+0000 7f2151c18740 10 client.0 _ll_register_callbacks cb 0x7f21380ba200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.753+0000 7f214d5db640 1 -- 172.21.15.133:0/165745517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d7d20 msgr2=0x5591055dadf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:48:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.753+0000 7f214d5db640 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d7d20 0x5591055dadf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.753+0000 7f214d5db640 1 -- 172.21.15.133:0/165745517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 msgr2=0x5591055e86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.753+0000 7f214d5db640 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055e86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.753+0000 7f214d5db640 1 -- 172.21.15.133:0/165745517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591055e8c20 con 0x5591055d67b0 2024-09-11T23:48:10.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.754+0000 7f214d5db640 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 0x5591055da8b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2148000b60 tx=0x7f214806cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f130c5492d112ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:10.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.755+0000 7f2130ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:10.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.763+0000 7f2130ff9640 1 -- 172.21.15.133:0/165745517 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214807b930 con 0x5591055d67b0 2024-09-11T23:48:10.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.763+0000 7f2130ff9640 1 -- 172.21.15.133:0/165745517 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7f214807bad0 con 0x5591055d67b0 2024-09-11T23:48:10.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.763+0000 7f2151c18740 1 -- 172.21.15.133:0/165745517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2148078040 con 0x5591055d67b0 2024-09-11T23:48:10.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.765+0000 7f2130ff9640 1 -- 172.21.15.133:0/165745517 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f214807bd10 con 0x5591055d67b0 2024-09-11T23:48:10.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.765+0000 7f2151c18740 10 client.18354 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:10.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.766+0000 7f2151c18740 1 -- 172.21.15.133:0/165745517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5591055e9e60 con 0x5591055d67b0 2024-09-11T23:48:10.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.771+0000 7f2130ff9640 1 -- 172.21.15.133:0/165745517 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f214807c8b0 con 0x5591055d67b0 2024-09-11T23:48:10.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.771+0000 7f2130ff9640 1 -- 172.21.15.133:0/165745517 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f214809f6c0 con 0x5591055d67b0 2024-09-11T23:48:10.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.771+0000 7f2151c18740 10 client.18354 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:10.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.771+0000 7f2151c18740 10 client.18354 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:48:10.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.771+0000 7f2151c18740 20 client.18354 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:10.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.771+0000 7f2151c18740 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55910565c090 0x55910565e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:10.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.771+0000 7f2151c18740 4 client.18354 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:48:10.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.771+0000 7f2151c18740 1 -- 172.21.15.133:0/165745517 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2148078040 con 0x55910565c090 2024-09-11T23:48:10.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.776+0000 7f214cdda640 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55910565c090 0x55910565e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:10.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.784+0000 7f214cdda640 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55910565c090 0x55910565e480 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ecaec1c9d7b2690 server_cookie=25dc9c2955aec8fc in_seq=0 out_seq=0 2024-09-11T23:48:10.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.784+0000 7f2130ff9640 10 client.18354 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:48:10.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.784+0000 7f2130ff9640 1 -- 172.21.15.133:0/165745517 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2148078040 con 0x55910565c090 2024-09-11T23:48:10.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.784+0000 7f2130ff9640 10 client.18354 handle_command_reply: tid=0 2024-09-11T23:48:10.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.784+0000 7f2151c18740 1 client.18354 shutdown 2024-09-11T23:48:10.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.787+0000 7f2151c18740 1 -- 172.21.15.133:0/165745517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 msgr2=0x5591055da8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:10.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.787+0000 7f2151c18740 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 0x5591055da8b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2148000b60 tx=0x7f214806cba0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.787+0000 7f2151c18740 1 -- 172.21.15.133:0/165745517 shutdown_connections 2024-09-11T23:48:10.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.787+0000 7f2151c18740 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x55910565c090 0x55910565e480 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.787+0000 7f2151c18740 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591055e1550 0x5591055e86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.787+0000 7f2151c18740 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591055d7d20 0x5591055dadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.787+0000 7f2151c18740 1 --2- 172.21.15.133:0/165745517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591055d67b0 0x5591055da8b0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.787+0000 7f2151c18740 1 -- 172.21.15.133:0/165745517 >> 172.21.15.133:0/165745517 conn(0x55910559f310 msgr2=0x5591055dc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:10.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.788+0000 7f2130ff9640 0 client.18354 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:48:10.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.788+0000 7f2151c18740 1 -- 172.21.15.133:0/165745517 shutdown_connections 2024-09-11T23:48:10.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.788+0000 7f2151c18740 1 -- 172.21.15.133:0/165745517 wait complete. 2024-09-11T23:48:10.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.788+0000 7f2151c18740 20 client.18354 trim_cache size 0 max 16384 2024-09-11T23:48:10.789 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:48:10.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.791+0000 7f214e5dd640 1 -- 172.21.15.133:0/2761348897 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f21340a2e40 msgr2=0x7f21340a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:10.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.791+0000 7f214e5dd640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f21340a2e40 0x7f21340a5300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2144005fd0 tx=0x7f2144005ec0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.791+0000 7f214e5dd640 1 -- 172.21.15.133:0/2761348897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2140095510 msgr2=0x7f21400aa220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:10.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.791+0000 7f214e5dd640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2140095510 0x7f21400aa220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2138034040 tx=0x7f2138005040 comp rx=0 tx=0).stop 2024-09-11T23:48:10.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.791+0000 7f214e5dd640 1 -- 172.21.15.133:0/2761348897 shutdown_connections 2024-09-11T23:48:10.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.791+0000 7f214e5dd640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f21340a2e40 0x7f21340a5300 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.791+0000 7f214e5dd640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2140099880 0x7f21400a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.791+0000 7f214e5dd640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2140095510 0x7f21400aa220 secure :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2138034040 tx=0x7f2138005040 comp rx=0 tx=0).stop 2024-09-11T23:48:10.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.791+0000 7f214e5dd640 1 --2- 172.21.15.133:0/2761348897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2140094b60 0x7f21400a9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:10.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.791+0000 7f214e5dd640 1 -- 172.21.15.133:0/2761348897 >> 172.21.15.133:0/2761348897 conn(0x7f214000a1b0 msgr2=0x7f2140092cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:10.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.791+0000 7f214e5dd640 1 -- 172.21.15.133:0/2761348897 shutdown_connections 2024-09-11T23:48:10.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:10.792+0000 7f214e5dd640 1 -- 172.21.15.133:0/2761348897 wait complete. 2024-09-11T23:48:11.182 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:10 smithi133 ceph-mon[31430]: pgmap v357: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 62 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 497 MiB/s wr, 1.86k op/s 2024-09-11T23:48:11.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:10 smithi138 ceph-mon[35142]: pgmap v357: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 62 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 497 MiB/s wr, 1.86k op/s 2024-09-11T23:48:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:10 smithi178 ceph-mon[35882]: pgmap v357: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 62 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 497 MiB/s wr, 1.86k op/s 2024-09-11T23:48:11.628 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:48:11.905 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:11.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:11 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:12.187 INFO:tasks.workunit.client.0.smithi133.stdout:stopping iogen 2024-09-11T23:48:12.187 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 'stopping iogen' 2024-09-11T23:48:12.187 INFO:tasks.workunit.client.0.smithi133.stderr:+ ./iogen -k 2024-09-11T23:48:12.191 INFO:tasks.workunit.client.0.smithi133.stdout:OK 2024-09-11T23:48:12.191 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo OK 2024-09-11T23:48:12.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:11 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:12.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:11 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:48:12.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.397+0000 7fcff2833640 1 -- 172.21.15.133:0/430998799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec102ef0 msgr2=0x7fcfec10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.397+0000 7fcff2833640 1 --2- 172.21.15.133:0/430998799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec102ef0 0x7fcfec10da40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009ca0 tx=0x7fcfe402a950 comp rx=0 tx=0).stop 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.397+0000 7fcff2833640 1 -- 172.21.15.133:0/430998799 shutdown_connections 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.397+0000 7fcff2833640 1 --2- 172.21.15.133:0/430998799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec102ef0 0x7fcfec10da40 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.397+0000 7fcff2833640 1 --2- 172.21.15.133:0/430998799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfec102570 0x7fcfec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.397+0000 7fcff2833640 1 --2- 172.21.15.133:0/430998799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfec100e20 0x7fcfec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.397+0000 7fcff2833640 1 -- 172.21.15.133:0/430998799 >> 172.21.15.133:0/430998799 conn(0x7fcfec0fc7d0 msgr2=0x7fcfec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.397+0000 7fcff2833640 1 -- 172.21.15.133:0/430998799 shutdown_connections 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.397+0000 7fcff2833640 1 -- 172.21.15.133:0/430998799 wait complete. 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcff2833640 1 Processor -- start 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcff2833640 1 -- start start 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcff2833640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfec100e20 0x7fcfec1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcff2833640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfec102570 0x7fcfec199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcff2833640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec102ef0 0x7fcfec199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcff2833640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec110f80 con 0x7fcfec102ef0 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcff2833640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec110e00 con 0x7fcfec100e20 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcff2833640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec111100 con 0x7fcfec102570 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcff0dd0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec102ef0 0x7fcfec199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcff0dd0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec102ef0 0x7fcfec199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57580/0 (socket says 172.21.15.133:57580) 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcfebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfec100e20 0x7fcfec1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcff0dd0640 1 -- 172.21.15.133:0/3497095613 learned_addr learned my addr 172.21.15.133:0/3497095613 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.398+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfec102570 0x7fcfec199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcff0dd0640 1 -- 172.21.15.133:0/3497095613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfec102570 msgr2=0x7fcfec199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcff0dd0640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfec102570 0x7fcfec199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcff0dd0640 1 -- 172.21.15.133:0/3497095613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfec100e20 msgr2=0x7fcfec1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:12.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcff0dd0640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfec100e20 0x7fcfec1993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcff0dd0640 1 -- 172.21.15.133:0/3497095613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec1a47d0 con 0x7fcfec102ef0 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcfebfff640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfec100e20 0x7fcfec1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfec102570 0x7fcfec199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcff0dd0640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec102ef0 0x7fcfec199e90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4004af0 tx=0x7fcfe402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcfe9ffb640 1 -- 172.21.15.133:0/3497095613 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4034040 con 0x7fcfec102ef0 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcff2833640 1 -- 172.21.15.133:0/3497095613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfec1a4a60 con 0x7fcfec102ef0 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcfe9ffb640 1 -- 172.21.15.133:0/3497095613 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe4030920 con 0x7fcfec102ef0 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcfe9ffb640 1 -- 172.21.15.133:0/3497095613 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4004630 con 0x7fcfec102ef0 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.399+0000 7fcff2833640 1 -- 172.21.15.133:0/3497095613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfec1a4f40 con 0x7fcfec102ef0 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.401+0000 7fcfe9ffb640 1 -- 172.21.15.133:0/3497095613 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcfe4037020 con 0x7fcfec102ef0 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcff4c0c740 1 Processor -- start 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcff4c0c740 1 -- start start 2024-09-11T23:48:12.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 0x55c484155090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c484155a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484160370 0x55c484162760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c4840bb2f0 con 0x55c484156b40 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcfe9ffb640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcfd80a2cf0 0x7fcfd80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c484062cf0 con 0x55c484160370 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c4840b72b0 con 0x55c4841555d0 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcfe3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 0x55c484155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcfe3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 0x55c484155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57588/0 (socket says 172.21.15.133:57588) 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcfe3fff640 1 -- 172.21.15.133:0/3122558134 learned_addr learned my addr 172.21.15.133:0/3122558134 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/3122558134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484160370 0x55c484162760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcfebfff640 1 --2- 172.21.15.133:0/3122558134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c484155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.402+0000 7fcff0dd0640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcfd80a2cf0 0x7fcfd80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.403+0000 7fcfe9ffb640 1 -- 172.21.15.133:0/3497095613 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7fcfe40f0090 con 0x7fcfec102ef0 2024-09-11T23:48:12.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.403+0000 7fcfe3fff640 1 -- 172.21.15.133:0/3122558134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 msgr2=0x55c484155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.403+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/3122558134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c484155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.403+0000 7fcfe3fff640 1 -- 172.21.15.133:0/3122558134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 msgr2=0x55c484155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.403+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/3122558134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 0x55c484155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.403+0000 7fcfe3fff640 1 -- 172.21.15.133:0/3122558134 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4840473a0 con 0x55c484160370 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.403+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/3122558134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484160370 0x55c484162760 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4013bb0 tx=0x7fcfd4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c36552f5573370 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.403+0000 7fcff0dd0640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcfd80a2cf0 0x7fcfd80a51b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40b8b30 tx=0x7fcfe40040d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.403+0000 7fcfe27fc640 1 -- 172.21.15.133:0/3122558134 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4013170 con 0x55c484160370 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.403+0000 7fcfe27fc640 1 -- 172.21.15.133:0/3122558134 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd4005500 con 0x55c484160370 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.403+0000 7fcfe27fc640 1 -- 172.21.15.133:0/3122558134 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4004b60 con 0x55c484160370 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.404+0000 7fcff4c0c740 1 -- 172.21.15.133:0/3122558134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484160370 msgr2=0x55c484162760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.404+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/3122558134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484160370 0x55c484162760 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4013bb0 tx=0x7fcfd4004730 comp rx=0 tx=0).stop 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.404+0000 7fcff4c0c740 1 -- 172.21.15.133:0/3122558134 shutdown_connections 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.404+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/3122558134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484160370 0x55c484162760 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.404+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/3122558134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c484155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.404+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/3122558134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 0x55c484155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.404+0000 7fcff4c0c740 1 -- 172.21.15.133:0/3122558134 >> 172.21.15.133:0/3122558134 conn(0x55c48411e310 msgr2=0x55c484151c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.404+0000 7fcff4c0c740 1 -- 172.21.15.133:0/3122558134 shutdown_connections 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.404+0000 7fcff4c0c740 1 -- 172.21.15.133:0/3122558134 wait complete. 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.404+0000 7fcff4c0c740 1 -- start start 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.404+0000 7fcff4c0c740 1 Processor -- start 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c48415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484156b40 0x55c48415d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c48415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c484163070 con 0x55c484160370 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c484076060 con 0x55c484156b40 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c4841631f0 con 0x55c4841555d0 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcff4c0c740 10 client.0 _ll_register_callbacks cb 0x55c483dc98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:12.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfe3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c48415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfe3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c48415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57604/0 (socket says 172.21.15.133:57604) 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfe3fff640 1 -- 172.21.15.133:0/864536317 learned_addr learned my addr 172.21.15.133:0/864536317 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c48415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfebfff640 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484156b40 0x55c48415d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfe3fff640 1 -- 172.21.15.133:0/864536317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 msgr2=0x55c48415fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c48415fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfe3fff640 1 -- 172.21.15.133:0/864536317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484156b40 msgr2=0x55c48415d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484156b40 0x55c48415d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfe3fff640 1 -- 172.21.15.133:0/864536317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c483e2c3c0 con 0x55c484160370 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c48415d5d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4001eb0 tx=0x7fcfd4004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200e9d87bed683d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:12.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/864536317 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4004920 con 0x55c484160370 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/864536317 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7fcfd401b040 con 0x55c484160370 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.405+0000 7fcff4c0c740 1 -- 172.21.15.133:0/864536317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd4005500 con 0x55c484160370 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/864536317 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcfd4036050 con 0x55c484160370 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcff4c0c740 10 client.18366 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcff4c0c740 1 -- 172.21.15.133:0/864536317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c48416eaf0 con 0x55c484160370 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/864536317 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd4005500 con 0x55c484160370 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/864536317 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcfd4022b40 con 0x55c484160370 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcff4c0c740 10 client.18366 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcff4c0c740 10 client.18366 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcff4c0c740 20 client.18366 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55c48416fe40 0x55c484172230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcff4c0c740 4 client.18366 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcff4c0c740 1 -- 172.21.15.133:0/864536317 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4840c8860 con 0x55c48416fe40 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.406+0000 7fcfebfff640 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55c48416fe40 0x55c484172230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.407+0000 7fcfebfff640 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55c48416fe40 0x55c484172230 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ed51c0276f293b server_cookie=4f4d0c5b599b053 in_seq=0 out_seq=0 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.407+0000 7fcfcb7fe640 10 client.18366 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.407+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/864536317 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4840c8860 con 0x55c48416fe40 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.407+0000 7fcfcb7fe640 10 client.18366 handle_command_reply: tid=0 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.407+0000 7fcff4c0c740 1 client.18366 shutdown 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.407+0000 7fcff4c0c740 1 -- 172.21.15.133:0/864536317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 msgr2=0x55c48415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.407+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c48415d5d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4001eb0 tx=0x7fcfd4004840 comp rx=0 tx=0).stop 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.408+0000 7fcff4c0c740 1 -- 172.21.15.133:0/864536317 shutdown_connections 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.408+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55c48416fe40 0x55c484172230 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.408+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c48415d5d0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.408+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484156b40 0x55c48415d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.408+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/864536317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c48415fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.408+0000 7fcff4c0c740 1 -- 172.21.15.133:0/864536317 >> 172.21.15.133:0/864536317 conn(0x55c48411e310 msgr2=0x55c484157960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.408+0000 7fcfcb7fe640 0 client.18366 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:48:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.408+0000 7fcff4c0c740 1 -- 172.21.15.133:0/864536317 shutdown_connections 2024-09-11T23:48:12.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.408+0000 7fcff4c0c740 1 -- 172.21.15.133:0/864536317 wait complete. 2024-09-11T23:48:12.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.408+0000 7fcff4c0c740 20 client.18366 trim_cache size 0 max 16384 2024-09-11T23:48:12.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcff4c0c740 1 Processor -- start 2024-09-11T23:48:12.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcff4c0c740 1 -- start start 2024-09-11T23:48:12.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c48416c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 0x55c48415c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484160370 0x55c484158120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1ad6e0 con 0x55c484156b40 2024-09-11T23:48:12.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1af220 con 0x55c484160370 2024-09-11T23:48:12.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1addb0 con 0x55c4841555d0 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcfebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 0x55c48415c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcfebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 0x55c48415c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57614/0 (socket says 172.21.15.133:57614) 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcfebfff640 1 -- 172.21.15.133:0/3961853374 learned_addr learned my addr 172.21.15.133:0/3961853374 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/3961853374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484160370 0x55c484158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/3961853374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c48416c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcfebfff640 1 -- 172.21.15.133:0/3961853374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 msgr2=0x55c48416c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcfebfff640 1 --2- 172.21.15.133:0/3961853374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c48416c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcfebfff640 1 -- 172.21.15.133:0/3961853374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484160370 msgr2=0x55c484158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcfebfff640 1 --2- 172.21.15.133:0/3961853374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484160370 0x55c484158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcfebfff640 1 -- 172.21.15.133:0/3961853374 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4841552d0 con 0x55c484156b40 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.421+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/3961853374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c48416c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcfebfff640 1 --2- 172.21.15.133:0/3961853374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 0x55c48415c000 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x55c48415e640 tx=0x7fcfdc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de04d32425742a47 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:12.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcfe1ffb640 1 -- 172.21.15.133:0/3961853374 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc004030 con 0x55c484156b40 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcfe1ffb640 1 -- 172.21.15.133:0/3961853374 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfdc00fdd0 con 0x55c484156b40 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcfe1ffb640 1 -- 172.21.15.133:0/3961853374 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc003b20 con 0x55c484156b40 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcff4c0c740 1 -- 172.21.15.133:0/3961853374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 msgr2=0x55c48415c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/3961853374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 0x55c48415c000 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x55c48415e640 tx=0x7fcfdc005f90 comp rx=0 tx=0).stop 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcff4c0c740 1 -- 172.21.15.133:0/3961853374 shutdown_connections 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/3961853374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c484160370 0x55c484158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/3961853374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484156b40 0x55c48415c000 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/3961853374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4841555d0 0x55c48416c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcff4c0c740 1 -- 172.21.15.133:0/3961853374 >> 172.21.15.133:0/3961853374 conn(0x55c48411e310 msgr2=0x55c48415b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.422+0000 7fcff4c0c740 1 -- 172.21.15.133:0/3961853374 shutdown_connections 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcff4c0c740 1 -- 172.21.15.133:0/3961853374 wait complete. 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcff4c0c740 1 -- start start 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcff4c0c740 1 Processor -- start 2024-09-11T23:48:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4841555d0 0x55c484168660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c484156b40 0x55c484168ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcff4c0c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c484159830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1aeb50 con 0x55c484160370 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1ae480 con 0x55c4841555d0 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcff4c0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1ab6e0 con 0x55c484156b40 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcff4c0c740 10 client.0 _ll_register_callbacks cb 0x7fcfd4009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcfe3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c484159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcfe3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c484159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57630/0 (socket says 172.21.15.133:57630) 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.423+0000 7fcfe3fff640 1 -- 172.21.15.133:0/179283990 learned_addr learned my addr 172.21.15.133:0/179283990 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfebfff640 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c484156b40 0x55c484168ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4841555d0 0x55c484168660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfe3fff640 1 -- 172.21.15.133:0/179283990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c484156b40 msgr2=0x55c484168ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:12.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c484156b40 0x55c484168ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfe3fff640 1 -- 172.21.15.133:0/179283990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4841555d0 msgr2=0x55c484168660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4841555d0 0x55c484168660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfe3fff640 1 -- 172.21.15.133:0/179283990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c48415a030 con 0x55c484160370 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfe3fff640 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c484159830 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4022f10 tx=0x7fcfd40047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65c904c749c9a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/179283990 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4017070 con 0x55c484160370 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/179283990 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8499+0+0 (secure 0 0 0) 0x7fcfd4013580 con 0x55c484160370 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcff4c0c740 1 -- 172.21.15.133:0/179283990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfdc00fdd0 con 0x55c484160370 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/179283990 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcfd40137a0 con 0x55c484160370 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcff4c0c740 10 client.18372 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.424+0000 7fcff4c0c740 1 -- 172.21.15.133:0/179283990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4841da510 con 0x55c484160370 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.425+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/179283990 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfdc00fdd0 con 0x55c484160370 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.425+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/179283990 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcfd4004130 con 0x55c484160370 2024-09-11T23:48:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.425+0000 7fcff4c0c740 10 client.18372 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.425+0000 7fcff4c0c740 10 client.18372 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.425+0000 7fcff4c0c740 20 client.18372 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.425+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55c4841dac90 0x55c4841dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.425+0000 7fcff4c0c740 4 client.18372 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.425+0000 7fcff4c0c740 1 -- 172.21.15.133:0/179283990 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4840c2d70 con 0x55c4841dac90 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.425+0000 7fcfebfff640 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55c4841dac90 0x55c4841dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.425+0000 7fcfcb7fe640 10 client.18372 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.425+0000 7fcfebfff640 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55c4841dac90 0x55c4841dd0b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d2c414ed438167a server_cookie=3112b91cc4240f05 in_seq=0 out_seq=0 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcfcb7fe640 1 -- 172.21.15.133:0/179283990 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4840c2d70 con 0x55c4841dac90 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcfcb7fe640 10 client.18372 handle_command_reply: tid=0 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 1 client.18372 shutdown 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 1 -- 172.21.15.133:0/179283990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 msgr2=0x55c484159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c484159830 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4022f10 tx=0x7fcfd40047a0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 1 -- 172.21.15.133:0/179283990 shutdown_connections 2024-09-11T23:48:12.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x55c4841dac90 0x55c4841dd0b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c484160370 0x55c484159830 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c484156b40 0x55c484168ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 1 --2- 172.21.15.133:0/179283990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4841555d0 0x55c484168660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcfcb7fe640 0 client.18372 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:48:12.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 1 -- 172.21.15.133:0/179283990 >> 172.21.15.133:0/179283990 conn(0x55c48411e310 msgr2=0x55c48415e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:12.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 1 -- 172.21.15.133:0/179283990 shutdown_connections 2024-09-11T23:48:12.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 1 -- 172.21.15.133:0/179283990 wait complete. 2024-09-11T23:48:12.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff4c0c740 20 client.18372 trim_cache size 0 max 16384 2024-09-11T23:48:12.433 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:48:12.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff2833640 1 -- 172.21.15.133:0/3497095613 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcfd80a2cf0 msgr2=0x7fcfd80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:12.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.426+0000 7fcff2833640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcfd80a2cf0 0x7fcfd80a51b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40b8b30 tx=0x7fcfe40040d0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.427+0000 7fcff2833640 1 -- 172.21.15.133:0/3497095613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec102ef0 msgr2=0x7fcfec199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:12.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.427+0000 7fcff2833640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec102ef0 0x7fcfec199e90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4004af0 tx=0x7fcfe402f3a0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.427+0000 7fcff2833640 1 -- 172.21.15.133:0/3497095613 shutdown_connections 2024-09-11T23:48:12.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.427+0000 7fcff2833640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcfd80a2cf0 0x7fcfd80a51b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.427+0000 7fcff2833640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec102ef0 0x7fcfec199e90 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.427+0000 7fcff2833640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfec102570 0x7fcfec199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.427+0000 7fcff2833640 1 --2- 172.21.15.133:0/3497095613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfec100e20 0x7fcfec1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:12.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.427+0000 7fcff2833640 1 -- 172.21.15.133:0/3497095613 >> 172.21.15.133:0/3497095613 conn(0x7fcfec0fc7d0 msgr2=0x7fcfec1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:12.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.427+0000 7fcff2833640 1 -- 172.21.15.133:0/3497095613 shutdown_connections 2024-09-11T23:48:12.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:12.427+0000 7fcff2833640 1 -- 172.21.15.133:0/3497095613 wait complete. 2024-09-11T23:48:13.072 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:12 smithi133 ceph-mon[31430]: pgmap v358: 97 pgs: 97 active+clean; 60 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 933 KiB/s rd, 405 MiB/s wr, 1.53k op/s 2024-09-11T23:48:13.072 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:12 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:13.072 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:12 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:13.072 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:12 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:48:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:12 smithi138 ceph-mon[35142]: pgmap v358: 97 pgs: 97 active+clean; 60 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 933 KiB/s rd, 405 MiB/s wr, 1.53k op/s 2024-09-11T23:48:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:12 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:12 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:13.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:12 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:48:13.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:48:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:12 smithi178 ceph-mon[35882]: pgmap v358: 97 pgs: 97 active+clean; 60 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 933 KiB/s rd, 405 MiB/s wr, 1.53k op/s 2024-09-11T23:48:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:12 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:48:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:12 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:48:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:12 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:48:13.476 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:14.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.099+0000 7fde98b8b640 1 -- 172.21.15.133:0/3498719407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde9410c180 msgr2=0x7fde9410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.099+0000 7fde98b8b640 1 --2- 172.21.15.133:0/3498719407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde9410c180 0x7fde9410e610 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009ca0 tx=0x7fde8c02a950 comp rx=0 tx=0).stop 2024-09-11T23:48:14.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.099+0000 7fde98b8b640 1 -- 172.21.15.133:0/3498719407 shutdown_connections 2024-09-11T23:48:14.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.099+0000 7fde98b8b640 1 --2- 172.21.15.133:0/3498719407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde9410c180 0x7fde9410e610 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.099+0000 7fde98b8b640 1 --2- 172.21.15.133:0/3498719407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde94103700 0x7fde9410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.099+0000 7fde98b8b640 1 --2- 172.21.15.133:0/3498719407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde94102de0 0x7fde941031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.099+0000 7fde98b8b640 1 -- 172.21.15.133:0/3498719407 >> 172.21.15.133:0/3498719407 conn(0x7fde940fc7a0 msgr2=0x7fde940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.100+0000 7fde98b8b640 1 -- 172.21.15.133:0/3498719407 shutdown_connections 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.100+0000 7fde98b8b640 1 -- 172.21.15.133:0/3498719407 wait complete. 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.100+0000 7fde98b8b640 1 Processor -- start 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.100+0000 7fde98b8b640 1 -- start start 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde98b8b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde94102de0 0x7fde94100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde98b8b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde94103700 0x7fde94100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde98b8b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9410c180 0x7fde941014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde98b8b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fde94111fb0 con 0x7fde94102de0 2024-09-11T23:48:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde98b8b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fde94111e30 con 0x7fde94103700 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde98b8b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde94112130 con 0x7fde9410c180 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde9259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde94102de0 0x7fde94100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde9259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde94102de0 0x7fde94100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57660/0 (socket says 172.21.15.133:57660) 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde9259c640 1 -- 172.21.15.133:0/612118625 learned_addr learned my addr 172.21.15.133:0/612118625 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde91d9b640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde94103700 0x7fde94100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde92d9d640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9410c180 0x7fde941014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde9259c640 1 -- 172.21.15.133:0/612118625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9410c180 msgr2=0x7fde941014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde9259c640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9410c180 0x7fde941014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde9259c640 1 -- 172.21.15.133:0/612118625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde94103700 msgr2=0x7fde94100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde9259c640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde94103700 0x7fde94100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde9259c640 1 -- 172.21.15.133:0/612118625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde941a69c0 con 0x7fde94102de0 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde91d9b640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde94103700 0x7fde94100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:48:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde9259c640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde94102de0 0x7fde94100a40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fde8400b4a0 tx=0x7fde8400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde837fe640 1 -- 172.21.15.133:0/612118625 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84014070 con 0x7fde94102de0 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde98b8b640 1 -- 172.21.15.133:0/612118625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde941a6cb0 con 0x7fde94102de0 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde837fe640 1 -- 172.21.15.133:0/612118625 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde84010af0 con 0x7fde94102de0 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.101+0000 7fde837fe640 1 -- 172.21.15.133:0/612118625 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84017d10 con 0x7fde94102de0 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.102+0000 7fde98b8b640 1 -- 172.21.15.133:0/612118625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde941a71a0 con 0x7fde94102de0 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.104+0000 7fde9af64740 1 Processor -- start 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde9af64740 1 -- start start 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde837fe640 1 -- 172.21.15.133:0/612118625 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fde84017070 con 0x7fde94102de0 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde9af64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d38ab40 0x55ab0d389090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde9af64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d3895d0 0x55ab0d389a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde9af64740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d396760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde9af64740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ab0d2ef2f0 con 0x55ab0d394370 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde9af64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ab0d296cf0 con 0x55ab0d3895d0 2024-09-11T23:48:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde9af64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ab0d2eb2b0 con 0x55ab0d38ab40 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde91d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d396760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde91d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d396760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57666/0 (socket says 172.21.15.133:57666) 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde91d9b640 1 -- 172.21.15.133:0/1515436206 learned_addr learned my addr 172.21.15.133:0/1515436206 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde9259c640 1 --2- 172.21.15.133:0/1515436206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d3895d0 0x55ab0d389a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.105+0000 7fde92d9d640 1 --2- 172.21.15.133:0/1515436206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d38ab40 0x55ab0d389090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde91d9b640 1 -- 172.21.15.133:0/1515436206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d38ab40 msgr2=0x55ab0d389090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde91d9b640 1 --2- 172.21.15.133:0/1515436206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d38ab40 0x55ab0d389090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde91d9b640 1 -- 172.21.15.133:0/1515436206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d3895d0 msgr2=0x55ab0d389a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde91d9b640 1 --2- 172.21.15.133:0/1515436206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d3895d0 0x55ab0d389a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde91d9b640 1 -- 172.21.15.133:0/1515436206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0d27b3a0 con 0x55ab0d394370 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde91d9b640 1 --2- 172.21.15.133:0/1515436206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d396760 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fde8800bcb0 tx=0x7fde8800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a93ca38ef35550b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde837fe640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fde7c0a2cf0 0x7fde7c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde80ff9640 1 -- 172.21.15.133:0/1515436206 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88011440 con 0x55ab0d394370 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde80ff9640 1 -- 172.21.15.133:0/1515436206 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde88011ac0 con 0x55ab0d394370 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde80ff9640 1 -- 172.21.15.133:0/1515436206 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8801bb10 con 0x55ab0d394370 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde92d9d640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fde7c0a2cf0 0x7fde7c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.106+0000 7fde837fe640 1 -- 172.21.15.133:0/612118625 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fde840ce720 con 0x7fde94102de0 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.107+0000 7fde9af64740 1 -- 172.21.15.133:0/1515436206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 msgr2=0x55ab0d396760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.107+0000 7fde92d9d640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fde7c0a2cf0 0x7fde7c0a51b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009430 tx=0x7fde8c009360 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.107+0000 7fde9af64740 1 --2- 172.21.15.133:0/1515436206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d396760 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fde8800bcb0 tx=0x7fde8800f730 comp rx=0 tx=0).stop 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.107+0000 7fde9af64740 1 -- 172.21.15.133:0/1515436206 shutdown_connections 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.107+0000 7fde9af64740 1 --2- 172.21.15.133:0/1515436206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d396760 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.107+0000 7fde9af64740 1 --2- 172.21.15.133:0/1515436206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d3895d0 0x55ab0d389a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.107+0000 7fde9af64740 1 --2- 172.21.15.133:0/1515436206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d38ab40 0x55ab0d389090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.107+0000 7fde9af64740 1 -- 172.21.15.133:0/1515436206 >> 172.21.15.133:0/1515436206 conn(0x55ab0d352310 msgr2=0x55ab0d385c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.107+0000 7fde9af64740 1 -- 172.21.15.133:0/1515436206 shutdown_connections 2024-09-11T23:48:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.107+0000 7fde9af64740 1 -- 172.21.15.133:0/1515436206 wait complete. 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.107+0000 7fde9af64740 1 -- start start 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde9af64740 1 Processor -- start 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde9af64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d3895d0 0x55ab0d393d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde9af64740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d38ab40 0x55ab0d390fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde9af64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d394370 0x55ab0d391570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde9af64740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ab0d397070 con 0x55ab0d38ab40 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde9af64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ab0d2aa060 con 0x55ab0d394370 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde9af64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ab0d3971f0 con 0x55ab0d3895d0 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde9af64740 10 client.0 _ll_register_callbacks cb 0x55ab0cffd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde9259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d3895d0 0x55ab0d393d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde91d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d38ab40 0x55ab0d390fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde9259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d3895d0 0x55ab0d393d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:47760/0 (socket says 172.21.15.133:47760) 2024-09-11T23:48:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde91d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d38ab40 0x55ab0d390fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57668/0 (socket says 172.21.15.133:57668) 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde9259c640 1 -- 172.21.15.133:0/2982833752 learned_addr learned my addr 172.21.15.133:0/2982833752 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde92d9d640 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d394370 0x55ab0d391570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde91d9b640 1 -- 172.21.15.133:0/2982833752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d3895d0 msgr2=0x55ab0d393d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde91d9b640 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d3895d0 0x55ab0d393d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde91d9b640 1 -- 172.21.15.133:0/2982833752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d394370 msgr2=0x55ab0d391570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde91d9b640 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d394370 0x55ab0d391570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde91d9b640 1 -- 172.21.15.133:0/2982833752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0d0603c0 con 0x55ab0d38ab40 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde91d9b640 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d38ab40 0x55ab0d390fd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fde8800fc30 tx=0x7fde8800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3467b2cf367a9789 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.108+0000 7fde79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde79ffb640 1 -- 172.21.15.133:0/2982833752 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8802a060 con 0x55ab0d38ab40 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde79ffb640 1 -- 172.21.15.133:0/2982833752 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fde88004250 con 0x55ab0d38ab40 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde9af64740 1 -- 172.21.15.133:0/2982833752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde88011ac0 con 0x55ab0d38ab40 2024-09-11T23:48:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde79ffb640 1 -- 172.21.15.133:0/2982833752 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde88004450 con 0x55ab0d38ab40 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde9af64740 10 client.18384 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde9af64740 1 -- 172.21.15.133:0/2982833752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab0d3a2ac0 con 0x55ab0d38ab40 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde79ffb640 1 -- 172.21.15.133:0/2982833752 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde88011ac0 con 0x55ab0d38ab40 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde79ffb640 1 -- 172.21.15.133:0/2982833752 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fde880035f0 con 0x55ab0d38ab40 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde9af64740 10 client.18384 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde9af64740 10 client.18384 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde9af64740 20 client.18384 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde9af64740 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55ab0d3a3e10 0x55ab0d3a6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde9af64740 4 client.18384 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde9af64740 1 -- 172.21.15.133:0/2982833752 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab0d2fc860 con 0x55ab0d3a3e10 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.109+0000 7fde9259c640 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55ab0d3a3e10 0x55ab0d3a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.110+0000 7fde9259c640 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55ab0d3a3e10 0x55ab0d3a6200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcc1cb203afa2e4c server_cookie=ed17e3cc427d024d in_seq=0 out_seq=0 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.110+0000 7fde79ffb640 10 client.18384 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:48:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.110+0000 7fde79ffb640 1 -- 172.21.15.133:0/2982833752 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab0d2fc860 con 0x55ab0d3a3e10 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.110+0000 7fde79ffb640 10 client.18384 handle_command_reply: tid=0 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 1 client.18384 shutdown 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 1 -- 172.21.15.133:0/2982833752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d38ab40 msgr2=0x55ab0d390fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d38ab40 0x55ab0d390fd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fde8800fc30 tx=0x7fde8800bfd0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 1 -- 172.21.15.133:0/2982833752 shutdown_connections 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55ab0d3a3e10 0x55ab0d3a6200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d394370 0x55ab0d391570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d38ab40 0x55ab0d390fd0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 1 --2- 172.21.15.133:0/2982833752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d3895d0 0x55ab0d393d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 1 -- 172.21.15.133:0/2982833752 >> 172.21.15.133:0/2982833752 conn(0x55ab0d352310 msgr2=0x55ab0d38b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde79ffb640 0 client.18384 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 1 -- 172.21.15.133:0/2982833752 shutdown_connections 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 1 -- 172.21.15.133:0/2982833752 wait complete. 2024-09-11T23:48:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.111+0000 7fde9af64740 20 client.18384 trim_cache size 0 max 16384 2024-09-11T23:48:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9af64740 1 Processor -- start 2024-09-11T23:48:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9af64740 1 -- start start 2024-09-11T23:48:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9af64740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d3895d0 0x55ab0d3a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9af64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 0x55ab0d38c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9af64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d394370 0x55ab0d38c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9af64740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fde941afaa0 con 0x55ab0d3895d0 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9af64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fde941b15e0 con 0x55ab0d38ab40 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9af64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde941b0170 con 0x55ab0d394370 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d3895d0 0x55ab0d3a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d3895d0 0x55ab0d3a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57684/0 (socket says 172.21.15.133:57684) 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9259c640 1 -- 172.21.15.133:0/3115306411 learned_addr learned my addr 172.21.15.133:0/3115306411 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde91d9b640 1 --2- 172.21.15.133:0/3115306411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 0x55ab0d38c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde92d9d640 1 --2- 172.21.15.133:0/3115306411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d394370 0x55ab0d38c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9259c640 1 -- 172.21.15.133:0/3115306411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d394370 msgr2=0x55ab0d38c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9259c640 1 --2- 172.21.15.133:0/3115306411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d394370 0x55ab0d38c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9259c640 1 -- 172.21.15.133:0/3115306411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 msgr2=0x55ab0d38c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9259c640 1 --2- 172.21.15.133:0/3115306411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 0x55ab0d38c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde9259c640 1 -- 172.21.15.133:0/3115306411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0d385ed0 con 0x55ab0d3895d0 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.124+0000 7fde91d9b640 1 --2- 172.21.15.133:0/3115306411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 0x55ab0d38c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.125+0000 7fde9259c640 1 --2- 172.21.15.133:0/3115306411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d3895d0 0x55ab0d3a0310 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fde84017fd0 tx=0x7fde840d4550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a0ff3651251990 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.125+0000 7fde7bfff640 1 -- 172.21.15.133:0/3115306411 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8401e040 con 0x55ab0d3895d0 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.125+0000 7fde7bfff640 1 -- 172.21.15.133:0/3115306411 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8409b490 con 0x55ab0d3895d0 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.125+0000 7fde7bfff640 1 -- 172.21.15.133:0/3115306411 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84099160 con 0x55ab0d3895d0 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 -- 172.21.15.133:0/3115306411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d3895d0 msgr2=0x55ab0d3a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 --2- 172.21.15.133:0/3115306411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d3895d0 0x55ab0d3a0310 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fde84017fd0 tx=0x7fde840d4550 comp rx=0 tx=0).stop 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 -- 172.21.15.133:0/3115306411 shutdown_connections 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 --2- 172.21.15.133:0/3115306411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d394370 0x55ab0d38c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 --2- 172.21.15.133:0/3115306411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 0x55ab0d38c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 --2- 172.21.15.133:0/3115306411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d3895d0 0x55ab0d3a0310 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 -- 172.21.15.133:0/3115306411 >> 172.21.15.133:0/3115306411 conn(0x55ab0d352310 msgr2=0x55ab0d393050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 -- 172.21.15.133:0/3115306411 shutdown_connections 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 -- 172.21.15.133:0/3115306411 wait complete. 2024-09-11T23:48:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 -- start start 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 Processor -- start 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d3895d0 0x55ab0d398470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 0x55ab0d3989b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d398ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fde941b0f10 con 0x55ab0d394370 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fde941b0840 con 0x55ab0d38ab40 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.126+0000 7fde9af64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde941adaa0 con 0x55ab0d3895d0 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde9af64740 10 client.0 _ll_register_callbacks cb 0x7fde88015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde92d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d398ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde92d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d398ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57700/0 (socket says 172.21.15.133:57700) 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde92d9d640 1 -- 172.21.15.133:0/184313739 learned_addr learned my addr 172.21.15.133:0/184313739 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde91d9b640 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 0x55ab0d3989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde9259c640 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d3895d0 0x55ab0d398470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde92d9d640 1 -- 172.21.15.133:0/184313739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d3895d0 msgr2=0x55ab0d398470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde92d9d640 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d3895d0 0x55ab0d398470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde92d9d640 1 -- 172.21.15.133:0/184313739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 msgr2=0x55ab0d3989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde92d9d640 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 0x55ab0d3989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde92d9d640 1 -- 172.21.15.133:0/184313739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0d3918e0 con 0x55ab0d394370 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde91d9b640 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 0x55ab0d3989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde92d9d640 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d398ef0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fde8c035000 tx=0x7fde8c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e84a0923081c2ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde79ffb640 1 -- 172.21.15.133:0/184313739 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c034040 con 0x55ab0d394370 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde79ffb640 1 -- 172.21.15.133:0/184313739 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fde8c041030 con 0x55ab0d394370 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde9af64740 1 -- 172.21.15.133:0/184313739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde8409b490 con 0x55ab0d394370 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.127+0000 7fde79ffb640 1 -- 172.21.15.133:0/184313739 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde8c02f760 con 0x55ab0d394370 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.128+0000 7fde9af64740 10 client.18390 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.128+0000 7fde9af64740 1 -- 172.21.15.133:0/184313739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab0d40eb00 con 0x55ab0d394370 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.128+0000 7fde79ffb640 1 -- 172.21.15.133:0/184313739 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8409b490 con 0x55ab0d394370 2024-09-11T23:48:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.128+0000 7fde79ffb640 1 -- 172.21.15.133:0/184313739 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fde8c05c8f0 con 0x55ab0d394370 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.128+0000 7fde9af64740 10 client.18390 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.128+0000 7fde9af64740 10 client.18390 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.128+0000 7fde9af64740 20 client.18390 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.128+0000 7fde9af64740 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55ab0d40f280 0x55ab0d4116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.128+0000 7fde9af64740 4 client.18390 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.128+0000 7fde9af64740 1 -- 172.21.15.133:0/184313739 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab0d2f6d70 con 0x55ab0d40f280 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.128+0000 7fde9259c640 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55ab0d40f280 0x55ab0d4116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde79ffb640 10 client.18390 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9259c640 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55ab0d40f280 0x55ab0d4116a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fc88eaa10fb4183 server_cookie=243981c4e607b8c1 in_seq=0 out_seq=0 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde79ffb640 1 -- 172.21.15.133:0/184313739 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab0d2f6d70 con 0x55ab0d40f280 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde79ffb640 10 client.18390 handle_command_reply: tid=0 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 1 client.18390 shutdown 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 1 -- 172.21.15.133:0/184313739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 msgr2=0x55ab0d398ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d398ef0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fde8c035000 tx=0x7fde8c02a950 comp rx=0 tx=0).stop 2024-09-11T23:48:14.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 1 -- 172.21.15.133:0/184313739 shutdown_connections 2024-09-11T23:48:14.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x55ab0d40f280 0x55ab0d4116a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab0d394370 0x55ab0d398ef0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0d38ab40 0x55ab0d3989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 1 --2- 172.21.15.133:0/184313739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab0d3895d0 0x55ab0d398470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 1 -- 172.21.15.133:0/184313739 >> 172.21.15.133:0/184313739 conn(0x55ab0d352310 msgr2=0x55ab0d38f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde79ffb640 0 client.18390 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:48:14.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 1 -- 172.21.15.133:0/184313739 shutdown_connections 2024-09-11T23:48:14.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 1 -- 172.21.15.133:0/184313739 wait complete. 2024-09-11T23:48:14.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.129+0000 7fde9af64740 20 client.18390 trim_cache size 0 max 16384 2024-09-11T23:48:14.135 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:48:14.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 -- 172.21.15.133:0/612118625 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fde7c0a2cf0 msgr2=0x7fde7c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fde7c0a2cf0 0x7fde7c0a51b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009430 tx=0x7fde8c009360 comp rx=0 tx=0).stop 2024-09-11T23:48:14.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 -- 172.21.15.133:0/612118625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde94102de0 msgr2=0x7fde94100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde94102de0 0x7fde94100a40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fde8400b4a0 tx=0x7fde8400b960 comp rx=0 tx=0).stop 2024-09-11T23:48:14.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 -- 172.21.15.133:0/612118625 shutdown_connections 2024-09-11T23:48:14.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fde7c0a2cf0 0x7fde7c0a51b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9410c180 0x7fde941014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde94103700 0x7fde94100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 --2- 172.21.15.133:0/612118625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde94102de0 0x7fde94100a40 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 -- 172.21.15.133:0/612118625 >> 172.21.15.133:0/612118625 conn(0x7fde940fc7a0 msgr2=0x7fde940fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 -- 172.21.15.133:0/612118625 shutdown_connections 2024-09-11T23:48:14.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:14.130+0000 7fde98b8b640 1 -- 172.21.15.133:0/612118625 wait complete. 2024-09-11T23:48:14.206 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T23:48:14.207 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T23:48:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.239+0000 7f6c494ba640 1 -- 172.21.15.133:0/4226809774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c44101370 msgr2=0x7f6c4410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.239+0000 7f6c494ba640 1 --2- 172.21.15.133:0/4226809774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c44101370 0x7f6c4410d8a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009cd0 tx=0x7f6c3c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.239+0000 7f6c494ba640 1 -- 172.21.15.133:0/4226809774 shutdown_connections 2024-09-11T23:48:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.239+0000 7f6c494ba640 1 --2- 172.21.15.133:0/4226809774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c44101370 0x7f6c4410d8a0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.239+0000 7f6c494ba640 1 --2- 172.21.15.133:0/4226809774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c441009d0 0x7f6c44100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.239+0000 7f6c494ba640 1 --2- 172.21.15.133:0/4226809774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c441069e0 0x7f6c44106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.239+0000 7f6c494ba640 1 -- 172.21.15.133:0/4226809774 >> 172.21.15.133:0/4226809774 conn(0x7f6c440fc6b0 msgr2=0x7f6c440fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.239+0000 7f6c494ba640 1 -- 172.21.15.133:0/4226809774 shutdown_connections 2024-09-11T23:48:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.239+0000 7f6c494ba640 1 -- 172.21.15.133:0/4226809774 wait complete. 2024-09-11T23:48:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.239+0000 7f6c494ba640 1 Processor -- start 2024-09-11T23:48:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.239+0000 7f6c494ba640 1 -- start start 2024-09-11T23:48:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c494ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c441009d0 0x7f6c44199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c494ba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c44101370 0x7f6c441996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c494ba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c441069e0 0x7f6c441a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c494ba640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44110e20 con 0x7f6c441009d0 2024-09-11T23:48:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c494ba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44110ca0 con 0x7f6c441069e0 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c494ba640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44110fa0 con 0x7f6c44101370 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c42ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c441009d0 0x7f6c44199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c42ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c441009d0 0x7f6c44199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57708/0 (socket says 172.21.15.133:57708) 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c42ffd640 1 -- 172.21.15.133:0/205177047 learned_addr learned my addr 172.21.15.133:0/205177047 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c427fc640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c44101370 0x7f6c441996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c437fe640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c441069e0 0x7f6c441a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c42ffd640 1 -- 172.21.15.133:0/205177047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c44101370 msgr2=0x7f6c441996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c44101370 0x7f6c441996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c42ffd640 1 -- 172.21.15.133:0/205177047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c441069e0 msgr2=0x7f6c441a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c441069e0 0x7f6c441a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c42ffd640 1 -- 172.21.15.133:0/205177047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c441a4750 con 0x7f6c441009d0 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c437fe640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c441069e0 0x7f6c441a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:48:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c441009d0 0x7f6c44199190 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400d950 tx=0x7f6c3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c2bfff640 1 -- 172.21.15.133:0/205177047 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34017070 con 0x7f6c441009d0 2024-09-11T23:48:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c2bfff640 1 -- 172.21.15.133:0/205177047 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c34009c30 con 0x7f6c441009d0 2024-09-11T23:48:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c494ba640 1 -- 172.21.15.133:0/205177047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c441a4920 con 0x7f6c441009d0 2024-09-11T23:48:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.240+0000 7f6c2bfff640 1 -- 172.21.15.133:0/205177047 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400f6b0 con 0x7f6c441009d0 2024-09-11T23:48:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.241+0000 7f6c494ba640 1 -- 172.21.15.133:0/205177047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c441a4e40 con 0x7f6c441009d0 2024-09-11T23:48:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c2bfff640 1 -- 172.21.15.133:0/205177047 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6c340109d0 con 0x7f6c441009d0 2024-09-11T23:48:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c4c25c740 1 Processor -- start 2024-09-11T23:48:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c4c25c740 1 -- start start 2024-09-11T23:48:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 0x56033f033dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f034310 0x56033f042b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f0454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56033ef982f0 con 0x56033f0430c0 2024-09-11T23:48:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56033ef40270 con 0x56033f0339f0 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56033ef942b0 con 0x56033f034310 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c427fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f0454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c437fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 0x56033f033dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c427fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f0454b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57724/0 (socket says 172.21.15.133:57724) 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c427fc640 1 -- 172.21.15.133:0/2816450679 learned_addr learned my addr 172.21.15.133:0/2816450679 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.244+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/2816450679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f034310 0x56033f042b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c2bfff640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c2c0a2da0 0x7f6c2c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c437fe640 1 -- 172.21.15.133:0/2816450679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f034310 msgr2=0x56033f042b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c437fe640 1 --2- 172.21.15.133:0/2816450679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f034310 0x56033f042b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c437fe640 1 -- 172.21.15.133:0/2816450679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 msgr2=0x56033f0454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c437fe640 1 --2- 172.21.15.133:0/2816450679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f0454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c437fe640 1 -- 172.21.15.133:0/2816450679 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56033ef3f340 con 0x56033f0339f0 2024-09-11T23:48:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c2bfff640 1 -- 172.21.15.133:0/205177047 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f6c340cef90 con 0x7f6c441009d0 2024-09-11T23:48:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c437fe640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c2c0a2da0 0x7f6c2c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c437fe640 1 --2- 172.21.15.133:0/2816450679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 0x56033f033dd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009ca0 tx=0x7f6c3c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de1f1c80209fe96 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c297fa640 1 -- 172.21.15.133:0/2816450679 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c04c070 con 0x56033f0339f0 2024-09-11T23:48:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c297fa640 1 -- 172.21.15.133:0/2816450679 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3c036040 con 0x56033f0339f0 2024-09-11T23:48:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.245+0000 7f6c297fa640 1 -- 172.21.15.133:0/2816450679 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0309e0 con 0x56033f0339f0 2024-09-11T23:48:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c4c25c740 1 -- 172.21.15.133:0/2816450679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 msgr2=0x56033f033dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c437fe640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c2c0a2da0 0x7f6c2c0a5260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c045000 tx=0x7f6c3c04a000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/2816450679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 0x56033f033dd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009ca0 tx=0x7f6c3c009a20 comp rx=0 tx=0).stop 2024-09-11T23:48:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c4c25c740 1 -- 172.21.15.133:0/2816450679 shutdown_connections 2024-09-11T23:48:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/2816450679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f0454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/2816450679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f034310 0x56033f042b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/2816450679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 0x56033f033dd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c4c25c740 1 -- 172.21.15.133:0/2816450679 >> 172.21.15.133:0/2816450679 conn(0x56033f030060 msgr2=0x56033f030440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c4c25c740 1 -- 172.21.15.133:0/2816450679 shutdown_connections 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c4c25c740 1 -- 172.21.15.133:0/2816450679 wait complete. 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c4c25c740 1 -- start start 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.246+0000 7f6c4c25c740 1 Processor -- start 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0339f0 0x56033f040050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f034310 0x56033f040590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f0430c0 0x56033f040ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56033f045d20 con 0x56033f0339f0 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56033ef21290 con 0x56033f034310 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56033f045ea0 con 0x56033f0430c0 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c4c25c740 10 client.0 _ll_register_callbacks cb 0x56033ef52140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c42ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0339f0 0x56033f040050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c42ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0339f0 0x56033f040050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57730/0 (socket says 172.21.15.133:57730) 2024-09-11T23:48:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c42ffd640 1 -- 172.21.15.133:0/3927935597 learned_addr learned my addr 172.21.15.133:0/3927935597 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c427fc640 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f034310 0x56033f040590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c437fe640 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f0430c0 0x56033f040ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c42ffd640 1 -- 172.21.15.133:0/3927935597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f0430c0 msgr2=0x56033f040ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f0430c0 0x56033f040ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c42ffd640 1 -- 172.21.15.133:0/3927935597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f034310 msgr2=0x56033f040590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f034310 0x56033f040590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c42ffd640 1 -- 172.21.15.133:0/3927935597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56033ecab210 con 0x56033f0339f0 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0339f0 0x56033f040050 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009610 tx=0x7f6c34005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471af1fb579eb316 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3927935597 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34020070 con 0x56033f0339f0 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3927935597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3c036040 con 0x56033f0339f0 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.247+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3927935597 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f6c340d10d0 con 0x56033f0339f0 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3927935597 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6c3409ae50 con 0x56033f0339f0 2024-09-11T23:48:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c4c25c740 10 client.18402 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3927935597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56033ed77620 con 0x56033f0339f0 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3927935597 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3c036040 con 0x56033f0339f0 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3927935597 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6c340b41f0 con 0x56033f0339f0 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c4c25c740 10 client.18402 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c4c25c740 10 client.18402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c4c25c740 20 client.18402 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56033f053180 0x56033f055570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c4c25c740 4 client.18402 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3927935597 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56033efa5a20 con 0x56033f053180 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.248+0000 7f6c427fc640 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56033f053180 0x56033f055570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.249+0000 7f6c427fc640 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56033f053180 0x56033f055570 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6751a179a33b8bd8 server_cookie=bfaa549210863ba4 in_seq=0 out_seq=0 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.249+0000 7f6c26ff5640 10 client.18402 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3927935597 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56033efa5a20 con 0x56033f053180 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c26ff5640 10 client.18402 handle_command_reply: tid=0 2024-09-11T23:48:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c4c25c740 1 client.18402 shutdown 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3927935597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0339f0 msgr2=0x56033f040050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0339f0 0x56033f040050 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009610 tx=0x7f6c34005f90 comp rx=0 tx=0).stop 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3927935597 shutdown_connections 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56033f053180 0x56033f055570 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f0430c0 0x56033f040ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f034310 0x56033f040590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3927935597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0339f0 0x56033f040050 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c26ff5640 0 client.18402 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.250+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3927935597 >> 172.21.15.133:0/3927935597 conn(0x56033f030060 msgr2=0x56033f035dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.251+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3927935597 shutdown_connections 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.251+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3927935597 wait complete. 2024-09-11T23:48:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.251+0000 7f6c4c25c740 20 client.18402 trim_cache size 0 max 16384 2024-09-11T23:48:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.265+0000 7f6c4c25c740 1 Processor -- start 2024-09-11T23:48:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.265+0000 7f6c4c25c740 1 -- start start 2024-09-11T23:48:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.265+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 0x56033f03ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.265+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f034310 0x56033f05ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.265+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f05b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.265+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ab570 con 0x56033f0430c0 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.265+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ad0b0 con 0x56033f0339f0 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.265+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441abc40 con 0x56033f034310 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c437fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f05b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c437fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f05b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57740/0 (socket says 172.21.15.133:57740) 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c437fe640 1 -- 172.21.15.133:0/2192294441 learned_addr learned my addr 172.21.15.133:0/2192294441 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c427fc640 1 --2- 172.21.15.133:0/2192294441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f034310 0x56033f05ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/2192294441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 0x56033f03ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c437fe640 1 -- 172.21.15.133:0/2192294441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f034310 msgr2=0x56033f05ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c437fe640 1 --2- 172.21.15.133:0/2192294441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f034310 0x56033f05ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c437fe640 1 -- 172.21.15.133:0/2192294441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 msgr2=0x56033f03ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c437fe640 1 --2- 172.21.15.133:0/2192294441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 0x56033f03ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c437fe640 1 -- 172.21.15.133:0/2192294441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56033f034140 con 0x56033f0430c0 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/2192294441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 0x56033f03ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c437fe640 1 --2- 172.21.15.133:0/2192294441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f05b3c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c006010 tx=0x7f6c3c04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0177442c7650cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c28ff9640 1 -- 172.21.15.133:0/2192294441 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c05b070 con 0x56033f0430c0 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c28ff9640 1 -- 172.21.15.133:0/2192294441 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3c036040 con 0x56033f0430c0 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.266+0000 7f6c28ff9640 1 -- 172.21.15.133:0/2192294441 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c04c040 con 0x56033f0430c0 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 -- 172.21.15.133:0/2192294441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 msgr2=0x56033f05b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/2192294441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f05b3c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c006010 tx=0x7f6c3c04f000 comp rx=0 tx=0).stop 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 -- 172.21.15.133:0/2192294441 shutdown_connections 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/2192294441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f05b3c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/2192294441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f034310 0x56033f05ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/2192294441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f0339f0 0x56033f03ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 -- 172.21.15.133:0/2192294441 >> 172.21.15.133:0/2192294441 conn(0x56033f030060 msgr2=0x56033f035dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 -- 172.21.15.133:0/2192294441 shutdown_connections 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 -- 172.21.15.133:0/2192294441 wait complete. 2024-09-11T23:48:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 -- start start 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 Processor -- start 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.267+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f0339f0 0x56033f04b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f034310 0x56033f04b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c4c25c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f0466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ac9e0 con 0x56033f0430c0 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441ac310 con 0x56033f034310 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c4c25c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441a9570 con 0x56033f0339f0 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c4c25c740 10 client.0 _ll_register_callbacks cb 0x7f6c3400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c437fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f0466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c437fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f0466b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57742/0 (socket says 172.21.15.133:57742) 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c437fe640 1 -- 172.21.15.133:0/3993698429 learned_addr learned my addr 172.21.15.133:0/3993698429 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f0339f0 0x56033f04b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c427fc640 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f034310 0x56033f04b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c437fe640 1 -- 172.21.15.133:0/3993698429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f0339f0 msgr2=0x56033f04b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c437fe640 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f0339f0 0x56033f04b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c437fe640 1 -- 172.21.15.133:0/3993698429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f034310 msgr2=0x56033f04b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c437fe640 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f034310 0x56033f04b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c437fe640 1 -- 172.21.15.133:0/3993698429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56033f046ee0 con 0x56033f0430c0 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c427fc640 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f034310 0x56033f04b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c437fe640 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f0466b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c02ae00 tx=0x7f6c3c030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b34c067bbe13ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3993698429 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c05b070 con 0x56033f0430c0 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3993698429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3c036040 con 0x56033f0430c0 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3993698429 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f6c3c0318c0 con 0x56033f0430c0 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3993698429 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6c3c04c070 con 0x56033f0430c0 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c4c25c740 10 client.18408 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.268+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3993698429 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56033f0bd890 con 0x56033f0430c0 2024-09-11T23:48:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.269+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3993698429 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3c036040 con 0x56033f0430c0 2024-09-11T23:48:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.269+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3993698429 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6c3c030070 con 0x56033f0430c0 2024-09-11T23:48:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.269+0000 7f6c4c25c740 10 client.18408 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.269+0000 7f6c4c25c740 10 client.18408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.269+0000 7f6c4c25c740 20 client.18408 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.269+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56033f0bdac0 0x56033f0bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.269+0000 7f6c4c25c740 4 client.18408 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:48:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.269+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3993698429 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56033ef9fd70 con 0x56033f0bdac0 2024-09-11T23:48:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.269+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56033f0bdac0 0x56033f0bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.269+0000 7f6c26ff5640 10 client.18408 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.269+0000 7f6c42ffd640 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56033f0bdac0 0x56033f0bfeb0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=269e33629e06ece2 server_cookie=e7e1f8d329a9a5fb in_seq=0 out_seq=0 2024-09-11T23:48:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.423+0000 7f6c26ff5640 1 -- 172.21.15.133:0/3993698429 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56033ef9fd70 con 0x56033f0bdac0 2024-09-11T23:48:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.423+0000 7f6c26ff5640 10 client.18408 handle_command_reply: tid=0 2024-09-11T23:48:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.423+0000 7f6c4c25c740 1 client.18408 shutdown 2024-09-11T23:48:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.423+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3993698429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 msgr2=0x56033f0466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.423+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f0466b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c02ae00 tx=0x7f6c3c030fc0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3993698429 shutdown_connections 2024-09-11T23:48:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56033f0bdac0 0x56033f0bfeb0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56033f0430c0 0x56033f0466b0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c26ff5640 0 client.18408 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56033f034310 0x56033f04b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c4c25c740 1 --2- 172.21.15.133:0/3993698429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56033f0339f0 0x56033f04b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3993698429 >> 172.21.15.133:0/3993698429 conn(0x56033f030060 msgr2=0x56033f040010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3993698429 shutdown_connections 2024-09-11T23:48:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c4c25c740 1 -- 172.21.15.133:0/3993698429 wait complete. 2024-09-11T23:48:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c4c25c740 20 client.18408 trim_cache size 0 max 16384 2024-09-11T23:48:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2024-09-11T23:48:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2024-09-11T23:48:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2024-09-11T23:48:14.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2024-09-11T23:48:14.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c494ba640 1 -- 172.21.15.133:0/205177047 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c2c0a2da0 msgr2=0x7f6c2c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c494ba640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c2c0a2da0 0x7f6c2c0a5260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c045000 tx=0x7f6c3c04a000 comp rx=0 tx=0).stop 2024-09-11T23:48:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c494ba640 1 -- 172.21.15.133:0/205177047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c441009d0 msgr2=0x7f6c44199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c494ba640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c441009d0 0x7f6c44199190 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400d950 tx=0x7f6c3400de10 comp rx=0 tx=0).stop 2024-09-11T23:48:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c494ba640 1 -- 172.21.15.133:0/205177047 shutdown_connections 2024-09-11T23:48:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c494ba640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6c2c0a2da0 0x7f6c2c0a5260 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c494ba640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c441069e0 0x7f6c441a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c494ba640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c44101370 0x7f6c441996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c494ba640 1 --2- 172.21.15.133:0/205177047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c441009d0 0x7f6c44199190 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.424+0000 7f6c494ba640 1 -- 172.21.15.133:0/205177047 >> 172.21.15.133:0/205177047 conn(0x7f6c440fc6b0 msgr2=0x7f6c440fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.425+0000 7f6c494ba640 1 -- 172.21.15.133:0/205177047 shutdown_connections 2024-09-11T23:48:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2024-09-11T23:48:14.425+0000 7f6c494ba640 1 -- 172.21.15.133:0/205177047 wait complete. 2024-09-11T23:48:14.437 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-09-11T23:48:14.437 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T23:48:14.802 DEBUG:teuthology.parallel:result is None 2024-09-11T23:48:14.802 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T23:48:14.999 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T23:48:14.999 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-11T23:48:15.010 INFO:tasks.quiescer:joining Quiescers 2024-09-11T23:48:15.010 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-11T23:48:15.010 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:48:15.076 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:14 smithi133 ceph-mon[31430]: pgmap v359: 97 pgs: 97 active+clean; 60 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 59 KiB/s rd, 229 MiB/s wr, 1.36k op/s 2024-09-11T23:48:15.076 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:14 smithi133 ceph-mon[31430]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T23:48:15.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:14 smithi138 ceph-mon[35142]: pgmap v359: 97 pgs: 97 active+clean; 60 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 59 KiB/s rd, 229 MiB/s wr, 1.36k op/s 2024-09-11T23:48:15.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:14 smithi138 ceph-mon[35142]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T23:48:15.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:14 smithi178 ceph-mon[35882]: pgmap v359: 97 pgs: 97 active+clean; 60 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 59 KiB/s rd, 229 MiB/s wr, 1.36k op/s 2024-09-11T23:48:15.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:14 smithi178 ceph-mon[35882]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T23:48:15.381 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:48:15.867 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.589+0000 7f7dd120e640 1 -- 172.21.15.133:0/3423035877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dcc112a30 msgr2=0x7f7dcc114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.589+0000 7f7dd120e640 1 --2- 172.21.15.133:0/3423035877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dcc112a30 0x7f7dcc114e20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc01a910 tx=0x7f7dbc03b470 comp rx=0 tx=0).stop 2024-09-11T23:48:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.590+0000 7f7dd120e640 1 -- 172.21.15.133:0/3423035877 shutdown_connections 2024-09-11T23:48:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.590+0000 7f7dd120e640 1 --2- 172.21.15.133:0/3423035877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dcc112a30 0x7f7dcc114e20 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.590+0000 7f7dd120e640 1 --2- 172.21.15.133:0/3423035877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dcc077040 0x7f7dcc1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.590+0000 7f7dd120e640 1 --2- 172.21.15.133:0/3423035877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dcc076720 0x7f7dcc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.590+0000 7f7dd120e640 1 -- 172.21.15.133:0/3423035877 >> 172.21.15.133:0/3423035877 conn(0x7f7dcc0ffa70 msgr2=0x7f7dcc101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.595+0000 7f7dd120e640 1 -- 172.21.15.133:0/3423035877 shutdown_connections 2024-09-11T23:48:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.595+0000 7f7dd120e640 1 -- 172.21.15.133:0/3423035877 wait complete. 2024-09-11T23:48:16.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.597+0000 7f7dd120e640 1 Processor -- start 2024-09-11T23:48:16.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.597+0000 7f7dd120e640 1 -- start start 2024-09-11T23:48:16.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.597+0000 7f7dd120e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dcc112a30 0x7f7dcc10e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:16.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.598+0000 7f7dd120e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dcc1125b0 0x7f7dcc10eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:16.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.598+0000 7f7dd120e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dcc10f170 0x7f7dcc1c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:16.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.598+0000 7f7dd120e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc106730 con 0x7f7dcc1125b0 2024-09-11T23:48:16.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.598+0000 7f7dd120e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc1065b0 con 0x7f7dcc112a30 2024-09-11T23:48:16.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.598+0000 7f7dd120e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc1068b0 con 0x7f7dcc10f170 2024-09-11T23:48:16.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.598+0000 7f7dcad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dcc112a30 0x7f7dcc10e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:16.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.598+0000 7f7dcad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dcc112a30 0x7f7dcc10e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52034/0 (socket says 172.21.15.133:52034) 2024-09-11T23:48:16.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.598+0000 7f7dcad9d640 1 -- 172.21.15.133:0/2511889951 learned_addr learned my addr 172.21.15.133:0/2511889951 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:16.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.598+0000 7f7dcb59e640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dcc10f170 0x7f7dcc1c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:16.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.600+0000 7f7dcad9d640 1 -- 172.21.15.133:0/2511889951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dcc10f170 msgr2=0x7f7dcc1c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:16.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.600+0000 7f7dcad9d640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dcc10f170 0x7f7dcc1c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:16.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.600+0000 7f7dcad9d640 1 -- 172.21.15.133:0/2511889951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dcc1125b0 msgr2=0x7f7dcc10eb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:48:16.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.600+0000 7f7dcad9d640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dcc1125b0 0x7f7dcc10eb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:16.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.600+0000 7f7dcad9d640 1 -- 172.21.15.133:0/2511889951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc1c5160 con 0x7f7dcc112a30 2024-09-11T23:48:16.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.602+0000 7f7dcad9d640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dcc112a30 0x7f7dcc10e650 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000b430 tx=0x7f7dc000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:16.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.605+0000 7f7db3fff640 1 -- 172.21.15.133:0/2511889951 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0015020 con 0x7f7dcc112a30 2024-09-11T23:48:16.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.605+0000 7f7dd120e640 1 -- 172.21.15.133:0/2511889951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dcc1c5450 con 0x7f7dcc112a30 2024-09-11T23:48:16.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.605+0000 7f7dd120e640 1 -- 172.21.15.133:0/2511889951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dcc1c5990 con 0x7f7dcc112a30 2024-09-11T23:48:16.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.605+0000 7f7db3fff640 1 -- 172.21.15.133:0/2511889951 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc0011d10 con 0x7f7dcc112a30 2024-09-11T23:48:16.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.605+0000 7f7db3fff640 1 -- 172.21.15.133:0/2511889951 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000f2e0 con 0x7f7dcc112a30 2024-09-11T23:48:16.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.607+0000 7f7dd120e640 1 -- 172.21.15.133:0/2511889951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db8005290 con 0x7f7dcc112a30 2024-09-11T23:48:16.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.613+0000 7f7db3fff640 1 -- 172.21.15.133:0/2511889951 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7dc0018020 con 0x7f7dcc112a30 2024-09-11T23:48:16.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.614+0000 7f7db3fff640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7db40a2f30 0x7f7db40a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:16.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.614+0000 7f7db3fff640 1 -- 172.21.15.133:0/2511889951 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f7dc00ceff0 con 0x7f7dcc112a30 2024-09-11T23:48:16.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.617+0000 7f7db3fff640 1 -- 172.21.15.133:0/2511889951 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7dc001f020 con 0x7f7dcc112a30 2024-09-11T23:48:16.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.617+0000 7f7dca59c640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7db40a2f30 0x7f7db40a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:16.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.619+0000 7f7dca59c640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7db40a2f30 0x7f7db40a53f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc077be0 tx=0x7f7dc4009250 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:16.638 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:16.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.893+0000 7f7dd120e640 1 -- 172.21.15.133:0/2511889951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7db8005580 con 0x7f7dcc112a30 2024-09-11T23:48:16.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.894+0000 7f7db3fff640 1 -- 172.21.15.133:0/2511889951 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11994 (secure 0 0 0) 0x7f7dc009bd80 con 0x7f7dcc112a30 2024-09-11T23:48:16.894 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:48:16.894 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":32,"btime":"2024-09-11T23:48:01:740980+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:48:01.740959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27616":{"gid":27616,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2029124747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2029124747},{"type":"v1","addr":"172.21.15.178:6835","nonce":2029124747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:48:16.894 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 32 2024-09-11T23:48:16.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 -- 172.21.15.133:0/2511889951 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7db40a2f30 msgr2=0x7f7db40a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:16.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7db40a2f30 0x7f7db40a53f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc077be0 tx=0x7f7dc4009250 comp rx=0 tx=0).stop 2024-09-11T23:48:16.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 -- 172.21.15.133:0/2511889951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dcc112a30 msgr2=0x7f7dcc10e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:16.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dcc112a30 0x7f7dcc10e650 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000b430 tx=0x7f7dc000b8f0 comp rx=0 tx=0).stop 2024-09-11T23:48:16.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 -- 172.21.15.133:0/2511889951 shutdown_connections 2024-09-11T23:48:16.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7db40a2f30 0x7f7db40a53f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:16.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dcc10f170 0x7f7dcc1c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:16.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dcc1125b0 0x7f7dcc10eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:16.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 --2- 172.21.15.133:0/2511889951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dcc112a30 0x7f7dcc10e650 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:16.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 -- 172.21.15.133:0/2511889951 >> 172.21.15.133:0/2511889951 conn(0x7f7dcc0ffa70 msgr2=0x7f7dcc113340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:16.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 -- 172.21.15.133:0/2511889951 shutdown_connections 2024-09-11T23:48:16.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:16.896+0000 7f7dd120e640 1 -- 172.21.15.133:0/2511889951 wait complete. 2024-09-11T23:48:17.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:16 smithi133 ceph-mon[31430]: pgmap v361: 97 pgs: 97 active+clean; 60 GiB data, 252 GiB used, 821 GiB / 1.0 TiB avail; 58 KiB/s rd, 148 MiB/s wr, 1.15k op/s 2024-09-11T23:48:17.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:16 smithi138 ceph-mon[35142]: pgmap v361: 97 pgs: 97 active+clean; 60 GiB data, 252 GiB used, 821 GiB / 1.0 TiB avail; 58 KiB/s rd, 148 MiB/s wr, 1.15k op/s 2024-09-11T23:48:17.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.241+0000 7fcf676d6640 1 -- 172.21.15.133:0/2146531870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf60106a70 msgr2=0x7fcf60106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:17.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.241+0000 7fcf676d6640 1 --2- 172.21.15.133:0/2146531870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf60106a70 0x7fcf60106e50 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009930 tx=0x7fcf5c02a420 comp rx=0 tx=0).stop 2024-09-11T23:48:17.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.241+0000 7fcf676d6640 1 -- 172.21.15.133:0/2146531870 shutdown_connections 2024-09-11T23:48:17.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.241+0000 7fcf676d6640 1 --2- 172.21.15.133:0/2146531870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf60101410 0x7fcf6010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.241+0000 7fcf676d6640 1 --2- 172.21.15.133:0/2146531870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60100a70 0x7fcf60100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.241+0000 7fcf676d6640 1 --2- 172.21.15.133:0/2146531870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf60106a70 0x7fcf60106e50 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.241+0000 7fcf676d6640 1 -- 172.21.15.133:0/2146531870 >> 172.21.15.133:0/2146531870 conn(0x7fcf600fc790 msgr2=0x7fcf600febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:17.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.241+0000 7fcf676d6640 1 -- 172.21.15.133:0/2146531870 shutdown_connections 2024-09-11T23:48:17.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.241+0000 7fcf676d6640 1 -- 172.21.15.133:0/2146531870 wait complete. 2024-09-11T23:48:17.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.241+0000 7fcf676d6640 1 Processor -- start 2024-09-11T23:48:17.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.241+0000 7fcf676d6640 1 -- start start 2024-09-11T23:48:17.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf676d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf60100a70 0x7fcf601984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf676d6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf60101410 0x7fcf601989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf676d6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60106a70 0x7fcf6019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf676d6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf60110e90 con 0x7fcf60100a70 2024-09-11T23:48:17.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf676d6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf60110d10 con 0x7fcf60106a70 2024-09-11T23:48:17.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf676d6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf60111010 con 0x7fcf60101410 2024-09-11T23:48:17.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf65472640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf60100a70 0x7fcf601984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf65472640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf60100a70 0x7fcf601984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57764/0 (socket says 172.21.15.133:57764) 2024-09-11T23:48:17.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf65472640 1 -- 172.21.15.133:0/2075416704 learned_addr learned my addr 172.21.15.133:0/2075416704 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:17.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf65c73640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60106a70 0x7fcf6019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf64c71640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf60101410 0x7fcf601989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf65472640 1 -- 172.21.15.133:0/2075416704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf60101410 msgr2=0x7fcf601989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:17.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf65472640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf60101410 0x7fcf601989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf65472640 1 -- 172.21.15.133:0/2075416704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60106a70 msgr2=0x7fcf6019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:17.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf65472640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60106a70 0x7fcf6019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.242+0000 7fcf65472640 1 -- 172.21.15.133:0/2075416704 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6019d460 con 0x7fcf60100a70 2024-09-11T23:48:17.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.243+0000 7fcf65472640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf60100a70 0x7fcf601984b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009230 tx=0x7fcf5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:17.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.243+0000 7fcf567fc640 1 -- 172.21.15.133:0/2075416704 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c03f030 con 0x7fcf60100a70 2024-09-11T23:48:17.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.243+0000 7fcf567fc640 1 -- 172.21.15.133:0/2075416704 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c003ad0 con 0x7fcf60100a70 2024-09-11T23:48:17.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.243+0000 7fcf567fc640 1 -- 172.21.15.133:0/2075416704 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c035070 con 0x7fcf60100a70 2024-09-11T23:48:17.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.243+0000 7fcf676d6640 1 -- 172.21.15.133:0/2075416704 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6019d6f0 con 0x7fcf60100a70 2024-09-11T23:48:17.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.243+0000 7fcf676d6640 1 -- 172.21.15.133:0/2075416704 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601a5030 con 0x7fcf60100a70 2024-09-11T23:48:17.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.245+0000 7fcf567fc640 1 -- 172.21.15.133:0/2075416704 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcf5c042030 con 0x7fcf60100a70 2024-09-11T23:48:17.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf69aaf740 1 Processor -- start 2024-09-11T23:48:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf69aaf740 1 -- start start 2024-09-11T23:48:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf567fc640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf4c0a2cf0 0x7fcf4c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9e1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9ec780 0x55e48a9eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e48a9432f0 con 0x55e48a9e2f50 2024-09-11T23:48:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e48a8eacf0 con 0x55e48a9ec780 2024-09-11T23:48:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e48a93f2b0 con 0x55e48a9e19e0 2024-09-11T23:48:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf65c73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf65c73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57776/0 (socket says 172.21.15.133:57776) 2024-09-11T23:48:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf567fc640 1 -- 172.21.15.133:0/2075416704 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fcf5c0ef8b0 con 0x7fcf60100a70 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf65c73640 1 -- 172.21.15.133:0/326225060 learned_addr learned my addr 172.21.15.133:0/326225060 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf65472640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf4c0a2cf0 0x7fcf4c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf65c73640 1 --2- 172.21.15.133:0/326225060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9ec780 0x55e48a9eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf64c71640 1 --2- 172.21.15.133:0/326225060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9e1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf65c73640 1 -- 172.21.15.133:0/326225060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 msgr2=0x55e48a9e1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf65c73640 1 --2- 172.21.15.133:0/326225060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9e1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf65c73640 1 -- 172.21.15.133:0/326225060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9ec780 msgr2=0x55e48a9eeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.246+0000 7fcf65c73640 1 --2- 172.21.15.133:0/326225060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9ec780 0x55e48a9eeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.247+0000 7fcf65c73640 1 -- 172.21.15.133:0/326225060 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48a8cf3a0 con 0x55e48a9e2f50 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.247+0000 7fcf64c71640 1 --2- 172.21.15.133:0/326225060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.247+0000 7fcf65472640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf4c0a2cf0 0x7fcf4c0a51b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c032c00 tx=0x7fcf5c005e50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.247+0000 7fcf65c73640 1 --2- 172.21.15.133:0/326225060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e14a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009fd0 tx=0x7fcf58009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd48f3fdabfea32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.247+0000 7fcf4b7fe640 1 -- 172.21.15.133:0/326225060 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58014040 con 0x55e48a9e2f50 2024-09-11T23:48:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.247+0000 7fcf4b7fe640 1 -- 172.21.15.133:0/326225060 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf58015e60 con 0x55e48a9e2f50 2024-09-11T23:48:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.247+0000 7fcf4b7fe640 1 -- 172.21.15.133:0/326225060 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58019380 con 0x55e48a9e2f50 2024-09-11T23:48:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.248+0000 7fcf69aaf740 1 -- 172.21.15.133:0/326225060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 msgr2=0x55e48a9e14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.248+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/326225060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e14a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009fd0 tx=0x7fcf58009650 comp rx=0 tx=0).stop 2024-09-11T23:48:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.248+0000 7fcf69aaf740 1 -- 172.21.15.133:0/326225060 shutdown_connections 2024-09-11T23:48:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.248+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/326225060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9ec780 0x55e48a9eeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.248+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/326225060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.248+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/326225060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e14a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.248+0000 7fcf69aaf740 1 -- 172.21.15.133:0/326225060 >> 172.21.15.133:0/326225060 conn(0x55e48a9a6310 msgr2=0x55e48a9de070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.248+0000 7fcf69aaf740 1 -- 172.21.15.133:0/326225060 shutdown_connections 2024-09-11T23:48:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.248+0000 7fcf69aaf740 1 -- 172.21.15.133:0/326225060 wait complete. 2024-09-11T23:48:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.248+0000 7fcf69aaf740 1 -- start start 2024-09-11T23:48:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.248+0000 7fcf69aaf740 1 Processor -- start 2024-09-11T23:48:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9ec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9e2f50 0x55e48a9e9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 0x55e48a9e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e48a9ef480 con 0x55e48a9ec780 2024-09-11T23:48:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e48a8fe060 con 0x55e48a9e2f50 2024-09-11T23:48:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e48a9ef600 con 0x55e48a9e19e0 2024-09-11T23:48:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf69aaf740 10 client.0 _ll_register_callbacks cb 0x55e48a6518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf65c73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 0x55e48a9e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf65c73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 0x55e48a9e9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57788/0 (socket says 172.21.15.133:57788) 2024-09-11T23:48:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf65c73640 1 -- 172.21.15.133:0/28649216 learned_addr learned my addr 172.21.15.133:0/28649216 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf64c71640 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9e2f50 0x55e48a9e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf65c73640 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9ec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf65c73640 1 -- 172.21.15.133:0/28649216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 msgr2=0x55e48a9ec170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf65c73640 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9ec170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf65c73640 1 -- 172.21.15.133:0/28649216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9e2f50 msgr2=0x55e48a9e9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf65c73640 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9e2f50 0x55e48a9e9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf65c73640 1 -- 172.21.15.133:0/28649216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e48a6b43c0 con 0x55e48a9ec780 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf65c73640 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 0x55e48a9e9a10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fcf58004a80 tx=0x7fcf58004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce506096c4cf750 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.249+0000 7fcf48ff9640 1 -- 172.21.15.133:0/28649216 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5801a5d0 con 0x55e48a9ec780 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf48ff9640 1 -- 172.21.15.133:0/28649216 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fcf5801a770 con 0x55e48a9ec780 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf69aaf740 1 -- 172.21.15.133:0/28649216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf58015e60 con 0x55e48a9ec780 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf48ff9640 1 -- 172.21.15.133:0/28649216 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf5801a970 con 0x55e48a9ec780 2024-09-11T23:48:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf69aaf740 10 client.18423 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf69aaf740 1 -- 172.21.15.133:0/28649216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e48a9fae90 con 0x55e48a9ec780 2024-09-11T23:48:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf48ff9640 1 -- 172.21.15.133:0/28649216 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf58015e60 con 0x55e48a9ec780 2024-09-11T23:48:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf48ff9640 1 -- 172.21.15.133:0/28649216 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcf58029400 con 0x55e48a9ec780 2024-09-11T23:48:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf69aaf740 10 client.18423 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf69aaf740 10 client.18423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf69aaf740 20 client.18423 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e48a9fc1e0 0x55e48a9fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf69aaf740 4 client.18423 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf69aaf740 1 -- 172.21.15.133:0/28649216 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e48a950860 con 0x55e48a9fc1e0 2024-09-11T23:48:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.250+0000 7fcf64c71640 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e48a9fc1e0 0x55e48a9fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.251+0000 7fcf48ff9640 10 client.18423 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:17.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.251+0000 7fcf64c71640 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e48a9fc1e0 0x55e48a9fe5d0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b85b40d9fc9f667 server_cookie=e3b71e5387773a5a in_seq=0 out_seq=0 2024-09-11T23:48:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.251+0000 7fcf48ff9640 1 -- 172.21.15.133:0/28649216 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e48a950860 con 0x55e48a9fc1e0 2024-09-11T23:48:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.251+0000 7fcf48ff9640 10 client.18423 handle_command_reply: tid=0 2024-09-11T23:48:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.251+0000 7fcf69aaf740 1 client.18423 shutdown 2024-09-11T23:48:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf69aaf740 1 -- 172.21.15.133:0/28649216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 msgr2=0x55e48a9e9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 0x55e48a9e9a10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fcf58004a80 tx=0x7fcf58004b50 comp rx=0 tx=0).stop 2024-09-11T23:48:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf69aaf740 1 -- 172.21.15.133:0/28649216 shutdown_connections 2024-09-11T23:48:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e48a9fc1e0 0x55e48a9fe5d0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 0x55e48a9e9a10 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9e2f50 0x55e48a9e9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/28649216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9ec170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf48ff9640 0 client.18423 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf69aaf740 1 -- 172.21.15.133:0/28649216 >> 172.21.15.133:0/28649216 conn(0x55e48a9a6310 msgr2=0x55e48a9e3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf69aaf740 1 -- 172.21.15.133:0/28649216 shutdown_connections 2024-09-11T23:48:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf69aaf740 1 -- 172.21.15.133:0/28649216 wait complete. 2024-09-11T23:48:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.252+0000 7fcf69aaf740 20 client.18423 trim_cache size 0 max 16384 2024-09-11T23:48:17.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.265+0000 7fcf69aaf740 1 Processor -- start 2024-09-11T23:48:17.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.265+0000 7fcf69aaf740 1 -- start start 2024-09-11T23:48:17.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.265+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9e19e0 0x55e48a9de110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.265+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e2f50 0x55e48a9f8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.265+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 0x55e48aa03db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.265+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601ab5f0 con 0x55e48a9ec780 2024-09-11T23:48:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.265+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601ad130 con 0x55e48a9e19e0 2024-09-11T23:48:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.265+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601abcc0 con 0x55e48a9e2f50 2024-09-11T23:48:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf65c73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 0x55e48aa03db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf64c71640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e2f50 0x55e48a9f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf64c71640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e2f50 0x55e48a9f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:47886/0 (socket says 172.21.15.133:47886) 2024-09-11T23:48:17.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf65c73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 0x55e48aa03db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57792/0 (socket says 172.21.15.133:57792) 2024-09-11T23:48:17.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf64c71640 1 -- 172.21.15.133:0/437234537 learned_addr learned my addr 172.21.15.133:0/437234537 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:17.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf65c73640 1 --2- 172.21.15.133:0/437234537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9e19e0 0x55e48a9de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf64c71640 1 -- 172.21.15.133:0/437234537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9e19e0 msgr2=0x55e48a9de110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:17.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf64c71640 1 --2- 172.21.15.133:0/437234537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9e19e0 0x55e48a9de110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf64c71640 1 -- 172.21.15.133:0/437234537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 msgr2=0x55e48aa03db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:17.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf64c71640 1 --2- 172.21.15.133:0/437234537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 0x55e48aa03db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf64c71640 1 -- 172.21.15.133:0/437234537 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48a9ec4e0 con 0x55e48a9e2f50 2024-09-11T23:48:17.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf64c71640 1 --2- 172.21.15.133:0/437234537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e2f50 0x55e48a9f8720 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55e48a9ec750 tx=0x7fcf500039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=852be117f2e0ad15 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:17.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf4affd640 1 -- 172.21.15.133:0/437234537 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000bc40 con 0x55e48a9e2f50 2024-09-11T23:48:17.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf4affd640 1 -- 172.21.15.133:0/437234537 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5000bde0 con 0x55e48a9e2f50 2024-09-11T23:48:17.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.266+0000 7fcf4affd640 1 -- 172.21.15.133:0/437234537 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50012450 con 0x55e48a9e2f50 2024-09-11T23:48:17.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.267+0000 7fcf69aaf740 1 -- 172.21.15.133:0/437234537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e2f50 msgr2=0x55e48a9f8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:17.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.267+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/437234537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e2f50 0x55e48a9f8720 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55e48a9ec750 tx=0x7fcf500039c0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.267+0000 7fcf69aaf740 1 -- 172.21.15.133:0/437234537 shutdown_connections 2024-09-11T23:48:17.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.267+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/437234537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9ec780 0x55e48aa03db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.267+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/437234537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e2f50 0x55e48a9f8720 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.267+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/437234537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9e19e0 0x55e48a9de110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.267+0000 7fcf69aaf740 1 -- 172.21.15.133:0/437234537 >> 172.21.15.133:0/437234537 conn(0x55e48a9a6310 msgr2=0x55e48a9dda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:17.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.267+0000 7fcf69aaf740 1 -- 172.21.15.133:0/437234537 shutdown_connections 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.267+0000 7fcf69aaf740 1 -- 172.21.15.133:0/437234537 wait complete. 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf69aaf740 1 -- start start 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf69aaf740 1 Processor -- start 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9f5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf69aaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9ec780 0x55e48a9e6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601aca60 con 0x55e48a9e2f50 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601ac390 con 0x55e48a9ec780 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf69aaf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a95f0 con 0x55e48a9e19e0 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf69aaf740 10 client.0 _ll_register_callbacks cb 0x7fcf5800a690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf64c71640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf64c71640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57798/0 (socket says 172.21.15.133:57798) 2024-09-11T23:48:17.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf64c71640 1 -- 172.21.15.133:0/3015568006 learned_addr learned my addr 172.21.15.133:0/3015568006 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:17.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf65c73640 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9ec780 0x55e48a9e6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.268+0000 7fcf65c73640 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf64c71640 1 -- 172.21.15.133:0/3015568006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 msgr2=0x55e48a9f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:17.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf64c71640 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf64c71640 1 -- 172.21.15.133:0/3015568006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9ec780 msgr2=0x55e48a9e6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:17.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf64c71640 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9ec780 0x55e48a9e6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf64c71640 1 -- 172.21.15.133:0/3015568006 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e48aa65d80 con 0x55e48a9e2f50 2024-09-11T23:48:17.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf65c73640 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9ec780 0x55e48a9e6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:17.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf64c71640 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e6340 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fcf50024000 tx=0x7fcf5000bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc70360d245009b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:17.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf48ff9640 1 -- 172.21.15.133:0/3015568006 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500046c0 con 0x55e48a9e2f50 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf48ff9640 1 -- 172.21.15.133:0/3015568006 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fcf50022040 con 0x55e48a9e2f50 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf69aaf740 1 -- 172.21.15.133:0/3015568006 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5000bde0 con 0x55e48a9e2f50 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf48ff9640 1 -- 172.21.15.133:0/3015568006 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf50004860 con 0x55e48a9e2f50 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf69aaf740 10 client.18429 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf69aaf740 1 -- 172.21.15.133:0/3015568006 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e48aa66f20 con 0x55e48a9e2f50 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.269+0000 7fcf48ff9640 1 -- 172.21.15.133:0/3015568006 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5000bde0 con 0x55e48a9e2f50 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf48ff9640 1 -- 172.21.15.133:0/3015568006 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcf500325e0 con 0x55e48a9e2f50 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf69aaf740 10 client.18429 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf69aaf740 10 client.18429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf69aaf740 20 client.18429 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e48aa67620 0x55e48aa69a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf69aaf740 4 client.18429 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf69aaf740 1 -- 172.21.15.133:0/3015568006 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e48a94ad70 con 0x55e48aa67620 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf65c73640 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e48aa67620 0x55e48aa69a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf65c73640 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e48aa67620 0x55e48aa69a40 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5244242026302c3b server_cookie=cc4b9f7c407befe3 in_seq=0 out_seq=0 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf48ff9640 10 client.18429 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf48ff9640 1 -- 172.21.15.133:0/3015568006 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e48a94ad70 con 0x55e48aa67620 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf48ff9640 10 client.18429 handle_command_reply: tid=0 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf69aaf740 1 client.18429 shutdown 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf69aaf740 1 -- 172.21.15.133:0/3015568006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 msgr2=0x55e48a9e6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.270+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e6340 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fcf50024000 tx=0x7fcf5000bc40 comp rx=0 tx=0).stop 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf69aaf740 1 -- 172.21.15.133:0/3015568006 shutdown_connections 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55e48aa67620 0x55e48aa69a40 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e48a9ec780 0x55e48a9e6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e48a9e2f50 0x55e48a9e6340 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf69aaf740 1 --2- 172.21.15.133:0/3015568006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e48a9e19e0 0x55e48a9f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf69aaf740 1 -- 172.21.15.133:0/3015568006 >> 172.21.15.133:0/3015568006 conn(0x55e48a9a6310 msgr2=0x55e48a9eac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:17.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf48ff9640 0 client.18429 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:17.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf69aaf740 1 -- 172.21.15.133:0/3015568006 shutdown_connections 2024-09-11T23:48:17.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf69aaf740 1 -- 172.21.15.133:0/3015568006 wait complete. 2024-09-11T23:48:17.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf69aaf740 20 client.18429 trim_cache size 0 max 16384 2024-09-11T23:48:17.278 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:48:17.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf676d6640 1 -- 172.21.15.133:0/2075416704 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf4c0a2cf0 msgr2=0x7fcf4c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:17.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf676d6640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf4c0a2cf0 0x7fcf4c0a51b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c032c00 tx=0x7fcf5c005e50 comp rx=0 tx=0).stop 2024-09-11T23:48:17.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf676d6640 1 -- 172.21.15.133:0/2075416704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf60100a70 msgr2=0x7fcf601984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:17.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.271+0000 7fcf676d6640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf60100a70 0x7fcf601984b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009230 tx=0x7fcf5c004270 comp rx=0 tx=0).stop 2024-09-11T23:48:17.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.272+0000 7fcf676d6640 1 -- 172.21.15.133:0/2075416704 shutdown_connections 2024-09-11T23:48:17.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.272+0000 7fcf676d6640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fcf4c0a2cf0 0x7fcf4c0a51b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.272+0000 7fcf676d6640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60106a70 0x7fcf6019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.272+0000 7fcf676d6640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf60101410 0x7fcf601989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.272+0000 7fcf676d6640 1 --2- 172.21.15.133:0/2075416704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf60100a70 0x7fcf601984b0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:17.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.272+0000 7fcf676d6640 1 -- 172.21.15.133:0/2075416704 >> 172.21.15.133:0/2075416704 conn(0x7fcf600fc790 msgr2=0x7fcf6006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:17.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.272+0000 7fcf676d6640 1 -- 172.21.15.133:0/2075416704 shutdown_connections 2024-09-11T23:48:17.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:17.272+0000 7fcf676d6640 1 -- 172.21.15.133:0/2075416704 wait complete. 2024-09-11T23:48:17.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:16 smithi178 ceph-mon[35882]: pgmap v361: 97 pgs: 97 active+clean; 60 GiB data, 252 GiB used, 821 GiB / 1.0 TiB avail; 58 KiB/s rd, 148 MiB/s wr, 1.15k op/s 2024-09-11T23:48:17.831 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24481 quiesce db --cancel --all' 2024-09-11T23:48:17.831 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24481 quiesce db --cancel --all 2024-09-11T23:48:18.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:17 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2511889951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:48:18.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:17 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2511889951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:48:18.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:17 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2511889951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:48:18.593 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:18.994 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:18 smithi133 ceph-mon[31430]: pgmap v362: 97 pgs: 97 active+clean; 60 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 57 KiB/s rd, 172 MiB/s wr, 1.37k op/s 2024-09-11T23:48:19.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.082+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/2156720824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a38100e20 msgr2=0x7f6a38101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:19.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.082+0000 7f6a3cfa3640 1 --2- 172.21.15.133:0/2156720824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a38100e20 0x7f6a38101200 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009900 tx=0x7f6a2802a480 comp rx=0 tx=0).stop 2024-09-11T23:48:19.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.082+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/2156720824 shutdown_connections 2024-09-11T23:48:19.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.082+0000 7f6a3cfa3640 1 --2- 172.21.15.133:0/2156720824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a38102ef0 0x7f6a3810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.082+0000 7f6a3cfa3640 1 --2- 172.21.15.133:0/2156720824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a38102570 0x7f6a381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.082+0000 7f6a3cfa3640 1 --2- 172.21.15.133:0/2156720824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a38100e20 0x7f6a38101200 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.082+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/2156720824 >> 172.21.15.133:0/2156720824 conn(0x7f6a380fc7d0 msgr2=0x7f6a380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:19.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.082+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/2156720824 shutdown_connections 2024-09-11T23:48:19.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.082+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/2156720824 wait complete. 2024-09-11T23:48:19.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.083+0000 7f6a3cfa3640 1 Processor -- start 2024-09-11T23:48:19.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.083+0000 7f6a3cfa3640 1 -- start start 2024-09-11T23:48:19.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.083+0000 7f6a3cfa3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a38100e20 0x7f6a38103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.083+0000 7f6a3cfa3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a38102570 0x7f6a38103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.083+0000 7f6a3cfa3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a38102ef0 0x7f6a381041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.083+0000 7f6a3cfa3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38110f80 con 0x7f6a38100e20 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.083+0000 7f6a3cfa3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38110e00 con 0x7f6a38102ef0 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.083+0000 7f6a3cfa3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38111100 con 0x7f6a38102570 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.083+0000 7f6a3659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a38100e20 0x7f6a38103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a3659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a38100e20 0x7f6a38103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57810/0 (socket says 172.21.15.133:57810) 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a3659c640 1 -- 172.21.15.133:0/3895578929 learned_addr learned my addr 172.21.15.133:0/3895578929 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a36d9d640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a38102ef0 0x7f6a381041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a2ffff640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a38102570 0x7f6a38103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a3659c640 1 -- 172.21.15.133:0/3895578929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a38102570 msgr2=0x7f6a38103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a3659c640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a38102570 0x7f6a38103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a3659c640 1 -- 172.21.15.133:0/3895578929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a38102ef0 msgr2=0x7f6a381041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a3659c640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a38102ef0 0x7f6a381041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a3659c640 1 -- 172.21.15.133:0/3895578929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a381a6a10 con 0x7f6a38100e20 2024-09-11T23:48:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a3659c640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a38100e20 0x7f6a38103750 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f6a28006010 tx=0x7f6a28003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a2f7fe640 1 -- 172.21.15.133:0/3895578929 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2803f070 con 0x7f6a38100e20 2024-09-11T23:48:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a2f7fe640 1 -- 172.21.15.133:0/3895578929 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a28031c30 con 0x7f6a38100e20 2024-09-11T23:48:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/3895578929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a381a6ca0 con 0x7f6a38100e20 2024-09-11T23:48:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a2f7fe640 1 -- 172.21.15.133:0/3895578929 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28035030 con 0x7f6a38100e20 2024-09-11T23:48:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.084+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/3895578929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a381a7180 con 0x7f6a38100e20 2024-09-11T23:48:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.087+0000 7f6a2f7fe640 1 -- 172.21.15.133:0/3895578929 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6a28042070 con 0x7f6a38100e20 2024-09-11T23:48:19.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.087+0000 7f6a3f37c740 1 Processor -- start 2024-09-11T23:48:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.087+0000 7f6a3f37c740 1 -- start start 2024-09-11T23:48:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a0fba00 0x56239a0fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a1066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a106be0 0x56239a108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56239a060200 con 0x56239a0fc320 2024-09-11T23:48:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56239a006dc0 con 0x56239a0fba00 2024-09-11T23:48:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56239a05c1c0 con 0x56239a106be0 2024-09-11T23:48:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a1066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a1066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57820/0 (socket says 172.21.15.133:57820) 2024-09-11T23:48:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3659c640 1 -- 172.21.15.133:0/504928596 learned_addr learned my addr 172.21.15.133:0/504928596 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a2f7fe640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6a240a2cf0 0x7f6a240a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a36d9d640 1 --2- 172.21.15.133:0/504928596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a0fba00 0x56239a0fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a2ffff640 1 --2- 172.21.15.133:0/504928596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a106be0 0x56239a108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a36d9d640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6a240a2cf0 0x7f6a240a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a2f7fe640 1 -- 172.21.15.133:0/3895578929 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f6a280efd20 con 0x7f6a38100e20 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3659c640 1 -- 172.21.15.133:0/504928596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a106be0 msgr2=0x56239a108fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3659c640 1 --2- 172.21.15.133:0/504928596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a106be0 0x56239a108fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3659c640 1 -- 172.21.15.133:0/504928596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a0fba00 msgr2=0x56239a0fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3659c640 1 --2- 172.21.15.133:0/504928596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a0fba00 0x56239a0fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a3659c640 1 -- 172.21.15.133:0/504928596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562399dd1620 con 0x56239a0fc320 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.088+0000 7f6a36d9d640 1 --2- 172.21.15.133:0/504928596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a0fba00 0x56239a0fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.089+0000 7f6a3659c640 1 --2- 172.21.15.133:0/504928596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a1066a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6a280bc210 tx=0x7f6a2802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9678a659bc0289e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.089+0000 7f6a36d9d640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6a240a2cf0 0x7f6a240a51b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6a30005fd0 tx=0x7f6a30004ab0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.089+0000 7f6a2cff9640 1 -- 172.21.15.133:0/504928596 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28049070 con 0x56239a0fc320 2024-09-11T23:48:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.089+0000 7f6a2cff9640 1 -- 172.21.15.133:0/504928596 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a28031de0 con 0x56239a0fc320 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.089+0000 7f6a2cff9640 1 -- 172.21.15.133:0/504928596 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a280ed730 con 0x56239a0fc320 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.089+0000 7f6a3f37c740 1 -- 172.21.15.133:0/504928596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 msgr2=0x56239a1066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.089+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/504928596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a1066a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6a280bc210 tx=0x7f6a2802a480 comp rx=0 tx=0).stop 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 -- 172.21.15.133:0/504928596 shutdown_connections 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/504928596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a106be0 0x56239a108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/504928596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a1066a0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/504928596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a0fba00 0x56239a0fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 -- 172.21.15.133:0/504928596 >> 172.21.15.133:0/504928596 conn(0x56239a0f7f90 msgr2=0x56239a0f8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 -- 172.21.15.133:0/504928596 shutdown_connections 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 -- 172.21.15.133:0/504928596 wait complete. 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 -- start start 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 Processor -- start 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a106490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a103740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a103c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56239a109570 con 0x56239a0fc320 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562399ffac50 con 0x56239a106be0 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56239a1096f0 con 0x56239a0fba00 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.090+0000 7f6a3f37c740 10 client.0 _ll_register_callbacks cb 0x56239a00cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a2ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a2ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a103740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57828/0 (socket says 172.21.15.133:57828) 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a2ffff640 1 -- 172.21.15.133:0/1954237059 learned_addr learned my addr 172.21.15.133:0/1954237059 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a3659c640 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a106490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a36d9d640 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a2ffff640 1 -- 172.21.15.133:0/1954237059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 msgr2=0x56239a106490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a2ffff640 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a106490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a2ffff640 1 -- 172.21.15.133:0/1954237059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 msgr2=0x56239a103c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a2ffff640 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a103c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a2ffff640 1 -- 172.21.15.133:0/1954237059 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239a104480 con 0x56239a0fc320 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a2ffff640 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a103740 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000da10 tx=0x7f6a2000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad8d237763fea34 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a1dffb640 1 -- 172.21.15.133:0/1954237059 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20011640 con 0x56239a0fc320 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a1dffb640 1 -- 172.21.15.133:0/1954237059 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f6a200117e0 con 0x56239a0fc320 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a3f37c740 1 -- 172.21.15.133:0/1954237059 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a28031de0 con 0x56239a0fc320 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a1dffb640 1 -- 172.21.15.133:0/1954237059 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a200119e0 con 0x56239a0fc320 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a3f37c740 10 client.18441 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.091+0000 7f6a3f37c740 1 -- 172.21.15.133:0/1954237059 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56239a114ff0 con 0x56239a0fc320 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.092+0000 7f6a1dffb640 1 -- 172.21.15.133:0/1954237059 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a28031de0 con 0x56239a0fc320 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.092+0000 7f6a1dffb640 1 -- 172.21.15.133:0/1954237059 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6a20019030 con 0x56239a0fc320 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.092+0000 7f6a3f37c740 10 client.18441 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.092+0000 7f6a3f37c740 10 client.18441 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.092+0000 7f6a3f37c740 20 client.18441 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.092+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56239a116500 0x56239a118940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.092+0000 7f6a3f37c740 4 client.18441 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.092+0000 7f6a3f37c740 1 -- 172.21.15.133:0/1954237059 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56239a06d770 con 0x56239a116500 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.092+0000 7f6a3659c640 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56239a116500 0x56239a118940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.092+0000 7f6a1dffb640 10 client.18441 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.092+0000 7f6a3659c640 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56239a116500 0x56239a118940 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa58299fdacba42c server_cookie=c5f4bd0f7580f086 in_seq=0 out_seq=0 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a1dffb640 1 -- 172.21.15.133:0/1954237059 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56239a06d770 con 0x56239a116500 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a1dffb640 10 client.18441 handle_command_reply: tid=0 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a3f37c740 1 client.18441 shutdown 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a3f37c740 1 -- 172.21.15.133:0/1954237059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 msgr2=0x56239a103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a103740 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000da10 tx=0x7f6a2000ded0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a3f37c740 1 -- 172.21.15.133:0/1954237059 shutdown_connections 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56239a116500 0x56239a118940 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a103740 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/1954237059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a3f37c740 1 -- 172.21.15.133:0/1954237059 >> 172.21.15.133:0/1954237059 conn(0x56239a0f7f90 msgr2=0x56239a0f71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:19.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.093+0000 7f6a1dffb640 0 client.18441 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:19.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.094+0000 7f6a3f37c740 1 -- 172.21.15.133:0/1954237059 shutdown_connections 2024-09-11T23:48:19.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.094+0000 7f6a3f37c740 1 -- 172.21.15.133:0/1954237059 wait complete. 2024-09-11T23:48:19.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.094+0000 7f6a3f37c740 20 client.18441 trim_cache size 0 max 16384 2024-09-11T23:48:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a3f37c740 1 Processor -- start 2024-09-11T23:48:19.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a3f37c740 1 -- start start 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a112660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a102720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a11e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381afa00 con 0x56239a0fc320 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b1540 con 0x56239a106be0 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b00d0 con 0x56239a0fba00 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a2ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a2ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a102720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57840/0 (socket says 172.21.15.133:57840) 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a2ffff640 1 -- 172.21.15.133:0/739492861 learned_addr learned my addr 172.21.15.133:0/739492861 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.111+0000 7f6a36d9d640 1 --2- 172.21.15.133:0/739492861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a11e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a3659c640 1 --2- 172.21.15.133:0/739492861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a112660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a2ffff640 1 -- 172.21.15.133:0/739492861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 msgr2=0x56239a112660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a2ffff640 1 --2- 172.21.15.133:0/739492861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a112660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a2ffff640 1 -- 172.21.15.133:0/739492861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 msgr2=0x56239a11e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a2ffff640 1 --2- 172.21.15.133:0/739492861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a11e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a2ffff640 1 -- 172.21.15.133:0/739492861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239a0f7dc0 con 0x56239a0fc320 2024-09-11T23:48:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a36d9d640 1 --2- 172.21.15.133:0/739492861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a11e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:48:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a2ffff640 1 --2- 172.21.15.133:0/739492861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a102720 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f6a2001ae90 tx=0x7f6a20003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2020a47ffdc9498 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a1ffff640 1 -- 172.21.15.133:0/739492861 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20030070 con 0x56239a0fc320 2024-09-11T23:48:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a1ffff640 1 -- 172.21.15.133:0/739492861 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2000f490 con 0x56239a0fc320 2024-09-11T23:48:19.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a1ffff640 1 -- 172.21.15.133:0/739492861 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20027040 con 0x56239a0fc320 2024-09-11T23:48:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a3f37c740 1 -- 172.21.15.133:0/739492861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 msgr2=0x56239a102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.112+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/739492861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a102720 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f6a2001ae90 tx=0x7f6a20003f20 comp rx=0 tx=0).stop 2024-09-11T23:48:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 -- 172.21.15.133:0/739492861 shutdown_connections 2024-09-11T23:48:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/739492861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a11e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/739492861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a102720 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/739492861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 -- 172.21.15.133:0/739492861 >> 172.21.15.133:0/739492861 conn(0x56239a0f7f90 msgr2=0x56239a102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 -- 172.21.15.133:0/739492861 shutdown_connections 2024-09-11T23:48:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 -- 172.21.15.133:0/739492861 wait complete. 2024-09-11T23:48:19.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 -- start start 2024-09-11T23:48:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 Processor -- start 2024-09-11T23:48:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a10bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a10c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a17f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b0e70 con 0x56239a0fc320 2024-09-11T23:48:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b07a0 con 0x56239a106be0 2024-09-11T23:48:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381ada00 con 0x56239a0fba00 2024-09-11T23:48:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3f37c740 10 client.0 _ll_register_callbacks cb 0x7f6a20004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:19.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a2ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a10c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a2ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a10c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57850/0 (socket says 172.21.15.133:57850) 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a2ffff640 1 -- 172.21.15.133:0/3594807610 learned_addr learned my addr 172.21.15.133:0/3594807610 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a36d9d640 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a17f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.113+0000 7f6a3659c640 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a10bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a2ffff640 1 -- 172.21.15.133:0/3594807610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 msgr2=0x56239a10bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a2ffff640 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a10bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a2ffff640 1 -- 172.21.15.133:0/3594807610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 msgr2=0x56239a17f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a2ffff640 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a17f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a2ffff640 1 -- 172.21.15.133:0/3594807610 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239a17f720 con 0x56239a0fc320 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a2ffff640 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a10c180 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6a20004550 tx=0x7f6a200039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6736c7dcde1d4d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a1dffb640 1 -- 172.21.15.133:0/3594807610 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20027070 con 0x56239a0fc320 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a1dffb640 1 -- 172.21.15.133:0/3594807610 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f6a20011640 con 0x56239a0fc320 2024-09-11T23:48:19.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a3f37c740 1 -- 172.21.15.133:0/3594807610 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a2000f490 con 0x56239a0fc320 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a1dffb640 1 -- 172.21.15.133:0/3594807610 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a20030040 con 0x56239a0fc320 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a3f37c740 10 client.18447 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.114+0000 7f6a3f37c740 1 -- 172.21.15.133:0/3594807610 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56239a180940 con 0x56239a0fc320 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.115+0000 7f6a1dffb640 1 -- 172.21.15.133:0/3594807610 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2000f490 con 0x56239a0fc320 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.115+0000 7f6a1dffb640 1 -- 172.21.15.133:0/3594807610 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6a20033020 con 0x56239a0fc320 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.115+0000 7f6a3f37c740 10 client.18447 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.115+0000 7f6a3f37c740 10 client.18447 resolve_mds: validated gid 24481 aka daemon mds.i 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.115+0000 7f6a3f37c740 20 client.18447 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.115+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56239a1810b0 0x56239a1834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.115+0000 7f6a3f37c740 4 client.18447 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.115+0000 7f6a3f37c740 1 -- 172.21.15.133:0/3594807610 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x56239a067c80 con 0x56239a1810b0 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.115+0000 7f6a3659c640 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56239a1810b0 0x56239a1834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:19.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.115+0000 7f6a3659c640 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56239a1810b0 0x56239a1834d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f487af109dd1d93b server_cookie=85d513d5b8c05109 in_seq=0 out_seq=0 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.115+0000 7f6a1dffb640 10 client.18447 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a1dffb640 1 -- 172.21.15.133:0/3594807610 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x56239a067c80 con 0x56239a1810b0 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a1dffb640 10 client.18447 handle_command_reply: tid=0 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 1 client.18447 shutdown 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 1 -- 172.21.15.133:0/3594807610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 msgr2=0x56239a10c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a10c180 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6a20004550 tx=0x7f6a200039c0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 1 -- 172.21.15.133:0/3594807610 shutdown_connections 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56239a1810b0 0x56239a1834d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239a106be0 0x56239a17f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239a0fc320 0x56239a10c180 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a1dffb640 0 client.18447 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 1 --2- 172.21.15.133:0/3594807610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56239a0fba00 0x56239a10bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 1 -- 172.21.15.133:0/3594807610 >> 172.21.15.133:0/3594807610 conn(0x56239a0f7f90 msgr2=0x56239a102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:19.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 1 -- 172.21.15.133:0/3594807610 shutdown_connections 2024-09-11T23:48:19.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 1 -- 172.21.15.133:0/3594807610 wait complete. 2024-09-11T23:48:19.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.116+0000 7f6a3f37c740 20 client.18447 trim_cache size 0 max 16384 2024-09-11T23:48:19.122 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:48:19.122 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 32, 2024-09-11T23:48:19.122 INFO:teuthology.orchestra.run.smithi133.stdout: "leader": 24481, 2024-09-11T23:48:19.122 INFO:teuthology.orchestra.run.smithi133.stdout: "set_version": 44, 2024-09-11T23:48:19.122 INFO:teuthology.orchestra.run.smithi133.stdout: "sets": {} 2024-09-11T23:48:19.122 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:48:19.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/3895578929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6a240a2cf0 msgr2=0x7f6a240a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:19.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6a240a2cf0 0x7f6a240a51b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6a30005fd0 tx=0x7f6a30004ab0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/3895578929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a38100e20 msgr2=0x7f6a38103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:19.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a38100e20 0x7f6a38103750 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f6a28006010 tx=0x7f6a28003c10 comp rx=0 tx=0).stop 2024-09-11T23:48:19.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/3895578929 shutdown_connections 2024-09-11T23:48:19.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f6a240a2cf0 0x7f6a240a51b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a38102ef0 0x7f6a381041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a38102570 0x7f6a38103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 --2- 172.21.15.133:0/3895578929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a38100e20 0x7f6a38103750 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:19.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/3895578929 >> 172.21.15.133:0/3895578929 conn(0x7f6a380fc7d0 msgr2=0x7f6a3806a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:19.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/3895578929 shutdown_connections 2024-09-11T23:48:19.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:19.117+0000 7f6a3cfa3640 1 -- 172.21.15.133:0/3895578929 wait complete. 2024-09-11T23:48:19.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:18 smithi138 ceph-mon[35142]: pgmap v362: 97 pgs: 97 active+clean; 60 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 57 KiB/s rd, 172 MiB/s wr, 1.37k op/s 2024-09-11T23:48:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:18 smithi178 ceph-mon[35882]: pgmap v362: 97 pgs: 97 active+clean; 60 GiB data, 253 GiB used, 820 GiB / 1.0 TiB avail; 57 KiB/s rd, 172 MiB/s wr, 1.37k op/s 2024-09-11T23:48:19.485 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:48:19.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:48:19.845 INFO:tasks.quiescer:done joining Quiescers 2024-09-11T23:48:19.845 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-11T23:48:19.855 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T23:48:19.864 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-11T23:48:19.888 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-11T23:48:19.888 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:48:19.888 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T23:48:19.888 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-11T23:48:19.922 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:19.934 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-11T23:48:20.002 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T23:48:20.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.151+0000 7f7924b91640 1 -- 172.21.15.133:0/3332517281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79201035c0 msgr2=0x7f792010baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.151+0000 7f7924b91640 1 --2- 172.21.15.133:0/3332517281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79201035c0 0x7f792010baa0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7914009900 tx=0x7f791402a410 comp rx=0 tx=0).stop 2024-09-11T23:48:20.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.151+0000 7f7924b91640 1 -- 172.21.15.133:0/3332517281 shutdown_connections 2024-09-11T23:48:20.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.151+0000 7f7924b91640 1 --2- 172.21.15.133:0/3332517281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f792010c0d0 0x7f792010e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.151+0000 7f7924b91640 1 --2- 172.21.15.133:0/3332517281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79201035c0 0x7f792010baa0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.151+0000 7f7924b91640 1 --2- 172.21.15.133:0/3332517281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7920102ca0 0x7f7920103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.151+0000 7f7924b91640 1 -- 172.21.15.133:0/3332517281 >> 172.21.15.133:0/3332517281 conn(0x7f79200fc6b0 msgr2=0x7f79200fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:20.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.151+0000 7f7924b91640 1 -- 172.21.15.133:0/3332517281 shutdown_connections 2024-09-11T23:48:20.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.151+0000 7f7924b91640 1 -- 172.21.15.133:0/3332517281 wait complete. 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.152+0000 7f7924b91640 1 Processor -- start 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.152+0000 7f7924b91640 1 -- start start 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f7924b91640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7920102ca0 0x7f7920198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f7924b91640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79201035c0 0x7f79201988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f7924b91640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f792010c0d0 0x7f792019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f7924b91640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7920111f20 con 0x7f79201035c0 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f7924b91640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7920111da0 con 0x7f792010c0d0 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f7924b91640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f79201120a0 con 0x7f7920102ca0 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f791dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79201035c0 0x7f79201988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f791dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79201035c0 0x7f79201988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57860/0 (socket says 172.21.15.133:57860) 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f791dd9b640 1 -- 172.21.15.133:0/2688749547 learned_addr learned my addr 172.21.15.133:0/2688749547 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:20.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f791e59c640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7920102ca0 0x7f7920198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f791dd9b640 1 -- 172.21.15.133:0/2688749547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7920102ca0 msgr2=0x7f7920198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f791dd9b640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7920102ca0 0x7f7920198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f791dd9b640 1 -- 172.21.15.133:0/2688749547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f792010c0d0 msgr2=0x7f792019cc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:48:20.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f791dd9b640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f792010c0d0 0x7f792019cc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.153+0000 7f791dd9b640 1 -- 172.21.15.133:0/2688749547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792019d350 con 0x7f79201035c0 2024-09-11T23:48:20.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.156+0000 7f791dd9b640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79201035c0 0x7f79201988a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f791402a930 tx=0x7f7914003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:20.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.157+0000 7f790f7fe640 1 -- 172.21.15.133:0/2688749547 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791403f070 con 0x7f79201035c0 2024-09-11T23:48:20.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.157+0000 7f790f7fe640 1 -- 172.21.15.133:0/2688749547 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7914031c90 con 0x7f79201035c0 2024-09-11T23:48:20.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.157+0000 7f790f7fe640 1 -- 172.21.15.133:0/2688749547 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914035030 con 0x7f79201035c0 2024-09-11T23:48:20.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.157+0000 7f7924b91640 1 -- 172.21.15.133:0/2688749547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792019d5e0 con 0x7f79201035c0 2024-09-11T23:48:20.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.157+0000 7f7924b91640 1 -- 172.21.15.133:0/2688749547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792019d9f0 con 0x7f79201035c0 2024-09-11T23:48:20.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.159+0000 7f7924b91640 1 -- 172.21.15.133:0/2688749547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7920103080 con 0x7f79201035c0 2024-09-11T23:48:20.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.160+0000 7f790f7fe640 1 -- 172.21.15.133:0/2688749547 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7914042070 con 0x7f79201035c0 2024-09-11T23:48:20.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.161+0000 7f790f7fe640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79080a2e20 0x7f79080a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.161+0000 7f790f7fe640 1 -- 172.21.15.133:0/2688749547 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f79140ef730 con 0x7f79201035c0 2024-09-11T23:48:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.161+0000 7f791e59c640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79080a2e20 0x7f79080a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.162+0000 7f791e59c640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79080a2e20 0x7f79080a52e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7910009bd0 tx=0x7f7910005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:20.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.168+0000 7f790f7fe640 1 -- 172.21.15.133:0/2688749547 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79140bc390 con 0x7f79201035c0 2024-09-11T23:48:20.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.533+0000 7f22f293f640 1 -- 172.21.15.133:0/3595521510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e409b680 msgr2=0x7f22e409ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.533+0000 7f22f293f640 1 --2- 172.21.15.133:0/3595521510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e409b680 0x7f22e409ba60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009860 tx=0x7f22e002a380 comp rx=0 tx=0).stop 2024-09-11T23:48:20.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.536+0000 7f22f293f640 1 -- 172.21.15.133:0/3595521510 shutdown_connections 2024-09-11T23:48:20.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.536+0000 7f22f293f640 1 --2- 172.21.15.133:0/3595521510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22e4095f70 0x7f22e40a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.536+0000 7f22f293f640 1 --2- 172.21.15.133:0/3595521510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22e40955d0 0x7f22e4095a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.536+0000 7f22f293f640 1 --2- 172.21.15.133:0/3595521510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e409b680 0x7f22e409ba60 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.536+0000 7f22f293f640 1 -- 172.21.15.133:0/3595521510 >> 172.21.15.133:0/3595521510 conn(0x7f22e4091270 msgr2=0x7f22e4093690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:20.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.537+0000 7f22f293f640 1 -- 172.21.15.133:0/3595521510 shutdown_connections 2024-09-11T23:48:20.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.537+0000 7f22f293f640 1 -- 172.21.15.133:0/3595521510 wait complete. 2024-09-11T23:48:20.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.538+0000 7f22f293f640 1 Processor -- start 2024-09-11T23:48:20.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.538+0000 7f22f293f640 1 -- start start 2024-09-11T23:48:20.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.539+0000 7f22f293f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22e40955d0 0x7f22e412be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.539+0000 7f22f293f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22e4095f70 0x7f22e412c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.539+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22e40955d0 0x7f22e412be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.539+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22e40955d0 0x7f22e412be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52178/0 (socket says 172.21.15.133:52178) 2024-09-11T23:48:20.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.539+0000 7f22ebfff640 1 -- 172.21.15.133:0/3626332278 learned_addr learned my addr 172.21.15.133:0/3626332278 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:20.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.540+0000 7f22f293f640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e409b680 0x7f22e4130710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.540+0000 7f22eb7fe640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22e4095f70 0x7f22e412c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.540+0000 7f22f293f640 1 -- 172.21.15.133:0/3626332278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22e40a5a00 con 0x7f22e409b680 2024-09-11T23:48:20.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.540+0000 7f22f293f640 1 -- 172.21.15.133:0/3626332278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22e40a5880 con 0x7f22e40955d0 2024-09-11T23:48:20.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.540+0000 7f22f293f640 1 -- 172.21.15.133:0/3626332278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f22e40a5b80 con 0x7f22e4095f70 2024-09-11T23:48:20.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.540+0000 7f7924b91640 1 -- 172.21.15.133:0/2688749547 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7920062630 con 0x7f79080a2e20 2024-09-11T23:48:20.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.540+0000 7f22eb7fe640 1 -- 172.21.15.133:0/3626332278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22e40955d0 msgr2=0x7f22e412be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.540+0000 7f22eb7fe640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22e40955d0 0x7f22e412be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.540+0000 7f22eb7fe640 1 -- 172.21.15.133:0/3626332278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e409b680 msgr2=0x7f22e4130710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.541+0000 7f22eb7fe640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e409b680 0x7f22e4130710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.541+0000 7f22eb7fe640 1 -- 172.21.15.133:0/3626332278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e4130df0 con 0x7f22e4095f70 2024-09-11T23:48:20.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.541+0000 7f22eb7fe640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22e4095f70 0x7f22e412c380 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009fd0 tx=0x7f22d800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:20.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.541+0000 7f22e97fa640 1 -- 172.21.15.133:0/3626332278 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8015020 con 0x7f22e4095f70 2024-09-11T23:48:20.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.541+0000 7f22e97fa640 1 -- 172.21.15.133:0/3626332278 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d8011d10 con 0x7f22e4095f70 2024-09-11T23:48:20.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.541+0000 7f22f293f640 1 -- 172.21.15.133:0/3626332278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e41310e0 con 0x7f22e4095f70 2024-09-11T23:48:20.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.542+0000 7f22e97fa640 1 -- 172.21.15.133:0/3626332278 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d800f2e0 con 0x7f22e4095f70 2024-09-11T23:48:20.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.543+0000 7f22f293f640 1 -- 172.21.15.133:0/3626332278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e41389c0 con 0x7f22e4095f70 2024-09-11T23:48:20.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.546+0000 7f22f4d18740 1 Processor -- start 2024-09-11T23:48:20.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.547+0000 7f22f4d18740 1 -- start start 2024-09-11T23:48:20.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.547+0000 7f22f4d18740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d47ffd70 0x5647d4800150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.547+0000 7f22f4d18740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 0x5647d480aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.547+0000 7f22f4d18740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d480af50 0x5647d480d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.547+0000 7f22f4d18740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5647d4765280 con 0x5647d480af50 2024-09-11T23:48:20.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.547+0000 7f22f4d18740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647d470cb10 con 0x5647d47ffd70 2024-09-11T23:48:20.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.547+0000 7f22f4d18740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5647d4761240 con 0x5647d4800690 2024-09-11T23:48:20.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.548+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 0x5647d480aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.548+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 0x5647d480aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:47984/0 (socket says 172.21.15.133:47984) 2024-09-11T23:48:20.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.548+0000 7f22ebfff640 1 -- 172.21.15.133:0/3007952741 learned_addr learned my addr 172.21.15.133:0/3007952741 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:20.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.548+0000 7f22e97fa640 1 -- 172.21.15.133:0/3626332278 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f22d8018020 con 0x7f22e4095f70 2024-09-11T23:48:20.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.548+0000 7f22eb7fe640 1 --2- 172.21.15.133:0/3007952741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d480af50 0x5647d480d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.548+0000 7f22ebfff640 1 -- 172.21.15.133:0/3007952741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d47ffd70 msgr2=0x5647d4800150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:20.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.548+0000 7f22ebfff640 1 --2- 172.21.15.133:0/3007952741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d47ffd70 0x5647d4800150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.548+0000 7f22ebfff640 1 -- 172.21.15.133:0/3007952741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d480af50 msgr2=0x5647d480d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:20.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.548+0000 7f22ebfff640 1 --2- 172.21.15.133:0/3007952741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d480af50 0x5647d480d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.548+0000 7f22ebfff640 1 -- 172.21.15.133:0/3007952741 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d47191f0 con 0x5647d4800690 2024-09-11T23:48:20.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.549+0000 7f22ebfff640 1 --2- 172.21.15.133:0/3007952741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 0x5647d480aa10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f22e00095c0 tx=0x7f22e0030590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b7b18c3ef49c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:20.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.549+0000 7f22d67fc640 1 -- 172.21.15.133:0/3007952741 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0037070 con 0x5647d4800690 2024-09-11T23:48:20.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.549+0000 7f22d67fc640 1 -- 172.21.15.133:0/3007952741 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e0005030 con 0x5647d4800690 2024-09-11T23:48:20.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.550+0000 7f22d67fc640 1 -- 172.21.15.133:0/3007952741 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0030cf0 con 0x5647d4800690 2024-09-11T23:48:20.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.550+0000 7f22e97fa640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22dc0a2d80 0x7f22dc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.550+0000 7f22e97fa640 1 -- 172.21.15.133:0/3626332278 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f22d80cea40 con 0x7f22e4095f70 2024-09-11T23:48:20.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.550+0000 7f22f4d18740 1 -- 172.21.15.133:0/3007952741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 msgr2=0x5647d480aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.550+0000 7f22f4d18740 1 --2- 172.21.15.133:0/3007952741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 0x5647d480aa10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f22e00095c0 tx=0x7f22e0030590 comp rx=0 tx=0).stop 2024-09-11T23:48:20.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.551+0000 7f22f0edc640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22dc0a2d80 0x7f22dc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.552+0000 7f22f4d18740 1 -- 172.21.15.133:0/3007952741 shutdown_connections 2024-09-11T23:48:20.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.552+0000 7f22f4d18740 1 --2- 172.21.15.133:0/3007952741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d480af50 0x5647d480d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.552+0000 7f22f4d18740 1 --2- 172.21.15.133:0/3007952741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 0x5647d480aa10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.552+0000 7f22f4d18740 1 --2- 172.21.15.133:0/3007952741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d47ffd70 0x5647d4800150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.552+0000 7f22f4d18740 1 -- 172.21.15.133:0/3007952741 >> 172.21.15.133:0/3007952741 conn(0x5647d47fc300 msgr2=0x5647d47fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:20.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.553+0000 7f22f0edc640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22dc0a2d80 0x7f22dc0a5240 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0637b0 tx=0x7f22ec0681c0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:20.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.554+0000 7f22f4d18740 1 -- 172.21.15.133:0/3007952741 shutdown_connections 2024-09-11T23:48:20.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.554+0000 7f22f4d18740 1 -- 172.21.15.133:0/3007952741 wait complete. 2024-09-11T23:48:20.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.554+0000 7f22f4d18740 1 -- start start 2024-09-11T23:48:20.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.555+0000 7f22f4d18740 1 Processor -- start 2024-09-11T23:48:20.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.555+0000 7f22f4d18740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d47ffd70 0x5647d480aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.555+0000 7f22f4d18740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d4800690 0x5647d4807d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22f4d18740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d480af50 0x5647d48084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d47ffd70 0x5647d480aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22f4d18740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5647d480dbc0 con 0x5647d4800690 2024-09-11T23:48:20.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d47ffd70 0x5647d480aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52200/0 (socket says 172.21.15.133:52200) 2024-09-11T23:48:20.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22f4d18740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647d471ff10 con 0x5647d47ffd70 2024-09-11T23:48:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22ebfff640 1 -- 172.21.15.133:0/1977946646 learned_addr learned my addr 172.21.15.133:0/1977946646 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22f4d18740 1 -- 172.21.15.133:0/1977946646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5647d480dd40 con 0x5647d480af50 2024-09-11T23:48:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22f4d18740 10 client.0 _ll_register_callbacks cb 0x5647d46e8a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22f0edc640 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d480af50 0x5647d48084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22ebfff640 1 -- 172.21.15.133:0/1977946646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d480af50 msgr2=0x5647d48084e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22ebfff640 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d480af50 0x5647d48084e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22ebfff640 1 -- 172.21.15.133:0/1977946646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d4800690 msgr2=0x5647d4807d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22ebfff640 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d4800690 0x5647d4807d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22ebfff640 1 -- 172.21.15.133:0/1977946646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d44d66a0 con 0x5647d47ffd70 2024-09-11T23:48:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22ebfff640 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d47ffd70 0x5647d480aa00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f22e0030820 tx=0x7f22e0030f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e164c535b80f1b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.556+0000 7f22cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:48:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.557+0000 7f22cb7fe640 1 -- 172.21.15.133:0/1977946646 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0055060 con 0x5647d47ffd70 2024-09-11T23:48:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.557+0000 7f22f4d18740 1 -- 172.21.15.133:0/1977946646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e0005030 con 0x5647d47ffd70 2024-09-11T23:48:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.557+0000 7f22cb7fe640 1 -- 172.21.15.133:0/1977946646 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f22e0047a60 con 0x5647d47ffd70 2024-09-11T23:48:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.557+0000 7f22cb7fe640 1 -- 172.21.15.133:0/1977946646 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f22e0037040 con 0x5647d47ffd70 2024-09-11T23:48:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.557+0000 7f22f4d18740 10 client.27989 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.557+0000 7f22f4d18740 1 -- 172.21.15.133:0/1977946646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5647d4819730 con 0x5647d47ffd70 2024-09-11T23:48:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.557+0000 7f22cb7fe640 1 -- 172.21.15.133:0/1977946646 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e0047c60 con 0x5647d47ffd70 2024-09-11T23:48:20.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.559+0000 7f22cb7fe640 1 -- 172.21.15.133:0/1977946646 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f22e003a040 con 0x5647d47ffd70 2024-09-11T23:48:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.559+0000 7f790f7fe640 1 -- 172.21.15.133:0/2688749547 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f7920062630 con 0x7f79080a2e20 2024-09-11T23:48:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.559+0000 7f22f4d18740 10 client.27989 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.559+0000 7f22f4d18740 10 client.27989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.559+0000 7f22f4d18740 20 client.27989 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.559+0000 7f22f4d18740 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5647d481ad20 0x5647d481d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.559+0000 7f22f4d18740 4 client.27989 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:20.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.559+0000 7f22f4d18740 1 -- 172.21.15.133:0/1977946646 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f22e0005030 con 0x5647d481ad20 2024-09-11T23:48:20.560 INFO:teuthology.orchestra.run.smithi133.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T23:36:58", "first": "2024-09-11T23:37:00", "last": "2024-09-11T23:48:00", "last_pruned": "2024-09-11T23:48:01", "created_count": 12, "pruned_count": 6, "active": true} 2024-09-11T23:48:20.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.560+0000 7f22eb7fe640 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5647d481ad20 0x5647d481d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.562+0000 7f7924b91640 1 -- 172.21.15.133:0/2688749547 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79080a2e20 msgr2=0x7f79080a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.562+0000 7f7924b91640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79080a2e20 0x7f79080a52e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7910009bd0 tx=0x7f7910005f90 comp rx=0 tx=0).stop 2024-09-11T23:48:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.562+0000 7f7924b91640 1 -- 172.21.15.133:0/2688749547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79201035c0 msgr2=0x7f79201988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.562+0000 7f7924b91640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79201035c0 0x7f79201988a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f791402a930 tx=0x7f7914003b20 comp rx=0 tx=0).stop 2024-09-11T23:48:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.563+0000 7f7924b91640 1 -- 172.21.15.133:0/2688749547 shutdown_connections 2024-09-11T23:48:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.563+0000 7f7924b91640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f79080a2e20 0x7f79080a52e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.563+0000 7f7924b91640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f792010c0d0 0x7f792019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.563+0000 7f7924b91640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79201035c0 0x7f79201988a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.563+0000 7f7924b91640 1 --2- 172.21.15.133:0/2688749547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7920102ca0 0x7f7920198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.563+0000 7f7924b91640 1 -- 172.21.15.133:0/2688749547 >> 172.21.15.133:0/2688749547 conn(0x7f79200fc6b0 msgr2=0x7f79200fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.563+0000 7f7924b91640 1 -- 172.21.15.133:0/2688749547 shutdown_connections 2024-09-11T23:48:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.563+0000 7f7924b91640 1 -- 172.21.15.133:0/2688749547 wait complete. 2024-09-11T23:48:20.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.564+0000 7f22cb7fe640 10 client.27989 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:20.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.564+0000 7f22eb7fe640 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5647d481ad20 0x5647d481d190 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f113d1d4d49946 server_cookie=bf3a6128269aa08d in_seq=0 out_seq=0 2024-09-11T23:48:20.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.568+0000 7f22cb7fe640 1 -- 172.21.15.133:0/1977946646 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f22e0005030 con 0x5647d481ad20 2024-09-11T23:48:20.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.568+0000 7f22cb7fe640 10 client.27989 handle_command_reply: tid=0 2024-09-11T23:48:20.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.568+0000 7f22f4d18740 1 client.27989 shutdown 2024-09-11T23:48:20.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.569+0000 7f22f4d18740 1 -- 172.21.15.133:0/1977946646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d47ffd70 msgr2=0x5647d480aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.569+0000 7f22f4d18740 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d47ffd70 0x5647d480aa00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f22e0030820 tx=0x7f22e0030f30 comp rx=0 tx=0).stop 2024-09-11T23:48:20.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.570+0000 7f22f4d18740 1 -- 172.21.15.133:0/1977946646 shutdown_connections 2024-09-11T23:48:20.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.570+0000 7f22f4d18740 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5647d481ad20 0x5647d481d190 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.570+0000 7f22f4d18740 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d480af50 0x5647d48084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.570+0000 7f22f4d18740 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d4800690 0x5647d4807d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.570+0000 7f22f4d18740 1 --2- 172.21.15.133:0/1977946646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d47ffd70 0x5647d480aa00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.570+0000 7f22f4d18740 1 -- 172.21.15.133:0/1977946646 >> 172.21.15.133:0/1977946646 conn(0x5647d47fc300 msgr2=0x5647d4801fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:20.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.570+0000 7f22cb7fe640 0 client.27989 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:20.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.571+0000 7f22f4d18740 1 -- 172.21.15.133:0/1977946646 shutdown_connections 2024-09-11T23:48:20.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.571+0000 7f22f4d18740 1 -- 172.21.15.133:0/1977946646 wait complete. 2024-09-11T23:48:20.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.572+0000 7f22f4d18740 20 client.27989 trim_cache size 0 max 16384 2024-09-11T23:48:20.580 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T23:48:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.600+0000 7f22f4d18740 1 Processor -- start 2024-09-11T23:48:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.601+0000 7f22f4d18740 1 -- start start 2024-09-11T23:48:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.601+0000 7f22f4d18740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d47ffd70 0x5647d4822a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.601+0000 7f22f4d18740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 0x5647d4822f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22f4d18740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d480af50 0x5647d4804fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d47ffd70 0x5647d4822a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22f4d18740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22e413ecf0 con 0x5647d47ffd70 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22f4d18740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22e4140830 con 0x5647d480af50 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d47ffd70 0x5647d4822a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57894/0 (socket says 172.21.15.133:57894) 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22f4d18740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f22e413f3c0 con 0x5647d4800690 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22ebfff640 1 -- 172.21.15.133:0/385177203 learned_addr learned my addr 172.21.15.133:0/385177203 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22eb7fe640 1 --2- 172.21.15.133:0/385177203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 0x5647d4822f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22ebfff640 1 -- 172.21.15.133:0/385177203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 msgr2=0x5647d4822f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22ebfff640 1 --2- 172.21.15.133:0/385177203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 0x5647d4822f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22ebfff640 1 -- 172.21.15.133:0/385177203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d480af50 msgr2=0x5647d4804fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22ebfff640 1 --2- 172.21.15.133:0/385177203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d480af50 0x5647d4804fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.602+0000 7f22ebfff640 1 -- 172.21.15.133:0/385177203 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d47fc130 con 0x5647d47ffd70 2024-09-11T23:48:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.603+0000 7f22ebfff640 1 --2- 172.21.15.133:0/385177203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d47ffd70 0x5647d4822a30 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005e00 tx=0x7f22e0030ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6a713104b3882f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:20.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.603+0000 7f22d5ffb640 1 -- 172.21.15.133:0/385177203 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0055070 con 0x5647d47ffd70 2024-09-11T23:48:20.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.603+0000 7f22d5ffb640 1 -- 172.21.15.133:0/385177203 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e0005850 con 0x5647d47ffd70 2024-09-11T23:48:20.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.603+0000 7f22d5ffb640 1 -- 172.21.15.133:0/385177203 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00046c0 con 0x5647d47ffd70 2024-09-11T23:48:20.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.604+0000 7f22f4d18740 1 -- 172.21.15.133:0/385177203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d47ffd70 msgr2=0x5647d4822a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.604+0000 7f22f4d18740 1 --2- 172.21.15.133:0/385177203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d47ffd70 0x5647d4822a30 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005e00 tx=0x7f22e0030ba0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.604+0000 7f22f4d18740 1 -- 172.21.15.133:0/385177203 shutdown_connections 2024-09-11T23:48:20.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.604+0000 7f22f4d18740 1 --2- 172.21.15.133:0/385177203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d480af50 0x5647d4804fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.604+0000 7f22f4d18740 1 --2- 172.21.15.133:0/385177203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d4800690 0x5647d4822f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.604+0000 7f22f4d18740 1 --2- 172.21.15.133:0/385177203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d47ffd70 0x5647d4822a30 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.604+0000 7f22f4d18740 1 -- 172.21.15.133:0/385177203 >> 172.21.15.133:0/385177203 conn(0x5647d47fc300 msgr2=0x5647d4803030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:20.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.604+0000 7f22f4d18740 1 -- 172.21.15.133:0/385177203 shutdown_connections 2024-09-11T23:48:20.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.604+0000 7f22f4d18740 1 -- 172.21.15.133:0/385177203 wait complete. 2024-09-11T23:48:20.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.605+0000 7f22f4d18740 1 -- start start 2024-09-11T23:48:20.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.605+0000 7f22f4d18740 1 Processor -- start 2024-09-11T23:48:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.606+0000 7f22f4d18740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d47ffd70 0x5647d4813000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.606+0000 7f22f4d18740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d4800690 0x5647d4813540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.606+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d47ffd70 0x5647d4813000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.606+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d47ffd70 0x5647d4813000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:48008/0 (socket says 172.21.15.133:48008) 2024-09-11T23:48:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.606+0000 7f22ebfff640 1 -- 172.21.15.133:0/2188164511 learned_addr learned my addr 172.21.15.133:0/2188164511 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.607+0000 7f22eb7fe640 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d4800690 0x5647d4813540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.607+0000 7f22f4d18740 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d480af50 0x5647d480dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.607+0000 7f22f4d18740 1 -- 172.21.15.133:0/2188164511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22e4140160 con 0x5647d480af50 2024-09-11T23:48:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.607+0000 7f22f4d18740 1 -- 172.21.15.133:0/2188164511 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22e413fa90 con 0x5647d4800690 2024-09-11T23:48:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.607+0000 7f22f4d18740 1 -- 172.21.15.133:0/2188164511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f22e413cda0 con 0x5647d47ffd70 2024-09-11T23:48:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.607+0000 7f22f4d18740 10 client.0 _ll_register_callbacks cb 0x7f22e00096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.608+0000 7f22ebfff640 1 -- 172.21.15.133:0/2188164511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d4800690 msgr2=0x5647d4813540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.608+0000 7f22ebfff640 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d4800690 0x5647d4813540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.608+0000 7f22ebfff640 1 -- 172.21.15.133:0/2188164511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d480af50 msgr2=0x5647d480dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.608+0000 7f22ebfff640 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d480af50 0x5647d480dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.608+0000 7f22ebfff640 1 -- 172.21.15.133:0/2188164511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d4805c50 con 0x5647d47ffd70 2024-09-11T23:48:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.608+0000 7f22f4d18740 1 -- 172.21.15.133:0/2188164511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e0005850 con 0x5647d47ffd70 2024-09-11T23:48:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.608+0000 7f22ebfff640 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d47ffd70 0x5647d4813000 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f22e0030cd0 tx=0x7f22e0047f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d85e6d6fdd4f60ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.609+0000 7f22cb7fe640 10 client.27784 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T23:48:20.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.609+0000 7f22cb7fe640 1 -- 172.21.15.133:0/2188164511 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0055040 con 0x5647d47ffd70 2024-09-11T23:48:20.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.609+0000 7f22cb7fe640 1 -- 172.21.15.133:0/2188164511 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f22e002acc0 con 0x5647d47ffd70 2024-09-11T23:48:20.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.609+0000 7f22cb7fe640 1 -- 172.21.15.133:0/2188164511 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f22e00046c0 con 0x5647d47ffd70 2024-09-11T23:48:20.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.609+0000 7f22f4d18740 10 client.27784 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:20.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.609+0000 7f22f4d18740 1 -- 172.21.15.133:0/2188164511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5647d4885d70 con 0x5647d47ffd70 2024-09-11T23:48:20.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.610+0000 7f22cb7fe640 1 -- 172.21.15.133:0/2188164511 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f22e003a030 con 0x5647d47ffd70 2024-09-11T23:48:20.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.610+0000 7f22f4d18740 10 client.27784 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.610+0000 7f22f4d18740 10 client.27784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.610+0000 7f22f4d18740 20 client.27784 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.610+0000 7f22f4d18740 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5647d48860b0 0x5647d4888510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.610+0000 7f22f4d18740 4 client.27784 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:48:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.610+0000 7f22f4d18740 1 -- 172.21.15.133:0/2188164511 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5647d47727f0 con 0x5647d48860b0 2024-09-11T23:48:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.611+0000 7f22eb7fe640 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5647d48860b0 0x5647d4888510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.613+0000 7f22eb7fe640 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5647d48860b0 0x5647d4888510 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d251649072006b7e server_cookie=8913f548564b16f6 in_seq=0 out_seq=0 2024-09-11T23:48:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.613+0000 7f22cb7fe640 10 client.27784 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.614+0000 7f22cb7fe640 1 -- 172.21.15.133:0/2188164511 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5647d47727f0 con 0x5647d48860b0 2024-09-11T23:48:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.614+0000 7f22cb7fe640 10 client.27784 handle_command_reply: tid=0 2024-09-11T23:48:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.614+0000 7f22f4d18740 1 client.27784 shutdown 2024-09-11T23:48:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.614+0000 7f22f4d18740 1 -- 172.21.15.133:0/2188164511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d47ffd70 msgr2=0x5647d4813000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.614+0000 7f22f4d18740 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d47ffd70 0x5647d4813000 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f22e0030cd0 tx=0x7f22e0047f40 comp rx=0 tx=0).stop 2024-09-11T23:48:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.614+0000 7f22f4d18740 1 -- 172.21.15.133:0/2188164511 shutdown_connections 2024-09-11T23:48:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.614+0000 7f22f4d18740 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x5647d48860b0 0x5647d4888510 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.615+0000 7f22cb7fe640 0 client.27784 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.615+0000 7f22f4d18740 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647d480af50 0x5647d480dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.615+0000 7f22f4d18740 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d4800690 0x5647d4813540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.615+0000 7f22f4d18740 1 --2- 172.21.15.133:0/2188164511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647d47ffd70 0x5647d4813000 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.615+0000 7f22f4d18740 1 -- 172.21.15.133:0/2188164511 >> 172.21.15.133:0/2188164511 conn(0x5647d47fc300 msgr2=0x5647d47fabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.615+0000 7f22f4d18740 1 -- 172.21.15.133:0/2188164511 shutdown_connections 2024-09-11T23:48:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.615+0000 7f22f4d18740 1 -- 172.21.15.133:0/2188164511 wait complete. 2024-09-11T23:48:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.615+0000 7f22f4d18740 20 client.27784 trim_cache size 0 max 16384 2024-09-11T23:48:20.617 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:48:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-09-11T23:48:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "c8609a9f-9ffe-4ded-8f9f-11584d843a1c", 2024-09-11T23:48:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-09-11T23:48:20.617 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:48:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.616+0000 7f22f293f640 1 -- 172.21.15.133:0/3626332278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22dc0a2d80 msgr2=0x7f22dc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.616+0000 7f22f293f640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22dc0a2d80 0x7f22dc0a5240 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0637b0 tx=0x7f22ec0681c0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.617+0000 7f22f293f640 1 -- 172.21.15.133:0/3626332278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22e4095f70 msgr2=0x7f22e412c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.617+0000 7f22f293f640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22e4095f70 0x7f22e412c380 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009fd0 tx=0x7f22d800b870 comp rx=0 tx=0).stop 2024-09-11T23:48:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.617+0000 7f22f293f640 1 -- 172.21.15.133:0/3626332278 shutdown_connections 2024-09-11T23:48:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.617+0000 7f22f293f640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f22dc0a2d80 0x7f22dc0a5240 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.617+0000 7f22f293f640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e409b680 0x7f22e4130710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.617+0000 7f22f293f640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22e4095f70 0x7f22e412c380 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.617+0000 7f22f293f640 1 --2- 172.21.15.133:0/3626332278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22e40955d0 0x7f22e412be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.617+0000 7f22f293f640 1 -- 172.21.15.133:0/3626332278 >> 172.21.15.133:0/3626332278 conn(0x7f22e4091270 msgr2=0x7f22e4092980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.618+0000 7f22f293f640 1 -- 172.21.15.133:0/3626332278 shutdown_connections 2024-09-11T23:48:20.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.618+0000 7f22f293f640 1 -- 172.21.15.133:0/3626332278 wait complete. 2024-09-11T23:48:20.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.737+0000 7fe16738d640 1 -- 172.21.15.133:0/3213595004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe160102d00 msgr2=0x7fe1601030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.737+0000 7fe16738d640 1 --2- 172.21.15.133:0/3213595004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe160102d00 0x7fe1601030e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009900 tx=0x7fe15c02a480 comp rx=0 tx=0).stop 2024-09-11T23:48:20.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.737+0000 7fe16738d640 1 -- 172.21.15.133:0/3213595004 shutdown_connections 2024-09-11T23:48:20.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.737+0000 7fe16738d640 1 --2- 172.21.15.133:0/3213595004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe16010c0d0 0x7fe16010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.737+0000 7fe16738d640 1 --2- 172.21.15.133:0/3213595004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe160103620 0x7fe16010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.737+0000 7fe16738d640 1 --2- 172.21.15.133:0/3213595004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe160102d00 0x7fe1601030e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.737+0000 7fe16738d640 1 -- 172.21.15.133:0/3213595004 >> 172.21.15.133:0/3213595004 conn(0x7fe160077990 msgr2=0x7fe1600fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:20.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.738+0000 7fe16738d640 1 -- 172.21.15.133:0/3213595004 shutdown_connections 2024-09-11T23:48:20.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.738+0000 7fe16738d640 1 -- 172.21.15.133:0/3213595004 wait complete. 2024-09-11T23:48:20.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe16738d640 1 Processor -- start 2024-09-11T23:48:20.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe16738d640 1 -- start start 2024-09-11T23:48:20.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe16738d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe160102d00 0x7fe160100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe16738d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160103620 0x7fe160100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe16738d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16010c0d0 0x7fe1601013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe16738d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe160111eb0 con 0x7fe160103620 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe16738d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe160111d30 con 0x7fe16010c0d0 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe16738d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe160112030 con 0x7fe160102d00 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe16592a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16010c0d0 0x7fe1601013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe16592a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16010c0d0 0x7fe1601013d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:52238/0 (socket says 172.21.15.133:52238) 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe16592a640 1 -- 172.21.15.133:0/2292845227 learned_addr learned my addr 172.21.15.133:0/2292845227 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.739+0000 7fe165129640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe160102d00 0x7fe160100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.740+0000 7fe164928640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160103620 0x7fe160100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.740+0000 7fe16592a640 1 -- 172.21.15.133:0/2292845227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe160102d00 msgr2=0x7fe160100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.740+0000 7fe16592a640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe160102d00 0x7fe160100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.740+0000 7fe16592a640 1 -- 172.21.15.133:0/2292845227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160103620 msgr2=0x7fe160100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.740+0000 7fe16592a640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160103620 0x7fe160100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:20.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.740+0000 7fe16592a640 1 -- 172.21.15.133:0/2292845227 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1601a6840 con 0x7fe16010c0d0 2024-09-11T23:48:20.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.740+0000 7fe164928640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160103620 0x7fe160100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:48:20.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.740+0000 7fe16592a640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16010c0d0 0x7fe1601013d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe15800eb10 tx=0x7fe15800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:20.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.740+0000 7fe1567fc640 1 -- 172.21.15.133:0/2292845227 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158004b20 con 0x7fe16010c0d0 2024-09-11T23:48:20.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.740+0000 7fe16738d640 1 -- 172.21.15.133:0/2292845227 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1601a6b30 con 0x7fe16010c0d0 2024-09-11T23:48:20.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.740+0000 7fe1567fc640 1 -- 172.21.15.133:0/2292845227 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe158004cc0 con 0x7fe16010c0d0 2024-09-11T23:48:20.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.741+0000 7fe16738d640 1 -- 172.21.15.133:0/2292845227 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1601a7080 con 0x7fe16010c0d0 2024-09-11T23:48:20.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.741+0000 7fe1567fc640 1 -- 172.21.15.133:0/2292845227 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158010760 con 0x7fe16010c0d0 2024-09-11T23:48:20.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.742+0000 7fe16738d640 1 -- 172.21.15.133:0/2292845227 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1601030e0 con 0x7fe16010c0d0 2024-09-11T23:48:20.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.744+0000 7fe1567fc640 1 -- 172.21.15.133:0/2292845227 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe158005020 con 0x7fe16010c0d0 2024-09-11T23:48:20.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.745+0000 7fe1567fc640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe14c0a2e20 0x7fe14c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:20.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.745+0000 7fe1567fc640 1 -- 172.21.15.133:0/2292845227 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fe1580cfa20 con 0x7fe16010c0d0 2024-09-11T23:48:20.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.745+0000 7fe165129640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe14c0a2e20 0x7fe14c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.746+0000 7fe165129640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe14c0a2e20 0x7fe14c0a52e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe15c005ab0 tx=0x7fe15c005a20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:20.752+0000 7fe1567fc640 1 -- 172.21.15.133:0/2292845227 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe15809c680 con 0x7fe16010c0d0 2024-09-11T23:48:20.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:20 smithi133 ceph-mon[31430]: pgmap v363: 97 pgs: 97 active+clean; 60 GiB data, 245 GiB used, 828 GiB / 1.0 TiB avail; 49 KiB/s rd, 216 MiB/s wr, 1.72k op/s 2024-09-11T23:48:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.072+0000 7fe16738d640 1 -- 172.21.15.133:0/2292845227 --> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe1600fd6f0 con 0x7fe14c0a2e20 2024-09-11T23:48:21.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.093+0000 7fe1567fc640 1 -- 172.21.15.133:0/2292845227 <== mgr.15255 v2:172.21.15.133:6800/2757204896 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe1600fd6f0 con 0x7fe14c0a2e20 2024-09-11T23:48:21.094 INFO:teuthology.orchestra.run.smithi133.stdout:/ 1m 6m3h 2024-09-11T23:48:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 -- 172.21.15.133:0/2292845227 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe14c0a2e20 msgr2=0x7fe14c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe14c0a2e20 0x7fe14c0a52e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe15c005ab0 tx=0x7fe15c005a20 comp rx=0 tx=0).stop 2024-09-11T23:48:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 -- 172.21.15.133:0/2292845227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16010c0d0 msgr2=0x7fe1601013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16010c0d0 0x7fe1601013d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe15800eb10 tx=0x7fe15800c2d0 comp rx=0 tx=0).stop 2024-09-11T23:48:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 -- 172.21.15.133:0/2292845227 shutdown_connections 2024-09-11T23:48:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fe14c0a2e20 0x7fe14c0a52e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16010c0d0 0x7fe1601013d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160103620 0x7fe160100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 --2- 172.21.15.133:0/2292845227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe160102d00 0x7fe160100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 -- 172.21.15.133:0/2292845227 >> 172.21.15.133:0/2292845227 conn(0x7fe160077990 msgr2=0x7fe1600fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:21.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 -- 172.21.15.133:0/2292845227 shutdown_connections 2024-09-11T23:48:21.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:21.096+0000 7fe16738d640 1 -- 172.21.15.133:0/2292845227 wait complete. 2024-09-11T23:48:21.112 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-11T23:48:21.121 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-11T23:48:21.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:20 smithi138 ceph-mon[35142]: pgmap v363: 97 pgs: 97 active+clean; 60 GiB data, 245 GiB used, 828 GiB / 1.0 TiB avail; 49 KiB/s rd, 216 MiB/s wr, 1.72k op/s 2024-09-11T23:48:21.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:20 smithi178 ceph-mon[35882]: pgmap v363: 97 pgs: 97 active+clean; 60 GiB data, 245 GiB used, 828 GiB / 1.0 TiB avail; 49 KiB/s rd, 216 MiB/s wr, 1.72k op/s 2024-09-11T23:48:21.568 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:48:21.955 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:22.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:21 smithi138 ceph-mon[35142]: from='client.18453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:22.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:21 smithi138 ceph-mon[35142]: scrub queued for path: / 2024-09-11T23:48:22.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:21 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:22.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:21 smithi138 ceph-mon[35142]: scrub summary: active paths [/] 2024-09-11T23:48:22.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:21 smithi138 ceph-mon[35142]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:22.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:21 smithi133 ceph-mon[31430]: from='client.18453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:22.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:21 smithi133 ceph-mon[31430]: scrub queued for path: / 2024-09-11T23:48:22.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:21 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:22.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:21 smithi133 ceph-mon[31430]: scrub summary: active paths [/] 2024-09-11T23:48:22.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:21 smithi133 ceph-mon[31430]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:22.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:21 smithi178 ceph-mon[35882]: from='client.18453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:22.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:21 smithi178 ceph-mon[35882]: scrub queued for path: / 2024-09-11T23:48:22.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:21 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:22.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:21 smithi178 ceph-mon[35882]: scrub summary: active paths [/] 2024-09-11T23:48:22.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:21 smithi178 ceph-mon[35882]: scrub summary: idle+waiting paths [/] 2024-09-11T23:48:22.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.451+0000 7feacca31640 1 -- 172.21.15.133:0/1926989369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feac8102d30 msgr2=0x7feac8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:22.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.451+0000 7feacca31640 1 --2- 172.21.15.133:0/1926989369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feac8102d30 0x7feac8103110 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7feab8009930 tx=0x7feab802a420 comp rx=0 tx=0).stop 2024-09-11T23:48:22.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.451+0000 7feacca31640 1 -- 172.21.15.133:0/1926989369 shutdown_connections 2024-09-11T23:48:22.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.451+0000 7feacca31640 1 --2- 172.21.15.133:0/1926989369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac81082a0 0x7feac810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.451+0000 7feacca31640 1 --2- 172.21.15.133:0/1926989369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feac81036e0 0x7feac8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.451+0000 7feacca31640 1 --2- 172.21.15.133:0/1926989369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feac8102d30 0x7feac8103110 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.451+0000 7feacca31640 1 -- 172.21.15.133:0/1926989369 >> 172.21.15.133:0/1926989369 conn(0x7feac80fc7d0 msgr2=0x7feac80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:22.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.451+0000 7feacca31640 1 -- 172.21.15.133:0/1926989369 shutdown_connections 2024-09-11T23:48:22.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.451+0000 7feacca31640 1 -- 172.21.15.133:0/1926989369 wait complete. 2024-09-11T23:48:22.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feacca31640 1 Processor -- start 2024-09-11T23:48:22.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feacca31640 1 -- start start 2024-09-11T23:48:22.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feacca31640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feac8102d30 0x7feac81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feacca31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac81036e0 0x7feac8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feacca31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feac81082a0 0x7feac819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feacca31640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feac81120a0 con 0x7feac81082a0 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feacca31640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feac8111f20 con 0x7feac81036e0 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feacca31640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feac8112220 con 0x7feac8102d30 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feac6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feac81082a0 0x7feac819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feac6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feac81082a0 0x7feac819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40656/0 (socket says 172.21.15.133:40656) 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feac6d9d640 1 -- 172.21.15.133:0/492205134 learned_addr learned my addr 172.21.15.133:0/492205134 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feac5d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac81036e0 0x7feac8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feac659c640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feac8102d30 0x7feac81985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feac6d9d640 1 -- 172.21.15.133:0/492205134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feac8102d30 msgr2=0x7feac81985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feac6d9d640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feac8102d30 0x7feac81985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feac6d9d640 1 -- 172.21.15.133:0/492205134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac81036e0 msgr2=0x7feac8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feac6d9d640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac81036e0 0x7feac8198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feac6d9d640 1 -- 172.21.15.133:0/492205134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac819d590 con 0x7feac81082a0 2024-09-11T23:48:22.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.452+0000 7feac5d9b640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac81036e0 0x7feac8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:22.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.453+0000 7feac6d9d640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feac81082a0 0x7feac819ceb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7feac000eb10 tx=0x7feac000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:22.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.453+0000 7feab77fe640 1 -- 172.21.15.133:0/492205134 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0004b20 con 0x7feac81082a0 2024-09-11T23:48:22.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.453+0000 7feab77fe640 1 -- 172.21.15.133:0/492205134 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac0004cc0 con 0x7feac81082a0 2024-09-11T23:48:22.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.453+0000 7feacca31640 1 -- 172.21.15.133:0/492205134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac819d880 con 0x7feac81082a0 2024-09-11T23:48:22.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.453+0000 7feab77fe640 1 -- 172.21.15.133:0/492205134 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0010760 con 0x7feac81082a0 2024-09-11T23:48:22.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.453+0000 7feacca31640 1 -- 172.21.15.133:0/492205134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac81a5160 con 0x7feac81082a0 2024-09-11T23:48:22.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feab77fe640 1 -- 172.21.15.133:0/492205134 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7feac0005020 con 0x7feac81082a0 2024-09-11T23:48:22.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feacee0a740 1 Processor -- start 2024-09-11T23:48:22.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feacee0a740 1 -- start start 2024-09-11T23:48:22.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feacee0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae04a0 0x563061ae0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feacee0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 0x563061aeb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feacee0a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061aeda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feacee0a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563061a462e0 con 0x563061aeb680 2024-09-11T23:48:22.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feacee0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5630619edce0 con 0x563061ae0dc0 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feacee0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563061a422a0 con 0x563061ae04a0 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feac5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061aeda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feac5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061aeda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40658/0 (socket says 172.21.15.133:40658) 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feac659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 0x563061aeb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feac5d9b640 1 -- 172.21.15.133:0/3887656969 learned_addr learned my addr 172.21.15.133:0/3887656969 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.456+0000 7feac5d9b640 1 --2- 172.21.15.133:0/3887656969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae04a0 0x563061ae0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feab77fe640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feab00a2cf0 0x7feab00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feac5d9b640 1 -- 172.21.15.133:0/3887656969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae04a0 msgr2=0x563061ae0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feac5d9b640 1 --2- 172.21.15.133:0/3887656969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae04a0 0x563061ae0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feac5d9b640 1 -- 172.21.15.133:0/3887656969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 msgr2=0x563061aeb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feac5d9b640 1 --2- 172.21.15.133:0/3887656969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 0x563061aeb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feac5d9b640 1 -- 172.21.15.133:0/3887656969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630619d2440 con 0x563061aeb680 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feac659c640 1 --2- 172.21.15.133:0/3887656969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 0x563061aeb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:22.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feac6d9d640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feab00a2cf0 0x7feab00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feab77fe640 1 -- 172.21.15.133:0/492205134 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7feac00cf9b0 con 0x7feac81082a0 2024-09-11T23:48:22.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feac5d9b640 1 --2- 172.21.15.133:0/3887656969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061aeda70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7feabc011aa0 tx=0x7feabc00aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc6293cb665a912 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:22.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feab4ff9640 1 -- 172.21.15.133:0/3887656969 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc00fa90 con 0x563061aeb680 2024-09-11T23:48:22.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feac6d9d640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feab00a2cf0 0x7feab00a51b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feac0098370 tx=0x7feac006ded0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:22.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feab4ff9640 1 -- 172.21.15.133:0/3887656969 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feabc00fc30 con 0x563061aeb680 2024-09-11T23:48:22.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.457+0000 7feab4ff9640 1 -- 172.21.15.133:0/3887656969 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc003ca0 con 0x563061aeb680 2024-09-11T23:48:22.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.458+0000 7feacee0a740 1 -- 172.21.15.133:0/3887656969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 msgr2=0x563061aeda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.458+0000 7feacee0a740 1 --2- 172.21.15.133:0/3887656969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061aeda70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7feabc011aa0 tx=0x7feabc00aa30 comp rx=0 tx=0).stop 2024-09-11T23:48:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.458+0000 7feacee0a740 1 -- 172.21.15.133:0/3887656969 shutdown_connections 2024-09-11T23:48:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.458+0000 7feacee0a740 1 --2- 172.21.15.133:0/3887656969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061aeda70 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.458+0000 7feacee0a740 1 --2- 172.21.15.133:0/3887656969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 0x563061aeb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.458+0000 7feacee0a740 1 --2- 172.21.15.133:0/3887656969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae04a0 0x563061ae0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.458+0000 7feacee0a740 1 -- 172.21.15.133:0/3887656969 >> 172.21.15.133:0/3887656969 conn(0x563061adca30 msgr2=0x563061adce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.458+0000 7feacee0a740 1 -- 172.21.15.133:0/3887656969 shutdown_connections 2024-09-11T23:48:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.458+0000 7feacee0a740 1 -- 172.21.15.133:0/3887656969 wait complete. 2024-09-11T23:48:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feacee0a740 1 -- start start 2024-09-11T23:48:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feacee0a740 1 Processor -- start 2024-09-11T23:48:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feacee0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae04a0 0x563061aeb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feacee0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 0x563061ae8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feacee0a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061ae8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feacee0a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563061aee250 con 0x563061aeb680 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feacee0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563061a01050 con 0x563061ae0dc0 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feacee0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563061aee3d0 con 0x563061ae04a0 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feacee0a740 10 client.0 _ll_register_callbacks cb 0x563061a41160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feac5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061ae8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feac659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 0x563061ae8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feac5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061ae8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40660/0 (socket says 172.21.15.133:40660) 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feac659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 0x563061ae8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:50070/0 (socket says 172.21.15.133:50070) 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feac5d9b640 1 -- 172.21.15.133:0/2863353255 learned_addr learned my addr 172.21.15.133:0/2863353255 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feac5d9b640 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae04a0 0x563061aeb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feac659c640 1 -- 172.21.15.133:0/2863353255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae04a0 msgr2=0x563061aeb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:22.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feac659c640 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae04a0 0x563061aeb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feac659c640 1 -- 172.21.15.133:0/2863353255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 msgr2=0x563061ae8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feac659c640 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061ae8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.459+0000 7feac659c640 1 -- 172.21.15.133:0/2863353255 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630617b73c0 con 0x563061ae0dc0 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feac659c640 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 0x563061ae8330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feab8009900 tx=0x7feab8031a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3783957be1886616 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feaadffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feaadffb640 1 -- 172.21.15.133:0/2863353255 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8035040 con 0x563061ae0dc0 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feaadffb640 1 -- 172.21.15.133:0/2863353255 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7feab8004240 con 0x563061ae0dc0 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feacee0a740 1 -- 172.21.15.133:0/2863353255 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feabc00fc30 con 0x563061ae0dc0 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feaadffb640 1 -- 172.21.15.133:0/2863353255 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feab8004440 con 0x563061ae0dc0 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feacee0a740 10 client.28010 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feacee0a740 1 -- 172.21.15.133:0/2863353255 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563061af9cb0 con 0x563061ae0dc0 2024-09-11T23:48:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feaadffb640 1 -- 172.21.15.133:0/2863353255 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feabc00fc30 con 0x563061ae0dc0 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feaadffb640 1 -- 172.21.15.133:0/2863353255 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7feab8004be0 con 0x563061ae0dc0 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feacee0a740 10 client.28010 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feacee0a740 10 client.28010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feacee0a740 20 client.28010 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feacee0a740 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563061afb0c0 0x563061afd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.460+0000 7feacee0a740 4 client.28010 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.461+0000 7feacee0a740 1 -- 172.21.15.133:0/2863353255 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563061a53850 con 0x563061afb0c0 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.461+0000 7feac5d9b640 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563061afb0c0 0x563061afd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.461+0000 7feaadffb640 10 client.28010 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.461+0000 7feac5d9b640 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563061afb0c0 0x563061afd4f0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5af698b77b6f6f72 server_cookie=eb45d295bcf5232e in_seq=0 out_seq=0 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feaadffb640 1 -- 172.21.15.133:0/2863353255 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563061a53850 con 0x563061afb0c0 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feaadffb640 10 client.28010 handle_command_reply: tid=0 2024-09-11T23:48:22.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 1 client.28010 shutdown 2024-09-11T23:48:22.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 1 -- 172.21.15.133:0/2863353255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 msgr2=0x563061ae8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:22.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 0x563061ae8330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feab8009900 tx=0x7feab8031a80 comp rx=0 tx=0).stop 2024-09-11T23:48:22.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 1 -- 172.21.15.133:0/2863353255 shutdown_connections 2024-09-11T23:48:22.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563061afb0c0 0x563061afd4f0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061ae8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae0dc0 0x563061ae8330 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feaadffb640 0 client.28010 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:22.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 1 --2- 172.21.15.133:0/2863353255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae04a0 0x563061aeb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 1 -- 172.21.15.133:0/2863353255 >> 172.21.15.133:0/2863353255 conn(0x563061adca30 msgr2=0x563061ae2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:22.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 1 -- 172.21.15.133:0/2863353255 shutdown_connections 2024-09-11T23:48:22.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 1 -- 172.21.15.133:0/2863353255 wait complete. 2024-09-11T23:48:22.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.462+0000 7feacee0a740 20 client.28010 trim_cache size 0 max 16384 2024-09-11T23:48:22.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.479+0000 7feacee0a740 1 Processor -- start 2024-09-11T23:48:22.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.479+0000 7feacee0a740 1 -- start start 2024-09-11T23:48:22.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.479+0000 7feacee0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae04a0 0x563061ae3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feacee0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae0dc0 0x563061af74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feacee0a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061b02dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feacee0a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feac81ad790 con 0x563061aeb680 2024-09-11T23:48:22.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feacee0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feac81af2d0 con 0x563061ae04a0 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feacee0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feac81ade60 con 0x563061ae0dc0 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061b02dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061b02dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40668/0 (socket says 172.21.15.133:40668) 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac5d9b640 1 -- 172.21.15.133:0/529180602 learned_addr learned my addr 172.21.15.133:0/529180602 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac659c640 1 --2- 172.21.15.133:0/529180602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae0dc0 0x563061af74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac5d9b640 1 --2- 172.21.15.133:0/529180602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae04a0 0x563061ae3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac659c640 1 -- 172.21.15.133:0/529180602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae04a0 msgr2=0x563061ae3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac659c640 1 --2- 172.21.15.133:0/529180602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae04a0 0x563061ae3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac659c640 1 -- 172.21.15.133:0/529180602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 msgr2=0x563061b02dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac659c640 1 --2- 172.21.15.133:0/529180602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061b02dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac659c640 1 -- 172.21.15.133:0/529180602 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563061ae6880 con 0x563061ae0dc0 2024-09-11T23:48:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac5d9b640 1 --2- 172.21.15.133:0/529180602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061b02dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:22.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feac659c640 1 --2- 172.21.15.133:0/529180602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae0dc0 0x563061af74b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feab80035f0 tx=0x7feab80039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e52efd624b42218c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:22.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.480+0000 7feaaffff640 1 -- 172.21.15.133:0/529180602 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8035070 con 0x563061ae0dc0 2024-09-11T23:48:22.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.481+0000 7feaaffff640 1 -- 172.21.15.133:0/529180602 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab80054e0 con 0x563061ae0dc0 2024-09-11T23:48:22.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.481+0000 7feaaffff640 1 -- 172.21.15.133:0/529180602 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8000e60 con 0x563061ae0dc0 2024-09-11T23:48:22.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.481+0000 7feacee0a740 1 -- 172.21.15.133:0/529180602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae0dc0 msgr2=0x563061af74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:22.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.481+0000 7feacee0a740 1 --2- 172.21.15.133:0/529180602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae0dc0 0x563061af74b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feab80035f0 tx=0x7feab80039d0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.481+0000 7feacee0a740 1 -- 172.21.15.133:0/529180602 shutdown_connections 2024-09-11T23:48:22.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.481+0000 7feacee0a740 1 --2- 172.21.15.133:0/529180602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061aeb680 0x563061b02dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.481+0000 7feacee0a740 1 --2- 172.21.15.133:0/529180602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae0dc0 0x563061af74b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.481+0000 7feacee0a740 1 --2- 172.21.15.133:0/529180602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061ae04a0 0x563061ae3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.481+0000 7feacee0a740 1 -- 172.21.15.133:0/529180602 >> 172.21.15.133:0/529180602 conn(0x563061adca30 msgr2=0x563061aea3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.481+0000 7feacee0a740 1 -- 172.21.15.133:0/529180602 shutdown_connections 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.481+0000 7feacee0a740 1 -- 172.21.15.133:0/529180602 wait complete. 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feacee0a740 1 -- start start 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feacee0a740 1 Processor -- start 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feacee0a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061ae04a0 0x563061af3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feacee0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae0dc0 0x563061aeee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feacee0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061aeb680 0x563061aef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feacee0a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feac81aec00 con 0x563061ae04a0 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feacee0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feac81ae530 con 0x563061aeb680 2024-09-11T23:48:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feacee0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feac81ab790 con 0x563061ae0dc0 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feacee0a740 10 client.0 _ll_register_callbacks cb 0x7feab8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feac5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061ae04a0 0x563061af3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feac5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061ae04a0 0x563061af3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40672/0 (socket says 172.21.15.133:40672) 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feac5d9b640 1 -- 172.21.15.133:0/3640193377 learned_addr learned my addr 172.21.15.133:0/3640193377 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feac659c640 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae0dc0 0x563061aeee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feac5d9b640 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061aeb680 0x563061aef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feac5d9b640 1 -- 172.21.15.133:0/3640193377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae0dc0 msgr2=0x563061aeee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feac5d9b640 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae0dc0 0x563061aeee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feac5d9b640 1 -- 172.21.15.133:0/3640193377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061aeb680 msgr2=0x563061aef3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feac5d9b640 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061aeb680 0x563061aef3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.482+0000 7feac5d9b640 1 -- 172.21.15.133:0/3640193377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563061aef960 con 0x563061ae04a0 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.483+0000 7feac5d9b640 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061ae04a0 0x563061af3b30 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7feabc00f850 tx=0x7feabc020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28dd83432a5df5f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.483+0000 7feaadffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.483+0000 7feaadffb640 1 -- 172.21.15.133:0/3640193377 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc031070 con 0x563061ae04a0 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.483+0000 7feaadffb640 1 -- 172.21.15.133:0/3640193377 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7feabc011020 con 0x563061ae04a0 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.483+0000 7feacee0a740 1 -- 172.21.15.133:0/3640193377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feab80054e0 con 0x563061ae04a0 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.483+0000 7feaadffb640 1 -- 172.21.15.133:0/3640193377 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feabc040050 con 0x563061ae04a0 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.483+0000 7feacee0a740 10 client.18483 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.483+0000 7feacee0a740 1 -- 172.21.15.133:0/3640193377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563061b65b70 con 0x563061ae04a0 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.483+0000 7feaadffb640 1 -- 172.21.15.133:0/3640193377 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab80054e0 con 0x563061ae04a0 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.483+0000 7feaadffb640 1 -- 172.21.15.133:0/3640193377 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7feabc037070 con 0x563061ae04a0 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feacee0a740 10 client.18483 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feacee0a740 10 client.18483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feacee0a740 20 client.18483 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feacee0a740 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563061b660b0 0x563061b684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feacee0a740 4 client.18483 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feacee0a740 1 -- 172.21.15.133:0/3640193377 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563061a4dd60 con 0x563061b660b0 2024-09-11T23:48:22.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feac659c640 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563061b660b0 0x563061b684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feaadffb640 10 client.18483 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feac659c640 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563061b660b0 0x563061b684a0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c51ba932a1971ac5 server_cookie=f642a2d8c89bec77 in_seq=0 out_seq=0 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feaadffb640 1 -- 172.21.15.133:0/3640193377 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563061a4dd60 con 0x563061b660b0 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feaadffb640 10 client.18483 handle_command_reply: tid=0 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feacee0a740 1 client.18483 shutdown 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feacee0a740 1 -- 172.21.15.133:0/3640193377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061ae04a0 msgr2=0x563061af3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.484+0000 7feacee0a740 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061ae04a0 0x563061af3b30 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7feabc00f850 tx=0x7feabc020040 comp rx=0 tx=0).stop 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacee0a740 1 -- 172.21.15.133:0/3640193377 shutdown_connections 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacee0a740 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x563061b660b0 0x563061b684a0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacee0a740 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563061aeb680 0x563061aef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacee0a740 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563061ae0dc0 0x563061aeee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacee0a740 1 --2- 172.21.15.133:0/3640193377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563061ae04a0 0x563061af3b30 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacee0a740 1 -- 172.21.15.133:0/3640193377 >> 172.21.15.133:0/3640193377 conn(0x563061adca30 msgr2=0x563061ae1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feaadffb640 0 client.18483 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacee0a740 1 -- 172.21.15.133:0/3640193377 shutdown_connections 2024-09-11T23:48:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacee0a740 1 -- 172.21.15.133:0/3640193377 wait complete. 2024-09-11T23:48:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacee0a740 20 client.18483 trim_cache size 0 max 16384 2024-09-11T23:48:22.492 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:48:22.492 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:48:22.492 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-09-11T23:48:22.492 INFO:teuthology.orchestra.run.smithi133.stdout: "c8609a9f-9ffe-4ded-8f9f-11584d843a1c": { 2024-09-11T23:48:22.493 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-09-11T23:48:22.493 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "c8609a9f-9ffe-4ded-8f9f-11584d843a1c", 2024-09-11T23:48:22.493 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2024-09-11T23:48:22.493 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:48:22.493 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:48:22.493 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:48:22.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacca31640 1 -- 172.21.15.133:0/492205134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feab00a2cf0 msgr2=0x7feab00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:22.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacca31640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feab00a2cf0 0x7feab00a51b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feac0098370 tx=0x7feac006ded0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacca31640 1 -- 172.21.15.133:0/492205134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feac81082a0 msgr2=0x7feac819ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:22.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacca31640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feac81082a0 0x7feac819ceb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7feac000eb10 tx=0x7feac000c2d0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacca31640 1 -- 172.21.15.133:0/492205134 shutdown_connections 2024-09-11T23:48:22.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacca31640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7feab00a2cf0 0x7feab00a51b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacca31640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feac81082a0 0x7feac819ceb0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacca31640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac81036e0 0x7feac8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.485+0000 7feacca31640 1 --2- 172.21.15.133:0/492205134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feac8102d30 0x7feac81985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:22.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.486+0000 7feacca31640 1 -- 172.21.15.133:0/492205134 >> 172.21.15.133:0/492205134 conn(0x7feac80fc7d0 msgr2=0x7feac8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:22.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.486+0000 7feacca31640 1 -- 172.21.15.133:0/492205134 shutdown_connections 2024-09-11T23:48:22.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:22.486+0000 7feacca31640 1 -- 172.21.15.133:0/492205134 wait complete. 2024-09-11T23:48:23.097 INFO:tasks.cephfs.filesystem:scrub status for tag:c8609a9f-9ffe-4ded-8f9f-11584d843a1c - {'path': '/', 'tag': 'c8609a9f-9ffe-4ded-8f9f-11584d843a1c', 'options': 'recursive,force'} 2024-09-11T23:48:23.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:22 smithi138 ceph-mon[35142]: from='client.27998 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:23.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:22 smithi138 ceph-mon[35142]: pgmap v364: 97 pgs: 97 active+clean; 60 GiB data, 243 GiB used, 829 GiB / 1.0 TiB avail; 50 KiB/s rd, 230 MiB/s wr, 1.82k op/s 2024-09-11T23:48:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:22 smithi133 ceph-mon[31430]: from='client.27998 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:22 smithi133 ceph-mon[31430]: pgmap v364: 97 pgs: 97 active+clean; 60 GiB data, 243 GiB used, 829 GiB / 1.0 TiB avail; 50 KiB/s rd, 230 MiB/s wr, 1.82k op/s 2024-09-11T23:48:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:22 smithi178 ceph-mon[35882]: from='client.27998 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:48:23.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:22 smithi178 ceph-mon[35882]: pgmap v364: 97 pgs: 97 active+clean; 60 GiB data, 243 GiB used, 829 GiB / 1.0 TiB avail; 50 KiB/s rd, 230 MiB/s wr, 1.82k op/s 2024-09-11T23:48:25.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:24 smithi138 ceph-mon[35142]: pgmap v365: 97 pgs: 97 active+clean; 60 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 48 KiB/s rd, 187 MiB/s wr, 1.44k op/s 2024-09-11T23:48:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:24 smithi133 ceph-mon[31430]: pgmap v365: 97 pgs: 97 active+clean; 60 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 48 KiB/s rd, 187 MiB/s wr, 1.44k op/s 2024-09-11T23:48:25.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:24 smithi178 ceph-mon[35882]: pgmap v365: 97 pgs: 97 active+clean; 60 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 48 KiB/s rd, 187 MiB/s wr, 1.44k op/s 2024-09-11T23:48:26.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:25 smithi138 ceph-mon[35142]: scrub summary: idle 2024-09-11T23:48:26.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:25 smithi133 ceph-mon[31430]: scrub summary: idle 2024-09-11T23:48:26.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:25 smithi178 ceph-mon[35882]: scrub summary: idle 2024-09-11T23:48:27.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:26 smithi138 ceph-mon[35142]: pgmap v366: 97 pgs: 97 active+clean; 60 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 41 KiB/s rd, 153 MiB/s wr, 1.19k op/s 2024-09-11T23:48:27.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:26 smithi133 ceph-mon[31430]: pgmap v366: 97 pgs: 97 active+clean; 60 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 41 KiB/s rd, 153 MiB/s wr, 1.19k op/s 2024-09-11T23:48:27.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:26 smithi178 ceph-mon[35882]: pgmap v366: 97 pgs: 97 active+clean; 60 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 41 KiB/s rd, 153 MiB/s wr, 1.19k op/s 2024-09-11T23:48:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:28 smithi138 ceph-mon[35142]: pgmap v367: 97 pgs: 97 active+clean; 60 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 41 KiB/s rd, 144 MiB/s wr, 1.12k op/s 2024-09-11T23:48:29.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:28 smithi133 ceph-mon[31430]: pgmap v367: 97 pgs: 97 active+clean; 60 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 41 KiB/s rd, 144 MiB/s wr, 1.12k op/s 2024-09-11T23:48:29.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:28 smithi178 ceph-mon[35882]: pgmap v367: 97 pgs: 97 active+clean; 60 GiB data, 229 GiB used, 844 GiB / 1.0 TiB avail; 41 KiB/s rd, 144 MiB/s wr, 1.12k op/s 2024-09-11T23:48:31.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:30 smithi138 ceph-mon[35142]: pgmap v368: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 43 KiB/s rd, 106 MiB/s wr, 844 op/s 2024-09-11T23:48:31.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:30 smithi133 ceph-mon[31430]: pgmap v368: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 43 KiB/s rd, 106 MiB/s wr, 844 op/s 2024-09-11T23:48:31.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:30 smithi178 ceph-mon[35882]: pgmap v368: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 43 KiB/s rd, 106 MiB/s wr, 844 op/s 2024-09-11T23:48:33.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:32 smithi138 ceph-mon[35142]: pgmap v369: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 259 op/s 2024-09-11T23:48:33.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:32 smithi133 ceph-mon[31430]: pgmap v369: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 259 op/s 2024-09-11T23:48:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:32 smithi178 ceph-mon[35882]: pgmap v369: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 259 op/s 2024-09-11T23:48:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:34 smithi138 ceph-mon[35142]: pgmap v370: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 18 KiB/s rd, 229 KiB/s wr, 178 op/s 2024-09-11T23:48:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:34 smithi133 ceph-mon[31430]: pgmap v370: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 18 KiB/s rd, 229 KiB/s wr, 178 op/s 2024-09-11T23:48:35.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:34 smithi178 ceph-mon[35882]: pgmap v370: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 18 KiB/s rd, 229 KiB/s wr, 178 op/s 2024-09-11T23:48:37.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:36 smithi138 ceph-mon[35142]: pgmap v371: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 7.2 KiB/s rd, 233 KiB/s wr, 92 op/s 2024-09-11T23:48:37.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:36 smithi133 ceph-mon[31430]: pgmap v371: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 7.2 KiB/s rd, 233 KiB/s wr, 92 op/s 2024-09-11T23:48:37.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:36 smithi178 ceph-mon[35882]: pgmap v371: 97 pgs: 97 active+clean; 60 GiB data, 225 GiB used, 848 GiB / 1.0 TiB avail; 7.2 KiB/s rd, 233 KiB/s wr, 92 op/s 2024-09-11T23:48:39.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:38 smithi138 ceph-mon[35142]: pgmap v372: 97 pgs: 97 active+clean; 60 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 313 KiB/s wr, 65 op/s 2024-09-11T23:48:39.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:38 smithi133 ceph-mon[31430]: pgmap v372: 97 pgs: 97 active+clean; 60 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 313 KiB/s wr, 65 op/s 2024-09-11T23:48:39.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:38 smithi178 ceph-mon[35882]: pgmap v372: 97 pgs: 97 active+clean; 60 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 313 KiB/s wr, 65 op/s 2024-09-11T23:48:41.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:40 smithi138 ceph-mon[35142]: pgmap v373: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 531 KiB/s wr, 64 op/s 2024-09-11T23:48:41.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:40 smithi133 ceph-mon[31430]: pgmap v373: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 531 KiB/s wr, 64 op/s 2024-09-11T23:48:41.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:40 smithi178 ceph-mon[35882]: pgmap v373: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 531 KiB/s wr, 64 op/s 2024-09-11T23:48:43.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:42 smithi138 ceph-mon[35142]: pgmap v374: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 508 KiB/s wr, 64 op/s 2024-09-11T23:48:43.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:42 smithi133 ceph-mon[31430]: pgmap v374: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 508 KiB/s wr, 64 op/s 2024-09-11T23:48:43.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:42 smithi178 ceph-mon[35882]: pgmap v374: 97 pgs: 97 active+clean; 60 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 508 KiB/s wr, 64 op/s 2024-09-11T23:48:45.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:44 smithi138 ceph-mon[35142]: pgmap v375: 97 pgs: 97 active+clean; 60 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 471 KiB/s wr, 94 op/s 2024-09-11T23:48:45.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:44 smithi133 ceph-mon[31430]: pgmap v375: 97 pgs: 97 active+clean; 60 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 471 KiB/s wr, 94 op/s 2024-09-11T23:48:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:44 smithi178 ceph-mon[35882]: pgmap v375: 97 pgs: 97 active+clean; 60 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 471 KiB/s wr, 94 op/s 2024-09-11T23:48:47.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:46 smithi138 ceph-mon[35142]: pgmap v376: 97 pgs: 97 active+clean; 60 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 438 KiB/s wr, 67 op/s 2024-09-11T23:48:47.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:46 smithi133 ceph-mon[31430]: pgmap v376: 97 pgs: 97 active+clean; 60 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 438 KiB/s wr, 67 op/s 2024-09-11T23:48:47.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:46 smithi178 ceph-mon[35882]: pgmap v376: 97 pgs: 97 active+clean; 60 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 438 KiB/s wr, 67 op/s 2024-09-11T23:48:49.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:48 smithi138 ceph-mon[35142]: pgmap v377: 97 pgs: 97 active+clean; 60 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 427 KiB/s wr, 59 op/s 2024-09-11T23:48:49.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:48 smithi133 ceph-mon[31430]: pgmap v377: 97 pgs: 97 active+clean; 60 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 427 KiB/s wr, 59 op/s 2024-09-11T23:48:49.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:48 smithi178 ceph-mon[35882]: pgmap v377: 97 pgs: 97 active+clean; 60 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 427 KiB/s wr, 59 op/s 2024-09-11T23:48:51.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:50 smithi138 ceph-mon[35142]: pgmap v378: 97 pgs: 97 active+clean; 60 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 347 KiB/s wr, 60 op/s 2024-09-11T23:48:51.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:50 smithi133 ceph-mon[31430]: pgmap v378: 97 pgs: 97 active+clean; 60 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 347 KiB/s wr, 60 op/s 2024-09-11T23:48:51.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:50 smithi178 ceph-mon[35882]: pgmap v378: 97 pgs: 97 active+clean; 60 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 347 KiB/s wr, 60 op/s 2024-09-11T23:48:53.098 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:48:53.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:52 smithi138 ceph-mon[35142]: pgmap v379: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 113 KiB/s wr, 54 op/s 2024-09-11T23:48:53.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:52 smithi133 ceph-mon[31430]: pgmap v379: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 113 KiB/s wr, 54 op/s 2024-09-11T23:48:53.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:52 smithi178 ceph-mon[35882]: pgmap v379: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 113 KiB/s wr, 54 op/s 2024-09-11T23:48:53.360 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:53.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.837+0000 7f663e1af640 1 -- 172.21.15.133:0/1386485018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6638101420 msgr2=0x7f663810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:53.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.837+0000 7f663e1af640 1 --2- 172.21.15.133:0/1386485018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6638101420 0x7f663810d950 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6630009cd0 tx=0x7f663002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.837+0000 7f663e1af640 1 -- 172.21.15.133:0/1386485018 shutdown_connections 2024-09-11T23:48:53.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.837+0000 7f663e1af640 1 --2- 172.21.15.133:0/1386485018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6638101420 0x7f663810d950 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.837+0000 7f663e1af640 1 --2- 172.21.15.133:0/1386485018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6638100a80 0x7f6638100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.837+0000 7f663e1af640 1 --2- 172.21.15.133:0/1386485018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6638106a80 0x7f6638106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.837+0000 7f663e1af640 1 -- 172.21.15.133:0/1386485018 >> 172.21.15.133:0/1386485018 conn(0x7f66380fc7a0 msgr2=0x7f66380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:53.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.837+0000 7f663e1af640 1 -- 172.21.15.133:0/1386485018 shutdown_connections 2024-09-11T23:48:53.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.837+0000 7f663e1af640 1 -- 172.21.15.133:0/1386485018 wait complete. 2024-09-11T23:48:53.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f663e1af640 1 Processor -- start 2024-09-11T23:48:53.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f663e1af640 1 -- start start 2024-09-11T23:48:53.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f663e1af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6638100a80 0x7f663810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f663e1af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6638101420 0x7f663810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f663e1af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6638106a80 0x7f663810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f663e1af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6638111fa0 con 0x7f6638100a80 2024-09-11T23:48:53.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f663e1af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6638111e20 con 0x7f6638106a80 2024-09-11T23:48:53.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f663e1af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6638112120 con 0x7f6638101420 2024-09-11T23:48:53.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f66377fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6638100a80 0x7f663810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f66377fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6638100a80 0x7f663810e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39302/0 (socket says 172.21.15.133:39302) 2024-09-11T23:48:53.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f66377fe640 1 -- 172.21.15.133:0/3076953023 learned_addr learned my addr 172.21.15.133:0/3076953023 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:53.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f6637fff640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6638106a80 0x7f663810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.838+0000 7f6636ffd640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6638101420 0x7f663810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.839+0000 7f66377fe640 1 -- 172.21.15.133:0/3076953023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6638101420 msgr2=0x7f663810e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:53.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.839+0000 7f66377fe640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6638101420 0x7f663810e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.839+0000 7f66377fe640 1 -- 172.21.15.133:0/3076953023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6638106a80 msgr2=0x7f663810ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:53.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.839+0000 7f66377fe640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6638106a80 0x7f663810ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.839+0000 7f66377fe640 1 -- 172.21.15.133:0/3076953023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66381a6820 con 0x7f6638100a80 2024-09-11T23:48:53.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.839+0000 7f66377fe640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6638100a80 0x7f663810e3e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f662800d950 tx=0x7f662800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:53.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.839+0000 7f6634ff9640 1 -- 172.21.15.133:0/3076953023 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628017030 con 0x7f6638100a80 2024-09-11T23:48:53.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.839+0000 7f6634ff9640 1 -- 172.21.15.133:0/3076953023 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6628004020 con 0x7f6638100a80 2024-09-11T23:48:53.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.839+0000 7f663e1af640 1 -- 172.21.15.133:0/3076953023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66381a6b10 con 0x7f6638100a80 2024-09-11T23:48:53.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.839+0000 7f6634ff9640 1 -- 172.21.15.133:0/3076953023 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628005650 con 0x7f6638100a80 2024-09-11T23:48:53.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.839+0000 7f663e1af640 1 -- 172.21.15.133:0/3076953023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66381a7050 con 0x7f6638100a80 2024-09-11T23:48:53.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6634ff9640 1 -- 172.21.15.133:0/3076953023 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f66280041d0 con 0x7f6638100a80 2024-09-11T23:48:53.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6640588740 1 Processor -- start 2024-09-11T23:48:53.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6640588740 1 -- start start 2024-09-11T23:48:53.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6640588740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d38c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6640588740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d397140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6640588740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d399a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6640588740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c53d2f22e0 con 0x55c53d397680 2024-09-11T23:48:53.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6640588740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c53d299ce0 con 0x55c53d38c4a0 2024-09-11T23:48:53.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6640588740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c53d2ee2a0 con 0x55c53d38cdc0 2024-09-11T23:48:53.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6636ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d399a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6637fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d38c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6636ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d399a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39316/0 (socket says 172.21.15.133:39316) 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6637fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d38c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:54122/0 (socket says 172.21.15.133:54122) 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6636ffd640 1 -- 172.21.15.133:0/958518054 learned_addr learned my addr 172.21.15.133:0/958518054 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f6634ff9640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66200a2cf0 0x7f66200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.842+0000 7f66377fe640 1 --2- 172.21.15.133:0/958518054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d397140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f6637fff640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66200a2cf0 0x7f66200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f6636ffd640 1 -- 172.21.15.133:0/958518054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 msgr2=0x55c53d397140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f6636ffd640 1 --2- 172.21.15.133:0/958518054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d397140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f6636ffd640 1 -- 172.21.15.133:0/958518054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 msgr2=0x55c53d38c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f6636ffd640 1 --2- 172.21.15.133:0/958518054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d38c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f6636ffd640 1 -- 172.21.15.133:0/958518054 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c53d27e440 con 0x55c53d397680 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f6637fff640 1 --2- 172.21.15.133:0/958518054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d38c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:53.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f6634ff9640 1 -- 172.21.15.133:0/3076953023 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f66280cecf0 con 0x7f6638100a80 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f6636ffd640 1 --2- 172.21.15.133:0/958518054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d399a70 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f662c00aca0 tx=0x7f662c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39a9f34aa2d3bce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f661dffb640 1 -- 172.21.15.133:0/958518054 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c01d070 con 0x55c53d397680 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f6637fff640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66200a2cf0 0x7f66200a51b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6630004ab0 tx=0x7f663002a8e0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f661dffb640 1 -- 172.21.15.133:0/958518054 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c018b20 con 0x55c53d397680 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.843+0000 7f661dffb640 1 -- 172.21.15.133:0/958518054 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c010c20 con 0x55c53d397680 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.844+0000 7f6640588740 1 -- 172.21.15.133:0/958518054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 msgr2=0x55c53d399a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.844+0000 7f6640588740 1 --2- 172.21.15.133:0/958518054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d399a70 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f662c00aca0 tx=0x7f662c00f790 comp rx=0 tx=0).stop 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.844+0000 7f6640588740 1 -- 172.21.15.133:0/958518054 shutdown_connections 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.844+0000 7f6640588740 1 --2- 172.21.15.133:0/958518054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d399a70 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.844+0000 7f6640588740 1 --2- 172.21.15.133:0/958518054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d397140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.844+0000 7f6640588740 1 --2- 172.21.15.133:0/958518054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d38c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.844+0000 7f6640588740 1 -- 172.21.15.133:0/958518054 >> 172.21.15.133:0/958518054 conn(0x55c53d388a30 msgr2=0x55c53d388e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:53.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.844+0000 7f6640588740 1 -- 172.21.15.133:0/958518054 shutdown_connections 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.844+0000 7f6640588740 1 -- 172.21.15.133:0/958518054 wait complete. 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.844+0000 7f6640588740 1 -- start start 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6640588740 1 Processor -- start 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6640588740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d396f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6640588740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d38cdc0 0x55c53d3941d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6640588740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d397680 0x55c53d394710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6640588740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c53d39a250 con 0x55c53d38cdc0 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6640588740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c53d2ad050 con 0x55c53d38c4a0 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6640588740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c53d39a3d0 con 0x55c53d397680 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6640588740 10 client.0 _ll_register_callbacks cb 0x55c53d2ed160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6636ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d38cdc0 0x55c53d3941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6636ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d38cdc0 0x55c53d3941d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39326/0 (socket says 172.21.15.133:39326) 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6636ffd640 1 -- 172.21.15.133:0/3100862770 learned_addr learned my addr 172.21.15.133:0/3100862770 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:53.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f66377fe640 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d396f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6637fff640 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d397680 0x55c53d394710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6636ffd640 1 -- 172.21.15.133:0/3100862770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d397680 msgr2=0x55c53d394710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6636ffd640 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d397680 0x55c53d394710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6636ffd640 1 -- 172.21.15.133:0/3100862770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 msgr2=0x55c53d396f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6636ffd640 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d396f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.845+0000 7f6636ffd640 1 -- 172.21.15.133:0/3100862770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c53d0633c0 con 0x55c53d38cdc0 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6636ffd640 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d38cdc0 0x55c53d3941d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f662c0107e0 tx=0x7f662c010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473c656bff5ce4e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6616ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6616ffd640 1 -- 172.21.15.133:0/3100862770 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c01d050 con 0x55c53d38cdc0 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6616ffd640 1 -- 172.21.15.133:0/3100862770 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f662c010db0 con 0x55c53d38cdc0 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6640588740 1 -- 172.21.15.133:0/3100862770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662c018b20 con 0x55c53d38cdc0 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6616ffd640 1 -- 172.21.15.133:0/3100862770 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f662c032050 con 0x55c53d38cdc0 2024-09-11T23:48:53.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6640588740 10 client.18495 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6640588740 1 -- 172.21.15.133:0/3100862770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c53d3a5b70 con 0x55c53d38cdc0 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6616ffd640 1 -- 172.21.15.133:0/3100862770 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c018b20 con 0x55c53d38cdc0 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6616ffd640 1 -- 172.21.15.133:0/3100862770 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f662c0046c0 con 0x55c53d38cdc0 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6640588740 10 client.18495 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6640588740 10 client.18495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6640588740 20 client.18495 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6640588740 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c53d3a6f20 0x55c53d3a9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6640588740 4 client.18495 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.846+0000 7f6640588740 1 -- 172.21.15.133:0/3100862770 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c53d2ff850 con 0x55c53d3a6f20 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.847+0000 7f66377fe640 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c53d3a6f20 0x55c53d3a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.847+0000 7f6616ffd640 10 client.18495 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.847+0000 7f66377fe640 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c53d3a6f20 0x55c53d3a9310 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55f03496c284691 server_cookie=9b0a668da37eb9a0 in_seq=0 out_seq=0 2024-09-11T23:48:53.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6616ffd640 1 -- 172.21.15.133:0/3100862770 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c53d2ff850 con 0x55c53d3a6f20 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6616ffd640 10 client.18495 handle_command_reply: tid=0 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 1 client.18495 shutdown 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 1 -- 172.21.15.133:0/3100862770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d38cdc0 msgr2=0x55c53d3941d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d38cdc0 0x55c53d3941d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f662c0107e0 tx=0x7f662c010660 comp rx=0 tx=0).stop 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 1 -- 172.21.15.133:0/3100862770 shutdown_connections 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c53d3a6f20 0x55c53d3a9310 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6616ffd640 0 client.18495 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d397680 0x55c53d394710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d38cdc0 0x55c53d3941d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 1 --2- 172.21.15.133:0/3100862770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d396f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 1 -- 172.21.15.133:0/3100862770 >> 172.21.15.133:0/3100862770 conn(0x55c53d388a30 msgr2=0x55c53d38e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 1 -- 172.21.15.133:0/3100862770 shutdown_connections 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 1 -- 172.21.15.133:0/3100862770 wait complete. 2024-09-11T23:48:53.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.848+0000 7f6640588740 20 client.18495 trim_cache size 0 max 16384 2024-09-11T23:48:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.865+0000 7f6640588740 1 Processor -- start 2024-09-11T23:48:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.865+0000 7f6640588740 1 -- start start 2024-09-11T23:48:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6640588740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d3a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6640588740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d38f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6640588740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d396140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6640588740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f66381af970 con 0x55c53d397680 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6640588740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66381b14b0 con 0x55c53d38c4a0 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6640588740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f66381b0040 con 0x55c53d38cdc0 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6637fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d396140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6637fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d396140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39330/0 (socket says 172.21.15.133:39330) 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6637fff640 1 -- 172.21.15.133:0/255971379 learned_addr learned my addr 172.21.15.133:0/255971379 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6636ffd640 1 --2- 172.21.15.133:0/255971379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d38f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f66377fe640 1 --2- 172.21.15.133:0/255971379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d3a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6637fff640 1 -- 172.21.15.133:0/255971379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 msgr2=0x55c53d38f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6637fff640 1 --2- 172.21.15.133:0/255971379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d38f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6637fff640 1 -- 172.21.15.133:0/255971379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 msgr2=0x55c53d3a34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6637fff640 1 --2- 172.21.15.133:0/255971379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d3a34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6637fff640 1 -- 172.21.15.133:0/255971379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c53d397290 con 0x55c53d397680 2024-09-11T23:48:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f66377fe640 1 --2- 172.21.15.133:0/255971379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d3a34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:48:53.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f6637fff640 1 --2- 172.21.15.133:0/255971379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d396140 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6630009cd0 tx=0x7f6630005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4459c3a531114967 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:53.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f661d7fa640 1 -- 172.21.15.133:0/255971379 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630004510 con 0x55c53d397680 2024-09-11T23:48:53.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f661d7fa640 1 -- 172.21.15.133:0/255971379 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6630036050 con 0x55c53d397680 2024-09-11T23:48:53.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.866+0000 7f661d7fa640 1 -- 172.21.15.133:0/255971379 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663004a690 con 0x55c53d397680 2024-09-11T23:48:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.867+0000 7f6640588740 1 -- 172.21.15.133:0/255971379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 msgr2=0x55c53d396140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.867+0000 7f6640588740 1 --2- 172.21.15.133:0/255971379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d396140 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6630009cd0 tx=0x7f6630005fa0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.867+0000 7f6640588740 1 -- 172.21.15.133:0/255971379 shutdown_connections 2024-09-11T23:48:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.867+0000 7f6640588740 1 --2- 172.21.15.133:0/255971379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d396140 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.867+0000 7f6640588740 1 --2- 172.21.15.133:0/255971379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d38f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.867+0000 7f6640588740 1 --2- 172.21.15.133:0/255971379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d3a34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.867+0000 7f6640588740 1 -- 172.21.15.133:0/255971379 >> 172.21.15.133:0/255971379 conn(0x55c53d388a30 msgr2=0x55c53d395a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.867+0000 7f6640588740 1 -- 172.21.15.133:0/255971379 shutdown_connections 2024-09-11T23:48:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.867+0000 7f6640588740 1 -- 172.21.15.133:0/255971379 wait complete. 2024-09-11T23:48:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.867+0000 7f6640588740 1 -- start start 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6640588740 1 Processor -- start 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6640588740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d39fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6640588740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d39ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6640588740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d39b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6640588740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f66381b0de0 con 0x55c53d397680 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6640588740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66381b0710 con 0x55c53d38c4a0 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6640588740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f66381ad970 con 0x55c53d38cdc0 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6640588740 10 client.0 _ll_register_callbacks cb 0x7f662c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6637fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d39b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6637fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d39b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39332/0 (socket says 172.21.15.133:39332) 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6637fff640 1 -- 172.21.15.133:0/3224562809 learned_addr learned my addr 172.21.15.133:0/3224562809 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6636ffd640 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d39ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f66377fe640 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d39fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6637fff640 1 -- 172.21.15.133:0/3224562809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 msgr2=0x55c53d39ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6637fff640 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d39ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6637fff640 1 -- 172.21.15.133:0/3224562809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 msgr2=0x55c53d39fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6637fff640 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d39fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f6637fff640 1 -- 172.21.15.133:0/3224562809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c53d39b810 con 0x55c53d397680 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.868+0000 7f66377fe640 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d39fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6637fff640 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d39b2d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f6630031860 tx=0x7f6630030e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320dba80fe68a2ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6616ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6616ffd640 1 -- 172.21.15.133:0/3224562809 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663005c060 con 0x55c53d397680 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6616ffd640 1 -- 172.21.15.133:0/3224562809 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f66300305a0 con 0x55c53d397680 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6640588740 1 -- 172.21.15.133:0/3224562809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6630036050 con 0x55c53d397680 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6616ffd640 1 -- 172.21.15.133:0/3224562809 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f66300307a0 con 0x55c53d397680 2024-09-11T23:48:53.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6640588740 10 client.18501 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6640588740 1 -- 172.21.15.133:0/3224562809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c53d411670 con 0x55c53d397680 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6616ffd640 1 -- 172.21.15.133:0/3224562809 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6630036050 con 0x55c53d397680 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6616ffd640 1 -- 172.21.15.133:0/3224562809 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6630030070 con 0x55c53d397680 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6640588740 10 client.18501 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6640588740 10 client.18501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6640588740 20 client.18501 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6640588740 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c53d3a6f20 0x55c53d413d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6640588740 4 client.18501 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.869+0000 7f6640588740 1 -- 172.21.15.133:0/3224562809 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c53d2f9d60 con 0x55c53d3a6f20 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f66377fe640 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c53d3a6f20 0x55c53d413d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6616ffd640 10 client.18501 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f66377fe640 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c53d3a6f20 0x55c53d413d20 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad1cee4dd8518b4f server_cookie=c50d571c4971dcb5 in_seq=0 out_seq=0 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6616ffd640 1 -- 172.21.15.133:0/3224562809 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c53d2f9d60 con 0x55c53d3a6f20 2024-09-11T23:48:53.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6616ffd640 10 client.18501 handle_command_reply: tid=0 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6640588740 1 client.18501 shutdown 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6640588740 1 -- 172.21.15.133:0/3224562809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 msgr2=0x55c53d39b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6640588740 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d39b2d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f6630031860 tx=0x7f6630030e70 comp rx=0 tx=0).stop 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6640588740 1 -- 172.21.15.133:0/3224562809 shutdown_connections 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6640588740 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x55c53d3a6f20 0x55c53d413d20 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6640588740 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c53d397680 0x55c53d39b2d0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6640588740 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c53d38cdc0 0x55c53d39ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6640588740 1 --2- 172.21.15.133:0/3224562809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c53d38c4a0 0x55c53d39fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6640588740 1 -- 172.21.15.133:0/3224562809 >> 172.21.15.133:0/3224562809 conn(0x55c53d388a30 msgr2=0x55c53d396250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.870+0000 7f6616ffd640 0 client.18501 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f6640588740 1 -- 172.21.15.133:0/3224562809 shutdown_connections 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f6640588740 1 -- 172.21.15.133:0/3224562809 wait complete. 2024-09-11T23:48:53.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f6640588740 20 client.18501 trim_cache size 0 max 16384 2024-09-11T23:48:53.877 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:48:53.877 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-09-11T23:48:53.877 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-09-11T23:48:53.877 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:48:53.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 -- 172.21.15.133:0/3076953023 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66200a2cf0 msgr2=0x7f66200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:53.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66200a2cf0 0x7f66200a51b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6630004ab0 tx=0x7f663002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 -- 172.21.15.133:0/3076953023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6638100a80 msgr2=0x7f663810e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:53.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6638100a80 0x7f663810e3e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f662800d950 tx=0x7f662800dd50 comp rx=0 tx=0).stop 2024-09-11T23:48:53.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 -- 172.21.15.133:0/3076953023 shutdown_connections 2024-09-11T23:48:53.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f66200a2cf0 0x7f66200a51b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6638106a80 0x7f663810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6638101420 0x7f663810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 --2- 172.21.15.133:0/3076953023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6638100a80 0x7f663810e3e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:53.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 -- 172.21.15.133:0/3076953023 >> 172.21.15.133:0/3076953023 conn(0x7f66380fc7a0 msgr2=0x7f66380fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:53.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 -- 172.21.15.133:0/3076953023 shutdown_connections 2024-09-11T23:48:53.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:53.871+0000 7f663e1af640 1 -- 172.21.15.133:0/3076953023 wait complete. 2024-09-11T23:48:54.754 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:48:54.754 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:48:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:55.126 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:54 smithi133 ceph-mon[31430]: pgmap v380: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 51 op/s 2024-09-11T23:48:55.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:54 smithi178 ceph-mon[35882]: pgmap v380: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 51 op/s 2024-09-11T23:48:55.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:54 smithi138 ceph-mon[35142]: pgmap v380: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 51 op/s 2024-09-11T23:48:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.479+0000 7f0bd9867640 1 -- 172.21.15.133:0/2105324182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd4102780 msgr2=0x7f0bd41067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.479+0000 7f0bd9867640 1 --2- 172.21.15.133:0/2105324182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd4102780 0x7f0bd41067d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009900 tx=0x7f0bc802a410 comp rx=0 tx=0).stop 2024-09-11T23:48:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.480+0000 7f0bd9867640 1 -- 172.21.15.133:0/2105324182 shutdown_connections 2024-09-11T23:48:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.480+0000 7f0bd9867640 1 --2- 172.21.15.133:0/2105324182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd4106f00 0x7f0bd410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.480+0000 7f0bd9867640 1 --2- 172.21.15.133:0/2105324182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd4102780 0x7f0bd41067d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.480+0000 7f0bd9867640 1 --2- 172.21.15.133:0/2105324182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd4101dd0 0x7f0bd41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.480+0000 7f0bd9867640 1 -- 172.21.15.133:0/2105324182 >> 172.21.15.133:0/2105324182 conn(0x7f0bd40fd8b0 msgr2=0x7f0bd40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.480+0000 7f0bd9867640 1 -- 172.21.15.133:0/2105324182 shutdown_connections 2024-09-11T23:48:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.480+0000 7f0bd9867640 1 -- 172.21.15.133:0/2105324182 wait complete. 2024-09-11T23:48:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.480+0000 7f0bd9867640 1 Processor -- start 2024-09-11T23:48:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd9867640 1 -- start start 2024-09-11T23:48:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd9867640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd4101dd0 0x7f0bd4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd9867640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd4102780 0x7f0bd41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd9867640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd4106f00 0x7f0bd419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd9867640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4111f60 con 0x7f0bd4102780 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd9867640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4111de0 con 0x7f0bd4106f00 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd9867640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41120e0 con 0x7f0bd4101dd0 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd4102780 0x7f0bd41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd4102780 0x7f0bd41989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39348/0 (socket says 172.21.15.133:39348) 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd27fc640 1 -- 172.21.15.133:0/1236666080 learned_addr learned my addr 172.21.15.133:0/1236666080 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd37fe640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd4106f00 0x7f0bd419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd2ffd640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd4101dd0 0x7f0bd4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd27fc640 1 -- 172.21.15.133:0/1236666080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd4101dd0 msgr2=0x7f0bd4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd27fc640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd4101dd0 0x7f0bd4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd27fc640 1 -- 172.21.15.133:0/1236666080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd4106f00 msgr2=0x7f0bd419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd27fc640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd4106f00 0x7f0bd419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:55.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd27fc640 1 -- 172.21.15.133:0/1236666080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd419d410 con 0x7f0bd4102780 2024-09-11T23:48:55.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd27fc640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd4102780 0x7f0bd41989a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8006010 tx=0x7f0bc80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:55.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.481+0000 7f0bd8865640 1 -- 172.21.15.133:0/1236666080 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc803f030 con 0x7f0bd4102780 2024-09-11T23:48:55.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.482+0000 7f0bd9867640 1 -- 172.21.15.133:0/1236666080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd419d6a0 con 0x7f0bd4102780 2024-09-11T23:48:55.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.482+0000 7f0bd8865640 1 -- 172.21.15.133:0/1236666080 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc8003ce0 con 0x7f0bd4102780 2024-09-11T23:48:55.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.482+0000 7f0bd8865640 1 -- 172.21.15.133:0/1236666080 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8035070 con 0x7f0bd4102780 2024-09-11T23:48:55.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.482+0000 7f0bd9867640 1 -- 172.21.15.133:0/1236666080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd41a4fe0 con 0x7f0bd4102780 2024-09-11T23:48:55.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.482+0000 7f0bd9867640 1 -- 172.21.15.133:0/1236666080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd41021b0 con 0x7f0bd4102780 2024-09-11T23:48:55.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.484+0000 7f0bd8865640 1 -- 172.21.15.133:0/1236666080 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0bc8042030 con 0x7f0bd4102780 2024-09-11T23:48:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.484+0000 7f0bd8865640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bbc0a2cf0 0x7f0bbc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.484+0000 7f0bd2ffd640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bbc0a2cf0 0x7f0bbc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.484+0000 7f0bd8865640 1 -- 172.21.15.133:0/1236666080 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f0bc80efe50 con 0x7f0bd4102780 2024-09-11T23:48:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.485+0000 7f0bd2ffd640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bbc0a2cf0 0x7f0bbc0a51b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4005fd0 tx=0x7f0bc4009870 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.488+0000 7f0bd8865640 1 -- 172.21.15.133:0/1236666080 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bc80bcb70 con 0x7f0bd4102780 2024-09-11T23:48:55.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.688+0000 7f0bd9867640 1 -- 172.21.15.133:0/1236666080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bd410a170 con 0x7f0bd4102780 2024-09-11T23:48:55.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.689+0000 7f0bd8865640 1 -- 172.21.15.133:0/1236666080 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11994 (secure 0 0 0) 0x7f0bc80bc470 con 0x7f0bd4102780 2024-09-11T23:48:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:48:55.689 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":32,"btime":"2024-09-11T23:48:01:740980+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:48:01.740959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27616":{"gid":27616,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2029124747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2029124747},{"type":"v1","addr":"172.21.15.178:6835","nonce":2029124747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:48:55.689 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 32 2024-09-11T23:48:55.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 -- 172.21.15.133:0/1236666080 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bbc0a2cf0 msgr2=0x7f0bbc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:55.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bbc0a2cf0 0x7f0bbc0a51b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4005fd0 tx=0x7f0bc4009870 comp rx=0 tx=0).stop 2024-09-11T23:48:55.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 -- 172.21.15.133:0/1236666080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd4102780 msgr2=0x7f0bd41989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:55.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd4102780 0x7f0bd41989a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8006010 tx=0x7f0bc80042f0 comp rx=0 tx=0).stop 2024-09-11T23:48:55.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 -- 172.21.15.133:0/1236666080 shutdown_connections 2024-09-11T23:48:55.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f0bbc0a2cf0 0x7f0bbc0a51b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:55.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd4106f00 0x7f0bd419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:55.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bd4102780 0x7f0bd41989a0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:55.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 --2- 172.21.15.133:0/1236666080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bd4101dd0 0x7f0bd4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:55.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 -- 172.21.15.133:0/1236666080 >> 172.21.15.133:0/1236666080 conn(0x7f0bd40fd8b0 msgr2=0x7f0bd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:55.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 -- 172.21.15.133:0/1236666080 shutdown_connections 2024-09-11T23:48:55.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:55.691+0000 7f0bd9867640 1 -- 172.21.15.133:0/1236666080 wait complete. 2024-09-11T23:48:56.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:55 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1236666080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:48:56.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:55 smithi133 ceph-mon[31430]: pgmap v381: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 16 op/s 2024-09-11T23:48:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:55 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1236666080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:48:56.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:55 smithi178 ceph-mon[35882]: pgmap v381: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 16 op/s 2024-09-11T23:48:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:55 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1236666080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:48:56.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:55 smithi138 ceph-mon[35142]: pgmap v381: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 16 op/s 2024-09-11T23:48:56.449 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:48:56.697 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:57.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.148+0000 7f13775c6640 1 -- 172.21.15.133:0/3633603575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370100e20 msgr2=0x7f1370101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:57.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.148+0000 7f13775c6640 1 --2- 172.21.15.133:0/3633603575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370100e20 0x7f1370101200 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f136c009900 tx=0x7f136c02a480 comp rx=0 tx=0).stop 2024-09-11T23:48:57.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.148+0000 7f13775c6640 1 -- 172.21.15.133:0/3633603575 shutdown_connections 2024-09-11T23:48:57.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.148+0000 7f13775c6640 1 --2- 172.21.15.133:0/3633603575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1370102ef0 0x7f137010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.148+0000 7f13775c6640 1 --2- 172.21.15.133:0/3633603575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1370102570 0x7f13701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.148+0000 7f13775c6640 1 --2- 172.21.15.133:0/3633603575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370100e20 0x7f1370101200 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.148+0000 7f13775c6640 1 -- 172.21.15.133:0/3633603575 >> 172.21.15.133:0/3633603575 conn(0x7f13700fc7d0 msgr2=0x7f13700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.148+0000 7f13775c6640 1 -- 172.21.15.133:0/3633603575 shutdown_connections 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.148+0000 7f13775c6640 1 -- 172.21.15.133:0/3633603575 wait complete. 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.148+0000 7f13775c6640 1 Processor -- start 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.148+0000 7f13775c6640 1 -- start start 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f13775c6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1370100e20 0x7f13700fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f13775c6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1370102570 0x7f13700fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f13775c6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370102ef0 0x7f13700fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f13775c6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1370110f50 con 0x7f1370102ef0 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f13775c6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1370110dd0 con 0x7f1370100e20 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f13775c6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13701110d0 con 0x7f1370102570 2024-09-11T23:48:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f1375b63640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370102ef0 0x7f13700fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f1375b63640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370102ef0 0x7f13700fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39374/0 (socket says 172.21.15.133:39374) 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f1375b63640 1 -- 172.21.15.133:0/2039327180 learned_addr learned my addr 172.21.15.133:0/2039327180 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f1374b61640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1370102570 0x7f13700fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f1375362640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1370100e20 0x7f13700fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f1375b63640 1 -- 172.21.15.133:0/2039327180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1370102570 msgr2=0x7f13700fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f1375b63640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1370102570 0x7f13700fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f1375b63640 1 -- 172.21.15.133:0/2039327180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1370100e20 msgr2=0x7f13700fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f1375b63640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1370100e20 0x7f13700fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f1375b63640 1 -- 172.21.15.133:0/2039327180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13701a6980 con 0x7f1370102ef0 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f1375b63640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370102ef0 0x7f13700fd700 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f136800eaa0 tx=0x7f136800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f13667fc640 1 -- 172.21.15.133:0/2039327180 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136800cdb0 con 0x7f1370102ef0 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f13775c6640 1 -- 172.21.15.133:0/2039327180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13701a6c70 con 0x7f1370102ef0 2024-09-11T23:48:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f13667fc640 1 -- 172.21.15.133:0/2039327180 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1368005020 con 0x7f1370102ef0 2024-09-11T23:48:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f13667fc640 1 -- 172.21.15.133:0/2039327180 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13680116b0 con 0x7f1370102ef0 2024-09-11T23:48:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.149+0000 7f13775c6640 1 -- 172.21.15.133:0/2039327180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13701a71b0 con 0x7f1370102ef0 2024-09-11T23:48:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.152+0000 7f13667fc640 1 -- 172.21.15.133:0/2039327180 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1368005560 con 0x7f1370102ef0 2024-09-11T23:48:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.152+0000 7f137999f740 1 Processor -- start 2024-09-11T23:48:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f137999f740 1 -- start start 2024-09-11T23:48:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f137999f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291530090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f137999f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2915305d0 0x55a291530a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f137999f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a29153b370 0x55a29153d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f137999f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a2914962f0 con 0x55a291531b40 2024-09-11T23:48:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f137999f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a29143dcf0 con 0x55a29153b370 2024-09-11T23:48:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f137999f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a2914922b0 con 0x55a2915305d0 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f1374b61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291530090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f1374b61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291530090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39388/0 (socket says 172.21.15.133:39388) 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f1374b61640 1 -- 172.21.15.133:0/1034350692 learned_addr learned my addr 172.21.15.133:0/1034350692 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f1375362640 1 --2- 172.21.15.133:0/1034350692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2915305d0 0x55a291530a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f13667fc640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f135c0a2cf0 0x7f135c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f1374b61640 1 -- 172.21.15.133:0/1034350692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2915305d0 msgr2=0x55a291530a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f1374b61640 1 --2- 172.21.15.133:0/1034350692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2915305d0 0x55a291530a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f1374b61640 1 -- 172.21.15.133:0/1034350692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a29153b370 msgr2=0x55a29153d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f1374b61640 1 --2- 172.21.15.133:0/1034350692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a29153b370 0x55a29153d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f1374b61640 1 -- 172.21.15.133:0/1034350692 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2914223a0 con 0x55a291531b40 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f1375b63640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f135c0a2cf0 0x7f135c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f1374b61640 1 --2- 172.21.15.133:0/1034350692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291530090 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f136000cca0 tx=0x7f1360010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb51b4f3753a399 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:57.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.153+0000 7f13667fc640 1 -- 172.21.15.133:0/2039327180 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f13680cf6d0 con 0x7f1370102ef0 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f135b7fe640 1 -- 172.21.15.133:0/1034350692 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360016070 con 0x55a291531b40 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f135b7fe640 1 -- 172.21.15.133:0/1034350692 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13600047a0 con 0x55a291531b40 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f135b7fe640 1 -- 172.21.15.133:0/1034350692 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13600128c0 con 0x55a291531b40 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f137999f740 1 -- 172.21.15.133:0/1034350692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 msgr2=0x55a291530090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f1375b63640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f135c0a2cf0 0x7f135c0a51b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f136800efd0 tx=0x7f136806dbf0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f137999f740 1 --2- 172.21.15.133:0/1034350692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291530090 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f136000cca0 tx=0x7f1360010740 comp rx=0 tx=0).stop 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f137999f740 1 -- 172.21.15.133:0/1034350692 shutdown_connections 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f137999f740 1 --2- 172.21.15.133:0/1034350692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a29153b370 0x55a29153d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f137999f740 1 --2- 172.21.15.133:0/1034350692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2915305d0 0x55a291530a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f137999f740 1 --2- 172.21.15.133:0/1034350692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291530090 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f137999f740 1 -- 172.21.15.133:0/1034350692 >> 172.21.15.133:0/1034350692 conn(0x55a2914f9310 msgr2=0x55a29152cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:57.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f137999f740 1 -- 172.21.15.133:0/1034350692 shutdown_connections 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.154+0000 7f137999f740 1 -- 172.21.15.133:0/1034350692 wait complete. 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f137999f740 1 -- start start 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f137999f740 1 Processor -- start 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f137999f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2915305d0 0x55a29153acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f137999f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291537f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f137999f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a29153b370 0x55a2915384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f137999f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a29153e070 con 0x55a291531b40 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f137999f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a291451060 con 0x55a2915305d0 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f137999f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a29153e1f0 con 0x55a29153b370 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f137999f740 10 client.0 _ll_register_callbacks cb 0x55a2911a48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f1375362640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291537f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f1375362640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291537f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39402/0 (socket says 172.21.15.133:39402) 2024-09-11T23:48:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f1375362640 1 -- 172.21.15.133:0/719970427 learned_addr learned my addr 172.21.15.133:0/719970427 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:57.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.155+0000 7f1374b61640 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2915305d0 0x55a29153acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1374b61640 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a29153b370 0x55a2915384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1375362640 1 -- 172.21.15.133:0/719970427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a29153b370 msgr2=0x55a2915384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:57.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1375362640 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a29153b370 0x55a2915384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1375362640 1 -- 172.21.15.133:0/719970427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2915305d0 msgr2=0x55a29153acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:57.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1375362640 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2915305d0 0x55a29153acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1375362640 1 -- 172.21.15.133:0/719970427 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2912073c0 con 0x55a291531b40 2024-09-11T23:48:57.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1374b61640 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2915305d0 0x55a29153acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:48:57.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1375362640 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291537f70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f136c005940 tx=0x7f136c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb55d8deb4cdece0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:57.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1358ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:57.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1358ff9640 1 -- 172.21.15.133:0/719970427 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c035030 con 0x55a291531b40 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f137999f740 1 -- 172.21.15.133:0/719970427 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13600047a0 con 0x55a291531b40 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1358ff9640 1 -- 172.21.15.133:0/719970427 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f136c0048c0 con 0x55a291531b40 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1358ff9640 1 -- 172.21.15.133:0/719970427 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f136c005d50 con 0x55a291531b40 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f137999f740 10 client.18519 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f137999f740 1 -- 172.21.15.133:0/719970427 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a291549ab0 con 0x55a291531b40 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1358ff9640 1 -- 172.21.15.133:0/719970427 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13600047a0 con 0x55a291531b40 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.156+0000 7f1358ff9640 1 -- 172.21.15.133:0/719970427 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f136c031540 con 0x55a291531b40 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.157+0000 7f137999f740 10 client.18519 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.157+0000 7f137999f740 10 client.18519 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.157+0000 7f137999f740 20 client.18519 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.157+0000 7f137999f740 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a29154af70 0x55a29154d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.157+0000 7f137999f740 4 client.18519 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.157+0000 7f137999f740 1 -- 172.21.15.133:0/719970427 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2914a3860 con 0x55a29154af70 2024-09-11T23:48:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.157+0000 7f1374b61640 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a29154af70 0x55a29154d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.157+0000 7f1358ff9640 10 client.18519 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.157+0000 7f1374b61640 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a29154af70 0x55a29154d3b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a0906efd528e3e05 server_cookie=8026782aba9ecac7 in_seq=0 out_seq=0 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f1358ff9640 1 -- 172.21.15.133:0/719970427 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2914a3860 con 0x55a29154af70 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f1358ff9640 10 client.18519 handle_command_reply: tid=0 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 1 client.18519 shutdown 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 1 -- 172.21.15.133:0/719970427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 msgr2=0x55a291537f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291537f70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f136c005940 tx=0x7f136c0052f0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 1 -- 172.21.15.133:0/719970427 shutdown_connections 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a29154af70 0x55a29154d3b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a29153b370 0x55a2915384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a291537f70 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f1358ff9640 0 client.18519 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 1 --2- 172.21.15.133:0/719970427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2915305d0 0x55a29153acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 1 -- 172.21.15.133:0/719970427 >> 172.21.15.133:0/719970427 conn(0x55a2914f9310 msgr2=0x55a291532930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 1 -- 172.21.15.133:0/719970427 shutdown_connections 2024-09-11T23:48:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 1 -- 172.21.15.133:0/719970427 wait complete. 2024-09-11T23:48:57.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.158+0000 7f137999f740 20 client.18519 trim_cache size 0 max 16384 2024-09-11T23:48:57.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.171+0000 7f137999f740 1 Processor -- start 2024-09-11T23:48:57.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.171+0000 7f137999f740 1 -- start start 2024-09-11T23:48:57.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.171+0000 7f137999f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2915305d0 0x55a291547340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.171+0000 7f137999f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a291531b40 0x55a291537ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.171+0000 7f137999f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a29153b370 0x55a291533120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.171+0000 7f137999f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f13701af930 con 0x55a29153b370 2024-09-11T23:48:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.171+0000 7f137999f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b1470 con 0x55a291531b40 2024-09-11T23:48:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.171+0000 7f137999f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b0000 con 0x55a2915305d0 2024-09-11T23:48:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.171+0000 7f1374b61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a29153b370 0x55a291533120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f1374b61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a29153b370 0x55a291533120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39408/0 (socket says 172.21.15.133:39408) 2024-09-11T23:48:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f1374b61640 1 -- 172.21.15.133:0/1521761310 learned_addr learned my addr 172.21.15.133:0/1521761310 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f1374b61640 1 --2- 172.21.15.133:0/1521761310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2915305d0 0x55a291547340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f1375362640 1 --2- 172.21.15.133:0/1521761310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a291531b40 0x55a291537ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f1374b61640 1 -- 172.21.15.133:0/1521761310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2915305d0 msgr2=0x55a291547340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f1374b61640 1 --2- 172.21.15.133:0/1521761310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2915305d0 0x55a291547340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f1374b61640 1 -- 172.21.15.133:0/1521761310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a291531b40 msgr2=0x55a291537ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f1374b61640 1 --2- 172.21.15.133:0/1521761310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a291531b40 0x55a291537ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f1374b61640 1 -- 172.21.15.133:0/1521761310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29152d960 con 0x55a29153b370 2024-09-11T23:48:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f1375362640 1 --2- 172.21.15.133:0/1521761310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a291531b40 0x55a291537ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:48:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f1374b61640 1 --2- 172.21.15.133:0/1521761310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a29153b370 0x55a291533120 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f136000c340 tx=0x7f1360014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb33544d40fd88a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f135affd640 1 -- 172.21.15.133:0/1521761310 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360016040 con 0x55a29153b370 2024-09-11T23:48:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f135affd640 1 -- 172.21.15.133:0/1521761310 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1360005450 con 0x55a29153b370 2024-09-11T23:48:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.172+0000 7f135affd640 1 -- 172.21.15.133:0/1521761310 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360024040 con 0x55a29153b370 2024-09-11T23:48:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.173+0000 7f137999f740 1 -- 172.21.15.133:0/1521761310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a29153b370 msgr2=0x55a291533120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.173+0000 7f137999f740 1 --2- 172.21.15.133:0/1521761310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a29153b370 0x55a291533120 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f136000c340 tx=0x7f1360014040 comp rx=0 tx=0).stop 2024-09-11T23:48:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.173+0000 7f137999f740 1 -- 172.21.15.133:0/1521761310 shutdown_connections 2024-09-11T23:48:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.173+0000 7f137999f740 1 --2- 172.21.15.133:0/1521761310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a29153b370 0x55a291533120 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.173+0000 7f137999f740 1 --2- 172.21.15.133:0/1521761310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a291531b40 0x55a291537ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.173+0000 7f137999f740 1 --2- 172.21.15.133:0/1521761310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2915305d0 0x55a291547340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.173+0000 7f137999f740 1 -- 172.21.15.133:0/1521761310 >> 172.21.15.133:0/1521761310 conn(0x55a2914f9310 msgr2=0x55a291537060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.173+0000 7f137999f740 1 -- 172.21.15.133:0/1521761310 shutdown_connections 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.173+0000 7f137999f740 1 -- 172.21.15.133:0/1521761310 wait complete. 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.173+0000 7f137999f740 1 -- start start 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.173+0000 7f137999f740 1 Processor -- start 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f137999f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2915305d0 0x55a291543ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f137999f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a29153ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f137999f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a29153b370 0x55a29153f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f137999f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b0da0 con 0x55a291531b40 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f137999f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b06d0 con 0x55a2915305d0 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f137999f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13701ad930 con 0x55a29153b370 2024-09-11T23:48:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f137999f740 10 client.0 _ll_register_callbacks cb 0x7f136c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1375362640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a29153ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1375362640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a29153ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39424/0 (socket says 172.21.15.133:39424) 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1375362640 1 -- 172.21.15.133:0/3558914240 learned_addr learned my addr 172.21.15.133:0/3558914240 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1374b61640 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a29153b370 0x55a29153f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1374b61640 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2915305d0 0x55a291543ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1375362640 1 -- 172.21.15.133:0/3558914240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a29153b370 msgr2=0x55a29153f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1375362640 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a29153b370 0x55a29153f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1375362640 1 -- 172.21.15.133:0/3558914240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2915305d0 msgr2=0x55a291543ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1375362640 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2915305d0 0x55a291543ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1375362640 1 -- 172.21.15.133:0/3558914240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29153fa40 con 0x55a291531b40 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1374b61640 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a29153b370 0x55a29153f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:48:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1375362640 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a29153ee10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f136c009850 tx=0x7f136c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e583666a9573206 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1358ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1358ff9640 1 -- 172.21.15.133:0/3558914240 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c051070 con 0x55a291531b40 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f1358ff9640 1 -- 172.21.15.133:0/3558914240 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f136c0046c0 con 0x55a291531b40 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.174+0000 7f137999f740 1 -- 172.21.15.133:0/3558914240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1360005450 con 0x55a291531b40 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f1358ff9640 1 -- 172.21.15.133:0/3558914240 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f136c035040 con 0x55a291531b40 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f137999f740 10 client.18525 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f137999f740 1 -- 172.21.15.133:0/3558914240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a2915b5b30 con 0x55a291531b40 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f1358ff9640 1 -- 172.21.15.133:0/3558914240 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1360005450 con 0x55a291531b40 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f1358ff9640 1 -- 172.21.15.133:0/3558914240 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f136c003b20 con 0x55a291531b40 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f137999f740 10 client.18525 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f137999f740 10 client.18525 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f137999f740 20 client.18525 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f137999f740 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a2915b6280 0x55a2915b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f137999f740 4 client.18525 mds_command: new command op to 24401 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:48:57.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f137999f740 1 -- 172.21.15.133:0/3558914240 --> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a29149dd70 con 0x55a2915b6280 2024-09-11T23:48:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.175+0000 7f1374b61640 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a2915b6280 0x55a2915b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f1358ff9640 10 client.18525 ms_handle_connect on v2:172.21.15.138:6832/1969946111 2024-09-11T23:48:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f1374b61640 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a2915b6280 0x55a2915b86a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=43db6829e3baa4b2 server_cookie=311852dda370c73 in_seq=0 out_seq=0 2024-09-11T23:48:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f1358ff9640 1 -- 172.21.15.133:0/3558914240 <== mds.4 v2:172.21.15.138:6832/1969946111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a29149dd70 con 0x55a2915b6280 2024-09-11T23:48:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f1358ff9640 10 client.18525 handle_command_reply: tid=0 2024-09-11T23:48:57.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 1 client.18525 shutdown 2024-09-11T23:48:57.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 1 -- 172.21.15.133:0/3558914240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 msgr2=0x55a29153ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:57.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a29153ee10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f136c009850 tx=0x7f136c004610 comp rx=0 tx=0).stop 2024-09-11T23:48:57.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 1 -- 172.21.15.133:0/3558914240 shutdown_connections 2024-09-11T23:48:57.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.138:6832/1969946111,v1:172.21.15.138:6833/1969946111] conn(0x55a2915b6280 0x55a2915b86a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a29153b370 0x55a29153f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a291531b40 0x55a29153ee10 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 1 --2- 172.21.15.133:0/3558914240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2915305d0 0x55a291543ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 1 -- 172.21.15.133:0/3558914240 >> 172.21.15.133:0/3558914240 conn(0x55a2914f9310 msgr2=0x55a29152b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:57.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f1358ff9640 0 client.18525 ms_handle_reset on v2:172.21.15.138:6832/1969946111 2024-09-11T23:48:57.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 1 -- 172.21.15.133:0/3558914240 shutdown_connections 2024-09-11T23:48:57.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 1 -- 172.21.15.133:0/3558914240 wait complete. 2024-09-11T23:48:57.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.176+0000 7f137999f740 20 client.18525 trim_cache size 0 max 16384 2024-09-11T23:48:57.193 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:48:57.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 -- 172.21.15.133:0/2039327180 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f135c0a2cf0 msgr2=0x7f135c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:57.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f135c0a2cf0 0x7f135c0a51b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f136800efd0 tx=0x7f136806dbf0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 -- 172.21.15.133:0/2039327180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370102ef0 msgr2=0x7f13700fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:57.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370102ef0 0x7f13700fd700 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f136800eaa0 tx=0x7f136800ef60 comp rx=0 tx=0).stop 2024-09-11T23:48:57.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 -- 172.21.15.133:0/2039327180 shutdown_connections 2024-09-11T23:48:57.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f135c0a2cf0 0x7f135c0a51b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370102ef0 0x7f13700fd700 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1370102570 0x7f13700fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 --2- 172.21.15.133:0/2039327180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1370100e20 0x7f13700fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:57.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 -- 172.21.15.133:0/2039327180 >> 172.21.15.133:0/2039327180 conn(0x7f13700fc7d0 msgr2=0x7f137006a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:57.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 -- 172.21.15.133:0/2039327180 shutdown_connections 2024-09-11T23:48:57.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:57.177+0000 7f13775c6640 1 -- 172.21.15.133:0/2039327180 wait complete. 2024-09-11T23:48:57.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:48:58.176 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:48:58.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.664+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/1161978058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2f4102570 msgr2=0x7fb2f41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:58.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.664+0000 7fb2fc0cb640 1 --2- 172.21.15.133:0/1161978058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2f4102570 0x7fb2f41029b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009900 tx=0x7fb2e802a480 comp rx=0 tx=0).stop 2024-09-11T23:48:58.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.664+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/1161978058 shutdown_connections 2024-09-11T23:48:58.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.664+0000 7fb2fc0cb640 1 --2- 172.21.15.133:0/1161978058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4102ef0 0x7fb2f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.664+0000 7fb2fc0cb640 1 --2- 172.21.15.133:0/1161978058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2f4102570 0x7fb2f41029b0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.664+0000 7fb2fc0cb640 1 --2- 172.21.15.133:0/1161978058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f4100e20 0x7fb2f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.664+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/1161978058 >> 172.21.15.133:0/1161978058 conn(0x7fb2f40fc7d0 msgr2=0x7fb2f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.664+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/1161978058 shutdown_connections 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.664+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/1161978058 wait complete. 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.664+0000 7fb2fc0cb640 1 Processor -- start 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.664+0000 7fb2fc0cb640 1 -- start start 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2fc0cb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f4100e20 0x7fb2f419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2fc0cb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2f4102570 0x7fb2f419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2fc0cb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4102ef0 0x7fb2f419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2fc0cb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4110f80 con 0x7fb2f4102570 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2fc0cb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4110e00 con 0x7fb2f4102ef0 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2fc0cb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4111100 con 0x7fb2f4100e20 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2f9666640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2f4102570 0x7fb2f419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2f9666640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2f4102570 0x7fb2f419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39442/0 (socket says 172.21.15.133:39442) 2024-09-11T23:48:58.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2f9666640 1 -- 172.21.15.133:0/735718854 learned_addr learned my addr 172.21.15.133:0/735718854 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2fa668640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4102ef0 0x7fb2f419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2f9e67640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f4100e20 0x7fb2f419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2f9666640 1 -- 172.21.15.133:0/735718854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f4100e20 msgr2=0x7fb2f419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2f9666640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f4100e20 0x7fb2f419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2f9666640 1 -- 172.21.15.133:0/735718854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4102ef0 msgr2=0x7fb2f419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2f9666640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4102ef0 0x7fb2f419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2f9666640 1 -- 172.21.15.133:0/735718854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f419f660 con 0x7fb2f4102570 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2fa668640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4102ef0 0x7fb2f419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.665+0000 7fb2f9666640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2f4102570 0x7fb2f419abf0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8006010 tx=0x7fb2e8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.666+0000 7fb2f2ffd640 1 -- 172.21.15.133:0/735718854 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e803f070 con 0x7fb2f4102570 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.666+0000 7fb2f2ffd640 1 -- 172.21.15.133:0/735718854 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e8031c30 con 0x7fb2f4102570 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.666+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/735718854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f419f8f0 con 0x7fb2f4102570 2024-09-11T23:48:58.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.666+0000 7fb2f2ffd640 1 -- 172.21.15.133:0/735718854 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8035030 con 0x7fb2f4102570 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.666+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/735718854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f41a7230 con 0x7fb2f4102570 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.668+0000 7fb2f2ffd640 1 -- 172.21.15.133:0/735718854 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb2e8042070 con 0x7fb2f4102570 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.668+0000 7fb2fe4a4740 1 Processor -- start 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.668+0000 7fb2fe4a4740 1 -- start start 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.668+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd3b40 0x558964fd2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.668+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd25d0 0x558964fd2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.668+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fdf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.668+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558964f382f0 con 0x558964fdd370 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.668+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558964edfcf0 con 0x558964fd3b40 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.668+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558964f342b0 con 0x558964fd25d0 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f9666640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fdf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f2ffd640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2dc0a2cf0 0x7fb2dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f9666640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fdf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39456/0 (socket says 172.21.15.133:39456) 2024-09-11T23:48:58.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f9666640 1 -- 172.21.15.133:0/1257670199 learned_addr learned my addr 172.21.15.133:0/1257670199 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f9e67640 1 --2- 172.21.15.133:0/1257670199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd25d0 0x558964fd2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2fa668640 1 --2- 172.21.15.133:0/1257670199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd3b40 0x558964fd2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2fa668640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2dc0a2cf0 0x7fb2dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f2ffd640 1 -- 172.21.15.133:0/735718854 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fb2e80efa70 con 0x7fb2f4102570 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f9666640 1 -- 172.21.15.133:0/1257670199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd25d0 msgr2=0x558964fd2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f9666640 1 --2- 172.21.15.133:0/1257670199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd25d0 0x558964fd2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f9666640 1 -- 172.21.15.133:0/1257670199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd3b40 msgr2=0x558964fd2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f9666640 1 --2- 172.21.15.133:0/1257670199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd3b40 0x558964fd2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f9666640 1 -- 172.21.15.133:0/1257670199 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558964ec43a0 con 0x558964fdd370 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f9e67640 1 --2- 172.21.15.133:0/1257670199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd25d0 0x558964fd2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2f9666640 1 --2- 172.21.15.133:0/1257670199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fdf760 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8046000 tx=0x7fb2e803b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ac43a48a6aeef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2fa668640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2dc0a2cf0 0x7fb2dc0a51b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009710 tx=0x7fb2ec005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:58.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2dbfff640 1 -- 172.21.15.133:0/1257670199 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8049070 con 0x558964fdd370 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2dbfff640 1 -- 172.21.15.133:0/1257670199 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e80bc080 con 0x558964fdd370 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.669+0000 7fb2dbfff640 1 -- 172.21.15.133:0/1257670199 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80ed480 con 0x558964fdd370 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.670+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/1257670199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 msgr2=0x558964fdf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.670+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/1257670199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fdf760 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8046000 tx=0x7fb2e803b790 comp rx=0 tx=0).stop 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.670+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/1257670199 shutdown_connections 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.670+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/1257670199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fdf760 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.670+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/1257670199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd25d0 0x558964fd2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.670+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/1257670199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd3b40 0x558964fd2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.670+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/1257670199 >> 172.21.15.133:0/1257670199 conn(0x558964f9b310 msgr2=0x558964fcec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.670+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/1257670199 shutdown_connections 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.670+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/1257670199 wait complete. 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fe4a4740 1 -- start start 2024-09-11T23:48:58.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fe4a4740 1 Processor -- start 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd25d0 0x558964fdce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd3b40 0x558964fda0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fda840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558964f382f0 con 0x558964fdd370 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558964ef3060 con 0x558964fd3b40 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558964fdfe30 con 0x558964fd25d0 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fe4a4740 10 client.0 _ll_register_callbacks cb 0x558964c468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fa668640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fda840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fa668640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fda840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39464/0 (socket says 172.21.15.133:39464) 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2fa668640 1 -- 172.21.15.133:0/2951573123 learned_addr learned my addr 172.21.15.133:0/2951573123 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2f9666640 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd3b40 0x558964fda0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.671+0000 7fb2f9e67640 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd25d0 0x558964fdce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fa668640 1 -- 172.21.15.133:0/2951573123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd25d0 msgr2=0x558964fdce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fa668640 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd25d0 0x558964fdce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fa668640 1 -- 172.21.15.133:0/2951573123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd3b40 msgr2=0x558964fda0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fa668640 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd3b40 0x558964fda0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fa668640 1 -- 172.21.15.133:0/2951573123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558964ca93c0 con 0x558964fdd370 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2f9666640 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd3b40 0x558964fda0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fa668640 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fda840 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec011f60 tx=0x7fb2ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c80e3cc71d43ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2951573123 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec01f460 con 0x558964fdd370 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2951573123 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fb2ec01f600 con 0x558964fdd370 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2951573123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e80bc080 con 0x558964fdd370 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2951573123 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb2ec01f800 con 0x558964fdd370 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fe4a4740 10 client.18537 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2951573123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558964feb860 con 0x558964fdd370 2024-09-11T23:48:58.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2951573123 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e80bc080 con 0x558964fdd370 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2951573123 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb2ec028070 con 0x558964fdd370 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fe4a4740 10 client.18537 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fe4a4740 10 client.18537 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fe4a4740 20 client.18537 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.672+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x558964fecd70 0x558964fef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.673+0000 7fb2fe4a4740 4 client.18537 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.673+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2951573123 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558964f45860 con 0x558964fecd70 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.673+0000 7fb2f9e67640 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x558964fecd70 0x558964fef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.673+0000 7fb2d97fa640 10 client.18537 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.673+0000 7fb2f9e67640 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x558964fecd70 0x558964fef190 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d6b11126ab018bc server_cookie=4f8e613edaecb16a in_seq=0 out_seq=0 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2951573123 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558964f45860 con 0x558964fecd70 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2d97fa640 10 client.18537 handle_command_reply: tid=0 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2fe4a4740 1 client.18537 shutdown 2024-09-11T23:48:58.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2951573123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 msgr2=0x558964fda840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:58.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fda840 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec011f60 tx=0x7fb2ec0046c0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2951573123 shutdown_connections 2024-09-11T23:48:58.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x558964fecd70 0x558964fef190 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fdd370 0x558964fda840 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd3b40 0x558964fda0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2d97fa640 0 client.18537 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:48:58.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2951573123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd25d0 0x558964fdce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.674+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2951573123 >> 172.21.15.133:0/2951573123 conn(0x558964f9b310 msgr2=0x558964fd4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:58.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.675+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2951573123 shutdown_connections 2024-09-11T23:48:58.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.675+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2951573123 wait complete. 2024-09-11T23:48:58.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.675+0000 7fb2fe4a4740 20 client.18537 trim_cache size 0 max 16384 2024-09-11T23:48:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2fe4a4740 1 Processor -- start 2024-09-11T23:48:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2fe4a4740 1 -- start start 2024-09-11T23:48:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd25d0 0x558964fdc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd3b40 0x558964fde760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fdd370 0x558964fdeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41af980 con 0x558964fd3b40 2024-09-11T23:48:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41b14c0 con 0x558964fd25d0 2024-09-11T23:48:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41b0050 con 0x558964fdd370 2024-09-11T23:48:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2f9666640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd3b40 0x558964fde760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2f9666640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd3b40 0x558964fde760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39480/0 (socket says 172.21.15.133:39480) 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2f9666640 1 -- 172.21.15.133:0/2689799743 learned_addr learned my addr 172.21.15.133:0/2689799743 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2f9e67640 1 --2- 172.21.15.133:0/2689799743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd25d0 0x558964fdc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.688+0000 7fb2fa668640 1 --2- 172.21.15.133:0/2689799743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fdd370 0x558964fdeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.689+0000 7fb2f9666640 1 -- 172.21.15.133:0/2689799743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fdd370 msgr2=0x558964fdeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.689+0000 7fb2f9666640 1 --2- 172.21.15.133:0/2689799743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fdd370 0x558964fdeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.689+0000 7fb2f9666640 1 -- 172.21.15.133:0/2689799743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd25d0 msgr2=0x558964fdc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.689+0000 7fb2f9666640 1 --2- 172.21.15.133:0/2689799743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd25d0 0x558964fdc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.689+0000 7fb2f9666640 1 -- 172.21.15.133:0/2689799743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558964fcfdd0 con 0x558964fd3b40 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.689+0000 7fb2fa668640 1 --2- 172.21.15.133:0/2689799743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fdd370 0x558964fdeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.689+0000 7fb2f9666640 1 --2- 172.21.15.133:0/2689799743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd3b40 0x558964fde760 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80ed800 tx=0x7fb2e80bbf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5e1d059c0f36f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.689+0000 7fb2db7fe640 1 -- 172.21.15.133:0/2689799743 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8049070 con 0x558964fd3b40 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.689+0000 7fb2db7fe640 1 -- 172.21.15.133:0/2689799743 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e80bc080 con 0x558964fd3b40 2024-09-11T23:48:58.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.689+0000 7fb2db7fe640 1 -- 172.21.15.133:0/2689799743 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e803b580 con 0x558964fd3b40 2024-09-11T23:48:58.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.690+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2689799743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd3b40 msgr2=0x558964fde760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:58.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.690+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2689799743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd3b40 0x558964fde760 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80ed800 tx=0x7fb2e80bbf80 comp rx=0 tx=0).stop 2024-09-11T23:48:58.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.690+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2689799743 shutdown_connections 2024-09-11T23:48:58.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.690+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2689799743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fdd370 0x558964fdeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.690+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2689799743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd3b40 0x558964fde760 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.690+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2689799743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fd25d0 0x558964fdc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.690+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2689799743 >> 172.21.15.133:0/2689799743 conn(0x558964f9b310 msgr2=0x558964fdb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.690+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2689799743 shutdown_connections 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.690+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2689799743 wait complete. 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.690+0000 7fb2fe4a4740 1 -- start start 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.690+0000 7fb2fe4a4740 1 Processor -- start 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd25d0 0x558964fde6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd3b40 0x558964fdebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2fe4a4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fdd370 0x558964fdf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41b0df0 con 0x558964fd25d0 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41b0720 con 0x558964fdd370 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2fe4a4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41ad980 con 0x558964fd3b40 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2fe4a4740 10 client.0 _ll_register_callbacks cb 0x7fb2ec011df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:48:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2f9e67640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd25d0 0x558964fde6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2f9e67640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd25d0 0x558964fde6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39496/0 (socket says 172.21.15.133:39496) 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2f9e67640 1 -- 172.21.15.133:0/2395631272 learned_addr learned my addr 172.21.15.133:0/2395631272 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2fa668640 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fdd370 0x558964fdf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2f9e67640 1 -- 172.21.15.133:0/2395631272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd3b40 msgr2=0x558964fdebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2f9666640 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd3b40 0x558964fdebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2f9e67640 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd3b40 0x558964fdebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2f9e67640 1 -- 172.21.15.133:0/2395631272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fdd370 msgr2=0x558964fdf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2f9e67640 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fdd370 0x558964fdf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2f9e67640 1 -- 172.21.15.133:0/2395631272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558964fe3560 con 0x558964fd25d0 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2fa668640 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fdd370 0x558964fdf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2f9e67640 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd25d0 0x558964fde6a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4004a80 tx=0x7fb2e40136e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c84c619cfc1427b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:48:58.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2395631272 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4004040 con 0x558964fd25d0 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2395631272 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fb2e40041e0 con 0x558964fd25d0 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.691+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2395631272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e80bc080 con 0x558964fd25d0 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2395631272 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb2e402e050 con 0x558964fd25d0 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2fe4a4740 10 client.18543 fetch_fsmap learned FSMap version 32 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2395631272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558965057d40 con 0x558964fd25d0 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2395631272 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e80bc080 con 0x558964fd25d0 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2395631272 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb2e400f040 con 0x558964fd25d0 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2fe4a4740 10 client.18543 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2fe4a4740 10 client.18543 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2fe4a4740 20 client.18543 populate_metadata read hostname 'smithi133' 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5589650584f0 0x55896505a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2fe4a4740 4 client.18543 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2395631272 --> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558964f3fd70 con 0x5589650584f0 2024-09-11T23:48:58.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.692+0000 7fb2f9666640 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5589650584f0 0x55896505a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2f9666640 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5589650584f0 0x55896505a910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b4e584c13d4a2b2e server_cookie=7a9c0e03cc49df2b in_seq=0 out_seq=0 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2d97fa640 10 client.18543 ms_handle_connect on v2:172.21.15.138:6834/137089847 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2d97fa640 1 -- 172.21.15.133:0/2395631272 <== mds.3 v2:172.21.15.138:6834/137089847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558964f3fd70 con 0x5589650584f0 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2d97fa640 10 client.18543 handle_command_reply: tid=0 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 1 client.18543 shutdown 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2395631272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd25d0 msgr2=0x558964fde6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd25d0 0x558964fde6a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4004a80 tx=0x7fb2e40136e0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2395631272 shutdown_connections 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.138:6834/137089847,v1:172.21.15.138:6835/137089847] conn(0x5589650584f0 0x55896505a910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558964fdd370 0x558964fdf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558964fd3b40 0x558964fdebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 1 --2- 172.21.15.133:0/2395631272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558964fd25d0 0x558964fde6a0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2d97fa640 0 client.18543 ms_handle_reset on v2:172.21.15.138:6834/137089847 2024-09-11T23:48:58.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2395631272 >> 172.21.15.133:0/2395631272 conn(0x558964f9b310 msgr2=0x558964fcdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:58.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2395631272 shutdown_connections 2024-09-11T23:48:58.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 1 -- 172.21.15.133:0/2395631272 wait complete. 2024-09-11T23:48:58.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.693+0000 7fb2fe4a4740 20 client.18543 trim_cache size 0 max 16384 2024-09-11T23:48:58.705 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:48:58.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/735718854 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2dc0a2cf0 msgr2=0x7fb2dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:58.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2dc0a2cf0 0x7fb2dc0a51b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009710 tx=0x7fb2ec005f90 comp rx=0 tx=0).stop 2024-09-11T23:48:58.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/735718854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2f4102570 msgr2=0x7fb2f419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:48:58.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2f4102570 0x7fb2f419abf0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8006010 tx=0x7fb2e8003c10 comp rx=0 tx=0).stop 2024-09-11T23:48:58.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/735718854 shutdown_connections 2024-09-11T23:48:58.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fb2dc0a2cf0 0x7fb2dc0a51b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4102ef0 0x7fb2f419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2f4102570 0x7fb2f419abf0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 --2- 172.21.15.133:0/735718854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f4100e20 0x7fb2f419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:48:58.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/735718854 >> 172.21.15.133:0/735718854 conn(0x7fb2f40fc7d0 msgr2=0x7fb2f406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:48:58.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/735718854 shutdown_connections 2024-09-11T23:48:58.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:48:58.694+0000 7fb2fc0cb640 1 -- 172.21.15.133:0/735718854 wait complete. 2024-09-11T23:48:58.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:48:58 smithi133 ceph-mon[31430]: pgmap v382: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:48:59.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:48:58 smithi178 ceph-mon[35882]: pgmap v382: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:48:59.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:48:58 smithi138 ceph-mon[35142]: pgmap v382: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-11T23:48:59.383 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:48:59.629 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:00.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.132+0000 7f7b8f593640 1 -- 172.21.15.133:0/1734545497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b88102570 msgr2=0x7f7b881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:00.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.132+0000 7f7b8f593640 1 --2- 172.21.15.133:0/1734545497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b88102570 0x7f7b881029b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009900 tx=0x7f7b7802a480 comp rx=0 tx=0).stop 2024-09-11T23:49:00.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.132+0000 7f7b8f593640 1 -- 172.21.15.133:0/1734545497 shutdown_connections 2024-09-11T23:49:00.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.132+0000 7f7b8f593640 1 --2- 172.21.15.133:0/1734545497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b88102ef0 0x7f7b8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.132+0000 7f7b8f593640 1 --2- 172.21.15.133:0/1734545497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b88102570 0x7f7b881029b0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.132+0000 7f7b8f593640 1 --2- 172.21.15.133:0/1734545497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b88100e20 0x7f7b88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.132+0000 7f7b8f593640 1 -- 172.21.15.133:0/1734545497 >> 172.21.15.133:0/1734545497 conn(0x7f7b880fc7d0 msgr2=0x7f7b880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:00.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.132+0000 7f7b8f593640 1 -- 172.21.15.133:0/1734545497 shutdown_connections 2024-09-11T23:49:00.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.132+0000 7f7b8f593640 1 -- 172.21.15.133:0/1734545497 wait complete. 2024-09-11T23:49:00.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.133+0000 7f7b8f593640 1 Processor -- start 2024-09-11T23:49:00.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.133+0000 7f7b8f593640 1 -- start start 2024-09-11T23:49:00.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.133+0000 7f7b8f593640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b88100e20 0x7f7b881992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.133+0000 7f7b8f593640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b88102570 0x7f7b881997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.133+0000 7f7b8f593640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b88102ef0 0x7f7b881a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.133+0000 7f7b8f593640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88110f80 con 0x7f7b88102ef0 2024-09-11T23:49:00.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.133+0000 7f7b8f593640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88110e00 con 0x7f7b88100e20 2024-09-11T23:49:00.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.133+0000 7f7b8f593640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88111100 con 0x7f7b88102570 2024-09-11T23:49:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.134+0000 7f7b8db30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b88102ef0 0x7f7b881a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.134+0000 7f7b8db30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b88102ef0 0x7f7b881a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39506/0 (socket says 172.21.15.133:39506) 2024-09-11T23:49:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.134+0000 7f7b8db30640 1 -- 172.21.15.133:0/3840880189 learned_addr learned my addr 172.21.15.133:0/3840880189 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.134+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b88102570 0x7f7b881997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.134+0000 7f7b8d32f640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b88100e20 0x7f7b881992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.134+0000 7f7b8db30640 1 -- 172.21.15.133:0/3840880189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b88102570 msgr2=0x7f7b881997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.134+0000 7f7b8db30640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b88102570 0x7f7b881997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.134+0000 7f7b8db30640 1 -- 172.21.15.133:0/3840880189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b88100e20 msgr2=0x7f7b881992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.134+0000 7f7b8db30640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b88100e20 0x7f7b881992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.134+0000 7f7b8db30640 1 -- 172.21.15.133:0/3840880189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b881a4870 con 0x7f7b88102ef0 2024-09-11T23:49:00.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.135+0000 7f7b8db30640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b88102ef0 0x7f7b881a4330 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000eb10 tx=0x7f7b8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:00.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.135+0000 7f7b7e7fc640 1 -- 172.21.15.133:0/3840880189 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80004b20 con 0x7f7b88102ef0 2024-09-11T23:49:00.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.135+0000 7f7b8f593640 1 -- 172.21.15.133:0/3840880189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b881a4a40 con 0x7f7b88102ef0 2024-09-11T23:49:00.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.135+0000 7f7b7e7fc640 1 -- 172.21.15.133:0/3840880189 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b80004cc0 con 0x7f7b88102ef0 2024-09-11T23:49:00.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.135+0000 7f7b7e7fc640 1 -- 172.21.15.133:0/3840880189 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80010760 con 0x7f7b88102ef0 2024-09-11T23:49:00.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.136+0000 7f7b8f593640 1 -- 172.21.15.133:0/3840880189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b881a4fa0 con 0x7f7b88102ef0 2024-09-11T23:49:00.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.138+0000 7f7b7e7fc640 1 -- 172.21.15.133:0/3840880189 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7b80005020 con 0x7f7b88102ef0 2024-09-11T23:49:00.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.139+0000 7f7b9196c740 1 Processor -- start 2024-09-11T23:49:00.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.139+0000 7f7b9196c740 1 -- start start 2024-09-11T23:49:00.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.139+0000 7f7b7e7fc640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b740a2cf0 0x7f7b740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.139+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 0x5618d6c33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.139+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c335d0 0x5618d6c33a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.139+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 0x5618d6c44b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.139+0000 7f7b9196c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5618d6b992f0 con 0x5618d6c42780 2024-09-11T23:49:00.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.139+0000 7f7b9196c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618d6b40cf0 con 0x5618d6c335d0 2024-09-11T23:49:00.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.139+0000 7f7b9196c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5618d6b952b0 con 0x5618d6c34b40 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.139+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b740a2cf0 0x7f7b740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.139+0000 7f7b7e7fc640 1 -- 172.21.15.133:0/3840880189 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f7b800cfaf0 con 0x7f7b88102ef0 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8db30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 0x5618d6c44b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8d32f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 0x5618d6c33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8d32f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 0x5618d6c33090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.133:39304/0 (socket says 172.21.15.133:39304) 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8d32f640 1 -- 172.21.15.133:0/4096809404 learned_addr learned my addr 172.21.15.133:0/4096809404 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/4096809404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c335d0 0x5618d6c33a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8d32f640 1 -- 172.21.15.133:0/4096809404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c335d0 msgr2=0x5618d6c33a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8d32f640 1 --2- 172.21.15.133:0/4096809404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c335d0 0x5618d6c33a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b740a2cf0 0x7f7b740a51b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009230 tx=0x7f7b78005e50 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8d32f640 1 -- 172.21.15.133:0/4096809404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 msgr2=0x5618d6c44b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8d32f640 1 --2- 172.21.15.133:0/4096809404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 0x5618d6c44b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8d32f640 1 -- 172.21.15.133:0/4096809404 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d6b253a0 con 0x5618d6c34b40 2024-09-11T23:49:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.140+0000 7f7b8d32f640 1 --2- 172.21.15.133:0/4096809404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 0x5618d6c33090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009c80 tx=0x7f7b840117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a488df1499baaf63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:00.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.141+0000 7f7b737fe640 1 -- 172.21.15.133:0/4096809404 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84012450 con 0x5618d6c34b40 2024-09-11T23:49:00.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.141+0000 7f7b737fe640 1 -- 172.21.15.133:0/4096809404 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b84012ad0 con 0x5618d6c34b40 2024-09-11T23:49:00.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.141+0000 7f7b737fe640 1 -- 172.21.15.133:0/4096809404 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8400fb00 con 0x5618d6c34b40 2024-09-11T23:49:00.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.141+0000 7f7b9196c740 1 -- 172.21.15.133:0/4096809404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 msgr2=0x5618d6c33090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:00.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.141+0000 7f7b9196c740 1 --2- 172.21.15.133:0/4096809404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 0x5618d6c33090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009c80 tx=0x7f7b840117b0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.141+0000 7f7b9196c740 1 -- 172.21.15.133:0/4096809404 shutdown_connections 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.141+0000 7f7b9196c740 1 --2- 172.21.15.133:0/4096809404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 0x5618d6c44b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.141+0000 7f7b9196c740 1 --2- 172.21.15.133:0/4096809404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c335d0 0x5618d6c33a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.141+0000 7f7b9196c740 1 --2- 172.21.15.133:0/4096809404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 0x5618d6c33090 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.141+0000 7f7b9196c740 1 -- 172.21.15.133:0/4096809404 >> 172.21.15.133:0/4096809404 conn(0x5618d6bfc310 msgr2=0x5618d6c2fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.142+0000 7f7b9196c740 1 -- 172.21.15.133:0/4096809404 shutdown_connections 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.142+0000 7f7b9196c740 1 -- 172.21.15.133:0/4096809404 wait complete. 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.142+0000 7f7b9196c740 1 -- start start 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.142+0000 7f7b9196c740 1 Processor -- start 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 0x5618d6c3b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 0x5618d6c3baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c42780 0x5618d6c3c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b9196c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5618d6c36550 con 0x5618d6c335d0 2024-09-11T23:49:00.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b9196c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618d6b54060 con 0x5618d6c42780 2024-09-11T23:49:00.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b9196c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5618d6c366d0 con 0x5618d6c34b40 2024-09-11T23:49:00.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b9196c740 10 client.0 _ll_register_callbacks cb 0x5618d68a78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:49:00.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8d32f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 0x5618d6c3b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8d32f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 0x5618d6c3b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39516/0 (socket says 172.21.15.133:39516) 2024-09-11T23:49:00.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8cb2e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 0x5618d6c3baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8d32f640 1 -- 172.21.15.133:0/2415938057 learned_addr learned my addr 172.21.15.133:0/2415938057 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:00.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8db30640 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c42780 0x5618d6c3c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8d32f640 1 -- 172.21.15.133:0/2415938057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 msgr2=0x5618d6c3baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:00.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8d32f640 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 0x5618d6c3baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8d32f640 1 -- 172.21.15.133:0/2415938057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c42780 msgr2=0x5618d6c3c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:00.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8d32f640 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c42780 0x5618d6c3c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8d32f640 1 -- 172.21.15.133:0/2415938057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d690a3c0 con 0x5618d6c335d0 2024-09-11T23:49:00.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8db30640 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c42780 0x5618d6c3c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:49:00.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.143+0000 7f7b8d32f640 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 0x5618d6c3b5b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009790 tx=0x7f7b84012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e582e88f54dc23e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:00.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b70ff9640 1 -- 172.21.15.133:0/2415938057 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84028020 con 0x5618d6c335d0 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b70ff9640 1 -- 172.21.15.133:0/2415938057 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f7b840052d0 con 0x5618d6c335d0 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b9196c740 1 -- 172.21.15.133:0/2415938057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b84012ad0 con 0x5618d6c335d0 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b70ff9640 1 -- 172.21.15.133:0/2415938057 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b84033050 con 0x5618d6c335d0 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b9196c740 10 client.18555 fetch_fsmap learned FSMap version 32 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b9196c740 1 -- 172.21.15.133:0/2415938057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5618d6c55210 con 0x5618d6c335d0 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b70ff9640 1 -- 172.21.15.133:0/2415938057 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b84012ad0 con 0x5618d6c335d0 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b70ff9640 1 -- 172.21.15.133:0/2415938057 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7b840037b0 con 0x5618d6c335d0 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b9196c740 10 client.18555 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b9196c740 10 client.18555 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b9196c740 20 client.18555 populate_metadata read hostname 'smithi133' 2024-09-11T23:49:00.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.144+0000 7f7b9196c740 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5618d6c56670 0x5618d6c58a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.145+0000 7f7b9196c740 4 client.18555 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:49:00.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.145+0000 7f7b9196c740 1 -- 172.21.15.133:0/2415938057 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618d6ba6860 con 0x5618d6c56670 2024-09-11T23:49:00.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.145+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5618d6c56670 0x5618d6c58a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.145+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5618d6c56670 0x5618d6c58a60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e696cba251ce817d server_cookie=66c5884ac9d936b2 in_seq=0 out_seq=0 2024-09-11T23:49:00.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.145+0000 7f7b70ff9640 10 client.18555 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:49:00.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b70ff9640 1 -- 172.21.15.133:0/2415938057 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618d6ba6860 con 0x5618d6c56670 2024-09-11T23:49:00.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b70ff9640 10 client.18555 handle_command_reply: tid=0 2024-09-11T23:49:00.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b9196c740 1 client.18555 shutdown 2024-09-11T23:49:00.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b9196c740 1 -- 172.21.15.133:0/2415938057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 msgr2=0x5618d6c3b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b9196c740 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 0x5618d6c3b5b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009790 tx=0x7f7b84012680 comp rx=0 tx=0).stop 2024-09-11T23:49:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b9196c740 1 -- 172.21.15.133:0/2415938057 shutdown_connections 2024-09-11T23:49:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b9196c740 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5618d6c56670 0x5618d6c58a60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b9196c740 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c42780 0x5618d6c3c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b9196c740 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c34b40 0x5618d6c3baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b9196c740 1 --2- 172.21.15.133:0/2415938057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 0x5618d6c3b5b0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b9196c740 1 -- 172.21.15.133:0/2415938057 >> 172.21.15.133:0/2415938057 conn(0x5618d6bfc310 msgr2=0x5618d6c35960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b70ff9640 0 client.18555 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:49:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b9196c740 1 -- 172.21.15.133:0/2415938057 shutdown_connections 2024-09-11T23:49:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.146+0000 7f7b9196c740 1 -- 172.21.15.133:0/2415938057 wait complete. 2024-09-11T23:49:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.147+0000 7f7b9196c740 20 client.18555 trim_cache size 0 max 16384 2024-09-11T23:49:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b9196c740 1 Processor -- start 2024-09-11T23:49:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b9196c740 1 -- start start 2024-09-11T23:49:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c335d0 0x5618d6c3be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 0x5618d6c5e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 0x5618d6c5e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b9196c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881ab660 con 0x5618d6c42780 2024-09-11T23:49:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b9196c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881ad1a0 con 0x5618d6c34b40 2024-09-11T23:49:00.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b9196c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881abd30 con 0x5618d6c335d0 2024-09-11T23:49:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b8db30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 0x5618d6c5e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b8db30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 0x5618d6c5e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39530/0 (socket says 172.21.15.133:39530) 2024-09-11T23:49:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b8db30640 1 -- 172.21.15.133:0/969625172 learned_addr learned my addr 172.21.15.133:0/969625172 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b8d32f640 1 --2- 172.21.15.133:0/969625172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c335d0 0x5618d6c3be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.163+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/969625172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 0x5618d6c5e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.164+0000 7f7b8cb2e640 1 -- 172.21.15.133:0/969625172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c335d0 msgr2=0x5618d6c3be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.164+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/969625172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c335d0 0x5618d6c3be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.164+0000 7f7b8cb2e640 1 -- 172.21.15.133:0/969625172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 msgr2=0x5618d6c5e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.164+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/969625172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 0x5618d6c5e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.164+0000 7f7b8cb2e640 1 -- 172.21.15.133:0/969625172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d6c36120 con 0x5618d6c34b40 2024-09-11T23:49:00.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.164+0000 7f7b8db30640 1 --2- 172.21.15.133:0/969625172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 0x5618d6c5e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:49:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.164+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/969625172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 0x5618d6c5e090 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7b78005940 tx=0x7f7b78040070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a61398e60a57c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.164+0000 7f7b72ffd640 1 -- 172.21.15.133:0/969625172 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78035050 con 0x5618d6c34b40 2024-09-11T23:49:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.164+0000 7f7b72ffd640 1 -- 172.21.15.133:0/969625172 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b78003e10 con 0x5618d6c34b40 2024-09-11T23:49:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.164+0000 7f7b72ffd640 1 -- 172.21.15.133:0/969625172 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78054880 con 0x5618d6c34b40 2024-09-11T23:49:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.165+0000 7f7b9196c740 1 -- 172.21.15.133:0/969625172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 msgr2=0x5618d6c5e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.165+0000 7f7b9196c740 1 --2- 172.21.15.133:0/969625172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 0x5618d6c5e090 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7b78005940 tx=0x7f7b78040070 comp rx=0 tx=0).stop 2024-09-11T23:49:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.165+0000 7f7b9196c740 1 -- 172.21.15.133:0/969625172 shutdown_connections 2024-09-11T23:49:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.165+0000 7f7b9196c740 1 --2- 172.21.15.133:0/969625172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c42780 0x5618d6c5e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.165+0000 7f7b9196c740 1 --2- 172.21.15.133:0/969625172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 0x5618d6c5e090 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.165+0000 7f7b9196c740 1 --2- 172.21.15.133:0/969625172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c335d0 0x5618d6c3be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.165+0000 7f7b9196c740 1 -- 172.21.15.133:0/969625172 >> 172.21.15.133:0/969625172 conn(0x5618d6bfc310 msgr2=0x5618d6c3a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.165+0000 7f7b9196c740 1 -- 172.21.15.133:0/969625172 shutdown_connections 2024-09-11T23:49:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.165+0000 7f7b9196c740 1 -- 172.21.15.133:0/969625172 wait complete. 2024-09-11T23:49:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b9196c740 1 -- start start 2024-09-11T23:49:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b9196c740 1 Processor -- start 2024-09-11T23:49:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 0x5618d6c4da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 0x5618d6c4df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b9196c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c42780 0x5618d6c4e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b9196c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881acad0 con 0x5618d6c335d0 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b9196c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881ac400 con 0x5618d6c34b40 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b9196c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881a9660 con 0x5618d6c42780 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b9196c740 10 client.0 _ll_register_callbacks cb 0x7f7b84009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b8cb2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 0x5618d6c4df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b8d32f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 0x5618d6c4da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b8cb2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 0x5618d6c4df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.133:54318/0 (socket says 172.21.15.133:54318) 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b8cb2e640 1 -- 172.21.15.133:0/1593151964 learned_addr learned my addr 172.21.15.133:0/1593151964 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.166+0000 7f7b8db30640 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c42780 0x5618d6c4e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b8cb2e640 1 -- 172.21.15.133:0/1593151964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c42780 msgr2=0x5618d6c4e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:00.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c42780 0x5618d6c4e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b8cb2e640 1 -- 172.21.15.133:0/1593151964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 msgr2=0x5618d6c4da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 0x5618d6c4da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b8cb2e640 1 -- 172.21.15.133:0/1593151964 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d6cbf150 con 0x5618d6c34b40 2024-09-11T23:49:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b8cb2e640 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 0x5618d6c4df90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7b7802a9a0 tx=0x7f7b7802ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b4706047aca2eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-11T23:49:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b70ff9640 1 -- 172.21.15.133:0/1593151964 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78063020 con 0x5618d6c34b40 2024-09-11T23:49:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b9196c740 1 -- 172.21.15.133:0/1593151964 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b78003e10 con 0x5618d6c34b40 2024-09-11T23:49:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b70ff9640 1 -- 172.21.15.133:0/1593151964 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f7b780547d0 con 0x5618d6c34b40 2024-09-11T23:49:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b70ff9640 1 -- 172.21.15.133:0/1593151964 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b78035070 con 0x5618d6c34b40 2024-09-11T23:49:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b9196c740 10 client.28082 fetch_fsmap learned FSMap version 32 2024-09-11T23:49:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.167+0000 7f7b9196c740 1 -- 172.21.15.133:0/1593151964 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5618d6cc0430 con 0x5618d6c34b40 2024-09-11T23:49:00.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.168+0000 7f7b70ff9640 1 -- 172.21.15.133:0/1593151964 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b78003e10 con 0x5618d6c34b40 2024-09-11T23:49:00.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.168+0000 7f7b70ff9640 1 -- 172.21.15.133:0/1593151964 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7b7805f440 con 0x5618d6c34b40 2024-09-11T23:49:00.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.168+0000 7f7b9196c740 10 client.28082 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:49:00.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.168+0000 7f7b9196c740 10 client.28082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-11T23:49:00.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.168+0000 7f7b9196c740 20 client.28082 populate_metadata read hostname 'smithi133' 2024-09-11T23:49:00.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.168+0000 7f7b9196c740 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5618d6cc0bc0 0x5618d6cc2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:00.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.168+0000 7f7b9196c740 4 client.28082 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:49:00.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.168+0000 7f7b9196c740 1 -- 172.21.15.133:0/1593151964 --> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618d6ba0d70 con 0x5618d6cc0bc0 2024-09-11T23:49:00.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.168+0000 7f7b8d32f640 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5618d6cc0bc0 0x5618d6cc2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:00.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.168+0000 7f7b8d32f640 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5618d6cc0bc0 0x5618d6cc2fe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9dd3a1e16b059d1e server_cookie=71b6e4ac75e41c40 in_seq=0 out_seq=0 2024-09-11T23:49:00.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.168+0000 7f7b70ff9640 10 client.28082 ms_handle_connect on v2:172.21.15.138:6838/2407895128 2024-09-11T23:49:00.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b70ff9640 1 -- 172.21.15.133:0/1593151964 <== mds.2 v2:172.21.15.138:6838/2407895128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618d6ba0d70 con 0x5618d6cc0bc0 2024-09-11T23:49:00.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b70ff9640 10 client.28082 handle_command_reply: tid=0 2024-09-11T23:49:00.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b9196c740 1 client.28082 shutdown 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b9196c740 1 -- 172.21.15.133:0/1593151964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 msgr2=0x5618d6c4df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b9196c740 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 0x5618d6c4df90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7b7802a9a0 tx=0x7f7b7802ac00 comp rx=0 tx=0).stop 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b9196c740 1 -- 172.21.15.133:0/1593151964 shutdown_connections 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b9196c740 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.138:6838/2407895128,v1:172.21.15.138:6839/2407895128] conn(0x5618d6cc0bc0 0x5618d6cc2fe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b9196c740 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618d6c42780 0x5618d6c4e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b9196c740 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d6c34b40 0x5618d6c4df90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b9196c740 1 --2- 172.21.15.133:0/1593151964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618d6c335d0 0x5618d6c4da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b9196c740 1 -- 172.21.15.133:0/1593151964 >> 172.21.15.133:0/1593151964 conn(0x5618d6bfc310 msgr2=0x5618d6c3be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.169+0000 7f7b70ff9640 0 client.28082 ms_handle_reset on v2:172.21.15.138:6838/2407895128 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.170+0000 7f7b9196c740 1 -- 172.21.15.133:0/1593151964 shutdown_connections 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.170+0000 7f7b9196c740 1 -- 172.21.15.133:0/1593151964 wait complete. 2024-09-11T23:49:00.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.170+0000 7f7b9196c740 20 client.28082 trim_cache size 0 max 16384 2024-09-11T23:49:00.176 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:49:00.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.170+0000 7f7b8f593640 1 -- 172.21.15.133:0/3840880189 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b740a2cf0 msgr2=0x7f7b740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:00.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.170+0000 7f7b8f593640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b740a2cf0 0x7f7b740a51b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009230 tx=0x7f7b78005e50 comp rx=0 tx=0).stop 2024-09-11T23:49:00.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.170+0000 7f7b8f593640 1 -- 172.21.15.133:0/3840880189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b88102ef0 msgr2=0x7f7b881a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:00.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.170+0000 7f7b8f593640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b88102ef0 0x7f7b881a4330 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000eb10 tx=0x7f7b8000c2d0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.171+0000 7f7b8f593640 1 -- 172.21.15.133:0/3840880189 shutdown_connections 2024-09-11T23:49:00.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.171+0000 7f7b8f593640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b740a2cf0 0x7f7b740a51b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.171+0000 7f7b8f593640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b88102ef0 0x7f7b881a4330 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.171+0000 7f7b8f593640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b88102570 0x7f7b881997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.171+0000 7f7b8f593640 1 --2- 172.21.15.133:0/3840880189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b88100e20 0x7f7b881992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:00.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.171+0000 7f7b8f593640 1 -- 172.21.15.133:0/3840880189 >> 172.21.15.133:0/3840880189 conn(0x7f7b880fc7d0 msgr2=0x7f7b880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:00.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.171+0000 7f7b8f593640 1 -- 172.21.15.133:0/3840880189 shutdown_connections 2024-09-11T23:49:00.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:00.171+0000 7f7b8f593640 1 -- 172.21.15.133:0/3840880189 wait complete. 2024-09-11T23:49:00.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:00 smithi133 ceph-mon[31430]: pgmap v383: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-11T23:49:01.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:00 smithi178 ceph-mon[35882]: pgmap v383: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-11T23:49:01.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:49:01.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:00 smithi138 ceph-mon[35142]: pgmap v383: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-11T23:49:01.317 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:01.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.767+0000 7f8100d8a640 1 -- 172.21.15.133:0/757854663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80fc1082a0 msgr2=0x7f80fc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:01.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.767+0000 7f8100d8a640 1 --2- 172.21.15.133:0/757854663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80fc1082a0 0x7f80fc10ec40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009cd0 tx=0x7f80f402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.767+0000 7f8100d8a640 1 -- 172.21.15.133:0/757854663 shutdown_connections 2024-09-11T23:49:01.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.767+0000 7f8100d8a640 1 --2- 172.21.15.133:0/757854663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80fc1082a0 0x7f80fc10ec40 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.767+0000 7f8100d8a640 1 --2- 172.21.15.133:0/757854663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80fc1036e0 0x7f80fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.767+0000 7f8100d8a640 1 --2- 172.21.15.133:0/757854663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80fc102d30 0x7f80fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.767+0000 7f8100d8a640 1 -- 172.21.15.133:0/757854663 >> 172.21.15.133:0/757854663 conn(0x7f80fc0fc7d0 msgr2=0x7f80fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:01.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.767+0000 7f8100d8a640 1 -- 172.21.15.133:0/757854663 shutdown_connections 2024-09-11T23:49:01.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.767+0000 7f8100d8a640 1 -- 172.21.15.133:0/757854663 wait complete. 2024-09-11T23:49:01.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.768+0000 7f8100d8a640 1 Processor -- start 2024-09-11T23:49:01.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.768+0000 7f8100d8a640 1 -- start start 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.768+0000 7f8100d8a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80fc102d30 0x7f80fc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.768+0000 7f8100d8a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80fc1036e0 0x7f80fc19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.768+0000 7f8100d8a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80fc1082a0 0x7f80fc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.768+0000 7f8100d8a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1120a0 con 0x7f80fc102d30 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.768+0000 7f8100d8a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc111f20 con 0x7f80fc1082a0 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.768+0000 7f8100d8a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc112220 con 0x7f80fc1036e0 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.768+0000 7f80fa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80fc102d30 0x7f80fc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80fa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80fc102d30 0x7f80fc19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39558/0 (socket says 172.21.15.133:39558) 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80fa59c640 1 -- 172.21.15.133:0/552053859 learned_addr learned my addr 172.21.15.133:0/552053859 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80f3fff640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80fc1036e0 0x7f80fc19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80fad9d640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80fc1082a0 0x7f80fc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80fa59c640 1 -- 172.21.15.133:0/552053859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80fc1036e0 msgr2=0x7f80fc19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:01.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80fa59c640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80fc1036e0 0x7f80fc19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80fa59c640 1 -- 172.21.15.133:0/552053859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80fc1082a0 msgr2=0x7f80fc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:01.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80fa59c640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80fc1082a0 0x7f80fc19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80fa59c640 1 -- 172.21.15.133:0/552053859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80fc19f6a0 con 0x7f80fc102d30 2024-09-11T23:49:01.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80fa59c640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80fc102d30 0x7f80fc19a6f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00c920 tx=0x7f80ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:01.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80f37fe640 1 -- 172.21.15.133:0/552053859 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec007c70 con 0x7f80fc102d30 2024-09-11T23:49:01.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80f37fe640 1 -- 172.21.15.133:0/552053859 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec007e10 con 0x7f80fc102d30 2024-09-11T23:49:01.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f80f37fe640 1 -- 172.21.15.133:0/552053859 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec018890 con 0x7f80fc102d30 2024-09-11T23:49:01.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f8100d8a640 1 -- 172.21.15.133:0/552053859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80fc19f990 con 0x7f80fc102d30 2024-09-11T23:49:01.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.769+0000 7f8100d8a640 1 -- 172.21.15.133:0/552053859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80fc1a7270 con 0x7f80fc102d30 2024-09-11T23:49:01.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f80f37fe640 1 -- 172.21.15.133:0/552053859 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f80ec018a70 con 0x7f80fc102d30 2024-09-11T23:49:01.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f8103163740 1 Processor -- start 2024-09-11T23:49:01.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f8103163740 1 -- start start 2024-09-11T23:49:01.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f8103163740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f1d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f8103163740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f1d5d0 0x5576c4f1da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f8103163740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f28370 0x5576c4f2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f8103163740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5576c4e832f0 con 0x5576c4f28370 2024-09-11T23:49:01.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f8103163740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5576c4e2acf0 con 0x5576c4f1eb40 2024-09-11T23:49:01.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f8103163740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5576c4e7f2b0 con 0x5576c4f1d5d0 2024-09-11T23:49:01.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f80f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f28370 0x5576c4f2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f80f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f28370 0x5576c4f2a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39570/0 (socket says 172.21.15.133:39570) 2024-09-11T23:49:01.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f80f3fff640 1 -- 172.21.15.133:0/679920092 learned_addr learned my addr 172.21.15.133:0/679920092 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:01.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f80fad9d640 1 --2- 172.21.15.133:0/679920092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f1d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f80f37fe640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80e80a2cf0 0x7f80e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.772+0000 7f80fa59c640 1 --2- 172.21.15.133:0/679920092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f1d5d0 0x5576c4f1da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80fad9d640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80e80a2cf0 0x7f80e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80f3fff640 1 -- 172.21.15.133:0/679920092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f1d5d0 msgr2=0x5576c4f1da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:01.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80f3fff640 1 --2- 172.21.15.133:0/679920092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f1d5d0 0x5576c4f1da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80f3fff640 1 -- 172.21.15.133:0/679920092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 msgr2=0x5576c4f1d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:01.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80f3fff640 1 --2- 172.21.15.133:0/679920092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f1d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80f3fff640 1 -- 172.21.15.133:0/679920092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576c4e0f3a0 con 0x5576c4f28370 2024-09-11T23:49:01.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80f37fe640 1 -- 172.21.15.133:0/552053859 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f80ec0cf4b0 con 0x7f80fc102d30 2024-09-11T23:49:01.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80f3fff640 1 --2- 172.21.15.133:0/679920092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f28370 0x5576c4f2a760 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f80e400bba0 tx=0x7f80e40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23112d406d1f7bc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:01.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80fad9d640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80e80a2cf0 0x7f80e80a51b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f80f4004ab0 tx=0x7f80f402a8e0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:01.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80f0ff9640 1 -- 172.21.15.133:0/679920092 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e401c480 con 0x5576c4f28370 2024-09-11T23:49:01.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80f0ff9640 1 -- 172.21.15.133:0/679920092 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e4011920 con 0x5576c4f28370 2024-09-11T23:49:01.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.773+0000 7f80f0ff9640 1 -- 172.21.15.133:0/679920092 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e401a8c0 con 0x5576c4f28370 2024-09-11T23:49:01.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.774+0000 7f8103163740 1 -- 172.21.15.133:0/679920092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f28370 msgr2=0x5576c4f2a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:01.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.774+0000 7f8103163740 1 --2- 172.21.15.133:0/679920092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f28370 0x5576c4f2a760 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f80e400bba0 tx=0x7f80e40107a0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.774+0000 7f8103163740 1 -- 172.21.15.133:0/679920092 shutdown_connections 2024-09-11T23:49:01.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.774+0000 7f8103163740 1 --2- 172.21.15.133:0/679920092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f28370 0x5576c4f2a760 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.774+0000 7f8103163740 1 --2- 172.21.15.133:0/679920092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f1d5d0 0x5576c4f1da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.774+0000 7f8103163740 1 --2- 172.21.15.133:0/679920092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f1d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.774+0000 7f8103163740 1 -- 172.21.15.133:0/679920092 >> 172.21.15.133:0/679920092 conn(0x5576c4ee6310 msgr2=0x5576c4f19c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:01.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.774+0000 7f8103163740 1 -- 172.21.15.133:0/679920092 shutdown_connections 2024-09-11T23:49:01.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.774+0000 7f8103163740 1 -- 172.21.15.133:0/679920092 wait complete. 2024-09-11T23:49:01.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.774+0000 7f8103163740 1 -- start start 2024-09-11T23:49:01.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.774+0000 7f8103163740 1 Processor -- start 2024-09-11T23:49:01.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f8103163740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f27d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f8103163740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f1eb40 0x5576c4f24fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f8103163740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f28370 0x5576c4f255d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f8103163740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5576c4f2b070 con 0x5576c4f1d5d0 2024-09-11T23:49:01.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f8103163740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5576c4e3e060 con 0x5576c4f28370 2024-09-11T23:49:01.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f8103163740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5576c4f2b1f0 con 0x5576c4f1eb40 2024-09-11T23:49:01.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f8103163740 10 client.0 _ll_register_callbacks cb 0x5576c4b918c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:49:01.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80fa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f27d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80fa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f27d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39572/0 (socket says 172.21.15.133:39572) 2024-09-11T23:49:01.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80fa59c640 1 -- 172.21.15.133:0/916828020 learned_addr learned my addr 172.21.15.133:0/916828020 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:01.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80f3fff640 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f1eb40 0x5576c4f24fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80fa59c640 1 -- 172.21.15.133:0/916828020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f1eb40 msgr2=0x5576c4f24fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:01.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80fa59c640 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f1eb40 0x5576c4f24fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80fa59c640 1 -- 172.21.15.133:0/916828020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f28370 msgr2=0x5576c4f255d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:01.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80fa59c640 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f28370 0x5576c4f255d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80fa59c640 1 -- 172.21.15.133:0/916828020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576c4bf43c0 con 0x5576c4f1d5d0 2024-09-11T23:49:01.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80fa59c640 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f27d30 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f80ec09b9a0 tx=0x7f80ec09b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8217b122ba802c75 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:01.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:49:01.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80e1ffb640 1 -- 172.21.15.133:0/916828020 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec09ba10 con 0x5576c4f1d5d0 2024-09-11T23:49:01.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80e1ffb640 1 -- 172.21.15.133:0/916828020 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f80ec098080 con 0x5576c4f1d5d0 2024-09-11T23:49:01.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f8103163740 1 -- 172.21.15.133:0/916828020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e4011920 con 0x5576c4f1d5d0 2024-09-11T23:49:01.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f80e1ffb640 1 -- 172.21.15.133:0/916828020 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80ec09fdc0 con 0x5576c4f1d5d0 2024-09-11T23:49:01.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f8103163740 10 client.18573 fetch_fsmap learned FSMap version 32 2024-09-11T23:49:01.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.775+0000 7f8103163740 1 -- 172.21.15.133:0/916828020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576c4f36b30 con 0x5576c4f1d5d0 2024-09-11T23:49:01.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.776+0000 7f80e1ffb640 1 -- 172.21.15.133:0/916828020 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e4011920 con 0x5576c4f1d5d0 2024-09-11T23:49:01.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.776+0000 7f80e1ffb640 1 -- 172.21.15.133:0/916828020 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f80ec0a41f0 con 0x5576c4f1d5d0 2024-09-11T23:49:01.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.776+0000 7f8103163740 10 client.18573 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:49:01.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.776+0000 7f8103163740 10 client.18573 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:49:01.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.776+0000 7f8103163740 20 client.18573 populate_metadata read hostname 'smithi133' 2024-09-11T23:49:01.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.776+0000 7f8103163740 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5576c4f37f40 0x5576c4f3a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.776+0000 7f8103163740 4 client.18573 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.776+0000 7f8103163740 1 -- 172.21.15.133:0/916828020 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576c4e90860 con 0x5576c4f37f40 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.776+0000 7f80f3fff640 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5576c4f37f40 0x5576c4f3a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f80f3fff640 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5576c4f37f40 0x5576c4f3a370 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57ed3bf05040c5cf server_cookie=e210d05e73800f31 in_seq=0 out_seq=0 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f80e1ffb640 10 client.18573 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f80e1ffb640 1 -- 172.21.15.133:0/916828020 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576c4e90860 con 0x5576c4f37f40 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f80e1ffb640 10 client.18573 handle_command_reply: tid=0 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f8103163740 1 client.18573 shutdown 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f8103163740 1 -- 172.21.15.133:0/916828020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 msgr2=0x5576c4f27d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f8103163740 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f27d30 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f80ec09b9a0 tx=0x7f80ec09b370 comp rx=0 tx=0).stop 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f8103163740 1 -- 172.21.15.133:0/916828020 shutdown_connections 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f8103163740 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5576c4f37f40 0x5576c4f3a370 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f8103163740 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f28370 0x5576c4f255d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f8103163740 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f1eb40 0x5576c4f24fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f80e1ffb640 0 client.18573 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:49:01.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f8103163740 1 --2- 172.21.15.133:0/916828020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f27d30 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f8103163740 1 -- 172.21.15.133:0/916828020 >> 172.21.15.133:0/916828020 conn(0x5576c4ee6310 msgr2=0x5576c4f1f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:01.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f8103163740 1 -- 172.21.15.133:0/916828020 shutdown_connections 2024-09-11T23:49:01.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.777+0000 7f8103163740 1 -- 172.21.15.133:0/916828020 wait complete. 2024-09-11T23:49:01.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.778+0000 7f8103163740 20 client.18573 trim_cache size 0 max 16384 2024-09-11T23:49:01.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.790+0000 7f8103163740 1 Processor -- start 2024-09-11T23:49:01.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.790+0000 7f8103163740 1 -- start start 2024-09-11T23:49:01.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.790+0000 7f8103163740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f34310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.790+0000 7f8103163740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f29760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f8103163740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f28370 0x5576c4f29ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f8103163740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1af820 con 0x5576c4f1d5d0 2024-09-11T23:49:01.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f8103163740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1b1360 con 0x5576c4f1eb40 2024-09-11T23:49:01.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f8103163740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1afef0 con 0x5576c4f28370 2024-09-11T23:49:01.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80fa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f34310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80fa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f34310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39582/0 (socket says 172.21.15.133:39582) 2024-09-11T23:49:01.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80fa59c640 1 -- 172.21.15.133:0/4066219800 learned_addr learned my addr 172.21.15.133:0/4066219800 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:01.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80fa59c640 1 -- 172.21.15.133:0/4066219800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f28370 msgr2=0x5576c4f29ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:01.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80fa59c640 1 --2- 172.21.15.133:0/4066219800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f28370 0x5576c4f29ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80f3fff640 1 --2- 172.21.15.133:0/4066219800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f29760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80fa59c640 1 -- 172.21.15.133:0/4066219800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 msgr2=0x5576c4f29760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80fa59c640 1 --2- 172.21.15.133:0/4066219800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f29760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80fa59c640 1 -- 172.21.15.133:0/4066219800 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576c4f3a9f0 con 0x5576c4f1d5d0 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80fa59c640 1 --2- 172.21.15.133:0/4066219800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f34310 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f80ec097e70 tx=0x7f80ec0cf330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abce7e6a546fd53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80e3fff640 1 -- 172.21.15.133:0/4066219800 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec0a4680 con 0x5576c4f1d5d0 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80e3fff640 1 -- 172.21.15.133:0/4066219800 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec0a41f0 con 0x5576c4f1d5d0 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.791+0000 7f80e3fff640 1 -- 172.21.15.133:0/4066219800 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec086390 con 0x5576c4f1d5d0 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.792+0000 7f8103163740 1 -- 172.21.15.133:0/4066219800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 msgr2=0x5576c4f34310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.792+0000 7f8103163740 1 --2- 172.21.15.133:0/4066219800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f34310 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f80ec097e70 tx=0x7f80ec0cf330 comp rx=0 tx=0).stop 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.792+0000 7f8103163740 1 -- 172.21.15.133:0/4066219800 shutdown_connections 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.792+0000 7f8103163740 1 --2- 172.21.15.133:0/4066219800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f28370 0x5576c4f29ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.792+0000 7f8103163740 1 --2- 172.21.15.133:0/4066219800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f29760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.792+0000 7f8103163740 1 --2- 172.21.15.133:0/4066219800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f34310 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.792+0000 7f8103163740 1 -- 172.21.15.133:0/4066219800 >> 172.21.15.133:0/4066219800 conn(0x5576c4ee6310 msgr2=0x5576c4f2a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:01.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.792+0000 7f8103163740 1 -- 172.21.15.133:0/4066219800 shutdown_connections 2024-09-11T23:49:01.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.792+0000 7f8103163740 1 -- 172.21.15.133:0/4066219800 wait complete. 2024-09-11T23:49:01.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.792+0000 7f8103163740 1 -- start start 2024-09-11T23:49:01.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f8103163740 1 Processor -- start 2024-09-11T23:49:01.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f8103163740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f30490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f8103163740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f309d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f8103163740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f28370 0x5576c4f2b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f8103163740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1b0c90 con 0x5576c4f1d5d0 2024-09-11T23:49:01.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f8103163740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1b05c0 con 0x5576c4f1eb40 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f8103163740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1ad820 con 0x5576c4f28370 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f8103163740 10 client.0 _ll_register_callbacks cb 0x7f80ec00fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80fa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f30490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80fa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f30490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39584/0 (socket says 172.21.15.133:39584) 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80fa59c640 1 -- 172.21.15.133:0/2302966377 learned_addr learned my addr 172.21.15.133:0/2302966377 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80fad9d640 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f28370 0x5576c4f2b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80f3fff640 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80fa59c640 1 -- 172.21.15.133:0/2302966377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f28370 msgr2=0x5576c4f2b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80fa59c640 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f28370 0x5576c4f2b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80fa59c640 1 -- 172.21.15.133:0/2302966377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 msgr2=0x5576c4f309d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80fa59c640 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f309d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80fa59c640 1 -- 172.21.15.133:0/2302966377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576c4f40750 con 0x5576c4f1d5d0 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80f3fff640 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f309d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80fa59c640 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f30490 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f80ec007590 tx=0x7f80ec00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fd468936508185 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.793+0000 7f80e1ffb640 1 -- 172.21.15.133:0/2302966377 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec098200 con 0x5576c4f1d5d0 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f80e1ffb640 1 -- 172.21.15.133:0/2302966377 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f80ec097760 con 0x5576c4f1d5d0 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f8103163740 1 -- 172.21.15.133:0/2302966377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ec0a41f0 con 0x5576c4f1d5d0 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f80e1ffb640 1 -- 172.21.15.133:0/2302966377 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80ec0aa070 con 0x5576c4f1d5d0 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f8103163740 10 client.18579 fetch_fsmap learned FSMap version 32 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f8103163740 1 -- 172.21.15.133:0/2302966377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576c4fa3120 con 0x5576c4f1d5d0 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f80e1ffb640 1 -- 172.21.15.133:0/2302966377 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec0a41f0 con 0x5576c4f1d5d0 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f80e1ffb640 1 -- 172.21.15.133:0/2302966377 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f80ec0ba220 con 0x5576c4f1d5d0 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f8103163740 10 client.18579 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:49:01.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f8103163740 10 client.18579 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f8103163740 20 client.18579 populate_metadata read hostname 'smithi133' 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f8103163740 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5576c4fa35f0 0x5576c4fa59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f8103163740 4 client.18579 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f8103163740 1 -- 172.21.15.133:0/2302966377 --> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576c4e8ad70 con 0x5576c4fa35f0 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.794+0000 7f80f3fff640 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5576c4fa35f0 0x5576c4fa59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f80f3fff640 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5576c4fa35f0 0x5576c4fa59e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79285768aa65c3e0 server_cookie=de34ffec5a4b7448 in_seq=0 out_seq=0 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f80e1ffb640 10 client.18579 ms_handle_connect on v2:172.21.15.178:6836/1040997001 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f80e1ffb640 1 -- 172.21.15.133:0/2302966377 <== mds.1 v2:172.21.15.178:6836/1040997001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576c4e8ad70 con 0x5576c4fa35f0 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f80e1ffb640 10 client.18579 handle_command_reply: tid=0 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 1 client.18579 shutdown 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 1 -- 172.21.15.133:0/2302966377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 msgr2=0x5576c4f30490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f30490 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f80ec007590 tx=0x7f80ec00ff30 comp rx=0 tx=0).stop 2024-09-11T23:49:01.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 1 -- 172.21.15.133:0/2302966377 shutdown_connections 2024-09-11T23:49:01.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.178:6836/1040997001,v1:172.21.15.178:6837/1040997001] conn(0x5576c4fa35f0 0x5576c4fa59e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576c4f28370 0x5576c4f2b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576c4f1eb40 0x5576c4f309d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 1 --2- 172.21.15.133:0/2302966377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c4f1d5d0 0x5576c4f30490 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 1 -- 172.21.15.133:0/2302966377 >> 172.21.15.133:0/2302966377 conn(0x5576c4ee6310 msgr2=0x5576c4f184e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:01.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f80e1ffb640 0 client.18579 ms_handle_reset on v2:172.21.15.178:6836/1040997001 2024-09-11T23:49:01.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 1 -- 172.21.15.133:0/2302966377 shutdown_connections 2024-09-11T23:49:01.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 1 -- 172.21.15.133:0/2302966377 wait complete. 2024-09-11T23:49:01.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.795+0000 7f8103163740 20 client.18579 trim_cache size 0 max 16384 2024-09-11T23:49:01.815 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:49:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 -- 172.21.15.133:0/552053859 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80e80a2cf0 msgr2=0x7f80e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80e80a2cf0 0x7f80e80a51b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f80f4004ab0 tx=0x7f80f402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 -- 172.21.15.133:0/552053859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80fc102d30 msgr2=0x7f80fc19a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80fc102d30 0x7f80fc19a6f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00c920 tx=0x7f80ec00cde0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 -- 172.21.15.133:0/552053859 shutdown_connections 2024-09-11T23:49:01.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f80e80a2cf0 0x7f80e80a51b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80fc1082a0 0x7f80fc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80fc1036e0 0x7f80fc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 --2- 172.21.15.133:0/552053859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80fc102d30 0x7f80fc19a6f0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:01.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 -- 172.21.15.133:0/552053859 >> 172.21.15.133:0/552053859 conn(0x7f80fc0fc7d0 msgr2=0x7f80fc0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:01.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 -- 172.21.15.133:0/552053859 shutdown_connections 2024-09-11T23:49:01.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:01.796+0000 7f8100d8a640 1 -- 172.21.15.133:0/552053859 wait complete. 2024-09-11T23:49:02.533 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:49:02.784 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:03.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:02 smithi133 ceph-mon[31430]: pgmap v384: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:49:03.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:02 smithi178 ceph-mon[35882]: pgmap v384: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:49:03.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:02 smithi138 ceph-mon[35142]: pgmap v384: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:49:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.190+0000 7ff15af97640 1 -- 172.21.15.133:0/712121349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff15410e4f0 msgr2=0x7ff154110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.190+0000 7ff15af97640 1 --2- 172.21.15.133:0/712121349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff15410e4f0 0x7ff154110900 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009900 tx=0x7ff13c02a410 comp rx=0 tx=0).stop 2024-09-11T23:49:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.190+0000 7ff15af97640 1 -- 172.21.15.133:0/712121349 shutdown_connections 2024-09-11T23:49:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.190+0000 7ff15af97640 1 --2- 172.21.15.133:0/712121349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff15410e4f0 0x7ff154110900 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.190+0000 7ff15af97640 1 --2- 172.21.15.133:0/712121349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff154101380 0x7ff15410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.190+0000 7ff15af97640 1 --2- 172.21.15.133:0/712121349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1541009d0 0x7ff154100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.190+0000 7ff15af97640 1 -- 172.21.15.133:0/712121349 >> 172.21.15.133:0/712121349 conn(0x7ff1540fc7d0 msgr2=0x7ff1540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.190+0000 7ff15af97640 1 -- 172.21.15.133:0/712121349 shutdown_connections 2024-09-11T23:49:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.190+0000 7ff15af97640 1 -- 172.21.15.133:0/712121349 wait complete. 2024-09-11T23:49:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff15af97640 1 Processor -- start 2024-09-11T23:49:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff15af97640 1 -- start start 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff15af97640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1541009d0 0x7ff15419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff15af97640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff154101380 0x7ff15419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff15af97640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15410e4f0 0x7ff15419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff15af97640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff154114230 con 0x7ff1541009d0 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff15af97640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1541140b0 con 0x7ff15410e4f0 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff15af97640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1541143b0 con 0x7ff154101380 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff158d33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1541009d0 0x7ff15419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff158d33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1541009d0 0x7ff15419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59576/0 (socket says 172.21.15.133:59576) 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff158d33640 1 -- 172.21.15.133:0/1283124445 learned_addr learned my addr 172.21.15.133:0/1283124445 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff159534640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15410e4f0 0x7ff15419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff158d33640 1 -- 172.21.15.133:0/1283124445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff154101380 msgr2=0x7ff15419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff153fff640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff154101380 0x7ff15419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff158d33640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff154101380 0x7ff15419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff158d33640 1 -- 172.21.15.133:0/1283124445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15410e4f0 msgr2=0x7ff15419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff158d33640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15410e4f0 0x7ff15419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff158d33640 1 -- 172.21.15.133:0/1283124445 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15419f600 con 0x7ff1541009d0 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff153fff640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff154101380 0x7ff15419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.191+0000 7ff159534640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15410e4f0 0x7ff15419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.192+0000 7ff158d33640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1541009d0 0x7ff15419a650 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c920 tx=0x7ff14c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.192+0000 7ff151ffb640 1 -- 172.21.15.133:0/1283124445 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c007c70 con 0x7ff1541009d0 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.192+0000 7ff15af97640 1 -- 172.21.15.133:0/1283124445 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15419f8f0 con 0x7ff1541009d0 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.192+0000 7ff151ffb640 1 -- 172.21.15.133:0/1283124445 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff14c007e10 con 0x7ff1541009d0 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.192+0000 7ff151ffb640 1 -- 172.21.15.133:0/1283124445 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c018890 con 0x7ff1541009d0 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.192+0000 7ff15af97640 1 -- 172.21.15.133:0/1283124445 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1541a71d0 con 0x7ff1541009d0 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.194+0000 7ff151ffb640 1 -- 172.21.15.133:0/1283124445 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff14c018ad0 con 0x7ff1541009d0 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.194+0000 7ff15d370740 1 Processor -- start 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.194+0000 7ff15d370740 1 -- start start 2024-09-11T23:49:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff15d370740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff15d370740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efb65d0 0x56196efb6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff15d370740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efc1370 0x56196efc3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff15d370740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56196ef1c2f0 con 0x56196efb7b40 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff15d370740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56196eec3cf0 con 0x56196efb65d0 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff15d370740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56196ef182b0 con 0x56196efc1370 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff151ffb640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff1400a2cf0 0x7ff1400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff159534640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff159534640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efb6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59580/0 (socket says 172.21.15.133:59580) 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff159534640 1 -- 172.21.15.133:0/50930705 learned_addr learned my addr 172.21.15.133:0/50930705 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff158d33640 1 --2- 172.21.15.133:0/50930705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efb65d0 0x56196efb6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff159534640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff1400a2cf0 0x7ff1400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff159534640 1 -- 172.21.15.133:0/50930705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efc1370 msgr2=0x56196efc3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff151ffb640 1 -- 172.21.15.133:0/1283124445 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7ff14c0cf670 con 0x7ff1541009d0 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff159534640 1 --2- 172.21.15.133:0/50930705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efc1370 0x56196efc3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff159534640 1 -- 172.21.15.133:0/50930705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efb65d0 msgr2=0x56196efb6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff159534640 1 --2- 172.21.15.133:0/50930705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efb65d0 0x56196efb6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff159534640 1 -- 172.21.15.133:0/50930705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56196eea83a0 con 0x56196efb7b40 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff159534640 1 --2- 172.21.15.133:0/50930705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efb6090 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff13c005940 tx=0x7ff13c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fdcf97689940ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff14affd640 1 -- 172.21.15.133:0/50930705 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c035070 con 0x56196efb7b40 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.195+0000 7ff159534640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff1400a2cf0 0x7ff1400a51b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff13c04b000 tx=0x7ff13c049000 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.196+0000 7ff14affd640 1 -- 172.21.15.133:0/50930705 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff13c04c070 con 0x56196efb7b40 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.196+0000 7ff14affd640 1 -- 172.21.15.133:0/50930705 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c03a940 con 0x56196efb7b40 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.196+0000 7ff15d370740 1 -- 172.21.15.133:0/50930705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 msgr2=0x56196efb6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.196+0000 7ff15d370740 1 --2- 172.21.15.133:0/50930705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efb6090 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff13c005940 tx=0x7ff13c0040c0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.196+0000 7ff15d370740 1 -- 172.21.15.133:0/50930705 shutdown_connections 2024-09-11T23:49:03.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.196+0000 7ff15d370740 1 --2- 172.21.15.133:0/50930705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efc1370 0x56196efc3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.196+0000 7ff15d370740 1 --2- 172.21.15.133:0/50930705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efb65d0 0x56196efb6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.196+0000 7ff15d370740 1 --2- 172.21.15.133:0/50930705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efb6090 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.196+0000 7ff15d370740 1 -- 172.21.15.133:0/50930705 >> 172.21.15.133:0/50930705 conn(0x56196ef7f310 msgr2=0x56196efb2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.196+0000 7ff15d370740 1 -- 172.21.15.133:0/50930705 shutdown_connections 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.196+0000 7ff15d370740 1 -- 172.21.15.133:0/50930705 wait complete. 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff15d370740 1 -- start start 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff15d370740 1 Processor -- start 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff15d370740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb65d0 0x56196efc0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff15d370740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efb7b40 0x56196efbe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff15d370740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efc1370 0x56196efbe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff15d370740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56196efc4070 con 0x56196efc1370 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff15d370740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56196eed7060 con 0x56196efb7b40 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff15d370740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56196efc41f0 con 0x56196efb65d0 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff15d370740 10 client.0 _ll_register_callbacks cb 0x56196ec2a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:49:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff159534640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efc1370 0x56196efbe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff159534640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efc1370 0x56196efbe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59592/0 (socket says 172.21.15.133:59592) 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff159534640 1 -- 172.21.15.133:0/403706556 learned_addr learned my addr 172.21.15.133:0/403706556 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff158d33640 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb65d0 0x56196efc0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff153fff640 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efb7b40 0x56196efbe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff159534640 1 -- 172.21.15.133:0/403706556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb65d0 msgr2=0x56196efc0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff159534640 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb65d0 0x56196efc0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff159534640 1 -- 172.21.15.133:0/403706556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efb7b40 msgr2=0x56196efbe070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff159534640 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efb7b40 0x56196efbe070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.197+0000 7ff159534640 1 -- 172.21.15.133:0/403706556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56196ec8d3c0 con 0x56196efc1370 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff159534640 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efc1370 0x56196efbe800 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff13c02a930 tx=0x7ff13c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8075a49b5576579 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff133fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff133fff640 1 -- 172.21.15.133:0/403706556 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c05e060 con 0x56196efc1370 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff15d370740 1 -- 172.21.15.133:0/403706556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff13c04c070 con 0x56196efc1370 2024-09-11T23:49:03.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff133fff640 1 -- 172.21.15.133:0/403706556 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7ff13c03a6c0 con 0x56196efc1370 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff133fff640 1 -- 172.21.15.133:0/403706556 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff13c035070 con 0x56196efc1370 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff15d370740 10 client.18591 fetch_fsmap learned FSMap version 32 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff15d370740 1 -- 172.21.15.133:0/403706556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56196efcfaf0 con 0x56196efc1370 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff133fff640 1 -- 172.21.15.133:0/403706556 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff13c04c070 con 0x56196efc1370 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff133fff640 1 -- 172.21.15.133:0/403706556 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff13c050500 con 0x56196efc1370 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff15d370740 10 client.18591 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff15d370740 10 client.18591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff15d370740 20 client.18591 populate_metadata read hostname 'smithi133' 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.198+0000 7ff15d370740 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56196efd0e40 0x56196efd3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.199+0000 7ff15d370740 4 client.18591 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.199+0000 7ff15d370740 1 -- 172.21.15.133:0/403706556 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56196ef29860 con 0x56196efd0e40 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.199+0000 7ff158d33640 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56196efd0e40 0x56196efd3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.199+0000 7ff158d33640 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56196efd0e40 0x56196efd3230 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace01126a18e6725 server_cookie=3c54cb7bd4a40707 in_seq=0 out_seq=0 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.199+0000 7ff133fff640 10 client.18591 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff133fff640 1 -- 172.21.15.133:0/403706556 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56196ef29860 con 0x56196efd0e40 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff133fff640 10 client.18591 handle_command_reply: tid=0 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 1 client.18591 shutdown 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 1 -- 172.21.15.133:0/403706556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efc1370 msgr2=0x56196efbe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efc1370 0x56196efbe800 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff13c02a930 tx=0x7ff13c005a20 comp rx=0 tx=0).stop 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 1 -- 172.21.15.133:0/403706556 shutdown_connections 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56196efd0e40 0x56196efd3230 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efc1370 0x56196efbe800 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efb7b40 0x56196efbe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 1 --2- 172.21.15.133:0/403706556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb65d0 0x56196efc0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff133fff640 0 client.18591 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:49:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 1 -- 172.21.15.133:0/403706556 >> 172.21.15.133:0/403706556 conn(0x56196ef7f310 msgr2=0x56196efb8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:03.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 1 -- 172.21.15.133:0/403706556 shutdown_connections 2024-09-11T23:49:03.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 1 -- 172.21.15.133:0/403706556 wait complete. 2024-09-11T23:49:03.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.200+0000 7ff15d370740 20 client.18591 trim_cache size 0 max 16384 2024-09-11T23:49:03.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff15d370740 1 Processor -- start 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff15d370740 1 -- start start 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff15d370740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb65d0 0x56196efbe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff15d370740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb7b40 0x56196efb9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff15d370740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efc1370 0x56196efb9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff15d370740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff1541af8f0 con 0x56196efb65d0 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff15d370740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1541b1430 con 0x56196efc1370 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff15d370740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1541affc0 con 0x56196efb7b40 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff158d33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb65d0 0x56196efbe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff158d33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb65d0 0x56196efbe040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59606/0 (socket says 172.21.15.133:59606) 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff158d33640 1 -- 172.21.15.133:0/1160930990 learned_addr learned my addr 172.21.15.133:0/1160930990 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff159534640 1 --2- 172.21.15.133:0/1160930990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efc1370 0x56196efb9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff153fff640 1 --2- 172.21.15.133:0/1160930990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb7b40 0x56196efb9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff158d33640 1 -- 172.21.15.133:0/1160930990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb7b40 msgr2=0x56196efb9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:03.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff158d33640 1 --2- 172.21.15.133:0/1160930990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb7b40 0x56196efb9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff158d33640 1 -- 172.21.15.133:0/1160930990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efc1370 msgr2=0x56196efb9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff158d33640 1 --2- 172.21.15.133:0/1160930990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efc1370 0x56196efb9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff158d33640 1 -- 172.21.15.133:0/1160930990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56196efc41b0 con 0x56196efb65d0 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.213+0000 7ff159534640 1 --2- 172.21.15.133:0/1160930990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efc1370 0x56196efb9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.214+0000 7ff158d33640 1 --2- 172.21.15.133:0/1160930990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb65d0 0x56196efbe040 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff14c007590 tx=0x7ff14c013720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9084dfed4501a2f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.214+0000 7ff14a7fc640 1 -- 172.21.15.133:0/1160930990 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c014210 con 0x56196efb65d0 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.214+0000 7ff14a7fc640 1 -- 172.21.15.133:0/1160930990 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff14c099c10 con 0x56196efb65d0 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.214+0000 7ff14a7fc640 1 -- 172.21.15.133:0/1160930990 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0981e0 con 0x56196efb65d0 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.214+0000 7ff15d370740 1 -- 172.21.15.133:0/1160930990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb65d0 msgr2=0x56196efbe040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.214+0000 7ff15d370740 1 --2- 172.21.15.133:0/1160930990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb65d0 0x56196efbe040 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff14c007590 tx=0x7ff14c013720 comp rx=0 tx=0).stop 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.214+0000 7ff15d370740 1 -- 172.21.15.133:0/1160930990 shutdown_connections 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.214+0000 7ff15d370740 1 --2- 172.21.15.133:0/1160930990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efc1370 0x56196efb9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.214+0000 7ff15d370740 1 --2- 172.21.15.133:0/1160930990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb7b40 0x56196efb9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.214+0000 7ff15d370740 1 --2- 172.21.15.133:0/1160930990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb65d0 0x56196efbe040 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.214+0000 7ff15d370740 1 -- 172.21.15.133:0/1160930990 >> 172.21.15.133:0/1160930990 conn(0x56196ef7f310 msgr2=0x56196efbd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.215+0000 7ff15d370740 1 -- 172.21.15.133:0/1160930990 shutdown_connections 2024-09-11T23:49:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.215+0000 7ff15d370740 1 -- 172.21.15.133:0/1160930990 wait complete. 2024-09-11T23:49:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.215+0000 7ff15d370740 1 -- start start 2024-09-11T23:49:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.215+0000 7ff15d370740 1 Processor -- start 2024-09-11T23:49:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.215+0000 7ff15d370740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb65d0 0x56196efc9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.215+0000 7ff15d370740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efbae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.215+0000 7ff15d370740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efc1370 0x56196efbb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.215+0000 7ff15d370740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff1541b0d60 con 0x56196efb7b40 2024-09-11T23:49:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.215+0000 7ff15d370740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1541b0690 con 0x56196efc1370 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.215+0000 7ff15d370740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1541ad8f0 con 0x56196efb65d0 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.215+0000 7ff15d370740 10 client.0 _ll_register_callbacks cb 0x7ff13c031a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff153fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efbae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff153fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efbae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59608/0 (socket says 172.21.15.133:59608) 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff153fff640 1 -- 172.21.15.133:0/4128064255 learned_addr learned my addr 172.21.15.133:0/4128064255 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff158d33640 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb65d0 0x56196efc9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff159534640 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efc1370 0x56196efbb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff153fff640 1 -- 172.21.15.133:0/4128064255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb65d0 msgr2=0x56196efc9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff153fff640 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb65d0 0x56196efc9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff153fff640 1 -- 172.21.15.133:0/4128064255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efc1370 msgr2=0x56196efbb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff153fff640 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efc1370 0x56196efbb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff153fff640 1 -- 172.21.15.133:0/4128064255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56196efbbbb0 con 0x56196efb7b40 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff153fff640 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efbae40 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff15410eda0 tx=0x7ff144011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3984060eaf442ba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff133fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff133fff640 1 -- 172.21.15.133:0/4128064255 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14401e060 con 0x56196efb7b40 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff133fff640 1 -- 172.21.15.133:0/4128064255 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7ff14400fd80 con 0x56196efb7b40 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff15d370740 1 -- 172.21.15.133:0/4128064255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff14c099c10 con 0x56196efb7b40 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff133fff640 1 -- 172.21.15.133:0/4128064255 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff144023470 con 0x56196efb7b40 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff15d370740 10 client.18597 fetch_fsmap learned FSMap version 32 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.216+0000 7ff15d370740 1 -- 172.21.15.133:0/4128064255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56196f03b8e0 con 0x56196efb7b40 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.217+0000 7ff133fff640 1 -- 172.21.15.133:0/4128064255 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff14c099c10 con 0x56196efb7b40 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.217+0000 7ff133fff640 1 -- 172.21.15.133:0/4128064255 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff144018030 con 0x56196efb7b40 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.217+0000 7ff15d370740 10 client.18597 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.217+0000 7ff15d370740 10 client.18597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.217+0000 7ff15d370740 20 client.18597 populate_metadata read hostname 'smithi133' 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.217+0000 7ff15d370740 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56196f03c0c0 0x56196f03e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.217+0000 7ff15d370740 4 client.18597 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.217+0000 7ff15d370740 1 -- 172.21.15.133:0/4128064255 --> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56196ef23d70 con 0x56196f03c0c0 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.217+0000 7ff158d33640 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56196f03c0c0 0x56196f03e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.217+0000 7ff158d33640 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56196f03c0c0 0x56196f03e4e0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa66898dda0b4008 server_cookie=c7972b828cf9b871 in_seq=0 out_seq=0 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.217+0000 7ff133fff640 10 client.18597 ms_handle_connect on v2:172.21.15.178:6838/3794667907 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff133fff640 1 -- 172.21.15.133:0/4128064255 <== mds.0 v2:172.21.15.178:6838/3794667907 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56196ef23d70 con 0x56196f03c0c0 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff133fff640 10 client.18597 handle_command_reply: tid=0 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 1 client.18597 shutdown 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 1 -- 172.21.15.133:0/4128064255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 msgr2=0x56196efbae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efbae40 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff15410eda0 tx=0x7ff144011d10 comp rx=0 tx=0).stop 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 1 -- 172.21.15.133:0/4128064255 shutdown_connections 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.178:6838/3794667907,v1:172.21.15.178:6839/3794667907] conn(0x56196f03c0c0 0x56196f03e4e0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56196efc1370 0x56196efbb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56196efb7b40 0x56196efbae40 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 1 --2- 172.21.15.133:0/4128064255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56196efb65d0 0x56196efc9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 1 -- 172.21.15.133:0/4128064255 >> 172.21.15.133:0/4128064255 conn(0x56196ef7f310 msgr2=0x56196efc00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff133fff640 0 client.18597 ms_handle_reset on v2:172.21.15.178:6838/3794667907 2024-09-11T23:49:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 1 -- 172.21.15.133:0/4128064255 shutdown_connections 2024-09-11T23:49:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 1 -- 172.21.15.133:0/4128064255 wait complete. 2024-09-11T23:49:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.218+0000 7ff15d370740 20 client.18597 trim_cache size 0 max 16384 2024-09-11T23:49:03.229 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 -- 172.21.15.133:0/1283124445 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff1400a2cf0 msgr2=0x7ff1400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff1400a2cf0 0x7ff1400a51b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff13c04b000 tx=0x7ff13c049000 comp rx=0 tx=0).stop 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 -- 172.21.15.133:0/1283124445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1541009d0 msgr2=0x7ff15419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1541009d0 0x7ff15419a650 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c920 tx=0x7ff14c00cde0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 -- 172.21.15.133:0/1283124445 shutdown_connections 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7ff1400a2cf0 0x7ff1400a51b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15410e4f0 0x7ff15419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff154101380 0x7ff15419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 --2- 172.21.15.133:0/1283124445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1541009d0 0x7ff15419a650 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 -- 172.21.15.133:0/1283124445 >> 172.21.15.133:0/1283124445 conn(0x7ff1540fc7d0 msgr2=0x7ff1540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 -- 172.21.15.133:0/1283124445 shutdown_connections 2024-09-11T23:49:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:03.219+0000 7ff15af97640 1 -- 172.21.15.133:0/1283124445 wait complete. 2024-09-11T23:49:03.938 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-11T23:49:03.939 INFO:tasks.fwd_scrub:done joining 2024-09-11T23:49:03.939 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-11T23:49:03.950 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T23:49:04.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.056+0000 7f82ae69b640 1 -- 172.21.15.133:0/4041387176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a8101aa0 msgr2=0x7f82a8101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.056+0000 7f82ae69b640 1 --2- 172.21.15.133:0/4041387176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a8101aa0 0x7f82a8101f00 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f829c009930 tx=0x7f829c02a420 comp rx=0 tx=0).stop 2024-09-11T23:49:04.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.056+0000 7f82ae69b640 1 -- 172.21.15.133:0/4041387176 shutdown_connections 2024-09-11T23:49:04.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.056+0000 7f82ae69b640 1 --2- 172.21.15.133:0/4041387176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82a8102440 0x7f82a810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.056+0000 7f82ae69b640 1 --2- 172.21.15.133:0/4041387176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a8101aa0 0x7f82a8101f00 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.056+0000 7f82ae69b640 1 --2- 172.21.15.133:0/4041387176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82a8107ab0 0x7f82a8107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.056+0000 7f82ae69b640 1 -- 172.21.15.133:0/4041387176 >> 172.21.15.133:0/4041387176 conn(0x7f82a80fd7c0 msgr2=0x7f82a80ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:04.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.056+0000 7f82ae69b640 1 -- 172.21.15.133:0/4041387176 shutdown_connections 2024-09-11T23:49:04.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.056+0000 7f82ae69b640 1 -- 172.21.15.133:0/4041387176 wait complete. 2024-09-11T23:49:04.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.057+0000 7f82ae69b640 1 Processor -- start 2024-09-11T23:49:04.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.057+0000 7f82ae69b640 1 -- start start 2024-09-11T23:49:04.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.057+0000 7f82ae69b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82a8101aa0 0x7f82a8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.057+0000 7f82ae69b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82a8102440 0x7f82a8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.057+0000 7f82ae69b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a8107ab0 0x7f82a819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.057+0000 7f82ae69b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82a8111ef0 con 0x7f82a8107ab0 2024-09-11T23:49:04.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.057+0000 7f82ae69b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82a8111d70 con 0x7f82a8102440 2024-09-11T23:49:04.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.057+0000 7f82ae69b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82a8112070 con 0x7f82a8101aa0 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.057+0000 7f82acc38640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a8107ab0 0x7f82a819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82acc38640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a8107ab0 0x7f82a819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59630/0 (socket says 172.21.15.133:59630) 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82acc38640 1 -- 172.21.15.133:0/4181404500 learned_addr learned my addr 172.21.15.133:0/4181404500 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82a7fff640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82a8101aa0 0x7f82a8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82a77fe640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82a8102440 0x7f82a8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82acc38640 1 -- 172.21.15.133:0/4181404500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82a8101aa0 msgr2=0x7f82a8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82acc38640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82a8101aa0 0x7f82a8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82acc38640 1 -- 172.21.15.133:0/4181404500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82a8102440 msgr2=0x7f82a8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82acc38640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82a8102440 0x7f82a8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82acc38640 1 -- 172.21.15.133:0/4181404500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a819d3f0 con 0x7f82a8107ab0 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82a7fff640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82a8101aa0 0x7f82a8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82a77fe640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82a8102440 0x7f82a8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:49:04.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82acc38640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a8107ab0 0x7f82a819ccc0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f82a000eaa0 tx=0x7f82a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:04.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82a57fa640 1 -- 172.21.15.133:0/4181404500 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a000cdb0 con 0x7f82a8107ab0 2024-09-11T23:49:04.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82ae69b640 1 -- 172.21.15.133:0/4181404500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a819d6e0 con 0x7f82a8107ab0 2024-09-11T23:49:04.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82a57fa640 1 -- 172.21.15.133:0/4181404500 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a0005020 con 0x7f82a8107ab0 2024-09-11T23:49:04.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82a57fa640 1 -- 172.21.15.133:0/4181404500 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a00116b0 con 0x7f82a8107ab0 2024-09-11T23:49:04.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.058+0000 7f82ae69b640 1 -- 172.21.15.133:0/4181404500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a81a4f70 con 0x7f82a8107ab0 2024-09-11T23:49:04.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.059+0000 7f82ae69b640 1 -- 172.21.15.133:0/4181404500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a8101f00 con 0x7f82a8107ab0 2024-09-11T23:49:04.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.060+0000 7f82a57fa640 1 -- 172.21.15.133:0/4181404500 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f82a0005560 con 0x7f82a8107ab0 2024-09-11T23:49:04.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.061+0000 7f82a57fa640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82900a2d00 0x7f82900a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.061+0000 7f82a7fff640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82900a2d00 0x7f82900a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.061+0000 7f82a57fa640 1 -- 172.21.15.133:0/4181404500 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f82a00cf710 con 0x7f82a8107ab0 2024-09-11T23:49:04.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.061+0000 7f82a7fff640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82900a2d00 0x7f82900a51c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8298005d90 tx=0x7f8298005d20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:04.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.065+0000 7f82a57fa640 1 -- 172.21.15.133:0/4181404500 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82a0016030 con 0x7f82a8107ab0 2024-09-11T23:49:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.233+0000 7f82ae69b640 1 -- 172.21.15.133:0/4181404500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82a8109a80 con 0x7f82a8107ab0 2024-09-11T23:49:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.234+0000 7f82a57fa640 1 -- 172.21.15.133:0/4181404500 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23150 (secure 0 0 0) 0x7f82a0086020 con 0x7f82a8107ab0 2024-09-11T23:49:04.234 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:49:04.234 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":96,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:48:13.969704+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/3324898762":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3596945316":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6801/3324898762":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/1658494809":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3947064111":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/802317131":"2024-09-12T23:36:47.051817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:49:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.236+0000 7f82ae69b640 1 -- 172.21.15.133:0/4181404500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82900a2d00 msgr2=0x7f82900a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.236+0000 7f82ae69b640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82900a2d00 0x7f82900a51c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8298005d90 tx=0x7f8298005d20 comp rx=0 tx=0).stop 2024-09-11T23:49:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.236+0000 7f82ae69b640 1 -- 172.21.15.133:0/4181404500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a8107ab0 msgr2=0x7f82a819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.236+0000 7f82ae69b640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a8107ab0 0x7f82a819ccc0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f82a000eaa0 tx=0x7f82a000ef60 comp rx=0 tx=0).stop 2024-09-11T23:49:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.236+0000 7f82ae69b640 1 -- 172.21.15.133:0/4181404500 shutdown_connections 2024-09-11T23:49:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.236+0000 7f82ae69b640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f82900a2d00 0x7f82900a51c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.236+0000 7f82ae69b640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a8107ab0 0x7f82a819ccc0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.236+0000 7f82ae69b640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82a8102440 0x7f82a8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.236+0000 7f82ae69b640 1 --2- 172.21.15.133:0/4181404500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82a8101aa0 0x7f82a8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.236+0000 7f82ae69b640 1 -- 172.21.15.133:0/4181404500 >> 172.21.15.133:0/4181404500 conn(0x7f82a80fd7c0 msgr2=0x7f82a80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.237+0000 7f82ae69b640 1 -- 172.21.15.133:0/4181404500 shutdown_connections 2024-09-11T23:49:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.237+0000 7f82ae69b640 1 -- 172.21.15.133:0/4181404500 wait complete. 2024-09-11T23:49:04.248 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:30:22.061012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T23:34:49.162653+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T23:34:51.723611+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:49:04.249 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T23:49:04.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.353+0000 7f7b60c27640 1 -- 172.21.15.133:0/2463753186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b5c0fffd0 msgr2=0x7f7b5c100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.353+0000 7f7b60c27640 1 --2- 172.21.15.133:0/2463753186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b5c0fffd0 0x7f7b5c100430 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009900 tx=0x7f7b5002a410 comp rx=0 tx=0).stop 2024-09-11T23:49:04.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.353+0000 7f7b60c27640 1 -- 172.21.15.133:0/2463753186 shutdown_connections 2024-09-11T23:49:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.353+0000 7f7b60c27640 1 --2- 172.21.15.133:0/2463753186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5c100970 0x7f7b5c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.353+0000 7f7b60c27640 1 --2- 172.21.15.133:0/2463753186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b5c0fffd0 0x7f7b5c100430 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.353+0000 7f7b60c27640 1 --2- 172.21.15.133:0/2463753186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c106ab0 0x7f7b5c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.353+0000 7f7b60c27640 1 -- 172.21.15.133:0/2463753186 >> 172.21.15.133:0/2463753186 conn(0x7f7b5c077170 msgr2=0x7f7b5c0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.353+0000 7f7b60c27640 1 -- 172.21.15.133:0/2463753186 shutdown_connections 2024-09-11T23:49:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.353+0000 7f7b60c27640 1 -- 172.21.15.133:0/2463753186 wait complete. 2024-09-11T23:49:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.354+0000 7f7b60c27640 1 Processor -- start 2024-09-11T23:49:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.354+0000 7f7b60c27640 1 -- start start 2024-09-11T23:49:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.354+0000 7f7b60c27640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c0fffd0 0x7f7b5c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.354+0000 7f7b60c27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5c100970 0x7f7b5c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.354+0000 7f7b60c27640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b5c106ab0 0x7f7b5c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.354+0000 7f7b60c27640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c111f80 con 0x7f7b5c106ab0 2024-09-11T23:49:04.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.354+0000 7f7b60c27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c111e00 con 0x7f7b5c100970 2024-09-11T23:49:04.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.354+0000 7f7b60c27640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c112100 con 0x7f7b5c0fffd0 2024-09-11T23:49:04.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.354+0000 7f7b5ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b5c106ab0 0x7f7b5c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b5ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b5c106ab0 0x7f7b5c19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59652/0 (socket says 172.21.15.133:59652) 2024-09-11T23:49:04.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b5ad9d640 1 -- 172.21.15.133:0/3542819010 learned_addr learned my addr 172.21.15.133:0/3542819010 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:04.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b59d9b640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5c100970 0x7f7b5c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b5a59c640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c0fffd0 0x7f7b5c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b5ad9d640 1 -- 172.21.15.133:0/3542819010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c0fffd0 msgr2=0x7f7b5c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b5ad9d640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c0fffd0 0x7f7b5c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b5ad9d640 1 -- 172.21.15.133:0/3542819010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5c100970 msgr2=0x7f7b5c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b5ad9d640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5c100970 0x7f7b5c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b5ad9d640 1 -- 172.21.15.133:0/3542819010 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c19d480 con 0x7f7b5c106ab0 2024-09-11T23:49:04.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b5ad9d640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b5c106ab0 0x7f7b5c19cd50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400eb10 tx=0x7f7b5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:04.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b4b7fe640 1 -- 172.21.15.133:0/3542819010 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54004b20 con 0x7f7b5c106ab0 2024-09-11T23:49:04.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b60c27640 1 -- 172.21.15.133:0/3542819010 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5c19d770 con 0x7f7b5c106ab0 2024-09-11T23:49:04.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b4b7fe640 1 -- 172.21.15.133:0/3542819010 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b54004cc0 con 0x7f7b5c106ab0 2024-09-11T23:49:04.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b4b7fe640 1 -- 172.21.15.133:0/3542819010 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54010760 con 0x7f7b5c106ab0 2024-09-11T23:49:04.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.355+0000 7f7b60c27640 1 -- 172.21.15.133:0/3542819010 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b5c1a5000 con 0x7f7b5c106ab0 2024-09-11T23:49:04.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.356+0000 7f7b60c27640 1 -- 172.21.15.133:0/3542819010 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b5c100430 con 0x7f7b5c106ab0 2024-09-11T23:49:04.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.357+0000 7f7b4b7fe640 1 -- 172.21.15.133:0/3542819010 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7b54010900 con 0x7f7b5c106ab0 2024-09-11T23:49:04.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.358+0000 7f7b4b7fe640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b440a2d00 0x7f7b440a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.358+0000 7f7b5a59c640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b440a2d00 0x7f7b440a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.358+0000 7f7b4b7fe640 1 -- 172.21.15.133:0/3542819010 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f7b540cfc40 con 0x7f7b5c106ab0 2024-09-11T23:49:04.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.358+0000 7f7b5a59c640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b440a2d00 0x7f7b440a51c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009b60 tx=0x7f7b4c005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:04.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.362+0000 7f7b4b7fe640 1 -- 172.21.15.133:0/3542819010 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b540541b0 con 0x7f7b5c106ab0 2024-09-11T23:49:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.526+0000 7f7b60c27640 1 -- 172.21.15.133:0/3542819010 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7b5c109fb0 con 0x7f7b5c106ab0 2024-09-11T23:49:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.527+0000 7f7b4b7fe640 1 -- 172.21.15.133:0/3542819010 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f7b5409c8a0 con 0x7f7b5c106ab0 2024-09-11T23:49:04.526 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-09-11T23:49:04.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 -- 172.21.15.133:0/3542819010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b440a2d00 msgr2=0x7f7b440a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b440a2d00 0x7f7b440a51c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009b60 tx=0x7f7b4c005f90 comp rx=0 tx=0).stop 2024-09-11T23:49:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 -- 172.21.15.133:0/3542819010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b5c106ab0 msgr2=0x7f7b5c19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b5c106ab0 0x7f7b5c19cd50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400eb10 tx=0x7f7b5400c2d0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 -- 172.21.15.133:0/3542819010 shutdown_connections 2024-09-11T23:49:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f7b440a2d00 0x7f7b440a51c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b5c106ab0 0x7f7b5c19cd50 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5c100970 0x7f7b5c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 --2- 172.21.15.133:0/3542819010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c0fffd0 0x7f7b5c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 -- 172.21.15.133:0/3542819010 >> 172.21.15.133:0/3542819010 conn(0x7f7b5c077170 msgr2=0x7f7b5c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 -- 172.21.15.133:0/3542819010 shutdown_connections 2024-09-11T23:49:04.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.529+0000 7f7b60c27640 1 -- 172.21.15.133:0/3542819010 wait complete. 2024-09-11T23:49:04.539 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T23:49:04.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.643+0000 7f7711f76640 1 -- 172.21.15.133:0/2764369651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770c102cc0 msgr2=0x7f770c1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.643+0000 7f7711f76640 1 --2- 172.21.15.133:0/2764369651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770c102cc0 0x7f770c1030a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009900 tx=0x7f76fc02a480 comp rx=0 tx=0).stop 2024-09-11T23:49:04.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.643+0000 7f7711f76640 1 -- 172.21.15.133:0/2764369651 shutdown_connections 2024-09-11T23:49:04.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.643+0000 7f7711f76640 1 --2- 172.21.15.133:0/2764369651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f770c10c090 0x7f770c10e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.643+0000 7f7711f76640 1 --2- 172.21.15.133:0/2764369651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c1035e0 0x7f770c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.643+0000 7f7711f76640 1 --2- 172.21.15.133:0/2764369651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770c102cc0 0x7f770c1030a0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.643+0000 7f7711f76640 1 -- 172.21.15.133:0/2764369651 >> 172.21.15.133:0/2764369651 conn(0x7f770c0fc6b0 msgr2=0x7f770c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.643+0000 7f7711f76640 1 -- 172.21.15.133:0/2764369651 shutdown_connections 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.643+0000 7f7711f76640 1 -- 172.21.15.133:0/2764369651 wait complete. 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f7711f76640 1 Processor -- start 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f7711f76640 1 -- start start 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f7711f76640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770c102cc0 0x7f770c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f7711f76640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c1035e0 0x7f770c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f7711f76640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f770c10c090 0x7f770c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f7711f76640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f770c111e90 con 0x7f770c102cc0 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f7711f76640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f770c111d10 con 0x7f770c1035e0 2024-09-11T23:49:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f7711f76640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f770c112010 con 0x7f770c10c090 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f770b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770c102cc0 0x7f770c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f770b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770c102cc0 0x7f770c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59670/0 (socket says 172.21.15.133:59670) 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f770b7fe640 1 -- 172.21.15.133:0/1595070731 learned_addr learned my addr 172.21.15.133:0/1595070731 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.644+0000 7f770bfff640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f770c10c090 0x7f770c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f770affd640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c1035e0 0x7f770c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f770b7fe640 1 -- 172.21.15.133:0/1595070731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f770c10c090 msgr2=0x7f770c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f770b7fe640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f770c10c090 0x7f770c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f770b7fe640 1 -- 172.21.15.133:0/1595070731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c1035e0 msgr2=0x7f770c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f770b7fe640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c1035e0 0x7f770c198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f770b7fe640 1 -- 172.21.15.133:0/1595070731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c19d330 con 0x7f770c102cc0 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f770affd640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c1035e0 0x7f770c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f770b7fe640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770c102cc0 0x7f770c198340 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f76fc006010 tx=0x7f76fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f7708ff9640 1 -- 172.21.15.133:0/1595070731 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc03f030 con 0x7f770c102cc0 2024-09-11T23:49:04.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f7708ff9640 1 -- 172.21.15.133:0/1595070731 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76fc003ce0 con 0x7f770c102cc0 2024-09-11T23:49:04.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f7708ff9640 1 -- 172.21.15.133:0/1595070731 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc035070 con 0x7f770c102cc0 2024-09-11T23:49:04.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f7711f76640 1 -- 172.21.15.133:0/1595070731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770c19d5c0 con 0x7f770c102cc0 2024-09-11T23:49:04.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.645+0000 7f7711f76640 1 -- 172.21.15.133:0/1595070731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770c1a4eb0 con 0x7f770c102cc0 2024-09-11T23:49:04.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.646+0000 7f7711f76640 1 -- 172.21.15.133:0/1595070731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770c1030a0 con 0x7f770c102cc0 2024-09-11T23:49:04.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.647+0000 7f7708ff9640 1 -- 172.21.15.133:0/1595070731 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f76fc042030 con 0x7f770c102cc0 2024-09-11T23:49:04.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.648+0000 7f7708ff9640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76f40a2d00 0x7f76f40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.648+0000 7f770affd640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76f40a2d00 0x7f76f40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.648+0000 7f7708ff9640 1 -- 172.21.15.133:0/1595070731 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f76fc0ef9a0 con 0x7f770c102cc0 2024-09-11T23:49:04.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.648+0000 7f770affd640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76f40a2d00 0x7f76f40a51c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f770c0fec40 tx=0x7f770000a5c0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.652+0000 7f7708ff9640 1 -- 172.21.15.133:0/1595070731 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76fc0bc6c0 con 0x7f770c102cc0 2024-09-11T23:49:04.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.820+0000 7f7711f76640 1 -- 172.21.15.133:0/1595070731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f770c107140 con 0x7f770c102cc0 2024-09-11T23:49:04.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.821+0000 7f7708ff9640 1 -- 172.21.15.133:0/1595070731 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f76fc0bbfc0 con 0x7f770c102cc0 2024-09-11T23:49:04.821 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 -- 172.21.15.133:0/1595070731 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76f40a2d00 msgr2=0x7f76f40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76f40a2d00 0x7f76f40a51c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f770c0fec40 tx=0x7f770000a5c0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 -- 172.21.15.133:0/1595070731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770c102cc0 msgr2=0x7f770c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770c102cc0 0x7f770c198340 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f76fc006010 tx=0x7f76fc0042f0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 -- 172.21.15.133:0/1595070731 shutdown_connections 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f76f40a2d00 0x7f76f40a51c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f770c10c090 0x7f770c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f770c1035e0 0x7f770c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 --2- 172.21.15.133:0/1595070731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770c102cc0 0x7f770c198340 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 -- 172.21.15.133:0/1595070731 >> 172.21.15.133:0/1595070731 conn(0x7f770c0fc6b0 msgr2=0x7f770c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 -- 172.21.15.133:0/1595070731 shutdown_connections 2024-09-11T23:49:04.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.823+0000 7f7711f76640 1 -- 172.21.15.133:0/1595070731 wait complete. 2024-09-11T23:49:04.832 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:04 smithi133 ceph-mon[31430]: pgmap v385: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-11T23:49:04.832 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:04 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/4181404500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:49:04.832 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:04 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3542819010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:49:04.833 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T23:49:04.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.937+0000 7f1ec7265640 1 -- 172.21.15.133:0/976663268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ec0101350 msgr2=0x7f1ec010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.937+0000 7f1ec7265640 1 --2- 172.21.15.133:0/976663268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ec0101350 0x7f1ec010d880 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009cd0 tx=0x7f1eb802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 -- 172.21.15.133:0/976663268 shutdown_connections 2024-09-11T23:49:04.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 --2- 172.21.15.133:0/976663268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ec0101350 0x7f1ec010d880 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 --2- 172.21.15.133:0/976663268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec01009b0 0x7f1ec0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 --2- 172.21.15.133:0/976663268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec01069c0 0x7f1ec0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 -- 172.21.15.133:0/976663268 >> 172.21.15.133:0/976663268 conn(0x7f1ec00fc690 msgr2=0x7f1ec00feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:04.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 -- 172.21.15.133:0/976663268 shutdown_connections 2024-09-11T23:49:04.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 -- 172.21.15.133:0/976663268 wait complete. 2024-09-11T23:49:04.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 Processor -- start 2024-09-11T23:49:04.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 -- start start 2024-09-11T23:49:04.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec01009b0 0x7f1ec0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec0101350 0x7f1ec0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ec01069c0 0x7f1ec019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0110e00 con 0x7f1ec01069c0 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0110c80 con 0x7f1ec01009b0 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.938+0000 7f1ec7265640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0110f80 con 0x7f1ec0101350 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec5802640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ec01069c0 0x7f1ec019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec5802640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ec01069c0 0x7f1ec019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59686/0 (socket says 172.21.15.133:59686) 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec5802640 1 -- 172.21.15.133:0/3730481607 learned_addr learned my addr 172.21.15.133:0/3730481607 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec4800640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec0101350 0x7f1ec0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec5001640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec01009b0 0x7f1ec0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec5802640 1 -- 172.21.15.133:0/3730481607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec0101350 msgr2=0x7f1ec0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec5802640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec0101350 0x7f1ec0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec5802640 1 -- 172.21.15.133:0/3730481607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec01009b0 msgr2=0x7f1ec0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec5802640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec01009b0 0x7f1ec0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:04.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec5802640 1 -- 172.21.15.133:0/3730481607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec019d3f0 con 0x7f1ec01069c0 2024-09-11T23:49:04.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec4800640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec0101350 0x7f1ec0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:49:04.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec5802640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ec01069c0 0x7f1ec019ccc0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8005cc0 tx=0x7f1eb8032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:04.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1eb27fc640 1 -- 172.21.15.133:0/3730481607 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8004040 con 0x7f1ec01069c0 2024-09-11T23:49:04.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1eb27fc640 1 -- 172.21.15.133:0/3730481607 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb80046c0 con 0x7f1ec01069c0 2024-09-11T23:49:04.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1eb27fc640 1 -- 172.21.15.133:0/3730481607 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8030e00 con 0x7f1ec01069c0 2024-09-11T23:49:04.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec7265640 1 -- 172.21.15.133:0/3730481607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec019d680 con 0x7f1ec01069c0 2024-09-11T23:49:04.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.939+0000 7f1ec7265640 1 -- 172.21.15.133:0/3730481607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec01a4f70 con 0x7f1ec01069c0 2024-09-11T23:49:04.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.940+0000 7f1ec7265640 1 -- 172.21.15.133:0/3730481607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec0100e10 con 0x7f1ec01069c0 2024-09-11T23:49:04.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.941+0000 7f1eb27fc640 1 -- 172.21.15.133:0/3730481607 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1eb80305a0 con 0x7f1ec01069c0 2024-09-11T23:49:04.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.942+0000 7f1eb27fc640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1eac0a2d80 0x7f1eac0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:04.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.942+0000 7f1ec5001640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1eac0a2d80 0x7f1eac0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:04.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.942+0000 7f1eb27fc640 1 -- 172.21.15.133:0/3730481607 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7f1eb802e030 con 0x7f1ec01069c0 2024-09-11T23:49:04.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.942+0000 7f1ec5001640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1eac0a2d80 0x7f1eac0a5240 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009a20 tx=0x7f1ebc005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:04.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:04.945+0000 7f1eb27fc640 1 -- 172.21.15.133:0/3730481607 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1eb804b060 con 0x7f1ec01069c0 2024-09-11T23:49:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:04 smithi178 ceph-mon[35882]: pgmap v385: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-11T23:49:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:04 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/4181404500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:49:05.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:04 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3542819010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:49:05.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.114+0000 7f1ec7265640 1 -- 172.21.15.133:0/3730481607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1ec01099a0 con 0x7f1ec01069c0 2024-09-11T23:49:05.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.114+0000 7f1eb27fc640 1 -- 172.21.15.133:0/3730481607 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f1eb80bd980 con 0x7f1ec01069c0 2024-09-11T23:49:05.114 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-09-11T23:49:05.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 -- 172.21.15.133:0/3730481607 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1eac0a2d80 msgr2=0x7f1eac0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:05.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1eac0a2d80 0x7f1eac0a5240 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009a20 tx=0x7f1ebc005f90 comp rx=0 tx=0).stop 2024-09-11T23:49:05.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 -- 172.21.15.133:0/3730481607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ec01069c0 msgr2=0x7f1ec019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:05.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ec01069c0 0x7f1ec019ccc0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8005cc0 tx=0x7f1eb8032f40 comp rx=0 tx=0).stop 2024-09-11T23:49:05.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 -- 172.21.15.133:0/3730481607 shutdown_connections 2024-09-11T23:49:05.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f1eac0a2d80 0x7f1eac0a5240 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:05.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ec01069c0 0x7f1ec019ccc0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:05.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ec0101350 0x7f1ec0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:05.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 --2- 172.21.15.133:0/3730481607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec01009b0 0x7f1ec0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:05.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 -- 172.21.15.133:0/3730481607 >> 172.21.15.133:0/3730481607 conn(0x7f1ec00fc690 msgr2=0x7f1ec006a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:05.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 -- 172.21.15.133:0/3730481607 shutdown_connections 2024-09-11T23:49:05.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.116+0000 7f1ec7265640 1 -- 172.21.15.133:0/3730481607 wait complete. 2024-09-11T23:49:05.126 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-11T23:49:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:04 smithi138 ceph-mon[35142]: pgmap v385: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-11T23:49:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:04 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/4181404500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:49:05.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:04 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3542819010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:49:05.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.228+0000 7fc3096a5640 1 -- 172.21.15.133:0/165670393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc304102450 msgr2=0x7fc30410e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:05.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.228+0000 7fc3096a5640 1 --2- 172.21.15.133:0/165670393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc304102450 0x7fc30410e980 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009cd0 tx=0x7fc2fc02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:49:05.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.228+0000 7fc3096a5640 1 -- 172.21.15.133:0/165670393 shutdown_connections 2024-09-11T23:49:05.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.228+0000 7fc3096a5640 1 --2- 172.21.15.133:0/165670393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc304102450 0x7fc30410e980 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:05.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.228+0000 7fc3096a5640 1 --2- 172.21.15.133:0/165670393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304101ab0 0x7fc304101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:05.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.228+0000 7fc3096a5640 1 --2- 172.21.15.133:0/165670393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc304107ac0 0x7fc304107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.228+0000 7fc3096a5640 1 -- 172.21.15.133:0/165670393 >> 172.21.15.133:0/165670393 conn(0x7fc3040fd7d0 msgr2=0x7fc3040ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.228+0000 7fc3096a5640 1 -- 172.21.15.133:0/165670393 shutdown_connections 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.228+0000 7fc3096a5640 1 -- 172.21.15.133:0/165670393 wait complete. 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc3096a5640 1 Processor -- start 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc3096a5640 1 -- start start 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc3096a5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc304101ab0 0x7fc304198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc3096a5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc304102450 0x7fc304198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc3096a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304107ac0 0x7fc30419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc3096a5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc304111f00 con 0x7fc304101ab0 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc3096a5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc304111d80 con 0x7fc304102450 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc3096a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc304112080 con 0x7fc304107ac0 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc304101ab0 0x7fc304198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc304101ab0 0x7fc304198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59702/0 (socket says 172.21.15.133:59702) 2024-09-11T23:49:05.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc302ffd640 1 -- 172.21.15.133:0/4086481353 learned_addr learned my addr 172.21.15.133:0/4086481353 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.229+0000 7fc3027fc640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc304102450 0x7fc304198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc302ffd640 1 -- 172.21.15.133:0/4086481353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304107ac0 msgr2=0x7fc30419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc3037fe640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304107ac0 0x7fc30419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc302ffd640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304107ac0 0x7fc30419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc302ffd640 1 -- 172.21.15.133:0/4086481353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc304102450 msgr2=0x7fc304198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc302ffd640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc304102450 0x7fc304198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc302ffd640 1 -- 172.21.15.133:0/4086481353 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30419d310 con 0x7fc304101ab0 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc302ffd640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc304101ab0 0x7fc304198320 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00e9f0 tx=0x7fc2ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc2fbfff640 1 -- 172.21.15.133:0/4086481353 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00cd80 con 0x7fc304101ab0 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc2fbfff640 1 -- 172.21.15.133:0/4086481353 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ec010910 con 0x7fc304101ab0 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc3096a5640 1 -- 172.21.15.133:0/4086481353 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30419d600 con 0x7fc304101ab0 2024-09-11T23:49:05.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc2fbfff640 1 -- 172.21.15.133:0/4086481353 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec018990 con 0x7fc304101ab0 2024-09-11T23:49:05.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.230+0000 7fc3096a5640 1 -- 172.21.15.133:0/4086481353 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3041a4e90 con 0x7fc304101ab0 2024-09-11T23:49:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.231+0000 7fc3096a5640 1 -- 172.21.15.133:0/4086481353 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc304101f10 con 0x7fc304101ab0 2024-09-11T23:49:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.232+0000 7fc2fbfff640 1 -- 172.21.15.133:0/4086481353 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc2ec010ac0 con 0x7fc304101ab0 2024-09-11T23:49:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.233+0000 7fc2fbfff640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc2f00a2d80 0x7fc2f00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.233+0000 7fc3027fc640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc2f00a2d80 0x7fc2f00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.233+0000 7fc2fbfff640 1 -- 172.21.15.133:0/4086481353 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8455+0+0 (secure 0 0 0) 0x7fc2ec0cf740 con 0x7fc304101ab0 2024-09-11T23:49:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.233+0000 7fc3027fc640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc2f00a2d80 0x7fc2f00a5240 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005e10 tx=0x7fc2f4005da0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:05.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.237+0000 7fc2fbfff640 1 -- 172.21.15.133:0/4086481353 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2ec09c3a0 con 0x7fc304101ab0 2024-09-11T23:49:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.448+0000 7fc3096a5640 1 -- 172.21.15.133:0/4086481353 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fc3041a5170 con 0x7fc304101ab0 2024-09-11T23:49:05.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.455+0000 7fc2fbfff640 1 -- 172.21.15.133:0/4086481353 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740913 (secure 0 0 0) 0x7fc2ec09bca0 con 0x7fc304101ab0 2024-09-11T23:49:05.465 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:49:05.465 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:49:05.465 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 25, 2024-09-11T23:49:05.465 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: "active_gid": 15255, 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: "active_name": "x", 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: "active_addrs": { 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:6800", 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2757204896 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:6801", 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2757204896 2024-09-11T23:49:05.466 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: "active_addr": "172.21.15.133:6801/2757204896", 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: "active_change": "2024-09-11T23:36:47.052072+0000", 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: "active_mgr_features": 4540701547738038271, 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: "available": true, 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: "standbys": [ 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: "gid": 24797, 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "y", 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr_features": 4540701547738038271, 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.467 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.468 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.469 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.470 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.471 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.472 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.473 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.474 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.475 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.476 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.477 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.478 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.479 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2024-09-11T23:49:05.480 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2024-09-11T23:49:05.481 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.482 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.483 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.484 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.485 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2024-09-11T23:49:05.486 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.487 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2024-09-11T23:49:05.488 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.489 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.490 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.491 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.492 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.493 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.495 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2024-09-11T23:49:05.496 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-09-11T23:49:05.497 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "read", 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap", 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap-read" 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2024-09-11T23:49:05.498 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:49:05.499 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:49:05.500 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:49:05.501 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.502 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.503 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2024-09-11T23:49:05.504 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2024-09-11T23:49:05.505 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.506 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.507 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.508 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm_log_destination": { 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm_log_destination", 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.509 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: "file", 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: "file,syslog", 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: "syslog" 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.510 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.511 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2024-09-11T23:49:05.512 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2024-09-11T23:49:05.513 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2024-09-11T23:49:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.515 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.516 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.517 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:49:05.518 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:49:05.519 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2024-09-11T23:49:05.520 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2024-09-11T23:49:05.521 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.523 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_nvmeof": { 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_nvmeof", 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.524 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Nvme-of container image", 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.525 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.526 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_samba": { 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_samba", 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.527 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.528 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.529 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.530 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.531 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.532 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.533 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.534 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2024-09-11T23:49:05.535 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "grafana_dashboards_path": { 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.536 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2024-09-11T23:49:05.537 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.538 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.538 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.538 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:05.538 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.538 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.538 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "hw_monitoring": { 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hw_monitoring", 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: "inventory_list_all": { 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "inventory_list_all", 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.540 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.541 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.542 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.543 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.544 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.545 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.546 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.547 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.548 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.550 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.551 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.552 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "oob_default_addr": { 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "oob_default_addr", 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "169.254.1.1", 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:49:05.554 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.555 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.556 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.556 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.556 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:49:05.556 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.556 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.556 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.556 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.556 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2024-09-11T23:49:05.556 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2024-09-11T23:49:05.556 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.556 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.557 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.558 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.559 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2024-09-11T23:49:05.560 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2024-09-11T23:49:05.561 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2024-09-11T23:49:05.562 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.563 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.564 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.565 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.566 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2024-09-11T23:49:05.567 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.568 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.569 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.570 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.573 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.574 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.575 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.576 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.577 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.578 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.580 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.581 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.584 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.587 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.588 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.589 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2024-09-11T23:49:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:49:05.591 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.592 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.593 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.594 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.595 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.596 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.597 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.598 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.599 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.600 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.601 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.602 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:49:05.603 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.604 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.605 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.606 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.608 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:49:05.611 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:49:05.611 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.611 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.611 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.611 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.611 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.611 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.611 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.611 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.611 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.611 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.612 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.613 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.614 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.615 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.616 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.617 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.618 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.619 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.620 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2024-09-11T23:49:05.621 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.622 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.623 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.624 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.625 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.626 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.627 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.628 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.629 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.630 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.631 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.632 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.633 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2024-09-11T23:49:05.634 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.635 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.637 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2024-09-11T23:49:05.638 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.638 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.638 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.638 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2024-09-11T23:49:05.638 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2024-09-11T23:49:05.638 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.638 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.639 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.640 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.641 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.643 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.644 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.645 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.645 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.645 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.645 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2024-09-11T23:49:05.645 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2024-09-11T23:49:05.645 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.645 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.645 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.645 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2024-09-11T23:49:05.648 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.648 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2024-09-11T23:49:05.649 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2024-09-11T23:49:05.650 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-09-11T23:49:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.652 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.653 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.654 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.655 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.656 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.657 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.658 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.659 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.660 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.661 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.662 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.663 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.664 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2024-09-11T23:49:05.665 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2024-09-11T23:49:05.666 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.667 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.668 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.669 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.670 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.672 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2024-09-11T23:49:05.673 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2024-09-11T23:49:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-09-11T23:49:05.675 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2024-09-11T23:49:05.676 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2024-09-11T23:49:05.677 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.678 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.679 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.680 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.681 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.683 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.684 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.685 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.686 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.687 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.688 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.689 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.691 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:05.692 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.693 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.694 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.695 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.696 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.697 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.698 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.699 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.701 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2024-09-11T23:49:05.704 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2024-09-11T23:49:05.705 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2024-09-11T23:49:05.706 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.708 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.709 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.710 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.711 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.712 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.713 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.714 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.715 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.716 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.717 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.718 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.719 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2024-09-11T23:49:05.720 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.721 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.722 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.723 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.724 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.725 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.726 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.727 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.728 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.729 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.730 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.731 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.733 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.734 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.735 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.736 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.737 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.738 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.738 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.738 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.738 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.738 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.738 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.738 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.738 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.738 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.738 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.738 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.739 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.739 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.739 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.739 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.739 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.739 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.739 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.739 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.740 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.741 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.742 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.743 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.744 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.745 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.746 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.747 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.748 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.749 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:05.750 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2024-09-11T23:49:05.751 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.752 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.753 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.754 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.755 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.756 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.757 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.758 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.759 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.760 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.761 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.762 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.763 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.764 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.765 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.767 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.768 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.769 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.770 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.771 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.772 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.773 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.774 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.775 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.776 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.777 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.778 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.779 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.780 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.781 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.782 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.783 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2024-09-11T23:49:05.784 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.785 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.786 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.787 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.788 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.788 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.788 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.788 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.788 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.788 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.788 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.788 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.788 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.789 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.790 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.791 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.792 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.793 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.794 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.795 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.796 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.797 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.798 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.799 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.801 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.802 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.803 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.804 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.805 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.806 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.807 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.808 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.809 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.810 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.811 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2024-09-11T23:49:05.812 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2024-09-11T23:49:05.813 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2024-09-11T23:49:05.814 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.815 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2024-09-11T23:49:05.816 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2024-09-11T23:49:05.817 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.818 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.819 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2024-09-11T23:49:05.820 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.821 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.822 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.823 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.824 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.825 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.826 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.827 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.828 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.829 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.831 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.832 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.833 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.835 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.836 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.837 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.838 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.839 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.839 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.839 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.839 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.839 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.839 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.839 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.839 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.839 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.840 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.841 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.842 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.843 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.844 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.845 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.846 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.847 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.848 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.849 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.852 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.852 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.852 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.852 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.852 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.852 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.853 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.854 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.855 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.856 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.857 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.858 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.859 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.860 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.861 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:49:05.862 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.864 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.865 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.866 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.867 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.868 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.869 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.870 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.871 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.872 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.873 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.874 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2024-09-11T23:49:05.875 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.876 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.877 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.878 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.879 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.880 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.881 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.882 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.883 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.884 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.885 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.886 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.887 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.888 INFO:teuthology.orchestra.run.smithi133.stdout: "periodic_async_work": { 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "periodic_async_work", 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Periodically check for async work", 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.889 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.890 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.891 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.892 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.893 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.894 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.895 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.897 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.898 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.899 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.901 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.902 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: "gid": 24811, 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "z", 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr_features": 4540701547738038271, 2024-09-11T23:49:05.903 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.904 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.906 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.907 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.908 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.909 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.910 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.912 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.913 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.914 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.915 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2024-09-11T23:49:05.918 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2024-09-11T23:49:05.919 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.920 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.921 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.922 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2024-09-11T23:49:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2024-09-11T23:49:05.924 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2024-09-11T23:49:05.925 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.926 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.927 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.928 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.929 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.930 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2024-09-11T23:49:05.931 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-09-11T23:49:05.932 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "read", 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap", 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap-read" 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2024-09-11T23:49:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:49:05.934 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.935 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:49:05.936 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.937 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:05.938 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.939 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.939 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-09-11T23:49:05.939 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.939 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.939 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.939 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:49:05.939 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.939 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.939 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.940 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.941 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.942 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.943 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.944 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm_log_destination": { 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm_log_destination", 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.945 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "file", 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "file,syslog", 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "syslog" 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.947 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.948 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:49:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:49:05.950 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:49:05.951 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:49:05.952 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:49:05.953 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:49:05.954 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:49:05.955 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:49:05.956 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:49:05.957 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:49:05.958 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:49:05.959 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_nvmeof": { 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_nvmeof", 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:49:05.960 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Nvme-of container image", 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:49:05.961 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:49:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_samba": { 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_samba", 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.963 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.964 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.965 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.966 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.967 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.968 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.969 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.970 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:05.971 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "grafana_dashboards_path": { 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:49:05.972 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:05.973 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "hw_monitoring": { 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hw_monitoring", 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.974 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "inventory_list_all": { 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "inventory_list_all", 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.975 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.976 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.977 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.978 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:05.981 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.983 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.984 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:05.985 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.986 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.987 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "oob_default_addr": { 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "oob_default_addr", 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.988 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "169.254.1.1", 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.989 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.991 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.992 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.993 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.994 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:05.995 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2024-09-11T23:49:05.996 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2024-09-11T23:49:05.997 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2024-09-11T23:49:05.998 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:49:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2024-09-11T23:49:06.000 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:49:06.001 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.002 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.003 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.004 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.005 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.006 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.007 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:49:06.008 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.009 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:49:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:49:06.011 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.012 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.013 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.014 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.015 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.016 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.017 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.019 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.020 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.021 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-09-11T23:49:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.037 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.038 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.039 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.040 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.041 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:49:06.042 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.043 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2024-09-11T23:49:06.044 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2024-09-11T23:49:06.045 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.046 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.047 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.048 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.049 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.050 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.051 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.052 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.053 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.053 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.053 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.053 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.053 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2024-09-11T23:49:06.053 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2024-09-11T23:49:06.053 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.053 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.053 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.053 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.056 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.056 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.057 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.058 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2024-09-11T23:49:06.059 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.060 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.061 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.062 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.063 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.064 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.065 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.066 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.067 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-09-11T23:49:06.068 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.069 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.070 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2024-09-11T23:49:06.071 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.072 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.073 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.074 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2024-09-11T23:49:06.075 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.076 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.077 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.078 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.079 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.080 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.081 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.082 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2024-09-11T23:49:06.083 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2024-09-11T23:49:06.084 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2024-09-11T23:49:06.085 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.086 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2024-09-11T23:49:06.087 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.088 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.089 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.090 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.091 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.092 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.093 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.094 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.095 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.096 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.097 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.099 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.100 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.101 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.102 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.103 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.104 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.105 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.106 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.107 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.108 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.109 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.110 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:49:06.111 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.112 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.113 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.114 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.115 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.116 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.117 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.118 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.119 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.120 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.121 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2024-09-11T23:49:06.123 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2024-09-11T23:49:06.124 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.125 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.126 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.128 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.130 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.131 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.132 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.133 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.134 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.135 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.136 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.137 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.138 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.139 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:49:06.139 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.139 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.139 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.139 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.139 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2024-09-11T23:49:06.139 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2024-09-11T23:49:06.139 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.139 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.139 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.139 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2024-09-11T23:49:06.140 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.140 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.140 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.140 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:49:06.140 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.140 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.140 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.140 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.140 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.141 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.142 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.143 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.144 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.145 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.146 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.147 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.148 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.149 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.150 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.151 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.152 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.153 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.154 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.155 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.156 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.157 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.158 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2024-09-11T23:49:06.159 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.160 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.161 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.162 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.163 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.164 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.165 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.166 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.167 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.168 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.169 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.170 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.171 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.172 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.173 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.174 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.175 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.176 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.177 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.178 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.179 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.180 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.181 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.182 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.183 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2024-09-11T23:49:06.184 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2024-09-11T23:49:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-09-11T23:49:06.186 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.187 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.188 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.190 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.191 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.192 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:06.193 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.194 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.195 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.196 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.197 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.198 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.199 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.200 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.201 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.203 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:06.204 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.205 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.207 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.208 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.209 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.210 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.211 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.213 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.214 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.215 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.216 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.217 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.219 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.220 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.221 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.222 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.223 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.224 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.225 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.226 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.227 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2024-09-11T23:49:06.228 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.229 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.230 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.231 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.232 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.233 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.234 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.236 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.237 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.238 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.239 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.240 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.241 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.242 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.243 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.244 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2024-09-11T23:49:06.245 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2024-09-11T23:49:06.246 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2024-09-11T23:49:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2024-09-11T23:49:06.248 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2024-09-11T23:49:06.249 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.250 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.251 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2024-09-11T23:49:06.252 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2024-09-11T23:49:06.253 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2024-09-11T23:49:06.254 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2024-09-11T23:49:06.255 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2024-09-11T23:49:06.256 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.257 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.258 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.259 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.260 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.261 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.262 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.263 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.264 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.265 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.266 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.267 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.268 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.269 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.270 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.271 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.272 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.273 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.274 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.275 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.277 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.278 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.279 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.280 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.281 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.282 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.283 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.284 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.285 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.286 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.287 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.288 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.289 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2024-09-11T23:49:06.290 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.291 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:06.292 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2024-09-11T23:49:06.293 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2024-09-11T23:49:06.294 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.295 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.296 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.297 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.298 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.299 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.300 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.301 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.302 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.304 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.305 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.306 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.307 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.308 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.309 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.310 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.311 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2024-09-11T23:49:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "periodic_async_work": { 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "periodic_async_work", 2024-09-11T23:49:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Periodically check for async work", 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:49:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:49:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "modules": [ 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2024-09-11T23:49:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "dashboard", 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "iostat", 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "nfs", 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "restful", 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "snap_schedule" 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2024-09-11T23:49:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2024-09-11T23:49:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2024-09-11T23:49:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2024-09-11T23:49:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2024-09-11T23:49:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:49:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:49:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:49:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:49:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.360 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "read", 2024-09-11T23:49:06.361 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap", 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap-read" 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.362 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:06.363 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.364 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.365 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.366 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.367 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.368 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.369 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:49:06.370 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.371 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:49:06.372 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm_log_destination": { 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm_log_destination", 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "file", 2024-09-11T23:49:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: "file,syslog", 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "syslog" 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.374 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.375 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.376 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.377 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.378 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.379 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.380 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:49:06.381 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.382 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.383 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:49:06.384 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2024-09-11T23:49:06.385 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.386 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2024-09-11T23:49:06.387 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_nvmeof": { 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_nvmeof", 2024-09-11T23:49:06.388 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Nvme-of container image", 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2024-09-11T23:49:06.389 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.390 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.391 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_samba": { 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_samba", 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.392 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.393 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.394 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.395 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:49:06.396 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.397 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.398 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.399 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.400 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "grafana_dashboards_path": { 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.401 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.402 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "hw_monitoring": { 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hw_monitoring", 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.403 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "inventory_list_all": { 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "inventory_list_all", 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.404 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.405 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2024-09-11T23:49:06.406 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.407 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.408 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.409 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.410 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.411 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:49:06.412 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:49:06.413 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:49:06.414 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.415 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.416 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "oob_default_addr": { 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "oob_default_addr", 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "169.254.1.1", 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.417 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.418 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.419 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.420 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.421 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.422 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.423 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.424 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.425 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.426 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.427 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.429 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.430 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2024-09-11T23:49:06.431 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.432 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.433 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.434 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.435 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.436 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.437 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:49:06.438 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.439 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.440 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:49:06.441 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:49:06.442 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:49:06.443 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:49:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.445 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.446 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.447 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.449 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.450 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.451 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:49:06.452 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:49:06.453 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:49:06.454 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.455 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:49:06.456 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:49:06.457 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:49:06.458 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.459 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.460 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.461 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.462 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.462 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.462 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.462 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.462 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.462 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.462 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.462 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.462 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.462 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.462 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:49:06.464 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:49:06.464 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.465 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.466 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.467 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.468 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.469 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.470 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.471 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.472 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.473 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.474 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.475 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.476 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.477 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.478 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.479 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.480 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.481 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.482 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.483 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.484 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.485 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.486 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.487 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.488 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.490 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.491 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.492 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.494 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.495 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.496 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-09-11T23:49:06.497 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2024-09-11T23:49:06.498 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-09-11T23:49:06.499 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.500 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.501 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.502 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.503 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.504 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.506 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.507 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.508 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.509 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.510 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.511 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.512 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.513 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.514 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.515 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.516 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.517 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.520 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2024-09-11T23:49:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-09-11T23:49:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2024-09-11T23:49:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2024-09-11T23:49:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.527 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2024-09-11T23:49:06.528 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.529 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.530 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.531 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.532 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.533 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.534 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.535 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.536 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:49:06.537 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:49:06.538 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:49:06.539 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.540 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.541 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.542 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.543 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.544 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.545 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.548 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.549 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.550 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.551 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.552 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.553 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.554 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.555 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.556 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.557 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.558 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2024-09-11T23:49:06.559 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.560 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.561 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.563 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.564 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.565 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.566 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.567 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.568 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.569 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.570 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.571 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.572 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.573 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.574 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.575 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.576 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.577 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.578 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.579 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.581 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.582 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.583 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.584 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.585 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.586 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2024-09-11T23:49:06.587 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.588 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.589 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.590 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.591 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.592 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.604 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.605 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.606 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.608 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.609 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.610 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.611 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.612 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.613 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-09-11T23:49:06.614 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.615 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.616 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.617 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.618 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.619 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.620 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.621 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.622 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.623 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.624 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.625 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.626 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.627 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.629 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.630 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.631 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.632 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.633 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.634 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.635 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-09-11T23:49:06.636 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.637 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.638 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.639 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.640 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.641 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.642 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.644 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.645 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2024-09-11T23:49:06.647 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-09-11T23:49:06.648 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.649 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.650 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.651 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.652 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.653 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.655 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.656 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.657 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.658 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.659 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.660 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.661 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.662 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.663 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.664 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.667 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.667 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.667 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.667 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.667 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.668 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.670 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.671 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.672 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.673 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.674 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.675 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.676 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.677 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.678 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.679 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.680 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.681 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.682 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.683 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-09-11T23:49:06.684 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.685 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.686 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.687 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.688 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.689 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.690 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.691 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.692 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.693 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.694 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.695 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.696 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.697 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.698 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.699 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.700 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.701 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.702 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.703 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.704 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.705 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.706 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.708 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.709 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.710 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.711 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.712 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.713 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.714 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.716 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.717 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.718 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.719 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.720 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.721 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.722 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.723 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.724 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.725 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.726 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.727 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.728 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.729 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.731 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.732 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.733 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.735 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.736 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.737 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.738 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.739 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.740 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.741 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.742 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.743 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.744 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.745 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.746 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2024-09-11T23:49:06.747 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: "periodic_async_work": { 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "periodic_async_work", 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Periodically check for async work", 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.749 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.750 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.751 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.752 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.753 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.754 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.755 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-09-11T23:49:06.756 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.757 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-09-11T23:49:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.759 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.760 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.761 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-09-11T23:49:06.762 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: "services": { 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: "dashboard": "https://172.21.15.133:8443/" 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: "always_on_modules": { 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: "octopus": [ 2024-09-11T23:49:06.763 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2024-09-11T23:49:06.764 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2024-09-11T23:49:06.764 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2024-09-11T23:49:06.764 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2024-09-11T23:49:06.764 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2024-09-11T23:49:06.764 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2024-09-11T23:49:06.764 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2024-09-11T23:49:06.764 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2024-09-11T23:49:06.764 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2024-09-11T23:49:06.764 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2024-09-11T23:49:06.764 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.764 INFO:teuthology.orchestra.run.smithi133.stdout: "pacific": [ 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "quincy": [ 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2024-09-11T23:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "reef": [ 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2024-09-11T23:49:06.766 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "squid": [ 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2024-09-11T23:49:06.767 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: "last_failure_osd_epoch": 82, 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: "active_clients": [ 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-09-11T23:49:06.768 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2287112023 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "libcephsqlite", 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1079235871 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:49:06.769 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 4138974730 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-09-11T23:49:06.770 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3457406725 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-09-11T23:49:06.771 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2148533549 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2205207650 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.772 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:49:06.773 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:06.773 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T23:49:06.773 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:49:06.774 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:05 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1595070731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:49:06.774 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:05 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3730481607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:49:06.774 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:05 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/4086481353' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:49:06.774 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:05 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1595070731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:49:06.774 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:05 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3730481607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:49:06.774 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:05 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/4086481353' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:49:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 -- 172.21.15.133:0/4086481353 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc2f00a2d80 msgr2=0x7fc2f00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc2f00a2d80 0x7fc2f00a5240 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005e10 tx=0x7fc2f4005da0 comp rx=0 tx=0).stop 2024-09-11T23:49:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 -- 172.21.15.133:0/4086481353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc304101ab0 msgr2=0x7fc304198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc304101ab0 0x7fc304198320 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00e9f0 tx=0x7fc2ec00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:49:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 -- 172.21.15.133:0/4086481353 shutdown_connections 2024-09-11T23:49:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc2f00a2d80 0x7fc2f00a5240 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304107ac0 0x7fc30419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc304102450 0x7fc304198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 --2- 172.21.15.133:0/4086481353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc304101ab0 0x7fc304198320 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 -- 172.21.15.133:0/4086481353 >> 172.21.15.133:0/4086481353 conn(0x7fc3040fd7d0 msgr2=0x7fc3040fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 -- 172.21.15.133:0/4086481353 shutdown_connections 2024-09-11T23:49:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:05.461+0000 7fc3096a5640 1 -- 172.21.15.133:0/4086481353 wait complete. 2024-09-11T23:49:06.840 DEBUG:tasks.check_counter:Getting stats from a 2024-09-11T23:49:06.841 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-11T23:49:06.841 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:05 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1595070731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:49:06.841 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:05 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3730481607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:49:06.841 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:05 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/4086481353' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:49:07.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:06 smithi178 ceph-mon[35882]: pgmap v386: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-11T23:49:07.100 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:07.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:06 smithi138 ceph-mon[35142]: pgmap v386: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-11T23:49:07.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:06 smithi133 ceph-mon[31430]: pgmap v386: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-11T23:49:07.584 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:49:07.584 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:07.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1107, 2024-09-11T23:49:07.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1092, 2024-09-11T23:49:07.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1788656, 2024-09-11T23:49:07.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 413092, 2024-09-11T23:49:07.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 5, 2024-09-11T23:49:07.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2024-09-11T23:49:07.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.221484429, 2024-09-11T23:49:07.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.072470881, 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.104315847, 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.047320727, 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1091, 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.359293348, 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000329324 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1507088, 2024-09-11T23:49:07.585 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 146608 2024-09-11T23:49:07.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.586 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:07.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 754, 2024-09-11T23:49:07.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 928, 2024-09-11T23:49:07.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 188004, 2024-09-11T23:49:07.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 717020, 2024-09-11T23:49:07.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 7, 2024-09-11T23:49:07.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2024-09-11T23:49:07.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.110636844, 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.031215154, 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.026197671, 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.040791821, 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 926, 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.107764711, 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000116376 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.587 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 479088 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 843, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 845, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 221965, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 268075, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.108609267, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.027527603, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.028021167, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.039129740, 2024-09-11T23:49:07.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 843, 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.076059361, 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000090224 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-09-11T23:49:07.589 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3858, 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.210524430, 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000054568 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.590 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 0, 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 0, 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-09-11T23:49:07.591 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2024-09-11T23:49:07.592 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2024-09-11T23:49:07.593 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 1, 2024-09-11T23:49:07.593 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 0, 2024-09-11T23:49:07.593 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 0, 2024-09-11T23:49:07.593 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-09-11T23:49:07.593 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-09-11T23:49:07.593 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:49:07.593 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-09-11T23:49:07.593 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-09-11T23:49:07.593 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-09-11T23:49:07.593 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-09-11T23:49:07.593 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-09-11T23:49:07.594 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-09-11T23:49:07.594 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-09-11T23:49:07.594 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-09-11T23:49:07.594 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-09-11T23:49:07.594 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:49:07.594 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-09-11T23:49:07.594 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 0, 2024-09-11T23:49:07.594 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 0, 2024-09-11T23:49:07.594 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-09-11T23:49:07.595 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:49:07.595 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-09-11T23:49:07.595 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:49:07.595 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:49:07.595 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:49:07.595 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:49:07.595 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:49:07.595 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-09-11T23:49:07.595 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:49:07.595 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:49:07.596 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:49:07.596 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:49:07.596 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:49:07.596 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:49:07.596 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 0, 2024-09-11T23:49:07.596 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:49:07.596 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2024-09-11T23:49:07.596 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 0 2024-09-11T23:49:07.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.596 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-09-11T23:49:07.596 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-09-11T23:49:07.597 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:49:07.598 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:49:07.598 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-09-11T23:49:07.598 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-09-11T23:49:07.598 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:49:07.598 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:49:07.598 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:49:07.598 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-09-11T23:49:07.598 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-09-11T23:49:07.598 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:49:07.598 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-09-11T23:49:07.598 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-09-11T23:49:07.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.599 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-09-11T23:49:07.599 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-09-11T23:49:07.599 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 0, 2024-09-11T23:49:07.599 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-09-11T23:49:07.599 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-09-11T23:49:07.599 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 4, 2024-09-11T23:49:07.599 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-09-11T23:49:07.599 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 0, 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 2, 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 2, 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 4195426, 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 4195285, 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-09-11T23:49:07.600 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.601 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.601 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.601 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.601 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 4 2024-09-11T23:49:07.601 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.601 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-09-11T23:49:07.601 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 3, 2024-09-11T23:49:07.601 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 3, 2024-09-11T23:49:07.601 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 0, 2024-09-11T23:49:07.601 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 1, 2024-09-11T23:49:07.601 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 1, 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 0, 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 0, 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 0, 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 0, 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 0, 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 0, 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 47776, 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215328 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-09-11T23:49:07.602 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 0, 2024-09-11T23:49:07.603 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-09-11T23:49:07.603 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 0, 2024-09-11T23:49:07.603 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 0, 2024-09-11T23:49:07.603 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-09-11T23:49:07.603 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-09-11T23:49:07.603 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.603 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.603 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.604 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.604 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-09-11T23:49:07.604 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.604 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.604 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.604 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.604 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-09-11T23:49:07.604 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.604 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.604 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.605 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.607 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-09-11T23:49:07.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.607 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.607 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-09-11T23:49:07.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.607 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.608 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.609 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-09-11T23:49:07.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-09-11T23:49:07.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.612 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 0, 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 1, 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 1, 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 1, 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-09-11T23:49:07.615 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 0, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 0, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 986, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:07.616 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:07.617 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:07.617 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:07.617 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:07.617 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:07.617 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:07.617 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:07.617 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:07.617 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:07.617 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:07.617 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:07.617 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-09-11T23:49:07.618 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 189450, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 53, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:07.619 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:07.620 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:07.620 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 7936, 2024-09-11T23:49:07.620 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 175, 2024-09-11T23:49:07.620 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:07.620 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:07.620 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:07.620 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-09-11T23:49:07.620 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 8512, 2024-09-11T23:49:07.620 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 13, 2024-09-11T23:49:07.620 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 2429, 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 21, 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 3, 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 2429, 2024-09-11T23:49:07.621 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2429, 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 4.364918183, 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001797002 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2429, 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2430 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 2429, 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 2429, 2024-09-11T23:49:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 0, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 1583, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 844, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 0, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 0, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-09-11T23:49:07.623 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 0, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 1, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-09-11T23:49:07.624 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-09-11T23:49:07.625 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 96, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 17, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 5, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 5, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-09-11T23:49:07.626 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 0, 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 2, 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 0 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-09-11T23:49:07.627 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 0, 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 2700, 2024-09-11T23:49:07.628 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1938904, 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 2700, 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 2700, 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1938904, 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:07.629 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 2429, 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 1127, 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 2429, 2024-09-11T23:49:07.630 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1127, 2024-09-11T23:49:07.631 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:07.631 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.631 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.631 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.631 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:07.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.631 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-09-11T23:49:07.631 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:07.631 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-09-11T23:49:07.632 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:07.632 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-11T23:49:07.632 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-11T23:49:07.632 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:07.632 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:07.632 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 2429, 2024-09-11T23:49:07.632 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 2429, 2024-09-11T23:49:07.632 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 2429, 2024-09-11T23:49:07.632 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 2429, 2024-09-11T23:49:07.632 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:07.633 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.633 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.633 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.633 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:07.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.633 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:49:07.634 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:07.634 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-09-11T23:49:07.634 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:07.634 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-11T23:49:07.634 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-11T23:49:07.634 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:07.635 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:07.635 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-11T23:49:07.635 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-11T23:49:07.635 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-09-11T23:49:07.635 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-09-11T23:49:07.636 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:07.636 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.636 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.636 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.636 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:07.636 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:07.637 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x5611eea6eb40": { 2024-09-11T23:49:07.637 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:07.637 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-09-11T23:49:07.637 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:07.637 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-11T23:49:07.637 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-11T23:49:07.638 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:07.638 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:07.638 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-11T23:49:07.638 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-11T23:49:07.638 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-09-11T23:49:07.638 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-09-11T23:49:07.639 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:07.639 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:07.639 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:07.639 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:07.639 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:07.640 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:07.640 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:49:08.389 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-11T23:49:08.390 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-11T23:49:08.390 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-11T23:49:08.390 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:49:08.390 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:49:08.390 DEBUG:tasks.check_counter:Getting stats from d 2024-09-11T23:49:08.390 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-11T23:49:08.637 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:09.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:08 smithi178 ceph-mon[35882]: pgmap v387: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-11T23:49:09.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:08 smithi133 ceph-mon[31430]: pgmap v387: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-11T23:49:09.145 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:49:09.145 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:09.145 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 4, 2024-09-11T23:49:09.145 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 4, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 3024, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 448, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.002976272, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000039782, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002134347, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.003935621, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001311873 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:09.146 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 247, 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 237, 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1450080, 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 145984, 2024-09-11T23:49:09.147 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2024-09-11T23:49:09.148 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2024-09-11T23:49:09.148 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.063915742, 2024-09-11T23:49:09.148 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.019951490, 2024-09-11T23:49:09.148 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.065014702, 2024-09-11T23:49:09.148 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:49:09.148 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:09.148 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 236, 2024-09-11T23:49:09.148 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.043773341, 2024-09-11T23:49:09.148 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000185480 2024-09-11T23:49:09.148 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1450080, 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 145984 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2, 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 182, 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 320, 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 189296, 2024-09-11T23:49:09.149 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 5, 2024-09-11T23:49:09.150 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-09-11T23:49:09.150 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.017463399, 2024-09-11T23:49:09.150 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.012864783, 2024-09-11T23:49:09.150 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002082835, 2024-09-11T23:49:09.150 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:49:09.150 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:09.152 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 179, 2024-09-11T23:49:09.152 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.026047776, 2024-09-11T23:49:09.152 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000145518 2024-09-11T23:49:09.152 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:09.152 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:09.152 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:09.152 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:09.152 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:09.152 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:09.152 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 189296 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:09.153 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:09.154 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:09.155 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 148994, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 31, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:09.157 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 249, 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1415258, 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 249, 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 249, 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1415258, 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:09.158 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:49:09.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:08 smithi138 ceph-mon[35142]: pgmap v387: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-11T23:49:09.854 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-11T23:49:09.855 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-11T23:49:09.855 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-11T23:49:09.855 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-11T23:49:09.855 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-11T23:49:09.855 DEBUG:tasks.check_counter:Getting stats from g 2024-09-11T23:49:09.856 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-11T23:49:10.103 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:10.603 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:49:10.604 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:10.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1104, 2024-09-11T23:49:10.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1089, 2024-09-11T23:49:10.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1787520, 2024-09-11T23:49:10.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 412782, 2024-09-11T23:49:10.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 5, 2024-09-11T23:49:10.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2024-09-11T23:49:10.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.216982401, 2024-09-11T23:49:10.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.078370610, 2024-09-11T23:49:10.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.100183541, 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.040653249, 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1088, 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.391381151, 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000359725 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1507632, 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 147888 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1572, 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1749, 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 402430, 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 978133, 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 7, 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.184717994, 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.054773385, 2024-09-11T23:49:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.046115677, 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.069056596, 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1746, 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.242449138, 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000138859 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 478816 2024-09-11T23:49:10.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 8, 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 8, 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 4653, 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 1732, 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 5, 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.003343431, 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000096324, 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.003150766, 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000063707, 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:10.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7, 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.006286502, 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000898071 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-09-11T23:49:10.609 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3871, 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.154057005, 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000039797 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-09-11T23:49:10.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 0, 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 0, 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-09-11T23:49:10.611 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2024-09-11T23:49:10.612 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 1, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-09-11T23:49:10.613 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:49:10.614 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 0, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 0 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-09-11T23:49:10.616 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-09-11T23:49:10.617 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 0, 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 9, 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 0, 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-09-11T23:49:10.618 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-09-11T23:49:10.619 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 2, 2024-09-11T23:49:10.619 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 2, 2024-09-11T23:49:10.619 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-09-11T23:49:10.619 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-09-11T23:49:10.619 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-09-11T23:49:10.619 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 4196802, 2024-09-11T23:49:10.619 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 4196661, 2024-09-11T23:49:10.619 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-09-11T23:49:10.619 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.620 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.620 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.620 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 9 2024-09-11T23:49:10.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.620 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-09-11T23:49:10.620 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 3, 2024-09-11T23:49:10.620 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 3, 2024-09-11T23:49:10.620 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 0, 2024-09-11T23:49:10.620 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 1, 2024-09-11T23:49:10.620 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 1, 2024-09-11T23:49:10.621 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 0, 2024-09-11T23:49:10.621 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 0, 2024-09-11T23:49:10.621 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 0, 2024-09-11T23:49:10.621 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 0, 2024-09-11T23:49:10.621 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2024-09-11T23:49:10.621 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 0, 2024-09-11T23:49:10.621 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 0, 2024-09-11T23:49:10.621 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 49212, 2024-09-11T23:49:10.621 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215328 2024-09-11T23:49:10.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.621 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 0, 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 0, 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 0, 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-09-11T23:49:10.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.624 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.625 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.626 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.627 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.630 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-09-11T23:49:10.631 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.632 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.632 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.632 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.632 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-09-11T23:49:10.632 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.632 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.632 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.632 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.632 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-09-11T23:49:10.632 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.632 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.633 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:10.634 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 4, 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 3, 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 4, 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 0, 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 0, 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 4734, 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:10.635 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:10.636 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:10.637 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 197503, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 55, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:10.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 7936, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 175, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 8512, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 13, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-09-11T23:49:10.639 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:10.640 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-09-11T23:49:10.640 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.640 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-09-11T23:49:10.640 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-09-11T23:49:10.640 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-09-11T23:49:10.640 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 2407, 2024-09-11T23:49:10.640 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 21, 2024-09-11T23:49:10.640 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 3, 2024-09-11T23:49:10.640 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 2407, 2024-09-11T23:49:10.640 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-09-11T23:49:10.640 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2407, 2024-09-11T23:49:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 12.272771003, 2024-09-11T23:49:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005098783 2024-09-11T23:49:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-09-11T23:49:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-09-11T23:49:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2407, 2024-09-11T23:49:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2408 2024-09-11T23:49:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 2407, 2024-09-11T23:49:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 2407, 2024-09-11T23:49:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 1564, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 841, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-09-11T23:49:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 1, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-09-11T23:49:10.643 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-09-11T23:49:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-09-11T23:49:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-09-11T23:49:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-09-11T23:49:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-09-11T23:49:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-09-11T23:49:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-09-11T23:49:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-09-11T23:49:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-09-11T23:49:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-09-11T23:49:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-09-11T23:49:10.644 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 96, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 17, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 6, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 6, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 0, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 2, 2024-09-11T23:49:10.645 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 0 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:49:10.646 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 0, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 2680, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1936578, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 2680, 2024-09-11T23:49:10.647 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 2680, 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1936578, 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-11T23:49:10.648 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 2407, 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 2503, 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 2407, 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 2503, 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.649 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 2407, 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 2407, 2024-09-11T23:49:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 2407, 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 2407, 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-11T23:49:10.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x557eda116b40": { 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:10.652 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:10.653 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:10.654 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:10.654 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:49:11.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:10 smithi138 ceph-mon[35142]: pgmap v388: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-11T23:49:11.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:10 smithi133 ceph-mon[31430]: pgmap v388: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-11T23:49:11.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:10 smithi178 ceph-mon[35882]: pgmap v388: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-11T23:49:11.380 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-11T23:49:11.380 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-11T23:49:11.380 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-11T23:49:11.380 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:49:11.380 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:49:11.380 DEBUG:tasks.check_counter:Getting stats from j 2024-09-11T23:49:11.381 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-11T23:49:11.803 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:12.079 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:11 smithi133 ceph-mon[31430]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T23:49:12.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:11 smithi138 ceph-mon[35142]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T23:49:12.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:11 smithi178 ceph-mon[35882]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T23:49:12.417 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T23:49:12.417 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:12.418 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2024-09-11T23:49:12.418 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2024-09-11T23:49:12.418 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2024-09-11T23:49:12.418 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2024-09-11T23:49:12.418 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-09-11T23:49:12.419 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2024-09-11T23:49:12.419 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.001158288, 2024-09-11T23:49:12.419 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T23:49:12.419 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000210269, 2024-09-11T23:49:12.419 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:49:12.419 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:12.419 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:12.419 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:12.419 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2, 2024-09-11T23:49:12.420 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 183, 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 320, 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 189472, 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 5, 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.019846080, 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.014644121, 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002200141, 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 180, 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.021596355, 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000119979 2024-09-11T23:49:12.421 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 189472 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 253, 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 241, 2024-09-11T23:49:12.422 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1453648, 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 147712, 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.065391163, 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.020929114, 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.057889766, 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 241, 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.082705828, 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000343177 2024-09-11T23:49:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1453648, 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 147712 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-09-11T23:49:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:12.426 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:12.426 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:12.426 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:12.426 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:12.426 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:12.426 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:12.426 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:12.426 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:12.426 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:12.426 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:12.427 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 148984, 2024-09-11T23:49:12.428 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 30, 2024-09-11T23:49:12.428 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:12.428 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:12.428 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-09-11T23:49:12.428 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-09-11T23:49:12.428 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:12.428 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:12.428 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:12.428 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:12.428 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2024-09-11T23:49:12.428 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-11T23:49:12.429 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 251, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1415518, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 251, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 251, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1415518, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-11T23:49:12.430 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-11T23:49:12.431 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:12.431 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T23:49:12.431 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T23:49:13.202 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:12 smithi138 ceph-mon[35142]: pgmap v390: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-11T23:49:13.203 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:12 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:13.203 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:12 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.203 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:12 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.203 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:12 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.203 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:12 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.203 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:12 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.203 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:12 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:12 smithi133 ceph-mon[31430]: pgmap v390: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-11T23:49:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:12 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:13.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:12 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:12 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:12 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:12 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:12 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.210 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:12 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.263 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-11T23:49:13.263 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-11T23:49:13.263 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-11T23:49:13.263 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-11T23:49:13.263 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-11T23:49:13.263 DEBUG:tasks.check_counter:Getting stats from b 2024-09-11T23:49:13.263 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-11T23:49:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:12 smithi178 ceph-mon[35882]: pgmap v390: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 60 GiB data, 186 GiB used, 886 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-11T23:49:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:12 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:49:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:12 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:12 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:12 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:12 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:12 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:12 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:13.516 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:49:13.903 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:13 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:13.904 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:13 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:13.904 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:13 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:14.013 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-11T23:49:14.013 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:14.013 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 69, 2024-09-11T23:49:14.013 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 126, 2024-09-11T23:49:14.013 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 19414, 2024-09-11T23:49:14.013 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 43745, 2024-09-11T23:49:14.013 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 11, 2024-09-11T23:49:14.013 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-09-11T23:49:14.013 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.022481366, 2024-09-11T23:49:14.014 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.005109087, 2024-09-11T23:49:14.014 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.011870347, 2024-09-11T23:49:14.014 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000836533, 2024-09-11T23:49:14.014 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:14.014 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 124, 2024-09-11T23:49:14.014 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.023845715, 2024-09-11T23:49:14.014 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000192304 2024-09-11T23:49:14.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.014 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:14.015 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 114, 2024-09-11T23:49:14.015 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000024537, 2024-09-11T23:49:14.015 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000215 2024-09-11T23:49:14.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.015 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 11856, 2024-09-11T23:49:14.015 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 28416 2024-09-11T23:49:14.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.015 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:14.015 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 82, 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 151, 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 19053, 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 572450, 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 44, 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 7, 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.030823947, 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.004305378, 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.024527563, 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000558550, 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 149, 2024-09-11T23:49:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.021976574, 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000147493 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 120, 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000037577, 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000313 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 8896, 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 142768 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 1616, 2024-09-11T23:49:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 1367, 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1892671, 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 1074074, 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 12, 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 5, 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.145643838, 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.055648336, 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.111343756, 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.004882179, 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1366, 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.076103559, 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000055712 2024-09-11T23:49:14.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2164, 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000458592, 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000211 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1540688, 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 508896 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-09-11T23:49:14.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 70, 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.002147586, 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000030679 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-11T23:49:14.020 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-09-11T23:49:14.021 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000014752, 2024-09-11T23:49:14.021 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000007376 2024-09-11T23:49:14.021 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:14.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.021 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-09-11T23:49:14.021 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 12, 2024-09-11T23:49:14.021 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 8, 2024-09-11T23:49:14.021 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-09-11T23:49:14.021 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8, 2024-09-11T23:49:14.021 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.002646446, 2024-09-11T23:49:14.021 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000330805 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 4, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 11, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 14, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 0, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 14, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-09-11T23:49:14.022 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 14, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 0, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 3, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 3, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 2, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 38, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 35, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 0, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 1, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-09-11T23:49:14.023 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 0, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 0, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 21, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 10738135935, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 6, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:49:14.024 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:49:14.025 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:49:14.025 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:49:14.025 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:49:14.025 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 0, 2024-09-11T23:49:14.025 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:49:14.025 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 16, 2024-09-11T23:49:14.025 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 11, 2024-09-11T23:49:14.025 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 16, 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 0, 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 1 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 0, 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 0, 2024-09-11T23:49:14.026 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 2, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-09-11T23:49:14.027 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 11, 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 55, 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 12, 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-09-11T23:49:14.028 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 12, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 2, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 2, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 4197729, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-09-11T23:49:14.029 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 11, 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.193428773, 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.017584433 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 17, 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 17, 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 0, 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 15, 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 16, 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 1, 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 14, 2024-09-11T23:49:14.030 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 14, 2024-09-11T23:49:14.031 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 0, 2024-09-11T23:49:14.031 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 3, 2024-09-11T23:49:14.031 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 6, 2024-09-11T23:49:14.031 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 3, 2024-09-11T23:49:14.031 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 51208, 2024-09-11T23:49:14.031 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 223520 2024-09-11T23:49:14.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.031 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-09-11T23:49:14.031 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 13, 2024-09-11T23:49:14.031 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-09-11T23:49:14.032 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 12, 2024-09-11T23:49:14.032 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 36, 2024-09-11T23:49:14.032 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 0, 2024-09-11T23:49:14.032 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-09-11T23:49:14.032 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.032 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.032 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 5, 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.002014561, 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000402912 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-09-11T23:49:14.033 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.034 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.034 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.034 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-09-11T23:49:14.034 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-09-11T23:49:14.034 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000457335, 2024-09-11T23:49:14.034 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000228667 2024-09-11T23:49:14.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.034 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-09-11T23:49:14.034 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.035 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.035 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.035 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-09-11T23:49:14.035 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1, 2024-09-11T23:49:14.035 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000174550, 2024-09-11T23:49:14.035 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000174550 2024-09-11T23:49:14.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.035 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-09-11T23:49:14.035 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.035 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.036 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-09-11T23:49:14.037 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.037 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.037 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.037 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-09-11T23:49:14.037 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.039 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.040 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.040 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-09-11T23:49:14.040 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.040 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.040 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.040 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.042 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-09-11T23:49:14.043 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.044 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:49:14.045 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:49:14.045 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-09-11T23:49:14.045 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.045 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.045 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.045 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:14.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.045 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-09-11T23:49:14.045 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 2, 2024-09-11T23:49:14.045 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 6, 2024-09-11T23:49:14.045 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 4, 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 2, 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 6, 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 1, 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 740, 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:14.047 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 262615, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 87, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 8056, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 178, 2024-09-11T23:49:14.049 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 68829, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 299, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-09-11T23:49:14.050 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 53, 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 14653, 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 30, 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 53, 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 53, 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.527988829, 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.009962053 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 53, 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 80 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 53, 2024-09-11T23:49:14.051 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 0, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 53, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 0, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 11, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 8, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 6, 2024-09-11T23:49:14.052 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:49:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-09-11T23:49:14.054 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 97, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 18, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 8, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 8, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 23, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 0, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 2 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-09-11T23:49:14.055 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-09-11T23:49:14.056 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:49:14.056 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-09-11T23:49:14.056 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-09-11T23:49:14.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.056 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-09-11T23:49:14.056 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 66 2024-09-11T23:49:14.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.056 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-09-11T23:49:14.056 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:14.057 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 1763, 2024-09-11T23:49:14.058 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1751249, 2024-09-11T23:49:14.058 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:14.058 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 1763, 2024-09-11T23:49:14.058 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:14.058 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:14.058 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 1763, 2024-09-11T23:49:14.058 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1751249, 2024-09-11T23:49:14.058 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:14.058 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.058 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.058 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 53, 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 14653, 2024-09-11T23:49:14.059 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 53, 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 14653, 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:14.060 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 53, 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 53, 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 53, 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 53, 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:14.061 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.062 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x56477c9e8960": { 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 12, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 3425, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 12, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 11, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 3425, 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:14.063 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:14.064 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:14.064 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:14.064 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:14.064 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:14.064 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-11T23:49:14.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:13 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:14.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:13 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:14.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:13 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:14.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:13 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:49:14.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:13 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:49:14.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:13 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' 2024-09-11T23:49:14.854 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-11T23:49:14.854 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-11T23:49:14.855 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-11T23:49:14.855 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:49:14.855 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:49:14.855 DEBUG:tasks.check_counter:Getting stats from e 2024-09-11T23:49:14.855 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-11T23:49:14.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:14 smithi138 ceph-mon[35142]: pgmap v391: 97 pgs: 4 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 90 active+clean; 53 GiB data, 176 GiB used, 897 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:15.118 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:49:15.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:14 smithi133 ceph-mon[31430]: pgmap v391: 97 pgs: 4 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 90 active+clean; 53 GiB data, 176 GiB used, 897 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:15.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:14 smithi178 ceph-mon[35882]: pgmap v391: 97 pgs: 4 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 90 active+clean; 53 GiB data, 176 GiB used, 897 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:15.626 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-11T23:49:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 66, 2024-09-11T23:49:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 66, 2024-09-11T23:49:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 15194, 2024-09-11T23:49:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 450923, 2024-09-11T23:49:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 45, 2024-09-11T23:49:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 1, 2024-09-11T23:49:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.025633491, 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.001532911, 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.019015962, 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000464363, 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 65, 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.016630110, 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000255847 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 47, 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000005579, 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000118 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 4608, 2024-09-11T23:49:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 1888 2024-09-11T23:49:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 327, 2024-09-11T23:49:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 308, 2024-09-11T23:49:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1539559, 2024-09-11T23:49:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 211128, 2024-09-11T23:49:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 9, 2024-09-11T23:49:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 4, 2024-09-11T23:49:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.049855776, 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.017429468, 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.048244892, 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000253688, 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 307, 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.024566697, 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000080021 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 4, 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000269, 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000067 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1535696, 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 156064 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 727, 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 894, 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 206652, 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 635719, 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 8, 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 4, 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.082579162, 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.034224353, 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.045143927, 2024-09-11T23:49:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.004885044, 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 891, 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.048202913, 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000054099 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1403, 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000350144, 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000249 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 482752 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 43, 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.001627205, 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000037841 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-09-11T23:49:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-11T23:49:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-11T23:49:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-09-11T23:49:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000009917, 2024-09-11T23:49:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000004958 2024-09-11T23:49:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-09-11T23:49:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 2, 2024-09-11T23:49:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 1, 2024-09-11T23:49:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-09-11T23:49:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1, 2024-09-11T23:49:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000169865, 2024-09-11T23:49:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000169865 2024-09-11T23:49:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-09-11T23:49:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 1, 2024-09-11T23:49:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-09-11T23:49:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-09-11T23:49:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 11, 2024-09-11T23:49:15.635 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-09-11T23:49:15.635 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-09-11T23:49:15.635 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 11, 2024-09-11T23:49:15.635 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 0, 2024-09-11T23:49:15.635 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 11, 2024-09-11T23:49:15.635 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-09-11T23:49:15.635 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 10, 2024-09-11T23:49:15.635 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 0, 2024-09-11T23:49:15.635 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 0, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 0, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 2, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 26, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 24, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 0, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 1, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-09-11T23:49:15.636 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 0, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 0, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 21, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 10738135935, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 6, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:49:15.637 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:49:15.638 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:49:15.638 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:49:15.638 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 0, 2024-09-11T23:49:15.638 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:49:15.638 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:49:15.638 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:49:15.638 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:49:15.638 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:49:15.638 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:49:15.638 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 0, 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 0, 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 1 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 0, 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 0, 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 2, 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-09-11T23:49:15.639 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-09-11T23:49:15.640 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:49:15.640 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-09-11T23:49:15.640 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-09-11T23:49:15.640 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-09-11T23:49:15.640 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-09-11T23:49:15.640 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-09-11T23:49:15.640 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:49:15.640 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:49:15.640 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-09-11T23:49:15.640 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 11, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 22, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 4, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-09-11T23:49:15.641 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 4, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 2, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 2, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-09-11T23:49:15.642 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 4195426, 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3, 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.005127678, 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001709226 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 14, 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 14, 2024-09-11T23:49:15.643 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 0, 2024-09-11T23:49:15.644 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 12, 2024-09-11T23:49:15.644 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 13, 2024-09-11T23:49:15.644 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 1, 2024-09-11T23:49:15.644 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 11, 2024-09-11T23:49:15.644 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 11, 2024-09-11T23:49:15.644 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 0, 2024-09-11T23:49:15.644 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 0, 2024-09-11T23:49:15.644 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 1, 2024-09-11T23:49:15.644 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 1, 2024-09-11T23:49:15.644 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 52928, 2024-09-11T23:49:15.645 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 223520 2024-09-11T23:49:15.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.645 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-09-11T23:49:15.645 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 3, 2024-09-11T23:49:15.645 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-09-11T23:49:15.645 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 2, 2024-09-11T23:49:15.645 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 4, 2024-09-11T23:49:15.645 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 0, 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1, 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000169865, 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000169865 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.646 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-09-11T23:49:15.647 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.648 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.649 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-09-11T23:49:15.650 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.650 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.650 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.650 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-09-11T23:49:15.650 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.650 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.652 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-09-11T23:49:15.652 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.652 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.652 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.652 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-09-11T23:49:15.652 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-09-11T23:49:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-09-11T23:49:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-09-11T23:49:15.655 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.655 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.655 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.655 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-09-11T23:49:15.655 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.655 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.655 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.655 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.656 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.657 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 0, 2024-09-11T23:49:15.658 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 1, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 1, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 0, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 1, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 1, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 87, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:15.659 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:15.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:15.661 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:15.661 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:15.661 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:15.661 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:15.661 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:15.661 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:15.661 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:15.661 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:15.661 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:15.661 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:15.661 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 237687, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 69, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:15.662 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 8056, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 178, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 55080, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 213, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-09-11T23:49:15.663 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 38, 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 11334, 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 33, 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 38, 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 38, 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.198995788, 2024-09-11T23:49:15.664 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005236731 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 38, 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 60 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 38, 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 0, 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 38, 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-09-11T23:49:15.665 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 0, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 3, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 6, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 6, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-09-11T23:49:15.666 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-09-11T23:49:15.667 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-09-11T23:49:15.668 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-09-11T23:49:15.668 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-09-11T23:49:15.668 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-09-11T23:49:15.668 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-09-11T23:49:15.668 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-09-11T23:49:15.668 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-09-11T23:49:15.668 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-09-11T23:49:15.668 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-09-11T23:49:15.668 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-09-11T23:49:15.668 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-09-11T23:49:15.668 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-09-11T23:49:15.669 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-09-11T23:49:15.669 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 97, 2024-09-11T23:49:15.669 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-09-11T23:49:15.669 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 18, 2024-09-11T23:49:15.669 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 8, 2024-09-11T23:49:15.669 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 8, 2024-09-11T23:49:15.669 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-09-11T23:49:15.669 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-09-11T23:49:15.669 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 14, 2024-09-11T23:49:15.669 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 0, 2024-09-11T23:49:15.669 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 1 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 33 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-09-11T23:49:15.670 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 1116, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1642008, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 1116, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:15.671 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 1116, 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1642008, 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:15.672 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 38, 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 11334, 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 38, 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 11334, 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:15.673 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 38, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 38, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 38, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 38, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.674 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.675 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:15.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.675 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:49:15.675 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:15.675 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-11T23:49:15.675 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:15.675 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:15.675 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:15.675 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:15.675 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:15.676 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:15.676 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:15.676 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-09-11T23:49:15.676 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-09-11T23:49:15.676 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55b399bae960": { 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 4, 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1122, 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 4, 2024-09-11T23:49:15.677 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:15.678 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:15.678 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 3, 2024-09-11T23:49:15.678 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1122, 2024-09-11T23:49:15.678 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:15.678 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:15.678 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:15.678 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:15.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:15.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:15.678 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-11T23:49:16.393 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-11T23:49:16.393 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-11T23:49:16.393 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-11T23:49:16.393 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:49:16.393 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:49:16.393 DEBUG:tasks.check_counter:Getting stats from h 2024-09-11T23:49:16.394 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-11T23:49:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:49:17.062 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:16 smithi138 ceph-mon[35142]: pgmap v392: 97 pgs: 4 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 90 active+clean; 51 GiB data, 175 GiB used, 897 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:17.168 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-11T23:49:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 2461, 2024-09-11T23:49:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 2461, 2024-09-11T23:49:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 699503, 2024-09-11T23:49:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 781781, 2024-09-11T23:49:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 6, 2024-09-11T23:49:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-09-11T23:49:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.196888600, 2024-09-11T23:49:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.057936049, 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.051772690, 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.072727869, 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2460, 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.076109309, 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000030938 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-11T23:49:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 4, 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 183, 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 876, 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 481482, 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 6, 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 4, 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.011449918, 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.006859654, 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.002714536, 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000059764, 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 180, 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.017069534, 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000094830 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 480784 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 281, 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 261, 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1534972, 2024-09-11T23:49:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 150302, 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 3, 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.046026317, 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.016383455, 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.042374743, 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000021155, 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 260, 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.014385501, 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000055328 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1534704, 2024-09-11T23:49:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 149984 2024-09-11T23:49:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-09-11T23:49:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-09-11T23:49:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-09-11T23:49:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 4123, 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.106023229, 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000025715 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 0, 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 0, 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 0, 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-09-11T23:49:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 0, 2024-09-11T23:49:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 1, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 0, 2024-09-11T23:49:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:49:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 0, 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 0 2024-09-11T23:49:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-09-11T23:49:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-09-11T23:49:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 0, 2024-09-11T23:49:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 0, 2024-09-11T23:49:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 0, 2024-09-11T23:49:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-09-11T23:49:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-09-11T23:49:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-09-11T23:49:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:49:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-09-11T23:49:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-09-11T23:49:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-09-11T23:49:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-09-11T23:49:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-09-11T23:49:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:49:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:49:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-09-11T23:49:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-09-11T23:49:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:49:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 0, 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-09-11T23:49:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 4, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 0, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 2, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-09-11T23:49:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 4195426, 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 4195285, 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 4 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 3, 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 3, 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 0, 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 1, 2024-09-11T23:49:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 1, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 0, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 0, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 0, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 0, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 0, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 0, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 0, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 47972, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 215328 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 0, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 0, 2024-09-11T23:49:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 0, 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 0, 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-09-11T23:49:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-09-11T23:49:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-09-11T23:49:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-09-11T23:49:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-09-11T23:49:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-09-11T23:49:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-09-11T23:49:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 0, 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 1, 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 1, 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 1, 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 0, 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 0, 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 1334, 2024-09-11T23:49:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-09-11T23:49:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 181303, 2024-09-11T23:49:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 46, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 8056, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 178, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 8512, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 13, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 2460, 2024-09-11T23:49:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 21, 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 3, 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 2460, 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2460, 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2.969938476, 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001207292 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2460, 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2461 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 2460, 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 2460, 2024-09-11T23:49:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 0, 2024-09-11T23:49:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-09-11T23:49:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-09-11T23:49:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 1600, 2024-09-11T23:49:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 0, 2024-09-11T23:49:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 858, 2024-09-11T23:49:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 0, 2024-09-11T23:49:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 0, 2024-09-11T23:49:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-09-11T23:49:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-09-11T23:49:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-09-11T23:49:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 1, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:49:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-09-11T23:49:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 97, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 18, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 4, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 4, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 0, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 2, 2024-09-11T23:49:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 0 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 0, 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:49:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 2742, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1971304, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 2742, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2742, 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1971304, 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 2460, 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 1127, 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2460, 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1127, 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 2460, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 2460, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2460, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 2460, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:49:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-11T23:49:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55f02c490b40": { 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:17.215 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-11T23:49:17.216 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:16 smithi133 ceph-mon[31430]: pgmap v392: 97 pgs: 4 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 90 active+clean; 51 GiB data, 175 GiB used, 897 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:17.292 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:16 smithi178 ceph-mon[35882]: pgmap v392: 97 pgs: 4 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 90 active+clean; 51 GiB data, 175 GiB used, 897 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:18.046 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-11T23:49:18.046 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-11T23:49:18.046 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-09-11T23:49:18.046 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:49:18.046 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:49:18.047 DEBUG:tasks.check_counter:Getting stats from k 2024-09-11T23:49:18.047 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-11T23:49:18.302 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:49:18.791 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-11T23:49:18.791 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:18.792 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 375, 2024-09-11T23:49:18.792 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 363, 2024-09-11T23:49:18.792 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1541479, 2024-09-11T23:49:18.792 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 585096, 2024-09-11T23:49:18.792 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 44, 2024-09-11T23:49:18.792 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 5, 2024-09-11T23:49:18.792 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.073575383, 2024-09-11T23:49:18.792 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.018332890, 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.063923054, 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000583420, 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 360, 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.028234507, 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000078429 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 52, 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000007521, 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000144 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1532384, 2024-09-11T23:49:18.793 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 161200 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 737, 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 809, 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 210520, 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 447815, 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 9, 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 4, 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.087713667, 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.030580605, 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.055930546, 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.004647796, 2024-09-11T23:49:18.794 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 809, 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.044172384, 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000054601 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1461, 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000347676, 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000237 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 5280, 2024-09-11T23:49:18.795 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 142304 2024-09-11T23:49:18.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.796 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:18.796 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 29, 2024-09-11T23:49:18.796 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 182, 2024-09-11T23:49:18.796 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 6919, 2024-09-11T23:49:18.796 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 442553, 2024-09-11T23:49:18.796 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 13, 2024-09-11T23:49:18.796 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 5, 2024-09-11T23:49:18.796 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.012610833, 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.006278232, 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.004132693, 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000589837, 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 180, 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.029132406, 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000161846 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 6, 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000487, 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000081 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.797 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 358944 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 52, 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.002406802, 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000046284 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.798 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000020563, 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000010281 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 4, 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 4, 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 4, 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.001639520, 2024-09-11T23:49:18.799 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000409880 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 0, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 11, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 11, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 0, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 11, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 10, 2024-09-11T23:49:18.800 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 1, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 1, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 1, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 2, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 30, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 26, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 0, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 2, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-09-11T23:49:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-09-11T23:49:18.802 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 0, 2024-09-11T23:49:18.802 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 0, 2024-09-11T23:49:18.802 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-09-11T23:49:18.802 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:49:18.802 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-09-11T23:49:18.802 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 21, 2024-09-11T23:49:18.802 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 10738135935, 2024-09-11T23:49:18.802 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 6, 2024-09-11T23:49:18.802 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:49:18.802 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-09-11T23:49:18.802 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:49:18.803 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:49:18.803 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:49:18.803 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:49:18.803 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:49:18.803 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 0, 2024-09-11T23:49:18.803 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:49:18.803 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 3, 2024-09-11T23:49:18.803 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:49:18.803 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:49:18.803 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:49:18.803 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 3, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 0, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 0 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 0, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 0, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 3, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:49:18.804 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 11, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 22, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-09-11T23:49:18.805 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:49:18.806 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-09-11T23:49:18.806 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-09-11T23:49:18.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.806 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-09-11T23:49:18.806 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-09-11T23:49:18.806 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 9, 2024-09-11T23:49:18.806 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-09-11T23:49:18.806 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-09-11T23:49:18.806 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 9, 2024-09-11T23:49:18.806 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-09-11T23:49:18.806 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 2, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 2, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 4196802, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.155187827, 2024-09-11T23:49:18.807 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.019398478 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 14, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 14, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 0, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 13, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 14, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 1, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 11, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 12, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 1, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 1, 2024-09-11T23:49:18.808 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 3, 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 2, 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 51140, 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 223520 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 8, 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 4, 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 22, 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 1, 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.809 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1, 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000950957, 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000950957 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.810 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000585115, 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000292557 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1, 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000103448, 2024-09-11T23:49:18.811 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000103448 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-09-11T23:49:18.812 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.813 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.813 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.813 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-09-11T23:49:18.813 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.813 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.813 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.813 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-09-11T23:49:18.814 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.814 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.814 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.815 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.816 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-09-11T23:49:18.817 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.817 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.817 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.817 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-09-11T23:49:18.817 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.817 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.817 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.817 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.819 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-09-11T23:49:18.820 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.820 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.820 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.820 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-09-11T23:49:18.820 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.820 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.820 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.820 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-09-11T23:49:18.821 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 1, 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 4, 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 3, 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 1, 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 3, 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 0, 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 724, 2024-09-11T23:49:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-09-11T23:49:18.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.823 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-09-11T23:49:18.823 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:18.823 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:18.823 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:18.823 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:18.823 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:18.823 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:18.823 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:18.823 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:18.824 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:18.825 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 246231, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 78, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 8056, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 178, 2024-09-11T23:49:18.826 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 56928, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 228, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-09-11T23:49:18.827 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 47, 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 13035, 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 33, 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 47, 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 47, 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.433990787, 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.009233846 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 47, 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 71 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 47, 2024-09-11T23:49:18.828 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 0, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 47, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 0, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 8, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 8, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-09-11T23:49:18.829 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 6, 2024-09-11T23:49:18.830 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-09-11T23:49:18.830 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-09-11T23:49:18.830 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-09-11T23:49:18.830 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 0, 2024-09-11T23:49:18.830 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-09-11T23:49:18.830 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:49:18.830 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-09-11T23:49:18.830 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:49:18.830 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-09-11T23:49:18.830 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-09-11T23:49:18.830 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-09-11T23:49:18.831 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 97, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 18, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 9, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 9, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-09-11T23:49:18.832 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 17, 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 0, 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 2 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-09-11T23:49:18.833 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 33 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:18.834 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 1137, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1636529, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 1137, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 1137, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1636529, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:18.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 47, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 13035, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 47, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 13035, 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:18.836 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.837 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.837 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.837 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:18.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.837 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-09-11T23:49:18.837 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:18.837 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-09-11T23:49:18.837 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:18.837 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:18.837 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 47, 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 47, 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 47, 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 47, 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:49:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:18.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x5642d71d2960": { 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 9, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 2498, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 9, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 8, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 2498, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-11T23:49:18.840 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-11T23:49:18.841 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-11T23:49:18.841 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:18.841 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-11T23:49:18.841 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-11T23:49:18.947 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:18 smithi138 ceph-mon[35142]: pgmap v393: 97 pgs: 97 active+clean; 50 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:19.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:18 smithi133 ceph-mon[31430]: pgmap v393: 97 pgs: 97 active+clean; 50 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:19.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:18 smithi178 ceph-mon[35882]: pgmap v393: 97 pgs: 97 active+clean; 50 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:19.662 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-11T23:49:19.662 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-11T23:49:19.662 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-09-11T23:49:19.662 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:49:19.662 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:49:19.662 DEBUG:tasks.check_counter:Getting stats from c 2024-09-11T23:49:19.663 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-11T23:49:19.923 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:49:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:19 smithi178 ceph-mon[35882]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T23:49:20.251 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:19 smithi178 ceph-mon[35882]: pgmap v395: 97 pgs: 97 active+clean; 50 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-09-11T23:49:20.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:19 smithi138 ceph-mon[35142]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T23:49:20.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:19 smithi138 ceph-mon[35142]: pgmap v395: 97 pgs: 97 active+clean; 50 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-09-11T23:49:20.455 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-11T23:49:20.456 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:20.456 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 376, 2024-09-11T23:49:20.456 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 358, 2024-09-11T23:49:20.456 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1561006, 2024-09-11T23:49:20.456 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 582341, 2024-09-11T23:49:20.456 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 43, 2024-09-11T23:49:20.456 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-09-11T23:49:20.456 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.075404691, 2024-09-11T23:49:20.456 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.019657041, 2024-09-11T23:49:20.456 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.065830014, 2024-09-11T23:49:20.456 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000288050, 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 355, 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.031647991, 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000089149 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 53, 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000006862, 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000129 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1552480, 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 160208 2024-09-11T23:49:20.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 722, 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 903, 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 202786, 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 636636, 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 8, 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.095109853, 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.041654822, 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.053090531, 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.004686609, 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:20.458 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 903, 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.059533422, 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000065928 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1403, 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000344250, 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000245 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 484368 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 23, 2024-09-11T23:49:20.459 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 25, 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 5565, 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 86757, 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 12, 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.005012858, 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.000433990, 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.003755499, 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000447501, 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 23, 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.014577216, 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000633792 2024-09-11T23:49:20.460 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 6, 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000465, 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000077 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-11T23:49:20.461 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 44, 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001328550, 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000030194 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000012154, 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000006077 2024-09-11T23:49:20.462 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 3, 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 3, 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3, 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001874600, 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000624866 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-11T23:49:20.463 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 11, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 11, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 11, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 10, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 1, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 2, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 30, 2024-09-11T23:49:20.464 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 26, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 2, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-11T23:49:20.465 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 21, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 10738135935, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 6, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 2, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:49:20.466 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 2, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 3, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-09-11T23:49:20.467 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 11, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 22, 2024-09-11T23:49:20.468 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 4, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 4, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-11T23:49:20.469 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 2, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4195426, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.047584859, 2024-09-11T23:49:20.470 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.015861619 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 14, 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 14, 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 0, 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 13, 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 14, 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 1, 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 11, 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 12, 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 1, 2024-09-11T23:49:20.471 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 2, 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 2, 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 50316, 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 223520 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 8, 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 3, 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 4, 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 1, 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-11T23:49:20.472 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1, 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001222539, 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001222539 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-11T23:49:20.473 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000652061, 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000326030 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.474 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.475 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-11T23:49:20.476 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.477 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:19 smithi133 ceph-mon[31430]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T23:49:20.477 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:19 smithi133 ceph-mon[31430]: pgmap v395: 97 pgs: 97 active+clean; 50 GiB data, 174 GiB used, 899 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-09-11T23:49:20.477 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.477 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-11T23:49:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-11T23:49:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.480 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.481 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-11T23:49:20.482 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.483 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-11T23:49:20.484 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 0, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 1, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 1, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 0, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 2, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 2, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 87, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:20.485 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:20.486 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:20.487 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 245719, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 71, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 7936, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 175, 2024-09-11T23:49:20.488 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 56568, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 227, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-11T23:49:20.489 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 41, 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 11569, 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 33, 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 41, 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 41, 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.329993774, 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.008048628 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 41, 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 65 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 41, 2024-09-11T23:49:20.490 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 0, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 41, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 0, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 10, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 0, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 3, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 7, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 6, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-11T23:49:20.491 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 20, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-11T23:49:20.492 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-11T23:49:20.493 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 98, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 19, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 9, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 9, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 17, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 2 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-09-11T23:49:20.494 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 33 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-09-11T23:49:20.495 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 1117, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1649544, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 1117, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1117, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1649544, 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:20.496 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 41, 2024-09-11T23:49:20.497 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 11569, 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 41, 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 11569, 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:20.498 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 41, 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 41, 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 41, 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 41, 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:20.499 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.500 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x5567d026c960": { 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 4, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1122, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 4, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 3, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1122, 2024-09-11T23:49:20.501 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:20.502 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:20.502 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-11T23:49:21.202 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-11T23:49:21.203 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-11T23:49:21.203 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-11T23:49:21.203 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:49:21.203 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:49:21.203 DEBUG:tasks.check_counter:Getting stats from f 2024-09-11T23:49:21.203 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-11T23:49:21.458 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:49:21.999 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-11T23:49:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 182, 2024-09-11T23:49:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 184, 2024-09-11T23:49:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1140009, 2024-09-11T23:49:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 57943, 2024-09-11T23:49:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-09-11T23:49:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-09-11T23:49:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.020647042, 2024-09-11T23:49:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.005538288, 2024-09-11T23:49:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.007688051, 2024-09-11T23:49:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.005004650, 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 182, 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.025307064, 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000139049 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:49:22.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 45, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 42, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 171106, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 18163, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.010724278, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.002163546, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.011530347, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000123288, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 42, 2024-09-11T23:49:22.002 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.054316647, 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001293253 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 169760, 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 17200 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 82, 2024-09-11T23:49:22.003 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 101, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 20796, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 142959, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.009906409, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.003543264, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.002866086, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.002618295, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 99, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.006761749, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000068300 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:22.004 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 117200 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T23:49:22.005 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 437, 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.011557868, 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000026448 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.006 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-11T23:49:22.007 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 72, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 23, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 49, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 49, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 2, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-09-11T23:49:22.008 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-09-11T23:49:22.009 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 6, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:49:22.010 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-11T23:49:22.011 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:49:22.012 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 154, 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-11T23:49:22.013 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 1083135314, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 1083441079, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 1083440967, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.014 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 154 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 31, 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 49, 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 18, 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 19, 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 19, 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 0, 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 72, 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 72, 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 0, 2024-09-11T23:49:22.015 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 44532, 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 215328 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.016 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.017 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-11T23:49:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.021 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.021 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.021 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.021 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-11T23:49:22.021 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.021 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.021 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-11T23:49:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 2, 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 0, 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 0, 2024-09-11T23:49:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 1972, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 189973, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 87, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 7936, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 175, 2024-09-11T23:49:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 188146, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 1643, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-11T23:49:22.033 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 266, 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 26, 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 4, 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 266, 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 266, 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.435992294, 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001639068 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 266, 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 267 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.034 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 266, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 266, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 160, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 103, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-11T23:49:22.035 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 2, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:49:22.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-11T23:49:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 98, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 4, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 5, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 5, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-09-11T23:49:22.038 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-11T23:49:22.039 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 305, 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1299538, 2024-09-11T23:49:22.040 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 305, 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 305, 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1299538, 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:22.041 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 266, 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 1081993, 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 266, 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1081993, 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.042 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 266, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 266, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 266, 2024-09-11T23:49:22.043 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 266, 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:22.044 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55b952c4a6e0": { 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:22.045 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:22.046 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:22.047 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-11T23:49:23.022 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:22 smithi178 ceph-mon[35882]: pgmap v396: 97 pgs: 97 active+clean; 50 GiB data, 173 GiB used, 899 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:23.047 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-11T23:49:23.047 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-11T23:49:23.047 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-11T23:49:23.047 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:49:23.047 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:49:23.047 DEBUG:tasks.check_counter:Getting stats from i 2024-09-11T23:49:23.047 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-11T23:49:23.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:22 smithi138 ceph-mon[35142]: pgmap v396: 97 pgs: 97 active+clean; 50 GiB data, 173 GiB used, 899 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:23.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:22 smithi133 ceph-mon[31430]: pgmap v396: 97 pgs: 97 active+clean; 50 GiB data, 173 GiB used, 899 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T23:49:23.297 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:49:23.791 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-11T23:49:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 268786, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 270247, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 106905379, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 472304150, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 22, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 6, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 14.978930135, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 6.990288465, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 20.704768151, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.418529078, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 270239, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 40.950113492, 2024-09-11T23:49:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000151532 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 401903, 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.080882012, 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000201 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 105285408, 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 105976488 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 4964, 2024-09-11T23:49:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 5317, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 3062060, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 385517951, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 48, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 6, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 1.534965581, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 1.080681695, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.193663404, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.268264819, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5319, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1.591147157, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000299144 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2114, 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000620776, 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000293 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1559440, 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 178096 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 3575, 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 3673, 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 853357, 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 214978410, 2024-09-11T23:49:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 233, 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 7, 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 1.141174390, 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.746026942, 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.226948270, 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.172164206, 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3674, 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1.309060735, 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000356303 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3112, 2024-09-11T23:49:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000678218, 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000217 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 46304, 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 495888 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5379, 2024-09-11T23:49:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 93.073332692, 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.017303092 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3034, 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.088632990, 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000029213 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-11T23:49:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 441, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 441, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 441, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 12.380493943, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.028073682 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 270, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 1, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-11T23:49:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 171, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 88, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 83, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 44, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 2, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 2, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 2, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 722, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 712, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-09-11T23:49:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 475, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 6, 2024-09-11T23:49:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 475, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 173, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 17, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 156, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 302, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 2, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 321, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 2894, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:49:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 262949, 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 262653, 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 8, 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 58 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 10, 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 4, 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-11T23:49:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 22, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 18, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 11, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 342, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 17, 2024-09-11T23:49:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 1, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 261800, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 261735, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 261735, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 65, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 89, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 379, 2024-09-11T23:49:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 378, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 378, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 1, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 1, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 1, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 1083135314, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 1083441079, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 4947, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 574.386343957, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.116108013 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-09-11T23:49:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 135, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 170, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 35, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 29, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 30, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 1, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 171, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 181, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 10, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 2, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 50, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 48, 2024-09-11T23:49:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 74968, 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 231712 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 616, 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 2, 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 441, 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 30, 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 4, 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 22, 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.018742398, 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000851927 2024-09-11T23:49:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 180, 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 10.078806180, 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.055993367 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 149, 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.927665170, 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.006225940 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-11T23:49:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-11T23:49:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 13, 2024-09-11T23:49:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001958153, 2024-09-11T23:49:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000150627 2024-09-11T23:49:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-11T23:49:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 10, 2024-09-11T23:49:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.015830287, 2024-09-11T23:49:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001583028 2024-09-11T23:49:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-11T23:49:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 13, 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1.087995020, 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.083691924 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000704177, 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000352088 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 12, 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.015708942, 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001309078 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.002685121, 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001342560 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.815 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 4, 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.102751147, 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.025687786 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-11T23:49:23.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 7, 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.067848252, 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.009692607 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1, 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001601205, 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001601205 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 8, 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.020498353, 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002562294 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 18, 2024-09-11T23:49:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.037699538, 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002094418 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 2, 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 4, 2024-09-11T23:49:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 2, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 2, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 69, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 34, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 746, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 343829, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 139, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 7936, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 175, 2024-09-11T23:49:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-11T23:49:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 500759, 2024-09-11T23:49:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 3681, 2024-09-11T23:49:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-11T23:49:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-11T23:49:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 10482, 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 1080455078, 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 36, 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 10482, 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 10482, 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 594.383932863, 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.056705202 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-11T23:49:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 10482, 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 11138 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 10482, 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 514, 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 9968, 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 240, 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 54, 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 22, 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 4952, 2024-09-11T23:49:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 179, 2024-09-11T23:49:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-11T23:49:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-11T23:49:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-11T23:49:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 1, 2024-09-11T23:49:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 4116, 2024-09-11T23:49:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-11T23:49:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-11T23:49:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-11T23:49:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 492, 2024-09-11T23:49:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 108, 2024-09-11T23:49:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-11T23:49:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-11T23:49:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 98, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 19, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 12, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 12, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 943, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 31 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-09-11T23:49:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 353 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 1025, 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 6, 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 3090, 2024-09-11T23:49:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 18, 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 277321, 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 72692011, 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 277321, 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 277321, 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 72692011, 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 10482, 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 1080457760, 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 10482, 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1080457760, 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-11T23:49:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 10482, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 10482, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 10482, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 10482, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 18, 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 2682, 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 18, 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 5, 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 2682, 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x56129450a960": { 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 261800, 2024-09-11T23:49:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1079246775, 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 261800, 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 4947, 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1079246775, 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:23.842 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-11T23:49:24.618 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-11T23:49:24.618 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-11T23:49:24.618 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-11T23:49:24.618 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=13 2024-09-11T23:49:24.618 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=7 2024-09-11T23:49:24.618 DEBUG:tasks.check_counter:Getting stats from l 2024-09-11T23:49:24.618 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-11T23:49:24.868 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:49:25.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:24 smithi178 ceph-mon[35882]: pgmap v397: 97 pgs: 97 active+clean; 50 GiB data, 168 GiB used, 905 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:49:25.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:24 smithi138 ceph-mon[35142]: pgmap v397: 97 pgs: 97 active+clean; 50 GiB data, 168 GiB used, 905 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:49:25.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:24 smithi133 ceph-mon[31430]: pgmap v397: 97 pgs: 97 active+clean; 50 GiB data, 168 GiB used, 905 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:49:25.387 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-11T23:49:25.388 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:49:25.388 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 843, 2024-09-11T23:49:25.388 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 845, 2024-09-11T23:49:25.388 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 224802, 2024-09-11T23:49:25.388 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 268138, 2024-09-11T23:49:25.388 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-09-11T23:49:25.388 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-09-11T23:49:25.388 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.074704089, 2024-09-11T23:49:25.388 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.022508239, 2024-09-11T23:49:25.388 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.020175818, 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.024037149, 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 843, 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.035318882, 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000041896 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.389 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 741, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 926, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 182136, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 720079, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 7, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.087083764, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.023826183, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.021113225, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.033929130, 2024-09-11T23:49:25.390 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 923, 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.041774017, 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000045258 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 485072 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:49:25.391 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1125, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1103, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1820544, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 420000, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 3, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.144847418, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.044665414, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.074370435, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.032290507, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1103, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.060175850, 2024-09-11T23:49:25.392 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000054556 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1540992, 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 155424 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-11T23:49:25.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 4036, 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.123973113, 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000030716 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.394 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-11T23:49:25.395 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 1, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-09-11T23:49:25.396 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-11T23:49:25.397 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:49:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-09-11T23:49:25.399 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-09-11T23:49:25.400 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-09-11T23:49:25.400 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-11T23:49:25.400 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-11T23:49:25.400 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:49:25.400 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-09-11T23:49:25.400 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-09-11T23:49:25.400 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-11T23:49:25.400 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-11T23:49:25.400 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-11T23:49:25.400 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:49:25.401 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:49:25.401 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-11T23:49:25.401 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-11T23:49:25.401 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:49:25.401 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:49:25.401 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:49:25.401 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-11T23:49:25.401 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-11T23:49:25.401 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:49:25.401 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 12, 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-11T23:49:25.402 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4197729, 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 4197588, 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 12 2024-09-11T23:49:25.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 3, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 3, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 0, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 1, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 1, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 0, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 0, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 0, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 0, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 48984, 2024-09-11T23:49:25.404 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 215328 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-11T23:49:25.405 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-11T23:49:25.406 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.407 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.408 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.409 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-11T23:49:25.409 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.409 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.409 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.409 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-11T23:49:25.409 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.410 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.410 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.410 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-11T23:49:25.410 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.410 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.410 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.411 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-11T23:49:25.411 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.411 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.411 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.411 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-11T23:49:25.411 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.411 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.413 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.414 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.415 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-11T23:49:25.416 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.416 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.416 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.416 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-11T23:49:25.416 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.416 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.416 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.416 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-11T23:49:25.416 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-11T23:49:25.417 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.418 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.418 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:25.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.418 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-11T23:49:25.418 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 2, 2024-09-11T23:49:25.418 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 6, 2024-09-11T23:49:25.418 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 4, 2024-09-11T23:49:25.418 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 5, 2024-09-11T23:49:25.418 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-11T23:49:25.418 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 3801, 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:49:25.419 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:49:25.420 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-11T23:49:25.421 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 189361, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 50, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:49:25.422 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 7936, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 175, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 8512, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 13, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.423 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 2414, 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 21, 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 3, 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 2414, 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2414, 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 31.090425068, 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.012879215 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-11T23:49:25.424 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2414, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2415 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 2414, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 2414, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 1569, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 843, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-09-11T23:49:25.425 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 1, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-11T23:49:25.426 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-11T23:49:25.427 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-11T23:49:25.428 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-11T23:49:25.428 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-11T23:49:25.428 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-11T23:49:25.428 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-11T23:49:25.428 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-11T23:49:25.428 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-11T23:49:25.428 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-11T23:49:25.428 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 98, 2024-09-11T23:49:25.428 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-11T23:49:25.428 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 19, 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 5, 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 5, 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:49:25.429 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-09-11T23:49:25.430 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-11T23:49:25.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.431 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:49:25.431 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:25.431 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T23:49:25.431 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:25.431 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 2705, 2024-09-11T23:49:25.431 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1966320, 2024-09-11T23:49:25.431 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:25.431 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 2705, 2024-09-11T23:49:25.431 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2705, 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1966320, 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:25.432 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 2414, 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 3430, 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2414, 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 3430, 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:25.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 2414, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 2414, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2414, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 2414, 2024-09-11T23:49:25.434 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:25.435 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55d7148a4b40": { 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T23:49:25.436 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T23:49:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:25.438 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T23:49:25.438 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-11T23:49:26.253 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-11T23:49:26.253 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-11T23:49:26.253 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-11T23:49:26.254 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:49:26.254 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:49:26.254 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-11T23:49:26.265 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-11T23:49:26.265 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:49:26.265 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:49:26.287 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T23:49:26.287 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:49:26.347 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T23:49:26.645 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:27.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:26 smithi133 ceph-mon[31430]: pgmap v398: 97 pgs: 97 active+clean; 50 GiB data, 166 GiB used, 907 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:49:27.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:26 smithi178 ceph-mon[35882]: pgmap v398: 97 pgs: 97 active+clean; 50 GiB data, 166 GiB used, 907 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:49:27.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.101+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/1202377039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f0100a80 msgr2=0x7fc4f0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.101+0000 7fc4f5bf1640 1 --2- 172.21.15.133:0/1202377039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f0100a80 0x7fc4f0100ee0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009930 tx=0x7fc4d802a420 comp rx=0 tx=0).stop 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.101+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/1202377039 shutdown_connections 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.101+0000 7fc4f5bf1640 1 --2- 172.21.15.133:0/1202377039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f0101420 0x7fc4f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.101+0000 7fc4f5bf1640 1 --2- 172.21.15.133:0/1202377039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f0100a80 0x7fc4f0100ee0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.101+0000 7fc4f5bf1640 1 --2- 172.21.15.133:0/1202377039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f0106a80 0x7fc4f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.101+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/1202377039 >> 172.21.15.133:0/1202377039 conn(0x7fc4f00fc7a0 msgr2=0x7fc4f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.102+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/1202377039 shutdown_connections 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.102+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/1202377039 wait complete. 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.102+0000 7fc4f5bf1640 1 Processor -- start 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.102+0000 7fc4f5bf1640 1 -- start start 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.102+0000 7fc4f5bf1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f0100a80 0x7fc4f0197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.102+0000 7fc4f5bf1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f0101420 0x7fc4f0197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.102+0000 7fc4f5bf1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f0106a80 0x7fc4f019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.102+0000 7fc4f5bf1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0110e80 con 0x7fc4f0106a80 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.102+0000 7fc4f5bf1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0110d00 con 0x7fc4f0101420 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.102+0000 7fc4f5bf1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0111000 con 0x7fc4f0100a80 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4effff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f0106a80 0x7fc4f019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4effff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f0106a80 0x7fc4f019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58584/0 (socket says 172.21.15.133:58584) 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4effff640 1 -- 172.21.15.133:0/901670623 learned_addr learned my addr 172.21.15.133:0/901670623 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4ef7fe640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f0100a80 0x7fc4f0197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4e7fff640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f0101420 0x7fc4f0197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4effff640 1 -- 172.21.15.133:0/901670623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f0100a80 msgr2=0x7fc4f0197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4effff640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f0100a80 0x7fc4f0197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4effff640 1 -- 172.21.15.133:0/901670623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f0101420 msgr2=0x7fc4f0197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:27.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4effff640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f0101420 0x7fc4f0197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4effff640 1 -- 172.21.15.133:0/901670623 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f019c2f0 con 0x7fc4f0106a80 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4e7fff640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f0101420 0x7fc4f0197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4effff640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f0106a80 0x7fc4f019bc10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800eca0 tx=0x7fc4e800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4ed7fa640 1 -- 172.21.15.133:0/901670623 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8004b20 con 0x7fc4f0106a80 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4ed7fa640 1 -- 172.21.15.133:0/901670623 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e8004cc0 con 0x7fc4f0106a80 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/901670623 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f019c5e0 con 0x7fc4f0106a80 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4ed7fa640 1 -- 172.21.15.133:0/901670623 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8017650 con 0x7fc4f0106a80 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.103+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/901670623 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f006a790 con 0x7fc4f0106a80 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.104+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/901670623 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f01021c0 con 0x7fc4f0106a80 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.105+0000 7fc4ed7fa640 1 -- 172.21.15.133:0/901670623 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc4e8005020 con 0x7fc4f0106a80 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.106+0000 7fc4ed7fa640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc4dc0a2cf0 0x7fc4dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.106+0000 7fc4ef7fe640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc4dc0a2cf0 0x7fc4dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.106+0000 7fc4ed7fa640 1 -- 172.21.15.133:0/901670623 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fc4e80d0f20 con 0x7fc4f0106a80 2024-09-11T23:49:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.106+0000 7fc4ef7fe640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc4dc0a2cf0 0x7fc4dc0a51b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009ae0 tx=0x7fc4e0005f90 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:27.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.110+0000 7fc4ed7fa640 1 -- 172.21.15.133:0/901670623 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4e809db80 con 0x7fc4f0106a80 2024-09-11T23:49:27.199 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:26 smithi138 ceph-mon[35142]: pgmap v398: 97 pgs: 97 active+clean; 50 GiB data, 166 GiB used, 907 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:49:27.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.288+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/901670623 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc4f006b1b0 con 0x7fc4f0106a80 2024-09-11T23:49:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.288+0000 7fc4ed7fa640 1 -- 172.21.15.133:0/901670623 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v98) ==== 81+0+958 (secure 0 0 0) 0x7fc4e809d480 con 0x7fc4f0106a80 2024-09-11T23:49:27.288 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/2855546700 2024-09-12T23:27:45.045954+0000 2024-09-11T23:49:27.288 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/3324898762 2024-09-12T23:36:47.051817+0000 2024-09-11T23:49:27.289 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/67600571 2024-09-12T23:27:45.045954+0000 2024-09-11T23:49:27.289 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/3596945316 2024-09-12T23:36:47.051817+0000 2024-09-11T23:49:27.289 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/3318533253 2024-09-12T23:27:45.045954+0000 2024-09-11T23:49:27.289 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/2152407978 2024-09-12T23:28:07.759499+0000 2024-09-11T23:49:27.289 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1011550148 2024-09-12T23:27:45.045954+0000 2024-09-11T23:49:27.289 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/2629770686 2024-09-12T23:28:07.759499+0000 2024-09-11T23:49:27.289 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/3324898762 2024-09-12T23:36:47.051817+0000 2024-09-11T23:49:27.289 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1658494809 2024-09-12T23:36:47.051817+0000 2024-09-11T23:49:27.289 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/294728041 2024-09-12T23:28:07.759499+0000 2024-09-11T23:49:27.289 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/2152407978 2024-09-12T23:28:07.759499+0000 2024-09-11T23:49:27.290 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/3985386459 2024-09-12T23:28:07.759499+0000 2024-09-11T23:49:27.290 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/2855546700 2024-09-12T23:27:45.045954+0000 2024-09-11T23:49:27.290 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/3947064111 2024-09-12T23:36:47.051817+0000 2024-09-11T23:49:27.290 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/802317131 2024-09-12T23:36:47.051817+0000 2024-09-11T23:49:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:listed 16 entries 2024-09-11T23:49:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.290+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/901670623 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc4dc0a2cf0 msgr2=0x7fc4dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.290+0000 7fc4f5bf1640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc4dc0a2cf0 0x7fc4dc0a51b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009ae0 tx=0x7fc4e0005f90 comp rx=0 tx=0).stop 2024-09-11T23:49:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.290+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/901670623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f0106a80 msgr2=0x7fc4f019bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.290+0000 7fc4f5bf1640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f0106a80 0x7fc4f019bc10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800eca0 tx=0x7fc4e800b690 comp rx=0 tx=0).stop 2024-09-11T23:49:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.291+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/901670623 shutdown_connections 2024-09-11T23:49:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.291+0000 7fc4f5bf1640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fc4dc0a2cf0 0x7fc4dc0a51b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.291+0000 7fc4f5bf1640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4f0106a80 0x7fc4f019bc10 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.291+0000 7fc4f5bf1640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4f0101420 0x7fc4f0197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.291+0000 7fc4f5bf1640 1 --2- 172.21.15.133:0/901670623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4f0100a80 0x7fc4f0197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:27.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.291+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/901670623 >> 172.21.15.133:0/901670623 conn(0x7fc4f00fc7a0 msgr2=0x7fc4f00fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:27.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.291+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/901670623 shutdown_connections 2024-09-11T23:49:27.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:27.291+0000 7fc4f5bf1640 1 -- 172.21.15.133:0/901670623 wait complete. 2024-09-11T23:49:28.019 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T23:49:28.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:49:28.019 DEBUG:teuthology.orchestra.run.smithi133:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T23:49:28.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:27 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/901670623' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:49:28.041 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:27 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/901670623' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:49:28.060 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T23:49:28.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:49:28.060 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T23:49:28.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:49:28.147 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-09-11T23:49:28.172 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T23:49:28.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:49:28.173 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T23:49:28.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:27 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/901670623' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:49:28.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:49:28.239 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-09-11T23:49:28.335 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T23:49:28.347 INFO:tasks.cephadm:Teardown begin 2024-09-11T23:49:28.347 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T23:49:28.348 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:49:28.596 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:28.995 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:28 smithi133 ceph-mon[31430]: pgmap v399: 97 pgs: 97 active+clean; 50 GiB data, 162 GiB used, 911 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:49:29.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:28 smithi178 ceph-mon[35882]: pgmap v399: 97 pgs: 97 active+clean; 50 GiB data, 162 GiB used, 911 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:49:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.077+0000 7fea643b2640 1 -- 172.21.15.133:0/2011745634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c1027a0 msgr2=0x7fea5c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.077+0000 7fea643b2640 1 --2- 172.21.15.133:0/2011745634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c1027a0 0x7fea5c1067f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fea58009900 tx=0x7fea5802a410 comp rx=0 tx=0).stop 2024-09-11T23:49:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.077+0000 7fea643b2640 1 -- 172.21.15.133:0/2011745634 shutdown_connections 2024-09-11T23:49:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.077+0000 7fea643b2640 1 --2- 172.21.15.133:0/2011745634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c106f20 0x7fea5c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.077+0000 7fea643b2640 1 --2- 172.21.15.133:0/2011745634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c1027a0 0x7fea5c1067f0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.077+0000 7fea643b2640 1 --2- 172.21.15.133:0/2011745634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea5c101df0 0x7fea5c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.077+0000 7fea643b2640 1 -- 172.21.15.133:0/2011745634 >> 172.21.15.133:0/2011745634 conn(0x7fea5c0fd8f0 msgr2=0x7fea5c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.077+0000 7fea643b2640 1 -- 172.21.15.133:0/2011745634 shutdown_connections 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.077+0000 7fea643b2640 1 -- 172.21.15.133:0/2011745634 wait complete. 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.077+0000 7fea643b2640 1 Processor -- start 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea643b2640 1 -- start start 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea643b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c101df0 0x7fea5c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea643b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea5c1027a0 0x7fea5c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea643b2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c106f20 0x7fea5c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea643b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c111fa0 con 0x7fea5c101df0 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea643b2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c111e20 con 0x7fea5c1027a0 2024-09-11T23:49:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea643b2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c112120 con 0x7fea5c106f20 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea6214e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c101df0 0x7fea5c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea6214e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c101df0 0x7fea5c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58598/0 (socket says 172.21.15.133:58598) 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea6214e640 1 -- 172.21.15.133:0/1300450523 learned_addr learned my addr 172.21.15.133:0/1300450523 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea6194d640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea5c1027a0 0x7fea5c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea6294f640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c106f20 0x7fea5c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea6214e640 1 -- 172.21.15.133:0/1300450523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c106f20 msgr2=0x7fea5c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea6214e640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c106f20 0x7fea5c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea6214e640 1 -- 172.21.15.133:0/1300450523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea5c1027a0 msgr2=0x7fea5c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea6214e640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea5c1027a0 0x7fea5c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea6214e640 1 -- 172.21.15.133:0/1300450523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c19d480 con 0x7fea5c101df0 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.078+0000 7fea6214e640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c101df0 0x7fea5c1984d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fea5000b4a0 tx=0x7fea5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.079+0000 7fea4f7fe640 1 -- 172.21.15.133:0/1300450523 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50014070 con 0x7fea5c101df0 2024-09-11T23:49:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.079+0000 7fea643b2640 1 -- 172.21.15.133:0/1300450523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c19d770 con 0x7fea5c101df0 2024-09-11T23:49:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.079+0000 7fea4f7fe640 1 -- 172.21.15.133:0/1300450523 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea50010af0 con 0x7fea5c101df0 2024-09-11T23:49:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.079+0000 7fea4f7fe640 1 -- 172.21.15.133:0/1300450523 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50017d10 con 0x7fea5c101df0 2024-09-11T23:49:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.079+0000 7fea643b2640 1 -- 172.21.15.133:0/1300450523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c1a5050 con 0x7fea5c101df0 2024-09-11T23:49:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.080+0000 7fea643b2640 1 -- 172.21.15.133:0/1300450523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5c1021d0 con 0x7fea5c101df0 2024-09-11T23:49:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.082+0000 7fea4f7fe640 1 -- 172.21.15.133:0/1300450523 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fea50017070 con 0x7fea5c101df0 2024-09-11T23:49:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.082+0000 7fea4f7fe640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea480a2cf0 0x7fea480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.082+0000 7fea6194d640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea480a2cf0 0x7fea480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.082+0000 7fea4f7fe640 1 -- 172.21.15.133:0/1300450523 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fea500cea60 con 0x7fea5c101df0 2024-09-11T23:49:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.083+0000 7fea6194d640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea480a2cf0 0x7fea480a51b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fea58000c00 tx=0x7fea580058d0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.086+0000 7fea4f7fe640 1 -- 172.21.15.133:0/1300450523 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea5009b7d0 con 0x7fea5c101df0 2024-09-11T23:49:29.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:28 smithi138 ceph-mon[35142]: pgmap v399: 97 pgs: 97 active+clean; 50 GiB data, 162 GiB used, 911 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:49:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.288+0000 7fea643b2640 1 -- 172.21.15.133:0/1300450523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea5c199830 con 0x7fea5c101df0 2024-09-11T23:49:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.289+0000 7fea4f7fe640 1 -- 172.21.15.133:0/1300450523 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11994 (secure 0 0 0) 0x7fea5009b0d0 con 0x7fea5c101df0 2024-09-11T23:49:29.289 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:49:29.289 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":32,"btime":"2024-09-11T23:48:01:740980+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:48:01.740959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27616":{"gid":27616,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2029124747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2029124747},{"type":"v1","addr":"172.21.15.178:6835","nonce":2029124747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:49:29.290 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 32 2024-09-11T23:49:29.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.292+0000 7fea643b2640 1 -- 172.21.15.133:0/1300450523 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea480a2cf0 msgr2=0x7fea480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:29.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.292+0000 7fea643b2640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea480a2cf0 0x7fea480a51b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fea58000c00 tx=0x7fea580058d0 comp rx=0 tx=0).stop 2024-09-11T23:49:29.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.292+0000 7fea643b2640 1 -- 172.21.15.133:0/1300450523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c101df0 msgr2=0x7fea5c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:29.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.292+0000 7fea643b2640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c101df0 0x7fea5c1984d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fea5000b4a0 tx=0x7fea5000b960 comp rx=0 tx=0).stop 2024-09-11T23:49:29.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.293+0000 7fea643b2640 1 -- 172.21.15.133:0/1300450523 shutdown_connections 2024-09-11T23:49:29.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.293+0000 7fea643b2640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fea480a2cf0 0x7fea480a51b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:29.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.293+0000 7fea643b2640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c106f20 0x7fea5c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:29.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.293+0000 7fea643b2640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea5c1027a0 0x7fea5c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:29.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.293+0000 7fea643b2640 1 --2- 172.21.15.133:0/1300450523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea5c101df0 0x7fea5c1984d0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:29.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.293+0000 7fea643b2640 1 -- 172.21.15.133:0/1300450523 >> 172.21.15.133:0/1300450523 conn(0x7fea5c0fd8f0 msgr2=0x7fea5c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:29.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.293+0000 7fea643b2640 1 -- 172.21.15.133:0/1300450523 shutdown_connections 2024-09-11T23:49:29.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:29.293+0000 7fea643b2640 1 -- 172.21.15.133:0/1300450523 wait complete. 2024-09-11T23:49:29.925 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:29 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1300450523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:29.955 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:49:30.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:29 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1300450523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:30.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:29 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1300450523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:30.204 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:30.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.650+0000 7f0728d22640 1 -- 172.21.15.133:0/2456884004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0724106f40 msgr2=0x7f072410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:30.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.650+0000 7f0728d22640 1 --2- 172.21.15.133:0/2456884004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0724106f40 0x7f072410eac0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f071c009900 tx=0x7f071c02a410 comp rx=0 tx=0).stop 2024-09-11T23:49:30.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.650+0000 7f0728d22640 1 -- 172.21.15.133:0/2456884004 shutdown_connections 2024-09-11T23:49:30.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.650+0000 7f0728d22640 1 --2- 172.21.15.133:0/2456884004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0724106f40 0x7f072410eac0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:30.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.650+0000 7f0728d22640 1 --2- 172.21.15.133:0/2456884004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07241027c0 0x7f0724106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:30.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.650+0000 7f0728d22640 1 --2- 172.21.15.133:0/2456884004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0724101e10 0x7f07241021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:30.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.650+0000 7f0728d22640 1 -- 172.21.15.133:0/2456884004 >> 172.21.15.133:0/2456884004 conn(0x7f07240fd8d0 msgr2=0x7f07240ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:30.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.650+0000 7f0728d22640 1 -- 172.21.15.133:0/2456884004 shutdown_connections 2024-09-11T23:49:30.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.650+0000 7f0728d22640 1 -- 172.21.15.133:0/2456884004 wait complete. 2024-09-11T23:49:30.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.650+0000 7f0728d22640 1 Processor -- start 2024-09-11T23:49:30.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0728d22640 1 -- start start 2024-09-11T23:49:30.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0728d22640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0724101e10 0x7f07240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:30.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0728d22640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07241027c0 0x7f0724074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:30.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0728d22640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0724106f40 0x7f0724074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0728d22640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0724111fc0 con 0x7f07241027c0 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0728d22640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0724111e40 con 0x7f0724106f40 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0728d22640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0724112140 con 0x7f0724101e10 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0721d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07241027c0 0x7f0724074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0721d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07241027c0 0x7f0724074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58618/0 (socket says 172.21.15.133:58618) 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0721d9b640 1 -- 172.21.15.133:0/2467471703 learned_addr learned my addr 172.21.15.133:0/2467471703 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0722d9d640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0724106f40 0x7f0724074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f072259c640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0724101e10 0x7f07240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0721d9b640 1 -- 172.21.15.133:0/2467471703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0724101e10 msgr2=0x7f07240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0721d9b640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0724101e10 0x7f07240744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0721d9b640 1 -- 172.21.15.133:0/2467471703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0724106f40 msgr2=0x7f0724074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:30.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0721d9b640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0724106f40 0x7f0724074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0721d9b640 1 -- 172.21.15.133:0/2467471703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07241a6900 con 0x7f07241027c0 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0722d9d640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0724106f40 0x7f0724074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.651+0000 7f0721d9b640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07241027c0 0x7f0724074a30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f070c00d950 tx=0x7f070c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.652+0000 7f071b7fe640 1 -- 172.21.15.133:0/2467471703 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c017070 con 0x7f07241027c0 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.652+0000 7f0728d22640 1 -- 172.21.15.133:0/2467471703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07241a6bf0 con 0x7f07241027c0 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.652+0000 7f071b7fe640 1 -- 172.21.15.133:0/2467471703 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070c009c30 con 0x7f07241027c0 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.652+0000 7f071b7fe640 1 -- 172.21.15.133:0/2467471703 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c00f6b0 con 0x7f07241027c0 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.652+0000 7f0728d22640 1 -- 172.21.15.133:0/2467471703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07241a7180 con 0x7f07241027c0 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.653+0000 7f0728d22640 1 -- 172.21.15.133:0/2467471703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07241021f0 con 0x7f07241027c0 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.654+0000 7f071b7fe640 1 -- 172.21.15.133:0/2467471703 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f070c0109d0 con 0x7f07241027c0 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.655+0000 7f071b7fe640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07100a2cf0 0x7f07100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.655+0000 7f072259c640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07100a2cf0 0x7f07100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:30.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.655+0000 7f071b7fe640 1 -- 172.21.15.133:0/2467471703 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f070c0cf110 con 0x7f07241027c0 2024-09-11T23:49:30.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.655+0000 7f072259c640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07100a2cf0 0x7f07100a51b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0714009f20 tx=0x7f07140094f0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:30.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.659+0000 7f071b7fe640 1 -- 172.21.15.133:0/2467471703 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f070c009de0 con 0x7f07241027c0 2024-09-11T23:49:30.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.858+0000 7f0728d22640 1 -- 172.21.15.133:0/2467471703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f072410a120 con 0x7f07241027c0 2024-09-11T23:49:30.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.859+0000 7f071b7fe640 1 -- 172.21.15.133:0/2467471703 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11994 (secure 0 0 0) 0x7f070c09bd70 con 0x7f07241027c0 2024-09-11T23:49:30.859 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:49:30.859 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":32,"btime":"2024-09-11T23:48:01:740980+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1603982530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1603982530},{"type":"v1","addr":"172.21.15.133:6835","nonce":1603982530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3098733497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3098733497},{"type":"v1","addr":"172.21.15.133:6837","nonce":3098733497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:34:53.462890+0000","modified":"2024-09-11T23:48:01.740959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24481,"mds_1":24472,"mds_2":24464,"mds_3":24419,"mds_4":24401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/570951791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":570951791},{"type":"v1","addr":"172.21.15.133:6839","nonce":570951791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6841/1294883246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1294883246},{"type":"v1","addr":"172.21.15.133:6841","nonce":1294883246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.138:6833/1969946111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1969946111},{"type":"v1","addr":"172.21.15.138:6833","nonce":1969946111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.138:6835/137089847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":137089847},{"type":"v1","addr":"172.21.15.138:6835","nonce":137089847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/3569562128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3569562128},{"type":"v1","addr":"172.21.15.178:6833","nonce":3569562128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/1115908278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":1115908278},{"type":"v1","addr":"172.21.15.138:6837","nonce":1115908278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2407895128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2407895128},{"type":"v1","addr":"172.21.15.138:6839","nonce":2407895128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6837/1040997001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1040997001},{"type":"v1","addr":"172.21.15.178:6837","nonce":1040997001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.178:6839/3794667907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":3794667907},{"type":"v1","addr":"172.21.15.178:6839","nonce":3794667907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27616":{"gid":27616,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2029124747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2029124747},{"type":"v1","addr":"172.21.15.178:6835","nonce":2029124747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24481,"qdb_cluster":[24401,24419,24464,24472,24481]},"id":1}]} 2024-09-11T23:49:30.860 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 32 2024-09-11T23:49:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 -- 172.21.15.133:0/2467471703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07100a2cf0 msgr2=0x7f07100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07100a2cf0 0x7f07100a51b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0714009f20 tx=0x7f07140094f0 comp rx=0 tx=0).stop 2024-09-11T23:49:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 -- 172.21.15.133:0/2467471703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07241027c0 msgr2=0x7f0724074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07241027c0 0x7f0724074a30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f070c00d950 tx=0x7f070c00de10 comp rx=0 tx=0).stop 2024-09-11T23:49:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 -- 172.21.15.133:0/2467471703 shutdown_connections 2024-09-11T23:49:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f07100a2cf0 0x7f07100a51b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0724106f40 0x7f0724074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07241027c0 0x7f0724074a30 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 --2- 172.21.15.133:0/2467471703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0724101e10 0x7f07240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 -- 172.21.15.133:0/2467471703 >> 172.21.15.133:0/2467471703 conn(0x7f07240fd8d0 msgr2=0x7f07240feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 -- 172.21.15.133:0/2467471703 shutdown_connections 2024-09-11T23:49:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:30.861+0000 7f0728d22640 1 -- 172.21.15.133:0/2467471703 wait complete. 2024-09-11T23:49:30.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:30 smithi133 ceph-mon[31430]: pgmap v400: 97 pgs: 97 active+clean; 50 GiB data, 159 GiB used, 914 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 477 B/s wr, 9 op/s 2024-09-11T23:49:31.040 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:30 smithi178 ceph-mon[35882]: pgmap v400: 97 pgs: 97 active+clean; 50 GiB data, 159 GiB used, 914 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 477 B/s wr, 9 op/s 2024-09-11T23:49:31.197 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:30 smithi138 ceph-mon[35142]: pgmap v400: 97 pgs: 97 active+clean; 50 GiB data, 159 GiB used, 914 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 477 B/s wr, 9 op/s 2024-09-11T23:49:31.685 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:49:31.933 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:31.959 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:31 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2467471703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:32.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:31 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2467471703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:32.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:31 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2467471703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:49:32.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.390+0000 7f2da1dc9640 1 -- 172.21.15.133:0/541000943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c105590 msgr2=0x7f2d9c10d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:32.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.390+0000 7f2da1dc9640 1 --2- 172.21.15.133:0/541000943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c105590 0x7f2d9c10d110 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009ca0 tx=0x7f2d9402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.391+0000 7f2da1dc9640 1 -- 172.21.15.133:0/541000943 shutdown_connections 2024-09-11T23:49:32.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.391+0000 7f2da1dc9640 1 --2- 172.21.15.133:0/541000943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c105590 0x7f2d9c10d110 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.391+0000 7f2da1dc9640 1 --2- 172.21.15.133:0/541000943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d9c100e10 0x7f2d9c104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.391+0000 7f2da1dc9640 1 --2- 172.21.15.133:0/541000943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c100460 0x7f2d9c100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.391+0000 7f2da1dc9640 1 -- 172.21.15.133:0/541000943 >> 172.21.15.133:0/541000943 conn(0x7f2d9c0fbee0 msgr2=0x7f2d9c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:32.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.391+0000 7f2da1dc9640 1 -- 172.21.15.133:0/541000943 shutdown_connections 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.391+0000 7f2da1dc9640 1 -- 172.21.15.133:0/541000943 wait complete. 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.391+0000 7f2da1dc9640 1 Processor -- start 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.391+0000 7f2da1dc9640 1 -- start start 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.391+0000 7f2da1dc9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c100460 0x7f2d9c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2da1dc9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d9c100e10 0x7f2d9c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2da1dc9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c105590 0x7f2d9c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2da1dc9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c10fd70 con 0x7f2d9c100460 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2da1dc9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c10fbf0 con 0x7f2d9c105590 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2da1dc9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c10fef0 con 0x7f2d9c100e10 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d9b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c100460 0x7f2d9c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d9b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c100460 0x7f2d9c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38588/0 (socket says 172.21.15.133:38588) 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/2239576149 learned_addr learned my addr 172.21.15.133:0/2239576149 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d9affd640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d9c100e10 0x7f2d9c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d9bfff640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c105590 0x7f2d9c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/2239576149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d9c100e10 msgr2=0x7f2d9c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d9b7fe640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d9c100e10 0x7f2d9c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/2239576149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c105590 msgr2=0x7f2d9c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d9b7fe640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c105590 0x7f2d9c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d9b7fe640 1 -- 172.21.15.133:0/2239576149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9c19d350 con 0x7f2d9c100460 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d9b7fe640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c100460 0x7f2d9c1983a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00ea20 tx=0x7f2d8c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d98ff9640 1 -- 172.21.15.133:0/2239576149 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c00cd40 con 0x7f2d9c100460 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d98ff9640 1 -- 172.21.15.133:0/2239576149 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d8c0168f0 con 0x7f2d9c100460 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2da1dc9640 1 -- 172.21.15.133:0/2239576149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9c19d640 con 0x7f2d9c100460 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2d98ff9640 1 -- 172.21.15.133:0/2239576149 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c00b920 con 0x7f2d9c100460 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.392+0000 7f2da1dc9640 1 -- 172.21.15.133:0/2239576149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9c1a4f20 con 0x7f2d9c100460 2024-09-11T23:49:32.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.394+0000 7f2da1dc9640 1 -- 172.21.15.133:0/2239576149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9c100840 con 0x7f2d9c100460 2024-09-11T23:49:32.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.394+0000 7f2d98ff9640 1 -- 172.21.15.133:0/2239576149 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2d8c016aa0 con 0x7f2d9c100460 2024-09-11T23:49:32.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.395+0000 7f2d98ff9640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d840a2cf0 0x7f2d840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:32.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.395+0000 7f2d9affd640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d840a2cf0 0x7f2d840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:32.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.395+0000 7f2d98ff9640 1 -- 172.21.15.133:0/2239576149 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f2d8c0cf260 con 0x7f2d9c100460 2024-09-11T23:49:32.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.396+0000 7f2d9affd640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d840a2cf0 0x7f2d840a51b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0fc300 tx=0x7f2d9000b390 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:32.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.399+0000 7f2d98ff9640 1 -- 172.21.15.133:0/2239576149 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d8c09b870 con 0x7f2d9c100460 2024-09-11T23:49:32.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.567+0000 7f2da1dc9640 1 -- 172.21.15.133:0/2239576149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d9c108770 con 0x7f2d9c100460 2024-09-11T23:49:32.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.569+0000 7f2d98ff9640 1 -- 172.21.15.133:0/2239576149 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23150 (secure 0 0 0) 0x7f2d8c09fb70 con 0x7f2d9c100460 2024-09-11T23:49:32.569 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T23:49:32.569 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":98,"fsid":"42321b88-7095-11ef-bcea-c7b262605968","created":"2024-09-11T23:27:21.704064+0000","modified":"2024-09-11T23:49:18.983426+0000","last_up_change":"2024-09-11T23:34:04.043979+0000","last_in_change":"2024-09-11T23:33:40.241309+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:30:22.061012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:34:49.162653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:34:51.723611+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba73339-5c3f-4621-85af-3c3addd0583c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6803","nonce":901342877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6805","nonce":901342877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6809","nonce":901342877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":901342877},{"type":"v1","addr":"172.21.15.133:6807","nonce":901342877}]},"public_addr":"172.21.15.133:6803/901342877","cluster_addr":"172.21.15.133:6805/901342877","heartbeat_back_addr":"172.21.15.133:6809/901342877","heartbeat_front_addr":"172.21.15.133:6807/901342877","state":["exists","up"]},{"osd":1,"uuid":"19075246-d736-48fd-b96d-68395df7eb8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6801","nonce":2975334926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6803","nonce":2975334926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6807","nonce":2975334926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2975334926},{"type":"v1","addr":"172.21.15.138:6805","nonce":2975334926}]},"public_addr":"172.21.15.138:6801/2975334926","cluster_addr":"172.21.15.138:6803/2975334926","heartbeat_back_addr":"172.21.15.138:6807/2975334926","heartbeat_front_addr":"172.21.15.138:6805/2975334926","state":["exists","up"]},{"osd":2,"uuid":"3ae0f14e-2263-4fd2-9848-7b23b8e2638c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6801","nonce":3161638816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6803","nonce":3161638816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6807","nonce":3161638816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3161638816},{"type":"v1","addr":"172.21.15.178:6805","nonce":3161638816}]},"public_addr":"172.21.15.178:6801/3161638816","cluster_addr":"172.21.15.178:6803/3161638816","heartbeat_back_addr":"172.21.15.178:6807/3161638816","heartbeat_front_addr":"172.21.15.178:6805/3161638816","state":["exists","up"]},{"osd":3,"uuid":"e2392e1a-af87-4af1-a362-f17ce49444bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6811","nonce":1056435874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6813","nonce":1056435874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6817","nonce":1056435874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1056435874},{"type":"v1","addr":"172.21.15.133:6815","nonce":1056435874}]},"public_addr":"172.21.15.133:6811/1056435874","cluster_addr":"172.21.15.133:6813/1056435874","heartbeat_back_addr":"172.21.15.133:6817/1056435874","heartbeat_front_addr":"172.21.15.133:6815/1056435874","state":["exists","up"]},{"osd":4,"uuid":"d5414771-6d7e-4ea1-9b0a-50175189b9a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6809","nonce":952086734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6811","nonce":952086734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6815","nonce":952086734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":952086734},{"type":"v1","addr":"172.21.15.138:6813","nonce":952086734}]},"public_addr":"172.21.15.138:6809/952086734","cluster_addr":"172.21.15.138:6811/952086734","heartbeat_back_addr":"172.21.15.138:6815/952086734","heartbeat_front_addr":"172.21.15.138:6813/952086734","state":["exists","up"]},{"osd":5,"uuid":"8d1bd3f9-6661-405e-ad49-71be51a3034f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6809","nonce":1922026536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6811","nonce":1922026536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6815","nonce":1922026536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1922026536},{"type":"v1","addr":"172.21.15.178:6813","nonce":1922026536}]},"public_addr":"172.21.15.178:6809/1922026536","cluster_addr":"172.21.15.178:6811/1922026536","heartbeat_back_addr":"172.21.15.178:6815/1922026536","heartbeat_front_addr":"172.21.15.178:6813/1922026536","state":["exists","up"]},{"osd":6,"uuid":"92c712b8-c5b7-4097-b108-1df572e8f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6819","nonce":3287486573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6821","nonce":3287486573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6825","nonce":3287486573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3287486573},{"type":"v1","addr":"172.21.15.133:6823","nonce":3287486573}]},"public_addr":"172.21.15.133:6819/3287486573","cluster_addr":"172.21.15.133:6821/3287486573","heartbeat_back_addr":"172.21.15.133:6825/3287486573","heartbeat_front_addr":"172.21.15.133:6823/3287486573","state":["exists","up"]},{"osd":7,"uuid":"46533b61-85a2-4c27-8fb0-140a2e34f03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6817","nonce":663031854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6819","nonce":663031854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6823","nonce":663031854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":663031854},{"type":"v1","addr":"172.21.15.138:6821","nonce":663031854}]},"public_addr":"172.21.15.138:6817/663031854","cluster_addr":"172.21.15.138:6819/663031854","heartbeat_back_addr":"172.21.15.138:6823/663031854","heartbeat_front_addr":"172.21.15.138:6821/663031854","state":["exists","up"]},{"osd":8,"uuid":"9e16c39c-28cf-427c-b40a-73cffe154908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6817","nonce":3469671700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6819","nonce":3469671700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6823","nonce":3469671700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3469671700},{"type":"v1","addr":"172.21.15.178:6821","nonce":3469671700}]},"public_addr":"172.21.15.178:6817/3469671700","cluster_addr":"172.21.15.178:6819/3469671700","heartbeat_back_addr":"172.21.15.178:6823/3469671700","heartbeat_front_addr":"172.21.15.178:6821/3469671700","state":["exists","up"]},{"osd":9,"uuid":"1436e241-5439-49db-a07a-8b80f4499da2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6827","nonce":1405512574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6829","nonce":1405512574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6833","nonce":1405512574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1405512574},{"type":"v1","addr":"172.21.15.133:6831","nonce":1405512574}]},"public_addr":"172.21.15.133:6827/1405512574","cluster_addr":"172.21.15.133:6829/1405512574","heartbeat_back_addr":"172.21.15.133:6833/1405512574","heartbeat_front_addr":"172.21.15.133:6831/1405512574","state":["exists","up"]},{"osd":10,"uuid":"46e606d4-591e-415d-8aae-06d311761798","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6825","nonce":678369034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6827","nonce":678369034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6831","nonce":678369034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":678369034},{"type":"v1","addr":"172.21.15.138:6829","nonce":678369034}]},"public_addr":"172.21.15.138:6825/678369034","cluster_addr":"172.21.15.138:6827/678369034","heartbeat_back_addr":"172.21.15.138:6831/678369034","heartbeat_front_addr":"172.21.15.138:6829/678369034","state":["exists","up"]},{"osd":11,"uuid":"652ee9fd-003e-4bc5-80db-25e7880754d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6825","nonce":3213367962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6827","nonce":3213367962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6831","nonce":3213367962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3213367962},{"type":"v1","addr":"172.21.15.178:6829","nonce":3213367962}]},"public_addr":"172.21.15.178:6825/3213367962","cluster_addr":"172.21.15.178:6827/3213367962","heartbeat_back_addr":"172.21.15.178:6831/3213367962","heartbeat_front_addr":"172.21.15.178:6829/3213367962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:29.653738+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:29:54.928596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:19.185295+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:30:44.446772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:09.385239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:34.094188+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:31:59.652216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:23.434901+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:32:48.446950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:13.420686+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:33:37.252102+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:34:02.209172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6800/3324898762":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/67600571":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3596945316":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/3318533253":"2024-09-12T23:27:45.045954+0000","172.21.15.133:6801/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/1011550148":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/2629770686":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6801/3324898762":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/1658494809":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/294728041":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6800/2152407978":"2024-09-12T23:28:07.759499+0000","172.21.15.133:0/3985386459":"2024-09-12T23:28:07.759499+0000","172.21.15.133:6801/2855546700":"2024-09-12T23:27:45.045954+0000","172.21.15.133:0/3947064111":"2024-09-12T23:36:47.051817+0000","172.21.15.133:0/802317131":"2024-09-12T23:36:47.051817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 -- 172.21.15.133:0/2239576149 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d840a2cf0 msgr2=0x7f2d840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d840a2cf0 0x7f2d840a51b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0fc300 tx=0x7f2d9000b390 comp rx=0 tx=0).stop 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 -- 172.21.15.133:0/2239576149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c100460 msgr2=0x7f2d9c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c100460 0x7f2d9c1983a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00ea20 tx=0x7f2d8c00eee0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 -- 172.21.15.133:0/2239576149 shutdown_connections 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f2d840a2cf0 0x7f2d840a51b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c105590 0x7f2d9c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d9c100e10 0x7f2d9c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 --2- 172.21.15.133:0/2239576149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c100460 0x7f2d9c1983a0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 -- 172.21.15.133:0/2239576149 >> 172.21.15.133:0/2239576149 conn(0x7f2d9c0fbee0 msgr2=0x7f2d9c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 -- 172.21.15.133:0/2239576149 shutdown_connections 2024-09-11T23:49:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:32.571+0000 7f2da1dc9640 1 -- 172.21.15.133:0/2239576149 wait complete. 2024-09-11T23:49:32.923 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:32 smithi133 ceph-mon[31430]: pgmap v401: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-11T23:49:32.923 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:32 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/2239576149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:49:33.099 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:32 smithi138 ceph-mon[35142]: pgmap v401: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-11T23:49:33.099 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:32 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/2239576149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:49:33.245 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-11T23:49:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:32 smithi178 ceph-mon[35882]: pgmap v401: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-11T23:49:33.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:32 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/2239576149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:49:33.491 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.000+0000 7f09b1520640 1 -- 172.21.15.133:0/2286432521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09ac108300 msgr2=0x7f09ac10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.000+0000 7f09b1520640 1 --2- 172.21.15.133:0/2286432521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09ac108300 0x7f09ac10ec70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009930 tx=0x7f09a402a420 comp rx=0 tx=0).stop 2024-09-11T23:49:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.000+0000 7f09b1520640 1 -- 172.21.15.133:0/2286432521 shutdown_connections 2024-09-11T23:49:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.000+0000 7f09b1520640 1 --2- 172.21.15.133:0/2286432521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09ac108300 0x7f09ac10ec70 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.000+0000 7f09b1520640 1 --2- 172.21.15.133:0/2286432521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ac103780 0x7f09ac107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.000+0000 7f09b1520640 1 --2- 172.21.15.133:0/2286432521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac102dd0 0x7f09ac1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:34.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.000+0000 7f09b1520640 1 -- 172.21.15.133:0/2286432521 >> 172.21.15.133:0/2286432521 conn(0x7f09ac0fc7d0 msgr2=0x7f09ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.000+0000 7f09b1520640 1 -- 172.21.15.133:0/2286432521 shutdown_connections 2024-09-11T23:49:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.000+0000 7f09b1520640 1 -- 172.21.15.133:0/2286432521 wait complete. 2024-09-11T23:49:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.000+0000 7f09b1520640 1 Processor -- start 2024-09-11T23:49:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.000+0000 7f09b1520640 1 -- start start 2024-09-11T23:49:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09b1520640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac102dd0 0x7f09ac198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09b1520640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ac103780 0x7f09ac198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09b1520640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09ac108300 0x7f09ac19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09b1520640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1120d0 con 0x7f09ac108300 2024-09-11T23:49:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09b1520640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac111f50 con 0x7f09ac103780 2024-09-11T23:49:34.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09b1520640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac112250 con 0x7f09ac102dd0 2024-09-11T23:49:34.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09ac108300 0x7f09ac19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:34.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09ab7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09ac108300 0x7f09ac19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38608/0 (socket says 172.21.15.133:38608) 2024-09-11T23:49:34.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09ab7fe640 1 -- 172.21.15.133:0/3760250368 learned_addr learned my addr 172.21.15.133:0/3760250368 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:34.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09aaffd640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac102dd0 0x7f09ac198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:34.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09ab7fe640 1 -- 172.21.15.133:0/3760250368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac102dd0 msgr2=0x7f09ac198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:34.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09ab7fe640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac102dd0 0x7f09ac198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:34.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09ab7fe640 1 -- 172.21.15.133:0/3760250368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ac103780 msgr2=0x7f09ac198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:34.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09aa7fc640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ac103780 0x7f09ac198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:34.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09ab7fe640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ac103780 0x7f09ac198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:34.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09ab7fe640 1 -- 172.21.15.133:0/3760250368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac19d540 con 0x7f09ac108300 2024-09-11T23:49:34.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f09ab7fe640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09ac108300 0x7f09ac19ce60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f09a4005ab0 tx=0x7f09a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:34.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f0993fff640 1 -- 172.21.15.133:0/3760250368 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a403f030 con 0x7f09ac108300 2024-09-11T23:49:34.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.001+0000 7f0993fff640 1 -- 172.21.15.133:0/3760250368 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a4003ce0 con 0x7f09ac108300 2024-09-11T23:49:34.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.002+0000 7f09b1520640 1 -- 172.21.15.133:0/3760250368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ac19d7d0 con 0x7f09ac108300 2024-09-11T23:49:34.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.002+0000 7f0993fff640 1 -- 172.21.15.133:0/3760250368 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4035070 con 0x7f09ac108300 2024-09-11T23:49:34.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.002+0000 7f09b1520640 1 -- 172.21.15.133:0/3760250368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ac1a5110 con 0x7f09ac108300 2024-09-11T23:49:34.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.003+0000 7f09b1520640 1 -- 172.21.15.133:0/3760250368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ac06a7b0 con 0x7f09ac108300 2024-09-11T23:49:34.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.004+0000 7f0993fff640 1 -- 172.21.15.133:0/3760250368 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f09a4042030 con 0x7f09ac108300 2024-09-11T23:49:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.004+0000 7f0993fff640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09940a2cf0 0x7f09940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.004+0000 7f09aaffd640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09940a2cf0 0x7f09940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.004+0000 7f0993fff640 1 -- 172.21.15.133:0/3760250368 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f09a40ef960 con 0x7f09ac108300 2024-09-11T23:49:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.005+0000 7f09aaffd640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09940a2cf0 0x7f09940a51b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f099c005c40 tx=0x7f099c005bd0 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.008+0000 7f0993fff640 1 -- 172.21.15.133:0/3760250368 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09a40bc5c0 con 0x7f09ac108300 2024-09-11T23:49:34.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.212+0000 7f09b1520640 1 -- 172.21.15.133:0/3760250368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f09ac10a290 con 0x7f09ac108300 2024-09-11T23:49:34.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.216+0000 7f0993fff640 1 -- 172.21.15.133:0/3760250368 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) ==== 161+0+0 (secure 0 0 0) 0x7f09a40bbec0 con 0x7f09ac108300 2024-09-11T23:49:34.216 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T23:49:34.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.219+0000 7f09b1520640 1 -- 172.21.15.133:0/3760250368 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09940a2cf0 msgr2=0x7f09940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:34.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.219+0000 7f09b1520640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09940a2cf0 0x7f09940a51b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f099c005c40 tx=0x7f099c005bd0 comp rx=0 tx=0).stop 2024-09-11T23:49:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.219+0000 7f09b1520640 1 -- 172.21.15.133:0/3760250368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09ac108300 msgr2=0x7f09ac19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.219+0000 7f09b1520640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09ac108300 0x7f09ac19ce60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f09a4005ab0 tx=0x7f09a40042f0 comp rx=0 tx=0).stop 2024-09-11T23:49:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.219+0000 7f09b1520640 1 -- 172.21.15.133:0/3760250368 shutdown_connections 2024-09-11T23:49:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.219+0000 7f09b1520640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f09940a2cf0 0x7f09940a51b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.219+0000 7f09b1520640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09ac108300 0x7f09ac19ce60 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.219+0000 7f09b1520640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09ac103780 0x7f09ac198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.219+0000 7f09b1520640 1 --2- 172.21.15.133:0/3760250368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac102dd0 0x7f09ac198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.220+0000 7f09b1520640 1 -- 172.21.15.133:0/3760250368 >> 172.21.15.133:0/3760250368 conn(0x7f09ac0fc7d0 msgr2=0x7f09ac105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.220+0000 7f09b1520640 1 -- 172.21.15.133:0/3760250368 shutdown_connections 2024-09-11T23:49:34.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:34.220+0000 7f09b1520640 1 -- 172.21.15.133:0/3760250368 wait complete. 2024-09-11T23:49:34.389 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b[65233]: ignoring --setuser ceph since I am not root 2024-09-11T23:49:34.389 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b[65233]: ignoring --setgroup ceph since I am not root 2024-09-11T23:49:34.389 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-mds[65237]: -- 172.21.15.138:0/4174137714 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563351beb860 con 0x563351bc8800 2024-09-11T23:49:34.389 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b[65233]: starting mds.b at 2024-09-11T23:49:34.389 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e[64632]: ignoring --setuser ceph since I am not root 2024-09-11T23:49:34.389 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e[64632]: ignoring --setgroup ceph since I am not root 2024-09-11T23:49:34.389 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e[64632]: starting mds.e at 2024-09-11T23:49:34.390 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h[64932]: ignoring --setuser ceph since I am not root 2024-09-11T23:49:34.390 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h[64932]: ignoring --setgroup ceph since I am not root 2024-09-11T23:49:34.390 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-mds[64936]: -- 172.21.15.138:0/2013944884 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557745a8d860 con 0x557745a6b400 2024-09-11T23:49:34.390 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h[64932]: starting mds.h at 2024-09-11T23:49:34.390 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k[64322]: ignoring --setuser ceph since I am not root 2024-09-11T23:49:34.390 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k[64322]: ignoring --setgroup ceph since I am not root 2024-09-11T23:49:34.390 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-mds[64326]: -- 172.21.15.138:0/2649089112 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ce5f4cd860 con 0x55ce5f4ab000 2024-09-11T23:49:34.390 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k[64322]: starting mds.k at 2024-09-11T23:49:34.521 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a[31426]: 2024-09-11T23:49:34.213+0000 7f69e88f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:49:34.522 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a[70146]: ignoring --setuser ceph since I am not root 2024-09-11T23:49:34.522 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a[70146]: ignoring --setgroup ceph since I am not root 2024-09-11T23:49:34.522 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-mds[70162]: -- 172.21.15.133:0/4010042688 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5aa6eb860 con 0x55f5aa6c8800 2024-09-11T23:49:34.522 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a[70146]: starting mds.a at 2024-09-11T23:49:34.522 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g[69809]: ignoring --setuser ceph since I am not root 2024-09-11T23:49:34.522 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g[69809]: ignoring --setgroup ceph since I am not root 2024-09-11T23:49:34.522 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-mds[69818]: -- 172.21.15.133:0/2905562880 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5642760c7c20 con 0x5642760a4800 2024-09-11T23:49:34.522 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g[69809]: starting mds.g at 2024-09-11T23:49:34.540 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l[65172]: ignoring --setuser ceph since I am not root 2024-09-11T23:49:34.540 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l[65172]: ignoring --setgroup ceph since I am not root 2024-09-11T23:49:34.540 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l[65172]: starting mds.l at 2024-09-11T23:49:34.540 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c[65772]: ignoring --setuser ceph since I am not root 2024-09-11T23:49:34.540 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c[65772]: ignoring --setgroup ceph since I am not root 2024-09-11T23:49:34.541 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-mds[65776]: -- 172.21.15.178:0/943241258 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55567f4bd860 con 0x55567f49b000 2024-09-11T23:49:34.541 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c[65772]: starting mds.c at 2024-09-11T23:49:34.541 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c[65772]: 2024-09-11T23:49:34.262+0000 7fb196987640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-11T23:49:34.541 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: ignoring --setuser ceph since I am not root 2024-09-11T23:49:34.541 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: ignoring --setgroup ceph since I am not root 2024-09-11T23:49:34.541 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-mds[65475]: -- 172.21.15.178:0/2488614497 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e64909860 con 0x563e648e6800 2024-09-11T23:49:34.541 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: starting mds.f at 2024-09-11T23:49:34.541 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i[66069]: ignoring --setuser ceph since I am not root 2024-09-11T23:49:34.541 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i[66069]: ignoring --setgroup ceph since I am not root 2024-09-11T23:49:34.542 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-mds[66073]: -- 172.21.15.178:0/3873425543 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c70a577c20 con 0x55c70a554800 2024-09-11T23:49:34.542 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i[66069]: starting mds.i at 2024-09-11T23:49:34.917 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T23:49:35.164 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-mon[35142]: pgmap v402: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-11T23:49:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3760250368' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:49:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-mon[35142]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:49:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-mon[35142]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:49:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3760250368' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:49:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-mon[35142]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T23:49:35.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:34 smithi138 ceph-mon[35142]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T23:49:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-mon[31430]: pgmap v402: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-11T23:49:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3760250368' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:49:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-mon[31430]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:49:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-mon[31430]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:49:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3760250368' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:49:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-mon[31430]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T23:49:35.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:34 smithi133 ceph-mon[31430]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T23:49:35.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-mon[35882]: pgmap v402: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-11T23:49:35.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3760250368' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:49:35.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-mon[35882]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:49:35.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-mon[35882]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:49:35.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3760250368' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:49:35.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-mon[35882]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T23:49:35.291 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:34 smithi178 ceph-mon[35882]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T23:49:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.642+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/2346898571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d8106f10 msgr2=0x7fa4d810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.642+0000 7fa4dd0d0640 1 --2- 172.21.15.133:0/2346898571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d8106f10 0x7fa4d810ea90 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009ca0 tx=0x7fa4d002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:49:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.642+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/2346898571 shutdown_connections 2024-09-11T23:49:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.642+0000 7fa4dd0d0640 1 --2- 172.21.15.133:0/2346898571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d8106f10 0x7fa4d810ea90 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.642+0000 7fa4dd0d0640 1 --2- 172.21.15.133:0/2346898571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4d8102790 0x7fa4d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.642+0000 7fa4dd0d0640 1 --2- 172.21.15.133:0/2346898571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4d8101de0 0x7fa4d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.642+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/2346898571 >> 172.21.15.133:0/2346898571 conn(0x7fa4d80fd8c0 msgr2=0x7fa4d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.642+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/2346898571 shutdown_connections 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.642+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/2346898571 wait complete. 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.642+0000 7fa4dd0d0640 1 Processor -- start 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4dd0d0640 1 -- start start 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4dd0d0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4d8101de0 0x7fa4d81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4dd0d0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d8102790 0x7fa4d8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4dd0d0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4d8106f10 0x7fa4d819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4dd0d0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d8111f70 con 0x7fa4d8102790 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4dd0d0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d8111df0 con 0x7fa4d8101de0 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4dd0d0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81120f0 con 0x7fa4d8106f10 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4d659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d8102790 0x7fa4d8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4d659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d8102790 0x7fa4d8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38684/0 (socket says 172.21.15.133:38684) 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4d659c640 1 -- 172.21.15.133:0/3349936997 learned_addr learned my addr 172.21.15.133:0/3349936997 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4d759e640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4d8106f10 0x7fa4d819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:35.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4d6d9d640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4d8101de0 0x7fa4d81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4d659c640 1 -- 172.21.15.133:0/3349936997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4d8106f10 msgr2=0x7fa4d819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4d659c640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4d8106f10 0x7fa4d819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4d659c640 1 -- 172.21.15.133:0/3349936997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4d8101de0 msgr2=0x7fa4d81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4d659c640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4d8101de0 0x7fa4d81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4d659c640 1 -- 172.21.15.133:0/3349936997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d819d3a0 con 0x7fa4d8102790 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.643+0000 7fa4d659c640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d8102790 0x7fa4d8198930 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00e9f0 tx=0x7fa4cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.644+0000 7fa4bffff640 1 -- 172.21.15.133:0/3349936997 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc00cd80 con 0x7fa4d8102790 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.644+0000 7fa4bffff640 1 -- 172.21.15.133:0/3349936997 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4cc005500 con 0x7fa4d8102790 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.644+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/3349936997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d819d690 con 0x7fa4d8102790 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.644+0000 7fa4bffff640 1 -- 172.21.15.133:0/3349936997 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc010690 con 0x7fa4d8102790 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.644+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/3349936997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d81a4f70 con 0x7fa4d8102790 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.645+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/3349936997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d81021c0 con 0x7fa4d8102790 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.646+0000 7fa4bffff640 1 -- 172.21.15.133:0/3349936997 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa4cc0046c0 con 0x7fa4d8102790 2024-09-11T23:49:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.646+0000 7fa4bffff640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4c00a2cf0 0x7fa4c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:35.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.647+0000 7fa4d6d9d640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4c00a2cf0 0x7fa4c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:35.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.647+0000 7fa4bffff640 1 -- 172.21.15.133:0/3349936997 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8841+0+0 (secure 0 0 0) 0x7fa4cc0cf930 con 0x7fa4d8102790 2024-09-11T23:49:35.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.647+0000 7fa4d6d9d640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4c00a2cf0 0x7fa4c00a51b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8005d00 tx=0x7fa4c800b410 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:35.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.651+0000 7fa4bffff640 1 -- 172.21.15.133:0/3349936997 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4cc09c410 con 0x7fa4d8102790 2024-09-11T23:49:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:35.851+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/3349936997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa4d802d7c0 con 0x7fa4d8102790 2024-09-11T23:49:36.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.225+0000 7fa4bffff640 1 -- 172.21.15.133:0/3349936997 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7fa4cc09bd10 con 0x7fa4d8102790 2024-09-11T23:49:36.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.227+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/3349936997 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4c00a2cf0 msgr2=0x7fa4c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:36.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.227+0000 7fa4dd0d0640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4c00a2cf0 0x7fa4c00a51b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8005d00 tx=0x7fa4c800b410 comp rx=0 tx=0).stop 2024-09-11T23:49:36.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.227+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/3349936997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d8102790 msgr2=0x7fa4d8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:36.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.227+0000 7fa4dd0d0640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d8102790 0x7fa4d8198930 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00e9f0 tx=0x7fa4cc00edf0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.227+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/3349936997 shutdown_connections 2024-09-11T23:49:36.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.227+0000 7fa4dd0d0640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7fa4c00a2cf0 0x7fa4c00a51b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.227+0000 7fa4dd0d0640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4d8106f10 0x7fa4d819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.227+0000 7fa4dd0d0640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4d8102790 0x7fa4d8198930 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.227+0000 7fa4dd0d0640 1 --2- 172.21.15.133:0/3349936997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4d8101de0 0x7fa4d81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:36.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.227+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/3349936997 >> 172.21.15.133:0/3349936997 conn(0x7fa4d80fd8c0 msgr2=0x7fa4d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:36.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.228+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/3349936997 shutdown_connections 2024-09-11T23:49:36.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:36.228+0000 7fa4dd0d0640 1 -- 172.21.15.133:0/3349936997 wait complete. 2024-09-11T23:49:36.529 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6833/2367998528,v1:172.21.15.178:6835/2367998528] up:boot 2024-09-11T23:49:36.529 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6832/2724861116,v1:172.21.15.178:6834/2724861116] up:boot 2024-09-11T23:49:36.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.138:6832/2042840820,v1:172.21.15.138:6833/2042840820] up:boot 2024-09-11T23:49:36.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.133:6838/1532095560,v1:172.21.15.133:6840/1532095560] up:boot 2024-09-11T23:49:36.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.133:6839/2035808648,v1:172.21.15.133:6841/2035808648] up:boot 2024-09-11T23:49:36.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.138:6834/959169656,v1:172.21.15.138:6835/959169656] up:boot 2024-09-11T23:49:36.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6836/1110691978,v1:172.21.15.178:6837/1110691978] up:boot 2024-09-11T23:49:36.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.178:6838/2325524434,v1:172.21.15.178:6839/2325524434] up:boot 2024-09-11T23:49:36.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.138:6836/466019107,v1:172.21.15.138:6837/466019107] up:boot 2024-09-11T23:49:36.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: mds.? [v2:172.21.15.138:6838/2562034420,v1:172.21.15.138:6839/2562034420] up:boot 2024-09-11T23:49:36.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T23:49:36.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:49:36.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:49:36.531 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:49:36.531 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:49:36.531 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:49:36.531 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:49:36.531 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:49:36.531 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:49:36.531 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:49:36.531 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:49:36.531 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: pgmap v404: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-11T23:49:36.531 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:36 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3349936997' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:49:36.539 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6833/2367998528,v1:172.21.15.178:6835/2367998528] up:boot 2024-09-11T23:49:36.539 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6832/2724861116,v1:172.21.15.178:6834/2724861116] up:boot 2024-09-11T23:49:36.539 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.138:6832/2042840820,v1:172.21.15.138:6833/2042840820] up:boot 2024-09-11T23:49:36.539 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.133:6838/1532095560,v1:172.21.15.133:6840/1532095560] up:boot 2024-09-11T23:49:36.539 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.133:6839/2035808648,v1:172.21.15.133:6841/2035808648] up:boot 2024-09-11T23:49:36.539 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.138:6834/959169656,v1:172.21.15.138:6835/959169656] up:boot 2024-09-11T23:49:36.539 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6836/1110691978,v1:172.21.15.178:6837/1110691978] up:boot 2024-09-11T23:49:36.539 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.178:6838/2325524434,v1:172.21.15.178:6839/2325524434] up:boot 2024-09-11T23:49:36.539 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.138:6836/466019107,v1:172.21.15.138:6837/466019107] up:boot 2024-09-11T23:49:36.539 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: mds.? [v2:172.21.15.138:6838/2562034420,v1:172.21.15.138:6839/2562034420] up:boot 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:49:36.540 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: pgmap v404: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-11T23:49:36.541 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:36 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3349936997' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:49:36.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6833/2367998528,v1:172.21.15.178:6835/2367998528] up:boot 2024-09-11T23:49:36.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6832/2724861116,v1:172.21.15.178:6834/2724861116] up:boot 2024-09-11T23:49:36.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.138:6832/2042840820,v1:172.21.15.138:6833/2042840820] up:boot 2024-09-11T23:49:36.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.133:6838/1532095560,v1:172.21.15.133:6840/1532095560] up:boot 2024-09-11T23:49:36.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.133:6839/2035808648,v1:172.21.15.133:6841/2035808648] up:boot 2024-09-11T23:49:36.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.138:6834/959169656,v1:172.21.15.138:6835/959169656] up:boot 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6836/1110691978,v1:172.21.15.178:6837/1110691978] up:boot 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.178:6838/2325524434,v1:172.21.15.178:6839/2325524434] up:boot 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.138:6836/466019107,v1:172.21.15.138:6837/466019107] up:boot 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: mds.? [v2:172.21.15.138:6838/2562034420,v1:172.21.15.138:6839/2562034420] up:boot 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:49:36.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:49:36.700 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: from='mgr.15255 172.21.15.133:0/132207764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:49:36.700 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: pgmap v404: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-11T23:49:36.700 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:36 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3349936997' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:49:36.883 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T23:49:37.130 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:37.533 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:37 smithi133 ceph-mon[31430]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:49:37.534 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:37 smithi133 ceph-mon[31430]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:49:37.534 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:37 smithi133 ceph-mon[31430]: Cluster is now healthy 2024-09-11T23:49:37.534 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:37 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/3349936997' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:49:37.534 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:37 smithi133 ceph-mon[31430]: fsmap 12 up:standby 2024-09-11T23:49:37.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:37 smithi178 ceph-mon[35882]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:49:37.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:37 smithi178 ceph-mon[35882]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:49:37.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:37 smithi178 ceph-mon[35882]: Cluster is now healthy 2024-09-11T23:49:37.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:37 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/3349936997' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:49:37.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:37 smithi178 ceph-mon[35882]: fsmap 12 up:standby 2024-09-11T23:49:37.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.617+0000 7f8f93eeb640 1 -- 172.21.15.133:0/3144138137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8c1008c0 msgr2=0x7f8f8c100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:37.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.617+0000 7f8f93eeb640 1 --2- 172.21.15.133:0/3144138137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8c1008c0 0x7f8f8c100ca0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009900 tx=0x7f8f8802a410 comp rx=0 tx=0).stop 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.617+0000 7f8f93eeb640 1 -- 172.21.15.133:0/3144138137 shutdown_connections 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.617+0000 7f8f93eeb640 1 --2- 172.21.15.133:0/3144138137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c105ec0 0x7f8f8c106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.617+0000 7f8f93eeb640 1 --2- 172.21.15.133:0/3144138137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f8c1011e0 0x7f8f8c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.617+0000 7f8f93eeb640 1 --2- 172.21.15.133:0/3144138137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8c1008c0 0x7f8f8c100ca0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.617+0000 7f8f93eeb640 1 -- 172.21.15.133:0/3144138137 >> 172.21.15.133:0/3144138137 conn(0x7f8f8c0fc7d0 msgr2=0x7f8f8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.617+0000 7f8f93eeb640 1 -- 172.21.15.133:0/3144138137 shutdown_connections 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.617+0000 7f8f93eeb640 1 -- 172.21.15.133:0/3144138137 wait complete. 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.618+0000 7f8f93eeb640 1 Processor -- start 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.618+0000 7f8f93eeb640 1 -- start start 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.618+0000 7f8f93eeb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f8c1008c0 0x7f8f8c1a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.618+0000 7f8f93eeb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8c1011e0 0x7f8f8c1a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f93eeb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c105ec0 0x7f8f8c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f93eeb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c118680 con 0x7f8f8c1011e0 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f93eeb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c118500 con 0x7f8f8c105ec0 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f93eeb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c118800 con 0x7f8f8c1008c0 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f91486640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8c1011e0 0x7f8f8c1a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f91486640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8c1011e0 0x7f8f8c1a4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38700/0 (socket says 172.21.15.133:38700) 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f91486640 1 -- 172.21.15.133:0/1502157702 learned_addr learned my addr 172.21.15.133:0/1502157702 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f91c87640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f8c1008c0 0x7f8f8c1a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f91486640 1 -- 172.21.15.133:0/1502157702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f8c1008c0 msgr2=0x7f8f8c1a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f91486640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f8c1008c0 0x7f8f8c1a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f91486640 1 -- 172.21.15.133:0/1502157702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c105ec0 msgr2=0x7f8f8c19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f91486640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c105ec0 0x7f8f8c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f91486640 1 -- 172.21.15.133:0/1502157702 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c19f100 con 0x7f8f8c1011e0 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f91486640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8c1011e0 0x7f8f8c1a4d50 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00e9f0 tx=0x7f8f7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f82ffd640 1 -- 172.21.15.133:0/1502157702 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c00cd80 con 0x7f8f8c1011e0 2024-09-11T23:49:37.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f93eeb640 1 -- 172.21.15.133:0/1502157702 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c19f3f0 con 0x7f8f8c1011e0 2024-09-11T23:49:37.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f82ffd640 1 -- 172.21.15.133:0/1502157702 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7c005500 con 0x7f8f8c1011e0 2024-09-11T23:49:37.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f82ffd640 1 -- 172.21.15.133:0/1502157702 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c010690 con 0x7f8f8c1011e0 2024-09-11T23:49:37.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.619+0000 7f8f93eeb640 1 -- 172.21.15.133:0/1502157702 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c1ab6e0 con 0x7f8f8c1011e0 2024-09-11T23:49:37.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.620+0000 7f8f93eeb640 1 -- 172.21.15.133:0/1502157702 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f8c06a850 con 0x7f8f8c1011e0 2024-09-11T23:49:37.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.621+0000 7f8f82ffd640 1 -- 172.21.15.133:0/1502157702 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8f7c0046c0 con 0x7f8f8c1011e0 2024-09-11T23:49:37.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.622+0000 7f8f82ffd640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f780a2cf0 0x7f8f780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:37.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.622+0000 7f8f91c87640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f780a2cf0 0x7f8f780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:37.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.622+0000 7f8f82ffd640 1 -- 172.21.15.133:0/1502157702 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8841+0+0 (secure 0 0 0) 0x7f8f7c0cfaf0 con 0x7f8f8c1011e0 2024-09-11T23:49:37.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.622+0000 7f8f91c87640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f780a2cf0 0x7f8f780a51b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8f88006010 tx=0x7f8f88005a20 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:37.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.626+0000 7f8f82ffd640 1 -- 172.21.15.133:0/1502157702 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f7c09c5d0 con 0x7f8f8c1011e0 2024-09-11T23:49:37.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:37 smithi138 ceph-mon[35142]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:49:37.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:37 smithi138 ceph-mon[35142]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:49:37.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:37 smithi138 ceph-mon[35142]: Cluster is now healthy 2024-09-11T23:49:37.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:37 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/3349936997' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:49:37.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:37 smithi138 ceph-mon[35142]: fsmap 12 up:standby 2024-09-11T23:49:37.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:37.795+0000 7f8f93eeb640 1 -- 172.21.15.133:0/1502157702 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8f8c1ab910 con 0x7f8f8c1011e0 2024-09-11T23:49:38.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.234+0000 7f8f82ffd640 1 -- 172.21.15.133:0/1502157702 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) ==== 181+0+0 (secure 0 0 0) 0x7f8f7c09bed0 con 0x7f8f8c1011e0 2024-09-11T23:49:38.234 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-09-11T23:49:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.236+0000 7f8f93eeb640 1 -- 172.21.15.133:0/1502157702 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f780a2cf0 msgr2=0x7f8f780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.236+0000 7f8f93eeb640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f780a2cf0 0x7f8f780a51b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8f88006010 tx=0x7f8f88005a20 comp rx=0 tx=0).stop 2024-09-11T23:49:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.236+0000 7f8f93eeb640 1 -- 172.21.15.133:0/1502157702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8c1011e0 msgr2=0x7f8f8c1a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.236+0000 7f8f93eeb640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8c1011e0 0x7f8f8c1a4d50 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00e9f0 tx=0x7f8f7c00edf0 comp rx=0 tx=0).stop 2024-09-11T23:49:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.237+0000 7f8f93eeb640 1 -- 172.21.15.133:0/1502157702 shutdown_connections 2024-09-11T23:49:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.237+0000 7f8f93eeb640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f8f780a2cf0 0x7f8f780a51b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.237+0000 7f8f93eeb640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c105ec0 0x7f8f8c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.237+0000 7f8f93eeb640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f8c1011e0 0x7f8f8c1a4d50 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.237+0000 7f8f93eeb640 1 --2- 172.21.15.133:0/1502157702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f8c1008c0 0x7f8f8c1a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.237+0000 7f8f93eeb640 1 -- 172.21.15.133:0/1502157702 >> 172.21.15.133:0/1502157702 conn(0x7f8f8c0fc7d0 msgr2=0x7f8f8c10f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.237+0000 7f8f93eeb640 1 -- 172.21.15.133:0/1502157702 shutdown_connections 2024-09-11T23:49:38.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:38.237+0000 7f8f93eeb640 1 -- 172.21.15.133:0/1502157702 wait complete. 2024-09-11T23:49:38.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:38 smithi178 ceph-mon[35882]: pgmap v405: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-11T23:49:38.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:38 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1502157702' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:49:38.612 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:38 smithi133 ceph-mon[31430]: pgmap v405: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-11T23:49:38.613 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:38 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1502157702' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:49:38.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:38 smithi138 ceph-mon[35142]: pgmap v405: 97 pgs: 97 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-11T23:49:38.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:38 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1502157702' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:49:39.047 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 42321b88-7095-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T23:49:39.299 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:49:39.420 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:39 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1502157702' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:49:39.420 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:39 smithi133 ceph-mon[31430]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T23:49:39.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:39 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1502157702' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:49:39.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:39 smithi178 ceph-mon[35882]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T23:49:39.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:39 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1502157702' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:49:39.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:39 smithi138 ceph-mon[35142]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T23:49:39.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.757+0000 7f3f02256640 1 -- 172.21.15.133:0/2896401401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc103660 msgr2=0x7f3efc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:39.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.757+0000 7f3f02256640 1 --2- 172.21.15.133:0/2896401401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc103660 0x7f3efc10bb50 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009900 tx=0x7f3ef002a480 comp rx=0 tx=0).stop 2024-09-11T23:49:39.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.757+0000 7f3f02256640 1 -- 172.21.15.133:0/2896401401 shutdown_connections 2024-09-11T23:49:39.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.757+0000 7f3f02256640 1 --2- 172.21.15.133:0/2896401401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc10c170 0x7f3efc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.757+0000 7f3f02256640 1 --2- 172.21.15.133:0/2896401401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc103660 0x7f3efc10bb50 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.757+0000 7f3f02256640 1 --2- 172.21.15.133:0/2896401401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc102d40 0x7f3efc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.757+0000 7f3f02256640 1 -- 172.21.15.133:0/2896401401 >> 172.21.15.133:0/2896401401 conn(0x7f3efc0fc7a0 msgr2=0x7f3efc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.757+0000 7f3f02256640 1 -- 172.21.15.133:0/2896401401 shutdown_connections 2024-09-11T23:49:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.757+0000 7f3f02256640 1 -- 172.21.15.133:0/2896401401 wait complete. 2024-09-11T23:49:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.758+0000 7f3f02256640 1 Processor -- start 2024-09-11T23:49:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.758+0000 7f3f02256640 1 -- start start 2024-09-11T23:49:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.759+0000 7f3f02256640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc102d40 0x7f3efc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.759+0000 7f3f02256640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc103660 0x7f3efc198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.759+0000 7f3f02256640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc10c170 0x7f3efc19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.759+0000 7f3f02256640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc112030 con 0x7f3efc103660 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.759+0000 7f3f02256640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc111eb0 con 0x7f3efc102d40 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.759+0000 7f3f02256640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1121b0 con 0x7f3efc10c170 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.759+0000 7f3efaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc103660 0x7f3efc198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.759+0000 7f3efaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc103660 0x7f3efc198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38714/0 (socket says 172.21.15.133:38714) 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.759+0000 7f3efaffd640 1 -- 172.21.15.133:0/1019347797 learned_addr learned my addr 172.21.15.133:0/1019347797 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.759+0000 7f3efb7fe640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc102d40 0x7f3efc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.759+0000 7f3efbfff640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc10c170 0x7f3efc19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.760+0000 7f3efaffd640 1 -- 172.21.15.133:0/1019347797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc10c170 msgr2=0x7f3efc19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.760+0000 7f3efaffd640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc10c170 0x7f3efc19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.760+0000 7f3efaffd640 1 -- 172.21.15.133:0/1019347797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc102d40 msgr2=0x7f3efc198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.760+0000 7f3efaffd640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc102d40 0x7f3efc198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.760+0000 7f3efaffd640 1 -- 172.21.15.133:0/1019347797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc19d640 con 0x7f3efc103660 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.760+0000 7f3efaffd640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc103660 0x7f3efc198bd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005ab0 tx=0x7f3ef00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:39.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.760+0000 7f3ef8ff9640 1 -- 172.21.15.133:0/1019347797 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef003f030 con 0x7f3efc103660 2024-09-11T23:49:39.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.760+0000 7f3f02256640 1 -- 172.21.15.133:0/1019347797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc19d8d0 con 0x7f3efc103660 2024-09-11T23:49:39.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.760+0000 7f3ef8ff9640 1 -- 172.21.15.133:0/1019347797 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef0003ce0 con 0x7f3efc103660 2024-09-11T23:49:39.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.760+0000 7f3ef8ff9640 1 -- 172.21.15.133:0/1019347797 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0035070 con 0x7f3efc103660 2024-09-11T23:49:39.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.760+0000 7f3f02256640 1 -- 172.21.15.133:0/1019347797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a5210 con 0x7f3efc103660 2024-09-11T23:49:39.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.761+0000 7f3f02256640 1 -- 172.21.15.133:0/1019347797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc06b070 con 0x7f3efc103660 2024-09-11T23:49:39.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.762+0000 7f3ef8ff9640 1 -- 172.21.15.133:0/1019347797 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3ef0042030 con 0x7f3efc103660 2024-09-11T23:49:39.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.763+0000 7f3ef8ff9640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ee40a2cf0 0x7f3ee40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:49:39.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.763+0000 7f3efb7fe640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ee40a2cf0 0x7f3ee40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:49:39.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.763+0000 7f3ef8ff9640 1 -- 172.21.15.133:0/1019347797 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8345+0+0 (secure 0 0 0) 0x7f3ef00efdc0 con 0x7f3efc103660 2024-09-11T23:49:39.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.764+0000 7f3efb7fe640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ee40a2cf0 0x7f3ee40a51b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00a400 tx=0x7f3eec00d040 comp rx=0 tx=0).ready entity=mgr.15255 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:49:39.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.766+0000 7f3ef8ff9640 1 -- 172.21.15.133:0/1019347797 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ef00bc440 con 0x7f3efc103660 2024-09-11T23:49:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:39.932+0000 7f3f02256640 1 -- 172.21.15.133:0/1019347797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3efc1071e0 con 0x7f3efc103660 2024-09-11T23:49:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.242+0000 7f3ef8ff9640 1 -- 172.21.15.133:0/1019347797 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) ==== 169+0+0 (secure 0 0 0) 0x7f3ef00c0740 con 0x7f3efc103660 2024-09-11T23:49:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-09-11T23:49:40.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.244+0000 7f3f02256640 1 -- 172.21.15.133:0/1019347797 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ee40a2cf0 msgr2=0x7f3ee40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:40.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.244+0000 7f3f02256640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ee40a2cf0 0x7f3ee40a51b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00a400 tx=0x7f3eec00d040 comp rx=0 tx=0).stop 2024-09-11T23:49:40.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.244+0000 7f3f02256640 1 -- 172.21.15.133:0/1019347797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc103660 msgr2=0x7f3efc198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:49:40.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.244+0000 7f3f02256640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc103660 0x7f3efc198bd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005ab0 tx=0x7f3ef00042f0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.245+0000 7f3f02256640 1 -- 172.21.15.133:0/1019347797 shutdown_connections 2024-09-11T23:49:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.245+0000 7f3f02256640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.133:6800/2757204896,v1:172.21.15.133:6801/2757204896] conn(0x7f3ee40a2cf0 0x7f3ee40a51b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.245+0000 7f3f02256640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc10c170 0x7f3efc19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.245+0000 7f3f02256640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3efc103660 0x7f3efc198bd0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.245+0000 7f3f02256640 1 --2- 172.21.15.133:0/1019347797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc102d40 0x7f3efc198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:49:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.245+0000 7f3f02256640 1 -- 172.21.15.133:0/1019347797 >> 172.21.15.133:0/1019347797 conn(0x7f3efc0fc7a0 msgr2=0x7f3efc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:49:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.245+0000 7f3f02256640 1 -- 172.21.15.133:0/1019347797 shutdown_connections 2024-09-11T23:49:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T23:49:40.245+0000 7f3f02256640 1 -- 172.21.15.133:0/1019347797 wait complete. 2024-09-11T23:49:40.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:40 smithi178 ceph-mon[35882]: pgmap v407: 65 pgs: 65 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail 2024-09-11T23:49:40.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:40 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1019347797' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:49:40.583 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:40 smithi133 ceph-mon[31430]: pgmap v407: 65 pgs: 65 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail 2024-09-11T23:49:40.583 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:40 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1019347797' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:49:40.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:40 smithi138 ceph-mon[35142]: pgmap v407: 65 pgs: 65 active+clean; 50 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail 2024-09-11T23:49:40.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:40 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1019347797' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:49:41.044 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:49:41.082 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:49:41.123 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:49:41.151 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T23:49:41.151 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:49:41.173 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:49:41.195 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:49:41.210 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T23:49:41.211 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T23:49:41.211 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.a 2024-09-11T23:49:41.263 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:41 smithi133 ceph-mon[31430]: from='client.? 172.21.15.133:0/1019347797' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:49:41.263 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:41 smithi133 ceph-mon[31430]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T23:49:41.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:41 smithi138 ceph-mon[35142]: from='client.? 172.21.15.133:0/1019347797' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:49:41.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:41 smithi138 ceph-mon[35142]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T23:49:41.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:41 smithi178 ceph-mon[35882]: from='client.? 172.21.15.133:0/1019347797' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:49:41.540 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:41 smithi178 ceph-mon[35882]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T23:49:41.717 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:41 smithi133 systemd[1]: Stopping Ceph mon.a for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:49:42.058 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:41 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a[31426]: 2024-09-11T23:49:41.725+0000 7f69ee0fe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:49:42.058 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:41 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a[31426]: 2024-09-11T23:49:41.725+0000 7f69ee0fe640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T23:49:42.058 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:41 smithi133 podman[115992]: 2024-09-11 23:49:41.798536612 +0000 UTC m=+0.244729589 container died 573b8e5cf42605cd63fa7fa87cea92bdca73da537420b5edbaf9e46d8317207e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T23:49:42.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:42 smithi133 podman[115992]: 2024-09-11 23:49:42.261386314 +0000 UTC m=+0.707579282 container cleanup 573b8e5cf42605cd63fa7fa87cea92bdca73da537420b5edbaf9e46d8317207e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-11T23:49:42.459 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:42 smithi133 bash[115992]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a 2024-09-11T23:49:42.875 INFO:journalctl@ceph.mon.a.smithi133.stdout:Sep 11 23:49:42 smithi133 podman[116004]: 2024-09-11 23:49:42.537906333 +0000 UTC m=+0.729018031 container remove 573b8e5cf42605cd63fa7fa87cea92bdca73da537420b5edbaf9e46d8317207e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T23:49:42.892 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T23:49:42.946 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:49:42.947 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T23:49:42.947 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-11T23:49:42.947 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.b 2024-09-11T23:49:43.263 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:43 smithi138 systemd[1]: Stopping Ceph mon.b for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:49:43.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:43 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-b[35138]: 2024-09-11T23:49:43.263+0000 7f3a2588b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:49:43.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:43 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-b[35138]: 2024-09-11T23:49:43.263+0000 7f3a2588b640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T23:49:43.698 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:43 smithi138 podman[70739]: 2024-09-11 23:49:43.457303149 +0000 UTC m=+0.301227637 container died 645e84e2dcdbe4917630c3d596c0568510fc2ea1a0f6a83572ee41e66f6aeb23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T23:49:44.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:43 smithi138 podman[70739]: 2024-09-11 23:49:43.822505336 +0000 UTC m=+0.666429831 container cleanup 645e84e2dcdbe4917630c3d596c0568510fc2ea1a0f6a83572ee41e66f6aeb23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T23:49:44.198 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 11 23:49:43 smithi138 bash[70739]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-b 2024-09-11T23:49:44.427 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T23:49:44.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:49:44.474 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-11T23:49:44.474 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-11T23:49:44.474 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.c 2024-09-11T23:49:44.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:44 smithi178 systemd[1]: Stopping Ceph mon.c for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:49:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-c[35878]: 2024-09-11T23:49:44.895+0000 7fbfdeb9c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:49:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-c[35878]: 2024-09-11T23:49:44.895+0000 7fbfdeb9c640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T23:49:45.290 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:44 smithi178 podman[72151]: 2024-09-11 23:49:44.927162956 +0000 UTC m=+0.173182999 container died e0fe9a71a8b5982cee49708dfd1d24e32119ca46c097e0f1435e9242b69a5692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T23:49:45.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:45 smithi178 podman[72151]: 2024-09-11 23:49:45.337348642 +0000 UTC m=+0.583368670 container cleanup e0fe9a71a8b5982cee49708dfd1d24e32119ca46c097e0f1435e9242b69a5692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:49:45.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:45 smithi178 bash[72151]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-c 2024-09-11T23:49:45.790 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 23:49:45 smithi178 podman[72163]: 2024-09-11 23:49:45.539081702 +0000 UTC m=+0.605507900 container remove e0fe9a71a8b5982cee49708dfd1d24e32119ca46c097e0f1435e9242b69a5692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:49:45.846 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T23:49:45.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:49:45.922 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-11T23:49:45.922 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T23:49:45.922 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.x 2024-09-11T23:49:46.209 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:49:45 smithi133 systemd[1]: Stopping Ceph mgr.x for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:49:46.573 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:49:46 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x[31659]: 2024-09-11T23:49:46.274+0000 7f05ccbad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:49:46.573 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:49:46 smithi133 podman[116107]: 2024-09-11 23:49:46.305496947 +0000 UTC m=+0.161151884 container died 85e3425cd9c0cb05067ae350762509afe76cfe703d68e9fd21647c960c4bb476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T23:49:46.959 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:49:46 smithi133 podman[116107]: 2024-09-11 23:49:46.774387434 +0000 UTC m=+0.630042371 container cleanup 85e3425cd9c0cb05067ae350762509afe76cfe703d68e9fd21647c960c4bb476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:49:46.959 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 11 23:49:46 smithi133 bash[116107]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-x 2024-09-11T23:49:47.268 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T23:49:47.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:49:47.306 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T23:49:47.306 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T23:49:47.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.y 2024-09-11T23:49:47.624 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:49:47 smithi138 systemd[1]: Stopping Ceph mgr.y for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:49:47.917 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:49:47 smithi138 podman[70861]: 2024-09-11 23:49:47.625226095 +0000 UTC m=+0.131691112 container died 1b7d14b3bbac10b61420cec7375b522ca8aa4dea65c1ef77fd3a581538e3a3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:49:48.198 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:49:47 smithi138 podman[70861]: 2024-09-11 23:49:47.918815794 +0000 UTC m=+0.425280810 container cleanup 1b7d14b3bbac10b61420cec7375b522ca8aa4dea65c1ef77fd3a581538e3a3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T23:49:48.198 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:49:47 smithi138 bash[70861]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y 2024-09-11T23:49:48.198 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:49:48 smithi138 podman[70874]: 2024-09-11 23:49:48.120412572 +0000 UTC m=+0.499478112 container remove 1b7d14b3bbac10b61420cec7375b522ca8aa4dea65c1ef77fd3a581538e3a3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T23:49:48.198 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 11 23:49:48 smithi138 systemd[1]: ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-11T23:49:48.368 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T23:49:48.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:49:48.404 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T23:49:48.404 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-11T23:49:48.404 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.z 2024-09-11T23:49:48.780 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:49:48 smithi178 systemd[1]: Stopping Ceph mgr.z for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:49:49.040 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:49:48 smithi178 podman[72269]: 2024-09-11 23:49:48.781500015 +0000 UTC m=+0.148660916 container died 44877262ab9275fd0841daba8e6b0cd92d2e4243ab06ef47ded6720e79286988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T23:49:49.527 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:49:49 smithi178 podman[72269]: 2024-09-11 23:49:49.192476075 +0000 UTC m=+0.559636983 container cleanup 44877262ab9275fd0841daba8e6b0cd92d2e4243ab06ef47ded6720e79286988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T23:49:49.527 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:49:49 smithi178 bash[72269]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z 2024-09-11T23:49:49.790 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:49:49 smithi178 podman[72281]: 2024-09-11 23:49:49.528080432 +0000 UTC m=+0.742959251 container remove 44877262ab9275fd0841daba8e6b0cd92d2e4243ab06ef47ded6720e79286988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-11T23:49:49.790 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 23:49:49 smithi178 systemd[1]: ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-11T23:49:49.808 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-11T23:49:49.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:49:49.843 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-11T23:49:49.843 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T23:49:49.843 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.0 2024-09-11T23:49:50.145 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:49:49 smithi133 systemd[1]: Stopping Ceph osd.0 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:49:50.459 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:49:50 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0[42562]: 2024-09-11T23:49:50.145+0000 7f314c76e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:49:50.459 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:49:50 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0[42562]: 2024-09-11T23:49:50.145+0000 7f314c76e640 -1 osd.0 101 *** Got signal Terminated *** 2024-09-11T23:49:50.459 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:49:50 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0[42562]: 2024-09-11T23:49:50.145+0000 7f314c76e640 -1 osd.0 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:50:16.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:16 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:16.139+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:15.868326+0000) 2024-09-11T23:50:17.145 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:16 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:16.886+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:16.536540+0000) 2024-09-11T23:50:17.198 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:16 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1[39607]: 2024-09-11T23:50:16.845+0000 7fe2c2cb8640 -1 osd.1 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:54.804903+0000 front 2024-09-11T23:49:54.804772+0000 (oldest deadline 2024-09-11T23:50:16.504715+0000) 2024-09-11T23:50:17.273 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:17 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:17.142+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:16.193887+0000) 2024-09-11T23:50:17.273 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:16 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2[40845]: 2024-09-11T23:50:16.970+0000 7fc548daa640 -1 osd.2 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:54.566761+0000 front 2024-09-11T23:49:54.566735+0000 (oldest deadline 2024-09-11T23:50:16.866944+0000) 2024-09-11T23:50:17.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:17 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:17.145+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:15.868326+0000) 2024-09-11T23:50:18.094 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:17 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1[39607]: 2024-09-11T23:50:17.819+0000 7fe2c2cb8640 -1 osd.1 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:54.804903+0000 front 2024-09-11T23:49:54.804772+0000 (oldest deadline 2024-09-11T23:50:16.504715+0000) 2024-09-11T23:50:18.128 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:17 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:17.849+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:16.536540+0000) 2024-09-11T23:50:18.155 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:17 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2[40845]: 2024-09-11T23:50:17.968+0000 7fc548daa640 -1 osd.2 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:54.566761+0000 front 2024-09-11T23:49:54.566735+0000 (oldest deadline 2024-09-11T23:50:16.866944+0000) 2024-09-11T23:50:18.448 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:18 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:18.094+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:17.400155+0000) 2024-09-11T23:50:18.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:18 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:18.129+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:15.868326+0000) 2024-09-11T23:50:18.540 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:18.155+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:16.193887+0000) 2024-09-11T23:50:18.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:18.247+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:17.452662+0000) 2024-09-11T23:50:19.125 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:18 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1[39607]: 2024-09-11T23:50:18.849+0000 7fe2c2cb8640 -1 osd.1 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:54.804903+0000 front 2024-09-11T23:49:54.804772+0000 (oldest deadline 2024-09-11T23:50:16.504715+0000) 2024-09-11T23:50:19.175 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:18 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:18.803+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:16.536540+0000) 2024-09-11T23:50:19.248 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:19.139+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:16.193887+0000) 2024-09-11T23:50:19.249 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2[40845]: 2024-09-11T23:50:18.928+0000 7fc548daa640 -1 osd.2 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:54.566761+0000 front 2024-09-11T23:49:54.566735+0000 (oldest deadline 2024-09-11T23:50:16.866944+0000) 2024-09-11T23:50:19.447 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:19 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:19.125+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:17.400155+0000) 2024-09-11T23:50:19.461 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:19 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:19.175+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:15.868326+0000) 2024-09-11T23:50:19.461 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:19 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3[49209]: 2024-09-11T23:50:19.418+0000 7fe86e1b6640 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.607615+0000 front 2024-09-11T23:49:53.607664+0000 (oldest deadline 2024-09-11T23:50:18.907792+0000) 2024-09-11T23:50:19.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:19.249+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:17.452662+0000) 2024-09-11T23:50:20.088 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:19 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1[39607]: 2024-09-11T23:50:19.814+0000 7fe2c2cb8640 -1 osd.1 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:54.804903+0000 front 2024-09-11T23:49:54.804772+0000 (oldest deadline 2024-09-11T23:50:16.504715+0000) 2024-09-11T23:50:20.156 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:19 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:19.827+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:16.536540+0000) 2024-09-11T23:50:20.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:20 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:20.186+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:16.193887+0000) 2024-09-11T23:50:20.290 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2[40845]: 2024-09-11T23:50:19.943+0000 7fc548daa640 -1 osd.2 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:54.566761+0000 front 2024-09-11T23:49:54.566735+0000 (oldest deadline 2024-09-11T23:50:16.866944+0000) 2024-09-11T23:50:20.447 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:20 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:20.088+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:17.400155+0000) 2024-09-11T23:50:20.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:20 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:20.211+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:15.868326+0000) 2024-09-11T23:50:20.459 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:20 smithi133 bash[116223]: time="2024-09-11T23:50:20Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0 in 30 seconds, resorting to SIGKILL" 2024-09-11T23:50:20.459 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:20 smithi133 podman[116223]: 2024-09-11 23:50:20.294061019 +0000 UTC m=+30.266136474 container died f490ce00575ee84f856aaba365f57a5b9286fc0bf54c6730321fcf2eb28dd98f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:50:20.959 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:20 smithi133 podman[116223]: 2024-09-11 23:50:20.768316303 +0000 UTC m=+30.740391803 container cleanup f490ce00575ee84f856aaba365f57a5b9286fc0bf54c6730321fcf2eb28dd98f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T23:50:20.959 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:20 smithi133 bash[116223]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0 2024-09-11T23:50:21.441 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:21 smithi133 podman[116242]: 2024-09-11 23:50:21.094589571 +0000 UTC m=+0.802740399 container remove f490ce00575ee84f856aaba365f57a5b9286fc0bf54c6730321fcf2eb28dd98f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T23:50:21.442 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:21 smithi133 systemd[1]: ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.0.service: Main process exited, code=exited, status=137/n/a 2024-09-11T23:50:21.710 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:21 smithi133 podman[116328]: 2024-09-11 23:50:21.345576433 +0000 UTC m=+0.018455361 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:50:21.710 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:21 smithi133 podman[116328]: 2024-09-11 23:50:21.445296782 +0000 UTC m=+0.118175707 container create f922795acf648eb974f42c5f774ee7e1ad79f348b4791360abc785664b7ddfee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-11T23:50:21.710 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:21 smithi133 podman[116328]: 2024-09-11 23:50:21.681288662 +0000 UTC m=+0.354167587 container init f922795acf648eb974f42c5f774ee7e1ad79f348b4791360abc785664b7ddfee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-11T23:50:21.710 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:21 smithi133 podman[116328]: 2024-09-11 23:50:21.684625614 +0000 UTC m=+0.357504541 container start f922795acf648eb974f42c5f774ee7e1ad79f348b4791360abc785664b7ddfee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T23:50:22.209 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:21 smithi133 podman[116328]: 2024-09-11 23:50:21.789849518 +0000 UTC m=+0.462728450 container attach f922795acf648eb974f42c5f774ee7e1ad79f348b4791360abc785664b7ddfee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T23:50:22.209 INFO:journalctl@ceph.osd.0.smithi133.stdout:Sep 11 23:50:21 smithi133 podman[116328]: 2024-09-11 23:50:21.879693408 +0000 UTC m=+0.552572344 container died f922795acf648eb974f42c5f774ee7e1ad79f348b4791360abc785664b7ddfee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T23:50:22.519 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T23:50:22.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:50:22.558 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T23:50:22.558 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T23:50:22.558 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.1 2024-09-11T23:50:22.905 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:22 smithi138 systemd[1]: Stopping Ceph osd.1 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:50:23.198 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:22 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1[39607]: 2024-09-11T23:50:22.904+0000 7fe2c66b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:50:23.198 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:22 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1[39607]: 2024-09-11T23:50:22.904+0000 7fe2c66b3640 -1 osd.1 101 *** Got signal Terminated *** 2024-09-11T23:50:23.198 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:22 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1[39607]: 2024-09-11T23:50:22.904+0000 7fe2c66b3640 -1 osd.1 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:50:29.814 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:29 smithi138 podman[70986]: 2024-09-11 23:50:29.49744383 +0000 UTC m=+6.741179435 container died dc20fe75352823396eac403692395ef32b37b9dc4be32c0b38a40030a37180c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-11T23:50:30.130 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:29 smithi138 podman[70986]: 2024-09-11 23:50:29.871330214 +0000 UTC m=+7.115065810 container cleanup dc20fe75352823396eac403692395ef32b37b9dc4be32c0b38a40030a37180c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:50:30.130 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:29 smithi138 bash[70986]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1 2024-09-11T23:50:30.448 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:30 smithi138 podman[71002]: 2024-09-11 23:50:30.131142617 +0000 UTC m=+0.628804834 container remove dc20fe75352823396eac403692395ef32b37b9dc4be32c0b38a40030a37180c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:50:30.783 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:30 smithi138 podman[71089]: 2024-09-11 23:50:30.47359188 +0000 UTC m=+0.108882316 container create 3c5cc409a9572f65a6d7c1e51d29baedc487d3303bc00f9f515e72e2f3fc85e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:50:30.783 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:30 smithi138 podman[71089]: 2024-09-11 23:50:30.38053786 +0000 UTC m=+0.015828297 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:50:30.783 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:30 smithi138 podman[71089]: 2024-09-11 23:50:30.709444799 +0000 UTC m=+0.344735235 container init 3c5cc409a9572f65a6d7c1e51d29baedc487d3303bc00f9f515e72e2f3fc85e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T23:50:30.783 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:30 smithi138 podman[71089]: 2024-09-11 23:50:30.712662835 +0000 UTC m=+0.347953284 container start 3c5cc409a9572f65a6d7c1e51d29baedc487d3303bc00f9f515e72e2f3fc85e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T23:50:30.783 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:30 smithi138 podman[71089]: 2024-09-11 23:50:30.784368508 +0000 UTC m=+0.419658945 container attach 3c5cc409a9572f65a6d7c1e51d29baedc487d3303bc00f9f515e72e2f3fc85e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-11T23:50:31.145 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 11 23:50:30 smithi138 podman[71112]: 2024-09-11 23:50:30.938430109 +0000 UTC m=+0.017101197 container died 3c5cc409a9572f65a6d7c1e51d29baedc487d3303bc00f9f515e72e2f3fc85e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T23:50:31.455 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T23:50:31.494 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:50:31.494 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T23:50:31.494 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T23:50:31.494 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.2 2024-09-11T23:50:31.790 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:31 smithi178 systemd[1]: Stopping Ceph osd.2 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:50:32.290 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:31 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2[40845]: 2024-09-11T23:50:31.820+0000 7fc54c7a5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:50:32.290 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:31 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2[40845]: 2024-09-11T23:50:31.820+0000 7fc54c7a5640 -1 osd.2 101 *** Got signal Terminated *** 2024-09-11T23:50:32.290 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:31 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2[40845]: 2024-09-11T23:50:31.820+0000 7fc54c7a5640 -1 osd.2 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:50:37.745 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:37 smithi178 podman[72406]: 2024-09-11 23:50:37.478493433 +0000 UTC m=+5.782187502 container died aead7c577c2f4b6ae6442bd05c9127aa03bd5196876be4ac8317441b72b3e519 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:50:38.040 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:37 smithi178 podman[72406]: 2024-09-11 23:50:37.845400121 +0000 UTC m=+6.149094177 container cleanup aead7c577c2f4b6ae6442bd05c9127aa03bd5196876be4ac8317441b72b3e519 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T23:50:38.040 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:37 smithi178 bash[72406]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2 2024-09-11T23:50:38.450 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:38 smithi178 podman[72423]: 2024-09-11 23:50:38.130346989 +0000 UTC m=+0.648586187 container remove aead7c577c2f4b6ae6442bd05c9127aa03bd5196876be4ac8317441b72b3e519 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T23:50:38.708 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:38 smithi178 podman[72515]: 2024-09-11 23:50:38.354684195 +0000 UTC m=+0.015721359 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:50:38.708 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:38 smithi178 podman[72515]: 2024-09-11 23:50:38.464638262 +0000 UTC m=+0.125675410 container create 143f7d7cc0dd6b4cef2af3b5e158ac47dde2123d4d9c162dcaaf85c3dea26e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T23:50:38.708 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:38 smithi178 podman[72515]: 2024-09-11 23:50:38.70903306 +0000 UTC m=+0.370070208 container init 143f7d7cc0dd6b4cef2af3b5e158ac47dde2123d4d9c162dcaaf85c3dea26e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:50:39.040 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:38 smithi178 podman[72515]: 2024-09-11 23:50:38.712883464 +0000 UTC m=+0.373920628 container start 143f7d7cc0dd6b4cef2af3b5e158ac47dde2123d4d9c162dcaaf85c3dea26e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-11T23:50:39.040 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:38 smithi178 podman[72515]: 2024-09-11 23:50:38.767165882 +0000 UTC m=+0.428203030 container attach 143f7d7cc0dd6b4cef2af3b5e158ac47dde2123d4d9c162dcaaf85c3dea26e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T23:50:39.040 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 23:50:38 smithi178 podman[72515]: 2024-09-11 23:50:38.895342632 +0000 UTC m=+0.556379791 container died 143f7d7cc0dd6b4cef2af3b5e158ac47dde2123d4d9c162dcaaf85c3dea26e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T23:50:39.528 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T23:50:39.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:50:39.570 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T23:50:39.570 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T23:50:39.570 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.3 2024-09-11T23:50:39.886 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:39 smithi133 systemd[1]: Stopping Ceph osd.3 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:50:40.209 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:39 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3[49209]: 2024-09-11T23:50:39.886+0000 7fe871bb1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:50:40.209 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:39 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3[49209]: 2024-09-11T23:50:39.886+0000 7fe871bb1640 -1 osd.3 101 *** Got signal Terminated *** 2024-09-11T23:50:40.209 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:39 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3[49209]: 2024-09-11T23:50:39.886+0000 7fe871bb1640 -1 osd.3 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:50:41.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:41 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:41.259+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:42.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:42.179+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:42.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:42.243+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:43.209 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:42 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:42.949+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:43.448 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:43 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:43.037+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:43.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:43.230+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:43.694 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:43.223+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:44.447 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:44 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:44.077+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:44.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:44 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:44.200+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:44.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:43 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:43.962+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:44.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:44.196+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:45.187 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:44 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:44.921+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:45.447 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:45 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:45.105+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:45.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:45.188+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:45.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:45.162+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:45.540 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:45.069+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:45.869 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:45 smithi133 podman[116431]: 2024-09-11 23:50:45.581996447 +0000 UTC m=+5.819482007 container died bf7577837d592f022da39ff2db0cf42745f23e94919cf39d02af040a2894b422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:50:46.208 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:45 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:45.952+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:46.208 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:46 smithi133 podman[116431]: 2024-09-11 23:50:46.052539094 +0000 UTC m=+6.290024653 container cleanup bf7577837d592f022da39ff2db0cf42745f23e94919cf39d02af040a2894b422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:50:46.208 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:46 smithi133 bash[116431]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3 2024-09-11T23:50:46.447 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:46 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:46.095+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:46.459 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:46 smithi133 podman[116458]: 2024-09-11 23:50:46.329328063 +0000 UTC m=+0.749211079 container remove bf7577837d592f022da39ff2db0cf42745f23e94919cf39d02af040a2894b422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-09-11T23:50:46.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:46 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:46.208+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:46.540 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:46 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:46.071+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:46.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:46 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:46.209+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:46.939 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:46 smithi133 podman[116548]: 2024-09-11 23:50:46.573212387 +0000 UTC m=+0.015557593 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:50:46.939 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:46 smithi133 podman[116548]: 2024-09-11 23:50:46.671816612 +0000 UTC m=+0.114161853 container create 3a794a34f34d88cdf3569b577ae071c120645e89da0c18a93732dbcc5c33f6a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T23:50:46.940 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:46 smithi133 podman[116548]: 2024-09-11 23:50:46.891484373 +0000 UTC m=+0.333829574 container init 3a794a34f34d88cdf3569b577ae071c120645e89da0c18a93732dbcc5c33f6a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T23:50:46.940 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:46 smithi133 podman[116548]: 2024-09-11 23:50:46.895075244 +0000 UTC m=+0.337420444 container start 3a794a34f34d88cdf3569b577ae071c120645e89da0c18a93732dbcc5c33f6a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-11T23:50:47.209 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:46 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:46.939+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:47.209 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:46 smithi133 podman[116548]: 2024-09-11 23:50:46.988091144 +0000 UTC m=+0.430436357 container attach 3a794a34f34d88cdf3569b577ae071c120645e89da0c18a93732dbcc5c33f6a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T23:50:47.210 INFO:journalctl@ceph.osd.3.smithi133.stdout:Sep 11 23:50:47 smithi133 podman[116548]: 2024-09-11 23:50:47.082210692 +0000 UTC m=+0.524555894 container died 3a794a34f34d88cdf3569b577ae071c120645e89da0c18a93732dbcc5c33f6a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T23:50:47.448 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:47 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:47.068+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:47.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:47 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:47.185+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:47.540 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:47 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:47.054+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:47.676 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:47.241+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:47.708 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T23:50:47.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:50:47.750 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T23:50:47.750 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T23:50:47.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.4 2024-09-11T23:50:48.036 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:47 smithi138 systemd[1]: Stopping Ceph osd.4 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:50:48.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:48 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:48.156+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:48.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:48 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:48.015+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:48.448 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:48 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:48.037+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:48.448 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:48 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4[46102]: 2024-09-11T23:50:48.082+0000 7fa828935640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:50:48.448 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:48 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4[46102]: 2024-09-11T23:50:48.082+0000 7fa828935640 -1 osd.4 101 *** Got signal Terminated *** 2024-09-11T23:50:48.448 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:48 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4[46102]: 2024-09-11T23:50:48.082+0000 7fa828935640 -1 osd.4 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:50:48.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:47 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:47.970+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:48.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:48.192+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:49.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:49 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:49.149+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:49.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:49 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:49.014+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:49.447 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:48 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:48.994+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:49.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:48 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:48.962+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:49.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:49.143+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:50.198 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:50:49 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:50:49.736+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:50:50.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:49 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:49.966+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:50.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:49 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:49.966+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:50:50.250 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:50 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:50.158+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:50.251 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:49 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:49.975+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:50.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:50 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:50.141+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:50.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:49 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:49.997+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:51.133 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:50 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:50.990+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:51.198 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:50:50 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:50:50.745+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:50:51.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:50 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:50.980+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:51.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:50 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:50.980+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:50:51.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:51 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:51.130+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:51.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:51 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:51.024+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:51.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:51 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:51.134+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:51.540 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:51 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:51.134+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:50:52.198 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:51 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4[46102]: 2024-09-11T23:50:51.895+0000 7fa820f32640 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.769297+0000 front 2024-09-11T23:50:25.769275+0000 (oldest deadline 2024-09-11T23:50:51.669516+0000) 2024-09-11T23:50:52.198 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:50:51 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:50:51.728+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:50:52.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:51 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:51.955+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:52.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:51 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:51.955+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:50:52.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:52 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:52.019+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:52.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:52 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:52.147+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:52.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:52 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:52.147+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:50:52.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:52 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:52.044+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:52.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:52 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:52.119+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:52.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:52 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:52.119+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:50:53.198 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:52 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4[46102]: 2024-09-11T23:50:52.915+0000 7fa820f32640 -1 osd.4 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.769297+0000 front 2024-09-11T23:50:25.769275+0000 (oldest deadline 2024-09-11T23:50:51.669516+0000) 2024-09-11T23:50:53.198 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:50:52 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:50:52.740+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:50:53.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:52 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:52.950+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:53.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:52 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:52.950+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:50:53.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:52 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:52.999+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:53.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:53 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:53.112+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:53.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:53 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:53.112+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:50:53.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:53 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:53.056+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:53.460 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:53 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:53.078+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:53.460 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:53 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:53.078+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:50:53.903 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:53 smithi138 podman[71205]: 2024-09-11 23:50:53.652990868 +0000 UTC m=+5.706871609 container died 25dc8ea4f57a938b39ec2ee6bc9959544d1c432c32c09456202f11f7182268c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T23:50:53.903 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:50:53 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:50:53.727+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:50:54.157 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:53 smithi138 podman[71205]: 2024-09-11 23:50:53.981989813 +0000 UTC m=+6.035870556 container cleanup 25dc8ea4f57a938b39ec2ee6bc9959544d1c432c32c09456202f11f7182268c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T23:50:54.157 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:53 smithi138 bash[71205]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4 2024-09-11T23:50:54.157 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:53 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:53.903+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:54.157 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:53 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:53.903+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:50:54.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:53 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:53.976+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:54.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:54 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:54.095+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:54.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:54 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:54.095+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:50:54.448 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:54 smithi138 podman[71220]: 2024-09-11 23:50:54.158179062 +0000 UTC m=+0.504572836 container remove 25dc8ea4f57a938b39ec2ee6bc9959544d1c432c32c09456202f11f7182268c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T23:50:54.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:54 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:54.082+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:54.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:54 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:54.082+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:50:54.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:54 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:54.083+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:54.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:54 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:54.083+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:50:54.869 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:54 smithi138 podman[71309]: 2024-09-11 23:50:54.488988819 +0000 UTC m=+0.016052588 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:50:54.869 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:54 smithi138 podman[71309]: 2024-09-11 23:50:54.600883532 +0000 UTC m=+0.127947327 container create d682eaab974e284a2a7296c6c47c593449fc00209a5247e95c2966b4d5f2eff5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T23:50:54.869 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:54 smithi138 podman[71309]: 2024-09-11 23:50:54.794907327 +0000 UTC m=+0.321971084 container init d682eaab974e284a2a7296c6c47c593449fc00209a5247e95c2966b4d5f2eff5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:50:54.869 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:54 smithi138 podman[71309]: 2024-09-11 23:50:54.798478117 +0000 UTC m=+0.325541867 container start d682eaab974e284a2a7296c6c47c593449fc00209a5247e95c2966b4d5f2eff5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T23:50:54.869 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:54 smithi138 podman[71309]: 2024-09-11 23:50:54.869625968 +0000 UTC m=+0.396689740 container attach d682eaab974e284a2a7296c6c47c593449fc00209a5247e95c2966b4d5f2eff5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:50:54.869 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:50:54 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:50:54.719+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:50:55.198 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 11 23:50:54 smithi138 podman[71309]: 2024-09-11 23:50:54.990692213 +0000 UTC m=+0.517755963 container died d682eaab974e284a2a7296c6c47c593449fc00209a5247e95c2966b4d5f2eff5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-11T23:50:55.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:54 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:54.888+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:55.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:54 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:54.888+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:50:55.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:54 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:54.973+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:55.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:50:54 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:50:54.977+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:50:55.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:55 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:55.092+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:55.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:55 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:55.092+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:50:55.460 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:55 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:55.106+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:55.460 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:55 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:55.106+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:50:55.461 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:55 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:55.124+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:55.461 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:55 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:55.124+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:50:55.585 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T23:50:55.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:50:55.623 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T23:50:55.623 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T23:50:55.623 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.5 2024-09-11T23:50:55.914 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:50:55 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:50:55.676+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:50:55.931 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:55 smithi178 systemd[1]: Stopping Ceph osd.5 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:50:56.197 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:55 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:55.915+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:56.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:55 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:55.915+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:50:56.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:50:55 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:50:55.954+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:50:56.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:56 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:56.115+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:56.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:56 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:56.115+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:50:56.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:55 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:55.931+0000 7f0d11b57640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:50:56.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:55 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:55.931+0000 7f0d11b57640 -1 osd.5 101 *** Got signal Terminated *** 2024-09-11T23:50:56.291 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:55 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:55.931+0000 7f0d11b57640 -1 osd.5 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:50:56.291 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:55 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:55.941+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:56.291 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:55 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:55.941+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.400758+0000 front 2024-09-11T23:50:26.400609+0000 (oldest deadline 2024-09-11T23:50:55.201101+0000) 2024-09-11T23:50:56.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:56 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:56.126+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:56.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:56 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:56.126+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:50:56.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:56 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:56.143+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:56.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:56 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:56.143+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:50:56.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:50:56 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:50:56.629+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:50:56.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:56 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:56.868+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:56.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:56 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:56.868+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:50:57.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:50:56 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:50:56.928+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:50:57.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:56 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:56.983+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:57.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:56 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:56.983+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.400758+0000 front 2024-09-11T23:50:26.400609+0000 (oldest deadline 2024-09-11T23:50:55.201101+0000) 2024-09-11T23:50:57.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:57 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:57.140+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:57.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:57 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:57.140+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:50:57.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:57 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:57.140+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:57.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:57 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:57.140+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:50:57.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:57 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:57.111+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:57.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:57 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:57.111+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:50:57.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:50:57 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:50:57.626+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:50:57.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:57 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:57.834+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:57.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:57 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:57.834+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:50:58.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:50:57 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:50:57.958+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:50:58.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:58 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:57.999+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:58.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:58 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:57.999+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.400758+0000 front 2024-09-11T23:50:26.400609+0000 (oldest deadline 2024-09-11T23:50:55.201101+0000) 2024-09-11T23:50:58.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:58 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:58.104+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:58.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:58 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:58.104+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:50:58.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:58 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:58.097+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:58.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:58 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:58.097+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:50:58.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:58 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:58.102+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:58.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:58 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:58.102+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:50:58.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:50:58 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:50:58.591+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:50:58.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:58 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:58.800+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:58.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:58 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:58.800+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:50:59.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:58 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:58.973+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:50:59.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:58 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:58.973+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.400758+0000 front 2024-09-11T23:50:26.400609+0000 (oldest deadline 2024-09-11T23:50:55.201101+0000) 2024-09-11T23:50:59.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:50:58 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:50:58.994+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:50:59.291 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:50:58 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:50:58.994+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:50:59.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:59 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:59.107+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:50:59.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:50:59 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:50:59.107+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:50:59.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:59 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:59.058+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:50:59.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:50:59 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:50:59.058+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:50:59.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:59 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:59.096+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:50:59.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:59 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:59.096+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:50:59.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:50:59 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:50:59.096+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:50:59.947 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:50:59 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:50:59.585+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:50:59.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:59 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:59.829+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:50:59.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:59 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:59.829+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:50:59.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:50:59 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:50:59.829+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:00.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:59 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:59.981+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.093791+0000 front 2024-09-11T23:49:52.093778+0000 (oldest deadline 2024-09-11T23:50:44.699432+0000) 2024-09-11T23:51:00.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:59 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:59.981+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.400758+0000 front 2024-09-11T23:50:26.400609+0000 (oldest deadline 2024-09-11T23:50:55.201101+0000) 2024-09-11T23:51:00.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:50:59 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5[47611]: 2024-09-11T23:50:59.981+0000 7f0d0e15c640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.201495+0000 front 2024-09-11T23:50:35.201446+0000 (oldest deadline 2024-09-11T23:50:59.301637+0000) 2024-09-11T23:51:00.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:50:59 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:50:59.993+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:00.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:50:59 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:50:59.993+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:00.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:00 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:00.136+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:00.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:00 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:00.136+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:00.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:00.024+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:51:00.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:00.024+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:51:00.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:00.024+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.774716+0000 front 2024-09-11T23:50:34.774705+0000 (oldest deadline 2024-09-11T23:50:59.474562+0000) 2024-09-11T23:51:00.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:00.083+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:00.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:00.083+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:00.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:00 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:00.083+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:00.856 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:00 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:00.558+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:00.856 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:00 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:00.558+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:01.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:00 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:00.856+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:01.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:00 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:00.856+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:01.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:00 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:00.856+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:01.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:00 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:00.957+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:01.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:00 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:00.957+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:01.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:01 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:01.103+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:01.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:01 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:01.103+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:01.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:01 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:01.103+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:01.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:01 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:01.008+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:51:01.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:01 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:01.008+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:51:01.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:01 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:01.008+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.774716+0000 front 2024-09-11T23:50:34.774705+0000 (oldest deadline 2024-09-11T23:50:59.474562+0000) 2024-09-11T23:51:01.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:01 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:01.107+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:01.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:01 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:01.107+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:01.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:01 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:01.107+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:01.790 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:51:01 smithi178 podman[72631]: 2024-09-11 23:51:01.538256767 +0000 UTC m=+5.715472709 container died 1baf5e1f3f23fe68b8c9cc8af979a7ce542c97cce0dade4d340ef550789672a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T23:51:01.834 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:01 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:01.581+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:01.834 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:01 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:01.581+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:02.157 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:51:01 smithi178 podman[72631]: 2024-09-11 23:51:01.906562386 +0000 UTC m=+6.083778314 container cleanup 1baf5e1f3f23fe68b8c9cc8af979a7ce542c97cce0dade4d340ef550789672a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-11T23:51:02.157 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:51:01 smithi178 bash[72631]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5 2024-09-11T23:51:02.157 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:01 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:01.961+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:02.157 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:01 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:01.961+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:02.157 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:02 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:02.063+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:02.158 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:02 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:02.063+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:02.158 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:02 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:02.063+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:02.197 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:01 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:01.834+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:02.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:01 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:01.834+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:02.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:01 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:01.834+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:02.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:02 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:02.034+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:51:02.460 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:02 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:02.034+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:51:02.460 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:02 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:02.034+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.774716+0000 front 2024-09-11T23:50:34.774705+0000 (oldest deadline 2024-09-11T23:50:59.474562+0000) 2024-09-11T23:51:02.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:02 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:02.069+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:02.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:02 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:02.069+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:02.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:02 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:02.069+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:02.514 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:51:02 smithi178 podman[72648]: 2024-09-11 23:51:02.157920767 +0000 UTC m=+0.615269473 container remove 1baf5e1f3f23fe68b8c9cc8af979a7ce542c97cce0dade4d340ef550789672a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T23:51:02.790 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:51:02 smithi178 podman[72735]: 2024-09-11 23:51:02.41783648 +0000 UTC m=+0.016117068 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:51:02.790 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:51:02 smithi178 podman[72735]: 2024-09-11 23:51:02.517274533 +0000 UTC m=+0.115555161 container create f1e11365d35d331623ed2604cd0a73f8f4fb8e02b4f3c6109697472599097783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:51:02.790 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:51:02 smithi178 podman[72735]: 2024-09-11 23:51:02.728312964 +0000 UTC m=+0.326593575 container init f1e11365d35d331623ed2604cd0a73f8f4fb8e02b4f3c6109697472599097783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:51:02.790 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:51:02 smithi178 podman[72735]: 2024-09-11 23:51:02.731915121 +0000 UTC m=+0.330195707 container start f1e11365d35d331623ed2604cd0a73f8f4fb8e02b4f3c6109697472599097783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T23:51:02.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:02 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:02.626+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:02.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:02 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:02.626+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:02.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:02 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:02.848+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:02.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:02 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:02.848+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:02.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:02 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:02.848+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:03.246 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:02 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:02.967+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:03.246 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:02 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:02.967+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:03.247 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:03 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:03.030+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:03.247 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:03 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:03.030+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:03.247 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:03 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:03.030+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:03.247 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:51:02 smithi178 podman[72735]: 2024-09-11 23:51:02.828067807 +0000 UTC m=+0.426348410 container attach f1e11365d35d331623ed2604cd0a73f8f4fb8e02b4f3c6109697472599097783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:51:03.247 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 23:51:02 smithi178 podman[72735]: 2024-09-11 23:51:02.925157074 +0000 UTC m=+0.523437674 container died f1e11365d35d331623ed2604cd0a73f8f4fb8e02b4f3c6109697472599097783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T23:51:03.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:03.064+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:51:03.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:03.064+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:51:03.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:03.064+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.774716+0000 front 2024-09-11T23:50:34.774705+0000 (oldest deadline 2024-09-11T23:50:59.474562+0000) 2024-09-11T23:51:03.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:03.097+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:03.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:03.097+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:03.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:03.097+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:03.532 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T23:51:03.570 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:51:03.571 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T23:51:03.571 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T23:51:03.571 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.6 2024-09-11T23:51:03.862 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:03 smithi133 systemd[1]: Stopping Ceph osd.6 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:51:03.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:03 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:03.607+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:03.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:03 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:03.607+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:03.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:03 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:03.847+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:03.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:03 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:03.847+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:03.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:03 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:03.847+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:04.131 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:03.862+0000 7fb20a2a3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:51:04.131 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:03.862+0000 7fb20a2a3640 -1 osd.6 101 *** Got signal Terminated *** 2024-09-11T23:51:04.131 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:03.862+0000 7fb20a2a3640 -1 osd.6 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:51:04.131 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:04 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:04.111+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:51:04.131 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:04 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:04.111+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:51:04.132 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:04 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:04.111+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.774716+0000 front 2024-09-11T23:50:34.774705+0000 (oldest deadline 2024-09-11T23:50:59.474562+0000) 2024-09-11T23:51:04.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:04 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:04.010+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:04.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:04 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:04.010+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:04.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:03 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:03.990+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:04.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:03 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:03.990+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:04.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:03 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:03.990+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:04.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:04 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:04.132+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:04.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:04 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:04.132+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:04.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:04 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:04.132+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:04.947 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:04 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:04.611+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:04.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:04 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:04.611+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:04.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:04 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:04.822+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:04.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:04 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:04.822+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:04.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:04 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:04.822+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:05.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:05 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:05.060+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:05.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:05 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:05.060+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:05.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:04 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:04.966+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:05.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:04 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:04.966+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:05.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:04 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:04.966+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:05.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:05 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:05.069+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:51:05.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:05 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:05.069+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:51:05.460 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:05 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:05.069+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.774716+0000 front 2024-09-11T23:50:34.774705+0000 (oldest deadline 2024-09-11T23:50:59.474562+0000) 2024-09-11T23:51:05.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:05 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:05.169+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:05.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:05 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:05.169+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:05.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:05 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:05.169+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:05.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:05 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:05.587+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:05.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:05 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:05.587+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:05.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:05 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:05.817+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:05.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:05 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:05.817+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:05.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:05 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:05.817+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:06.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:06 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:06.075+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:06.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:06 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:06.075+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:06.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:05 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:05.952+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:06.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:05 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:05.952+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:06.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:05 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:05.952+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:06.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:06.110+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:51:06.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:06.110+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:51:06.460 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:06.110+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.774716+0000 front 2024-09-11T23:50:34.774705+0000 (oldest deadline 2024-09-11T23:50:59.474562+0000) 2024-09-11T23:51:06.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:06.213+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:06.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:06.213+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:06.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:06 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:06.213+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:06.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:06 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:06.593+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:06.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:06 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:06.593+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:06.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:06 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:06.794+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:06.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:06 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:06.794+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:06.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:06 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:06.794+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:06.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:06 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:06.794+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:07.207 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:07 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:07.079+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:07.207 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:07 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:07.079+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:07.207 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:06 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:06.952+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:07.208 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:06 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:06.952+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:07.208 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:06 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:06.952+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:07.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:07.093+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:51:07.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:07.093+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:51:07.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:07.093+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.774716+0000 front 2024-09-11T23:50:34.774705+0000 (oldest deadline 2024-09-11T23:50:59.474562+0000) 2024-09-11T23:51:07.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:07.234+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:07.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:07.234+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:07.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:07 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:07.234+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:07.839 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:07.561+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:07.840 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:07.561+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:07.840 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:07.561+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:41.305139+0000 front 2024-09-11T23:50:41.305054+0000 (oldest deadline 2024-09-11T23:51:06.605003+0000) 2024-09-11T23:51:08.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:07.839+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:08.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:07.839+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:08.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:07.839+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:08.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:07 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:07.839+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:08.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:08.090+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:08.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:08.090+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:08.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:08.090+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:08.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:07 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:07.911+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:08.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:07 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:07.911+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:08.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:07 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:07.911+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:08.461 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:08 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:08.044+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.368117+0000 front 2024-09-11T23:49:52.368100+0000 (oldest deadline 2024-09-11T23:50:41.471878+0000) 2024-09-11T23:51:08.516 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:08 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:08.044+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.073049+0000 front 2024-09-11T23:50:26.073064+0000 (oldest deadline 2024-09-11T23:50:51.373032+0000) 2024-09-11T23:51:08.517 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:08 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6[56152]: 2024-09-11T23:51:08.044+0000 7fb2068a8640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.774716+0000 front 2024-09-11T23:50:34.774705+0000 (oldest deadline 2024-09-11T23:50:59.474562+0000) 2024-09-11T23:51:08.517 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:08 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:08.279+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:08.517 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:08 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:08.279+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:08.517 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:08 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:08.279+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:08.806 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:08.538+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:08.807 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:08.538+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:08.807 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:08.538+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:41.305139+0000 front 2024-09-11T23:50:41.305054+0000 (oldest deadline 2024-09-11T23:51:06.605003+0000) 2024-09-11T23:51:09.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:08.807+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:09.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:08.807+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:09.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:08.807+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:09.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:08.807+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:09.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:09 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:09.067+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:09.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:09 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:09.067+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:09.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:09 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:09.067+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:09.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:08.882+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:09.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:08.882+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:09.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:08 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:08.882+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:09.538 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:09.262+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:09.538 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:09.262+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:09.538 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:09 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:09.262+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:09.844 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:09 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:09.575+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:09.844 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:09 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:09.575+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:09.844 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:09 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:09.575+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:41.305139+0000 front 2024-09-11T23:50:41.305054+0000 (oldest deadline 2024-09-11T23:51:06.605003+0000) 2024-09-11T23:51:09.935 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:09 smithi133 podman[116649]: 2024-09-11 23:51:09.539321983 +0000 UTC m=+5.769466521 container died 60edf2443a3539babe558d156ba78c7bdb681eabd3efcdbaed754b02e97f3b5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T23:51:10.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:09 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:09.844+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:10.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:09 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:09.844+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:10.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:09 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:09.844+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:10.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:09 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:09.844+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:10.209 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:09 smithi133 podman[116649]: 2024-09-11 23:51:09.936992123 +0000 UTC m=+6.167136658 container cleanup 60edf2443a3539babe558d156ba78c7bdb681eabd3efcdbaed754b02e97f3b5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T23:51:10.209 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:09 smithi133 bash[116649]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6 2024-09-11T23:51:10.209 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:10 smithi133 podman[116664]: 2024-09-11 23:51:10.17203562 +0000 UTC m=+0.635193275 container remove 60edf2443a3539babe558d156ba78c7bdb681eabd3efcdbaed754b02e97f3b5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T23:51:10.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:10 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:10.078+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:10.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:10 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:10.078+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:10.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:10 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:10.078+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:10.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:09 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:09.856+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:10.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:09 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:09.856+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:10.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:09 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:09.856+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:10.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:09 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:09.856+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:10.486 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:10 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:10.212+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:10.486 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:10 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:10.212+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:10.486 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:10 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:10.212+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:10.486 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:10 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:10.212+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:10.816 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:10 smithi133 podman[116755]: 2024-09-11 23:51:10.390157336 +0000 UTC m=+0.016025562 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:51:10.816 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:10 smithi133 podman[116755]: 2024-09-11 23:51:10.52277693 +0000 UTC m=+0.148645141 container create 0fe52304bb791ee9e2ba4d9a05ec3093a5cc6da502fc2f892b0904211354bf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-11T23:51:10.816 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:10 smithi133 podman[116755]: 2024-09-11 23:51:10.733393592 +0000 UTC m=+0.359261802 container init 0fe52304bb791ee9e2ba4d9a05ec3093a5cc6da502fc2f892b0904211354bf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T23:51:10.816 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:10 smithi133 podman[116755]: 2024-09-11 23:51:10.736841547 +0000 UTC m=+0.362709760 container start 0fe52304bb791ee9e2ba4d9a05ec3093a5cc6da502fc2f892b0904211354bf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:51:10.816 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:10 smithi133 podman[116755]: 2024-09-11 23:51:10.816694825 +0000 UTC m=+0.442563051 container attach 0fe52304bb791ee9e2ba4d9a05ec3093a5cc6da502fc2f892b0904211354bf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T23:51:10.867 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:10.577+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:10.867 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:10.577+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:10.867 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:10.577+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:41.305139+0000 front 2024-09-11T23:50:41.305054+0000 (oldest deadline 2024-09-11T23:51:06.605003+0000) 2024-09-11T23:51:11.193 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:10 smithi133 podman[116755]: 2024-09-11 23:51:10.923784439 +0000 UTC m=+0.549652654 container died 0fe52304bb791ee9e2ba4d9a05ec3093a5cc6da502fc2f892b0904211354bf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:51:11.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:10.867+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:11.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:10.867+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:11.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:10.867+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:11.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:10 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:10.867+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:11.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:11 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:11.127+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:11.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:11 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:11.127+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:11.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:11 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:11.127+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:11.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:10 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:10.890+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:11.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:10 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:10.890+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:11.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:10 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:10.890+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:11.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:10 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:10.890+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:11.444 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T23:51:11.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:11 smithi133 podman[116778]: 2024-09-11 23:51:11.428195525 +0000 UTC m=+0.495185000 container remove 0fe52304bb791ee9e2ba4d9a05ec3093a5cc6da502fc2f892b0904211354bf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T23:51:11.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:11 smithi133 systemd[1]: ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.6.service: Deactivated successfully. 2024-09-11T23:51:11.459 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:11 smithi133 systemd[1]: Stopped Ceph osd.6 for 42321b88-7095-11ef-bcea-c7b262605968. 2024-09-11T23:51:11.460 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 11 23:51:11 smithi133 systemd[1]: ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.6.service: Consumed 6min 21.234s CPU time. 2024-09-11T23:51:11.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:11 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:11.221+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:11.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:11 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:11.221+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:11.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:11 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:11.221+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:11.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:11 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:11.221+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:11.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:51:11.485 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T23:51:11.485 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T23:51:11.486 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.7 2024-09-11T23:51:11.792 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:11 smithi138 systemd[1]: Stopping Ceph osd.7 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:51:11.792 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:11.618+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:11.792 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:11.618+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:11.792 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:11.618+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:41.305139+0000 front 2024-09-11T23:50:41.305054+0000 (oldest deadline 2024-09-11T23:51:06.605003+0000) 2024-09-11T23:51:11.792 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:11.791+0000 7fb5225ec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:51:11.792 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:11.791+0000 7fb5225ec640 -1 osd.7 101 *** Got signal Terminated *** 2024-09-11T23:51:11.792 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:11.791+0000 7fb5225ec640 -1 osd.7 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:51:12.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:11.825+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:12.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:11.825+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:12.199 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:11.825+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:12.199 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:11.825+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:12.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:12 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:12.125+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:12.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:12 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:12.125+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:12.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:12 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:12.125+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:12.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:11 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:11.932+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:12.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:11 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:11.932+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:12.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:11 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:11.932+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:12.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:11 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:11.932+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:12.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:12 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:12.235+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:12.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:12 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:12.235+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:12.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:12 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:12.235+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:12.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:12 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:12.235+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:12.848 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:12 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:12.652+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:12.849 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:12 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:12.652+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:12.849 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:12 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:12.652+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:41.305139+0000 front 2024-09-11T23:50:41.305054+0000 (oldest deadline 2024-09-11T23:51:06.605003+0000) 2024-09-11T23:51:13.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:12 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:12.848+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:13.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:12 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:12.848+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:13.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:12 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:12.848+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:13.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:12 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:12.848+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:13.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:13 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:13.094+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:13.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:13 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:13.094+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:13.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:13 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:13.094+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:13.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:12 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:12.903+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:13.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:12 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:12.903+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:13.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:12 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:12.903+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:13.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:12 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:12.903+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:13.461 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:13 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:13.199+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:13.462 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:13 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:13.199+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:13.462 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:13 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:13.199+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:13.462 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:13 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:13.199+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:13.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:13 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:13.680+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:13.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:13 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:13.680+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:13.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:13 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:13.680+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:41.305139+0000 front 2024-09-11T23:50:41.305054+0000 (oldest deadline 2024-09-11T23:51:06.605003+0000) 2024-09-11T23:51:13.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:13 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:13.814+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:13.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:13 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:13.814+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:13.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:13 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:13.814+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:13.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:13 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:13.814+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:14.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:14 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:14.116+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:14.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:14 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:14.116+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:14.290 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:14 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:14.116+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:14.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:13 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:13.866+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:14.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:13 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:13.866+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:14.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:13 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:13.866+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:14.291 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:13 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:13.866+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:14.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:14 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:14.199+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:14.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:14 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:14.199+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:14.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:14 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:14.199+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:14.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:14 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:14.199+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:14.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:14 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:14.199+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:14.949 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:14 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:14.661+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:14.949 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:14 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:14.661+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:14.949 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:14 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:14.661+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:41.305139+0000 front 2024-09-11T23:50:41.305054+0000 (oldest deadline 2024-09-11T23:51:06.605003+0000) 2024-09-11T23:51:14.950 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:14 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:14.820+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:14.950 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:14 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:14.820+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:14.950 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:14 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:14.820+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:14.950 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:14 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:14.820+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:14.950 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:14 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:14.820+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:15.146 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:14 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:14.885+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:15.146 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:14 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:14.885+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:15.146 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:14 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:14.885+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:15.146 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:14 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:14.885+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:15.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:15 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:15.183+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:15.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:15 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:15.183+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:15.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:15 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:15.183+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:15.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:15 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:15.183+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:15.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:15 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:15.183+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:15.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:15 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:15.145+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:15.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:15 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:15.145+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:15.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:15 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:15.145+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:15.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:15 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:15.688+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:15.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:15 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:15.688+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:15.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:15 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:15.688+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:41.305139+0000 front 2024-09-11T23:50:41.305054+0000 (oldest deadline 2024-09-11T23:51:06.605003+0000) 2024-09-11T23:51:15.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:15 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:15.770+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:15.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:15 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:15.770+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:15.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:15 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:15.770+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:15.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:15 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:15.770+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:15.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:15 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:15.770+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:16.147 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:15 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:15.847+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:16.147 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:15 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:15.847+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:16.148 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:15 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:15.847+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:16.148 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:15 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:15.847+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:16.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:16 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:16.148+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:16.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:16 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:16.148+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:16.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:16 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:16.148+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:16.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:16 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:16.148+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.833082+0000 front 2024-09-11T23:50:51.833068+0000 (oldest deadline 2024-09-11T23:51:15.332875+0000) 2024-09-11T23:51:16.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:16 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:16.216+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:16.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:16 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:16.216+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:16.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:16 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:16.216+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:16.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:16 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:16.216+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:16.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:16 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:16.216+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:16.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:16 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:16.648+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.702881+0000 front 2024-09-11T23:50:26.702917+0000 (oldest deadline 2024-09-11T23:50:49.602995+0000) 2024-09-11T23:51:16.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:16 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:16.648+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.303951+0000 front 2024-09-11T23:50:34.303976+0000 (oldest deadline 2024-09-11T23:50:59.604035+0000) 2024-09-11T23:51:16.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:16 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7[52960]: 2024-09-11T23:51:16.648+0000 7fb51abe9640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:41.305139+0000 front 2024-09-11T23:50:41.305054+0000 (oldest deadline 2024-09-11T23:51:06.605003+0000) 2024-09-11T23:51:16.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:16 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:16.764+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:16.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:16 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:16.764+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:16.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:16 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:16.764+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:16.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:16 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:16.764+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:16.949 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:16 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:16.764+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:17.127 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:16 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:16.814+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:17.127 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:16 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:16.814+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:17.128 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:16 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:16.814+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:17.128 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:16 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:16.814+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:17.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:17 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:17.127+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:17.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:17 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:17.127+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:17.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:17 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:17.127+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:17.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:17 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:17.127+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.833082+0000 front 2024-09-11T23:50:51.833068+0000 (oldest deadline 2024-09-11T23:51:15.332875+0000) 2024-09-11T23:51:17.698 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:17 smithi138 podman[71410]: 2024-09-11 23:51:17.381404443 +0000 UTC m=+5.701644367 container died c1d761cfd9fcc5729526b5e93de5d4ee17a8290d50e5c10447cf8198d60deebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:51:17.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:17 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:17.252+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:17.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:17 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:17.252+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:17.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:17 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:17.252+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:17.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:17 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:17.252+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:17.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:17 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:17.252+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:18.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:17 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:17.775+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:18.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:17 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:17.775+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:18.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:17 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:17.775+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:18.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:17 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:17.775+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:18.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:17 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:17.775+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:18.198 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:17 smithi138 podman[71410]: 2024-09-11 23:51:17.700424817 +0000 UTC m=+6.020664732 container cleanup c1d761cfd9fcc5729526b5e93de5d4ee17a8290d50e5c10447cf8198d60deebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T23:51:18.198 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:17 smithi138 bash[71410]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7 2024-09-11T23:51:18.198 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:17 smithi138 podman[71438]: 2024-09-11 23:51:17.918764306 +0000 UTC m=+0.538122660 container remove c1d761cfd9fcc5729526b5e93de5d4ee17a8290d50e5c10447cf8198d60deebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T23:51:18.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:17 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:17.734+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:18.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:17 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:17.734+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:18.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:17 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:17.734+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:18.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:17 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:17.734+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:18.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:17 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:17.734+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:18.509 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:18.150+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:18.509 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:18.150+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:18.509 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:18.150+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:18.510 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:18.150+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.833082+0000 front 2024-09-11T23:50:51.833068+0000 (oldest deadline 2024-09-11T23:51:15.332875+0000) 2024-09-11T23:51:18.654 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:18 smithi138 podman[71528]: 2024-09-11 23:51:18.253716768 +0000 UTC m=+0.015773013 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:51:18.654 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:18 smithi138 podman[71528]: 2024-09-11 23:51:18.369751317 +0000 UTC m=+0.131807548 container create 543017d8fbf37fd6e4643393e7183822edaec74bc2bb7f21bdd9be92f0715d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-11T23:51:18.654 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:18 smithi138 podman[71528]: 2024-09-11 23:51:18.572034388 +0000 UTC m=+0.334090626 container init 543017d8fbf37fd6e4643393e7183822edaec74bc2bb7f21bdd9be92f0715d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T23:51:18.654 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:18 smithi138 podman[71528]: 2024-09-11 23:51:18.575369522 +0000 UTC m=+0.337425755 container start 543017d8fbf37fd6e4643393e7183822edaec74bc2bb7f21bdd9be92f0715d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-11T23:51:18.654 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:18 smithi138 podman[71528]: 2024-09-11 23:51:18.655139495 +0000 UTC m=+0.417195739 container attach 543017d8fbf37fd6e4643393e7183822edaec74bc2bb7f21bdd9be92f0715d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-11T23:51:18.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:18 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:18.272+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:18.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:18 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:18.272+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:18.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:18 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:18.272+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:18.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:18 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:18.272+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:18.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:18 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:18.272+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:18.948 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 11 23:51:18 smithi138 podman[71528]: 2024-09-11 23:51:18.761871447 +0000 UTC m=+0.523927680 container died 543017d8fbf37fd6e4643393e7183822edaec74bc2bb7f21bdd9be92f0715d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T23:51:18.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:18 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:18.737+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:18.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:18 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:18.737+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:18.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:18 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:18.737+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:18.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:18 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:18.737+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:18.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:18 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:18.737+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:19.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:18.807+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:19.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:18.807+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:19.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:18.807+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:19.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:18.807+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:19.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:18 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:18.807+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:19.340 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T23:51:19.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:51:19.379 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T23:51:19.379 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-11T23:51:19.379 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.8 2024-09-11T23:51:19.432 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:19.175+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:19.432 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:19.175+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:19.432 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:19.175+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:19.432 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:19.175+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.833082+0000 front 2024-09-11T23:50:51.833068+0000 (oldest deadline 2024-09-11T23:51:15.332875+0000) 2024-09-11T23:51:19.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:19 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:19.247+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:19.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:19 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:19.247+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:19.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:19 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:19.247+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:19.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:19 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:19.247+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:19.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:19 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:19.247+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:19.790 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:19 smithi178 systemd[1]: Stopping Ceph osd.8 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:51:19.790 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:19.675+0000 7fa787b94640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:51:19.790 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:19.675+0000 7fa787b94640 -1 osd.8 101 *** Got signal Terminated *** 2024-09-11T23:51:19.790 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:19.675+0000 7fa787b94640 -1 osd.8 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:51:20.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:19 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:19.736+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:20.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:19 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:19.736+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:20.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:19 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:19.736+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:20.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:19 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:19.736+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:20.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:19 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:19.736+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:20.203 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:19.818+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:20.203 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:19.818+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:20.203 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:19.818+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:20.203 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:19.818+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:20.203 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:19 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:19.818+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:20.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:20 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:20.203+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:20.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:20 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:20.203+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:20.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:20 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:20.203+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:20.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:20 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:20.203+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.833082+0000 front 2024-09-11T23:50:51.833068+0000 (oldest deadline 2024-09-11T23:51:15.332875+0000) 2024-09-11T23:51:20.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:20 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:20.228+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:20.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:20 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:20.228+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:20.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:20 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:20.228+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:20.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:20 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:20.228+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:20.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:20 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:20.228+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:21.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:20 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:20.713+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:21.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:20 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:20.713+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:21.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:20 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:20.713+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:21.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:20 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:20.713+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:21.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:20 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:20.713+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:21.232 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:20 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:20.791+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:21.232 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:20 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:20.791+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:21.232 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:20 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:20.791+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:21.232 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:20 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:20.791+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:21.233 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:20 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:20.791+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:21.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:21 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:21.232+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:21.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:21 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:21.232+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:21.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:21 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:21.232+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:21.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:21 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:21.232+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.833082+0000 front 2024-09-11T23:50:51.833068+0000 (oldest deadline 2024-09-11T23:51:15.332875+0000) 2024-09-11T23:51:21.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:21 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:21.241+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:21.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:21 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:21.241+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:21.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:21 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:21.241+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:21.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:21 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:21.241+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:21.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:21 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:21.241+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:21.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:21 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:21.688+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:21.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:21 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:21.688+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:21.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:21 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:21.688+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:21.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:21 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:21.688+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:21.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:21 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:21.688+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:22.261 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:21 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:21.833+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:22.261 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:21 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:21.833+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:22.261 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:21 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:21.833+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:22.261 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:21 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:21.833+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:22.261 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:21 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:21.833+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:22.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:22 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:22.260+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:22.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:22 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:22.260+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:22.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:22 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:22.260+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:22.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:22 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:22.260+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.833082+0000 front 2024-09-11T23:50:51.833068+0000 (oldest deadline 2024-09-11T23:51:15.332875+0000) 2024-09-11T23:51:22.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:22 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:22.260+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.534324+0000 front 2024-09-11T23:51:00.534250+0000 (oldest deadline 2024-09-11T23:51:21.634427+0000) 2024-09-11T23:51:22.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:22 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:22.249+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:22.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:22 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:22.249+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:22.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:22 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:22.249+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:22.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:22 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:22.249+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:22.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:22 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:22.249+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:23.197 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:22 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:22.712+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:23.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:22 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:22.712+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:23.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:22 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:22.712+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:23.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:22 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:22.712+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:23.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:22 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:22.712+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:23.235 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:22 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:22.793+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:23.235 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:22 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:22.793+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:23.235 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:22 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:22.793+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:23.235 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:22 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:22.793+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:23.235 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:22 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:22.793+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:23.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:23.236+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:23.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:23.236+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:23.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:23.236+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:23.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:23.236+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.833082+0000 front 2024-09-11T23:50:51.833068+0000 (oldest deadline 2024-09-11T23:51:15.332875+0000) 2024-09-11T23:51:23.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:23.236+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.534324+0000 front 2024-09-11T23:51:00.534250+0000 (oldest deadline 2024-09-11T23:51:21.634427+0000) 2024-09-11T23:51:23.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:23 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:23.259+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:23.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:23 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:23.259+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:23.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:23 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:23.259+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:23.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:23 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:23.259+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:23.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:23 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:23.259+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:24.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:23 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:23.756+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:24.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:23 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:23.756+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:24.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:23 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:23.756+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:24.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:23 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:23.756+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:24.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:23 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:23.756+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:24.200 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:23.817+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:24.200 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:23.817+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:24.200 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:23.817+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:24.200 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:23.817+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:24.200 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:23.817+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:24.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:24 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:24.200+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.027385+0000 front 2024-09-11T23:50:27.027520+0000 (oldest deadline 2024-09-11T23:50:54.027991+0000) 2024-09-11T23:51:24.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:24 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:24.200+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:35.728980+0000 front 2024-09-11T23:50:35.729093+0000 (oldest deadline 2024-09-11T23:50:58.629079+0000) 2024-09-11T23:51:24.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:24 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:24.200+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:43.830567+0000 front 2024-09-11T23:50:43.830582+0000 (oldest deadline 2024-09-11T23:51:07.930652+0000) 2024-09-11T23:51:24.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:24 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:24.200+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.833082+0000 front 2024-09-11T23:50:51.833068+0000 (oldest deadline 2024-09-11T23:51:15.332875+0000) 2024-09-11T23:51:24.540 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:24 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8[54292]: 2024-09-11T23:51:24.200+0000 7fa784199640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.534324+0000 front 2024-09-11T23:51:00.534250+0000 (oldest deadline 2024-09-11T23:51:21.634427+0000) 2024-09-11T23:51:24.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:24 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:24.275+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:24.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:24 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:24.275+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:24.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:24 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:24.275+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:24.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:24 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:24.275+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:24.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:24 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:24.275+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:25.197 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:24 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:24.715+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:25.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:24 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:24.715+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:25.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:24 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:24.715+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:25.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:24 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:24.715+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:25.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:24 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:24.715+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:25.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:24 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:24.809+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:25.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:24 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:24.809+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:25.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:24 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:24.809+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:25.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:24 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:24.809+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:25.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:24 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:24.809+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:25.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:24 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:24.809+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:25.600 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:25 smithi178 podman[72848]: 2024-09-11 23:51:25.312633489 +0000 UTC m=+5.732510119 container died 6274f2da20b7d373d99bc917706ea1955eaa4925a10481f9af1b57070325d4db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:51:25.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:25 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:25.268+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:25.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:25 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:25.268+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:25.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:25 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:25.268+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:25.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:25 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:25.268+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:25.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:25 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:25.268+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:25.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:25 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:25.268+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.448038+0000 front 2024-09-11T23:51:00.448066+0000 (oldest deadline 2024-09-11T23:51:25.148041+0000) 2024-09-11T23:51:25.884 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:25 smithi178 podman[72848]: 2024-09-11 23:51:25.692252059 +0000 UTC m=+6.112128693 container cleanup 6274f2da20b7d373d99bc917706ea1955eaa4925a10481f9af1b57070325d4db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:51:25.884 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:25 smithi178 bash[72848]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8 2024-09-11T23:51:25.885 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:25 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:25.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:25.885 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:25 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:25.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:25.885 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:25 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:25.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:25.885 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:25 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:25.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:25.885 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:25 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:25.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:25.885 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:25 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:25.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:26.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:25 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:25.756+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:26.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:25 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:25.756+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:26.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:25 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:25.756+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:26.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:25 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:25.756+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:26.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:25 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:25.756+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:26.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:25 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:25.756+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:26.289 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:25 smithi178 podman[72865]: 2024-09-11 23:51:25.885522543 +0000 UTC m=+0.574681451 container remove 6274f2da20b7d373d99bc917706ea1955eaa4925a10481f9af1b57070325d4db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-11T23:51:26.604 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:26 smithi178 podman[72957]: 2024-09-11 23:51:26.303127996 +0000 UTC m=+0.111286904 container create 2821ec6f8a85d312f44a73946b1b2107b54e1ab73b3b5039ec4428218ab7b41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:51:26.604 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:26 smithi178 podman[72957]: 2024-09-11 23:51:26.207642182 +0000 UTC m=+0.015801089 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:51:26.604 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:26 smithi178 podman[72957]: 2024-09-11 23:51:26.523064414 +0000 UTC m=+0.331223323 container init 2821ec6f8a85d312f44a73946b1b2107b54e1ab73b3b5039ec4428218ab7b41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T23:51:26.604 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:26 smithi178 podman[72957]: 2024-09-11 23:51:26.526491679 +0000 UTC m=+0.334650593 container start 2821ec6f8a85d312f44a73946b1b2107b54e1ab73b3b5039ec4428218ab7b41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T23:51:26.604 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:26 smithi178 podman[72957]: 2024-09-11 23:51:26.605483787 +0000 UTC m=+0.413642708 container attach 2821ec6f8a85d312f44a73946b1b2107b54e1ab73b3b5039ec4428218ab7b41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T23:51:26.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:26 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:26.274+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:26.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:26 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:26.274+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:26.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:26 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:26.274+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:26.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:26 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:26.274+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:26.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:26 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:26.274+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:26.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:26 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:26.274+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.448038+0000 front 2024-09-11T23:51:00.448066+0000 (oldest deadline 2024-09-11T23:51:25.148041+0000) 2024-09-11T23:51:27.025 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 23:51:26 smithi178 podman[72957]: 2024-09-11 23:51:26.719646042 +0000 UTC m=+0.527804956 container died 2821ec6f8a85d312f44a73946b1b2107b54e1ab73b3b5039ec4428218ab7b41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-11T23:51:27.025 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:26 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:26.781+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:27.025 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:26 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:26.781+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:27.025 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:26 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:26.781+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:27.025 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:26 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:26.781+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:27.025 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:26 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:26.781+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:27.025 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:26 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:26.781+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:27.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:26 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:26.730+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:27.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:26 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:26.730+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:27.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:26 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:26.730+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:27.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:26 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:26.730+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:27.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:26 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:26.730+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:27.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:26 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:26.730+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:27.345 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.8.service' 2024-09-11T23:51:27.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:51:27.388 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-11T23:51:27.388 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-11T23:51:27.388 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.9 2024-09-11T23:51:27.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:27 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:27.273+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:27.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:27 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:27.273+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:27.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:27 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:27.273+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:27.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:27 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:27.273+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:27.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:27 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:27.273+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:27.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:27 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:27.273+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.448038+0000 front 2024-09-11T23:51:00.448066+0000 (oldest deadline 2024-09-11T23:51:25.148041+0000) 2024-09-11T23:51:27.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:27 smithi133 systemd[1]: Stopping Ceph osd.9 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:51:28.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:27 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:27.805+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:28.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:27 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:27.805+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:28.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:27 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:27.805+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:28.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:27 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:27.805+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:28.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:27 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:27.805+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:28.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:27 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:27.805+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:28.197 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:27 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:27.779+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:28.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:27 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:27.779+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:28.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:27 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:27.779+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:28.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:27 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:27.779+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:28.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:27 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:27.779+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:28.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:27 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:27.779+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:28.209 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:27 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:27.721+0000 7f52a82b0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:51:28.209 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:27 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:27.721+0000 7f52a82b0640 -1 osd.9 101 *** Got signal Terminated *** 2024-09-11T23:51:28.209 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:27 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:27.721+0000 7f52a82b0640 -1 osd.9 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:51:28.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:28 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:28.228+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:28.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:28 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:28.228+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:28.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:28 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:28.228+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:28.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:28 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:28.228+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:28.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:28 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:28.228+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:28.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:28 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:28.228+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.448038+0000 front 2024-09-11T23:51:00.448066+0000 (oldest deadline 2024-09-11T23:51:25.148041+0000) 2024-09-11T23:51:29.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:28 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:28.782+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:29.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:28 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:28.782+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:29.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:28 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:28.782+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:29.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:28 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:28.782+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:29.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:28 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:28.782+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:29.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:28 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:28.782+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:29.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:28 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:28.829+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:29.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:28 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:28.829+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:29.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:28 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:28.829+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:29.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:28 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:28.829+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:29.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:28 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:28.829+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:29.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:28 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:28.829+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:29.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:29 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:29.204+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:29.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:29 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:29.204+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:29.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:29 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:29.204+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:29.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:29 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:29.204+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:29.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:29 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:29.204+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:29.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:29 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:29.204+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.448038+0000 front 2024-09-11T23:51:00.448066+0000 (oldest deadline 2024-09-11T23:51:25.148041+0000) 2024-09-11T23:51:30.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:29 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:29.741+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:30.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:29 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:29.741+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:30.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:29 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:29.741+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:30.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:29 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:29.741+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:30.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:29 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:29.741+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:30.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:29 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:29.741+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:30.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:29 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:29.865+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:30.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:29 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:29.865+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:30.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:29 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:29.865+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:30.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:29 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:29.865+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:30.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:29 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:29.865+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:30.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:29 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:29.865+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:30.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:30.211+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:30.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:30.211+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:30.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:30.211+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:30.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:30.211+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:30.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:30.211+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:30.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:30 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:30.211+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.448038+0000 front 2024-09-11T23:51:00.448066+0000 (oldest deadline 2024-09-11T23:51:25.148041+0000) 2024-09-11T23:51:31.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:30 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:30.783+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:31.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:30 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:30.783+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:31.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:30 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:30.783+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:31.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:30 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:30.783+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:31.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:30 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:30.783+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:31.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:30 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:30.783+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:31.197 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:30 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:30.862+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:31.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:30 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:30.862+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:31.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:30 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:30.862+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:31.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:30 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:30.862+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:31.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:30 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:30.862+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:31.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:30 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:30.862+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:31.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:31 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:31.224+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:31.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:31 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:31.224+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:31.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:31 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:31.224+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:31.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:31 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:31.224+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:31.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:31 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:31.224+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:31.710 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:31 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:31.224+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.448038+0000 front 2024-09-11T23:51:00.448066+0000 (oldest deadline 2024-09-11T23:51:25.148041+0000) 2024-09-11T23:51:32.197 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:31 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:31.815+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:32.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:31 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:31.815+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:32.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:31 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:31.815+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:32.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:31 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:31.815+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:32.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:31 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:31.815+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:32.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:31 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:31.815+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:32.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:31 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:31.827+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:32.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:31 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:31.827+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:32.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:31 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:31.827+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:32.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:31 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:31.827+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:32.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:31 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:31.827+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:32.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:31 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:31.827+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:32.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:32 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:32.184+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:53.036346+0000 front 2024-09-11T23:49:53.036366+0000 (oldest deadline 2024-09-11T23:50:42.442692+0000) 2024-09-11T23:51:32.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:32 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:32.184+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:25.343548+0000 front 2024-09-11T23:50:25.343462+0000 (oldest deadline 2024-09-11T23:50:53.543997+0000) 2024-09-11T23:51:32.459 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:32 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:32.184+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:33.544645+0000 front 2024-09-11T23:50:33.544550+0000 (oldest deadline 2024-09-11T23:50:58.844382+0000) 2024-09-11T23:51:32.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:32 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:32.184+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.745282+0000 front 2024-09-11T23:50:44.745328+0000 (oldest deadline 2024-09-11T23:51:09.345832+0000) 2024-09-11T23:51:32.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:32 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:32.184+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:49.346645+0000 front 2024-09-11T23:50:49.346775+0000 (oldest deadline 2024-09-11T23:51:14.046583+0000) 2024-09-11T23:51:32.460 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:32 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9[63023]: 2024-09-11T23:51:32.184+0000 7f52a08ad640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.448038+0000 front 2024-09-11T23:51:00.448066+0000 (oldest deadline 2024-09-11T23:51:25.148041+0000) 2024-09-11T23:51:33.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:32 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:32.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:33.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:32 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:32.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:33.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:32 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:32.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:33.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:32 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:32.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:33.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:32 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:32.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:33.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:32 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:32.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:33.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:32 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:32.770+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:33.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:32 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:32.770+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:33.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:32 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:32.770+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:33.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:32 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:32.770+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:33.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:32 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:32.770+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:33.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:32 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:32.770+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:33.646 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:33 smithi133 podman[116868]: 2024-09-11 23:51:33.349032797 +0000 UTC m=+5.754922841 container died fabafb439d44852bb69b184fe01e6e7ee09dffcc50b4c3a8f0d408f5b47c2b01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-11T23:51:33.959 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:33 smithi133 podman[116868]: 2024-09-11 23:51:33.813368782 +0000 UTC m=+6.219258867 container cleanup fabafb439d44852bb69b184fe01e6e7ee09dffcc50b4c3a8f0d408f5b47c2b01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T23:51:33.959 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:33 smithi133 bash[116868]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9 2024-09-11T23:51:34.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:33 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:33.763+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:34.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:33 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:33.763+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:34.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:33 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:33.763+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:34.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:33 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:33.763+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:34.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:33 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:33.763+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:34.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:33 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:33.763+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:34.197 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:33 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:33.768+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:34.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:33 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:33.768+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:34.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:33 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:33.768+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:34.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:33 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:33.768+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:34.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:33 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:33.768+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:34.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:33 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:33.768+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:34.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:33 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:33.768+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-09-11T23:51:08.212116+0000 front 2024-09-11T23:51:08.212174+0000 (oldest deadline 2024-09-11T23:51:32.912060+0000) 2024-09-11T23:51:34.421 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:34 smithi133 podman[116884]: 2024-09-11 23:51:34.08119947 +0000 UTC m=+0.731260942 container remove fabafb439d44852bb69b184fe01e6e7ee09dffcc50b4c3a8f0d408f5b47c2b01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T23:51:34.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:34 smithi133 podman[116970]: 2024-09-11 23:51:34.325633581 +0000 UTC m=+0.015648174 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:51:34.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:34 smithi133 podman[116970]: 2024-09-11 23:51:34.440192214 +0000 UTC m=+0.130206804 container create 91f0b107ace65ff9369d35a5e9245ba59d883a9c2e920c4492284d128f183907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T23:51:34.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:34 smithi133 podman[116970]: 2024-09-11 23:51:34.667433053 +0000 UTC m=+0.357447644 container init 91f0b107ace65ff9369d35a5e9245ba59d883a9c2e920c4492284d128f183907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9-deactivate, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T23:51:34.709 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:34 smithi133 podman[116970]: 2024-09-11 23:51:34.67060247 +0000 UTC m=+0.360617067 container start 91f0b107ace65ff9369d35a5e9245ba59d883a9c2e920c4492284d128f183907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:51:35.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:34.730+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:35.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:34.730+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:35.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:34.730+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:35.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:34.730+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:35.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:34.730+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:35.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:34 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:34.730+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:35.194 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:34 smithi133 podman[116970]: 2024-09-11 23:51:34.759153807 +0000 UTC m=+0.449168395 container attach 91f0b107ace65ff9369d35a5e9245ba59d883a9c2e920c4492284d128f183907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T23:51:35.195 INFO:journalctl@ceph.osd.9.smithi133.stdout:Sep 11 23:51:34 smithi133 podman[116970]: 2024-09-11 23:51:34.857191058 +0000 UTC m=+0.547205649 container died 91f0b107ace65ff9369d35a5e9245ba59d883a9c2e920c4492284d128f183907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:51:35.197 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:34.738+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:35.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:34.738+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:35.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:34.738+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:35.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:34.738+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:35.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:34.738+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:35.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:34.738+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:35.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:34 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:34.738+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-09-11T23:51:08.212116+0000 front 2024-09-11T23:51:08.212174+0000 (oldest deadline 2024-09-11T23:51:32.912060+0000) 2024-09-11T23:51:35.592 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.9.service' 2024-09-11T23:51:35.630 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:51:35.630 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-11T23:51:35.630 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-11T23:51:35.631 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.10 2024-09-11T23:51:35.936 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:35 smithi138 systemd[1]: Stopping Ceph osd.10 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:51:35.936 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:35 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:35.710+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:35.936 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:35 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:35.710+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:35.936 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:35 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:35.710+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:35.936 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:35 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:35.710+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:35.936 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:35 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:35.710+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:35.936 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:35 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:35.710+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:35.936 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:35 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:35.710+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-09-11T23:51:08.212116+0000 front 2024-09-11T23:51:08.212174+0000 (oldest deadline 2024-09-11T23:51:32.912060+0000) 2024-09-11T23:51:35.936 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:35 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:35.936+0000 7f88af5e3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:51:35.936 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:35 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:35.936+0000 7f88af5e3640 -1 osd.10 101 *** Got signal Terminated *** 2024-09-11T23:51:35.936 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:35 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:35.936+0000 7f88af5e3640 -1 osd.10 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:51:36.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:35 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:35.754+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:36.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:35 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:35.754+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:36.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:35 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:35.754+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:36.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:35 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:35.754+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:36.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:35 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:35.754+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:36.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:35 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:35.754+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:36.947 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:36 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:36.675+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:36.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:36 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:36.675+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:36.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:36 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:36.675+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:36.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:36 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:36.675+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:36.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:36 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:36.675+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:36.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:36 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:36.675+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:36.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:36 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:36.675+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-09-11T23:51:08.212116+0000 front 2024-09-11T23:51:08.212174+0000 (oldest deadline 2024-09-11T23:51:32.912060+0000) 2024-09-11T23:51:37.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:36 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:36.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:37.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:36 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:36.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:37.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:36 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:36.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:37.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:36 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:36.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:37.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:36 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:36.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:37.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:36 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:36.784+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:38.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:37 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:37.748+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:38.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:37 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:37.748+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:38.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:37 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:37.748+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:38.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:37 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:37.748+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:38.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:37 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:37.748+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:38.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:37 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:37.748+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:38.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:37 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:37.720+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:38.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:37 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:37.720+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:38.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:37 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:37.720+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:38.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:37 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:37.720+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:38.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:37 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:37.720+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:38.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:37 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:37.720+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:38.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:37 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:37.720+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-09-11T23:51:08.212116+0000 front 2024-09-11T23:51:08.212174+0000 (oldest deadline 2024-09-11T23:51:32.912060+0000) 2024-09-11T23:51:38.947 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:38 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:38.683+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:38.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:38 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:38.683+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:38.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:38 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:38.683+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:38.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:38 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:38.683+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:38.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:38 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:38.683+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:38.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:38 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:38.683+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:38.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:38 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:38.683+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-09-11T23:51:08.212116+0000 front 2024-09-11T23:51:08.212174+0000 (oldest deadline 2024-09-11T23:51:32.912060+0000) 2024-09-11T23:51:39.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:38 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:38.749+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:39.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:38 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:38.749+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:39.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:38 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:38.749+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:39.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:38 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:38.749+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:39.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:38 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:38.749+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:39.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:38 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:38.749+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:39.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:39 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:39.690+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:39.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:39 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:39.690+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:39.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:39 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:39.690+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:39.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:39 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:39.690+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:39.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:39 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:39.690+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:39.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:39 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:39.690+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:39.948 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:39 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:39.690+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-09-11T23:51:08.212116+0000 front 2024-09-11T23:51:08.212174+0000 (oldest deadline 2024-09-11T23:51:32.912060+0000) 2024-09-11T23:51:40.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:39 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:39.750+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:40.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:39 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:39.750+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:40.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:39 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:39.750+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:40.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:39 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:39.750+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:40.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:39 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:39.750+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:40.042 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:39 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:39.750+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:41.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:40 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:40.725+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:41.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:40 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:40.725+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:41.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:40 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:40.725+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:41.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:40 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:40.725+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:41.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:40 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:40.725+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:41.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:40 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:40.725+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:41.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:40 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:40.723+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:52.100286+0000 front 2024-09-11T23:49:52.100223+0000 (oldest deadline 2024-09-11T23:50:42.904687+0000) 2024-09-11T23:51:41.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:40 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:40.723+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:27.306898+0000 front 2024-09-11T23:50:27.306928+0000 (oldest deadline 2024-09-11T23:50:49.607041+0000) 2024-09-11T23:51:41.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:40 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:40.723+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:34.307888+0000 front 2024-09-11T23:50:34.307945+0000 (oldest deadline 2024-09-11T23:50:59.607776+0000) 2024-09-11T23:51:41.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:40 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:40.723+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:39.608168+0000 front 2024-09-11T23:50:39.608151+0000 (oldest deadline 2024-09-11T23:51:06.008830+0000) 2024-09-11T23:51:41.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:40 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:40.723+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:48.909636+0000 front 2024-09-11T23:50:48.909661+0000 (oldest deadline 2024-09-11T23:51:14.209728+0000) 2024-09-11T23:51:41.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:40 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:40.723+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:50:59.510917+0000 front 2024-09-11T23:50:59.510802+0000 (oldest deadline 2024-09-11T23:51:25.410755+0000) 2024-09-11T23:51:41.198 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:40 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10[59858]: 2024-09-11T23:51:40.723+0000 7f88abbe8640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-09-11T23:51:08.212116+0000 front 2024-09-11T23:51:08.212174+0000 (oldest deadline 2024-09-11T23:51:32.912060+0000) 2024-09-11T23:51:41.810 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:41 smithi138 podman[71630]: 2024-09-11 23:51:41.52635701 +0000 UTC m=+5.693105707 container died 01642201aab59b5a6d585e3b1d765128720c39f61b3ca80ce49c58a8a22eafb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T23:51:42.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:41 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:41.762+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:42.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:41 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:41.762+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:42.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:41 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:41.762+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:42.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:41 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:41.762+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:42.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:41 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:41.762+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:42.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:41 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:41.762+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:42.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:41 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:41.762+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-11T23:51:15.267443+0000 front 2024-09-11T23:51:15.267551+0000 (oldest deadline 2024-09-11T23:51:41.667756+0000) 2024-09-11T23:51:42.143 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:41 smithi138 podman[71630]: 2024-09-11 23:51:41.910490686 +0000 UTC m=+6.077239383 container cleanup 01642201aab59b5a6d585e3b1d765128720c39f61b3ca80ce49c58a8a22eafb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:51:42.143 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:41 smithi138 bash[71630]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10 2024-09-11T23:51:42.447 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:42 smithi138 podman[71649]: 2024-09-11 23:51:42.144875474 +0000 UTC m=+0.619293422 container remove 01642201aab59b5a6d585e3b1d765128720c39f61b3ca80ce49c58a8a22eafb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T23:51:42.768 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:42 smithi138 podman[71737]: 2024-09-11 23:51:42.454124368 +0000 UTC m=+0.110976243 container create 75ca90177084cd3db7f517ee7e3155766ad08af97196f1c45602916424890b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T23:51:42.768 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:42 smithi138 podman[71737]: 2024-09-11 23:51:42.363333621 +0000 UTC m=+0.020185510 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:51:42.768 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:42 smithi138 podman[71737]: 2024-09-11 23:51:42.673633056 +0000 UTC m=+0.330484938 container init 75ca90177084cd3db7f517ee7e3155766ad08af97196f1c45602916424890b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T23:51:42.768 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:42 smithi138 podman[71737]: 2024-09-11 23:51:42.677062075 +0000 UTC m=+0.333913953 container start 75ca90177084cd3db7f517ee7e3155766ad08af97196f1c45602916424890b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T23:51:42.769 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:42 smithi138 podman[71737]: 2024-09-11 23:51:42.769167576 +0000 UTC m=+0.426019459 container attach 75ca90177084cd3db7f517ee7e3155766ad08af97196f1c45602916424890b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T23:51:43.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:42.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:43.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:42.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:43.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:42.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:43.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:42.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:43.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:42.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:43.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:42.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:43.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:42 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:42.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-11T23:51:15.267443+0000 front 2024-09-11T23:51:15.267551+0000 (oldest deadline 2024-09-11T23:51:41.667756+0000) 2024-09-11T23:51:43.150 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 11 23:51:42 smithi138 podman[71737]: 2024-09-11 23:51:42.865058805 +0000 UTC m=+0.521910679 container died 75ca90177084cd3db7f517ee7e3155766ad08af97196f1c45602916424890b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T23:51:43.431 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.10.service' 2024-09-11T23:51:43.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:51:43.470 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-11T23:51:43.470 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-11T23:51:43.471 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.11 2024-09-11T23:51:43.778 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:43 smithi178 systemd[1]: Stopping Ceph osd.11 for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:51:43.778 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:43.684+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:43.778 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:43.684+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:43.778 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:43.684+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:43.778 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:43.684+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:43.778 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:43.684+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:43.778 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:43.684+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:43.778 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:43.684+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-11T23:51:15.267443+0000 front 2024-09-11T23:51:15.267551+0000 (oldest deadline 2024-09-11T23:51:41.667756+0000) 2024-09-11T23:51:43.779 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:43.778+0000 7fd5f0440640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:51:43.779 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:43.778+0000 7fd5f0440640 -1 osd.11 101 *** Got signal Terminated *** 2024-09-11T23:51:43.779 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:43 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:43.778+0000 7fd5f0440640 -1 osd.11 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:51:45.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:44.683+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:45.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:44.683+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:45.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:44.683+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:45.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:44.683+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:45.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:44.683+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:45.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:44.683+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:45.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:44 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:44.683+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-11T23:51:15.267443+0000 front 2024-09-11T23:51:15.267551+0000 (oldest deadline 2024-09-11T23:51:41.667756+0000) 2024-09-11T23:51:46.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:45.717+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:46.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:45.717+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:46.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:45.717+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:46.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:45.717+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:46.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:45.717+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:46.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:45.717+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:46.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:45 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:45.717+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-11T23:51:15.267443+0000 front 2024-09-11T23:51:15.267551+0000 (oldest deadline 2024-09-11T23:51:41.667756+0000) 2024-09-11T23:51:47.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:46 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:46.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:47.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:46 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:46.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:47.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:46 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:46.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:47.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:46 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:46.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:47.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:46 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:46.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:47.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:46 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:46.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:47.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:46 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:46.720+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-11T23:51:15.267443+0000 front 2024-09-11T23:51:15.267551+0000 (oldest deadline 2024-09-11T23:51:41.667756+0000) 2024-09-11T23:51:48.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:47 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:47.751+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-09-11T23:49:51.552622+0000 front 2024-09-11T23:49:51.552586+0000 (oldest deadline 2024-09-11T23:50:40.657377+0000) 2024-09-11T23:51:48.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:47 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:47.751+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-11T23:50:26.359495+0000 front 2024-09-11T23:50:26.359561+0000 (oldest deadline 2024-09-11T23:50:50.459588+0000) 2024-09-11T23:51:48.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:47 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:47.751+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T23:50:36.261119+0000 front 2024-09-11T23:50:36.261107+0000 (oldest deadline 2024-09-11T23:51:00.861711+0000) 2024-09-11T23:51:48.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:47 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:47.751+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-09-11T23:50:44.863390+0000 front 2024-09-11T23:50:44.863407+0000 (oldest deadline 2024-09-11T23:51:08.963256+0000) 2024-09-11T23:51:48.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:47 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:47.751+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-11T23:50:51.864319+0000 front 2024-09-11T23:50:51.864259+0000 (oldest deadline 2024-09-11T23:51:17.764403+0000) 2024-09-11T23:51:48.040 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:47 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:47.751+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T23:51:00.065354+0000 front 2024-09-11T23:51:00.065433+0000 (oldest deadline 2024-09-11T23:51:24.765421+0000) 2024-09-11T23:51:48.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:47 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:47.751+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-11T23:51:15.267443+0000 front 2024-09-11T23:51:15.267551+0000 (oldest deadline 2024-09-11T23:51:41.667756+0000) 2024-09-11T23:51:48.041 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:47 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11[60916]: 2024-09-11T23:51:47.751+0000 7fd5eca45640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.178:6820 osd.8 since back 2024-09-11T23:51:21.668113+0000 front 2024-09-11T23:51:21.668088+0000 (oldest deadline 2024-09-11T23:51:47.468405+0000) 2024-09-11T23:51:49.620 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:49 smithi178 podman[73072]: 2024-09-11 23:51:49.359354384 +0000 UTC m=+5.691609932 container died 182e022c208c2e37e74e224d37750a66eb56bd0c3cf12521794f596fcde7ae8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T23:51:49.986 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:49 smithi178 podman[73072]: 2024-09-11 23:51:49.719405724 +0000 UTC m=+6.051661271 container cleanup 182e022c208c2e37e74e224d37750a66eb56bd0c3cf12521794f596fcde7ae8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-11T23:51:49.986 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:49 smithi178 bash[73072]: ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11 2024-09-11T23:51:50.290 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:49 smithi178 podman[73087]: 2024-09-11 23:51:49.987355898 +0000 UTC m=+0.629788200 container remove 182e022c208c2e37e74e224d37750a66eb56bd0c3cf12521794f596fcde7ae8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-11T23:51:50.623 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:50 smithi178 podman[73178]: 2024-09-11 23:51:50.32157585 +0000 UTC m=+0.109111585 container create e99267a1dc53ec3ae44e3e8006579672ab0b590f39dc4c6e3f83dc13fb8cb25d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-11T23:51:50.623 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:50 smithi178 podman[73178]: 2024-09-11 23:51:50.228513616 +0000 UTC m=+0.016049363 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:51:50.623 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:50 smithi178 podman[73178]: 2024-09-11 23:51:50.540905777 +0000 UTC m=+0.328441524 container init e99267a1dc53ec3ae44e3e8006579672ab0b590f39dc4c6e3f83dc13fb8cb25d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:51:50.623 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:50 smithi178 podman[73178]: 2024-09-11 23:51:50.544200066 +0000 UTC m=+0.331735808 container start e99267a1dc53ec3ae44e3e8006579672ab0b590f39dc4c6e3f83dc13fb8cb25d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:51:50.623 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:50 smithi178 podman[73178]: 2024-09-11 23:51:50.624139459 +0000 UTC m=+0.411675214 container attach e99267a1dc53ec3ae44e3e8006579672ab0b590f39dc4c6e3f83dc13fb8cb25d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T23:51:51.017 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:50 smithi178 podman[73178]: 2024-09-11 23:51:50.728233516 +0000 UTC m=+0.515769251 container died e99267a1dc53ec3ae44e3e8006579672ab0b590f39dc4c6e3f83dc13fb8cb25d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T23:51:51.281 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 23:51:51 smithi178 podman[73201]: 2024-09-11 23:51:51.260975119 +0000 UTC m=+0.527377793 container remove e99267a1dc53ec3ae44e3e8006579672ab0b590f39dc4c6e3f83dc13fb8cb25d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T23:51:51.286 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@osd.11.service' 2024-09-11T23:51:51.326 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:51:51.327 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-11T23:51:51.327 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-11T23:51:51.327 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.a 2024-09-11T23:51:51.621 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:51:51 smithi133 systemd[1]: Stopping Ceph mds.a for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:51:51.875 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:51:51 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a[70146]: 2024-09-11T23:51:51.621+0000 7f8d45839640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:51:51.875 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:51:51 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a[70146]: 2024-09-11T23:51:51.621+0000 7f8d45839640 -1 mds.a *** got signal Terminated *** 2024-09-11T23:51:54.588 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:51:54 smithi133 podman[117072]: 2024-09-11 23:51:54.291678646 +0000 UTC m=+2.771777022 container died f47ac7b17b43902a563b4032b6dceb62030ec34a22edc7a4c8e8271449649595 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:51:54.959 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:51:54 smithi133 podman[117072]: 2024-09-11 23:51:54.79674469 +0000 UTC m=+3.276843108 container cleanup f47ac7b17b43902a563b4032b6dceb62030ec34a22edc7a4c8e8271449649595 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T23:51:54.959 INFO:journalctl@ceph.mds.a.smithi133.stdout:Sep 11 23:51:54 smithi133 bash[117072]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-a 2024-09-11T23:51:55.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.a.service' 2024-09-11T23:51:55.233 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:51:55.233 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-11T23:51:55.233 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-11T23:51:55.234 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.d 2024-09-11T23:51:55.567 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:51:55 smithi133 systemd[1]: Stopping Ceph mds.d for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:51:55.959 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:51:55 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-d[69037]: 2024-09-11T23:51:55.567+0000 7fe79e98b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:51:55.959 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:51:55 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-d[69037]: 2024-09-11T23:51:55.567+0000 7fe79e98b640 -1 mds.d *** got signal Terminated *** 2024-09-11T23:51:58.692 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:51:58 smithi133 podman[117207]: 2024-09-11 23:51:58.384406721 +0000 UTC m=+2.929436280 container died bca343033f22a6c507425826e32a0330e3c71d21b5980fa76fb2241beb9af19e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T23:51:58.959 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:51:58 smithi133 podman[117207]: 2024-09-11 23:51:58.867564442 +0000 UTC m=+3.412594006 container cleanup bca343033f22a6c507425826e32a0330e3c71d21b5980fa76fb2241beb9af19e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T23:51:58.959 INFO:journalctl@ceph.mds.d.smithi133.stdout:Sep 11 23:51:58 smithi133 bash[117207]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-d 2024-09-11T23:51:59.305 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.d.service' 2024-09-11T23:51:59.339 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:51:59.339 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-11T23:51:59.339 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-11T23:51:59.339 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.g 2024-09-11T23:51:59.664 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:51:59 smithi133 systemd[1]: Stopping Ceph mds.g for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:51:59.959 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:51:59 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g[69809]: 2024-09-11T23:51:59.662+0000 7fa7fe49b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:51:59.959 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:51:59 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g[69809]: 2024-09-11T23:51:59.662+0000 7fa7fe49b640 -1 mds.g *** got signal Terminated *** 2024-09-11T23:52:02.545 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:52:02 smithi133 podman[117329]: 2024-09-11 23:52:02.290026061 +0000 UTC m=+2.730340188 container died 0fa68f80ac2e774301cbfa86b677e307c55c76f559cf4bb6bd6031c3bc953353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T23:52:02.900 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:52:02 smithi133 podman[117329]: 2024-09-11 23:52:02.737812711 +0000 UTC m=+3.178126886 container cleanup 0fa68f80ac2e774301cbfa86b677e307c55c76f559cf4bb6bd6031c3bc953353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T23:52:02.901 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:52:02 smithi133 bash[117329]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g 2024-09-11T23:52:03.200 INFO:journalctl@ceph.mds.g.smithi133.stdout:Sep 11 23:52:03 smithi133 podman[117343]: 2024-09-11 23:52:03.056077307 +0000 UTC m=+0.765778323 container remove 0fa68f80ac2e774301cbfa86b677e307c55c76f559cf4bb6bd6031c3bc953353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-g, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T23:52:03.223 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.g.service' 2024-09-11T23:52:03.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:52:03.256 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-11T23:52:03.257 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-11T23:52:03.257 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.j 2024-09-11T23:52:03.574 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:52:03 smithi133 systemd[1]: Stopping Ceph mds.j for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:52:03.962 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:52:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-j[68623]: 2024-09-11T23:52:03.573+0000 7f0acd2b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:52:03.962 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:52:03 smithi133 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-j[68623]: 2024-09-11T23:52:03.573+0000 7f0acd2b3640 -1 mds.j *** got signal Terminated *** 2024-09-11T23:52:07.249 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:52:06 smithi133 podman[117449]: 2024-09-11 23:52:06.974399412 +0000 UTC m=+3.495083937 container died 34093403e5e3a7ec22f8899242df800d9566e1884540e185666d3169935ac3f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-j, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T23:52:07.700 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:52:07 smithi133 podman[117449]: 2024-09-11 23:52:07.433363409 +0000 UTC m=+3.954047976 container cleanup 34093403e5e3a7ec22f8899242df800d9566e1884540e185666d3169935ac3f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-j, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T23:52:07.700 INFO:journalctl@ceph.mds.j.smithi133.stdout:Sep 11 23:52:07 smithi133 bash[117449]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-j 2024-09-11T23:52:07.857 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.j.service' 2024-09-11T23:52:07.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:52:07.891 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-11T23:52:07.891 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-11T23:52:07.891 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.b 2024-09-11T23:52:08.198 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:52:07 smithi138 systemd[1]: Stopping Ceph mds.b for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:52:08.698 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:52:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b[65233]: 2024-09-11T23:52:08.277+0000 7f962e2f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:52:08.698 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:52:08 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b[65233]: 2024-09-11T23:52:08.277+0000 7f962e2f6640 -1 mds.b *** got signal Terminated *** 2024-09-11T23:52:10.647 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:52:10 smithi138 podman[71853]: 2024-09-11 23:52:10.293258272 +0000 UTC m=+2.195802156 container died dfb088c5ff20c2c81fc92f76bf589f8bd98a68ab3af5b3dea38a9876cee53bd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T23:52:10.951 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:52:10 smithi138 podman[71853]: 2024-09-11 23:52:10.727325814 +0000 UTC m=+2.629869697 container cleanup dfb088c5ff20c2c81fc92f76bf589f8bd98a68ab3af5b3dea38a9876cee53bd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T23:52:10.951 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 11 23:52:10 smithi138 bash[71853]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-b 2024-09-11T23:52:11.216 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.b.service' 2024-09-11T23:52:11.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:52:11.253 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-11T23:52:11.253 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-11T23:52:11.253 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.e 2024-09-11T23:52:11.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:52:11 smithi138 systemd[1]: Stopping Ceph mds.e for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:52:11.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:52:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e[64632]: 2024-09-11T23:52:11.555+0000 7f6674380640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:52:11.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:52:11 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e[64632]: 2024-09-11T23:52:11.555+0000 7f6674380640 -1 mds.e *** got signal Terminated *** 2024-09-11T23:52:14.589 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:52:14 smithi138 podman[71976]: 2024-09-11 23:52:14.302125091 +0000 UTC m=+2.821217774 container died 4a6d2a34381e67a76b11d4afe9149ac32fc17811a4bfcf14361ee42cfcfddd42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-11T23:52:14.897 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:52:14 smithi138 podman[71976]: 2024-09-11 23:52:14.697999601 +0000 UTC m=+3.217092285 container cleanup 4a6d2a34381e67a76b11d4afe9149ac32fc17811a4bfcf14361ee42cfcfddd42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T23:52:14.897 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 11 23:52:14 smithi138 bash[71976]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-e 2024-09-11T23:52:15.074 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.e.service' 2024-09-11T23:52:15.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:52:15.110 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-11T23:52:15.110 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-11T23:52:15.110 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.h 2024-09-11T23:52:15.447 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:52:15 smithi138 systemd[1]: Stopping Ceph mds.h for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:52:15.948 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:52:15 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h[64932]: 2024-09-11T23:52:15.484+0000 7f4142797640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:52:15.948 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:52:15 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h[64932]: 2024-09-11T23:52:15.484+0000 7f4142797640 -1 mds.h *** got signal Terminated *** 2024-09-11T23:52:18.643 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:52:18 smithi138 podman[72107]: 2024-09-11 23:52:18.294016299 +0000 UTC m=+2.911375442 container died f93610347ef8ddfd745a23897a6ef05ad3d850d91d1a8b1853fa6e7a52be216d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T23:52:18.947 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:52:18 smithi138 podman[72107]: 2024-09-11 23:52:18.735253998 +0000 UTC m=+3.352613140 container cleanup f93610347ef8ddfd745a23897a6ef05ad3d850d91d1a8b1853fa6e7a52be216d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T23:52:18.948 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 11 23:52:18 smithi138 bash[72107]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-h 2024-09-11T23:52:19.163 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.h.service' 2024-09-11T23:52:19.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:52:19.198 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-11T23:52:19.198 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-11T23:52:19.198 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.k 2024-09-11T23:52:19.702 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:52:19 smithi138 systemd[1]: Stopping Ceph mds.k for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:52:19.702 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:52:19 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k[64322]: 2024-09-11T23:52:19.505+0000 7f59c182a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:52:19.702 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:52:19 smithi138 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k[64322]: 2024-09-11T23:52:19.505+0000 7f59c182a640 -1 mds.k *** got signal Terminated *** 2024-09-11T23:52:22.697 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:52:22 smithi138 podman[72227]: 2024-09-11 23:52:22.281660223 +0000 UTC m=+2.864220260 container died 53cbe519fe351e375efd60720fcc9844fea3cdc63ebdfa6e1e5554f96c1632ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:52:23.005 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:52:22 smithi138 podman[72227]: 2024-09-11 23:52:22.780353807 +0000 UTC m=+3.362913849 container cleanup 53cbe519fe351e375efd60720fcc9844fea3cdc63ebdfa6e1e5554f96c1632ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T23:52:23.005 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 11 23:52:22 smithi138 bash[72227]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-k 2024-09-11T23:52:23.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.k.service' 2024-09-11T23:52:23.165 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:52:23.166 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-11T23:52:23.166 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-11T23:52:23.166 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.c 2024-09-11T23:52:23.459 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:52:23 smithi178 systemd[1]: Stopping Ceph mds.c for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:52:23.789 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:52:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c[65772]: 2024-09-11T23:52:23.460+0000 7fb194983640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:52:23.790 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:52:23 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c[65772]: 2024-09-11T23:52:23.460+0000 7fb194983640 -1 mds.c *** got signal Terminated *** 2024-09-11T23:52:26.540 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:52:26 smithi178 podman[73279]: 2024-09-11 23:52:26.284659191 +0000 UTC m=+2.923690779 container died 0440d39b47ca86517301520df1f869db6203a87a4254b429b2001437e729ea4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-11T23:52:26.909 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:52:26 smithi178 podman[73279]: 2024-09-11 23:52:26.651398156 +0000 UTC m=+3.290429742 container cleanup 0440d39b47ca86517301520df1f869db6203a87a4254b429b2001437e729ea4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T23:52:26.910 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 23:52:26 smithi178 bash[73279]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-c 2024-09-11T23:52:27.090 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.c.service' 2024-09-11T23:52:27.126 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:52:27.126 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-11T23:52:27.126 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-11T23:52:27.126 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.f 2024-09-11T23:52:27.438 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:52:27 smithi178 systemd[1]: Stopping Ceph mds.f for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:52:27.790 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:52:27 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: 2024-09-11T23:52:27.438+0000 7f448135b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:52:27.790 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:52:27 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f[65471]: 2024-09-11T23:52:27.438+0000 7f448135b640 -1 mds.f *** got signal Terminated *** 2024-09-11T23:52:30.540 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:52:30 smithi178 podman[73413]: 2024-09-11 23:52:30.285968412 +0000 UTC m=+2.942537514 container died 486f09cfb7211226893b4a388965c75d5895c2265d4eb7dfdbcc962b73abfb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-11T23:52:30.913 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:52:30 smithi178 podman[73413]: 2024-09-11 23:52:30.688294078 +0000 UTC m=+3.344863179 container cleanup 486f09cfb7211226893b4a388965c75d5895c2265d4eb7dfdbcc962b73abfb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-11T23:52:30.913 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 23:52:30 smithi178 bash[73413]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-f 2024-09-11T23:52:31.059 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.f.service' 2024-09-11T23:52:31.095 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:52:31.095 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-11T23:52:31.095 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-11T23:52:31.096 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.i 2024-09-11T23:52:31.540 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:52:31 smithi178 systemd[1]: Stopping Ceph mds.i for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:52:31.540 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:52:31 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i[66069]: 2024-09-11T23:52:31.426+0000 7f137a814640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:52:31.540 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:52:31 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i[66069]: 2024-09-11T23:52:31.426+0000 7f137a814640 -1 mds.i *** got signal Terminated *** 2024-09-11T23:52:34.559 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:52:34 smithi178 podman[73533]: 2024-09-11 23:52:34.295883172 +0000 UTC m=+2.970091134 container died ac19066487aa6cab926dbd00506e81016cef9f4dc0fc551b746319d27db0f232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T23:52:34.909 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:52:34 smithi178 podman[73533]: 2024-09-11 23:52:34.650922883 +0000 UTC m=+3.325130844 container cleanup ac19066487aa6cab926dbd00506e81016cef9f4dc0fc551b746319d27db0f232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T23:52:34.909 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 23:52:34 smithi178 bash[73533]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-i 2024-09-11T23:52:35.083 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.i.service' 2024-09-11T23:52:35.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:52:35.116 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-11T23:52:35.116 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-11T23:52:35.116 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.l 2024-09-11T23:52:35.437 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:52:35 smithi178 systemd[1]: Stopping Ceph mds.l for 42321b88-7095-11ef-bcea-c7b262605968... 2024-09-11T23:52:35.787 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:52:35 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l[65172]: 2024-09-11T23:52:35.437+0000 7f5694514640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:52:35.787 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:52:35 smithi178 ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l[65172]: 2024-09-11T23:52:35.437+0000 7f5694514640 -1 mds.l *** got signal Terminated *** 2024-09-11T23:52:38.553 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:52:38 smithi178 podman[73656]: 2024-09-11 23:52:38.295835789 +0000 UTC m=+2.961185034 container died d605213c28229bc6a123fb2068816a8750826699b59129e4bb8328d9ae5f7649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:52:38.913 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:52:38 smithi178 podman[73656]: 2024-09-11 23:52:38.646275111 +0000 UTC m=+3.311624400 container cleanup d605213c28229bc6a123fb2068816a8750826699b59129e4bb8328d9ae5f7649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T23:52:38.913 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 23:52:38 smithi178 bash[73656]: ceph-42321b88-7095-11ef-bcea-c7b262605968-mds-l 2024-09-11T23:52:39.062 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-42321b88-7095-11ef-bcea-c7b262605968@mds.l.service' 2024-09-11T23:52:39.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:52:39.094 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-11T23:52:39.095 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42321b88-7095-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:52:39.295 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:52:41.941 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42321b88-7095-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:52:42.142 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:52:44.487 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42321b88-7095-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:52:44.687 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:52:47.035 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:47.069 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:47.096 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:52:47.124 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T23:52:47.126 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477/remote/smithi133/crash 2024-09-11T23:52:47.127 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:52:47.154 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:52:47.154 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:52:47.156 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477/remote/smithi138/crash 2024-09-11T23:52:47.157 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:52:47.261 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:52:47.261 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:52:47.263 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477/remote/smithi178/crash 2024-09-11T23:52:47.264 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:52:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/42321b88-7095-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:52:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:52:47.320 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T23:52:47.320 DEBUG:teuthology.orchestra.run.smithi133:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-11T23:52:47.357 INFO:tasks.cephadm:Compressing logs... 2024-09-11T23:52:47.357 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:52:47.400 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:52:47.402 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:52:47.425 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T23:52:47.426 INFO:teuthology.orchestra.run.smithi133.stderr:: No such file or directory 2024-09-11T23:52:47.426 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-11T23:52:47.426 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:52:47.426 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:52:47.426 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-11T23:52:47.432 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-11T23:52:47.432 INFO:teuthology.orchestra.run.smithi138.stderr:: No such file or directory 2024-09-11T23:52:47.432 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T23:52:47.432 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T23:52:47.433 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:52:47.433 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-11T23:52:47.433 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-11T23:52:47.433 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.b.log: --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T23:52:47.433 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:52:47.436 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:52:47.436 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T23:52:47.436 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-11T23:52:47.436 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mon.a.log: 84.5% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T23:52:47.437 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T23:52:47.437 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:52:47.437 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.g.log: /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.4% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:52:47.437 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-11T23:52:47.437 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.log: 85.9% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:52:47.437 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T23:52:47.441 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:52:47.441 INFO:teuthology.orchestra.run.smithi178.stderr:: No such file or directory 2024-09-11T23:52:47.441 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr: 81.8% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.log: 90.6% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.l.log: 85.9% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:52:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-11T23:52:47.443 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:52:47.444 INFO:teuthology.orchestra.run.smithi133.stderr: 94.5% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-11T23:52:47.444 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-11T23:52:47.444 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T23:52:47.446 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:52:47.446 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-11T23:52:47.448 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.4.log: 91.0% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:52:47.449 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:52:47.449 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:52:47.449 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T23:52:47.449 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.cephadm.log: 81.8% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T23:52:47.449 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:52:47.449 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.6% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-11T23:52:47.450 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:52:47.451 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-11T23:52:47.453 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.c.log: 92.1% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-11T23:52:47.453 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:52:47.457 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mgr.y.log: 92.0%gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-11T23:52:47.461 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.log: 85.9% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T23:52:47.461 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:52:47.462 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T23:52:47.468 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:52:47.481 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-11T23:52:47.488 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:52:47.491 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:52:47.501 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:52:47.501 INFO:teuthology.orchestra.run.smithi133.stderr: 94.5% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-11T23:52:47.501 INFO:teuthology.orchestra.run.smithi133.stderr: 94.4% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-11T23:52:47.503 INFO:teuthology.orchestra.run.smithi178.stderr: 94.4% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-11T23:52:47.508 INFO:teuthology.orchestra.run.smithi138.stderr: 92.7% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-11T23:52:47.551 INFO:teuthology.orchestra.run.smithi138.stderr: 94.6% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-11T23:52:47.554 INFO:teuthology.orchestra.run.smithi178.stderr: 93.0% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-11T23:52:47.557 INFO:teuthology.orchestra.run.smithi138.stderr: 92.6% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-11T23:52:47.596 INFO:teuthology.orchestra.run.smithi138.stderr: 92.9% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-11T23:52:47.730 INFO:teuthology.orchestra.run.smithi133.stderr: 90.2% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T23:52:47.814 INFO:teuthology.orchestra.run.smithi178.stderr: 91.6% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T23:52:47.868 INFO:teuthology.orchestra.run.smithi138.stderr: 91.6% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T23:52:48.659 INFO:teuthology.orchestra.run.smithi133.stderr: 90.3% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T23:52:52.643 INFO:teuthology.orchestra.run.smithi133.stderr: 93.1% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-11T23:52:52.956 INFO:teuthology.orchestra.run.smithi138.stderr: 93.3% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T23:52:53.806 INFO:teuthology.orchestra.run.smithi178.stderr: 93.2% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-11T23:52:54.219 INFO:teuthology.orchestra.run.smithi178.stderr: 93.2% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T23:52:55.017 INFO:teuthology.orchestra.run.smithi138.stderr: 93.2% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-11T23:52:55.209 INFO:teuthology.orchestra.run.smithi133.stderr: 93.1% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T23:52:55.886 INFO:teuthology.orchestra.run.smithi138.stderr: 93.2% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T23:52:56.246 INFO:teuthology.orchestra.run.smithi133.stderr: 93.2% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T23:52:57.061 INFO:teuthology.orchestra.run.smithi138.stderr: 93.2% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T23:52:57.063 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-09-11T23:52:57.064 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m9.648s 2024-09-11T23:52:57.064 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m30.571s 2024-09-11T23:52:57.064 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m1.202s 2024-09-11T23:52:57.106 INFO:teuthology.orchestra.run.smithi178.stderr: 93.3% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T23:52:57.191 INFO:teuthology.orchestra.run.smithi133.stderr: 93.1% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T23:52:57.193 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-09-11T23:52:57.193 INFO:teuthology.orchestra.run.smithi133.stderr:real 0m9.779s 2024-09-11T23:52:57.193 INFO:teuthology.orchestra.run.smithi133.stderr:user 0m31.989s 2024-09-11T23:52:57.193 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m1.237s 2024-09-11T23:52:57.818 INFO:teuthology.orchestra.run.smithi178.stderr: 97.4% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-11T23:52:58.403 INFO:teuthology.orchestra.run.smithi178.stderr: 93.3% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-11T23:54:21.339 INFO:teuthology.orchestra.run.smithi178.stderr: 94.7% -- replaced with /var/log/ceph/42321b88-7095-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-11T23:54:21.340 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-11T23:54:21.340 INFO:teuthology.orchestra.run.smithi178.stderr:real 1m33.922s 2024-09-11T23:54:21.340 INFO:teuthology.orchestra.run.smithi178.stderr:user 2m13.503s 2024-09-11T23:54:21.340 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m4.866s 2024-09-11T23:54:21.341 INFO:tasks.cephadm:Archiving logs... 2024-09-11T23:54:21.341 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477/remote/smithi133/log 2024-09-11T23:54:21.342 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:54:26.345 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477/remote/smithi138/log 2024-09-11T23:54:26.346 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:54:30.906 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477/remote/smithi178/log 2024-09-11T23:54:30.907 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:54:48.471 INFO:tasks.cephadm:Removing cluster... 2024-09-11T23:54:48.472 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42321b88-7095-11ef-bcea-c7b262605968 --force 2024-09-11T23:54:48.675 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:54:49.023 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42321b88-7095-11ef-bcea-c7b262605968 --force 2024-09-11T23:54:49.224 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:54:49.594 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 42321b88-7095-11ef-bcea-c7b262605968 --force 2024-09-11T23:54:49.798 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 42321b88-7095-11ef-bcea-c7b262605968 2024-09-11T23:54:50.213 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T23:54:50.214 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:54:50.234 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:54:50.252 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:54:50.268 INFO:tasks.cephadm:Teardown complete 2024-09-11T23:54:50.268 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T23:54:50.280 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T23:54:50.280 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:54:50.283 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:54:50.295 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:54:50.437 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T23:54:50.449 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T23:54:50.449 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:54:50.452 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:54:50.455 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:54:50.466 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:54:50.468 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:54:50.469 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:54:50.471 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:54:50.472 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-09-11T23:54:50.472 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +1947us[+1974us] +/- 50ms 2024-09-11T23:54:50.472 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 -1439us[-1439us] +/- 76ms 2024-09-11T23:54:50.472 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 -1496us[-1469us] +/- 82ms 2024-09-11T23:54:50.472 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:54:50.473 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:54:50.473 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-09-11T23:54:50.474 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +1987us[+1989us] +/- 50ms 2024-09-11T23:54:50.474 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 -1408us[-1408us] +/- 76ms 2024-09-11T23:54:50.474 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 -1451us[-1449us] +/- 82ms 2024-09-11T23:54:50.474 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:54:50.475 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:54:50.475 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-11T23:54:50.475 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +1939us[+1971us] +/- 50ms 2024-09-11T23:54:50.475 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 -1457us[-1457us] +/- 76ms 2024-09-11T23:54:50.475 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 -1519us[-1519us] +/- 82ms 2024-09-11T23:54:50.475 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:54:50.475 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T23:54:50.488 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T23:54:50.488 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T23:54:50.573 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:54:50.660 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:54:50.733 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:54:50.800 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:54:50.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:54:50.831 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:54:50.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:54:50.860 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:54:50.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:54:50.889 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T23:54:50.900 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T23:54:50.931 INFO:teuthology.task.internal:Duration was 2051.843426 seconds 2024-09-11T23:54:50.931 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T23:54:50.969 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T23:54:50.969 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:54:50.972 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:54:50.974 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:54:51.009 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:54:51.012 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:54:51.017 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:54:51.497 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T23:54:51.497 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T23:54:51.498 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:54:51.557 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-09-11T23:54:51.557 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:54:51.594 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-09-11T23:54:51.594 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:54:51.618 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T23:54:51.618 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:54:51.620 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:54:51.636 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:54:51.685 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T23:54:51.685 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:54:51.687 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:54:51.703 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:54:53.112 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T23:54:53.124 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T23:54:53.124 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:54:53.150 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:54:53.180 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:54:53.206 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T23:54:53.252 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:54:53.255 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:54:53.257 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:54:53.276 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-09-11T23:54:53.281 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-09-11T23:54:53.282 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-09-11T23:54:53.343 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:54:53.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:54:53.358 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:54:53.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:54:53.374 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:54:53.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:54:53.399 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T23:54:53.412 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T23:54:53.413 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477/remote/smithi133 2024-09-11T23:54:53.414 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:54:53.489 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477/remote/smithi138 2024-09-11T23:54:53.489 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:54:53.527 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901477/remote/smithi178 2024-09-11T23:54:53.528 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:54:53.566 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T23:54:53.566 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:54:53.568 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:54:53.571 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:54:53.615 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T23:54:53.630 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T23:54:53.630 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T23:54:53.710 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T23:54:53.711 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:54:53.713 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:54:53.715 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:54:53.725 INFO:teuthology.orchestra.run.smithi133.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:54 /home/ubuntu/cephtest 2024-09-11T23:54:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:54 /home/ubuntu/cephtest 2024-09-11T23:54:53.730 INFO:teuthology.orchestra.run.smithi178.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:54 /home/ubuntu/cephtest 2024-09-11T23:54:53.732 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T23:54:53.744 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T23:54:53.829 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iogen}} duration: 2051.843425989151 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-11T23:54:53.829 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T23:54:53.920 INFO:teuthology.run:pass